OpenSearch
WilliamsNet OpenSearch Architecture
For convenience, and to enable this cluster to be usable as a Test and Development System for the HPCMP OpenSearch cluster, the configuration and naming conventions here will mirror that used in the HPCMP environment:
- Indices will be named using this style:
<data source>-<schema version>-<system name>-YYYY.MM.DD - Data will be routed through logstash to enable processing and augmentation prior to indexing in the cluster
OpenSearch Cluster & Dashboards
The OpenSearch Cluster is composed of these node types:
| Node Type | Servers | System
RAM (GB) |
Heap
Size (GB) |
|---|---|---|---|
| Master/Ingest | poggin | 16 | 4 |
| aravis | 8 | 4 | |
| lamppost | 8 | 4 | |
| Hot Data | caspian | 72 | 16 |
| belisar | 72 | 16 | |
| uvilas | 72 | 16 | |
| Cold Data | calormen | 16 | 6 |
| trumpkin | 12 | 6 | |
| Dashboards | pro1 | 40 | |
| Monitoring | pro1 (cerebro) | 40 |
All nodes have access to the /work filesystem (NFS shared by pro6), and use /work/osdata/<hostname> for logs and data (if appropriate).
Aggregation and Processing
Logstash is installed on calormen and serves as the primary ingest route for the cluster.
Data Collection
All servers should have metricbeat installed, feeding data to logstash on calormen
OpenSearch How-Tos
Topics related to creating, maintaining, and using an elastic cluster