OpenSearch

WilliamsNet OpenSearch Architecture

For convenience, and to enable this cluster to be usable as a Test and Development System for the HPCMP OpenSearch cluster, the configuration and naming conventions here will mirror that used in the HPCMP environment:

• Indices will be named using this style: <data source>-<schema version>-<system name>-YYYY.MM.DD
• Data will be routed through logstash to enable processing and augmentation prior to indexing in the cluster

OpenSearch Cluster & Dashboards

The OpenSearch Cluster is composed of these node types:

Node Type	Servers	System RAM (GB)	Heap Size (GB)
Master/Ingest	poggin	16	4
	aravis	8	4
	lamppost	8	4
Hot Data	caspian	72	16
	belisar	72	16
	uvilas	72	16
Cold Data	calormen	16	6
	trumpkin	12	6
Dashboards	pro1	40
Monitoring	pro1 (cerebro)	40

All nodes have access to the /work filesystem (NFS shared by pro6), and use /work/osdata/<hostname> for logs and data (if appropriate).

Aggregation and Processing

Logstash is installed on calormen and serves as the primary ingest route for the cluster.

Data Collection

All servers should have metricbeat installed, feeding data to logstash on calormen

OpenSearch How-Tos

Topics related to creating, maintaining, and using an elastic cluster

• Installing an OpenSearch Cluster & Dashboards
• Data Collection Methods
• Cluster Monitoring