OpenSearch

WilliamsNet OpenSearch Architecture

For convenience, and to enable this cluster to be usable as a Test and Development System for the HPCMP OpenSearch cluster, the configuration and naming conventions here will mirror that used in the HPCMP environment:

• Indices will be named using this style: <data source>-<schema version>-<system name>-YYYY.MM.DD
• Data will be routed through logstash to enable processing and augmentation prior to indexing in the cluster

OpenSearch Cluster & Dashboards

The OpenSearch Cluster is composed of these node types:

Node Type	Servers	System RAM (GB)	Heap Size (GB)
Master/Ingest	poggin	16	4
	aravis	8	4
	lamppost	8	4
Hot Data	caspian	72	16
	belisar	72	16
	uvilas	72	16
Cold Data	calormen	16	6
	trumpkin	12	6
Dashboards	pro1	40
Monitoring	pro1 (cerebro)	40

All nodes have access to the /work filesystem (NFS shared by pro6), and use /work/osdata/<hostname> for logs and data (if appropriate).

All communications with (port 9200) and within (port 9300) the cluster is SSL encrypted using certificates generated based on a self-signed CA. All certs are located in /work/osdata/certs.

Aggregation and Processing

Logstash is installed on calormen and serves as the primary ingest route for the cluster. Additionally, another instance of logstash is installed on aslan for testing/validation purposes.

Data Collection

• All servers should have metricbeat installed, feeding data to logstash on calormen
• Servers with CUDA-capable NVIDIA GPUs have the NVIDIA collection script installed:
  • poggin (Quadro GP100)
  • aslan (GTX 1080 Ti)
  • rogin (GTX 1080)
  • compute1 (GTX 1070)
  • griffle (GTX 1060, GTX 950)
  • trumpkin (GTX 950)

Cluster Monitoring

The OpenSearch cluster is monitored using the methods described in the Cluster Monitoring page:

• OpenSearch node logs are collected by a single instance of filebeat running on poggin, accessing the logs directly from the shared filesystem. The log data is sent through logstash on calormen, where the JSON log data is expanded and time-tagged
• Dashboards and visualizations are set up to view the log data and the audit data

Authentication

• User authentication to Dashboards is username/password using the OpenSearch internal user database
• API authentication uses username/password and certificate authentication methods
• Authentication of nodes in the cluster uses node certificates and the DNs contained in the opensearch.yml configuration files
• All certs are located in /work/osdata/certs

OpenSearch How-Tos

Topics related to creating, maintaining, and using an elastic cluster

• Installing an OpenSearch Cluster & Dashboards
• Data Collection Methods
• Cluster Monitoring
• Cluster Operations