OpenSearch - Revision history

DrEdWilliams: /* OpenSearch How-Tos */

2021-11-26T12:51:19Z

OpenSearch How-Tos

← Older revision		Revision as of 12:51, 26 November 2021
Line 102:		Line 102:
	* [[Data Collection Methods]]		* [[Data Collection Methods]]
	* [[OpenSearch Cluster Monitoring\|Cluster Monitoring]]		* [[OpenSearch Cluster Monitoring\|Cluster Monitoring]]
			*[[Cluster Operations]]

DrEdWilliams: /* Authentication */

2021-11-20T20:24:15Z

Authentication

← Older revision		Revision as of 20:24, 20 November 2021
Line 92:		Line 92:

	* User authentication to Dashboards is username/password using the OpenSearch internal user database		* User authentication to Dashboards is username/password using the OpenSearch internal user database
	* API authentication uses username/password and certificate authentication methods.		* API authentication uses username/password and certificate authentication methods
	* Authentication of nodes in the cluster uses node certificates and the DNs contained in the <code>opensearch.yml</code> configuration files		* Authentication of nodes in the cluster uses node certificates and the DNs contained in the <code>opensearch.yml</code> configuration files
			*All certs are located in <code>/work/osdata/certs</code>

	== OpenSearch How-Tos ==		== OpenSearch How-Tos ==

DrEdWilliams: /* Data Collection */

2021-11-20T19:52:01Z

Data Collection

← Older revision		Revision as of 19:52, 20 November 2021
Line 65:		Line 65:
	\|		\|
	\|}		\|}
	All nodes have access to the /work filesystem (NFS shared by pro6), and use /work/osdata/<hostname> for logs and data (if appropriate).		All nodes have access to the <code>/work</code> filesystem (NFS shared by '''pro6'''), and use <code>/work/osdata/<hostname></code> for logs and data (if appropriate).

			All communications with (port 9200) and within (port 9300) the cluster is SSL encrypted using certificates generated based on a self-signed CA. All certs are located in <code>/work/osdata/certs</code>.

	=== Aggregation and Processing ===		=== Aggregation and Processing ===
	Logstash is installed on calormen and serves as the primary ingest route for the cluster.		Logstash is installed on '''calormen''' and serves as the primary ingest route for the cluster. Additionally, another instance of logstash is installed on '''aslan''' for testing/validation purposes.

	=== Data Collection ===		=== Data Collection ===
	All servers should have metricbeat installed, feeding data to logstash on calormen
			* All servers should have metricbeat installed, feeding data to logstash on calormen
			* Servers with CUDA-capable NVIDIA GPUs have the NVIDIA collection script installed:
			** '''poggin''' (Quadro GP100)
			** '''aslan''' (GTX 1080 Ti)
			** '''rogin''' (GTX 1080)
			** '''compute1''' (GTX 1070)
			** '''griffle''' (GTX 1060, GTX 950)
			** '''trumpkin''' (GTX 950)

			=== Cluster Monitoring ===
			The OpenSearch cluster is monitored using the methods described in the [[Cluster Monitoring]] page:

			* OpenSearch node logs are collected by a single instance of filebeat running on '''poggin''', accessing the logs directly from the shared filesystem. The log data is sent through logstash on '''calormen''', where the JSON log data is expanded and time-tagged
			* Dashboards and visualizations are set up to view the log data and the audit data

			=== Authentication ===

			* User authentication to Dashboards is username/password using the OpenSearch internal user database
			* API authentication uses username/password and certificate authentication methods.
			* Authentication of nodes in the cluster uses node certificates and the DNs contained in the <code>opensearch.yml</code> configuration files

	== OpenSearch How-Tos ==		== OpenSearch How-Tos ==

DrEdWilliams at 19:10, 20 November 2021

2021-11-20T19:10:52Z

@@ Line 1: / Line 1: @@
 Topics related to creating, maintaining, and using an elastic cluster
-* [[OpenSearch Cluster]]
+* [[OpenSearch Cluster|Installing an OpenSearch Cluster & Dashboards]]
 * [[Data Collection Methods]]
 * [[OpenSearch Cluster Monitoring|Cluster Monitoring]]

DrEdWilliams: Created page with "Topics related to creating, maintaining, and using an elastic cluster * OpenSearch Cluster * Data Collection Methods * OpenSearch Cluster Monitoring|Cluster Monitor..."

2021-11-18T01:35:35Z

Created page with "Topics related to creating, maintaining, and using an elastic cluster * OpenSearch Cluster * Data Collection Methods * OpenSearch Cluster Monitoring|Cluster Monitor..."

New page

Topics related to creating, maintaining, and using an elastic cluster

* [[OpenSearch Cluster]]
* [[Data Collection Methods]]
* [[OpenSearch Cluster Monitoring|Cluster Monitoring]]