Data Collection Methods - Revision history

DrEdWilliams: /* Elastic Products */

2021-11-18T00:15:51Z

Elastic Products

← Older revision		Revision as of 00:15, 18 November 2021
Line 10:		Line 10:
	The elastic suite of applications includes tools that are usable for both the Elastic tools and the open source alternatives (such as OpenSearch):		The elastic suite of applications includes tools that are usable for both the Elastic tools and the open source alternatives (such as OpenSearch):

	* Logstash		*Logstash - aggregate and process data through pipelines before sending to OpenSearch
	* Filebeat		*Metricbeat - system data from Linux/Windows systems
	* ~~Metricbeat~~		*Filebeat - capturing text from logfiles
			**capture OpenSearch logs into OpenSearch for better visibility and analysis
			**capture syslogs from the log server (pro6) for all systems

	=== Prometheus Exporters: ===		=== Prometheus Exporters: ===
	[https://prometheus.io/ Prometheus] implements a pull method vs the push method used by the Elastic products. Metricbeat serves as an interface between the two methods -- one of its modules can read prometheus format metrics -- either produced natively by applications or by specific exporter applications. A list of exporters from the prometheus website can be found at https://prometheus.io/docs/instrumenting/exporters/		[https://prometheus.io/ Prometheus] implements a pull method vs the push method used by the Elastic products. Metricbeat serves as an interface between the two methods -- one of its modules can read prometheus format metrics -- either produced natively by applications or by specific exporter applications. A list of exporters from the prometheus website can be found at https://prometheus.io/docs/instrumenting/exporters/

	A general-purpose exporter is [https://github.com/prometheus/node_exporter node_exporter] -- it can export a large range of metrics across many areas of system operations.		* A general-purpose exporter is [https://github.com/prometheus/node_exporter node_exporter] -- it can export a large range of metrics across many areas of system operations.

			=== Custom Tools ===

			* NVIDIA - simple script to get GPU data

DrEdWilliams at 00:03, 18 November 2021

2021-11-18T00:03:05Z

← Older revision		Revision as of 00:03, 18 November 2021
Line 16:		Line 16:
	=== Prometheus Exporters: ===		=== Prometheus Exporters: ===
	[https://prometheus.io/ Prometheus] implements a pull method vs the push method used by the Elastic products. Metricbeat serves as an interface between the two methods -- one of its modules can read prometheus format metrics -- either produced natively by applications or by specific exporter applications. A list of exporters from the prometheus website can be found at https://prometheus.io/docs/instrumenting/exporters/		[https://prometheus.io/ Prometheus] implements a pull method vs the push method used by the Elastic products. Metricbeat serves as an interface between the two methods -- one of its modules can read prometheus format metrics -- either produced natively by applications or by specific exporter applications. A list of exporters from the prometheus website can be found at https://prometheus.io/docs/instrumenting/exporters/

			A general-purpose exporter is [https://github.com/prometheus/node_exporter node_exporter] -- it can export a large range of metrics across many areas of system operations.

DrEdWilliams: Created page with "(notes -- work in progress) Getting data for OpenSearch/ElasticSearch is a matter of: * Identifying the data to be gathered * Finding a way to surface and collect that data..."

2021-11-17T23:49:24Z

Created page with "(notes -- work in progress) Getting data for OpenSearch/ElasticSearch is a matter of: * Identifying the data to be gathered * Finding a way to surface and collect that data..."

New page

(notes -- work in progress)

Getting data for OpenSearch/ElasticSearch is a matter of:

* Identifying the data to be gathered
* Finding a way to surface and collect that data
* Send the data to the ElasticSearch/OpenSearch cluster

=== Elastic Products ===
The elastic suite of applications includes tools that are usable for both the Elastic tools and the open source alternatives (such as OpenSearch):

* Logstash
* Filebeat
* Metricbeat

=== Prometheus Exporters: ===
[https://prometheus.io/ Prometheus] implements a pull method vs the push method used by the Elastic products. Metricbeat serves as an interface between the two methods -- one of its modules can read prometheus format metrics -- either produced natively by applications or by specific exporter applications. A list of exporters from the prometheus website can be found at https://prometheus.io/docs/instrumenting/exporters/