Data Collection Methods

(notes -- work in progress)

Getting data for OpenSearch/ElasticSearch is a matter of:

• Identifying the data to be gathered
• Finding a way to surface and collect that data
• Send the data to the ElasticSearch/OpenSearch cluster

Elastic Products

The elastic suite of applications includes tools that are usable for both the Elastic tools and the open source alternatives (such as OpenSearch):

• Logstash
• Filebeat
• Metricbeat

Prometheus Exporters:

Prometheus implements a pull method vs the push method used by the Elastic products. Metricbeat serves as an interface between the two methods -- one of its modules can read prometheus format metrics -- either produced natively by applications or by specific exporter applications. A list of exporters from the prometheus website can be found at https://prometheus.io/docs/instrumenting/exporters/

A general-purpose exporter is node_exporter -- it can export a large range of metrics across many areas of system operations.