Elasticsearch 7.9+

Elasticsearch is an open-source search server, based on the Lucene search library. It runs in a Java virtual machine on top of a number of operating systems. The elasticsearch receiver collects node- and cluster-level telemetry from your Elasticsearch instances.

For more information about Elasticsearch, see the Elasticsearch 7.9+ documentation.

Prerequisites

To collect Elasticsearch telemetry, you must install the Ops Agent:

• For metrics, install version 2.21.0 or higher.
• For logs, install version 2.9.0 or higher.

This integration supports Elasticsearch version 7.9+.

Configure your Elasticsearch instance

If you enable Elasticsearch security features, you must configure a user with the monitor or manage cluster privilege.

Configure the Ops Agent for Elasticsearch

Following the guide to Configure the Ops Agent, add the required elements to collect telemetry from Elasticsearch instances, and restart the agent.

Example configuration

The following commands create the configuration to collect and ingest telemetry for Elasticsearch:

# Configures Ops Agent to collect telemetry from the app. You must restart the agent for the configuration to take effect.

set -e

# Check if the file exists
if [ ! -f /etc/google-cloud-ops-agent/config.yaml ]; then
  # Create the file if it doesn't exist.
  sudo mkdir -p /etc/google-cloud-ops-agent
  sudo touch /etc/google-cloud-ops-agent/config.yaml
fi

# Create a back up of the existing file so existing configurations are not lost.
sudo cp /etc/google-cloud-ops-agent/config.yaml /etc/google-cloud-ops-agent/config.yaml.bak

# Configure the Ops Agent.
sudo tee /etc/google-cloud-ops-agent/config.yaml > /dev/null << EOF
metrics:
  receivers:
    elasticsearch:
      type: elasticsearch
  service:
    pipelines:
      elasticsearch:
        receivers:
          - elasticsearch

logging:
  receivers:
    elasticsearch_json:
      type: elasticsearch_json
    elasticsearch_gc:
      type: elasticsearch_gc
  service:
    pipelines:
      elasticsearch:
        receivers:
          - elasticsearch_json
          - elasticsearch_gc
EOF

For these changes to take effect, you must restart the Ops Agent:

Configure logs collection

To ingest logs from Elasticsearch, you must create a receiver for the logs that Elasticsearch produces and then create a pipeline for the new receiver.

To configure a receiver for your elasticsearch_json logs, specify the following fields:

To configure a receiver for your elasticsearch_gc logs, specify the following fields:

What is logged

The logName is derived from the receiver IDs specified in the configuration. Detailed fields inside the LogEntry are as follows.

The elasticsearch_json logs contain the following fields in the LogEntry:

The elasticsearch_gc logs contain the following fields in the LogEntry:

Configure metrics collection

To ingest metrics from Elasticsearch, you must create a receiver for the metrics that Elasticsearch produces and then create a pipeline for the new receiver.

This receiver does not support the use of multiple instances in the configuration, for example, to monitor multiple endpoints. All such instances write to the same time series, and Cloud Monitoring has no way to distinguish among them.

To configure a receiver for your elasticsearch metrics, specify the following fields:

What is monitored

The following table provides the list of metrics that the Ops Agent collects from the Elasticsearch instance.