Project: SkyWalking (Observability and Analysis / Observability)
File: backend-elasticsearch-monitoring.md

SkyWalking leverages elasticsearch-exporter for collecting metrics data from Elasticsearch. It leverages OpenTelemetry Collector to transfer the metrics to and into the . The elasticsearch-exporter collect metrics data from Elasticsearch. OpenTelemetry Collector fetches metrics from elasticsearch-exporter via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via OpenTelemetry gRPC exporter. The SkyWalking OAP Server parses the expression with to filter/calculate/aggregate and store the results. Setup . Set up . The example for OpenTelemetry Collector configuration, refer to . Config SkyWalking . Elasticsearch monitoring provides multidimensional metrics monitoring of Elasticsearch clusters as `Layer: ELASTICSEARCH` `Service` in the OAP. In each cluster, the nodes are represented as `Instance` and indices are `Endpoint`s. | Monitoring Panel | Metric Name | Description | Data Source | ||-|-|| | Cluster Health | meterelasticsearchclusterhealthstatus | Whether all primary and replica shards are allocated | elasticsearch-exporter | | Tripped Of Breakers | meterelasticsearchclusterbreakerstripped | Tripped for breaker | elasticsearch-exporter | | Nodes | meterelasticsearchcluster_nodes | Number of nodes in the cluster. | elasticsearch-exporter | | Data Nodes | meterelasticsearchclusterdatanodes | Number of data nodes in the cluster | elasticsearch-exporter | | Pending Tasks | meterelasticsearchclusterpendingtasks_total | Cluster level changes which have not yet been executed | elasticsearch-exporter | | CPU Usage Avg. (%) | meterelasticsearchclustercpuusage_avg | Cluster level percent CPU used by process | elasticsearch-exporter | | JVM Memory Used Avg. (%) | meterelasticsearchclusterjvmmemoryusedavg | Cluster level percent JVM memory used | elasticsearch-exporter | | Open Files | meterelasticsearchclusteropenfile_count | Open file descriptors | elasticsearch-exporter | | Active Primary Shards | meterelasticsearchclusterprimaryshards_total | The number of primary shards in your

This is an aggregate total across all indices | elasticsearch-exporter | | Active Shards | meterelasticsearchclustershardstotal | Aggregate total of all shards across all indices, which includes replica shards | elasticsearch-exporter | | Initializing Shards | meterelasticsearchclusterinitializingshards_total | Count of shards that are being freshly created | elasticsearch-exporter | | Delayed Unassigned Shards | meterelasticsearchclusterdelayedunassignedshardstotal | Shards delayed to reduce reallocation overhead | elasticsearch-exporter | | Relocating Shards | meterelasticsearchclusterrelocatingshards_total | The number of shards that are currently moving from one node to another node | elasticsearch-exporter | | Unassigned Shards | meterelasticsearchclusterunassignedshards_total | The number of shards that exist in the cluster state, but cannot be found in the cluster itself | elasticsearch-exporter | | Monitoring Panel | Unit | Metric Name | Description | Data Source | ||--||-|| | Node Rules | | meterelasticsearchnode_rules | Node roles | elasticsearch-exporter | | JVM Memory Used | MB | meterelasticsearchnodejvmmemory_used | Node level JVM memory used size | elasticsearch-exporter | | CPU Percent | % | meterelasticsearchnodeprocesscpu_percent | Node level percent CPU used by process | elasticsearch-exporter | | Documents | | meterelasticsearchnodeindicesdocs | Count of index documents on this node | elasticsearch-exporter | | Segments | | meterelasticsearchnodesegmentcount | Count of index segments on this node | elasticsearch-exporter | | Disk Free Space | GB | meterelasticsearchnodealldiskfreespace | Available space on all block device | elasticsearch-exporter | | Open Files | | meterelasticsearchnodeopenfile_count | Open file descriptors | elasticsearch-exporter | | Process CPU Usage Percent | % | meterelasticsearchnodeprocesscpu_percent | Percent CPU used by process | elasticsearch-exporter | | OS CPU usage percent | % | meterelasticsearchnodeoscpu_percent | Percent CPU used by the OS | elasticsearch-exporter | | Load Average | | meterelasticsearchnodeosload1 <br/>meterelasticsearchnodeosload5<br/>meterelasticsearchnodeosload15 | Shortterm, Midterm, Longterm load average | elasticsearch-exporter | | JVM Memory Usage | MB | meterelasticsearchnodejvmmemorynonheapused<br />meterelasticsearchnodejvmmemoryheapused<br/>meterelasticsearchnodejvmmemoryheapmax | JVM memory currently usage by area | elasticsearch-exporter | | JVM Pool Peak Used | MB | meterelasticsearchnodejvmmemorypoolpeak_used | JVM memory currently used by pool | elasticsearch-exporter | | GC Count | | meterelasticsearchnodejvmgc_count | Count of JVM GC runs | elasticsearch-exporter | | GC Time | ms/min | meterelasticsearchnodejvmgc_time | GC run time | elasticsearch-exporter | | All Operations ReqRate | | meterelasticsearchnodeindices*reqrate | All Operations ReqRate on node | elasticsearch-exporter | | Indexing Rate | reqps | meterelasticsearchnodeindicesindexingindextotalreqrate<br />meterelasticsearchnodeindicesindexingindextotalprocrate | Indexing rate on node | elasticsearch-exporter | | Searching Rate | reqps | meterelasticsearchnodeindicessearchfetchtotalreqrate<br />meterelasticsearchnodeindicessearchquerytimesecondsproc_rate | Searching rate on node | elasticsearch-exporter | | Total Translog Operations | | meterelasticsearchnodeindicestranslog_operations | Total translog operations | elasticsearch-exporter | | Total Translog Size | MB | meterelasticsearchnodeindicestranslog_size | Total translog size | elasticsearch-exporter | | Tripped For Breakers | | meterelasticsearchnodebreakerstripped | Tripped for breaker | elasticsearch-exporter | | Estimated Size Of Breaker | MB | meterelasticsearchnodebreakersestimated_size | Estimated size of breaker | elasticsearch-exporter | | Documents Count | KB/s | meterelasticsearchnodeindicesdocs | Count of documents on this node | elasticsearch-exporter | | Merged Documents Count | count/s | meterelasticsearchnodeindicesmergesdocstotal | Cumulative docs merged | elasticsearch-exporter | | Deleted Documents Count | | meterelasticsearchnodeindicesdocsdeletedtotal | Count of deleted documents on this node | elasticsearch-exporter | | Documents Index Rate | calls/s | meterelasticsearchnodeindicesindexingindextotalreqrate | Total index calls per second | elasticsearch-exporter | | Merged Documents Rate | MB / s | meterelasticsearchnodeindicesmergestotalsizebytestotal | Total merge size per second | elasticsearch-exporter | | Documents Deleted Rate | docs/s | meterelasticsearchnodeindicesdocs_deleted | Count of deleted documents per second on this node | elasticsearch-exporter | | Count Of Index Segments | | meterelasticsearchnodesegmentcount | Count of index segments on this node | elasticsearch-exporter | | Current Memory Size Of