Project: SkyWalking (Observability and Analysis / Observability)
File: backend-apisix-monitoring.md

SkyWalking leverages OpenTelemetry Collector to transfer the metrics to and into the . collects metrics data from APSIX. OpenTelemetry Collector fetches metrics from via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via OpenTelemetry gRPC exporter. The SkyWalking OAP Server parses the expression with to filter/calculate/aggregate and store the results. Enable APISIX . Set up . For details on Prometheus Receiver in OpenTelemetry Collector, refer to . Config SkyWalking . provide multiple dimensions metrics for APISIX server , upstream , route , etc. Accordingly, SkyWalking observes the status, payload, and latency of the APISIX server, which is cataloged as a `LAYER: APISIX` `Service` in the OAP. Meanwhile, the instances would be recognized as `LAYER: APISIX` `instance`s. The route rules and nodes would be recognized as `endpoint`s with `route/` and `upstream/` prefixes. SkyWalking expects OTEL Collector attribute `skywalking_service` to be the `Service` name. Make sure `skywalkingservice` attribute exists through `staticconfigs` of OTEL Prometheus scape config. ```yaml receivers: prometheus: config: scrape_configs: job_name: 'apisix-monitoring' static_configs: targets: ['apisix:9091'] labels: skywalkingservice: exmpleservice_name # Specify SkyWalking Service name ``` You also could leverage OTEL Collector processor to add `skywalking_service` attribute , as following : ```yaml processors: resource/skywalking-service: attributes: key: skywalking_service value:

# Specify Skywalking Service name action: insert ``` Notice , if you don't specify `skywalking_service` attribute, SkyWalking OAP would use `APISIX` as the default service name | Monitoring Panel | Unit | Metric Name | Catalog | Description | Data Source | |-||-|-||--| | HTTP status | | meterapisixsvhttpstatus | Service | The increment rate of the status of HTTP requests | APISIX Prometheus plugin | | HTTP latency | | meterapisixsvhttplatency | Service | The increment rate of the latency of HTTP requests | APISIX Prometheus plugin | | HTTP bandwidth | KB | meterapisixsv_bandwidth | Service | The increment rate of the bandwidth of HTTP requests | APISIX Prometheus plugin | | HTTP status of non-matched requests | | meterapisixsvhttpstatus | Service | The increment rate of the status of HTTP requests, which don't match any route | APISIX Prometheus plugin | | HTTP latency non-matched requests | | meterapisixsvhttplatency | Service | The increment rate of the latency of HTTP requests, which don't match any route | APISIX Prometheus plugin | | HTTP bandwidth non-matched requests | KB | meterapisixsv_bandwidth | Service | The increment rate of the bandwidth of HTTP requests ,which don't match any route | APISIX Prometheus plugin | | HTTP connection | | meterapisixsvhttpconnections | Service | The avg number of the connections | APISIX Prometheus plugin | | HTTP Request Trend | | meterapisixhttp_requests | Service | The increment rate of HTTP requests | APISIX Prometheus plugin | | HTTP status | | meterapisixinstancehttpstatus | Instance | The increment rate of the status of HTTP requests | APISIX Prometheus plugin | | HTTP latency | | meterapisixinstancehttplatency | Instance | The increment rate of the latency of HTTP requests | APISIX Prometheus plugin | | HTTP bandwidth | KB | meterapisixinstance_bandwidth | Instance | The increment rate of the bandwidth of HTTP requests | APISIX Prometheus plugin | | HTTP status of non-matched requests | | meterapisixinstancehttpstatus | Instance | The increment rate of the status of HTTP requests, which don't match any route | APISIX Prometheus plugin | | HTTP latency non-matched requests | | meterapisixinstancehttplatency | Instance | The increment rate of the latency of HTTP requests, which don't match any route | APISIX Prometheus plugin | | HTTP bandwidth non-matched requests | KB | meterapisixinstance_bandwidth | Instance | The increment rate of the bandwidth of HTTP requests ,which don't match any route | APISIX Prometheus plugin | | HTTP connection | | meterapisixinstancehttpconnections | Instance | The avg number of the connections | APISIX Prometheus plugin | | HTTP Request Trend | | meterapisixinstancehttprequests | Instance | The increment rate of HTTP requests | APISIX Prometheus plugin | | Shared dict capacity | MB | meterapisixinstanceshareddictcapacitybytes | Instance | The avg capacity of shared dict capacity | APISIX Prometheus plugin | | Shared free space | MB | meterapisixinstanceshareddictfreespace_bytes | Instance | The avg free space of shared dict capacity | APISIX Prometheus plugin | | etcd index | | meterapisixinstancesvetcd_indexes | Instance | etcd modify index for APISIX keys | APISIX Prometheus plugin | | etcd latest reachability | | meterapisixinstancesvetcd_reachable | Instance | etcd latest reachable , Refer to | APISIX Prometheus plugin | | HTTP status | | meterapisixendpointnodehttp_status | Endpoint | The increment rate of the status of HTTP requests | APISIX Prometheus plugin | | HTTP latency | | meterapisixendpointnodehttp_latency | Endpoint | The increment rate of the latency of HTTP requests | APISIX Prometheus plugin | | HTTP bandwidth | KB | meterapisixendpointnodebandwidth | Endpoint | The increment rate of the bandwidth of