Project: SkyWalking (Observability and Analysis / Observability)
File: dashboards-so11y.md

SkyWalking itself collects and exports metrics in Prometheus format for consuming, it also provides a dashboard to visualize the self-observability metrics. SkyWalking OAP collects metrics data internally and exposes a Prometheus http endpoint to retrieve the metrics. SkyWalking OAP itself (or OpenTelemetry Collector, prefered in Kubernetes scenarios) fetches metrics from the Prometheus endpoint in step (1). OAP (or OpenTelemetry Collector) pushes metrics to SkyWalking OAP Server via OpenTelemetry gRPC exporter. The SkyWalking OAP Server parses the expression with to filter/calculate/aggregate and store the results. Follow to set up OAP and OpenTelemetry Collector. Self observability monitoring provides monitoring of the status and resources of the OAP server itself. `oap-server` is a `Service` in OAP, and land on the `Layer: SO11Y_OAP`. | Unit | Metric Name | Description | Data Source | |||--|--| | Count Per Minute | meteroapinstancejvmgc_count | GC Count | oap self observability | | MB | meteroapinstancejvmmemorybytesused | Memory | oap self observability | | ms / min | meteroapinstancejvmyounggctime | GC Time (ms / min) | oap self observability | | ms / min | meteroapinstancejvmoldgctime | GC Time (ms / min) | oap self observability | | Count Per Minute | meteroapinstancemeshcount | Mesh Analysis Count (Per Minute) | oap self observability | | Count Per Minute | meteroapinstancemeshanalysiserrorcount | Mesh Analysis Count (Per Minute) | oap self observability | | ms | meteroapinstancetracelatency_percentile | Trace Analysis Latency (ms) | oap self observability | | Count | meteroapjvmclassloaded_count | Class Count | oap self observability | | Count | meteroapjvmclasstotalunloadedcount | Class Count | oap self observability | | Count | meteroapjvmclasstotalloadedcount | Class Count | oap self observability | | Count | meteroapinstancepersistenceprepare_count | Persistence Count (Per 5 Minutes) | oap self observability | | Count | meteroapinstancepersistenceexecute_count | Persistence Count (Per 5 Minutes) | oap self observability | | Count | meteroapjvmthreadlive_count | Thread Count | oap self observability | | Count | meteroapjvmthreadpeak_count | Thread Count | oap self observability | | Count | meteroapjvmthreaddaemon_count | Thread Count | oap self observability | | ms | meteroapinstancepersistenceexecute_percentile | Persistence Execution Latency Per Metric Type (ms) | oap self observability | | ms | meteroapinstancepersistenceprepare_percentile | Persistence Preparing Latency Per Metric Type (ms) | oap self observability | | Count | meteroapjvmthreadrunnable_count | Thread State Count | oap self observability | | Count | meteroapjvmthreadtimedwaitingcount | Thread State Count | oap self observability | | Count | meteroapjvmthreadblocked_count | Thread State Count | oap self observability | | Count | meteroapjvmthreadwaiting_count | Thread State Count | oap self observability | | Count per minute | meteroapinstancemetricsaggregation | Aggregation (Per Minute) | oap self observability | | ms | meteroapinstancemeshlatency_percentile | Mesh Analysis Latency (ms) | oap self observability | | Count per minute | meteroapinstancetracecount | Trace Analysis Count (Per Minute) | oap self observability | | Count per minute | meteroapinstancetraceanalysiserrorcount | Trace Analysis Count (Per Minute) | oap self observability | | Percentage | meteroapinstancecpupercentage | CPU (%) | oap self observability | | Count | meteroapinstancemetricspersistent_cache | count of metrics cache hit and no-hit |oap self observability| You can customize your own metrics/expression/dashboard panel. The metrics definition and expression rules are found in `/config/fetcher-prom-rules/self.yaml` and `config/otel-rules/oap.yaml`. The self observability dashboard panel configurations are found in `/config/ui-initialized-templates/so11y_oap`.