Project: SkyWalking (Observability and Analysis / Observability)
File: backend-k8s-monitoring-metrics-cadvisor.md

SkyWalking leverages K8s kube-state-metrics (KSM) and cAdvisor for collecting metrics data from K8s. It leverages OpenTelemetry Collector to transfer the metrics to and into the . This feature requires authorizing the OAP Server to access K8s's `API Server`. K8s kube-state-metrics and cAdvisor collect metrics data from K8s. OpenTelemetry Collector fetches metrics from kube-state-metrics and cAdvisor via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via OpenTelemetry gRPC exporter. The SkyWalking OAP Server access to K8s's `API Server` gets meta info and parses the expression with to filter/calculate/aggregate and store the results. Setup . cAdvisor is integrated into `kubelet` by default. Set up . For details on Prometheus Receiver in OpenTelemetry Collector for K8s, refer to . For a quick start, we have provided a complete example of configuration and recommended version; you can refer to . Config SkyWalking . K8s cluster monitoring provides monitoring of the status and resources of the whole cluster and each node. K8s cluster as a `Service` in OAP, K8s node as an `Instance` in OAP, and land on the `Layer:

| Monitoring Panel | Unit | Metric Name | Description | Data Source | |--||||| | Node Total | | k8sclusternode_total | The number of nodes | K8s kube-state-metrics | | Namespace Total | | k8sclusternamespace_total | The number of namespaces | K8s kube-state-metrics | | Deployment Total | | k8sclusterdeployment_total | The number of deployments | K8s kube-state-metrics | | StatefulSet Total | | k8sclusterstatefulset_total | The number of statefulsets | K8s kube-state-metrics | | DaemonSet Total | | k8sclusterdaemonset_total | The number of daemonsets | K8s kube-state-metrics | | Service Total | | k8sclusterservice_total | The number of services | K8s kube-state-metrics | | Pod Total | | k8sclusterpod_total | The number of pods | K8s kube-state-metrics | | Container Total | | k8sclustercontainer_total | The number of containers | K8s kube-state-metrics | | CPU Resources | m | k8sclustercpucores<br />k8sclustercpucoresrequests<br />k8sclustercpucoreslimits<br />k8sclustercpucores_allocatable | The capacity and the Requests / Limits / Allocatable of the CPU | K8s kube-state-metrics | | Memory Resources | Gi | k8sclustermemorytotal<br />k8sclustermemoryrequests<br />k8sclustermemorylimits<br />k8sclustermemoryallocatable | The capacity and the Requests / Limits / Allocatable of the memory | K8s kube-state-metrics | | Storage Resources | Gi | k8sclusterstoragetotal<br />k8sclusterstorageallocatable | The capacity and allocatable of the storage | K8s kube-state-metrics | | Node Status | | k8sclusternode_status | The current status of the nodes | K8s kube-state-metrics | | Deployment Status | | k8sclusterdeployment_status | The current status of the deployment | K8s kube-state-metrics | | Deployment Spec Replicas | | k8sclusterdeploymentspecreplicas | The number of desired pods for a deployment | K8s kube-state-metrics | | Service Status | | k8sclusterservicepodstatus | The services current status, depending on the related pods' status | K8s kube-state-metrics | | Pod Status Not Running | | k8sclusterpodstatusnot_running | The pods which are not running in the current phase | K8s kube-state-metrics | | Pod Status Waiting | | k8sclusterpodstatuswaiting | The pods and containers which are currently in the waiting status, with reasons shown | K8s kube-state-metrics | | Pod Status Terminated | | k8sclustercontainerstatusterminated | The pods and containers which are currently in the terminated status, with reasons shown | K8s kube-state-metrics | | Monitoring Panel | Unit | Metric Name | Description | Data Source | |--||--|--|--| | Pod Total | | k8snodepod_total | The number of pods in this node | K8s kube-state-metrics | | Node Status | | k8snodenode_status | The current status of this node | K8s kube-state-metrics | | CPU Resources | m | k8snodecpucores<br />k8snodecpucoresallocatable<br />k8snodecpucoresrequests<br />k8snodecpucores_limits | The capacity and the requests / Limits / Allocatable of the CPU | K8s kube-state-metrics | | Memory Resources | Gi | k8snodememorytotal<br />k8snodememoryallocatable<br />k8snodememoryrequests<br />k8snodememorylimits | The capacity and the requests / Limits / Allocatable of the memory | K8s kube-state-metrics | | Storage Resources | Gi | k8snodestoragetotal<br />k8snodestorageallocatable | The capacity and allocatable of the storage | K8s kube-state-metrics | | CPU Usage | m | k8snodecpu_usage | The total usage of the CPU core, if there are 2 cores the maximum usage is 2000m | cAdvisor | | Memory Usage | Gi | k8snodememory_usage | The totaly memory usage | cAdvisor | | Network I/O| KB/s | k8snodenetworkreceive<br />k8snodenetworktransmit | The network receive and transmit | cAdvisor | K8s Service Monitoring provides observabilities into service status and resources from Kubernetes. K8s Service as a `Service` in OAP and land on the `Layer: K8S_SERVICE`. | Monitoring Panel | Unit | Metric Name | Description | Data Source | |--|--|--|--|--| | Service Pod Total | | k8sservicepod_total | The number of pods | K8s kube-state-metrics | | Service