Project: SkyWalking (Observability and Analysis / Observability)
File: backend-bookkeeper-monitoring.md

SkyWalking leverages OpenTelemetry Collector to collect metrics data from the BookKeeper and leverages OpenTelemetry Collector to transfer the metrics to and into the . Kafka entity as a `Service` in OAP and on the `Layer: BOOKKEEPER. BookKeeper exposes metrics through Prometheus endpoint. OpenTelemetry Collector fetches metrics from BookKeeper cluster via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via OpenTelemetry gRPC exporter. The SkyWalking OAP Server parses the expression with to filter/calculate/aggregate and store the results.` Set up . Set up . The example for OpenTelemetry Collector configuration, refer to . Config SkyWalking . Bookkeeper monitoring provides multidimensional metrics monitoring of BookKeeper cluster as `Layer: BOOKKEEPER` `Service` in the OAP. In each cluster, the nodes are represented as `Instance`. | Monitoring Panel | Metric Name | Description | Data Source | |--|||| | Bookie Ledgers Count | meterbookkeeperbookieledgerscount | The number of the bookie ledgers. | Bookkeeper Cluster | | Bookie Ledger Writable Dirs | meterbookkeeperbookieledgerwritable_dirs | The number of writable directories in the bookie. | Bookkeeper Cluster | | Bookie Ledger Dir Usage | meterbookkeeperbookieledgerdirdatabookkeeperledgersusage | The number of successfully created connections. | Bookkeeper Cluster | | Bookie Entries Count | meterbookkeeperbookieentriescount | The number of the bookie write entries. | Bookkeeper Cluster | | Bookie Write Cache Size | meterbookkeeperbookiewritecache_size | The size of the bookie write cache (MB). | Bookkeeper Cluster | | Bookie Write Cache Entry Count | meterbookkeeperbookiewritecache_count | The entry count in the bookie write cache. | Bookkeeper Cluster | | Bookie Read Cache Size | meterbookkeeperbookiereadcache_size | The size of the bookie read cache (MB). | Bookkeeper Cluster | | Bookie Read Cache Entry Count | meterbookkeeperbookiereadcache_count | The entry count in the bookie read cache. | Bookkeeper Cluster | | Bookie Read Rate | meterbookkeeperbookiereadrate | The bookie read rate (bytes/s). | Bookkeeper Cluster | | Bookie Write Rate | meterbookkeeperbookiewriterate | The bookie write rate (bytes/s). | Bookkeeper Cluster | | Monitoring Panel | Metric Name | Description | Data Source | |-|||--| | JVM Memory Pool Used | meterbookkeepernodejvmmemorypoolused | The usage of the broker jvm memory pool. | Bookkeeper Bookie | | JVM Memory | meterbookkeepernodejvmmemoryused <br /> meterbookkeepernodejvmmemorycommitted <br /> meterbookkeepernodejvmmemory_init | The usage of the broker jvm memory. | Bookkeeper Bookie | | JVM Threads | meterbookkeepernodejvmthreadscurrent <br /> meterbookkeepernodejvmthreadsdaemon <br /> meterbookkeepernodejvmthreadspeak <br /> meterbookkeepernodejvmthreadsdeadlocked | The count of the jvm threads. | Bookkeeper Bookie | | GC Time | meterbookkeepernodejvmgccollectionseconds_sum | Time spent in a given JVM garbage collector in seconds. | Bookkeeper Bookie | | GC Count | meterbookkeepernodejvmgccollectionseconds_count | The count of a given JVM garbage. | Bookkeeper Bookie | | Thread Executor Completed | meterbookkeepernodethreadexecutor_completed | The count of the executor thread. | Bookkeeper Bookie | | Thread Executor Tasks | meterbookkeepernodethreadexecutortaskscompleted <br /> meterbookkeepernodethreadexecutortasksrejected <br /> meterbookkeepernodethreadexecutortasksfailed | The count of the executor tasks. | Bookkeeper Bookie | | Pooled Threads | meterbookkeepernodehighprioritythreads <br /> meterbookkeepernodereadthreadpool_threads | The count of the pooled thread. | Bookkeeper Bookie | | Pooled Threads Max Queue Size | meterbookkeepernodehighprioritythreadmaxqueuesize <br /> meterbookkeepernodereadthreadpoolmaxqueuesize | The count of the pooled threads max queue size. | Bookkeeper Bookie | You can customize your own metrics/expression/dashboard panel. The metrics definition and expression rules are found in `otel-rules/bookkeeper/bookkeeper-cluster.yaml, otel-rules/bookkeeper/bookkeeper-node.yaml`. The Bookkeeper dashboard panel configurations are found in `/config/ui-initialized-templates/bookkeeper`.