Project: SkyWalking (Observability and Analysis / Observability)
File: backend-rabbitmq-monitoring.md

SkyWalking leverages `rabbitmq_prometheus` plugin for collecting metrics data from RabbitMQ. It leverages OpenTelemetry Collector to transfer the metrics to and into the . The `rabbitmq_prometheus` plugin collect metrics data from RabbitMQ. Note: The RabbitMQ version is required to be 3.8.0+. The `rabbitmq_prometheus` plugin is built-in since RabbitMQ v3.8.0. OpenTelemetry Collector fetches metrics from `rabbitmq_prometheus` plugin via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via OpenTelemetry gRPC exporter. The SkyWalking OAP Server parses the expression with to filter/calculate/aggregate and store the results. Setup . Set up . The example for OpenTelemetry Collector configuration, refer to . Config SkyWalking . RabbitMQ monitoring provides multidimensional metrics monitoring of RabbitMQ cluster as `Layer: RABBITMQ` `Service` in the OAP. In each cluster, the nodes are represented as `Instance`. | Monitoring Panel | Metric Name | Description | Data Source | |--|--|-|-| | Memory Available Before Publishers Blocked (MB) | meterrabbitmqmemoryavailablebeforepublisherblocked | If the value is zero or less, the memory alarm will be triggered and all publishing connections across all cluster nodes will be blocked. | rabbitmq_prometheus plugin | | Disk Space Available Before Publishers Blocked (GB) | meterrabbitmqdiskspaceavailablebeforepublisherblocked | This metric is reported for the partition where the RabbitMQ data directory is stored. | rabbitmqprometheus plugin | | File Descriptors Available | meterrabbitmqfiledescriptorsavailable | When this value reaches zero, new connections will not be accepted and disk write operations may fail. | rabbitmq_prometheus plugin | | TCP Sockets Available | meterrabbitmqtcpsocketavailable | When this value reaches zero, new connections will not be accepted. | rabbitmq_prometheus plugin | | Messages Ready To Be Delivered To Consumers | meterrabbitmqmessagereadydeliveredconsumers | Total number of ready messages ready to be delivered to consumers. | rabbitmqprometheus plugin | | Messages Pending Consumer Acknowledgement | meterrabbitmqmessageunacknowledgeddeliveredconsumers | The total number of messages that are either in-flight to consumers, currently being processed by consumers or simply waiting for the consumer acknowledgements to be processed by the queue. Until the queue processes the message acknowledgement, the message will remain unacknowledged. | rabbitmqprometheus plugin | | Messages Published | meterrabbitmqmessagespublished | The incoming message rate before any routing rules are applied. | rabbitmqprometheus plugin | | Messages Confirmed To Publishers | meterrabbitmqmessagesconfirmed | The rate of messages confirmed by the broker to publishers. Publishers must opt-in to receive message confirmations. | rabbitmqprometheus plugin | | Messages Unconfirmed To Publishers | meterrabbitmqmessagesunconfirmed | The rate of messages received from publishers that have publisher confirms enabled and the broker has not confirmed yet. | rabbitmqprometheus plugin | | Messages Routed To Queues | meterrabbitmqmessagesrouted | The rate of messages received from publishers and successfully routed to the master queue replicas. | rabbitmqprometheus plugin | | Unroutable Messages Returned To Publishers | meterrabbitmqmessagesunroutablereturned | The rate of messages that cannot be routed and are returned back to publishers. | rabbitmq_prometheus plugin | | Unroutable Messages Dropped | meterrabbitmqmessagesunroutabledropped | The rate of messages that cannot be routed and are dropped. | rabbitmq_prometheus plugin | | Queues Total | meterrabbitmqqueues | Total number of queue masters per node. | rabbitmq_prometheus plugin | | Queues Declared | meterrabbitmqqueuesdeclaredtotal | The rate of queue declarations performed by clients. | rabbitmq_prometheus plugin | | Queues Created | meterrabbitmqqueuescreatedtotal | The rate of new queues created (as opposed to redeclarations). | rabbitmq_prometheus plugin | | Queues Deleted | meterrabbitmqqueuesdeletedtotal | The rate of queues

| rabbitmq_prometheus plugin | | Channels Total | meterrabbitmqchannels | Total number of channels on all currently opened connections. | rabbitmq_prometheus plugin | | Channels Opened | meterrabbitmqchannelsopenedtotal | The rate of new channels opened by applications across all connections. Channels are expected to be long-lived. | rabbitmq_prometheus plugin | | Channels Closed | meterrabbitmqchannelsclosedtotal | The rate of channels closed by applications across all connections. Channels are expected to be long-lived. | rabbitmq_prometheus plugin | | Connections Total | meterrabbitmqconnections | Total number of client connections. | rabbitmq_prometheus plugin | | Connections Opened | meterrabbitmqconnectionsopenedtotal | The rate of new connections opened by clients. Connections are expected to be long-lived. | rabbitmq_prometheus plugin | | Connections Closed | meterrabbitmqconnectionsclosedtotal | The rate of connections closed. Connections are expected to be long-lived. | rabbitmq_prometheus plugin | | Monitoring Panel | Unit | Metric Name | Description | Data Source | |-||--|--|-| | Ready Messages | | meterrabbitmqnodequeuemessagesready | Total number of ready messages ready to be delivered to consumers. | rabbitmqprometheus plugin | | Unacknowledged Messages | | meterrabbitmqnodeunacknowledgedmessages | Messages delivered to consumers but not yet acknowledged | rabbitmq_prometheus plugin | | Incoming Messages | | meterrabbitmqnodeincomingmessages | The incoming message rate before any routing rules are applied. | rabbitmq_prometheus plugin | | Outgoing Messages | | meterrabbitmqnodeoutgoingmessagestotal | The outgoing message rate before any routing rules are applied. | rabbitmqprometheus plugin | | Publishers | | meterrabbitmqnodepublishertotal | Publishers | rabbitmq_prometheus plugin | | Consumers | | meterrabbitmqnodeconsumertotal | Consumers currently connect | rabbitmq_prometheus plugin | | Collections | | meterrabbitmqnodeconnectionstotal | Connections currently open | rabbitmq_prometheus plugin | | Channels | | meterrabbitmqnodechanneltotal | Channels currently open | rabbitmq_prometheus plugin | | Queues | | meterrabbitmqnodequeuetotal | Queues available | rabbitmq_prometheus plugin | | Allocated Used | % | meterrabbitmqnodeallocatedusedpercent | Erlang VM memory utilisation from ertsalloc perspective | rabbitmq_prometheus plugin | | Allocated Unused | % | meterrabbitmqnodeallocatedunusedpercent | Erlang VM memory utilisation from ertsalloc perspective | rabbitmq_prometheus plugin | | Allocated Used | MB | meterrabbitmqnodeallocatedusedbytes | Erlang VM memory utilisation from ertsalloc perspective | rabbitmq_prometheus plugin | | Allocated Unused | MB | meterrabbitmqnodeallocatedunusedbytes | Erlang VM memory utilisation from ertsalloc perspective | rabbitmq_prometheus plugin | | Allocated Total | MB | meterrabbitmqnodeallocatedtotalbytes | Erlang VM memory utilisation from ertsalloc perspective | rabbitmq_prometheus plugin | | Resident Set Size | MB | meterrabbitmqnodeprocessresidentmemorybytes | Erlang VM Resident Set Size (RSS) As reported by the OS | rabbitmq_prometheus plugin | | Allocators | MB | meterrabbitmqnodeallocatedunusedbytes <br />meterrabbitmqnodeallocatedtotalbytes <br /> meterrabbitmqnodeprocessresidentmemorybytes | | rabbitmq_prometheus plugin | | Allocated By Type | MB | meterrabbitmqnodeallocatedbytype | Allocated by allocator type | rabbitmqprometheus plugin | | Multiblock Used | MB | meterrabbitmqnodeallocatedmultiblockused | Multi block used | rabbitmqprometheus plugin | | Multiblock Unused | MB | meterrabbitmqnodeallocatedmultiblockunused | Multi block used | rabbitmqprometheus plugin | | Multiblock Pool Used | MB | meterrabbitmqnodeallocatedmultiblockpoolused | Multi block pool used | rabbitmq_prometheus plugin | | Multiblock Pool Unused | MB | meterrabbitmqnodeallocatedmultiblockpoolunused | Multi block pool unused | rabbitmq_prometheus plugin | | Singleblock Used | MB | meterrabbitmqnodeallocatedsingleblockused | Single block used | rabbitmqprometheus plugin | | Singleblock Unused | MB | meterrabbitmqnodeallocatedsingleblockunused | Single block unused | rabbitmqprometheus plugin | You can customize your own metrics/expression/dashboard panel. The metrics definition and expression rules are found in `/config/otel-rules/rabbitmq/rabbitmq-cluster.yaml, /config/otel-rules/rabbitmq/rabbitmq-node.yaml`. The RabbitMQ dashboard panel configurations are found in `/config/ui-initialized-templates/rabbitmq`.