Repository: redpanda-data/docs
Pull request #1014: DOC-933 Document new consumer group lag metrics and configs
Author: login: JakeSCahill
Created: 2025-03-14T13:47:22Z | Merged: 2025-03-19T10:24:01Z

## Description

Resolves https://redpandadata.atlassian.net/browse/DOC-933
Partially resolves https://redpandadata.atlassian.net/browse/DOC-1115

Review deadline: March 17

## Page previews

https://deploy-preview-1014--redpanda-docs-preview.netlify.app/25.1/manage/monitoring/#consumers

## Checks

- [X] New feature
- [ ] Content gap
- [ ] Support Follow-up
- [ ] Small fix (typos, links, copyedits, etc)



Diff:
diff --git a/modules/get-started/pages/whats-new.adoc b/modules/get-started/pages/whats-new.adoc
index 9153390cfc..3c90d674ee 100644
--- a/modules/get-started/pages/whats-new.adoc
+++ b/modules/get-started/pages/whats-new.adoc
@@ -39,6 +39,22 @@ Redpanda now supports normalization of Protobuf schemas in the Schema Registry.
 
 You now can configure Kafka clients to authenticate using xref:manage:security/authentication#enable-sasl.adoc[SASL/PLAIN] with a single account using the same username and password. Unlike SASL/SCRAM, which uses a challenge response with hashed credentials, SASL/PLAIN transmits plaintext passwords. You enable SASL/PLAIN by appending `PLAIN` to the list of SASL mechanisms.
 
+== New metrics
+
+The following metrics are new in this version:
+
+=== Consumer lag gauges
+
+Redpanda can now expose dedicated consumer lag gauges that eliminate the need to calculate lag manually. These metrics provide real-time insights into consumer group performance and help identify issues. The following metrics are available:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+Reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential performance or configuration issues.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+Aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+See xref:manage:monitoring.adoc#consumers[Monitor consumer group lag] for more information.
+
 == New cluster properties
 
 The following cluster properties are new in this version:
diff --git a/modules/manage/partials/monitor-health.adoc b/modules/manage/partials/monitor-health.adoc
index a3181026a5..465788c853 100644
--- a/modules/manage/partials/monitor-health.adoc
+++ b/modules/manage/partials/monitor-health.adoc
@@ -209,13 +209,132 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag indicates how current the consumed data is relative to real-time production. A high or increasing lag means that consumers are processing messages slower than producers are generating them. A decreasing or stable lag implies that consumers are keeping pace with producers, ensuring real-time or near-real-time data consumption.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+By monitoring consumer lag, you can identify performance bottlenecks and make informed decisions about scaling consumers, tuning configurations, and improving processing efficiency.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+A high maximum lag may indicate that a consumer is experiencing connectivity problems or cannot keep up with the incoming workload.
+
+A high or increasing total lag (lag sum) suggests that the consumer group lacks sufficient resources to process messages at the rate they are produced. In such cases, scaling the number of consumers within the group can help, but only up to the number of partitions available in the topic. If lag persists despite increasing consumers, repartitioning the topic may be necessary to distribute the workload more effectively and improve processing efficiency.
+
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda brokers can internally calculate consumer group lag and expose two dedicated gauges. This method is recommended for environments where your observability platform does not support complex queries required to calculate the lag from offset metrics.
++
+Enabling these gauges may add a small amount of additional processing overhead to the brokers.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag from offset metrics. Use this method if your observability platform supports functions, such as `max()`, and you prefer to avoid additional processing overhead on the broker.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+Reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential performance or configuration issues.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+Aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+To enable these dedicated gauges, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
++
+Set this value equal to the scrape interval of your metrics collection system. Aligning these intervals ensures synchronized data collection, reducing the likelihood of missing or misaligned lag measurements.
+
+For example:
+
+ifndef::env-kubernetes[]
+[,bash]
+----
+rpk cluster config set enable_consumer_group_metrics '["group", "partition", "consumer_lag"]'
+----
+endif::[]
+
+ifdef::env-kubernetes[]
+[tabs]
+======
+Helm + Operator::
++
+--
+.`redpanda-cluster.yaml`
+[,yaml]
+----
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+spec:
+  chartRef: {}
+  clusterSpec:
+    config:
+      cluster:
+        enable_consumer_group_metrics:
+          - group
+          - partition
+          - consumer_lag
+----
+
+```bash
+kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
+```
+
+--
+Helm::
++
+--
+[tabs]
+====
+--values::
++
+.`enable-consumer-metrics.yaml`
+[,yaml]
+----
+config:
+  cluster:
+    enable_consumer_group_metrics:
+      - group
+      - partition
+      - consumer_lag
+----
++
+```bash
+helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
+--values enable-consumer-metrics.yaml --reuse-values
+```
+
+--set::
++
+[,bash]
+----
+helm upgrade --install redpanda redpanda/redpanda \
+  --namespace <namespace> \
+  --create-namespace \
+  --set config.cluster.enable_consumer_group_metrics[0]=group \
+  --set config.cluster.enable_consumer_group_metrics[1]=partition \
+  --set config.cluster.enable_consumer_group_metrics[2]=consumer_lag
+----
+
+====
+--
+======
+endif::[]
+
+
+When these properties are enabled, Redpanda computes and exposes the `redpanda_kafka_consumer_group_lag_max` and `redpanda_kafka_consumer_group_lag_sum` gauges to the `/public_metrics` endpoint.
+
+==== Offset-based calculation
+
+If your environment is sensitive to the performance overhead of the <<dedicated-gauges, dedicated gauges>>, use the offset-based calculation method to calculate consumer group lag. This method requires your observability platform to support functions like `max()`.
+
+Redpanda provides two metrics to calculate consumer group lag:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`]: The broker's latest offset for a partition.
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`]: The last committed offset for a consumer group on that partition.
+
+For example, here's a typical query to compute consumer lag:
 
 [,promql]
 ----
diff --git a/modules/reference/pages/properties/cluster-properties.adoc b/modules/reference/pages/properties/cluster-properties.adoc
index 2f9b857a48..b31c959730 100644
--- a/modules/reference/pages/properties/cluster-properties.adoc
+++ b/modules/reference/pages/properties/cluster-properties.adoc
@@ -414,6 +414,26 @@ This is an internal-only configuration and should be enabled only after consulti
 
 ---
 
+=== consumer_group_lag_collection_interval_sec
+
+How often to run the collection loop when <<enable_consumer_group_metrics,`enable_consumer_group_metrics`>> contains `consumer_lag`.
+
+Reducing the value of `consumer_group_lag_collection_interval_sec` increases the metric collection frequency, which may raise resource utilization. In most environments, this impact is minimal, but it's best practice to monitor broker resource usage in high-scale settings.
+
+*Unit:* seconds
+
+*Requires restart:* No
+
+*Visibility:* `tunable`
+
+*Type:* integer
+
+*Accepted values:* [`-17179869184`, `17179869183`]
+
+*Default:* `60`
+
+---
+
 === controller_backend_housekeeping_interval_ms
 
 Interval between iterations of controller backend housekeeping loop.
@@ -812,6 +832,54 @@ Maximum amount of time the coordinator waits to snapshot after a command appears
 
 ---
 
+=== datalake_scheduler_block_size_bytes
+
+Size, in bytes, of each memory block reserved for record translation, as tracked by the datalake scheduler.
+
+*Unit:* bytes
+
+*Requires restart:* Yes
+
+*Visibility:* `tunable`
+
+*Type:* integer
+
+*Default:* `4_mib`
+
+---
+
+=== datalake_scheduler_max_concurrent_translations
+
+The maximum number of translations that the datalake scheduler will allow to run at a given time. If a translation is requested, but the number of running translations exceeds this value, the request will be put to sleep temporarily, polling until capacity becomes available.
+
+*Requires restart:* Yes
+
+*Visibility:* `tunable`
+
+*Type:* integer
+
+*Default:* `4`
+
+---
+
+=== datalake_scheduler_time_slice_ms
+
+Time, in milliseconds, for a datalake translation as scheduled by the datalake scheduler. After a translation is scheduled, it will run until either the time specified has elapsed or all pending records on its source partition have been translated.
+
+*Unit:* milliseconds
+
+*Requires restart:* Yes
+
+*Visibility:* `tunable`
+
+*Type:* integer
+
+*Accepted values:* [`-17592186044416`, `17592186044415`]
+
+*Default:* `30000`
+
+---
+
 === debug_bundle_auto_removal_seconds
 
 If set, how long debug bundles are kept in the debug bundle storage directory after they are created. If not set, debug bundles are kept indefinitely.
@@ -1061,7 +1129,15 @@ Enables cluster metadata uploads. Required for xref:manage:whole-cluster-restore
 
 === enable_consumer_group_metrics
 
-List of enabled consumer group metrics. Accepted Values: `group`, `partition`, `consumer_lag`.
+List of enabled consumer group metrics. Accepted values include:
+
+- `group`: Enables the xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_consumers[`redpanda_kafka_consumer_group_consumers`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_topics[`redpanda_kafka_consumer_group_topics`] metrics.
+- `partition`: Enables the xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] metric.
+- `consumer_lag`: Enables the xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`] metrics
++
+Enabling `consumer_lag` may add a small amount of additional processing overhead to the brokers, especially in environments with a high number of consumer groups or partitions.
++
+Use the xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] property to control the frequency of consumer lag metric collection.
 
 *Requires restart:* No
 
@@ -1071,6 +1147,9 @@ List of enabled consumer group metrics. Accepted Values: `group`, `partition`, `
 
 *Default:* `["group", "partition"]`
 
+*Related topics*:
+
+- xref:manage:monitoring.adoc#consumers[Monitor consumer group lag]
 ---
 
 === enable_controller_log_rate_limiting
@@ -1712,6 +1791,20 @@ Default value for the `redpanda.iceberg.delete` topic property that determines i
 
 ---
 
+=== iceberg_disable_automatic_snapshot_expiry
+
+Whether to disable automatic Iceberg snapshot expiry. This property may be useful if the Iceberg catalog expects to perform snapshot expiry on its own.
+
+*Requires restart:* No
+
+*Visibility:* `user`
+
+*Type:* boolean
+
+*Default:* `false`
+
+---
+
 === iceberg_disable_snapshot_tagging
 
 Whether to disable tagging of Iceberg snapshots. These tags are used to ensure that the snapshots that Redpanda writes are retained during snapshot removal, which in turn, helps Redpanda ensure exactly-once delivery of records. Disabling tags is therefore not recommended, but may be useful if the Iceberg catalog does not support tags.
diff --git a/modules/reference/pages/public-metrics-reference.adoc b/modules/reference/pages/public-metrics-reference.adoc
index 389a703668..bff50b08c7 100644
--- a/modules/reference/pages/public-metrics-reference.adoc
+++ b/modules/reference/pages/public-metrics-reference.adoc
@@ -863,15 +863,16 @@ Histogram of client quota throughput per quota rule and type.
 
 Committed offset for a consumer group, segmented by topic and partition.
 
+To enable this metric, you must include the `partition` option in the xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`] cluster property.
+
 *Type*: gauge
 
 *Labels*:
 
-* `group`
-
-* `topic`
-
-* `partition`
+- `redpanda_group`
+- `redpanda_partition`
+- `redpanda_topic`
+- `shard`
 
 ---
 
@@ -879,11 +880,50 @@ Committed offset for a consumer group, segmented by topic and partition.
 
 Number of active consumers within a consumer group.
 
+To enable this metric, you must include the `group` option in the xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`] cluster property.
+
 *Type*: gauge
 
 *Labels*:
 
-* `group`
+- `redpanda_group`
+- `shard`
+
+---
+
+=== redpanda_kafka_consumer_group_lag_max
+
+Maximum consumer group lag across topic partitions. This metric is useful for identifying the most delayed partition in the consumer group.
+
+To enable this metric, you must include the `consumer_lag` option in the xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`] cluster property.
+
+*Type*: gauge
+
+*Labels*:
+
+- `redpanda_group`
+
+*Related topics*:
+
+- xref:manage:monitoring.adoc#consumer-group-lag[Consumer group lag]
+
+---
+
+=== redpanda_kafka_consumer_group_lag_sum
+
+Sum of consumer group lag for all topic partitions. This metric is useful for tracking the total lag across all partitions.
+
+To enable this metric, you must include the `consumer_lag` option in the xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`] cluster property.
+
+*Type*: gauge
+
+*Labels*:
+
+- `redpanda_group`
+
+*Related topics*:
+
+- xref:manage:monitoring.adoc#consumer-group-lag[Consumer group lag]
 
 ---
 
@@ -891,11 +931,14 @@ Number of active consumers within a consumer group.
 
 Number of topics being consumed by a consumer group.
 
+To enable this metric, you must include the `group` option in the xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`] cluster property.
+
 *Type*: gauge
 
 *Labels*:
 
-* `group`
+- `redpanda_group`
+- `shard`
 
 == REST proxy metrics
 
@@ -1004,7 +1047,7 @@ Build information for Redpanda, including the revision and version details.
 
 ---
 
-=== redpanda_application_fips_mode 
+=== redpanda_application_fips_mode
 
 Indicates whether Redpanda is running in FIPS mode. Possible values:
 
diff --git a/tests/docker-compose/bootstrap.yml b/tests/docker-compose/bootstrap.yml
index e94edf9433..98ab6f6a54 100644
--- a/tests/docker-compose/bootstrap.yml
+++ b/tests/docker-compose/bootstrap.yml
@@ -41,4 +41,13 @@ cloud_storage_bucket: redpanda
 cloud_storage_segment_max_upload_interval_sec: 60
 # Continuous Data Balancing (enterprise feature) continuously monitors your node and rack availability and disk usage. This enables self-healing clusters that dynamically balance partitions, ensuring smooth operations and optimal cluster performance.
 # https://docs.redpanda.com/current/manage/cluster-maintenance/continuous-data-balancing/
-partition_autobalancing_mode: continuous
\ No newline at end of file
+partition_autobalancing_mode: continuous
+# Enable Redpanda to collect consumer group metrics.
+# https://docs.redpanda.com/current/reference/properties/cluster-properties/#enable_consumer_group_metrics
+enable_consumer_group_metrics:
+  - "group"
+  - "partition"
+  - "consumer_lag"
+# Lower the interval for the quickstart
+# https://docs.redpanda.com/current/reference/properties/cluster-properties/#consumer_group_lag_collection_interval_sec
+consumer_group_lag_collection_interval_sec: 60
\ No newline at end of file
diff --git a/tests/docker-compose/docker-compose.yml b/tests/docker-compose/docker-compose.yml
index 542076f937..36c416332c 100644
--- a/tests/docker-compose/docker-compose.yml
+++ b/tests/docker-compose/docker-compose.yml
@@ -56,7 +56,8 @@ services:
       timeout: 15s
       retries: 10
     depends_on:
-      - minio
+      minio:
+        condition: service_healthy
   redpanda-1:
     command:
       - redpanda
@@ -295,6 +296,25 @@ services:
       redpanda-0:
         condition: service_healthy
   ####################
+  # rpk container to deploy a consumer group #
+  # See https://docs.redpanda.com/current/reference/rpk/rpk-topic/rpk-topic-consume/
+  ####################
+  consumergroup:
+    command:
+      - topic
+      - consume
+      - transactions
+      - --group
+      - transactions-consumer
+      - -X user=superuser
+      - -X pass=secretpassword
+      - -X brokers=redpanda-0:9092
+    image: docker.redpanda.com/redpandadata/${REDPANDA_DOCKER_REPO:-redpanda}:${REDPANDA_VERSION:-latest}
+    networks:
+      - redpanda_network
+    depends_on:
+    - deploytransform
+  ####################
   # rpk container to deploy the pre-built data transform #
   # See https://docs.redpanda.com/current/develop/data-transforms/deploy/
   ####################
@@ -350,9 +370,15 @@ services:
       redpanda_network:
         aliases:
           - redpanda.minio
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/ready"]
+      interval: 10s
+      timeout: 5s
+      retries: 3
   mc:
     depends_on:
-      - minio
+      minio:
+        condition: service_healthy
     image: minio/mc
     container_name: mc
     networks:


Reviews:
user: login: Feediver1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T14:02:33Z
user: login: Feediver1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T14:03:50Z
user: login: Feediver1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T14:05:59Z
user: login: Feediver1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T14:07:30Z
user: login: Feediver1
body: lgtm
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T14:13:47Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T14:18:01Z
user: login: BenPope
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-17T11:04:36Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T11:30:20Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T11:34:19Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T11:36:12Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T11:50:36Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T12:05:27Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T13:27:30Z
user: login: BenPope
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-17T14:02:49Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T14:52:50Z
user: login: IoannisRP
body: The behavior of `enable_consumer_group_metrics` is:
- `group` : enables `redpanda_kafka_consumer_group_consumers and `redpanda_kafka_consumer_group_topics`
- `partition` : enables `redpanda_kafka_consumer_group_committed_offset`
- `consumer_lag` : enables `redpanda_kafka_consumer_group_lag_max` and `redpanda_kafka_consumer_group_lag_sum`

The `group` and `partition` values are tagged against the wrong metrics in public-metrics-reference.adoc.

It might be helpful to add this cluster-properties.adoc, as well, as @BenPope suggested.


state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T14:57:05Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T14:58:12Z
user: login: IoannisRP
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T15:07:41Z
user: login: JakeSCahill
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T15:40:53Z
user: login: IoannisRP
body: Lgtm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T15:53:36Z
user: login: IoannisRP
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T16:53:53Z
user: login: BenPope
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-18T19:19:23Z

Review comments:
diff_hunk: @@ -209,19 +209,137 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag by combining two offset metrics: `redpanda_kafka_max_offset` and `redpanda_kafka_consumer_group_committed_offset`. This method requires complex queries to calculate lag and may not be supported in all observability platforms.
path: modules/manage/partials/monitor-health.adoc
user: login: Feediver1
body: This mentions the complexity of this method, but nothing about the benefit or use case for why you would use it over the dedicated gauge. Is there a good use case for using this one over the dedicated one?

created_at: 2025-03-14T14:02:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,19 +209,137 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag by combining two offset metrics: `redpanda_kafka_max_offset` and `redpanda_kafka_consumer_group_committed_offset`. This method requires complex queries to calculate lag and may not be supported in all observability platforms.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge reporting the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+Using these gauges eliminates the need for complicated calculations in your observability tool.
+
+To have these dedicated gauges available, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
path: modules/manage/partials/monitor-health.adoc
user: login: Feediver1
body: ```suggestion
To make these dedicated gauges available, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
```
created_at: 2025-03-14T14:03:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,19 +209,137 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag by combining two offset metrics: `redpanda_kafka_max_offset` and `redpanda_kafka_consumer_group_committed_offset`. This method requires complex queries to calculate lag and may not be supported in all observability platforms.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge reporting the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+Using these gauges eliminates the need for complicated calculations in your observability tool.
+
+To have these dedicated gauges available, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+For example:
+
+ifndef::env-kubernetes[]
+[,bash]
+----
+rpk cluster config set enable_consumer_group_metrics '["group", "partition", "consumer_lag"]'
+----
+endif::[]
+
+ifdef::env-kubernetes[]
+[tabs]
+======
+Helm + Operator::
++
+--
+.`redpanda-cluster.yaml`
+[,yaml]
+----
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+spec:
+  chartRef: {}
+  clusterSpec:
+    config:
+      cluster:
+        enable_consumer_group_metrics:
+          - group
+          - partition
+          - consumer_lag
+----
+
+```bash
+kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
+```
+
+--
+Helm::
++
+--
+[tabs]
+====
+--values::
++
+.`enable-consumer-metrics.yaml`
+[,yaml]
+----
+config:
+  cluster:
+    enable_consumer_group_metrics:
+      - group
+      - partition
+      - consumer_lag
+----
++
+```bash
+helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
+--values enable-consumer-metrics.yaml --reuse-values
+```
+
+--set::
++
+[,bash]
+----
+helm upgrade --install redpanda redpanda/redpanda \
+  --namespace <namespace> \
+  --create-namespace \
+  --set config.cluster.enable_consumer_group_metrics[0]=group \
+  --set config.cluster.enable_consumer_group_metrics[1]=partition \
+  --set config.cluster.enable_consumer_group_metrics[2]=consumer_lag
+----
+
+====
+--
+======
+endif::[]
+
+When these properties are enabled, Redpanda computes and exposes the `redpanda_kafka_consumer_group_lag_max` and `redpanda_kafka_consumer_group_lag_sum` gauges in the public metrics endpoint.
+
+Enabling these additional metrics may add extra processing overhead to the broker, especially in environments with a very high number of consumer groups or partitions.
+The increased frequency of metric collection could result in higher resource utilization.
+If your environment is sensitive to performance overhead or if you prefer to avoid any additional load, you may use the <<offset-based-calculation, offset-based calculation method>>. Otherwise, enabling these properties can simplify monitoring and improve the accuracy of consumer group lag measurements across diverse observability platforms.
+
+==== Offset-based calculation
+
+If you prefer not to enable the <<dedicated-gauges, dedicated gauges>>, you can calculate consumer group lag using offset metrics.
path: modules/manage/partials/monitor-health.adoc
user: login: Feediver1
body: Again--what are the use cases for selecting this calculation instead of dedicated?
created_at: 2025-03-14T14:05:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,19 +209,137 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag by combining two offset metrics: `redpanda_kafka_max_offset` and `redpanda_kafka_consumer_group_committed_offset`. This method requires complex queries to calculate lag and may not be supported in all observability platforms.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge reporting the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+Using these gauges eliminates the need for complicated calculations in your observability tool.
+
+To have these dedicated gauges available, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+For example:
+
+ifndef::env-kubernetes[]
+[,bash]
+----
+rpk cluster config set enable_consumer_group_metrics '["group", "partition", "consumer_lag"]'
+----
+endif::[]
+
+ifdef::env-kubernetes[]
+[tabs]
+======
+Helm + Operator::
++
+--
+.`redpanda-cluster.yaml`
+[,yaml]
+----
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+spec:
+  chartRef: {}
+  clusterSpec:
+    config:
+      cluster:
+        enable_consumer_group_metrics:
+          - group
+          - partition
+          - consumer_lag
+----
+
+```bash
+kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
+```
+
+--
+Helm::
++
+--
+[tabs]
+====
+--values::
++
+.`enable-consumer-metrics.yaml`
+[,yaml]
+----
+config:
+  cluster:
+    enable_consumer_group_metrics:
+      - group
+      - partition
+      - consumer_lag
+----
++
+```bash
+helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
+--values enable-consumer-metrics.yaml --reuse-values
+```
+
+--set::
++
+[,bash]
+----
+helm upgrade --install redpanda redpanda/redpanda \
+  --namespace <namespace> \
+  --create-namespace \
+  --set config.cluster.enable_consumer_group_metrics[0]=group \
+  --set config.cluster.enable_consumer_group_metrics[1]=partition \
+  --set config.cluster.enable_consumer_group_metrics[2]=consumer_lag
+----
+
+====
+--
+======
+endif::[]
+
+When these properties are enabled, Redpanda computes and exposes the `redpanda_kafka_consumer_group_lag_max` and `redpanda_kafka_consumer_group_lag_sum` gauges in the public metrics endpoint.
+
+Enabling these additional metrics may add extra processing overhead to the broker, especially in environments with a very high number of consumer groups or partitions.
+The increased frequency of metric collection could result in higher resource utilization.
+If your environment is sensitive to performance overhead or if you prefer to avoid any additional load, you may use the <<offset-based-calculation, offset-based calculation method>>. Otherwise, enabling these properties can simplify monitoring and improve the accuracy of consumer group lag measurements across diverse observability platforms.
+
+==== Offset-based calculation
+
+If you prefer not to enable the <<dedicated-gauges, dedicated gauges>>, you can calculate consumer group lag using offset metrics.
+
+Redpanda provides two metrics to calculate consumer group lag:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`]: The broker's latest offset for a partition.
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`]: The last committed offset for a consumer group on that partition.
+
+For example, here's a typical query to compute consumer lag:
 
 [,promql]
 ----
 max by(redpanda_namespace, redpanda_topic, redpanda_partition)(redpanda_kafka_max_offset{redpanda_namespace="kafka"}) - on(redpanda_topic, redpanda_partition) group_right max by(redpanda_group, redpanda_topic, redpanda_partition)(redpanda_kafka_consumer_group_committed_offset)
 ----
 
+This approach works well if your observability platform supports functions, such as `max()`, on calculated values. However, it may not be suitable for all platforms, as some may not support complex queries or functions. In such cases, consider enabling the <<dedicated gauges, dedicated gauges>> for a simpler and more accurate monitoring experience.
path: modules/manage/partials/monitor-health.adoc
user: login: Feediver1
body: Seems like you already mentioned that is may not be suitable up above. This paragraph seems to duplicate some of what is said earlier. I'm keen to know the use cases where this is preferable to dedicated. 
created_at: 2025-03-14T14:07:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,19 +209,137 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag by combining two offset metrics: `redpanda_kafka_max_offset` and `redpanda_kafka_consumer_group_committed_offset`. This method requires complex queries to calculate lag and may not be supported in all observability platforms.
path: modules/manage/partials/monitor-health.adoc
user: login: JakeSCahill
body: Updated, thanks 👍 
created_at: 2025-03-14T14:18:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries. However, enabling these gauges may add extra processing overhead to the broker.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag. Use this method if your observability platform supports functions, such as `max()`, and you prefer to avoid additional processing overhead on the broker.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge that reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+To enable these dedicated gauges, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
path: modules/manage/partials/monitor-health.adoc
user: login: BenPope
body: Might suggest that setting this the same as the scrape interval is a good default.
created_at: 2025-03-14T14:37:46Z
author_association: MEMBER
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
path: modules/manage/partials/monitor-health.adoc
user: login: BenPope
body: I think being more direct is better, perhaps something like:
```suggestion
A high maximum lag suggests that a consumer might be having a problem such as connectivity or inability to keep up with the workload.

A high or increasing lag sum may indicate the the group has insufficient resources to process the incoming produced workload. In such cases, it may be possible to increase the number of consumers within the group (up to the number of partitions the group is consuming). If there is still lag, you may need to repartition topics.

created_at: 2025-03-14T14:48:28Z
author_association: MEMBER
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
path: modules/manage/partials/monitor-health.adoc
user: login: BenPope
body: An increasing lag would imply falling behind producers. A reducing lag would mean catching up with producers.

The point being that the group may be new.
created_at: 2025-03-17T10:28:14Z
author_association: MEMBER
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries. However, enabling these gauges may add extra processing overhead to the broker.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag. Use this method if your observability platform supports functions, such as `max()`, and you prefer to avoid additional processing overhead on the broker.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge that reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+To enable these dedicated gauges, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+For example:
+
+ifndef::env-kubernetes[]
+[,bash]
+----
+rpk cluster config set enable_consumer_group_metrics '["group", "partition", "consumer_lag"]'
+----
+endif::[]
+
+ifdef::env-kubernetes[]
+[tabs]
+======
+Helm + Operator::
++
+--
+.`redpanda-cluster.yaml`
+[,yaml]
+----
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+spec:
+  chartRef: {}
+  clusterSpec:
+    config:
+      cluster:
+        enable_consumer_group_metrics:
+          - group
+          - partition
+          - consumer_lag
+----
+
+```bash
+kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
+```
+
+--
+Helm::
++
+--
+[tabs]
+====
+--values::
++
+.`enable-consumer-metrics.yaml`
+[,yaml]
+----
+config:
+  cluster:
+    enable_consumer_group_metrics:
+      - group
+      - partition
+      - consumer_lag
+----
++
+```bash
+helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
+--values enable-consumer-metrics.yaml --reuse-values
+```
+
+--set::
++
+[,bash]
+----
+helm upgrade --install redpanda redpanda/redpanda \
+  --namespace <namespace> \
+  --create-namespace \
+  --set config.cluster.enable_consumer_group_metrics[0]=group \
+  --set config.cluster.enable_consumer_group_metrics[1]=partition \
+  --set config.cluster.enable_consumer_group_metrics[2]=consumer_lag
+----
+
+====
+--
+======
+endif::[]
+
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions.
+The increased frequency of metric collection could result in higher resource utilization. Monitor the broker's resource usage after enabling these properties to ensure that the broker can handle the additional load.
+
+When these properties are enabled, Redpanda computes and exposes the `redpanda_kafka_consumer_group_lag_max` and `redpanda_kafka_consumer_group_lag_sum` gauges in the public metrics endpoint.
+
+==== Offset-based calculation
+
+If your environment is sensitive to the performance overhead of the <<dedicated-gauges, dedicated gauges>>, use the offset-based calculation method to calculate consumer group lag. This method requires your observability platform to support functions like `max()`.
+
+Redpanda provides two metrics to calculate consumer group lag:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`]: The broker's latest offset for a partition.
path: modules/manage/partials/monitor-health.adoc
user: login: BenPope
body: This is enabled by including `partition` in `enable_consumer_group_metrics`
created_at: 2025-03-17T10:34:04Z
author_association: MEMBER
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries. However, enabling these gauges may add extra processing overhead to the broker.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag. Use this method if your observability platform supports functions, such as `max()`, and you prefer to avoid additional processing overhead on the broker.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge that reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+To enable these dedicated gauges, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+For example:
+
+ifndef::env-kubernetes[]
+[,bash]
+----
+rpk cluster config set enable_consumer_group_metrics '["group", "partition", "consumer_lag"]'
+----
+endif::[]
+
+ifdef::env-kubernetes[]
+[tabs]
+======
+Helm + Operator::
++
+--
+.`redpanda-cluster.yaml`
+[,yaml]
+----
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+spec:
+  chartRef: {}
+  clusterSpec:
+    config:
+      cluster:
+        enable_consumer_group_metrics:
+          - group
+          - partition
+          - consumer_lag
+----
+
+```bash
+kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
+```
+
+--
+Helm::
++
+--
+[tabs]
+====
+--values::
++
+.`enable-consumer-metrics.yaml`
+[,yaml]
+----
+config:
+  cluster:
+    enable_consumer_group_metrics:
+      - group
+      - partition
+      - consumer_lag
+----
++
+```bash
+helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
+--values enable-consumer-metrics.yaml --reuse-values
+```
+
+--set::
++
+[,bash]
+----
+helm upgrade --install redpanda redpanda/redpanda \
+  --namespace <namespace> \
+  --create-namespace \
+  --set config.cluster.enable_consumer_group_metrics[0]=group \
+  --set config.cluster.enable_consumer_group_metrics[1]=partition \
+  --set config.cluster.enable_consumer_group_metrics[2]=consumer_lag
+----
+
+====
+--
+======
+endif::[]
+
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions.
+The increased frequency of metric collection could result in higher resource utilization. Monitor the broker's resource usage after enabling these properties to ensure that the broker can handle the additional load.
+
+When these properties are enabled, Redpanda computes and exposes the `redpanda_kafka_consumer_group_lag_max` and `redpanda_kafka_consumer_group_lag_sum` gauges in the public metrics endpoint.
+
+==== Offset-based calculation
+
+If your environment is sensitive to the performance overhead of the <<dedicated-gauges, dedicated gauges>>, use the offset-based calculation method to calculate consumer group lag. This method requires your observability platform to support functions like `max()`.
+
+Redpanda provides two metrics to calculate consumer group lag:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`]: The broker's latest offset for a partition.
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`]: The last committed offset for a consumer group on that partition.
path: modules/manage/partials/monitor-health.adoc
user: login: BenPope
body: This is enabled by including `group` in `enable_consumer_group_metrics`
created_at: 2025-03-17T10:34:15Z
author_association: MEMBER
diff_hunk: @@ -414,6 +414,24 @@ This is an internal-only configuration and should be enabled only after consulti
 
 ---
 
+=== consumer_group_lag_collection_interval_sec
+
+How often to run the collection loop when <<enable_consumer_group_metrics,`enable_consumer_group_metrics`>> contains `consumer_lag`.
+
+*Unit:* seconds
+
+*Requires restart:* No
+
+*Visibility:* `tunable`
+
+*Type:* integer
+
+*Accepted values:* [`-17179869184`, `17179869183`]
path: modules/reference/pages/properties/cluster-properties.adoc
user: login: BenPope
body: These bounds are wild, maybe I should make it bounded.
created_at: 2025-03-17T10:56:24Z
author_association: MEMBER
diff_hunk: @@ -1063,6 +1129,9 @@ Enables cluster metadata uploads. Required for xref:manage:whole-cluster-restore
 
 List of enabled consumer group metrics. Accepted Values: `group`, `partition`, `consumer_lag`.
 
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions.
+The increased frequency of metric collection could result in higher resource utilization. See xref:manage:monitoring.adoc#consumers[Consumer group lag] for more information.
path: modules/reference/pages/properties/cluster-properties.adoc
user: login: BenPope
body: "The increased frequency of metric collection" applies more to `consumer_group_lag_collection_interval_sec` than this one,
created_at: 2025-03-17T10:57:59Z
author_association: MEMBER
diff_hunk: @@ -883,7 +882,52 @@ Number of active consumers within a consumer group.
 
 *Labels*:
 
-* `group`
+- `redpanda_group`
+- `shard`
+
+---
+
+=== redpanda_kafka_consumer_group_lag_max
+
+Maximum consumer group lag across topic partitions. This metric is useful for identifying the most delayed consumer group.
+
+To enable this metric, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+*Type*: gauge
+
+*Labels*:
+
+- `redpanda_group`
+- `shard`
path: modules/reference/pages/public-metrics-reference.adoc
user: login: BenPope
body: [I'm going to eliminate shard for this one](https://github.com/redpanda-data/redpanda/pull/25383)
created_at: 2025-03-17T11:02:38Z
author_association: MEMBER
diff_hunk: @@ -883,7 +882,52 @@ Number of active consumers within a consumer group.
 
 *Labels*:
 
-* `group`
+- `redpanda_group`
+- `shard`
+
+---
+
+=== redpanda_kafka_consumer_group_lag_max
+
+Maximum consumer group lag across topic partitions. This metric is useful for identifying the most delayed consumer group.
+
+To enable this metric, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+*Type*: gauge
+
+*Labels*:
+
+- `redpanda_group`
+- `shard`
+
+*Related topics*:
+
+- xref:manage:monitoring.adoc#consumer-group-lag[Consumer group lag]
+
+---
+
+=== redpanda_kafka_consumer_group_lag_sum
+
+Sum of consumer group lag for all topic partitions. This metric is useful for tracking the total lag across all partitions.
+
+To enable this metric, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+*Type*: gauge
+
+*Labels*:
+
+- `redpanda_group`
+- `shard`
path: modules/reference/pages/public-metrics-reference.adoc
user: login: BenPope
body: [I'm going to eliminate shard for this one](https://github.com/redpanda-data/redpanda/pull/25383)
created_at: 2025-03-17T11:02:51Z
author_association: MEMBER
diff_hunk: @@ -41,4 +41,13 @@ cloud_storage_bucket: redpanda
 cloud_storage_segment_max_upload_interval_sec: 60
 # Continuous Data Balancing (enterprise feature) continuously monitors your node and rack availability and disk usage. This enables self-healing clusters that dynamically balance partitions, ensuring smooth operations and optimal cluster performance.
 # https://docs.redpanda.com/current/manage/cluster-maintenance/continuous-data-balancing/
-partition_autobalancing_mode: continuous
\ No newline at end of file
+partition_autobalancing_mode: continuous
+# Enable Redpanda to collect consumer group metrics.
+# https://docs.redpanda.com/current/reference/properties/cluster-properties/#enable_consumer_group_metrics
+enable_consumer_group_metrics:
+  - "group"
+  - "partition"
+  - "consumer_lag"
+# Lower the interval for the quickstart
+# https://docs.redpanda.com/current/reference/properties/cluster-properties/#consumer_group_lag_collection_interval_sec
+consumer_group_lag_collection_interval_sec: 5
path: tests/docker-compose/bootstrap.yml
user: login: BenPope
body: 5s is pretty often, I'd rather not have that written down somewhere!
```suggestion
consumer_group_lag_collection_interval_sec: 60
```
created_at: 2025-03-17T11:04:19Z
author_association: MEMBER
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
path: modules/manage/partials/monitor-health.adoc
user: login: JakeSCahill
body: ```suggestion
Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag indicates how current the consumed data is relative to real-time production. A high or increasing lag means that consumers are processing messages slower than producers are generating them. A decreasing or stable lag implies that consumers are keeping pace with producers, ensuring real-time or near-real-time data consumption.

Monitoring consumer lag is essential for identifying performance bottlenecks, optimizing consumer throughput, and maintaining overall system health. By analyzing lag trends, you can make informed decisions about scaling consumers, tuning configurations, and improving processing efficiency.
```
created_at: 2025-03-17T11:30:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
path: modules/manage/partials/monitor-health.adoc
user: login: JakeSCahill
body: ```suggestion
A high maximum lag may indicate that a consumer is experiencing connectivity problems or cannot keep up with the incoming workload.

A high or increasing total lag (lag sum) suggests that the consumer group lacks sufficient resources to process messages at the rate they are produced. In such cases, scaling the number of consumers within the group can help, but only up to the number of partitions available in the topic. If lag persists despite increasing consumers, repartitioning the topic may be necessary to distribute the workload more effectively and improve processing efficiency.
```
created_at: 2025-03-17T11:34:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries. However, enabling these gauges may add extra processing overhead to the broker.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag. Use this method if your observability platform supports functions, such as `max()`, and you prefer to avoid additional processing overhead on the broker.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge that reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+To enable these dedicated gauges, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
path: modules/manage/partials/monitor-health.adoc
user: login: JakeSCahill
body: ```suggestion
- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
Set this value equal to the scrape interval of your metrics collection system. Aligning these intervals ensures synchronized data collection, reducing the likelihood of missing or misaligned lag measurements.
```
created_at: 2025-03-17T11:36:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,13 +209,126 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag is a performance indicator of how fresh the data being consumed is. A higher lag means consumers are falling behind producers.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+While a higher lag may be acceptable for archival consumers, a high lag for real-time consumers suggests that they might be overloaded. In such cases, you may need to repartition topics or increase the number of consumers to distribute the load more evenly.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries. However, enabling these gauges may add extra processing overhead to the broker.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag. Use this method if your observability platform supports functions, such as `max()`, and you prefer to avoid additional processing overhead on the broker.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge that reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+To enable these dedicated gauges, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+For example:
+
+ifndef::env-kubernetes[]
+[,bash]
+----
+rpk cluster config set enable_consumer_group_metrics '["group", "partition", "consumer_lag"]'
+----
+endif::[]
+
+ifdef::env-kubernetes[]
+[tabs]
+======
+Helm + Operator::
++
+--
+.`redpanda-cluster.yaml`
+[,yaml]
+----
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+spec:
+  chartRef: {}
+  clusterSpec:
+    config:
+      cluster:
+        enable_consumer_group_metrics:
+          - group
+          - partition
+          - consumer_lag
+----
+
+```bash
+kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
+```
+
+--
+Helm::
++
+--
+[tabs]
+====
+--values::
++
+.`enable-consumer-metrics.yaml`
+[,yaml]
+----
+config:
+  cluster:
+    enable_consumer_group_metrics:
+      - group
+      - partition
+      - consumer_lag
+----
++
+```bash
+helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
+--values enable-consumer-metrics.yaml --reuse-values
+```
+
+--set::
++
+[,bash]
+----
+helm upgrade --install redpanda redpanda/redpanda \
+  --namespace <namespace> \
+  --create-namespace \
+  --set config.cluster.enable_consumer_group_metrics[0]=group \
+  --set config.cluster.enable_consumer_group_metrics[1]=partition \
+  --set config.cluster.enable_consumer_group_metrics[2]=consumer_lag
+----
+
+====
+--
+======
+endif::[]
+
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions.
+The increased frequency of metric collection could result in higher resource utilization. Monitor the broker's resource usage after enabling these properties to ensure that the broker can handle the additional load.
+
+When these properties are enabled, Redpanda computes and exposes the `redpanda_kafka_consumer_group_lag_max` and `redpanda_kafka_consumer_group_lag_sum` gauges in the public metrics endpoint.
+
+==== Offset-based calculation
+
+If your environment is sensitive to the performance overhead of the <<dedicated-gauges, dedicated gauges>>, use the offset-based calculation method to calculate consumer group lag. This method requires your observability platform to support functions like `max()`.
+
+Redpanda provides two metrics to calculate consumer group lag:
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`]: The broker's latest offset for a partition.
path: modules/manage/partials/monitor-health.adoc
user: login: JakeSCahill
body: Added to the metrics reference doc since both `partition` and `group` are assigned by default.
created_at: 2025-03-17T11:50:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,13 +209,132 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag indicates how current the consumed data is relative to real-time production. A high or increasing lag means that consumers are processing messages slower than producers are generating them. A decreasing or stable lag implies that consumers are keeping pace with producers, ensuring real-time or near-real-time data consumption.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+Monitoring consumer lag is essential for identifying performance bottlenecks, optimizing consumer throughput, and maintaining overall system health. By analyzing lag trends, you can make informed decisions about scaling consumers, tuning configurations, and improving processing efficiency.
path: modules/manage/partials/monitor-health.adoc
user: login: JakeSCahill
body: ```suggestion
By monitoring consumer lag, you can identify performance bottlenecks and make informed decisions about scaling consumers, tuning configurations, and improving processing efficiency.
```
created_at: 2025-03-17T12:05:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -883,7 +882,52 @@ Number of active consumers within a consumer group.
 
 *Labels*:
 
-* `group`
+- `redpanda_group`
+- `shard`
+
+---
+
+=== redpanda_kafka_consumer_group_lag_max
+
+Maximum consumer group lag across topic partitions. This metric is useful for identifying the most delayed consumer group.
+
+To enable this metric, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
+
+*Type*: gauge
+
+*Labels*:
+
+- `redpanda_group`
+- `shard`
path: modules/reference/pages/public-metrics-reference.adoc
user: login: JakeSCahill
body: Removed
created_at: 2025-03-17T13:27:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -39,6 +39,22 @@ Redpanda now supports normalization of Protobuf schemas in the Schema Registry.
 
 You now can configure Kafka clients to authenticate using xref:manage:security/authentication#enable-sasl.adoc[SASL/PLAIN] with a single account using the same username and password. Unlike SASL/SCRAM, which uses a challenge response with hashed credentials, SASL/PLAIN transmits plaintext passwords. You enable SASL/PLAIN by appending `PLAIN` to the list of SASL mechanisms.
 
+== New metrics
+
+The following metrics are new in this version:
+
+=== Consumer lag gauges
+
+Redpanda now exposes dedicated consumer lag gauges that are computed internally by the broker. These metrics eliminate the need for complex calculations based on offset metrics and work with observability platforms that lack advanced query support.
path: modules/get-started/pages/whats-new.adoc
user: login: BenPope
body: This is not the default, so perhaps "can now expose"
created_at: 2025-03-17T13:47:09Z
author_association: MEMBER
diff_hunk: @@ -39,6 +39,22 @@ Redpanda now supports normalization of Protobuf schemas in the Schema Registry.
 
 You now can configure Kafka clients to authenticate using xref:manage:security/authentication#enable-sasl.adoc[SASL/PLAIN] with a single account using the same username and password. Unlike SASL/SCRAM, which uses a challenge response with hashed credentials, SASL/PLAIN transmits plaintext passwords. You enable SASL/PLAIN by appending `PLAIN` to the list of SASL mechanisms.
 
+== New metrics
+
+The following metrics are new in this version:
+
+=== Consumer lag gauges
+
+Redpanda now exposes dedicated consumer lag gauges that are computed internally by the broker. These metrics eliminate the need for complex calculations based on offset metrics and work with observability platforms that lack advanced query support.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge that reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
path: modules/get-started/pages/whats-new.adoc
user: login: BenPope
body: It might be more than a bottleneck, perhaps "problems" or "issues".
created_at: 2025-03-17T13:48:05Z
author_association: MEMBER
diff_hunk: @@ -209,13 +209,132 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag indicates how current the consumed data is relative to real-time production. A high or increasing lag means that consumers are processing messages slower than producers are generating them. A decreasing or stable lag implies that consumers are keeping pace with producers, ensuring real-time or near-real-time data consumption.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+By monitoring consumer lag, you can identify performance bottlenecks and make informed decisions about scaling consumers, tuning configurations, and improving processing efficiency.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+A high maximum lag may indicate that a consumer is experiencing connectivity problems or cannot keep up with the incoming workload.
+
+A high or increasing total lag (lag sum) suggests that the consumer group lacks sufficient resources to process messages at the rate they are produced. In such cases, scaling the number of consumers within the group can help, but only up to the number of partitions available in the topic. If lag persists despite increasing consumers, repartitioning the topic may be necessary to distribute the workload more effectively and improve processing efficiency.
+
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries. However, enabling these gauges may add extra processing overhead to the broker.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag. Use this method if your observability platform supports functions, such as `max()`, and you prefer to avoid additional processing overhead on the broker.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge that reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+To enable these dedicated gauges, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
++
+Set this value equal to the scrape interval of your metrics collection system. Aligning these intervals ensures synchronized data collection, reducing the likelihood of missing or misaligned lag measurements.
+
+For example:
+
+ifndef::env-kubernetes[]
+[,bash]
+----
+rpk cluster config set enable_consumer_group_metrics '["group", "partition", "consumer_lag"]'
+----
+endif::[]
+
+ifdef::env-kubernetes[]
+[tabs]
+======
+Helm + Operator::
++
+--
+.`redpanda-cluster.yaml`
+[,yaml]
+----
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+spec:
+  chartRef: {}
+  clusterSpec:
+    config:
+      cluster:
+        enable_consumer_group_metrics:
+          - group
+          - partition
+          - consumer_lag
+----
+
+```bash
+kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
+```
+
+--
+Helm::
++
+--
+[tabs]
+====
+--values::
++
+.`enable-consumer-metrics.yaml`
+[,yaml]
+----
+config:
+  cluster:
+    enable_consumer_group_metrics:
+      - group
+      - partition
+      - consumer_lag
+----
++
+```bash
+helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
+--values enable-consumer-metrics.yaml --reuse-values
+```
+
+--set::
++
+[,bash]
+----
+helm upgrade --install redpanda redpanda/redpanda \
+  --namespace <namespace> \
+  --create-namespace \
+  --set config.cluster.enable_consumer_group_metrics[0]=group \
+  --set config.cluster.enable_consumer_group_metrics[1]=partition \
+  --set config.cluster.enable_consumer_group_metrics[2]=consumer_lag
+----
+
+====
+--
+======
+endif::[]
+
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions.
+The lower the value of `consumer_group_lag_collection_interval_sec`, the higher the frequency of metric collection, which could result in higher resource utilization. Monitor the broker's resource usage after enabling these properties to ensure that the broker can handle the additional load.
path: modules/manage/partials/monitor-health.adoc
user: login: BenPope
body: This feels a little overstated, the overhead should be pretty minimal, and not likely to be any more than using an external tool like Burrow.
created_at: 2025-03-17T13:54:03Z
author_association: MEMBER
diff_hunk: @@ -1063,6 +1131,8 @@ Enables cluster metadata uploads. Required for xref:manage:whole-cluster-restore
 
 List of enabled consumer group metrics. Accepted Values: `group`, `partition`, `consumer_lag`.
 
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions. See xref:manage:monitoring.adoc#consumers[Consumer group lag] for more information.
path: modules/reference/pages/properties/cluster-properties.adoc
user: login: BenPope
body: I wonder if a brief description of each of these options is useful here?
created_at: 2025-03-17T13:59:21Z
author_association: MEMBER
diff_hunk: @@ -883,7 +886,50 @@ Number of active consumers within a consumer group.
 
 *Labels*:
 
-* `group`
+- `redpanda_group`
+- `shard`
+
+---
+
+=== redpanda_kafka_consumer_group_lag_max
+
+Maximum consumer group lag across topic partitions. This metric is useful for identifying the most delayed consumer group.
path: modules/reference/pages/public-metrics-reference.adoc
user: login: BenPope
body: It's the most delayed partition in the group
created_at: 2025-03-17T14:00:39Z
author_association: MEMBER
diff_hunk: @@ -863,15 +865,16 @@ Histogram of client quota throughput per quota rule and type.
 
 Committed offset for a consumer group, segmented by topic and partition.
 
+To enable this metric, you must include the `group` option in the xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`] cluster property.
+
path: modules/reference/pages/public-metrics-reference.adoc
user: login: IoannisRP
body: `redpanda_kafka_consumer_group_committed_offset` is being controlled by `partition` property.
created_at: 2025-03-17T14:35:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -607,6 +607,8 @@ Histogram capturing the latency (in seconds) for Schema Registry requests.
 
 High watermark offset for a partition, used to calculate consumer group lag.
 
+To enable this metric, you must include the `partition` option in the xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`] cluster property.
+
path: modules/reference/pages/public-metrics-reference.adoc
user: login: IoannisRP
body: I think this is misplaced.
created_at: 2025-03-17T14:50:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1063,6 +1131,8 @@ Enables cluster metadata uploads. Required for xref:manage:whole-cluster-restore
 
 List of enabled consumer group metrics. Accepted Values: `group`, `partition`, `consumer_lag`.
 
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions. See xref:manage:monitoring.adoc#consumers[Consumer group lag] for more information.
path: modules/reference/pages/properties/cluster-properties.adoc
user: login: JakeSCahill
body: That would be nice. Please could you advise what those descriptions should include?
created_at: 2025-03-17T14:52:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -209,13 +209,132 @@ Leaderless partitions can be caused by unresponsive brokers. When an alert on `r
 
 Redpanda's Raft implementation exchanges periodic status RPCs between a broker and its peers. The xref:reference:public-metrics-reference.adoc#redpanda_node_status_rpcs_timed_out[`redpanda_node_status_rpcs_timed_out`] gauge increases when a status RPC times out for a peer, which indicates that a peer may be unresponsive and may lead to problems with partition replication that Raft manages. Monitor for non-zero values of this gauge, and correlate it with any logged errors or changes in partition replication.
 
-=== Consumers
+[[consumers]]
+=== Consumer group lag
 
-==== Consumer group lag
+Consumer group lag is an important performance indicator that measures the difference between the broker's latest (max) offset and the consumer group's last committed offset. The lag indicates how current the consumed data is relative to real-time production. A high or increasing lag means that consumers are processing messages slower than producers are generating them. A decreasing or stable lag implies that consumers are keeping pace with producers, ensuring real-time or near-real-time data consumption.
 
-When working with Kafka consumer groups, the consumer group lag&mdash;the difference between the broker's latest (max) offset and the group's last committed offset&mdash;is a performance indicator of how fresh the data being consumed is. While higher lag for archival consumers is expected, high lag for real-time consumers could indicate that the consumers are overloaded and thus may need their topics to be partitioned more, or to spread the load to more consumers.
+By monitoring consumer lag, you can identify performance bottlenecks and make informed decisions about scaling consumers, tuning configurations, and improving processing efficiency.
 
-To monitor consumer group lag, create a query with the xref:reference:public-metrics-reference.adoc#redpanda_kafka_max_offset[`redpanda_kafka_max_offset`] and xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_committed_offset[`redpanda_kafka_consumer_group_committed_offset`] gauges:
+A high maximum lag may indicate that a consumer is experiencing connectivity problems or cannot keep up with the incoming workload.
+
+A high or increasing total lag (lag sum) suggests that the consumer group lacks sufficient resources to process messages at the rate they are produced. In such cases, scaling the number of consumers within the group can help, but only up to the number of partitions available in the topic. If lag persists despite increasing consumers, repartitioning the topic may be necessary to distribute the workload more effectively and improve processing efficiency.
+
+Redpanda provides the following methods for monitoring consumer group lag:
+
+- <<dedicated-gauges, Dedicated gauges>>: Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges. These gauges simplify monitoring by providing direct lag metrics without the need for complex calculations. This method is recommended for environments where observability platforms may not support complex queries. However, enabling these gauges may add extra processing overhead to the broker.
+- <<offset-based-calculation, Offset-based calculation>>: You can use your observability platform to calculate consumer group lag. Use this method if your observability platform supports functions, such as `max()`, and you prefer to avoid additional processing overhead on the broker.
+
+==== Dedicated gauges
+
+Redpanda can internally calculate consumer group lag and expose it as two dedicated gauges.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_max[`redpanda_kafka_consumer_group_lag_max`]:
+A gauge that reports the maximum lag observed among all partitions for a consumer group. This metric helps pinpoint the partition with the greatest delay, indicating potential bottlenecks.
+
+- xref:reference:public-metrics-reference.adoc#redpanda_kafka_consumer_group_lag_sum[`redpanda_kafka_consumer_group_lag_sum`]:
+A gauge that aggregates the lag across all partitions, providing an overall view of data consumption delay for the consumer group.
+
+To enable these dedicated gauges, you must enable consumer group metrics in your cluster properties. Add the following settings to your Redpanda configuration:
+
+- xref:reference:properties/cluster-properties.adoc#enable_consumer_group_metrics[`enable_consumer_group_metrics`]: A list of properties to enable for consumer group metrics. You must add the `consumer_lag` property to enable consumer group lag metrics.
+- xref:reference:properties/cluster-properties.adoc#consumer_group_lag_collection_interval_sec[`consumer_group_lag_collection_interval_sec`] (optional): The interval in seconds for collecting consumer group lag metrics. The default is 60 seconds.
++
+Set this value equal to the scrape interval of your metrics collection system. Aligning these intervals ensures synchronized data collection, reducing the likelihood of missing or misaligned lag measurements.
+
+For example:
+
+ifndef::env-kubernetes[]
+[,bash]
+----
+rpk cluster config set enable_consumer_group_metrics '["group", "partition", "consumer_lag"]'
+----
+endif::[]
+
+ifdef::env-kubernetes[]
+[tabs]
+======
+Helm + Operator::
++
+--
+.`redpanda-cluster.yaml`
+[,yaml]
+----
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+spec:
+  chartRef: {}
+  clusterSpec:
+    config:
+      cluster:
+        enable_consumer_group_metrics:
+          - group
+          - partition
+          - consumer_lag
+----
+
+```bash
+kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
+```
+
+--
+Helm::
++
+--
+[tabs]
+====
+--values::
++
+.`enable-consumer-metrics.yaml`
+[,yaml]
+----
+config:
+  cluster:
+    enable_consumer_group_metrics:
+      - group
+      - partition
+      - consumer_lag
+----
++
+```bash
+helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
+--values enable-consumer-metrics.yaml --reuse-values
+```
+
+--set::
++
+[,bash]
+----
+helm upgrade --install redpanda redpanda/redpanda \
+  --namespace <namespace> \
+  --create-namespace \
+  --set config.cluster.enable_consumer_group_metrics[0]=group \
+  --set config.cluster.enable_consumer_group_metrics[1]=partition \
+  --set config.cluster.enable_consumer_group_metrics[2]=consumer_lag
+----
+
+====
+--
+======
+endif::[]
+
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions.
+The lower the value of `consumer_group_lag_collection_interval_sec`, the higher the frequency of metric collection, which could result in higher resource utilization. Monitor the broker's resource usage after enabling these properties to ensure that the broker can handle the additional load.
path: modules/manage/partials/monitor-health.adoc
user: login: JakeSCahill
body: In that case, should we recommend that users enable `consumer_lag` instead of calculating lag themselves?

Why would users choose not to? Is this not enabled by default for some reason?
created_at: 2025-03-17T14:58:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1063,6 +1131,8 @@ Enables cluster metadata uploads. Required for xref:manage:whole-cluster-restore
 
 List of enabled consumer group metrics. Accepted Values: `group`, `partition`, `consumer_lag`.
 
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions. See xref:manage:monitoring.adoc#consumers[Consumer group lag] for more information.
path: modules/reference/pages/properties/cluster-properties.adoc
user: login: IoannisRP
body: https://github.com/redpanda-data/docs/pull/1014#pullrequestreview-2690772141

i think something along these lines should be sufficient?
created_at: 2025-03-17T15:07:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1063,6 +1131,8 @@ Enables cluster metadata uploads. Required for xref:manage:whole-cluster-restore
 
 List of enabled consumer group metrics. Accepted Values: `group`, `partition`, `consumer_lag`.
 
+Enabling `consumer_lag` may add extra processing overhead to the broker, especially in environments with a high number of consumer groups or partitions. See xref:manage:monitoring.adoc#consumers[Consumer group lag] for more information.
path: modules/reference/pages/properties/cluster-properties.adoc
user: login: JakeSCahill
body: Thanks - updated!
created_at: 2025-03-17T15:40:53Z
author_association: CONTRIBUTOR

PR comments:
user: login: netlify[bot]
created_at: 2025-03-14T13:47:42Z
body: ### <span aria-hidden="true">✅</span> Deploy Preview for *redpanda-docs-preview* ready!


|  Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 24e87e5922040a6dd2e6ae1043047cd0f906ce8b |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/redpanda-docs-preview/deploys/67d841b8d3aeac0008f9f346 |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-1014--redpanda-docs-preview.netlify.app](https://deploy-preview-1014--redpanda-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br />![QR Code](https://app.netlify.com/qr-code/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cmwiOiJodHRwczovL2RlcGxveS1wcmV2aWV3LTEwMTQtLXJlZHBhbmRhLWRvY3MtcHJldmlldy5uZXRsaWZ5LmFwcCJ9.wXCT4FS3WgOcxFv5MSqWAxGJQB1OUR2FxkhXuSJ1YiY)<br /><br />_Use your smartphone camera to open QR code link._</details> |
---
<!-- [redpanda-docs-preview Preview](https://deploy-preview-1014--redpanda-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify site configuration](https://app.netlify.com/sites/redpanda-docs-preview/configuration/notifications#deploy-webhooks)._
author_association: NONE
user: login: hyperlint-ai-deprecated[bot]
created_at: 2025-03-14T13:51:58Z
body: ## PR Change Summary

Documented new consumer group lag metrics and configurations in the Redpanda documentation.
- Introduced consumer group lag as a key performance indicator for monitoring data freshness.
- Added dedicated gauges for consumer group lag metrics to simplify monitoring.
- Updated configuration properties to enable consumer group lag metrics collection.
- Provided examples for enabling consumer group metrics in both Kubernetes and Helm environments.


**Modified Files**

- `modules/manage/partials/monitor-health.adoc`
- `modules/reference/pages/properties/cluster-properties.adoc`
- `modules/reference/pages/public-metrics-reference.adoc`


 







---

<details>

<summary>How can I customize these reviews?</summary>

Check out the [Hyperlint AI Reviewer docs](https://docs.hyperlint.com/guides/ai-reviewer/) for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the `hyperlint-ignore` label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add `hyperlint-ignore` to the PR to ignore the link check for this PR.
</details>

<details>
<summary>What is Hyperlint?</summary>

[Hyperlint](https://hyperlint.com) is an AI agent that helps you write, edit, and maintain your documentation.

Learn more about the [Hyperlint AI reviewer](https://docs.hyperlint.com/guides/ai-reviewer/) and the checks that we can run on your documentation.

</details>
author_association: CONTRIBUTOR