Project: Grafana Mimir (Observability and Analysis / Observability)
File: v2.11.md

title: "Grafana Mimir version 2.11 release notes" menuTitle: "V2.11 release notes" description: "Release notes for Grafana Mimir version 2.11" weight: 1000 Grafana Labs is excited to announce version 2.11 of Grafana Mimir. The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the . Sampled logging of errors in the ingester. A high-traffic Mimir cluster can occasionally become bogged down logging high volumes of repeated errors. You can now reduce the amount of errors outputted to logs by setting a sample rate via the `-ingester.error-sample-rate` CLI flag. Add total request size instance limit for ingesters. This limit protects the ingesters against requests that together may cause an OOM. Enable this feature by setting the `-ingester.instance-limits.max-inflight-push-requests-bytes` CLI flag in combination with the `-ingester.limit-inflight-requests-using-grpc-method-limiter` CLI flag. Reduce the resolution of incoming native histograms samples if the incoming sample has too many buckets compared to `-validation.max-native-histogram-buckets`. This is enabled by default but can be turned off by setting the `-validation.reduce-native-histogram-over-max-buckets` CLI flag to `false`. Improved query-scheduler performance under load. This is particularly apparent for clusters with large numbers of queriers. Ingester to querier chunks streaming reduces the memory utilization of queriers and reduces the likelihood of OOMs. Ingester query request minimization reduces the number of query requests to ingesters, improving performance and resource utilization for both ingesters and queriers. The `markblocks` tool is released in compiled form together with the rest of Mimir. It is used to mark blocks for deletion or as "non-compactable". Grafana Mimir 2.11 includes new features that are considered experimental and disabled by default. Please use them with caution and report any issue you encounter: Block specified queries on a per-tenant basis. This is configured via the `blocked_queries` limit. See for more information. Store metadata when ingesting metrics via OTLP. This makes metric description and type available when ingesting metrics via OTLP. You can enable this feature by setting the CLI flag `-distributor.enable-otlp-metadata-storage` to `true`. Reject gRPC push requests that the ingester/distributor is unable to accept before reading them into memory. You can enable this feature by using the `-ingester.limit-inflight-requests-using-grpc-method-limiter` and/or the `-distributor.limit-inflight-requests-using-grpc-method-limiter` CLI flags for the ingester and/or the distributor, respectively. Customize the memcached client write and read buffer

The buffer allocated for each memcached connection can be configured via the following CLI flags: For the blocks storage: `-blocks-storage.bucket-store.chunks-cache.memcached.read-buffer-size-bytes` `-blocks-storage.bucket-store.chunks-cache.memcached.write-buffer-size-bytes` `-blocks-storage.bucket-store.index-cache.memcached.read-buffer-size-bytes` `-blocks-storage.bucket-store.index-cache.memcached.write-buffer-size-bytes` `-blocks-storage.bucket-store.metadata-cache.memcached.read-buffer-size-bytes` `-blocks-storage.bucket-store.metadata-cache.memcached.write-buffer-size-bytes` For the query frontend: `-query-frontend.results-cache.memcached.read-buffer-size-bytes` `-query-frontend.results-cache.memcached.write-buffer-size-bytes` For the ruler storage: `-ruler-storage.cache.memcached.read-buffer-size-bytes` `-ruler-storage.cache.memcached.write-buffer-size-bytes` Configure the number of long-living workers used to process gRPC requests. This can decrease CPU usage by reducing the number of stack allocations. Configure this feature by using the `-server.grpc.num-workers` CLI flag. Enforce a limit in bytes on the `PostingsForMatchers` cache used by ingesters. This limit can be configured via the `-blocks-storage.tsdb.head-postings-for-matchers-cache-max-bytes` and `-blocks-storage.tsdb.block-postings-for-matchers-cache-max-bytes` CLI flags. Pre-allocate the pool of workers in the distributor that are used to send push requests to ingesters. This can decrease CPU usage by reducing the number of stack allocations. You can enable this feature by using the `-distributor.reusable-ingester-push-worker` flag. Include a `Retry-After` header in recoverable error responses from the distributor. This can protect your Mimir cluster from clients including Prometheus that default to retrying very quickly. Enable this feature by setting the `-distributor.retry-after-header.enabled` CLI flag. The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the . In Grafana Mimir 2.11 the following behavior has changed: The utilization-based read path limiter now operates on Go heap size instead of RSS from the Linux proc file system. The following configuration options had been previously deprecated and are removed in Grafana Mimir 2.11: The CLI flag `-querier.iterators`. The CLI flag `-query.batch-iterators`. The CLI flag `-blocks-storage.bucket-store.bucket-index.enabled`. The CLI flag `-blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes`. The CLI flag `-blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes`. The CLI flag `-blocks-storage.bucket-store.max-chunk-pool-bytes`. The following configuration options are deprecated and will be removed in Grafana Mimir 2.13: The CLI flag `-log.buffered`; this is now the default behavior. The following metrics are removed: `cortexqueryfrontendworkersenqueuedrequeststotal`; use `cortexqueryfrontendenqueuedurationsecondscount` instead. The following configuration option defaults were changed: The CLI flag `-blocks-storage.bucket-store.index-header.sparse-persistence-enabled` now defaults to true. The default value for the CLI flag `-blocks-storage.bucket-store.index-header.lazy-loading-concurrency` was changed from `0` to `4`. The default value for the CLI flag `-blocks-storage.tsdb.series-hash-cache-max-size-bytes` was changed from `1GB` to `350MB`. The default value for the CLI flag `-blocks-storage.tsdb.early-head-compaction-min-estimated-series-reduction-percentage` was changed from `10` to `15`. Ingester: Respect context cancelation during query execution. Distributor: Return 529 when ingestion rate limit is hit and the `distributor.serviceoverloadstatuscodeonratelimit_enabled` flag is active. Query-scheduler: Prevent accumulation of stale querier connections. Packaging: Fix preremove script preventing upgrades on RHEL based OS. Ingester: Fixed possible series matcher corruption leading to wrong series being included in query results. Mimirtool: Fixed missing .metricsUsed field in the output of analyze rule-file.