Project: Grafana Mimir (Observability and Analysis / Observability)
File: about-runtime-configuration.md

aliases: ../configuring/about-runtime-configuration/ # /docs/mimir/<MIMIR_VERSION>/configuring/about-runtime-configuration/ ../operators-guide/configure/about-runtime-configuration/ # /docs/mimir/<MIMIR_VERSION>/operators-guide/configure/about-runtime-configuration/ ../operators-guide/configuring/about-runtime-configuration/ # /docs/mimir/<MIMIR_VERSION>/operators-guide/configuring/about-runtime-configuration/ description: Runtime configuration enables you to change a subset of configurations without restarting Grafana Mimir. menuTitle: Runtime configuration title: About Grafana Mimir runtime configuration weight: 40 A runtime configuration file is a file that contains configuration parameters, which is periodically reloaded while Mimir is running. It allows you to change a subset of Grafana Mimirs configuration without having to restart the Grafana Mimir component or instance. Runtime configuration is available for a subset of the configuration that was set at startup. A Grafana Mimir operator can observe the configuration and use runtime configuration to make immediate adjustments to Grafana Mimir. Runtime configuration values take precedence over command-line options. If multiple runtime configuration files are specified the runtime configuration files will be merged in a left to right order. To enable runtime configuration, specify a comma-separated list of file paths upon startup by using the `-runtime-config.file=<filepath>,<filepath>` CLI flag or from within your YAML configuration file in the `runtime_config` block. By default, Grafana Mimir reloads the contents of these files every 10 seconds and merges these files from left to right. You can configure this interval by using the `-runtime-config.reload-period=<duration>` CLI flag or by specifying the `period` value in your YAML configuration file. When running Grafana Mimir on Kubernetes, store the runtime configuration files in a and mount the ConfigMaps in each container. Use Grafana Mimirs `/runtimeconfig` endpoint to see the current value of the runtime configuration, including the overrides. To see only the non-default values of the configuration, specify the endpoint with `/runtimeconfig?mode=diff`. The runtime configuration file is primarily used to set and adjust limits that are appropriate for each tenant based on their ingest and query needs. The values that are defined in the limits section of your YAML configuration define the default set of limits that are applied to tenants. For example, if you set the `ingestion_rate` to `25,000` in your YAML configuration file, any tenant in your cluster that sends more than 25,000 samples per second (SPS) is rate limited. You can use the runtime configuration file to override this

For example, if you have a tenant (`tenant1`) that needs to send twice as many data points as the current limit, and you have another tenant (`tenant2`) that needs to send three times as many data points, you can modify the contents of your runtime configuration file as follows: ```yaml overrides: tenant1: ingestion_rate: 50000 tenant2: ingestion_rate: 75000 ``` As a result, Grafana Mimir allows `tenant1` to send 50,000 SPS, and `tenant2` to send 75,000 SPS, while maintaining a 25,000 SPS rate limit on all other tenants. On a per-tenant basis, you can override all of the limits listed in the block within the runtime configuration file. For each tenant, you can override different limits. For any tenant or limit that is not overridden in the runtime configuration file, you can inherit the limit values that are specified in the `limits` block. The runtime configuration file can be used to dynamically adjust Grafana Mimir ingester instance limits. While per-tenant limits are limits applied to each tenant, per-ingester-instance limits are limits applied to each ingester process. Ingester limits ensure individual ingesters are not overwhelmed, regardless of any per-tenant limits. These limits can be set under the `ingester.instancelimits` block in the global configuration file, with CLI flags, or under the `ingesterlimits` field in the runtime configuration file. The runtime configuration allows you to override initial values, which is useful for advanced operators who need to dynamically change them in response to changes in ingest or query load. Everything under the `instance_limits` section within the block can be overridden via runtime configuration. The following example shows a portion of the runtime configuration that changes the ingester limits: ```yaml ingester_limits: maxingestionrate: 20000 max_series: 1500000 max_tenants: 1000 maxinflightpush_requests: 30000 ``` The runtime configuration file can be used to dynamically adjust Grafana Mimir distributor instance limits. While per-tenant limits are limits applied to each tenant, per-distributor-instance limits are limits applied to each distributor process. Distributor limits ensure individual distributors are not overwhelmed, regardless of any per-tenant limits. These limits can be set under the `distributor.instancelimits` block in the global configuration file, with CLI flags, or under the `distributorlimits` field in the runtime configuration file. The runtime configuration allows you to override initial values, which is useful for advanced operators who need to dynamically change them in response to changes in ingest load. Everything under the `instance_limits` section within the block can be overridden via runtime configuration. The following example shows a portion of the runtime configuration that changes the distributor limits: ```yaml distributor_limits: maxingestionrate: 75000 maxinflightpush_requests: 1500 maxinflightpushrequestsbytes: 314572800 ``` An advanced runtime configuration option controls if ingesters transfer encoded chunks (the default) or transfer decoded series to queriers at query time. The parameter `ingesterstreamchunkswhenusing_blocks` might only be used in runtime configuration. A value of `true` transfers encoded chunks, and a value of `false` transfers decoded series. {{< admonition type="note" >}} By default, `-ingester.stream-chunks-when-using-blocks` is `true` which enables transfer of encoded chunks. In runtime configuration, the parameter `ingesterstreamchunkswhenusing_blocks` overrides the CLI flag `-ingester.stream-chunks-when-using-blocks`. It's strongly recommended that you keep the transfer of encoded chunks enabled, except in rare cases where you observe rules evaluation slowing down. {{< /admonition >}}