Project: Flink (App Definition and Development / Streaming & Messaging)
File: metric_reporters.md

title: "Metric Reporters" weight: 7 type: docs aliases: /deployment/metric_reporters.html <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Flink allows reporting metrics to external systems. For more information about Flink's metric system go to the . Metrics can be exposed to an external system by configuring one or several reporters in . These reporters will be instantiated on each job and task manager when they are started. Below is a list of parameters that are generally applicable to all reporters. All properties are configured by setting `metrics.reporter.<reporter_name>.<property>` in the configuration. Reporters may additionally offer implementation-specific parameters, which are documented in the respective reporter's section. {{< includereporterconfig "layouts/shortcodes/generated/metricreporterssection.html" >}} All reporter configurations must contain the `factory.class` property. Some reporters (referred to as `Scheduled`) allow specifying a reporting `interval`. Example reporter configuration that specifies multiple reporters: ```yaml metrics.reporters: myjmxreporter,myotherreporter metrics.reporter.myjmxreporter.factory.class: org.apache.flink.metrics.jmx.JMXReporterFactory metrics.reporter.myjmxreporter.port: 9020-9040 metrics.reporter.myjmxreporter.scope.variables.excludes: jobid;taskattempt_num metrics.reporter.myjmxreporter.scope.variables.additional: clustername:mytestcluster,tagname:tag_value metrics.reporter.myotherreporter.factory.class: org.apache.flink.metrics.graphite.GraphiteReporterFactory metrics.reporter.myotherreporter.host: 192.168.1.1 metrics.reporter.myotherreporter.port: 10000 ``` Important: The jar containing the reporter must be accessible when Flink is started. Reporters are loaded as . All reporters documented on this page are available by default. You can write your own `Reporter` by implementing the `org.apache.flink.metrics.reporter.MetricReporter` interface. If the Reporter should send out reports regularly you have to implement the `Scheduled` interface as well. Be careful that `report()` method must not block for a significant amount of time, and any reporter needing more time should instead run the operation asynchronously. By additionally implementing a `MetricReporterFactory` your reporter can also be loaded as a plugin. There are generally 2 formats for how reporters export metrics. Identifier-based reporters assemble a flat string containing all scope information and the metric name. An example could be `job.MyJobName.numRestarts`. Tag-based reporters on the other hand define a generic class of metrics consisting of a logical scope and metric name (e.g.,

and report a particular instance of said metric as a set of `key-value` pairs, so called "tags" or "variables" (e.g., "jobName=MyJobName"). Metrics are exported either via pushes or pulls. Push-based reporters usually implement the `Scheduled` interface and periodically send a summary of current metrics to an external system. Pull-based reporters are queried from an external system instead. The following sections list the supported reporters. Type: pull/tags Parameters: `port` - (optional) the port on which JMX listens for connections. In order to be able to run several instances of the reporter on one host (e.g. when one TaskManager is colocated with the JobManager) it is advisable to use a port range like `9250-9260`. When a range is specified the actual port is shown in the relevant job or task manager log. If this setting is set Flink will start an extra JMX connector for the given port/range. Metrics are always available on the default local JMX interface. Example configuration: ```yaml metrics.reporter.jmx.factory.class: org.apache.flink.metrics.jmx.JMXReporterFactory metrics.reporter.jmx.port: 8789 ``` Metrics exposed through JMX are identified by a domain and a list of key-properties, which together form the object name. The domain always begins with `org.apache.flink` followed by a generalized metric identifier. In contrast to the usual identifier it is not affected by scope-formats, does not contain any variables and is constant across jobs. An example for such a domain would be `org.apache.flink.job.task.numBytesOut`. The key-property list contains the values for all variables, regardless of configured scope formats, that are associated with a given metric. An example for such a list would be `host=localhost,jobname=MyJob,taskname=MyTask`. The domain thus identifies a metric class, while the key-property list identifies one (or multiple) instances of that metric. Type: push/identifier Parameters: `host` - the Graphite server host `port` - the Graphite server port `protocol` - protocol to use (TCP/UDP) Example configuration: ```yaml metrics.reporter.grph.factory.class: org.apache.flink.metrics.graphite.GraphiteReporterFactory metrics.reporter.grph.host: localhost metrics.reporter.grph.port: 2003 metrics.reporter.grph.protocol: TCP metrics.reporter.grph.interval: 60 SECONDS ``` Type: push/tags Parameters: {{< includereporterconfig "layouts/shortcodes/generated/influxdbreporterconfiguration.html" >}} Example configuration: ```yaml metrics.reporter.influxdb.factory.class: org.apache.flink.metrics.influxdb.InfluxdbReporterFactory metrics.reporter.influxdb.scheme: http metrics.reporter.influxdb.host: localhost metrics.reporter.influxdb.port: 8086 metrics.reporter.influxdb.db: flink metrics.reporter.influxdb.username: flink-metrics metrics.reporter.influxdb.password: qwerty metrics.reporter.influxdb.retentionPolicy: one_hour metrics.reporter.influxdb.consistency: ANY metrics.reporter.influxdb.connectTimeout: 60000 metrics.reporter.influxdb.writeTimeout: 60000 metrics.reporter.influxdb.interval: 60 SECONDS ``` The reporter would send metrics using http protocol to the InfluxDB server with the specified retention policy (or the default policy specified on the server). All Flink metrics variables (see ) are exported as InfluxDB tags. Type: pull/tags Parameters: `port` - (optional) the port the Prometheus exporter listens on, defaults to

In order to be able to run several instances of the reporter on one host (e.g. when one TaskManager is colocated with the JobManager) it is advisable to use a port range like `9250-9260`. `filterLabelValueCharacters` - (optional) Specifies whether to filter label value characters. If enabled, all characters not matching \. Example configuration: ```yaml metrics.reporter.prom.factory.class: org.apache.flink.metrics.prometheus.PrometheusReporterFactory ``` Flink metric types are mapped to Prometheus metric types as follows: | Flink | Prometheus | Note | | ||| | Counter | Gauge |Prometheus counters cannot be decremented.| | Gauge | Gauge |Only numbers and booleans are supported. | | Histogram | Summary |Quantiles .5, .75, .95, .98, .99 and .999 | | Meter | Gauge |The gauge exports the meter's rate. | All Flink metrics variables (see ) are exported to Prometheus as labels. Type: push/tags Parameters: {{< includereporterconfig "layouts/shortcodes/generated/prometheuspushgatewayreporterconfiguration.html" >}} Example configuration: ```yaml metrics.reporter.promgateway.factory.class: org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporterFactory metrics.reporter.promgateway.hostUrl: http://localhost:9091 metrics.reporter.promgateway.jobName: myJob metrics.reporter.promgateway.randomJobNameSuffix: true metrics.reporter.promgateway.deleteOnShutdown: false metrics.reporter.promgateway.groupingKey: k1=v1;k2=v2 metrics.reporter.promgateway.interval: 60 SECONDS ``` The PrometheusPushGatewayReporter pushes metrics to a , which can be scraped by Prometheus. Please see the for use-cases. Type: push/identifier Parameters: `host` - the StatsD server host `port` - the StatsD server port Example configuration: ```yaml metrics.reporter.stsd.factory.class: org.apache.flink.metrics.statsd.StatsDReporterFactory metrics.reporter.stsd.host: localhost metrics.reporter.stsd.port: 8125 metrics.reporter.stsd.interval: 60 SECONDS ``` Type: push/tags Note any variables in Flink metrics, such as `<host>`, `<jobname>`, `<tmid>`, `<subtaskindex>`, `<taskname>`, and `<operator_name>`, will be sent to Datadog as tags. Tags will look like `host:localhost` and `job_name:myjobname`. <span class="label label-danger">Note</span> For legacy reasons the reporter uses both the metric identifier and tags. This redundancy can be avoided by enabling `useLogicalIdentifier`. <span class="label label-info">Note</span> Histograms are exposed as a series of gauges following the naming convention of Datadog histograms (`<metric_name>.<aggregation>`). The `min` aggregation is reported by default, whereas `sum` is not available. In contrast to Datadog-provided Histograms the reported aggregations are not computed for a specific reporting interval. Parameters: `apikey` - the Datadog API key `proxyHost` - (optional) The proxy host to use when sending to Datadog. `proxyPort` - (optional) The proxy port to use when sending to Datadog, defaults to 8080. `dataCenter` - (optional) The data center (`EU`/`US`) to connect to, defaults to `US`. `maxMetricsPerRequest` - (optional) The maximum number of metrics to include in each request, defaults to 2000. `useLogicalIdentifier` -> (optional) Whether the reporter uses a logical metric identifier, defaults to `false`. Example configuration: ```yaml metrics.reporter.dghttp.factory.class: org.apache.flink.metrics.datadog.DatadogHttpReporterFactory metrics.reporter.dghttp.apikey: xxx metrics.reporter.dghttp.proxyHost: my.web.proxy.com metrics.reporter.dghttp.proxyPort: 8080 metrics.reporter.dghttp.dataCenter: US metrics.reporter.dghttp.maxMetricsPerRequest: 2000 metrics.reporter.dghttp.interval: 60 SECONDS metrics.reporter.dghttp.useLogicalIdentifier: true ``` Type: push/identifier Example configuration: ```yaml metrics.reporter.slf4j.factory.class: org.apache.flink.metrics.slf4j.Slf4jReporterFactory metrics.reporter.slf4j.interval: 60 SECONDS ``` {{< top >}}