Project: M3 (Observability and Analysis / Observability)
File: m3query.md

title: Query Data with M3 Query weight: 4 {{< fileinclude file="m3query_intro.md" >}} Before running M3 Query, you need to have at least . To start M3 Query, you need a YAML configuration file that specifies the M3 nodes it connects to. {{% notice tip %}} You can use for experimenting with a local cluster that uses an embedded etcd cluster. for more details on running in production with an external etcd cluster. {{% /notice %}} {{< tabs name="run_m3query" >}} {{% tab name="Prebuilt Binaries" %}} M3 Query has pre-built binaries available for Linux and macOS. . mentioned above. ```shell ./bin/m3query -f .m3query-local-etcd.yml ``` {{% /tab %}} {{% tab name="Self-built binary" %}} {{< fileinclude file="build_prerequisites.md" >}} Build and run the binary: ```shell make m3query ./bin/m3query -f ./src/query/config/m3query-local-etcd.yml ``` {{% /tab %}} {{% tab name="Docker" %}} ```shell docker run quay.io/m3db/m3query:v1.1.0 ``` {{% /tab %}} {{< /tabs >}} You will notice that in the setup linked above, M3DB has just one unaggregated namespace configured. If you want aggregated metrics, you will need to set up an aggregated namespace. It is important to note that all writes go to all namespaces marked as ready. Aggregation is done strictly by the query service. As an example, to configure an aggregated namespace named `metrics10s48h`, you can execute the following API call: ```shell curl -X POST {{% apiendpoint %}}services/m3db/namespace -d '{ "name": "metrics10s48h", "options": { "bootstrapEnabled": true, "flushEnabled": true, "writesToCommitLog": true, "cleanupEnabled": true, "snapshotEnabled": true, "repairEnabled": false, "retentionOptions": { "retentionPeriodDuration": "48h", "blockSizeDuration": "2h", "bufferFutureDuration": "10m", "bufferPastDuration": "10m", "blockDataExpiry": true, "blockDataExpiryAfterNotAccessedPeriodDuration": "5m" }, "indexOptions": { "enabled": true, "blockSizeDuration": "2h" }, "aggregationOptions": { "aggregations": [ { "aggregated": true, "attributes": { "resolutionDuration": "10s" } } ] } } }' ``` If you run Statsite, m3agg, or some other aggregation tier, you will want to set the `all` flag under `downsample` to `false`. Otherwise, you will be aggregating metrics that have already been aggregated. Using the example above, `aggregationOptions` would be configured as follows ```shell ... "aggregationOptions": { "aggregations": [ { "aggregated": true, "attributes": { "resolutionDuration": "10s", "downsampleOptions": { "all": false } } } ] } ... ``` As of 1.0, the default generation scheme for IDs is `quoted`. If you are using the deprecated `legacy` scheme, please contact the M3 project contributors on the open source Slack channel. The `quoted` generation scheme yields the most human-readable IDs, whereas `prepend_meta` is better for more compact IDs, or if tags are expected to contain non-ASCII

To set the ID generation scheme, add the following to your m3coordinator configuration yaml file: ```yaml tagOptions: idScheme: <name> ``` As an example of how these schemes generate IDs, consider a series with the following 4 tags, `[{"t1":v1}, {t2:"v2"}, {t3:v3}, {t4:v4}]`. The following is an example of how different schemes will generate IDs. ```shell legacy: "t1"=v1,t2="v2",t3=v3,t4=v4, prepend_meta: 4,2,2,4,2,2,2,2!"t1"v1t2"v2"t3v3t4v4 quoted: {\"t1\"="v1",t2="\"v2\"",t3="v3",t4="v4"} ``` If there is a chance that your metric tags will contain "control" characters, specifically `,` and `=`, it is highly recommended that one of either the `quoted` or `prepend_meta` schemes are specified, as the `legacy` scheme may cause ID collisions. As a general guideline, we suggest `quoted`, as it mirrors the more familiar Prometheus style IDs. We technically have a fourth ID generation scheme that is used for Graphite IDs, but it is exclusive to the Graphite ingestion path and is not selectable as a general scheme. WARNING: Once a scheme is selected, be very careful about changing it. If changed, all incoming metrics will resolve to a new ID, effectively doubling the metric cardinality until all of the older-style metric IDs fall out of retention. We recently updated our ID generation scheme in m3coordinator to avoid the collision issues discussed above. To ease migration, we're temporarily enforcing that an ID generation scheme be explicitly provided in the m3coordinator configuration files. If you have been running m3query or m3coordinator already, you may want to counterintuitively select the collision-prone `legacy` scheme, as all the IDs for all of your current metrics would have already been generated with this scheme, and choosing another will effectively double your index size. If the twofold increase in cardinality is an acceptable increase (and unfortunately, this is likely to mean doubled cardinality until your longest retention cluster rotates out), it's suggested to choose a collision-resistant scheme instead. An example of a configuration file for a standalone m3query instance with the ID generation scheme can be found . If you're running m3query or m3coordinator embedded, these configuration options should be nested under the `coordinator:` heading, as seen . If none of these options work for you, or you would like further clarification, please stop by our and we'll be happy to help you. You can also set up m3query as a . To do this, add a new datasource with a type of `Prometheus`. The URL should point to the host/port running m3query. By default, m3query runs on port `7201`.