Project: SkyWalking (Observability and Analysis / Observability)
File: metrics-query-expression.md

MQE is a string that consists of one or more expressions. Each expression could be a combination of one or more operations. The expression allows users to do simple query-stage calculation through . ```text Expression = <Operation> Expression1 <Operation> Expression2 <Operation> Expression3 ... ``` The following document lists the operations supported by MQE. Metrics Expression will return a collection of time-series values. Expression: ```text <metric_name> ``` For example: If we want to query the `service_sla` metric, we can use the following expression: ```text service_sla ``` The `ExpressionResultType` of the expression is `TIMESERIESVALUES`. Since v10.0.0, SkyWalking supports multiple labels metrics. We could query the specific labels of the metric by the following expression. Expression: ```text <metricname>{<label1name>='<label1value1>,...', <label2name>='<label2value_1>,...',<label2...} ``` `{<label1name>='<labelvalue_1>,...'}` is the selected label name/value of the metric. If is not specified, all label values of the metric will be selected. For example: The `k8sclusterdeployment_status` metric has labels `namespace`, `deployment` and `status`. If we want to query all deployment metric value with `namespace=skywalking-showcase` and `status=true`, we can use the following expression: ```text k8sclusterdeployment_status{namespace='skywalking-showcase', status='true'} ``` We also could query the label with multiple values by separating the values with `,`: If we want to query the `service_percentile` metric with the label name `p` and values `50,75,90,95,99`, we can use the following expression: ```text service_percentile{p='50,75,90,95,99'} ``` If we want to rename the label values to `P50,P75,P90,P95,P99`, see . The `ExpressionResultType` of the expression is `TIMESERIESVALUES` and with labels. The Binary Operation is an operation that takes two expressions and performs a calculation on their results. The following table lists the binary operations supported by MQE. Expression: ```text Expression1 <Binary-Operator> Expression2 ``` | Operator | Definition | |-|-| | + | addition | | - | subtraction | | * | multiplication | | / | division | | % | modulo | For example: If we want to transform the service_sla metric value to percent, we can use the following expression: ```text service_sla / 100 ``` For the result type of the expression, please refer to the following table. The following table lists if the different result types of the input expressions could do this operation and the result type after the operation. The expression could be on the left or right side of the operator. Note: If the expressions result on both sides of the operator are `with labels`, they should have the same labels for calculation. If the labels match, will reserve left expression result labels and the calculated value. Otherwise, will return empty value. | Expression | Expression | Yes/No | ExpressionResultType | |-||--|--| | SINGLEVALUE | SINGLEVALUE | Yes | SINGLE_VALUE | | SINGLEVALUE | TIMESERIESVALUES | Yes | TIMESERIES_VALUES | | SINGLEVALUE | SORTEDLIST/RECORDLIST | Yes | SORTEDLIST/RECORD_LIST | | TIMESERIESVALUES | TIMESERIESVALUES | Yes | TIMESERIESVALUES | | TIMESERIESVALUES | SORTEDLIST/RECORDLIST | no | | | SORTEDLIST/RECORDLIST | SORTEDLIST/RECORDLIST | no | | Compare Operation takes two expressions and compares their results. The following table lists the compare operations supported by

Expression: ```text Expression1 <Compare-Operator> Expression2 ``` | Operator | Definition | |-|--| | \> | greater than | | \>= | greater than or equal | | < | less than | | <= | less than or equal | | == | equal | | != | not equal | The result of the compare operation is an int value: 1: true 0: false For example: Compare the `serviceresptime` metric value if greater than 3000, if the `serviceresptime` result is: ```json { "data": { "execExpression": { "type": "TIMESERIESVALUES", "error": null, "results": [ { "metric": { "labels": [] }, "values": [{"id": "1691658000000", "value": "2500", "traceID": null}, {"id": "1691661600000", "value": 3500, "traceID": null}] } ] } } } ``` we can use the following expression: ```text serviceresptime > 3000 ``` and get result: ```json { "data": { "execExpression": { "type": "TIMESERIESVALUES", "error": null, "results": [ { "metric": { "labels": [] }, "values": [{"id": "1691658000000", "value": "0", "traceID": null}, {"id": "1691661600000", "value": 1, "traceID": null}] } ] } } } ``` Same as the . Aggregation Operation takes an expression and performs aggregate calculations on its results. Expression: ```text <Aggregation-Operator>(Expression) ``` | Operator | Definition | ExpressionResultType | |-||-| | avg | average the result | SINGLE_VALUE | | count | count number of the result | SINGLE_VALUE | | latest | select the latest non-null value from the result | SINGLE_VALUE | | sum | sum the result | SINGLE_VALUE | | max | select maximum from the result | SINGLE_VALUE | | min | select minimum from the result | SINGLE_VALUE | For example: If we want to query the average value of the `service_cpm` metric, we can use the following expression: ```text avg(service_cpm) ``` The different operators could impact the `ExpressionResultType`, please refer to the above table. Mathematical Operation takes an expression and performs mathematical calculations on its results. Expression: ```text <Mathematical-Operator>(Expression, parameters) ``` | Operator | Definition | parameters | ExpressionResultType | |-||--|-| | abs | returns the absolute value of the result | | follow the input expression | | ceil | returns the smallest integer value that is greater or equal to the result | | follow the input expression | | floor | returns the largest integer value that is greater or equal to the result | | follow the input expression | | round | returns result round to specific decimal places | `places`: a positive integer specific decimal places of the result | follow the input expression | For example: If we want to query the average value of the `service_cpm` metric in seconds, and round the result to 2 decimal places, we can use the following expression: ```text round(service_cpm / 60 , 2) ``` The different operators could impact the `ExpressionResultType`, please refer to the above table. TopN Operation takes an expression and performs calculation to get the TopN of Services/Instances/Endpoints. The result depends on the `entity` condition in the query. Global TopN: The `entity` is

The result is the topN Services/Instances/Endpoints in the whole traffics. Notice: If query the Endpoints metric, the global candidate set could be huge, please use it carefully. Service's Instances/Endpoints TopN: The `serviceName` in the `entity` is not empty. The result is the topN Instances/Endpoints of the service. Expression: ```text topn(<metricname>, <top_number>, <order>) ``` `top_number` is the number of the top results, should be a positive integer. `order` is the order of the top results. The value of `order` can be `asc` or `des`. For example: If we want to query the current service's top 10 instances with the highest `serviceinstancecpm` metric value, we can use the following expression under specific service: ```text topn(serviceinstance_cpm, 10, des) ``` According to the type of the metric, the `ExpressionResultType` of the expression will be `SORTEDLIST` or `RECORDLIST`. Relabel Operation takes an expression and replaces the label values with new label values on its results. Since v10.0.0, SkyWalking supports relabel multiple labels. Expression: ```text relabel(Expression, <targetlabelname>='<originlabelvalue1>,...', <newlabelname>='<newlabelvalue1>,...') ``` The order of the new label values should be the same as the order of the label values in the input expression result. For example: If we want to query the `service_percentile` metric with the label values `50,75,90,95,99`, and rename the label name to `percentile` and the label values to `P50,P75,P90,P95,P99`, we can use the following expression: ```text relabel(service_percentile{p='50,75,90,95,99'}, p='50,75,90,95,99', percentile='P50,P75,P90,P95,P99') ``` Follow the input expression. AggregateLabels Operation takes an expression and performs an aggregate calculation on its `Labeled Value Metrics` results. It aggregates a group of `TIMESERIESVALUES` into a single `TIMESERIESVALUES`. Expression: ```text aggregatelabels(Expression, <AggregateType>(<label1name>,<label2_name>...)) ``` `AggregateType` is the type of the aggregation operation. `<label1name>,<label2name>...` is the label names that need to be aggregated. If not specified, all labels will be aggregated. Optional. | AggregateType | Definition | ExpressionResultType | ||-|-| | avg | calculate avg value of a `Labeled Value Metrics` | TIMESERIESVALUES | | sum | calculate sum value of a `Labeled Value Metrics` | TIMESERIESVALUES | | max | select the maximum value from a `Labeled Value Metrics` | TIMESERIESVALUES | | min | select the minimum value from a `Labeled Value Metrics` | TIMESERIESVALUES | For example: If we want to query all Redis command total rates, we can use the following expression(`totalcommandsrate` is a metric which recorded every command rate in labeled value): Aggregating all the labels: ```text aggregatelabels(totalcommands_rate, sum) ``` Also, we can aggregate by the `cmd` label: ```text aggregatelabels(totalcommands_rate, sum(cmd)) ``` The ExpressionResultType of the aggregateLabels operation is TIMESERIESVALUES. ViewAsSequence operation represents the first not-null metric from the listing metrics in the given prioritized sequence(left to right). It could also be considered as a `short-circuit` of given metrics for the first value existing metric. Expression: ```text viewasseq([<expression1>, <expression2>, ...]) ``` For example: if the first expression value is empty but the second one is not empty, it would return the result from the second expression. The following example would return the content of the service_cpm metric. ```text viewasseq(notexisting, servicecpm) ``` The result type is determined by the type of selected not-null metric

IsPresent operation represents that in a list of metrics, if any expression has a value, it would return `1` in the result; otherwise, it would return `0`. Expression: ```text ispresent([<expression1>, <expression_2>, ...]) ``` For example: When the meter does not exist or the metrics has no value, it would return `0`. However, if the metrics list contains meter with values, it would return `1`. ```text ispresent(notexisting, existingwithoutvalue, existingwithvalue) ``` The result type is `SINGLE_VALUE`, and the result(`1` or `0`) in the first value. Trend Operation takes an expression and performs a trend calculation on its results. Expression: ```text <Trend-Operator>(Metrics Expression, time_range) ``` `time_range` is the positive int of the calculated range. The unit will automatically align with to the query , for example, if the query Step is `MINUTE`, the unit of `time_range` is `minute`. | Operator | Definition | ExpressionResultType | |-||-| | increase | returns the increase in the time range in the time series | TIMESERIESVALUES | | rate | returns the per-second average rate of increase in the time range in the time series | TIMESERIESVALUES | For example: If we want to query the increase value of the `service_cpm` metric in 2 minute(assume the query Step is MINUTE), we can use the following expression: ```text increase(service_cpm, 2) ``` If the query duration is 3 minutes, from (T1 to T3) and the metric has values in time series: ```text V(T1-2), V(T1-1), V(T1), V(T2), V(T3) ``` then the expression result is: ```text V(T1)-V(T1-2), V(T2)-V(T1-1), V(T3)-V(T1) ``` Note: If the calculated metric value is empty, the result will be empty. Assume in the T3 point, the increase value = V(T3)-V(T1), If the metric V(T3) or V(T1) is empty, the result value in T3 will be empty. TIMESERIESVALUES. SortValues Operation takes an expression and sorts the values of the input expression result. Expression: ```text sort_values(Expression, <limit>, <order>) ``` `limit` is the number of the sort results, should be a positive integer, if not specified, will return all results. Optional. `order` is the order of the sort results. The value of `order` can be `asc` or `des`. For example: If we want to sort the `serviceresptime` metric values in descending order and get the top 10 values, we can use the following expression: ```text sortvalues(serviceresp_time, 10, des) ``` The result type follows the input expression. SortLabelValues Operation takes an expression and sorts the label values of the input expression result. This function uses `natural sort order`. Expression: ```text sortlabelvalues(Expression, <order>, <label1name>, <label2name> ...) ``` `order` is the order of the sort results. The value of `order` can be `asc` or `des`. `<label1name>, <label2name> ...` is the label names that need to be sorted by their values. At least one label name should be specified. The labels in the head of the list will be sorted first, and if the label not be included in the expression result will be

For example: If we want to sort the `service_percentile` metric label values in descending order by the `p` label, we can use the following expression: ```text sortlabelvalues(service_percentile{p='50,75,90,95,99'}, des, p) ``` For multiple labels, assume the metric has 2 labels ```text metric{label1='a', label2='2a'} metric{label1='a', label2='2c'} metric{label1='b', label2='2a'} metric{label1='b', label2='2c'} ``` If we want to sort the `metric` metric label values in descending order by the `label1` and `label2` labels, we can use the following expression: ```text sortlabelvalues(metric, des, label1, label2) ``` And the result will be: ```text metric{label1='b', label2='2c'} metric{label1='b', label2='2a'} metric{label1='a', label2='2c'} metric{label1='a', label2='2a'} ``` ```text service_percentile{p='50,95'} ``` The example result is: ```json { "data": { "execExpression": { "type": "TIMESERIESVALUES", "error": null, "results": [ { "metric": { "labels": [{"key": "p", "value": "50"}] }, "values": [{"id": "1691658000000", "value": "1000", "traceID": null}, {"id": "1691661600000", "value": 2000, "traceID": null}] }, { "metric": { "labels": [{"key": "p", "value": "75"}] }, "values": [{"id": "1691658000000", "value": "2000", "traceID": null}, {"id": "1691661600000", "value": 3000, "traceID": null}] } ] } } } ``` If we want to transform the percentile value unit from `ms` to `s` the expression is: ```text service_percentile{p='50,75'} / 1000 ``` ```json { "data": { "execExpression": { "type": "TIMESERIESVALUES", "error": null, "results": [ { "metric": { "labels": [{"key": "p", "value": "50"}] }, "values": [{"id": "1691658000000", "value": "1", "traceID": null}, {"id": "1691661600000", "value": 2, "traceID": null}] }, { "metric": { "labels": [{"key": "p", "value": "75"}] }, "values": [{"id": "1691658000000", "value": "2", "traceID": null}, {"id": "1691661600000", "value": 3, "traceID": null}] } ] } } } ``` Get the average value of each percentile, the expression is: ```text avg(service_percentile{p='50,75'}) ``` ```json { "data": { "execExpression": { "type": "SINGLE_VALUE", "error": null, "results": [ { "metric": { "labels": [{"key": "p", "value": "50"}] }, "values": [{"id": null, "value": "1500", "traceID": null}] }, { "metric": { "labels": [{"key": "p", "value": "75"}] }, "values": [{"id": null, "value": "2500", "traceID": null}] } ] } } } ``` Calculate the difference between the percentile and the average value, the expression is: ```text servicepercentile{p='50,75'} - avg(servicepercentile{p='50,75'}) ``` ```json { "data": { "execExpression": { "type": "TIMESERIESVALUES", "error": null, "results": [ { "metric": { "labels": [{"key": "p", "value": "50"}] }, "values": [{"id": "1691658000000", "value": "-500", "traceID": null}, {"id": "1691661600000", "value": 500, "traceID": null}] }, { "metric": { "labels": [{"key": "p", "value": "75"}] }, "values": [{"id": "1691658000000", "value": "-500", "traceID": null}, {"id": "1691661600000", "value": 500, "traceID": null}] } ] } } } ``` Calculate the difference between the `serviceresptime` and the `servicepercentile`, if the `serviceresp_time` result is: ```json { "data": { "execExpression": { "type": "TIMESERIESVALUES", "error": null, "results": [ { "metric": { "labels": [] }, "values": [{"id": "1691658000000", "value": "2500", "traceID": null}, {"id": "1691661600000", "value": 3500, "traceID": null}] } ] } } } ``` The expression is: ```text serviceresptime - service_percentile{p='50,75'} ``` ```json { "data": { "execExpression": { "type": "TIMESERIESVALUES", "error": null, "results": [ { "metric": { "labels": [{"key": "p", "value": "50"}] }, "values": [{"id": "1691658000000", "value": "1500", "traceID": null}, {"id": "1691661600000", "value": "1500", "traceID": null}] }, { "metric": { "labels": [{"key": "p", "value": "75"}] }, "values": [{"id": "1691658000000", "value": "500", "traceID": null}, {"id": "1691661600000", "value": "500", "traceID": null}] } ] } } } ```