Project: VictoriaMetrics (Observability and Analysis / Observability)
File: vmalert-tool.md

sort: 12 weight: 12 menu: docs: parent: 'victoriametrics' weight: 12 title: vmalert-tool VMAlert command-line tool You can use `vmalert-tool` to run unit tests for alerting and recording rules. It will perform the following actions: sets up an isolated VictoriaMetrics instance; simulates the periodic ingestion of time series; queries the ingested data for recording and alerting rules evaluation like ; checks whether the firing alerts or resulting recording rules match the expected results. See how to run vmalert-tool for unit test below: ``` ./vmalert-tool unittest --files test1.yaml --files test2.yaml ``` vmalert-tool unittest is compatible with except `promqlexprtest` field. Use `metricsqlexprtest` field name instead. The name is different because vmalert-tool validates and executes expressions, which aren't always backward compatible with . vmalert-tool evaluates all the groups defined in `rulefiles` using `evaluationinterval`(default `1m`) instead of `interval` under each rule group. vmalert-tool shares the same limitation with on chaining rules under one group: by default, rules execution is sequential within one group, but persistence of execution results to remote storage is asynchronous. Hence, user shouldnt rely on chaining of recording rules when result of previous recording rule is reused in the next one; For example, you have recording rule A and alerting rule B in the same group, and rule B's expression is based on A's results. Rule B won't get the latest data of A, since data didn't persist to remote storage yet. The workaround is to divide them in two groups and put groupA in front of groupB (or use `groupevalorder` to define the evaluation order). In this way, vmalert-tool makes sure that the results of groupA must be written to storage before evaluating groupB: ```yaml groups: name: groupA rules: record: A expr: sum(xxx) name: groupB rules: alert: B expr: A >=

for: 1m ``` The configuration format for files specified in `--files` cmd-line flag is the following: ```yaml rule_files: [ - <string> ] [ evaluation_interval: <duration> | default = 1m ] groupevalorder: [ - <string> ] tests: [ - <test_group> ] ``` ```yaml interval: <duration> input_series: [ - <series> ] [ name: <string> ] alertruletest: [ - <alerttestcase> ] metricsqlexprtest: [ - <metricsqlexprtest> ] external_labels: [ <labelname>: <string> ... ] ``` ```yaml series: <string> values: <string> ``` vmalert by default adds `alertgroup` and `alertname` to the generated alerts and time series. So you will need to specify both `groupname` and `alertname` under a single `<alerttestcase>`, but no need to add them under `exp_alerts`. You can also pass `--disableAlertgroupLabel` to skip `alertgroup` check. ```yaml eval_time: <duration> groupname: <string> alertname: <string> exp_alerts: [ - <alert> ] ``` ```yaml exp_labels: [ <labelname>: <string> ] exp_annotations: [ <labelname>: <string> ] ``` ```yaml expr: <string> eval_time: <duration> exp_samples: [ - <sample> ] ``` ```yaml labels: <string> value: <number> ``` This is an example input file for unit testing which will pass. `test.yaml` is the test file which follows the syntax above and `alerts.yaml` contains the alerting rules. With `rules.yaml` in the same directory, run `./vmalert-tool unittest --files=./unittest/testdata/test.yaml`. ```yaml rule_files: rules.yaml evaluation_interval: 1m tests: interval: 1m input_series: series: 'up{job="prometheus", instance="localhost:9090"}' values: "0+0x1440" metricsqlexprtest: expr: suqueryintervaltest eval_time: 4m exp_samples: labels: '{name="suqueryintervaltest", datacenter="dc-123", instance="localhost:9090", job="prometheus"}' value: 1 alertruletest: eval_time: 2h groupname: group1 alertname: InstanceDown exp_alerts: exp_labels: job: prometheus severity: page instance: localhost:9090 datacenter: dc-123 exp_annotations: summary: "Instance localhost:9090 down" description: "localhost:9090 of job prometheus has been down for more than 5 minutes." eval_time: 0 groupname: group1 alertname: AlwaysFiring exp_alerts: exp_labels: datacenter: dc-123 eval_time: 0 groupname: group1 alertname: InstanceDown exp_alerts: [] external_labels: datacenter: dc-123 ``` ```yaml groups: name: group1 rules: alert: InstanceDown expr: up == 0 for: 5m labels: severity: page annotations: summary: "Instance {{ $labels.instance }} down" description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes." alert: AlwaysFiring expr: 1 name: group2 rules: record: job:test:countovertime1m expr: sum without(instance) (countovertime(test[1m])) record: suqueryintervaltest expr: countovertime(up[5m:]) ```