Project: VictoriaMetrics (Observability and Analysis / Observability)
File: guide-delete-or-replace-metrics.md

weight: 7 title: How to delete or replace metrics in VictoriaMetrics menu: docs: parent: "guides" weight: 7 aliases: /guides/guide-delete-or-replace-metrics.html Data deletion is an operation people expect a database to have. supports but to a limited extent. Due to implementation details, VictoriaMetrics remains an , which perfectly fits the case for storing time series data. But the drawback of such architecture is that it is extremely expensive to mutate the data. Hence, `delete` or `update` operations support is very limited. In this guide, we'll walk through the possible workarounds for deleting or changing already written data in VictoriaMetrics. ; ; Warning: time series deletion is not recommended to use on a regular basis. Each call to delete API could have a performance penalty. The API was provided for one-off operations to deleting malformed data or to satisfy GDPR compliance. expects from user to specify . So the first thing to do before the deletion is to verify whether the selector matches the correct series. To check that metrics are present in VictoriaMetrics Cluster run the following command: Warning: response can return many metrics, so be careful with series selector. ```curl curl -s 'http://vmselect:8481/select/0/prometheus/api/v1/series?match[]=processcpucores_available' | jq ``` The expected output: ```json { "status": "success", "isPartial": false, "data": [ { "name": "processcpucores_available", "job": "vmagent", "instance": "vmagent:8429" }, { "name": "processcpucores_available", "job": "vmalert", "instance": "vmalert:8880" }, { "name": "processcpucores_available", "job": "vminsert", "instance": "vminsert:8480" }, { "name": "processcpucores_available", "job": "vmselect", "instance": "vmselect:8481" }, { "name": "processcpucores_available", "job": "vmstorage", "instance": "vmstorage-1:8482" }, { "name": "processcpucores_available", "job": "vmstorage", "instance": "vmstorage-2:8482" } ] } ``` When you're sure is correct, send a POST request to with argument. For example: ```curl curl -s 'http://vmselect:8481/delete/0/prometheus/api/v1/admin/tsdb/deleteseries?match[]=processcpucoresavailable' ``` If operation was successful, the deleted series will stop being . Storage space for the deleted time series isn't freed instantly - it is freed during subsequent

The background merges may never occur for data from previous months, so storage space won't be freed for historical data. In this case may help freeing up storage space. To trigger on VictoriaMetrics Cluster run the following command: ```curl curl -v -X POST http://vmstorage:8482/internal/force_merge ``` After the merge is complete, the data will be permanently deleted from the disk. By default, VictoriaMetrics doesn't provide a mechanism for replacing or updating data. As a workaround, take the following actions: ; change the values of time series in the file and save it; ; . For example, let's export metric for `nodememoryMemTotal_bytes` with labels `instance="node-exporter:9100"` and `job="hostname.com"`: ```curl curl -X POST -g http://vmselect:8481/select/0/prometheus/api/v1/export -d 'match[]=nodememoryMemTotal_bytes{instance="node-exporter:9100", job="hostname.com"}' > data.jsonl ``` To check that exported file contains time series we can use and ```curl cat data.jsonl | jq ``` The expected output will look like: ```json { "metric": { "name": "nodememoryMemTotal_bytes", "job": "hostname.com", "instance": "node-exporter:9100" }, "values": [ 33604390912, 33604390912, 33604390912, 33604390912 ], "timestamps": [ 1656669031378, 1656669032378, 1656669033378, 1656669034378 ] } ``` In this example, we will replace the values of `nodememoryMemTotal_bytes` from `33604390912` to `17179869184` (from 32Gb to 16Gb) via , but it can be done in any of the available ways. ```sh sed -i 's/33604390912/17179869184/g' data.jsonl ``` Let's check the changes in data.jsonl with `cat`: ```sh cat data.jsonl | jq ``` The expected output will be the next: ```json { "metric": { "name": "nodememoryMemTotal_bytes", "job": "hostname.com", "instance": "node-exporter:9100" }, "values": [ 17179869184, 17179869184, 17179869184, 17179869184 ], "timestamps": [ 1656669031378, 1656669032378, 1656669033378, 1656669034378 ] } ``` See from the previous paragraph Victoriametrics supports a lot of and we will use . The next command will import metrics from `data.jsonl` to VictoriaMetrics: ```curl curl -v -X POST http://vminsert:8480/insert/0/prometheus/api/v1/import -T data.jsonl ``` ```curk curl -X POST -g http://vmselect:8481/select/0/prometheus/api/v1/export -d match[]=nodememoryMemTotal_bytes ``` The expected output will look like: ```json { "metric": { "name": "nodememoryMemTotal_bytes", "job": "hostname.com", "instance": "node-exporter:9100" }, "values": [ 17179869184, 17179869184, 17179869184, 17179869184 ], "timestamps": [ 1656669031378, 1656669032378, 1656669033378, 1656669034378 ] } ```