Project: VictoriaMetrics (Observability and Analysis / Observability)
File: grafana-datasource.md

sort: 38 weight: 38 title: Grafana datasource menu: docs: parent: victoriametrics weight: 38 aliases: /grafana-datasource.html The datasource plugin allows you to query and visualize data from VictoriaMetrics in Grafana. Thanks to VictoriaMetrics compatibility with Prometheus API users can use for Grafana to query data from VictoriaMetrics. But with time, Prometheus and VictoriaMetrics diverge more and more. After some unexpected changes to Prometheus datasource we decided to create a datasource plugin specifically for VictoriaMetrics. The benefits of using VictoriaMetrics plugin are the following: functions support; Supports in Explore mode or right in panel's expressions; Supports ; Plugin fixes issue; Integration with . Installing VictoriaMetrics Grafana datasource the following changes to Grafana's `grafana.ini` config: ``` ini [plugins] allowloadingunsigned_plugins = victoriametrics-datasource ``` For `grafana-operator` users, please adjust `config:` section in your `kind=Grafana` resource as below ``` config: plugins: allowloadingunsigned_plugins: "victoriametrics-datasource" ``` See . For detailed instructions on how to install the plugin on Grafana Cloud or locally, please checkout the . provides a complete environment to spin-up all required components via Docker. To begin, clone repository and follow steps described in the . Provision of Grafana plugin requires to create . Example of config file for provisioning VictoriaMetrics datasource is the following: ```yaml apiVersion: 1 datasources: name: VictoriaMetrics type: victoriametrics-datasource access: proxy url: http://victoriametrics:8428 isDefault: true name: VictoriaMetrics - cluster type: victoriametrics-datasource access: proxy url: http://vmselect:8481/select/0/prometheus isDefault: false ``` Please find the example of provisioning Grafana instance with VictoriaMetrics datasource below: Create folder `./provisioning/datasource` with datasource example file: Download the latest release: ``` bash ver=$(curl -s https://api.github.com/repos/VictoriaMetrics/grafana-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1) curl -L https://github.com/VictoriaMetrics/grafana-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o plugin.tar.gz tar -xf plugin.tar.gz -C ./ rm plugin.tar.gz ``` Define Grafana installation via docker-compose: ```yaml version: '3.0' services: grafana: container_name: 'grafana-datasource' build: context: ./.config args: grafanaversion: ${GRAFANAVERSION:-9.1.2} ports: 3000:3000/tcp volumes: ./victoriametrics-datasource:/var/lib/grafana/plugins/grafana-datasource ./provisioning:/etc/grafana/provisioning ``` Run docker-compose file: ``` docker-compose -f docker-compose.yaml up ``` When Grafana starts successfully datasources should be present on the datasources tab <img src="provision_datasources.webp" width="800" alt="Configuration"> Example with Grafana : ``` yaml extraInitContainers: name: "load-vm-ds-plugin" image: "curlimages/curl:7.85.0" command: [ "/bin/sh" ] workingDir: "/var/lib/grafana" securityContext: runAsUser: 472 runAsNonRoot: true runAsGroup: 472 args: "-c" | set -ex mkdir -p /var/lib/grafana/plugins/ ver=$(curl -s https://api.github.com/repos/VictoriaMetrics/grafana-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1) curl -L https://github.com/VictoriaMetrics/grafana-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/plugin.tar.gz tar -xf /var/lib/grafana/plugins/plugin.tar.gz -C /var/lib/grafana/plugins/ rm /var/lib/grafana/plugins/plugin.tar.gz volumeMounts: name: storage mountPath: /var/lib/grafana ``` For `grafana-operator` users, the above configuration should be done for the part `/spec/deployment/spec/template/spec/initContainers` of your `kind=Grafana` resource. This example uses init container to download and install plugin. To allow Grafana using this container as a sidecar set the following config: ```yaml sidecar: datasources: initDatasources: true enabled: true ``` See more about chart settings Another option would be to build custom Grafana image with plugin based on same installation instructions. Example with Grafana : ```yaml apiVersion: grafana.integreatly.org/v1beta1 kind: Grafana metadata: name: grafana-vm spec: persistentVolumeClaim: spec: accessModes: ReadWriteOnce resources: requests: storage: 200Mi deployment: spec: template: spec: initContainers: name: "load-vm-ds-plugin" image: "curlimages/curl:7.85.0" command: [ "/bin/sh" ] workingDir: "/var/lib/grafana" securityContext: runAsUser: 10001 runAsNonRoot: true runAsGroup: 10001 args: "-c" | set -ex mkdir -p /var/lib/grafana/plugins/ ver=$(curl -s https://api.github.com/repos/VictoriaMetrics/grafana-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1) curl -L https://github.com/VictoriaMetrics/grafana-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/plugin.tar.gz tar -xf /var/lib/grafana/plugins/plugin.tar.gz -C /var/lib/grafana/plugins/ rm /var/lib/grafana/plugins/plugin.tar.gz volumeMounts: name: grafana-data mountPath: /var/lib/grafana config: plugins: allowloadingunsigned_plugins: victoriametrics-datasource ``` See to find more about Grafana operator. This example uses init container to download and install plugin. To download plugin build and move contents into Grafana plugins directory: ``` bash ver=$(curl -s https://api.github.com/repos/VictoriaMetrics/grafana-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1) curl -L https://github.com/VictoriaMetrics/grafana-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/plugin.tar.gz tar -xf /var/lib/grafana/plugins/plugin.tar.gz -C /var/lib/grafana/plugins/ rm

``` Restart Grafana Installing dev version of Grafana plugin requires to change `grafana.ini` config to allow loading unsigned plugins: ``` ini plugins = {{path to directory with plugin}} ``` ``` ini [plugins] allowloadingunsigned_plugins = victoriametrics-datasource ``` In the project directory, you can run: ``` yarn install yarn dev yarn build:zip ``` From the root folder of the project run the following command: ``` make victoriametrics-backend-plugin-build ``` This command will build executable multi-platform files to the `dist` folder for the following platforms: linux/amd64 linux/arm64 linux/arm linux/386 amd64 arm64 windows From the root folder of the project run the following command: ``` make victorimetrics-frontend-plugin-build ``` This command will build all frontend app into `dist` folder. When frontend and backend parts of the plugin is required, run the following command from the root folder of the project: ``` make victoriametrics-datasource-plugin-build ``` This command will build frontend part and backend part or the plugin and locate both parts into `dist` folder. The `WITH` templates feature simplifies the construction and management of complex queries. You can try this feature in the . The "WITH templates" section allows you to create expressions with templates that can be used in dashboards. WITH expressions are stored in the datasource object. If the dashboard gets exported, the associated WITH templates will not be included in the resulting JSON (due to technical limitations) and need to be migrated separately. Navigate to the dashboard where you want to add a template.<br/> Note: templates are available within the dashboard scope. Click the `WITH templates` button. Enter the expression in the input field. Once done, press the `Save` button to apply the changes. For example: ``` commonFilters = {instance=~"$node:$port",job=~"$job"}, cpuCount = count(count(nodecpuseconds_total{commonFilters}) by (cpu)), cpuIdle = sum(rate(nodecpuseconds_total{mode='idle',commonFilters}[5m])) ``` You can specify a comment before the variable and use markdown in it. The comment will be displayed as a hint during auto-completion. The comment can span multiple lines. After saving the template, you can enter it into the query editor field: ``` ((cpuCount - cpuIdle) * 100) / cpuCount ``` Thus, the entire query will look as follows: ``` WITH ( commonFilters = {instance=~"$node:$port",job=~"$job"}, cpuCount = count(count(nodecpuseconds_total{commonFilters}) by (cpu)), cpuIdle = sum(rate(nodecpuseconds_total{mode='idle',commonFilters}[5m])) ) ((cpuCount - cpuIdle) * 100) / cpuCount ``` To view the raw query in the interface, enable the `Raw` toggle. Make sure there are no open security issues. Create a release tag: `git tag -s v1.xx.y` in `master` branch Run `TAG=v1.xx.y make build-release` to build and package binaries in `*.tar.gz` release archives. Run `git push origin v1.xx.y` to push the tag created `v1.xx.y` at step 2 to public GitHub repository Go to <https://github.com/VictoriaMetrics/grafana-datasource/releases> and verify that draft release with the name `TAG` has been created and this release contains all the needed binaries and checksums. Remove the `draft` checkbox for the `TAG` release and manually publish it. Based on our previous experience of the signing procedure was a formal act. But when we tried we were told by GrafanaLabs representative the plugin falls into a Commercial signature level. It matters not if plugin or VictoriaMetrics itself are opensource. The announced cost of Commercial signature level was much higher than expected, so we interrupted the procedure. Make sure that VictoriaMetrics datasource plugin is , and a new datasource is created from the plugin. Each panel in Grafana dashboard has a datasource dropdown when in Edit mode. Just choose the VictoriaMetrics datasource instead of Prometheus datasource in dropdown. If datasource is configured via Grafana variable, then change variable to VictoriaMetrics datasource type. Grafana doesn't allow forwarding Alert requests to alerting API /api/v1/rules for plugins which are not of Prometheus or Loki type. See