Repository: VictoriaMetrics/VictoriaMetrics
Pull request #5363 (2023-11-21 10:25:42)

deployment/docker: add grafana datasource to the docker-compose files
Add [VictoriaMetrics datasource](https://github.com/VictoriaMetrics/grafana-datasource) to the docker-compose environment.

I have separated environments for data sources and the default setup. I used the docker compose merge functionality, and it helps to avoid many tricks via bash

Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3920, https://github.com/VictoriaMetrics/grafana-datasource/issues/113

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/5363?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base [(`41f7940`)](https://app.codecov.io/gh/VictoriaMetrics/VictoriaMetrics/commit/41f7940f973404a7b93f446f2f9055842d72c103?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) 60.09% compared to head [(`df21f02`)](https://app.codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/5363?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) 60.08%.


<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #5363      +/-   ##
==========================================
- Coverage   60.09%   60.08%   -0.01%     
==========================================
  Files         401      401              
  Lines       75241    75241              
==========================================
- Hits        45215    45210       -5     
- Misses      27509    27511       +2     
- Partials     2517     2520       +3     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/5363?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics).

hagen1778: @dmitryk-dk is it possible to use something similar to init container - a script which will download and install the plugin as we recommend doing here https://github.com/VictoriaMetrics/grafana-datasource#install-in-kubernetes ?
dmitryk-dk: > @dmitryk-dk is it possible to use something similar to init container - a script which will download and install the plugin as we recommend doing here https://github.com/VictoriaMetrics/grafana-datasource#install-in-kubernetes ?

As far as I know, Docker doesn't have that concept as Kubernetes has. So it would be not possible to make it in that manner.

Maybe @zekker6 knows more about it.
Haleygo: > > @dmitryk-dk is it possible to use something similar to init container - a script which will download and install the plugin as we recommend doing here https://github.com/VictoriaMetrics/grafana-datasource#install-in-kubernetes ?
> 
> As far as I know, Docker doesn't have that concept as Kubernetes has. So it would be not possible to make it in that manner.
> 
> Maybe @zekker6 knows more about it.

Yeah, for docker, we can override images's default entrypoint and cmd with our script.
Could be something like
```
services:
  grafana:
    container_name: grafana
    image: grafana/grafana:latest
    ports:
      - 3000:3000
    entrypoint: ["/bin/bash", "-c"]
    command: ["chmod +x /download-vm-datasourcet.sh && /download-vm-datasource.sh && sleep 1000"]
    volumes:
      - ./download-vm-datasource.sh:/download-vm-datasource.sh
```
 
ps. we should also upgrade grafana image to [9.5.0+](https://hub.docker.com/layers/grafana/grafana/9.5.0/images/sha256-258a4c283403a26848bd6a9abb6b6867d21ec83ca228d14da4e66589a49676ef?context=explore) if we use this way, since it [gained `curl`](https://github.com/grafana/grafana/pull/65547) about that time. 
dmitryk-dk: > > > @dmitryk-dk is it possible to use something similar to init container - a script which will download and install the plugin as we recommend doing here https://github.com/VictoriaMetrics/grafana-datasource#install-in-kubernetes ?
> > 
> > 
> > As far as I know, Docker doesn't have that concept as Kubernetes has. So it would be not possible to make it in that manner.
> > Maybe @zekker6 knows more about it.
> 
> Yeah, for docker, we can override images's default entrypoint and cmd with our script in [docker-compose yaml](https://docs.docker.com/compose/compose-file/05-services/#entrypoint). Could be something like
> 
> ```
> services:
>   grafana:
>     container_name: grafana
>     image: grafana/grafana:latest
>     ports:
>       - 3000:3000
>     entrypoint: ["/bin/bash", "-c"]
>     command: ["chmod +x /downloadvmdatasource.sh && /downloadvmdatasource.sh && /run.sh"]
>     volumes:
>       - ./downloadvmdatasource.sh:/downloadvmdatasource.sh
> ```
> 
> ps. we should also upgrade grafana image to [9.5.0+](https://hub.docker.com/layers/grafana/grafana/9.5.0/images/sha256-258a4c283403a26848bd6a9abb6b6867d21ec83ca228d14da4e66589a49676ef?context=explore) if we use this way, since it [gained `curl`](https://github.com/grafana/grafana/pull/65547) about that time.

Hi @Haleygo ! I have simplified logic and added the download script as you suggested. can you take a look please.

I tested it on my local env it works well
Haleygo: Looking good!
Little proposal, how about switching them and making victoriametrics datasource as the default one with `docker-cluster-up`? 
dmitryk-dk: > Looking good! Little proposal, how about switching them and making victoriametrics datasource as the default one with `docker-cluster-up`?

We splitted realisation for two envs specially, because our plugin is unsigned and some of the users do not want to use it. So they can choose what of the datasource to use.

Patch:
diff --git a/deployment/docker/Makefile b/deployment/docker/Makefile
index 166407e04..2b60d2435 100644
--- a/deployment/docker/Makefile
+++ b/deployment/docker/Makefile
@@ -187,8 +187,20 @@ docker-single-up:
 docker-single-down:
 	$(DOCKER_COMPOSE) -f deployment/docker/docker-compose.yml down -v
 
+docker-single-vm-datasource-up:
+	$(DOCKER_COMPOSE) -f deployment/docker/docker-compose.yml -f deployment/docker/vm-datasource/docker-compose-with-datasource.yml up -d
+
+docker-single-vm-datasource-down:
+	$(DOCKER_COMPOSE) -f deployment/docker/docker-compose.yml -f deployment/docker/vm-datasource/docker-compose-with-datasource.yml down -v
+
 docker-cluster-up:
 	$(DOCKER_COMPOSE) -f deployment/docker/docker-compose-cluster.yml up -d
 
 docker-cluster-down:
 	$(DOCKER_COMPOSE) -f deployment/docker/docker-compose-cluster.yml down -v
+
+docker-cluster-vm-datasource-up:
+	$(DOCKER_COMPOSE) -f deployment/docker/docker-compose-cluster.yml -f deployment/docker/vm-datasource/docker-compose-cluster-with-datasource.yml up -d
+
+docker-cluster-vm-datasource-down:
+	$(DOCKER_COMPOSE) -f deployment/docker/docker-compose-cluster.yml -f deployment/docker/vm-datasource/docker-compose-cluster-with-datasource.yml down -v
diff --git a/deployment/docker/docker-compose-cluster.yml b/deployment/docker/docker-compose-cluster.yml
index b50e35b15..584e46bcc 100644
--- a/deployment/docker/docker-compose-cluster.yml
+++ b/deployment/docker/docker-compose-cluster.yml
@@ -17,7 +17,7 @@ services:
 
   grafana:
     container_name: grafana
-    image: grafana/grafana:9.2.7
+    image: grafana/grafana:9.5.15
     depends_on:
       - "vmauth"
     ports:
@@ -25,7 +25,8 @@ services:
     restart: always
     volumes:
       - grafanadata:/var/lib/grafana
-      - ./provisioning/:/etc/grafana/provisioning/
+      - ./provisioning/datasources/prometheus-datasource:/etc/grafana/provisioning/datasources
+      - ./provisioning/dashboards:/etc/grafana/provisioning/dashboards
       - ./../../dashboards/victoriametrics-cluster.json:/var/lib/grafana/dashboards/vm.json
       - ./../../dashboards/vmagent.json:/var/lib/grafana/dashboards/vmagent.json
       - ./../../dashboards/vmalert.json:/var/lib/grafana/dashboards/vmalert.json
diff --git a/deployment/docker/docker-compose.yml b/deployment/docker/docker-compose.yml
index 125349135..dd786c7ec 100644
--- a/deployment/docker/docker-compose.yml
+++ b/deployment/docker/docker-compose.yml
@@ -40,14 +40,15 @@ services:
     restart: always
   grafana:
     container_name: grafana
-    image: grafana/grafana:9.2.7
+    image: grafana/grafana:9.5.15
     depends_on:
       - "victoriametrics"
     ports:
       - 3000:3000
     volumes:
       - grafanadata:/var/lib/grafana
-      - ./provisioning/:/etc/grafana/provisioning/
+      - ./provisioning/datasources/prometheus-datasource:/etc/grafana/provisioning/datasources
+      - ./provisioning/dashboards:/etc/grafana/provisioning/dashboards
       - ./../../dashboards/victoriametrics.json:/var/lib/grafana/dashboards/vm.json
       - ./../../dashboards/victorialogs.json:/var/lib/grafana/dashboards/vl.json
       - ./../../dashboards/vmagent.json:/var/lib/grafana/dashboards/vmagent.json
diff --git a/deployment/docker/provisioning/datasources/datasource.yml b/deployment/docker/provisioning/datasources/prometheus-datasource/prometheus-datasource.yml
similarity index 100%
rename from deployment/docker/provisioning/datasources/datasource.yml
rename to deployment/docker/provisioning/datasources/prometheus-datasource/prometheus-datasource.yml
diff --git a/deployment/docker/vm-datasource/config/victoriametrics-datasource.yml b/deployment/docker/vm-datasource/config/victoriametrics-datasource.yml
new file mode 100644
index 000000000..8749c63b1
--- /dev/null
+++ b/deployment/docker/vm-datasource/config/victoriametrics-datasource.yml
@@ -0,0 +1,36 @@
+apiVersion: 1
+
+# List of data sources to insert/update depending on what's
+# available in the database.
+datasources:
+  # <string, required> Name of the VictoriaMetrics datasource
+  # displayed in Grafana panels and queries.
+  - name: VictoriaMetrics
+    # <string, required> Sets the data source type.
+    type: victoriametrics-datasource
+      # <string, required> Sets the access mode, either
+      # proxy or direct (Server or Browser in the UI).
+    # Some data sources are incompatible with any setting
+    # but proxy (Server).
+    access: proxy
+    # <string> Sets default URL of the single node version of VictoriaMetrics
+    url: http://victoriametrics:8428
+    # <string> Sets the pre-selected datasource for new panels.
+    # You can set only one default data source per organization.
+    isDefault: true
+
+    # <string, required> Name of the VictoriaMetrics datasource
+    # displayed in Grafana panels and queries.
+  - name: VictoriaMetrics - cluster
+    # <string, required> Sets the data source type.
+    type: victoriametrics-datasource
+    # <string, required> Sets the access mode, either
+    # proxy or direct (Server or Browser in the UI).
+    # Some data sources are incompatible with any setting
+    # but proxy (Server).
+    access: proxy
+    # <string> Sets default URL of the cluster version of VictoriaMetrics
+    url: http://vmauth:8427/select/0/prometheus
+    # <string> Sets the pre-selected datasource for new panels.
+    # You can set only one default data source per organization.
+    isDefault: false
diff --git a/deployment/docker/vm-datasource/docker-compose-cluster-with-datasource.yml b/deployment/docker/vm-datasource/docker-compose-cluster-with-datasource.yml
new file mode 100644
index 000000000..2da5a8ae2
--- /dev/null
+++ b/deployment/docker/vm-datasource/docker-compose-cluster-with-datasource.yml
@@ -0,0 +1,24 @@
+version: "3.5"
+services:
+  grafana:
+    container_name: grafana
+    image: grafana/grafana:9.5.15
+    depends_on:
+      - "vmauth"
+    ports:
+      - 3000:3000
+    entrypoint: ["/bin/bash", "-c"]
+    command: ["chmod +x /download-vm-datasource.sh && /download-vm-datasource.sh && /run.sh"]
+    volumes:
+      - grafanadata:/var/lib/grafana
+      - ./vm-datasource/config:/etc/grafana/provisioning/datasources
+      - ./provisioning/dashboards:/etc/grafana/provisioning/dashboards
+      - ./provisioning/plugins/:/var/lib/grafana/plugins
+      - ./../../dashboards/vm/victoriametrics-cluster.json:/var/lib/grafana/dashboards/vm.json
+      - ./../../dashboards/vm/vmagent.json:/var/lib/grafana/dashboards/vmagent.json
+      - ./../../dashboards/vm/vmalert.json:/var/lib/grafana/dashboards/vmalert.json
+      - ./vm-datasource/scripts/download-vm-datasource.sh:/download-vm-datasource.sh
+    environment:
+      - "GF_ALLOW_LOADING_UNSIGNED_PLUGINS=victoriametrics-datasource"
+      - "GF_DEFAULT_APP_MODE=development"
+    restart: always
diff --git a/deployment/docker/vm-datasource/docker-compose-with-datasource.yml b/deployment/docker/vm-datasource/docker-compose-with-datasource.yml
new file mode 100644
index 000000000..4822803b2
--- /dev/null
+++ b/deployment/docker/vm-datasource/docker-compose-with-datasource.yml
@@ -0,0 +1,26 @@
+version: "3.5"
+services:
+  grafana:
+    container_name: grafana
+    image: grafana/grafana:9.5.15
+    depends_on:
+      - "victoriametrics"
+    ports:
+      - 3000:3000
+    entrypoint: [ "/bin/bash", "-c" ]
+    command: [ "chmod +x /download-vm-datasource.sh && /download-vm-datasource.sh && /run.sh" ]
+    volumes:
+      - grafanadata:/var/lib/grafana
+      - ./vm-datasource/config:/etc/grafana/provisioning/datasources
+      - ./provisioning/dashboards:/etc/grafana/provisioning/dashboards
+      - ./provisioning/plugins/:/var/lib/grafana/plugins
+      - ./../../dashboards/vm/victoriametrics.json:/var/lib/grafana/dashboards/vm.json
+      - ./../../dashboards/vm/vmagent.json:/var/lib/grafana/dashboards/vmagent.json
+      - ./../../dashboards/vm/vmalert.json:/var/lib/grafana/dashboards/vmalert.json
+      - ./vm-datasource/scripts/download-vm-datasource.sh:/download-vm-datasource.sh
+    environment:
+      - "GF_ALLOW_LOADING_UNSIGNED_PLUGINS=victoriametrics-datasource"
+      - "GF_DEFAULT_APP_MODE=development"
+    networks:
+      - vm_net
+    restart: always
diff --git a/deployment/docker/vm-datasource/scripts/download-vm-datasource.sh b/deployment/docker/vm-datasource/scripts/download-vm-datasource.sh
new file mode 100755
index 000000000..21f9f1ae6
--- /dev/null
+++ b/deployment/docker/vm-datasource/scripts/download-vm-datasource.sh
@@ -0,0 +1,22 @@
+#!/bin/bash
+set -ex
+
+VM_DS_PATH='/var/lib/grafana/plugins/victoriametrics-datasource'
+PLUGIN_PATH='/var/lib/grafana/plugins'
+
+if [[ -f ${VM_DS_PATH}/plugin.json ]]; then
+    ver=$(cat ${VM_DS_PATH}/plugin.json)
+    if [[ ! -z "$ver" ]]; then
+    exit
+    fi
+fi
+
+echo "Victoriametrics datasource is not installed. Installing datasource..."
+rm -rf ${VM_DS_PATH}/* || true
+mkdir -p ${VM_DS_PATH}
+
+export LATEST_VERSION=$(curl https://api.github.com/repos/VictoriaMetrics/grafana-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1); \
+curl -L https://github.com/VictoriaMetrics/grafana-datasource/releases/download/${LATEST_VERSION}/victoriametrics-datasource-${LATEST_VERSION}.tar.gz -o ${PLUGIN_PATH}/plugin.tar.gz && \
+tar -xzf ${PLUGIN_PATH}/plugin.tar.gz -C ${PLUGIN_PATH}
+echo "Victoriametrics datasource has been installed."
+rm ${PLUGIN_PATH}/plugin.tar.gz
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 60f50c545..59babec40 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -53,6 +53,7 @@ The sandbox cluster installation is running under the constant load generated by
 * FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): rename cmd-line flag `vm-native-disable-retries` to `vm-native-disable-per-metric-migration` to better reflect its meaning.
 * FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): add `-vm-native-src-insecure-skip-verify` and `-vm-native-dst-insecure-skip-verify` command-line flags for native protocol. It can be used for skipping TLS certificate verification when connecting to the source or destination addresses.
 * FEATURE: [Alerting rules for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts): add `job` label to `DiskRunsOutOfSpace` alerting rule, so it is easier to understand to which installation the triggered instance belongs.
+* FEATURE: add [VictoriaMetrics datasource](https://github.com/VictoriaMetrics/grafana-datasource) to docker compose environment. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5363).
 
 * BUGFIX: properly return errors from [export APIs](https://docs.victoriametrics.com/#how-to-export-time-series). Previously these errors were silently suppressed. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5649).
 * BUGFIX: [VictoriaMetrics cluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html): properly return full results when `-search.skipSlowReplicas` command-line flag is passed to `vmselect` and when [vmstorage groups](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#vmstorage-groups-at-vmselect) are in use. Previously partial results could be returned in this case.


Test patch:
diff --git a/app/vmctl/backoff/backoff_test.go b/app/vmctl/backoff/backoff_test.go
index 6d6d001a1..3506c4a18 100644
--- a/app/vmctl/backoff/backoff_test.go
+++ b/app/vmctl/backoff/backoff_test.go
@@ -15,7 +15,6 @@ func TestRetry_Do(t *testing.T) {
 		backoffFactor      float64
 		backoffMinDuration time.Duration
 		retryableFunc      retryableFunc
-		ctx                context.Context
 		cancelTimeout      time.Duration
 		want               uint64
 		wantErr            bool
@@ -25,7 +24,6 @@ func TestRetry_Do(t *testing.T) {
 			retryableFunc: func() error {
 				return ErrBadRequest
 			},
-			ctx:     context.Background(),
 			want:    0,
 			wantErr: true,
 		},
@@ -35,7 +33,6 @@ func TestRetry_Do(t *testing.T) {
 				time.Sleep(time.Millisecond * 100)
 				return nil
 			},
-			ctx:     context.Background(),
 			want:    0,
 			wantErr: true,
 		},
@@ -58,7 +55,6 @@ func TestRetry_Do(t *testing.T) {
 				}
 				return nil
 			},
-			ctx:     context.Background(),
 			want:    1,
 			wantErr: false,
 		},
@@ -75,7 +71,6 @@ func TestRetry_Do(t *testing.T) {
 				}
 				return nil
 			},
-			ctx:     context.Background(),
 			want:    5,
 			wantErr: true,
 		},
@@ -85,14 +80,8 @@ func TestRetry_Do(t *testing.T) {
 			backoffFactor:      1.7,
 			backoffMinDuration: time.Millisecond * 10,
 			retryableFunc: func() error {
-				t := time.NewTicker(time.Millisecond * 5)
-				defer t.Stop()
-				for range t.C {
-					return fmt.Errorf("got some error")
-				}
-				return nil
+				return fmt.Errorf("got some error")
 			},
-			ctx:           context.Background(),
 			cancelTimeout: time.Millisecond * 40,
 			want:          3,
 			wantErr:       true,
@@ -101,12 +90,13 @@ func TestRetry_Do(t *testing.T) {
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			r := &Backoff{retries: tt.backoffRetries, factor: tt.backoffFactor, minDuration: tt.backoffMinDuration}
+			ctx := context.Background()
 			if tt.cancelTimeout != 0 {
-				newCtx, cancelFn := context.WithTimeout(tt.ctx, tt.cancelTimeout)
-				tt.ctx = newCtx
+				newCtx, cancelFn := context.WithTimeout(context.Background(), tt.cancelTimeout)
+				ctx = newCtx
 				defer cancelFn()
 			}
-			got, err := r.Retry(tt.ctx, tt.retryableFunc)
+			got, err := r.Retry(ctx, tt.retryableFunc)
 			if (err != nil) != tt.wantErr {
 				t.Errorf("Retry() error = %v, wantErr %v", err, tt.wantErr)
 				return