Repository: VictoriaMetrics/VictoriaMetrics
Pull request #1226 (2021-04-16 11:39:10)

adds query params support for vmauth urlPrefix

codecov[bot]: # [Codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) Report
> Merging [#1226](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) (d7f4931) into [master](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/commit/650c143f6d7ff63ad6ffc07f0acbbb54964d62f2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) (650c143) will **decrease** coverage by `0.03%`.
> The diff coverage is `86.66%`.

[![Impacted file tree graph](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226/graphs/tree.svg?width=650&height=150&src=pr&token=YND8dYVcXn&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics)

```diff
@@            Coverage Diff             @@
##           master    #1226      +/-   ##
==========================================
- Coverage   59.40%   59.37%   -0.04%     
==========================================
  Files         215      215              
  Lines       25557    25570      +13     
==========================================
- Hits        15183    15181       -2     
- Misses       8970     8982      +12     
- Partials     1404     1407       +3     
```


| [Impacted Files](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) | Coverage Δ | |
|---|---|---|
| [app/vmauth/target\_url.go](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics#diff-YXBwL3ZtYXV0aC90YXJnZXRfdXJsLmdv) | `83.33% <86.66%> (+1.51%)` | :arrow_up: |
| [app/vmselect/promql/aggr\_incremental.go](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics#diff-YXBwL3Ztc2VsZWN0L3Byb21xbC9hZ2dyX2luY3JlbWVudGFsLmdv) | `84.88% <0.00%> (-3.11%)` | :arrow_down: |
| [lib/mergeset/table.go](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics#diff-bGliL21lcmdlc2V0L3RhYmxlLmdv) | `64.05% <0.00%> (-1.08%)` | :arrow_down: |
| [app/vmalert/remotewrite/remotewrite.go](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics#diff-YXBwL3ZtYWxlcnQvcmVtb3Rld3JpdGUvcmVtb3Rld3JpdGUuZ28=) | `58.87% <0.00%> (+1.86%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics). Last update [b3d8861...d7f4931](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/1226?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics).


Patch:
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 445254bcd..f75a7ef1b 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -62,6 +62,5 @@ jobs:
       - name: Publish coverage
         uses: codecov/codecov-action@v1.0.6
         with:
-          token: ${{secrets.CODECOV_TOKEN}}
           file: ./coverage.txt
 
diff --git a/app/vmauth/example_config.yml b/app/vmauth/example_config.yml
index ba8e12217..5eae8e1b2 100644
--- a/app/vmauth/example_config.yml
+++ b/app/vmauth/example_config.yml
@@ -11,6 +11,14 @@ users:
   password: "***"
   url_prefix: "http://localhost:8428"
 
+  # The user for querying local single-node VictoriaMetrics with extra_label team=dev.
+  # All the requests to http://vmauth:8427 with the given Basic Auth (username:password)
+  # will be routed to http://localhost:8428 with extra_label=team=dev query arg.
+  # For example, http://vmauth:8427/api/v1/query is routed to http://localhost:8428/api/v1/query?extra_label=team=dev
+- username: "local-single-node"
+  password: "***"
+  url_prefix: "http://localhost:8428?extra_label=team=dev"
+
   # The user for querying account 123 in VictoriaMetrics cluster
   # See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format
   # All the requests to http://vmauth:8427 with the given Basic Auth (username:password)
@@ -28,4 +36,3 @@ users:
 - username: "cluster-insert-account-42"
   password: "***"
   url_prefix: "http://vminsert:8480/insert/42/prometheus"
-
diff --git a/app/vmauth/target_url.go b/app/vmauth/target_url.go
index 9f4db4dcf..28fbdb47c 100644
--- a/app/vmauth/target_url.go
+++ b/app/vmauth/target_url.go
@@ -7,6 +7,32 @@ import (
 	"strings"
 )
 
+func mergeURLs(uiURL string, requestURI *url.URL) (string, error) {
+	prefixURL, err := url.Parse(uiURL)
+	if err != nil {
+		return "", fmt.Errorf("BUG - cannot parse userInfo url: %q, err: %w", uiURL, err)
+	}
+	prefixURL.Path += requestURI.Path
+	requestParams := requestURI.Query()
+	// fast path
+	if len(requestParams) == 0 {
+		return prefixURL.String(), nil
+	}
+	// merge query parameters from requests.
+	userInfoParams := prefixURL.Query()
+	for k, v := range requestParams {
+		// skip clashed query params from original request
+		if exist := userInfoParams.Get(k); len(exist) > 0 {
+			continue
+		}
+		for i := range v {
+			userInfoParams.Add(k, v[i])
+		}
+	}
+	prefixURL.RawQuery = userInfoParams.Encode()
+	return prefixURL.String(), nil
+}
+
 func createTargetURL(ui *UserInfo, uOrig *url.URL) (string, error) {
 	u, err := url.Parse(uOrig.String())
 	if err != nil {
@@ -20,12 +46,12 @@ func createTargetURL(ui *UserInfo, uOrig *url.URL) (string, error) {
 	for _, e := range ui.URLMap {
 		for _, sp := range e.SrcPaths {
 			if sp.match(u.Path) {
-				return e.URLPrefix + u.RequestURI(), nil
+				return mergeURLs(e.URLPrefix, u)
 			}
 		}
 	}
 	if len(ui.URLPrefix) > 0 {
-		return ui.URLPrefix + u.RequestURI(), nil
+		return mergeURLs(ui.URLPrefix, u)
 	}
 	return "", fmt.Errorf("missing route for %q", u)
 }
diff --git a/dashboards/victoriametrics.json b/dashboards/victoriametrics.json
index 9bab26979..8a6182751 100644
--- a/dashboards/victoriametrics.json
+++ b/dashboards/victoriametrics.json
@@ -3028,7 +3028,7 @@
           "yaxes": [
             {
               "decimals": 2,
-              "format": "percentunit",
+              "format": "short",
               "label": null,
               "logBase": 1,
               "max": null,
@@ -3399,6 +3399,7 @@
           "dashLength": 10,
           "dashes": false,
           "datasource": "$ds",
+          "description": "Shows avg GC duration",
           "fieldConfig": {
             "defaults": {
               "custom": {},
@@ -3440,10 +3441,10 @@
           "steppedLine": false,
           "targets": [
             {
-              "expr": "sum(rate(go_gc_duration_seconds_sum{job=\"$job\", instance=\"$instance\"}[2m]))",
+              "expr": "sum(rate(go_gc_duration_seconds_sum{job=\"$job\", instance=\"$instance\"}[5m]))\n/\nsum(rate(go_gc_duration_seconds_count{job=\"$job\", instance=\"$instance\"}[5m]))",
               "format": "time_series",
               "intervalFactor": 2,
-              "legendFormat": "gc duration",
+              "legendFormat": "avg gc duration",
               "refId": "A"
             }
           ],
diff --git a/docs/Cluster-VictoriaMetrics.md b/docs/Cluster-VictoriaMetrics.md
index 0368eb35b..b6905263b 100644
--- a/docs/Cluster-VictoriaMetrics.md
+++ b/docs/Cluster-VictoriaMetrics.md
@@ -418,7 +418,7 @@ curl -s http://vminsert:8480/debug/pprof/heap > mem.pprof
 
 ## Community and contributions
 
-We are open to third-party pull requests provided they follow [KISS design principle](https://en.wikipedia.org/wiki/KISS_principle):
+We are open to third-party pull requests provided they follow the [KISS design principle](https://en.wikipedia.org/wiki/KISS_principle):
 
 - Prefer simple code and architecture.
 - Avoid complex abstractions.
@@ -427,13 +427,13 @@ We are open to third-party pull requests provided they follow [KISS design princ
 - Minimize the number of moving parts in the distributed system.
 - Avoid automated decisions, which may hurt cluster availability, consistency or performance.
 
-Adhering `KISS` principle simplifies the resulting code and architecture, so it can be reviewed, understood and verified by many people.
+Adhering to the `KISS` principle simplifies the resulting code and architecture, so it can be reviewed, understood and verified by many people.
 
-Due to `KISS` cluster version of VictoriaMetrics has no the following "features" popular in distributed computing world:
+Due to `KISS`, cluster version of VictoriaMetrics has no the following "features" popular in distributed computing world:
 
 - Fragile gossip protocols. See [failed attempt in Thanos](https://github.com/improbable-eng/thanos/blob/030bc345c12c446962225221795f4973848caab5/docs/proposals/completed/201809_gossip-removal.md).
 - Hard-to-understand-and-implement-properly [Paxos protocols](https://www.quora.com/In-distributed-systems-what-is-a-simple-explanation-of-the-Paxos-algorithm).
-- Complex replication schemes, which may go nuts in unforesseen edge cases. See [replication docs](#replication-and-data-safety) for details.
+- Complex replication schemes, which may go nuts in unforeseen edge cases. See [replication docs](#replication-and-data-safety) for details.
 - Automatic data reshuffling between storage nodes, which may hurt cluster performance and availability.
 - Automatic cluster resizing, which may cost you a lot of money if improperly configured.
 - Automatic discovering and addition of new nodes in the cluster, which may mix data between dev and prod clusters :)


Test patch:
diff --git a/app/vmauth/target_url_test.go b/app/vmauth/target_url_test.go
index 34d2e6c33..9c8d454a5 100644
--- a/app/vmauth/target_url_test.go
+++ b/app/vmauth/target_url_test.go
@@ -38,7 +38,7 @@ func TestCreateTargetURLSuccess(t *testing.T) {
 	}, "/../../aaa", "https://sss:3894/x/y/aaa")
 	f(&UserInfo{
 		URLPrefix: "https://sss:3894/x/y",
-	}, "/./asd/../../aaa?a=d&s=s/../d", "https://sss:3894/x/y/aaa?a=d&s=s/../d")
+	}, "/./asd/../../aaa?a=d&s=s/../d", "https://sss:3894/x/y/aaa?a=d&s=s%2F..%2Fd")
 
 	// Complex routing with `url_map`
 	ui := &UserInfo{
@@ -77,6 +77,13 @@ func TestCreateTargetURLSuccess(t *testing.T) {
 	f(ui, "/api/v1/label/foo/values", "http://vmselect/0/prometheus/api/v1/label/foo/values")
 	f(ui, "/api/v1/write", "http://vminsert/0/prometheus/api/v1/write")
 	f(ui, "/api/v1/foo/bar", "http://default-server/api/v1/foo/bar")
+	f(&UserInfo{
+		URLPrefix: "http://foo.bar?extra_label=team=dev",
+	}, "/api/v1/query", "http://foo.bar/api/v1/query?extra_label=team=dev")
+	f(&UserInfo{
+		URLPrefix: "http://foo.bar?extra_label=team=mobile",
+	}, "/api/v1/query?extra_label=team=dev", "http://foo.bar/api/v1/query?extra_label=team%3Dmobile")
+
 }
 
 func TestCreateTargetURLFailure(t *testing.T) {