Repository: VictoriaMetrics/VictoriaMetrics
Pull request #3983 (2023-03-20 22:08:50)

app/vmctl: follow up after aed59b90298a6b5e44ffea4543a6aa3b9e402e65
Fixed windows build for vmctl

codecov[bot]: ## [Codecov](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/3983?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) Report
Patch coverage has no change and project coverage change: **`-0.07`** :warning:
> Comparison is base [(`9153353`)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/commit/91533531f56d32206b711a15ebaadddf310ddd34?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) 58.59% compared to head [(`ddf36e4`)](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/3983?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics) 58.52%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #3983      +/-   ##
==========================================
- Coverage   58.59%   58.52%   -0.07%     
==========================================
  Files         314      313       -1     
  Lines       52575    52571       -4     
==========================================
- Hits        30806    30769      -37     
- Misses      19861    19895      +34     
+ Partials     1908     1907       -1     
```


[see 8 files with indirect coverage changes](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/3983/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics)

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics)

</details>

[:umbrella: View full report in Codecov by Sentry](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics/pull/3983?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=VictoriaMetrics).


Patch:
diff --git a/app/vmagent/README.md b/app/vmagent/README.md
index d3727a6d0..b6b954354 100644
--- a/app/vmagent/README.md
+++ b/app/vmagent/README.md
@@ -637,18 +637,19 @@ provide the following tools for debugging target-level and metric-level relabeli
 - Target-level debugging (e.g. `relabel_configs` section at [scrape_configs](https://docs.victoriametrics.com/sd_configs.html#scrape_configs))
   can be performed by navigating to `http://vmagent:8429/targets` page (`http://victoriametrics:8428/targets` page for single-node VictoriaMetrics)
   and clicking the `debug target relabeling` link at the target, which must be debugged.
-  The opened page will show step-by-step results for the actual target relabeling rules applied to the discovered target labels.
+  The opened page shows step-by-step results for the actual target relabeling rules applied to the discovered target labels.
+  The page shows also the target URL generated after applying all the relabeling rules.
 
   The `http://vmagent:8429/targets` page shows only active targets. If you need to understand why some target
   is dropped during the relabeling, then navigate to `http://vmagent:8428/service-discovery` page
   (`http://victoriametrics:8428/service-discovery` for single-node VictoriaMetrics), find the dropped target
-  and click the `debug` link there. The opened page will show step-by-step results for the actual relabeling rules,
+  and click the `debug` link there. The opened page shows step-by-step results for the actual relabeling rules,
   which result to target drop.
 
 - Metric-level debugging (e.g. `metric_relabel_configs` section at [scrape_configs](https://docs.victoriametrics.com/sd_configs.html#scrape_configs)
   can be performed by navigating to `http://vmagent:8429/targets` page (`http://victoriametrics:8428/targets` page for single-node VictoriaMetrics)
   and clicking the `debug metrics relabeling` link at the target, which must be debugged.
-  The opened page will show step-by-step results for the actual metric relabeling rules applied to the given target labels.
+  The opened page shows step-by-step results for the actual metric relabeling rules applied to the given target labels.
 
 ## Prometheus staleness markers
 
diff --git a/app/vmctl/terminal/terminal.go b/app/vmctl/terminal/terminal.go
index 2f3e26eaa..6b3f9ad85 100644
--- a/app/vmctl/terminal/terminal.go
+++ b/app/vmctl/terminal/terminal.go
@@ -1,14 +1,6 @@
-//go:build darwin || linux || solaris
-// +build darwin linux solaris
-
 package terminal
 
-import (
-	"golang.org/x/sys/unix"
-)
-
 // IsTerminal returns true if the file descriptor is terminal
 func IsTerminal(fd int) bool {
-	_, err := unix.IoctlGetTermios(fd, ioctlReadTermios)
-	return err == nil
+	return isTerminal(fd)
 }
diff --git a/app/vmctl/terminal/unix.go b/app/vmctl/terminal/unix.go
index 610e17f8c..ad8e64b07 100644
--- a/app/vmctl/terminal/unix.go
+++ b/app/vmctl/terminal/unix.go
@@ -6,3 +6,8 @@ package terminal
 import "golang.org/x/sys/unix"
 
 const ioctlReadTermios = unix.TCGETS
+
+func isTerminal(fd int) bool {
+	_, err := unix.IoctlGetTermios(fd, ioctlReadTermios)
+	return err == nil
+}
diff --git a/app/vmctl/terminal/unix_bsd.go b/app/vmctl/terminal/unix_bsd.go
index 7f866d087..6e5ebae0e 100644
--- a/app/vmctl/terminal/unix_bsd.go
+++ b/app/vmctl/terminal/unix_bsd.go
@@ -6,3 +6,8 @@ package terminal
 import "golang.org/x/sys/unix"
 
 const ioctlReadTermios = unix.TIOCGETA
+
+func isTerminal(fd int) bool {
+	_, err := unix.IoctlGetTermios(fd, ioctlReadTermios)
+	return err == nil
+}
diff --git a/app/vmctl/terminal/windows..go b/app/vmctl/terminal/windows..go
new file mode 100644
index 000000000..c3af23e40
--- /dev/null
+++ b/app/vmctl/terminal/windows..go
@@ -0,0 +1,8 @@
+//go:build windows
+// +build windows
+
+package terminal
+
+func isTerminal(fd int) bool {
+	return true
+}
diff --git a/app/vmselect/netstorage/netstorage.go b/app/vmselect/netstorage/netstorage.go
index 85caed314..4fb850379 100644
--- a/app/vmselect/netstorage/netstorage.go
+++ b/app/vmselect/netstorage/netstorage.go
@@ -148,40 +148,28 @@ func timeseriesWorker(qt *querytracer.Tracer, workChs []chan *timeseriesWork, wo
 	// Then help others with the remaining work.
 	rowsProcessed = 0
 	seriesProcessed = 0
-	idx := int(workerID)
-	for {
-		tsw, idxNext := stealTimeseriesWork(workChs, idx)
-		if tsw == nil {
-			// There is no more work
-			break
+	for i := uint(1); i < uint(len(workChs)); i++ {
+		idx := (i + workerID) % uint(len(workChs))
+		ch := workChs[idx]
+		for len(ch) > 0 {
+			// Give a chance other goroutines to perform their work.
+			runtime.Gosched()
+			// It is expected that every channel in the workChs is already closed,
+			// so the next line should return immediately.
+			tsw, ok := <-ch
+			if !ok {
+				break
+			}
+			tsw.err = tsw.do(&tmpResult.rs, workerID)
+			rowsProcessed += tsw.rowsProcessed
+			seriesProcessed++
 		}
-		tsw.err = tsw.do(&tmpResult.rs, workerID)
-		rowsProcessed += tsw.rowsProcessed
-		seriesProcessed++
-		idx = idxNext
 	}
 	qt.Printf("others work processed: series=%d, samples=%d", seriesProcessed, rowsProcessed)
 
 	putTmpResult(tmpResult)
 }
 
-func stealTimeseriesWork(workChs []chan *timeseriesWork, startIdx int) (*timeseriesWork, int) {
-	for i := startIdx; i < startIdx+len(workChs); i++ {
-		// Give a chance other goroutines to perform their work
-		runtime.Gosched()
-
-		idx := i % len(workChs)
-		ch := workChs[idx]
-		// It is expected that every channel in the workChs is already closed,
-		// so the next line should return immediately.
-		tsw, ok := <-ch
-		if ok {
-			return tsw, idx
-		}
-	}
-	return nil, startIdx
-}
-
 func getTmpResult() *result {
 	v := resultPool.Get()
 	if v == nil {
@@ -207,10 +195,17 @@ type result struct {
 
 var resultPool sync.Pool
 
+// MaxWorkers returns the maximum number of workers netstorage can spin when calling RunParallel()
+func MaxWorkers() int {
+	return gomaxprocs
+}
+
+var gomaxprocs = cgroup.AvailableCPUs()
+
 // RunParallel runs f in parallel for all the results from rss.
 //
 // f shouldn't hold references to rs after returning.
-// workerID is the id of the worker goroutine that calls f.
+// workerID is the id of the worker goroutine that calls f. The workerID is in the range [0..MaxWorkers()-1].
 // Data processing is immediately stopped if f returns non-nil error.
 //
 // rss becomes unusable after the call to RunParallel.
@@ -244,7 +239,8 @@ func (rss *Results) runParallel(qt *querytracer.Tracer, f func(rs *Result, worke
 		tsw.f = f
 		tsw.mustStop = &mustStop
 	}
-	if gomaxprocs == 1 || tswsLen == 1 {
+	maxWorkers := MaxWorkers()
+	if maxWorkers == 1 || tswsLen == 1 {
 		// It is faster to process time series in the current goroutine.
 		tsw := getTimeseriesWork()
 		tmpResult := getTmpResult()
@@ -280,8 +276,8 @@ func (rss *Results) runParallel(qt *querytracer.Tracer, f func(rs *Result, worke
 
 	// Prepare worker channels.
 	workers := len(tsws)
-	if workers > gomaxprocs {
-		workers = gomaxprocs
+	if workers > maxWorkers {
+		workers = maxWorkers
 	}
 	itemsPerWorker := (len(tsws) + workers - 1) / workers
 	workChs := make([]chan *timeseriesWork, workers)
@@ -333,8 +329,6 @@ var (
 	seriesReadPerQuery = metrics.NewHistogram(`vm_series_read_per_query`)
 )
 
-var gomaxprocs = cgroup.AvailableCPUs()
-
 type packedTimeseries struct {
 	metricName string
 	brs        []blockRef
@@ -391,37 +385,25 @@ func unpackWorker(workChs []chan *unpackWork, workerID uint) {
 	}
 
 	// Then help others with their work.
-	idx := int(workerID)
-	for {
-		upw, idxNext := stealUnpackWork(workChs, idx)
-		if upw == nil {
-			// There is no more work
-			break
+	for i := uint(1); i < uint(len(workChs)); i++ {
+		idx := (i + workerID) % uint(len(workChs))
+		ch := workChs[idx]
+		for len(ch) > 0 {
+			// Give a chance other goroutines to perform their work
+			runtime.Gosched()
+			// It is expected that every channel in the workChs is already closed,
+			// so the next line should return immediately.
+			upw, ok := <-ch
+			if !ok {
+				break
+			}
+			upw.unpack(tmpBlock)
 		}
-		upw.unpack(tmpBlock)
-		idx = idxNext
 	}
 
 	putTmpStorageBlock(tmpBlock)
 }
 
-func stealUnpackWork(workChs []chan *unpackWork, startIdx int) (*unpackWork, int) {
-	for i := startIdx; i < startIdx+len(workChs); i++ {
-		// Give a chance other goroutines to perform their work
-		runtime.Gosched()
-
-		idx := i % len(workChs)
-		ch := workChs[idx]
-		// It is expected that every channel in the workChs is already closed,
-		// so the next line should return immediately.
-		upw, ok := <-ch
-		if ok {
-			return upw, idx
-		}
-	}
-	return nil, startIdx
-}
-
 func getTmpStorageBlock() *storage.Block {
 	v := tmpStorageBlockPool.Get()
 	if v == nil {
@@ -1017,7 +999,6 @@ func ExportBlocks(qt *querytracer.Tracer, sq *storage.SearchQuery, deadline sear
 	indexSearchDuration.UpdateDuration(startTime)
 
 	// Start workers that call f in parallel on available CPU cores.
-	gomaxprocs := cgroup.AvailableCPUs()
 	workCh := make(chan *exportWork, gomaxprocs*8)
 	var (
 		errGlobal     error
diff --git a/app/vmselect/promql/aggr_incremental.go b/app/vmselect/promql/aggr_incremental.go
index 26f91ed0e..f3d5bd416 100644
--- a/app/vmselect/promql/aggr_incremental.go
+++ b/app/vmselect/promql/aggr_incremental.go
@@ -3,8 +3,9 @@ package promql
 import (
 	"math"
 	"strings"
-	"sync"
+	"unsafe"
 
+	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
 	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
 	"github.com/VictoriaMetrics/metricsql"
 )
@@ -63,31 +64,36 @@ var incrementalAggrFuncCallbacksMap = map[string]*incrementalAggrFuncCallbacks{
 	},
 }
 
+type incrementalAggrContextMap struct {
+	m map[string]*incrementalAggrContext
+
+	// The padding prevents false sharing on widespread platforms with
+	// 128 mod (cache line size) = 0 .
+	_ [128 - unsafe.Sizeof(map[string]*incrementalAggrContext{})%128]byte
+}
+
 type incrementalAggrFuncContext struct {
 	ae *metricsql.AggrFuncExpr
 
-	m sync.Map
+	byWorkerID []incrementalAggrContextMap
 
 	callbacks *incrementalAggrFuncCallbacks
 }
 
 func newIncrementalAggrFuncContext(ae *metricsql.AggrFuncExpr, callbacks *incrementalAggrFuncCallbacks) *incrementalAggrFuncContext {
 	return &incrementalAggrFuncContext{
-		ae:        ae,
-		callbacks: callbacks,
+		ae:         ae,
+		byWorkerID: make([]incrementalAggrContextMap, netstorage.MaxWorkers()),
+		callbacks:  callbacks,
 	}
 }
 
 func (iafc *incrementalAggrFuncContext) updateTimeseries(tsOrig *timeseries, workerID uint) {
-	v, ok := iafc.m.Load(workerID)
-	if !ok {
-		// It is safe creating and storing m in iafc.m without locking,
-		// since it is guaranteed that only a single goroutine can execute
-		// code for the given workerID at a time.
-		v = make(map[string]*incrementalAggrContext, 1)
-		iafc.m.Store(workerID, v)
+	v := &iafc.byWorkerID[workerID]
+	if v.m == nil {
+		v.m = make(map[string]*incrementalAggrContext, 1)
 	}
-	m := v.(map[string]*incrementalAggrContext)
+	m := v.m
 
 	ts := tsOrig
 	keepOriginal := iafc.callbacks.keepOriginal
@@ -128,9 +134,9 @@ func (iafc *incrementalAggrFuncContext) updateTimeseries(tsOrig *timeseries, wor
 func (iafc *incrementalAggrFuncContext) finalizeTimeseries() []*timeseries {
 	mGlobal := make(map[string]*incrementalAggrContext)
 	mergeAggrFunc := iafc.callbacks.mergeAggrFunc
-	iafc.m.Range(func(k, v interface{}) bool {
-		m := v.(map[string]*incrementalAggrContext)
-		for k, iac := range m {
+	byWorkerID := iafc.byWorkerID
+	for i := range byWorkerID {
+		for k, iac := range byWorkerID[i].m {
 			iacGlobal := mGlobal[k]
 			if iacGlobal == nil {
 				if iafc.ae.Limit > 0 && len(mGlobal) >= iafc.ae.Limit {
@@ -142,8 +148,7 @@ func (iafc *incrementalAggrFuncContext) finalizeTimeseries() []*timeseries {
 			}
 			mergeAggrFunc(iacGlobal, iac)
 		}
-		return true
-	})
+	}
 	tss := make([]*timeseries, 0, len(mGlobal))
 	finalizeAggrFunc := iafc.callbacks.finalizeAggrFunc
 	for _, iac := range mGlobal {
diff --git a/app/vmselect/promql/eval.go b/app/vmselect/promql/eval.go
index ded6e9577..c4c32081e 100644
--- a/app/vmselect/promql/eval.go
+++ b/app/vmselect/promql/eval.go
@@ -908,7 +908,7 @@ func evalRollupFuncWithSubquery(qt *querytracer.Tracer, ec *EvalConfig, funcName
 	var tssLock sync.Mutex
 	var samplesScannedTotal uint64
 	keepMetricNames := getKeepMetricNames(expr)
-	doParallel(tssSQ, func(tsSQ *timeseries, values []float64, timestamps []int64) ([]float64, []int64) {
+	doParallel(tssSQ, func(tsSQ *timeseries, values []float64, timestamps []int64, workerID uint) ([]float64, []int64) {
 		values, timestamps = removeNanValues(values[:0], timestamps[:0], tsSQ.Values, tsSQ.Timestamps)
 		preFunc(values, timestamps)
 		for _, rc := range rcs {
@@ -952,28 +952,36 @@ func getKeepMetricNames(expr metricsql.Expr) bool {
 	return false
 }
 
-func doParallel(tss []*timeseries, f func(ts *timeseries, values []float64, timestamps []int64) ([]float64, []int64)) {
-	concurrency := cgroup.AvailableCPUs()
-	if concurrency > len(tss) {
-		concurrency = len(tss)
+func doParallel(tss []*timeseries, f func(ts *timeseries, values []float64, timestamps []int64, workerID uint) ([]float64, []int64)) {
+	workers := netstorage.MaxWorkers()
+	if workers > len(tss) {
+		workers = len(tss)
 	}
-	workCh := make(chan *timeseries, concurrency)
+	seriesPerWorker := (len(tss) + workers - 1) / workers
+	workChs := make([]chan *timeseries, workers)
+	for i := range workChs {
+		workChs[i] = make(chan *timeseries, seriesPerWorker)
+	}
+	for i, ts := range tss {
+		idx := i % len(workChs)
+		workChs[idx] <- ts
+	}
+	for _, workCh := range workChs {
+		close(workCh)
+	}
+
 	var wg sync.WaitGroup
-	wg.Add(concurrency)
-	for i := 0; i < concurrency; i++ {
-		go func() {
+	wg.Add(workers)
+	for i := 0; i < workers; i++ {
+		go func(workerID uint) {
 			defer wg.Done()
 			var tmpValues []float64
 			var tmpTimestamps []int64
-			for ts := range workCh {
-				tmpValues, tmpTimestamps = f(ts, tmpValues, tmpTimestamps)
+			for ts := range workChs[workerID] {
+				tmpValues, tmpTimestamps = f(ts, tmpValues, tmpTimestamps, workerID)
 			}
-		}()
-	}
-	for _, ts := range tss {
-		workCh <- ts
+		}(uint(i))
 	}
-	close(workCh)
 	wg.Wait()
 }
 
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 4d8d9f8fa..f262ef197 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -21,6 +21,7 @@ created by v1.90.0 or newer versions. The solution is to upgrade to v1.90.0 or n
 
 * FEATURE: publish VictoriaMetrics binaries for Windows. See [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3236), [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3821) and [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70) issues.
 * FEATURE: log metrics with truncated labels if the length of label value in the ingested metric exceeds `-maxLabelValueLen`. This should simplify debugging for this case.
+* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): show target URL when debugging [target relabeling](https://docs.victoriametrics.com/vmagent.html#relabel-debug). This should simplify target relabel debugging a bit. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3882).
 * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add support for [VictoriaMetrics remote write protocol](https://docs.victoriametrics.com/vmagent.html#victoriametrics-remote-write-protocol) when [sending / receiving data to / from Kafka](https://docs.victoriametrics.com/vmagent.html#kafka-integration). This protocol allows saving egress network bandwidth costs when sending data from `vmagent` to `Kafka` located in another datacenter or availability zone. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225).
 * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `--kafka.consumer.topic.concurrency` command-line flag. It controls the number of Kafka consumer workers to use by `vmagent`. It should eliminate the need to start multiple `vmagent` instances to improve data transfer rate. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1957).
 * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add support for [Kafka producer and consumer](https://docs.victoriametrics.com/vmagent.html#kafka-integration) on `arm64` machines. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2271).
diff --git a/docs/vmagent.md b/docs/vmagent.md
index 428d88147..92f68de85 100644
--- a/docs/vmagent.md
+++ b/docs/vmagent.md
@@ -641,18 +641,19 @@ provide the following tools for debugging target-level and metric-level relabeli
 - Target-level debugging (e.g. `relabel_configs` section at [scrape_configs](https://docs.victoriametrics.com/sd_configs.html#scrape_configs))
   can be performed by navigating to `http://vmagent:8429/targets` page (`http://victoriametrics:8428/targets` page for single-node VictoriaMetrics)
   and clicking the `debug target relabeling` link at the target, which must be debugged.
-  The opened page will show step-by-step results for the actual target relabeling rules applied to the discovered target labels.
+  The opened page shows step-by-step results for the actual target relabeling rules applied to the discovered target labels.
+  The page shows also the target URL generated after applying all the relabeling rules.
 
   The `http://vmagent:8429/targets` page shows only active targets. If you need to understand why some target
   is dropped during the relabeling, then navigate to `http://vmagent:8428/service-discovery` page
   (`http://victoriametrics:8428/service-discovery` for single-node VictoriaMetrics), find the dropped target
-  and click the `debug` link there. The opened page will show step-by-step results for the actual relabeling rules,
+  and click the `debug` link there. The opened page shows step-by-step results for the actual relabeling rules,
   which result to target drop.
 
 - Metric-level debugging (e.g. `metric_relabel_configs` section at [scrape_configs](https://docs.victoriametrics.com/sd_configs.html#scrape_configs)
   can be performed by navigating to `http://vmagent:8429/targets` page (`http://victoriametrics:8428/targets` page for single-node VictoriaMetrics)
   and clicking the `debug metrics relabeling` link at the target, which must be debugged.
-  The opened page will show step-by-step results for the actual metric relabeling rules applied to the given target labels.
+  The opened page shows step-by-step results for the actual metric relabeling rules applied to the given target labels.
 
 ## Prometheus staleness markers
 
diff --git a/lib/promrelabel/debug.go b/lib/promrelabel/debug.go
index 360792d13..d4de9e3e5 100644
--- a/lib/promrelabel/debug.go
+++ b/lib/promrelabel/debug.go
@@ -21,30 +21,32 @@ func writeRelabelDebug(w io.Writer, isTargetRelabel bool, targetID, metric, rela
 	if metric == "" {
 		metric = "{}"
 	}
+	targetURL := ""
 	if err != nil {
-		WriteRelabelDebugSteps(w, isTargetRelabel, targetID, nil, metric, relabelConfigs, err)
+		WriteRelabelDebugSteps(w, targetURL, targetID, nil, metric, relabelConfigs, err)
 		return
 	}
 	labels, err := promutils.NewLabelsFromString(metric)
 	if err != nil {
 		err = fmt.Errorf("cannot parse metric: %s", err)
-		WriteRelabelDebugSteps(w, isTargetRelabel, targetID, nil, metric, relabelConfigs, err)
+		WriteRelabelDebugSteps(w, targetURL, targetID, nil, metric, relabelConfigs, err)
 		return
 	}
 	pcs, err := ParseRelabelConfigsData([]byte(relabelConfigs))
 	if err != nil {
 		err = fmt.Errorf("cannot parse relabel configs: %s", err)
-		WriteRelabelDebugSteps(w, isTargetRelabel, targetID, nil, metric, relabelConfigs, err)
+		WriteRelabelDebugSteps(w, targetURL, targetID, nil, metric, relabelConfigs, err)
 		return
 	}
 
-	dss := newDebugRelabelSteps(pcs, labels, isTargetRelabel)
-	WriteRelabelDebugSteps(w, isTargetRelabel, targetID, dss, metric, relabelConfigs, nil)
+	dss, targetURL := newDebugRelabelSteps(pcs, labels, isTargetRelabel)
+	WriteRelabelDebugSteps(w, targetURL, targetID, dss, metric, relabelConfigs, nil)
 }
 
-func newDebugRelabelSteps(pcs *ParsedConfigs, labels *promutils.Labels, isTargetRelabel bool) []DebugStep {
-	// The target relabeling below must be in sync with the code at scrapeWorkConfig.getScrapeWork if isTragetRelabeling=true
+func newDebugRelabelSteps(pcs *ParsedConfigs, labels *promutils.Labels, isTargetRelabel bool) ([]DebugStep, string) {
+	// The target relabeling below must be in sync with the code at scrapeWorkConfig.getScrapeWork if isTargetRelabel=true
 	// and with the code at scrapeWork.addRowToTimeseries when isTargetRelabeling=false
+	targetURL := ""
 
 	// Prevent from modifying the original labels
 	labels = labels.Clone()
@@ -70,6 +72,9 @@ func newDebugRelabelSteps(pcs *ParsedConfigs, labels *promutils.Labels, isTarget
 			}
 		}
 
+		// Generate targetURL
+		targetURL, _ = GetScrapeURL(labels, nil)
+
 		// Remove labels with __ prefix
 		inStr := outStr
 		labels.RemoveLabelsWithDoubleUnderscorePrefix()
@@ -96,7 +101,7 @@ func newDebugRelabelSteps(pcs *ParsedConfigs, labels *promutils.Labels, isTarget
 	}
 
 	// There is no need in labels' sorting, since LabelsToString() automatically sorts labels.
-	return dss
+	return dss, targetURL
 }
 
 func getChangedLabelNames(in, out *promutils.Labels) map[string]struct{} {
diff --git a/lib/promrelabel/debug.qtpl b/lib/promrelabel/debug.qtpl
index 2f8244d25..1d894c0ae 100644
--- a/lib/promrelabel/debug.qtpl
+++ b/lib/promrelabel/debug.qtpl
@@ -5,7 +5,7 @@
 
 {% stripspace %}
 
-{% func RelabelDebugSteps(isTargetRelabel bool, targetID string, dss []DebugStep, metric, relabelConfigs string, err error) %}
+{% func RelabelDebugSteps(targetURL string, targetID string, dss []DebugStep, metric, relabelConfigs string, err error) %}
 <!DOCTYPE html>
 <html lang="en">
 <head>
@@ -27,7 +27,7 @@ function submitRelabelDebugForm(e) {
     <div class="container-fluid">
         <a href="https://docs.victoriametrics.com/relabeling.html" target="_blank">Relabeling docs</a>{% space %}
 
-        {% if isTargetRelabel %}
+        {% if targetURL != "" %}
           <a href="metric-relabel-debug{% if targetID != "" %}?id={%s targetID %}{% endif %}">Metric relabel debug</a>
         {% else %}
           <a href="target-relabel-debug{% if targetID != "" %}?id={%s targetID %}{% endif %}">Target relabel debug</a>
@@ -53,7 +53,7 @@ function submitRelabelDebugForm(e) {
 
         <div class="row">
             <main class="col-12">
-              {%= relabelDebugSteps(dss) %}
+              {%= relabelDebugSteps(dss, targetURL, targetID) %}
             </main>
         </div>
     </div>
@@ -73,10 +73,10 @@ function submitRelabelDebugForm(e) {
     </div>
 {% endfunc %}
 
-{% func relabelDebugSteps(dss []DebugStep) %}
+{% func relabelDebugSteps(dss []DebugStep, targetURL, targetID string) %}
 {% if len(dss) > 0 %}
 <div class="m-3">
-        <b>Original labels:</b> <samp>{%= mustFormatLabels(dss[0].In) %}</samp>
+    <b>Original labels:</b> <samp>{%= mustFormatLabels(dss[0].In) %}</samp>
 </div>
 {% endif %}
 <table class="table table-striped table-hover table-bordered table-sm">
@@ -114,7 +114,16 @@ function submitRelabelDebugForm(e) {
 </table>
 {% if len(dss) > 0 %}
 <div class="m-3">
-        <b>Resulting labels:</b> <samp>{%= mustFormatLabels(dss[len(dss)-1].Out) %}</samp>
+    <b>Resulting labels:</b> <samp>{%= mustFormatLabels(dss[len(dss)-1].Out) %}</samp>
+    {% if targetURL != "" %}
+        <div>
+            <b>Target URL:</b>{% space %}<a href="{%s targetURL %}" target="_blank">{%s targetURL %}</a>
+            {% if targetID != "" %}
+                {% space %}
+                (<a href="target_response?id={%s targetID %}" target="_blank" title="click to fetch target response on behalf of the scraper">response</a>)
+            {% endif %}
+        </div>
+    {% endif %}
 </div>
 {% endif %}
 {% endfunc %}
diff --git a/lib/promrelabel/debug.qtpl.go b/lib/promrelabel/debug.qtpl.go
index 4e78569b9..e2d680eff 100644
--- a/lib/promrelabel/debug.qtpl.go
+++ b/lib/promrelabel/debug.qtpl.go
@@ -24,7 +24,7 @@ var (
 )
 
 //line lib/promrelabel/debug.qtpl:8
-func StreamRelabelDebugSteps(qw422016 *qt422016.Writer, isTargetRelabel bool, targetID string, dss []DebugStep, metric, relabelConfigs string, err error) {
+func StreamRelabelDebugSteps(qw422016 *qt422016.Writer, targetURL string, targetID string, dss []DebugStep, metric, relabelConfigs string, err error) {
 //line lib/promrelabel/debug.qtpl:8
 	qw422016.N().S(`<!DOCTYPE html><html lang="en"><head>`)
 //line lib/promrelabel/debug.qtpl:12
@@ -38,7 +38,7 @@ func StreamRelabelDebugSteps(qw422016 *qt422016.Writer, isTargetRelabel bool, ta
 //line lib/promrelabel/debug.qtpl:28
 	qw422016.N().S(` `)
 //line lib/promrelabel/debug.qtpl:30
-	if isTargetRelabel {
+	if targetURL != "" {
 //line lib/promrelabel/debug.qtpl:30
 		qw422016.N().S(`<a href="metric-relabel-debug`)
 //line lib/promrelabel/debug.qtpl:31
@@ -104,29 +104,29 @@ func StreamRelabelDebugSteps(qw422016 *qt422016.Writer, isTargetRelabel bool, ta
 //line lib/promrelabel/debug.qtpl:50
 	qw422016.N().S(`</form></div><div class="row"><main class="col-12">`)
 //line lib/promrelabel/debug.qtpl:56
-	streamrelabelDebugSteps(qw422016, dss)
+	streamrelabelDebugSteps(qw422016, dss, targetURL, targetID)
 //line lib/promrelabel/debug.qtpl:56
 	qw422016.N().S(`</main></div></div></body></html>`)
 //line lib/promrelabel/debug.qtpl:62
 }
 
 //line lib/promrelabel/debug.qtpl:62
-func WriteRelabelDebugSteps(qq422016 qtio422016.Writer, isTargetRelabel bool, targetID string, dss []DebugStep, metric, relabelConfigs string, err error) {
+func WriteRelabelDebugSteps(qq422016 qtio422016.Writer, targetURL string, targetID string, dss []DebugStep, metric, relabelConfigs string, err error) {
 //line lib/promrelabel/debug.qtpl:62
 	qw422016 := qt422016.AcquireWriter(qq422016)
 //line lib/promrelabel/debug.qtpl:62
-	StreamRelabelDebugSteps(qw422016, isTargetRelabel, targetID, dss, metric, relabelConfigs, err)
+	StreamRelabelDebugSteps(qw422016, targetURL, targetID, dss, metric, relabelConfigs, err)
 //line lib/promrelabel/debug.qtpl:62
 	qt422016.ReleaseWriter(qw422016)
 //line lib/promrelabel/debug.qtpl:62
 }
 
 //line lib/promrelabel/debug.qtpl:62
-func RelabelDebugSteps(isTargetRelabel bool, targetID string, dss []DebugStep, metric, relabelConfigs string, err error) string {
+func RelabelDebugSteps(targetURL string, targetID string, dss []DebugStep, metric, relabelConfigs string, err error) string {
 //line lib/promrelabel/debug.qtpl:62
 	qb422016 := qt422016.AcquireByteBuffer()
 //line lib/promrelabel/debug.qtpl:62
-	WriteRelabelDebugSteps(qb422016, isTargetRelabel, targetID, dss, metric, relabelConfigs, err)
+	WriteRelabelDebugSteps(qb422016, targetURL, targetID, dss, metric, relabelConfigs, err)
 //line lib/promrelabel/debug.qtpl:62
 	qs422016 := string(qb422016.B)
 //line lib/promrelabel/debug.qtpl:62
@@ -178,7 +178,7 @@ func relabelDebugFormInputs(metric, relabelConfigs string) string {
 }
 
 //line lib/promrelabel/debug.qtpl:76
-func streamrelabelDebugSteps(qw422016 *qt422016.Writer, dss []DebugStep) {
+func streamrelabelDebugSteps(qw422016 *qt422016.Writer, dss []DebugStep, targetURL, targetID string) {
 //line lib/promrelabel/debug.qtpl:77
 	if len(dss) > 0 {
 //line lib/promrelabel/debug.qtpl:77
@@ -227,41 +227,75 @@ func streamrelabelDebugSteps(qw422016 *qt422016.Writer, dss []DebugStep) {
 //line lib/promrelabel/debug.qtpl:117
 		streammustFormatLabels(qw422016, dss[len(dss)-1].Out)
 //line lib/promrelabel/debug.qtpl:117
-		qw422016.N().S(`</samp></div>`)
-//line lib/promrelabel/debug.qtpl:119
-	}
+		qw422016.N().S(`</samp>`)
+//line lib/promrelabel/debug.qtpl:118
+		if targetURL != "" {
+//line lib/promrelabel/debug.qtpl:118
+			qw422016.N().S(`<div><b>Target URL:</b>`)
 //line lib/promrelabel/debug.qtpl:120
-}
-
+			qw422016.N().S(` `)
 //line lib/promrelabel/debug.qtpl:120
-func writerelabelDebugSteps(qq422016 qtio422016.Writer, dss []DebugStep) {
+			qw422016.N().S(`<a href="`)
 //line lib/promrelabel/debug.qtpl:120
-	qw422016 := qt422016.AcquireWriter(qq422016)
+			qw422016.E().S(targetURL)
 //line lib/promrelabel/debug.qtpl:120
-	streamrelabelDebugSteps(qw422016, dss)
+			qw422016.N().S(`" target="_blank">`)
 //line lib/promrelabel/debug.qtpl:120
-	qt422016.ReleaseWriter(qw422016)
+			qw422016.E().S(targetURL)
 //line lib/promrelabel/debug.qtpl:120
+			qw422016.N().S(`</a>`)
+//line lib/promrelabel/debug.qtpl:121
+			if targetID != "" {
+//line lib/promrelabel/debug.qtpl:122
+				qw422016.N().S(` `)
+//line lib/promrelabel/debug.qtpl:122
+				qw422016.N().S(`(<a href="target_response?id=`)
+//line lib/promrelabel/debug.qtpl:123
+				qw422016.E().S(targetID)
+//line lib/promrelabel/debug.qtpl:123
+				qw422016.N().S(`" target="_blank" title="click to fetch target response on behalf of the scraper">response</a>)`)
+//line lib/promrelabel/debug.qtpl:124
+			}
+//line lib/promrelabel/debug.qtpl:124
+			qw422016.N().S(`</div>`)
+//line lib/promrelabel/debug.qtpl:126
+		}
+//line lib/promrelabel/debug.qtpl:126
+		qw422016.N().S(`</div>`)
+//line lib/promrelabel/debug.qtpl:128
+	}
+//line lib/promrelabel/debug.qtpl:129
 }
 
-//line lib/promrelabel/debug.qtpl:120
-func relabelDebugSteps(dss []DebugStep) string {
-//line lib/promrelabel/debug.qtpl:120
+//line lib/promrelabel/debug.qtpl:129
+func writerelabelDebugSteps(qq422016 qtio422016.Writer, dss []DebugStep, targetURL, targetID string) {
+//line lib/promrelabel/debug.qtpl:129
+	qw422016 := qt422016.AcquireWriter(qq422016)
+//line lib/promrelabel/debug.qtpl:129
+	streamrelabelDebugSteps(qw422016, dss, targetURL, targetID)
+//line lib/promrelabel/debug.qtpl:129
+	qt422016.ReleaseWriter(qw422016)
+//line lib/promrelabel/debug.qtpl:129
+}
+
+//line lib/promrelabel/debug.qtpl:129
+func relabelDebugSteps(dss []DebugStep, targetURL, targetID string) string {
+//line lib/promrelabel/debug.qtpl:129
 	qb422016 := qt422016.AcquireByteBuffer()
-//line lib/promrelabel/debug.qtpl:120
-	writerelabelDebugSteps(qb422016, dss)
-//line lib/promrelabel/debug.qtpl:120
+//line lib/promrelabel/debug.qtpl:129
+	writerelabelDebugSteps(qb422016, dss, targetURL, targetID)
+//line lib/promrelabel/debug.qtpl:129
 	qs422016 := string(qb422016.B)
-//line lib/promrelabel/debug.qtpl:120
+//line lib/promrelabel/debug.qtpl:129
 	qt422016.ReleaseByteBuffer(qb422016)
-//line lib/promrelabel/debug.qtpl:120
+//line lib/promrelabel/debug.qtpl:129
 	return qs422016
-//line lib/promrelabel/debug.qtpl:120
+//line lib/promrelabel/debug.qtpl:129
 }
 
-//line lib/promrelabel/debug.qtpl:122
+//line lib/promrelabel/debug.qtpl:131
 func streamlabelsWithHighlight(qw422016 *qt422016.Writer, labels *promutils.Labels, highlight map[string]struct{}, color string) {
-//line lib/promrelabel/debug.qtpl:124
+//line lib/promrelabel/debug.qtpl:133
 	labelsList := labels.GetLabels()
 	metricName := ""
 	for i, label := range labelsList {
@@ -272,137 +306,137 @@ func streamlabelsWithHighlight(qw422016 *qt422016.Writer, labels *promutils.Labe
 		}
 	}
 
-//line lib/promrelabel/debug.qtpl:134
+//line lib/promrelabel/debug.qtpl:143
 	if metricName != "" {
-//line lib/promrelabel/debug.qtpl:135
+//line lib/promrelabel/debug.qtpl:144
 		if _, ok := highlight["__name__"]; ok {
-//line lib/promrelabel/debug.qtpl:135
+//line lib/promrelabel/debug.qtpl:144
 			qw422016.N().S(`<span style="font-weight:bold;color:`)
-//line lib/promrelabel/debug.qtpl:136
+//line lib/promrelabel/debug.qtpl:145
 			qw422016.E().S(color)
-//line lib/promrelabel/debug.qtpl:136
+//line lib/promrelabel/debug.qtpl:145
 			qw422016.N().S(`">`)
-//line lib/promrelabel/debug.qtpl:136
+//line lib/promrelabel/debug.qtpl:145
 			qw422016.E().S(metricName)
-//line lib/promrelabel/debug.qtpl:136
+//line lib/promrelabel/debug.qtpl:145
 			qw422016.N().S(`</span>`)
-//line lib/promrelabel/debug.qtpl:137
+//line lib/promrelabel/debug.qtpl:146
 		} else {
-//line lib/promrelabel/debug.qtpl:138
+//line lib/promrelabel/debug.qtpl:147
 			qw422016.E().S(metricName)
-//line lib/promrelabel/debug.qtpl:139
+//line lib/promrelabel/debug.qtpl:148
 		}
-//line lib/promrelabel/debug.qtpl:140
+//line lib/promrelabel/debug.qtpl:149
 		if len(labelsList) == 0 {
-//line lib/promrelabel/debug.qtpl:140
+//line lib/promrelabel/debug.qtpl:149
 			return
-//line lib/promrelabel/debug.qtpl:140
+//line lib/promrelabel/debug.qtpl:149
 		}
-//line lib/promrelabel/debug.qtpl:141
+//line lib/promrelabel/debug.qtpl:150
 	}
-//line lib/promrelabel/debug.qtpl:141
+//line lib/promrelabel/debug.qtpl:150
 	qw422016.N().S(`{`)
-//line lib/promrelabel/debug.qtpl:143
+//line lib/promrelabel/debug.qtpl:152
 	for i, label := range labelsList {
-//line lib/promrelabel/debug.qtpl:144
+//line lib/promrelabel/debug.qtpl:153
 		if _, ok := highlight[label.Name]; ok {
-//line lib/promrelabel/debug.qtpl:144
+//line lib/promrelabel/debug.qtpl:153
 			qw422016.N().S(`<span style="font-weight:bold;color:`)
-//line lib/promrelabel/debug.qtpl:145
+//line lib/promrelabel/debug.qtpl:154
 			qw422016.E().S(color)
-//line lib/promrelabel/debug.qtpl:145
+//line lib/promrelabel/debug.qtpl:154
 			qw422016.N().S(`">`)
-//line lib/promrelabel/debug.qtpl:145
+//line lib/promrelabel/debug.qtpl:154
 			qw422016.E().S(label.Name)
-//line lib/promrelabel/debug.qtpl:145
+//line lib/promrelabel/debug.qtpl:154
 			qw422016.N().S(`=`)
-//line lib/promrelabel/debug.qtpl:145
+//line lib/promrelabel/debug.qtpl:154
 			qw422016.E().Q(label.Value)
-//line lib/promrelabel/debug.qtpl:145
+//line lib/promrelabel/debug.qtpl:154
 			qw422016.N().S(`</span>`)
-//line lib/promrelabel/debug.qtpl:146
+//line lib/promrelabel/debug.qtpl:155
 		} else {
-//line lib/promrelabel/debug.qtpl:147
+//line lib/promrelabel/debug.qtpl:156
 			qw422016.E().S(label.Name)
-//line lib/promrelabel/debug.qtpl:147
+//line lib/promrelabel/debug.qtpl:156
 			qw422016.N().S(`=`)
-//line lib/promrelabel/debug.qtpl:147
+//line lib/promrelabel/debug.qtpl:156
 			qw422016.E().Q(label.Value)
-//line lib/promrelabel/debug.qtpl:148
+//line lib/promrelabel/debug.qtpl:157
 		}
-//line lib/promrelabel/debug.qtpl:149
+//line lib/promrelabel/debug.qtpl:158
 		if i < len(labelsList)-1 {
-//line lib/promrelabel/debug.qtpl:149
+//line lib/promrelabel/debug.qtpl:158
 			qw422016.N().S(`,`)
-//line lib/promrelabel/debug.qtpl:149
+//line lib/promrelabel/debug.qtpl:158
 			qw422016.N().S(` `)
-//line lib/promrelabel/debug.qtpl:149
+//line lib/promrelabel/debug.qtpl:158
 		}
-//line lib/promrelabel/debug.qtpl:150
+//line lib/promrelabel/debug.qtpl:159
 	}
-//line lib/promrelabel/debug.qtpl:150
+//line lib/promrelabel/debug.qtpl:159
 	qw422016.N().S(`}`)
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 }
 
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 func writelabelsWithHighlight(qq422016 qtio422016.Writer, labels *promutils.Labels, highlight map[string]struct{}, color string) {
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 	qw422016 := qt422016.AcquireWriter(qq422016)
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 	streamlabelsWithHighlight(qw422016, labels, highlight, color)
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 	qt422016.ReleaseWriter(qw422016)
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 }
 
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 func labelsWithHighlight(labels *promutils.Labels, highlight map[string]struct{}, color string) string {
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 	qb422016 := qt422016.AcquireByteBuffer()
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 	writelabelsWithHighlight(qb422016, labels, highlight, color)
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 	qs422016 := string(qb422016.B)
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 	qt422016.ReleaseByteBuffer(qb422016)
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 	return qs422016
-//line lib/promrelabel/debug.qtpl:152
+//line lib/promrelabel/debug.qtpl:161
 }
 
-//line lib/promrelabel/debug.qtpl:154
+//line lib/promrelabel/debug.qtpl:163
 func streammustFormatLabels(qw422016 *qt422016.Writer, s string) {
-//line lib/promrelabel/debug.qtpl:155
+//line lib/promrelabel/debug.qtpl:164
 	labels := promutils.MustNewLabelsFromString(s)
 
-//line lib/promrelabel/debug.qtpl:156
+//line lib/promrelabel/debug.qtpl:165
 	streamlabelsWithHighlight(qw422016, labels, nil, "")
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 }
 
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 func writemustFormatLabels(qq422016 qtio422016.Writer, s string) {
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 	qw422016 := qt422016.AcquireWriter(qq422016)
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 	streammustFormatLabels(qw422016, s)
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 	qt422016.ReleaseWriter(qw422016)
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 }
 
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 func mustFormatLabels(s string) string {
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 	qb422016 := qt422016.AcquireByteBuffer()
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 	writemustFormatLabels(qb422016, s)
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 	qs422016 := string(qb422016.B)
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 	qt422016.ReleaseByteBuffer(qb422016)
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 	return qs422016
-//line lib/promrelabel/debug.qtpl:157
+//line lib/promrelabel/debug.qtpl:166
 }
diff --git a/lib/promrelabel/relabel.go b/lib/promrelabel/relabel.go
index 9c55d76a3..35a47e74d 100644
--- a/lib/promrelabel/relabel.go
+++ b/lib/promrelabel/relabel.go
@@ -49,7 +49,7 @@ type DebugStep struct {
 	// Rule contains string representation of the rule step
 	Rule string
 
-	// In contains the input labels before the exeuction of the rule step
+	// In contains the input labels before the execution of the rule step
 	In string
 
 	// Out contains the output labels after the execution of the rule step
diff --git a/lib/promrelabel/scrape_url.go b/lib/promrelabel/scrape_url.go
new file mode 100644
index 000000000..55d7d5d5d
--- /dev/null
+++ b/lib/promrelabel/scrape_url.go
@@ -0,0 +1,114 @@
+package promrelabel
+
+import (
+	"net/url"
+	"strings"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutils"
+)
+
+// GetScrapeURL makes scrape url and __address_ labels for the given labels and extraParams.
+func GetScrapeURL(labels *promutils.Labels, extraParams map[string][]string) (string, string) {
+	// See https://www.robustperception.io/life-of-a-label
+	scheme := labels.Get("__scheme__")
+	if len(scheme) == 0 {
+		scheme = "http"
+	}
+	metricsPath := labels.Get("__metrics_path__")
+	if len(metricsPath) == 0 {
+		metricsPath = "/metrics"
+	}
+	address := labels.Get("__address__")
+	if len(address) == 0 {
+		return "", ""
+	}
+	// Usability extension to Prometheus behavior: extract optional scheme and metricsPath from __address__.
+	// Prometheus silently drops targets with __address__ containing scheme or metricsPath
+	// according to https://www.robustperception.io/life-of-a-label/ .
+	if strings.HasPrefix(address, "http://") {
+		scheme = "http"
+		address = address[len("http://"):]
+	} else if strings.HasPrefix(address, "https://") {
+		scheme = "https"
+		address = address[len("https://"):]
+	}
+	if n := strings.IndexByte(address, '/'); n >= 0 {
+		metricsPath = address[n:]
+		address = address[:n]
+	}
+	address = addMissingPort(address, scheme == "https")
+
+	if !strings.HasPrefix(metricsPath, "/") {
+		metricsPath = "/" + metricsPath
+	}
+	params := getParamsFromLabels(labels, extraParams)
+	optionalQuestion := ""
+	if len(params) > 0 {
+		optionalQuestion = "?"
+		if strings.Contains(metricsPath, "?") {
+			optionalQuestion = "&"
+		}
+	}
+	paramsStr := url.Values(params).Encode()
+	scrapeURL := buildScrapeURL(scheme, address, metricsPath, optionalQuestion, paramsStr)
+	return scrapeURL, address
+}
+
+func getParamsFromLabels(labels *promutils.Labels, extraParams map[string][]string) map[string][]string {
+	// See https://www.robustperception.io/life-of-a-label
+	var m map[string][]string
+	for _, label := range labels.GetLabels() {
+		if !strings.HasPrefix(label.Name, "__param_") {
+			continue
+		}
+		name := label.Name[len("__param_"):]
+		values := []string{label.Value}
+		if p := extraParams[name]; len(p) > 1 {
+			values = append(values, p[1:]...)
+		}
+		if m == nil {
+			m = make(map[string][]string)
+		}
+		m[name] = values
+	}
+	return m
+}
+
+func buildScrapeURL(scheme, address, metricsPath, optionalQuestion, paramsStr string) string {
+	bb := bbPool.Get()
+	b := bb.B[:0]
+	b = append(b, scheme...)
+	b = append(b, "://"...)
+	b = append(b, address...)
+	b = append(b, metricsPath...)
+	b = append(b, optionalQuestion...)
+	b = append(b, paramsStr...)
+	s := bytesutil.InternBytes(b)
+	bb.B = b
+	bbPool.Put(bb)
+	return s
+}
+
+func addMissingPort(addr string, isTLS bool) string {
+	if strings.Contains(addr, ":") {
+		return addr
+	}
+	if isTLS {
+		return concatTwoStrings(addr, ":443")
+	}
+	return concatTwoStrings(addr, ":80")
+}
+
+func concatTwoStrings(x, y string) string {
+	bb := bbPool.Get()
+	b := bb.B[:0]
+	b = append(b, x...)
+	b = append(b, y...)
+	s := bytesutil.InternBytes(b)
+	bb.B = b
+	bbPool.Put(bb)
+	return s
+}
+
+var bbPool bytesutil.ByteBufferPool
diff --git a/lib/promscrape/config.go b/lib/promscrape/config.go
index 4ae3369e9..3914c5f71 100644
--- a/lib/promscrape/config.go
+++ b/lib/promscrape/config.go
@@ -1258,36 +1258,15 @@ func (swc *scrapeWorkConfig) getScrapeWork(target string, extraLabels, metaLabel
 		droppedTargetsMap.Register(originalLabels, swc.relabelConfigs)
 		return nil, nil
 	}
-	// See https://www.robustperception.io/life-of-a-label
-	scheme := labels.Get("__scheme__")
-	if len(scheme) == 0 {
-		scheme = "http"
-	}
-	metricsPath := labels.Get("__metrics_path__")
-	if len(metricsPath) == 0 {
-		metricsPath = "/metrics"
-	}
-	address := labels.Get("__address__")
-	if len(address) == 0 {
-		// Drop target without scrape address.
+	scrapeURL, address := promrelabel.GetScrapeURL(labels, swc.params)
+	if scrapeURL == "" {
+		// Drop target without URL.
 		droppedTargetsMap.Register(originalLabels, swc.relabelConfigs)
 		return nil, nil
 	}
-	// Usability extension to Prometheus behavior: extract optional scheme and metricsPath from __address__.
-	// Prometheus silently drops targets with __address__ containing scheme or metricsPath
-	// according to https://www.robustperception.io/life-of-a-label/ .
-	if strings.HasPrefix(address, "http://") {
-		scheme = "http"
-		address = address[len("http://"):]
-	} else if strings.HasPrefix(address, "https://") {
-		scheme = "https"
-		address = address[len("https://"):]
-	}
-	if n := strings.IndexByte(address, '/'); n >= 0 {
-		metricsPath = address[n:]
-		address = address[:n]
+	if _, err := url.Parse(scrapeURL); err != nil {
+		return nil, fmt.Errorf("invalid target url=%q for job=%q: %w", scrapeURL, swc.jobName, err)
 	}
-	address = addMissingPort(address, scheme == "https")
 
 	var at *auth.Token
 	tenantID := labels.Get("__tenant_id__")
@@ -1299,23 +1278,6 @@ func (swc *scrapeWorkConfig) getScrapeWork(target string, extraLabels, metaLabel
 		at = newToken
 	}
 
-	if !strings.HasPrefix(metricsPath, "/") {
-		metricsPath = "/" + metricsPath
-	}
-	params := getParamsFromLabels(labels, swc.params)
-	optionalQuestion := ""
-	if len(params) > 0 {
-		optionalQuestion = "?"
-		if strings.Contains(metricsPath, "?") {
-			optionalQuestion = "&"
-		}
-	}
-	paramsStr := url.Values(params).Encode()
-	scrapeURL := getScrapeURL(scheme, address, metricsPath, optionalQuestion, paramsStr)
-	if _, err := url.Parse(scrapeURL); err != nil {
-		return nil, fmt.Errorf("invalid url %q for scheme=%q, target=%q, address=%q, metrics_path=%q for job=%q: %w",
-			scrapeURL, scheme, target, address, metricsPath, swc.jobName, err)
-	}
 	// Read __scrape_interval__ and __scrape_timeout__ from labels.
 	scrapeInterval := swc.scrapeInterval
 	if s := labels.Get("__scrape_interval__"); len(s) > 0 {
@@ -1398,41 +1360,6 @@ func (swc *scrapeWorkConfig) getScrapeWork(target string, extraLabels, metaLabel
 	return sw, nil
 }
 
-func getScrapeURL(scheme, address, metricsPath, optionalQuestion, paramsStr string) string {
-	bb := bbPool.Get()
-	b := bb.B[:0]
-	b = append(b, scheme...)
-	b = append(b, "://"...)
-	b = append(b, address...)
-	b = append(b, metricsPath...)
-	b = append(b, optionalQuestion...)
-	b = append(b, paramsStr...)
-	s := bytesutil.InternBytes(b)
-	bb.B = b
-	bbPool.Put(bb)
-	return s
-}
-
-func getParamsFromLabels(labels *promutils.Labels, paramsOrig map[string][]string) map[string][]string {
-	// See https://www.robustperception.io/life-of-a-label
-	var m map[string][]string
-	for _, label := range labels.GetLabels() {
-		if !strings.HasPrefix(label.Name, "__param_") {
-			continue
-		}
-		name := label.Name[len("__param_"):]
-		values := []string{label.Value}
-		if p := paramsOrig[name]; len(p) > 1 {
-			values = append(values, p[1:]...)
-		}
-		if m == nil {
-			m = make(map[string][]string)
-		}
-		m[name] = values
-	}
-	return m
-}
-
 func mergeLabels(dst *promutils.Labels, swc *scrapeWorkConfig, target string, extraLabels, metaLabels *promutils.Labels) {
 	if n := dst.Len(); n > 0 {
 		logger.Panicf("BUG: len(dst.Labels) must be 0; got %d", n)


Test patch:
diff --git a/app/vmselect/promql/aggr_incremental_test.go b/app/vmselect/promql/aggr_incremental_test.go
index 781296511..167629758 100644
--- a/app/vmselect/promql/aggr_incremental_test.go
+++ b/app/vmselect/promql/aggr_incremental_test.go
@@ -8,6 +8,7 @@ import (
 	"sync"
 	"testing"
 
+	"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
 	"github.com/VictoriaMetrics/metricsql"
 )
 
@@ -99,7 +100,7 @@ func TestIncrementalAggr(t *testing.T) {
 }
 
 func testIncrementalParallelAggr(iafc *incrementalAggrFuncContext, tssSrc, tssExpected []*timeseries) error {
-	const workersCount = 3
+	workersCount := netstorage.MaxWorkers()
 	tsCh := make(chan *timeseries)
 	var wg sync.WaitGroup
 	wg.Add(workersCount)
diff --git a/lib/promrelabel/scrape_url_test.go b/lib/promrelabel/scrape_url_test.go
new file mode 100644
index 000000000..de8da3a32
--- /dev/null
+++ b/lib/promrelabel/scrape_url_test.go
@@ -0,0 +1,54 @@
+package promrelabel
+
+import (
+	"testing"
+
+	"github.com/VictoriaMetrics/VictoriaMetrics/lib/promutils"
+)
+
+func TestGetScrapeURL(t *testing.T) {
+	f := func(labelsStr, expectedScrapeURL, expectedAddress string) {
+		t.Helper()
+		labels := promutils.MustNewLabelsFromString(labelsStr)
+		scrapeURL, address := GetScrapeURL(labels, nil)
+		if scrapeURL != expectedScrapeURL {
+			t.Fatalf("unexpected scrapeURL; got %q; want %q", scrapeURL, expectedScrapeURL)
+		}
+		if address != expectedAddress {
+			t.Fatalf("unexpected address; got %q; want %q", address, expectedAddress)
+		}
+	}
+
+	// Missing __address__
+	f("{}", "", "")
+	f(`{foo="bar"}`, "", "")
+
+	// __address__ without port
+	f(`{__address__="foo"}`, "http://foo:80/metrics", "foo:80")
+
+	// __address__ with explicit port
+	f(`{__address__="foo:1234"}`, "http://foo:1234/metrics", "foo:1234")
+
+	// explicit __scheme__
+	f(`{__address__="foo",__scheme__="https"}`, "https://foo:443/metrics", "foo:443")
+	f(`{__address__="foo:1234",__scheme__="https"}`, "https://foo:1234/metrics", "foo:1234")
+
+	// explicit __metrics_path__
+	f(`{__address__="foo",__metrics_path__="abc"}`, "http://foo:80/abc", "foo:80")
+	f(`{__address__="foo",__metrics_path__="/abc"}`, "http://foo:80/abc", "foo:80")
+	f(`{__address__="foo",__metrics_path__="/ab/c?d=ef&aa=bb"}`, "http://foo:80/ab/c?d=ef&aa=bb", "foo:80")
+
+	// explitit __param_*
+	f(`{__address__="foo",__param_x="y"}`, "http://foo:80/metrics?x=y", "foo:80")
+	f(`{__address__="foo",__param_x="y",__param_y="aa"}`, "http://foo:80/metrics?x=y&y=aa", "foo:80")
+	f(`{__address__="foo",__param_x="y",__metrics_path__="?abc=de"}`, "http://foo:80/?abc=de&x=y", "foo:80")
+	f(`{__address__="foo",__param_abc="y",__metrics_path__="?abc=de"}`, "http://foo:80/?abc=de&abc=y", "foo:80")
+
+	// __address__ with metrics path and/or scheme
+	f(`{__address__="foo/bar/baz?abc=de"}`, "http://foo:80/bar/baz?abc=de", "foo:80")
+	f(`{__address__="foo:784/bar/baz?abc=de"}`, "http://foo:784/bar/baz?abc=de", "foo:784")
+	f(`{__address__="foo:784/bar/baz?abc=de",__param_xx="yy"}`, "http://foo:784/bar/baz?abc=de&xx=yy", "foo:784")
+	f(`{__address__="foo:784/bar/baz?abc=de",__param_xx="yy",__scheme__="https"}`, "https://foo:784/bar/baz?abc=de&xx=yy", "foo:784")
+	f(`{__address__="http://foo/bar/baz?abc=de",__param_xx="yy"}`, "http://foo:80/bar/baz?abc=de&xx=yy", "foo:80")
+	f(`{__address__="https://foo/bar/baz?abc=de",__param_xx="yy"}`, "https://foo:443/bar/baz?abc=de&xx=yy", "foo:443")
+}