Repository: go
Issue: runtime: frequent ReadMemStats will cause heap_live to exceed next_gc 
Labels: NeedsInvestigation,compiler/runtime

<!--
Please answer these questions before submitting your issue. Thanks!
-->

### What version of Go are you using (`go version`)?

<pre>
$ go version
go version go1.16.8 darwin/amd64
</pre>

### Does this issue reproduce with the latest release?
Maybe, I am not sure, [GC Pacer Redesign](https://github.com/golang/proposal/blob/329650d4723a558c2b76b81b4995fc5c267e6bc1/design/44167-gc-pacer-redesign.md#heap-goal) maybe reduce the probability of it


### What operating system and processor architecture are you using (`go env`)?
darwin/amd64

### What did you do?

<!--
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on go.dev/play is best.
-->

A test similar to https://go-review.googlesource.com/c/go/+/227767/3
```go
package main_test

import (
	"runtime"
	"testing"
)

var setGCPercentBallast interface{}
var setGCPercentSink interface{}

func TestMemStats(t *testing.T) {
	defer func() {
		setGCPercentBallast = nil
		setGCPercentSink = nil
		runtime.GC()
	}()

	runtime.GC()
	var m1 runtime.MemStats
	runtime.ReadMemStats(&m1)

	// Create 50 MB of live heap as a baseline.
	const baseline = 50 << 20
	setGCPercentBallast = make([]byte, baseline-m1.Alloc)
	// Allocate a bunch and check that we stay under the limit.
	runtime.ReadMemStats(&m1)
	var m2 runtime.MemStats
	for i := 0; i < 2000; i++ {
		setGCPercentSink = make([]byte, 1<<20)
		runtime.ReadMemStats(&m2)
	}
	if m1.NumGC == m2.NumGC {
	}
}
```

```bash
go test -c .
GODEBUG=gctrace=1 ./demo.test -test.v -test.trace=./16_trace_1 -test.run=^TestMemStats$
```


### What did you expect to see?
heap_live <= next_gc


### What did you see instead?
probabilistic recurrence  
```bash
gc 4 @0.028s 1%: 0.015+0.87+0.042 ms clock, 0.18+0.095/0.18/0.19+0.50 ms cpu, 102->103->53 MB, 104 MB goal, 12 P
gc 5 @0.037s 1%: 0.015+0.50+0.018 ms clock, 0.19+0.12/0.26/0.089+0.21 ms cpu, 104->108->56 MB, 106 MB goal, 12 P
gc 6 @0.043s 1%: 0.021+0.40+0.025 ms clock, 0.25+0.055/0.20/0.17+0.30 ms cpu, 109->110->53 MB, 112 MB goal, 12 P
```
and  
```bash
gc 23 @0.217s 1%: 0.19+0.80+0.033 ms clock, 2.3+0.076/0.25/0.13+0.39 ms cpu, 106->107->53 MB, 108 MB goal, 12 P
gc 24 @0.242s 1%: 0.063+4.0+0.051 ms clock, 0.76+0.081/0.33/0.001+0.61 ms cpu, 104->111->59 MB, 106 MB goal, 12 P
gc 25 @0.256s 1%: 0.046+1.1+0.008 ms clock, 0.55+0.13/0.13/0.24+0.10 ms cpu, 113->114->53 MB, 118 MB goal, 12 P
```  
While this example is extreme, does this result meet expectations?  
There is only one user Goroutine running in the demo,
In this scenario, should the GC pacer mechanism ensure that heap_live <= next_gc ?  
some brief analysis:  
I guess it is related to this commit https://go-review.googlesource.com/c/go/+/216557 ,  
Frequent ReadMemStats/STW causes GC mark goroutine to execute too slowly.    
I also found some interesting phenomena in the positioning of this problem (maybe it is normal, but maybe some strategy adjustments can be made if necessary)  
1. M is awakened but cannot execute bgscavenge in Runnable. Maybe it is caused by directly entering the following code in findrunnable?  
```go
	if sched.gcwaiting != 0 {
		gcstopm()
		goto top
	}
```

![image](https://user-images.githubusercontent.com/43537346/149280832-dabac28f-213d-4ad7-9ed0-fd99b99a1e43.png)  

2. In this domo scenario, since there is only one user goroutine, after M is woken up, the findRunnableGCWorker will not execute in the findrunnable, and the gcMarkWorker of DedicatedMode starts too slowly.