Repository: go
Issue: runtime: bgsweep does not yield to netpoller-readied goroutines
Labels: NeedsInvestigation,compiler/runtime

When `bgsweep` yields, it puts itself on the global run queue. When its P looks for work, the P finds that same goroutine in the global run queue before considering other Ps' local run queues. This results in bgsweep running at higher priority than any sort of work that gets enqueued in a P's local run queue (such as from the netpoller, which may be the easiest type to see in execution traces).

For a 4-thread process, this means the GC consumes 1/4 of available processing time for longer than I'd expect (and perhaps longer than necessary). And if I understand this behavior correctly, a 2-thread process could end up spending 1/2 of its available processing time running GC-related code, in excess of the GC's "25% overhead" design goal.

CC @golang/runtime 

### What version of Go are you using (`go version`)?

<pre>
$ go version
go version go1.19.1 linux/amd64
</pre>

### Does this issue reproduce with the latest release?

I haven't checked since this week's release of go1.19.2 (though the change log for that doesn't mention anything related). I also haven't checked after https://go.dev/cl/429615 (for issue #54767), which isn't part of a stable release yet. That change still attempts to yield using `mcall(gosched_m)`.

### What operating system and processor architecture are you using (`go env`)?

The production system runs on a Linux / AMD64 VM that presents four hardware threads.

### What did you do?

The service receives inbound HTTP requests, makes outbound HTTP requests to its dependencies, and does some calculation/processing. This allocates memory, and under load it does a GC cycle about twice per second.

### What did you expect to see?

I expected the `runtime.bgsweep` goroutine to run at a very low priority, to yield its processor to other goroutines if any are runnable. I expected that to include goroutines that are made runnable as a result of inbound network traffic.

### What did you see instead?

In execution traces, I see the program keeping 3 of the 4 processors busy with user goroutines and 1 of the 4 processors busy with `runtime.bgsweep` (Proc 3 in the image below). I see the count of runnable goroutines increase beyond zero, while bgsweep continues to yield and re-acquire its processor. I see goroutines made runnable by the netpoller wait up to several milliseconds before they're picked up by a processor (Proc 0 in the image below), during which the bgsweep goroutine is rescheduled over 1000 times (Proc 3).

In Go 1.19, bgsweep calls `Gosched`. In tip, it calls `goschedIfBusy`. Both lead to `mcall(gosched_m)`, which leads to `goschedImpl(gp)`, which leads to `globrunqput(gp)`, which puts `gp` on the scheduler's global run queue.

When a P finishes its current work and gets more from `findRunnable`, it will sometimes find no "special" work (safe point function, trace reader, GC mark worker, finalizer), no work in its local run queue, and no work in the global run queue. It will then poll the network and enqueue it via `injectglist`. When that has a P, and finds that there are no (other) idle Ps, it will place as many as it can (up to 256?) of the newly-runnable goroutines in its own local run queue. (I think the netpoller will only return 128 at a time. But this is a problem with even a small number of goroutines enqueued this way.)

The P that had been running the `bgsweep` worker needs new work, so it calls `findRunnable`. It finds no "special" work, finds nothing in _its own_ local run queue, and then checks the global run queue. There it finds the `bgsweep` goroutine that it dropped off a moment earlier. It picks it back up and continues the sweep work.

In the meantime, the goroutines that another P made runnable (because it grabbed everything that the netpoller had to offer) remain in a different local run queue. No P ever gets to the point of calling `stealWork`.

<img width="1450" alt="bgsweep hogs P, leading to delay in processing inbound network traffic" src="https://user-images.githubusercontent.com/230685/194171778-0357315c-620a-4150-8e9a-21cef30266a9.png">