Repository: go
Issue: runtime/pprof: GOOS=darwin over-reports CPU usage in mutex-heavy workload
Labels: NeedsInvestigation,compiler/runtime

### Go version

```
$ ~/sdk/go1.23/bin/go version
go version go1.23rc2 darwin/arm64
```

### Output of `go env` in your module/workspace:

```shell
$ ~/sdk/go1.23/bin/go env -changed
$
```

### What did you do?

On an M1 Macbook Air, which I understand to have 8 cores of 1 thread each (4 performance, 4 efficiency), I ran the runtime's ChanContended benchmark while varying GOMAXPROCS, collecting a CPU profile for each.

```
$ for i in 1 2 4 8; do ~/sdk/go1.23/bin/go test runtime -run=^$ -bench=ChanContended -cpu=$i -cpuprofile=/tmp/cpu$i; done
```

### What did you see happen?

The profile reports using an average of 0.62 threads of on-CPU time when GOMAXPROCS is 1, and 1.21 when GOMAXPROCS is 2.

However, when GOMAXPROCS is 4 and 8, it reports using 125.54 and 403.14 threads of on-CPU time respectively. It claims that the program consumed 930 seconds of on-CPU time over a period of 2.31 seconds of wall-clock time.

That's more than 50 times more hardware threads than the machine really has.

### What did you expect to see?

I expected the profile to report no more than 100 CPU profile samples per thread-second.

---

This may be related to #57722 . In that issue, GOOS=darwin CPU profiles included the right number of samples but skewed them towards a particular portion of the workload. But here, the workload itself is experiencing extreme over-sampling.

```
$ for i in 1 2 4 8; do ~/sdk/go1.23/bin/go test runtime -run=^$ -bench=ChanContended -cpu=$i -cpuprofile=/tmp/cpu$i; done
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M1
BenchmarkChanContended 	  446238	      2638 ns/op
PASS
ok  	runtime	2.113s
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M1
BenchmarkChanContended-2   	  354793	      3331 ns/op
PASS
ok  	runtime	2.241s
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M1
BenchmarkChanContended-4   	  231181	      5051 ns/op
PASS
ok  	runtime	1.560s
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M1
BenchmarkChanContended-8   	  202797	      5902 ns/op
PASS
ok  	runtime	2.468s

$ for i in 1 2 4 8; do ~/sdk/go1.23/bin/go tool pprof /tmp/cpu$i </dev/null 2>&1 | grep Duration; done
Duration: 1.95s, Total samples = 1.21s (62.01%)
Duration: 2.09s, Total samples = 2.53s (121.18%)
Duration: 1.40s, Total samples = 175.74s (12553.94%)
Duration: 2.31s, Total samples = 929.99s (40314.39%)
```

A combined execution trace and CPU profile show a change in the density of the magenta "CPU profile sample" annotations at the time when the test transitions from GOMAXPROCS=2 to GOMAXPROCS=3. (It's like it's the opposite of #35057 , where GOOS=linux profiles _under-sampled_ when the program's CPU usage crossed the 250% boundary.)

```
$ ~/sdk/go1.23/bin/go test runtime -run=^$ -bench=ChanContended -cpu=1,2,3,4,5,6,7,8 -cpuprofile=/tmp/cpu -trace=/tmp/cct
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M1
BenchmarkChanContended     	  447841	      2642 ns/op
BenchmarkChanContended-2   	  354488	      3363 ns/op
BenchmarkChanContended-3   	  265177	      4586 ns/op
BenchmarkChanContended-4   	  237844	      5083 ns/op
BenchmarkChanContended-5   	  215295	      5618 ns/op
BenchmarkChanContended-6   	  207518	      5817 ns/op
BenchmarkChanContended-7   	  209977	      5872 ns/op
BenchmarkChanContended-8   	  206673	      5850 ns/op
PASS
ok  	runtime	11.268s

$ ~/sdk/go1.23/bin/go tool trace /tmp/cct
```

<img width="1368" alt="Screenshot 2024-07-16 at 4 20 20 PM" src="https://github.com/user-attachments/assets/f4473cce-7bea-45bf-8834-175c182e1aa1">