Repository: go
Issue: runtime: frequent GC related asserts seen on illumos based OSes (but only on AMD!)
Labels: NeedsInvestigation,OS-illumos,compiler/runtime

### What version of Go are you using (`go version`)?

This problem reproduces for several versions of 1.16, release 1.17, and release 1.18.

### Does this issue reproduce with the latest release?

Yes (`go1.18.3 solaris/amd64`)

### What operating system and processor architecture are you using (`go env`)?

We've seen errors on AMD Ryzen and EPYC, running OmniOS, and have **not** been able to reproduce on Xeon systems.

<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/export/home/james/.cache/go-build"
GOENV="/export/home/james/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="solaris"
GOINSECURE=""
GOMODCACHE="/export/home/james/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="solaris"
GOPATH="/export/home/james/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/export/home/james/goroot"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/export/home/james/goroot/pkg/tool/solaris_amd64"
GOVCS=""
GOVERSION="go1.18.3"
GCCGO="/usr/bin/gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2037678958=/tmp/go-build -gno-record-gcc-switches"
</pre></details>

### What did you do?

The errors we saw are in the "Testing packages" phase of `all.bash`. To reproduce from a fresh OmniOS system, the following script was run:

```bash
#!/bin/bash
set -e

unset GOBIN
unset GOPATH

# assuming a fresh system, bootstrap go 1.4 from C toolchain
# requires `pkg install git build-essential`
if [[ ! -e bootstrap/ ]];
then
    git clone \
        --recurse-submodules \
        -b release-branch.go1.4 \
        --depth=1 \
        https://go.googlesource.com/go bootstrap

    (
     cd bootstrap/src/
     export CGO_ENABLED=0
     bash ./make.bash
    )
fi

#VERSION="go1.16.10"
#VERSION="go1.16.11"
#VERSION="go1.16.15"
#VERSION="release-branch.go1.17"
VERSION="release-branch.go1.18"

# build $VERSION using go 1.4
rm -rf bootstrap2/

if [[ ! -e bootstrap2 ]];
then
    export GOROOT_BOOTSTRAP=${HOME}/bootstrap/
    $GOROOT_BOOTSTRAP/bin/go version

    git clone \
        --recurse-submodules \
        -b $VERSION \
        --depth=1 \
        https://go.googlesource.com/go bootstrap2

    ( cd bootstrap2/src && bash ./make.bash )

    unset GOROOT_BOOTSTRAP
fi

export PATH="${HOME}/bootstrap2/bin:$PATH"

# build $VERSION in a loop
while :;
do
        rm -rf goroot/
        rm -rf /tmp/go-* /tmp/cgo*

        git clone \
                --recurse-submodules \
                -b $VERSION \
                --depth=1 \
                https://go.googlesource.com/go goroot

        # core files, please
        sed -i -e '/ulimit -c 0/d' goroot/src/run.bash

        ( cd goroot/src && bash ./all.bash )
done;
```

See https://github.com/golang/go/issues/45775#issuecomment-1149099750 for instructions on reproducing this on OmniOS under qemu.

### What did you expect to see?

`ALL TESTS PASSED`

### What did you see instead?

Eventually, the while loop in that reproduction script fails. We've seen the following:

`fatal error: s.allocCount != s.nelems && freeIndex == s.nelems`

```
2022/06/07 16:51:53 loading imports: exit status 2
runtime: marked free object in span 0xfffffc7fecc2d158, elemsize=4096 freeindex=1 (bad use of unsafe.Pointer? try -d=checkptr)
0xc0002fa000 alloc unmarked
0xc0002fb000 free  unmarked
fatal error: found pointer to free object
```

```
##### ../misc/cgo/testso
--- FAIL: TestSO (0.37s)
    so_test.go:100: gcc -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3726058972=/tmp/go-build -gno-record-gcc-switches -shared -o libcgosotest.so cgoso_c.c:
    so_test.go:117: go build -o main.exe main.go: exit status 2
        runtime: marked free object in span 0xfffffc7fef178a78, elemsize=144 freeindex=1 (bad use of unsafe.Pointer? try -d=checkptr)
```