Repository: go
Issue: net: sending UDP datagrams back and forth fails with "sendmsg: operation not permitted"
Labels: NeedsInvestigation

<!--
Please answer these questions before submitting your issue. Thanks!
-->

### What version of Go are you using (`go version`)?

<pre>
$ go version
go1.21.0 linux/arm64
</pre>

### Does this issue reproduce with the latest release?

Yes

### What operating system and processor architecture are you using (`go env`)?

<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/home/parallels/.cache/go-build'
GOENV='/home/parallels/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/parallels/src/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/parallels/src/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/parallels/bin/go1.21ex'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/parallels/bin/go1.21ex/pkg/tool/linux_arm64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/parallels/src/go/src/github.com/libp2p/go-libp2p/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build618110419=/tmp/go-build -gno-record-gcc-switches'

</pre></details>

### What did you do?

I created two UDP sockets, and send UDP datagrams from one to the other, and vice versa:

```go
func newUDPConn(t *testing.T) *net.UDPConn {
	addr, err := net.ResolveUDPAddr("udp", "localhost:0")
	if err != nil {
		t.Fatal(err)
	}
	conn, err := net.ListenUDP("udp", addr)
	if err != nil {
		t.Fatal(err)
	}
	t.Cleanup(func() { conn.Close() })
	return conn
}

func TestUDPSending(t *testing.T) {
	c1 := newUDPConn(t)
	c2 := newUDPConn(t)

	const num = 1000
	done := make(chan struct{}, 2)
	var failed1, failed2 bool
	go func() {
		defer func() { done <- struct{}{} }()
		for i := 0; i < num; i++ {
			_, _, err := c1.WriteMsgUDP(make([]byte, 1000), nil, c2.LocalAddr().(*net.UDPAddr))
			if err != nil {
				failed1 = true
				t.Logf("c1 send failed %d: %v\n", i, err)
			}
			time.Sleep(200 * time.Microsecond)
		}
	}()

	go func() {
		defer func() { done <- struct{}{} }()
		for i := 0; i < num; i++ {
			_, _, err := c2.WriteMsgUDP(make([]byte, 1000), nil, c1.LocalAddr().(*net.UDPAddr))
			if err != nil {
				failed2 = true
				t.Logf("c2 send failed %d: %v\n", i, err)
			}
			time.Sleep(200 * time.Microsecond)
		}
	}()

	<-done
	<-done

	if failed1 || failed2 {
		t.Fail()
	}
}
```


### What did you expect to see?

I expected UDP datagrams to be sent on both sockets.

### What did you see instead?

More often than not, this test fails on Linux:
```
c1 send failed 0: write udp 127.0.0.1:47716->127.0.0.1:34183: sendmsg: operation not permitted
```

It's either the first or the second connection that fails, but if there's a failure, it's always only the first call that fails. All subsequent calls complete successfully.

On macOS, this code doesn't produce any errors.