Repository: quic-go/quic-go

Issue:
Failing proxy tests, not covered by CI
I was trying to run all tests using `go test ./...`, but they unexpectedly fail while CI is happy. It appears that CI is not running all tests, is that intentional?

[integration.yml](https://github.com/quic-go/quic-go/blob/master/.github/workflows/integration.yml) is configured to only run `./integrationtests/tools`, `./integrationtests/versionnegotiation`, `./integrationtests/self`. It notably skips `./integrationtests/tools/proxy` as you can see in [this GitHub Actions output](https://github.com/quic-go/quic-go/actions/runs/14020194115/job/39251115118).

I bisected the proxy_test.go failures to commit 3e87ea3f50dfd3e02c8f04b254a95f2a63841e2a (broken on both Go 1.23.7 and Go 1.24.1). Should I investigate any further or is this a known issue?

This was my initial output on current git master (before I [fixed the receive buffer sysctl](https://github.com/quic-go/quic-go/issues/3801#issuecomment-2758099664)):
```
$ docker run --rm -it -v "$PWD:/w" -w /w golang:1.24
root@fcb9b6db4aed:/w# time go test ./...
Kernel Version: 6.10

2025/03/27 12:52:30 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details.
Running Suite: QUIC Suite - /w
==============================
Random Seed: 1743079950

Will run 127 of 127 specs
•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••

Ran 127 of 127 Specs in 1.507 seconds
SUCCESS! -- 127 Passed | 0 Failed | 0 Pending | 0 Skipped
--- FAIL: TestForcingReceiveBufferSize (0.00s)
    sys_conn_helper_linux_test.go:33:
        	Error Trace:	/w/sys_conn_helper_linux_test.go:33
        	Error:      	Received unexpected error:
        	            	operation not permitted
        	Test:       	TestForcingReceiveBufferSize
--- FAIL: TestForcingSendBufferSize (0.00s)
    sys_conn_helper_linux_test.go:60:
        	Error Trace:	/w/sys_conn_helper_linux_test.go:60
        	Error:      	Received unexpected error:
        	            	operation not permitted
        	Test:       	TestForcingSendBufferSize
FAIL
FAIL	github.com/quic-go/quic-go	2.628s
?   	github.com/quic-go/quic-go/example	[no test files]
?   	github.com/quic-go/quic-go/example/client	[no test files]
?   	github.com/quic-go/quic-go/example/echo	[no test files]
?   	github.com/quic-go/quic-go/fuzzing/frames	[no test files]
?   	github.com/quic-go/quic-go/fuzzing/frames/cmd	[no test files]
?   	github.com/quic-go/quic-go/fuzzing/handshake	[no test files]
?   	github.com/quic-go/quic-go/fuzzing/handshake/cmd	[no test files]
?   	github.com/quic-go/quic-go/fuzzing/header	[no test files]
?   	github.com/quic-go/quic-go/fuzzing/header/cmd	[no test files]
ok  	github.com/quic-go/quic-go/fuzzing/internal/helper	0.002s
?   	github.com/quic-go/quic-go/fuzzing/tokens	[no test files]
?   	github.com/quic-go/quic-go/fuzzing/transportparameters	[no test files]
?   	github.com/quic-go/quic-go/fuzzing/transportparameters/cmd	[no test files]
ok  	github.com/quic-go/quic-go/http3	1.747s
using QUIC version: v1
2025/03/27 12:52:31 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details.
--- FAIL: TestDatagramLoss (0.20s)
    datagram_test.go:230: dropped 12 incoming and 8 outgoing out of 205 packets
    datagram_test.go:233: server received 85 out of 100 sent datagrams
    datagram_test.go:235: client received 44 out of 100 sent datagrams
    datagram_test.go:236:
        	Error Trace:	/w/integrationtests/self/datagram_test.go:236
        	Error:      	Max difference between 92 and 44 allowed is 5, but difference was 48
        	Test:       	TestDatagramLoss
        	Messages:   	datagrams received by the client
--- FAIL: TestTimeoutAfterInactivity (0.19s)
    timeout_test.go:296:
        	Error Trace:	/w/integrationtests/self/timeout_test.go:296
        	Error:      	"185.445333ms" is not less than "180ms"
        	Test:       	TestTimeoutAfterInactivity
FAIL
FAIL	github.com/quic-go/quic-go/integrationtests/self	46.562s
?   	github.com/quic-go/quic-go/integrationtests/tools	[no test files]
?   	github.com/quic-go/quic-go/integrationtests/tools/israce	[no test files]
--- FAIL: TestProxyingBackAndForth (1.00s)
    proxy_test.go:107: timeout
--- FAIL: TestDropIncomingPackets (1.10s)
    proxy_test.go:107: timeout
--- FAIL: TestDropOutgoingPackets (1.10s)
    proxy_test.go:107: timeout
--- FAIL: TestDelayIncomingPackets (1.61s)
    proxy_test.go:107: timeout
--- FAIL: TestPacketReordering (1.61s)
    proxy_test.go:107: timeout
--- FAIL: TestConstantDelay (1.11s)
    proxy_test.go:107: timeout
--- FAIL: TestDelayOutgoingPackets (1.61s)
    proxy_test.go:107: timeout
FAIL
FAIL	github.com/quic-go/quic-go/integrationtests/tools/proxy	9.160s
ok  	github.com/quic-go/quic-go/integrationtests/versionnegotiation	0.516s
ok  	github.com/quic-go/quic-go/internal/ackhandler	0.019s
ok  	github.com/quic-go/quic-go/internal/congestion	0.033s
ok  	github.com/quic-go/quic-go/internal/flowcontrol	0.001s
ok  	github.com/quic-go/quic-go/internal/handshake	0.160s
?   	github.com/quic-go/quic-go/internal/mocks	[no test files]
?   	github.com/quic-go/quic-go/internal/mocks/ackhandler	[no test files]
?   	github.com/quic-go/quic-go/internal/mocks/logging	[no test files]
?   	github.com/quic-go/quic-go/internal/mocks/logging/internal	[no test files]
?   	github.com/quic-go/quic-go/internal/mocks/quic	[no test files]
ok  	github.com/quic-go/quic-go/internal/protocol	0.004s
ok  	github.com/quic-go/quic-go/internal/qerr	0.002s
ok  	github.com/quic-go/quic-go/internal/qtls	0.014s
ok  	github.com/quic-go/quic-go/internal/utils	0.244s
?   	github.com/quic-go/quic-go/internal/utils/linkedlist	[no test files]
ok  	github.com/quic-go/quic-go/internal/utils/ringbuffer	0.002s
ok  	github.com/quic-go/quic-go/internal/wire	0.022s
?   	github.com/quic-go/quic-go/interop/client	[no test files]
ok  	github.com/quic-go/quic-go/interop/http09	0.010s
?   	github.com/quic-go/quic-go/interop/server	[no test files]
?   	github.com/quic-go/quic-go/interop/utils	[no test files]
ok  	github.com/quic-go/quic-go/logging	0.002s
?   	github.com/quic-go/quic-go/metrics	[no test files]
ok  	github.com/quic-go/quic-go/qlog	0.004s
ok  	github.com/quic-go/quic-go/quicvarint	0.002s
?   	github.com/quic-go/quic-go/testutils	[no test files]
FAIL

real	0m54.591s
user	0m52.358s
sys	0m7.519s
```

Hints from issue comments:
@marten-seemann: You're right, we should execute tests recursively here:
https://github.com/quic-go/quic-go/blob/623e59b2a22e862ef2a5702acb0de2bf96373d14/.github/workflows/integration.yml#L49-L50@Lekensteyn: > You're right, we should execute tests recursively here:
> 
> https://github.com/quic-go/quic-go/blob/623e59b2a22e862ef2a5702acb0de2bf96373d14/.github/workflows/integration.yml#L49-L50

I have updated PR #5010 to run all tests within that package. There are still lots of other tests that are not being run. Should these be included too?

```
$ git -P grep -lne 'func Test' ':**_test.go' ':!integrationtests'
```
<details>
<summary>(127 files)</summary>

```
buffer_pool_test.go
client_test.go
closed_conn_test.go
config_test.go
conn_id_generator_test.go
conn_id_manager_test.go
connection_logging_test.go
connection_test.go
connection_timer_test.go
crypto_stream_manager_test.go
crypto_stream_test.go
datagram_queue_test.go
errors_test.go
frame_sorter_test.go
framer_test.go
fuzzing/internal/helper/helper_test.go
http3/error_codes_test.go
http3/error_test.go
http3/http3_suite_test.go
http3/transport_test.go
internal/ackhandler/ack_eliciting_test.go
internal/ackhandler/ackhandler_suite_test.go
internal/ackhandler/received_packet_handler_test.go
internal/ackhandler/received_packet_history_test.go
internal/ackhandler/received_packet_tracker_test.go
internal/ackhandler/send_mode_test.go
internal/ackhandler/sent_packet_handler_test.go
internal/ackhandler/sent_packet_history_test.go
internal/congestion/congestion_suite_test.go
internal/congestion/pacer_test.go
internal/flowcontrol/connection_flow_controller_test.go
internal/flowcontrol/stream_flow_controller_test.go
internal/handshake/aead_test.go
internal/handshake/crypto_setup_test.go
internal/handshake/handshake_helpers_test.go
internal/handshake/hkdf_test.go
internal/handshake/initial_aead_test.go
internal/handshake/retry_test.go
internal/handshake/session_ticket_test.go
internal/handshake/tls_config_test.go
internal/handshake/token_generator_test.go
internal/handshake/token_protector_test.go
internal/handshake/updatable_aead_test.go
internal/protocol/connection_id_test.go
internal/protocol/encryption_level_test.go
internal/protocol/key_phase_test.go
internal/protocol/packet_number_test.go
internal/protocol/params_test.go
internal/protocol/perspective_test.go
internal/protocol/protocol_test.go
internal/protocol/stream_test.go
internal/protocol/version_test.go
internal/qerr/errorcodes_test.go
internal/qerr/errors_test.go
internal/qtls/cipher_suite_test.go
internal/testdata/cert_test.go
internal/utils/buffered_write_closer_test.go
internal/utils/log_test.go
internal/utils/rand_test.go
internal/utils/ringbuffer/ringbuffer_test.go
internal/utils/rtt_stats_test.go
internal/utils/timer_test.go
internal/wire/ack_frame_test.go
internal/wire/ack_range_test.go
internal/wire/connection_close_frame_test.go
internal/wire/crypto_frame_test.go
internal/wire/data_blocked_frame_test.go
internal/wire/datagram_frame_test.go
internal/wire/extended_header_test.go
internal/wire/frame_parser_test.go
internal/wire/frame_test.go
internal/wire/handshake_done_frame_test.go
internal/wire/header_test.go
internal/wire/log_test.go
internal/wire/max_data_frame_test.go
internal/wire/max_stream_data_frame_test.go
internal/wire/max_streams_frame_test.go
internal/wire/new_connection_id_frame_test.go
internal/wire/new_token_frame_test.go
internal/wire/path_challenge_frame_test.go
internal/wire/path_response_frame_test.go
internal/wire/ping_frame_test.go
internal/wire/pool_test.go
internal/wire/reset_stream_frame_test.go
internal/wire/retire_connection_id_frame_test.go
internal/wire/short_header_test.go
internal/wire/stop_sending_frame_test.go
internal/wire/stream_data_blocked_frame_test.go
internal/wire/stream_frame_test.go
internal/wire/streams_blocked_frame_test.go
internal/wire/transport_parameter_test.go
internal/wire/version_negotiation_test.go
interop/http09/http_test.go
logging/connection_tracer_test.go
logging/packet_header_test.go
logging/tracer_test.go
mtu_discoverer_test.go
packet_handler_map_test.go
packet_packer_test.go
packet_unpacker_test.go
path_manager_outgoing_test.go
path_manager_test.go
qlog/connection_tracer_test.go
qlog/event_test.go
qlog/frame_test.go
qlog/packet_header_test.go
qlog/qlog_dir_test.go
qlog/tracer_test.go
qlog/types_test.go
qlog/writer_test.go
quic_suite_test.go
quicvarint/io_test.go
quicvarint/varint_test.go
receive_stream_test.go
send_conn_test.go
send_queue_test.go
send_stream_test.go
server_test.go
stateless_reset_test.go
stream_test.go
sys_conn_df_darwin_test.go
sys_conn_helper_linux_test.go
sys_conn_oob_test.go
sys_conn_test.go
sys_conn_windows_test.go
token_store_test.go
transport_test.go
```
</details>

Solution patch:
diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml
index dcfc4eda..f762af33 100644
--- a/.github/workflows/integration.yml
+++ b/.github/workflows/integration.yml
@@ -47,7 +47,7 @@ jobs:
         run: echo "GOARCH=386" >> $GITHUB_ENV
       - run: go version
       - name: Run tools tests
-        run: go test ${{ env.RACEFLAG }} -v -timeout 30s -shuffle=on ./integrationtests/tools
+        run: go test ${{ env.RACEFLAG }} -v -timeout 30s -shuffle=on ./integrationtests/tools/...
       - name: Run version negotiation tests
         run: go test ${{ env.RACEFLAG }} -v -timeout 30s -shuffle=on ./integrationtests/versionnegotiation ${{ env.QLOGFLAG }}
       - name: Run self tests, using QUIC v1



Test patch:
diff --git a/integrationtests/tools/proxy/proxy_test.go b/integrationtests/tools/proxy/proxy_test.go
index b53a214b..4211512e 100644
--- a/integrationtests/tools/proxy/proxy_test.go
+++ b/integrationtests/tools/proxy/proxy_test.go
@@ -79,10 +79,17 @@ func readPacketNumber(t *testing.T, b []byte) protocol.PacketNumber {
 // Set up a dumb UDP server.
 // In production this would be a QUIC server.
 func runServer(t *testing.T) (*net.UDPAddr, chan []byte) {
-	serverConn := newUPDConnLocalhost(t)
+	done := make(chan struct{})
+	t.Cleanup(func() {
+		select {
+		case <-done:
+		case <-time.After(time.Second):
+			t.Fatal("timeout")
+		}
+	})
 
+	serverConn := newUPDConnLocalhost(t)
 	serverReceivedPackets := make(chan []byte, 100)
-	done := make(chan struct{})
 	go func() {
 		defer close(done)
 		for {
@@ -100,14 +107,6 @@ func runServer(t *testing.T) (*net.UDPAddr, chan []byte) {
 		}
 	}()
 
-	t.Cleanup(func() {
-		select {
-		case <-done:
-		case <-time.After(time.Second):
-			t.Fatalf("timeout")
-		}
-	})
-
 	return serverConn.LocalAddr().(*net.UDPAddr), serverReceivedPackets
 }
 
@@ -433,11 +432,8 @@ func TestProxySwitchConn(t *testing.T) {
 		Data []byte
 		Addr *net.UDPAddr
 	}
-
 	serverReceivedPackets := make(chan packet, 1)
-	done := make(chan struct{})
 	go func() {
-		defer close(done)
 		for {
 			buf := make([]byte, 1000)
 			n, addr, err := serverConn.ReadFromUDP(buf)