Change:
// TestClientReceivedGOAWAY tests the in-flight watch requests will not be affected and new requests use a new // connection after client received GOAWAY. func TestClientReceivedGOAWAY(t *testing.T) { <del> t.Skip("disabled because of https://github.com/kubernetes/kubernetes/issues/94622") </del> s, err := newTestGOAWAYServer() if err != nil { t.Fatalf("failed to set-up test GOAWAY http server, err: %v", err)

Discussion:
Before merged, TestClientReceivedGOAWAY failed with data race on . However, as far as I know, it's not the root cause of TestClientReceivedGOAWAY flake, because dial a new connection only when client send a new request, and in this case, requests are sent one by one. In other words, if data race on , then the additional unexpected connection set up, which means the test case or the configuration has problem. After merged, TestClientReceivedGOAWAY still failed in many jobs, such as , , which may be a good proof of my suspect. And I got a failed job with data race issue that read at (main testing goroutine) and write at , which told us the http client try to set-up a new connection even when we had already finished all requests(may caused by goroutine leak/block in http client). I'm still debug the root cause of TestClientReceivedGOAWAY flake, for now, I doubt that I used an incorrect http2 server/client configuration( or SETTINGSMAXCONCURRENT_STREAMS limited), or the dial function ignored context.Context which will cause more than one goroutine dial the connection when the first canceled(not found any where will cancel the requests). I still can not reproduce this failed case and it's hard to find the root cause. Couple of days may be needed to find the root cause, and I will post a PR to skip TestClientReceivedGOAWAY before I solved the problem. Sorry for any inconvenient.
/assign /sig testing
cc
Thank you for working on this!! How often does it flake? I'm just wondering if we should disable it in the mean time to avoid angry contributors.
On my PC, about 1/1000. I will post a PR to disable it for now.
Thank you so much! That's not a bad rate at all, but we run a lot of tests. This is super helpful!
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or .

/lifecycle stale
/lifecycle stale I'm still working on it, a new golang version(1.16 or 1.15.7+) release required
/sig api-machinery /remove-sig testing AFAICT this is specific to these tests, not the test framework or CI infrastructure.
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
the tests re-enabled in started flaking again immediately Example failure: /milestone v1.21 the tests should be deflaked or disabled for 1.21
/cc
/triage accepted
+1 in recent PR tests
Sorry for my slow response. I'm working on it, looks like I need to debug golang HTTP client again :-P
Shall we revert in the meantime?
No, has already fixed by golang 1.15.8 I think. This one is caused by another problem.
I will disable TestClientReceivedGOAWAY case again if I can not find the root cause this night.
Can not reproduce on my PC, disable this test case before I figure out the root cause:
I run this test case with this command and can not reproduce. Does any thing different with command ?
moving out of the milestone and lowering priority becuase merged, leaving open because the fundamental issue is unresolved /milestone clear
will do something comparable under make. There's a bunch of stuff in there, generally it's not relevant for a unit test bug like this but the answer to what exactly it does is not easily concise. Note that CI may have more load than your local machine running it once, and also currently runs as root.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at . /lifecycle stale
/lifecycle active /assign ref:

This fails close to 100% of the time when run with race mode: Disabled in It is unclear if the test or the component needs to be corrected /sig api-machinery /priority critical-urgent
/assign /cc
GitHub didn't allow me to assign the following users: answer1991. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
The test is also flaky. but this one is hard to reproduce locally.
I will debug and try to fix it later, thanks for the report.
I doubt data race is caused by golang http/http2 pkg, let me try to make some reproduce cases and may open an issue to golang repo.
data race is caused by golang http/http2 pkg, I have already opened an issue at golang repo. I will continue to debug tomorrow.
another case:
For case, change to be would be a solution: But I would rather figure out why client will set-up the additional TCP connections.
Fix TestGOAWAYConcurrency flaky may need a new golang version after golang/go fixed. TestClientReceivedGOAWAY will be tracked at .
Great find-- can we skip the test for now while we wait for the go fix to land?
Yes, I think so.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/lifecycle stale I'm still working on it, a new golang version(1.16 or 1.15.7+) release required
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with .

Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
As clients in k/k repo use to config http2 transports instead of use golang SDK h2_bundle, a new version of (with ) will fix this issue. However, for other clients which use default http2 configuration in golang SDK, golang 1.15.8 or 1.16+ required to avoid HTTP2 un-expected error when received GOAWAY frame.
15.8 should be landed now
yes, I have already post a pr to enable goaway feature test cases: