Change:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --pull-kubernetes-integration TestMutatingWebhookResetsInvalidManagedFields Not sure. Example failures from recent PRs: (PR: ) (PR: ) /sig api-machinery /kind flake
There's even a few failures in the post-submit master integration tests job: (ignore the test/integration/job failures, I have an open PR to fix that).
cc
Thanks! /cc /wg api-expression
Oh no, what did I do :-/ I'll have a look. Might be that the admission webhook is not up quick enough. /assign
/triage accepted
I tried this locally and it passes, so maybe I can try stress testing this
Would increasing the timelimit here give more time to the admissionhook to come up ?
I think so. Looked at it a few days ago, I'm not sure how much we should increase it. I don't want to slow down the tests a lot more if we can avoid it. Do we know if the other admission tests (duplicate owner ref) have the same issue?
Can we make the webhook blocking so that the request will wait for it to be ready? Can we make sure that it's up-and-running somehow before performing the actual test? Having an arbitrary "wait" before we start the test seems wrong.
The poll and timeout are fine as is. lastErr is only getting set, never unset. If validateManagedFieldsAndDecode returns nil, lastErr should get cleared.
Hmm.. actually, the one failing is the Update request, after the Patch request has already been checked. The patch request passing means the webhook was effective and working, the update request should work consistently the first time. If the patch request really is ensuring the webhook was installed, then a timeout or retry on the update request is not the right approach.
But isnt the patch request alos failing to report the adequate warnings
this continues to be the flakiest test in the integration job in by a wide margin
So sorry for letting this slip for so long :-/ I'll work on it right when I get back home in an hour.

Discussion:
// Make sure reset happens on patch requests // wait until new webhook is called if err := wait.PollImmediate(time.Millisecond*5, wait.ForeverTestTimeout, func() (bool, error) { <ins> defer recordedWarnings.Reset() </ins> pod, err = client.CoreV1().Pods("default").Patch(context.TODO(), invalidManagedFieldsMarkerFixture.Name, types.JSONPatchType, []byte("[]"), metav1.PatchOptions{}) if err != nil { return false, err

lastErr = err return false, nil } <ins> if warningWriter.WarningCount() != 1 { lastErr = fmt.Errorf("expected one warning, got: %v", warningWriter.WarningCount()) return false, nil } if !strings.Contains(recordedWarnings.String(), expectedWarning) { lastErr = fmt.Errorf("unexpected warning, expected: n%vn, got: n%v", expectedWarning, recordedWarnings.String()) return false, nil } lastErr = nil </ins> return true, nil <del> }); err != nil { </del> <ins> }); err != nil || lastErr != nil { </ins> t.Fatalf("failed to wait for apiserver handling webhook mutation: %v, last error: %v", err, lastErr) } <del> if lastErr != nil { t.Fatal(lastErr) } if warningWriter.WarningCount() != 1 { t.Errorf("expected one warning, got: %v", warningWriter.WarningCount()) } if !strings.Contains(recordedWarnings.String(), expectedWarning) { t.Errorf("unexpected warning, expected: n%vn, got: n%v", expectedWarning, recordedWarnings.String()) } recordedWarnings.Reset() </del> // Make sure reset happens in update requests pod, err = client.CoreV1().Pods("default").Update(context.TODO(), pod, metav1.UpdateOptions{})