Change:
Which jobs are flaking: Which test(s) are flaking: Testgrid link: Reason for failure: Anything else we need to know: Triage links: Spyglass links: Related ginkgo test: /sig cloud-provider-gcp
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : Which jobs are flaking: Anything else we need to know: Triage links: Spyglass links: /sig cloud provider gcp Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cloud-provider
/area provider/gcp cc
cc
do we have an assignee to resolve this?
/assign
GitHub didn't allow me to assign the following users: DangerOnTheRanger. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The flake seems to be caused by an intermittent race condition related to pod creation and waiting for pod readiness. Right now in the E2E testing framework, there's an if a pod that the caller is trying to wait on doesn't exist. If the pod is created after that branch is executed, the test will prematurely fail. We can see an example of that with (test log edited to show relevant lines): Note the timestamp of pod creation versus when the test failed. Another example from a : And for good measure:
/assign

Discussion:
// WaitForPodCondition waits a pods to be matched to the given condition. func WaitForPodCondition(c clientset.Interface, ns, podName, desc string, timeout time.Duration, condition podCondition) error { e2elog.Logf("Waiting up to %v for pod %q in namespace %q to be %q", timeout, podName, ns, desc) <ins> var lastPodError error </ins> for start := time.Now(); time.Since(start) < timeout; time.Sleep(poll) { pod, err := c.CoreV1().Pods(ns).Get(context.TODO(), podName, metav1.GetOptions{}) <ins> lastPodError = err </ins> if err != nil { if apierrors.IsNotFound(err) { e2elog.Logf("Pod %q in namespace %q not found. Error: %v", podName, ns, err) <del> return err </del> <ins> } else { e2elog.Logf("Get pod %q in namespace %q failed, ignoring for %v. Error: %v", podName, ns, poll, err) </ins> } <del> e2elog.Logf("Get pod %q in namespace %q failed, ignoring for %v. Error: %v", podName, ns, poll, err) </del> continue } // log now so that current pod info is reported before calling `condition()`

return err } } <ins> if apierrors.IsNotFound(lastPodError) { // return for compatbility with other functions testing for IsNotFound return lastPodError } </ins> return fmt.Errorf("Gave up after waiting %v for pod %q to be %q", timeout, podName, desc) }