Change:
/kind failing-test <!-- If this is a flake, please add: /kind flake --/sig node
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
<img width="1312" alt="image" src=""
This looks like a test you a few days ago: /assign Can you look into these failures?
GitHub didn't allow me to assign the following users: chengjoey. Note that only with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : This looks like a test you a few days ago: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
You can't reopen an issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
As mentioned in the previous comments, there should be only one expected state in the end. From the logs of these failed tests, after stopping the container, the phase changes to Failed. I think this does not meet expectations.
Thanks to for revert my previous PR. I think the pod final state should always be in one state. From the current e2e test results, the final state may be Succeed or Failed, seems pod.Status.Phase is empty. I think this seems to be wrong. I will study whether the node status (sufficient memory) and other factors interfere with the final state. Thanks again to for his help.
Add the log with phase "Succeed": The normal update status steps are 1-2-3-4, but the result of failure tests is that step 2 is executed last, which seems a bit strange. Because phase is empty
feel free to reopen the PR. But let's make sure to run the tests that this reported as failing before we merge. Thank you for your help on improving test coverage!
The links to the failed E2E tests in the overview of this issue all show the Pod phase as , but is there a link available somewhere for cases where it's empty string?
In fact, these failed tests are all empty strings, because not terminated: should be followed by phase Because my original test was written like this as I commented earlier, different order of updating status may lead to different final state
The Pod's final state is shown as in the dumped result immediately after the test fails. The referenced inside - is that coming from the Pod information fetched within ? Could it be that it's showing up as an empty string because it's not able to access the value externally?
In the tests submitted previously, and in my local(minnikube) tests, the pod status will change from to again see this log step 3 So I think the order of state changes affects the final result, and this is not very stable
I think this is a separate issue, so as I mentioned in , I thought it might be fine to wait for either the or phase. However, after looking at , it seems that we may need to resolve this unstable final result first. WDYT?
If we can explain different results, it is fine. I don't want unexplained check for either condition in our tests

Discussion:
return checkMirrorPodDisappear(ctx, f.ClientSet, pod.Name, pod.Namespace) }, f.Timeouts.PodDelete, f.Timeouts.Poll).Should(gomega.BeNil()) }) <del> // Regression test for an extended scenario for https://issues.k8s.io/123980 ginkgo.It("should evict running pods that do not meet the affinity after the kubelet restart", func(ctx context.Context) { nodeLabelKey := string(uuid.NewUUID()) nodeLabelValueRequired := string(uuid.NewUUID()) podName := "affinity-pod" + string(uuid.NewUUID()) nodeName := getNodeName(ctx, f) ginkgo.By(fmt.Sprintf("Adding node label for node (%s) to satisify pod (%s/%s) affinity", nodeName, f.Namespace.Name, podName)) e2enode.AddOrUpdateLabelOnNode(f.ClientSet, nodeName, nodeLabelKey, nodeLabelValueRequired) ginkgo.DeferCleanup(func() { e2enode.RemoveLabelOffNode(f.ClientSet, nodeName, nodeLabelKey) }) pod := e2epod.MustMixinRestrictedPodSecurity(&v1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name:      podName, Namespace: f.Namespace.Name, }, Spec: v1.PodSpec{ Affinity: &v1.Affinity{ NodeAffinity: &v1.NodeAffinity{ RequiredDuringSchedulingIgnoredDuringExecution: &v1.NodeSelector{ NodeSelectorTerms: []v1.NodeSelectorTerm{ { MatchExpressions: []v1.NodeSelectorRequirement{ { Key:      nodeLabelKey, Operator: v1.NodeSelectorOpIn, Values:   []string{nodeLabelValueRequired}, }, }, }, }, }, }, }, Containers: []v1.Container{ { Name:  podName, Image: imageutils.GetPauseImageName(), }, }, }, }) // Create the pod bound to the node. It will start, but will be rejected after kubelet restart. ginkgo.By(fmt.Sprintf("Creating a pod (%s/%s)", f.Namespace.Name, podName)) e2epod.NewPodClient(f).Create(ctx, pod) ginkgo.By(fmt.Sprintf("Waiting for the pod (%s/%s) to be running", f.Namespace.Name, pod.Name)) err := e2epod.WaitForPodNameRunningInNamespace(ctx, f.ClientSet, pod.Name, f.Namespace.Name) framework.ExpectNoError(err, "Failed to await for the pod to be running: (%v/%v)", f.Namespace.Name, pod.Name) // Remove node label e2enode.RemoveLabelOffNode(f.ClientSet, nodeName, nodeLabelKey) ginkgo.By("Restart the kubelet") restartKubelet(true) gomega.Eventually(ctx, func() bool { return kubeletHealthCheck(kubeletHealthCheckURL) }, recoverTimeout, f.Timeouts.Poll). Should(gomega.BeTrueBecause("kubelet should be healthy after restart")) // Pod should be terminated, maybe not immediately, should allow a few seconds for the kubelet to kill the pod // after kubelet restart, pod admission denied, kubelet will reject the pod and kill container. gomega.Eventually(ctx, func() bool { pod, err = e2epod.NewPodClient(f).Get(ctx, podName, metav1.GetOptions{}) framework.ExpectNoError(err) // pod is in a final state, the following are the behaviors of pods after kubelet restarted: // 1. kubelet `canAdmitPod` reject pod by reason Pod admission denied by nodeAffinity // 2. kubelet stop/kill container // the final state of the pod is related to the kill container. // if an error occurs in the preStop of the container or the exitCode is not 0, // the phase is PodFailed. if the exitCode of the StopContainer is 0, the phase is PodSucceeded. // in this case, stop and kill container is successful(exitCode 0), the pod phase should be PodSucceeded. return pod.Status.Phase == v1.PodSucceeded }, recoverTimeout, f.Timeouts.Poll).Should(gomega.BeTrueBecause("Pod %s not terminated: %s", pod.Name, pod.Status.Phase)) }) </del> }) })