Change:
ContainersToKill:  getKillMapWithInitContainers(basePod, baseStatus, []int{}), }, }, <ins> "Pod sandbox not ready, init container failed, but RestartPolicy == Never; kill pod only": { mutatePodFn: func(pod *v1.Pod) { pod.Spec.RestartPolicy = v1.RestartPolicyNever }, mutateStatusFn: func(status *kubecontainer.PodStatus) { status.SandboxStatuses[0].State = runtimeapi.PodSandboxState_SANDBOX_NOTREADY }, actions: podActions{ KillPod:           true, CreateSandbox:     false, SandboxID:         baseStatus.SandboxStatuses[0].Id, Attempt:           uint32(1), ContainersToStart: []int{}, ContainersToKill:  getKillMapWithInitContainers(basePod, baseStatus, []int{}), }, }, "Pod sandbox not ready, and RestartPolicy == Never, but no visible init containers;  create a new pod sandbox": { mutatePodFn: func(pod *v1.Pod) { pod.Spec.RestartPolicy = v1.RestartPolicyNever }, mutateStatusFn: func(status *kubecontainer.PodStatus) { status.SandboxStatuses[0].State = runtimeapi.PodSandboxState_SANDBOX_NOTREADY status.ContainerStatuses = []*kubecontainer.ContainerStatus{} }, actions: podActions{ KillPod:                  true, CreateSandbox:            true, SandboxID:                baseStatus.SandboxStatuses[0].Id, Attempt:                  uint32(1), NextInitContainerToStart: &basePod.Spec.InitContainers[0], ContainersToStart:        []int{}, ContainersToKill:         getKillMapWithInitContainers(basePod, baseStatus, []int{}), }, }, </ins> } { pod, status := makeBasePodAndStatusWithInitContainers() if test.mutatePodFn != nil {

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: If a pod's restart policy is "Never", and kubelet failed to create the PodSandbox (for whatever reason) once, the pod will be stuck at "ContainerCreating" forever. What you expected to happen: Since the user/application containers were never created, kubelet should retry the PodSandbox creation. The fact that the pod remains in a ContainerCreating state while kubelet explicitly does nothing to resolve the situation also confuses the user. How to reproduce it (as minimally and precisely as possible): I don't have an easy way to trigger a podsandbox creation failure off the top of my head, but this should be doable by messing with the CNI binaries/configurations. Anything else we need to know?: Relevant code snippet: Environment: Kubernetes version (use ): Only checked the master branch (1.16), but all supported release branches should have the issue Cloud provider or hardware configuration: not relevant OS (e.g: ): not relevant Kernel (e.g. ): not relevant Install tools: Network plugin and version (if this is a network-related bug): not relevant; though the initial PodSandbox creation failure could be caused by networking issues. Others:
thanks for posting! I'm looking at the code, and it seems like one fix could be checking if this pod has an associated sandbox - if it does, we want to respect the current behavior. If it doesn't, we want to attempt to create the sandbox. I was thinking something similar to: Thoughts? Is there a simplex fix?
Hi! Thank you for filing this - it matches the behaviour we've been seeing. Am I correct that this is a regression from (), and so only affects Kubernetes =1.13 ?
Your solution may not work for the one specific failure I checked, where the PodSandbox container was created, but call to the CNI plugin failed (for some reason; could be caused by a race condition). This left us a not-ready PodSandbox with no IP. Probably have to actually determine whether the user containers were created before. Need to check the code closer though.
Thanks. That's probably correct. I saw this in happening in 1.13
Gotcha - thanks! Taking a look at your pr now :)