Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Let's say, I have a following sample static pod yaml, <details<summaryClick to expand pod yaml</summary</detailsI want to simulate the pod update, so we will change the container image tag from to <details<summaryClick to expand modified pod yaml</summary</detailsWhen I replace the original pod yaml with the modified pod yaml in , the original pod gets terminated and is replaced with the modified pod. But when the modified pod is coming up I see following errors in This is happening because the original pod has set to 60 seconds. So the original pod stays up for 60 seconds while the replace pod is also brought up causing the errors we see above. Ideally, the modified pod should not come up until the original pod is completely terminated. This behaviour is observed only with static pods. If I use the same pod yamls above to provision and update a regular pod I don't see any errors. <details<summaryClick to see behaviour with regular pods </summary</detailsWhat you expected to happen: The static pod should be updated by respecting How to reproduce it (as minimally and precisely as possible): Take the original pod yaml and create the static pod. Then take the modified pod yaml to simulate the update. Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig node
May I try this as my first contribution? As a result of the test so far, static HTTP Pod also has the same issue.
I can't reproduce this phenomenon with the v1.20.0 version. What is your version? I think you should be able to remove the ?
Sure, thanks. If you find anything useful to fix this, please feel free to share it here. Yes, static HTTP pods will have the same issue.
20.0 Yes, but this is just a sample workload to demonstrate that while updating the static pods do not honor .
While going over following kubelet logs, notice that the mirror pod deletion doesn't respect . The mirror pod is immediately deleted. There was this PR by that should have prevented this from happening. Looking into why the is not respected while deleting the mirror pod.
/cc
/triage accepted
/assign
I've been digging for a few days and find the key causes. <delIf the contents of the static file or HTTP response change, the hash linking the static pod and its mirror pod is changed. As a result, the unlinked mirror pod is deleted not gracefully.</delDeleting a mirror pod when changing a static pod is the intended behavior. File-specific? Depending on your file editor program(vim in my case), may make the backup file("[filepath]~"). Then, the hash changed because the filename is used to make the hash. Therefore, the mirror pod is deleted not gracefully. some reason, when updating the file, it may rename your file, and then the pod is gracefully deleted but an error will occur because the pod was deleted not updated(rename file cause the pod deletion). I think cause has to be fixed by distinguishing the hash for identifying the static pod(uid hash) and the hash for being used to check if the internal spec differs. However, cause and cause are outside of the kubernetes, so I'm not sure if the fix is needed. To fix cause /assign
thanks
Reproduced at master, release-1.22. The current source doesn't respect terminationGracePeriodSeconds of static pods. And even worse, if we update a static pod twice before the grace period seconds are running out, the pod is stuck in the ERROR status forever or is not created forever. (Maybe there is a deadlock) /open
Looking for a workaround. /reopen
Reopened this issue. <detailsIn response to : Looking for a workaround. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale PR:

Discussion:
return nil } <ins> // If the pod is a static pod and its mirror pod is still gracefully terminating, // we do not want to start the new static pod until the old static pod is gracefully terminated. podFullName := kubecontainer.GetPodFullName(pod) if kl.podKiller.IsMirrorPodPendingTerminationByPodName(podFullName) { return fmt.Errorf("pod %q is pending termination", podFullName) } </ins> // Latency measurements for the main workflow are relative to the // first time the pod was seen by the API server. var firstSeenTime time.Time

// Create Mirror Pod for Static Pod if it doesn't already exist if kubetypes.IsStaticPod(pod) { <del> podFullName := kubecontainer.GetPodFullName(pod) </del> deleted := false if mirrorPod != nil { if mirrorPod.DeletionTimestamp != nil || !kl.podManager.IsMirrorPodOf(mirrorPod, pod) {

} podPair := kubecontainer.PodPair{APIPod: pod, RunningPod: &runningPod} <ins> if _, ok := kl.podManager.GetMirrorPodByPod(pod); ok { kl.podKiller.MarkMirrorPodPendingTermination(pod) } </ins> kl.podKiller.KillPod(&podPair) // We leave the volume/directory cleanup to the periodic cleanup routine.

kl.handleMirrorPod(pod, start) continue } <del> if _, ok := kl.podManager.GetMirrorPodByPod(pod); ok { kl.podKiller.MarkMirrorPodPendingTermination(pod) } </del> // Deletion is allowed to fail because the periodic cleanup routine // will trigger deletion again. if err := kl.deletePod(pod); err != nil {