Change:
work not well. Not sure how to fix it. This issue has spreaded to the currently. <img width="550" alt="image" src=""
/triage accepted
/language en
For a likely fix, see

Discussion:
container runtime's management service is restarted while waiting for processes to terminate, the cluster retries from the start including the full original grace period. <del> An example flow: </del> <ins> Pod termination flow, illustrated with an example: </ins> 1. You use the `kubectl` tool to manually delete a specific Pod, with the default grace period (30 seconds).

If the `preStop` hook is still running after the grace period expires, the kubelet requests a small, one-off grace period extension of 2 seconds. <del> {{< note >}} If the `preStop` hook needs longer to complete than the default grace period allows, you must modify `terminationGracePeriodSeconds` to suit this. {{< /note >}} </del> <ins> {{% note %}} If the `preStop` hook needs longer to complete than the default grace period allows, you must modify `terminationGracePeriodSeconds` to suit this. {{% /note %}} </ins> 1. The kubelet triggers the container runtime to send a TERM signal to process 1 inside each container. <del> {{< note >}} The containers in the Pod receive the TERM signal at different times and in an arbitrary order. If the order of shutdowns matters, consider using a `preStop` hook to synchronize. {{< /note >}} </del> <ins> There is [special ordering](#termination-with-sidecars) if the Pod has any {{< glossary_tooltip text="sidecar containers" term_id="sidecar-container" >}} defined. Otherwise, the containers in the Pod receive the TERM signal at different times and in an arbitrary order. If the order of shutdowns matters, consider using a `preStop` hook to synchronize (or switch to using sidecar containers). </ins> 1. At the same time as the kubelet is starting graceful shutdown of the Pod, the control plane evaluates whether to remove that shutting-down Pod from EndpointSlice (and Endpoints) objects,

condition `serving`.  You can find more details on how to implement connections draining in the tutorial [Pods And Endpoints Termination Flow](/docs/tutorials/services/pods-and-endpoint-termination-flow/) <del> {{<note>}} If you don't have the `EndpointSliceTerminatingCondition` feature gate enabled in your cluster (the gate is on by default from Kubernetes 1.22, and locked to default in 1.26), then the Kubernetes control plane removes a Pod from any relevant EndpointSlices as soon as the Pod's termination grace period _begins_. The behavior above is described when the feature gate `EndpointSliceTerminatingCondition` is enabled. {{</note>}} {{<note>}} Beginning with Kubernetes 1.29, if your Pod includes one or more sidecar containers (init containers with an Always restart policy), the kubelet will delay sending the TERM signal to these sidecar containers until the last main container has fully terminated. The sidecar containers will be terminated in the reverse order they are defined in the Pod spec. This ensures that sidecar containers continue serving the other containers in the Pod until they are no longer needed. Note that slow termination of a main container will also delay the termination of the sidecar containers. If the grace period expires before the termination process is complete, the Pod may enter emergency termination. In this case, all remaining containers in the Pod will be terminated simultaneously with a short grace period. </del> <ins> <a id="pod-termination-beyond-grace-period" /> </ins> <del> Similarly, if the Pod has a preStop hook that exceeds the termination grace period, emergency termination may occur. In general, if you have used preStop hooks to control the termination order without sidecar containers, you can now remove them and allow the kubelet to manage sidecar termination automatically. {{</note>}} </del> <ins> 1. The kubelet ensures the Pod is shut down and terminated 1. When the grace period expires, if there is still any container running in the Pod, the kubelet triggers forcible shutdown. The container runtime sends `SIGKILL` to any processes still running in any container in the Pod. The kubelet also cleans up a hidden `pause` container if that container runtime uses one. 1. The kubelet transitions the Pod into a terminal phase (`Failed` or `Succeeded` depending on the end state of its containers). 1. The kubelet triggers forcible removal of the Pod object from the API server, by setting grace period to 0 (immediate deletion). 1. The API server deletes the Pod's API object, which is then no longer visible from any client. </ins> <del> 1. When the grace period expires, the kubelet triggers forcible shutdown. The container runtime sends `SIGKILL` to any processes still running in any container in the Pod. The kubelet also cleans up a hidden `pause` container if that container runtime uses one. 1. The kubelet transitions the Pod into a terminal phase (`Failed` or `Succeeded` depending on the end state of its containers). This step is guaranteed since version 1.27. 1. The kubelet triggers forcible removal of Pod object from the API server, by setting grace period to 0 (immediate deletion). 1. The API server deletes the Pod's API object, which is then no longer visible from any client. </del> ### Forced Pod termination {#pod-termination-forced}

server. If the Pod was still running on a node, that forcible deletion triggers the kubelet to begin immediate cleanup. <del> {{< note >}} You must specify an additional flag `--force` along with `--grace-period=0` </del> <ins> Using kubectl, You must specify an additional flag `--force` along with `--grace-period=0` </ins> in order to perform force deletions. <del> {{< /note >}} </del> When a force deletion is performed, the API server does not wait for confirmation from the kubelet that the Pod has been terminated on the node it was running on. It

documentation for [deleting Pods from a StatefulSet](/docs/tasks/run-application/force-delete-stateful-set-pod/). <ins> ### Pod shutdown and sidecar containers {##termination-with-sidecars} If your Pod includes one or more [sidecar containers](/docs/concepts/workloads/pods/sidecar-containers/) (init containers with an Always restart policy), the kubelet will delay sending the TERM signal to these sidecar containers until the last main container has fully terminated. The sidecar containers will be terminated in the reverse order they are defined in the Pod spec. This ensures that sidecar containers continue serving the other containers in the Pod until they are no longer needed. This means that slow termination of a main container will also delay the termination of the sidecar containers. If the grace period expires before the termination process is complete, the Pod may enter [forced termination](#pod-termination-beyond-grace-period). In this case, all remaining containers in the Pod will be terminated simultaneously with a short grace period. Similarly, if the Pod has a `preStop` hook that exceeds the termination grace period, emergency termination may occur. In general, if you have used `preStop` hooks to control the termination order without sidecar containers, you can now remove them and allow the kubelet to manage sidecar termination automatically. </ins> ### Garbage collection of Pods {#pod-garbage-collection} For failed Pods, the API objects remain in the cluster's API until a human or