Repository: kubernetes
Issue: PDB views pods in Terminating state as available
Labels: kind/bug,priority/important-longterm,wg/policy,triage/accepted

### What happened?

1. I have a statefulset with 2 pods. The grace period for these pods is long, so the terminating state can endure over hours, but that's not important.
2. I have a basic PDB of minAvailable: 1
3. I have a service that distributes across both pods.
4. I rollout restart the sts, which first restarts pod-1, then tries to restart pod-0.
5. While pod-0 is still in status "Terminating", I issue a second rollout restart to the sts.
6. K8s restarts pod-1 again, while pod-0 is still Terminating from the first restart.
7. My service is effectively down, as my service returns 503 Service Unavailable, because all pods are Terminating.

### What did you expect to happen?

6. K8s waits for pod-0 to be restarted and ready again before continuing to restart pod-1
7. My service stays available, as there is always at least one pod that is ready AND Running, which is what I expect from the PDB

### How can we reproduce it (as minimally and precisely as possible)?

```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: pdb-fail
  name: pdb-fail
spec:
  podManagementPolicy: Parallel # Without this, the STS stays available regardless of whether there is a PDB in place or not
  replicas: 2
  selector:
    matchLabels:
      app: pdb-fail
  serviceName: pdb-fail
  template:
    metadata:
      labels:
        app: pdb-fail
      name: pdb-fail
    spec:
      containers:
        - name: pdb-fail
          image: nginx
          lifecycle:
            preStop:
              exec:
                command:
                  - sleep
                  - "15"
      terminationGracePeriodSeconds: 1000

---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: pdb-fail
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: pdb-fail

```
After successful deployment of the sts, you can use the following script or run these manually to force both pods into "Terminating" state:
```bash
kubectl get pod,sts,pdb
kubectl rollout restart sts pdb-fail
kubectl get pod,sts,pdb
sleep 25
kubectl get pod,sts,pdb
kubectl rollout restart sts pdb-fail
kubectl get pod,sts,pdb
echo Tada
```

Output looks as follows:
```text
$ bash trigger-issue.sh
NAME             READY   STATUS    RESTARTS   AGE
pod/pdb-fail-0   1/1     Running   0          63s
pod/pdb-fail-1   1/1     Running   0          82s

NAME                        READY   AGE
statefulset.apps/pdb-fail   2/2     5m3s

NAME                                  MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
poddisruptionbudget.policy/pdb-fail   N/A             1                 1                     5m3s
statefulset.apps/pdb-fail restarted
NAME             READY   STATUS        RESTARTS   AGE
pod/pdb-fail-0   1/1     Running       0          63s
pod/pdb-fail-1   1/1     Terminating   0          82s

NAME                        READY   AGE
statefulset.apps/pdb-fail   2/2     5m3s

NAME                                  MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
poddisruptionbudget.policy/pdb-fail   N/A             1                 0                     5m3s
NAME             READY   STATUS        RESTARTS   AGE
pod/pdb-fail-0   1/1     Terminating   0          84s
pod/pdb-fail-1   1/1     Running       0          5s

NAME                        READY   AGE
statefulset.apps/pdb-fail   2/2     5m24s

NAME                                  MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
poddisruptionbudget.policy/pdb-fail   N/A             1                 0                     5m24s
statefulset.apps/pdb-fail restarted
NAME             READY   STATUS        RESTARTS   AGE
pod/pdb-fail-0   1/1     Terminating   0          84s
pod/pdb-fail-1   1/1     Terminating   0          5s

NAME                        READY   AGE
statefulset.apps/pdb-fail   2/2     5m24s

NAME                                  MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
poddisruptionbudget.policy/pdb-fail   N/A             1                 0                     5m24s
Tada
```
### Anything else we need to know?

I don't want to allow my service to route traffic to terminating pods, though I know that's an option.
I want the PDB to view terminating pods as unavailable, just like the default behavior of a service.

### Kubernetes version

<details>

```console
$ kubectl version
Client Version: v1.29.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.3
```

</details>


### Cloud provider

<details>
Azure Kubernetes Service
</details>


### OS version

<details>

```console
# On Linux (In WSL):
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
$ uname -a
Linux DESKTOP-J27URRA 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

# On Windows (Host for WSL):
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
BuildNumber  Caption                          OSArchitecture  Version
22631        Microsoft Windows 11 Enterprise  64-bit          10.0.22631
```

</details>


### Install tools

<details>

</details>


### Container runtime (CRI) and version (if applicable)

<details>

</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>

</details>