Repository: kubernetes
Issue: No enough disk space in the os disk but there is no alert in node's DiskPressure     
Labels: kind/bug,priority/important-soon,sig/windows,help wanted,lifecycle/frozen,needs-triage

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!

If the matter is security related, please disclose it privately via https://kubernetes.io/security/
-->


#### What happened:
No enough disk space in the os disk but there is no alert in node's DiskPressure     

```
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  DiskPressure     False   Mon, 21 Jun 2021 11:37:50 +0800   Fri, 18 Jun 2021 15:29:20 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
```

#### What you expected to happen:
Node is not ready since there is no enough disk space to host new pods.

#### How to reproduce it (as minimally and precisely as possible):
Generate large files in the OS disk in the Windows node.
```
for($i=0; $i -lt 70; $i++) {
 Write-Host "Create file $i"
 $out = new-object byte[] 2000048576; (new-Object Random).NextBytes($out);[IO.File]::WriteAllBytes("c:\,ypowerfile51-$i.txt", $out)
}
```

#### Anything else we need to know?:

#### Environment:
- Kubernetes version (use `kubectl version`): 1.19.11
- Cloud provider or hardware configuration: Azure
- OS (e.g: `cat /etc/os-release`): Windows Server 2019
- Kernel (e.g. `uname -a`): 10.0.17763.1971
- Install tools: AKS
- Network plugin and version (if this is a network-related bug): Azure CNI
- Others:

Kubelet.err.log:
```
I0618 07:15:12.603468    4976 image_gc_manager.go:304] [imageGCManager]: Disk usage on image filesystem is at 94% which is over the high threshold (85%). Trying to free 18956696780 bytes down to the low threshold (80%).
I0618 07:15:12.623447    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:ad675c9cb2d58c2b42b387143a182a79b3dd9506e7efd2b5cab2c6d2ca77f904" to free 252024907 bytes
I0618 07:15:15.710317    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:f0d83b4dfa18444e8cfc593185eb6aaf5e738b53bb7ca2e56db2d1cefa5396e2" to free 294765124 bytes
I0618 07:15:16.030449    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:5da9b97692475de2cc717b91b6badc2ba5fce1ef4a992ec6c3a0d9987f4d1b95" to free 302053528 bytes
I0618 07:15:16.420524    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:f401c064d0b828ba247c8dc3ba0cb396a82001d0a66a17cbc8830195639d8b87" to free 304232501 bytes
I0618 07:15:19.581151    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:6db76cecfbae97dda7d7f020f2e9f82882a085aaf0a7ed3b046812d4367d08a4" to free 295492277 bytes
I0618 07:15:19.933694    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:69dea4a0249e21c4031b88ad664a15306253bf45b9ee68efeb89d917046c2b1b" to free 299225156 bytes
I0618 07:15:20.433381    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:cc6fa6a61f3147f7860268c50662a62cf55ec93e6de6ba72cc8d06bb5441ccbf" to free 290941764 bytes
I0618 07:15:20.928340    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:eaad28baf4b2844d3909fdf323f6e843c683cf2b5e3c967b388ed086c3eb553c" to free 270003827 bytes
I0618 07:15:21.572742    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:3a7f23e29bd731305680773c5818276bb888cb024f80068923886cfa1e328b0a" to free 5277076693 bytes
I0618 07:17:01.366118    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:f8d6ba053e57ba4d65ed6b8ceccdb732202dc5630d6bf406cc41588219d52b18" to free 294577220 bytes
I0618 07:17:01.818293    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:84162105ffe3375bb0493c4026d85adeb16a658c3551310a8d5b4e4983ab40d9" to free 268728368 bytes
I0618 07:17:05.825079    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:bc1b7c74446d06b36da6bedd570563aca939465edfaebbad43dcf30102dc95fe" to free 509345724 bytes
I0618 07:17:08.685489    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:06cd0b8cb7ee21e578585d489fe04718ef75496debf3966f278d9acb155421c6" to free 308642348 bytes
I0618 07:17:12.288085    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:ca273de9d00e285648a7aaf9ea70cd9aec841f27cb14fc705df7329be0507c6d" to free 301778868 bytes
I0618 07:17:12.727789    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:978d75aa605732dd92a7cacc3dd8fa6087e7852b9e9843abccc666d0fcf9f5e2" to free 268076147 bytes
I0618 07:17:35.727837    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:077fe47c8702ab4f09ae9fbe6d11777dcdf7d0b111b9edf406c2c3c0c8f77058" to free 267426993 bytes
I0618 07:17:39.336893    4976 image_gc_manager.go:375] [imageGCManager]: Removing image "sha256:5e6dfdfe4bea95d6b03f6661635c1e7712fb674fcf65d2cdbdb0fd171a4f9629" to free 292636341 bytes
E0618 07:17:43.563796    4976 kubelet.go:1238] Image garbage collection failed once. Stats initialization may not have completed yet: failed to garbage collect required amount of images. Wanted to free 18956696780 bytes, but freed 10097027786 bytes
I0618 07:22:43.567803    4976 image_gc_manager.go:304] [imageGCManager]: Disk usage on image filesystem is at 97% which is over the high threshold (85%). Trying to free 22999923916 bytes down to the low threshold (80%).
E0618 07:22:43.589792    4976 kubelet.go:1234] Image garbage collection failed multiple times in a row: failed to garbage collect required amount of images. Wanted to free 22999923916 bytes, but freed 0 bytes
time="2021-06-18T07:23:21Z" level=error msg="failed to waitForNotification: callbackNumber does not exist in callbackMap" callbackNumber=3667 spanID=b5bbeaa83b8e4033 traceID=86c32c2e83114b918e22244bd36a6565
```


Disk usage in drive C: in the Windows node:
```
Size(GB)       : 127.509761810303
Free Space(GB) : 1.27705383300781
Free (%)       :     1%
DeviceID       : C:
DriveType      : 3
```

Node description:
```
Name:               aksnpwin000000
Roles:              agent
Labels:             agentpool=npwin
                    beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/instance-type=Standard_D2s_v3
                    beta.kubernetes.io/os=windows
                    failure-domain.beta.kubernetes.io/region=westus2
                    failure-domain.beta.kubernetes.io/zone=0
                    kubernetes.azure.com/cluster=MC_abeltestdisk_k8s_westus2
                    kubernetes.azure.com/node-image-version=AKSWindows-2019-17763.1971.210526
                    kubernetes.azure.com/role=agent
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=aksnpwin000000
                    kubernetes.io/os=windows
                    kubernetes.io/role=agent
                    node-role.kubernetes.io/agent=
                    node.kubernetes.io/instance-type=Standard_D2s_v3
                    node.kubernetes.io/windows-build=10.0.17763
                    storageprofile=managed
                    storagetier=Premium_LRS
                    topology.kubernetes.io/region=westus2
                    topology.kubernetes.io/zone=0
Annotations:        node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Fri, 18 Jun 2021 13:55:12 +0800
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  aksnpwin000000
  AcquireTime:     <unset>
  RenewTime:       Mon, 21 Jun 2021 11:40:16 +0800
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Mon, 21 Jun 2021 11:37:50 +0800   Fri, 18 Jun 2021 15:29:20 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Mon, 21 Jun 2021 11:37:50 +0800   Fri, 18 Jun 2021 15:29:20 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Mon, 21 Jun 2021 11:37:50 +0800   Fri, 18 Jun 2021 15:29:20 +0800   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Mon, 21 Jun 2021 11:37:50 +0800   Fri, 18 Jun 2021 15:29:20 +0800   KubeletReady                 kubelet is posting ready status
Addresses:
  Hostname:    aksnpwin000000
  InternalIP:  10.240.0.35
Capacity:
  attachable-volumes-azure-disk:  4
  cpu:                            2
  ephemeral-storage:              133703676Ki
  memory:                         8388148Ki
  pods:                           30
Allocatable:
  attachable-volumes-azure-disk:  4
  cpu:                            1900m
  ephemeral-storage:              133703676Ki
  memory:                         4403764Ki
  pods:                           30
System Info:
  Machine ID:                 aksnpwin000000
  System UUID:                19344145-27C1-49F1-BF7E-1F3AC0FCD96D
  Boot ID:
  Kernel Version:             10.0.17763.1971
  OS Image:                   Windows Server 2019 Datacenter
  Operating System:           windows
  Architecture:               amd64
  Container Runtime Version:  docker://19.3.14
  Kubelet Version:            v1.19.11
  Kube-Proxy Version:         v1.19.11
ProviderID:                   azure:///subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/mc_abeltestdisk_k8s_westus2/providers/Microsoft.Compute/virtualMachineScaleSets/aksnpwin/virtualMachines/0
Non-terminated Pods:          (2 in total)
  Namespace                   Name                       CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                       ------------  ----------  ---------------  -------------  ---
  default                     sample-545fb69bb6-nskpj    200m (10%)    200m (10%)  300M (6%)        300M (6%)      2d21h
  kube-system                 omsagent-win-x8qdn         200m (10%)    200m (10%)  600Mi (13%)      600Mi (13%)    2d21h
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource                       Requests         Limits
  --------                       --------         ------
  cpu                            400m (21%)       400m (21%)
  memory                         929145600 (20%)  929145600 (20%)
  ephemeral-storage              0 (0%)           0 (0%)
  attachable-volumes-azure-disk  0                0
Events:
  Type     Reason         Age                    From     Message
  ----     ------         ----                   ----     -------
  Warning  ImageGCFailed  46s (x809 over 2d19h)  kubelet  (combined from similar events): failed to garbage collect required amount of images. Wanted to free 26937920716 bytes, but freed 0 bytes
```