Repository: kubernetes
Issue: Pod volume file system is corrupted, causing the program to get stuck when accessing files, leading to program freeze
Labels: kind/bug,sig/storage,sig/node,priority/important-longterm,lifecycle/stale,needs-triage

### What happened?

The mounted file system of the pod is corrupted, causing the program to get stuck when accessing files. The program gets stuck while HandlePodCleanups method is polling for requests from OrphanedPodCgroups, and during the podVolumesExist method, there is a call to getMountedVolumePathListFromDisk. The program may get stuck at the line 'isNotMount, err = kl.mounter.IsLikelyNotMountPoint(volumePath).


https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kubelet_getters.go#L363


### What did you expect to happen?


https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kubelet_getters.go#L363


```
func (kl *Kubelet) IsLikelyNotMountPoint(ctx context.Context, volumePath string)  (bool, error) {
	select {
	case <-ctx.Done():
		klog.Errorf("filesystem %s may be corrupted", volumePath)
		return false, fmt.Errorf("filesystem %s may be corrupted", volumePath)
	default:
		// 使用上下文调用访问磁盘的函数
		return kl.mounter.IsLikelyNotMountPoint(volumePath)
	}
}
	
func (kl *Kubelet) getMountedVolumePathListFromDisk(podUID types.UID) ([]string, error) {
	mountedVolumes := []string{}
	volumePaths, err := kl.getPodVolumePathListFromDisk(podUID)
	if err != nil {
		return mountedVolumes, err
	}
	// Only use IsLikelyNotMountPoint to check might not cover all cases. For CSI volumes that
	// either: 1) don't mount or 2) bind mount in the rootfs, the mount check will not work as expected.
	// We plan to remove this mountpoint check as a condition before deleting pods since it is
	// not reliable and the condition might be different for different types of volumes. But it requires
	// a reliable way to clean up unused volume dir to avoid problems during pod deletion. See discussion in issue #74650
	// 5s，may be 30s？
        ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
	defer cancel()

	for _, volumePath := range volumePaths {
		isNotMount, err := kl.IsLikelyNotMountPoint(ctx,  volumePath)
		if err != nil {
			return mountedVolumes, fmt.Errorf("fail to check mount point %q: %v", volumePath, err)
		}
		if !isNotMount {
			mountedVolumes = append(mountedVolumes, volumePath)
		}
	}
	return mountedVolumes, nil
}	
```
	

### How can we reproduce it (as minimally and precisely as possible)?

1. pod volumes File system exception
2. If the program gets stuck while accessing a file or directory

https://github.com/kubernetes/kubernetes/pull/119530/files

### Anything else we need to know?

_No response_

### Kubernetes version

<details>
master：https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kubelet_getters.go#L363
console          
   

k8s version: master

```
func (kl *Kubelet) getMountedVolumePathListFromDisk(podUID types.UID) ([]string, error) {
	mountedVolumes := []string{}
	volumePaths, err := kl.getPodVolumePathListFromDisk(podUID)
	if err != nil {
		return mountedVolumes, err
	}
	// Only use IsLikelyNotMountPoint to check might not cover all cases. For CSI volumes that
	// either: 1) don't mount or 2) bind mount in the rootfs, the mount check will not work as expected.
	// We plan to remove this mountpoint check as a condition before deleting pods since it is
	// not reliable and the condition might be different for different types of volumes. But it requires
	// a reliable way to clean up unused volume dir to avoid problems during pod deletion. See discussion in issue #74650
	for _, volumePath := range volumePaths {
		isNotMount, err := kl.mounter.IsLikelyNotMountPoint(volumePath)
		if err != nil {
			return mountedVolumes, fmt.Errorf("fail to check mount point %q: %v", volumePath, err)
		}
		if !isNotMount {
			mountedVolumes = append(mountedVolumes, volumePath)
		}
	}
	return mountedVolumes, nil
}
```

</details>


### Cloud provider

<details>

</details>


### OS version

<details>

```console
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here
```

</details>


### Install tools

<details>

</details>


### Container runtime (CRI) and version (if applicable)

<details>

</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>

</details>