Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --We have even encountered such case that pod continuously reports “Unable to attach or mount volumes for pod;...” when kubelet restarted. The reasons are listed in details as below: , as result of apiserver instantaneous unstablility, dswp get pvc‘s pv spec failed, then the pvc volume does not be to dsw. , syncStates() lists all the volume directories locally, and recontructs the volume from local volume, if the volume isn't in dsw, it will put the recontructed volume to asw, but now the volume.OuterVolumeSpecName isn't correct. chance, at this time, In , the fail to pvc volume is to dsw again normally in reconcile() -mountAttachVolumes(), it compares dsw and asw by volume's innerVolumename, it's all the same. but it could not sense the OuterVolumeSpecName dismatch. so it will not re-mount again to update asw cache , it checks pod's volume attach or mount success by comparing volume.OuterVolumeSpecName and , but this pv'c OuterVolumeSpecName isn't correct in asw. so pod continuously reports “Unable to attach or mount volumes for pod;...” reconcile() could sense the reconstruct volume's dismatch to the dsw, then re-mount again to make volume cache consistency Kubernetes version (use ): 1.19.4 Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig node
/triage accepted
/assign
CC
I think this is a point that can be optimized. /cc
I will give a fix for this
Thanks for reporting the issue, this innerSpecName and outerSpecName are confusing and cause issues. Please ping me for review when you have a fix. Thanks!
Please don't triage your own issues immediately, give someone else a chance to review them :( /sig storage
okay, I know

Discussion:
// or have a mount/unmount operation pending. GetAttachedVolumes() []AttachedVolume <del> // SyncReconstructedVolume check the volume.outerVolumeSpecName in asw and // the one populated from dsw, if they do not match, update this field from the value from dsw. SyncReconstructedVolume(volumeName v1.UniqueVolumeName, podName volumetypes.UniquePodName, outerVolumeSpecName string) </del> // Add the specified volume to ASW as uncertainly attached. AddAttachUncertainReconstructedVolume(volumeName v1.UniqueVolumeName, volumeSpec *volume.Spec, nodeName types.NodeName, devicePath string) error

return unmountedVolumes } <del> func (asw *actualStateOfWorld) SyncReconstructedVolume(volumeName v1.UniqueVolumeName, podName volumetypes.UniquePodName, outerVolumeSpecName string) { asw.Lock() defer asw.Unlock() if volumeObj, volumeExists := asw.attachedVolumes[volumeName]; volumeExists { if podObj, podExists := volumeObj.mountedPods[podName]; podExists { if podObj.outerVolumeSpecName != outerVolumeSpecName { podObj.outerVolumeSpecName = outerVolumeSpecName asw.attachedVolumes[volumeName].mountedPods[podName] = podObj } } } } </del> func (asw *actualStateOfWorld) newAttachedVolume( attachedVolume *attachedVolume) AttachedVolume { seLinuxMountContext := ""

func (rc *reconciler) Run(stopCh <-chan struct{}) { rc.reconstructVolumes() klog.InfoS("Reconciler: start to sync state") <del> wait.Until(rc.reconcileNew, rc.loopSleepDuration, stopCh) </del> <ins> wait.Until(rc.reconcile, rc.loopSleepDuration, stopCh) </ins> } <del> func (rc *reconciler) reconcileNew() { </del> <ins> func (rc *reconciler) reconcile() { </ins> readyToUnmount := rc.readyToUnmount() if readyToUnmount { // Unmounts are triggered before mounts so that a volume that was

reconciler.volumesNeedUpdateFromNodeStatus = append(reconciler.volumesNeedUpdateFromNodeStatus, volumeName1, volumeName2) // Act - run reconcile loop just once. // "volumesNeedUpdateFromNodeStatus" is not empty, so no unmount will be triggered. <del> reconciler.reconcileNew() </del> <ins> reconciler.reconcile() </ins> // Assert assert.True(t, reconciler.StatesHasBeenSynced())

if len(reconstructedVolumes) > 0 { // Add the volumes to ASW <del> rc.updateStatesNew(reconstructedVolumes) </del> <ins> rc.updateStates(reconstructedVolumes) </ins> // The reconstructed volumes are mounted, hence a previous kubelet must have already put it into node.status.volumesInUse. // Remember to update DSW with this information.

klog.V(2).InfoS("Volume reconstruction finished") } <del> func (rc *reconciler) updateStatesNew(reconstructedVolumes map[v1.UniqueVolumeName]*globalVolumeInfo) { </del> <ins> func (rc *reconciler) updateStates(reconstructedVolumes map[v1.UniqueVolumeName]*globalVolumeInfo) { </ins> for _, gvl := range reconstructedVolumes { err := rc.actualStateOfWorld.AddAttachUncertainReconstructedVolume( //TODO: the devicePath might not be correct for some volume plugins: see issue #54108

volumeSpec: volumeSpec, // volume.volumeSpecName is actually InnerVolumeSpecName. It will not be used // for volume cleanup. <del> // in case pod is added back to desired state, outerVolumeSpecName will be updated from dsw information. // See issue #103143 and its fix for details. </del> <ins> // in case reconciler calls mountOrAttachVolumes, outerVolumeSpecName will // be updated from dsw information in ASW.MarkVolumeAsMounted(). </ins> outerVolumeSpecName: volume.volumeSpecName, pod:                 pod, deviceMounter:       deviceMounter,

rcInstance.volumesNeedUpdateFromNodeStatus = nil // Act 2 - reconcile once <del> rcInstance.reconcileNew() </del> <ins> rcInstance.reconcile() </ins> // Assert 2 // MountDevice was attempted