Change:
if err != nil { // Skip detaching this volume if unable to update node status klog.ErrorS(err, "UpdateNodeStatusForNode failed while attempting to report volume as attached", "volume", attachedVolume) <ins> // Add volume back to ReportAsAttached if UpdateNodeStatusForNode call failed so that node status updater will add it back to VolumeAttached list. // It is needed here too because DetachVolume is not call actually and we keep the data consistency for every reconcile. rc.actualStateOfWorld.AddVolumeToReportAsAttached(attachedVolume.VolumeName, attachedVolume.NodeName) </ins> continue }

Discussion:
Recently, the cpu load of my kube-apiserver gets higher after I expanded more pods and nodes into the cluster. And I found that some volumes declared by pods could not be mounted after the restart of pods, even though the pods were successful scheduled. here's the kubectl describe information: After checking the logs of attachdetachcontroller, I found that in , return an error, thus the was skipped. That should be related to the high load on my kube-apiserver as it occasionally returns 500 http code (but this may not be the root cause). Here is the error message: Here comes the problem: In my opinion, the next loop of would retry and , since the state of actualStateOfWorld is different from the state of desiredStateOfWorld. but it didn't happened. By recording some additional logs, I notice that the pod was scheduled to the destination node by the scheduler right after above error message appeared, which cause executed. That would write the volume info back to . After that, the next loop of would skip the detach operation for that volume, since desiredStateOfWorld was 'equal to' actualStateOfWorld. The log snippet is as follows: Finally, attachdetachcontroller would not attach the volume neither, because the states of desiredStateOfWorld and actualStateOfWorld are the same. attachdetachcontroller mistakenly thought the volume was attached, but it' not the case. when fail, attachdetachcontroller should retry to detach the volume until the the detach operation completed. However, it was interrupted by the update of desiredStateOfWorld. a log and before a pod mounting pv Then you'll found log message "force skip detaching" would disappear after a few loops of No response <details</details<detailsN/A </detailsNo response <details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository.

</details
/sig storage
If fails, that means the volume has not been detached and the volumeattachment is still on the nodeStatus, then kubelet should see that the volume still attaches to the node, so I don't understand why your pod failed to mount, it may not be caused by the controller. what's the error for kubelet?
! There are . The one before was failed, but another which is at the end of was successfully executed. Thus it cause the kubelet coundn't get the volume in .
kubelet log repeat the message below infinitely:
Okay, I know what you said, actually this problem is similar with the one which was fixed by , but it is not completely. Give another pr below to fix this.
Can we have the fix () back-ported to 1.21? Given that this completes the , the initial fix for which was , it would make sense to have this follow-up fix also be to 1.21. Looks like there is a that we could get this into. We have a 100+ clusters deployed in production that are running 1.21. We see this bug at least once a week and it's a headache to deal with. Upgrading to 1.22 or higher is a big undertaking for us.