Code change:
} } <ins> func TestAttacherWaitForAttach(t *testing.T) { tests := []struct { name             string driver           string makeAttachment   func() *storage.VolumeAttachment expectedAttachID string expectError      bool }{ { name:   "successful attach", driver: "attachable", makeAttachment: func() *storage.VolumeAttachment { testAttachID := getAttachmentName("test-vol", "attachable", "node") successfulAttachment := makeTestAttachment(testAttachID, "node", "test-pv") successfulAttachment.Status.Attached = true return successfulAttachment }, expectedAttachID: getAttachmentName("test-vol", "attachable", "node"), expectError:      false, }, { name:        "failed attach", driver:      "attachable", expectError: true, }, } for _, test := range tests { t.Run(test.name, func(t *testing.T) { plug, _, tmpDir, _ := newTestWatchPlugin(t, nil) defer os.RemoveAll(tmpDir) attacher, err := plug.NewAttacher() if err != nil { t.Fatalf("failed to create new attacher: %v", err) } csiAttacher := attacher.(*csiAttacher) spec := volume.NewSpecFromPersistentVolume(makeTestPV("test-pv", 10, test.driver, "test-vol"), false) if test.makeAttachment != nil { attachment := test.makeAttachment() _, err = csiAttacher.k8s.StorageV1beta1().VolumeAttachments().Create(attachment) if err != nil { t.Fatalf("failed to create VolumeAttachment: %v", err) } gotAttachment, err := csiAttacher.k8s.StorageV1beta1().VolumeAttachments().Get(attachment.Name, meta.GetOptions{}) if err != nil { t.Fatalf("failed to get created VolumeAttachment: %v", err) } t.Logf("created test VolumeAttachment %+v", gotAttachment) } attachID, err := csiAttacher.WaitForAttach(spec, "", nil, time.Second) if err != nil && !test.expectError { t.Errorf("Unexpected error: %s", err) } if err == nil && test.expectError { t.Errorf("Expected error, got none") } if attachID != test.expectedAttachID { t.Errorf("Expected attachID %q, got %q", test.expectedAttachID, attachID) } }) } } </ins> func TestAttacherWaitForVolumeAttachment(t *testing.T) { nodeName := "test-node" testCases := []struct {

Related discussion:
text: Could there be a race with AD controller or external attacher if they try to detach the volume? is another race around this area. Unclear if it's the same yet.
Michelle Au <: No idea. Even if it isn't, it's at least related to the same piece of code. Probably worth fixing both together. Can we get someone who has worked in this area to fix these issues? Today is the first day that I've ever seen this code, so I'm a bit afraid that it'll take me a long time to get it right, and I'm worried about regressions.
I've asked to help investigate
It is not a blocker for beta or freeze. Can be addressed in stabilization. Let's keep the 1.13 tag on it. And based on progress by stabilization we can reassess if it is even a 1.13 blocker (at this point I don't believe it is).
The in the actual state of the world gets reset during unmounting:
That removes the device path looks wrong to me. I've submitted PR which changes that. Let's see how much that breaks...
/assign The conclusion in the CSI working group was that while my analysis and the resulting fix might be correct, further debugging and a second pair of eyes would be good. Let me reassign, because I have done what I could for this issue.
The RT considers this a blocker now, because flakes are happening often enough to interfere with getting clear signal on several jobs. Even if the underlying problem isn't that serious.
/assign I will help triage as well.
Ack Thanks
Instead of which may have negative side effects for non-CSI volumes, let's do this instead: Modify the CSI method in to ignore the passed in (which is populated from devicePath. Instead let's have that method generate the from scratch, e.g. attachID := getAttachmentName(volID, driverName, string(nodeName)) This will remove any CSI dependency on devicePath and not effect other volume plugins.
I am trying to twist my head around the code. IMO it affects also AWS that uses devicePath stored in and it should be reproducible there too. So far I was unsuccessful. In the logs it's clearly visible that UnmountDevice succeeded. So why kubelet continues with MountVolume instead of MountDevice? That's wrong.