Change:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /priority failing-test What happened: https://k8s- please investigate ASAP, this is a release-blocking test group.
Umbrella issue -
From the test log: kube-controller- only has this single message:
cc
thanks for those logs. seems there's a race in the external kubernetes side appears to be working just fine, though I'm not 100% sure because unfortunately the pv controller isn't logging anything I1114 21:30:55. 7 ] POST /api/v1/persistentvolumes: (9.545259ms) 201 [[nfs-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format] 35.185.45.68:] provisioner creates PV I1114 21:30:55. 7 ] POST /api/v1/namespaces/e2e-tests-volume-provisioning-vtd74/events: (2.074502ms) 201 [[nfs-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format] 35.185.45.68:] provisioner posts ProvisioningSucceeded to PVC I1114 21:30:55. 7 ] DELETE /api/v1/persistentvolumes/pvc--c983-11e7-8959-: (3.533325ms) 200 [[nfs-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format] 35.185.45.68:] provisioner immediately deletes the PV?!?! why? the volume should be in Available or Pending or Bound state I1114 21:30:55. 7 ] PUT /api/v1/namespaces/e2e-tests-volume-provisioning-vtd74/persistentvolumeclaims/pvc-9brzx/status: (3.281342ms) 200 [[kube-controller-manager/v1.9.0 (linux/amd64) kubernetes//system:serviceaccount:kube-system:persistent-volume-binder] [::1]:] PV controller updates claim to I1114 21:30:55. 7 ] POST /api/v1/namespaces/e2e-tests-volume-provisioning-vtd74/events: (2.119586ms) 201 [[kube-controller-manager/v1.9.0 (linux/amd64) kubernetes//system:serviceaccount:kube-system:persistent-volume-binder] [::1]:] PV controller posts event Is there any way to get the pod logs? else I will need to modify the test to get the provisioner pod's logs to figure out why it would delete a PV right after creating it edit: actualyl the pod logs would not help... this condition seems pretty fool-proof:
I don't think pod logs are automatically collected.
ok, I see Here is my current theory for what happened, difficult to confirm without pv controller logs though PVC is created but not processed by PV controller and not to its cache yet PV is and processed by PV controller, this is possible since the PVC and PV queue are distinct (?) Instead of going here and setting PV to Bound, the PV controller goes here because the claim is not in the cache and presumed deleted, and sets PV to Released Provisioner deletes PV because it is Released PV controller finally processes PVC and sets it to Lost need your expertise. Solution maybe is for external provisioners to leave volume.Spec.ClaimRef.UID blank.
The PV controller is supposed to set the annotation to signal to external provisioners that they can start provisioning: Does the external provisioner library wait for that annotation? But I guess it's also possible that the user could create the PVC with the annotation already set.
Oh yeah you are right! The provisioner is supposed to wait for the annotation, but we do not wait atm, we simply fallthrough to checking claim.Class which is wrong will fix.
That should not happen, the controller set the provisioning annotation so it must be in its cache...
/priority critical-urgent this blocks
was this fixed on the external provisioner side?
Ping?
looks like the external provisioner code still hasn't been fixed. Do you plan on making that change soon?
fix will be done today. I will merge , update nfs-provisioner image, then update the test to point to the new image

Discussion:
Containers: []v1.Container{ { Name:  "nfs-provisioner", <del> Image: "quay.io/kubernetes_incubator/nfs-provisioner:v1.0.6", </del> <ins> Image: "quay.io/kubernetes_incubator/nfs-provisioner:v1.0.9", </ins> SecurityContext: &v1.SecurityContext{ Capabilities: &v1.Capabilities{ Add: []v1.Capability{"DAC_READ_SEARCH"},