Change:
Deployment/Service manifest: logs from events: In aws, the volume is attached, On the box: In only one is mounted: Where can I debug this further?
Take a look at the on your master for events related to your volume . That will show you when the volume was attached/detached. It appears that your volume attached without issue so there should be nothing unusual here. Then take a look at on the node that the volume is supposed to be attached to. Looks for the UID of your pod (). I suspect one of your mounts is failing. It looks like you are using hyperkube so it might be a mount namespace propagation issue.
I believe this happens when the RC/Pod is being created shortly after the PV or PVC. The latter two appear to need several seconds to be fully available (even if the status is already set to "Bound"). I ran into the same issue with a NFS PV. When I delete and create PV+PVC+SVC+RC all in one go, the associated Pod will be in ContainerCreating state for several seconds (10,<60), then log the warning event and afterwards be created without any problems or further delays. When I first create PV+PVC, wait for several seconds (30) and only then create SVC+RC, the pod will be created immediately and no warning event will be logged. One oddity I noticed is that the PVC will have capacity 0 and empty access modes (as shown by ), even though it was setup to request ROX for 1Gi and the PV provides 1Gi with ROX. This might be a different bug, though.
Thanks for digging into it The capacity 0 issue () was fixed by PR and is in v1.3.5. This is more or less expected behavior. The mount logic has exponential backoff so if it rapidly encounters PVC/PV does not exist, it'll back off the retries resulting in a longer mount time. We can consider optimizing this particular scenario. But I'll lower the priority on this for now and remove the milestone.
Maybe the "Bound" status could be delayed, so the RC would be held back for a while and be scheduled only when the PVC is actually ready?
I don't think the RC actually waits for PVC to be bound before scheduling.
I have also this problem using deployment with HELM, I deploy all my pv,pvc,deploy,svc in one Charts, and my pod who requiere a volume from the PVC get scheduled and stay in ContainerCreating mode. I need to wait minutes or delete pods for they correctly get rescheduled using the volume. Here is the log when the pod get automaticly deleted (after minutes) : It would be awesome if the deployment or RC wait for the PVC to be bound before schedule !
may slightly improve this situation as the maximum backoff time of 2 minutes will not fire anymore.
Note that the same error is thrown when node-size == C5 or M5 for kubectl and with kops . Solution is to set node as t2 machines. Due to it seems.
Reading from the new solution is:

Discussion:
return } <ins> allVolumesAdded := true </ins> // Process volume spec for each volume defined in pod for _, podVolume := range pod.Spec.Volumes { volumeSpec, volumeGidValue, err :=

podVolume.Name, format.Pod(pod), err) <ins> allVolumesAdded = false </ins> continue }

volumeSpec.Name(), uniquePodName, err) <ins> allVolumesAdded = false </ins> } glog.V(10).Infof(

uniquePodName) } <del> dswp.markPodProcessed(uniquePodName) </del> <ins> // some of the volume additions may have failed, should not mark this pod as fully processed if allVolumesAdded { dswp.markPodProcessed(uniquePodName) } </ins> } // podPreviouslyProcessed returns true if the volumes for this pod have already

// createVolumeSpec creates and returns a mutatable volume.Spec object for the // specified volume. It dereference any PVC to get PV objects, if needed. <ins> // Returns an error if unable to obtain the volume at this time. </ins> func (dswp *desiredStateOfWorldPopulator) createVolumeSpec( podVolume v1.Volume, podNamespace string) (*volume.Spec, string, error) { if pvcSource :=

} if pvc.Status.Phase != v1.ClaimBound || pvc.Spec.VolumeName == "" { return "", "", fmt.Errorf( "PVC %s/%s has non-bound phase (%q) or empty pvc.Spec.VolumeName (%q)", namespace,

} } <ins> func TestInitialPendingVolumesForPodAndGetVolumesInUse(t *testing.T) { tmpDir, err := utiltesting.MkTmpdir("volumeManagerTest") if err != nil { t.Fatalf("can't make a temp dir: %v", err) } defer os.RemoveAll(tmpDir) podManager := kubepod.NewBasicPodManager(podtest.NewFakeMirrorClient(), secret.NewFakeManager(), configmap.NewFakeManager()) node, pod, pv, claim := createObjects() claim.Status = v1.PersistentVolumeClaimStatus{ Phase: v1.ClaimPending, } kubeClient := fake.NewSimpleClientset(node, pod, pv, claim) manager := newTestVolumeManager(tmpDir, podManager, kubeClient) stopCh := runVolumeManager(manager) defer close(stopCh) podManager.SetPods([]*v1.Pod{pod}) // Fake node status update go simulateVolumeInUseUpdate( v1.UniqueVolumeName(node.Status.VolumesAttached[0].Name), stopCh, manager) // delayed claim binding go delayClaimBecomesBound(kubeClient, claim.GetNamespace(), claim.ObjectMeta.Name) err = manager.WaitForAttachAndMount(pod) if err != nil { t.Errorf("Expected success: %v", err) } } </ins> func TestGetExtraSupplementalGroupsForPod(t *testing.T) { tmpDir, err := utiltesting.MkTmpdir("volumeManagerTest") if err != nil {

} } <ins> func delayClaimBecomesBound( kubeClient clientset.Interface, namespace, claimName string, ) { time.Sleep(500 * time.Millisecond) volumeClaim, _ := kubeClient.Core().PersistentVolumeClaims(namespace).Get(claimName, metav1.GetOptions{}) volumeClaim.Status = v1.PersistentVolumeClaimStatus{ Phase: v1.ClaimBound, } kubeClient.Core().PersistentVolumeClaims(namespace).Update(volumeClaim) return } </ins> func runVolumeManager(manager VolumeManager) chan struct{} { stopCh := make(chan struct{}) //readyCh := make(chan bool, 1)