Issue 1:
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/29986/kubernetes-pull-build-test-e2e-gce/51596/

Failed: [k8s.io] Docker Containers should be able to override the image's default commmand (docker entrypoint) [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/docker_containers.go:54
Error creating Pod
Expected error:
<*url.Error | 0xc82040c210>: {
Op: "Post",
URL: "https://104.155.177.28/api/v1/namespaces/e2e-tests-containers-nkgxg/pods",
Err: {
Op: "dial",
Net: "tcp",
Source: nil,
Addr: {
IP: "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xffh\x9b\xb1\x1c",
Port: 443,
Zone: "",
},
Err: {},
},
}
Post https://104.155.177.28/api/v1/namespaces/e2e-tests-containers-nkgxg/pods: dial tcp 104.155.177.28:443: i/o timeout
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:50

Happened on a presubmit run in #29986.

Issue 2:
Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.): No

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): Completed Status POD Unmount

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.5+$Format:%h$", GitCommit:"dc1efecced758680dc5aa29dc317e366b6792a48", GitTreeState:"dirty", BuildDate:"2016-10-31T10:12:09Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"linux/amd64"}

Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.5+$Format:%h$", GitCommit:"996dae74127a3e5a2c3da136dd846356e612b18f", GitTreeState:"dirty", BuildDate:"2016-11-01T03:03:10Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration:

VMs on softlayer. 1 Master nodes 3 worker nodes

OS (e.g. from /etc/os-release):

NAME="Ubuntu"

VERSION="14.04.5 LTS, Trusty Tahr"

ID=ubuntu

ID_LIKE=debian

PRETTY_NAME="Ubuntu 14.04.5 LTS"

VERSION_ID="14.04"

HOME_URL="http://www.ubuntu.com/"

SUPPORT_URL="http://help.ubuntu.com/"

BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

Kernel (e.g. uname -a):

Linux cdsdev-dal09-stony03-api-01.softlayer.com 3.16.0-76-generic #98~14.04.1-Ubuntu SMP Fri Jun 24 17:04:54 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Install tools: Ansible

Others: No.

What happened:

Create a bunch of PVs using nfs

Create PVC in a pvc_storage.yaml

resources:
requests:
storage: 1Ti

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: dashmpp-local2
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Ti

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: dashmpp-data2
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Ti

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: dashmpp-local3
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Ti

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: dashmpp-data3
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Ti

After use kubectl create -f pvc_storage.yaml

NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE
ppvl01 15Gi RWX Recycle Available 6d
ppvl02 1Ti RWO Recycle Bound default/dashmpp-local1 6d
ppvl04 1Ti RWO Recycle Bound default/dashmpp-data1 6d
ppvl05 1Ti RWO Recycle Bound default/dashmpp-data2 27s
ppvl06 1Ti RWO Recycle Bound default/dashmpp-local2 1m
spv1 10Gi RWX Recycle Bound default/dashmpp-head 6d
spv2 10Gi RWX Recycle Available 6d
spv3 10Gi RWX Recycle Bound default/dashmpp-scratch 6d
spv4 10Gi RWX Recycle Available 6d

After run kubectl delete -f pv_storage.yaml

persistentvolumeclaim "dashmpp-head" deleted
persistentvolumeclaim "dashmpp-scratch" deleted
persistentvolumeclaim "dashmpp-local1" deleted
persistentvolumeclaim "dashmpp-data1" deleted
persistentvolumeclaim "dashmpp-local2" deleted
persistentvolumeclaim "dashmpp-data2" deleted
persistentvolumeclaim "dashmpp-local3" deleted
persistentvolumeclaim "dashmpp-data3" deleted

Using kubectl get pod -a -o wide some of the recyclers are in Completed status.

NAME READY STATUS RESTARTS AGE IP NODE

recycler-for-ppvl04 0/1 Completed 0 1m 10.121.55.231

recycler-for-spv1 0/1 Completed 0 1m 10.121.55.102

recycler-for-spv3 0/1 Completed 0 1m 10.121.55.70

Login one of the recycler Node, e.g. 10.121.55.70 using df -h, we got

169.55.11.79:/gpfs/fs01/shared/prod/democluster01/dashdb/gamestop/spv3 99T 18T 81T 19% /var/lib/kubelet/pods/1e5c2676-a045-11e6-8264-06d7ef65f0fb/volumes/kubernetes.io~nfs/ppvl02

As you can see the mount point still exist.

If we delelet PVCs one by one, such as

kubectl delete pvc dashmpp-local1
kubectl delete pvc dashmpp-local2
kubectl delete pvc dashmpp-data1
kubectl delete pvc dashmpp-data2
kubectl delete pvc dashmpp-head
kubectl delete pvc dashmpp-scratch

We do not get such problems

What you expected to happen:

Using kubectl delete -f pvc_storage.yaml, all the recycler could not inCompleted status, and umount the mount point from worker node.

How to reproduce it (as minimally and precisely as possible):

Describe in what happened section

Anything else do we need to know:

No.