Issue 1:
Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.):

No

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.):

kubernetes, openstack, cloud-config, loadbalancer

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

BUG REPORT

Kubernetes version (use kubectl version):

stack@kube-2:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:48:38Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:42:39Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration:

VM

OS (e.g. from /etc/os-release):

Ubuntu 16.04

Kernel (e.g. uname -a):

stack@kube-2:~$ uname -a
Linux kube-2 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Install tools:

Installed K8S cluster using kubeadm

Others:

I am trying to use openstack's load balancer as an external load balancer for k8s services(type:loadbalancer). Currently my setup is as follows:

One node k8s cluster installed on a VM with Ubuntu 16.04. I have another node where devstack is installed. I have the following details in the cloud config file:

[Global]
auth-url=http://10.10.40.152/identity_admin
username=admin
password=password
tenant-name=admin
region=regionOne

I have updated the kube-controller-manager.json and kube-apiserver.json with the following configuration:

"--cloud-provider=openstack",

"--cloud-config=/etc/sysconfig/kube_openstack_config",

What happened:

As soon as I updated the above config files, kube-controller-manager container crashes.

What you expected to happen:

kube-controller-manager is supposed to read the cloud config file and listen to kubernetes api calls and create load balancer on openstack.

How to reproduce it (as minimally and precisely as possible):

Update the above files with cloud-provider and cloud-config options.

Anything else do we need to know:

The k8s cluster is not running inside openstack/devstack. One node runs k8s cluster and another node is running devstack.

stack@kube-2:~$ kubectl get pods --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
kube-system calico-node-k9wsj 2/2 Running 0 1h 10.10.40.151 kube-2
kube-system calico-policy-controller-2snnc 1/1 Running 0 1h 10.10.40.151 kube-2
kube-system dummy-2088944543-l5cmf 1/1 Running 0 1h 10.10.40.151 kube-2
kube-system etcd-kube-2 1/1 Running 0 1h 10.10.40.151 kube-2
kube-system kube-apiserver-kube-2 1/1 Running 2 1h 10.10.40.151 kube-2
**kube-system kube-controller-manager-kube-2 0/1 CrashLoopBackOff 28 3s 10.10.40.151 kube-2**
kube-system kube-discovery-1150918428-iwg0p 1/1 Running 0 1h 10.10.40.151 kube-2
kube-system kube-dns-654381707-cv1f1 3/3 Running 0 1h 192.168.212.198 kube-2
kube-system kube-proxy-32pls 1/1 Running 0 1h 10.10.40.151 kube-2
kube-system kube-scheduler-kube-2 1/1 Running 0 1h 10.10.40.151 kube-2

Issue 2:
Possibly a duplicate of an earlier issue. I can't delete pods and I have to keep re-trying kubectl delete which often fails. GCE.

satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-0
pods/serve-hostname-0-1
pods/serve-hostname-0-10
F0331 17:06:00.464445 15992 delete.go:67] Client error: 501: All the given peers are not reachable (failed to propose on members [http://127.0.0.1:4001] twice [last error: dial tcp 127.0.0.1:4001: too many open files]) [0]
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-11
pods/serve-hostname-0-12
pods/serve-hostname-0-13
pods/serve-hostname-0-14
pods/serve-hostname-0-15
pods/serve-hostname-0-16
pods/serve-hostname-0-17
F0331 17:06:21.511183 16000 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-0-18?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-18
pods/serve-hostname-0-19
pods/serve-hostname-0-2
pods/serve-hostname-0-20
pods/serve-hostname-0-21
pods/serve-hostname-0-22
pods/serve-hostname-0-23
pods/serve-hostname-0-24
pods/serve-hostname-0-25
pods/serve-hostname-0-26
pods/serve-hostname-0-27
pods/serve-hostname-0-28
pods/serve-hostname-0-29
pods/serve-hostname-0-3
pods/serve-hostname-0-4
pods/serve-hostname-0-5
pods/serve-hostname-0-6
pods/serve-hostname-0-7
pods/serve-hostname-0-8
pods/serve-hostname-0-9
pods/serve-hostname-1-0
pods/serve-hostname-1-1
pods/serve-hostname-1-10
pods/serve-hostname-1-11
pods/serve-hostname-1-12
pods/serve-hostname-1-13
F0331 17:06:50.483268 16008 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-14?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-1-14
pods/serve-hostname-1-15
pods/serve-hostname-1-16
pods/serve-hostname-1-17
pods/serve-hostname-1-18
pods/serve-hostname-1-19
pods/serve-hostname-1-2
pods/serve-hostname-1-20
pods/serve-hostname-1-21
pods/serve-hostname-1-22
pods/serve-hostname-1-23
pods/serve-hostname-1-24
pods/serve-hostname-1-25
pods/serve-hostname-1-26
pods/serve-hostname-1-27
pods/serve-hostname-1-28
pods/serve-hostname-1-29
F0331 17:07:17.744491 16017 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-3?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:07:36.299212 16025 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-3?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
7pods/serve-hostname-1-3
pods/serve-hostname-1-4 pods/serve-hostname-1-5
pods/serve-hostname-1-6
pods/serve-hostname-1-7
pods/serve-hostname-1-8
F0331 17:08:21.421952 16049 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-9?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:08:51.879079 16057 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506?quorum=false&recursive=true&sorted=true: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:08:55.015498 16068 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506?quorum=false&recursive=true&sorted=true: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:09:16.905368 16077 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-9?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-1-9
F0331 17:14:20.492245 16139 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-0?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:15:04.572098 16149 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-0?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-10-0
pods/serve-hostname-10-1
pods/serve-hostname-10-10
pods/serve-hostname-10-11
pods/serve-hostname-10-12
pods/serve-hostname-10-13
pods/serve-hostname-10-14
pods/serve-hostname-10-15
pods/serve-hostname-10-16
pods/serve-hostname-10-17
pods/serve-hostname-10-18
pods/serve-hostname-10-19
pods/serve-hostname-10-2
pods/serve-hostname-10-20
pods/serve-hostname-10-21
pods/serve-hostname-10-22
pods/serve-hostname-10-23
F0331 17:17:15.837176 16174 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-24?quorum=false&recursive=false&sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$