Issue 1:
Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.):

No.

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.):

kubeadm join

kubeadm join crash

kubeadm close of closed channel

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

BUG REPORT

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:48:38Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}

kubeadm version (use kubeadm version):

kubeadm version: version.Info{Major:"1", Minor:"5+", GitVersion:"v1.5.0-alpha.2.380+85fe0f1aadf91e", GitCommit:"85fe0f1aadf91e134102cf3c01a9eed11a7e257f", GitTreeState:"clean", BuildDate:"2016-11-02T14:58:17Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration:

OS (e.g. from /etc/os-release): CentOS

Kernel (e.g. uname -a): 3.10.0-327.36.1.el7.x86_64 #1 SMP Sun Sep 18 13:04:29 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Install tools: yum

Others:

What happened:

kubeadm crash when join mulitple API endpoints

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

join mulitple API endpoints

Anything else do we need to know:

kubeadm output:

# kubeadm join --token=ab6000.d5be12581d4bc52d 10.37.0.8
Running pre-flight checks
<util/tokens> validating provided token
<node/discovery> created cluster info discovery client, requesting info from "http://10.37.0.8:9898/cluster-info/v1/?token-id=ab6000"
<node/discovery> cluster info object received, verifying signature using given token
<node/discovery> cluster info signature and contents are valid, will use API endpoints [https://10.37.0.8:6443 https://10.37.0.10:6443 https://10.37.0.11:6443 https://10.37.0.12:6443]
<node/bootstrap> trying to connect to endpoint https://10.37.0.8:6443
<node/bootstrap> trying to connect to endpoint https://10.37.0.10:6443
<node/bootstrap> trying to connect to endpoint https://10.37.0.11:6443
<node/bootstrap> trying to connect to endpoint https://10.37.0.12:6443
<node/bootstrap> detected server version v1.4.5
<node/bootstrap> detected server version v1.4.5
<node/bootstrap> detected server version v1.4.5
<node/bootstrap> detected server version v1.4.5
<node/bootstrap> successfully established connection with endpoint https://10.37.0.11:6443
<node/csr> created API client to obtain unique certificate for this node, generating keys and certificate signing request
<node/bootstrap> successfully established connection with endpoint https://10.37.0.12:6443
E1117 19:59:42.617610 3072 runtime.go:64] Observed a panic: "close of closed channel" (close of closed channel)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:70
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:63
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:49
/usr/local/go/src/runtime/asm_amd64.s:479
/usr/local/go/src/runtime/panic.go:458
/usr/local/go/src/runtime/chan.go:311
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/node/bootstrap.go:83
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:87
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:88
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:49
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/node/bootstrap.go:90
/usr/local/go/src/runtime/asm_amd64.s:2086
panic: close of closed channel [recovered]
panic: close of closed channel

goroutine 14 [running]:
panic(0x2065ce0, 0xc42033b990)
/usr/local/go/src/runtime/panic.go:500 +0x1a1
k8s.io/kubernetes/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:56 +0x126
panic(0x2065ce0, 0xc42033b990)
/usr/local/go/src/runtime/panic.go:458 +0x243
k8s.io/kubernetes/cmd/kubeadm/app/node.EstablishMasterConnection.func1.1()
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/node/bootstrap.go:83 +0x29d
k8s.io/kubernetes/pkg/util/wait.JitterUntil.func1(0xc4205f3ed0)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:87 +0x5e
k8s.io/kubernetes/pkg/util/wait.JitterUntil(0xc4205f3ed0, 0x12a05f200, 0x0, 0xc420272601, 0xc4203e8fc0)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:88 +0xad
k8s.io/kubernetes/pkg/util/wait.Until(0xc4205f3ed0, 0x12a05f200, 0xc4203e8fc0)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:49 +0x4d
k8s.io/kubernetes/cmd/kubeadm/app/node.EstablishMasterConnection.func1(0xc42044a6b0, 0xc4201ea180, 0xc4203e8fc0, 0xc4203e9020, 0xc420172000, 0x401, 0x480, 0xc42044a6a0, 0x4, 0xc4203edca0, ...)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/node/bootstrap.go:90 +0x100
created by k8s.io/kubernetes/cmd/kubeadm/app/node.EstablishMasterConnection
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/node/bootstrap.go:91 +0x3ed

Issue 2:
We want to deprecate today's debian-based container-vm image, and switch to GCI image by default. Before we do that, we need to validate the image using Kubernetes testing infrastructure. GCI image has three different release channels: dev, beta, and stable. Besides that they also build ci release nightly. Today we already run the the following jenkins jobs against each image release:

kubernetes-gce-e2e,

kubernetes-gce-e2e-slow,

kubernetes-gce-e2e-serial

But we are missing kubernetes-gce-e2e-soak, kubernetes-gce-e2e-scalability, probably also kubernetes-gce-e2e-upgrade (? not sure how to handle this since it is the first release), and kubernetes-gce-e2e-features.

We also need to include gci image to node-e2e test suite too.

Next step we should to identify the critical builds for gci to block merge-bot.

Eventually, we can switch GCI image as the default image on GCE, but keep debian-based container-vm image testing until we can deprecate it.

@andyzheng0831 and @wonderfly are going to write detail plan on this.

cc/ @roberthbailey from GKE side.

cc/ @mansoorj from PM side