Change:
BUG REPORT: /kind bug What happened: After upgrading the kubernetes cluster from 1.6.4 with the VSphere Cloud Provider enabled to 1.6.5/1.6.6 or 1.6.7, the cluster nodes don't have anymore InternalIPs Several issues occur at this step. For example, it breaks heapster with such an error : What you expected to happen: The node should have an InternalIP. How to reproduce it (as minimally and precisely as possible): Deploy a 1 node kubernetes cluster in a VM, and enable the vsphere cloud provider. Anything else we need to know?: with kubernetes 1.6.4 we get : Downgrading to 1.6.4 fixes the issue. Environment: Kubernetes version (use ):Cloud provider or hardware configuration: vsphere OS (e.g. from /etc/os-release):Kernel (e.g. ):Install tools: Others: It may be related to some changes around PR kubernetes /area platform/vsphere cloudprovider
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <bre.g., for API Machinery<br(2) specifying the label manually: <bre.g., for sig/scalability<br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
/sig cluster-ops /sig network /area platform/vsphere /area cloudprovider
Performing some more tests on a mini instance outside of the production cluster, I think I've identified the issue. The configuration required to start kubelet with the option (otherwise the wrong IP/interface is used to communicate between nodes). The issue is triggered when this option is set. The cloud provider looks OK. But the issue is located here : When the is not , the loop iterates on the node addresses and returns only the first one found. Since PR kubernetes, the list always returns an ExternalIP first. I think the loop in should iterate on all the addresses to rebuild the list, so that it will include the ExternalIP and the InternalIP. Can someone confirm the analysis ?
/sig node
ping
Modifying the code like this solved the issue : which returns : I guess it can be done in a cleaner way, but it was to quickly test the fix.
CC:
Thanks for filing this issue related to vSphere cloud provider. I agree with your analysis. It seems the current function assumes that the IP address specified by user can only be InternalIP. Your fix to is a better solution than work-around in cloud provider.
Hi, what are the next steps to discuss about the PR ? Are you able to make some tests ?

Discussion:
"//pkg/api:go_default_library", "//pkg/api/install:go_default_library", "//pkg/capabilities:go_default_library", <ins> "//pkg/cloudprovider/providers/fake:go_default_library", </ins> "//pkg/kubelet/apis:go_default_library", "//pkg/kubelet/apis/kubeletconfig:go_default_library", "//pkg/kubelet/cadvisor/testing:go_default_library",

return fmt.Errorf("failed to get node address from cloud provider: %v", err) } if kl.nodeIP != nil { <ins> enforcedNodeAddresses := []v1.NodeAddress{} </ins> for _, nodeAddress := range nodeAddresses { if nodeAddress.Address == kl.nodeIP.String() { <del> node.Status.Addresses = []v1.NodeAddress{ {Type: nodeAddress.Type, Address: nodeAddress.Address}, {Type: v1.NodeHostName, Address: kl.GetHostname()}, } return nil </del> <ins> enforcedNodeAddresses = append(enforcedNodeAddresses, v1.NodeAddress{Type: nodeAddress.Type, Address: nodeAddress.Address}) </ins> } } <ins> if len(enforcedNodeAddresses) > 0 { enforcedNodeAddresses = append(enforcedNodeAddresses, v1.NodeAddress{Type: v1.NodeHostName, Address: kl.GetHostname()}) node.Status.Addresses = enforcedNodeAddresses return nil } </ins> return fmt.Errorf("failed to get node address from cloud provider that matches ip: %v", kl.nodeIP) }

import ( "encoding/json" "fmt" <ins> "net" </ins> goruntime "runtime" "sort" "strconv"

"k8s.io/apimachinery/pkg/util/wait" "k8s.io/client-go/kubernetes/fake" core "k8s.io/client-go/testing" <ins> fakecloud "k8s.io/kubernetes/pkg/cloudprovider/providers/fake" </ins> kubeletapis "k8s.io/kubernetes/pkg/kubelet/apis" "k8s.io/kubernetes/pkg/kubelet/cm" kubecontainer "k8s.io/kubernetes/pkg/kubelet/container"

return lcm.capacity } <ins> func TestNodeStatusWithCloudProviderNodeIP(t *testing.T) { testKubelet := newTestKubelet(t, false /* controllerAttachDetachEnabled */) defer testKubelet.Cleanup() kubelet := testKubelet.kubelet kubelet.hostname = testKubeletHostname existingNode := v1.Node{ ObjectMeta: metav1.ObjectMeta{Name: testKubeletHostname, Annotations: make(map[string]string)}, Spec:       v1.NodeSpec{}, } // TODO : is it possible to mock kubelet.validateNodeIP() to avoid relying on the host interface addresses ? addrs, err := net.InterfaceAddrs() assert.NoError(t, err) for _, addr := range addrs { var ip net.IP switch v := addr.(type) { case *net.IPNet: ip = v.IP case *net.IPAddr: ip = v.IP } if ip != nil && !ip.IsLoopback() && ip.To4() != nil { kubelet.nodeIP = ip break } } assert.NotNil(t, kubelet.nodeIP) fakeCloud := &fakecloud.FakeCloud{ Addresses: []v1.NodeAddress{ { Type:    v1.NodeExternalIP, Address: "132.143.154.163", }, { Type:    v1.NodeExternalIP, Address: kubelet.nodeIP.String(), }, { Type:    v1.NodeInternalIP, Address: "132.143.154.164", }, { Type:    v1.NodeInternalIP, Address: kubelet.nodeIP.String(), }, { Type:    v1.NodeInternalIP, Address: "132.143.154.165", }, { Type:    v1.NodeHostName, Address: testKubeletHostname, }, }, Err: nil, } kubelet.cloud = fakeCloud kubelet.setNodeAddress(&existingNode) expectedAddresses := []v1.NodeAddress{ { Type:    v1.NodeExternalIP, Address: kubelet.nodeIP.String(), }, { Type:    v1.NodeInternalIP, Address: kubelet.nodeIP.String(), }, { Type:    v1.NodeHostName, Address: testKubeletHostname, }, } assert.True(t, apiequality.Semantic.DeepEqual(expectedAddresses, existingNode.Status.Addresses), "%s", diff.ObjectDiff(expectedAddresses, existingNode.Status.Addresses)) } </ins> func TestUpdateNewNodeStatus(t *testing.T) { // generate one more than maxImagesInNodeStatus in inputImageList inputImageList, expectedImageList := generateTestingImageList(maxImagesInNodeStatus + 1)