Change:
What happened: when initializing a kubernetes cluster in Google Cloud manually (not using GKE), internal (and external if available/applicable) node IP is not populated in "kubectl get nodes" if the interface used to connect to the master and passed to kubelet does not correspond to the first network interface, nic0 on google cloud console UI. I cannot "kubectl edit node" and patch in the node IP information into the "addresses" field either. This issue does not occur when kubelet --node-ip=(ip of nic0). kubelet on all members of the cluster declare cloud-provider="gce". What you expected to happen: when kubelet is given a node-ip argument, it is checked against all nics of the google cloud instance and populated in node information even if a nic other than nic0 is used. Or at least I am allowed to override nodeIP somehow after the node is brought up. How to reproduce it (as minimally and precisely as possible): We have scripts that automate this "manual kubernetes" process, I will describe it as generally as I can but i can share chunks of the script as needed. Given two VPCs, initialize an instance for master, one nic on VPC1 initialize an instance for worker with two nics, one on VPC2, the other on VPC1. install required dependencies on both (apt-get docker-ce kubelet kubeadm kubectl ...) on master, mkdir -p /etc/default/kubelet; echo "KUBELETEXTRAARGS=--cloud-provider='gce' /etc/default/kubelet; kubeadm init on worker, mkdir -p /etc/default/kubelet; echo "KUBELETEXTRAARGS=--node-ip=(nic1 ip) --cloud-provider='gce' /etc/default/kubelet; kubeadm join wait a bit. kubectl get nodes -o wide, verify no internal IP or external IP information is populated for the worker journalctl -xef on worker to see something like ] Failed to set some node status fields: failed to get node address from cloud provider that matches ip: (nic1 ip) Anything else we need to know?: i'm guessing the problem stems from something like where we only check nic0 and return that, and then this bit here only loops through nic0's info, doesn't match, and then we return an error. if the first bit returned an array of IPs instead of just a single set, that would probably fix this. Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2020-04-08T17:30:47Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: Google Cloud, all n1-standard-4 instances OS (e.g: ): NAME="Ubuntu" VERSION="18.04.4 LTS (Bionic Beaver)" ID=ubuntu IDLIKE=debian PRETTYNAME="Ubuntu 18.04.4 LTS" VERSIONID="18.04" HOMEURL="" SUPPORTURL="" BUGREPORTURL="" PRIVACYPOLICYURL="" VERSIONCODENAME=bionic UBUNTUCODENAME=bionic Kernel (e.g. ): Linux 5.0.0-1033-gcp -Ubuntu SMP Tue Mar 3 04:36:26 UTC 2020 x8664 x8664 x86_64 GNU/Linux Install tools: installed kubeadm kubelet docker and whatever else through apt-get Network plugin and version (if this is a network-related bug): Others:
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig cloud-provider ? Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
w.r.t. NodeAddresses in We can keep calling Get for 1/ip, 2/ip, ... until err is non-nil. Is there a better way to find how many interfaces there are ?
i haven't used the gcloud go metadata client, but looping until error or hitting instance/network-interfaces directly for the interface indices available is probably about the same since NIC count is limited by VM tier anyways.
I will create a PR with a fix for this

Discussion:
) const ( <del> defaultZone = "" </del> <ins> defaultZone                   = "" networkInterfaceIP            = "instance/network-interfaces/%s/ip" networkInterfaceAccessConfigs = "instance/network-interfaces/%s/access-configs" networkInterfaceExternalIP    = "instance/network-interfaces/%s/access-configs/%s/external-ip" </ins> ) func newInstancesMetricContext(request, zone string) *metricContext {

// Use metadata server if possible if g.isCurrentInstance(instanceName) { <del> internalIP, err := metadata.Get("instance/network-interfaces/0/ip") </del> <ins> nics, err := metadata.Get("instance/network-interfaces/") </ins> if err != nil { <del> return nil, fmt.Errorf("couldn't get internal IP: %v", err) </del> <ins> return nil, fmt.Errorf("couldn't get network interfaces: %v", err) </ins> } <del> externalIP, err := metadata.Get("instance/network-interfaces/0/access-configs/0/external-ip") if err != nil { return nil, fmt.Errorf("couldn't get external IP: %v", err) } addresses := []v1.NodeAddress{ {Type: v1.NodeInternalIP, Address: internalIP}, {Type: v1.NodeExternalIP, Address: externalIP}, </del> <ins> nicsArr := strings.Split(nics, "/n") nodeAddresses := []v1.NodeAddress{} for _, nic := range nicsArr { if nic == "" { continue } internalIP, err := metadata.Get(fmt.Sprintf(networkInterfaceIP, nic)) if err != nil { return nil, fmt.Errorf("couldn't get internal IP: %v", err) } nodeAddresses = append(nodeAddresses, v1.NodeAddress{Type: v1.NodeInternalIP, Address: internalIP}) acs, err := metadata.Get(fmt.Sprintf(networkInterfaceAccessConfigs, nic)) if err != nil { return nil, fmt.Errorf("couldn't get access configs: %v", err) } acsArr := strings.Split(acs, "/n") for _, ac := range acsArr { if ac == "" { continue } externalIP, err := metadata.Get(fmt.Sprintf(networkInterfaceExternalIP, nic, ac)) if err != nil { return nil, fmt.Errorf("couldn't get external IP: %v", err) } if externalIP != "" { nodeAddresses = append(nodeAddresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: externalIP}) } } </ins> } <del> if internalDNSFull, err := metadata.Get("instance/hostname"); err != nil { </del> <ins> internalDNSFull, err := metadata.Get("instance/hostname") if err != nil { </ins> klog.Warningf("couldn't get full internal DNS name: %v", err) } else { <del> addresses = append(addresses, </del> <ins> nodeAddresses = append(nodeAddresses, </ins> v1.NodeAddress{Type: v1.NodeInternalDNS, Address: internalDNSFull}, v1.NodeAddress{Type: v1.NodeHostName, Address: internalDNSFull}, ) } <del> return addresses, nil </del> <ins> return nodeAddresses, nil </ins> } }

if len(instance.NetworkInterfaces) < 1 { return nil, fmt.Errorf("could not find network interfaces for instanceID %q", instance.Id) } <del> networkInterface := instance.NetworkInterfaces[0] </del> <ins> nodeAddresses := []v1.NodeAddress{} </ins> <del> nodeAddresses := []v1.NodeAddress{{Type: v1.NodeInternalIP, Address: networkInterface.NetworkIP}} for _, config := range networkInterface.AccessConfigs { nodeAddresses = append(nodeAddresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: config.NatIP}) </del> <ins> for _, nic := range instance.NetworkInterfaces { nodeAddresses = append(nodeAddresses, v1.NodeAddress{Type: v1.NodeInternalIP, Address: nic.NetworkIP}) for _, config := range nic.AccessConfigs { nodeAddresses = append(nodeAddresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: config.NatIP}) } </ins> } return nodeAddresses, nil }