Code change:
Is this a BUG REPORT or FEATURE REQUEST?: What happened: After upgrade of Kubernetes 1.11.3 to 1.12.1 via kubeadm in bare-metal setup I see these errors in kubelet logs: What you expected to happen: Since this is a bare-metal setup I expect no messages concerning Azure cloud How to reproduce it (as minimally and precisely as possible): Do an upgrade from 1.11.3 to 1.12.1 Anything else we need to know?: kubelet-1.12.1- was downloaded from Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-10-05T16:36:14Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: bare-metal OS (e.g. from /etc/os-release): Oracle Linux Server 7.4 Kernel (e.g. ): 62 Install tools: Others:
Lines from kubelet log which say that no cloud provider was specified (that is a bare-metal setup): ..
/sig azure
have the same issue related to the azure.. also a bare-metal cluster.. these messages are flooding my centralized logging elasticsearch (though they aren't really problematic more than being an annoyance). i also upgraded from 1.11.3 to 1.12.1 on a 3-master HA set up. i also noted (perhaps worthy of a separate issue) that the two of my three control-planes that are showing the azure error also previously had been having cert authority errors from the api-server container, when having used kubeadm upgrade.. (on the second two control plane systems, with the issue) (and on the first control without the issue) (and i needed to kubeadm reset and then kubeadm init all over again with 1.12.1 on the 2nd two control-plane nodes).. the first control-plane node that didn't need to do a reset/init didn't have the azure errors (nor the cert errors). so i kind of suspect it might not be an kubeadm upgrade issue but a general issue going from 1.11.3 to 1.12.1 ?
Same problems +1
Same problem on brand new bm 1.12.1 cluster built with kubeadm and latest 1alpha3 config based off of customized kubeadm config print-default example
/assign
Same problems +1
Oct 9 11:11:32 localhost kubelet: E1009 11:11:32. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:11:42 localhost kubelet: E1009 11:11:42. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:11:52 localhost kubelet: E1009 11:11:52. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:12:02 localhost kubelet: E1009 11:12:02. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:12:12 localhost kubelet: E1009 11:12:12. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:12:23 localhost kubelet: E1009 11:12:23. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:12:33 localhost kubelet: E1009 11:12:33. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:12:43 localhost kubelet: E1009 11:12:43. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:12:53 localhost kubelet: E1009 11:12:53. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1 Oct 9 11:13:03 localhost kubelet: E1009 11:13:03. 3342 ] failed to get azure cloud in GetVolumeLimits, kubernetes-1
how often did you hit this error log: I am planning to make this log as , is that ok for you? Usually V(2) is the default info(see ) log for k8s on azure I am not sure what the default info log by kubeadmin installation.
It is logged every 10 seconds. I disagree. At least previous incarnation of GetVolumeLimits in version 1.11.3 : would have logged an error "No cloudprovider present". Which I didn't see in kubelet log. In fact, I didn't see any message in kubelet logs with 'azure' substring at all. I think it means that AzureDataDisk plugin was not instantiated at all in 1.11.3. So the question is: why azuredd is instantiated now in 1.12.1 in bare-metal setup (that is without cloud provider)? Do you mind checking this idea? (Besides your method of lowering message priority won't work as I said this is a bare-metal setup where kubelet systemd unit file has no --v option at all )
That's because feature reaches beta in v1.12, in v1.11.3, even the code is there, it won't be invoked since feature is still alpha in v1.11
And in v1.12, aws, gce, and azure disk plugins are all loaded, take aws ebs for example, it will return error directly: And logging as V(4): I suppose V(4) logs won't display in logs. Let me make the same change as aws and gce.
also tries to address this error.
The same mistake, I also encountered NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME 172.16.16.13 Ready <none5m27s v1.12.1 172.16.16.13 <noneCentOS Linux 7 (Core) 4.18.12- docker://18.6.0 failed to get azure cloud in GetVolumeLimits, 172.16.16.13 failed to get azure cloud in GetVolumeLimits, 172.16.16.13 failed to get azure cloud in GetVolumeLimits, 172.16.16.13
I have cherry picked the fix to v1.12
I understand that the feature Node-specific Volume Limits has now reached beta in 1.12, but for those of us running Kubernetes on-premises, this log error is pretty useless. In fact, I would even want to disable this plugin alltogether, since it will never be used on-prem. Is there a way to stop the plugin from loading, or in the very least make sure the log does not appear? Right now, it is polluting the system log file quite a lot.
I second that. That was the key message of my bug report. That is we must not even instantiate cloud-specific plugins if no cloud provider is configured.
I'm having exactly the same problem, doing an installation from scratch on DigitalOcean
+1
+1
Hi I have the same issue. It would be really nice to know how to disable this error message that floods the logs for no reason.
For those using logstash it was sufficient to just temporarily add this to your syslog filter: if [message] =~ "" { drop { } }
After a fresh installation of Raspbian and kubernetes, same issue is occurring in my 4 node cluster. Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/arm"} Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"", GitTreeState:"clean", BuildDate:"2018-10-05T16:36:14Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/arm"} flood of this: epsilon kubelet[2412]: E1024 12:25:58. 2412 ] failed to get azure cloud in GetVolumeLimits, epsilon
disable in controller plane should work: "--feature-gates=AttachVolumeLimit=false"
I have tried "--feature-gates=AttachVolumeLimit=false" and it's not working. I still see messages kubelet[]: E1024 22:51:02. ] failed to get azure cloud in GetVolumeLimits,
I have the same problem here.., ] failed to get azure cloud in GetVolumeLimits,
+1 same case here, also tried with --feature-gates=AttachVolumeLimit=false
Will this be disabled by default in the next release?
It's already fixed in v1.12.2 and v1.13.0. do you know how to disable in kubelet in v1.12? Would add in kubelet command arguments work?
I haven't updated to 1.12.2 yet (still on 1.12.1) but just to confirm, disabling that feature works. For those that are also running on CentOS and/or RHEL, just to the following: Edit file on your master server (control plane), and add the following: If you already have extra arguments in there, just add this one at the end of the line. Then restart kubelet on your master server:
Yes that will work.
--feature-gates=AttachVolumeLimit property doesn't exist in the documentation
Update tho 1.12.2 resolve for me Em qui, 1 de nov de 2018 9:22 PM, Abhilash Pallerlamudi < escreveu:
add follow line to "your kubelet.service" file. like this: [root ~]# systemctl daemon-reload [root ~]# sudo systemctl restart kubelet then it's ok
So to get rid of a bug we should disable an entire feature? Says it is supported by Azure-Disk. Why isn't node reporting the limit? It is getting instance metadata okay as all my nodes are tagged with SKU. Being that SKU determines attach limit as well, I do not understand why this cannot be made working? I am on 1.14 in Azure on Ubuntu.
how do you deploy k8s on Azure? To get volume limit, kubelet on agent node needs azure identity, e.g. service principal, usually it's under file
I deployed with kubespray. I have cloudconfig set as such: KUBELETCLOUDPROVIDER="--cloud-provider=azure --cloud-config=/etc/kubernetes/cloudconfig" Service principle is set in cloudconfig file.
I triple checked and location was misspelled on some of the nodes for the cloud provider. Thanks!
the get vm size list call depends on location, so now it works?
Yes it does. Thanks!

Related discussion:
text: // hoping external CCM or admin can set it. Returning // default values from here will mean, no one can // override them. <del> glog.Errorf("failed to get azure cloud in GetVolumeLimits, plugin.host: %s", plugin.host.GetHostName()) return volumeLimits, nil </del> <ins> return nil, fmt.Errorf("failed to get azure cloud in GetVolumeLimits, plugin.host: %s", plugin.host.GetHostName()) </ins> } instances, ok := az.Instances()