Change:
PR introduced an annotation for fluentd migration from static pod to the daemonset to avoid two instances of fluentd on a single node during the transition stage. I didn't read the pr carefully, but based on the code at , every node object has one label be default: "". I think this is wrong. 1) Not everyone use fluentd as the logging agent. 2) It is hard-coded in Kubelet code as part of kubeletnodestatus. No matter running fluentd as a static pod or a daemonset on the node, Kubelet shouldn't treat it different from other pods. The right approach is introducing an external controller to apply this cloud-provider / kubernetes vendor (i.e. GKE) specific label to the node. cc/
The string "fluentd" shouldn't appear anywhere in our code. Please remove it ASAP.
Absolutely agree with you, thanks for bringing this up! Is it possible to apply labels to the node as part of the node startup script/salt?
To clarify, I'm thinking about possible ways to provide a fix for that ASAP. Writing external controller is a best solution, but here we have a release coming and this label is intended to be only a temporary measure to perform upgrade from using fluentd as static pods to using daemon set.
There are many ways to achieve this outside the core system. But I believe GKE already has an existing mechanism to add this kind of labels to the node, and the consistency is important for GKE management. cc/
OK, thanks! However, I believe we should be sure not to break everyone's upgrade process, i.e. after master upgrade to 1.6 on any platform, there should be only one fluentd agent on each node. I propose changing the configure scripts and salt to set up needed labels in kubelet's flag (). WDYT?
It seems to me like upgrades could survive without the explicit dependency on fluentd: I'm not sure if upgrades call initialNode or not, my own ignorance ? If they do, and this is removed, can't an external controller just do the work of re--adding that label if its dropped?
thanks for raising this. Let's sync f2f
cc
WDYT about exposing kubelet version as a label and have a selector in fluentd ds config targeting version 1.6+?
and I talked about this issue a bit for migrating kube-proxy. One idea we came up with was that of running a controller that will identify kubelet versions and apply taints to nodes running static pods. The daemonset version would not tolerate that taint and that way we keep daemon set pods off of nodes running static pods. Have you considered that approach?
It doesn't differ from a controller applying labels. I'd like to avoid necessity of introducing such controller, since it's not clear what is it's responsibility (generic controller for applying labels on nodes based on some rules? a custom controller just for fluentd purposes). In GKE we can do it on GKE control plane side, but we need a story for open source.
Move applying labels to startup scripts/salt seems to be the least intrusive and easiest to implement way.
Actually what I proposed in is not an option since nodeSelector in PodSpec is old style selector.
I'm not a big fan of approach with salt, but it seems there is no better option.
Re: Instead of exposing Kubelet version as a label, why not using field selector. Kubelet version is part of NodeStatus already. I just simply don't want to have a duplicate info for the same object.
is it possible to schedule based on fieldselector? I'm only aware of using field selector while listing objects.
Also node-labels option is in alpha
But it's present since 1.2 What is the status of this flag? Can we use it for fluentd purposes or better avoid this?
Maybe it does make sense to implement a temporary ad-hoc controller? It's the easiest to deploy solution, just another in fluentd addon.
I know. It totally doesn't make sense to introduce anything custom. I'll use just the flag. It's rather hacky but unfortunately I can't see anything better.
Cool thing about controller is that you can deploy it into any cluster and it will work, with CA also. You don't have to set up your node labels manually then or restart the cluster.
There is a number of open question with having a new controller: it run on the master or a node? we guarantee that it will be always running? it have any resource requirements? I'd like to avoid discussing it right before the release.
Having a controller will help with kube-proxy transition too in the future. , Piotr Szczesniak <:
is this a 1.6 release blocker?
I think yes. The PR is already sent and approved. I need to fix e2e tests because it doesn't work well with the test infra (e2e tests passes locally). EDIT: I'll do it hopefully today.
The PR is ready to be merged and lgtmed. Waiting for approvals.

Discussion:
#   standalone     - Heapster only. Metrics available via Heapster REST API. ENABLE_CLUSTER_MONITORING="${KUBE_ENABLE_CLUSTER_MONITORING:-influxdb}" <ins> # Historically fluentd was a manifest pod and then was migrated to DaemonSet. # To avoid situation during cluster upgrade when there are two instances # of fluentd running on a node, kubelet need to mark node on which # fluentd is not running as a manifest pod with appropriate label. # TODO(piosz): remove this in 1.8 NODE_LABELS="${KUBE_NODE_LABELS:-alpha.kubernetes.io/fluentd-ds-ready=true}" </ins> # Optional: Enable node logging. ENABLE_NODE_LOGGING="${KUBE_ENABLE_NODE_LOGGING:-true}" LOGGING_DESTINATION="${KUBE_LOGGING_DESTINATION:-gcp}" # options: elasticsearch, gcp

SCHEDULER_TEST_ARGS="${SCHEDULER_TEST_ARGS:-} ${TEST_CLUSTER_API_CONTENT_TYPE}" KUBEPROXY_TEST_ARGS="${KUBEPROXY_TEST_ARGS:-} ${TEST_CLUSTER_API_CONTENT_TYPE}" <ins> # Historically fluentd was a manifest pod and then was migrated to DaemonSet. # To avoid situation during cluster upgrade when there are two instances # of fluentd running on a node, kubelet need to mark node on which # fluentd is not running as a manifest pod with appropriate label. # TODO(piosz): remove this in 1.8 NODE_LABELS="${KUBE_NODE_LABELS:-alpha.kubernetes.io/fluentd-ds-ready=true}" </ins> # Optional: Enable node logging. ENABLE_NODE_LOGGING="${KUBE_ENABLE_NODE_LOGGING:-true}" LOGGING_DESTINATION="${KUBE_LOGGING_DESTINATION:-gcp}" # options: elasticsearch, gcp

ObjectMeta: metav1.ObjectMeta{ Name: string(kl.nodeName), Labels: map[string]string{ <del> metav1.LabelHostname:       kl.hostname, metav1.LabelOS:             goruntime.GOOS, metav1.LabelArch:           goruntime.GOARCH, metav1.LabelFluentdDsReady: "true", </del> <ins> metav1.LabelHostname: kl.hostname, metav1.LabelOS:       goruntime.GOOS, metav1.LabelArch:     goruntime.GOARCH, </ins> }, }, Spec: v1.NodeSpec{