Change:
https://k8s- is failing to run the follow up e2es because the ip-masq-agent daemonset is scheduling pods that get stuck in a pending state on 1.5 nodes. This short-circuits the logic for the skewed e2es, and they never run. from https://k8s-
So I see that in 1.7.0 we start setting the non-masq-cidr to 0/0 gce/config-"0.0.0.0/0" We set the addon if non-masq-cidr is 0/0 if [[ "${NONMASQUERADECIDR:-}" == "0.0.0.0/0" ]]; then setup-addon-manifests "addons" "ip-masq-agent" fi Also the agent cannot run on nodes prior to 1.7 so this seems to be correct. Once the nodes are upgraded to 1.7 then the ip-masq agent should be running and should get out of the pending state. I think this is just a general problem with the way addons are handled. I may want to enable a new addon when a master is upgraded to 1.7 by default. That addon may be a DaemonSet that wants to run on nodes meeting a certain criteria. That criteria may not be met in 1.5 but may be met when they are updated to 1.7. A few things that come to mind to fix this: -Fix the test's expectations to allow this until the nodes are updated to 1.7. -rollback the PR that made this the default: -set NONMASQUERADECIDR=10.0.0.0/8 in the test properties so that the ip-masq-agent doesn't get .
It seems like the most conservative way forward is to revert the PR (). Customers who want to use the feature can set the variable () during to be able to get the feature. What do you guys think?
I think I asked the similar question related to ip-masq-agent, but my question was dropped. At this stage, I think reverting the preferred fix.
ip-masq-agent was a daemonset introduced in 1.7 which I questioned at The problem is that after master is upgraded to 1.7, addons is trying to schedule ip-masq-agent daemonset to every node no matter which Kubelet version the node is on. The problem is that ip-masq-agent can only run on node with 1.7 release.
Even if we fix the test it doesn't fix GKE or any other environment where the master is upgraded to 1.7 and the nodes are < 1.7. I think the real problem here is that there is no distinction between staging an addon and enabling an addon. Essentially staging = running. So we end up adding labels to nodes as a way of keeping the DaemonSet disabled until those labels are to nodes. While that works fine for addons that are DaemonDets, it doesn't work for deployments and replica sets. I feel like this is going to happen again regardless of what we do here with ip-masq-agent. It also creates a secondary step. Who applies those addons? What if they forget? what if it's deleted?
Fixed by (the formatting wasn't quite right in the IR to auto-close this issue).
got reverted as it broke tests.

Discussion:
volumeMounts: - name: config mountPath: /etc/config <ins> nodeSelector: beta.kubernetes.io/masq-agent-ds-ready: "true" </ins> volumes: - name: config configMap:

# TODO(piosz): remove this in 1.8 NODE_LABELS="${KUBE_NODE_LABELS:-beta.kubernetes.io/fluentd-ds-ready=true}" <ins> # To avoid running the DaemonSet on older version make sure the ip-masq-agent # only runs when the readiness label is set. NODE_LABELS="${NODE_LABELS},beta.kubernetes.io/masq-agent-ds-ready=true" </ins> # To avoid running Calico on a node that is not configured appropriately, # label each Node so that the DaemonSet can run the Pods only on ready Nodes. if [[ ${NETWORK_POLICY_PROVIDER:-} == "calico" ]]; then

# Network Policy plugin specific settings. NETWORK_POLICY_PROVIDER="${NETWORK_POLICY_PROVIDER:-none}" # calico <ins> # Should the kubelet configure egress masquerade (old way) or let a daemonset do it? NON_MASQUERADE_CIDR="0.0.0.0/0" </ins> # How should the kubelet configure hairpin mode? HAIRPIN_MODE="${HAIRPIN_MODE:-promiscuous-bridge}" # promiscuous-bridge, hairpin-veth, none