Repository: kubernetes
Issue: Don't schedule all addons on the first node
Labels: sig/scheduling,lifecycle/frozen,needs-priority,needs-triage

This used to not happen but it does now. The following pods landed on the first node that became ready on kube-up (jenkins-e2e-minion-50tl):

```
14:28:59 Feb 23 14:28:53.406: INFO: downwardapi-volume-8c06635c-da7c-11e5-b51c-42010af01555  jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:27:11 -0800 PST  }]
14:28:59 Feb 23 14:28:53.406: INFO: nettest-fs46h                                            jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:28:51 -0800 PST  }]
14:28:59 Feb 23 14:28:53.406: INFO: nodeport-test-mnxrw                                      jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:26:46 -0800 PST  }]
14:28:59 Feb 23 14:28:53.406: INFO: service1-tihva                                           jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:27:08 -0800 PST  }]
14:28:59 Feb 23 14:28:53.406: INFO: service2-sn40n                                           jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:27:30 -0800 PST  }]
14:28:59 Feb 23 14:28:53.406: INFO: elasticsearch-logging-v1-0r22z                           jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:25:40 -0800 PST  }]
14:28:59 Feb 23 14:28:53.406: INFO: elasticsearch-logging-v1-9rbre                           jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:25:40 -0800 PST  }]
14:28:59 Feb 23 14:28:53.406: INFO: fluentd-elasticsearch-jenkins-e2e-minion-50tl            jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:25:39 -0800 PST  }]
14:28:59 Feb 23 14:28:53.407: INFO: heapster-v14-g6e8b                                       jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:24:51 -0800 PST  }]
14:28:59 Feb 23 14:28:53.407: INFO: kibana-logging-v1-omtbg                                  jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:26:02 -0800 PST  }]
14:28:59 Feb 23 14:28:53.407: INFO: kube-dns-v10-jq6pf                                       jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:25:56 -0800 PST  }]
14:28:59 Feb 23 14:28:53.407: INFO: kube-proxy-jenkins-e2e-minion-50tl                       jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:24:03 -0800 PST  }]
14:28:59 Feb 23 14:28:53.407: INFO: kubernetes-dashboard-v0.1.0-k1hi2                        jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:24:25 -0800 PST  }]
14:28:59 Feb 23 14:28:53.407: INFO: l7-lb-controller-v0.5.2-cnntu                            jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:25:47 -0800 PST  }]
14:28:59 Feb 23 14:28:53.407: INFO: monitoring-influxdb-grafana-v3-rnb9q                     jenkins-e2e-minion-50tl  Running         [{Ready True 0001-01-01 00:00:00 +0000 UTC 2016-02-23 14:25:53 -0800 PST  }]
```

Scheduler logs:
https://pantheon.corp.google.com/m/cloudstorage/b/kubernetes-jenkins/o/logs/kubernetes-e2e-gce/12147/artifacts/104.197.127.170%3A22-kube-scheduler.log

Test logs: http://kubekins.dls.corp.google.com/view/Critical%20Builds/job/kubernetes-e2e-gce/12147/consoleFull

I'm marking this kind/flake because it leads to a lot of unpredictablity during e2es from a single pod/node, I suspect some addons are bluffing about their limits (see https://github.com/kubernetes/kubernetes/issues/21820#issuecomment-188546116).