Change:
I have 3 identical nodes, 1 replication controller with 2 replicas of a pod. I simulate failure of one of the nodes to where pods have been scheduled by simply unplugging the node. Kubernetes realizes that the node has failed, but the pod gets not re-scheduled to the another available node. The documentation at says the following: and So is that a mistake in the documentation or a missing or a bug? I use v0.10.
That's definitely a bug, and something that used to work correctly. I will investigate today, and follow up on this bug. Thanks for the report. Brendan On Feb 5, 2015 8:11 AM, "peter-braun" wrote:
Actually, it's really a missing feature -- . I believe the last of the mechanisms we needed were ~last week. said he would start on it.
Is there a rudimentary solution that we could get in quickly, and then improve next week?
Just sent a simple PR with the fix.
Note that this issue is about a replication-controller set of pods. This isn't which is a broader "fix pods on failed nodes in general" issue.
Ah, a cache bug. Will take a look.
Re-opening this, because I reverted the relevant code in the replication controller with
I'll comment on the other thread later. Sorry I was out yesterday. I'm a bit free this weekend and should be able to work on the problem for at least one day.
Fixed by . Re-open if not fixed.
was merged a few days ago. It referenced , but not this issue.

Discussion:
var result []api.Pod for _, value := range pods { if api.PodSucceeded != value.Status.Phase && <del> api.PodFailed != value.Status.Phase { </del> <ins> api.PodFailed != value.Status.Phase && api.PodUnknown != value.Status.Phase { </ins> result = append(result, value) } }

return &node.Status, nil } <ins> func (p *PodCache) clearNodeStatus() { p.lock.Lock() defer p.lock.Unlock() p.currentNodes = map[objKey]api.NodeStatus{} } </ins> // TODO: once Host gets moved to spec, this can take a podSpec + metadata instead of an // entire pod? func (p *PodCache) updatePodStatus(pod *api.Pod) error {

// calling again, or risk having new info getting clobbered by delayed // old info. func (p *PodCache) UpdateAllContainers() { <ins> // TODO: this is silly, we should pro-actively update the pod status when // the API server makes changes. p.clearNodeStatus() </ins> ctx := api.NewContext() pods, err := p.pods.ListPods(ctx, labels.Everything()) if err != nil {

} } <ins> func TestPodUpdateAllContainersClearsNodeStatus(t *testing.T) { node := makeHealthyNode("machine", "1.2.3.5") pod1 := makePod(api.NamespaceDefault, "foo", "machine", "bar") pod2 := makePod(api.NamespaceDefault, "baz", "machine", "qux") config := podCacheTestConfig{ kubeletContainerInfo: api.PodStatus{ Info: api.PodInfo{"bar": api.ContainerStatus{}}}, nodes: []api.Node{*node}, pods:  []api.Pod{*pod1, *pod2}, } cache := config.Construct() if len(cache.currentNodes) != 0 { t.Errorf("unexpected node cache: %v", cache.currentNodes) } key := objKey{"", "machine"} cache.currentNodes[key] = makeUnhealthyNode("machine").Status cache.UpdateAllContainers() if len(cache.currentNodes) != 1 { t.Errorf("unexpected empty node cache: %v", cache.currentNodes) } if !reflect.DeepEqual(cache.currentNodes[key], node.Status) { t.Errorf("unexpected status:n%#vnexpected:n%#vn", cache.currentNodes[key], node.Status) } } </ins> func TestPodUpdateAllContainers(t *testing.T) { pod1 := makePod(api.NamespaceDefault, "foo", "machine", "bar") pod2 := makePod(api.NamespaceDefault, "baz", "machine", "qux")