Code change:
pair{FROM, FROM}: true, } <del> pod := func(name, check string) *api.Pod { return &api.Pod{ </del> <ins> pod := func(name, check string, final bool) *api.Pod { p := &api.Pod{ </ins> ObjectMeta: api.ObjectMeta{ Name:   name, Labels: map[string]string{"check": check}, }, } <ins> if final { p.Labels["final"] = "true" } return p } deletePod := func(p *api.Pod) bool { return p.Labels["final"] == "true" </ins> } tests := []func(string){ func(name string) { name = "a-" + name <del> source.Add(pod(name, FROM)) source.Modify(pod(name, TO)) </del> <ins> source.Add(pod(name, FROM, false)) source.Modify(pod(name, TO, true)) </ins> }, func(name string) { name = "b-" + name <del> source.Add(pod(name, FROM)) source.ModifyDropWatch(pod(name, TO)) </del> <ins> source.Add(pod(name, FROM, false)) source.ModifyDropWatch(pod(name, TO, true)) </ins> }, func(name string) { name = "c-" + name <del> source.AddDropWatch(pod(name, FROM)) source.Modify(pod(name, ADD_MISSED)) source.Modify(pod(name, TO)) </del> <ins> source.AddDropWatch(pod(name, FROM, false)) source.Modify(pod(name, ADD_MISSED, false)) source.Modify(pod(name, TO, true)) </ins> }, func(name string) { name = "d-" + name <del> source.Add(pod(name, FROM)) </del> <ins> source.Add(pod(name, FROM, true)) </ins> }, }

Related discussion:
text: I got this by running the following against commit : There have been previous attempts to fix this: It appears it's still an issue :cry: We're also seeing it in OpenShift (), although our code hasn't pulled in the fix from quite yet.
cc
What go version are you using? Previously this test has been catching legit bugs, so I'm not sure about the "flaky" moniker.
go 1.5.1. I noticed you had a comment in another issue or PR about this possibly being a bug in Go itself (or did I misread)?
I doubt it's a bug in go. (go1.5 had an unrelated bug, maybe that's what you're thinking of.) We haven't actually flipped 1.5 on so I may sit on this for a bit.
I instrumented this code, and this is being caused by duplicate deletions causing the wait group to decrement below 0. Are duplicate deletions a bug? Or should the wait group be guarded against that?
ping If duplicate deletes are not a bug, I can put together a quick PR to guard against that. If they are a bug, I could try to track it down sometime this week (even thought I'm not that familiar with this code) if no one else is going to look into it.
Duplicate deletes are a bug. (assuming that the test harness isn't actually issuing multiple deletes for the same item.)
That would be great if you could take this, since I won't have time.
So it would be an error in the test if was called multiple times for the same pod? Or should the framework be de-duping that?
IMO, that's a test error. ...If the test is doing that because the framework dup'd an update or an add, then it's a framework error. (adds and updates shouldn't be duped, either) , krousey wrote:
from what I gather, this test is just trying to delete after each pod reaches it's final intended state. This code is deleting on every event, including intermediate ones. In particular, this test case is occasionally causing duplicate deletes to be issued: Sometimes it catches the transition from FROM to ADDMISSED, and then gets ADDMISSED to TO. I think the fix is to tell the test to only issue the delete when the final state is TO.