Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When developing unit tests related to PDBs/Deletion, mutates pod resource via adding deletion timestamp, and this directly conflicts with checking resource version. What you expected to happen: No resource version mutation. How to reproduce it (as minimally and precisely as possible): I'm attempting to add preconditions to delete call in eviction API: Anything else we need to know?: Happens locally and in CI. Environment: Kubernetes version (use ): Head of master: Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
There are no sig labels on this issue. Please add a sig label by either: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. See the . The in method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
can you provide more details about what you're observing?
You can see an example here: Here's the relevant section of the tests: There really shouldn't be anything modifying the pod after creation until the delete request, thus I don't believe we should be hitting this precondition failure. Resource Version in precondition I believe to be correct, but when we contact etcd to verify, it's increased by one. It's not clear why this is happening, it appears to be a bug.
Okay, I've sorted this out. We have a conflict with 'graceful' deletion with this approach: This updates the object to add a deletion timestamp, thus the first call to eviction will always fail if we need to set ResourceVersion.

Related discussion:
text: } } <ins> func TestStoreGracefulDeleteWithResourceVersion(t *testing.T) { podA := &example.Pod{ ObjectMeta: metav1.ObjectMeta{Name: "foo"}, Spec:       example.PodSpec{NodeName: "machine"}, } testContext := genericapirequest.WithNamespace(genericapirequest.NewContext(), "test") destroyFunc, registry := NewTestGenericStoreRegistry(t) defer destroyFunc() defaultDeleteStrategy := testRESTStrategy{scheme, names.SimpleNameGenerator, true, false, true} registry.DeleteStrategy = testGracefulStrategy{defaultDeleteStrategy} // test failure condition _, _, err := registry.Delete(testContext, podA.Name, rest.ValidateAllObjectFunc, nil) if !errors.IsNotFound(err) { t.Errorf("Unexpected error: %v", err) } // create pod _, err = registry.Create(testContext, podA, rest.ValidateAllObjectFunc, &metav1.CreateOptions{}) if err != nil { t.Errorf("Unexpected error: %v", err) } // try to get a item which should be deleted obj, err := registry.Get(testContext, podA.Name, &metav1.GetOptions{}) if errors.IsNotFound(err) { t.Errorf("Unexpected error: %v", err) } accessor, err := meta.Accessor(obj) if err != nil { t.Errorf("Unexpected error: %v", err) } resourceVersion := accessor.GetResourceVersion() options := metav1.NewDeleteOptions(0) options.Preconditions = &metav1.Preconditions{ResourceVersion: &resourceVersion} // delete object _, wasDeleted, err := registry.Delete(testContext, podA.Name, rest.ValidateAllObjectFunc, options) if err != nil { t.Errorf("Unexpected error: %v", err) } if !wasDeleted { t.Errorf("unexpected, pod %s should have been deleted immediately", podA.Name) } // try to get a item which should be deleted _, err = registry.Get(testContext, podA.Name, &metav1.GetOptions{}) if !errors.IsNotFound(err) { t.Errorf("Unexpected error: %v", err) } } </ins> // TestGracefulStoreCanDeleteIfExistingGracePeriodZero tests recovery from // race condition where the graceful delete is unable to complete // in prior operation, but the pod remains with deletion timestamp