Change:
When continuously re-applying the same object via a controller or kubectl with SSA the resourceVersion increases every time. I expected the resourceVersion to stay the same if the client always sends the same object via SSA, especially if the resulting object doesn't change (apart from resourceVersion & managedFields, but these are managed by the apiserver). Apply this CRD: <details</summary</detailsApply this CR: <details</summary</detailsGet state of the CR after first apply Apply the CR again Get state of the CR after second apply Diff =The resourceVersion and the "time" field of the managedField entry of the "test-manager" increases. The use case we have is a lot more complex than this simple scenario. The tl;dr is that we want to implement caching for our SSA calls, so that we don't have SSA calls on every reconcile of our controller if nothing actually changes. This is relatively hard to do if the resourceVersion increases after every apply. I took a closer look at the APIserver and ~ the following happens: "incoming" SSA patch is applied over the live object from etcd this leads to a change because has which means the field will be dropped Because of that change the "time" field in the managedField entry gets set to "time.Now()" later during admission the namespace field gets defaulted again (I also tried this with a mutating webhook, same result) On a high-level: client always applies the same object defaulting always results in the same object resourceVersion and managedFields[].time continuously go up I think there are many similar cases that would lead to this behavior. Basically whenever the admission chain mutates a field previously set via SSA. Notes: This only happens if there is 1s between each apply, otherwise the managedField[].time is identical and then the resourceVersion stays the same (because we then don't write to etcd) <detailsI also tried this with client-go v0.30 and kube-apiserver v1.30. Same result. </details<detailsRunning this natively on Apple Silicon M2. But can't imagine this makes a difference. Also got the same results in Docker Desktop on Mac. </details
(cc might be interesting for you)
/sig api-machinery
It looks less a bug but more of a intended behavior to me. /cc /triage accepted
Bypassing the writes triggered by noop applies would break storage version migration (), so we're committed to supporting this behavior as a feature.
I am confused. Are you saying this is intended behavior for a NOP SSA apply to increment resourceVersion? This seems to not actually happen in all cases, only for certain objects/edge cases (see ). For example, try this: No resourceVersion is incremented above). Surprisingly I couldn't find any clear statement that SSA doesn't increment the resource version, but does say "a no-op apply is not very much more work for the API server than an extra GET" which seems impossible if resourceVersion is incremented
is probably only updating it because either the fieldManager changes or in the patch body the apiVersion is different (or both, perhaps)?
We need this in storage version migration to make sure we correctly handles the CRD schema changes. See Joe's comments here - If it helps to understand this better, we have tests to validate this.
I may have misspoke. When storage versions changes, a NOP SSA apply will trigger a resourceVersion bump. If the storage version is unchanged and an update would be a NOP for etcd, I believe we short-circuit the write (?). This might explain the configMap case.
For the reported issue: We've seen cases like this before where the time gets bumped causing a NOP to become a write. This could be improved. Note that this isn't technically a bug because the API server is allowed to bump the resourceVersion for any write (the discussion about storage version changes causing this is a good example of where this can happen in a way that the caller wouldn't expect). But I'd like to see the etcd write optimized away for cases like this.
Sounds similar to perhaps which had some investigation. FWIW as a controller author my expectation is that repeatedly applying does not repeatedly trigger resourceVersion increments. It's fine if it happens rarely (storage version change, etc) but if it happens consistently that means our own writes re-trigger reconciliation and we have an infinite loop of writes. AFAIK all the marketing around writing SSA controllers has been that you just apply what you want and don't try to do a get+diff+apply. Even if we did want to do that, it doesn't really work well currently (). If we aren't able to rely on NOP writes (usually) being NOPs, I don't see how we are supposed to use SSA to write a controller at all?
This is a really good point. I agree.
looks like it was intended fix this sort of problem. I'm looking into why it doesn't work for this case.
doesn't work for CRs since the managed fields are inside of a unstructured object and are not typed as (for the slow path equality check in IgnoreManagedFieldsTimestampsTransformer).
is a draft proposal for a fix. Feedback welcome on how to make this fix cleaner. I'll test carefully before moving it out of draft.

Discussion:
"k8s.io/apimachinery/pkg/api/equality" "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" <ins> "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" </ins> "k8s.io/apimachinery/pkg/conversion" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apiserver/pkg/endpoints/metrics"

} var eqs = equality.Semantic.Copy() <del> err := eqs.AddFunc( </del> <ins> err := eqs.AddFuncs( </ins> func(a, b metav1.ManagedFieldsEntry) bool { // Two objects' managed fields are equivalent if, ignoring timestamp, //	the objects are deeply equal.

b.Time = nil return reflect.DeepEqual(a, b) }, <ins> func(a, b unstructured.Unstructured) bool { // Check if the managed fields are equal by converting to structured types and leveraging the above // function, then, ignoring the managed fields, equality check the rest of the unstructured data. if !avoidTimestampEqualities.DeepEqual(a.GetManagedFields(), b.GetManagedFields()) { return false } return equalIgnoringValueAtPath(a.Object, b.Object, []string{"metadata", "managedFields"}) }, </ins> ) if err != nil {

return avoidTimestampEqualities } <ins> func equalIgnoringValueAtPath(a, b any, path []string) bool { if len(path) == 0 { // found the value to ignore return true } aMap, aOk := a.(map[string]any) bMap, bOk := b.(map[string]any) if !aOk || !bOk { // Can't traverse into non-maps, ignore return true } if len(aMap) != len(bMap) { return false } pathHead := path[0] for k, aVal := range aMap { bVal, ok := bMap[k] if !ok { return false } if k == pathHead { if !equalIgnoringValueAtPath(aVal, bVal, path[1:]) { return false } } else if !avoidTimestampEqualities.DeepEqual(aVal, bVal) { return false } } return true } </ins> // IgnoreManagedFieldsTimestampsTransformer reverts timestamp updates // if the non-managed parts of the object are equivalent func IgnoreManagedFieldsTimestampsTransformer(

<ins> /* Copyright 2024 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package fieldmanager import "testing" func TestEqualIgnoringFieldValueAtPath(t *testing.T) { cases := []struct { name string a, b map[string]any want bool }{ { name: "identical objects", a: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field": "value", }, }, b: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field": "value", }, }, want: true, }, { name: "different metadata label value", a: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field": "value", }, }, b: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "prod"}, "managedFields": []any{}, }, "spec": map[string]any{ "field": "value", }, }, want: false, }, { name: "different spec value", a: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field": "value1", }, }, b: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field": "value2", }, }, want: false, }, { name: "extra spec field", a: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field":      "value1", "otherField": "other", }, }, b: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field": "value1", }, }, want: false, }, { name: "different spec field", a: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field": "value1", }, }, b: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{}, }, "spec": map[string]any{ "field":      "value1", "otherField": "other", }, }, want: false, }, { name: "different managed fields should be ignored", a: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{map[string]any{"manager": "client1"}}, }, "spec": map[string]any{ "field": "value", }, }, b: map[string]any{ "metadata": map[string]any{ "labels":        map[string]any{"env": "dev"}, "managedFields": []any{map[string]any{"manager": "client2"}}, }, "spec": map[string]any{ "field": "value", }, }, want: true, }, { name: "wrong type for metadata result in differences being ignored", a: map[string]any{ "metadata": []any{ "something", }, "spec": map[string]any{ "field": "value", }, }, b: map[string]any{ "metadata": []any{ "something else", }, "spec": map[string]any{ "field": "value", }, }, want: true, }, } path := []string{"metadata", "managedFields"} for _, c := range cases { t.Run(c.name, func(t *testing.T) { actual := equalIgnoringValueAtPath(c.a, c.b, path) if actual != c.want { t.Error("Expected equality check to return ", c.want, ", but got ", actual) } // Check that equality is commutative actual = equalIgnoringValueAtPath(c.b, c.a, path) if actual != c.want { t.Error("Expected equality check to return ", c.want, ", but got ", actual) } }) } } </ins>

"testing" "time" <ins> "k8s.io/apimachinery/pkg/api/meta" "k8s.io/apimachinery/pkg/runtime/schema" </ins> "k8s.io/apimachinery/pkg/util/wait" genericapirequest "k8s.io/apiserver/pkg/endpoints/request"

t.Fatalf("Expecting to get one conflict when a different applier updates existing list item, got: %v", status.Status().Details.Causes) } } <ins> func TestNoOpApplyWithDefaultsSameResourceVersionCRD(t *testing.T) { // https://github.com/kubernetes/kubernetes/issues/124605 server, err := apiservertesting.StartTestServer(t, apiservertesting.NewDefaultTestServerOptions(), nil, framework.SharedEtcd()) if err != nil { t.Fatal(err) } defer server.TearDownFn() config := server.ClientConfig apiExtensionClient, err := clientset.NewForConfig(config) if err != nil { t.Fatal(err) } dynamicClient, err := dynamic.NewForConfig(config) if err != nil { t.Fatal(err) } noxuDefinition := fixtures.NewNoxuV1CustomResourceDefinition(apiextensionsv1.ClusterScoped) err = json.Unmarshal([]byte(`{ "openAPIV3Schema": { "type": "object", "properties": { "spec": { "type": "object", "properties": { "infrastructureRef": { "type": "object", "properties": { "name": { "type": "string" }, "namespace": { "type": "string", "default": "default-namespace" } }, "x-kubernetes-map-type": "atomic" } } } } } }`), &noxuDefinition.Spec.Versions[0].Schema) if err != nil { t.Fatal(err) } noxuDefinition, err = fixtures.CreateNewV1CustomResourceDefinition(noxuDefinition, apiExtensionClient, dynamicClient) if err != nil { t.Fatal(err) } group := noxuDefinition.Spec.Group kind := noxuDefinition.Spec.Names.Kind resource := noxuDefinition.Spec.Names.Plural version := noxuDefinition.Spec.Versions[0].Name apiVersion := noxuDefinition.Spec.Group + "/" + version gvr := schema.GroupVersionResource{Group: group, Version: version, Resource: resource} name := "mytest" // fieldWithDefault will be defaulted applyConfiguration := &unstructured.Unstructured{ Object: map[string]interface{}{ "apiVersion": apiVersion, "kind":       kind, "metadata": map[string]interface{}{ "name": name, }, "spec": map[string]interface{}{"infrastructureRef": map[string]interface{}{"name": "infrastructure-machine-1n"}}, }, } created, err := dynamicClient.Resource(gvr).Apply(context.TODO(), name, applyConfiguration, metav1.ApplyOptions{FieldManager: "apply_test"}) if err != nil { t.Fatalf("failed to create custom resource with apply: %v:n%v", err, created) } createdAccessor, err := meta.Accessor(created) if err != nil { t.Fatalf("Failed to get meta accessor for updated object: %v", err) } // Sleep for one second to make sure that the times of each update operation is different. time.Sleep(1 * time.Second) updated, err := dynamicClient.Resource(gvr).Apply(context.TODO(), name, applyConfiguration, metav1.ApplyOptions{FieldManager: "apply_test"}) if err != nil { t.Fatalf("failed to update custom resource with apply: %v:n%v", err, updated) } updatedAccessor, err := meta.Accessor(updated) if err != nil { t.Fatalf("Failed to get meta accessor for updated object: %v", err) } if createdAccessor.GetResourceVersion() != updatedAccessor.GetResourceVersion() { t.Fatalf("Expected same resource version to be %v but got: %vnold object:n%vnnew object:n%v", createdAccessor.GetResourceVersion(), updatedAccessor.GetResourceVersion(), created, updated, ) } } </ins>