Repository: kubernetes
Issue: Unexpected annotations merge instead of replace with server-side PATCH using pure API
Labels: kind/bug,sig/api-machinery,triage/accepted

### What happened?

This looks to me like a bug. Whenever annotations are managed by the same field manager the server-side PATCH that includes annotation change, the changes are merged instead of complete replacement of the annotations. 

### What did you expect to happen?

Annotations to be replaced when PATCHed on the server-side by the same field manager as the one that introduced them.
If the behavior is correct, what then is the mechanism to replace the entire mapping with a *server-side* PATCH?

### How can we reproduce it (as minimally and precisely as possible)?

The test case to be provided

### Anything else we need to know?

Here is the conversation capture:

In the below example there is a namespace with annotations. Namespace is added with annotations "a: x" and the PATCH is supposed to replace them with "b: y" (same field manager). Instead they are merged: `"annotations":{"a":"x","b":"y"}`. 

Send:

```
POST https://192.168.121.2:8443/api/v1/namespaces?fieldManager=kubernator&fieldValidation=Warn

Accept: application/json
User-Agent: OpenAPI-Generator/28.1.0/python
Content-Type: application/json

{"apiVersion": "v1", "kind": "Namespace", "metadata": {"name": "ns1", "annotations": {"a": "x"}}}
```

Recv:
```

201 Created

Audit-Id: 53d3660e-11f8-4acb-8614-66ee7ef69451
Cache-Control: no-cache, private
Content-Type: application/json
X-Kubernetes-Pf-Flowschema-Uid: 5c8f085c-4351-4402-a445-6dd0bd78700c
X-Kubernetes-Pf-Prioritylevel-Uid: 48c26814-1353-4e9f-b65b-5d5542e5c295
Date: Sun, 07 Jan 2024 23:23:03 GMT
Content-Length: 566

{"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ns1","uid":"0a95b535-34ad-48a4-9fe3-8812d09b4343","resourceVersion":"276","creationTimestamp":"2024-01-07T23:23:03Z","labels":{"kubernetes.io/metadata.name":"ns1"},"annotations":{"a":"x"},"managedFields":[{"manager":"kubernator","operation":"Update","apiVersion":"v1","time":"2024-01-07T23:23:03Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:a":{}},"f:labels":{".":{},"f:kubernetes.io/metadata.name":{}}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}}
```

Send:
```
PATCH https://192.168.121.2:8443/api/v1/namespaces/ns1?dryRun=All&fieldManager=kubernator&fieldValidation=Warn&force=True

Accept: application/json
Content-Type: application/apply-patch+yaml
User-Agent: OpenAPI-Generator/28.1.0/python

{"apiVersion": "v1", "kind": "Namespace", "metadata": {"name": "ns1", "annotations": {"b": "y"}}}
```

Recv:
```
200 OK

Audit-Id: 8caa380e-e3cf-4379-8188-12b1b73304c9
Cache-Control: no-cache, private
Content-Type: application/json
X-Kubernetes-Pf-Flowschema-Uid: 5c8f085c-4351-4402-a445-6dd0bd78700c
X-Kubernetes-Pf-Prioritylevel-Uid: 48c26814-1353-4e9f-b65b-5d5542e5c295
Date: Sun, 07 Jan 2024 23:23:05 GMT
Content-Length: 746

{"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ns1","uid":"0a95b535-34ad-48a4-9fe3-8812d09b4343","resourceVersion":"276","creationTimestamp":"2024-01-07T23:23:03Z","labels":{"kubernetes.io/metadata.name":"ns1"},"annotations":{"a":"x","b":"y"},"managedFields":[{"manager":"kubernator","operation":"Apply","apiVersion":"v1","time":"2024-01-07T23:23:05Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:b":{}}}}},{"manager":"kubernator","operation":"Update","apiVersion":"v1","time":"2024-01-07T23:23:03Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:a":{}},"f:labels":{".":{},"f:kubernetes.io/metadata.name":{}}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}}
```

### Kubernetes version

1.28.4


### Cloud provider

N/A

### OS version

N/A

### Install tools

<details>

</details>


### Container runtime (CRI) and version (if applicable)

<details>

</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>

</details>