Repository: kubernetes/kubernetes (language: go)

Issue:
Dueling writes to extension-apiserver-authentication configmap during 1.31 → 1.32.0-rc.1 upgrade
### What happened?

We had an alert firing for high memory usage for 1.32. At the time of increase we changed the churn to create at v1.31 (So update to v1.32). Previously it was creating at v1.30 and updating to v1.31

We can see some etcd are using more memory than others
```
kubx-etcd-02          etcd-csprq8020q6fal7635s0-6zxcz72w5v                         24m          420Mi           
kubx-etcd-02          etcd-csprq8020q6fal7635s0-hm5px7dkc5                         19m          416Mi           
kubx-etcd-02          etcd-csprq8020q6fal7635s0-jhmsq75wc5                         60m          566Mi  

...

kubx-etcd-04          etcd-cssndj620tea3foqjl5g-5qqscgqp2g                         28m          101Mi           
kubx-etcd-04          etcd-cssndj620tea3foqjl5g-c45kz2lprz                         23m          117Mi           
kubx-etcd-04          etcd-cssndj620tea3foqjl5g-wsbwcb9fp8                         50m          103Mi 
```

### What did you expect to happen?

When a cluster updating to 1.32 from 1.31, we expected (roughly) th same etcd memory usage as 1.31

### How can we reproduce it (as minimally and precisely as possible)?

- Deploy a 1.31 cluster
- upgrade to 1.32
- Observer memory usage on etcd

### Anything else we need to know?

A cluster that was created at v1.32 has an etcd `container_memory_working_set_bytes` of around 130 MB.
A v1.31 cluster is also around 130 MB. After the upgrade to v1.32 it increased to around 450 MB (Caused by a short increase in etcd_mvcc_db_total_size_in_use_in_bytes to 430 MB). `etcd_mvcc_db_total_size_in_bytes` permanently increased to 450 MB at upgrade time too. Over time some of the etcds saw an increase in `container_memory_working_set_bytes` to 600MB, but `etcd_mvcc_db_total_size_in_bytes` did not change.

When it was defragged the `etcd_mvcc_db_total_size_in_bytes` dropped to 11 MB (Which is just above the value of `etcd_mvcc_db_total_size_in_use_in_bytes` but `container_memory_working_set_bytes` only dropped to around 450MB.

After further investigation it requires a defrag, followed by restart of etcd pods to return the memory usage to normal after an update to v1.32.

### Kubernetes version

<details>

```console
$ kubectl version
Client Version: v1.31.1
Kustomize Version: v5.4.2
Server Version: v1.32.0-beta.0+IKS
```

</details>


### Cloud provider

<details>
IBM Cloud
</details>


### OS version

<details>

```console
# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.1 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo

$ uname -a
Linux test-csug3jq20lvmbeb4nq2g-standard132-default-00000172 6.8.0-48-generic #48-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 14:04:52 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
```

</details>


### Install tools

<details>

</details>


### Container runtime (CRI) and version (if applicable)

<details>

</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>

</details>



Discussion / hints:
/sig etcd
/sig api-machinery
- v1.31 is using etcd 3.5.15-0. https://github.com/kubernetes/kubernetes/blob/v1.31.0/cmd/kubeadm/app/constants/constants.go#L325
- master is using etcd 3.5.16-0

Any other diff in file `/etc/kubernetes/manifests/etcd.yaml` after the upgrade?

/cc 
I don't think this is caused by an etcd issue. When the apiservers restart I see a brief large spike in `etcd_mvcc_db_total_size_in_use_in_bytes` to over 400 MB:
<img width="2030" alt="Screenshot 2024-11-19 at 10 53 13" src="https://github.com/user-attachments/assets/86391669-11e1-43e0-9313-ab030e3391d6">

In apiserver logs after the upgrade I see thousands of these messages about updates to ConfigMap `extension-apiserver-authentication` - I'm assuming this causes a large build up of key history which causes etcd to expand.

```
I1118 08:48:50.943402       1 cluster_authentication_trust_controller.go:170] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1118 08:48:50.954528       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="10.328526ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="9a802596-81ce-4ba3-a8a8-4d6e682a287e" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="9.721029ms" resp=200
I1118 08:48:50.976156       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="19.169336ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="a2847ef5-3ebc-4da8-8c4b-7ccc395dfdf8" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="18.511827ms" resp=200
I1118 08:48:51.000467       1 cluster_authentication_trust_controller.go:170] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1118 08:48:51.009391       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="8.392264ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="b50c7394-43f6-46bb-821e-0271a99cea69" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="8.00145ms" resp=200
I1118 08:48:51.028497       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="17.407801ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="c5f8098d-7368-49f0-85ee-7fbbe4a3066d" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="16.73501ms" resp=200
I1118 08:48:51.053583       1 cluster_authentication_trust_controller.go:170] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1118 08:48:51.064116       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="9.540177ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="e3175bbb-c99d-431b-bbba-842aa5474d07" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="8.599835ms" resp=200
I1118 08:48:51.091760       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="25.822272ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="0db17246-63cc-41f6-871d-15f479a523bd" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="25.22635ms" resp=200
I1118 08:48:51.141871       1 cluster_authentication_trust_controller.go:170] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1118 08:48:51.151858       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="9.35584ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="4c5f7059-900c-4167-884e-13575b9982cf" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="8.5531ms" resp=200
I1118 08:48:51.182038       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="28.603253ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="ca116057-2fd6-4d4d-ad40-9cea1a22b489" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="28.020921ms" resp=200
I1118 08:48:51.210561       1 cluster_authentication_trust_controller.go:170] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1118 08:48:51.219817       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="8.505724ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="40f0a95e-6ea4-427f-b3d9-997fa8922eac" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="7.900848ms" resp=200
I1118 08:48:51.253855       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="32.764828ms" userAgent="kube-apiserver/v1.32.0 (linux/amd64) kubernetes/2da9c36" audit-ID="ea864834-e147-461e-b48d-b3b367f662fb" srcIP="[::1]:47446" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="31.982292ms" resp=200
I1118 08:48:51.276608       1 cluster_authentication_trust_controller.go:170] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
```

It looks to me like we must be stuck a loop where this is never evaluating to true even though we keep updating it?
https://github.com/kubernetes/kubernetes/blob/master/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller.go#L166
Logs showing the high number of updates on restart of apiserver -> 
[IKS_v132_restart.zip](https://github.com/user-attachments/files/17931253/IKS_v132_restart.zip)

Can you try v1.32.0-rc.0 if possible to include some fixes and reverts? 
@pacoxu yes, we will try v1.32.0-rc.0 next week after the US Thanksgiving holiday.
cc @mengjiao-liu @chaochn47 
@pacoxu I tried with `v1.32.0-rc.0` and we see same behaviour - jump in memory and large number of updates to `kube-system configmaps/extension-apiserver-authentication`
https://github.com/kubernetes/kubernetes/pull/115834 the PR that may be related when I search using [`extension-apiserver-authentication milestone:v1.32`](https://github.com/kubernetes/kubernetes/pulls?q=extension-apiserver-authentication+milestone%3Av1.32)

/cc @liggitt @stlaz
@stlaz if old servers are still active, will they duel with a new server to set this key in the configmap?

can you dig into the behavior of a single 1.32 apiserver in steady state, and when combined with a 1.31 server that doesn't know about the uid configmap field?
/assign @stlaz 
cc @fsmunoz 
Thank you @pacoxu  @liggitt , following this closely, even considering the possible release date impact.
@liggitt good shout - I checked the logs during upgrade, and can indeed see similar messages about it updating the ConfigMap in the remaning v1.31 apiserver logs:

```I1204 10:05:15.001765       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:05:15.007256       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="5.150721ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="b35cc293-09f6-4900-b74c-86d9dc9b076a" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="4.900009ms" resp=200
I1204 10:05:15.017101       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="9.336193ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="03d113c8-a2f3-40d0-ba4c-e299a0ac80de" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="9.112163ms" resp=200
I1204 10:05:15.033367       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:05:15.038519       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="4.875444ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="4efd7566-9b12-43a4-aecc-3612bd4c4080" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="4.596182ms" resp=200
I1204 10:05:15.046208       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="7.031578ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="ea6bd372-5025-4731-b7cf-715c3ae8a385" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="6.790984ms" resp=200
I1204 10:05:15.064069       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:05:15.069587       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="5.239949ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="a4e54a7e-fc55-4939-85e9-85136d669659" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="4.939084ms" resp=200
I1204 10:05:15.077954       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="7.630473ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="344dfb85-fd63-4bb3-8913-ddc28b77d3e2" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="7.3828ms" resp=200
I1204 10:05:15.096316       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:05:15.101305       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="4.735979ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="c029331b-1a3c-445d-b09b-a81b38e4d8e4" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="4.433505ms" resp=200
I1204 10:05:15.108865       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="7.011051ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="1f214b5f-51cd-4688-88f4-721030eee19e" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="6.74948ms" resp=200
I1204 10:05:15.126585       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:05:15.131665       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="4.792554ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="61c79f29-5c0d-4598-9f65-e81b593314fd" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="4.468172ms" resp=200
I1204 10:05:15.139461       1 httplog.go:134] "HTTP" verb="PUT" URI="/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication" latency="7.214051ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="940b87a6-2b43-4c3d-b0b9-88e006294890" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="6.987153ms" resp=200
I1204 10:05:15.156627       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:05:15.161588       1 httplog.go:134] "HTTP" verb="GET" URI="/api/v1/namespaces/kube-system" latency="4.678607ms" userAgent="kube-apiserver/v1.31.3+IKS (linux/amd64) kubernetes/8cc6b5f" audit-ID="da7d91ca-cf58-4756-b1cd-f3d8813f7b38" srcIP="[::1]:60266" apf_pl="exempt" apf_fs="exempt" apf_iseats=1 apf_fseats=0 apf_additionalLatency="0s" apf_execution_time="4.415127ms" resp=200
```

Once the last v1.31 apiserver has been deleted and replaced by v1.32 apiserver the messages in the v1.32 apiserver logs stop.

(We have 3 apiserver replicas, and they get upgraded 1 by 1, so do have v1.31 & v1.32 apiservers co-existing for a few minutes)
> if old servers are still active, will they duel with a new server to set this key in the configmap?

I can confirm that such a duel indeed occurs :face_exhaling: 

My reproducer:
1. build a `kind` image from master
2. `kind create cluster --config=/home/standa/kind-configs/ha-cluster.yaml --image=kindest/node:latest` - I'll add the config used below
3. `docker exec -ti kind-control-plane sed -i 's#image: .*$#image: registry.k8s.io/kube-apiserver:v1.31.0#' /etc/kubernetes/manifests/kube-apiserver.yaml` to replace one of the kube-apiservers with an old version.

After kubelet picks up the changes and restarts the KAS, the `kube-system/extension-apiserver-authentication` does get modified in a loop and the new instance loop-logs:
```
I1204 10:37:49.678660       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:37:49.710061       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:37:49.730578       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
I1204 10:37:49.757051       1 cluster_authentication_trust_controller.go:168] writing updated authentication info to  kube-system configmaps/extension-apiserver-authentication
```

I'll have to think a bit on how to fix the issue.

`ha-cluster.yaml`:
```
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- &control-plane-role
  role: control-plane
  kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    apiServer:
        extraArgs:
          v: "2"
- *control-plane-role
- *control-plane-role
```
Our priority is to restore 1.32 to stable behavior as quickly and low risk as possible

The immediate options that come to mind:
1. Full revert
2. Put a feature gate around the surface area (command-line options to add uid headers, publishing uid headers to and consuming uid headers from the configmap) and mark it alpha / disabled by default in 1.32

Then in 1.33 we can work on making the configmap publishing controller tolerate unknown keys so it won't duel, which would make it safe to re-enable in 1.34

Fix patch:
diff --git a/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller.go b/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller.go
index 12e6b250e0462..3c86b026f5920 100644
--- a/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller.go
+++ b/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller.go
@@ -35,7 +35,9 @@ import (
 	"k8s.io/apimachinery/pkg/util/sets"
 	"k8s.io/apimachinery/pkg/util/wait"
 	"k8s.io/apiserver/pkg/authentication/request/headerrequest"
+	"k8s.io/apiserver/pkg/features"
 	"k8s.io/apiserver/pkg/server/dynamiccertificates"
+	utilfeature "k8s.io/apiserver/pkg/util/feature"
 	corev1informers "k8s.io/client-go/informers/core/v1"
 	"k8s.io/client-go/kubernetes"
 	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
@@ -262,10 +264,13 @@ func getConfigMapDataFor(authenticationInfo ClusterAuthenticationInfo) (map[stri
 		if err != nil {
 			return nil, err
 		}
-		data["requestheader-uid-headers"], err = jsonSerializeStringSlice(authenticationInfo.RequestHeaderUIDHeaders.Value())
-		if err != nil {
-			return nil, err
+		if utilfeature.DefaultFeatureGate.Enabled(features.RemoteRequestHeaderUID) && len(authenticationInfo.RequestHeaderUIDHeaders.Value()) > 0 {
+			data["requestheader-uid-headers"], err = jsonSerializeStringSlice(authenticationInfo.RequestHeaderUIDHeaders.Value())
+			if err != nil {
+				return nil, err
+			}
 		}
+
 		data["requestheader-group-headers"], err = jsonSerializeStringSlice(authenticationInfo.RequestHeaderGroupHeaders.Value())
 		if err != nil {
 			return nil, err
@@ -305,9 +310,12 @@ func getClusterAuthenticationInfoFor(data map[string]string) (ClusterAuthenticat
 	if err != nil {
 		return ClusterAuthenticationInfo{}, err
 	}
-	ret.RequestHeaderUIDHeaders, err = jsonDeserializeStringSlice(data["requestheader-uid-headers"])
-	if err != nil {
-		return ClusterAuthenticationInfo{}, err
+
+	if utilfeature.DefaultFeatureGate.Enabled(features.RemoteRequestHeaderUID) {
+		ret.RequestHeaderUIDHeaders, err = jsonDeserializeStringSlice(data["requestheader-uid-headers"])
+		if err != nil {
+			return ClusterAuthenticationInfo{}, err
+		}
 	}
 
 	if caBundle := data["requestheader-client-ca-file"]; len(caBundle) > 0 {
diff --git a/pkg/features/versioned_kube_features.go b/pkg/features/versioned_kube_features.go
index 9b8bbc6958f5c..ecee44abf7b30 100644
--- a/pkg/features/versioned_kube_features.go
+++ b/pkg/features/versioned_kube_features.go
@@ -306,6 +306,10 @@ var defaultVersionedKubernetesFeatureGates = map[featuregate.Feature]featuregate
 		{Version: version.MustParse("1.29"), Default: true, PreRelease: featuregate.GA, LockToDefault: true},
 	},
 
+	genericfeatures.RemoteRequestHeaderUID: {
+		{Version: version.MustParse("1.32"), Default: false, PreRelease: featuregate.Alpha},
+	},
+
 	genericfeatures.ResilientWatchCacheInitialization: {
 		{Version: version.MustParse("1.31"), Default: true, PreRelease: featuregate.Beta},
 	},
diff --git a/staging/src/k8s.io/apiserver/pkg/features/kube_features.go b/staging/src/k8s.io/apiserver/pkg/features/kube_features.go
index d0b064dc2c901..c23343346e461 100644
--- a/staging/src/k8s.io/apiserver/pkg/features/kube_features.go
+++ b/staging/src/k8s.io/apiserver/pkg/features/kube_features.go
@@ -149,6 +149,13 @@ const (
 	// to a chunking list request.
 	RemainingItemCount featuregate.Feature = "RemainingItemCount"
 
+	// owner: @stlaz
+	//
+	// Enable kube-apiserver to accept UIDs via request header authentication.
+	// This will also make the kube-apiserver's API aggregator add UIDs via standard
+	// headers when forwarding requests to the servers serving the aggregated API.
+	RemoteRequestHeaderUID featuregate.Feature = "RemoteRequestHeaderUID"
+
 	// owner: @wojtek-t
 	//
 	// Enables resilient watchcache initialization to avoid controlplane
@@ -359,6 +366,10 @@ var defaultVersionedKubernetesFeatureGates = map[featuregate.Feature]featuregate
 		{Version: version.MustParse("1.29"), Default: true, PreRelease: featuregate.GA, LockToDefault: true},
 	},
 
+	RemoteRequestHeaderUID: {
+		{Version: version.MustParse("1.32"), Default: false, PreRelease: featuregate.Alpha},
+	},
+
 	ResilientWatchCacheInitialization: {
 		{Version: version.MustParse("1.31"), Default: true, PreRelease: featuregate.Beta},
 	},
diff --git a/staging/src/k8s.io/apiserver/pkg/server/options/authentication.go b/staging/src/k8s.io/apiserver/pkg/server/options/authentication.go
index 6b53908355c7e..f88f73e72b1ef 100644
--- a/staging/src/k8s.io/apiserver/pkg/server/options/authentication.go
+++ b/staging/src/k8s.io/apiserver/pkg/server/options/authentication.go
@@ -29,8 +29,10 @@ import (
 	"k8s.io/apiserver/pkg/apis/apiserver"
 	"k8s.io/apiserver/pkg/authentication/authenticatorfactory"
 	"k8s.io/apiserver/pkg/authentication/request/headerrequest"
+	"k8s.io/apiserver/pkg/features"
 	"k8s.io/apiserver/pkg/server"
 	"k8s.io/apiserver/pkg/server/dynamiccertificates"
+	utilfeature "k8s.io/apiserver/pkg/util/feature"
 	"k8s.io/client-go/kubernetes"
 	"k8s.io/client-go/rest"
 	"k8s.io/client-go/tools/clientcmd"
@@ -68,9 +70,6 @@ func (s *RequestHeaderAuthenticationOptions) Validate() []error {
 	if err := checkForWhiteSpaceOnly("requestheader-username-headers", s.UsernameHeaders...); err != nil {
 		allErrors = append(allErrors, err)
 	}
-	if err := checkForWhiteSpaceOnly("requestheader-uid-headers", s.UIDHeaders...); err != nil {
-		allErrors = append(allErrors, err)
-	}
 	if err := checkForWhiteSpaceOnly("requestheader-group-headers", s.GroupHeaders...); err != nil {
 		allErrors = append(allErrors, err)
 	}
@@ -84,10 +83,6 @@ func (s *RequestHeaderAuthenticationOptions) Validate() []error {
 	if len(s.UsernameHeaders) > 0 && !caseInsensitiveHas(s.UsernameHeaders, "X-Remote-User") {
 		klog.Warningf("--requestheader-username-headers is set without specifying the standard X-Remote-User header - API aggregation will not work")
 	}
-	if len(s.UIDHeaders) > 0 && !caseInsensitiveHas(s.UIDHeaders, "X-Remote-Uid") {
-		// this was added later and so we are able to error out
-		allErrors = append(allErrors, fmt.Errorf("--requestheader-uid-headers is set without specifying the standard X-Remote-Uid header - API aggregation will not work"))
-	}
 	if len(s.GroupHeaders) > 0 && !caseInsensitiveHas(s.GroupHeaders, "X-Remote-Group") {
 		klog.Warningf("--requestheader-group-headers is set without specifying the standard X-Remote-Group header - API aggregation will not work")
 	}
@@ -95,6 +90,20 @@ func (s *RequestHeaderAuthenticationOptions) Validate() []error {
 		klog.Warningf("--requestheader-extra-headers-prefix is set without specifying the standard X-Remote-Extra- header prefix - API aggregation will not work")
 	}
 
+	if !utilfeature.DefaultFeatureGate.Enabled(features.RemoteRequestHeaderUID) {
+		if len(s.UIDHeaders) > 0 {
+			allErrors = append(allErrors, fmt.Errorf("--requestheader-uid-headers requires the %q feature to be enabled", features.RemoteRequestHeaderUID))
+		}
+	} else {
+		if err := checkForWhiteSpaceOnly("requestheader-uid-headers", s.UIDHeaders...); err != nil {
+			allErrors = append(allErrors, err)
+		}
+		if len(s.UIDHeaders) > 0 && !caseInsensitiveHas(s.UIDHeaders, "X-Remote-Uid") {
+			// this was added later and so we are able to error out
+			allErrors = append(allErrors, fmt.Errorf("--requestheader-uid-headers is set without specifying the standard X-Remote-Uid header - API aggregation will not work"))
+		}
+	}
+
 	return allErrors
 }
 
@@ -126,7 +135,7 @@ func (s *RequestHeaderAuthenticationOptions) AddFlags(fs *pflag.FlagSet) {
 		"List of request headers to inspect for usernames. X-Remote-User is common.")
 
 	fs.StringSliceVar(&s.UIDHeaders, "requestheader-uid-headers", s.UIDHeaders, ""+
-		"List of request headers to inspect for UIDs. X-Remote-Uid is suggested.")
+		"List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled.")
 
 	fs.StringSliceVar(&s.GroupHeaders, "requestheader-group-headers", s.GroupHeaders, ""+
 		"List of request headers to inspect for groups. X-Remote-Group is suggested.")
diff --git a/staging/src/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy.go b/staging/src/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy.go
index 5292ec8648947..d95a271af76ee 100644
--- a/staging/src/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy.go
+++ b/staging/src/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy.go
@@ -159,7 +159,12 @@ func (r *proxyHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
 	proxyRoundTripper := handlingInfo.proxyRoundTripper
 	upgrade := httpstream.IsUpgradeRequest(req)
 
-	proxyRoundTripper = transport.NewAuthProxyRoundTripper(user.GetName(), user.GetUID(), user.GetGroups(), user.GetExtra(), proxyRoundTripper)
+	var userUID string
+	if utilfeature.DefaultFeatureGate.Enabled(genericfeatures.RemoteRequestHeaderUID) {
+		userUID = user.GetUID()
+	}
+
+	proxyRoundTripper = transport.NewAuthProxyRoundTripper(user.GetName(), userUID, user.GetGroups(), user.GetExtra(), proxyRoundTripper)
 
 	if utilfeature.DefaultFeatureGate.Enabled(genericfeatures.APIServerTracing) && !upgrade {
 		tracingWrapper := tracing.WrapperFor(r.tracerProvider)
@@ -170,7 +175,7 @@ func (r *proxyHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
 	// NOT use the proxyRoundTripper.  It's a direct dial that bypasses the proxyRoundTripper.  This means that we have to
 	// attach the "correct" user headers to the request ahead of time.
 	if upgrade {
-		transport.SetAuthProxyHeaders(newReq, user.GetName(), user.GetUID(), user.GetGroups(), user.GetExtra())
+		transport.SetAuthProxyHeaders(newReq, user.GetName(), userUID, user.GetGroups(), user.GetExtra())
 	}
 
 	handler := proxy.NewUpgradeAwareHandler(location, proxyRoundTripper, true, upgrade, &responder{w: w})
diff --git a/staging/src/k8s.io/sample-apiserver/main.go b/staging/src/k8s.io/sample-apiserver/main.go
index 7dd8718c80488..6550efa560e43 100644
--- a/staging/src/k8s.io/sample-apiserver/main.go
+++ b/staging/src/k8s.io/sample-apiserver/main.go
@@ -27,7 +27,7 @@ import (
 func main() {
 	ctx := genericapiserver.SetupSignalContext()
 	options := server.NewWardleServerOptions(os.Stdout, os.Stderr)
-	cmd := server.NewCommandStartWardleServer(ctx, options)
+	cmd := server.NewCommandStartWardleServer(ctx, options, false)
 	code := cli.Run(cmd)
 	os.Exit(code)
 }
diff --git a/staging/src/k8s.io/sample-apiserver/pkg/cmd/server/start.go b/staging/src/k8s.io/sample-apiserver/pkg/cmd/server/start.go
index 79df51f015b7d..852252dbb0df7 100644
--- a/staging/src/k8s.io/sample-apiserver/pkg/cmd/server/start.go
+++ b/staging/src/k8s.io/sample-apiserver/pkg/cmd/server/start.go
@@ -90,12 +90,15 @@ func NewWardleServerOptions(out, errOut io.Writer) *WardleServerOptions {
 
 // NewCommandStartWardleServer provides a CLI handler for 'start master' command
 // with a default WardleServerOptions.
-func NewCommandStartWardleServer(ctx context.Context, defaults *WardleServerOptions) *cobra.Command {
+func NewCommandStartWardleServer(ctx context.Context, defaults *WardleServerOptions, skipDefaultComponentGlobalsRegistrySet bool) *cobra.Command {
 	o := *defaults
 	cmd := &cobra.Command{
 		Short: "Launch a wardle API server",
 		Long:  "Launch a wardle API server",
 		PersistentPreRunE: func(*cobra.Command, []string) error {
+			if skipDefaultComponentGlobalsRegistrySet {
+				return nil
+			}
 			return featuregate.DefaultComponentGlobalsRegistry.Set()
 		},
 		RunE: func(c *cobra.Command, args []string) error {


Test patch:
diff --git a/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller_test.go b/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller_test.go
index d9e3a217dd6f5..0dfe2cbca13c9 100644
--- a/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller_test.go
+++ b/pkg/controlplane/controller/clusterauthenticationtrust/cluster_authentication_trust_controller_test.go
@@ -30,11 +30,14 @@ import (
 	"k8s.io/apimachinery/pkg/util/dump"
 	"k8s.io/apimachinery/pkg/util/validation/field"
 	"k8s.io/apiserver/pkg/authentication/request/headerrequest"
+	"k8s.io/apiserver/pkg/features"
 	"k8s.io/apiserver/pkg/server/dynamiccertificates"
+	utilfeature "k8s.io/apiserver/pkg/util/feature"
 	"k8s.io/client-go/kubernetes/fake"
 	corev1listers "k8s.io/client-go/listers/core/v1"
 	clienttesting "k8s.io/client-go/testing"
 	"k8s.io/client-go/tools/cache"
+	featuregatetesting "k8s.io/component-base/featuregate/testing"
 )
 
 var (
@@ -95,6 +98,7 @@ func TestWriteClientCAs(t *testing.T) {
 		preexistingObjs    []runtime.Object
 		expectedConfigMaps map[string]*corev1.ConfigMap
 		expectCreate       bool
+		uidGate            bool
 	}{
 		{
 			name: "basic",
@@ -107,6 +111,32 @@ func TestWriteClientCAs(t *testing.T) {
 				RequestHeaderCA:                  anotherRandomCAProvider,
 				RequestHeaderAllowedNames:        headerrequest.StaticStringSlice{"first", "second"},
 			},
+			expectedConfigMaps: map[string]*corev1.ConfigMap{
+				"extension-apiserver-authentication": {
+					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
+					Data: map[string]string{
+						"client-ca-file":                     string(someRandomCA),
+						"requestheader-username-headers":     `["alfa","bravo","charlie"]`,
+						"requestheader-group-headers":        `["delta"]`,
+						"requestheader-extra-headers-prefix": `["echo","foxtrot"]`,
+						"requestheader-client-ca-file":       string(anotherRandomCA),
+						"requestheader-allowed-names":        `["first","second"]`,
+					},
+				},
+			},
+			expectCreate: true,
+		},
+		{
+			name: "basic with feature gate",
+			clusterAuthInfo: ClusterAuthenticationInfo{
+				ClientCA:                         someRandomCAProvider,
+				RequestHeaderUsernameHeaders:     headerrequest.StaticStringSlice{"alfa", "bravo", "charlie"},
+				RequestHeaderUIDHeaders:          headerrequest.StaticStringSlice{"golf", "hotel", "india"},
+				RequestHeaderGroupHeaders:        headerrequest.StaticStringSlice{"delta"},
+				RequestHeaderExtraHeaderPrefixes: headerrequest.StaticStringSlice{"echo", "foxtrot"},
+				RequestHeaderCA:                  anotherRandomCAProvider,
+				RequestHeaderAllowedNames:        headerrequest.StaticStringSlice{"first", "second"},
+			},
 			expectedConfigMaps: map[string]*corev1.ConfigMap{
 				"extension-apiserver-authentication": {
 					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
@@ -122,6 +152,7 @@ func TestWriteClientCAs(t *testing.T) {
 				},
 			},
 			expectCreate: true,
+			uidGate:      true,
 		},
 		{
 			name: "skip extension-apiserver-authentication",
@@ -134,7 +165,6 @@ func TestWriteClientCAs(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
 					Data: map[string]string{
 						"requestheader-username-headers":     `[]`,
-						"requestheader-uid-headers":          `[]`,
 						"requestheader-group-headers":        `[]`,
 						"requestheader-extra-headers-prefix": `[]`,
 						"requestheader-client-ca-file":       string(anotherRandomCA),
@@ -169,7 +199,6 @@ func TestWriteClientCAs(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
 					Data: map[string]string{
 						"requestheader-username-headers":     `[]`,
-						"requestheader-uid-headers":          `[]`,
 						"requestheader-group-headers":        `[]`,
 						"requestheader-extra-headers-prefix": `[]`,
 						"requestheader-client-ca-file":       string(anotherRandomCA),
@@ -205,7 +234,6 @@ func TestWriteClientCAs(t *testing.T) {
 			name: "overwrite extension-apiserver-authentication requestheader",
 			clusterAuthInfo: ClusterAuthenticationInfo{
 				RequestHeaderUsernameHeaders:     headerrequest.StaticStringSlice{},
-				RequestHeaderUIDHeaders:          headerrequest.StaticStringSlice{},
 				RequestHeaderGroupHeaders:        headerrequest.StaticStringSlice{},
 				RequestHeaderExtraHeaderPrefixes: headerrequest.StaticStringSlice{},
 				RequestHeaderCA:                  anotherRandomCAProvider,
@@ -216,7 +244,6 @@ func TestWriteClientCAs(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
 					Data: map[string]string{
 						"requestheader-username-headers":     `[]`,
-						"requestheader-uid-headers":          `[]`,
 						"requestheader-group-headers":        `[]`,
 						"requestheader-extra-headers-prefix": `[]`,
 						"requestheader-client-ca-file":       string(someRandomCA),
@@ -229,7 +256,6 @@ func TestWriteClientCAs(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
 					Data: map[string]string{
 						"requestheader-username-headers":     `[]`,
-						"requestheader-uid-headers":          `[]`,
 						"requestheader-group-headers":        `[]`,
 						"requestheader-extra-headers-prefix": `[]`,
 						"requestheader-client-ca-file":       string(someRandomCA) + string(anotherRandomCA),
@@ -260,7 +286,6 @@ func TestWriteClientCAs(t *testing.T) {
 			name: "skip on no change",
 			clusterAuthInfo: ClusterAuthenticationInfo{
 				RequestHeaderUsernameHeaders:     headerrequest.StaticStringSlice{},
-				RequestHeaderUIDHeaders:          headerrequest.StaticStringSlice{},
 				RequestHeaderGroupHeaders:        headerrequest.StaticStringSlice{},
 				RequestHeaderExtraHeaderPrefixes: headerrequest.StaticStringSlice{},
 				RequestHeaderCA:                  anotherRandomCAProvider,
@@ -271,7 +296,6 @@ func TestWriteClientCAs(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
 					Data: map[string]string{
 						"requestheader-username-headers":     `[]`,
-						"requestheader-uid-headers":          `[]`,
 						"requestheader-group-headers":        `[]`,
 						"requestheader-extra-headers-prefix": `[]`,
 						"requestheader-client-ca-file":       string(anotherRandomCA),
@@ -282,10 +306,126 @@ func TestWriteClientCAs(t *testing.T) {
 			expectedConfigMaps: map[string]*corev1.ConfigMap{},
 			expectCreate:       false,
 		},
+		{
+			name: "drop uid without feature gate",
+			clusterAuthInfo: ClusterAuthenticationInfo{
+				RequestHeaderUsernameHeaders:     headerrequest.StaticStringSlice{},
+				RequestHeaderUIDHeaders:          headerrequest.StaticStringSlice{"panda"},
+				RequestHeaderGroupHeaders:        headerrequest.StaticStringSlice{},
+				RequestHeaderExtraHeaderPrefixes: headerrequest.StaticStringSlice{},
+				RequestHeaderCA:                  anotherRandomCAProvider,
+				RequestHeaderAllowedNames:        headerrequest.StaticStringSlice{},
+			},
+			preexistingObjs: []runtime.Object{
+				&corev1.ConfigMap{
+					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
+					Data: map[string]string{
+						"requestheader-username-headers":     `[]`,
+						"requestheader-uid-headers":          `["snorlax"]`,
+						"requestheader-group-headers":        `[]`,
+						"requestheader-extra-headers-prefix": `[]`,
+						"requestheader-client-ca-file":       string(anotherRandomCA),
+						"requestheader-allowed-names":        `[]`,
+					},
+				},
+			},
+			expectedConfigMaps: map[string]*corev1.ConfigMap{
+				"extension-apiserver-authentication": {
+					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
+					Data: map[string]string{
+						"requestheader-username-headers":     `[]`,
+						"requestheader-group-headers":        `[]`,
+						"requestheader-extra-headers-prefix": `[]`,
+						"requestheader-client-ca-file":       string(anotherRandomCA),
+						"requestheader-allowed-names":        `[]`,
+					},
+				},
+			},
+			expectCreate: false,
+		},
+		{
+			name: "add uid with feature gate",
+			clusterAuthInfo: ClusterAuthenticationInfo{
+				RequestHeaderUsernameHeaders:     headerrequest.StaticStringSlice{},
+				RequestHeaderUIDHeaders:          headerrequest.StaticStringSlice{"panda"},
+				RequestHeaderGroupHeaders:        headerrequest.StaticStringSlice{},
+				RequestHeaderExtraHeaderPrefixes: headerrequest.StaticStringSlice{},
+				RequestHeaderCA:                  anotherRandomCAProvider,
+				RequestHeaderAllowedNames:        headerrequest.StaticStringSlice{},
+			},
+			preexistingObjs: []runtime.Object{
+				&corev1.ConfigMap{
+					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
+					Data: map[string]string{
+						"requestheader-username-headers":     `[]`,
+						"requestheader-group-headers":        `[]`,
+						"requestheader-extra-headers-prefix": `[]`,
+						"requestheader-client-ca-file":       string(anotherRandomCA),
+						"requestheader-allowed-names":        `[]`,
+					},
+				},
+			},
+			expectedConfigMaps: map[string]*corev1.ConfigMap{
+				"extension-apiserver-authentication": {
+					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
+					Data: map[string]string{
+						"requestheader-username-headers":     `[]`,
+						"requestheader-uid-headers":          `["panda"]`,
+						"requestheader-group-headers":        `[]`,
+						"requestheader-extra-headers-prefix": `[]`,
+						"requestheader-client-ca-file":       string(anotherRandomCA),
+						"requestheader-allowed-names":        `[]`,
+					},
+				},
+			},
+			expectCreate: false,
+			uidGate:      true,
+		},
+		{
+			name: "append uid with feature gate",
+			clusterAuthInfo: ClusterAuthenticationInfo{
+				RequestHeaderUsernameHeaders:     headerrequest.StaticStringSlice{},
+				RequestHeaderUIDHeaders:          headerrequest.StaticStringSlice{"panda"},
+				RequestHeaderGroupHeaders:        headerrequest.StaticStringSlice{},
+				RequestHeaderExtraHeaderPrefixes: headerrequest.StaticStringSlice{},
+				RequestHeaderCA:                  anotherRandomCAProvider,
+				RequestHeaderAllowedNames:        headerrequest.StaticStringSlice{},
+			},
+			preexistingObjs: []runtime.Object{
+				&corev1.ConfigMap{
+					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
+					Data: map[string]string{
+						"requestheader-username-headers":     `[]`,
+						"requestheader-uid-headers":          `["snorlax"]`,
+						"requestheader-group-headers":        `[]`,
+						"requestheader-extra-headers-prefix": `[]`,
+						"requestheader-client-ca-file":       string(anotherRandomCA),
+						"requestheader-allowed-names":        `[]`,
+					},
+				},
+			},
+			expectedConfigMaps: map[string]*corev1.ConfigMap{
+				"extension-apiserver-authentication": {
+					ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
+					Data: map[string]string{
+						"requestheader-username-headers":     `[]`,
+						"requestheader-uid-headers":          `["snorlax","panda"]`,
+						"requestheader-group-headers":        `[]`,
+						"requestheader-extra-headers-prefix": `[]`,
+						"requestheader-client-ca-file":       string(anotherRandomCA),
+						"requestheader-allowed-names":        `[]`,
+					},
+				},
+			},
+			expectCreate: false,
+			uidGate:      true,
+		},
 	}
 
 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
+			featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.RemoteRequestHeaderUID, test.uidGate)
+
 			client := fake.NewSimpleClientset(test.preexistingObjs...)
 			configMapIndexer := cache.NewIndexer(cache.MetaNamespaceKeyFunc, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc})
 			for _, obj := range test.preexistingObjs {
@@ -341,7 +481,6 @@ func TestWriteConfigMapDeleted(t *testing.T) {
 		ObjectMeta: metav1.ObjectMeta{Namespace: metav1.NamespaceSystem, Name: "extension-apiserver-authentication"},
 		Data: map[string]string{
 			"requestheader-username-headers":     `[]`,
-			"requestheader-uid-headers":          `[]`,
 			"requestheader-group-headers":        `[]`,
 			"requestheader-extra-headers-prefix": `[]`,
 			"requestheader-client-ca-file":       string(anotherRandomCA),
diff --git a/staging/src/k8s.io/component-base/featuregate/testing/feature_gate.go b/staging/src/k8s.io/component-base/featuregate/testing/feature_gate.go
index eb28217aff51a..e9d452fb8cbe4 100644
--- a/staging/src/k8s.io/component-base/featuregate/testing/feature_gate.go
+++ b/staging/src/k8s.io/component-base/featuregate/testing/feature_gate.go
@@ -99,7 +99,7 @@ func SetFeatureGateEmulationVersionDuringTest(tb TB, gate featuregate.FeatureGat
 	detectParallelOverrideCleanup := detectParallelOverrideEmulationVersion(tb, ver)
 	originalEmuVer := gate.(featuregate.MutableVersionedFeatureGate).EmulationVersion()
 	if err := gate.(featuregate.MutableVersionedFeatureGate).SetEmulationVersion(ver); err != nil {
-		tb.Fatalf("failed to set emulation version to %s during test", ver.String())
+		tb.Fatalf("failed to set emulation version to %s during test: %v", ver.String(), err)
 	}
 	tb.Cleanup(func() {
 		tb.Helper()
diff --git a/staging/src/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy_test.go b/staging/src/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy_test.go
index 632727018a581..a19cdd22fad60 100644
--- a/staging/src/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy_test.go
+++ b/staging/src/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy_test.go
@@ -34,31 +34,34 @@ import (
 	"sync/atomic"
 	"testing"
 
-	"k8s.io/apiserver/pkg/audit"
-	"k8s.io/apiserver/pkg/server/dynamiccertificates"
-	"k8s.io/client-go/transport"
-
-	"golang.org/x/net/websocket"
-
+	"github.com/google/go-cmp/cmp"
 	"go.opentelemetry.io/otel/propagation"
 	sdktrace "go.opentelemetry.io/otel/sdk/trace"
 	"go.opentelemetry.io/otel/sdk/trace/tracetest"
 	"go.opentelemetry.io/otel/trace"
+	"golang.org/x/net/websocket"
+
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/types"
 	utilnet "k8s.io/apimachinery/pkg/util/net"
 	"k8s.io/apimachinery/pkg/util/proxy"
 	"k8s.io/apimachinery/pkg/util/sets"
+	"k8s.io/apiserver/pkg/audit"
 	"k8s.io/apiserver/pkg/authentication/user"
 	"k8s.io/apiserver/pkg/endpoints/filters"
 	genericapirequest "k8s.io/apiserver/pkg/endpoints/request"
+	"k8s.io/apiserver/pkg/features"
+	"k8s.io/apiserver/pkg/server/dynamiccertificates"
 	"k8s.io/apiserver/pkg/server/egressselector"
+	utilfeature "k8s.io/apiserver/pkg/util/feature"
 	utilflowcontrol "k8s.io/apiserver/pkg/util/flowcontrol"
 	apiserverproxyutil "k8s.io/apiserver/pkg/util/proxy"
+	"k8s.io/client-go/transport"
+	"k8s.io/component-base/featuregate"
+	featuregatetesting "k8s.io/component-base/featuregate/testing"
 	"k8s.io/component-base/metrics"
 	"k8s.io/component-base/metrics/legacyregistry"
 	apiregistration "k8s.io/kube-aggregator/pkg/apis/apiregistration/v1"
-	"k8s.io/utils/pointer"
 	"k8s.io/utils/ptr"
 )
 
@@ -130,6 +133,8 @@ func TestProxyHandler(t *testing.T) {
 		expectedBody       string
 		expectedCalled     bool
 		expectedHeaders    map[string][]string
+
+		enableFeatureGates []featuregate.Feature
 	}{
 		"no target": {
 			expectedStatusCode: http.StatusNotFound,
@@ -138,7 +143,7 @@ func TestProxyHandler(t *testing.T) {
 			apiService: &apiregistration.APIService{
 				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
 				Spec: apiregistration.APIServiceSpec{
-					Service: &apiregistration.ServiceReference{Port: pointer.Int32Ptr(443)},
+					Service: &apiregistration.ServiceReference{Port: ptr.To[int32](443)},
 					Group:   "foo",
 					Version: "v1",
 				},
@@ -161,7 +166,40 @@ func TestProxyHandler(t *testing.T) {
 			apiService: &apiregistration.APIService{
 				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
 				Spec: apiregistration.APIServiceSpec{
-					Service:               &apiregistration.ServiceReference{Port: pointer.Int32Ptr(443)},
+					Service:               &apiregistration.ServiceReference{Port: ptr.To[int32](443)},
+					Group:                 "foo",
+					Version:               "v1",
+					InsecureSkipTLSVerify: true,
+				},
+				Status: apiregistration.APIServiceStatus{
+					Conditions: []apiregistration.APIServiceCondition{
+						{Type: apiregistration.Available, Status: apiregistration.ConditionTrue},
+					},
+				},
+			},
+			expectedStatusCode: http.StatusOK,
+			expectedCalled:     true,
+			expectedHeaders: map[string][]string{
+				"X-Forwarded-Proto": {"https"},
+				"X-Forwarded-Uri":   {"/request/path"},
+				"X-Forwarded-For":   {"127.0.0.1"},
+				"X-Remote-User":     {"username"},
+				"User-Agent":        {"Go-http-client/1.1"},
+				"Accept-Encoding":   {"gzip"},
+				"X-Remote-Group":    {"one", "two"},
+			},
+		},
+		"[RemoteRequestHeaderUID] proxy with user, insecure": {
+			user: &user.DefaultInfo{
+				Name:   "username",
+				UID:    "6b60d791-1af9-4513-92e5-e4252a1e0a78",
+				Groups: []string{"one", "two"},
+			},
+			path: "/request/path",
+			apiService: &apiregistration.APIService{
+				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
+				Spec: apiregistration.APIServiceSpec{
+					Service:               &apiregistration.ServiceReference{Port: ptr.To[int32](443)},
 					Group:                 "foo",
 					Version:               "v1",
 					InsecureSkipTLSVerify: true,
@@ -172,6 +210,7 @@ func TestProxyHandler(t *testing.T) {
 					},
 				},
 			},
+			enableFeatureGates: []featuregate.Feature{features.RemoteRequestHeaderUID},
 			expectedStatusCode: http.StatusOK,
 			expectedCalled:     true,
 			expectedHeaders: map[string][]string{
@@ -195,7 +234,7 @@ func TestProxyHandler(t *testing.T) {
 			apiService: &apiregistration.APIService{
 				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
 				Spec: apiregistration.APIServiceSpec{
-					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: pointer.Int32Ptr(443)},
+					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: ptr.To[int32](443)},
 					Group:    "foo",
 					Version:  "v1",
 					CABundle: testCACrt,
@@ -208,6 +247,40 @@ func TestProxyHandler(t *testing.T) {
 			},
 			expectedStatusCode: http.StatusOK,
 			expectedCalled:     true,
+			expectedHeaders: map[string][]string{
+				"X-Forwarded-Proto": {"https"},
+				"X-Forwarded-Uri":   {"/request/path"},
+				"X-Forwarded-For":   {"127.0.0.1"},
+				"X-Remote-User":     {"username"},
+				"User-Agent":        {"Go-http-client/1.1"},
+				"Accept-Encoding":   {"gzip"},
+				"X-Remote-Group":    {"one", "two"},
+			},
+		},
+		"[RemoteRequestHeaderUID] proxy with user, cabundle": {
+			user: &user.DefaultInfo{
+				Name:   "username",
+				UID:    "6b60d791-1af9-4513-92e5-e4252a1e0a78",
+				Groups: []string{"one", "two"},
+			},
+			path: "/request/path",
+			apiService: &apiregistration.APIService{
+				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
+				Spec: apiregistration.APIServiceSpec{
+					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: ptr.To[int32](443)},
+					Group:    "foo",
+					Version:  "v1",
+					CABundle: testCACrt,
+				},
+				Status: apiregistration.APIServiceStatus{
+					Conditions: []apiregistration.APIServiceCondition{
+						{Type: apiregistration.Available, Status: apiregistration.ConditionTrue},
+					},
+				},
+			},
+			enableFeatureGates: []featuregate.Feature{features.RemoteRequestHeaderUID},
+			expectedStatusCode: http.StatusOK,
+			expectedCalled:     true,
 			expectedHeaders: map[string][]string{
 				"X-Forwarded-Proto": {"https"},
 				"X-Forwarded-Uri":   {"/request/path"},
@@ -229,7 +302,7 @@ func TestProxyHandler(t *testing.T) {
 			apiService: &apiregistration.APIService{
 				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
 				Spec: apiregistration.APIServiceSpec{
-					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: pointer.Int32Ptr(443)},
+					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: ptr.To[int32](443)},
 					Group:    "foo",
 					Version:  "v1",
 					CABundle: testCACrt,
@@ -253,7 +326,7 @@ func TestProxyHandler(t *testing.T) {
 			apiService: &apiregistration.APIService{
 				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
 				Spec: apiregistration.APIServiceSpec{
-					Service:  &apiregistration.ServiceReference{Name: "bad-service", Namespace: "test-ns", Port: pointer.Int32Ptr(443)},
+					Service:  &apiregistration.ServiceReference{Name: "bad-service", Namespace: "test-ns", Port: ptr.To[int32](443)},
 					Group:    "foo",
 					Version:  "v1",
 					CABundle: testCACrt,
@@ -276,7 +349,7 @@ func TestProxyHandler(t *testing.T) {
 			apiService: &apiregistration.APIService{
 				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
 				Spec: apiregistration.APIServiceSpec{
-					Service: &apiregistration.ServiceReference{Port: pointer.Int32Ptr(443)},
+					Service: &apiregistration.ServiceReference{Port: ptr.To[int32](443)},
 					Group:   "foo",
 					Version: "v1",
 				},
@@ -298,7 +371,7 @@ func TestProxyHandler(t *testing.T) {
 			apiService: &apiregistration.APIService{
 				ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
 				Spec: apiregistration.APIServiceSpec{
-					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: pointer.Int32Ptr(443)},
+					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: ptr.To[int32](443)},
 					Group:    "foo",
 					Version:  "v1",
 					CABundle: testCACrt,
@@ -320,7 +393,11 @@ func TestProxyHandler(t *testing.T) {
 		target.Reset()
 		legacyregistry.Reset()
 
-		func() {
+		t.Run(name, func(t *testing.T) {
+			for _, f := range tc.enableFeatureGates {
+				featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, f, true)
+			}
+
 			targetServer := httptest.NewUnstartedServer(target)
 			serviceCert := tc.serviceCertOverride
 			if serviceCert == nil {
@@ -354,37 +431,37 @@ func TestProxyHandler(t *testing.T) {
 
 			resp, err := http.Get(server.URL + tc.path)
 			if err != nil {
-				t.Errorf("%s: %v", name, err)
+				t.Errorf("%v", err)
 				return
 			}
 			if e, a := tc.expectedStatusCode, resp.StatusCode; e != a {
 				body, _ := httputil.DumpResponse(resp, true)
-				t.Logf("%s: %v", name, string(body))
-				t.Errorf("%s: expected %v, got %v", name, e, a)
+				t.Logf("%v", string(body))
+				t.Errorf("expected %v, got %v", e, a)
 				return
 			}
 			bytes, err := io.ReadAll(resp.Body)
 			if err != nil {
-				t.Errorf("%s: %v", name, err)
+				t.Errorf("%v", err)
 				return
 			}
 			if !strings.Contains(string(bytes), tc.expectedBody) {
-				t.Errorf("%s: expected %q, got %q", name, tc.expectedBody, string(bytes))
+				t.Errorf("expected %q, got %q", tc.expectedBody, string(bytes))
 				return
 			}
 
 			if e, a := tc.expectedCalled, target.called; e != a {
-				t.Errorf("%s: expected %v, got %v", name, e, a)
+				t.Errorf("expected %v, got %v", e, a)
 				return
 			}
 			// this varies every test
 			delete(target.headers, "X-Forwarded-Host")
 			if e, a := tc.expectedHeaders, target.headers; !reflect.DeepEqual(e, a) {
-				t.Errorf("%s: expected %v, got %v", name, e, a)
+				t.Errorf("expected != got %v", cmp.Diff(e, a))
 				return
 			}
 			if e, a := targetServer.Listener.Addr().String(), target.host; tc.expectedCalled && !reflect.DeepEqual(e, a) {
-				t.Errorf("%s: expected %v, got %v", name, e, a)
+				t.Errorf("expected %v, got %v", e, a)
 				return
 			}
 
@@ -397,7 +474,7 @@ func TestProxyHandler(t *testing.T) {
 					t.Errorf("expected the x509_missing_san_total to be 1, but it's %d", errorCounter)
 				}
 			}
-		}()
+		})
 	}
 }
 
@@ -432,6 +509,8 @@ func newBrokenDialerAndSelector() (*mockEgressDialer, *egressselector.EgressSele
 }
 
 func TestProxyUpgrade(t *testing.T) {
+	featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.RemoteRequestHeaderUID, true)
+
 	upgradeUser := "upgradeUser"
 	upgradeUID := "upgradeUser-UID"
 	testcases := map[string]struct {
@@ -446,7 +525,7 @@ func TestProxyUpgrade(t *testing.T) {
 					CABundle: testCACrt,
 					Group:    "mygroup",
 					Version:  "v1",
-					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: pointer.Int32Ptr(443)},
+					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: ptr.To[int32](443)},
 				},
 				Status: apiregistration.APIServiceStatus{
 					Conditions: []apiregistration.APIServiceCondition{
@@ -463,7 +542,7 @@ func TestProxyUpgrade(t *testing.T) {
 					InsecureSkipTLSVerify: true,
 					Group:                 "mygroup",
 					Version:               "v1",
-					Service:               &apiregistration.ServiceReference{Name: "invalid-service", Namespace: "invalid-ns", Port: pointer.Int32Ptr(443)},
+					Service:               &apiregistration.ServiceReference{Name: "invalid-service", Namespace: "invalid-ns", Port: ptr.To[int32](443)},
 				},
 				Status: apiregistration.APIServiceStatus{
 					Conditions: []apiregistration.APIServiceCondition{
@@ -480,7 +559,7 @@ func TestProxyUpgrade(t *testing.T) {
 					CABundle: testCACrt,
 					Group:    "mygroup",
 					Version:  "v1",
-					Service:  &apiregistration.ServiceReference{Name: "invalid-service", Namespace: "invalid-ns", Port: pointer.Int32Ptr(443)},
+					Service:  &apiregistration.ServiceReference{Name: "invalid-service", Namespace: "invalid-ns", Port: ptr.To[int32](443)},
 				},
 				Status: apiregistration.APIServiceStatus{
 					Conditions: []apiregistration.APIServiceCondition{
@@ -497,7 +576,7 @@ func TestProxyUpgrade(t *testing.T) {
 					CABundle: testCACrt,
 					Group:    "mygroup",
 					Version:  "v1",
-					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: pointer.Int32Ptr(443)},
+					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: ptr.To[int32](443)},
 				},
 				Status: apiregistration.APIServiceStatus{
 					Conditions: []apiregistration.APIServiceCondition{
@@ -515,7 +594,7 @@ func TestProxyUpgrade(t *testing.T) {
 					CABundle: testCACrt,
 					Group:    "mygroup",
 					Version:  "v1",
-					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: pointer.Int32Ptr(443)},
+					Service:  &apiregistration.ServiceReference{Name: "test-service", Namespace: "test-ns", Port: ptr.To[int32](443)},
 				},
 				Status: apiregistration.APIServiceStatus{
 					Conditions: []apiregistration.APIServiceCondition{
@@ -996,7 +1075,7 @@ func TestProxyCertReload(t *testing.T) {
 	apiService := &apiregistration.APIService{
 		ObjectMeta: metav1.ObjectMeta{Name: "v1.foo"},
 		Spec: apiregistration.APIServiceSpec{
-			Service:  &apiregistration.ServiceReference{Name: "test-service2", Namespace: "test-ns", Port: pointer.Int32Ptr(443)},
+			Service:  &apiregistration.ServiceReference{Name: "test-service2", Namespace: "test-ns", Port: ptr.To[int32](443)},
 			Group:    "foo",
 			Version:  "v1",
 			CABundle: backendCaCertificate(), // used to validate backendCertificate()
diff --git a/test/featuregates_linter/test_data/versioned_feature_list.yaml b/test/featuregates_linter/test_data/versioned_feature_list.yaml
index 3e355898dd771..a25a113071af6 100644
--- a/test/featuregates_linter/test_data/versioned_feature_list.yaml
+++ b/test/featuregates_linter/test_data/versioned_feature_list.yaml
@@ -1058,6 +1058,12 @@
     lockToDefault: true
     preRelease: GA
     version: "1.29"
+- name: RemoteRequestHeaderUID
+  versionedSpecs:
+  - default: false
+    lockToDefault: false
+    preRelease: Alpha
+    version: "1.32"
 - name: ResilientWatchCacheInitialization
   versionedSpecs:
   - default: true
diff --git a/test/integration/auth/requestheader_test.go b/test/integration/auth/requestheader_test.go
index 65198f2bc544d..4654a7950918e 100644
--- a/test/integration/auth/requestheader_test.go
+++ b/test/integration/auth/requestheader_test.go
@@ -27,11 +27,14 @@ import (
 
 	authnv1 "k8s.io/api/authentication/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apiserver/pkg/features"
+	utilfeature "k8s.io/apiserver/pkg/util/feature"
 	"k8s.io/client-go/kubernetes"
 	"k8s.io/client-go/kubernetes/scheme"
 	"k8s.io/client-go/rest"
 	certutil "k8s.io/client-go/util/cert"
 	"k8s.io/client-go/util/keyutil"
+	featuregatetesting "k8s.io/component-base/featuregate/testing"
 	"k8s.io/kubernetes/cmd/kube-apiserver/app/options"
 	"k8s.io/kubernetes/test/integration/framework"
 	testutils "k8s.io/kubernetes/test/utils"
@@ -39,6 +42,8 @@ import (
 )
 
 func TestAuthnToKAS(t *testing.T) {
+	featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.RemoteRequestHeaderUID, true)
+
 	tCtx := ktesting.Init(t)
 
 	frontProxyCA, frontProxyClient, frontProxyKey, err := newTestCAWithClient(
diff --git a/test/integration/examples/apiserver_test.go b/test/integration/examples/apiserver_test.go
index 36e3f7721d010..a50a8a846a5a7 100644
--- a/test/integration/examples/apiserver_test.go
+++ b/test/integration/examples/apiserver_test.go
@@ -45,8 +45,10 @@ import (
 	"k8s.io/apiserver/pkg/authentication/serviceaccount"
 	"k8s.io/apiserver/pkg/authentication/user"
 	genericapirequest "k8s.io/apiserver/pkg/endpoints/request"
+	"k8s.io/apiserver/pkg/features"
 	"k8s.io/apiserver/pkg/server/dynamiccertificates"
 	genericapiserveroptions "k8s.io/apiserver/pkg/server/options"
+	utilfeature "k8s.io/apiserver/pkg/util/feature"
 	client "k8s.io/client-go/kubernetes"
 	"k8s.io/client-go/rest"
 	"k8s.io/client-go/tools/clientcmd"
@@ -54,6 +56,7 @@ import (
 	"k8s.io/client-go/transport"
 	"k8s.io/client-go/util/cert"
 	"k8s.io/component-base/featuregate"
+	featuregatetesting "k8s.io/component-base/featuregate/testing"
 	utilversion "k8s.io/component-base/version"
 	apiregistrationv1 "k8s.io/kube-aggregator/pkg/apis/apiregistration/v1"
 	aggregatorclient "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset"
@@ -260,6 +263,7 @@ func TestFrontProxyConfig(t *testing.T) {
 		testFrontProxyConfig(t, false)
 	})
 	t.Run("WithUID", func(t *testing.T) {
+		featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.RemoteRequestHeaderUID, true)
 		testFrontProxyConfig(t, true)
 	})
 }
@@ -282,7 +286,7 @@ func testFrontProxyConfig(t *testing.T, withUID bool) {
 	kubeBinaryVersion := sampleserver.WardleVersionToKubeVersion(version.MustParse(wardleBinaryVersion)).String()
 
 	// start up the KAS and prepare the options for the wardle API server
-	testKAS, wardleOptions, wardlePort := prepareAggregatedWardleAPIServer(ctx, t, testNamespace, kubeBinaryVersion, wardleBinaryVersion, extraKASFlags)
+	testKAS, wardleOptions, wardlePort := prepareAggregatedWardleAPIServer(ctx, t, testNamespace, kubeBinaryVersion, wardleBinaryVersion, extraKASFlags, withUID)
 	kubeConfig := getKubeConfig(testKAS)
 
 	// create the SA that we will use to query the aggregated API
@@ -337,12 +341,12 @@ func testFrontProxyConfig(t *testing.T, withUID bool) {
 		transport.WrapperFunc(func(rt http.RoundTripper) http.RoundTripper {
 			return roundTripperFunc(func(req *http.Request) (*http.Response, error) {
 				gotUser, ok := genericapirequest.UserFrom(req.Context())
-				if !ok || gotUser.GetName() == "system:anonymous" {
+				if !ok {
 					return nil, fmt.Errorf("got an unauthenticated request")
 				}
 
 				// this is likely the KAS checking the OpenAPI endpoints
-				if gotUser.GetName() == "system:anonymous" || gotUser.GetName() == "system:aggregator" {
+				if gotUser.GetName() == "system:anonymous" || gotUser.GetName() == "system:aggregator" || gotUser.GetName() == "system:kube-aggregator" {
 					return rt.RoundTrip(req)
 				}
 
@@ -368,7 +372,7 @@ func testFrontProxyConfig(t *testing.T, withUID bool) {
 	wardleCertDir, _ := os.MkdirTemp("", "test-integration-wardle-server")
 	defer os.RemoveAll(wardleCertDir)
 
-	runPreparedWardleServer(ctx, t, wardleOptions, wardleCertDir, wardlePort, false, true, wardleBinaryVersion, kubeConfig)
+	runPreparedWardleServer(ctx, t, wardleOptions, wardleCertDir, wardlePort, false, true, wardleBinaryVersion, kubeConfig, withUID)
 	waitForWardleAPIServiceReady(ctx, t, kubeConfig, wardleCertDir, testNamespace)
 
 	// get the wardle API client using our SA token
@@ -401,13 +405,13 @@ func testAggregatedAPIServer(t *testing.T, setWardleFeatureGate, banFlunder bool
 	// each wardle binary is bundled with a specific kube binary.
 	kubeBinaryVersion := sampleserver.WardleVersionToKubeVersion(version.MustParse(wardleBinaryVersion)).String()
 
-	testKAS, wardleOptions, wardlePort := prepareAggregatedWardleAPIServer(ctx, t, testNamespace, kubeBinaryVersion, wardleBinaryVersion, nil)
+	testKAS, wardleOptions, wardlePort := prepareAggregatedWardleAPIServer(ctx, t, testNamespace, kubeBinaryVersion, wardleBinaryVersion, nil, false)
 	kubeClientConfig := getKubeConfig(testKAS)
 
 	wardleCertDir, _ := os.MkdirTemp("", "test-integration-wardle-server")
 	defer os.RemoveAll(wardleCertDir)
 
-	directWardleClientConfig := runPreparedWardleServer(ctx, t, wardleOptions, wardleCertDir, wardlePort, setWardleFeatureGate, banFlunder, wardleEmulationVersion, kubeClientConfig)
+	directWardleClientConfig := runPreparedWardleServer(ctx, t, wardleOptions, wardleCertDir, wardlePort, setWardleFeatureGate, banFlunder, wardleEmulationVersion, kubeClientConfig, false)
 
 	// now we're finally ready to test. These are what's run by default now
 	wardleDirectClient := client.NewForConfigOrDie(directWardleClientConfig)
@@ -681,7 +685,7 @@ func TestAggregatedAPIServerRejectRedirectResponse(t *testing.T) {
 	}
 }
 
-func prepareAggregatedWardleAPIServer(ctx context.Context, t *testing.T, namespace, kubebinaryVersion, wardleBinaryVersion string, kubeAPIServerFlags []string) (*kastesting.TestServer, *sampleserver.WardleServerOptions, int) {
+func prepareAggregatedWardleAPIServer(ctx context.Context, t *testing.T, namespace, kubebinaryVersion, wardleBinaryVersion string, kubeAPIServerFlags []string, withUID bool) (*kastesting.TestServer, *sampleserver.WardleServerOptions, int) {
 	// makes the kube-apiserver very responsive.  it's normally a minute
 	dynamiccertificates.FileRefreshDuration = 1 * time.Second
 
@@ -693,6 +697,11 @@ func prepareAggregatedWardleAPIServer(ctx context.Context, t *testing.T, namespa
 	// endpoints cannot have loopback IPs so we need to override the resolver itself
 	t.Cleanup(app.SetServiceResolverForTests(staticURLServiceResolver(fmt.Sprintf("https://127.0.0.1:%d", wardlePort))))
 
+	// TODO figure out how to actually make BinaryVersion/EmulationVersion work with Wardle and KAS at the same time when Alpha FG are being set
+	if withUID {
+		kubebinaryVersion = ""
+	}
+
 	testServer := kastesting.StartTestServerOrDie(t,
 		&kastesting.TestServerInstanceOptions{
 			EnableCertAuth: true,
@@ -751,6 +760,7 @@ func runPreparedWardleServer(
 	banFlunder bool,
 	emulationVersion string,
 	kubeConfig *rest.Config,
+	withUID bool,
 ) *rest.Config {
 
 	// start the wardle server to prove we can aggregate it
@@ -769,7 +779,8 @@ func runPreparedWardleServer(
 		if flunderBanningFeatureGate {
 			args = append(args, "--feature-gates", fmt.Sprintf("wardle:BanFlunder=%v", banFlunder))
 		}
-		wardleCmd := sampleserver.NewCommandStartWardleServer(ctx, wardleOptions)
+		// TODO figure out how to actually make BinaryVersion/EmulationVersion work with Wardle and KAS at the same time when Alpha FG are being set
+		wardleCmd := sampleserver.NewCommandStartWardleServer(ctx, wardleOptions, withUID)
 		wardleCmd.SetArgs(args)
 		if err := wardleCmd.Execute(); err != nil {
 			t.Error(err)