Repository: kubernetes-sigs/kueue

Issue:
Merge the FlowSchema and PriorityLevelConfiguration v1 manifests into the installation manifests
<!-- Please only use this template for submitting enhancement requests -->

**What would you like to be added**:

As we discussed [here](https://github.com/kubernetes-sigs/kueue/pull/3043#discussion_r1764680475) and [here](https://github.com/kubernetes-sigs/kueue/pull/3043#discussion_r1764915102) we should merge the FlowSchema and PriorityLevelConfiguration v1 manifests into the installation manifests and remove v1beta3 manifests once the latest Kubernetes version (1.28) no longer supports v1beta3.

**Why is this needed**:

 - During the second iteration of [Graduation criteria](https://github.com/kubernetes-sigs/kueue/tree/3a428360818bab0f1618edcbdea26dd94b4ca6e5/keps/168-2-pending-workloads-visibility#beta) of visibility we need to merge manifests.

 - Due to flowcontrol.apiserver.k8s.io/v1beta3 FlowSchema is deprecated in v1.29+ and unavailable in v1.32+ we don't need to support v1beta3 anymore.

**Completion requirements**:

This enhancement requires the following artifacts:

- [ ] Design doc
- [ ] API change
- [ ] Docs update

The artifacts should be linked in subsequent comments.


Discussion:
@mimowo: Question regarding the support: https://github.com/kubernetes-sigs/kueue/pull/3043#discussion_r1760626660. I'm wondering what happens if 1.32 is released, so we release 0.10 Kueue according to the plan. IIUC this will complicate deployment of the latest Kueue on 1.31 which will be still used by majority of clusters at that point. @tenzen-y: @mbobrovskyi Could you update this issue to align with the latest decision?
@k8s-triage-robot: The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity, `lifecycle/stale` is applied
- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied
- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed

You can:
- Mark this issue as fresh with `/remove-lifecycle stale`
- Close this issue with `/close`
- Offer to help out with [Issue Triage][1]

Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community).

/lifecycle stale

[1]: https://www.kubernetes.dev/docs/guide/issue-triage/@mimowo: /remove-lifecycle stale@mimowo: As discussed in https://github.com/kubernetes-sigs/kueue/pull/3983#discussion_r1925026803 I prefer to keep the APF configuration as opt-in for a while longer after the last EOL release of Kubernetes. 

I'm ok some features of Kueue not working ideally on older versions, but not being to install at all is a flag,  because we claim Kueue can be at least installed on older versions (as in the main README: `A Kubernetes cluster with version 1.25 or newer is running. Learn how to [install the Kubernetes tools](https://kubernetes.io/docs/tasks/tools/).`. I would prefer to keep it that way, as the additional release will be very helpful for some users to migrate away for a version that just became EOL.

I propose to merge the configuration only when 1.30 gets EOL (+1 release). Then we update the README to claim Kueue installation is supported on 1.29+. 

Unless there are other ideas, or other factors at play.

Solution patch:
diff --git a/Makefile b/Makefile
index 83018b63b..53d885386 100644
--- a/Makefile
+++ b/Makefile
@@ -265,6 +265,8 @@ artifacts: kustomize yq helm ## Generate release artifacts.
 	$(KUSTOMIZE) build config/dev -o artifacts/manifests-dev.yaml
 	$(KUSTOMIZE) build config/alpha-enabled -o artifacts/manifests-alpha-enabled.yaml
 	$(KUSTOMIZE) build config/prometheus -o artifacts/prometheus.yaml
+	$(KUSTOMIZE) build config/visibility-apf/default -o artifacts/visibility-apf.yaml
+	$(KUSTOMIZE) build config/visibility-apf/1_28 -o artifacts/visibility-apf-1-28.yaml
 	@$(call clean-manifests)
 	# Update the image tag and policy
 	$(YQ)  e  '.controllerManager.manager.image.repository = "$(IMAGE_REPO)" | .controllerManager.manager.image.tag = "$(GIT_TAG)" | .controllerManager.manager.image.pullPolicy = "IfNotPresent"' -i charts/kueue/values.yaml
diff --git a/apis/kueue/v1alpha1/cohort_types.go b/apis/kueue/v1alpha1/cohort_types.go
index 1450bd957..e197a38e0 100644
--- a/apis/kueue/v1alpha1/cohort_types.go
+++ b/apis/kueue/v1alpha1/cohort_types.go
@@ -62,12 +62,6 @@ type CohortSpec struct {
 	ResourceGroups []kueuebeta.ResourceGroup `json:"resourceGroups,omitempty"`
 }
 
-const (
-	// Condition indicating that a Cohort is correctly configured,
-	// for example, there is no cycle.
-	CohortActive = "Active"
-)
-
 // CohortStatus defines the observed state of Cohort
 type CohortStatus struct {
 
@@ -82,7 +76,9 @@ type CohortStatus struct {
 //+kubebuilder:subresource:status
 //+kubebuilder:resource:scope=Cluster
 
-// Cohort is the Schema for the cohorts API
+// Cohort is the Schema for the cohorts API. Using Hierarchical
+// Cohorts (any Cohort which has a parent) with Fair Sharing
+// results in undefined behavior in 0.9
 type Cohort struct {
 	metav1.TypeMeta   `json:",inline"`
 	metav1.ObjectMeta `json:"metadata,omitempty"`
diff --git a/charts/kueue/README.md b/charts/kueue/README.md
index 11fb03fff..7e4bdb29b 100644
--- a/charts/kueue/README.md
+++ b/charts/kueue/README.md
@@ -48,6 +48,7 @@ The following table lists the configurable parameters of the kueue chart and the
 | `fullnameOverride`                                     | override the resource name                             | ``                                          |
 | `enablePrometheus`                                     | enable Prometheus                                      | `false`                                     |
 | `enableCertManager`                                    | enable CertManager                                     | `false`                                     |
+| `enableVisibilityAPF`                                  | enable APF for the visibility API                      | `false`                                     |
 | `controllerManager.kubeRbacProxy.image`                | controllerManager.kubeRbacProxy's image                | `gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0` |
 | `controllerManager.manager.image`                      | controllerManager.manager's image                      | `us-central1-docker.pkg.dev/k8s-staging-images/kueue/kueue` |
 | `controllerManager.manager.tag`                        | controllerManager.manager's tag                        | `main`                                      |
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_cohorts.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_cohorts.yaml
index 309ac1e15..bf9c532ed 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_cohorts.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_cohorts.yaml
@@ -32,7 +32,10 @@ spec:
   - name: v1alpha1
     schema:
       openAPIV3Schema:
-        description: Cohort is the Schema for the cohorts API
+        description: |-
+          Cohort is the Schema for the cohorts API. Using Hierarchical
+          Cohorts (any Cohort which has a parent) with Fair Sharing
+          results in undefined behavior in 0.9
         properties:
           apiVersion:
             description: |-
diff --git a/charts/kueue/templates/visibility-apf/flowschema.yaml b/charts/kueue/templates/visibility-apf/flowschema.yaml
new file mode 100644
index 000000000..7b5e90e28
--- /dev/null
+++ b/charts/kueue/templates/visibility-apf/flowschema.yaml
@@ -0,0 +1,33 @@
+{{- if .Values.enableVisibilityAPF }}
+apiVersion: flowcontrol.apiserver.k8s.io/{{ and (eq .Capabilities.KubeVersion.Major "1") (eq .Capabilities.KubeVersion.Minor "28") | ternary "v1beta3" "v1" }}
+kind: FlowSchema
+metadata:
+  labels:
+  {{- include "kueue.labels" . | nindent 4 }}
+  name: '{{ include "kueue.fullname" . }}-visibility'
+  namespace: '{{ .Release.Namespace }}'
+spec:
+  distinguisherMethod:
+    type: ByUser
+  matchingPrecedence: 9000
+  priorityLevelConfiguration:
+    name: kueue-visibility
+  rules:
+    - resourceRules:
+        - apiGroups:
+            - 'visibility.kueue.x-k8s.io'
+          clusterScope: true
+          namespaces:
+            - '*'
+          resources:
+            - '*'
+          verbs:
+            - '*'
+      subjects:
+        - group:
+            name: system:unauthenticated
+          kind: Group
+        - group:
+            name: system:authenticated
+          kind: Group
+{{- end }}
diff --git a/charts/kueue/templates/visibility-apf/prioritylevelconfigurations.yaml b/charts/kueue/templates/visibility-apf/prioritylevelconfigurations.yaml
new file mode 100644
index 000000000..b855863c6
--- /dev/null
+++ b/charts/kueue/templates/visibility-apf/prioritylevelconfigurations.yaml
@@ -0,0 +1,20 @@
+{{- if .Values.enableVisibilityAPF }}
+apiVersion: flowcontrol.apiserver.k8s.io/{{ and (eq .Capabilities.KubeVersion.Major "1") (eq .Capabilities.KubeVersion.Minor "28") | ternary "v1beta3" "v1" }}
+kind: PriorityLevelConfiguration
+metadata:
+  labels:
+  {{- include "kueue.labels" . | nindent 4 }}
+  name: '{{ include "kueue.fullname" . }}-visibility'
+  namespace: '{{ .Release.Namespace }}'
+spec:
+  limited:
+    lendablePercent: 90
+    limitResponse:
+      queuing:
+        handSize: 4
+        queueLengthLimit: 50
+        queues: 16
+      type: Queue
+    nominalConcurrencyShares: 10
+  type: Limited
+{{- end }}
diff --git a/charts/kueue/values.yaml b/charts/kueue/values.yaml
index 0fa2ef3f9..7be554d4a 100644
--- a/charts/kueue/values.yaml
+++ b/charts/kueue/values.yaml
@@ -7,6 +7,8 @@ fullnameOverride: ""
 enablePrometheus: false
 # Enable x509 automated certificate management using cert-manager (cert-manager.io)
 enableCertManager: false
+# Enable API Priority and Fairness configuration for the visibility API
+enableVisibilityAPF: false
 # Customize controllerManager
 controllerManager:
   #featureGates:
diff --git a/config/components/crd/bases/kueue.x-k8s.io_cohorts.yaml b/config/components/crd/bases/kueue.x-k8s.io_cohorts.yaml
index 2716830c8..550c984ee 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_cohorts.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_cohorts.yaml
@@ -17,7 +17,10 @@ spec:
   - name: v1alpha1
     schema:
       openAPIV3Schema:
-        description: Cohort is the Schema for the cohorts API
+        description: |-
+          Cohort is the Schema for the cohorts API. Using Hierarchical
+          Cohorts (any Cohort which has a parent) with Fair Sharing
+          results in undefined behavior in 0.9
         properties:
           apiVersion:
             description: |-
diff --git a/config/components/visibility-apf/flowschema.yaml b/config/components/visibility-apf/flowschema.yaml
new file mode 100644
index 000000000..091cb3105
--- /dev/null
+++ b/config/components/visibility-apf/flowschema.yaml
@@ -0,0 +1,29 @@
+apiVersion: flowcontrol.apiserver.k8s.io/v1
+kind: FlowSchema
+metadata:
+  name: visibility
+  namespace: system
+spec:
+  distinguisherMethod:
+    type: ByUser
+  matchingPrecedence: 9000
+  priorityLevelConfiguration:
+    name: kueue-visibility
+  rules:
+    - resourceRules:
+        - apiGroups:
+            - 'visibility.kueue.x-k8s.io'
+          clusterScope: true
+          namespaces:
+            - '*'
+          resources:
+            - '*'
+          verbs:
+            - '*'
+      subjects:
+        - group:
+            name: system:unauthenticated
+          kind: Group
+        - group:
+            name: system:authenticated
+          kind: Group
diff --git a/config/components/visibility-apf/kustomization.yaml b/config/components/visibility-apf/kustomization.yaml
new file mode 100644
index 000000000..40d982fc9
--- /dev/null
+++ b/config/components/visibility-apf/kustomization.yaml
@@ -0,0 +1,3 @@
+resources:
+- flowschema.yaml
+- prioritylevelconfigurations.yaml
diff --git a/config/components/visibility-apf/prioritylevelconfigurations.yaml b/config/components/visibility-apf/prioritylevelconfigurations.yaml
new file mode 100644
index 000000000..d8faf5fb6
--- /dev/null
+++ b/config/components/visibility-apf/prioritylevelconfigurations.yaml
@@ -0,0 +1,16 @@
+apiVersion: flowcontrol.apiserver.k8s.io/v1
+kind: PriorityLevelConfiguration
+metadata:
+  name: visibility
+  namespace: system
+spec:
+  limited:
+    lendablePercent: 90
+    limitResponse:
+      queuing:
+        handSize: 4
+        queueLengthLimit: 50
+        queues: 16
+      type: Queue
+    nominalConcurrencyShares: 10
+  type: Limited
\ No newline at end of file
diff --git a/config/visibility-apf/1_28/kustomization.yaml b/config/visibility-apf/1_28/kustomization.yaml
new file mode 100644
index 000000000..23832e5e1
--- /dev/null
+++ b/config/visibility-apf/1_28/kustomization.yaml
@@ -0,0 +1,19 @@
+# This overlay builds the APF components for Kubernetes version 1.28 to be used in combination
+# with other overlays.
+
+resources:
+- ../default/
+
+patches:
+  - target:
+      kind: FlowSchema
+    patch: |
+      - op: replace
+        path: /apiVersion
+        value: flowcontrol.apiserver.k8s.io/v1beta3
+  - target:
+      kind: PriorityLevelConfiguration
+    patch: |
+      - op: replace
+        path: /apiVersion
+        value: flowcontrol.apiserver.k8s.io/v1beta3
\ No newline at end of file
diff --git a/config/visibility-apf/default/kustomization.yaml b/config/visibility-apf/default/kustomization.yaml
new file mode 100644
index 000000000..914c81bdf
--- /dev/null
+++ b/config/visibility-apf/default/kustomization.yaml
@@ -0,0 +1,14 @@
+# This overlay builds the APF components to be used in combination
+# with other overlays.
+
+namespace: kueue-system
+namePrefix: kueue-
+
+labels:
+  - pairs:
+      app.kubernetes.io/name: kueue
+      app.kubernetes.io/component: controller
+      control-plane: controller-manager
+
+resources:
+  - ../../components/visibility-apf/
\ No newline at end of file
diff --git a/hack/e2e-common.sh b/hack/e2e-common.sh
index d28a44769..dbeb5ef20 100644
--- a/hack/e2e-common.sh
+++ b/hack/e2e-common.sh
@@ -19,6 +19,8 @@ export GINKGO="$ROOT_DIR"/bin/ginkgo
 export KIND="$ROOT_DIR"/bin/kind
 export YQ="$ROOT_DIR"/bin/yq
 
+export KIND_VERSION="${E2E_KIND_VERSION/"kindest/node:v"/}"
+
 export JOBSET_MANIFEST="https://github.com/kubernetes-sigs/jobset/releases/download/${JOBSET_VERSION}/manifests.yaml"
 export JOBSET_IMAGE=registry.k8s.io/jobset/jobset:${JOBSET_VERSION}
 export JOBSET_CRDS=${ROOT_DIR}/dep-crds/jobset-operator/
@@ -75,7 +77,11 @@ function cluster_kind_load_image {
 # $1 cluster
 function cluster_kueue_deploy {
     kubectl config use-context "kind-${1}"
-    kubectl apply --server-side -k test/e2e/config
+    if [ "${KIND_VERSION%.*}" = "1.28" ]; then
+        kubectl apply --server-side -k test/e2e/config/1_28
+    else
+        kubectl apply --server-side -k test/e2e/config/default
+    fi
 }
 
 #$1 - cluster name
diff --git a/hack/multikueue-e2e-test.sh b/hack/multikueue-e2e-test.sh
index 309bb25f3..0d51346a6 100755
--- a/hack/multikueue-e2e-test.sh
+++ b/hack/multikueue-e2e-test.sh
@@ -51,8 +51,7 @@ function startup {
         if [ ! -d "$ARTIFACTS" ]; then
             mkdir -p "$ARTIFACTS"
         fi
-	
-        KIND_VERSION=${E2E_KIND_VERSION/"kindest/node:v"/}
+
         MANAGER_KIND_CONFIG="${SOURCE_DIR}/multikueue/manager-cluster.kind-${KIND_VERSION}.yaml"
         if [ !  -f "$MANAGER_KIND_CONFIG" ]; then
             MANAGER_KIND_CONFIG="${SOURCE_DIR}/multikueue/manager-cluster.kind.yaml"
diff --git a/hack/update-helm.sh b/hack/update-helm.sh
index 08afc41ff..6b066dc2f 100755
--- a/hack/update-helm.sh
+++ b/hack/update-helm.sh
@@ -23,11 +23,13 @@ SRC_CRD_DIR=config/components/crd/bases
 SRC_RBAC_DIR=config/components/rbac
 SRC_WEBHOOK_DIR=config/components/webhook
 SRC_VISIBILITY_DIR=config/components/visibility
+SRC_VISIBILITY_APF_DIR=config/components/visibility-apf
 
 DEST_CRD_DIR=charts/kueue/templates/crd
 DEST_RBAC_DIR=charts/kueue/templates/rbac
 DEST_WEBHOOK_DIR=charts/kueue/templates/webhook
 DEST_VISIBILITY_DIR=charts/kueue/templates/visibility
+DEST_VISIBILITY_APF_DIR=charts/kueue/templates/visibility-apf
 
 YQ=./bin/yq
 SED=${SED:-/usr/bin/sed}
@@ -48,6 +50,8 @@ find $SRC_RBAC_DIR -name "*.yaml" $EXCLUDE_FILES_ARGS -exec cp "{}" $DEST_RBAC_D
 find $SRC_WEBHOOK_DIR -name "*.yaml" $EXCLUDE_FILES_ARGS -exec cp "{}" $DEST_WEBHOOK_DIR \;
 # shellcheck disable=SC2086
 find $SRC_VISIBILITY_DIR -name "*.yaml" $EXCLUDE_FILES_ARGS -exec cp "{}" $DEST_VISIBILITY_DIR \;
+# shellcheck disable=SC2086
+find $SRC_VISIBILITY_APF_DIR -name "*.yaml" $EXCLUDE_FILES_ARGS -exec cp "{}" $DEST_VISIBILITY_APF_DIR \;
 $YQ -N -s '.kind' ${DEST_WEBHOOK_DIR}/manifests.yaml
 rm ${DEST_WEBHOOK_DIR}/manifests.yaml
 files=("MutatingWebhookConfiguration.yml" "ValidatingWebhookConfiguration.yml")
@@ -298,3 +302,17 @@ EOT
   fi
   $SED -i '/^metadata:.*/a\  labels:\n  {{- include "kueue.labels" . | nindent 4 }}' "$output_file"
 done
+
+# Replace flowcontrol version on visibility-apf directory
+for output_file in "${DEST_VISIBILITY_APF_DIR}"/*.yaml; do
+  $YQ -N -i '.apiVersion = "flowcontrol.apiserver.k8s.io/{{ and (eq .Capabilities.KubeVersion.Major \"1\") (eq .Capabilities.KubeVersion.Minor \"28\") | ternary \"v1beta3\" \"v1\" }}"' "$output_file"
+  $YQ -N -i '.metadata.name |= "{{ include \"kueue.fullname\" . }}-" + .' "$output_file"
+  $YQ -N -i '.metadata.namespace = "{{ .Release.Namespace }}"' "$output_file"
+  $SED -i '/^metadata:.*/a\  labels:\n  {{- include "kueue.labels" . | nindent 4 }}' "$output_file"
+  {
+    echo '{{- if .Values.enableVisibilityAPF }}'
+    cat "$output_file"
+    echo "{{- end }}"
+  } > "${output_file}.tmp"
+  mv "${output_file}.tmp" "${output_file}"
+done
\ No newline at end of file
diff --git a/site/content/en/docs/installation/_index.md b/site/content/en/docs/installation/_index.md
index 236d68bd5..962c84321 100644
--- a/site/content/en/docs/installation/_index.md
+++ b/site/content/en/docs/installation/_index.md
@@ -16,6 +16,7 @@ description: >
   - [Uninstall](#uninstall-1)
 - [Build and install from source](#build-and-install-from-source)
   - [Add metrics scraping for prometheus-operator](#add-metrics-scraping-for-prometheus-operator-1)
+  - [Add API Priority and Fairness configuration for the visibility API](#add-api-priority-and-fairness-configuration-for-the-visibility-api)
   - [Uninstall](#uninstall-2)
 - [Install via Helm](#install-via-helm)
 - [Change the feature gates configuration](#change-the-feature-gates-configuration)
@@ -74,6 +75,10 @@ We can follow `https://prometheus-operator.dev/docs/prologue/quick-start/` to in
 kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/{{< param "version" >}}/prometheus.yaml
 ```
 
+### Add API Priority and Fairness configuration for the visibility API
+
+See [Configure API Priority and Fairness](/docs/tasks/manage/monitor_pending_workloads/pending_workloads_on_demand/#configure-api-priority-and-fairness) for more details.
+
 ### Uninstall
 
 To uninstall a released version of Kueue from your cluster, run the following command:
diff --git a/site/content/en/docs/reference/kueue-alpha.v1alpha1.md b/site/content/en/docs/reference/kueue-alpha.v1alpha1.md
index e759b679f..de80e5296 100644
--- a/site/content/en/docs/reference/kueue-alpha.v1alpha1.md
+++ b/site/content/en/docs/reference/kueue-alpha.v1alpha1.md
@@ -81,7 +81,9 @@ description: Generated API reference documentation for kueue.x-k8s.io/v1alpha1.
 
 
 
-<p>Cohort is the Schema for the cohorts API</p>
+<p>Cohort is the Schema for the cohorts API. Using Hierarchical
+Cohorts (any Cohort which has a parent) with Fair Sharing
+results in undefined behavior in 0.9</p>
 
 
 <table class="table">
diff --git a/site/content/en/docs/tasks/manage/monitor_pending_workloads/pending_workloads_on_demand.md b/site/content/en/docs/tasks/manage/monitor_pending_workloads/pending_workloads_on_demand.md
index d509fc48e..19bee460a 100644
--- a/site/content/en/docs/tasks/manage/monitor_pending_workloads/pending_workloads_on_demand.md
+++ b/site/content/en/docs/tasks/manage/monitor_pending_workloads/pending_workloads_on_demand.md
@@ -26,6 +26,15 @@ Make sure the following conditions are met:
 
 VisibilityOnDemand is an `Alpha` feature disabled by default. To use the visibility API  change [the feature gates configuration](/docs/installation/#change-the-feature-gates-configuration) and set `VisibilityOnDemand=true`.
 
+### Configure API Priority and Fairness:
+
+To install the [API Priority and Fairness](https://kubernetes.io/docs/concepts/cluster-administration/flow-control/) configuration for the visibility API apply one of the manifests, depending on your Kubernetes version:
+
+{{< tabpane lang="shell" persist=disabled >}}
+{{< tab header="Kubernetes 1.29 or newer" >}} kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/{{< param "version" >}}/visibility-apf.yaml {{< /tab >}}
+{{< tab header="Kubernetes 1.28" >}} kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/{{< param "version" >}}/visibility-apf-1-28.yaml {{< /tab >}}
+{{< /tabpane >}}
+
 ## Monitor pending workloads on demand
 
 {{< feature-state state="alpha" for_version="v0.6" >}}



Test patch:
diff --git a/test/e2e/config/1_28/kustomization.yaml b/test/e2e/config/1_28/kustomization.yaml
new file mode 100644
index 000000000..90bbf9daa
--- /dev/null
+++ b/test/e2e/config/1_28/kustomization.yaml
@@ -0,0 +1,3 @@
+resources:
+- ../common
+- ../../../../config/visibility-apf/1_28
\ No newline at end of file
diff --git a/test/e2e/config/controller_manager_config.yaml b/test/e2e/config/common/controller_manager_config.yaml
similarity index 100%
rename from test/e2e/config/controller_manager_config.yaml
rename to test/e2e/config/common/controller_manager_config.yaml
diff --git a/test/e2e/config/kustomization.yaml b/test/e2e/config/common/kustomization.yaml
similarity index 94%
rename from test/e2e/config/kustomization.yaml
rename to test/e2e/config/common/kustomization.yaml
index 56657fe34..dcf86970a 100644
--- a/test/e2e/config/kustomization.yaml
+++ b/test/e2e/config/common/kustomization.yaml
@@ -2,7 +2,7 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 
 resources:
-- ../../../config/dev
+- ../../../../config/dev
 
 replicas:
 - name: kueue-controller-manager
diff --git a/test/e2e/config/manager_e2e_patch.yaml b/test/e2e/config/common/manager_e2e_patch.yaml
similarity index 100%
rename from test/e2e/config/manager_e2e_patch.yaml
rename to test/e2e/config/common/manager_e2e_patch.yaml
diff --git a/test/e2e/config/default/kustomization.yaml b/test/e2e/config/default/kustomization.yaml
new file mode 100644
index 000000000..3c6314485
--- /dev/null
+++ b/test/e2e/config/default/kustomization.yaml
@@ -0,0 +1,3 @@
+resources:
+- ../common
+- ../../../../config/visibility-apf/default
\ No newline at end of file