Repository: kubernetes-sigs/kueue

Issue:
Parametrize the priority of kueue-controller-manager in Helm
**What would you like to be added**:

Helm parameter to allow setting the priority class for `kueue-controller-manager`.

Name proposal: `controllerManager.manager.priorityClassName`.

**Why is this needed**:

On some deployments Kueue is a critical Pod,  and to reflect that users would like to set the priority as `system-cluster-critical`. 

This, for example, allows to make sure Kueue can find quickly replacement node when the original node fails.


Discussion:
@mimowo: cc @tenzen-y @mbobrovskyi @mimowo: cc @mwysokin @mimowo: cc @kaisoz in case you have free cycles@kaisoz: /assign 

Yes! I can work on this thanks!

Solution patch:
diff --git a/Makefile b/Makefile
index c45770aeb..b3e87fb80 100644
--- a/Makefile
+++ b/Makefile
@@ -178,6 +178,8 @@ helm-verify: helm helm-lint ## run helm template and detect any rendering failur
 	$(HELM) template charts/kueue --set enableKueueViz=true --set enableCertManager=true --set enablePrometheus=true > /dev/null
 # test added managedJobsNamespaceSelector option
 	$(HELM) template charts/kueue --set managerConfig.controllerManagerConfigYaml="managedJobsNamespaceSelector:\n  matchExpressions:\n    - key: kubernetes.io/metadata.name\n      operator: In\n      values: [ kube-system ]" > /dev/null
+# test priorityClassName option
+	$(HELM) template charts/kueue --set controllerManager.manager.priorityClassName="system-cluster-critical" > /dev/null
 
 # test
 .PHONY: helm-unit-test
diff --git a/charts/kueue/README.md b/charts/kueue/README.md
index e286c3e37..9240f2fe6 100644
--- a/charts/kueue/README.md
+++ b/charts/kueue/README.md
@@ -108,6 +108,7 @@ The following table lists the configurable parameters of the kueue chart and the
 | `kueueViz.backend.image`                               | KueueViz dashboard backend image                       | `us-central1-docker.pkg.dev/k8s-staging-images/kueue/kueueviz-backend:main`  |
 | `kueueViz.frontend.image`                              | KueueViz dashboard frontend image                      | `us-central1-docker.pkg.dev/k8s-staging-images/kueue/kueueviz-frontend:main` |
 | `controllerManager.manager.featureGates`               | controllerManager.manager's feature gates              | `[]`                                                                         |
+| `controllerManager.manager.priorityClassName`          | controllerManager.manager's Pod priorityClassName      | ``                                          |
 | `controllerManager.manager.image.repository`           | controllerManager.manager's repository and image       | `us-central1-docker.pkg.dev/k8s-staging-images/kueue/kueue` |
 | `controllerManager.manager.image.tag`                  | controllerManager.manager's tag                        | `main`                                      |
 | `controllerManager.manager.resources`                  | controllerManager.manager's resources                  | abbr.                                       |
diff --git a/charts/kueue/templates/manager/manager.yaml b/charts/kueue/templates/manager/manager.yaml
index 98d2c81b4..8c0f1999e 100644
--- a/charts/kueue/templates/manager/manager.yaml
+++ b/charts/kueue/templates/manager/manager.yaml
@@ -21,6 +21,9 @@ spec:
         {{- toYaml .Values.controllerManager.manager.podAnnotations | nindent 8 }}
         {{- end }}
     spec:
+      {{- with  .Values.controllerManager.manager.priorityClassName }}
+      priorityClassName: {{ . }}
+      {{- end }}
       containers:
       - args:
         - --config=/controller_manager_config.yaml
diff --git a/charts/kueue/values.yaml b/charts/kueue/values.yaml
index 8f34d3aa4..76c544f76 100644
--- a/charts/kueue/values.yaml
+++ b/charts/kueue/values.yaml
@@ -15,6 +15,7 @@ controllerManager:
   #  - name: PartialAdmission
   #    enabled: true
   manager:
+    # priorityClassName: "system-cluster-critical"
     image:
       repository: us-central1-docker.pkg.dev/k8s-staging-images/kueue/kueue
       # This should be set to 'IfNotPresent' for released version



Test patch:
diff --git a/charts/kueue/tests/manager_test.yaml b/charts/kueue/tests/manager_test.yaml
index 875dcf28c..aa76775b0 100644
--- a/charts/kueue/tests/manager_test.yaml
+++ b/charts/kueue/tests/manager_test.yaml
@@ -13,3 +13,22 @@ tests:
       - equal:
           path: spec.replicas
           value: 2
+  - it: should set the pod priorityClassName when provided
+    template: manager/manager.yaml
+    set:
+      controllerManager:
+        manager:
+          priorityClassName: "foo"
+    asserts:
+      - equal:
+          path: spec.template.spec.priorityClassName
+          value: foo
+  - it: should not render the pod priorityClassName if not set
+    template: manager/manager.yaml
+    set:
+      controllerManager:
+        manager:
+          priorityClassName: ""
+    asserts:
+      - notExists:
+          path: spec.template.spec.priorityClassName