Repository: kubernetes-sigs/kueue

Issue:
[flaky integration test] Preemption is enabled within ClusterQueue should preempt the low and mid priority workloads to fit the high-priority workload
/kind flake 

**What happened**:

failure on unrelated branch https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/5394/pull-kueue-test-integration-baseline-main/1927649254336106496

**What you expected to happen**:
no failure
**How to reproduce it (as minimally and precisely as possible)**:
ci
**Anything else we need to know?**:

```
TopologyAwareScheduling Suite: [It] Topology Aware Scheduling when Single TAS Resource Flavor when Preemption is enabled within ClusterQueue should preempt the low and mid priority workloads to fit the high-priority workload expand_less	13s
{Timed out after 10.001s.
The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/util/util.go:365 with:
Not enough workloads are admitted
Expected
    <int>: 1
to equal
    <int>: 2 failed [FAILED] Timed out after 10.001s.
The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/util/util.go:365 with:
Not enough workloads are admitted
Expected
    <int>: 1
to equal
    <int>: 2
In [It] at: /home/prow/go/src/sigs.k8s.io/kueue/test/integration/singlecluster/tas/tas_test.go:1374 @ 05/28/25 09:06:47.971
}
```

Discussion:
@mimowo: cc @mbobrovskyi @mykysha ptal@tenzen-y: We observed the same failure on #5416, again.@mimowo: cc @mszadkow @mszadkow: /assign@mszadkow: Something very strange happened in that test run.
`wl1` got admitted and later it was verified, when `wl2` was evaluated for admission `wl1` disappeared without a trace...
What I mean is there was no `delete` or `preemption` recorded, maybe just this attempt to preempt `wl1` which meanwhile disappear.
 
```
2025-05-28T09:06:37.97242884Z	LEVEL(-2)	scheduler	scheduler/scheduler.go:226	Attempting to schedule workload	{"schedulingCycle": 38, "workload": {"name":"wl2","namespace":"tas-94dv5"}, "clusterQueue": {"name":"cluster-queue"}}
  2025-05-28T09:06:37.975342209Z	LEVEL(-2)	scheduler	scheduler/scheduler.go:572	Workload not admitted because it was deleted	{"schedulingCycle": 37, "workload": {"name":"wl1","namespace":"tas-7vrc8"}, "clusterQueue": {"name":"cluster-queue"}}
  2025-05-28T09:06:37.978364299Z	ERROR	scheduler	scheduler/scheduler.go:268	Failed to preempt workloads	{"schedulingCycle": 38, "workload": {"name":"wl2","namespace":"tas-94dv5"}, "clusterQueue": {"name":"cluster-queue"}, "error": "workloads.kueue.x-k8s.io \"wl1\" not found"}
  sigs.k8s.io/kueue/pkg/scheduler.(*Scheduler).schedule
  	/home/prow/go/src/sigs.k8s.io/kueue/pkg/scheduler/scheduler.go:268
  sigs.k8s.io/kueue/pkg/util/wait.untilWithBackoff.func1
  	/home/prow/go/src/sigs.k8s.io/kueue/pkg/util/wait/backoff.go:43
  k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
  	/home/prow/go/src/sigs.k8s.io/kueue/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:233
  k8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1
  	/home/prow/go/src/sigs.k8s.io/kueue/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:255
  k8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext
  	/home/prow/go/src/sigs.k8s.io/kueue/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:256
  k8s.io/apimachinery/pkg/util/wait.BackoffUntil
  	/home/prow/go/src/sigs.k8s.io/kueue/vendor/k8s.io/apimachinery/pkg/util/wait/backoff.go:233
  sigs.k8s.io/kueue/pkg/util/wait.untilWithBackoff
  	/home/prow/go/src/sigs.k8s.io/kueue/pkg/util/wait/backoff.go:42
  sigs.k8s.io/kueue/pkg/util/wait.UntilWithBackoff
  	/home/prow/go/src/sigs.k8s.io/kueue/pkg/util/wait/backoff.go:34
  2025-05-28T09:06:37.978562602Z	LEVEL(-3)	scheduler	scheduler/logging.go:43	Workload evaluated for admission	{"schedulingCycle": 38, "workload": {"name":"wl2","namespace":"tas-94dv5"}, "clusterQueue": {"name":"cluster-queue"}, "status": "", "reason": "couldn't assign flavors to pod set worker: topology \"default\" doesn't allow to fit any of 1 pod(s)"}
```@mimowo: Could you try to repro locally by looping the test under some load? Maybe we could capture it locally.@mszadkow: I have tried already, nothing happens, works smoothly.
But I will keep trying, I just wanted to share this "weird" thing that `wl1` namespace somehow got changed.
From `tas-94dv5` to `tas-7vrc8`, which is from the previous test case...

```
  2025-05-28T09:06:37.710423748Z	LEVEL(-2)	workload-reconciler	core/workload_controller.go:747	Workload update event	{"workload": {"name":"wl1","namespace":"tas-94dv5"}, "queue": "local-queue", "status": "admitted", "prevStatus": "pending", "clusterQueue": "cluster-queue"}
...
2025-05-28T09:06:37.975342209Z	LEVEL(-2)	scheduler	scheduler/scheduler.go:572	Workload not admitted because it was deleted	{"schedulingCycle": 37, "workload": {"name":"wl1","namespace":"tas-7vrc8"}, "clusterQueue": {"name":"cluster-queue"}}
```
My suspect is that we share lq and cq as pointers between tests...
But I would like to get repro anyway@mimowo: Thank you for investigating that. Indeed, I don't exclude the possibility of cross-test interaction, this would be hard to capture. Even if you are out of ideas for now, please think if we could improve the logging (either in tests) or in code which could be helpful next time.@mimowo: another recent occurence: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/5433/pull-kueue-test-integration-baseline-release-0-12/1929436649175912448@mszadkow: Yes, I believe the fix is ready to be merged - https://github.com/kubernetes-sigs/kueue/pull/5423@mbobrovskyi: Looks like we still have an issue https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/5443/pull-kueue-test-integration-baseline-main/1929489390048907264.@mbobrovskyi: /reopen@k8s-ci-robot: @mbobrovskyi: Reopened this issue.

<details>

In response to [this](https://github.com/kubernetes-sigs/kueue/issues/5395#issuecomment-2930073887):

>/reopen


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>@mbobrovskyi: One more

https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/5225/pull-kueue-test-integration-baseline-main/1929477214840033280@mszadkow: ack, looking into logs@mimowo: same here: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/5440/pull-kueue-test-integration-baseline-main/1929495861822230528@mszadkow: The next suspect is the `queue manager` as it seems that `wl1` from the previous test is considered for preemption.
While it should be `wl1` from the current test and `wl1` from previous test should be removed from the queue already.
Investigating there...although still it's not possible to reproduce locally.@mimowo: Hmm, this might be a bug related to this code: https://github.com/kubernetes-sigs/kueue/blob/e3bbc66fa7c95df49668a33934e9ae9c97d9b65a/pkg/queue/manager.go#L793-L795

It might be the goroutine is spawned to add the workload after 1s and it does not consider the context is already cancelled when test ended.

Another fix might be to do not enqueue if the CQ no longer exists. 

To test this hypothesis locally maybe you could extend 1s to 2s.@tenzen-y: We observed this again.

https://github.com/kubernetes-sigs/kueue/pull/5432
https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/5432/pull-kueue-test-integration-baseline-main/1930110922383167488@tenzen-y: Once more:

- https://github.com/kubernetes-sigs/kueue/pull/5491
- https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/5491/pull-kueue-test-integration-baseline-release-0-12/1930119799250096128

Solution patch:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_admissionchecks.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_admissionchecks.yaml
index 02420e935..9d366c765 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_admissionchecks.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_admissionchecks.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: admissionchecks.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_clusterqueues.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_clusterqueues.yaml
index 2b4ded851..2267a0910 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_clusterqueues.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_clusterqueues.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: clusterqueues.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_cohorts.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_cohorts.yaml
index 53226b1a7..e85c6c619 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_cohorts.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_cohorts.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: cohorts.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_localqueues.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_localqueues.yaml
index b917d07f7..46379fe6e 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_localqueues.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_localqueues.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: localqueues.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_multikueueclusters.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_multikueueclusters.yaml
index efa955c99..52ddfdc8e 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_multikueueclusters.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_multikueueclusters.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: multikueueclusters.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_multikueueconfigs.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_multikueueconfigs.yaml
index 13dd68eda..102a1d4f8 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_multikueueconfigs.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_multikueueconfigs.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: multikueueconfigs.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_provisioningrequestconfigs.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_provisioningrequestconfigs.yaml
index 4badc39ae..2f9651c0b 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_provisioningrequestconfigs.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_provisioningrequestconfigs.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: provisioningrequestconfigs.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_resourceflavors.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_resourceflavors.yaml
index 059cf076b..92ee1e2df 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_resourceflavors.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_resourceflavors.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: resourceflavors.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_topologies.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_topologies.yaml
index dbb037579..abb911050 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_topologies.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_topologies.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: topologies.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_workloadpriorityclasses.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_workloadpriorityclasses.yaml
index cd10a55ea..5fd06997c 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_workloadpriorityclasses.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_workloadpriorityclasses.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: workloadpriorityclasses.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/charts/kueue/templates/crd/kueue.x-k8s.io_workloads.yaml b/charts/kueue/templates/crd/kueue.x-k8s.io_workloads.yaml
index f598bd662..c00943ce9 100644
--- a/charts/kueue/templates/crd/kueue.x-k8s.io_workloads.yaml
+++ b/charts/kueue/templates/crd/kueue.x-k8s.io_workloads.yaml
@@ -7,7 +7,7 @@ metadata:
     {{- if .Values.enableCertManager }}
     cert-manager.io/inject-ca-from: '{{ .Release.Namespace }}/{{ include "kueue.fullname" . }}-serving-cert'
     {{- end }}
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: workloads.kueue.x-k8s.io
 spec:
   conversion:
diff --git a/config/components/crd/bases/kueue.x-k8s.io_admissionchecks.yaml b/config/components/crd/bases/kueue.x-k8s.io_admissionchecks.yaml
index 91df8df24..76b0bc2d2 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_admissionchecks.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_admissionchecks.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: admissionchecks.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_clusterqueues.yaml b/config/components/crd/bases/kueue.x-k8s.io_clusterqueues.yaml
index b65162200..c17d9291b 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_clusterqueues.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_clusterqueues.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: clusterqueues.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_cohorts.yaml b/config/components/crd/bases/kueue.x-k8s.io_cohorts.yaml
index 7e394e8d8..f99a8589a 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_cohorts.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_cohorts.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: cohorts.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_localqueues.yaml b/config/components/crd/bases/kueue.x-k8s.io_localqueues.yaml
index caeaf663d..11f823509 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_localqueues.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_localqueues.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: localqueues.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_multikueueclusters.yaml b/config/components/crd/bases/kueue.x-k8s.io_multikueueclusters.yaml
index 4ee3a353c..df4436884 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_multikueueclusters.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_multikueueclusters.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: multikueueclusters.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_multikueueconfigs.yaml b/config/components/crd/bases/kueue.x-k8s.io_multikueueconfigs.yaml
index f41933f1e..7e4a9c353 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_multikueueconfigs.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_multikueueconfigs.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: multikueueconfigs.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_provisioningrequestconfigs.yaml b/config/components/crd/bases/kueue.x-k8s.io_provisioningrequestconfigs.yaml
index 5f431a12c..9104178f5 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_provisioningrequestconfigs.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_provisioningrequestconfigs.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: provisioningrequestconfigs.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_resourceflavors.yaml b/config/components/crd/bases/kueue.x-k8s.io_resourceflavors.yaml
index 8fed8f34e..92ec29b28 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_resourceflavors.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_resourceflavors.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: resourceflavors.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_topologies.yaml b/config/components/crd/bases/kueue.x-k8s.io_topologies.yaml
index 14b6debc6..8a73dac4a 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_topologies.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_topologies.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: topologies.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_workloadpriorityclasses.yaml b/config/components/crd/bases/kueue.x-k8s.io_workloadpriorityclasses.yaml
index 7d4513ece..52a25e37b 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_workloadpriorityclasses.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_workloadpriorityclasses.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: workloadpriorityclasses.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/config/components/crd/bases/kueue.x-k8s.io_workloads.yaml b/config/components/crd/bases/kueue.x-k8s.io_workloads.yaml
index 9b951f50b..34e0f1dad 100644
--- a/config/components/crd/bases/kueue.x-k8s.io_workloads.yaml
+++ b/config/components/crd/bases/kueue.x-k8s.io_workloads.yaml
@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.17.3
+    controller-gen.kubebuilder.io/version: v0.18.0
   name: workloads.kueue.x-k8s.io
 spec:
   group: kueue.x-k8s.io
diff --git a/go.mod b/go.mod
index 217174620..3714a00df 100644
--- a/go.mod
+++ b/go.mod
@@ -5,7 +5,7 @@ go 1.24.0
 require (
 	github.com/cert-manager/cert-manager v1.17.2
 	github.com/fsnotify/fsnotify v1.9.0
-	github.com/go-logr/logr v1.4.2
+	github.com/go-logr/logr v1.4.3
 	github.com/google/go-cmp v0.7.0
 	github.com/json-iterator/go v1.1.12
 	github.com/kubeflow/mpi-operator v0.6.0
diff --git a/go.sum b/go.sum
index c9ab68560..a5b36fb15 100644
--- a/go.sum
+++ b/go.sum
@@ -54,8 +54,8 @@ github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXE
 github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
 github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
 github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
-github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
-github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
+github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
+github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
 github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
 github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
 github.com/go-logr/zapr v1.3.0 h1:XGdV8XW8zdwFiwOA2Dryh1gj2KRQyOOoNmBy4EplIcQ=
diff --git a/hack/internal/tools/go.mod b/hack/internal/tools/go.mod
index 3b0af3781..418994d06 100644
--- a/hack/internal/tools/go.mod
+++ b/hack/internal/tools/go.mod
@@ -15,7 +15,7 @@ require (
 	helm.sh/helm/v3 v3.18.1
 	k8s.io/code-generator v0.33.1
 	sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20250407161039-6ad5c1dd4418
-	sigs.k8s.io/controller-tools v0.17.3
+	sigs.k8s.io/controller-tools v0.18.0
 	sigs.k8s.io/kind v0.29.0
 	sigs.k8s.io/kustomize/kustomize/v5 v5.6.0
 	sigs.k8s.io/mdtoc v1.4.0
diff --git a/hack/internal/tools/go.sum b/hack/internal/tools/go.sum
index 04a5eaa74..472899b15 100644
--- a/hack/internal/tools/go.sum
+++ b/hack/internal/tools/go.sum
@@ -179,8 +179,8 @@ github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
 github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=
 github.com/onsi/ginkgo/v2 v2.23.4 h1:ktYTpKJAVZnDT4VjxSbiBenUjmlL/5QkBEocaWXiQus=
 github.com/onsi/ginkgo/v2 v2.23.4/go.mod h1:Bt66ApGPBFzHyR+JO10Zbt0Gsp4uWxu5mIOTusL46e8=
-github.com/onsi/gomega v1.36.3 h1:hID7cr8t3Wp26+cYnfcjR6HpJ00fdogN6dqZ1t6IylU=
-github.com/onsi/gomega v1.36.3/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=
+github.com/onsi/gomega v1.37.0 h1:CdEG8g0S133B4OswTDC/5XPSzE1OeP29QOioj2PID2Y=
+github.com/onsi/gomega v1.37.0/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=
 github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
 github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
 github.com/peterbourgon/diskv v2.0.1+incompatible h1:UBdAOUP5p4RWqPBg048CAvpKN+vxiaj6gdUUzhl4XmI=
@@ -343,8 +343,8 @@ k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 h1:M3sRQVHv7vB20Xc2ybTt7ODCeFj6J
 k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
 sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20250407161039-6ad5c1dd4418 h1:04bBdoLjAbJzwbBTmcu/PukrGLTBj2suuFmQSEpmCH8=
 sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20250407161039-6ad5c1dd4418/go.mod h1:bJr92VVEbIDLODpQ2NPN7Wlwsdv7uGzTAUvi9/gCgcc=
-sigs.k8s.io/controller-tools v0.17.3 h1:lwFPLicpBKLgIepah+c8ikRBubFW5kOQyT88r3EwfNw=
-sigs.k8s.io/controller-tools v0.17.3/go.mod h1:1ii+oXcYZkxcBXzwv3YZBlzjt1fvkrCGjVF73blosJI=
+sigs.k8s.io/controller-tools v0.18.0 h1:rGxGZCZTV2wJreeRgqVoWab/mfcumTMmSwKzoM9xrsE=
+sigs.k8s.io/controller-tools v0.18.0/go.mod h1:gLKoiGBriyNh+x1rWtUQnakUYEujErjXs9pf+x/8n1U=
 sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 h1:/Rv+M11QRah1itp8VhT6HoVx1Ray9eB4DBr+K+/sCJ8=
 sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3/go.mod h1:18nIHnGi6636UCz6m8i4DhaJ65T6EruyzmoQqI2BVDo=
 sigs.k8s.io/kind v0.29.0 h1:3TpCsyh908IkXXpcSnsMjWdwdWjIl7o9IMZImZCWFnI=
diff --git a/vendor/github.com/go-logr/logr/.golangci.yaml b/vendor/github.com/go-logr/logr/.golangci.yaml
index 0cffafa7b..0ed62c1a1 100644
--- a/vendor/github.com/go-logr/logr/.golangci.yaml
+++ b/vendor/github.com/go-logr/logr/.golangci.yaml
@@ -1,26 +1,28 @@
+version: "2"
+
 run:
   timeout: 1m
   tests: true
 
 linters:
-  disable-all: true
-  enable:
+  default: none
+  enable: # please keep this alphabetized
+    - asasalint
     - asciicheck
+    - copyloopvar
+    - dupl
     - errcheck
     - forcetypeassert
+    - goconst
     - gocritic
-    - gofmt
-    - goimports
-    - gosimple
     - govet
     - ineffassign
     - misspell
+    - musttag
     - revive
     - staticcheck
-    - typecheck
     - unused
 
 issues:
-  exclude-use-default: false
   max-issues-per-linter: 0
   max-same-issues: 10
diff --git a/vendor/github.com/go-logr/logr/funcr/funcr.go b/vendor/github.com/go-logr/logr/funcr/funcr.go
index 30568e768..b22c57d71 100644
--- a/vendor/github.com/go-logr/logr/funcr/funcr.go
+++ b/vendor/github.com/go-logr/logr/funcr/funcr.go
@@ -77,7 +77,7 @@ func newSink(fn func(prefix, args string), formatter Formatter) logr.LogSink {
 		write:     fn,
 	}
 	// For skipping fnlogger.Info and fnlogger.Error.
-	l.Formatter.AddCallDepth(1)
+	l.AddCallDepth(1) // via Formatter
 	return l
 }
 
@@ -164,17 +164,17 @@ type fnlogger struct {
 }
 
 func (l fnlogger) WithName(name string) logr.LogSink {
-	l.Formatter.AddName(name)
+	l.AddName(name) // via Formatter
 	return &l
 }
 
 func (l fnlogger) WithValues(kvList ...any) logr.LogSink {
-	l.Formatter.AddValues(kvList)
+	l.AddValues(kvList) // via Formatter
 	return &l
 }
 
 func (l fnlogger) WithCallDepth(depth int) logr.LogSink {
-	l.Formatter.AddCallDepth(depth)
+	l.AddCallDepth(depth) // via Formatter
 	return &l
 }
 
diff --git a/vendor/modules.txt b/vendor/modules.txt
index 08a73b0ee..c8a83dd70 100644
--- a/vendor/modules.txt
+++ b/vendor/modules.txt
@@ -77,7 +77,7 @@ github.com/fxamacker/cbor/v2
 # github.com/go-errors/errors v1.4.2
 ## explicit; go 1.14
 github.com/go-errors/errors
-# github.com/go-logr/logr v1.4.2
+# github.com/go-logr/logr v1.4.3
 ## explicit; go 1.18
 github.com/go-logr/logr
 github.com/go-logr/logr/funcr



Test patch:
diff --git a/test/e2e/tas/node_failure_test.go b/test/e2e/tas/node_failure_test.go
index fd487574c..14c4938b5 100644
--- a/test/e2e/tas/node_failure_test.go
+++ b/test/e2e/tas/node_failure_test.go
@@ -137,9 +137,8 @@ var _ = ginkgo.Describe("NodeFailure Controller", ginkgo.Ordered, func() {
 			ginkgo.DeferCleanup(func() {
 				ginkgo.By(fmt.Sprintf("Restoring original Ready status of node %s", node.Name))
 				util.SetNodeCondition(ctx, k8sClient, node, &corev1.NodeCondition{
-					Type:               corev1.NodeReady,
-					Status:             corev1.ConditionTrue,
-					LastTransitionTime: metav1.NewTime(time.Now()),
+					Type:   corev1.NodeReady,
+					Status: corev1.ConditionTrue,
 				})
 			})
 
@@ -191,9 +190,8 @@ var _ = ginkgo.Describe("NodeFailure Controller", ginkgo.Ordered, func() {
 				util.MustCreate(ctx, k8sClient, originalNode)
 
 				util.SetNodeCondition(ctx, k8sClient, originalNode, &corev1.NodeCondition{
-					Type:               corev1.NodeReady,
-					Status:             corev1.ConditionTrue,
-					LastTransitionTime: metav1.NewTime(time.Now()),
+					Type:   corev1.NodeReady,
+					Status: corev1.ConditionTrue,
 				})
 			})
 
diff --git a/test/integration/singlecluster/tas/tas_test.go b/test/integration/singlecluster/tas/tas_test.go
index 22d5efc88..02eed7b73 100644
--- a/test/integration/singlecluster/tas/tas_test.go
+++ b/test/integration/singlecluster/tas/tas_test.go
@@ -992,14 +992,10 @@ var _ = ginkgo.Describe("Topology Aware Scheduling", ginkgo.Ordered, func() {
 				ginkgo.By("making the node NotReady", func() {
 					nodeToUpdate := &corev1.Node{}
 					gomega.Expect(k8sClient.Get(ctx, client.ObjectKey{Name: nodeName}, nodeToUpdate)).Should(gomega.Succeed())
-					for i, cond := range nodeToUpdate.Status.Conditions {
-						if cond.Type == corev1.NodeReady {
-							nodeToUpdate.Status.Conditions[i].Status = corev1.ConditionFalse
-							nodeToUpdate.Status.Conditions[i].LastTransitionTime = metav1.NewTime(time.Now())
-							break
-						}
-					}
-					gomega.Expect(k8sClient.Status().Update(ctx, nodeToUpdate)).Should(gomega.Succeed())
+					util.SetNodeCondition(ctx, k8sClient, nodeToUpdate, &corev1.NodeCondition{
+						Type:   corev1.NodeReady,
+						Status: corev1.ConditionFalse,
+					})
 				})
 
 				ginkgo.By("verify the workload does not have the NodeToReplaceAnnotation", func() {
diff --git a/test/util/util.go b/test/util/util.go
index c02019073..ff578eabb 100644
--- a/test/util/util.go
+++ b/test/util/util.go
@@ -1072,7 +1072,6 @@ func DeactivateWorkload(ctx context.Context, c client.Client, key client.ObjectK
 	}, Timeout, Interval).Should(gomega.Succeed())
 }
 
-// conditionType corev1.NodeConditionType, conditionStatus corev1.ConditionStatus, time time.Time
 func SetNodeCondition(ctx context.Context, k8sClient client.Client, node *corev1.Node, newCondition *corev1.NodeCondition) {
 	gomega.EventuallyWithOffset(1, func(g gomega.Gomega) {
 		var updatedNode corev1.Node
@@ -1082,11 +1081,18 @@ func SetNodeCondition(ctx context.Context, k8sClient client.Client, node *corev1
 		if condition == nil {
 			updatedNode.Status.Conditions = append(updatedNode.Status.Conditions, *newCondition)
 			changed = true
-		} else if condition.Status != newCondition.Status {
-			condition.Status = newCondition.Status
+		}
+		if newCondition.LastTransitionTime.IsZero() {
+			condition.LastTransitionTime = metav1.Now()
+			changed = true
+		} else if condition.LastTransitionTime != newCondition.LastTransitionTime {
 			condition.LastTransitionTime = newCondition.LastTransitionTime
 			changed = true
 		}
+		if condition.Status != newCondition.Status {
+			condition.Status = newCondition.Status
+			changed = true
+		}
 		if changed {
 			g.Expect(k8sClient.Status().Update(ctx, &updatedNode)).To(gomega.Succeed())
 		}
diff --git a/vendor/github.com/go-logr/logr/testr/testr.go b/vendor/github.com/go-logr/logr/testr/testr.go
index 5eabe2b6f..28168e0ae 100644
--- a/vendor/github.com/go-logr/logr/testr/testr.go
+++ b/vendor/github.com/go-logr/logr/testr/testr.go
@@ -130,12 +130,12 @@ type testloggerInterface struct {
 }
 
 func (l testloggerInterface) WithName(name string) logr.LogSink {
-	l.Formatter.AddName(name)
+	l.AddName(name) // via Formatter
 	return &l
 }
 
 func (l testloggerInterface) WithValues(kvList ...any) logr.LogSink {
-	l.Formatter.AddValues(kvList)
+	l.AddValues(kvList) // via Formatter
 	return &l
 }