Repository: kubernetes-sigs/kueue

Issue:
[Flaky] Scheduler when Using AdmissionFairSharing at Cohort level should preempt a workload from LQ with higher recent usage
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!

If the matter is security related, please disclose it privately via https://kubernetes.io/security/
-->


**What happened**:
Scheduler Fair Sharing Suite: [It] Scheduler when Using AdmissionFairSharing at Cohort level should preempt a workload from LQ with higher recent usage

```
{Timed out after 10.001s.
The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/util/util.go:431 with:
Not enough workloads are preempted
Expected
    <int>: 0
to equal
    <int>: 1 failed [FAILED] Timed out after 10.001s.
The function passed to Eventually failed at /home/prow/go/src/sigs.k8s.io/kueue/test/util/util.go:431 with:
Not enough workloads are preempted
Expected
    <int>: 0
to equal
    <int>: 1
In [It] at: /home/prow/go/src/sigs.k8s.io/kueue/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go:508 @ 07/28/25 09:04:04.895
}
```

**What you expected to happen**:
No errors

**How to reproduce it (as minimally and precisely as possible)**:
https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/5174/pull-kueue-test-integration-baseline-main/1949754497840451584

**Anything else we need to know?**:

/kind flake

**Environment**:
- Kubernetes version (use `kubectl version`):
- Kueue version (use `git describe --tags --dirty --always`):
- Cloud provider or hardware configuration:
- OS (e.g: `cat /etc/os-release`):
- Kernel (e.g. `uname -a`):
- Install tools:
- Others:

Discussion:
@mimowo: cc @PBundyra ptal@mimowo: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/6283/pull-kueue-test-integration-baseline-main/1950559248463171584@IrvingMg: /assign@mimowo: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/6297/pull-kueue-test-integration-baseline-main/1950867693955452928 @IrvingMg: > https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/6297/pull-kueue-test-integration-baseline-main/1950867693955452928

Actually, this is a different test. I've opened #6324.
@mimowo: https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-kueue-test-integration-main/1951171153938616320
happened on nightly@mbobrovskyi: One more https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/6345/pull-kueue-test-integration-baseline-main/1951184148903235584

Solution patch:
diff --git a/CHANGELOG/CHANGELOG-0.12.md b/CHANGELOG/CHANGELOG-0.12.md
index d6d9e7cbb..0223d522f 100644
--- a/CHANGELOG/CHANGELOG-0.12.md
+++ b/CHANGELOG/CHANGELOG-0.12.md
@@ -1,3 +1,36 @@
+## v0.12.6
+
+Changes since `v0.12.5`:
+
+## Urgent Upgrade Notes
+
+### (No, really, you MUST read this before you upgrade)
+
+- Rename kueue-metrics-certs to kueue-metrics-cert cert-manager.io/v1 Certificate name in cert-manager manifests when installing Kueue using the Kustomize configuration.
+
+  If you're using cert-manager and have deployed Kueue using the Kustomize configuration, you must delete the existing kueue-metrics-certs cert-manager.io/v1 Certificate before applying the new changes to avoid conflicts. (#6361, @mbobrovskyi)
+
+## Changes by Kind
+
+### Bug or Regression
+
+- Fix accounting for the `evicted_workloads_once_total` metric:
+  - the metric wasn't incremented for workloads evicted due to stopped LocalQueue (LocalQueueStopped reason)
+  - the reason used for the metric was "Deactivated" for workloads deactivated by users and Kueue, now the reason label can have the following values: Deactivated, DeactivatedDueToAdmissionCheck, DeactivatedDueToMaximumExecutionTimeExceeded, DeactivatedDueToRequeuingLimitExceeded. This approach aligns the metric with `evicted_workloads_total`.
+  - the metric was incremented during preemption before the preemption request was issued. Thus, it could be incorrectly over-counted in case of the preemption request failure.
+  - the metric was not incremented for workload evicted due to NodeFailures (TAS)
+
+  The existing and introduced DeactivatedDueToXYZ reason label values will be replaced by the single "Deactivated" reason label value and underlying_cause in the future release. (#6363, @mimowo)
+- Fix the bug which could occasionally cause workloads evicted by the built-in AdmissionChecks
+  (ProvisioningRequest and MultiKueue) to get stuck in the evicted state which didn't allow re-scheduling.
+  This could happen when the AdmissionCheck controller would trigger eviction by setting the
+  Admission check state to "Retry". (#6301, @mimowo)
+- Fixed a bug that prevented adding the kueue- prefix to the secretName field in cert-manager manifests when installing Kueue using the Kustomize configuration. (#6344, @mbobrovskyi)
+- ProvisioningRequest: Fix a bug that Kueue didn't recreate the next ProvisioningRequest instance after the
+  second (and consecutive) failed attempt. (#6330, @PBundyra)
+- Support disabling client-side ratelimiting in Config API clientConnection.qps with a negative value (e.g., -1) (#6306, @tenzen-y)
+- TAS: Fix a bug that the node failure controller tries to re-schedule Pods on the failure node even after the Node is recovered and reappears (#6348, @pajakd)
+
 ## v0.12.5
 
 Changes since `v0.12.4`:



Test patch:
diff --git a/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go b/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go
index 067ae807d..4fa932f1f 100644
--- a/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go
+++ b/test/integration/singlecluster/scheduler/fairsharing/fair_sharing_test.go
@@ -17,6 +17,8 @@ limitations under the License.
 package fairsharing
 
 import (
+	"time"
+
 	"github.com/onsi/ginkgo/v2"
 	"github.com/onsi/gomega"
 	corev1 "k8s.io/api/core/v1"
@@ -26,6 +28,7 @@ import (
 	"k8s.io/utils/ptr"
 	"sigs.k8s.io/controller-runtime/pkg/client"
 
+	config "sigs.k8s.io/kueue/apis/config/v1beta1"
 	kueue "sigs.k8s.io/kueue/apis/kueue/v1beta1"
 	"sigs.k8s.io/kueue/pkg/features"
 	"sigs.k8s.io/kueue/pkg/metrics"
@@ -34,7 +37,7 @@ import (
 	"sigs.k8s.io/kueue/test/util"
 )
 
-var _ = ginkgo.Describe("Scheduler", func() {
+var _ = ginkgo.Describe("Scheduler", ginkgo.Ordered, ginkgo.ContinueOnFailure, func() {
 	var (
 		defaultFlavor *kueue.ResourceFlavor
 		ns            *corev1.Namespace
@@ -75,6 +78,23 @@ var _ = ginkgo.Describe("Scheduler", func() {
 		return createWorkloadWithPriority(queue, cpuRequests, 0)
 	}
 
+	ginkgo.BeforeAll(func() {
+		fwk.StartManager(ctx, cfg, managerAndSchedulerSetup(
+			&config.AdmissionFairSharing{
+				UsageHalfLifeTime: metav1.Duration{
+					Duration: 10 * time.Second,
+				},
+				UsageSamplingInterval: metav1.Duration{
+					Duration: 1 * time.Second,
+				},
+			},
+		))
+	})
+
+	ginkgo.AfterAll(func() {
+		fwk.StopManager(ctx)
+	})
+
 	ginkgo.BeforeEach(func() {
 		defaultFlavor = testing.MakeResourceFlavor("default").Obj()
 		util.MustCreate(ctx, k8sClient, defaultFlavor)
@@ -394,121 +414,6 @@ var _ = ginkgo.Describe("Scheduler", func() {
 		})
 	})
 
-	ginkgo.When("Using AdmissionFairSharing at Cohort level", func() {
-		var (
-			cq1 *kueue.ClusterQueue
-			cq2 *kueue.ClusterQueue
-			lqA *kueue.LocalQueue
-			lqB *kueue.LocalQueue
-			lqC *kueue.LocalQueue
-		)
-
-		ginkgo.BeforeEach(func() {
-			gomega.Expect(features.SetEnable(features.AdmissionFairSharing, true)).To(gomega.Succeed())
-
-			cq1 = createQueue(testing.MakeClusterQueue("cq1").
-				Cohort("all").
-				ResourceGroup(*testing.MakeFlavorQuotas(defaultFlavor.Name).Resource(corev1.ResourceCPU, "16").Obj()).
-				Preemption(kueue.ClusterQueuePreemption{WithinClusterQueue: kueue.PreemptionPolicyNever}).
-				QueueingStrategy(kueue.StrictFIFO).
-				AdmissionMode(kueue.UsageBasedAdmissionFairSharing).
-				Obj())
-
-			cq2 = createQueue(testing.MakeClusterQueue("cq2").
-				Cohort("all").
-				ResourceGroup(
-					*testing.MakeFlavorQuotas("default").Resource(corev1.ResourceCPU, "16").Obj(),
-				).Preemption(kueue.ClusterQueuePreemption{
-				ReclaimWithinCohort: kueue.PreemptionPolicyAny,
-				WithinClusterQueue:  kueue.PreemptionPolicyLowerPriority,
-			}).Obj())
-
-			lqA = testing.MakeLocalQueue("lq-a", ns.Name).
-				FairSharing(&kueue.FairSharing{Weight: ptr.To(resource.MustParse("1"))}).
-				ClusterQueue(cq1.Name).Obj()
-			lqB = testing.MakeLocalQueue("lq-b", ns.Name).
-				FairSharing(&kueue.FairSharing{Weight: ptr.To(resource.MustParse("1"))}).
-				ClusterQueue(cq1.Name).Obj()
-			lqC = testing.MakeLocalQueue("lq-c", ns.Name).
-				FairSharing(&kueue.FairSharing{Weight: ptr.To(resource.MustParse("1"))}).
-				ClusterQueue(cq1.Name).Obj()
-			lqs = append(lqs, lqA)
-			lqs = append(lqs, lqB)
-			lqs = append(lqs, lqC)
-			util.MustCreate(ctx, k8sClient, lqA)
-			util.MustCreate(ctx, k8sClient, lqB)
-			util.MustCreate(ctx, k8sClient, lqC)
-		})
-
-		ginkgo.AfterEach(func() {
-			gomega.Expect(features.SetEnable(features.AdmissionFairSharing, false)).To(gomega.Succeed())
-		})
-
-		ginkgo.It("should promote a workload from LQ with lower recent usage", func() {
-			ginkgo.By("Creating a workload")
-			wl := createWorkload("lq-a", "32")
-
-			ginkgo.By("Admitting the workload")
-			util.ExpectWorkloadsToBeAdmitted(ctx, k8sClient, wl)
-			util.ExpectReservingActiveWorkloadsMetric(cq1, 1)
-
-			ginkgo.By("Checking that LQ's resource usage is updated", func() {
-				gomega.Eventually(func(g gomega.Gomega) {
-					g.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(lqA), lqA)).Should(gomega.Succeed())
-					g.Expect(lqA.Status.FairSharing).ShouldNot(gomega.BeNil())
-					g.Expect(lqA.Status.FairSharing.AdmissionFairSharingStatus).ShouldNot(gomega.BeNil())
-					g.Expect(lqA.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources).Should(gomega.HaveLen(1))
-					usage := lqA.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources[corev1.ResourceCPU]
-					g.Expect(usage.MilliValue()).To(gomega.BeNumerically(">=", 0))
-				}, util.Timeout, util.Interval).Should(gomega.Succeed())
-			})
-
-			ginkgo.By("Creating two pending workloads")
-			wlA := createWorkload("lq-a", "32")
-			wlB := createWorkload("lq-b", "32")
-
-			ginkgo.By("Finish the previous workload")
-			util.FinishWorkloads(ctx, k8sClient, wl)
-
-			ginkgo.By("Admitting the workload from LQ-b, which has lower recent usage")
-			util.ExpectWorkloadsToBeAdmitted(ctx, k8sClient, wlB)
-			util.ExpectWorkloadsToBePending(ctx, k8sClient, wlA)
-		})
-
-		ginkgo.It("should preempt a workload from LQ with higher recent usage", func() {
-			ginkgo.By("Creating workloads in CQ1 that borrow from CQ2")
-			wlHighA := createWorkloadWithPriority("lq-a", "20", 10)
-			_ = createWorkloadWithPriority("lq-b", "12", 1)
-			util.ExpectReservingActiveWorkloadsMetric(cq1, 2)
-
-			ginkgo.By("Checking that LQs' resource usage is updated", func() {
-				gomega.Eventually(func(g gomega.Gomega) {
-					g.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(lqA), lqA)).Should(gomega.Succeed())
-					g.Expect(lqA.Status.FairSharing).ShouldNot(gomega.BeNil())
-					g.Expect(lqA.Status.FairSharing.AdmissionFairSharingStatus).ShouldNot(gomega.BeNil())
-					g.Expect(lqA.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources).Should(gomega.HaveLen(1))
-					usage := lqA.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources[corev1.ResourceCPU]
-					g.Expect(usage.MilliValue()).To(gomega.BeNumerically(">", 0))
-				}, util.Timeout, util.Interval).Should(gomega.Succeed())
-
-				gomega.Eventually(func(g gomega.Gomega) {
-					g.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(lqB), lqB)).Should(gomega.Succeed())
-					g.Expect(lqB.Status.FairSharing).ShouldNot(gomega.BeNil())
-					g.Expect(lqB.Status.FairSharing.AdmissionFairSharingStatus).ShouldNot(gomega.BeNil())
-					g.Expect(lqB.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources).Should(gomega.HaveLen(1))
-					usage := lqB.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources[corev1.ResourceCPU]
-					g.Expect(usage.MilliValue()).To(gomega.BeNumerically(">", 0))
-				}, util.Timeout, util.Interval).Should(gomega.Succeed())
-			})
-
-			ginkgo.By("Creating a workload in CQ2 that reclaims the quota")
-			_ = createWorkload(kueue.LocalQueueName(cq2.Name), "10")
-
-			ginkgo.By("Checking that the workload from lq-A is preempted despite having bigger priority")
-			util.ExpectWorkloadsToBePreempted(ctx, k8sClient, wlHighA)
-		})
-	})
-
 	ginkgo.When("Using AdmissionFairSharing at ClusterQueue level", func() {
 		var (
 			cq1 *kueue.ClusterQueue
@@ -693,3 +598,186 @@ func expectCohortWeightedShare(cohortName string, weightedShare int64) {
 		g.ExpectWithOffset(1, int64(v)).Should(gomega.Equal(weightedShare))
 	}, util.Timeout, util.Interval).Should(gomega.Succeed())
 }
+
+var _ = ginkgo.Describe("Scheduler", ginkgo.Ordered, ginkgo.ContinueOnFailure, func() {
+	var (
+		defaultFlavor *kueue.ResourceFlavor
+		ns            *corev1.Namespace
+
+		cohorts []*kueue.Cohort
+		cqs     []*kueue.ClusterQueue
+		lqs     []*kueue.LocalQueue
+		wls     []*kueue.Workload
+	)
+
+	var createQueue = func(cq *kueue.ClusterQueue) *kueue.ClusterQueue {
+		util.MustCreate(ctx, k8sClient, cq)
+		cqs = append(cqs, cq)
+
+		lq := testing.MakeLocalQueue(cq.Name, ns.Name).ClusterQueue(cq.Name).Obj()
+		util.MustCreate(ctx, k8sClient, lq)
+		lqs = append(lqs, lq)
+		return cq
+	}
+
+	var createWorkloadWithPriority = func(queue kueue.LocalQueueName, cpuRequests string, priority int32) *kueue.Workload {
+		wl := testing.MakeWorkloadWithGeneratedName("workload-", ns.Name).
+			Priority(priority).
+			Queue(queue).
+			Request(corev1.ResourceCPU, cpuRequests).Obj()
+		wls = append(wls, wl)
+		util.MustCreate(ctx, k8sClient, wl)
+		return wl
+	}
+
+	var createWorkload = func(queue kueue.LocalQueueName, cpuRequests string) *kueue.Workload {
+		return createWorkloadWithPriority(queue, cpuRequests, 0)
+	}
+
+	ginkgo.BeforeAll(func() {
+		fwk.StartManager(ctx, cfg, managerAndSchedulerSetup(
+			&config.AdmissionFairSharing{
+				UsageHalfLifeTime: metav1.Duration{
+					Duration: 1 * time.Second,
+				},
+				UsageSamplingInterval: metav1.Duration{
+					Duration: 1 * time.Second,
+				},
+			},
+		))
+	})
+
+	ginkgo.AfterAll(func() {
+		fwk.StopManager(ctx)
+	})
+
+	ginkgo.BeforeEach(func() {
+		defaultFlavor = testing.MakeResourceFlavor("default").Obj()
+		util.MustCreate(ctx, k8sClient, defaultFlavor)
+
+		ns = util.CreateNamespaceFromPrefixWithLog(ctx, k8sClient, "core-")
+	})
+
+	ginkgo.AfterEach(func() {
+		for _, wl := range wls {
+			util.ExpectObjectToBeDeleted(ctx, k8sClient, wl, true)
+		}
+		for _, lq := range lqs {
+			util.ExpectObjectToBeDeleted(ctx, k8sClient, lq, true)
+		}
+		gomega.Expect(util.DeleteNamespace(ctx, k8sClient, ns)).To(gomega.Succeed())
+		for _, cq := range cqs {
+			util.ExpectObjectToBeDeleted(ctx, k8sClient, cq, true)
+		}
+		for _, cohort := range cohorts {
+			util.ExpectObjectToBeDeleted(ctx, k8sClient, cohort, true)
+		}
+		util.ExpectObjectToBeDeleted(ctx, k8sClient, defaultFlavor, true)
+	})
+
+	ginkgo.When("Using AdmissionFairSharing at Cohort level", func() {
+		var (
+			cq1 *kueue.ClusterQueue
+			cq2 *kueue.ClusterQueue
+			lqA *kueue.LocalQueue
+			lqB *kueue.LocalQueue
+			lqC *kueue.LocalQueue
+		)
+
+		ginkgo.BeforeEach(func() {
+			gomega.Expect(features.SetEnable(features.AdmissionFairSharing, true)).To(gomega.Succeed())
+
+			cq1 = createQueue(testing.MakeClusterQueue("cq1").
+				Cohort("all").
+				ResourceGroup(*testing.MakeFlavorQuotas(defaultFlavor.Name).Resource(corev1.ResourceCPU, "16").Obj()).
+				Preemption(kueue.ClusterQueuePreemption{WithinClusterQueue: kueue.PreemptionPolicyNever}).
+				QueueingStrategy(kueue.StrictFIFO).
+				AdmissionMode(kueue.UsageBasedAdmissionFairSharing).
+				Obj())
+
+			cq2 = createQueue(testing.MakeClusterQueue("cq2").
+				Cohort("all").
+				ResourceGroup(
+					*testing.MakeFlavorQuotas("default").Resource(corev1.ResourceCPU, "16").Obj(),
+				).Preemption(kueue.ClusterQueuePreemption{
+				ReclaimWithinCohort: kueue.PreemptionPolicyAny,
+				WithinClusterQueue:  kueue.PreemptionPolicyLowerPriority,
+			}).Obj())
+
+			lqA = testing.MakeLocalQueue("lq-a", ns.Name).
+				FairSharing(&kueue.FairSharing{Weight: ptr.To(resource.MustParse("1"))}).
+				ClusterQueue(cq1.Name).Obj()
+			lqB = testing.MakeLocalQueue("lq-b", ns.Name).
+				FairSharing(&kueue.FairSharing{Weight: ptr.To(resource.MustParse("1"))}).
+				ClusterQueue(cq1.Name).Obj()
+			lqC = testing.MakeLocalQueue("lq-c", ns.Name).
+				FairSharing(&kueue.FairSharing{Weight: ptr.To(resource.MustParse("1"))}).
+				ClusterQueue(cq1.Name).Obj()
+			lqs = append(lqs, lqA)
+			lqs = append(lqs, lqB)
+			lqs = append(lqs, lqC)
+			util.MustCreate(ctx, k8sClient, lqA)
+			util.MustCreate(ctx, k8sClient, lqB)
+			util.MustCreate(ctx, k8sClient, lqC)
+		})
+
+		ginkgo.AfterEach(func() {
+			gomega.Expect(features.SetEnable(features.AdmissionFairSharing, false)).To(gomega.Succeed())
+		})
+
+		ginkgo.It("should promote a workload from LQ with lower recent usage", func() {
+			ginkgo.By("Creating a workload")
+			wl := createWorkload("lq-a", "32")
+
+			ginkgo.By("Admitting the workload")
+			util.ExpectWorkloadsToBeAdmitted(ctx, k8sClient, wl)
+			util.ExpectReservingActiveWorkloadsMetric(cq1, 1)
+
+			ginkgo.By("Checking that LQ's resource usage is updated", func() {
+				gomega.Eventually(func(g gomega.Gomega) {
+					g.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(lqA), lqA)).Should(gomega.Succeed())
+					g.Expect(lqA.Status.FairSharing).ShouldNot(gomega.BeNil())
+					g.Expect(lqA.Status.FairSharing.AdmissionFairSharingStatus).ShouldNot(gomega.BeNil())
+					g.Expect(lqA.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources).Should(gomega.HaveLen(1))
+					usage := lqA.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources[corev1.ResourceCPU]
+					g.Expect(usage.MilliValue()).To(gomega.BeNumerically(">=", 0))
+				}, util.Timeout, util.Interval).Should(gomega.Succeed())
+			})
+
+			ginkgo.By("Creating two pending workloads")
+			wlA := createWorkload("lq-a", "32")
+			wlB := createWorkload("lq-b", "32")
+
+			ginkgo.By("Finish the previous workload")
+			util.FinishWorkloads(ctx, k8sClient, wl)
+
+			ginkgo.By("Admitting the workload from LQ-b, which has lower recent usage")
+			util.ExpectWorkloadsToBeAdmitted(ctx, k8sClient, wlB)
+			util.ExpectWorkloadsToBePending(ctx, k8sClient, wlA)
+		})
+
+		ginkgo.It("should preempt a workload from LQ with higher recent usage", func() {
+			ginkgo.By("Creating workloads in CQ1 that borrow from CQ2")
+			wlHighA := createWorkloadWithPriority("lq-a", "20", 10)
+			_ = createWorkloadWithPriority("lq-b", "12", 1)
+			util.ExpectReservingActiveWorkloadsMetric(cq1, 2)
+
+			ginkgo.By("Checking that LQs' resource usage is updated", func() {
+				gomega.Eventually(func(g gomega.Gomega) {
+					g.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(lqA), lqA)).Should(gomega.Succeed())
+					g.Expect(lqA.Status.FairSharing).ShouldNot(gomega.BeNil())
+					g.Expect(lqA.Status.FairSharing.AdmissionFairSharingStatus).ShouldNot(gomega.BeNil())
+					g.Expect(lqA.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources).Should(gomega.HaveLen(1))
+					usage := lqA.Status.FairSharing.AdmissionFairSharingStatus.ConsumedResources[corev1.ResourceCPU]
+					g.Expect(usage.MilliValue()).To(gomega.BeNumerically(">", 12))
+				}, util.Timeout, util.Interval).Should(gomega.Succeed())
+			})
+
+			ginkgo.By("Creating a workload in CQ2 that reclaims the quota")
+			_ = createWorkload(kueue.LocalQueueName(cq2.Name), "10")
+
+			ginkgo.By("Checking that the workload from lq-A is preempted despite having bigger priority")
+			util.ExpectWorkloadsToBePreempted(ctx, k8sClient, wlHighA)
+		})
+	})
+})
diff --git a/test/integration/singlecluster/scheduler/fairsharing/suite_test.go b/test/integration/singlecluster/scheduler/fairsharing/suite_test.go
index c9fe0dd66..0f9404a23 100644
--- a/test/integration/singlecluster/scheduler/fairsharing/suite_test.go
+++ b/test/integration/singlecluster/scheduler/fairsharing/suite_test.go
@@ -19,11 +19,9 @@ package fairsharing
 import (
 	"context"
 	"testing"
-	"time"
 
 	"github.com/onsi/ginkgo/v2"
 	"github.com/onsi/gomega"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/client-go/rest"
 	"sigs.k8s.io/controller-runtime/pkg/client"
 	"sigs.k8s.io/controller-runtime/pkg/manager"
@@ -63,48 +61,41 @@ var _ = ginkgo.BeforeSuite(func() {
 	}
 	cfg = fwk.Init()
 	ctx, k8sClient = fwk.SetupClient(cfg)
-	fwk.StartManager(ctx, cfg, managerAndSchedulerSetup)
 })
 
 var _ = ginkgo.AfterSuite(func() {
 	fwk.Teardown()
 })
 
-func managerAndSchedulerSetup(ctx context.Context, mgr manager.Manager) {
-	fairSharing := &config.FairSharing{
-		Enable: true,
-	}
-	admissionFairSharing := &config.AdmissionFairSharing{
-		UsageHalfLifeTime: metav1.Duration{
-			Duration: 10 * time.Second,
-		},
-		UsageSamplingInterval: metav1.Duration{
-			Duration: 1 * time.Second,
-		},
-	}
+func managerAndSchedulerSetup(admissionFairSharing *config.AdmissionFairSharing) framework.ManagerSetup {
+	return func(ctx context.Context, mgr manager.Manager) {
+		fairSharing := &config.FairSharing{
+			Enable: true,
+		}
 
-	err := indexer.Setup(ctx, mgr.GetFieldIndexer())
-	gomega.Expect(err).NotTo(gomega.HaveOccurred())
+		err := indexer.Setup(ctx, mgr.GetFieldIndexer())
+		gomega.Expect(err).NotTo(gomega.HaveOccurred())
 
-	_ = features.SetEnable(features.AdmissionFairSharing, true)
-	cCache := cache.New(mgr.GetClient(), cache.WithFairSharing(fairSharing.Enable), cache.WithAdmissionFairSharing(admissionFairSharing))
-	queues := queue.NewManager(mgr.GetClient(), cCache, queue.WithAdmissionFairSharing(admissionFairSharing))
+		_ = features.SetEnable(features.AdmissionFairSharing, true)
+		cCache := cache.New(mgr.GetClient(), cache.WithFairSharing(fairSharing.Enable), cache.WithAdmissionFairSharing(admissionFairSharing))
+		queues := queue.NewManager(mgr.GetClient(), cCache, queue.WithAdmissionFairSharing(admissionFairSharing))
 
-	configuration := &config.Configuration{FairSharing: fairSharing, AdmissionFairSharing: admissionFairSharing}
-	configuration.Metrics.EnableClusterQueueResources = true
-	mgr.GetScheme().Default(configuration)
+		configuration := &config.Configuration{FairSharing: fairSharing, AdmissionFairSharing: admissionFairSharing}
+		configuration.Metrics.EnableClusterQueueResources = true
+		mgr.GetScheme().Default(configuration)
 
-	failedCtrl, err := core.SetupControllers(mgr, queues, cCache, configuration)
-	gomega.Expect(err).ToNot(gomega.HaveOccurred(), "controller", failedCtrl)
+		failedCtrl, err := core.SetupControllers(mgr, queues, cCache, configuration)
+		gomega.Expect(err).ToNot(gomega.HaveOccurred(), "controller", failedCtrl)
 
-	failedWebhook, err := webhooks.Setup(mgr, config.MultiKueueDispatcherModeAllAtOnce)
-	gomega.Expect(err).ToNot(gomega.HaveOccurred(), "webhook", failedWebhook)
+		failedWebhook, err := webhooks.Setup(mgr, config.MultiKueueDispatcherModeAllAtOnce)
+		gomega.Expect(err).ToNot(gomega.HaveOccurred(), "webhook", failedWebhook)
 
-	err = workloadjob.SetupIndexes(ctx, mgr.GetFieldIndexer())
-	gomega.Expect(err).NotTo(gomega.HaveOccurred())
+		err = workloadjob.SetupIndexes(ctx, mgr.GetFieldIndexer())
+		gomega.Expect(err).NotTo(gomega.HaveOccurred())
 
-	sched := scheduler.New(queues, cCache, mgr.GetClient(), mgr.GetEventRecorderFor(constants.AdmissionName),
-		scheduler.WithFairSharing(fairSharing), scheduler.WithAdmissionFairSharing(admissionFairSharing))
-	err = sched.Start(ctx)
-	gomega.Expect(err).NotTo(gomega.HaveOccurred())
+		sched := scheduler.New(queues, cCache, mgr.GetClient(), mgr.GetEventRecorderFor(constants.AdmissionName),
+			scheduler.WithFairSharing(fairSharing), scheduler.WithAdmissionFairSharing(admissionFairSharing))
+		err = sched.Start(ctx)
+		gomega.Expect(err).NotTo(gomega.HaveOccurred())
+	}
 }