Repository: kubernetes-sigs/kueue
Pull request #146 (2022-03-24 13:15:03)

Abstract cluster queue interface
Signed-off-by: Alex Wang <wangqingcan1990@gmail.com>

<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?
/kind feature
<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

#### What this PR does / why we need it:
Make a `ClusterQueue` interface  so that each queueing strategy implements and gets invoked agnostically by queue manager. 


#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:



k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/kueue/pull/146#" title="Author self-approved">denkensk</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **alculquicondor** after the PR has been reviewed.
You can assign the PR to them by writing `/assign @alculquicondor` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fkueue).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/kueue/blob/main/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["alculquicondor"]} -->
k8s-ci-robot: @denkensk: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kueue-test-integration-main | b8539b2ef8d0e174800558577d3251e6ae9ad74c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/146/pull-kueue-test-integration-main/1506982935721414656) | true | `/test pull-kueue-test-integration-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kueue&pr=146). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adenkensk). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kueue/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
denkensk: This is the first version of the implementation that ensures the right direction.
Need your feedback. @ahg-g @alculquicondor 
k8s-ci-robot: @denkensk: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kueue-test-integration-main | b8539b2ef8d0e174800558577d3251e6ae9ad74c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/146/pull-kueue-test-integration-main/1506982935721414656) | true | `/test pull-kueue-test-integration-main`
pull-kueue-verify-main | b8539b2ef8d0e174800558577d3251e6ae9ad74c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/146/pull-kueue-verify-main/1506982935767552000) | true | `/test pull-kueue-verify-main`
pull-kueue-test-unit-main | b8539b2ef8d0e174800558577d3251e6ae9ad74c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/146/pull-kueue-test-unit-main/1506982935687860224) | true | `/test pull-kueue-test-unit-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kueue&pr=146). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adenkensk). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kueue/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @denkensk: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kueue-test-integration-main | b8539b2ef8d0e174800558577d3251e6ae9ad74c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/146/pull-kueue-test-integration-main/1506982935721414656) | true | `/test pull-kueue-test-integration-main`
pull-kueue-test-unit-main | b8539b2ef8d0e174800558577d3251e6ae9ad74c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/146/pull-kueue-test-unit-main/1506982935687860224) | true | `/test pull-kueue-test-unit-main`
pull-kueue-verify-main | bc4bdd94df9b783992a1b1499bb4ad70dd58fe82 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/146/pull-kueue-verify-main/1507254858179153920) | true | `/test pull-kueue-verify-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kueue&pr=146). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adenkensk). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kueue/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @denkensk: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-kueue-verify-main | 202d53fb51a97136af0ca2119003cb7a3098d3c4 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/146/pull-kueue-verify-main/1507257141818298368) | true | `/test pull-kueue-verify-main`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=kueue&pr=146). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Adenkensk). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/kueue/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/api/v1alpha1/clusterqueue_types.go b/api/v1alpha1/clusterqueue_types.go
index 1f77ffe9f..18bca462b 100644
--- a/api/v1alpha1/clusterqueue_types.go
+++ b/api/v1alpha1/clusterqueue_types.go
@@ -247,8 +247,8 @@ type Quota struct {
 	// ceiling is the upper limit on the amount of resource requests that
 	// could be used by running workloads assigned to this quota at a point in time.
 	// Resources can be borrowed from unused guaranteed quota of other
-	// ClusterQueues in the same cohort. When null, it is unlimited.
-	Ceiling *resource.Quantity `json:"ceiling,omitempty"`
+	// ClusterQueues in the same cohort. When not set, it is unlimited.
+	Ceiling resource.Quantity `json:"ceiling,omitempty"`
 }
 
 // ClusterQueueStatus defines the observed state of ClusterQueue
diff --git a/api/v1alpha1/zz_generated.deepcopy.go b/api/v1alpha1/zz_generated.deepcopy.go
index 59fff2bca..054cbe3a0 100644
--- a/api/v1alpha1/zz_generated.deepcopy.go
+++ b/api/v1alpha1/zz_generated.deepcopy.go
@@ -445,11 +445,7 @@ func (in *QueuedWorkloadStatus) DeepCopy() *QueuedWorkloadStatus {
 func (in *Quota) DeepCopyInto(out *Quota) {
 	*out = *in
 	out.Guaranteed = in.Guaranteed.DeepCopy()
-	if in.Ceiling != nil {
-		in, out := &in.Ceiling, &out.Ceiling
-		x := (*in).DeepCopy()
-		*out = &x
-	}
+	out.Ceiling = in.Ceiling.DeepCopy()
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Quota.
diff --git a/config/crd/bases/kueue.x-k8s.io_clusterqueues.yaml b/config/crd/bases/kueue.x-k8s.io_clusterqueues.yaml
index 350857d74..36d4e5923 100644
--- a/config/crd/bases/kueue.x-k8s.io_clusterqueues.yaml
+++ b/config/crd/bases/kueue.x-k8s.io_clusterqueues.yaml
@@ -199,7 +199,7 @@ spec:
                                   workloads assigned to this quota at a point in time.
                                   Resources can be borrowed from unused guaranteed
                                   quota of other ClusterQueues in the same cohort.
-                                  When null, it is unlimited.
+                                  When not set, it is unlimited.
                                 pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                 x-kubernetes-int-or-string: true
                               guaranteed:
diff --git a/config/samples/single-clusterqueue-setup.yaml b/config/samples/single-clusterqueue-setup.yaml
index 0d0a1b902..203521b11 100644
--- a/config/samples/single-clusterqueue-setup.yaml
+++ b/config/samples/single-clusterqueue-setup.yaml
@@ -15,11 +15,13 @@ spec:
     - resourceFlavor: default
       quota:
         guaranteed: 9
+        ceiling: 9
   - name: "memory"
     flavors:
     - resourceFlavor: default
       quota:
         guaranteed: 36Gi
+        ceiling: 36Gi
 ---
 apiVersion: kueue.x-k8s.io/v1alpha1
 kind: Queue
diff --git a/docs/tasks/administer_cluster_resources.md b/docs/tasks/administer_cluster_resources.md
index 8a9083476..2bb085deb 100644
--- a/docs/tasks/administer_cluster_resources.md
+++ b/docs/tasks/administer_cluster_resources.md
@@ -44,16 +44,21 @@ spec:
     - resourceFlavor: default
       quota:
         guaranteed: 9
+        ceiling: 9
   - name: "memory"
     flavors:
     - resourceFlavor: default
       quota:
         guaranteed: 36Gi
+        ceiling: 36Gi
 ```
 
 This ClusterQueue governs the usage of [resource types](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#resource-types)
 `cpu` and `memory`. Each resource type has a single [resource flavor](/docs/concepts/cluster_queue.md#resource-flavors),
 named `default` with a guaranteed quota.
+As there is a single ClusterQueue in the cluster, it is not possible for the
+ClusterQueue to borrow resources from anywhere. Thus, the `ceiling` is equal
+to the `guaranteed` quota.
 
 The empty `namespaceSelector` allows any namespace to use these resources.
 
diff --git a/pkg/cache/cache.go b/pkg/cache/cache.go
index a5e46e9d4..d8bc033bc 100644
--- a/pkg/cache/cache.go
+++ b/pkg/cache/cache.go
@@ -97,7 +97,7 @@ type ClusterQueue struct {
 type FlavorLimits struct {
 	Name       string
 	Guaranteed int64
-	Ceiling    *int64
+	Ceiling    int64
 }
 
 func (c *Cache) newClusterQueue(cq *kueue.ClusterQueue) (*ClusterQueue, error) {
@@ -138,7 +138,7 @@ func (c *ClusterQueue) update(in *kueue.ClusterQueue, resourceFlavors map[string
 	return nil
 }
 
-// UpdateLabelKeys updates a ClusterQueue's LabelKeys based on the passed ResourceFlavors set.
+// UpdateLabelkeys updates a ClusterQueue's LabelKeys based on the passed ResourceFlavors set.
 // Exported only for testing.
 func (c *ClusterQueue) UpdateLabelKeys(flavors map[string]*kueue.ResourceFlavor) {
 	labelKeys := map[corev1.ResourceName]sets.String{}
@@ -473,9 +473,7 @@ func resourceLimitsByName(in []kueue.Resource) map[corev1.ResourceName][]FlavorL
 			fLimits := FlavorLimits{
 				Name:       string(f.ResourceFlavor),
 				Guaranteed: workload.ResourceValue(r.Name, f.Quota.Guaranteed),
-			}
-			if f.Quota.Ceiling != nil {
-				fLimits.Ceiling = pointer.Int64(workload.ResourceValue(r.Name, *f.Quota.Ceiling))
+				Ceiling:    workload.ResourceValue(r.Name, f.Quota.Ceiling),
 			}
 			flavors[i] = fLimits
 
diff --git a/pkg/controller/core/clusterqueue_controller.go b/pkg/controller/core/clusterqueue_controller.go
index e9806902e..d32221a39 100644
--- a/pkg/controller/core/clusterqueue_controller.go
+++ b/pkg/controller/core/clusterqueue_controller.go
@@ -140,71 +140,52 @@ func (r *ClusterQueue) Generic(e event.GenericEvent) bool {
 	return true
 }
 
-// workloadHandler signals the controller to reconcile the ClusterQueue
+// assignedWorkloadHandler signals the controller to reconcile the ClusterQueue
 // assigned to the workload in the event.
-type workloadHandler struct {
-	qManager *queue.Manager
-}
+type assignedWorkloadHandler struct{}
 
-func (h *workloadHandler) Create(e event.CreateEvent, q workqueue.RateLimitingInterface) {
+func (h *assignedWorkloadHandler) Create(e event.CreateEvent, q workqueue.RateLimitingInterface) {
 	w := e.Object.(*kueue.QueuedWorkload)
-	req := h.requestForWorkloadClusterQueue(w)
-	if req != nil {
-		q.AddAfter(*req, constants.UpdatesBatchPeriod)
+	if w.Spec.Admission != nil {
+		q.AddAfter(requestForWorkloadClusterQueue(w), constants.UpdatesBatchPeriod)
 	}
 }
 
-func (h *workloadHandler) Update(e event.UpdateEvent, q workqueue.RateLimitingInterface) {
+func (h *assignedWorkloadHandler) Update(e event.UpdateEvent, q workqueue.RateLimitingInterface) {
 	oldW := e.ObjectOld.(*kueue.QueuedWorkload)
-	oldReq := h.requestForWorkloadClusterQueue(oldW)
-	if oldReq != nil {
-		q.AddAfter(*oldReq, constants.UpdatesBatchPeriod)
+	if oldW.Spec.Admission != nil {
+		q.AddAfter(requestForWorkloadClusterQueue(oldW), constants.UpdatesBatchPeriod)
 	}
 	newW := e.ObjectNew.(*kueue.QueuedWorkload)
-	newReq := h.requestForWorkloadClusterQueue(newW)
-	// Enqueue the CQ for the new object if it's different from the old one.
-	if newReq != nil && (oldReq == nil || oldReq.Name != newReq.Name) {
-		q.AddAfter(*newReq, constants.UpdatesBatchPeriod)
+	if newW.Spec.Admission != nil && (oldW.Spec.Admission == nil || workloadFinished(newW) ||
+		newW.Spec.Admission.ClusterQueue != oldW.Spec.Admission.ClusterQueue) {
+		q.AddAfter(requestForWorkloadClusterQueue(newW), constants.UpdatesBatchPeriod)
 	}
 }
 
-func (h *workloadHandler) Delete(e event.DeleteEvent, q workqueue.RateLimitingInterface) {
+func (h *assignedWorkloadHandler) Delete(e event.DeleteEvent, q workqueue.RateLimitingInterface) {
 	w := e.Object.(*kueue.QueuedWorkload)
-	req := h.requestForWorkloadClusterQueue(w)
-	if req != nil {
-		q.AddAfter(*req, constants.UpdatesBatchPeriod)
+	if w.Spec.Admission != nil {
+		q.AddAfter(requestForWorkloadClusterQueue(w), constants.UpdatesBatchPeriod)
 	}
 }
 
-func (h *workloadHandler) Generic(e event.GenericEvent, q workqueue.RateLimitingInterface) {
+func (h *assignedWorkloadHandler) Generic(e event.GenericEvent, q workqueue.RateLimitingInterface) {
 }
 
-func (h *workloadHandler) requestForWorkloadClusterQueue(w *kueue.QueuedWorkload) *reconcile.Request {
-	var name string
-	if w.Spec.Admission != nil {
-		name = string(w.Spec.Admission.ClusterQueue)
-	} else {
-		var ok bool
-		name, ok = h.qManager.ClusterQueueForWorkload(w)
-		if !ok {
-			return nil
-		}
-	}
-	return &reconcile.Request{
+func requestForWorkloadClusterQueue(w *kueue.QueuedWorkload) reconcile.Request {
+	return reconcile.Request{
 		NamespacedName: types.NamespacedName{
-			Name: name,
+			Name: string(w.Spec.Admission.ClusterQueue),
 		},
 	}
 }
 
 // SetupWithManager sets up the controller with the Manager.
 func (r *ClusterQueue) SetupWithManager(mgr ctrl.Manager) error {
-	wHandler := workloadHandler{
-		qManager: r.qManager,
-	}
 	return ctrl.NewControllerManagedBy(mgr).
 		For(&kueue.ClusterQueue{}).
-		Watches(&source.Kind{Type: &kueue.QueuedWorkload{}}, &wHandler).
+		Watches(&source.Kind{Type: &kueue.QueuedWorkload{}}, &assignedWorkloadHandler{}).
 		WithEventFilter(r).
 		Complete(r)
 }
diff --git a/pkg/controller/core/queuedworkload_controller.go b/pkg/controller/core/queuedworkload_controller.go
index 9274a4d6d..a5e791551 100644
--- a/pkg/controller/core/queuedworkload_controller.go
+++ b/pkg/controller/core/queuedworkload_controller.go
@@ -73,15 +73,16 @@ func (r *QueuedWorkloadReconciler) Reconcile(ctx context.Context, req ctrl.Reque
 	log.V(2).Info("Reconciling QueuedWorkload")
 
 	status := workloadStatus(&wl)
-	if status == pending && !r.queues.QueueForWorkloadExists(&wl) {
+	q, qOK := r.queues.QueueExists(&wl)
+	if status == pending && !qOK {
 		err := workload.UpdateWorkloadStatusIfChanged(ctx, r.client, &wl, kueue.QueuedWorkloadAdmitted, corev1.ConditionFalse,
-			"Inadmissible", fmt.Sprintf("Queue %s doesn't exist", wl.Spec.QueueName))
+			"Inadmissible", fmt.Sprintf("Queue %s for workload doesn't exist", wl.Spec.QueueName))
 		return ctrl.Result{}, client.IgnoreNotFound(err)
 	}
-	cqName, cqOk := r.queues.ClusterQueueForWorkload(&wl)
-	if status == pending && !cqOk {
+
+	if status == pending && !r.queues.ClusterQueueExists(&wl) {
 		err := workload.UpdateWorkloadStatusIfChanged(ctx, r.client, &wl, kueue.QueuedWorkloadAdmitted, corev1.ConditionFalse,
-			"Inadmissible", fmt.Sprintf("ClusterQueue %s doesn't exist", cqName))
+			"Inadmissible", fmt.Sprintf("ClusterQueue %s doesn't exist", q.ClusterQueue))
 		return ctrl.Result{}, client.IgnoreNotFound(err)
 	}
 
diff --git a/pkg/queue/cluster_queue_impl.go b/pkg/queue/cluster_queue_impl.go
new file mode 100644
index 000000000..17e9c61ee
--- /dev/null
+++ b/pkg/queue/cluster_queue_impl.go
@@ -0,0 +1,108 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package queue
+
+import (
+	"container/heap"
+
+	"k8s.io/apimachinery/pkg/util/sets"
+
+	kueue "sigs.k8s.io/kueue/api/v1alpha1"
+	"sigs.k8s.io/kueue/pkg/workload"
+)
+
+// ClusterQueueImpl is the base implementation of ClusterQueue interface.
+// It can be inherited and overwritten by other class.
+type ClusterQueueImpl struct {
+	// QueueingStrategy indicates the queueing strategy of the workloads
+	// across the queues in this ClusterQueue.
+	QueueingStrategy kueue.QueueingStrategy
+
+	heap heapImpl
+}
+
+var _ ClusterQueue = &ClusterQueueImpl{}
+
+func (c *ClusterQueueImpl) Update(apiCQ *kueue.ClusterQueue) {
+	c.QueueingStrategy = apiCQ.Spec.QueueingStrategy
+}
+
+func (c *ClusterQueueImpl) AddFromQueue(q *Queue) bool {
+	added := false
+	for _, w := range q.items {
+		if c.PushIfNotPresent(w) {
+			added = true
+		}
+	}
+	return added
+}
+
+func (c *ClusterQueueImpl) DeleteFromQueue(q *Queue) {
+	for _, w := range q.items {
+		c.Delete(w.Obj)
+	}
+}
+
+func (c *ClusterQueueImpl) PushIfNotPresent(info *workload.Info) bool {
+	item := c.heap.items[workload.Key(info.Obj)]
+	if item != nil {
+		return false
+	}
+	heap.Push(&c.heap, *info)
+	return true
+}
+
+func (c *ClusterQueueImpl) PushOrUpdate(w *kueue.QueuedWorkload) {
+	item := c.heap.items[workload.Key(w)]
+	info := *workload.NewInfo(w)
+	if item == nil {
+		heap.Push(&c.heap, info)
+		return
+	}
+	item.obj = info
+	heap.Fix(&c.heap, item.index)
+}
+
+func (c *ClusterQueueImpl) Delete(w *kueue.QueuedWorkload) {
+	item := c.heap.items[workload.Key(w)]
+	if item != nil {
+		heap.Remove(&c.heap, item.index)
+	}
+}
+
+func (c *ClusterQueueImpl) Pop() *workload.Info {
+	if c.heap.Len() == 0 {
+		return nil
+	}
+	w := heap.Pop(&c.heap).(workload.Info)
+	return &w
+}
+
+func (c *ClusterQueueImpl) Pending() int32 {
+	return int32(len(c.heap.heap))
+}
+
+func (c *ClusterQueueImpl) Dump() (sets.String, bool) {
+	if len(c.heap.items) == 0 {
+		return sets.NewString(), false
+	}
+	elements := make(sets.String, len(c.heap.items))
+	for _, e := range c.heap.items {
+		elements.Insert(e.obj.Obj.Name)
+	}
+	return elements, true
+}
diff --git a/pkg/queue/cluster_queue_interface.go b/pkg/queue/cluster_queue_interface.go
new file mode 100644
index 000000000..a86ee496d
--- /dev/null
+++ b/pkg/queue/cluster_queue_interface.go
@@ -0,0 +1,72 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package queue
+
+import (
+	"fmt"
+
+	"k8s.io/apimachinery/pkg/util/sets"
+	kueue "sigs.k8s.io/kueue/api/v1alpha1"
+	"sigs.k8s.io/kueue/pkg/workload"
+)
+
+// ClusterQueue is an interface for a cluster queue to store workloads waiting
+// to be scheduled.
+type ClusterQueue interface {
+	// Update updates the properties of this ClusterQueue.
+	Update(*kueue.ClusterQueue)
+
+	// AddFromQueue pushes all workloads belonging to this queue to
+	// the ClusterQueue. If at least one workload is added, returns true.
+	// Otherwise returns false.
+	AddFromQueue(*Queue) bool
+	// DeleteFromQueue removes all workloads belonging to this queue from
+	// the ClusterQueue.
+	DeleteFromQueue(*Queue)
+
+	// PushIfNotPresent pushes the workload to ClusterQueue.
+	// If the workload is already present, returns false. Otherwise returns true.
+	PushIfNotPresent(*workload.Info) bool
+	// PushOrUpdate pushes the workload to ClusterQueue.
+	// If the workload is already present, updates with the new one.
+	PushOrUpdate(*kueue.QueuedWorkload)
+	// Delete removes the workload from ClusterQueue.
+	Delete(*kueue.QueuedWorkload)
+	// Pop removes the head of the queue and returns it. It returns nil if the
+	// queue is empty.
+	Pop() *workload.Info
+
+	// Pending returns the number of pending workloads.
+	Pending() int32
+	// Dump produces a dump of the current workloads in the heap of
+	// this ClusterQueue. It returns false if the queue is empty.
+	// Otherwise returns true.
+	Dump() (sets.String, bool)
+}
+
+var registry = map[kueue.QueueingStrategy]func(cq *kueue.ClusterQueue) (ClusterQueue, error){
+	StrictFIFO: newClusterQueueStrictFIFO,
+}
+
+func newClusterQueue(cq *kueue.ClusterQueue) (ClusterQueue, error) {
+	strategy := cq.Spec.QueueingStrategy
+	f, exist := registry[strategy]
+	if !exist {
+		return nil, fmt.Errorf("invalid QueueingStrategy %q", cq.Spec.QueueingStrategy)
+	}
+	return f(cq)
+}
diff --git a/pkg/queue/cluster_queue_strict_fifo.go b/pkg/queue/cluster_queue_strict_fifo.go
new file mode 100644
index 000000000..b24d4beac
--- /dev/null
+++ b/pkg/queue/cluster_queue_strict_fifo.go
@@ -0,0 +1,57 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package queue
+
+import (
+	kueue "sigs.k8s.io/kueue/api/v1alpha1"
+	utilpriority "sigs.k8s.io/kueue/pkg/util/priority"
+	"sigs.k8s.io/kueue/pkg/workload"
+)
+
+// ClusterQueueStrictFIFO is the implementation for the ClusterQueue for
+// StrictFIFO.
+type ClusterQueueStrictFIFO struct {
+	ClusterQueueImpl
+}
+
+var _ ClusterQueue = &ClusterQueueStrictFIFO{}
+
+const StrictFIFO = kueue.StrictFIFO
+
+func newClusterQueueStrictFIFO(cq *kueue.ClusterQueue) (ClusterQueue, error) {
+	cqImpl := &ClusterQueueImpl{
+		heap: heapImpl{
+			less:  strictFIFO,
+			items: make(map[string]*heapItem),
+		},
+	}
+	cqImpl.Update(cq)
+	return cqImpl, nil
+}
+
+// strictFIFO is the function used by the clusterQueue heap algorithm to sort
+// workloads. It sorts workloads based on their priority.
+// When priorities are equal, it uses workloads.creationTimestamp.
+func strictFIFO(a, b workload.Info) bool {
+	p1 := utilpriority.Priority(a.Obj)
+	p2 := utilpriority.Priority(b.Obj)
+
+	if p1 != p2 {
+		return p1 > p2
+	}
+	return a.Obj.CreationTimestamp.Before(&b.Obj.CreationTimestamp)
+}
diff --git a/pkg/queue/queue_test.go b/pkg/queue/cluster_queue_strict_fifo_test.go
similarity index 100%
rename from pkg/queue/queue_test.go
rename to pkg/queue/cluster_queue_strict_fifo_test.go
diff --git a/pkg/queue/manager.go b/pkg/queue/manager.go
index fe45201ca..b06ba4764 100644
--- a/pkg/queue/manager.go
+++ b/pkg/queue/manager.go
@@ -46,7 +46,7 @@ type Manager struct {
 	cond sync.Cond
 
 	client        client.Client
-	clusterQueues map[string]*ClusterQueue
+	clusterQueues map[string]ClusterQueue
 	queues        map[string]*Queue
 }
 
@@ -54,7 +54,7 @@ func NewManager(client client.Client) *Manager {
 	m := &Manager{
 		client:        client,
 		queues:        make(map[string]*Queue),
-		clusterQueues: make(map[string]*ClusterQueue),
+		clusterQueues: make(map[string]ClusterQueue),
 	}
 	m.cond.L = &m.RWMutex
 	return m
@@ -107,7 +107,7 @@ func (m *Manager) UpdateClusterQueue(cq *kueue.ClusterQueue) error {
 		return errClusterQueueDoesNotExist
 	}
 	// TODO(#8): recreate heap based on a change of queueing policy.
-	cqImpl.update(cq)
+	cqImpl.Update(cq)
 	return nil
 }
 
@@ -203,29 +203,22 @@ func (m *Manager) PendingWorkloads(q *kueue.Queue) (int32, error) {
 func (m *Manager) Pending(cq *kueue.ClusterQueue) int32 {
 	m.RLock()
 	defer m.RUnlock()
-	return int32(len(m.clusterQueues[cq.Name].heap.heap))
+	return m.clusterQueues[cq.Name].Pending()
 }
 
-func (m *Manager) QueueForWorkloadExists(wl *kueue.QueuedWorkload) bool {
+func (m *Manager) QueueExists(wl *kueue.QueuedWorkload) (*Queue, bool) {
 	m.RLock()
 	defer m.RUnlock()
-	_, ok := m.queues[queueKeyForWorkload(wl)]
-	return ok
+	q, ok := m.queues[queueKeyForWorkload(wl)]
+	return q, ok
 
 }
 
-// ClusterQueueForWorkload returns the name of the ClusterQueue where the
-// workload should be queued and whether it exists.
-// Returns empty string if the queue doesn't exist.
-func (m *Manager) ClusterQueueForWorkload(wl *kueue.QueuedWorkload) (string, bool) {
+func (m *Manager) ClusterQueueExists(wl *kueue.QueuedWorkload) bool {
 	m.RLock()
 	defer m.RUnlock()
-	q, ok := m.queues[queueKeyForWorkload(wl)]
-	if !ok {
-		return "", false
-	}
-	_, ok = m.clusterQueues[q.ClusterQueue]
-	return q.ClusterQueue, ok
+	_, ok := m.clusterQueues[m.queues[queueKeyForWorkload(wl)].ClusterQueue]
+	return ok
 }
 
 // AddOrUpdateWorkload adds or updates workload to the corresponding queue.
@@ -358,14 +351,9 @@ func (m *Manager) Dump() map[string]sets.String {
 	}
 	dump := make(map[string]sets.String, len(m.queues))
 	for key, cq := range m.clusterQueues {
-		if len(cq.heap.items) == 0 {
-			continue
-		}
-		elements := make(sets.String, len(cq.heap.items))
-		for _, e := range cq.heap.items {
-			elements.Insert(e.obj.Obj.Name)
+		if elements, ok := cq.Dump(); ok {
+			dump[key] = elements
 		}
-		dump[key] = elements
 	}
 	if len(dump) == 0 {
 		return nil
diff --git a/pkg/queue/queue.go b/pkg/queue/queue.go
index 024237434..9bca1f54f 100644
--- a/pkg/queue/queue.go
+++ b/pkg/queue/queue.go
@@ -17,11 +17,9 @@ limitations under the License.
 package queue
 
 import (
-	"container/heap"
 	"fmt"
 
 	kueue "sigs.k8s.io/kueue/api/v1alpha1"
-	utilpriority "sigs.k8s.io/kueue/pkg/util/priority"
 	"sigs.k8s.io/kueue/pkg/workload"
 )
 
@@ -64,104 +62,6 @@ func (q *Queue) AddOrUpdate(w *kueue.QueuedWorkload) {
 	q.items[key] = workload.NewInfo(w)
 }
 
-// ClusterQueue is the internal implementation of kueue.ClusterQueue that
-// holds pending workloads.
-type ClusterQueue struct {
-	// QueueingStrategy indicates the queueing strategy of the workloads
-	// across the queues in this ClusterQueue.
-	QueueingStrategy kueue.QueueingStrategy
-
-	heap heapImpl
-}
-
-func newClusterQueue(cq *kueue.ClusterQueue) (*ClusterQueue, error) {
-	var less lessFunc
-
-	switch cq.Spec.QueueingStrategy {
-	case kueue.StrictFIFO:
-		less = strictFIFO
-	default:
-		return nil, fmt.Errorf("invalid QueueingStrategy %q", cq.Spec.QueueingStrategy)
-	}
-
-	cqImpl := &ClusterQueue{
-		heap: heapImpl{
-			less:  less,
-			items: make(map[string]*heapItem),
-		},
-	}
-	cqImpl.update(cq)
-	return cqImpl, nil
-}
-
-func (cq *ClusterQueue) update(apiCQ *kueue.ClusterQueue) {
-	cq.QueueingStrategy = apiCQ.Spec.QueueingStrategy
-}
-
-func (cq *ClusterQueue) AddFromQueue(q *Queue) bool {
-	added := false
-	for _, w := range q.items {
-		if cq.PushIfNotPresent(w) {
-			added = true
-		}
-	}
-	return added
-}
-
-func (cq *ClusterQueue) DeleteFromQueue(q *Queue) {
-	for _, w := range q.items {
-		cq.Delete(w.Obj)
-	}
-}
-
-func (cq *ClusterQueue) PushIfNotPresent(info *workload.Info) bool {
-	item := cq.heap.items[workload.Key(info.Obj)]
-	if item != nil {
-		return false
-	}
-	heap.Push(&cq.heap, *info)
-	return true
-}
-
-func (cq *ClusterQueue) PushOrUpdate(w *kueue.QueuedWorkload) {
-	item := cq.heap.items[workload.Key(w)]
-	info := *workload.NewInfo(w)
-	if item == nil {
-		heap.Push(&cq.heap, info)
-		return
-	}
-	item.obj = info
-	heap.Fix(&cq.heap, item.index)
-}
-
-func (cq *ClusterQueue) Delete(w *kueue.QueuedWorkload) {
-	item := cq.heap.items[workload.Key(w)]
-	if item != nil {
-		heap.Remove(&cq.heap, item.index)
-	}
-}
-
-func (cq *ClusterQueue) Pop() *workload.Info {
-	if cq.heap.Len() == 0 {
-		return nil
-	}
-	w := heap.Pop(&cq.heap).(workload.Info)
-	return &w
-}
-
-// strictFIFO is the function used by the clusterQueue heap algorithm to sort
-// workloads. It sorts workloads based on their priority.
-// When priorities are equal, it uses workloads.creationTimestamp.
-func strictFIFO(a, b workload.Info) bool {
-	p1 := utilpriority.Priority(a.Obj)
-	p2 := utilpriority.Priority(b.Obj)
-
-	if p1 != p2 {
-		return p1 > p2
-	}
-	return a.Obj.CreationTimestamp.Before(&b.Obj.CreationTimestamp)
-}
-
 // heap.Interface implementation inspired by
 // https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/internal/heap/heap.go
 
diff --git a/pkg/scheduler/scheduler.go b/pkg/scheduler/scheduler.go
index a00f68c49..f544c265b 100644
--- a/pkg/scheduler/scheduler.go
+++ b/pkg/scheduler/scheduler.go
@@ -348,7 +348,7 @@ func flavorSelector(spec *corev1.PodSpec, allowedKeys sets.String) nodeaffinity.
 // If it fits, also returns any borrowing required.
 func fitsFlavorLimits(name corev1.ResourceName, val int64, cq *cache.ClusterQueue, flavor *cache.FlavorLimits) (bool, int64) {
 	used := cq.UsedResources[name][flavor.Name]
-	if flavor.Ceiling != nil && used+val > *flavor.Ceiling {
+	if used+val > flavor.Ceiling {
 		// Past borrowing limit.
 		return false, 0
 	}
diff --git a/pkg/util/pointer/pointer.go b/pkg/util/pointer/pointer.go
index 63d6a0b0c..00f9e8b45 100644
--- a/pkg/util/pointer/pointer.go
+++ b/pkg/util/pointer/pointer.go
@@ -16,17 +16,8 @@ limitations under the License.
 
 package pointer
 
-import (
-	"k8s.io/apimachinery/pkg/api/resource"
-	"k8s.io/utils/pointer"
-)
+import "k8s.io/apimachinery/pkg/api/resource"
 
 func Quantity(q resource.Quantity) *resource.Quantity {
 	return &q
 }
-
-var (
-	Int32 = pointer.Int32
-	Int64 = pointer.Int64
-	Bool  = pointer.Bool
-)


Test patch:
diff --git a/pkg/cache/cache_test.go b/pkg/cache/cache_test.go
index adcfe542d..cdeb94e31 100644
--- a/pkg/cache/cache_test.go
+++ b/pkg/cache/cache_test.go
@@ -45,15 +45,16 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 				RequestableResources: []kueue.Resource{
 					{
 						Name: corev1.ResourceCPU,
-						Flavors: []kueue.Flavor{{
-							ResourceFlavor: "default",
-							Quota: kueue.Quota{
-								Guaranteed: resource.MustParse("10"),
-								Ceiling:    pointer.Quantity(resource.MustParse("20")),
+						Flavors: []kueue.Flavor{
+							{
+								ResourceFlavor: "default",
+								Quota: kueue.Quota{
+									Guaranteed: resource.MustParse("10"),
+									Ceiling:    resource.MustParse("20"),
+								},
 							},
-						}},
-					},
-				},
+						},
+					}},
 				Cohort: "one",
 			},
 		},
@@ -62,15 +63,9 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 			Spec: kueue.ClusterQueueSpec{
 				RequestableResources: []kueue.Resource{
 					{
-						Name: corev1.ResourceCPU,
-						Flavors: []kueue.Flavor{{
-							ResourceFlavor: "default",
-							Quota: kueue.Quota{
-								Guaranteed: resource.MustParse("15"),
-							},
-						}},
-					},
-				},
+						Name:    corev1.ResourceCPU,
+						Flavors: []kueue.Flavor{{ResourceFlavor: "default"}},
+					}},
 				Cohort: "one",
 			},
 		},
@@ -112,7 +107,7 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 				"a": {
 					Name: "a",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Guaranteed: 10000, Ceiling: pointer.Int64(20000)}},
+						corev1.ResourceCPU: {{Name: "default", Guaranteed: 10000, Ceiling: 20000}},
 					},
 					NamespaceSelector: labels.Nothing(),
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
@@ -121,7 +116,7 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 				"b": {
 					Name: "b",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Guaranteed: 15000}},
+						corev1.ResourceCPU: {{Name: "default"}},
 					},
 					NamespaceSelector: labels.Nothing(),
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
@@ -146,7 +141,7 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 			},
 		},
 		{
-			name: "add flavors after queue capacities",
+			name: "add flavors after queue capacties",
 			operation: func(cache *Cache) {
 				for _, c := range initialClusterQueues {
 					if err := cache.AddClusterQueue(context.Background(), &c); err != nil {
@@ -162,7 +157,7 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 				"a": {
 					Name: "a",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Guaranteed: 10000, Ceiling: pointer.Int64(20000)}},
+						corev1.ResourceCPU: {{Name: "default", Guaranteed: 10000, Ceiling: 20000}},
 					},
 					NamespaceSelector: labels.Nothing(),
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
@@ -171,7 +166,7 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 				"b": {
 					Name: "b",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Guaranteed: 15000}},
+						corev1.ResourceCPU: {{Name: "default"}},
 					},
 					NamespaceSelector: labels.Nothing(),
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
@@ -211,7 +206,7 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 											ResourceFlavor: "default",
 											Quota: kueue.Quota{
 												Guaranteed: resource.MustParse("5"),
-												Ceiling:    pointer.Quantity(resource.MustParse("10")),
+												Ceiling:    resource.MustParse("10"),
 											},
 										},
 									},
@@ -242,7 +237,7 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 				"a": {
 					Name: "a",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Guaranteed: 5000, Ceiling: pointer.Int64(10000)}},
+						corev1.ResourceCPU: {{Name: "default", Guaranteed: 5000, Ceiling: 10000}},
 					},
 					NamespaceSelector: labels.Nothing(),
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType", "region")},
@@ -288,7 +283,7 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 				"b": {
 					Name: "b",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Guaranteed: 15000}},
+						corev1.ResourceCPU: {{Name: "default"}},
 					},
 					NamespaceSelector: labels.Nothing(),
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
@@ -804,7 +799,7 @@ func TestClusterQueueUsage(t *testing.T) {
 							ResourceFlavor: "default",
 							Quota: kueue.Quota{
 								Guaranteed: resource.MustParse("10"),
-								Ceiling:    pointer.Quantity(resource.MustParse("20")),
+								Ceiling:    resource.MustParse("20"),
 							},
 						},
 					},
@@ -816,14 +811,14 @@ func TestClusterQueueUsage(t *testing.T) {
 							ResourceFlavor: "model_a",
 							Quota: kueue.Quota{
 								Guaranteed: resource.MustParse("5"),
-								Ceiling:    pointer.Quantity(resource.MustParse("10")),
+								Ceiling:    resource.MustParse("10"),
 							},
 						},
 						{
 							ResourceFlavor: "model_b",
 							Quota: kueue.Quota{
 								Guaranteed: resource.MustParse("5"),
-								// No ceiling.
+								Ceiling:    resource.MustParse("10"),
 							},
 						},
 					},
diff --git a/pkg/queue/manager_test.go b/pkg/queue/manager_test.go
index cf81ac2be..b35d3a07b 100644
--- a/pkg/queue/manager_test.go
+++ b/pkg/queue/manager_test.go
@@ -774,7 +774,7 @@ func TestHeadsCancelled(t *testing.T) {
 
 // popNamesFromCQ pops all the workloads from the clusterQueue and returns
 // the keyed names in the order they are popped.
-func popNamesFromCQ(cq *ClusterQueue) []string {
+func popNamesFromCQ(cq ClusterQueue) []string {
 	var names []string
 	for w := cq.Pop(); w != nil; w = cq.Pop() {
 		names = append(names, workload.Key(w.Obj))
diff --git a/pkg/scheduler/scheduler_test.go b/pkg/scheduler/scheduler_test.go
index 6f44f0a92..541235707 100644
--- a/pkg/scheduler/scheduler_test.go
+++ b/pkg/scheduler/scheduler_test.go
@@ -33,7 +33,6 @@ import (
 	"k8s.io/client-go/tools/record"
 	ctrl "sigs.k8s.io/controller-runtime"
 	"sigs.k8s.io/controller-runtime/pkg/client/fake"
-	"sigs.k8s.io/kueue/pkg/util/pointer"
 
 	kueue "sigs.k8s.io/kueue/api/v1alpha1"
 	"sigs.k8s.io/kueue/pkg/cache"
@@ -78,7 +77,7 @@ func TestSchedule(t *testing.T) {
 								ResourceFlavor: "default",
 								Quota: kueue.Quota{
 									Guaranteed: resource.MustParse("50"),
-									Ceiling:    pointer.Quantity(resource.MustParse("50")),
+									Ceiling:    resource.MustParse("50"),
 								},
 							},
 						},
@@ -108,14 +107,14 @@ func TestSchedule(t *testing.T) {
 								ResourceFlavor: "on-demand",
 								Quota: kueue.Quota{
 									Guaranteed: resource.MustParse("50"),
-									Ceiling:    pointer.Quantity(resource.MustParse("100")),
+									Ceiling:    resource.MustParse("100"),
 								},
 							},
 							{
 								ResourceFlavor: "spot",
 								Quota: kueue.Quota{
 									Guaranteed: resource.MustParse("100"),
-									Ceiling:    pointer.Quantity(resource.MustParse("100")),
+									Ceiling:    resource.MustParse("100"),
 								},
 							},
 						},
@@ -145,14 +144,14 @@ func TestSchedule(t *testing.T) {
 								ResourceFlavor: "on-demand",
 								Quota: kueue.Quota{
 									Guaranteed: resource.MustParse("50"),
-									Ceiling:    pointer.Quantity(resource.MustParse("60")),
+									Ceiling:    resource.MustParse("60"),
 								},
 							},
 							{
 								ResourceFlavor: "spot",
 								Quota: kueue.Quota{
 									Guaranteed: resource.MustParse("0"),
-									Ceiling:    pointer.Quantity(resource.MustParse("100")),
+									Ceiling:    resource.MustParse("100"),
 								},
 							},
 						},
@@ -164,7 +163,7 @@ func TestSchedule(t *testing.T) {
 								ResourceFlavor: "model-a",
 								Quota: kueue.Quota{
 									Guaranteed: resource.MustParse("20"),
-									Ceiling:    pointer.Quantity(resource.MustParse("20")),
+									Ceiling:    resource.MustParse("20"),
 								},
 							},
 						},
@@ -843,8 +842,8 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU:    {{Name: "default", Guaranteed: 1000}},
-					corev1.ResourceMemory: {{Name: "default", Guaranteed: 2 * utiltesting.Mi}},
+					corev1.ResourceCPU:    defaultFlavorNoBorrowing(1000),
+					corev1.ResourceMemory: defaultFlavorNoBorrowing(2 * utiltesting.Mi),
 				},
 			},
 			wantFits: true,
@@ -868,20 +867,18 @@ func TestEntryAssignFlavors(t *testing.T) {
 								},
 							},
 						},
-						Tolerations: []corev1.Toleration{{
+						Tolerations: []corev1.Toleration{corev1.Toleration{
 							Key:      "instance",
 							Operator: corev1.TolerationOpEqual,
 							Value:    "spot",
 							Effect:   corev1.TaintEffectNoSchedule,
 						}},
-					},
-				},
-			},
+					}}},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "tainted", Guaranteed: 4000},
-					},
+					}),
 				},
 			},
 			wantFits: true,
@@ -903,7 +900,7 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {{Name: "default", Guaranteed: 4000}},
+					corev1.ResourceCPU: defaultFlavorNoBorrowing(4000),
 				},
 				UsedResources: cache.Resources{
 					corev1.ResourceCPU: {
@@ -925,14 +922,14 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: 2000},
 						{Name: "two", Guaranteed: 4000},
-					},
-					corev1.ResourceMemory: {
+					}),
+					corev1.ResourceMemory: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: utiltesting.Gi},
 						{Name: "two", Guaranteed: 5 * utiltesting.Mi},
-					},
+					}),
 				},
 			},
 			wantFits: true,
@@ -956,14 +953,14 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: 2000},
 						{Name: "two", Guaranteed: 4000},
-					},
-					corev1.ResourceMemory: {
+					}),
+					corev1.ResourceMemory: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: utiltesting.Gi},
 						{Name: "two", Guaranteed: 5 * utiltesting.Mi},
-					},
+					}),
 				},
 			},
 		},
@@ -979,10 +976,10 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "tainted", Guaranteed: 4000},
 						{Name: "two", Guaranteed: 4000},
-					},
+					}),
 				},
 			},
 			wantFits: true,
@@ -1004,10 +1001,10 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "non-existent", Guaranteed: 4000},
 						{Name: "two", Guaranteed: 4000},
-					},
+					}),
 				},
 			},
 			wantFits: true,
@@ -1053,10 +1050,10 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: 4000},
 						{Name: "two", Guaranteed: 4000},
-					},
+					}),
 				},
 				LabelKeys: map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
 			},
@@ -1104,14 +1101,14 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: 4000},
 						{Name: "two", Guaranteed: 4000},
-					},
-					corev1.ResourceMemory: {
+					}),
+					corev1.ResourceMemory: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: utiltesting.Gi},
 						{Name: "two", Guaranteed: utiltesting.Gi},
-					},
+					}),
 				},
 			},
 			wantFits: true,
@@ -1168,10 +1165,10 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: 4000},
 						{Name: "two", Guaranteed: 4000},
-					},
+					}),
 				},
 			},
 			wantFits: true,
@@ -1214,10 +1211,10 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: 4000},
 						{Name: "two", Guaranteed: 4000},
-					},
+					}),
 				},
 				LabelKeys: map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
 			},
@@ -1242,10 +1239,10 @@ func TestEntryAssignFlavors(t *testing.T) {
 			},
 			clusterQueue: cache.ClusterQueue{
 				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {
+					corev1.ResourceCPU: noBorrowing([]cache.FlavorLimits{
 						{Name: "one", Guaranteed: 4000},
 						{Name: "two", Guaranteed: 10_000},
-					},
+					}),
 				},
 			},
 			wantFits: true,
@@ -1283,14 +1280,14 @@ func TestEntryAssignFlavors(t *testing.T) {
 						{
 							Name:       "default",
 							Guaranteed: 2000,
-							Ceiling:    pointer.Int64(100_000),
+							Ceiling:    100_000,
 						},
 					},
 					corev1.ResourceMemory: {
 						{
 							Name:       "default",
 							Guaranteed: 2 * utiltesting.Gi,
-							// No ceiling.
+							Ceiling:    100 * utiltesting.Gi,
 						},
 					},
 				},
@@ -1341,7 +1338,7 @@ func TestEntryAssignFlavors(t *testing.T) {
 						{
 							Name:       "one",
 							Guaranteed: 1000,
-							// No ceiling.
+							Ceiling:    10_000,
 						},
 					},
 				},
@@ -1371,7 +1368,7 @@ func TestEntryAssignFlavors(t *testing.T) {
 						{
 							Name:       "one",
 							Guaranteed: 1000,
-							Ceiling:    pointer.Int64(10_000),
+							Ceiling:    10_000,
 						},
 					},
 				},
@@ -1475,3 +1472,14 @@ func TestEntryOrdering(t *testing.T) {
 		t.Errorf("Unexpected order (-want,+got):\n%s", diff)
 	}
 }
+
+func defaultFlavorNoBorrowing(guaranteed int64) []cache.FlavorLimits {
+	return noBorrowing([]cache.FlavorLimits{{Name: "default", Guaranteed: guaranteed}})
+}
+
+func noBorrowing(flavors []cache.FlavorLimits) []cache.FlavorLimits {
+	for i := range flavors {
+		flavors[i].Ceiling = flavors[i].Guaranteed
+	}
+	return flavors
+}
diff --git a/pkg/util/testing/wrappers.go b/pkg/util/testing/wrappers.go
index bf48e11d2..2aa4ecedd 100644
--- a/pkg/util/testing/wrappers.go
+++ b/pkg/util/testing/wrappers.go
@@ -24,10 +24,10 @@ import (
 	schedulingv1 "k8s.io/api/scheduling/v1"
 	"k8s.io/apimachinery/pkg/api/resource"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/utils/pointer"
 
 	kueue "sigs.k8s.io/kueue/api/v1alpha1"
 	"sigs.k8s.io/kueue/pkg/constants"
-	"sigs.k8s.io/kueue/pkg/util/pointer"
 )
 
 // JobWrapper wraps a Job.
@@ -42,8 +42,8 @@ func MakeJob(name, ns string) *JobWrapper {
 			Annotations: make(map[string]string, 1),
 		},
 		Spec: batchv1.JobSpec{
-			Parallelism: pointer.Int32(1),
-			Suspend:     pointer.Bool(true),
+			Parallelism: pointer.Int32Ptr(1),
+			Suspend:     pointer.BoolPtr(true),
 			Template: corev1.PodTemplateSpec{
 				Spec: corev1.PodSpec{
 					RestartPolicy: "Never",
@@ -69,13 +69,13 @@ func (j *JobWrapper) Obj() *batchv1.Job {
 
 // Suspend updates the suspend status of the job
 func (j *JobWrapper) Suspend(s bool) *JobWrapper {
-	j.Spec.Suspend = pointer.Bool(s)
+	j.Spec.Suspend = pointer.BoolPtr(s)
 	return j
 }
 
 // Parallelism updates job parallelism.
 func (j *JobWrapper) Parallelism(p int32) *JobWrapper {
-	j.Spec.Parallelism = pointer.Int32(p)
+	j.Spec.Parallelism = pointer.Int32Ptr(p)
 	return j
 }
 
@@ -315,6 +315,7 @@ func MakeFlavor(rf, guaranteed string) *FlavorWrapper {
 		ResourceFlavor: kueue.ResourceFlavorReference(rf),
 		Quota: kueue.Quota{
 			Guaranteed: resource.MustParse(guaranteed),
+			Ceiling:    resource.MustParse(guaranteed),
 		},
 	}}
 }
@@ -326,7 +327,7 @@ func (f *FlavorWrapper) Obj() *kueue.Flavor {
 
 // Ceiling updates the flavor ceiling.
 func (f *FlavorWrapper) Ceiling(c string) *FlavorWrapper {
-	f.Quota.Ceiling = pointer.Quantity(resource.MustParse(c))
+	f.Quota.Ceiling = resource.MustParse(c)
 	return f
 }
 
diff --git a/test/integration/controller/core/clusterqueue_controller_test.go b/test/integration/controller/core/clusterqueue_controller_test.go
index e39b60ca8..8498675c3 100644
--- a/test/integration/controller/core/clusterqueue_controller_test.go
+++ b/test/integration/controller/core/clusterqueue_controller_test.go
@@ -43,19 +43,10 @@ const (
 
 var _ = ginkgo.Describe("ClusterQueue controller", func() {
 	var (
-		ns                 *corev1.Namespace
-		clusterQueue       *kueue.ClusterQueue
-		queue              *kueue.Queue
-		emptyUsedResources = kueue.UsedResources{
-			corev1.ResourceCPU: {
-				flavorOnDemand: {Total: pointer.Quantity(resource.MustParse("0"))},
-				flavorSpot:     {Total: pointer.Quantity(resource.MustParse("0"))},
-			},
-			resourceGPU: {
-				flavorModelA: {Total: pointer.Quantity(resource.MustParse("0"))},
-				flavorModelB: {Total: pointer.Quantity(resource.MustParse("0"))},
-			},
-		}
+		ns            *corev1.Namespace
+		clusterQueue  *kueue.ClusterQueue
+		queue         *kueue.Queue
+		emptyCqStatus kueue.ClusterQueueStatus
 	)
 
 	ginkgo.BeforeEach(func() {
@@ -82,6 +73,19 @@ var _ = ginkgo.Describe("ClusterQueue controller", func() {
 		gomega.Expect(k8sClient.Create(ctx, clusterQueue)).To(gomega.Succeed())
 		queue = testing.MakeQueue("queue", ns.Name).ClusterQueue(clusterQueue.Name).Obj()
 		gomega.Expect(k8sClient.Create(ctx, queue)).To(gomega.Succeed())
+		emptyCqStatus = kueue.ClusterQueueStatus{
+			AdmittedWorkloads: 0,
+			UsedResources: kueue.UsedResources{
+				corev1.ResourceCPU: {
+					flavorOnDemand: {Total: pointer.Quantity(resource.MustParse("0"))},
+					flavorSpot:     {Total: pointer.Quantity(resource.MustParse("0"))},
+				},
+				resourceGPU: {
+					flavorModelA: {Total: pointer.Quantity(resource.MustParse("0"))},
+					flavorModelB: {Total: pointer.Quantity(resource.MustParse("0"))},
+				},
+			},
+		}
 	})
 
 	ginkgo.AfterEach(func() {
@@ -98,7 +102,7 @@ var _ = ginkgo.Describe("ClusterQueue controller", func() {
 				Request(corev1.ResourceCPU, "1").Request(resourceGPU, "1").Obj(),
 			testing.MakeQueuedWorkload("four", ns.Name).Queue(queue.Name).
 				Request(corev1.ResourceCPU, "1").Request(resourceGPU, "1").Obj(),
-			testing.MakeQueuedWorkload("five", ns.Name).Queue("other").
+			testing.MakeQueuedWorkload("five", ns.Name).Queue(queue.Name).
 				Request(corev1.ResourceCPU, "1").Request(resourceGPU, "1").Obj(),
 			testing.MakeQueuedWorkload("six", ns.Name).Queue(queue.Name).
 				Request(corev1.ResourceCPU, "1").Request(resourceGPU, "1").Obj(),
@@ -112,10 +116,7 @@ var _ = ginkgo.Describe("ClusterQueue controller", func() {
 			var updatedCq kueue.ClusterQueue
 			gomega.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(clusterQueue), &updatedCq)).To(gomega.Succeed())
 			return updatedCq.Status
-		}, framework.Timeout, framework.Interval).Should(testing.Equal(kueue.ClusterQueueStatus{
-			PendingWorkloads: 5,
-			UsedResources:    emptyUsedResources,
-		}))
+		}, framework.Timeout, framework.Interval).Should(testing.Equal(emptyCqStatus))
 
 		ginkgo.By("Admitting workloads")
 		admissions := []*kueue.Admission{
@@ -183,8 +184,6 @@ var _ = ginkgo.Describe("ClusterQueue controller", func() {
 			var updatedCq kueue.ClusterQueue
 			gomega.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(clusterQueue), &updatedCq)).To(gomega.Succeed())
 			return updatedCq.Status
-		}, framework.Timeout, framework.Interval).Should(testing.Equal(kueue.ClusterQueueStatus{
-			UsedResources: emptyUsedResources,
-		}))
+		}, framework.Timeout, framework.Interval).Should(testing.Equal(emptyCqStatus))
 	})
 })
diff --git a/test/integration/controller/core/queuedworkload_controller_test.go b/test/integration/controller/core/queuedworkload_controller_test.go
index 071a660e6..846f0fcf2 100644
--- a/test/integration/controller/core/queuedworkload_controller_test.go
+++ b/test/integration/controller/core/queuedworkload_controller_test.go
@@ -57,7 +57,7 @@ var _ = ginkgo.Describe("QueuedWorkload controller", func() {
 		})
 		ginkgo.It("Should update status when workloads are created", func() {
 			wl = testing.MakeQueuedWorkload("one", ns.Name).Request(corev1.ResourceCPU, "1").Obj()
-			message = fmt.Sprintf("Queue %s doesn't exist", "")
+			message = fmt.Sprintf("Queue %s for workload doesn't exist", "")
 			gomega.Expect(k8sClient.Create(ctx, wl)).To(gomega.Succeed())
 			gomega.Eventually(func() int {
 				gomega.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(wl), &updatedQueueWorkload)).To(gomega.Succeed())
@@ -74,7 +74,7 @@ var _ = ginkgo.Describe("QueuedWorkload controller", func() {
 		})
 		ginkgo.It("Should update status when workloads are created", func() {
 			wl = testing.MakeQueuedWorkload("two", ns.Name).Queue("nonCreatedQueue").Request(corev1.ResourceCPU, "1").Obj()
-			message = fmt.Sprintf("Queue %s doesn't exist", "nonCreatedQueue")
+			message = fmt.Sprintf("Queue %s for workload doesn't exist", "nonCreatedQueue")
 			gomega.Expect(k8sClient.Create(ctx, wl)).To(gomega.Succeed())
 			gomega.Eventually(func() int {
 				gomega.Expect(k8sClient.Get(ctx, client.ObjectKeyFromObject(wl), &updatedQueueWorkload)).To(gomega.Succeed())
diff --git a/test/integration/framework/framework.go b/test/integration/framework/framework.go
index e6f0cc03c..110b9e134 100644
--- a/test/integration/framework/framework.go
+++ b/test/integration/framework/framework.go
@@ -24,7 +24,6 @@ import (
 	batchv1 "k8s.io/api/batch/v1"
 	corev1 "k8s.io/api/core/v1"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/client-go/kubernetes/scheme"
 	"k8s.io/client-go/rest"
 	ctrl "sigs.k8s.io/controller-runtime"
@@ -116,8 +115,7 @@ func DeleteNamespace(ctx context.Context, c client.Client, ns *corev1.Namespace)
 	if ns == nil {
 		return nil
 	}
-	err := c.DeleteAllOf(ctx, &batchv1.Job{}, client.InNamespace(ns.Name), client.PropagationPolicy(metav1.DeletePropagationBackground))
-	if err != nil && !apierrors.IsNotFound(err) {
+	if err := c.DeleteAllOf(ctx, &batchv1.Job{}, client.InNamespace(ns.Name)); err != nil && !apierrors.IsNotFound(err) {
 		return err
 	}
 	if err := c.DeleteAllOf(ctx, &kueue.Queue{}, client.InNamespace(ns.Name)); err != nil && !apierrors.IsNotFound(err) {
diff --git a/test/integration/scheduler/scheduler_test.go b/test/integration/scheduler/scheduler_test.go
index 996184c06..159aca133 100644
--- a/test/integration/scheduler/scheduler_test.go
+++ b/test/integration/scheduler/scheduler_test.go
@@ -46,7 +46,6 @@ var _ = ginkgo.Describe("Scheduler", func() {
 		onDemandFlavor      *kueue.ResourceFlavor
 		spotTaintedFlavor   *kueue.ResourceFlavor
 		spotUntaintedFlavor *kueue.ResourceFlavor
-		spotToleration      corev1.Toleration
 	)
 
 	ginkgo.BeforeEach(func() {
@@ -68,12 +67,6 @@ var _ = ginkgo.Describe("Scheduler", func() {
 				Effect: corev1.TaintEffectNoSchedule,
 			}).Obj()
 		gomega.Expect(k8sClient.Create(ctx, spotTaintedFlavor)).Should(gomega.Succeed())
-		spotToleration = corev1.Toleration{
-			Key:      instanceKey,
-			Operator: corev1.TolerationOpEqual,
-			Value:    spotTaintedFlavor.Name,
-			Effect:   corev1.TaintEffectNoSchedule,
-		}
 
 		spotUntaintedFlavor = testing.MakeResourceFlavor("spot-untainted").Label(instanceKey, "spot-untainted").Obj()
 		gomega.Expect(k8sClient.Create(ctx, spotUntaintedFlavor)).Should(gomega.Succeed())
@@ -81,8 +74,8 @@ var _ = ginkgo.Describe("Scheduler", func() {
 		prodClusterQ = testing.MakeClusterQueue("prod-cq").
 			Cohort("prod").
 			Resource(testing.MakeResource(corev1.ResourceCPU).
-				Flavor(testing.MakeFlavor(spotTaintedFlavor.Name, "5").Ceiling("5").Obj()).
-				Flavor(testing.MakeFlavor(onDemandFlavor.Name, "5").Obj()).
+				Flavor(testing.MakeFlavor(spotTaintedFlavor.Name, "5").Obj()).
+				Flavor(testing.MakeFlavor(onDemandFlavor.Name, "5").Ceiling("10").Obj()).
 				Obj()).
 			Obj()
 		gomega.Expect(k8sClient.Create(ctx, prodClusterQ)).Should(gomega.Succeed())
@@ -182,7 +175,12 @@ var _ = ginkgo.Describe("Scheduler", func() {
 
 		ginkgo.By("checking a third job with toleration starts")
 		job3 := testing.MakeJob("spot-job3", ns.Name).Queue(prodQueue.Name).
-			Toleration(spotToleration).
+			Toleration(corev1.Toleration{
+				Key:      instanceKey,
+				Operator: corev1.TolerationOpEqual,
+				Value:    spotTaintedFlavor.Name,
+				Effect:   corev1.TaintEffectNoSchedule,
+			}).
 			Request(corev1.ResourceCPU, "5").Obj()
 		gomega.Expect(k8sClient.Create(ctx, job3)).Should(gomega.Succeed())
 		createdJob3 := &batchv1.Job{}
@@ -195,8 +193,7 @@ var _ = ginkgo.Describe("Scheduler", func() {
 
 	ginkgo.It("Should schedule jobs using borrowed ClusterQueue", func() {
 		ginkgo.By("checking a no-fit job does not start")
-		job := testing.MakeJob("job", ns.Name).Queue(prodQueue.Name).
-			Request(corev1.ResourceCPU, "10").Toleration(spotToleration).Obj()
+		job := testing.MakeJob("job", ns.Name).Queue(prodQueue.Name).Request(corev1.ResourceCPU, "10").Obj()
 		gomega.Expect(k8sClient.Create(ctx, job)).Should(gomega.Succeed())
 		lookupKey := types.NamespacedName{Name: job.Name, Namespace: job.Namespace}
 		createdJob := &batchv1.Job{}
@@ -208,8 +205,7 @@ var _ = ginkgo.Describe("Scheduler", func() {
 		fallbackClusterQueue := testing.MakeClusterQueue("fallback-cq").
 			Cohort(prodClusterQ.Spec.Cohort).
 			Resource(testing.MakeResource(corev1.ResourceCPU).
-				Flavor(testing.MakeFlavor(spotTaintedFlavor.Name, "5").Obj()). // prod-cq can't borrow this flavor due to its ceiling.
-				Flavor(testing.MakeFlavor(onDemandFlavor.Name, "5").Obj()).
+				Flavor(testing.MakeFlavor(onDemandFlavor.Name, "5").Ceiling("10").Obj()).
 				Obj()).
 			Obj()
 		gomega.Expect(k8sClient.Create(ctx, fallbackClusterQueue)).Should(gomega.Succeed())