Repository: kubernetes-sigs/kueue
Pull request #248 (2022-05-04 23:16:49)

fix: forget to cleanup inadmissibleWorkloads when deleteFromQueue
Signed-off-by: kerthcet <kerthcet@gmail.com>

<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?
/kind bug

<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

#### What this PR does / why we need it:
Met this panic several times in testing:
```
goroutine 560 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic({0x21d9e40?, 0x31c25b0})
        /Users/kerthcet/Workspaces/pkg/mod/k8s.io/apimachinery@v0.23.4/pkg/util/runtime/runtime.go:74 +0x86
k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc0000ddc00?})
        /Users/kerthcet/Workspaces/pkg/mod/k8s.io/apimachinery@v0.23.4/pkg/util/runtime/runtime.go:48 +0x75
panic({0x21d9e40, 0x31c25b0})
        /Users/kerthcet/sdk/go1.18/src/runtime/panic.go:838 +0x207
sigs.k8s.io/kueue/pkg/queue.(*Manager).heads(0xc0000ddc00)
        /Users/kerthcet/Workspaces/kueue/pkg/queue/manager.go:452 +0x32b
sigs.k8s.io/kueue/pkg/queue.(*Manager).Heads(0xc0000ddc00, {0x261b390, 0xc0009be210})
        /Users/kerthcet/Workspaces/kueue/pkg/queue/manager.go:407 +0x105
sigs.k8s.io/kueue/pkg/scheduler.(*Scheduler).schedule(0xc000902640, {0x261b390, 0xc0009be210})
        /Users/kerthcet/Workspaces/kueue/pkg/scheduler/scheduler.go:85 +0x96
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
        /Users/kerthcet/Workspaces/pkg/mod/k8s.io/apimachinery@v0.23.4/pkg/util/wait/wait.go:188 +0x25
```
#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:



k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/kueue/pull/248#" title="Author self-approved">kerthcet</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **ahg-g** after the PR has been reviewed.
You can assign the PR to them by writing `/assign @ahg-g` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fkueue).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/kueue/blob/main/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["ahg-g"]} -->
ahg-g: integration or unit tests you faced this? and which test exactly?
denkensk: Maybe checking if it is nil or not  is not enough here. 
If the queue of workload is nil, it shouldn't been put into ClusterQueue.
kerthcet: I'll try to reproduce this error and find the cause here.
alculquicondor: When we Delete a queue, we remove all the workloads related to it from the ClusterQueue. So this shouldn't be happening, meaning that we might have a bug somewhere else.

Let's not merge this as it will obscure a real bug.
kerthcet: Recently error occurred here, just for trace https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/245/pull-kueue-test-integration-main/1522576998889689088
alculquicondor: ```
Scheduler Should schedule jobs as they fit in their ClusterQueue
/home/prow/go/src/sigs.k8s.io/kueue/test/integration/scheduler/scheduler_test.go:116
```
alculquicondor: I'm thinking that this error could be coming not from the test above itself, but from the cleanups of other tests.

Two reasons:
- The test itself doesn't remove any queues.
- There are no log lines other than the crash, suggesting that the test failed right at the beginning.
alculquicondor: @kerthcet, I think you said you have encountered this while developing locally.

Could you try running the suite with ginkgo?

```sh
ginkgo -v ./test/integration/scheduler
```
kerthcet: This maybe a bug with `ClusterQueueBestEffortFIFO`:

https://github.com/kubernetes-sigs/kueue/blob/20406a45269c12e6f37bfbaff0fb5ad4625a8b9a/test/integration/scheduler/scheduler_test.go#L304-L355

Refer to above spec, here's the possible sequence of events:
1. LowPriorityWorkload and HighPriorityWorkload created
2. Since this is a BestEffortFIFO, unadmitted LowPriorityWorkload will be stored in inadmissibleWorkloads
3. Test passed, will enter cleanup phase.
4. When deleting the namespace, we will delete the queues, but forget to cleanup the LowPriorityWorkload in  inadmissibleWorkloads, this is a bug I think.
5. Deleting workloads, but since queues are already deleted, we will directly return in `deleteWorkloadFromQueueAndClusterQueue`, and the LowPriorityWorkload is still in inadmissibleWorkloads.
6. Somehow triggered the broadcast, drive the workloads from inadmissibleWorkloads and went into the `Heads()` process, since clusterQueues haven't been deleted, but queues are already deleted, panic then. We can find the proofs from the log below: the queues are all deleted, together with the workloads, but clusterQueues haven't been deleted for namespace is still in terminating.
```
2022-05-07T10:39:45.932609+08:00        LEVEL(-2)       queue-reconciler        Queue delete event      {"queue": {"name":"dev-queue","namespace":"core-xhfrf"}}
2022-05-07T10:39:45.933959+08:00        LEVEL(-2)       queue-reconciler        Queue delete event      {"queue": {"name":"prod-queue","namespace":"core-xhfrf"}}
2022-05-07T10:39:45.936032+08:00        LEVEL(-2)       queue-reconciler        Queue delete event      {"queue": {"name":"queue","namespace":"core-xhfrf"}}
2022-05-07T10:39:45.943974+08:00        LEVEL(-2)       workload-reconciler     Workload delete event   {"workload": {"name":"job-high-priority","namespace":"core-xhfrf"}, "queue": "queue", "status": "admitted"}
2022-05-07T10:39:45.944129+08:00        LEVEL(-3)       cluster-queue-reconciler        Got Workload event      {"workload": {"name":"job-high-priority","namespace":"core-xhfrf"}}
2022-05-07T10:39:45.944115+08:00        LEVEL(-3)       queue-reconciler        Got Workload event      {"workload": {"name":"job-high-priority","namespace":"core-xhfrf"}}
2022-05-07T10:39:45.947496+08:00        LEVEL(-2)       workload-reconciler     Workload delete event   {"workload": {"name":"job-low-priority","namespace":"core-xhfrf"}, "queue": "queue", "status": "pending"}
2022-05-07T10:39:45.947631+08:00        LEVEL(-3)       queue-reconciler        Got Workload event      {"workload": {"name":"job-low-priority","namespace":"core-xhfrf"}}
2022-05-07T10:39:45.947669+08:00        LEVEL(-3)       cluster-queue-reconciler        Got Workload event      {"workload": {"name":"job-low-priority","namespace":"core-xhfrf"}}
2022-05-07T10:39:45.949348+08:00        LEVEL(-2)       controller.clusterqueue Reconciling ClusterQueue        {"reconciler group": "kueue.x-k8s.io", "reconciler kind": "ClusterQueue", "name": "prod-cq", "namespace": "", "clusterQueue":
{"name":"prod-cq"}}
2022-05-07T10:39:45.955853+08:00        LEVEL(-2)       cluster-queue-reconciler        ClusterQueue update event       {"clusterQueue": {"name":"prod-cq"}}
E0507 10:39:45.956005   34767 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 679 [running]:
```

However, I haven't figure out why clusterQueue got an update event in the last phase. 
ahg-g: Thanks @kerthcet for the analysis.

I believe we need to override DeleteFromQueue in the fifo implementation to also delete workloads from the inadmissibleWorkloads map: https://github.com/kubernetes-sigs/kueue/blob/b515c85fabbd343ed104b933bc6a0e67f3278253/pkg/queue/cluster_queue_impl.go#L65
kerthcet: > Thanks @kerthcet for the analysis.
> 
> I believe we need to override DeleteFromQueue in the fifo implementation to also delete workloads from the inadmissibleWorkloads map:
> 
> https://github.com/kubernetes-sigs/kueue/blob/b515c85fabbd343ed104b933bc6a0e67f3278253/pkg/queue/cluster_queue_impl.go#L65

yes, exactly.
kerthcet: addressed.
ahg-g: Thanks, pls squash.

Patch:
diff --git a/.github/ISSUE_TEMPLATE/NEW_RELEASE.md b/.github/ISSUE_TEMPLATE/NEW_RELEASE.md
index 4cd8a652d..e8d66667e 100644
--- a/.github/ISSUE_TEMPLATE/NEW_RELEASE.md
+++ b/.github/ISSUE_TEMPLATE/NEW_RELEASE.md
@@ -22,7 +22,7 @@ Please do not remove items from the checklist
 - [ ] An OWNER [prepares a draft release](https://github.com/kubernetes-sigs/kueue/releases)
   - [ ] Write the change log into the draft release.
   - [ ] Run
-      `make artifacts IMAGE_REGISTRY=registry.k8s.io/kueue GIT_TAG=$VERSION`
+      `make artifacts IMAGE_REGISTRY=k8s.gcr.io/kueue GIT_TAG=$VERSION`
       to generate the artifacts and upload the files in the `artifacts` folder
       to the draft release.
 - [ ] An OWNER creates a signed tag running
@@ -37,7 +37,7 @@ Please do not remove items from the checklist
       updating `k8s.gcr.io/images/k8s-staging-kueue/images.yaml` to
       [promote the container images](https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io#image-promoter)
       to production: <!-- example kubernetes/k8s.io#3612-->
-- [ ] Wait for the PR to be merged and verify that the image `registry.k8s.io/kueue/kueue:$VERSION` is available.
+- [ ] Wait for the PR to be merged and verify that the image `k8s.gcr.io/kueue/kueue:$VERSION` is available.
 - [ ] Publish the draft release prepared at the [Github releases page](https://github.com/kubernetes-sigs/kueue/releases).
 - [ ] Add a link to the tagged release in this issue: <!-- example https://github.com/kubernetes-sigs/kueue/releases/tag/v0.1.0 -->
 - [ ] Send an announcement email to `sig-scheduling@kubernetes.io` and `wg-batch@kubernetes.io` with the subject `[ANNOUNCE] kueue $VERSION is released`
diff --git a/Makefile b/Makefile
index 3cd1564eb..29e84db42 100644
--- a/Makefile
+++ b/Makefile
@@ -180,9 +180,10 @@ undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/confi
 .PHONY: artifacts
 artifacts: kustomize
 	cd config/manager && $(KUSTOMIZE) edit set image controller=${IMAGE_TAG}
-	if [ -d artifacts ]; then rm -rf artifacts; fi
 	@mkdir -p artifacts
+	@rm -r artifacts/*
 	$(KUSTOMIZE) build config/default -o artifacts/manifests.yaml
+	@$(call clean-manifests)
 
 ##@ Tools
 PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))
diff --git a/main.go b/main.go
index 2eddd4499..8057bef33 100644
--- a/main.go
+++ b/main.go
@@ -26,7 +26,6 @@ import (
 	// to ensure that exec-entrypoint and run can make use of them.
 	_ "k8s.io/client-go/plugin/pkg/client/auth"
 
-	zaplog "go.uber.org/zap"
 	"go.uber.org/zap/zapcore"
 	schedulingv1 "k8s.io/api/scheduling/v1"
 	"k8s.io/apimachinery/pkg/runtime"
@@ -71,7 +70,6 @@ func main() {
 
 	opts := zap.Options{
 		TimeEncoder: zapcore.RFC3339NanoTimeEncoder,
-		ZapOpts:     []zaplog.Option{zaplog.AddCaller(), zaplog.AddCallerSkip(-1)},
 	}
 	opts.BindFlags(flag.CommandLine)
 	flag.Parse()
diff --git a/pkg/cache/cache.go b/pkg/cache/cache.go
index 7bdbd9b2b..2554c750c 100644
--- a/pkg/cache/cache.go
+++ b/pkg/cache/cache.go
@@ -80,19 +80,6 @@ func newCohort(name string, size int) *Cohort {
 	}
 }
 
-type ClusterQueueStatus int
-
-const (
-	// Pending means the ClusterQueue is accepted but not yet active,
-	// this can be because of a missing ResourceFlavor referenced by the ClusterQueue.
-	// In this state, the ClusterQueue can't admit new workloads and its quota can't be borrowed
-	// by other active ClusterQueues in the cohort.
-	Pending ClusterQueueStatus = iota
-	// Active means the ClusterQueue can admit new workloads and its quota
-	// can be borrowed by other ClusterQueues in the cohort.
-	Active
-)
-
 // ClusterQueue is the internal implementation of kueue.ClusterQueue that
 // holds admitted workloads.
 type ClusterQueue struct {
@@ -106,7 +93,6 @@ type ClusterQueue struct {
 	// Those keys define the affinity terms of a workload
 	// that can be matched against the flavors.
 	LabelKeys map[corev1.ResourceName]sets.String
-	Status    ClusterQueueStatus
 }
 
 // FlavorLimits holds a processed ClusterQueue flavor quota.
@@ -128,10 +114,6 @@ func (c *Cache) newClusterQueue(cq *kueue.ClusterQueue) (*ClusterQueue, error) {
 	return cqImpl, nil
 }
 
-func (c *ClusterQueue) Active() bool {
-	return c.Status == Active
-}
-
 func (c *ClusterQueue) update(in *kueue.ClusterQueue, resourceFlavors map[string]*kueue.ResourceFlavor) error {
 	c.RequestableResources = resourceLimitsByName(in.Spec.Resources)
 	nsSelector, err := metav1.LabelSelectorAsSelector(in.Spec.NamespaceSelector)
@@ -154,22 +136,13 @@ func (c *ClusterQueue) update(in *kueue.ClusterQueue, resourceFlavors map[string
 		usedResources[r.Name] = usedFlavors
 	}
 	c.UsedResources = usedResources
-	c.UpdateWithFlavors(resourceFlavors)
+	c.UpdateLabelKeys(resourceFlavors)
 	return nil
 }
 
-// UpdateWithFlavors updates a ClusterQueue based on the passed ResourceFlavors set.
+// UpdateLabelKeys updates a ClusterQueue's LabelKeys based on the passed ResourceFlavors set.
 // Exported only for testing.
-func (c *ClusterQueue) UpdateWithFlavors(flavors map[string]*kueue.ResourceFlavor) {
-	if flavorNotFound := c.updateLabelKeys(flavors); flavorNotFound {
-		c.Status = Pending
-		return
-	}
-	c.Status = Active
-}
-
-func (c *ClusterQueue) updateLabelKeys(flavors map[string]*kueue.ResourceFlavor) bool {
-	var flavorNotFound bool
+func (c *ClusterQueue) UpdateLabelKeys(flavors map[string]*kueue.ResourceFlavor) {
 	labelKeys := map[corev1.ResourceName]sets.String{}
 	for rName, flvLimits := range c.RequestableResources {
 		if len(flvLimits) == 0 {
@@ -181,9 +154,9 @@ func (c *ClusterQueue) updateLabelKeys(flavors map[string]*kueue.ResourceFlavor)
 				for k := range flv.Labels {
 					resKeys.Insert(k)
 				}
-			} else {
-				flavorNotFound = true
 			}
+			// We don't error here on missing flavor since ResourceFlavor add events may
+			// come after ClusterQueue add/update.
 		}
 
 		if len(resKeys) != 0 {
@@ -195,8 +168,6 @@ func (c *ClusterQueue) updateLabelKeys(flavors map[string]*kueue.ResourceFlavor)
 	if len(labelKeys) != 0 {
 		c.LabelKeys = labelKeys
 	}
-
-	return flavorNotFound
 }
 
 func (c *ClusterQueue) addWorkload(w *kueue.Workload) error {
@@ -234,45 +205,22 @@ func (c *ClusterQueue) updateWorkloadUsage(wi *workload.Info, m int64) {
 	}
 }
 
-func (c *Cache) updateClusterQueues() sets.String {
-	cqs := sets.NewString()
-
+func (c *Cache) AddOrUpdateResourceFlavor(rf *kueue.ResourceFlavor) {
+	c.Lock()
+	c.resourceFlavors[rf.Name] = rf
 	for _, cq := range c.clusterQueues {
-		prevStatus := cq.Status
 		// We call update on all ClusterQueues irrespective of which CQ actually use this flavor
 		// because it is not expensive to do so, and is not worth tracking which ClusterQueues use
 		// which flavors.
-		cq.UpdateWithFlavors(c.resourceFlavors)
-		curStatus := cq.Status
-		if prevStatus == Pending && curStatus == Active {
-			cqs.Insert(cq.Name)
-		}
+		cq.UpdateLabelKeys(c.resourceFlavors)
 	}
-	return cqs
-}
-
-func (c *Cache) AddOrUpdateResourceFlavor(rf *kueue.ResourceFlavor) sets.String {
-	c.Lock()
-	defer c.Unlock()
-	c.resourceFlavors[rf.Name] = rf
-	return c.updateClusterQueues()
+	c.Unlock()
 }
 
-func (c *Cache) DeleteResourceFlavor(rf *kueue.ResourceFlavor) sets.String {
+func (c *Cache) DeleteResourceFlavor(rf *kueue.ResourceFlavor) {
 	c.Lock()
-	defer c.Unlock()
 	delete(c.resourceFlavors, rf.Name)
-	return c.updateClusterQueues()
-}
-
-func (c *Cache) ClusterQueueActive(name string) bool {
-	c.RLock()
-	defer c.RUnlock()
-	cq, exists := c.clusterQueues[name]
-	if !exists {
-		return false
-	}
-	return cq != nil && cq.Active()
+	c.Unlock()
 }
 
 func (c *Cache) AddClusterQueue(ctx context.Context, cq *kueue.ClusterQueue) error {
@@ -316,14 +264,12 @@ func (c *Cache) UpdateClusterQueue(cq *kueue.ClusterQueue) error {
 	if err := cqImpl.update(cq, c.resourceFlavors); err != nil {
 		return err
 	}
-
-	if cqImpl.Cohort == nil {
-		c.addClusterQueueToCohort(cqImpl, cq.Spec.Cohort)
-		return nil
-	}
-
-	if cqImpl.Cohort.Name != cq.Spec.Cohort {
-		c.deleteClusterQueueFromCohort(cqImpl)
+	if cqImpl.Cohort != nil {
+		if cqImpl.Cohort.Name != cq.Spec.Cohort {
+			c.deleteClusterQueueFromCohort(cqImpl)
+			c.addClusterQueueToCohort(cqImpl, cq.Spec.Cohort)
+		}
+	} else {
 		c.addClusterQueueToCohort(cqImpl, cq.Spec.Cohort)
 	}
 	return nil
@@ -394,14 +340,14 @@ func (c *Cache) DeleteWorkload(w *kueue.Workload) error {
 		return errWorkloadNotAdmitted
 	}
 
-	cq, ok := c.clusterQueues[string(w.Spec.Admission.ClusterQueue)]
+	qc, ok := c.clusterQueues[string(w.Spec.Admission.ClusterQueue)]
 	if !ok {
 		return errCqNotFound
 	}
 
 	c.cleanupAssumedState(w)
 
-	cq.deleteWorkload(w)
+	qc.deleteWorkload(w)
 	return nil
 }
 
@@ -461,7 +407,6 @@ func (c *Cache) Usage(cqObj *kueue.ClusterQueue) (kueue.UsedResources, int, erro
 	if cq == nil {
 		return nil, 0, errCqNotFound
 	}
-
 	usage := make(kueue.UsedResources, len(cq.UsedResources))
 	for rName, usedRes := range cq.UsedResources {
 		rUsage := make(map[string]kueue.Usage)
diff --git a/pkg/cache/snapshot.go b/pkg/cache/snapshot.go
index ef684e04a..e82239824 100644
--- a/pkg/cache/snapshot.go
+++ b/pkg/cache/snapshot.go
@@ -17,16 +17,13 @@ limitations under the License.
 package cache
 
 import (
-	"k8s.io/apimachinery/pkg/util/sets"
-
 	kueue "sigs.k8s.io/kueue/apis/kueue/v1alpha1"
 	"sigs.k8s.io/kueue/pkg/workload"
 )
 
 type Snapshot struct {
-	ClusterQueues            map[string]*ClusterQueue
-	ResourceFlavors          map[string]*kueue.ResourceFlavor
-	InactiveClusterQueueSets sets.String
+	ClusterQueues   map[string]*ClusterQueue
+	ResourceFlavors map[string]*kueue.ResourceFlavor
 }
 
 func (c *Cache) Snapshot() Snapshot {
@@ -34,15 +31,10 @@ func (c *Cache) Snapshot() Snapshot {
 	defer c.RUnlock()
 
 	snap := Snapshot{
-		ClusterQueues:            make(map[string]*ClusterQueue, len(c.clusterQueues)),
-		ResourceFlavors:          make(map[string]*kueue.ResourceFlavor, len(c.resourceFlavors)),
-		InactiveClusterQueueSets: sets.NewString(),
+		ClusterQueues:   make(map[string]*ClusterQueue, len(c.clusterQueues)),
+		ResourceFlavors: make(map[string]*kueue.ResourceFlavor, len(c.resourceFlavors)),
 	}
 	for _, cq := range c.clusterQueues {
-		if !cq.Active() {
-			snap.InactiveClusterQueueSets.Insert(cq.Name)
-			continue
-		}
 		snap.ClusterQueues[cq.Name] = cq.snapshot()
 	}
 	for _, rf := range c.resourceFlavors {
@@ -52,12 +44,10 @@ func (c *Cache) Snapshot() Snapshot {
 	for _, cohort := range c.cohorts {
 		cohortCopy := newCohort(cohort.Name, len(cohort.members))
 		for cq := range cohort.members {
-			if cq.Active() {
-				cqCopy := snap.ClusterQueues[cq.Name]
-				cqCopy.accumulateResources(cohortCopy)
-				cqCopy.Cohort = cohortCopy
-				cohortCopy.members[cqCopy] = struct{}{}
-			}
+			cqCopy := snap.ClusterQueues[cq.Name]
+			cqCopy.accumulateResources(cohortCopy)
+			cqCopy.Cohort = cohortCopy
+			cohortCopy.members[cqCopy] = struct{}{}
 		}
 	}
 	return snap
@@ -73,7 +63,6 @@ func (c *ClusterQueue) snapshot() *ClusterQueue {
 		Workloads:            make(map[string]*workload.Info, len(c.Workloads)),
 		LabelKeys:            c.LabelKeys, // Shallow copy is enough.
 		NamespaceSelector:    c.NamespaceSelector,
-		Status:               c.Status,
 	}
 	for res, flavors := range c.UsedResources {
 		flavorsCopy := make(map[string]int64, len(flavors))
diff --git a/pkg/controller/core/clusterqueue_controller.go b/pkg/controller/core/clusterqueue_controller.go
index 45c4d9c4f..cd316d2ad 100644
--- a/pkg/controller/core/clusterqueue_controller.go
+++ b/pkg/controller/core/clusterqueue_controller.go
@@ -107,7 +107,6 @@ func (r *ClusterQueueReconciler) Create(e event.CreateEvent) bool {
 	if err := r.cache.AddClusterQueue(ctx, cq); err != nil {
 		log.Error(err, "Failed to add clusterQueue to cache")
 	}
-
 	if err := r.qManager.AddClusterQueue(ctx, cq); err != nil {
 		log.Error(err, "Failed to add clusterQueue to queue manager")
 	}
@@ -134,7 +133,6 @@ func (r *ClusterQueueReconciler) Update(e event.UpdateEvent) bool {
 	}
 	log := r.log.WithValues("clusterQueue", klog.KObj(cq))
 	log.V(2).Info("ClusterQueue update event")
-
 	if err := r.cache.UpdateClusterQueue(cq); err != nil {
 		log.Error(err, "Failed to update clusterQueue in cache")
 	}
diff --git a/pkg/controller/core/core.go b/pkg/controller/core/core.go
index edb2aa5b9..617766dfd 100644
--- a/pkg/controller/core/core.go
+++ b/pkg/controller/core/core.go
@@ -37,7 +37,7 @@ func SetupControllers(mgr ctrl.Manager, qManager *queue.Manager, cc *cache.Cache
 	if err := NewWorkloadReconciler(mgr.GetClient(), qManager, cc, qRec, cqRec).SetupWithManager(mgr); err != nil {
 		return "Workload", err
 	}
-	if err := NewResourceFlavorReconciler(qManager, cc).SetupWithManager(mgr); err != nil {
+	if err := NewResourceFlavorReconciler(cc).SetupWithManager(mgr); err != nil {
 		return "ResourceFlavor", err
 	}
 	return "", nil
diff --git a/pkg/controller/core/resourceflavor_controller.go b/pkg/controller/core/resourceflavor_controller.go
index 402918c53..8fb51ab35 100644
--- a/pkg/controller/core/resourceflavor_controller.go
+++ b/pkg/controller/core/resourceflavor_controller.go
@@ -26,21 +26,18 @@ import (
 
 	kueue "sigs.k8s.io/kueue/apis/kueue/v1alpha1"
 	"sigs.k8s.io/kueue/pkg/cache"
-	"sigs.k8s.io/kueue/pkg/queue"
 )
 
 // ResourceFlavorReconciler reconciles a ResourceFlavor object
 type ResourceFlavorReconciler struct {
-	log      logr.Logger
-	qManager *queue.Manager
-	cache    *cache.Cache
+	log   logr.Logger
+	cache *cache.Cache
 }
 
-func NewResourceFlavorReconciler(qMgr *queue.Manager, cache *cache.Cache) *ResourceFlavorReconciler {
+func NewResourceFlavorReconciler(cache *cache.Cache) *ResourceFlavorReconciler {
 	return &ResourceFlavorReconciler{
-		log:      ctrl.Log.WithName("resourceflavor-reconciler"),
-		cache:    cache,
-		qManager: qMgr,
+		log:   ctrl.Log.WithName("resourceflavor-reconciler"),
+		cache: cache,
 	}
 }
 
@@ -56,15 +53,9 @@ func (r *ResourceFlavorReconciler) Create(e event.CreateEvent) bool {
 	if !match {
 		return false
 	}
-
 	log := r.log.WithValues("resourceFlavor", klog.KObj(flv))
 	log.V(2).Info("ResourceFlavor create event")
-
-	// As long as one clusterQueue becomes active,
-	// we should inform clusterQueue controller to broadcast the event.
-	if cqNames := r.cache.AddOrUpdateResourceFlavor(flv.DeepCopy()); len(cqNames) > 0 {
-		r.qManager.QueueInadmissibleWorkloads(cqNames)
-	}
+	r.cache.AddOrUpdateResourceFlavor(flv.DeepCopy())
 	return false
 }
 
@@ -73,13 +64,9 @@ func (r *ResourceFlavorReconciler) Delete(e event.DeleteEvent) bool {
 	if !match {
 		return false
 	}
-
 	log := r.log.WithValues("resourceFlavor", klog.KObj(flv))
 	log.V(2).Info("ResourceFlavor delete event")
-
-	if cqNames := r.cache.DeleteResourceFlavor(flv); len(cqNames) > 0 {
-		r.qManager.QueueInadmissibleWorkloads(cqNames)
-	}
+	r.cache.DeleteResourceFlavor(flv)
 	return false
 }
 
@@ -88,13 +75,9 @@ func (r *ResourceFlavorReconciler) Update(e event.UpdateEvent) bool {
 	if !match {
 		return false
 	}
-
 	log := r.log.WithValues("resourceFlavor", klog.KObj(flv))
 	log.V(2).Info("ResourceFlavor update event")
-
-	if cqNames := r.cache.AddOrUpdateResourceFlavor(flv.DeepCopy()); len(cqNames) > 0 {
-		r.qManager.QueueInadmissibleWorkloads(cqNames)
-	}
+	r.cache.AddOrUpdateResourceFlavor(flv.DeepCopy())
 	return false
 }
 
diff --git a/pkg/controller/core/workload_controller.go b/pkg/controller/core/workload_controller.go
index 6f71e54ad..71ac5e97d 100644
--- a/pkg/controller/core/workload_controller.go
+++ b/pkg/controller/core/workload_controller.go
@@ -94,12 +94,6 @@ func (r *WorkloadReconciler) Reconcile(ctx context.Context, req ctrl.Request) (c
 		return ctrl.Result{}, client.IgnoreNotFound(err)
 	}
 
-	if status == pending && !r.cache.ClusterQueueActive(cqName) {
-		err := workload.UpdateStatusIfChanged(ctx, r.client, &wl, kueue.WorkloadAdmitted, corev1.ConditionFalse,
-			"Inadmissible", fmt.Sprintf("ClusterQueue %s is inactive", cqName))
-		return ctrl.Result{}, client.IgnoreNotFound(err)
-	}
-
 	if status == admitted {
 		err := workload.UpdateStatusIfChanged(ctx, r.client, &wl, kueue.WorkloadAdmitted, corev1.ConditionTrue, "", "")
 		return ctrl.Result{}, client.IgnoreNotFound(err)
diff --git a/pkg/queue/cluster_queue_best_effort_fifo.go b/pkg/queue/cluster_queue_best_effort_fifo.go
index 2d380472d..d49332a49 100644
--- a/pkg/queue/cluster_queue_best_effort_fifo.go
+++ b/pkg/queue/cluster_queue_best_effort_fifo.go
@@ -69,6 +69,16 @@ func (cq *ClusterQueueBestEffortFIFO) Delete(w *kueue.Workload) {
 	cq.ClusterQueueImpl.Delete(w)
 }
 
+func (cq *ClusterQueueBestEffortFIFO) DeleteFromQueue(q *Queue) {
+	for _, w := range q.items {
+		key := workload.Key(w.Obj)
+		if wl := cq.inadmissibleWorkloads[key]; wl != nil {
+			delete(cq.inadmissibleWorkloads, key)
+		}
+	}
+	cq.ClusterQueueImpl.DeleteFromQueue(q)
+}
+
 // RequeueIfNotPresent inserts a workload that cannot be admitted into
 // ClusterQueue, unless it is already in the queue. If immediate is true,
 // the workload will be pushed back to heap directly. If not,
diff --git a/pkg/queue/manager.go b/pkg/queue/manager.go
index 0c45cbd70..f0c07db88 100644
--- a/pkg/queue/manager.go
+++ b/pkg/queue/manager.go
@@ -77,6 +77,7 @@ func (m *Manager) AddClusterQueue(ctx context.Context, cq *kueue.ClusterQueue) e
 	if err != nil {
 		return err
 	}
+
 	m.clusterQueues[cq.Name] = cqImpl
 
 	cohort := cq.Spec.Cohort
@@ -105,7 +106,7 @@ func (m *Manager) AddClusterQueue(ctx context.Context, cq *kueue.ClusterQueue) e
 
 	queued := m.queueAllInadmissibleWorkloadsInCohort(cqImpl)
 	if queued || addedWorkloads {
-		m.Broadcast()
+		m.cond.Broadcast()
 	}
 	return nil
 }
@@ -117,8 +118,8 @@ func (m *Manager) UpdateClusterQueue(cq *kueue.ClusterQueue) error {
 	if !ok {
 		return errClusterQueueDoesNotExist
 	}
-
 	oldCohort := cqImpl.Cohort()
+
 	// TODO(#8): recreate heap based on a change of queueing policy.
 	cqImpl.Update(cq)
 	newCohort := cqImpl.Cohort()
@@ -128,7 +129,7 @@ func (m *Manager) UpdateClusterQueue(cq *kueue.ClusterQueue) error {
 
 	// TODO(#8): Selectively move workloads based on the exact event.
 	if m.queueAllInadmissibleWorkloadsInCohort(cqImpl) {
-		m.Broadcast()
+		m.cond.Broadcast()
 	}
 
 	return nil
@@ -144,7 +145,9 @@ func (m *Manager) DeleteClusterQueue(cq *kueue.ClusterQueue) {
 	delete(m.clusterQueues, cq.Name)
 
 	cohort := cq.Spec.Cohort
-	m.deleteCohort(cohort, cq.Name)
+	if cohort != "" {
+		m.deleteCohort(cohort, cq.Name)
+	}
 }
 
 func (m *Manager) AddQueue(ctx context.Context, q *kueue.Queue) error {
@@ -173,7 +176,7 @@ func (m *Manager) AddQueue(ctx context.Context, q *kueue.Queue) error {
 	}
 	cq := m.clusterQueues[qImpl.ClusterQueue]
 	if cq != nil && cq.AddFromQueue(qImpl) {
-		m.Broadcast()
+		m.cond.Broadcast()
 	}
 	qImpl.reportPendingWorkloads()
 	return nil
@@ -194,7 +197,7 @@ func (m *Manager) UpdateQueue(q *kueue.Queue) error {
 		}
 		newCQ := m.clusterQueues[string(q.Spec.ClusterQueue)]
 		if newCQ != nil && newCQ.AddFromQueue(qImpl) {
-			m.Broadcast()
+			m.cond.Broadcast()
 		}
 	}
 	qImpl.update(q)
@@ -279,7 +282,7 @@ func (m *Manager) addOrUpdateWorkload(w *kueue.Workload) bool {
 		return false
 	}
 	cq.PushOrUpdate(w)
-	m.Broadcast()
+	m.cond.Broadcast()
 	return true
 }
 
@@ -313,7 +316,7 @@ func (m *Manager) RequeueWorkload(ctx context.Context, info *workload.Info, imme
 	info = workload.NewInfo(&w)
 	added := cq.RequeueIfNotPresent(info, immediate)
 	if added {
-		m.Broadcast()
+		m.cond.Broadcast()
 	}
 	return added
 }
@@ -355,33 +358,7 @@ func (m *Manager) QueueAssociatedInadmissibleWorkloads(w *kueue.Workload) {
 	}
 
 	if m.queueAllInadmissibleWorkloadsInCohort(cq) {
-		m.Broadcast()
-	}
-}
-
-// QueueInadmissibleWorkloads moves all inadmissibleWorkloads in
-// corresponding ClusterQueues to heap. If at least one workload queued,
-// we will broadcast the event.
-func (m *Manager) QueueInadmissibleWorkloads(cqNames sets.String) {
-	m.Lock()
-	defer m.Unlock()
-	if len(cqNames) == 0 {
-		return
-	}
-
-	var queued bool
-	for name := range cqNames {
-		cq, exists := m.clusterQueues[name]
-		if !exists {
-			continue
-		}
-		if m.queueAllInadmissibleWorkloadsInCohort(cq) {
-			queued = true
-		}
-	}
-
-	if queued {
-		m.Broadcast()
+		m.cond.Broadcast()
 	}
 }
 
@@ -426,7 +403,7 @@ func (m *Manager) UpdateWorkload(oldW, w *kueue.Workload) bool {
 // Heads.
 func (m *Manager) CleanUpOnContext(ctx context.Context) {
 	<-ctx.Done()
-	m.Broadcast()
+	m.cond.Broadcast()
 }
 
 // Heads returns the heads of the queues, along with their associated ClusterQueue.
@@ -495,9 +472,6 @@ func (m *Manager) addCohort(cohort string, cqName string) {
 }
 
 func (m *Manager) deleteCohort(cohort string, cqName string) {
-	if cohort == "" {
-		return
-	}
 	if m.cohorts[cohort] != nil {
 		m.cohorts[cohort].Delete(cqName)
 		if len(m.cohorts[cohort]) == 0 {
@@ -511,10 +485,6 @@ func (m *Manager) updateCohort(oldCohort string, newCohort string, cqName string
 	m.addCohort(newCohort, cqName)
 }
 
-func (m *Manager) Broadcast() {
-	m.cond.Broadcast()
-}
-
 func SetupIndexes(indexer client.FieldIndexer) error {
 	err := indexer.IndexField(context.Background(), &kueue.Workload{}, workloadQueueKey, func(o client.Object) []string {
 		wl := o.(*kueue.Workload)
diff --git a/pkg/scheduler/scheduler.go b/pkg/scheduler/scheduler.go
index fab6b3d43..96933c2c4 100644
--- a/pkg/scheduler/scheduler.go
+++ b/pkg/scheduler/scheduler.go
@@ -181,10 +181,8 @@ func (s *Scheduler) nominate(ctx context.Context, workloads []workload.Info, sna
 		cq := snap.ClusterQueues[w.ClusterQueue]
 		ns := corev1.Namespace{}
 		e := entry{Info: w}
-		if snap.InactiveClusterQueueSets.Has(w.ClusterQueue) {
-			e.inadmissibleReason = fmt.Sprintf("ClusterQueue %s is inactive", w.ClusterQueue)
-		} else if cq == nil {
-			e.inadmissibleReason = fmt.Sprintf("ClusterQueue %s not found", w.ClusterQueue)
+		if cq == nil {
+			e.inadmissibleReason = "ClusterQueue not found"
 		} else if err := s.client.Get(ctx, types.NamespacedName{Name: w.Obj.Namespace}, &ns); err != nil {
 			e.inadmissibleReason = fmt.Sprintf("Could not obtain workload namespace: %v", err)
 		} else if !cq.NamespaceSelector.Matches(labels.Set(ns.Labels)) {
@@ -490,7 +488,7 @@ func (e entryOrdering) Less(i, j int) bool {
 
 func (s *Scheduler) requeueAndUpdate(log logr.Logger, ctx context.Context, e entry) {
 	added := s.queues.RequeueWorkload(ctx, &e.Info, e.status != "")
-	log.V(2).Info("Workload re-queued", "workload", klog.KObj(e.Obj), "clusterQueue", e.ClusterQueue, "queue", klog.KRef(e.Obj.Namespace, e.Obj.Spec.QueueName), "added", added, "status", e.status)
+	log.V(2).Info("Workload re-queued", "workload", klog.KObj(e.Obj), "queue", klog.KRef(e.Obj.Namespace, e.Obj.Spec.QueueName), "added", added, "status", e.status)
 
 	if e.status == "" {
 		err := workload.UpdateStatus(ctx, s.client, e.Obj, kueue.WorkloadAdmitted, corev1.ConditionFalse, "Pending", e.inadmissibleReason)


Test patch:
diff --git a/pkg/cache/cache_test.go b/pkg/cache/cache_test.go
index 6b906bb03..470e77d46 100644
--- a/pkg/cache/cache_test.go
+++ b/pkg/cache/cache_test.go
@@ -81,23 +81,6 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 		{
 			ObjectMeta: metav1.ObjectMeta{Name: "d"},
 		},
-		{
-			ObjectMeta: metav1.ObjectMeta{Name: "e"},
-			Spec: kueue.ClusterQueueSpec{
-				Resources: []kueue.Resource{
-					{
-						Name: corev1.ResourceCPU,
-						Flavors: []kueue.Flavor{{
-							Name: "nonexistent-flavor",
-							Quota: kueue.Quota{
-								Min: resource.MustParse("15"),
-							},
-						}},
-					},
-				},
-				Cohort: "two",
-			},
-		},
 	}
 	setup := func(cache *Cache) {
 		cache.AddOrUpdateResourceFlavor(&kueue.ResourceFlavor{
@@ -134,7 +117,6 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 					NamespaceSelector: labels.Nothing(),
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
-					Status:            Active,
 				},
 				"b": {
 					Name: "b",
@@ -144,36 +126,23 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 					NamespaceSelector: labels.Nothing(),
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
-					Status:            Active,
 				},
 				"c": {
 					Name:                 "c",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
 					NamespaceSelector:    labels.Nothing(),
 					UsedResources:        Resources{},
-					Status:               Active,
 				},
 				"d": {
 					Name:                 "d",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
 					NamespaceSelector:    labels.Nothing(),
 					UsedResources:        Resources{},
-					Status:               Active,
-				},
-				"e": {
-					Name: "e",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "nonexistent-flavor", Min: 15000}},
-					},
-					NamespaceSelector: labels.Nothing(),
-					UsedResources:     Resources{corev1.ResourceCPU: {"nonexistent-flavor": 0}},
-					LabelKeys:         nil,
-					Status:            Pending,
 				},
 			},
 			wantCohorts: map[string]sets.String{
 				"one": sets.NewString("a", "b"),
-				"two": sets.NewString("c", "e"),
+				"two": sets.NewString("c"),
 			},
 		},
 		{
@@ -198,7 +167,6 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 					NamespaceSelector: labels.Nothing(),
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
-					Status:            Active,
 				},
 				"b": {
 					Name: "b",
@@ -208,36 +176,23 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 					NamespaceSelector: labels.Nothing(),
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
-					Status:            Active,
 				},
 				"c": {
 					Name:                 "c",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
 					NamespaceSelector:    labels.Nothing(),
 					UsedResources:        Resources{},
-					Status:               Active,
 				},
 				"d": {
 					Name:                 "d",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
 					NamespaceSelector:    labels.Nothing(),
 					UsedResources:        Resources{},
-					Status:               Active,
-				},
-				"e": {
-					Name: "e",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "nonexistent-flavor", Min: 15000}},
-					},
-					NamespaceSelector: labels.Nothing(),
-					UsedResources:     Resources{corev1.ResourceCPU: {"nonexistent-flavor": 0}},
-					LabelKeys:         nil,
-					Status:            Pending,
 				},
 			},
 			wantCohorts: map[string]sets.String{
 				"one": sets.NewString("a", "b"),
-				"two": sets.NewString("c", "e"),
+				"two": sets.NewString("c"),
 			},
 		},
 		{
@@ -272,25 +227,6 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 							NamespaceSelector: &metav1.LabelSelector{}, // everything
 						},
 					},
-					{
-						ObjectMeta: metav1.ObjectMeta{Name: "e"},
-						Spec: kueue.ClusterQueueSpec{
-							Resources: []kueue.Resource{
-								{
-									Name: corev1.ResourceCPU,
-									Flavors: []kueue.Flavor{
-										{
-											Name: "default",
-											Quota: kueue.Quota{
-												Min: resource.MustParse("5"),
-												Max: pointer.Quantity(resource.MustParse("10")),
-											},
-										},
-									},
-								}},
-							Cohort: "two",
-						},
-					},
 				}
 				for _, c := range clusterQueues {
 					if err := cache.UpdateClusterQueue(&c); err != nil {
@@ -311,43 +247,29 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 					NamespaceSelector: labels.Nothing(),
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType", "region")},
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
-					Status:            Active,
 				},
 				"b": {
 					Name:                 "b",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
 					NamespaceSelector:    labels.Everything(),
 					UsedResources:        Resources{},
-					Status:               Active,
 				},
 				"c": {
 					Name:                 "c",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
 					NamespaceSelector:    labels.Nothing(),
 					UsedResources:        Resources{},
-					Status:               Active,
 				},
 				"d": {
 					Name:                 "d",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
 					NamespaceSelector:    labels.Nothing(),
 					UsedResources:        Resources{},
-					Status:               Active,
-				},
-				"e": {
-					Name: "e",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Min: 5000, Max: pointer.Int64(10000)}},
-					},
-					NamespaceSelector: labels.Nothing(),
-					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
-					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType", "region")},
-					Status:            Active,
 				},
 			},
 			wantCohorts: map[string]sets.String{
 				"one": sets.NewString("b"),
-				"two": sets.NewString("a", "c", "e"),
+				"two": sets.NewString("a", "c"),
 			},
 		},
 		{
@@ -371,88 +293,17 @@ func TestCacheClusterQueueOperations(t *testing.T) {
 					NamespaceSelector: labels.Nothing(),
 					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
 					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
-					Status:            Active,
 				},
 				"c": {
 					Name:                 "c",
 					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
 					NamespaceSelector:    labels.Nothing(),
 					UsedResources:        Resources{},
-					Status:               Active,
-				},
-				"e": {
-					Name: "e",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "nonexistent-flavor", Min: 15000}},
-					},
-					NamespaceSelector: labels.Nothing(),
-					UsedResources:     Resources{corev1.ResourceCPU: {"nonexistent-flavor": 0}},
-					LabelKeys:         nil,
-					Status:            Pending,
 				},
 			},
 			wantCohorts: map[string]sets.String{
 				"one": sets.NewString("b"),
-				"two": sets.NewString("c", "e"),
-			},
-		},
-		{
-			name: "add resource flavors",
-			operation: func(cache *Cache) {
-				setup(cache)
-				cache.AddOrUpdateResourceFlavor(&kueue.ResourceFlavor{
-					ObjectMeta: metav1.ObjectMeta{Name: "nonexistent-flavor"},
-				})
-			},
-			wantClusterQueues: map[string]*ClusterQueue{
-				"a": {
-					Name: "a",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Min: 10000, Max: pointer.Int64(20000)}},
-					},
-					NamespaceSelector: labels.Nothing(),
-					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
-					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
-					Status:            Active,
-				},
-				"b": {
-					Name: "b",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "default", Min: 15000}},
-					},
-					NamespaceSelector: labels.Nothing(),
-					UsedResources:     Resources{corev1.ResourceCPU: {"default": 0}},
-					LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
-					Status:            Active,
-				},
-				"c": {
-					Name:                 "c",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
-					NamespaceSelector:    labels.Nothing(),
-					UsedResources:        Resources{},
-					Status:               Active,
-				},
-				"d": {
-					Name:                 "d",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{},
-					NamespaceSelector:    labels.Nothing(),
-					UsedResources:        Resources{},
-					Status:               Active,
-				},
-				"e": {
-					Name: "e",
-					RequestableResources: map[corev1.ResourceName][]FlavorLimits{
-						corev1.ResourceCPU: {{Name: "nonexistent-flavor", Min: 15000}},
-					},
-					NamespaceSelector: labels.Nothing(),
-					UsedResources:     Resources{corev1.ResourceCPU: {"nonexistent-flavor": 0}},
-					LabelKeys:         nil,
-					Status:            Active,
-				},
-			},
-			wantCohorts: map[string]sets.String{
-				"one": sets.NewString("a", "b"),
-				"two": sets.NewString("c", "e"),
+				"two": sets.NewString("c"),
 			},
 		},
 	}
diff --git a/pkg/cache/snapshot_test.go b/pkg/cache/snapshot_test.go
index 0e1234d1c..a590ff764 100644
--- a/pkg/cache/snapshot_test.go
+++ b/pkg/cache/snapshot_test.go
@@ -101,27 +101,6 @@ func TestSnapshot(t *testing.T) {
 				},
 			},
 		},
-		{
-			ObjectMeta: metav1.ObjectMeta{
-				Name: "flavor-nonexistent-cq",
-			},
-			Spec: kueue.ClusterQueueSpec{
-				Cohort: "foo",
-				Resources: []kueue.Resource{
-					{
-						Name: corev1.ResourceCPU,
-						Flavors: []kueue.Flavor{
-							{
-								Name: "nonexistent-flavor",
-								Quota: kueue.Quota{
-									Min: resource.MustParse("100"),
-								},
-							},
-						},
-					},
-				},
-			},
-		},
 		{
 			ObjectMeta: metav1.ObjectMeta{
 				Name: "bar",
@@ -159,12 +138,7 @@ func TestSnapshot(t *testing.T) {
 			ObjectMeta: metav1.ObjectMeta{Name: "spot"},
 			Labels:     map[string]string{"baz": "bar", "instance": "spot"},
 		},
-		{
-			ObjectMeta: metav1.ObjectMeta{Name: "default"},
-			Labels:     nil,
-		},
 	}
-
 	for i := range flavors {
 		cache.AddOrUpdateResourceFlavor(&flavors[i])
 	}
@@ -248,44 +222,31 @@ func TestSnapshot(t *testing.T) {
 				},
 			},
 		},
-		{
-			ObjectMeta: metav1.ObjectMeta{Name: "sigma"},
-			Spec: kueue.WorkloadSpec{
-				PodSets: []kueue.PodSet{
-					{
-						Name:  "main",
-						Count: 5,
-						Spec: utiltesting.PodSpecForRequest(map[corev1.ResourceName]string{
-							corev1.ResourceCPU: "1",
-						}),
-					},
-				},
-				Admission: nil,
-			},
-		},
 	}
 	for _, w := range workloads {
 		cache.AddOrUpdateWorkload(w.DeepCopy())
 	}
 	snapshot := cache.Snapshot()
-	wantCohort := Cohort{
-		Name: "foo",
-		RequestableResources: Resources{
-			corev1.ResourceCPU: map[string]int64{
-				"demand": 100_000,
-				"spot":   300_000,
-			},
-			"example.com/gpu": map[string]int64{
-				"default": 50,
-			},
-		},
-		UsedResources: Resources{
-			corev1.ResourceCPU: map[string]int64{
-				"demand": 10_000,
-				"spot":   10_000,
+	wantCohorts := []Cohort{
+		{
+			Name: "foo",
+			RequestableResources: Resources{
+				corev1.ResourceCPU: map[string]int64{
+					"demand": 100_000,
+					"spot":   300_000,
+				},
+				"example.com/gpu": map[string]int64{
+					"default": 50,
+				},
 			},
-			"example.com/gpu": map[string]int64{
-				"default": 15,
+			UsedResources: Resources{
+				corev1.ResourceCPU: map[string]int64{
+					"demand": 10_000,
+					"spot":   10_000,
+				},
+				"example.com/gpu": map[string]int64{
+					"default": 15,
+				},
 			},
 		},
 	}
@@ -293,7 +254,7 @@ func TestSnapshot(t *testing.T) {
 		ClusterQueues: map[string]*ClusterQueue{
 			"foofoo": {
 				Name:   "foofoo",
-				Cohort: &wantCohort,
+				Cohort: &wantCohorts[0],
 				RequestableResources: map[corev1.ResourceName][]FlavorLimits{
 					corev1.ResourceCPU: {
 						{
@@ -317,11 +278,10 @@ func TestSnapshot(t *testing.T) {
 				},
 				LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: {"baz": {}, "foo": {}, "instance": {}}},
 				NamespaceSelector: labels.Nothing(),
-				Status:            Active,
 			},
 			"foobar": {
 				Name:   "foobar",
-				Cohort: &wantCohort,
+				Cohort: &wantCohorts[0],
 				RequestableResources: map[corev1.ResourceName][]FlavorLimits{
 					corev1.ResourceCPU: {
 						{
@@ -350,7 +310,6 @@ func TestSnapshot(t *testing.T) {
 				},
 				NamespaceSelector: labels.Nothing(),
 				LabelKeys:         map[corev1.ResourceName]sets.String{corev1.ResourceCPU: {"baz": {}, "instance": {}}},
-				Status:            Active,
 			},
 			"bar": {
 				Name: "bar",
@@ -367,14 +326,9 @@ func TestSnapshot(t *testing.T) {
 				},
 				Workloads:         map[string]*workload.Info{},
 				NamespaceSelector: labels.Nothing(),
-				Status:            Active,
 			},
 		},
 		ResourceFlavors: map[string]*kueue.ResourceFlavor{
-			"default": {
-				ObjectMeta: metav1.ObjectMeta{Name: "default"},
-				Labels:     nil,
-			},
 			"demand": {
 				ObjectMeta: metav1.ObjectMeta{Name: "demand"},
 				Labels:     map[string]string{"foo": "bar", "instance": "demand"},
@@ -384,7 +338,6 @@ func TestSnapshot(t *testing.T) {
 				Labels:     map[string]string{"baz": "bar", "instance": "spot"},
 			},
 		},
-		InactiveClusterQueueSets: sets.String{"flavor-nonexistent-cq": {}},
 	}
 	if diff := cmp.Diff(wantSnapshot, snapshot, cmpopts.IgnoreUnexported(Cohort{})); diff != "" {
 		t.Errorf("Unexpected Snapshot (-want,+got):\n%s", diff)
diff --git a/pkg/queue/cluster_queue_best_effort_fifo_test.go b/pkg/queue/cluster_queue_best_effort_fifo_test.go
index 4ef842fda..8e5b45cec 100644
--- a/pkg/queue/cluster_queue_best_effort_fifo_test.go
+++ b/pkg/queue/cluster_queue_best_effort_fifo_test.go
@@ -135,3 +135,43 @@ func TestClusterQueueBestEffortFIFO(t *testing.T) {
 		})
 	}
 }
+
+func TestDeleteFromQueue(t *testing.T) {
+	cq := utiltesting.MakeClusterQueue("cq").Obj()
+	cqImpl, err := newClusterQueueBestEffortFIFO(cq)
+	if err != nil {
+		t.Fatalf("Failed creating ClusterQueue %v", err)
+	}
+	q := utiltesting.MakeQueue("foo", "").ClusterQueue(cq.Name).Obj()
+	qImpl := newQueue(q)
+	wl1 := utiltesting.MakeWorkload("wl1", "").Queue(q.Name).Obj()
+	wl2 := utiltesting.MakeWorkload("wl2", "").Queue(q.Name).Obj()
+	wl3 := utiltesting.MakeWorkload("wl3", "").Queue(q.Name).Obj()
+	wl4 := utiltesting.MakeWorkload("wl4", "").Queue(q.Name).Obj()
+	admissibleworkloads := []*kueue.Workload{wl1, wl2}
+	inadmissibleWorkloads := []*kueue.Workload{wl3, wl4}
+
+	for _, w := range admissibleworkloads {
+		cqImpl.PushOrUpdate(w)
+		qImpl.AddOrUpdate(w)
+	}
+
+	for _, w := range inadmissibleWorkloads {
+		cqImpl.RequeueIfNotPresent(workload.NewInfo(w), false)
+		qImpl.AddOrUpdate(w)
+	}
+
+	wantPending := len(admissibleworkloads) + len(inadmissibleWorkloads)
+	if pending := cqImpl.Pending(); pending != int32(wantPending) {
+		t.Errorf("clusterQueue's workload number not right, want %v, got %v", wantPending, pending)
+	}
+	fifo := cqImpl.(*ClusterQueueBestEffortFIFO)
+	if len(fifo.inadmissibleWorkloads) != len(inadmissibleWorkloads) {
+		t.Errorf("clusterQueue's workload number in inadmissibleWorkloads not right, want %v, got %v", len(inadmissibleWorkloads), len(fifo.inadmissibleWorkloads))
+	}
+
+	cqImpl.DeleteFromQueue(qImpl)
+	if cqImpl.Pending() != 0 {
+		t.Error("clusterQueue should be empty")
+	}
+}
diff --git a/pkg/scheduler/scheduler_test.go b/pkg/scheduler/scheduler_test.go
index 886153cf5..0d7773eb0 100644
--- a/pkg/scheduler/scheduler_test.go
+++ b/pkg/scheduler/scheduler_test.go
@@ -175,25 +175,6 @@ func TestSchedule(t *testing.T) {
 				},
 			},
 		},
-		{
-			ObjectMeta: metav1.ObjectMeta{Name: "flavor-nonexistent-cq"},
-			Spec: kueue.ClusterQueueSpec{
-				QueueingStrategy: kueue.StrictFIFO,
-				Resources: []kueue.Resource{
-					{
-						Name: corev1.ResourceCPU,
-						Flavors: []kueue.Flavor{
-							{
-								Name: "nonexistent-flavor",
-								Quota: kueue.Quota{
-									Min: resource.MustParse("50"),
-								},
-							},
-						},
-					},
-				},
-			},
-		},
 	}
 	queues := []kueue.Queue{
 		{
@@ -232,24 +213,6 @@ func TestSchedule(t *testing.T) {
 				ClusterQueue: "eng-beta",
 			},
 		},
-		{
-			ObjectMeta: metav1.ObjectMeta{
-				Namespace: "sales",
-				Name:      "flavor-nonexistent-queue",
-			},
-			Spec: kueue.QueueSpec{
-				ClusterQueue: "flavor-nonexistent-cq",
-			},
-		},
-		{
-			ObjectMeta: metav1.ObjectMeta{
-				Namespace: "sales",
-				Name:      "cq-nonexistent-queue",
-			},
-			Spec: kueue.QueueSpec{
-				ClusterQueue: "nonexistent-cq",
-			},
-		},
 	}
 	cases := map[string]struct {
 		workloads []kueue.Workload
@@ -725,53 +688,6 @@ func TestSchedule(t *testing.T) {
 			},
 			wantScheduled: []string{"eng-alpha/new"},
 		},
-		"workload should not fit in nonexistent clusterQueue": {
-			workloads: []kueue.Workload{
-				{
-					ObjectMeta: metav1.ObjectMeta{
-						Namespace: "sales",
-						Name:      "foo",
-					},
-					Spec: kueue.WorkloadSpec{
-						QueueName: "cq-nonexistent-queue",
-						PodSets: []kueue.PodSet{
-							{
-								Name:  "one",
-								Count: 10,
-								Spec: utiltesting.PodSpecForRequest(map[corev1.ResourceName]string{
-									corev1.ResourceCPU: "1",
-								}),
-							},
-						},
-					},
-				},
-			},
-		},
-		"workload should not fit in flavor nonexistent clusterQueue": {
-			workloads: []kueue.Workload{
-				{
-					ObjectMeta: metav1.ObjectMeta{
-						Namespace: "sales",
-						Name:      "foo",
-					},
-					Spec: kueue.WorkloadSpec{
-						QueueName: "flavor-nonexistent-queue",
-						PodSets: []kueue.PodSet{
-							{
-								Name:  "one",
-								Count: 10,
-								Spec: utiltesting.PodSpecForRequest(map[corev1.ResourceName]string{
-									corev1.ResourceCPU: "1",
-								}),
-							},
-						},
-					},
-				},
-			},
-			wantLeft: map[string]sets.String{
-				"flavor-nonexistent-cq": sets.NewString("foo"),
-			},
-		},
 	}
 	for name, tc := range cases {
 		t.Run(name, func(t *testing.T) {
@@ -805,9 +721,6 @@ func TestSchedule(t *testing.T) {
 					t.Fatalf("Inserting queue %s/%s in manager: %v", q.Namespace, q.Name, err)
 				}
 			}
-			for i := range resourceFlavors {
-				cqCache.AddOrUpdateResourceFlavor(resourceFlavors[i])
-			}
 			for _, cq := range clusterQueues {
 				if err := cqCache.AddClusterQueue(ctx, &cq); err != nil {
 					t.Fatalf("Inserting clusterQueue %s in cache: %v", cq.Name, err)
@@ -816,6 +729,9 @@ func TestSchedule(t *testing.T) {
 					t.Fatalf("Inserting clusterQueue %s in manager: %v", cq.Name, err)
 				}
 			}
+			for i := range resourceFlavors {
+				cqCache.AddOrUpdateResourceFlavor(resourceFlavors[i])
+			}
 			workloadWatch, err := cl.Watch(ctx, &kueue.WorkloadList{})
 			if err != nil {
 				t.Fatalf("Failed setting up watch: %v", err)
@@ -999,7 +915,6 @@ func TestEntryAssignFlavors(t *testing.T) {
 					},
 				},
 			},
-			wantMsg: "insufficient quota for flavor default, 1000 more needed",
 		},
 		"multiple flavors, fits": {
 			wlPods: []kueue.PodSet{
@@ -1055,7 +970,6 @@ func TestEntryAssignFlavors(t *testing.T) {
 					},
 				},
 			},
-			wantMsg: "insufficient quota for flavor one, 2100 more needed, insufficient quota for flavor two, 100 more needed",
 		},
 		"multiple flavors, fits while skipping tainted flavor": {
 			wlPods: []kueue.PodSet{
@@ -1312,7 +1226,7 @@ func TestEntryAssignFlavors(t *testing.T) {
 				LabelKeys: map[corev1.ResourceName]sets.String{corev1.ResourceCPU: sets.NewString("cpuType")},
 			},
 			wantFits: false,
-			wantMsg:  "flavor one doesn't match with node affinity",
+			wantMsg:  "doesn't match with node affinity",
 		},
 		"multiple specs, fit different flavors": {
 			wlPods: []kueue.PodSet{
@@ -1445,7 +1359,6 @@ func TestEntryAssignFlavors(t *testing.T) {
 					},
 				},
 			},
-			wantMsg: "insufficient quota for flavor one, 1000 more needed after borrowing",
 		},
 		"past max": {
 			wlPods: []kueue.PodSet{
@@ -1479,7 +1392,6 @@ func TestEntryAssignFlavors(t *testing.T) {
 					},
 				},
 			},
-			wantMsg: "borrowing limit for flavor one exceeded",
 		},
 		"resource not listed in clusterQueue": {
 			wlPods: []kueue.PodSet{
@@ -1501,23 +1413,6 @@ func TestEntryAssignFlavors(t *testing.T) {
 			wantFits: false,
 			wantMsg:  "resource unavailable in ClusterQueue",
 		},
-		"flavor not found": {
-			wlPods: []kueue.PodSet{
-				{
-					Count: 1,
-					Name:  "main",
-					Spec: utiltesting.PodSpecForRequest(map[corev1.ResourceName]string{
-						corev1.ResourceCPU: "1",
-					}),
-				},
-			},
-			clusterQueue: cache.ClusterQueue{
-				RequestableResources: map[corev1.ResourceName][]cache.FlavorLimits{
-					corev1.ResourceCPU: {{Name: "nonexistent-flavor", Min: 1000}},
-				},
-			},
-			wantMsg: "flavor nonexistent-flavor not found",
-		},
 	}
 	for name, tc := range cases {
 		t.Run(name, func(t *testing.T) {
@@ -1531,13 +1426,13 @@ func TestEntryAssignFlavors(t *testing.T) {
 					},
 				}),
 			}
-			tc.clusterQueue.UpdateWithFlavors(resourceFlavors)
+			tc.clusterQueue.UpdateLabelKeys(resourceFlavors)
 			status := e.assignFlavors(log, resourceFlavors, &tc.clusterQueue)
 			if status.IsSuccess() != tc.wantFits {
 				t.Errorf("e.assignFlavors(_)=%t, want %t", status.IsSuccess(), tc.wantFits)
 			}
 			if !tc.wantFits {
-				if len(tc.wantMsg) == 0 || !strings.Contains(status.Message(), tc.wantMsg) {
+				if !strings.Contains(status.Message(), tc.wantMsg) {
 					t.Errorf("got msg %s, want msg containing %s", status.Message(), tc.wantMsg)
 				}
 			}
diff --git a/test/integration/controller/core/workload_controller_test.go b/test/integration/controller/core/workload_controller_test.go
index 3b65bc5ff..947ff9552 100644
--- a/test/integration/controller/core/workload_controller_test.go
+++ b/test/integration/controller/core/workload_controller_test.go
@@ -112,8 +112,6 @@ var _ = ginkgo.Describe("Workload controller", func() {
 
 	ginkgo.When("the workload is admitted", func() {
 		ginkgo.BeforeEach(func() {
-			flavor := testing.MakeResourceFlavor(flavorOnDemand).Obj()
-			gomega.Expect(k8sClient.Create(ctx, flavor)).Should(gomega.Succeed())
 			clusterQueue = testing.MakeClusterQueue("cluster-queue").
 				Resource(testing.MakeResource(resourceGPU).
 					Flavor(testing.MakeFlavor(flavorOnDemand, "5").Max("10").Obj()).Obj()).
diff --git a/test/integration/framework/framework.go b/test/integration/framework/framework.go
index 36c95ee46..b6c89a072 100644
--- a/test/integration/framework/framework.go
+++ b/test/integration/framework/framework.go
@@ -25,7 +25,6 @@ import (
 
 	"github.com/onsi/ginkgo/v2"
 	"github.com/onsi/gomega"
-	zaplog "go.uber.org/zap"
 	"go.uber.org/zap/zapcore"
 	batchv1 "k8s.io/api/batch/v1"
 	corev1 "k8s.io/api/core/v1"
@@ -59,16 +58,10 @@ type Framework struct {
 }
 
 func (f *Framework) Setup() (context.Context, *rest.Config, client.Client) {
-	opts := zap.Options{
-		TimeEncoder: zapcore.RFC3339NanoTimeEncoder,
-		ZapOpts:     []zaplog.Option{zaplog.AddCaller(), zaplog.AddCallerSkip(-1)},
+	logTimeEncodeOpt := func(o *zap.Options) {
+		o.TimeEncoder = zapcore.RFC3339NanoTimeEncoder
 	}
-	ctrl.SetLogger(zap.New(
-		zap.WriteTo(ginkgo.GinkgoWriter),
-		zap.UseDevMode(true),
-		zap.Level(zapcore.Level(-3)),
-		zap.UseFlagOptions(&opts)),
-	)
+	ctrl.SetLogger(zap.New(zap.WriteTo(ginkgo.GinkgoWriter), zap.UseDevMode(true), zap.Level(zapcore.Level(-3)), logTimeEncodeOpt))
 
 	ginkgo.By("bootstrapping test environment")
 	f.testEnv = &envtest.Environment{
@@ -228,26 +221,6 @@ func ExpectWorkloadsToBePending(ctx context.Context, k8sClient client.Client, wl
 	}, Timeout, Interval).Should(gomega.Equal(len(wls)), "Not enough workloads are pending")
 }
 
-func ExpectWorkloadsToBeFrozen(ctx context.Context, k8sClient client.Client, cq string, wls ...*kueue.Workload) {
-	gomega.EventuallyWithOffset(1, func() int {
-		frozen := 0
-		var updatedWorkload kueue.Workload
-		for _, wl := range wls {
-			gomega.ExpectWithOffset(1, k8sClient.Get(ctx, client.ObjectKeyFromObject(wl), &updatedWorkload)).To(gomega.Succeed())
-			idx := workload.FindConditionIndex(&updatedWorkload.Status, kueue.WorkloadAdmitted)
-			if idx == -1 {
-				continue
-			}
-			msg := fmt.Sprintf("ClusterQueue %s is inactive", cq)
-			cond := updatedWorkload.Status.Conditions[idx]
-			if cond.Status == corev1.ConditionFalse && cond.Reason == "Inadmissible" && wl.Spec.Admission == nil && cond.Message == msg {
-				frozen++
-			}
-		}
-		return frozen
-	}, Timeout, Interval).Should(gomega.Equal(len(wls)), "Not enough workloads are frozen")
-}
-
 func ExpectPendingWorkloadsMetric(q *kueue.Queue, v int) {
 	metric := metrics.PendingWorkloads.WithLabelValues(string(q.Spec.ClusterQueue), queue.Key(q))
 	gomega.EventuallyWithOffset(1, func() int {
diff --git a/test/integration/scheduler/scheduler_test.go b/test/integration/scheduler/scheduler_test.go
index e2eeb66d4..c4600dc30 100644
--- a/test/integration/scheduler/scheduler_test.go
+++ b/test/integration/scheduler/scheduler_test.go
@@ -627,7 +627,8 @@ var _ = ginkgo.Describe("Scheduler", func() {
 		strictFIFOClusterQ := testing.MakeClusterQueue("strict-fifo-cq").
 			QueueingStrategy(kueue.StrictFIFO).
 			Resource(testing.MakeResource(corev1.ResourceCPU).
-				Flavor(testing.MakeFlavor(onDemandFlavor.Name, "5").Max("5").Obj()).
+				Flavor(testing.MakeFlavor(onDemandFlavor.Name,
+					"5").Max("5").Obj()).
 				Obj()).
 			Obj()
 		gomega.Expect(k8sClient.Create(ctx, strictFIFOClusterQ)).Should(gomega.Succeed())
@@ -658,60 +659,4 @@ var _ = ginkgo.Describe("Scheduler", func() {
 			return wl3.Spec.Admission == nil
 		}, framework.ConsistentDuration, framework.Interval).Should(gomega.Equal(true))
 	})
-
-	ginkgo.Context("When clusterQueue references nonexistent flavors", func() {
-		var (
-			fooFlavor *kueue.ResourceFlavor
-			fooQ      *kueue.Queue
-			fooCq     *kueue.ClusterQueue
-		)
-
-		ginkgo.BeforeEach(func() {
-			fooCq = testing.MakeClusterQueue("foo-cq").
-				QueueingStrategy(kueue.BestEffortFIFO).
-				Resource(testing.MakeResource(corev1.ResourceCPU).
-					Flavor(testing.MakeFlavor("foo-flavor", "15").Obj()).
-					Obj()).
-				Obj()
-			gomega.Expect(k8sClient.Create(ctx, fooCq)).Should(gomega.Succeed())
-			fooQ = testing.MakeQueue("foo-queue", ns.Name).ClusterQueue(fooCq.Name).Obj()
-			gomega.Expect(k8sClient.Create(ctx, fooQ)).Should(gomega.Succeed())
-		})
-
-		ginkgo.AfterEach(func() {
-			gomega.Expect(framework.DeleteClusterQueue(ctx, k8sClient, fooCq)).To(gomega.Succeed())
-			gomega.Expect(framework.DeleteResourceFlavor(ctx, k8sClient, fooFlavor)).To(gomega.Succeed())
-		})
-
-		ginkgo.It("Should be inactive until the flavor is created", func() {
-			ginkgo.By("Creating one workload")
-			wl := testing.MakeWorkload("workload", ns.Name).Queue(fooQ.Name).Request(corev1.ResourceCPU, "1").Obj()
-			gomega.Expect(k8sClient.Create(ctx, wl)).Should(gomega.Succeed())
-			framework.ExpectWorkloadsToBeFrozen(ctx, k8sClient, fooCq.Name, wl)
-
-			ginkgo.By("Creating foo flavor")
-			fooFlavor = testing.MakeResourceFlavor("foo-flavor").Obj()
-			gomega.Expect(k8sClient.Create(ctx, fooFlavor)).Should(gomega.Succeed())
-			framework.ExpectWorkloadsToBeAdmitted(ctx, k8sClient, fooCq.Name, wl)
-		})
-
-		ginkgo.It("Should be inactive when corresponding flavors deleted", func() {
-			ginkgo.By("Creating foo flavors")
-			fooFlavor = testing.MakeResourceFlavor("foo-flavor").Obj()
-			gomega.Expect(k8sClient.Create(ctx, fooFlavor)).Should(gomega.Succeed())
-
-			ginkgo.By("Creating one workload")
-			wl1 := testing.MakeWorkload("workload", ns.Name).Queue(fooQ.Name).Request(corev1.ResourceCPU, "1").Obj()
-			gomega.Expect(k8sClient.Create(ctx, wl1)).Should(gomega.Succeed())
-			framework.ExpectWorkloadsToBeAdmitted(ctx, k8sClient, fooCq.Name, wl1)
-
-			ginkgo.By("Deleting foo flavors")
-			gomega.Expect(framework.DeleteResourceFlavor(ctx, k8sClient, fooFlavor)).To(gomega.Succeed())
-
-			ginkgo.By("Creating another workloads")
-			wl2 := testing.MakeWorkload("wl-2", ns.Name).Queue(fooQ.Name).Request(corev1.ResourceCPU, "1").Obj()
-			gomega.Expect(k8sClient.Create(ctx, wl2)).Should(gomega.Succeed())
-			framework.ExpectWorkloadsToBeFrozen(ctx, k8sClient, fooCq.Name, wl2)
-		})
-	})
 })