Repository: operator-framework/operator-lifecycle-manager
Pull request #3028 (2023-09-07 16:21:13)

OCPBUGS-17157: *: detect when all objects are labelled, restart
When all of the k8s objects that need labels have them, we are good to exit the process. The next Pod that start up will detect that all labels are present and be able to filter informers going forward.


openshift-ci-robot: @stevekuznetsov: This pull request references [Jira Issue OCPBUGS-17157](https://issues.redhat.com//browse/OCPBUGS-17157), which is valid.

<details><summary>3 validation(s) were run on this bug</summary>

* bug is open, matching expected state (open)
* bug target version (4.14.0) matches configured target version for branch (4.14.0)
* bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)</details>

Requesting review from QA contact:
/cc @jianzhangbjz

The bug has been updated to refer to the pull request using the external bug tracker.

<details>

In response to [this](https://github.com/operator-framework/operator-lifecycle-manager/pull/3028):

>When all of the k8s objects that need labels have them, we are good to exit the process. The next Pod that start up will detect that all labels are present and be able to filter informers going forward.
>


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/operator-framework/operator-lifecycle-manager/pull/3028#" title="Author self-approved">stevekuznetsov</a>*
**Once this PR has been reviewed and has the lgtm label**, please assign [ecordell](https://github.com/ecordell) for approval. For more information see [the Kubernetes Code Review Process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process).

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=operator-framework%2Foperator-lifecycle-manager).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/operator-framework/operator-lifecycle-manager/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["ecordell"]} -->
openshift-merge-robot: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
tmshort: /retest
openshift-merge-robot: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/operator-framework/operator-lifecycle-manager/pull/3028#pullrequestreview-1637912313" title="Approved">perdasilva</a>*, *<a href="https://github.com/operator-framework/operator-lifecycle-manager/pull/3028#" title="Author self-approved">stevekuznetsov</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=operator-framework%2Foperator-lifecycle-manager).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/operator-framework/operator-lifecycle-manager/blob/master/OWNERS)~~ [perdasilva]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
stevekuznetsov: This should now only fail one e2e - `User defined service account [It] with retry`. That test does the following:

 - create an `OperatorGroup` with a user-provided service account
 - omit any permissions for that service account
 - create a `Subscription` and wait for the `InstallPlan` to enter a terminal failure state
 - create permissions
 - expect the `InstallPlan` to exit the terminal state

We either need to not wait for the terminal state or allow permissions errors to be re-tried forever. @joelanford please advise.
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/operator-framework/operator-lifecycle-manager/pull/3028#issuecomment-1739654327" title="Approved">joelanford</a>*, *<a href="https://github.com/operator-framework/operator-lifecycle-manager/pull/3028#pullrequestreview-1637912313" title="Approved">perdasilva</a>*, *<a href="https://github.com/operator-framework/operator-lifecycle-manager/pull/3028#" title="Author self-approved">stevekuznetsov</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=operator-framework%2Foperator-lifecycle-manager).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/operator-framework/operator-lifecycle-manager/blob/master/OWNERS)~~ [joelanford,perdasilva]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
openshift-ci[bot]: New changes are detected. LGTM label has been removed.
openshift-ci-robot: @stevekuznetsov: This pull request references [Jira Issue OCPBUGS-17157](https://issues.redhat.com//browse/OCPBUGS-17157), which is valid.

<details><summary>3 validation(s) were run on this bug</summary>

* bug is open, matching expected state (open)
* bug target version (4.15.0) matches configured target version for branch (4.15.0)
* bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)</details>

Requesting review from QA contact:
/cc @jianzhangbjz

<details>

In response to [this](https://github.com/operator-framework/operator-lifecycle-manager/pull/3028):

>When all of the k8s objects that need labels have them, we are good to exit the process. The next Pod that start up will detect that all labels are present and be able to filter informers going forward.
>


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>

Patch:
diff --git a/cmd/olm/main.go b/cmd/olm/main.go
index 447349678..6d76606dc 100644
--- a/cmd/olm/main.go
+++ b/cmd/olm/main.go
@@ -173,7 +173,7 @@ func main() {
 		olm.WithExternalClient(crClient),
 		olm.WithMetadataClient(metadataClient),
 		olm.WithOperatorClient(opClient),
-		olm.WithRestConfig(config),
+		olm.WithRestConfig(validatingConfig),
 		olm.WithConfigClient(versionedConfigClient),
 		olm.WithProtectedCopiedCSVNamespaces(*protectedCopiedCSVNamespaces),
 	)
diff --git a/cmd/olm/manager.go b/cmd/olm/manager.go
index fbca8e2d9..dbb6d0d22 100644
--- a/cmd/olm/manager.go
+++ b/cmd/olm/manager.go
@@ -3,11 +3,15 @@ package main
 import (
 	"context"
 
+	appsv1 "k8s.io/api/apps/v1"
 	corev1 "k8s.io/api/core/v1"
+	rbacv1 "k8s.io/api/rbac/v1"
+	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/apimachinery/pkg/selection"
+	apiregistrationv1 "k8s.io/kube-aggregator/pkg/apis/apiregistration/v1"
 	ctrl "sigs.k8s.io/controller-runtime"
 	"sigs.k8s.io/controller-runtime/pkg/cache"
 	"sigs.k8s.io/controller-runtime/pkg/client"
@@ -53,7 +57,34 @@ func Manager(ctx context.Context, debug bool) (ctrl.Manager, error) {
 		MetricsBindAddress: "0", // TODO(njhale): Enable metrics on non-conflicting port (not 8080)
 		Cache: cache.Options{
 			ByObject: map[client.Object]cache.ByObject{
-				&corev1.Secret{}: {
+				&appsv1.Deployment{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&corev1.Service{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&apiextensionsv1.CustomResourceDefinition{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&apiregistrationv1.APIService{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&corev1.ConfigMap{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&corev1.ServiceAccount{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&rbacv1.Role{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&rbacv1.RoleBinding{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&rbacv1.ClusterRole{}: {
+					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
+				},
+				&rbacv1.ClusterRoleBinding{}: {
 					Label: labels.SelectorFromValidatedSet(map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}),
 				},
 				&operatorsv1alpha1.ClusterServiceVersion{}: {
diff --git a/deploy/chart/templates/0000_50_olm_07-olm-operator.deployment.yaml b/deploy/chart/templates/0000_50_olm_07-olm-operator.deployment.yaml
index 825d39f0d..76f7ca1ee 100644
--- a/deploy/chart/templates/0000_50_olm_07-olm-operator.deployment.yaml
+++ b/deploy/chart/templates/0000_50_olm_07-olm-operator.deployment.yaml
@@ -110,6 +110,10 @@ spec:
                 fieldPath: metadata.namespace
           - name: OPERATOR_NAME
             value: olm-operator
+          {{- if .Values.debug }}
+          - name: CI
+            value: "true"
+          {{- end }}
           {{- if .Values.olm.resources }}
           resources:
 {{ toYaml .Values.olm.resources | indent 12 }}
diff --git a/deploy/chart/templates/0000_50_olm_08-catalog-operator.deployment.yaml b/deploy/chart/templates/0000_50_olm_08-catalog-operator.deployment.yaml
index 48e882d4e..2b2d4534d 100644
--- a/deploy/chart/templates/0000_50_olm_08-catalog-operator.deployment.yaml
+++ b/deploy/chart/templates/0000_50_olm_08-catalog-operator.deployment.yaml
@@ -90,6 +90,11 @@ spec:
           - --set-workload-user-id=false
           {{ end }}
           image: {{ .Values.catalog.image.ref }}
+          {{- if .Values.debug }}
+          env:
+            - name: CI
+              value: "true"
+          {{- end }}
           imagePullPolicy: {{ .Values.catalog.image.pullPolicy }}
           ports:
             - containerPort: {{ .Values.olm.service.internalPort }}
diff --git a/pkg/controller/bundle/bundle_unpacker.go b/pkg/controller/bundle/bundle_unpacker.go
index 79f67ab17..cfe46187b 100644
--- a/pkg/controller/bundle/bundle_unpacker.go
+++ b/pkg/controller/bundle/bundle_unpacker.go
@@ -4,6 +4,7 @@ import (
 	"context"
 	"crypto/sha256"
 	"fmt"
+	"sort"
 	"strings"
 	"time"
 
@@ -18,6 +19,7 @@ import (
 	"k8s.io/apimachinery/pkg/api/resource"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	k8slabels "k8s.io/apimachinery/pkg/labels"
+	"k8s.io/apiserver/pkg/storage/names"
 	"k8s.io/client-go/kubernetes"
 	listersbatchv1 "k8s.io/client-go/listers/batch/v1"
 	listerscorev1 "k8s.io/client-go/listers/core/v1"
@@ -41,6 +43,13 @@ const (
 	// e.g 1m30s
 	BundleUnpackTimeoutAnnotationKey = "operatorframework.io/bundle-unpack-timeout"
 	BundleUnpackPodLabel             = "job-name"
+
+	// BundleUnpackRetryMinimumIntervalAnnotationKey sets a minimum interval to wait before
+	// attempting to recreate a failed unpack job for a bundle.
+	BundleUnpackRetryMinimumIntervalAnnotationKey = "operatorframework.io/bundle-unpack-min-retry-interval"
+
+	// bundleUnpackRefLabel is used to filter for all unpack jobs for a specific bundle.
+	bundleUnpackRefLabel = "operatorframework.io/bundle-unpack-ref"
 )
 
 type BundleUnpackResult struct {
@@ -89,6 +98,7 @@ func (c *ConfigMapUnpacker) job(cmRef *corev1.ObjectReference, bundlePath string
 		ObjectMeta: metav1.ObjectMeta{
 			Labels: map[string]string{
 				install.OLMManagedLabelKey: install.OLMManagedLabelValue,
+				bundleUnpackRefLabel:       cmRef.Name,
 			},
 		},
 		Spec: batchv1.JobSpec{
@@ -287,7 +297,7 @@ func (c *ConfigMapUnpacker) job(cmRef *corev1.ObjectReference, bundlePath string
 //go:generate go run github.com/maxbrunsfeld/counterfeiter/v6 . Unpacker
 
 type Unpacker interface {
-	UnpackBundle(lookup *operatorsv1alpha1.BundleLookup, timeout time.Duration) (result *BundleUnpackResult, err error)
+	UnpackBundle(lookup *operatorsv1alpha1.BundleLookup, timeout, retryInterval time.Duration) (result *BundleUnpackResult, err error)
 }
 
 type ConfigMapUnpacker struct {
@@ -448,7 +458,7 @@ const (
 	NotUnpackedMessage          = "bundle contents have not yet been persisted to installplan status"
 )
 
-func (c *ConfigMapUnpacker) UnpackBundle(lookup *operatorsv1alpha1.BundleLookup, timeout time.Duration) (result *BundleUnpackResult, err error) {
+func (c *ConfigMapUnpacker) UnpackBundle(lookup *operatorsv1alpha1.BundleLookup, timeout, retryInterval time.Duration) (result *BundleUnpackResult, err error) {
 	result = newBundleUnpackResult(lookup)
 
 	// if bundle lookup failed condition already present, then there is nothing more to do
@@ -510,7 +520,7 @@ func (c *ConfigMapUnpacker) UnpackBundle(lookup *operatorsv1alpha1.BundleLookup,
 		secrets = append(secrets, corev1.LocalObjectReference{Name: secretName})
 	}
 	var job *batchv1.Job
-	job, err = c.ensureJob(cmRef, result.Path, secrets, timeout)
+	job, err = c.ensureJob(cmRef, result.Path, secrets, timeout, retryInterval)
 	if err != nil || job == nil {
 		// ensureJob can return nil if the job present does not match the expected job (spec and ownerefs)
 		// The current job is deleted in that case so UnpackBundle needs to be retried
@@ -649,16 +659,39 @@ func (c *ConfigMapUnpacker) ensureConfigmap(csRef *corev1.ObjectReference, name
 	return
 }
 
-func (c *ConfigMapUnpacker) ensureJob(cmRef *corev1.ObjectReference, bundlePath string, secrets []corev1.LocalObjectReference, timeout time.Duration) (job *batchv1.Job, err error) {
+func (c *ConfigMapUnpacker) ensureJob(cmRef *corev1.ObjectReference, bundlePath string, secrets []corev1.LocalObjectReference, timeout time.Duration, unpackRetryInterval time.Duration) (job *batchv1.Job, err error) {
 	fresh := c.job(cmRef, bundlePath, secrets, timeout)
-	job, err = c.jobLister.Jobs(fresh.GetNamespace()).Get(fresh.GetName())
+	var jobs, toDelete []*batchv1.Job
+	jobs, err = c.jobLister.Jobs(fresh.GetNamespace()).List(k8slabels.ValidatedSetSelector{bundleUnpackRefLabel: cmRef.Name})
 	if err != nil {
-		if apierrors.IsNotFound(err) {
-			job, err = c.client.BatchV1().Jobs(fresh.GetNamespace()).Create(context.TODO(), fresh, metav1.CreateOptions{})
-		}
-
 		return
 	}
+	if len(jobs) == 0 {
+		job, err = c.client.BatchV1().Jobs(fresh.GetNamespace()).Create(context.TODO(), fresh, metav1.CreateOptions{})
+		return
+	}
+
+	maxRetainedJobs := 5                                  // TODO: make this configurable
+	job, toDelete = sortUnpackJobs(jobs, maxRetainedJobs) // choose latest or on-failed job attempt
+
+	// only check for retries if an unpackRetryInterval is specified
+	if unpackRetryInterval > 0 {
+		if _, isFailed := getCondition(job, batchv1.JobFailed); isFailed {
+			// Look for other unpack jobs for the same bundle
+			if cond, failed := getCondition(job, batchv1.JobFailed); failed {
+				if time.Now().After(cond.LastTransitionTime.Time.Add(unpackRetryInterval)) {
+					fresh.SetName(names.SimpleNameGenerator.GenerateName(fresh.GetName()))
+					job, err = c.client.BatchV1().Jobs(fresh.GetNamespace()).Create(context.TODO(), fresh, metav1.CreateOptions{})
+				}
+			}
+
+			// cleanup old failed jobs, but don't clean up successful jobs to avoid repeat unpacking
+			for _, j := range toDelete {
+				_ = c.client.BatchV1().Jobs(j.GetNamespace()).Delete(context.TODO(), j.GetName(), metav1.DeleteOptions{})
+			}
+			return
+		}
+	}
 
 	if equality.Semantic.DeepDerivative(fresh.GetOwnerReferences(), job.GetOwnerReferences()) && equality.Semantic.DeepDerivative(fresh.Spec, job.Spec) {
 		return
@@ -801,6 +834,37 @@ func getCondition(job *batchv1.Job, conditionType batchv1.JobConditionType) (con
 	return
 }
 
+func sortUnpackJobs(jobs []*batchv1.Job, maxRetainedJobs int) (latest *batchv1.Job, toDelete []*batchv1.Job) {
+	if len(jobs) == 0 {
+		return
+	}
+	// sort jobs so that latest job is first
+	// with preference for non-failed jobs
+	sort.Slice(jobs, func(i, j int) bool {
+		condI, failedI := getCondition(jobs[i], batchv1.JobFailed)
+		condJ, failedJ := getCondition(jobs[j], batchv1.JobFailed)
+		if failedI != failedJ {
+			return !failedI // non-failed job goes first
+		}
+		return condI.LastTransitionTime.After(condJ.LastTransitionTime.Time)
+	})
+	latest = jobs[0]
+	if len(jobs) <= maxRetainedJobs {
+		return
+	}
+	if maxRetainedJobs == 0 {
+		toDelete = jobs[1:]
+		return
+	}
+
+	// cleanup old failed jobs, n-1 recent jobs and the oldest job
+	for i := 0; i < maxRetainedJobs && i+maxRetainedJobs < len(jobs); i++ {
+		toDelete = append(toDelete, jobs[maxRetainedJobs+i])
+	}
+
+	return
+}
+
 // OperatorGroupBundleUnpackTimeout returns bundle timeout from annotation if specified.
 // If the timeout annotation is not set, return timeout < 0 which is subsequently ignored.
 // This is to overrides the --bundle-unpack-timeout flag value on per-OperatorGroup basis.
@@ -827,3 +891,28 @@ func OperatorGroupBundleUnpackTimeout(ogLister v1listers.OperatorGroupNamespaceL
 
 	return d, nil
 }
+
+// OperatorGroupBundleUnpackRetryInterval returns bundle unpack retry interval from annotation if specified.
+// If the retry annotation is not set, return retry = 0 which is subsequently ignored. This interval, if > 0,
+// determines the minimum interval between recreating a failed unpack job.
+func OperatorGroupBundleUnpackRetryInterval(ogLister v1listers.OperatorGroupNamespaceLister) (time.Duration, error) {
+	ogs, err := ogLister.List(k8slabels.Everything())
+	if err != nil {
+		return 0, err
+	}
+	if len(ogs) != 1 {
+		return 0, fmt.Errorf("found %d operatorGroups, expected 1", len(ogs))
+	}
+
+	timeoutStr, ok := ogs[0].GetAnnotations()[BundleUnpackRetryMinimumIntervalAnnotationKey]
+	if !ok {
+		return 0, nil
+	}
+
+	d, err := time.ParseDuration(timeoutStr)
+	if err != nil {
+		return 0, fmt.Errorf("failed to parse unpack retry annotation(%s: %s): %w", BundleUnpackRetryMinimumIntervalAnnotationKey, timeoutStr, err)
+	}
+
+	return d, nil
+}
diff --git a/pkg/controller/bundle/bundlefakes/fake_unpacker.go b/pkg/controller/bundle/bundlefakes/fake_unpacker.go
index 56b666c54..791df3ea6 100644
--- a/pkg/controller/bundle/bundlefakes/fake_unpacker.go
+++ b/pkg/controller/bundle/bundlefakes/fake_unpacker.go
@@ -10,11 +10,12 @@ import (
 )
 
 type FakeUnpacker struct {
-	UnpackBundleStub        func(*v1alpha1.BundleLookup, time.Duration) (*bundle.BundleUnpackResult, error)
+	UnpackBundleStub        func(*v1alpha1.BundleLookup, time.Duration, time.Duration) (*bundle.BundleUnpackResult, error)
 	unpackBundleMutex       sync.RWMutex
 	unpackBundleArgsForCall []struct {
 		arg1 *v1alpha1.BundleLookup
 		arg2 time.Duration
+		arg3 time.Duration
 	}
 	unpackBundleReturns struct {
 		result1 *bundle.BundleUnpackResult
@@ -28,17 +29,18 @@ type FakeUnpacker struct {
 	invocationsMutex sync.RWMutex
 }
 
-func (fake *FakeUnpacker) UnpackBundle(arg1 *v1alpha1.BundleLookup, arg2 time.Duration) (*bundle.BundleUnpackResult, error) {
+func (fake *FakeUnpacker) UnpackBundle(arg1 *v1alpha1.BundleLookup, arg2 time.Duration, arg3 time.Duration) (*bundle.BundleUnpackResult, error) {
 	fake.unpackBundleMutex.Lock()
 	ret, specificReturn := fake.unpackBundleReturnsOnCall[len(fake.unpackBundleArgsForCall)]
 	fake.unpackBundleArgsForCall = append(fake.unpackBundleArgsForCall, struct {
 		arg1 *v1alpha1.BundleLookup
 		arg2 time.Duration
-	}{arg1, arg2})
-	fake.recordInvocation("UnpackBundle", []interface{}{arg1, arg2})
+		arg3 time.Duration
+	}{arg1, arg2, arg3})
+	fake.recordInvocation("UnpackBundle", []interface{}{arg1, arg2, arg3})
 	fake.unpackBundleMutex.Unlock()
 	if fake.UnpackBundleStub != nil {
-		return fake.UnpackBundleStub(arg1, arg2)
+		return fake.UnpackBundleStub(arg1, arg2, arg3)
 	}
 	if specificReturn {
 		return ret.result1, ret.result2
@@ -53,17 +55,17 @@ func (fake *FakeUnpacker) UnpackBundleCallCount() int {
 	return len(fake.unpackBundleArgsForCall)
 }
 
-func (fake *FakeUnpacker) UnpackBundleCalls(stub func(*v1alpha1.BundleLookup, time.Duration) (*bundle.BundleUnpackResult, error)) {
+func (fake *FakeUnpacker) UnpackBundleCalls(stub func(*v1alpha1.BundleLookup, time.Duration, time.Duration) (*bundle.BundleUnpackResult, error)) {
 	fake.unpackBundleMutex.Lock()
 	defer fake.unpackBundleMutex.Unlock()
 	fake.UnpackBundleStub = stub
 }
 
-func (fake *FakeUnpacker) UnpackBundleArgsForCall(i int) (*v1alpha1.BundleLookup, time.Duration) {
+func (fake *FakeUnpacker) UnpackBundleArgsForCall(i int) (*v1alpha1.BundleLookup, time.Duration, time.Duration) {
 	fake.unpackBundleMutex.RLock()
 	defer fake.unpackBundleMutex.RUnlock()
 	argsForCall := fake.unpackBundleArgsForCall[i]
-	return argsForCall.arg1, argsForCall.arg2
+	return argsForCall.arg1, argsForCall.arg2, argsForCall.arg3
 }
 
 func (fake *FakeUnpacker) UnpackBundleReturns(result1 *bundle.BundleUnpackResult, result2 error) {
diff --git a/pkg/controller/install/apiservice.go b/pkg/controller/install/apiservice.go
index 9d17e1664..09e3b5237 100644
--- a/pkg/controller/install/apiservice.go
+++ b/pkg/controller/install/apiservice.go
@@ -61,6 +61,7 @@ func (i *StrategyDeploymentInstaller) createOrUpdateAPIService(caPEM []byte, des
 	if err := ownerutil.AddOwnerLabels(apiService, i.owner); err != nil {
 		return err
 	}
+	apiService.Labels[OLMManagedLabelKey] = OLMManagedLabelValue
 
 	// Create a service for the deployment
 	containerPort := int32(443)
diff --git a/pkg/controller/install/certresources.go b/pkg/controller/install/certresources.go
index 1a2dcbca7..45bc78d0c 100644
--- a/pkg/controller/install/certresources.go
+++ b/pkg/controller/install/certresources.go
@@ -456,7 +456,7 @@ func (i *StrategyDeploymentInstaller) installCertRequirementsForDeployment(deplo
 			Name:     "system:auth-delegator",
 		},
 	}
-	authDelegatorClusterRoleBinding.SetName(service.GetName() + "-system:auth-delegator")
+	authDelegatorClusterRoleBinding.SetName(AuthDelegatorClusterRoleBindingName(service.GetName()))
 	authDelegatorClusterRoleBinding.SetLabels(map[string]string{OLMManagedLabelKey: OLMManagedLabelValue})
 
 	existingAuthDelegatorClusterRoleBinding, err := i.strategyClient.GetOpLister().RbacV1().ClusterRoleBindingLister().Get(authDelegatorClusterRoleBinding.GetName())
@@ -504,7 +504,7 @@ func (i *StrategyDeploymentInstaller) installCertRequirementsForDeployment(deplo
 			Name:     "extension-apiserver-authentication-reader",
 		},
 	}
-	authReaderRoleBinding.SetName(service.GetName() + "-auth-reader")
+	authReaderRoleBinding.SetName(AuthReaderRoleBindingName(service.GetName()))
 	authReaderRoleBinding.SetNamespace(KubeSystem)
 	authReaderRoleBinding.SetLabels(map[string]string{OLMManagedLabelKey: OLMManagedLabelValue})
 
@@ -543,6 +543,14 @@ func (i *StrategyDeploymentInstaller) installCertRequirementsForDeployment(deplo
 	return &depSpec, caPEM, nil
 }
 
+func AuthDelegatorClusterRoleBindingName(serviceName string) string {
+	return serviceName + "-system:auth-delegator"
+}
+
+func AuthReaderRoleBindingName(serviceName string) string {
+	return serviceName + "-auth-reader"
+}
+
 func SetCAAnnotation(depSpec *appsv1.DeploymentSpec, caHash string) {
 	if len(depSpec.Template.ObjectMeta.GetAnnotations()) == 0 {
 		depSpec.Template.ObjectMeta.SetAnnotations(map[string]string{OLMCAHashAnnotationKey: caHash})
diff --git a/pkg/controller/install/deployment.go b/pkg/controller/install/deployment.go
index 12e8044b2..8d5e72626 100644
--- a/pkg/controller/install/deployment.go
+++ b/pkg/controller/install/deployment.go
@@ -152,11 +152,11 @@ func (i *StrategyDeploymentInstaller) deploymentForSpec(name string, spec appsv1
 	dep.Spec.Template.SetAnnotations(annotations)
 
 	// Set custom labels before CSV owner labels
+	dep.SetLabels(specLabels)
 	if dep.Labels == nil {
 		dep.Labels = map[string]string{}
 	}
 	dep.Labels[OLMManagedLabelKey] = OLMManagedLabelValue
-	dep.SetLabels(specLabels)
 
 	ownerutil.AddNonBlockingOwner(dep, i.owner)
 	ownerutil.AddOwnerLabelsForKind(dep, i.owner, v1alpha1.ClusterServiceVersionKind)
diff --git a/pkg/controller/operators/catalog/operator.go b/pkg/controller/operators/catalog/operator.go
index 615f2267e..dbbc581ce 100644
--- a/pkg/controller/operators/catalog/operator.go
+++ b/pkg/controller/operators/catalog/operator.go
@@ -185,7 +185,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 		return nil, err
 	}
 
-	canFilter, err := labeller.Validate(ctx, logger, metadataClient)
+	canFilter, err := labeller.Validate(ctx, logger, metadataClient, crClient)
 	if err != nil {
 		return nil, err
 	}
@@ -207,10 +207,10 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 		ogQueueSet:               queueinformer.NewEmptyResourceQueueSet(),
 		catalogSubscriberIndexer: map[string]cache.Indexer{},
 		serviceAccountQuerier:    scoped.NewUserDefinedServiceAccountQuerier(logger, crClient),
-		clientAttenuator:         scoped.NewClientAttenuator(logger, config, opClient),
+		clientAttenuator:         scoped.NewClientAttenuator(logger, validatingConfig, opClient),
 		installPlanTimeout:       installPlanTimeout,
 		bundleUnpackTimeout:      bundleUnpackTimeout,
-		clientFactory:            clients.NewFactory(config),
+		clientFactory:            clients.NewFactory(validatingConfig),
 	}
 	op.sources = grpc.NewSourceStore(logger, 10*time.Second, 10*time.Minute, op.syncSourceState)
 	op.sourceInvalidator = resolver.SourceProviderFromRegistryClientProvider(op.sources, logger)
@@ -380,10 +380,25 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 	op.lister.RbacV1().RegisterRoleLister(metav1.NamespaceAll, roleInformer.Lister())
 	sharedIndexInformers = append(sharedIndexInformers, roleInformer.Informer())
 
-	labelObjects := func(gvr schema.GroupVersionResource, informer cache.SharedIndexInformer, sync queueinformer.LegacySyncHandler) error {
+	complete := map[schema.GroupVersionResource][]bool{}
+	completeLock := &sync.Mutex{}
+
+	labelObjects := func(gvr schema.GroupVersionResource, informer cache.SharedIndexInformer, sync func(done func() bool) queueinformer.LegacySyncHandler) error {
 		if canFilter {
 			return nil
 		}
+
+		// for each GVR, we may have more than one labelling controller active; each of which detects
+		// when it is done; we allocate a space in complete[gvr][idx] to hold that outcome and track it
+		var idx int
+		if _, exists := complete[gvr]; exists {
+			idx = len(complete[gvr])
+			complete[gvr] = append(complete[gvr], false)
+		} else {
+			idx = 0
+			complete[gvr] = []bool{false}
+		}
+
 		queue := workqueue.NewRateLimitingQueueWithConfig(workqueue.DefaultControllerRateLimiter(), workqueue.RateLimitingQueueConfig{
 			Name: gvr.String(),
 		})
@@ -392,7 +407,23 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 			queueinformer.WithQueue(queue),
 			queueinformer.WithLogger(op.logger),
 			queueinformer.WithInformer(informer),
-			queueinformer.WithSyncer(sync.ToSyncer()),
+			queueinformer.WithSyncer(sync(func() bool {
+				// this function is called by the processor when it detects that it's work is done - so, for that
+				// particular labelling action on that particular GVR, all objects are in the correct state. when
+				// that action is done, we need to further know if that was the last action to be completed, as
+				// when every action we know about has been completed, we re-start the process to allow the future
+				// invocation of this process to filter informers (canFilter = true) and elide all this logic
+				completeLock.Lock()
+				complete[gvr][idx] = true
+				allDone := true
+				for _, items := range complete {
+					for _, done := range items {
+						allDone = allDone && done
+					}
+				}
+				completeLock.Unlock()
+				return allDone
+			}).ToSyncer()),
 		)
 		if err != nil {
 			return err
@@ -408,6 +439,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 	rolesgvk := rbacv1.SchemeGroupVersion.WithResource("roles")
 	if err := labelObjects(rolesgvk, roleInformer.Informer(), labeller.ObjectLabeler[*rbacv1.Role, *rbacv1applyconfigurations.RoleApplyConfiguration](
 		ctx, op.logger, labeller.Filter(rolesgvk),
+		roleInformer.Lister().List,
 		rbacv1applyconfigurations.Role,
 		func(namespace string, ctx context.Context, cfg *rbacv1applyconfigurations.RoleApplyConfiguration, opts metav1.ApplyOptions) (*rbacv1.Role, error) {
 			return op.opClient.KubernetesInterface().RbacV1().Roles(namespace).Apply(ctx, cfg, opts)
@@ -420,6 +452,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 		func(role *rbacv1.Role) (string, error) {
 			return resolver.PolicyRuleHashLabelValue(role.Rules)
 		},
+		roleInformer.Lister().List,
 		rbacv1applyconfigurations.Role,
 		func(namespace string, ctx context.Context, cfg *rbacv1applyconfigurations.RoleApplyConfiguration, opts metav1.ApplyOptions) (*rbacv1.Role, error) {
 			return op.opClient.KubernetesInterface().RbacV1().Roles(namespace).Apply(ctx, cfg, opts)
@@ -436,6 +469,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 	rolebindingsgvk := rbacv1.SchemeGroupVersion.WithResource("rolebindings")
 	if err := labelObjects(rolebindingsgvk, roleBindingInformer.Informer(), labeller.ObjectLabeler[*rbacv1.RoleBinding, *rbacv1applyconfigurations.RoleBindingApplyConfiguration](
 		ctx, op.logger, labeller.Filter(rolebindingsgvk),
+		roleBindingInformer.Lister().List,
 		rbacv1applyconfigurations.RoleBinding,
 		func(namespace string, ctx context.Context, cfg *rbacv1applyconfigurations.RoleBindingApplyConfiguration, opts metav1.ApplyOptions) (*rbacv1.RoleBinding, error) {
 			return op.opClient.KubernetesInterface().RbacV1().RoleBindings(namespace).Apply(ctx, cfg, opts)
@@ -448,6 +482,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 		func(roleBinding *rbacv1.RoleBinding) (string, error) {
 			return resolver.RoleReferenceAndSubjectHashLabelValue(roleBinding.RoleRef, roleBinding.Subjects)
 		},
+		roleBindingInformer.Lister().List,
 		rbacv1applyconfigurations.RoleBinding,
 		func(namespace string, ctx context.Context, cfg *rbacv1applyconfigurations.RoleBindingApplyConfiguration, opts metav1.ApplyOptions) (*rbacv1.RoleBinding, error) {
 			return op.opClient.KubernetesInterface().RbacV1().RoleBindings(namespace).Apply(ctx, cfg, opts)
@@ -463,7 +498,19 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 
 	serviceaccountsgvk := corev1.SchemeGroupVersion.WithResource("serviceaccounts")
 	if err := labelObjects(serviceaccountsgvk, serviceAccountInformer.Informer(), labeller.ObjectLabeler[*corev1.ServiceAccount, *corev1applyconfigurations.ServiceAccountApplyConfiguration](
-		ctx, op.logger, labeller.Filter(serviceaccountsgvk),
+		ctx, op.logger, labeller.ServiceAccountFilter(func(namespace, name string) bool {
+			operatorGroups, err := operatorGroupInformer.Lister().OperatorGroups(namespace).List(labels.Everything())
+			if err != nil {
+				return false
+			}
+			for _, operatorGroup := range operatorGroups {
+				if operatorGroup.Spec.ServiceAccountName == name {
+					return true
+				}
+			}
+			return false
+		}),
+		serviceAccountInformer.Lister().List,
 		corev1applyconfigurations.ServiceAccount,
 		func(namespace string, ctx context.Context, cfg *corev1applyconfigurations.ServiceAccountApplyConfiguration, opts metav1.ApplyOptions) (*corev1.ServiceAccount, error) {
 			return op.opClient.KubernetesInterface().CoreV1().ServiceAccounts(namespace).Apply(ctx, cfg, opts)
@@ -480,6 +527,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 	servicesgvk := corev1.SchemeGroupVersion.WithResource("services")
 	if err := labelObjects(servicesgvk, serviceInformer.Informer(), labeller.ObjectLabeler[*corev1.Service, *corev1applyconfigurations.ServiceApplyConfiguration](
 		ctx, op.logger, labeller.Filter(servicesgvk),
+		serviceInformer.Lister().List,
 		corev1applyconfigurations.Service,
 		func(namespace string, ctx context.Context, cfg *corev1applyconfigurations.ServiceApplyConfiguration, opts metav1.ApplyOptions) (*corev1.Service, error) {
 			return op.opClient.KubernetesInterface().CoreV1().Services(namespace).Apply(ctx, cfg, opts)
@@ -505,6 +553,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 	podsgvk := corev1.SchemeGroupVersion.WithResource("pods")
 	if err := labelObjects(podsgvk, csPodInformer.Informer(), labeller.ObjectLabeler[*corev1.Pod, *corev1applyconfigurations.PodApplyConfiguration](
 		ctx, op.logger, labeller.Filter(podsgvk),
+		csPodInformer.Lister().List,
 		corev1applyconfigurations.Pod,
 		func(namespace string, ctx context.Context, cfg *corev1applyconfigurations.PodApplyConfiguration, opts metav1.ApplyOptions) (*corev1.Pod, error) {
 			return op.opClient.KubernetesInterface().CoreV1().Pods(namespace).Apply(ctx, cfg, opts)
@@ -542,6 +591,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 		ctx, op.logger, labeller.JobFilter(func(namespace, name string) (metav1.Object, error) {
 			return configMapInformer.Lister().ConfigMaps(namespace).Get(name)
 		}),
+		jobInformer.Lister().List,
 		batchv1applyconfigurations.Job,
 		func(namespace string, ctx context.Context, cfg *batchv1applyconfigurations.JobApplyConfiguration, opts metav1.ApplyOptions) (*batchv1.Job, error) {
 			return op.opClient.KubernetesInterface().BatchV1().Jobs(namespace).Apply(ctx, cfg, opts)
@@ -617,6 +667,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 	customresourcedefinitionsgvk := apiextensionsv1.SchemeGroupVersion.WithResource("customresourcedefinitions")
 	if err := labelObjects(customresourcedefinitionsgvk, crdInformer, labeller.ObjectPatchLabeler(
 		ctx, op.logger, labeller.Filter(customresourcedefinitionsgvk),
+		crdLister.List,
 		op.opClient.ApiextensionsInterface().ApiextensionsV1().CustomResourceDefinitions().Patch,
 	)); err != nil {
 		return nil, err
@@ -1112,6 +1163,11 @@ func (o *Operator) syncResolvingNamespace(obj interface{}) error {
 		return err
 	}
 
+	minUnpackRetryInterval, err := bundle.OperatorGroupBundleUnpackRetryInterval(ogLister)
+	if err != nil {
+		return err
+	}
+
 	// TODO: parallel
 	maxGeneration := 0
 	subscriptionUpdated := false
@@ -1207,7 +1263,7 @@ func (o *Operator) syncResolvingNamespace(obj interface{}) error {
 		logger.Debug("unpacking bundles")
 
 		var unpacked bool
-		unpacked, steps, bundleLookups, err = o.unpackBundles(namespace, steps, bundleLookups, unpackTimeout)
+		unpacked, steps, bundleLookups, err = o.unpackBundles(namespace, steps, bundleLookups, unpackTimeout, minUnpackRetryInterval)
 		if err != nil {
 			// If the error was fatal capture and fail
 			if olmerrors.IsFatal(err) {
@@ -1664,7 +1720,7 @@ type UnpackedBundleReference struct {
 	Properties             string `json:"properties"`
 }
 
-func (o *Operator) unpackBundles(namespace string, installPlanSteps []*v1alpha1.Step, bundleLookups []v1alpha1.BundleLookup, unpackTimeout time.Duration) (bool, []*v1alpha1.Step, []v1alpha1.BundleLookup, error) {
+func (o *Operator) unpackBundles(namespace string, installPlanSteps []*v1alpha1.Step, bundleLookups []v1alpha1.BundleLookup, unpackTimeout, unpackRetryInterval time.Duration) (bool, []*v1alpha1.Step, []v1alpha1.BundleLookup, error) {
 	unpacked := true
 
 	outBundleLookups := make([]v1alpha1.BundleLookup, len(bundleLookups))
@@ -1679,7 +1735,7 @@ func (o *Operator) unpackBundles(namespace string, installPlanSteps []*v1alpha1.
 	var errs []error
 	for i := 0; i < len(outBundleLookups); i++ {
 		lookup := outBundleLookups[i]
-		res, err := o.bundleUnpacker.UnpackBundle(&lookup, unpackTimeout)
+		res, err := o.bundleUnpacker.UnpackBundle(&lookup, unpackTimeout, unpackRetryInterval)
 		if err != nil {
 			errs = append(errs, err)
 			continue
@@ -1988,13 +2044,15 @@ func transitionInstallPlanState(log logrus.FieldLogger, transitioner installPlan
 		}
 		log.Debug("attempting to install")
 		if err := transitioner.ExecutePlan(out); err != nil {
-			if now.Sub(out.Status.StartTime.Time) >= timeout {
+			if apierrors.IsForbidden(err) || now.Sub(out.Status.StartTime.Time) < timeout {
+				// forbidden problems are never terminal since we don't know when a user might provide
+				// the service account they specified with more permissions
+				out.Status.Message = fmt.Sprintf("retrying execution due to error: %s", err.Error())
+			} else {
 				out.Status.SetCondition(v1alpha1.ConditionFailed(v1alpha1.InstallPlanInstalled,
 					v1alpha1.InstallPlanReasonComponentFailed, err.Error(), &now))
 				out.Status.Phase = v1alpha1.InstallPlanPhaseFailed
 				out.Status.Message = err.Error()
-			} else {
-				out.Status.Message = fmt.Sprintf("retrying execution due to error: %s", err.Error())
 			}
 			return out, err
 		} else if !out.Status.NeedsRequeue() {
diff --git a/pkg/controller/operators/labeller/filters.go b/pkg/controller/operators/labeller/filters.go
index 4bdbfece3..11e04a24d 100644
--- a/pkg/controller/operators/labeller/filters.go
+++ b/pkg/controller/operators/labeller/filters.go
@@ -6,6 +6,7 @@ import (
 	"strings"
 	"sync"
 
+	operators "github.com/operator-framework/operator-lifecycle-manager/pkg/api/client/clientset/versioned"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/reconciler"
 	"github.com/sirupsen/logrus"
 	"golang.org/x/sync/errgroup"
@@ -16,6 +17,8 @@ import (
 	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/runtime/schema"
+	"k8s.io/apimachinery/pkg/types"
+	"k8s.io/apimachinery/pkg/util/sets"
 	"k8s.io/client-go/metadata"
 
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators/internal/alongside"
@@ -49,15 +52,18 @@ func JobFilter(getConfigMap func(namespace, name string) (metav1.Object, error))
 	}
 }
 
+func ServiceAccountFilter(isServiceAccountReferenced func(namespace, name string) bool) func(object metav1.Object) bool {
+	return func(object metav1.Object) bool {
+		return HasOLMOwnerRef(object) || HasOLMLabel(object) || isServiceAccountReferenced(object.GetNamespace(), object.GetName())
+	}
+}
+
 var filters = map[schema.GroupVersionResource]func(metav1.Object) bool{
 	corev1.SchemeGroupVersion.WithResource("services"): HasOLMOwnerRef,
 	corev1.SchemeGroupVersion.WithResource("pods"): func(object metav1.Object) bool {
 		_, ok := object.GetLabels()[reconciler.CatalogSourceLabelKey]
 		return ok
 	},
-	corev1.SchemeGroupVersion.WithResource("serviceaccounts"): func(object metav1.Object) bool {
-		return HasOLMOwnerRef(object) || HasOLMLabel(object)
-	},
 	appsv1.SchemeGroupVersion.WithResource("deployments"):         HasOLMOwnerRef,
 	rbacv1.SchemeGroupVersion.WithResource("roles"):               HasOLMOwnerRef,
 	rbacv1.SchemeGroupVersion.WithResource("rolebindings"):        HasOLMOwnerRef,
@@ -73,9 +79,9 @@ var filters = map[schema.GroupVersionResource]func(metav1.Object) bool{
 	},
 }
 
-func Validate(ctx context.Context, logger *logrus.Logger, metadataClient metadata.Interface) (bool, error) {
+func Validate(ctx context.Context, logger *logrus.Logger, metadataClient metadata.Interface, operatorClient operators.Interface) (bool, error) {
 	okLock := sync.Mutex{}
-	var ok bool
+	ok := true
 	g, ctx := errgroup.WithContext(ctx)
 	allFilters := map[schema.GroupVersionResource]func(metav1.Object) bool{}
 	for gvr, filter := range filters {
@@ -96,6 +102,27 @@ func Validate(ctx context.Context, logger *logrus.Logger, metadataClient metadat
 			return previous != nil && previous(object) && ContentHashFilter(object)
 		}
 	}
+
+	operatorGroups, err := operatorClient.OperatorsV1().OperatorGroups(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
+	if err != nil {
+		return false, err
+	}
+	userProvidedServiceAccounts := sets.New[types.NamespacedName]()
+	for _, operatorGroup := range operatorGroups.Items {
+		if operatorGroup.Spec.ServiceAccountName != "" {
+			userProvidedServiceAccounts.Insert(types.NamespacedName{
+				Namespace: operatorGroup.Namespace,
+				Name:      operatorGroup.Spec.ServiceAccountName,
+			})
+		}
+	}
+	allFilters[corev1.SchemeGroupVersion.WithResource("serviceaccounts")] = ServiceAccountFilter(func(namespace, name string) bool {
+		return userProvidedServiceAccounts.Has(types.NamespacedName{
+			Namespace: namespace,
+			Name:      name,
+		})
+	})
+
 	for gvr, filter := range allFilters {
 		gvr, filter := gvr, filter
 		g.Go(func() error {
@@ -124,5 +151,6 @@ func Validate(ctx context.Context, logger *logrus.Logger, metadataClient metadat
 	if err := g.Wait(); err != nil {
 		return false, err
 	}
+	logger.WithField("canFilter", ok).Info("detected ability to filter informers")
 	return ok, nil
 }
diff --git a/pkg/controller/operators/labeller/labels.go b/pkg/controller/operators/labeller/labels.go
index 8c2af67c9..f131ed105 100644
--- a/pkg/controller/operators/labeller/labels.go
+++ b/pkg/controller/operators/labeller/labels.go
@@ -4,12 +4,14 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
+	"os"
 	"strings"
 
 	jsonpatch "github.com/evanphx/json-patch"
 	"github.com/sirupsen/logrus"
 	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/apimachinery/pkg/runtime/schema"
 	"k8s.io/apimachinery/pkg/types"
 
@@ -40,28 +42,50 @@ func ObjectLabeler[T metav1.Object, A ApplyConfig[A]](
 	ctx context.Context,
 	logger *logrus.Logger,
 	check func(metav1.Object) bool,
+	list func(options labels.Selector) ([]T, error),
 	applyConfigFor func(name, namespace string) A,
 	apply func(namespace string, ctx context.Context, cfg A, opts metav1.ApplyOptions) (T, error),
-) queueinformer.LegacySyncHandler {
-	return func(obj interface{}) error {
-		cast, ok := obj.(T)
-		if !ok {
-			err := fmt.Errorf("wrong type %T, expected %T: %#v", obj, new(T), obj)
-			logger.WithError(err).Error("casting failed")
-			return fmt.Errorf("casting failed: %w", err)
-		}
+) func(done func() bool) queueinformer.LegacySyncHandler {
+	return func(done func() bool) queueinformer.LegacySyncHandler {
+		return func(obj interface{}) error {
+			cast, ok := obj.(T)
+			if !ok {
+				err := fmt.Errorf("wrong type %T, expected %T: %#v", obj, new(T), obj)
+				logger.WithError(err).Error("casting failed")
+				return fmt.Errorf("casting failed: %w", err)
+			}
 
-		if !check(cast) || hasLabel(cast) {
-			return nil
-		}
+			if !check(cast) || hasLabel(cast) {
+				// if the object we're processing does not need us to label it, it's possible that every object that requires
+				// the label already has it; in which case we should exit the process, so the Pod that succeeds us can filter
+				// the informers used to drive the controller and stop having to track extraneous objects
+				items, err := list(labels.Everything())
+				if err != nil {
+					logger.WithError(err).Warn("failed to list all objects to check for labelling completion")
+					return nil
+				}
+				gvrFullyLabelled := true
+				for _, item := range items {
+					gvrFullyLabelled = gvrFullyLabelled && (!check(item) || hasLabel(item))
+				}
+				if gvrFullyLabelled {
+					allObjectsLabelled := done()
+					if allObjectsLabelled {
+						logrus.Info("detected that every object is labelled, exiting to re-start the process...")
+						os.Exit(0)
+					}
+				}
+				return nil
+			}
 
-		cfg := applyConfigFor(cast.GetName(), cast.GetNamespace())
-		cfg.WithLabels(map[string]string{
-			install.OLMManagedLabelKey: install.OLMManagedLabelValue,
-		})
+			cfg := applyConfigFor(cast.GetName(), cast.GetNamespace())
+			cfg.WithLabels(map[string]string{
+				install.OLMManagedLabelKey: install.OLMManagedLabelValue,
+			})
 
-		_, err := apply(cast.GetNamespace(), ctx, cfg, metav1.ApplyOptions{})
-		return err
+			_, err := apply(cast.GetNamespace(), ctx, cfg, metav1.ApplyOptions{})
+			return err
+		}
 	}
 }
 
@@ -71,58 +95,78 @@ func ObjectPatchLabeler(
 	ctx context.Context,
 	logger *logrus.Logger,
 	check func(metav1.Object) bool,
+	list func(selector labels.Selector) (ret []*metav1.PartialObjectMetadata, err error),
 	patch func(ctx context.Context, name string, pt types.PatchType, data []byte, opts metav1.PatchOptions, subresources ...string) (result *apiextensionsv1.CustomResourceDefinition, err error),
-) func(
-	obj interface{},
-) error {
-	return func(obj interface{}) error {
-		cast, ok := obj.(*apiextensionsv1.CustomResourceDefinition)
-		if !ok {
-			err := fmt.Errorf("wrong type %T, expected %T: %#v", obj, new(*apiextensionsv1.CustomResourceDefinition), obj)
-			logger.WithError(err).Error("casting failed")
-			return fmt.Errorf("casting failed: %w", err)
-		}
+) func(done func() bool) queueinformer.LegacySyncHandler {
+	return func(done func() bool) queueinformer.LegacySyncHandler {
+		return func(obj interface{}) error {
+			cast, ok := obj.(*apiextensionsv1.CustomResourceDefinition)
+			if !ok {
+				err := fmt.Errorf("wrong type %T, expected %T: %#v", obj, new(*apiextensionsv1.CustomResourceDefinition), obj)
+				logger.WithError(err).Error("casting failed")
+				return fmt.Errorf("casting failed: %w", err)
+			}
 
-		if !check(cast) || hasLabel(cast) {
-			return nil
-		}
+			if !check(cast) || hasLabel(cast) {
+				// if the object we're processing does not need us to label it, it's possible that every object that requires
+				// the label already has it; in which case we should exit the process, so the Pod that succeeds us can filter
+				// the informers used to drive the controller and stop having to track extraneous objects
+				items, err := list(labels.Everything())
+				if err != nil {
+					logger.WithError(err).Warn("failed to list all objects to check for labelling completion")
+					return nil
+				}
+				gvrFullyLabelled := true
+				for _, item := range items {
+					gvrFullyLabelled = gvrFullyLabelled && (!check(item) || hasLabel(item))
+				}
+				if gvrFullyLabelled {
+					allObjectsLabelled := done()
+					if allObjectsLabelled {
+						logrus.Info("detected that every object is labelled, exiting to re-start the process...")
+						os.Exit(0)
+					}
+				}
+				return nil
+			}
 
-		uid := cast.GetUID()
-		rv := cast.GetResourceVersion()
+			uid := cast.GetUID()
+			rv := cast.GetResourceVersion()
 
-		// to ensure they appear in the patch as preconditions
-		previous := cast.DeepCopy()
-		previous.SetUID("")
-		previous.SetResourceVersion("")
+			// to ensure they appear in the patch as preconditions
+			previous := cast.DeepCopy()
+			previous.SetUID("")
+			previous.SetResourceVersion("")
 
-		oldData, err := json.Marshal(previous)
-		if err != nil {
-			return fmt.Errorf("failed to Marshal old data for %s/%s: %w", previous.GetNamespace(), previous.GetName(), err)
-		}
+			oldData, err := json.Marshal(previous)
+			if err != nil {
+				return fmt.Errorf("failed to Marshal old data for %s/%s: %w", previous.GetNamespace(), previous.GetName(), err)
+			}
 
-		// to ensure they appear in the patch as preconditions
-		updated := cast.DeepCopy()
-		updated.SetUID(uid)
-		updated.SetResourceVersion(rv)
-		labels := updated.GetLabels()
-		if labels == nil {
-			labels = map[string]string{}
-		}
-		labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
-		updated.SetLabels(labels)
+			// to ensure they appear in the patch as preconditions
+			updated := cast.DeepCopy()
+			updated.SetUID(uid)
+			updated.SetResourceVersion(rv)
+			labels := updated.GetLabels()
+			if labels == nil {
+				labels = map[string]string{}
+			}
+			labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
+			updated.SetLabels(labels)
 
-		newData, err := json.Marshal(updated)
-		if err != nil {
-			return fmt.Errorf("failed to Marshal old data for %s/%s: %w", updated.GetNamespace(), updated.GetName(), err)
-		}
+			newData, err := json.Marshal(updated)
+			if err != nil {
+				return fmt.Errorf("failed to Marshal old data for %s/%s: %w", updated.GetNamespace(), updated.GetName(), err)
+			}
 
-		patchBytes, err := jsonpatch.CreateMergePatch(oldData, newData)
-		if err != nil {
-			return fmt.Errorf("failed to create patch for %s/%s: %w", cast.GetNamespace(), cast.GetName(), err)
-		}
+			patchBytes, err := jsonpatch.CreateMergePatch(oldData, newData)
+			if err != nil {
+				return fmt.Errorf("failed to create patch for %s/%s: %w", cast.GetNamespace(), cast.GetName(), err)
+			}
 
-		_, err = patch(ctx, cast.GetName(), types.MergePatchType, patchBytes, metav1.PatchOptions{})
-		return err
+			_, err = patch(ctx, cast.GetName(), types.MergePatchType, patchBytes, metav1.PatchOptions{})
+			return err
+		}
 	}
 }
 
diff --git a/pkg/controller/operators/labeller/rbac.go b/pkg/controller/operators/labeller/rbac.go
index 6e64791ca..939ea8242 100644
--- a/pkg/controller/operators/labeller/rbac.go
+++ b/pkg/controller/operators/labeller/rbac.go
@@ -3,6 +3,7 @@ package labeller
 import (
 	"context"
 	"fmt"
+	"os"
 
 	"github.com/operator-framework/api/pkg/operators/v1alpha1"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/resolver"
@@ -10,6 +11,7 @@ import (
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/queueinformer"
 	"github.com/sirupsen/logrus"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/labels"
 )
 
 func hasHashLabel(obj metav1.Object) bool {
@@ -22,35 +24,57 @@ func ContentHashLabeler[T metav1.Object, A ApplyConfig[A]](
 	logger *logrus.Logger,
 	check func(metav1.Object) bool,
 	hasher func(object T) (string, error),
+	list func(options labels.Selector) ([]T, error),
 	applyConfigFor func(name, namespace string) A,
 	apply func(namespace string, ctx context.Context, cfg A, opts metav1.ApplyOptions) (T, error),
-) queueinformer.LegacySyncHandler {
-	return func(obj interface{}) error {
-		cast, ok := obj.(T)
-		if !ok {
-			err := fmt.Errorf("wrong type %T, expected %T: %#v", obj, new(T), obj)
-			logger.WithError(err).Error("casting failed")
-			return fmt.Errorf("casting failed: %w", err)
-		}
+) func(done func() bool) queueinformer.LegacySyncHandler {
+	return func(done func() bool) queueinformer.LegacySyncHandler {
+		return func(obj interface{}) error {
+			cast, ok := obj.(T)
+			if !ok {
+				err := fmt.Errorf("wrong type %T, expected %T: %#v", obj, new(T), obj)
+				logger.WithError(err).Error("casting failed")
+				return fmt.Errorf("casting failed: %w", err)
+			}
 
-		if _, _, ok := ownerutil.GetOwnerByKindLabel(cast, v1alpha1.ClusterServiceVersionKind); !ok {
-			return nil
-		}
+			if _, _, ok := ownerutil.GetOwnerByKindLabel(cast, v1alpha1.ClusterServiceVersionKind); !ok {
+				// if the object we're processing does not need us to label it, it's possible that every object that requires
+				// the label already has it; in which case we should exit the process, so the Pod that succeeds us can filter
+				// the informers used to drive the controller and stop having to track extraneous objects
+				items, err := list(labels.Everything())
+				if err != nil {
+					logger.WithError(err).Warn("failed to list all objects to check for labelling completion")
+					return nil
+				}
+				gvrFullyLabelled := true
+				for _, item := range items {
+					gvrFullyLabelled = gvrFullyLabelled && (!check(item) || hasLabel(item))
+				}
+				if gvrFullyLabelled {
+					allObjectsLabelled := done()
+					if allObjectsLabelled {
+						logrus.Info("detected that every object is labelled, exiting to re-start the process...")
+						os.Exit(0)
+					}
+				}
+				return nil
+			}
 
-		if !check(cast) || hasHashLabel(cast) {
-			return nil
-		}
+			if !check(cast) || hasHashLabel(cast) {
+				return nil
+			}
 
-		hash, err := hasher(cast)
-		if err != nil {
-			return fmt.Errorf("failed to calculate hash: %w", err)
-		}
+			hash, err := hasher(cast)
+			if err != nil {
+				return fmt.Errorf("failed to calculate hash: %w", err)
+			}
 
-		cfg := applyConfigFor(cast.GetName(), cast.GetNamespace())
-		cfg.WithLabels(map[string]string{
-			resolver.ContentHashLabelKey: hash,
-		})
-		_, err = apply(cast.GetNamespace(), ctx, cfg, metav1.ApplyOptions{})
-		return err
+			cfg := applyConfigFor(cast.GetName(), cast.GetNamespace())
+			cfg.WithLabels(map[string]string{
+				resolver.ContentHashLabelKey: hash,
+			})
+			_, err = apply(cast.GetNamespace(), ctx, cfg, metav1.ApplyOptions{})
+			return err
+		}
 	}
 }
diff --git a/pkg/controller/operators/olm/apiservices.go b/pkg/controller/operators/olm/apiservices.go
index cb052f510..69d49e144 100644
--- a/pkg/controller/operators/olm/apiservices.go
+++ b/pkg/controller/operators/olm/apiservices.go
@@ -6,7 +6,6 @@ import (
 
 	log "github.com/sirupsen/logrus"
 	appsv1 "k8s.io/api/apps/v1"
-	rbacv1 "k8s.io/api/rbac/v1"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/labels"
@@ -47,7 +46,6 @@ func (a *Operator) checkAPIServiceResources(csv *v1alpha1.ClusterServiceVersion,
 	})
 
 	errs := []error{}
-	ruleChecker := install.NewCSVRuleChecker(a.lister.RbacV1().RoleLister(), a.lister.RbacV1().RoleBindingLister(), a.lister.RbacV1().ClusterRoleLister(), a.lister.RbacV1().ClusterRoleBindingLister(), csv)
 	for _, desc := range csv.GetOwnedAPIServiceDescriptions() {
 		apiServiceName := desc.GetName()
 		logger := logger.WithFields(log.Fields{
@@ -164,60 +162,27 @@ func (a *Operator) checkAPIServiceResources(csv *v1alpha1.ClusterServiceVersion,
 		if serviceAccountName == "" {
 			serviceAccountName = "default"
 		}
-		serviceAccount, err := a.lister.CoreV1().ServiceAccountLister().ServiceAccounts(deployment.GetNamespace()).Get(serviceAccountName)
+		_, err = a.opClient.KubernetesInterface().CoreV1().ServiceAccounts(deployment.GetNamespace()).Get(context.TODO(), serviceAccountName, metav1.GetOptions{})
 		if err != nil {
-			logger.WithField("serviceaccount", serviceAccountName).Warnf("could not retrieve ServiceAccount")
+			logger.WithError(err).WithField("serviceaccount", serviceAccountName).Warnf("could not retrieve ServiceAccount")
 			errs = append(errs, err)
-			continue
 		}
 
-		// Ensure RBAC permissions for the APIService are correct
-		rulesMap := map[string][]rbacv1.PolicyRule{
-			// Serving cert Secret Rule
-			csv.GetNamespace(): {
-				{
-					Verbs:         []string{"get"},
-					APIGroups:     []string{""},
-					Resources:     []string{"secrets"},
-					ResourceNames: []string{secret.GetName()},
-				},
-			},
-			install.KubeSystem:  {},
-			metav1.NamespaceAll: {},
+		if _, err := a.lister.RbacV1().RoleLister().Roles(secret.GetNamespace()).Get(secret.GetName()); err != nil {
+			logger.WithError(err).Warnf("could not retrieve role %s/%s", secret.GetNamespace(), secret.GetName())
+			errs = append(errs, err)
 		}
-
-		// extension-apiserver-authentication-reader
-		authReaderRole, err := a.lister.RbacV1().RoleLister().Roles(install.KubeSystem).Get("extension-apiserver-authentication-reader")
-		if err != nil {
-			logger.Warnf("could not retrieve Role extension-apiserver-authentication-reader")
+		if _, err := a.lister.RbacV1().RoleBindingLister().RoleBindings(secret.GetNamespace()).Get(secret.GetName()); err != nil {
+			logger.WithError(err).Warnf("could not retrieve role binding %s/%s", secret.GetNamespace(), secret.GetName())
 			errs = append(errs, err)
-			continue
 		}
-		rulesMap[install.KubeSystem] = append(rulesMap[install.KubeSystem], authReaderRole.Rules...)
-
-		// system:auth-delegator
-		authDelegatorClusterRole, err := a.lister.RbacV1().ClusterRoleLister().Get("system:auth-delegator")
-		if err != nil {
-			logger.Warnf("could not retrieve ClusterRole system:auth-delegator")
+		if _, err := a.lister.RbacV1().ClusterRoleBindingLister().Get(install.AuthDelegatorClusterRoleBindingName(service.GetName())); err != nil {
+			logger.WithError(err).Warnf("could not retrieve auth delegator cluster role binding %s", install.AuthDelegatorClusterRoleBindingName(service.GetName()))
 			errs = append(errs, err)
-			continue
 		}
-		rulesMap[metav1.NamespaceAll] = append(rulesMap[metav1.NamespaceAll], authDelegatorClusterRole.Rules...)
-
-		for namespace, rules := range rulesMap {
-			for _, rule := range rules {
-				satisfied, err := ruleChecker.RuleSatisfied(serviceAccount, namespace, rule)
-				if err != nil {
-					logger.WithField("rule", fmt.Sprintf("%+v", rule)).Warnf("error checking Rule")
-					errs = append(errs, err)
-					continue
-				}
-				if !satisfied {
-					logger.WithField("rule", fmt.Sprintf("%+v", rule)).Warnf("Rule not satisfied")
-					errs = append(errs, fmt.Errorf("rule %+v not satisfied", rule))
-					continue
-				}
-			}
+		if _, err := a.lister.RbacV1().RoleBindingLister().RoleBindings(install.KubeSystem).Get(install.AuthReaderRoleBindingName(service.GetName())); err != nil {
+			logger.WithError(err).Warnf("could not retrieve role binding %s/%s", install.KubeSystem, install.AuthReaderRoleBindingName(service.GetName()))
+			errs = append(errs, err)
 		}
 	}
 
diff --git a/pkg/controller/operators/olm/operator.go b/pkg/controller/operators/olm/operator.go
index 9855e8d10..d30005b67 100644
--- a/pkg/controller/operators/olm/operator.go
+++ b/pkg/controller/operators/olm/operator.go
@@ -5,6 +5,7 @@ import (
 	"errors"
 	"fmt"
 	"strings"
+	"sync"
 	"time"
 
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators/labeller"
@@ -102,12 +103,57 @@ type Operator struct {
 	clientFactory                clients.Factory
 	plugins                      []plugins.OperatorPlugin
 	informersByNamespace         map[string]*plugins.Informers
+	informersFiltered            bool
+
+	ruleChecker     func(*v1alpha1.ClusterServiceVersion) *install.CSVRuleChecker
+	ruleCheckerLock sync.RWMutex
+	resyncPeriod    func() time.Duration
+	ctx             context.Context
 }
 
 func (a *Operator) Informers() map[string]*plugins.Informers {
 	return a.informersByNamespace
 }
 
+func (a *Operator) getRuleChecker() func(*v1alpha1.ClusterServiceVersion) *install.CSVRuleChecker {
+	var ruleChecker func(*v1alpha1.ClusterServiceVersion) *install.CSVRuleChecker
+	a.ruleCheckerLock.RLock()
+	ruleChecker = a.ruleChecker
+	a.ruleCheckerLock.RUnlock()
+	if ruleChecker != nil {
+		return ruleChecker
+	}
+
+	a.ruleCheckerLock.Lock()
+	defer a.ruleCheckerLock.Unlock()
+	if a.ruleChecker != nil {
+		return a.ruleChecker
+	}
+
+	sif := informers.NewSharedInformerFactoryWithOptions(a.opClient.KubernetesInterface(), a.resyncPeriod())
+	rolesLister := sif.Rbac().V1().Roles().Lister()
+	roleBindingsLister := sif.Rbac().V1().RoleBindings().Lister()
+	clusterRolesLister := sif.Rbac().V1().ClusterRoles().Lister()
+	clusterRoleBindingsLister := sif.Rbac().V1().ClusterRoleBindings().Lister()
+
+	sif.Start(a.ctx.Done())
+	sif.WaitForCacheSync(a.ctx.Done())
+
+	if a.ctx.Err() != nil {
+		a.ruleChecker = nil
+		return nil
+	}
+
+	a.ruleChecker = func(csv *v1alpha1.ClusterServiceVersion) *install.CSVRuleChecker {
+		return install.NewCSVRuleChecker(
+			rolesLister, roleBindingsLister,
+			clusterRolesLister, clusterRoleBindingsLister,
+			csv,
+		)
+	}
+	return a.ruleChecker
+}
+
 func NewOperator(ctx context.Context, options ...OperatorOption) (*Operator, error) {
 	config := defaultOperatorConfig()
 	config.apply(options)
@@ -140,7 +186,7 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 		return nil, err
 	}
 
-	canFilter, err := labeller.Validate(ctx, config.logger, config.metadataClient)
+	canFilter, err := labeller.Validate(ctx, config.logger, config.metadataClient, config.externalClient)
 	if err != nil {
 		return nil, err
 	}
@@ -171,6 +217,10 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 		serviceAccountQuerier:        scoped.NewUserDefinedServiceAccountQuerier(config.logger, config.externalClient),
 		clientFactory:                clients.NewFactory(config.restConfig),
 		protectedCopiedCSVNamespaces: config.protectedCopiedCSVNamespaces,
+		resyncPeriod:                 config.resyncPeriod,
+		ruleCheckerLock:              sync.RWMutex{},
+		ctx:                          ctx,
+		informersFiltered:            canFilter,
 	}
 
 	informersByNamespace := map[string]*plugins.Informers{}
@@ -447,10 +497,24 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 		}
 	}
 
-	labelObjects := func(gvr schema.GroupVersionResource, informer cache.SharedIndexInformer, sync queueinformer.LegacySyncHandler) error {
+	complete := map[schema.GroupVersionResource][]bool{}
+	completeLock := &sync.Mutex{}
+
+	labelObjects := func(gvr schema.GroupVersionResource, informer cache.SharedIndexInformer, sync func(done func() bool) queueinformer.LegacySyncHandler) error {
 		if canFilter {
 			return nil
 		}
+
+		// for each GVR, we may have more than one labelling controller active; each of which detects
+		// when it is done; we allocate a space in complete[gvr][idx] to hold that outcome and track it
+		var idx int
+		if _, exists := complete[gvr]; exists {
+			idx = len(complete[gvr])
+			complete[gvr] = append(complete[gvr], false)
+		} else {
+			idx = 0
+			complete[gvr] = []bool{false}
+		}
 		queue := workqueue.NewRateLimitingQueueWithConfig(workqueue.DefaultControllerRateLimiter(), workqueue.RateLimitingQueueConfig{
 			Name: gvr.String(),
 		})
@@ -459,7 +523,23 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 			queueinformer.WithQueue(queue),
 			queueinformer.WithLogger(op.logger),
 			queueinformer.WithInformer(informer),
-			queueinformer.WithSyncer(sync.ToSyncer()),
+			queueinformer.WithSyncer(sync(func() bool {
+				// this function is called by the processor when it detects that it's work is done - so, for that
+				// particular labelling action on that particular GVR, all objects are in the correct state. when
+				// that action is done, we need to further know if that was the last action to be completed, as
+				// when every action we know about has been completed, we re-start the process to allow the future
+				// invocation of this process to filter informers (canFilter = true) and elide all this logic
+				completeLock.Lock()
+				complete[gvr][idx] = true
+				allDone := true
+				for _, items := range complete {
+					for _, done := range items {
+						allDone = allDone && done
+					}
+				}
+				completeLock.Unlock()
+				return allDone
+			}).ToSyncer()),
 		)
 		if err != nil {
 			return err
@@ -475,6 +555,7 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 	deploymentsgvk := appsv1.SchemeGroupVersion.WithResource("deployments")
 	if err := labelObjects(deploymentsgvk, informersByNamespace[metav1.NamespaceAll].DeploymentInformer.Informer(), labeller.ObjectLabeler[*appsv1.Deployment, *appsv1applyconfigurations.DeploymentApplyConfiguration](
 		ctx, op.logger, labeller.Filter(deploymentsgvk),
+		informersByNamespace[metav1.NamespaceAll].DeploymentInformer.Lister().List,
 		appsv1applyconfigurations.Deployment,
 		func(namespace string, ctx context.Context, cfg *appsv1applyconfigurations.DeploymentApplyConfiguration, opts metav1.ApplyOptions) (*appsv1.Deployment, error) {
 			return op.opClient.KubernetesInterface().AppsV1().Deployments(namespace).Apply(ctx, cfg, opts)
@@ -547,6 +628,7 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 	clusterrolesgvk := rbacv1.SchemeGroupVersion.WithResource("clusterroles")
 	if err := labelObjects(clusterrolesgvk, clusterRoleInformer.Informer(), labeller.ObjectLabeler[*rbacv1.ClusterRole, *rbacv1applyconfigurations.ClusterRoleApplyConfiguration](
 		ctx, op.logger, labeller.Filter(clusterrolesgvk),
+		clusterRoleInformer.Lister().List,
 		func(name, _ string) *rbacv1applyconfigurations.ClusterRoleApplyConfiguration {
 			return rbacv1applyconfigurations.ClusterRole(name)
 		},
@@ -561,6 +643,7 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 		func(clusterRole *rbacv1.ClusterRole) (string, error) {
 			return resolver.PolicyRuleHashLabelValue(clusterRole.Rules)
 		},
+		clusterRoleInformer.Lister().List,
 		func(name, _ string) *rbacv1applyconfigurations.ClusterRoleApplyConfiguration {
 			return rbacv1applyconfigurations.ClusterRole(name)
 		},
@@ -590,6 +673,7 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 	clusterrolebindingssgvk := rbacv1.SchemeGroupVersion.WithResource("clusterrolebindings")
 	if err := labelObjects(clusterrolebindingssgvk, clusterRoleBindingInformer.Informer(), labeller.ObjectLabeler[*rbacv1.ClusterRoleBinding, *rbacv1applyconfigurations.ClusterRoleBindingApplyConfiguration](
 		ctx, op.logger, labeller.Filter(clusterrolebindingssgvk),
+		clusterRoleBindingInformer.Lister().List,
 		func(name, _ string) *rbacv1applyconfigurations.ClusterRoleBindingApplyConfiguration {
 			return rbacv1applyconfigurations.ClusterRoleBinding(name)
 		},
@@ -604,6 +688,7 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 		func(clusterRoleBinding *rbacv1.ClusterRoleBinding) (string, error) {
 			return resolver.RoleReferenceAndSubjectHashLabelValue(clusterRoleBinding.RoleRef, clusterRoleBinding.Subjects)
 		},
+		clusterRoleBindingInformer.Lister().List,
 		func(name, _ string) *rbacv1applyconfigurations.ClusterRoleBindingApplyConfiguration {
 			return rbacv1applyconfigurations.ClusterRoleBinding(name)
 		},
@@ -614,8 +699,9 @@ func newOperatorWithConfig(ctx context.Context, config *operatorConfig) (*Operat
 		return nil, err
 	}
 
-	// register namespace queueinformer
-	namespaceInformer := k8sInformerFactory.Core().V1().Namespaces()
+	// register namespace queueinformer using a new informer factory - since namespaces won't have the labels
+	// that other k8s objects will
+	namespaceInformer := informers.NewSharedInformerFactory(op.opClient.KubernetesInterface(), config.resyncPeriod()).Core().V1().Namespaces()
 	informersByNamespace[metav1.NamespaceAll].NamespaceInformer = namespaceInformer
 	op.lister.CoreV1().RegisterNamespaceLister(namespaceInformer.Lister())
 	op.nsQueueSet = workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "resolver")
diff --git a/pkg/controller/operators/olm/operatorgroup.go b/pkg/controller/operators/olm/operatorgroup.go
index bcf27c952..a23803a2d 100644
--- a/pkg/controller/operators/olm/operatorgroup.go
+++ b/pkg/controller/operators/olm/operatorgroup.go
@@ -487,7 +487,6 @@ func (a *Operator) ensureRBACInTargetNamespace(csv *v1alpha1.ClusterServiceVersi
 	if !ok {
 		return fmt.Errorf("could not cast install strategy as type %T", strategyDetailsDeployment)
 	}
-	ruleChecker := install.NewCSVRuleChecker(a.lister.RbacV1().RoleLister(), a.lister.RbacV1().RoleBindingLister(), a.lister.RbacV1().ClusterRoleLister(), a.lister.RbacV1().ClusterRoleBindingLister(), csv)
 
 	logger := a.logger.WithField("opgroup", operatorGroup.GetName()).WithField("csv", csv.GetName())
 
@@ -498,7 +497,7 @@ func (a *Operator) ensureRBACInTargetNamespace(csv *v1alpha1.ClusterServiceVersi
 		// synthesize cluster permissions to verify rbac
 		strategyDetailsDeployment.ClusterPermissions = append(strategyDetailsDeployment.ClusterPermissions, strategyDetailsDeployment.Permissions...)
 		strategyDetailsDeployment.Permissions = nil
-		permMet, _, err := a.permissionStatus(strategyDetailsDeployment, ruleChecker, corev1.NamespaceAll, csv)
+		permMet, _, err := a.permissionStatus(strategyDetailsDeployment, corev1.NamespaceAll, csv)
 		if err != nil {
 			return err
 		}
@@ -653,6 +652,7 @@ func (a *Operator) ensureTenantRBAC(operatorNamespace, targetNamespace string, c
 			return err
 		}
 		targetRole.SetLabels(utillabels.AddLabel(targetRole.GetLabels(), v1alpha1.CopiedLabelKey, operatorNamespace))
+		targetRole.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 		if _, err := a.opClient.CreateRole(targetRole); err != nil {
 			return err
 		}
@@ -697,6 +697,7 @@ func (a *Operator) ensureTenantRBAC(operatorNamespace, targetNamespace string, c
 			return err
 		}
 		ownedRoleBinding.SetLabels(utillabels.AddLabel(ownedRoleBinding.GetLabels(), v1alpha1.CopiedLabelKey, operatorNamespace))
+		ownedRoleBinding.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 		if _, err := a.opClient.CreateRoleBinding(ownedRoleBinding); err != nil {
 			return err
 		}
@@ -711,8 +712,6 @@ func (a *Operator) ensureCSVsInNamespaces(csv *v1alpha1.ClusterServiceVersion, o
 	}
 
 	strategyDetailsDeployment := &csv.Spec.InstallStrategy.StrategySpec
-	ruleChecker := install.NewCSVRuleChecker(a.lister.RbacV1().RoleLister(), a.lister.RbacV1().RoleBindingLister(), a.lister.RbacV1().ClusterRoleLister(), a.lister.RbacV1().ClusterRoleBindingLister(), csv)
-
 	logger := a.logger.WithField("opgroup", operatorGroup.GetName()).WithField("csv", csv.GetName())
 
 	targetCSVs := make(map[string]*v1alpha1.ClusterServiceVersion)
@@ -728,20 +727,20 @@ func (a *Operator) ensureCSVsInNamespaces(csv *v1alpha1.ClusterServiceVersion, o
 		if targets.Contains(ns.GetName()) {
 			var targetCSV *v1alpha1.ClusterServiceVersion
 			if targetCSV, err = a.copyToNamespace(&copyPrototype, csv.GetNamespace(), ns.GetName(), nonstatus, status); err != nil {
-				a.logger.WithError(err).Debug("error copying to target")
+				logger.WithError(err).Debug("error copying to target")
 				continue
 			}
 			targetCSVs[ns.GetName()] = targetCSV
 		} else {
 			if err := a.pruneFromNamespace(operatorGroup.GetName(), ns.GetName()); err != nil {
-				a.logger.WithError(err).Debug("error pruning from old target")
+				logger.WithError(err).Debug("error pruning from old target")
 			}
 		}
 	}
 
 	targetNamespaces := operatorGroup.Status.Namespaces
 	if targetNamespaces == nil {
-		a.logger.Errorf("operatorgroup '%v' should have non-nil status", operatorGroup.GetName())
+		logger.Errorf("operatorgroup '%v' should have non-nil status", operatorGroup.GetName())
 		return nil
 	}
 	if len(targetNamespaces) == 1 && targetNamespaces[0] == corev1.NamespaceAll {
@@ -750,7 +749,7 @@ func (a *Operator) ensureCSVsInNamespaces(csv *v1alpha1.ClusterServiceVersion, o
 	}
 	for _, ns := range targetNamespaces {
 		// create roles/rolebindings for each target namespace
-		permMet, _, err := a.permissionStatus(strategyDetailsDeployment, ruleChecker, ns, csv)
+		permMet, _, err := a.permissionStatus(strategyDetailsDeployment, ns, csv)
 		if err != nil {
 			logger.WithError(err).Debug("permission status")
 			return err
@@ -1051,6 +1050,7 @@ func (a *Operator) ensureOpGroupClusterRole(op *operatorsv1.OperatorGroup, suffi
 	if err := ownerutil.AddOwnerLabels(clusterRole, op); err != nil {
 		return err
 	}
+	clusterRole.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 
 	a.logger.Infof("creating cluster role: %s owned by operator group: %s/%s", clusterRole.GetName(), op.GetNamespace(), op.GetName())
 	_, err = a.opClient.KubernetesInterface().RbacV1().ClusterRoles().Create(context.TODO(), clusterRole, metav1.CreateOptions{})
diff --git a/pkg/controller/operators/olm/requirements.go b/pkg/controller/operators/olm/requirements.go
index 0c288b8e3..890fb06cb 100644
--- a/pkg/controller/operators/olm/requirements.go
+++ b/pkg/controller/operators/olm/requirements.go
@@ -3,19 +3,22 @@ package olm
 import (
 	"context"
 	"encoding/json"
+	"errors"
 	"fmt"
 	"strings"
 
 	"github.com/coreos/go-semver/semver"
-	"github.com/sirupsen/logrus"
-	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-
 	"github.com/operator-framework/api/pkg/operators/v1alpha1"
 	listersv1alpha1 "github.com/operator-framework/operator-lifecycle-manager/pkg/api/client/listers/operators/v1alpha1"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/install"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/operators/internal/alongside"
+	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/resolver"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/ownerutil"
+	"github.com/sirupsen/logrus"
+	rbacv1 "k8s.io/api/rbac/v1"
+	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/labels"
 )
 
 func (a *Operator) minKubeVersionStatus(name string, minKubeVersion string) (met bool, statuses []v1alpha1.RequirementStatus) {
@@ -254,7 +257,7 @@ func (a *Operator) requirementStatus(strategyDetailsDeployment *v1alpha1.Strateg
 }
 
 // permissionStatus checks whether the given CSV's RBAC requirements are met in its namespace
-func (a *Operator) permissionStatus(strategyDetailsDeployment *v1alpha1.StrategyDetailsDeployment, ruleChecker install.RuleChecker, targetNamespace string, csv *v1alpha1.ClusterServiceVersion) (bool, []v1alpha1.RequirementStatus, error) {
+func (a *Operator) permissionStatus(strategyDetailsDeployment *v1alpha1.StrategyDetailsDeployment, targetNamespace string, csv *v1alpha1.ClusterServiceVersion) (bool, []v1alpha1.RequirementStatus, error) {
 	statusesSet := map[string]v1alpha1.RequirementStatus{}
 
 	checkPermissions := func(permissions []v1alpha1.StrategyDeploymentPermissions, namespace string) (bool, error) {
@@ -296,6 +299,60 @@ func (a *Operator) permissionStatus(strategyDetailsDeployment *v1alpha1.Strategy
 			}
 
 			// Check if PolicyRules are satisfied
+			if a.informersFiltered {
+				// we don't hold the whole set of RBAC in memory, so we can't use the authorizer:
+				// check for rules we would have created ourselves first
+				var err error
+				var permissionMet bool
+				if namespace == metav1.NamespaceAll {
+					permissionMet, err = permissionsPreviouslyCreated[*rbacv1.ClusterRole, *rbacv1.ClusterRoleBinding](
+						perm, csv,
+						a.lister.RbacV1().ClusterRoleLister().List, a.lister.RbacV1().ClusterRoleBindingLister().List,
+					)
+				} else {
+					permissionMet, err = permissionsPreviouslyCreated[*rbacv1.Role, *rbacv1.RoleBinding](
+						perm, csv,
+						a.lister.RbacV1().RoleLister().List, a.lister.RbacV1().RoleBindingLister().List,
+					)
+				}
+				if err != nil {
+					return false, err
+				}
+				if permissionMet {
+					// OLM previously made all the permissions we need, exit early
+					for _, rule := range perm.Rules {
+						dependent := v1alpha1.DependentStatus{
+							Group:   "rbac.authorization.k8s.io",
+							Kind:    "PolicyRule",
+							Version: "v1",
+							Status:  v1alpha1.DependentStatusReasonSatisfied,
+						}
+						marshalled, err := json.Marshal(rule)
+						if err != nil {
+							dependent.Status = v1alpha1.DependentStatusReasonNotSatisfied
+							dependent.Message = "rule unmarshallable"
+							status.Dependents = append(status.Dependents, dependent)
+							continue
+						}
+
+						var scope string
+						if namespace == metav1.NamespaceAll {
+							scope = "cluster"
+						} else {
+							scope = "namespaced"
+						}
+						dependent.Message = fmt.Sprintf("%s rule:%s", scope, marshalled)
+						status.Dependents = append(status.Dependents, dependent)
+					}
+					continue
+				}
+			}
+			// if we have not filtered our informers or if we were unable to detect the correct permissions, we have
+			// no choice but to page in the world and see if the user pre-created permissions for this CSV
+			ruleChecker := a.getRuleChecker()(csv)
+			if ruleChecker == nil {
+				return false, errors.New("could not create a rule checker (are we shutting down?)")
+			}
 			for _, rule := range perm.Rules {
 				dependent := v1alpha1.DependentStatus{
 					Group:   "rbac.authorization.k8s.io",
@@ -358,6 +415,54 @@ func (a *Operator) permissionStatus(strategyDetailsDeployment *v1alpha1.Strategy
 	return permMet && clusterPermMet, statuses, nil
 }
 
+func permissionsPreviouslyCreated[T, U metav1.Object](
+	permission v1alpha1.StrategyDeploymentPermissions,
+	csv *v1alpha1.ClusterServiceVersion,
+	listRoles func(labels.Selector) ([]T, error),
+	listBindings func(labels.Selector) ([]U, error),
+) (bool, error) {
+	// first, find the (cluster)role
+	ruleHash, err := resolver.PolicyRuleHashLabelValue(permission.Rules)
+	if err != nil {
+		return false, fmt.Errorf("failed to hash permission rules: %w", err)
+	}
+	roleSelectorMap := ownerutil.OwnerLabel(csv, v1alpha1.ClusterServiceVersionKind)
+	roleSelectorMap[resolver.ContentHashLabelKey] = ruleHash
+	roles, err := listRoles(labels.SelectorFromSet(roleSelectorMap))
+	if err != nil {
+		return false, err
+	}
+
+	if len(roles) == 0 {
+		return false, nil
+	}
+
+	// then, find the (cluster)rolebinding, if we found the role
+	bindingHash, err := resolver.RoleReferenceAndSubjectHashLabelValue(rbacv1.RoleRef{
+		Kind:     "Role",
+		Name:     roles[0].GetName(),
+		APIGroup: rbacv1.GroupName,
+	},
+		[]rbacv1.Subject{{
+			Kind:      "ServiceAccount",
+			Name:      permission.ServiceAccountName,
+			Namespace: csv.GetNamespace(),
+		}},
+	)
+	if err != nil {
+		return false, fmt.Errorf("failed to hash binding content: %w", err)
+	}
+	bindingSelectorMap := ownerutil.OwnerLabel(csv, v1alpha1.ClusterServiceVersionKind)
+	bindingSelectorMap[resolver.ContentHashLabelKey] = bindingHash
+	bindingSelectorSet := labels.Set{}
+	for key, value := range bindingSelectorMap {
+		bindingSelectorSet[key] = value
+	}
+	bindingSelector := labels.SelectorFromSet(bindingSelectorSet)
+	bindings, err := listBindings(bindingSelector)
+	return len(roles) > 0 && len(bindings) > 0, err
+}
+
 // requirementAndPermissionStatus returns the aggregate requirement and permissions statuses for the given CSV
 func (a *Operator) requirementAndPermissionStatus(csv *v1alpha1.ClusterServiceVersion) (bool, []v1alpha1.RequirementStatus, error) {
 	allReqStatuses := []v1alpha1.RequirementStatus{}
@@ -383,14 +488,7 @@ func (a *Operator) requirementAndPermissionStatus(csv *v1alpha1.ClusterServiceVe
 	reqMet, reqStatuses := a.requirementStatus(strategyDetailsDeployment, csv)
 	allReqStatuses = append(allReqStatuses, reqStatuses...)
 
-	rbacLister := a.lister.RbacV1()
-	roleLister := rbacLister.RoleLister()
-	roleBindingLister := rbacLister.RoleBindingLister()
-	clusterRoleLister := rbacLister.ClusterRoleLister()
-	clusterRoleBindingLister := rbacLister.ClusterRoleBindingLister()
-
-	ruleChecker := install.NewCSVRuleChecker(roleLister, roleBindingLister, clusterRoleLister, clusterRoleBindingLister, csv)
-	permMet, permStatuses, err := a.permissionStatus(strategyDetailsDeployment, ruleChecker, csv.GetNamespace(), csv)
+	permMet, permStatuses, err := a.permissionStatus(strategyDetailsDeployment, csv.GetNamespace(), csv)
 	if err != nil {
 		return false, nil, err
 	}
diff --git a/pkg/lib/scoped/syncer.go b/pkg/lib/scoped/syncer.go
index dec0a0d0e..15208fd24 100644
--- a/pkg/lib/scoped/syncer.go
+++ b/pkg/lib/scoped/syncer.go
@@ -5,12 +5,14 @@ import (
 	"fmt"
 	"reflect"
 
+	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/install"
 	"github.com/sirupsen/logrus"
 	corev1 "k8s.io/api/core/v1"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
-	meta "k8s.io/apimachinery/pkg/api/meta"
+	"k8s.io/apimachinery/pkg/api/meta"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/runtime"
+	corev1applyconfigurations "k8s.io/client-go/applyconfigurations/core/v1"
 	"k8s.io/client-go/tools/reference"
 	"k8s.io/utils/clock"
 
@@ -98,6 +100,14 @@ func (s *UserDefinedServiceAccountSyncer) SyncOperatorGroup(in *v1.OperatorGroup
 		return
 	}
 
+	// A service account has been specified, but likely does not have the labels we require it to have to ensure it
+	// shows up in our listers, so let's add that and queue again later
+	config := corev1applyconfigurations.ServiceAccount(serviceAccountName, namespace)
+	config.Labels = map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue}
+	if _, err := s.client.KubernetesInterface().CoreV1().ServiceAccounts(namespace).Apply(context.TODO(), config, metav1.ApplyOptions{FieldManager: "operator-lifecycle-manager"}); err != nil {
+		return out, fmt.Errorf("failed to apply labels[%s]=%s to serviceaccount %s/%s: %w", install.OLMManagedLabelKey, install.OLMManagedLabelValue, namespace, serviceAccountName, err)
+	}
+
 	ref, err := reference.GetReference(s.scheme, sa)
 	if err != nil {
 		return


Test patch:
diff --git a/pkg/controller/bundle/bundle_unpacker_test.go b/pkg/controller/bundle/bundle_unpacker_test.go
index 2bb0d4702..a583eb541 100644
--- a/pkg/controller/bundle/bundle_unpacker_test.go
+++ b/pkg/controller/bundle/bundle_unpacker_test.go
@@ -208,7 +208,7 @@ func TestConfigMapUnpacker(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name:      pathHash,
 							Namespace: "ns-a",
-							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue, bundleUnpackRefLabel: pathHash},
 							OwnerReferences: []metav1.OwnerReference{
 								{
 									APIVersion:         "v1",
@@ -441,7 +441,7 @@ func TestConfigMapUnpacker(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name:      digestHash,
 							Namespace: "ns-a",
-							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue, bundleUnpackRefLabel: digestHash},
 							OwnerReferences: []metav1.OwnerReference{
 								{
 									APIVersion:         "v1",
@@ -712,7 +712,7 @@ func TestConfigMapUnpacker(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name:      digestHash,
 							Namespace: "ns-a",
-							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue, bundleUnpackRefLabel: digestHash},
 							OwnerReferences: []metav1.OwnerReference{
 								{
 									APIVersion:         "v1",
@@ -978,7 +978,7 @@ func TestConfigMapUnpacker(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name:      pathHash,
 							Namespace: "ns-a",
-							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue, bundleUnpackRefLabel: pathHash},
 							OwnerReferences: []metav1.OwnerReference{
 								{
 									APIVersion:         "v1",
@@ -1213,7 +1213,7 @@ func TestConfigMapUnpacker(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name:      pathHash,
 							Namespace: "ns-a",
-							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue, bundleUnpackRefLabel: pathHash},
 							OwnerReferences: []metav1.OwnerReference{
 								{
 									APIVersion:         "v1",
@@ -1459,7 +1459,7 @@ func TestConfigMapUnpacker(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name:      pathHash,
 							Namespace: "ns-a",
-							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+							Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue, bundleUnpackRefLabel: pathHash},
 							OwnerReferences: []metav1.OwnerReference{
 								{
 									APIVersion:         "v1",
@@ -1673,7 +1673,7 @@ func TestConfigMapUnpacker(t *testing.T) {
 			)
 			require.NoError(t, err)
 
-			res, err := unpacker.UnpackBundle(tt.args.lookup, tt.args.annotationTimeout)
+			res, err := unpacker.UnpackBundle(tt.args.lookup, tt.args.annotationTimeout, 0)
 			require.Equal(t, tt.expected.err, err)
 
 			if tt.expected.res == nil {
@@ -1839,3 +1839,213 @@ func TestOperatorGroupBundleUnpackTimeout(t *testing.T) {
 		})
 	}
 }
+
+func TestOperatorGroupBundleUnpackRetryInterval(t *testing.T) {
+	nsName := "fake-ns"
+
+	for _, tc := range []struct {
+		name            string
+		operatorGroups  []*operatorsv1.OperatorGroup
+		expectedTimeout time.Duration
+		expectedError   error
+	}{
+		{
+			name:            "No operator groups exist",
+			expectedTimeout: 0,
+			expectedError:   errors.New("found 0 operatorGroups, expected 1"),
+		},
+		{
+			name: "Multiple operator groups exist",
+			operatorGroups: []*operatorsv1.OperatorGroup{
+				{
+					TypeMeta: metav1.TypeMeta{
+						Kind:       operatorsv1.OperatorGroupKind,
+						APIVersion: operatorsv1.GroupVersion.String(),
+					},
+					ObjectMeta: metav1.ObjectMeta{
+						Name:      "og1",
+						Namespace: nsName,
+					},
+				},
+				{
+					TypeMeta: metav1.TypeMeta{
+						Kind:       operatorsv1.OperatorGroupKind,
+						APIVersion: operatorsv1.GroupVersion.String(),
+					},
+					ObjectMeta: metav1.ObjectMeta{
+						Name:      "og2",
+						Namespace: nsName,
+					},
+				},
+			},
+			expectedTimeout: 0,
+			expectedError:   errors.New("found 2 operatorGroups, expected 1"),
+		},
+		{
+			name: "One operator group exists with valid unpack retry annotation",
+			operatorGroups: []*operatorsv1.OperatorGroup{
+				{
+					TypeMeta: metav1.TypeMeta{
+						Kind:       operatorsv1.OperatorGroupKind,
+						APIVersion: operatorsv1.GroupVersion.String(),
+					},
+					ObjectMeta: metav1.ObjectMeta{
+						Name:        "og",
+						Namespace:   nsName,
+						Annotations: map[string]string{BundleUnpackRetryMinimumIntervalAnnotationKey: "1m"},
+					},
+				},
+			},
+			expectedTimeout: 1 * time.Minute,
+			expectedError:   nil,
+		},
+		{
+			name: "One operator group exists with no unpack retry annotation",
+			operatorGroups: []*operatorsv1.OperatorGroup{
+				{
+					TypeMeta: metav1.TypeMeta{
+						Kind:       operatorsv1.OperatorGroupKind,
+						APIVersion: operatorsv1.GroupVersion.String(),
+					},
+					ObjectMeta: metav1.ObjectMeta{
+						Name:      "og",
+						Namespace: nsName,
+					},
+				},
+			},
+			expectedTimeout: 0,
+			expectedError:   nil,
+		},
+		{
+			name: "One operator group exists with invalid unpack retry annotation",
+			operatorGroups: []*operatorsv1.OperatorGroup{
+				{
+					TypeMeta: metav1.TypeMeta{
+						Kind:       operatorsv1.OperatorGroupKind,
+						APIVersion: operatorsv1.GroupVersion.String(),
+					},
+					ObjectMeta: metav1.ObjectMeta{
+						Name:        "og",
+						Namespace:   nsName,
+						Annotations: map[string]string{BundleUnpackRetryMinimumIntervalAnnotationKey: "invalid"},
+					},
+				},
+			},
+			expectedTimeout: 0,
+			expectedError:   fmt.Errorf("failed to parse unpack retry annotation(operatorframework.io/bundle-unpack-min-retry-interval: invalid): %w", errors.New("time: invalid duration \"invalid\"")),
+		},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			ogIndexer := cache.NewIndexer(cache.MetaNamespaceKeyFunc, cache.Indexers{})
+			ogLister := v1listers.NewOperatorGroupLister(ogIndexer).OperatorGroups(nsName)
+
+			for _, og := range tc.operatorGroups {
+				err := ogIndexer.Add(og)
+				assert.NoError(t, err)
+			}
+
+			timeout, err := OperatorGroupBundleUnpackRetryInterval(ogLister)
+
+			assert.Equal(t, tc.expectedTimeout, timeout)
+			assert.Equal(t, tc.expectedError, err)
+		})
+	}
+}
+
+func TestSortUnpackJobs(t *testing.T) {
+	// if there is a non-failed job, it should be first
+	// otherwise, the latest job should be first
+	//first n-1 jobs and oldest job are preserved
+	testJob := func(name string, failed bool, ts int64) *batchv1.Job {
+		conditions := []batchv1.JobCondition{}
+		if failed {
+			conditions = append(conditions, batchv1.JobCondition{
+				Type:               batchv1.JobFailed,
+				Status:             corev1.ConditionTrue,
+				LastTransitionTime: metav1.Time{Time: time.Unix(ts, 0)},
+			})
+		}
+		return &batchv1.Job{
+			ObjectMeta: metav1.ObjectMeta{
+				Name:   name,
+				Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue, bundleUnpackRefLabel: "test"},
+			},
+			Status: batchv1.JobStatus{
+				Conditions: conditions,
+			},
+		}
+	}
+	failedJobs := []*batchv1.Job{
+		testJob("f-1", true, 1),
+		testJob("f-2", true, 2),
+		testJob("f-3", true, 3),
+		testJob("f-4", true, 4),
+		testJob("f-5", true, 5),
+	}
+	nonFailedJob := testJob("s-1", false, 1)
+	for _, tc := range []struct {
+		name             string
+		jobs             []*batchv1.Job
+		maxRetained      int
+		expectedLatest   *batchv1.Job
+		expectedToDelete []*batchv1.Job
+	}{
+		{
+			name:        "no job history",
+			maxRetained: 0,
+			jobs: []*batchv1.Job{
+				failedJobs[1],
+				failedJobs[2],
+				failedJobs[0],
+			},
+			expectedLatest: failedJobs[2],
+			expectedToDelete: []*batchv1.Job{
+				failedJobs[1],
+				failedJobs[0],
+			},
+		}, {
+			name:        "empty job list",
+			maxRetained: 1,
+		}, {
+			name:        "retain oldest",
+			maxRetained: 1,
+			jobs: []*batchv1.Job{
+				failedJobs[2],
+				failedJobs[0],
+				failedJobs[1],
+			},
+			expectedToDelete: []*batchv1.Job{
+				failedJobs[1],
+			},
+			expectedLatest: failedJobs[2],
+		}, {
+			name:        "multiple old jobs",
+			maxRetained: 2,
+			jobs: []*batchv1.Job{
+				failedJobs[1],
+				failedJobs[0],
+				failedJobs[2],
+				failedJobs[3],
+				failedJobs[4],
+			},
+			expectedLatest: failedJobs[4],
+			expectedToDelete: []*batchv1.Job{
+				failedJobs[1],
+				failedJobs[2],
+			},
+		}, {
+			name:        "select non-failed as latest",
+			maxRetained: 3,
+			jobs: []*batchv1.Job{
+				failedJobs[0],
+				failedJobs[1],
+				nonFailedJob,
+			},
+			expectedLatest: nonFailedJob,
+		},
+	} {
+		latest, toDelete := sortUnpackJobs(tc.jobs, tc.maxRetained)
+		assert.Equal(t, tc.expectedLatest, latest)
+		assert.ElementsMatch(t, tc.expectedToDelete, toDelete)
+	}
+}
diff --git a/pkg/controller/install/deployment_test.go b/pkg/controller/install/deployment_test.go
index d9ec66cc2..cceea7deb 100644
--- a/pkg/controller/install/deployment_test.go
+++ b/pkg/controller/install/deployment_test.go
@@ -353,6 +353,7 @@ func TestInstallStrategyDeploymentCheckInstallErrors(t *testing.T) {
 			dep.Spec.Template.SetAnnotations(map[string]string{"test": "annotation"})
 			dep.Spec.RevisionHistoryLimit = &revisionHistoryLimit
 			dep.SetLabels(labels.CloneAndAddLabel(dep.ObjectMeta.GetLabels(), DeploymentSpecHashLabelKey, HashDeploymentSpec(dep.Spec)))
+			dep.Labels[OLMManagedLabelKey] = OLMManagedLabelValue
 			dep.Status.Conditions = append(dep.Status.Conditions, appsv1.DeploymentCondition{
 				Type:   appsv1.DeploymentAvailable,
 				Status: corev1.ConditionTrue,
diff --git a/pkg/controller/operators/adoption_controller_test.go b/pkg/controller/operators/adoption_controller_test.go
index 271e75477..2fd4cb651 100644
--- a/pkg/controller/operators/adoption_controller_test.go
+++ b/pkg/controller/operators/adoption_controller_test.go
@@ -6,6 +6,7 @@ import (
 
 	. "github.com/onsi/ginkgo/v2"
 	. "github.com/onsi/gomega"
+	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/install"
 	appsv1 "k8s.io/api/apps/v1"
 	corev1 "k8s.io/api/core/v1"
 	rbacv1 "k8s.io/api/rbac/v1"
@@ -350,6 +351,12 @@ var _ = Describe("Adoption Controller", func() {
 								),
 							}
 							for _, component := range components {
+								labels := component.GetLabels()
+								if labels == nil {
+									labels = map[string]string{}
+								}
+								labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
+								component.SetLabels(labels)
 								Eventually(func() error {
 									return k8sClient.Create(ctx, component)
 								}, timeout, interval).Should(Succeed())
diff --git a/pkg/controller/operators/catalog/subscriptions_test.go b/pkg/controller/operators/catalog/subscriptions_test.go
index b84d24c9e..94fefe61e 100644
--- a/pkg/controller/operators/catalog/subscriptions_test.go
+++ b/pkg/controller/operators/catalog/subscriptions_test.go
@@ -1172,7 +1172,7 @@ func TestSyncSubscriptions(t *testing.T) {
 			defer cancel()
 
 			fakeBundleUnpacker := &bundlefakes.FakeUnpacker{
-				UnpackBundleStub: func(lookup *v1alpha1.BundleLookup, timeout time.Duration) (*bundle.BundleUnpackResult, error) {
+				UnpackBundleStub: func(lookup *v1alpha1.BundleLookup, timeout, retryInterval time.Duration) (*bundle.BundleUnpackResult, error) {
 					return &bundle.BundleUnpackResult{BundleLookup: lookup.DeepCopy()}, tt.fields.unpackBundleErr
 				},
 			}
diff --git a/pkg/controller/operators/olm/operator_test.go b/pkg/controller/operators/olm/operator_test.go
index 99dca3fd9..e6dbe5014 100644
--- a/pkg/controller/operators/olm/operator_test.go
+++ b/pkg/controller/operators/olm/operator_test.go
@@ -383,6 +383,7 @@ func deployment(deploymentName, namespace, serviceAccountName string, templateAn
 		ObjectMeta: metav1.ObjectMeta{
 			Name:      deploymentName,
 			Namespace: namespace,
+			Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 		},
 		Spec: appsv1.DeploymentSpec{
 			Selector: &metav1.LabelSelector{
@@ -432,6 +433,7 @@ func serviceAccount(name, namespace string) *corev1.ServiceAccount {
 		ObjectMeta: metav1.ObjectMeta{
 			Name:      name,
 			Namespace: namespace,
+			Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 		},
 	}
 
@@ -440,6 +442,9 @@ func serviceAccount(name, namespace string) *corev1.ServiceAccount {
 
 func service(name, namespace, deploymentName string, targetPort int, ownerReferences ...metav1.OwnerReference) *corev1.Service {
 	service := &corev1.Service{
+		ObjectMeta: metav1.ObjectMeta{
+			Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+		},
 		Spec: corev1.ServiceSpec{
 			Ports: []corev1.ServicePort{
 				{
@@ -461,6 +466,9 @@ func service(name, namespace, deploymentName string, targetPort int, ownerRefere
 
 func clusterRoleBinding(name, clusterRoleName, serviceAccountName, serviceAccountNamespace string) *rbacv1.ClusterRoleBinding {
 	clusterRoleBinding := &rbacv1.ClusterRoleBinding{
+		ObjectMeta: metav1.ObjectMeta{
+			Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+		},
 		Subjects: []rbacv1.Subject{
 			{
 				Kind:      "ServiceAccount",
@@ -482,6 +490,9 @@ func clusterRoleBinding(name, clusterRoleName, serviceAccountName, serviceAccoun
 
 func clusterRole(name string, rules []rbacv1.PolicyRule) *rbacv1.ClusterRole {
 	clusterRole := &rbacv1.ClusterRole{
+		ObjectMeta: metav1.ObjectMeta{
+			Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+		},
 		Rules: rules,
 	}
 	clusterRole.SetName(name)
@@ -491,6 +502,9 @@ func clusterRole(name string, rules []rbacv1.PolicyRule) *rbacv1.ClusterRole {
 
 func role(name, namespace string, rules []rbacv1.PolicyRule) *rbacv1.Role {
 	role := &rbacv1.Role{
+		ObjectMeta: metav1.ObjectMeta{
+			Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+		},
 		Rules: rules,
 	}
 	role.SetName(name)
@@ -501,6 +515,9 @@ func role(name, namespace string, rules []rbacv1.PolicyRule) *rbacv1.Role {
 
 func roleBinding(name, namespace, roleName, serviceAccountName, serviceAccountNamespace string) *rbacv1.RoleBinding {
 	roleBinding := &rbacv1.RoleBinding{
+		ObjectMeta: metav1.ObjectMeta{
+			Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+		},
 		Subjects: []rbacv1.Subject{
 			{
 				Kind:      "ServiceAccount",
@@ -523,6 +540,9 @@ func roleBinding(name, namespace, roleName, serviceAccountName, serviceAccountNa
 
 func tlsSecret(name, namespace string, certPEM, privPEM []byte) *corev1.Secret {
 	secret := &corev1.Secret{
+		ObjectMeta: metav1.ObjectMeta{
+			Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
+		},
 		Data: map[string][]byte{
 			"tls.crt": certPEM,
 			"tls.key": privPEM,
@@ -846,7 +866,8 @@ func apiService(group, version, serviceName, serviceNamespace, deploymentName st
 func crd(name, version, group string) *apiextensionsv1.CustomResourceDefinition {
 	return &apiextensionsv1.CustomResourceDefinition{
 		ObjectMeta: metav1.ObjectMeta{
-			Name: name + "." + group,
+			Name:   name + "." + group,
+			Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 		},
 		Spec: apiextensionsv1.CustomResourceDefinitionSpec{
 			Group: group,
@@ -4343,6 +4364,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 	annotatedDeployment := ownedDeployment.DeepCopy()
 	annotatedDeployment.Spec.Template.SetAnnotations(map[string]string{operatorsv1.OperatorGroupTargetsAnnotationKey: operatorNamespace + "," + targetNamespace, operatorsv1.OperatorGroupAnnotationKey: "operator-group-1", operatorsv1.OperatorGroupNamespaceAnnotationKey: operatorNamespace})
 	annotatedDeployment.SetLabels(map[string]string{
+		"olm.managed":                      "true",
 		"olm.owner":                        "csv1",
 		"olm.owner.namespace":              "operator-ns",
 		"olm.owner.kind":                   "ClusterServiceVersion",
@@ -4352,6 +4374,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 	annotatedGlobalDeployment := ownedDeployment.DeepCopy()
 	annotatedGlobalDeployment.Spec.Template.SetAnnotations(map[string]string{operatorsv1.OperatorGroupTargetsAnnotationKey: "", operatorsv1.OperatorGroupAnnotationKey: "operator-group-1", operatorsv1.OperatorGroupNamespaceAnnotationKey: operatorNamespace})
 	annotatedGlobalDeployment.SetLabels(map[string]string{
+		"olm.managed":                      "true",
 		"olm.owner":                        "csv1",
 		"olm.owner.namespace":              "operator-ns",
 		"olm.owner.kind":                   "ClusterServiceVersion",
@@ -4371,6 +4394,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 		},
 		Rules: permissions[0].Rules,
 	}
+	role.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 
 	roleBinding := &rbacv1.RoleBinding{
 		TypeMeta: metav1.TypeMeta{
@@ -4397,6 +4421,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 			Name:     role.GetName(),
 		},
 	}
+	roleBinding.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 
 	type initial struct {
 		operatorGroup *operatorsv1.OperatorGroup
@@ -4562,6 +4587,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.admin-8rdAjL0E35JMMAkOqYmoorzjpIIihfnj3DcgDU",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4572,6 +4598,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.edit-9lBEUxqAYE7CX7wZfFEPYutTfQTo43WarB08od",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4582,6 +4609,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.view-1l6ymczPK5SceF4d0DCtAnWZuvmKn6s8oBUxHr",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4623,6 +4651,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "operator-group-1-admin",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4633,6 +4662,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "operator-group-1-view",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4643,6 +4673,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "operator-group-1-edit",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4661,6 +4692,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.admin-8rdAjL0E35JMMAkOqYmoorzjpIIihfnj3DcgDU",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4671,6 +4703,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.edit-9lBEUxqAYE7CX7wZfFEPYutTfQTo43WarB08od",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4681,6 +4714,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.view-1l6ymczPK5SceF4d0DCtAnWZuvmKn6s8oBUxHr",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4691,6 +4725,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "operator-group-1-admin",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4701,6 +4736,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "operator-group-1-view",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4711,6 +4747,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "operator-group-1-edit",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4752,6 +4789,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.admin-8rdAjL0E35JMMAkOqYmoorzjpIIihfnj3DcgDU",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns-bob",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4763,6 +4801,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.view-1l6ymczPK5SceF4d0DCtAnWZuvmKn6s8oBUxHr",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-5",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4775,6 +4814,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.edit-9lBEUxqAYE7CX7wZfFEPYutTfQTo43WarB08od",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroupKind",
@@ -4793,6 +4833,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.admin-8rdAjL0E35JMMAkOqYmoorzjpIIihfnj3DcgDU",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4804,6 +4845,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.edit-9lBEUxqAYE7CX7wZfFEPYutTfQTo43WarB08od",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4814,6 +4856,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.view-1l6ymczPK5SceF4d0DCtAnWZuvmKn6s8oBUxHr",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4857,6 +4900,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.admin-8rdAjL0E35JMMAkOqYmoorzjpIIihfnj3DcgDU",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4867,6 +4911,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.edit-9lBEUxqAYE7CX7wZfFEPYutTfQTo43WarB08od",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4877,6 +4922,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.view-1l6ymczPK5SceF4d0DCtAnWZuvmKn6s8oBUxHr",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4894,6 +4940,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.admin-8rdAjL0E35JMMAkOqYmoorzjpIIihfnj3DcgDU",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4904,6 +4951,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.edit-9lBEUxqAYE7CX7wZfFEPYutTfQTo43WarB08od",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4914,6 +4962,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "olm.og.operator-group-1.view-1l6ymczPK5SceF4d0DCtAnWZuvmKn6s8oBUxHr",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "operator-group-1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "OperatorGroup",
@@ -4985,6 +5034,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 							Name:            "csv-role",
 							Namespace:       targetNamespace,
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.copiedFrom":      "operator-ns",
 								"olm.owner":           "csv1",
 								"olm.owner.namespace": "target-ns",
@@ -5006,6 +5056,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 							Name:            "csv-rolebinding",
 							Namespace:       targetNamespace,
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.copiedFrom":      "operator-ns",
 								"olm.owner":           "csv1",
 								"olm.owner.namespace": "target-ns",
@@ -5088,6 +5139,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 							Name:            "csv-role",
 							Namespace:       targetNamespace,
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.copiedFrom":      "operator-ns",
 								"olm.owner":           "csv1",
 								"olm.owner.namespace": "target-ns",
@@ -5109,6 +5161,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 							Name:            "csv-rolebinding",
 							Namespace:       targetNamespace,
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.copiedFrom":      "operator-ns",
 								"olm.owner":           "csv1",
 								"olm.owner.namespace": "target-ns",
@@ -5188,6 +5241,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "csv-role",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "csv1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "ClusterServiceVersion",
@@ -5207,6 +5261,7 @@ func TestSyncOperatorGroups(t *testing.T) {
 						ObjectMeta: metav1.ObjectMeta{
 							Name: "csv-rolebinding",
 							Labels: map[string]string{
+								"olm.managed":         "true",
 								"olm.owner":           "csv1",
 								"olm.owner.namespace": "operator-ns",
 								"olm.owner.kind":      "ClusterServiceVersion",
@@ -5507,17 +5562,14 @@ func TestSyncOperatorGroups(t *testing.T) {
 				}
 
 				for _, csv := range csvs.Items {
-					t.Logf("%s/%s", csv.Namespace, csv.Name)
 					if csv.Status.Phase == v1alpha1.CSVPhaseInstalling {
 						simulateSuccessfulRollout(&csv)
 					}
 
-					t.Log("op.syncClusterServiceVersion")
 					if err := op.syncClusterServiceVersion(&csv); err != nil {
 						return false, fmt.Errorf("failed to syncClusterServiceVersion: %w", err)
 					}
 
-					t.Log("op.syncCopyCSV")
 					if err := op.syncCopyCSV(&csv); err != nil && !tt.ignoreCopyError {
 						return false, fmt.Errorf("failed to syncCopyCSV: %w", err)
 					}
diff --git a/pkg/controller/operators/olm/requirements_test.go b/pkg/controller/operators/olm/requirements_test.go
index e3712d695..32239963e 100644
--- a/pkg/controller/operators/olm/requirements_test.go
+++ b/pkg/controller/operators/olm/requirements_test.go
@@ -5,6 +5,7 @@ import (
 	"fmt"
 	"testing"
 
+	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/install"
 	"github.com/stretchr/testify/require"
 	corev1 "k8s.io/api/core/v1"
 	rbacv1 "k8s.io/api/rbac/v1"
@@ -80,6 +81,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 						Name:      "sa",
 						Namespace: namespace,
 						UID:       types.UID("sa"),
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 						OwnerReferences: []metav1.OwnerReference{
 							{
 								Kind: v1alpha1.ClusterServiceVersionKind,
@@ -92,6 +94,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "role",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Rules: []rbacv1.PolicyRule{
 						{
@@ -105,6 +108,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "roleBinding",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Subjects: []rbacv1.Subject{
 						{
@@ -122,7 +126,8 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 				},
 				&rbacv1.ClusterRole{
 					ObjectMeta: metav1.ObjectMeta{
-						Name: "clusterRole",
+						Name:   "clusterRole",
+						Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Rules: []rbacv1.PolicyRule{
 						{
@@ -133,7 +138,8 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 				},
 				&rbacv1.ClusterRoleBinding{
 					ObjectMeta: metav1.ObjectMeta{
-						Name: "clusterRoleBinding",
+						Name:   "clusterRoleBinding",
+						Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Subjects: []rbacv1.Subject{
 						{
@@ -224,6 +230,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 						Name:      "sa",
 						Namespace: namespace,
 						UID:       types.UID("sa"),
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 						OwnerReferences: []metav1.OwnerReference{
 							{
 								Kind: v1alpha1.ClusterServiceVersionKind,
@@ -236,6 +243,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "role",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Rules: []rbacv1.PolicyRule{
 						{
@@ -249,6 +257,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "roleBinding",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Subjects: []rbacv1.Subject{
 						{
@@ -266,7 +275,8 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 				},
 				&rbacv1.ClusterRole{
 					ObjectMeta: metav1.ObjectMeta{
-						Name: "clusterRole",
+						Name:   "clusterRole",
+						Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Rules: []rbacv1.PolicyRule{
 						{
@@ -277,7 +287,8 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 				},
 				&rbacv1.ClusterRoleBinding{
 					ObjectMeta: metav1.ObjectMeta{
-						Name: "clusterRoleBinding",
+						Name:   "clusterRoleBinding",
+						Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Subjects: []rbacv1.Subject{
 						{
@@ -368,6 +379,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 						Name:      "sa",
 						Namespace: namespace,
 						UID:       types.UID("sa"),
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 						OwnerReferences: []metav1.OwnerReference{
 							{
 								Kind: v1alpha1.ClusterServiceVersionKind,
@@ -380,6 +392,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "role",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Rules: []rbacv1.PolicyRule{
 						{
@@ -393,6 +406,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "roleBinding",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Subjects: []rbacv1.Subject{
 						{
@@ -410,7 +424,8 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 				},
 				&rbacv1.ClusterRole{
 					ObjectMeta: metav1.ObjectMeta{
-						Name: "clusterRole",
+						Name:   "clusterRole",
+						Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Rules: []rbacv1.PolicyRule{
 						{
@@ -421,7 +436,8 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 				},
 				&rbacv1.ClusterRoleBinding{
 					ObjectMeta: metav1.ObjectMeta{
-						Name: "clusterRoleBinding",
+						Name:   "clusterRoleBinding",
+						Labels: map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Subjects: []rbacv1.Subject{
 						{
@@ -491,6 +507,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 						Name:      "sa",
 						Namespace: namespace,
 						UID:       types.UID("sa"),
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 						OwnerReferences: []metav1.OwnerReference{
 							{
 								Kind: v1alpha1.ClusterServiceVersionKind,
@@ -503,6 +520,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "role",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Rules: []rbacv1.PolicyRule{
 						{
@@ -516,6 +534,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "roleBinding",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 					},
 					Subjects: []rbacv1.Subject{
 						{
@@ -770,6 +789,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 						Name:      "sa",
 						Namespace: namespace,
 						UID:       types.UID("sa"),
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 						OwnerReferences: []metav1.OwnerReference{
 							{
 								Kind: v1alpha1.ClusterServiceVersionKind,
@@ -824,6 +844,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 						Name:      "sa",
 						Namespace: namespace,
 						UID:       types.UID("sa"),
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 						OwnerReferences: []metav1.OwnerReference{
 							{
 								Kind: v1alpha1.SubscriptionKind, // arbitrary non-CSV kind
@@ -877,6 +898,7 @@ func TestRequirementAndPermissionStatus(t *testing.T) {
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "sa",
 						Namespace: namespace,
+						Labels:    map[string]string{install.OLMManagedLabelKey: install.OLMManagedLabelValue},
 						UID:       types.UID("sa"),
 					},
 				},
diff --git a/pkg/controller/operators/openshift/suite_test.go b/pkg/controller/operators/openshift/suite_test.go
index da1043978..e639720b5 100644
--- a/pkg/controller/operators/openshift/suite_test.go
+++ b/pkg/controller/operators/openshift/suite_test.go
@@ -108,5 +108,5 @@ var _ = BeforeSuite(func() {
 var _ = AfterSuite(func() {
 	By("tearing down the test environment")
 	close(syncCh)
-	Expect(testEnv.Stop()).To(Succeed())
+	testEnv.Stop()
 })
diff --git a/pkg/controller/operators/suite_test.go b/pkg/controller/operators/suite_test.go
index 2ddfb18ae..c75cd69b9 100644
--- a/pkg/controller/operators/suite_test.go
+++ b/pkg/controller/operators/suite_test.go
@@ -162,8 +162,7 @@ var _ = AfterSuite(func() {
 	ctx.Done()
 
 	By("tearing down the test environment")
-	err := testEnv.Stop()
-	Expect(err).ToNot(HaveOccurred())
+	testEnv.Stop()
 })
 
 func newOperator(name string) *decorators.Operator {
diff --git a/test/e2e/fbc_provider.go b/test/e2e/fbc_provider.go
index 178718744..362ef19c3 100644
--- a/test/e2e/fbc_provider.go
+++ b/test/e2e/fbc_provider.go
@@ -31,3 +31,9 @@ func NewFileBasedFiledBasedCatalogProvider(path string) (FileBasedCatalogProvide
 func (f *fileBasedFileBasedCatalogProvider) GetCatalog() string {
 	return f.fbc
 }
+
+func NewRawFileBasedCatalogProvider(data string) (FileBasedCatalogProvider, error) {
+	return &fileBasedFileBasedCatalogProvider{
+		fbc: string(data),
+	}, nil
+}
diff --git a/test/e2e/installplan_e2e_test.go b/test/e2e/installplan_e2e_test.go
index 2de113cb2..c8aa2f758 100644
--- a/test/e2e/installplan_e2e_test.go
+++ b/test/e2e/installplan_e2e_test.go
@@ -3638,11 +3638,9 @@ var _ = Describe("Install Plan", func() {
 		}
 		Expect(ctx.Ctx().Client().Status().Update(context.Background(), newPlan)).To(Succeed())
 
-		newKey := client.ObjectKeyFromObject(newPlan)
-
-		Eventually(func() (*operatorsv1alpha1.InstallPlan, error) {
-			return newPlan, ctx.Ctx().Client().Get(context.Background(), newKey, newPlan)
-		}).Should(HavePhase(operatorsv1alpha1.InstallPlanPhaseFailed))
+		ipPhaseCheckerFunc := buildInstallPlanMessageCheckFunc(`cannot create resource "services" in API group`)
+		_, err = fetchInstallPlanWithNamespace(GinkgoT(), crc, newPlan.Name, newPlan.Namespace, ipPhaseCheckerFunc)
+		require.NoError(GinkgoT(), err)
 
 		Expect(client.IgnoreNotFound(ctx.Ctx().Client().Delete(context.Background(), &crd))).To(Succeed())
 		Eventually(func() error {
@@ -3928,6 +3926,19 @@ func validateCRDVersions(t GinkgoTInterface, c operatorclient.ClientInterface, n
 	require.Equal(t, 0, len(expectedVersions), "Actual CRD versions do not match expected")
 }
 
+func buildInstallPlanMessageCheckFunc(substring string) checkInstallPlanFunc {
+	var lastMessage string
+	lastTime := time.Now()
+	return func(fip *operatorsv1alpha1.InstallPlan) bool {
+		if fip.Status.Message != lastMessage {
+			ctx.Ctx().Logf("waiting %s for installplan %s/%s to have message substring %q, have message %q", time.Since(lastTime), fip.Namespace, fip.Name, substring, fip.Status.Message)
+			lastMessage = fip.Status.Message
+			lastTime = time.Now()
+		}
+		return strings.Contains(fip.Status.Message, substring)
+	}
+}
+
 func buildInstallPlanPhaseCheckFunc(phases ...operatorsv1alpha1.InstallPlanPhase) checkInstallPlanFunc {
 	var lastPhase operatorsv1alpha1.InstallPlanPhase
 	lastTime := time.Now()
diff --git a/test/e2e/operator_groups_e2e_test.go b/test/e2e/operator_groups_e2e_test.go
index 940b06e5e..a3575fba5 100644
--- a/test/e2e/operator_groups_e2e_test.go
+++ b/test/e2e/operator_groups_e2e_test.go
@@ -3,6 +3,7 @@ package e2e
 import (
 	"context"
 	"fmt"
+	"os"
 	"strings"
 	"time"
 
@@ -1420,19 +1421,31 @@ var _ = Describe("Operator Group", func() {
 				Name: role.GetName(),
 			},
 		}
-		_, err = c.CreateServiceAccount(serviceAccount)
+		serviceAccount, err = c.CreateServiceAccount(serviceAccount)
 		require.NoError(GinkgoT(), err)
 		defer func() {
+			if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+				fmt.Printf("Skipping cleanup of serviceaccount %s/%s...\n", serviceAccount.GetNamespace(), serviceAccount.GetName())
+				return
+			}
 			c.DeleteServiceAccount(serviceAccount.GetNamespace(), serviceAccount.GetName(), metav1.NewDeleteOptions(0))
 		}()
 		createdRole, err := c.CreateRole(role)
 		require.NoError(GinkgoT(), err)
 		defer func() {
+			if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+				fmt.Printf("Skipping cleanup of role %s/%s...\n", role.GetNamespace(), role.GetName())
+				return
+			}
 			c.DeleteRole(role.GetNamespace(), role.GetName(), metav1.NewDeleteOptions(0))
 		}()
 		createdRoleBinding, err := c.CreateRoleBinding(roleBinding)
 		require.NoError(GinkgoT(), err)
 		defer func() {
+			if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+				fmt.Printf("Skipping cleanup of role binding %s/%s...\n", roleBinding.GetNamespace(), roleBinding.GetName())
+				return
+			}
 			c.DeleteRoleBinding(roleBinding.GetNamespace(), roleBinding.GetName(), metav1.NewDeleteOptions(0))
 		}()
 		// Create a new NamedInstallStrategy
@@ -1448,11 +1461,13 @@ var _ = Describe("Operator Group", func() {
 
 		err = ownerutil.AddOwnerLabels(createdRole, createdCSV)
 		require.NoError(GinkgoT(), err)
+		createdRole.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 		_, err = c.UpdateRole(createdRole)
 		require.NoError(GinkgoT(), err)
 
 		err = ownerutil.AddOwnerLabels(createdRoleBinding, createdCSV)
 		require.NoError(GinkgoT(), err)
+		createdRoleBinding.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 		_, err = c.UpdateRoleBinding(createdRoleBinding)
 		require.NoError(GinkgoT(), err)
 		GinkgoT().Log("wait for CSV to succeed")
@@ -1903,16 +1918,28 @@ var _ = Describe("Operator Group", func() {
 		_, err = c.CreateServiceAccount(serviceAccount)
 		require.NoError(GinkgoT(), err)
 		defer func() {
+			if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+				fmt.Printf("Skipping cleanup of serviceaccount %s/%s...\n", serviceAccount.GetNamespace(), serviceAccount.GetName())
+				return
+			}
 			c.DeleteServiceAccount(serviceAccount.GetNamespace(), serviceAccount.GetName(), metav1.NewDeleteOptions(0))
 		}()
 		createdRole, err := c.CreateRole(role)
 		require.NoError(GinkgoT(), err)
 		defer func() {
+			if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+				fmt.Printf("Skipping cleanup of role %s/%s...\n", role.GetNamespace(), role.GetName())
+				return
+			}
 			c.DeleteRole(role.GetNamespace(), role.GetName(), metav1.NewDeleteOptions(0))
 		}()
 		createdRoleBinding, err := c.CreateRoleBinding(roleBinding)
 		require.NoError(GinkgoT(), err)
 		defer func() {
+			if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+				fmt.Printf("Skipping cleanup of role binding %s/%s...\n", roleBinding.GetNamespace(), roleBinding.GetName())
+				return
+			}
 			c.DeleteRoleBinding(roleBinding.GetNamespace(), roleBinding.GetName(), metav1.NewDeleteOptions(0))
 		}()
 		// Create a new NamedInstallStrategy
@@ -1928,11 +1955,13 @@ var _ = Describe("Operator Group", func() {
 
 		err = ownerutil.AddOwnerLabels(createdRole, createdCSV)
 		require.NoError(GinkgoT(), err)
+		createdRole.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 		_, err = c.UpdateRole(createdRole)
 		require.NoError(GinkgoT(), err)
 
 		err = ownerutil.AddOwnerLabels(createdRoleBinding, createdCSV)
 		require.NoError(GinkgoT(), err)
+		createdRoleBinding.Labels[install.OLMManagedLabelKey] = install.OLMManagedLabelValue
 		_, err = c.UpdateRoleBinding(createdRoleBinding)
 		require.NoError(GinkgoT(), err)
 		GinkgoT().Log("wait for CSV to succeed")
@@ -1950,6 +1979,7 @@ var _ = Describe("Operator Group", func() {
 			}
 			return true, nil
 		})
+		require.NoError(GinkgoT(), err)
 		require.EqualValues(GinkgoT(), append(role.Rules, rbacv1.PolicyRule{
 			Verbs:     []string{"get", "list", "watch"},
 			APIGroups: []string{""},
@@ -1966,6 +1996,7 @@ var _ = Describe("Operator Group", func() {
 			}
 			return true, nil
 		})
+		require.NoError(GinkgoT(), err)
 		require.EqualValues(GinkgoT(), roleBinding.Subjects, fetchedRoleBinding.Subjects)
 		require.EqualValues(GinkgoT(), roleBinding.RoleRef.Name, fetchedRoleBinding.RoleRef.Name)
 		require.EqualValues(GinkgoT(), "rbac.authorization.k8s.io", fetchedRoleBinding.RoleRef.APIGroup)
diff --git a/test/e2e/operator_test.go b/test/e2e/operator_test.go
index 6824adcdd..90caea0f1 100644
--- a/test/e2e/operator_test.go
+++ b/test/e2e/operator_test.go
@@ -3,12 +3,15 @@ package e2e
 import (
 	"context"
 	"fmt"
+	"os"
 	"time"
 
 	. "github.com/onsi/ginkgo/v2"
 	. "github.com/onsi/gomega"
 	"github.com/onsi/gomega/format"
 	gomegatypes "github.com/onsi/gomega/types"
+	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/install"
+	"github.com/stretchr/testify/require"
 	corev1 "k8s.io/api/core/v1"
 	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
@@ -74,6 +77,7 @@ var _ = Describe("Operator API", func() {
 	It("should surface components in its status", func() {
 		o := &operatorsv1.Operator{}
 		o.SetName(genName("o-"))
+		By(fmt.Sprintf("Creating an Operator resource %s", o.GetName()))
 
 		Consistently(o).ShouldNot(ContainCopiedCSVReferences())
 
@@ -83,6 +87,10 @@ var _ = Describe("Operator API", func() {
 
 		defer func() {
 			Eventually(func() error {
+				if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+					fmt.Printf("Skipping cleanup of operator %s...\n", o.GetName())
+					return nil
+				}
 				err := client.Delete(clientCtx, o)
 				if apierrors.IsNotFound(err) {
 					return nil
@@ -116,11 +124,11 @@ var _ = Describe("Operator API", func() {
 		}))
 		defer w.Stop()
 
-		// Create namespaces ns-a and ns-b
 		nsA := &corev1.Namespace{}
 		nsA.SetName(genName("ns-a-"))
 		nsB := &corev1.Namespace{}
 		nsB.SetName(genName("ns-b-"))
+		By(fmt.Sprintf("Create namespaces ns-a: (%s) and ns-b: (%s)", nsA.GetName(), nsB.GetName()))
 
 		for _, ns := range []*corev1.Namespace{nsA, nsB} {
 			Eventually(func() error {
@@ -128,6 +136,10 @@ var _ = Describe("Operator API", func() {
 			}).Should(Succeed())
 
 			defer func(n *corev1.Namespace) {
+				if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+					fmt.Printf("Skipping cleanup of namespace %s...\n", n.GetName())
+					return
+				}
 				Eventually(func() error {
 					err := client.Delete(clientCtx, n)
 					if apierrors.IsNotFound(err) {
@@ -138,24 +150,27 @@ var _ = Describe("Operator API", func() {
 			}(ns)
 		}
 
-		// Label ns-a with o's component label
+		By(fmt.Sprintf("Label ns-a (%s) with o's (%s) component label (%s)", nsA.GetName(), o.GetName(), expectedKey))
 		setComponentLabel := func(m metav1.Object) error {
-			m.SetLabels(map[string]string{expectedKey: ""})
+			m.SetLabels(map[string]string{
+				install.OLMManagedLabelKey: install.OLMManagedLabelValue,
+				expectedKey:                "",
+			})
 			return nil
 		}
 		Eventually(Apply(nsA, setComponentLabel)).Should(Succeed())
 
-		// Ensure o's status.components.refs field eventually contains a reference to ns-a
+		By("Ensure o's status.components.refs field eventually contains a reference to ns-a")
 		By("eventually listing a single component reference")
 		componentRefEventuallyExists(w, true, getReference(scheme, nsA))
 
-		// Create ServiceAccounts sa-a and sa-b in namespaces ns-a and ns-b respectively
 		saA := &corev1.ServiceAccount{}
 		saA.SetName(genName("sa-a-"))
 		saA.SetNamespace(nsA.GetName())
 		saB := &corev1.ServiceAccount{}
 		saB.SetName(genName("sa-b-"))
 		saB.SetNamespace(nsB.GetName())
+		By(fmt.Sprintf("Create ServiceAccounts sa-a (%s/%s) and sa-b (%s/%s) in namespaces ns-a and ns-b respectively", saA.GetNamespace(), saA.GetName(), saB.GetNamespace(), saB.GetName()))
 
 		for _, sa := range []*corev1.ServiceAccount{saA, saB} {
 			Eventually(func() error {
@@ -163,6 +178,10 @@ var _ = Describe("Operator API", func() {
 			}).Should(Succeed())
 			defer func(sa *corev1.ServiceAccount) {
 				Eventually(func() error {
+					if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+						fmt.Printf("Skipping cleanup of serviceaccount %s/%s...\n", sa.GetNamespace(), sa.GetName())
+						return nil
+					}
 					err := client.Delete(clientCtx, sa)
 					if apierrors.IsNotFound(err) {
 						return nil
@@ -172,26 +191,26 @@ var _ = Describe("Operator API", func() {
 			}(sa)
 		}
 
-		// Label sa-a and sa-b with o's component label
+		By("Label sa-a and sa-b with o's component label")
 		Eventually(Apply(saA, setComponentLabel)).Should(Succeed())
 		Eventually(Apply(saB, setComponentLabel)).Should(Succeed())
 
-		// Ensure o's status.components.refs field eventually contains references to sa-a and sa-b
+		By("Ensure o's status.components.refs field eventually contains references to sa-a and sa-b")
 		By("eventually listing multiple component references")
 		componentRefEventuallyExists(w, true, getReference(scheme, saA))
 		componentRefEventuallyExists(w, true, getReference(scheme, saB))
 
-		// Remove the component label from sa-b
+		By("Remove the component label from sa-b")
 		Eventually(Apply(saB, func(m metav1.Object) error {
 			m.SetLabels(nil)
 			return nil
 		})).Should(Succeed())
 
-		// Ensure the reference to sa-b is eventually removed from o's status.components.refs field
+		By("Ensure the reference to sa-b is eventually removed from o's status.components.refs field")
 		By("removing a component's reference when it no longer bears the component label")
 		componentRefEventuallyExists(w, false, getReference(scheme, saB))
 
-		// Delete o
+		By("Delete o")
 		Eventually(func() error {
 			err := client.Delete(clientCtx, o)
 			if err != nil && !apierrors.IsNotFound(err) {
@@ -200,13 +219,13 @@ var _ = Describe("Operator API", func() {
 			return nil
 		}).Should(Succeed())
 
-		// Ensure that o is eventually recreated (because some of its components still exist).
+		By("Ensure that o is eventually recreated (because some of its components still exist).")
 		By("recreating the Operator when any components still exist")
 		Eventually(func() error {
 			return client.Get(clientCtx, types.NamespacedName{Name: o.GetName()}, o)
 		}).Should(Succeed())
 
-		// Delete ns-a
+		By("Delete ns-a")
 		Eventually(func() error {
 			err := client.Delete(clientCtx, nsA)
 			if apierrors.IsNotFound(err) {
@@ -215,11 +234,11 @@ var _ = Describe("Operator API", func() {
 			return err
 		}).Should(Succeed())
 
-		// Ensure the reference to ns-a is eventually removed from o's status.components.refs field
+		By("Ensure the reference to ns-a is eventually removed from o's status.components.refs field")
 		By("removing a component's reference when it no longer exists")
 		componentRefEventuallyExists(w, false, getReference(scheme, nsA))
 
-		// Delete o
+		By("Delete o")
 		Eventually(func() error {
 			err := client.Delete(clientCtx, o)
 			if apierrors.IsNotFound(err) {
@@ -228,7 +247,7 @@ var _ = Describe("Operator API", func() {
 			return err
 		}).Should(Succeed())
 
-		// Ensure that o is consistently not found
+		By("Ensure that o is consistently not found")
 		By("verifying the Operator is permanently deleted if it has no components")
 		Consistently(func() error {
 			err := client.Get(clientCtx, types.NamespacedName{Name: o.GetName()}, o)
@@ -248,20 +267,22 @@ var _ = Describe("Operator API", func() {
 		)
 
 		BeforeEach(func() {
-			// Subscribe to a package and await a successful install
+			By("Subscribe to a package and await a successful install")
 			ns = &corev1.Namespace{}
 			ns.SetName(genName("ns-"))
 			Eventually(func() error {
 				return client.Create(clientCtx, ns)
 			}).Should(Succeed())
+			By(fmt.Sprintf("created namespace %s", ns.Name))
 
-			// Default to AllNamespaces
+			By("Default to AllNamespaces")
 			og := &operatorsv1.OperatorGroup{}
 			og.SetNamespace(ns.GetName())
 			og.SetName(genName("og-"))
 			Eventually(func() error {
 				return client.Create(clientCtx, og)
 			}).Should(Succeed())
+			By(fmt.Sprintf("created operator group %s/%s", og.Namespace, og.Name))
 
 			cs := &operatorsv1alpha1.CatalogSource{
 				Spec: operatorsv1alpha1.CatalogSourceSpec{
@@ -277,8 +298,9 @@ var _ = Describe("Operator API", func() {
 			Eventually(func() error {
 				return client.Create(clientCtx, cs)
 			}).Should(Succeed())
+			By(fmt.Sprintf("created catalog source %s/%s", cs.Namespace, cs.Name))
 
-			// Wait for the CatalogSource to be ready
+			By("Wait for the CatalogSource to be ready")
 			_, err := fetchCatalogSourceOnStatus(newCRClient(), cs.GetName(), cs.GetNamespace(), catalogSourceRegistryPodSynced())
 			Expect(err).ToNot(HaveOccurred())
 
@@ -296,38 +318,31 @@ var _ = Describe("Operator API", func() {
 			Eventually(func() error {
 				return client.Create(clientCtx, sub)
 			}).Should(Succeed())
+			By(fmt.Sprintf("created subscription %s/%s", sub.Namespace, sub.Name))
 
-			Eventually(func() (operatorsv1alpha1.SubscriptionState, error) {
-				s := sub.DeepCopy()
-				if err := client.Get(clientCtx, testobj.NamespacedName(s), s); err != nil {
-					return operatorsv1alpha1.SubscriptionStateNone, err
-				}
-
-				return s.Status.State, nil
-			}).Should(BeEquivalentTo(operatorsv1alpha1.SubscriptionStateAtLatest))
+			_, err = fetchSubscription(newCRClient(), sub.Namespace, sub.Name, subscriptionStateAtLatestChecker())
+			require.NoError(GinkgoT(), err)
 
-			var ipRef *corev1.ObjectReference
-			Eventually(func() (*corev1.ObjectReference, error) {
-				if err := client.Get(clientCtx, testobj.NamespacedName(sub), sub); err != nil {
-					return nil, err
-				}
-				ipRef = sub.Status.InstallPlanRef
+			subscriptionWithInstallPLan, err := fetchSubscription(newCRClient(), sub.Namespace, sub.Name, subscriptionHasInstallPlanChecker())
+			require.NoError(GinkgoT(), err)
+			require.NotNil(GinkgoT(), subscriptionWithInstallPLan)
+			ipRef := subscriptionWithInstallPLan.Status.InstallPlanRef
 
-				return ipRef, nil
-			}).ShouldNot(BeNil())
-
-			ip = &operatorsv1alpha1.InstallPlan{}
-			Eventually(func() error {
-				return client.Get(clientCtx, types.NamespacedName{Namespace: ipRef.Namespace, Name: ipRef.Name}, ip)
-			}).Should(Succeed())
+			ip, err = fetchInstallPlan(GinkgoT(), newCRClient(), ipRef.Name, ipRef.Namespace, buildInstallPlanPhaseCheckFunc(operatorsv1alpha1.InstallPlanPhaseComplete))
+			Expect(err).To(BeNil())
 
 			operator, err := operatorFactory.NewPackageOperator(sub.Spec.Package, sub.GetNamespace())
 			Expect(err).ToNot(HaveOccurred())
 			operatorName = testobj.NamespacedName(operator)
+			By(fmt.Sprintf("waiting for operator %s/%s to exist", operator.Namespace, operator.Name))
 		})
 
 		AfterEach(func() {
 			Eventually(func() error {
+				if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+					fmt.Printf("Skipping cleanup of namespace %s...\n", ns.Name)
+					return nil
+				}
 				err := client.Delete(clientCtx, ns)
 				if apierrors.IsNotFound(err) {
 					return nil
@@ -368,7 +383,7 @@ var _ = Describe("Operator API", func() {
 			var newNs *corev1.Namespace
 
 			BeforeEach(func() {
-				// Subscribe to a package and await a successful install
+				By("Subscribe to a package and await a successful install")
 				newNs = &corev1.Namespace{}
 				newNs.SetName(genName("ns-"))
 				Eventually(func() error {
diff --git a/test/e2e/subscription_e2e_test.go b/test/e2e/subscription_e2e_test.go
index a3421b12d..d212c39a5 100644
--- a/test/e2e/subscription_e2e_test.go
+++ b/test/e2e/subscription_e2e_test.go
@@ -4,6 +4,7 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
+	"os"
 	"path/filepath"
 	"reflect"
 	"strings"
@@ -2524,6 +2525,202 @@ var _ = Describe("Subscription", func() {
 			})
 		})
 	})
+	When("bundle unpack retries are enabled", func() {
+		It("should retry failing unpack jobs", func() {
+			if ok, err := inKind(c); ok && err == nil {
+				Skip("This spec fails when run using KIND cluster. See https://github.com/operator-framework/operator-lifecycle-manager/issues/2420 for more details")
+			} else if err != nil {
+				Skip("Could not determine whether running in a kind cluster. Skipping.")
+			}
+			By("Ensuring a registry to host bundle images")
+			local, err := Local(c)
+			Expect(err).NotTo(HaveOccurred(), "cannot determine if test running locally or on CI: %s", err)
+
+			var registryURL string
+			var copyImage func(dst, dstTag, src, srcTag string) error
+			if local {
+				registryURL, err = createDockerRegistry(c, generatedNamespace.GetName())
+				Expect(err).NotTo(HaveOccurred(), "error creating container registry: %s", err)
+				defer deleteDockerRegistry(c, generatedNamespace.GetName())
+
+				// ensure registry pod is ready before attempting port-forwarding
+				_ = awaitPod(GinkgoT(), c, generatedNamespace.GetName(), registryName, podReady)
+
+				err = registryPortForward(generatedNamespace.GetName())
+				Expect(err).NotTo(HaveOccurred(), "port-forwarding local registry: %s", err)
+				copyImage = func(dst, dstTag, src, srcTag string) error {
+					if !strings.HasPrefix(src, "docker://") {
+						src = fmt.Sprintf("docker://%s", src)
+					}
+					if !strings.HasPrefix(dst, "docker://") {
+						dst = fmt.Sprintf("docker://%s", dst)
+					}
+					_, err := skopeoLocalCopy(dst, dstTag, src, srcTag)
+					return err
+				}
+			} else {
+				registryURL = fmt.Sprintf("%s/%s", openshiftregistryFQDN, generatedNamespace.GetName())
+				registryAuth, err := openshiftRegistryAuth(c, generatedNamespace.GetName())
+				Expect(err).NotTo(HaveOccurred(), "error getting openshift registry authentication: %s", err)
+				copyImage = func(dst, dstTag, src, srcTag string) error {
+					if !strings.HasPrefix(src, "docker://") {
+						src = fmt.Sprintf("docker://%s", src)
+					}
+					if !strings.HasPrefix(dst, "docker://") {
+						dst = fmt.Sprintf("docker://%s", dst)
+					}
+					skopeoArgs := skopeoCopyCmd(dst, dstTag, src, srcTag, registryAuth)
+					err = createSkopeoPod(c, skopeoArgs, generatedNamespace.GetName())
+					if err != nil {
+						return fmt.Errorf("error creating skopeo pod: %v", err)
+					}
+
+					// wait for skopeo pod to exit successfully
+					awaitPod(GinkgoT(), c, generatedNamespace.GetName(), skopeo, func(pod *corev1.Pod) bool {
+						return pod.Status.Phase == corev1.PodSucceeded
+					})
+
+					if err := deleteSkopeoPod(c, generatedNamespace.GetName()); err != nil {
+						return fmt.Errorf("error deleting skopeo pod: %s", err)
+					}
+					return nil
+				}
+			}
+
+			// The remote image to be copied onto the local registry
+			srcImage := "quay.io/olmtest/example-operator-bundle:"
+			srcTag := "0.1.0"
+
+			// on-cluster image ref
+			bundleImage := registryURL + "/unpack-retry-bundle:"
+			bundleTag := genName("x")
+
+			unpackRetryCatalog := fmt.Sprintf(`
+schema: olm.package
+name: unpack-retry-package
+defaultChannel: stable
+---
+schema: olm.channel
+package: unpack-retry-package
+name: stable
+entries:
+  - name: example-operator.v0.1.0
+---
+schema: olm.bundle
+name: example-operator.v0.1.0
+package: unpack-retry-package
+image: %s%s
+properties:
+  - type: olm.package
+    value:
+      packageName: unpack-retry-package
+      version: 1.0.0
+`, bundleImage, bundleTag)
+
+			By("creating a catalog referencing a non-existent bundle image")
+			unpackRetryProvider, err := NewRawFileBasedCatalogProvider(unpackRetryCatalog)
+			Expect(err).ToNot(HaveOccurred())
+			catalogSourceName := fmt.Sprintf("%s-catsrc", generatedNamespace.GetName())
+			magicCatalog := NewMagicCatalog(ctx.Ctx().Client(), generatedNamespace.GetName(), catalogSourceName, unpackRetryProvider)
+			Expect(magicCatalog.DeployCatalog(context.Background())).To(BeNil())
+
+			By("patching the OperatorGroup to reduce the bundle unpacking timeout")
+			ogNN := types.NamespacedName{Name: operatorGroup.GetName(), Namespace: generatedNamespace.GetName()}
+			addBundleUnpackTimeoutOGAnnotation(context.Background(), ctx.Ctx().Client(), ogNN, "1s")
+
+			By("creating a subscription for the missing bundle")
+			unpackRetrySubName := fmt.Sprintf("%s-unpack-retry-package-sub", generatedNamespace.GetName())
+			createSubscriptionForCatalog(crc, generatedNamespace.GetName(), unpackRetrySubName, catalogSourceName, "unpack-retry-package", stableChannel, "", operatorsv1alpha1.ApprovalAutomatic)
+
+			By("waiting for bundle unpack to fail")
+			Eventually(
+				func() error {
+					fetched, err := crc.OperatorsV1alpha1().Subscriptions(generatedNamespace.GetName()).Get(context.Background(), unpackRetrySubName, metav1.GetOptions{})
+					if err != nil {
+						return err
+					}
+					if cond := fetched.Status.GetCondition(v1alpha1.SubscriptionBundleUnpackFailed); cond.Status != corev1.ConditionTrue || cond.Reason != "BundleUnpackFailed" {
+						return fmt.Errorf("%s condition not found", v1alpha1.SubscriptionBundleUnpackFailed)
+					}
+					return nil
+				},
+				5*time.Minute,
+				interval,
+			).Should(Succeed())
+
+			By("pushing missing bundle image")
+			Expect(copyImage(bundleImage, bundleTag, srcImage, srcTag)).To(Succeed())
+
+			By("patching the OperatorGroup to increase the bundle unpacking timeout")
+			addBundleUnpackTimeoutOGAnnotation(context.Background(), ctx.Ctx().Client(), ogNN, "") // revert to default unpack timeout
+
+			By("patching operator group to enable unpack retries")
+			setBundleUnpackRetryMinimumIntervalAnnotation(context.Background(), ctx.Ctx().Client(), ogNN, "1s")
+
+			By("waiting until the subscription has an IP reference")
+			subscription, err := fetchSubscription(crc, generatedNamespace.GetName(), unpackRetrySubName, subscriptionHasInstallPlanChecker())
+			Expect(err).Should(BeNil())
+
+			By("waiting for the v0.1.0 CSV to report a succeeded phase")
+			_, err = fetchCSV(crc, subscription.Status.CurrentCSV, generatedNamespace.GetName(), buildCSVConditionChecker(operatorsv1alpha1.CSVPhaseSucceeded))
+			Expect(err).ShouldNot(HaveOccurred())
+
+			By("checking if old unpack conditions on subscription are removed")
+			Eventually(func() error {
+				fetched, err := crc.OperatorsV1alpha1().Subscriptions(generatedNamespace.GetName()).Get(context.Background(), unpackRetrySubName, metav1.GetOptions{})
+				if err != nil {
+					return err
+				}
+				if cond := fetched.Status.GetCondition(v1alpha1.SubscriptionBundleUnpacking); cond.Status != corev1.ConditionFalse {
+					return fmt.Errorf("subscription condition %s has unexpected value %s, expected %s", v1alpha1.SubscriptionBundleUnpacking, cond.Status, corev1.ConditionFalse)
+				}
+				if cond := fetched.Status.GetCondition(v1alpha1.SubscriptionBundleUnpackFailed); cond.Status != corev1.ConditionUnknown {
+					return fmt.Errorf("unexpected condition %s on subscription", v1alpha1.SubscriptionBundleUnpackFailed)
+				}
+				return nil
+			}).Should(Succeed())
+		})
+
+		It("should not retry successful unpack jobs", func() {
+			By("deploying the testing catalog")
+			provider, err := NewFileBasedFiledBasedCatalogProvider(filepath.Join(testdataDir, failForwardTestDataBaseDir, "example-operator.v0.1.0.yaml"))
+			Expect(err).To(BeNil())
+			catalogSourceName := fmt.Sprintf("%s-catsrc", generatedNamespace.GetName())
+			magicCatalog := NewMagicCatalog(ctx.Ctx().Client(), generatedNamespace.GetName(), catalogSourceName, provider)
+			Expect(magicCatalog.DeployCatalog(context.Background())).To(BeNil())
+
+			By("creating the testing subscription")
+			subName := fmt.Sprintf("%s-packagea-sub", generatedNamespace.GetName())
+			createSubscriptionForCatalog(crc, generatedNamespace.GetName(), subName, catalogSourceName, "packageA", stableChannel, "", operatorsv1alpha1.ApprovalAutomatic)
+
+			By("waiting until the subscription has an IP reference")
+			subscription, err := fetchSubscription(crc, generatedNamespace.GetName(), subName, subscriptionHasInstallPlanChecker())
+			Expect(err).Should(BeNil())
+
+			By("waiting for the v0.1.0 CSV to report a succeeded phase")
+			_, err = fetchCSV(crc, subscription.Status.CurrentCSV, generatedNamespace.GetName(), buildCSVConditionChecker(operatorsv1alpha1.CSVPhaseSucceeded))
+			Expect(err).ShouldNot(HaveOccurred())
+
+			By("patching operator group to enable unpack retries")
+			ogNN := types.NamespacedName{Name: operatorGroup.GetName(), Namespace: generatedNamespace.GetName()}
+			setBundleUnpackRetryMinimumIntervalAnnotation(context.Background(), ctx.Ctx().Client(), ogNN, "1s")
+
+			By("Ensuring successful bundle unpack jobs are not retried")
+			Consistently(func() error {
+				fetched, err := crc.OperatorsV1alpha1().Subscriptions(generatedNamespace.GetName()).Get(context.Background(), subName, metav1.GetOptions{})
+				if err != nil {
+					return err
+				}
+				if cond := fetched.Status.GetCondition(v1alpha1.SubscriptionBundleUnpacking); cond.Status == corev1.ConditionTrue {
+					return fmt.Errorf("unexpected condition status for %s on subscription %s", v1alpha1.SubscriptionBundleUnpacking, subName)
+				}
+				if cond := fetched.Status.GetCondition(v1alpha1.SubscriptionBundleUnpackFailed); cond.Status == corev1.ConditionTrue {
+					return fmt.Errorf("unexpected condition status for %s on subscription %s", v1alpha1.SubscriptionBundleUnpackFailed, subName)
+				}
+				return nil
+			}).Should(Succeed())
+		})
+	})
 })
 
 const (
@@ -2889,9 +3086,9 @@ func fetchSubscription(crc versioned.Interface, namespace, name string, checker
 			return false, err
 		}
 		thisState, thisCSV, thisInstallPlanRef := fetchedSubscription.Status.State, fetchedSubscription.Status.CurrentCSV, fetchedSubscription.Status.InstallPlanRef
-		if thisState != lastState || thisCSV != lastCSV || equality.Semantic.DeepEqual(thisInstallPlanRef, lastInstallPlanRef) {
+		if thisState != lastState || thisCSV != lastCSV || !equality.Semantic.DeepEqual(thisInstallPlanRef, lastInstallPlanRef) {
 			lastState, lastCSV, lastInstallPlanRef = thisState, thisCSV, thisInstallPlanRef
-			log(fmt.Sprintf("%s (%s): %s", thisState, thisCSV, thisInstallPlanRef))
+			log(fmt.Sprintf("subscription %s/%s state: %s (csv %s): installPlanRef: %#v", namespace, name, thisState, thisCSV, thisInstallPlanRef))
 		}
 		return checker(fetchedSubscription), nil
 	})
@@ -2904,6 +3101,11 @@ func fetchSubscription(crc versioned.Interface, namespace, name string, checker
 
 func buildSubscriptionCleanupFunc(crc versioned.Interface, subscription *operatorsv1alpha1.Subscription) cleanupFunc {
 	return func() {
+		if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+			fmt.Printf("Skipping cleanup of install plan for subscription %s/%s...\n", subscription.GetNamespace(), subscription.GetName())
+			return
+		}
+
 		if installPlanRef := subscription.Status.InstallPlanRef; installPlanRef != nil {
 			installPlan, err := crc.OperatorsV1alpha1().InstallPlans(subscription.GetNamespace()).Get(context.Background(), installPlanRef.Name, metav1.GetOptions{})
 			if err == nil {
diff --git a/test/e2e/user_defined_sa_test.go b/test/e2e/user_defined_sa_test.go
index 1e05304b9..ef82ac648 100644
--- a/test/e2e/user_defined_sa_test.go
+++ b/test/e2e/user_defined_sa_test.go
@@ -3,6 +3,7 @@ package e2e
 import (
 	"context"
 	"fmt"
+	"os"
 
 	"github.com/blang/semver/v4"
 	. "github.com/onsi/ginkgo/v2"
@@ -80,17 +81,12 @@ var _ = Describe("User defined service account", func() {
 		require.NoError(GinkgoT(), err)
 		require.NotNil(GinkgoT(), subscription)
 
-		By("We expect the InstallPlan to be in status: Failed.")
+		By("We expect the InstallPlan to be in status: Installing.")
 		ipName := subscription.Status.Install.Name
-		ipPhaseCheckerFunc := buildInstallPlanPhaseCheckFunc(v1alpha1.InstallPlanPhaseFailed)
+		ipPhaseCheckerFunc := buildInstallPlanMessageCheckFunc(`cannot create resource`)
 		ipGot, err := fetchInstallPlanWithNamespace(GinkgoT(), crc, ipName, generatedNamespace.GetName(), ipPhaseCheckerFunc)
 		require.NoError(GinkgoT(), err)
-
-		conditionGot := mustHaveCondition(GinkgoT(), ipGot, v1alpha1.InstallPlanInstalled)
-		assert.Equal(GinkgoT(), corev1.ConditionFalse, conditionGot.Status)
-		assert.Equal(GinkgoT(), v1alpha1.InstallPlanReasonComponentFailed, conditionGot.Reason)
-		assert.Contains(GinkgoT(), conditionGot.Message, fmt.Sprintf("is forbidden: User \"system:serviceaccount:%s:%s\" cannot create resource", generatedNamespace.GetName(), saName))
-
+		
 		By("Verify that all step resources are in Unknown state.")
 		for _, step := range ipGot.Status.Plan {
 			assert.Equal(GinkgoT(), v1alpha1.StepStatusUnknown, step.Status)
@@ -185,12 +181,11 @@ var _ = Describe("User defined service account", func() {
 		require.NoError(GinkgoT(), err)
 		require.NotNil(GinkgoT(), subscription)
 
-		By("We expect the InstallPlan to be in status: Failed.")
+		By("We expect the InstallPlan to expose the permissions error.")
 		ipNameOld := subscription.Status.InstallPlanRef.Name
-		ipPhaseCheckerFunc := buildInstallPlanPhaseCheckFunc(v1alpha1.InstallPlanPhaseFailed)
-		ipGotOld, err := fetchInstallPlanWithNamespace(GinkgoT(), crc, ipNameOld, generatedNamespace.GetName(), ipPhaseCheckerFunc)
+		ipPhaseCheckerFunc := buildInstallPlanMessageCheckFunc(`cannot create resource "clusterserviceversions" in API group "operators.coreos.com" in the namespace`)
+		_, err = fetchInstallPlanWithNamespace(GinkgoT(), crc, ipNameOld, generatedNamespace.GetName(), ipPhaseCheckerFunc)
 		require.NoError(GinkgoT(), err)
-		require.Equal(GinkgoT(), v1alpha1.InstallPlanPhaseFailed, ipGotOld.Status.Phase)
 
 		By("Grant permission now and this should trigger an retry of InstallPlan.")
 		cleanupPerm := grantPermission(GinkgoT(), c, generatedNamespace.GetName(), saName)
@@ -242,6 +237,10 @@ func newServiceAccount(client operatorclient.ClientInterface, namespace, name st
 	Expect(sa).ToNot(BeNil())
 
 	cleanup = func() {
+		if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+			fmt.Printf("Skipping cleanup of service account %s/%s...\n", sa.GetNamespace(), sa.GetName())
+			return
+		}
 		err := client.KubernetesInterface().CoreV1().ServiceAccounts(sa.GetNamespace()).Delete(context.TODO(), sa.GetName(), metav1.DeleteOptions{})
 		Expect(err).ToNot(HaveOccurred())
 	}
@@ -268,6 +267,10 @@ func newOperatorGroupWithServiceAccount(client versioned.Interface, namespace, n
 	Expect(og).ToNot(BeNil())
 
 	cleanup = func() {
+		if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+			fmt.Printf("Skipping cleanup of operator group %s/%s...\n", og.GetNamespace(), og.GetName())
+			return
+		}
 		err := client.OperatorsV1().OperatorGroups(og.GetNamespace()).Delete(context.TODO(), og.GetName(), metav1.DeleteOptions{})
 		Expect(err).ToNot(HaveOccurred())
 	}
@@ -538,6 +541,14 @@ func grantPermission(t GinkgoTInterface, client operatorclient.ClientInterface,
 	require.NoError(t, err)
 
 	cleanup = func() {
+		if env := os.Getenv("SKIP_CLEANUP"); env != "" {
+			fmt.Printf("Skipping cleanup of role %s/%s...\n", role.GetNamespace(), role.GetName())
+			fmt.Printf("Skipping cleanup of role binding %s/%s...\n", binding.GetNamespace(), binding.GetName())
+			fmt.Printf("Skipping cleanup of cluster role %s...\n", clusterrole.GetName())
+			fmt.Printf("Skipping cleanup of cluster role binding %s...\n", clusterbinding.GetName())
+			return
+		}
+
 		err := client.KubernetesInterface().RbacV1().Roles(role.GetNamespace()).Delete(context.TODO(), role.GetName(), metav1.DeleteOptions{})
 		require.NoError(t, err)
 
diff --git a/test/e2e/util.go b/test/e2e/util.go
index 04878b321..ecd622ad4 100644
--- a/test/e2e/util.go
+++ b/test/e2e/util.go
@@ -50,7 +50,7 @@ import (
 )
 
 const (
-	pollInterval = 1 * time.Second
+	pollInterval = 100 * time.Millisecond
 	pollDuration = 5 * time.Minute
 
 	olmConfigMap = "olm-operators" // No-longer used, how long do we keep this around?
@@ -94,13 +94,25 @@ func objectRefToNamespacedName(ip *corev1.ObjectReference) types.NamespacedName
 // adding the "operatorframework.io/bundle-unpack-timeout" annotation to an OperatorGroup
 // resource.
 func addBundleUnpackTimeoutOGAnnotation(ctx context.Context, c k8scontrollerclient.Client, ogNN types.NamespacedName, timeout string) {
+	setOGAnnotation(ctx, c, ogNN, bundle.BundleUnpackTimeoutAnnotationKey, timeout)
+}
+
+func setBundleUnpackRetryMinimumIntervalAnnotation(ctx context.Context, c k8scontrollerclient.Client, ogNN types.NamespacedName, interval string) {
+	setOGAnnotation(ctx, c, ogNN, bundle.BundleUnpackRetryMinimumIntervalAnnotationKey, interval)
+}
+
+func setOGAnnotation(ctx context.Context, c k8scontrollerclient.Client, ogNN types.NamespacedName, key, value string) {
 	Eventually(func() error {
 		og := &operatorsv1.OperatorGroup{}
 		if err := c.Get(ctx, ogNN, og); err != nil {
 			return err
 		}
 		annotations := og.GetAnnotations()
-		annotations[bundle.BundleUnpackTimeoutAnnotationKey] = timeout
+		if len(value) == 0 {
+			delete(annotations, key)
+		} else {
+			annotations[key] = value
+		}
 		og.SetAnnotations(annotations)
 
 		return c.Update(ctx, og)