Repository: operator-framework/operator-lifecycle-manager
Pull request #1734 (2020-08-20 21:07:19)

[release-4.5] Bug 1870782: Retry conflicting catalog source pod update after fetching latest pod spec
**Description of the change:**
Updating the catalog source pod was intermittently failing due to resource version conflicts. This change switches to using Server side apply to do perform the catalog source pod update.

cherry-pick of https://github.com/operator-framework/operator-lifecycle-manager/pull/1624

openshift-ci-robot: @ankitathomas: This pull request references [Bugzilla bug 1870782](https://bugzilla.redhat.com/show_bug.cgi?id=1870782), which is invalid:
 - expected dependent [Bugzilla bug 1853601](https://bugzilla.redhat.com/show_bug.cgi?id=1853601) to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is ON_QA instead

Comment <code>/bugzilla refresh</code> to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

<details>

In response to [this](https://github.com/operator-framework/operator-lifecycle-manager/pull/1734):

>[release-4.5] Bug 1870782: Retry conflicting catalog source pod update after fetching latest pod spec


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
openshift-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/operator-framework/operator-lifecycle-manager/pull/1734#" title="Author self-approved">ankitathomas</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **njhale**
You can assign the PR to them by writing `/assign @njhale` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=operator-framework%2Foperator-lifecycle-manager).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/operator-framework/operator-lifecycle-manager/blob/release-4.5/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["njhale"]} -->
openshift-ci-robot: @ankitathomas: The following test **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
ci/prow/verify | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-verify/1296554437745053696) | `/test verify`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=operator-framework&repo=operator-lifecycle-manager&pr=1734). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:ankitathomas).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci-robot: @ankitathomas: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
ci/prow/verify | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-verify/1296554437745053696) | `/test verify`
ci/prow/unit | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-unit/1296554437715693568) | `/test unit`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=operator-framework&repo=operator-lifecycle-manager&pr=1734). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:ankitathomas).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci-robot: @ankitathomas: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
ci/prow/verify | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-verify/1296554437745053696) | `/test verify`
ci/prow/unit | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-unit/1296554437715693568) | `/test unit`
ci/prow/e2e-aws-olm | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-e2e-aws-olm/1296554437619224576) | `/test e2e-aws-olm`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=operator-framework&repo=operator-lifecycle-manager&pr=1734). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:ankitathomas).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci-robot: @ankitathomas: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
ci/prow/verify | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-verify/1296554437745053696) | `/test verify`
ci/prow/unit | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-unit/1296554437715693568) | `/test unit`
ci/prow/e2e-aws-olm | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-e2e-aws-olm/1296554437619224576) | `/test e2e-aws-olm`
ci/prow/e2e-gcp-upgrade | 0c275207ff7a76edd9f5360399a073aa1337cf68 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-e2e-gcp-upgrade/1296554437669556224) | `/test e2e-gcp-upgrade`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=operator-framework&repo=operator-lifecycle-manager&pr=1734). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:ankitathomas).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
ankitathomas: /retest
openshift-ci-robot: @ankitathomas: The following test **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
ci/prow/e2e-aws-olm | d1ecc798f754a3d23f545bab3761cd0dc6234dae | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-e2e-aws-olm/1297691157035225088) | `/test e2e-aws-olm`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=operator-framework&repo=operator-lifecycle-manager&pr=1734). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:ankitathomas).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci-robot: @ankitathomas: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
ci/prow/e2e-aws-olm | d1ecc798f754a3d23f545bab3761cd0dc6234dae | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-e2e-aws-olm/1297691157035225088) | `/test e2e-aws-olm`
ci/prow/e2e-gcp-upgrade | d1ecc798f754a3d23f545bab3761cd0dc6234dae | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-e2e-gcp-upgrade/1297691157093945344) | `/test e2e-gcp-upgrade`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=operator-framework&repo=operator-lifecycle-manager&pr=1734). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:ankitathomas).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
ankitathomas: /retest
ankitathomas: /test run-e2e-tests
openshift-ci-robot: @ankitathomas: The specified target(s) for `/test` were not found.
The following commands are available to trigger jobs:
* `/test e2e-aws-console-olm`
* `/test e2e-aws-olm`
* `/test e2e-gcp`
* `/test e2e-gcp-upgrade`
* `/test images`
* `/test unit`
* `/test verify`

Use `/test all` to run all jobs.


<details>

In response to [this](https://github.com/operator-framework/operator-lifecycle-manager/pull/1734#issuecomment-679163909):

>/test run-e2e-tests


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
openshift-ci-robot: @ankitathomas: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
ci/prow/e2e-aws-olm | d1ecc798f754a3d23f545bab3761cd0dc6234dae | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-e2e-aws-olm/1297691157035225088) | `/test e2e-aws-olm`
ci/prow/e2e-gcp-upgrade | cdb6201bfb0dbba104395d4f0523b0ab6f622635 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/operator-framework_operator-lifecycle-manager/1734/pull-ci-operator-framework-operator-lifecycle-manager-release-4.5-e2e-gcp-upgrade/1297889408384503808) | `/test e2e-gcp-upgrade`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=operator-framework&repo=operator-lifecycle-manager&pr=1734). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:ankitathomas).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
ankitathomas: /test e2e-gcp-upgrade

Patch:
diff --git a/cmd/catalog/main.go b/cmd/catalog/main.go
index 70b3a67c3..917b8290e 100644
--- a/cmd/catalog/main.go
+++ b/cmd/catalog/main.go
@@ -13,6 +13,7 @@ import (
 	"github.com/prometheus/client_golang/prometheus/promhttp"
 	log "github.com/sirupsen/logrus"
 	utilclock "k8s.io/apimachinery/pkg/util/clock"
+	k8sscheme "k8s.io/client-go/kubernetes/scheme"
 	"k8s.io/client-go/tools/clientcmd"
 
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/api/client"
@@ -172,7 +173,7 @@ func main() {
 	}
 
 	// Create a new instance of the operator.
-	op, err := catalog.NewOperator(ctx, *kubeConfigPath, utilclock.RealClock{}, logger, *wakeupInterval, *configmapServerImage, *utilImage, *catalogNamespace)
+	op, err := catalog.NewOperator(ctx, *kubeConfigPath, utilclock.RealClock{}, logger, *wakeupInterval, *configmapServerImage, *utilImage, *catalogNamespace, k8sscheme.Scheme)
 	if err != nil {
 		log.Panicf("error configuring operator: %s", err.Error())
 	}
diff --git a/pkg/controller/operators/catalog/operator.go b/pkg/controller/operators/catalog/operator.go
index e3746ca0d..22905d1f0 100644
--- a/pkg/controller/operators/catalog/operator.go
+++ b/pkg/controller/operators/catalog/operator.go
@@ -11,6 +11,8 @@ import (
 	"sync"
 	"time"
 
+	"k8s.io/apimachinery/pkg/runtime"
+
 	errorwrap "github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
 	"google.golang.org/grpc/connectivity"
@@ -47,6 +49,7 @@ import (
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/grpc"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/reconciler"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/resolver"
+	controllerclient "github.com/operator-framework/operator-lifecycle-manager/pkg/lib/controller-runtime/client"
 	index "github.com/operator-framework/operator-lifecycle-manager/pkg/lib/index"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorclient"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorlister"
@@ -71,6 +74,7 @@ const (
 	generatedByKey         = "olm.generated-by"
 	maxInstallPlanCount    = 5
 	maxDeletesPerSweep     = 5
+	RegistryFieldManager   = "olm.registry"
 )
 
 // Operator represents a Kubernetes operator that executes InstallPlans by
@@ -103,7 +107,7 @@ type Operator struct {
 type CatalogSourceSyncFunc func(logger *logrus.Entry, in *v1alpha1.CatalogSource) (out *v1alpha1.CatalogSource, continueSync bool, syncError error)
 
 // NewOperator creates a new Catalog Operator.
-func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clock, logger *logrus.Logger, resync time.Duration, configmapRegistryImage, utilImage string, operatorNamespace string) (*Operator, error) {
+func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clock, logger *logrus.Logger, resync time.Duration, configmapRegistryImage, utilImage string, operatorNamespace string, scheme *runtime.Scheme) (*Operator, error) {
 	resyncPeriod := queueinformer.ResyncWithJitter(resync, 0.2)
 	config, err := clientcmd.BuildConfigFromFlags("", kubeconfigPath)
 	if err != nil {
@@ -132,6 +136,11 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 	// Create an OperatorLister
 	lister := operatorlister.NewLister()
 
+	ssaClient, err := controllerclient.NewForConfig(config, scheme, RegistryFieldManager)
+	if err != nil {
+		return nil, err
+	}
+
 	// Allocate the new instance of an Operator.
 	op := &Operator{
 		Operator:                 queueOperator,
@@ -152,7 +161,7 @@ func NewOperator(ctx context.Context, kubeconfigPath string, clock utilclock.Clo
 		clientAttenuator:         scoped.NewClientAttenuator(logger, config, opClient, crClient, dynamicClient),
 	}
 	op.sources = grpc.NewSourceStore(logger, 10*time.Second, 10*time.Minute, op.syncSourceState)
-	op.reconciler = reconciler.NewRegistryReconcilerFactory(lister, opClient, configmapRegistryImage, op.now)
+	op.reconciler = reconciler.NewRegistryReconcilerFactory(lister, opClient, configmapRegistryImage, op.now, ssaClient)
 
 	// Wire OLM CR sharedIndexInformers
 	crInformerFactory := externalversions.NewSharedInformerFactoryWithOptions(op.client, resyncPeriod())
diff --git a/pkg/controller/registry/reconciler/grpc.go b/pkg/controller/registry/reconciler/grpc.go
index d0c28b302..f67b4b1b0 100644
--- a/pkg/controller/registry/reconciler/grpc.go
+++ b/pkg/controller/registry/reconciler/grpc.go
@@ -3,8 +3,8 @@ package reconciler
 import (
 	"context"
 	"fmt"
-
 	"github.com/operator-framework/api/pkg/operators/v1alpha1"
+	controllerclient "github.com/operator-framework/operator-lifecycle-manager/pkg/lib/controller-runtime/client"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorclient"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorlister"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/ownerutil"
@@ -70,9 +70,10 @@ func (s *grpcCatalogSourceDecorator) Pod() *v1.Pod {
 }
 
 type GrpcRegistryReconciler struct {
-	now      nowFunc
-	Lister   operatorlister.OperatorLister
-	OpClient operatorclient.ClientInterface
+	now       nowFunc
+	Lister    operatorlister.OperatorLister
+	OpClient  operatorclient.ClientInterface
+	SSAClient *controllerclient.ServerSideApplier
 }
 
 var _ RegistryReconciler = &GrpcRegistryReconciler{}
@@ -194,16 +195,17 @@ func (c *GrpcRegistryReconciler) ensureUpdatePod(source grpcCatalogSourceDecorat
 			logrus.WithField("CatalogSource", source.GetName()).Info("detect image update for catalogsource pod")
 
 			updateFlag = true
-			updatePod.Labels[CatalogSourceLabelKey] = source.GetName()
-			updatePod.Labels[CatalogSourceUpdateKey] = ""
 
 			// Update the update pod to promote it to serving pod
-			_, err := c.OpClient.KubernetesInterface().CoreV1().Pods(source.GetNamespace()).Update(context.TODO(), updatePod, metav1.UpdateOptions{})
+			err := c.SSAClient.Apply(context.TODO(), updatePod, func(p *v1.Pod) error {
+				p.Labels[CatalogSourceLabelKey] = source.GetName()
+				p.Labels[CatalogSourceUpdateKey] = ""
+				return nil
+			})()
+
 			if err != nil {
-				return errors.Wrapf(err, "error creating new pod: %s", source.Pod().GetName())
+				return errors.Wrapf(err, "error updating catalog source pod: %s", source.Pod().GetName())
 			}
-
-			break
 		}
 	}
 
diff --git a/pkg/controller/registry/reconciler/reconciler.go b/pkg/controller/registry/reconciler/reconciler.go
index 20c49f9c3..c3351abc7 100644
--- a/pkg/controller/registry/reconciler/reconciler.go
+++ b/pkg/controller/registry/reconciler/reconciler.go
@@ -3,6 +3,7 @@ package reconciler
 
 import (
 	"github.com/operator-framework/api/pkg/operators/v1alpha1"
+	controllerclient "github.com/operator-framework/operator-lifecycle-manager/pkg/lib/controller-runtime/client"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorclient"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorlister"
 	v1 "k8s.io/api/core/v1"
@@ -46,6 +47,7 @@ type registryReconcilerFactory struct {
 	Lister               operatorlister.OperatorLister
 	OpClient             operatorclient.ClientInterface
 	ConfigMapServerImage string
+	SSAClient            *controllerclient.ServerSideApplier
 }
 
 // ReconcilerForSource returns a RegistryReconciler based on the configuration of the given CatalogSource.
@@ -62,9 +64,10 @@ func (r *registryReconcilerFactory) ReconcilerForSource(source *v1alpha1.Catalog
 	case v1alpha1.SourceTypeGrpc:
 		if source.Spec.Image != "" {
 			return &GrpcRegistryReconciler{
-				now:      r.now,
-				Lister:   r.Lister,
-				OpClient: r.OpClient,
+				now:       r.now,
+				Lister:    r.Lister,
+				OpClient:  r.OpClient,
+				SSAClient: r.SSAClient,
 			}
 		} else if source.Spec.Address != "" {
 			return &GrpcAddressRegistryReconciler{
@@ -76,12 +79,13 @@ func (r *registryReconcilerFactory) ReconcilerForSource(source *v1alpha1.Catalog
 }
 
 // NewRegistryReconcilerFactory returns an initialized RegistryReconcilerFactory.
-func NewRegistryReconcilerFactory(lister operatorlister.OperatorLister, opClient operatorclient.ClientInterface, configMapServerImage string, now nowFunc) RegistryReconcilerFactory {
+func NewRegistryReconcilerFactory(lister operatorlister.OperatorLister, opClient operatorclient.ClientInterface, configMapServerImage string, now nowFunc, ssaClient *controllerclient.ServerSideApplier) RegistryReconcilerFactory {
 	return &registryReconcilerFactory{
 		now:                  now,
 		Lister:               lister,
 		OpClient:             opClient,
 		ConfigMapServerImage: configMapServerImage,
+		SSAClient:            ssaClient,
 	}
 }
 
diff --git a/pkg/lib/controller-runtime/client/fake_ssa.go b/pkg/lib/controller-runtime/client/fake_ssa.go
new file mode 100644
index 000000000..560933ab8
--- /dev/null
+++ b/pkg/lib/controller-runtime/client/fake_ssa.go
@@ -0,0 +1,56 @@
+package client
+
+import (
+	"context"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/types"
+	k8scontrollerclient "sigs.k8s.io/controller-runtime/pkg/client"
+	fakecontrollerclient "sigs.k8s.io/controller-runtime/pkg/client/fake"
+)
+
+// FakeApplier provides a wrapper around the fake k8s controller client to convert the unsupported apply-type patches to merge patches.
+func NewFakeApplier(scheme *runtime.Scheme, owner string, objs ...runtime.Object) *ServerSideApplier {
+	return &ServerSideApplier{
+		client: &fakeApplier{fakecontrollerclient.NewFakeClientWithScheme(scheme, objs...)},
+		Scheme: scheme,
+		Owner:  k8scontrollerclient.FieldOwner(owner),
+	}
+}
+
+type fakeApplier struct {
+	k8scontrollerclient.Client
+}
+
+func (c *fakeApplier) Patch(ctx context.Context, obj runtime.Object, patch k8scontrollerclient.Patch, opts ...k8scontrollerclient.PatchOption) error {
+	patch, opts = convertApplyToMergePatch(patch, opts...)
+	return c.Client.Patch(ctx, obj, patch, opts...)
+}
+
+func (c *fakeApplier) Status() k8scontrollerclient.StatusWriter {
+	return fakeStatusWriter{c.Client.Status()}
+}
+
+type fakeStatusWriter struct {
+	k8scontrollerclient.StatusWriter
+}
+
+func (c fakeStatusWriter) Patch(ctx context.Context, obj runtime.Object, patch k8scontrollerclient.Patch, opts ...k8scontrollerclient.PatchOption) error {
+	patch, opts = convertApplyToMergePatch(patch, opts...)
+	return c.StatusWriter.Patch(ctx, obj, patch, opts...)
+}
+
+func convertApplyToMergePatch(patch k8scontrollerclient.Patch, opts ...k8scontrollerclient.PatchOption) (k8scontrollerclient.Patch, []k8scontrollerclient.PatchOption) {
+	// Apply patch type is not supported on the fake controller
+	if patch.Type() == types.ApplyPatchType {
+		patch = k8scontrollerclient.Merge
+		patchOptions := make([]k8scontrollerclient.PatchOption, 0, len(opts))
+		for _, opt := range opts {
+			if opt == k8scontrollerclient.ForceOwnership {
+				continue
+			}
+			patchOptions = append(patchOptions, opt)
+		}
+		opts = patchOptions
+	}
+	return patch, opts
+}
diff --git a/pkg/lib/controller-runtime/client/ssa.go b/pkg/lib/controller-runtime/client/ssa.go
new file mode 100644
index 000000000..cae89e56d
--- /dev/null
+++ b/pkg/lib/controller-runtime/client/ssa.go
@@ -0,0 +1,196 @@
+package client
+
+import (
+	"context"
+	"fmt"
+	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+	"k8s.io/apimachinery/pkg/runtime"
+	kscheme "k8s.io/client-go/kubernetes/scheme"
+	"k8s.io/client-go/rest"
+	apiregistrationv1 "k8s.io/kube-aggregator/pkg/apis/apiregistration/v1"
+	"reflect"
+	k8scontrollerclient "sigs.k8s.io/controller-runtime/pkg/client"
+)
+
+const (
+	defaultOwner = "olm.registry"
+)
+
+type Object interface {
+	runtime.Object
+	metav1.Object
+}
+
+func NewForConfig(c *rest.Config, scheme *runtime.Scheme, owner string) (*ServerSideApplier, error) {
+	if scheme == nil {
+		scheme = runtime.NewScheme()
+		localSchemeBuilder := runtime.NewSchemeBuilder(
+			kscheme.AddToScheme,
+			apiextensionsv1.AddToScheme,
+			apiregistrationv1.AddToScheme,
+		)
+		if err := localSchemeBuilder.AddToScheme(scheme); err != nil {
+			return nil, err
+		}
+	}
+	restClient, err := k8scontrollerclient.New(c, k8scontrollerclient.Options{
+		Scheme: scheme,
+	})
+	if err != nil {
+		return nil, err
+	}
+
+	if len(owner) == 0 {
+		owner = defaultOwner
+	}
+
+	return &ServerSideApplier{
+		client: restClient,
+		Scheme: scheme,
+		Owner:  k8scontrollerclient.FieldOwner(owner),
+	}, nil
+}
+
+func SetDefaultGroupVersionKind(obj Object, s *runtime.Scheme) {
+	gvk := obj.GetObjectKind().GroupVersionKind()
+	if gvk.Empty() && s != nil {
+		// Best-effort guess the GVK
+		gvks, _, err := s.ObjectKinds(obj)
+		if err != nil {
+			panic(fmt.Sprintf("unable to get gvks for object %T: %s", obj, err))
+		}
+		if len(gvks) == 0 || gvks[0].Empty() {
+			panic(fmt.Sprintf("unexpected gvks registered for object %T: %v", obj, gvks))
+		}
+		// TODO: The same object can be registered for multiple group versions
+		// (although in practise this doesn't seem to be used).
+		// In such case, the version set may not be correct.
+		gvk = gvks[0]
+	}
+	obj.GetObjectKind().SetGroupVersionKind(gvk)
+}
+
+type ServerSideApplier struct {
+	client k8scontrollerclient.Client
+	Scheme *runtime.Scheme
+	// Owner becomes the Field Manager for whatever field the Server-Side apply acts on
+	Owner k8scontrollerclient.FieldOwner
+}
+
+// Apply returns a function that invokes a change func on an object and performs a server-side apply patch with the result and its status subresource.
+// The given resource must be a pointer to a struct that specifies its Name, Namespace, APIVersion, and Kind at minimum.
+// The given change function must be unary, matching the signature: "func(<obj type>) error".
+// The returned function is suitable for use w/ asyncronous assertions.
+// The underlying value of the given resource pointer is updated to reflect the latest cluster state each time the closure is successfully invoked.
+// Ex. Change the spec of an existing InstallPlan
+//
+// plan := &InstallPlan{}
+// plan.SetNamespace("ns")
+// plan.SetName("install-123def")
+// Eventually(c.Apply(plan, func(p *v1alpha1.InstallPlan) error {
+//		p.Spec.Approved = true
+//		return nil
+// })).Should(Succeed())
+func (c *ServerSideApplier) Apply(ctx context.Context, obj Object, changeFunc interface{}) func() error {
+	// Ensure given object is a pointer
+	objType := reflect.TypeOf(obj)
+	if objType.Kind() != reflect.Ptr {
+		panic(fmt.Sprintf("argument object must be a pointer"))
+	}
+
+	// Ensure given function matches expected signature
+	var (
+		change     = reflect.ValueOf(changeFunc)
+		changeType = change.Type()
+	)
+	if n := changeType.NumIn(); n != 1 {
+		panic(fmt.Sprintf("unexpected number of formal parameters in change function signature: expected 1, present %d", n))
+	}
+	if pt := changeType.In(0); pt.Kind() != reflect.Interface {
+		if objType != pt {
+			panic(fmt.Sprintf("argument object type does not match the change function parameter type: argument %s, parameter: %s", objType, pt))
+		}
+	} else if !objType.Implements(pt) {
+		panic(fmt.Sprintf("argument object type does not implement the change function parameter type: argument %s, parameter: %s", objType, pt))
+	}
+	if n := changeType.NumOut(); n != 1 {
+		panic(fmt.Sprintf("unexpected number of return values in change function signature: expected 1, present %d", n))
+	}
+	var err error
+	if rt := changeType.Out(0); !rt.Implements(reflect.TypeOf((*error)(nil)).Elem()) {
+		panic(fmt.Sprintf("unexpected return type in change function signature: expected %t, present %s", err, rt))
+	}
+
+	// Determine if we need to apply a status subresource
+	_, applyStatus := objType.Elem().FieldByName("Status")
+
+	if unstructuredObj, ok := obj.(*unstructured.Unstructured); ok {
+		_, applyStatus = unstructuredObj.Object["status"]
+	}
+
+	key, err := k8scontrollerclient.ObjectKeyFromObject(obj)
+	if err != nil {
+		panic(fmt.Sprintf("unable to extract key from resource: %s", err))
+	}
+
+	// Ensure the GVK is set before patching
+	SetDefaultGroupVersionKind(obj, c.Scheme)
+
+	return func() error {
+		changed := func(obj Object) (Object, error) {
+			cp := obj.DeepCopyObject().(Object)
+			if err := c.client.Get(ctx, key, cp); err != nil {
+				return nil, err
+			}
+			// Reset the GVK after the client call strips it
+			SetDefaultGroupVersionKind(cp, c.Scheme)
+			cp.SetManagedFields(nil)
+
+			out := change.Call([]reflect.Value{reflect.ValueOf(cp)})
+			if len(out) != 1 {
+				panic(fmt.Sprintf("unexpected number of return values from apply mutation func: expected 1, returned %d", len(out)))
+			}
+
+			if err := out[0].Interface(); err != nil {
+				return nil, err.(error)
+			}
+
+			return cp, nil
+		}
+
+		cp, err := changed(obj)
+		if err != nil {
+			return err
+		}
+
+		if len(c.Owner) == 0 {
+			c.Owner = defaultOwner
+		}
+
+		if err := c.client.Patch(ctx, cp, k8scontrollerclient.Apply, k8scontrollerclient.ForceOwnership, c.Owner); err != nil {
+			fmt.Printf("first patch error: %s\n", err)
+			return err
+		}
+
+		if !applyStatus {
+			reflect.ValueOf(obj).Elem().Set(reflect.ValueOf(cp).Elem())
+			return nil
+		}
+
+		cp, err = changed(cp)
+		if err != nil {
+			return err
+		}
+
+		if err := c.client.Status().Patch(ctx, cp, k8scontrollerclient.Apply, k8scontrollerclient.ForceOwnership, c.Owner); err != nil {
+			fmt.Printf("second patch error: %s\n", err)
+			return err
+		}
+
+		reflect.ValueOf(obj).Elem().Set(reflect.ValueOf(cp).Elem())
+
+		return nil
+	}
+}
diff --git a/vendor/sigs.k8s.io/controller-runtime/pkg/client/fake/client.go b/vendor/sigs.k8s.io/controller-runtime/pkg/client/fake/client.go
new file mode 100644
index 000000000..be688d5e8
--- /dev/null
+++ b/vendor/sigs.k8s.io/controller-runtime/pkg/client/fake/client.go
@@ -0,0 +1,405 @@
+/*
+Copyright 2018 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package fake
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"strconv"
+	"strings"
+
+	apierrors "k8s.io/apimachinery/pkg/api/errors"
+	"k8s.io/apimachinery/pkg/api/meta"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/runtime/schema"
+	utilrand "k8s.io/apimachinery/pkg/util/rand"
+	"k8s.io/apimachinery/pkg/util/validation/field"
+	"k8s.io/client-go/kubernetes/scheme"
+	"k8s.io/client-go/testing"
+
+	"sigs.k8s.io/controller-runtime/pkg/client"
+	"sigs.k8s.io/controller-runtime/pkg/client/apiutil"
+	"sigs.k8s.io/controller-runtime/pkg/internal/objectutil"
+)
+
+type versionedTracker struct {
+	testing.ObjectTracker
+}
+
+type fakeClient struct {
+	tracker versionedTracker
+	scheme  *runtime.Scheme
+}
+
+var _ client.Client = &fakeClient{}
+
+const (
+	maxNameLength          = 63
+	randomLength           = 5
+	maxGeneratedNameLength = maxNameLength - randomLength
+)
+
+// NewFakeClient creates a new fake client for testing.
+// You can choose to initialize it with a slice of runtime.Object.
+// Deprecated: use NewFakeClientWithScheme.  You should always be
+// passing an explicit Scheme.
+func NewFakeClient(initObjs ...runtime.Object) client.Client {
+	return NewFakeClientWithScheme(scheme.Scheme, initObjs...)
+}
+
+// NewFakeClientWithScheme creates a new fake client with the given scheme
+// for testing.
+// You can choose to initialize it with a slice of runtime.Object.
+func NewFakeClientWithScheme(clientScheme *runtime.Scheme, initObjs ...runtime.Object) client.Client {
+	tracker := testing.NewObjectTracker(clientScheme, scheme.Codecs.UniversalDecoder())
+	for _, obj := range initObjs {
+		err := tracker.Add(obj)
+		if err != nil {
+			panic(fmt.Errorf("failed to add object %v to fake client: %w", obj, err))
+		}
+	}
+	return &fakeClient{
+		tracker: versionedTracker{tracker},
+		scheme:  clientScheme,
+	}
+}
+
+func (t versionedTracker) Create(gvr schema.GroupVersionResource, obj runtime.Object, ns string) error {
+	accessor, err := meta.Accessor(obj)
+	if err != nil {
+		return err
+	}
+	if accessor.GetName() == "" {
+		return apierrors.NewInvalid(
+			obj.GetObjectKind().GroupVersionKind().GroupKind(),
+			accessor.GetName(),
+			field.ErrorList{field.Required(field.NewPath("metadata.name"), "name is required")})
+	}
+	if accessor.GetResourceVersion() != "" {
+		return apierrors.NewBadRequest("resourceVersion can not be set for Create requests")
+	}
+	accessor.SetResourceVersion("1")
+	return t.ObjectTracker.Create(gvr, obj, ns)
+}
+
+func (t versionedTracker) Update(gvr schema.GroupVersionResource, obj runtime.Object, ns string) error {
+	accessor, err := meta.Accessor(obj)
+	if err != nil {
+		return fmt.Errorf("failed to get accessor for object: %v", err)
+	}
+	if accessor.GetName() == "" {
+		return apierrors.NewInvalid(
+			obj.GetObjectKind().GroupVersionKind().GroupKind(),
+			accessor.GetName(),
+			field.ErrorList{field.Required(field.NewPath("metadata.name"), "name is required")})
+	}
+	oldObject, err := t.ObjectTracker.Get(gvr, ns, accessor.GetName())
+	if err != nil {
+		return err
+	}
+	oldAccessor, err := meta.Accessor(oldObject)
+	if err != nil {
+		return err
+	}
+	if accessor.GetResourceVersion() != oldAccessor.GetResourceVersion() {
+		return apierrors.NewConflict(gvr.GroupResource(), accessor.GetName(), errors.New("object was modified"))
+	}
+	if oldAccessor.GetResourceVersion() == "" {
+		oldAccessor.SetResourceVersion("0")
+	}
+	intResourceVersion, err := strconv.ParseUint(oldAccessor.GetResourceVersion(), 10, 64)
+	if err != nil {
+		return fmt.Errorf("can not convert resourceVersion %q to int: %v", oldAccessor.GetResourceVersion(), err)
+	}
+	intResourceVersion++
+	accessor.SetResourceVersion(strconv.FormatUint(intResourceVersion, 10))
+	return t.ObjectTracker.Update(gvr, obj, ns)
+}
+
+func (c *fakeClient) Get(ctx context.Context, key client.ObjectKey, obj runtime.Object) error {
+	gvr, err := getGVRFromObject(obj, c.scheme)
+	if err != nil {
+		return err
+	}
+	o, err := c.tracker.Get(gvr, key.Namespace, key.Name)
+	if err != nil {
+		return err
+	}
+
+	gvk, err := apiutil.GVKForObject(obj, c.scheme)
+	if err != nil {
+		return err
+	}
+	ta, err := meta.TypeAccessor(o)
+	if err != nil {
+		return err
+	}
+	ta.SetKind(gvk.Kind)
+	ta.SetAPIVersion(gvk.GroupVersion().String())
+
+	j, err := json.Marshal(o)
+	if err != nil {
+		return err
+	}
+	decoder := scheme.Codecs.UniversalDecoder()
+	_, _, err = decoder.Decode(j, nil, obj)
+	return err
+}
+
+func (c *fakeClient) List(ctx context.Context, obj runtime.Object, opts ...client.ListOption) error {
+	gvk, err := apiutil.GVKForObject(obj, c.scheme)
+	if err != nil {
+		return err
+	}
+
+	OriginalKind := gvk.Kind
+
+	if !strings.HasSuffix(gvk.Kind, "List") {
+		return fmt.Errorf("non-list type %T (kind %q) passed as output", obj, gvk)
+	}
+	// we need the non-list GVK, so chop off the "List" from the end of the kind
+	gvk.Kind = gvk.Kind[:len(gvk.Kind)-4]
+
+	listOpts := client.ListOptions{}
+	listOpts.ApplyOptions(opts)
+
+	gvr, _ := meta.UnsafeGuessKindToResource(gvk)
+	o, err := c.tracker.List(gvr, gvk, listOpts.Namespace)
+	if err != nil {
+		return err
+	}
+
+	ta, err := meta.TypeAccessor(o)
+	if err != nil {
+		return err
+	}
+	ta.SetKind(OriginalKind)
+	ta.SetAPIVersion(gvk.GroupVersion().String())
+
+	j, err := json.Marshal(o)
+	if err != nil {
+		return err
+	}
+	decoder := scheme.Codecs.UniversalDecoder()
+	_, _, err = decoder.Decode(j, nil, obj)
+	if err != nil {
+		return err
+	}
+
+	if listOpts.LabelSelector != nil {
+		objs, err := meta.ExtractList(obj)
+		if err != nil {
+			return err
+		}
+		filteredObjs, err := objectutil.FilterWithLabels(objs, listOpts.LabelSelector)
+		if err != nil {
+			return err
+		}
+		err = meta.SetList(obj, filteredObjs)
+		if err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func (c *fakeClient) Create(ctx context.Context, obj runtime.Object, opts ...client.CreateOption) error {
+	createOptions := &client.CreateOptions{}
+	createOptions.ApplyOptions(opts)
+
+	for _, dryRunOpt := range createOptions.DryRun {
+		if dryRunOpt == metav1.DryRunAll {
+			return nil
+		}
+	}
+
+	gvr, err := getGVRFromObject(obj, c.scheme)
+	if err != nil {
+		return err
+	}
+	accessor, err := meta.Accessor(obj)
+	if err != nil {
+		return err
+	}
+
+	if accessor.GetName() == "" && accessor.GetGenerateName() != "" {
+		base := accessor.GetGenerateName()
+		if len(base) > maxGeneratedNameLength {
+			base = base[:maxGeneratedNameLength]
+		}
+		accessor.SetName(fmt.Sprintf("%s%s", base, utilrand.String(randomLength)))
+	}
+
+	return c.tracker.Create(gvr, obj, accessor.GetNamespace())
+}
+
+func (c *fakeClient) Delete(ctx context.Context, obj runtime.Object, opts ...client.DeleteOption) error {
+	gvr, err := getGVRFromObject(obj, c.scheme)
+	if err != nil {
+		return err
+	}
+	accessor, err := meta.Accessor(obj)
+	if err != nil {
+		return err
+	}
+	delOptions := client.DeleteOptions{}
+	delOptions.ApplyOptions(opts)
+
+	//TODO: implement propagation
+	return c.tracker.Delete(gvr, accessor.GetNamespace(), accessor.GetName())
+}
+
+func (c *fakeClient) DeleteAllOf(ctx context.Context, obj runtime.Object, opts ...client.DeleteAllOfOption) error {
+	gvk, err := apiutil.GVKForObject(obj, scheme.Scheme)
+	if err != nil {
+		return err
+	}
+
+	dcOptions := client.DeleteAllOfOptions{}
+	dcOptions.ApplyOptions(opts)
+
+	gvr, _ := meta.UnsafeGuessKindToResource(gvk)
+	o, err := c.tracker.List(gvr, gvk, dcOptions.Namespace)
+	if err != nil {
+		return err
+	}
+
+	objs, err := meta.ExtractList(o)
+	if err != nil {
+		return err
+	}
+	filteredObjs, err := objectutil.FilterWithLabels(objs, dcOptions.LabelSelector)
+	if err != nil {
+		return err
+	}
+	for _, o := range filteredObjs {
+		accessor, err := meta.Accessor(o)
+		if err != nil {
+			return err
+		}
+		err = c.tracker.Delete(gvr, accessor.GetNamespace(), accessor.GetName())
+		if err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func (c *fakeClient) Update(ctx context.Context, obj runtime.Object, opts ...client.UpdateOption) error {
+	updateOptions := &client.UpdateOptions{}
+	updateOptions.ApplyOptions(opts)
+
+	for _, dryRunOpt := range updateOptions.DryRun {
+		if dryRunOpt == metav1.DryRunAll {
+			return nil
+		}
+	}
+
+	gvr, err := getGVRFromObject(obj, c.scheme)
+	if err != nil {
+		return err
+	}
+	accessor, err := meta.Accessor(obj)
+	if err != nil {
+		return err
+	}
+	return c.tracker.Update(gvr, obj, accessor.GetNamespace())
+}
+
+func (c *fakeClient) Patch(ctx context.Context, obj runtime.Object, patch client.Patch, opts ...client.PatchOption) error {
+	patchOptions := &client.PatchOptions{}
+	patchOptions.ApplyOptions(opts)
+
+	for _, dryRunOpt := range patchOptions.DryRun {
+		if dryRunOpt == metav1.DryRunAll {
+			return nil
+		}
+	}
+
+	gvr, err := getGVRFromObject(obj, c.scheme)
+	if err != nil {
+		return err
+	}
+	accessor, err := meta.Accessor(obj)
+	if err != nil {
+		return err
+	}
+	data, err := patch.Data(obj)
+	if err != nil {
+		return err
+	}
+
+	reaction := testing.ObjectReaction(c.tracker)
+	handled, o, err := reaction(testing.NewPatchAction(gvr, accessor.GetNamespace(), accessor.GetName(), patch.Type(), data))
+	if err != nil {
+		return err
+	}
+	if !handled {
+		panic("tracker could not handle patch method")
+	}
+
+	gvk, err := apiutil.GVKForObject(obj, c.scheme)
+	if err != nil {
+		return err
+	}
+	ta, err := meta.TypeAccessor(o)
+	if err != nil {
+		return err
+	}
+	ta.SetKind(gvk.Kind)
+	ta.SetAPIVersion(gvk.GroupVersion().String())
+
+	j, err := json.Marshal(o)
+	if err != nil {
+		return err
+	}
+	decoder := scheme.Codecs.UniversalDecoder()
+	_, _, err = decoder.Decode(j, nil, obj)
+	return err
+}
+
+func (c *fakeClient) Status() client.StatusWriter {
+	return &fakeStatusWriter{client: c}
+}
+
+func getGVRFromObject(obj runtime.Object, scheme *runtime.Scheme) (schema.GroupVersionResource, error) {
+	gvk, err := apiutil.GVKForObject(obj, scheme)
+	if err != nil {
+		return schema.GroupVersionResource{}, err
+	}
+	gvr, _ := meta.UnsafeGuessKindToResource(gvk)
+	return gvr, nil
+}
+
+type fakeStatusWriter struct {
+	client *fakeClient
+}
+
+func (sw *fakeStatusWriter) Update(ctx context.Context, obj runtime.Object, opts ...client.UpdateOption) error {
+	// TODO(droot): This results in full update of the obj (spec + status). Need
+	// a way to update status field only.
+	return sw.client.Update(ctx, obj, opts...)
+}
+
+func (sw *fakeStatusWriter) Patch(ctx context.Context, obj runtime.Object, patch client.Patch, opts ...client.PatchOption) error {
+	// TODO(droot): This results in full update of the obj (spec + status). Need
+	// a way to update status field only.
+	return sw.client.Patch(ctx, obj, patch, opts...)
+}
diff --git a/vendor/sigs.k8s.io/controller-runtime/pkg/client/fake/doc.go b/vendor/sigs.k8s.io/controller-runtime/pkg/client/fake/doc.go
new file mode 100644
index 000000000..a45d70332
--- /dev/null
+++ b/vendor/sigs.k8s.io/controller-runtime/pkg/client/fake/doc.go
@@ -0,0 +1,33 @@
+/*
+Copyright 2018 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+/*
+Package fake provides a fake client for testing.
+
+Deprecated: please use pkg/envtest for testing. This package will be dropped
+before the v1.0.0 release.
+
+An fake client is backed by its simple object store indexed by GroupVersionResource.
+You can create a fake client with optional objects.
+
+	client := NewFakeClient(initObjs...) // initObjs is a slice of runtime.Object
+
+You can invoke the methods defined in the Client interface.
+
+When it doubt, it's almost always better not to use this package and instead use
+envtest.Environment with a real client and API server.
+*/
+package fake
diff --git a/vendor/sigs.k8s.io/controller-runtime/pkg/internal/objectutil/filter.go b/vendor/sigs.k8s.io/controller-runtime/pkg/internal/objectutil/filter.go
new file mode 100644
index 000000000..8513846e2
--- /dev/null
+++ b/vendor/sigs.k8s.io/controller-runtime/pkg/internal/objectutil/filter.go
@@ -0,0 +1,42 @@
+/*
+Copyright 2018 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package objectutil
+
+import (
+	apimeta "k8s.io/apimachinery/pkg/api/meta"
+	"k8s.io/apimachinery/pkg/labels"
+	"k8s.io/apimachinery/pkg/runtime"
+)
+
+// FilterWithLabels returns a copy of the items in objs matching labelSel
+func FilterWithLabels(objs []runtime.Object, labelSel labels.Selector) ([]runtime.Object, error) {
+	outItems := make([]runtime.Object, 0, len(objs))
+	for _, obj := range objs {
+		meta, err := apimeta.Accessor(obj)
+		if err != nil {
+			return nil, err
+		}
+		if labelSel != nil {
+			lbls := labels.Set(meta.GetLabels())
+			if !labelSel.Matches(lbls) {
+				continue
+			}
+		}
+		outItems = append(outItems, obj.DeepCopyObject())
+	}
+	return outItems, nil
+}


Test patch:
diff --git a/pkg/controller/operators/catalog/operator_test.go b/pkg/controller/operators/catalog/operator_test.go
index 0db22732a..d500d829b 100644
--- a/pkg/controller/operators/catalog/operator_test.go
+++ b/pkg/controller/operators/catalog/operator_test.go
@@ -50,6 +50,7 @@ import (
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/resolver"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/fakes"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/clientfake"
+	controllerclient "github.com/operator-framework/operator-lifecycle-manager/pkg/lib/controller-runtime/client"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorclient"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorlister"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/ownerutil"
@@ -1247,7 +1248,14 @@ func NewFakeOperator(ctx context.Context, namespace string, namespaces []string,
 	}
 	op.sources = grpc.NewSourceStore(config.logger, 1*time.Second, 5*time.Second, op.syncSourceState)
 	if op.reconciler == nil {
-		op.reconciler = reconciler.NewRegistryReconcilerFactory(lister, op.opClient, "test:pod", op.now)
+		s := runtime.NewScheme()
+		err := k8sfake.AddToScheme(s)
+		if err != nil {
+			return nil, err
+		}
+		applier := controllerclient.NewFakeApplier(s, "testowner")
+
+		op.reconciler = reconciler.NewRegistryReconcilerFactory(lister, op.opClient, "test:pod", op.now, applier)
 	}
 
 	op.RunInformers(ctx)
diff --git a/pkg/lib/controller-runtime/client/ssa_test.go b/pkg/lib/controller-runtime/client/ssa_test.go
new file mode 100644
index 000000000..13ab2cf74
--- /dev/null
+++ b/pkg/lib/controller-runtime/client/ssa_test.go
@@ -0,0 +1,157 @@
+package client
+
+import (
+	"context"
+	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+	"testing"
+
+	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/testobj"
+	"github.com/stretchr/testify/require"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/runtime/schema"
+)
+
+func TestSetDefaultGroupVersionKind(t *testing.T) {
+	tests := []struct {
+		name          string
+		schemebuilder runtime.SchemeBuilder
+		obj           Object
+		want          schema.GroupVersionKind
+	}{
+		{
+			name:          "DefaultGVK",
+			schemebuilder: corev1.SchemeBuilder,
+			obj:           &corev1.ServiceAccount{},
+			want: schema.GroupVersionKind{
+				Group:   "",
+				Version: "v1",
+				Kind:    "ServiceAccount",
+			},
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			s := runtime.NewScheme()
+			err := tt.schemebuilder.AddToScheme(s)
+			require.NoError(t, err)
+
+			SetDefaultGroupVersionKind(tt.obj, s)
+			require.EqualValues(t, tt.want, tt.obj.GetObjectKind().GroupVersionKind())
+		})
+	}
+}
+
+func TestServerSideApply(t *testing.T) {
+	tests := []struct {
+		name          string
+		obj           Object
+		schemebuilder *runtime.SchemeBuilder
+		changeFunc    interface{}
+		want          Object
+	}{
+		{
+			name: "ServerSideMetadataApply",
+			obj: &corev1.Pod{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "testpod",
+					Namespace: "testns",
+					Labels: map[string]string{
+						"testlabel": "oldvalue",
+					},
+				},
+			},
+			schemebuilder: &corev1.SchemeBuilder,
+			changeFunc: func(p *corev1.Pod) error {
+				p.Labels["testlabel"] = "newvalue"
+				return nil
+			},
+			want: &corev1.Pod{
+				TypeMeta: metav1.TypeMeta{
+					Kind:       "Pod",
+					APIVersion: "v1",
+				},
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "testpod",
+					Namespace: "testns",
+					Labels: map[string]string{
+						"testlabel": "newvalue",
+					},
+				},
+			},
+		},
+		{
+			name: "ServerSideStatusApply",
+			obj: &corev1.Pod{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "testpod",
+					Namespace: "testns",
+				},
+				Status: corev1.PodStatus{Message: "new"},
+			},
+			schemebuilder: &corev1.SchemeBuilder,
+			changeFunc: func(p *corev1.Pod) error {
+				p.Status.Message = "new"
+				return nil
+			},
+			want: &corev1.Pod{
+				TypeMeta: metav1.TypeMeta{
+					Kind:       "Pod",
+					APIVersion: "v1",
+				},
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "testpod",
+					Namespace: "testns",
+				},
+				Status: corev1.PodStatus{Message: "new"},
+			},
+		},
+		{
+			name: "ServerSideUnstructuredApply",
+			obj: &unstructured.Unstructured{
+				Object: map[string]interface{}{
+					"kind":       "Pod",
+					"apiVersion": "v1",
+					"metadata": map[string]interface{}{
+						"name":      "testpod",
+						"namespace": "testns",
+					},
+					"status": map[string]interface{}{
+						"message": "old",
+					},
+				},
+			},
+			schemebuilder: &corev1.SchemeBuilder,
+			changeFunc: func(u *unstructured.Unstructured) error {
+				return unstructured.SetNestedField(u.Object, "new", "status", "message")
+			},
+			want: &corev1.Pod{
+				TypeMeta: metav1.TypeMeta{
+					Kind:       "Pod",
+					APIVersion: "v1",
+				},
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "testpod",
+					Namespace: "testns",
+				},
+				Status: corev1.PodStatus{Message: "new"},
+			},
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			s := runtime.NewScheme()
+			require.NoError(t, tt.schemebuilder.AddToScheme(s))
+
+			applier := NewFakeApplier(s, "testowner", tt.obj)
+			require.NoError(t, applier.Apply(context.TODO(), tt.obj, tt.changeFunc)())
+
+			newObj := corev1.Pod{}
+			require.NoError(t, applier.client.Get(context.TODO(), testobj.NamespacedName(tt.obj), &newObj))
+
+			newObj.ResourceVersion = ""
+			require.EqualValues(t, tt.want, &newObj)
+		})
+	}
+}
diff --git a/test/e2e/ctx/ctx.go b/test/e2e/ctx/ctx.go
index 1f98903dd..889db77fd 100644
--- a/test/e2e/ctx/ctx.go
+++ b/test/e2e/ctx/ctx.go
@@ -5,10 +5,16 @@ import (
 	"strings"
 
 	. "github.com/onsi/ginkgo"
+	apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
+	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/client-go/dynamic"
+	kscheme "k8s.io/client-go/kubernetes/scheme"
 	"k8s.io/client-go/rest"
 
+	operatorsv1 "github.com/operator-framework/api/pkg/operators/v1"
+	operatorsv1alpha1 "github.com/operator-framework/api/pkg/operators/v1alpha1"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/api/client/clientset/versioned"
+	controllerclient "github.com/operator-framework/operator-lifecycle-manager/pkg/lib/controller-runtime/client"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorclient"
 	pversioned "github.com/operator-framework/operator-lifecycle-manager/pkg/package-server/client/clientset/versioned"
 )
@@ -23,6 +29,9 @@ type TestContext struct {
 	operatorClient versioned.Interface
 	dynamicClient  dynamic.Interface
 	packageClient  pversioned.Interface
+	ssaClient      *controllerclient.ServerSideApplier
+
+	scheme *runtime.Scheme
 }
 
 // Ctx returns a pointer to the global test context. During parallel
@@ -60,6 +69,9 @@ func (ctx TestContext) PackageClient() pversioned.Interface {
 	return ctx.packageClient
 }
 
+func (ctx TestContext) SSAClient() *controllerclient.ServerSideApplier {
+	return ctx.ssaClient
+}
 func setDerivedFields(ctx *TestContext) error {
 	if ctx == nil {
 		return fmt.Errorf("nil test context")
@@ -93,5 +105,22 @@ func setDerivedFields(ctx *TestContext) error {
 	}
 	ctx.packageClient = packageClient
 
+	ctx.scheme = runtime.NewScheme()
+	localSchemeBuilder := runtime.NewSchemeBuilder(
+		apiextensionsv1.AddToScheme,
+		kscheme.AddToScheme,
+		operatorsv1alpha1.AddToScheme,
+		operatorsv1.AddToScheme,
+		apiextensionsv1.AddToScheme,
+	)
+
+	if err := localSchemeBuilder.AddToScheme(ctx.scheme); err != nil {
+		return err
+	}
+
+	ctx.ssaClient, err = controllerclient.NewForConfig(ctx.restConfig, ctx.scheme, "test.olm.registry")
+	if err != nil {
+		return err
+	}
 	return nil
 }
diff --git a/test/e2e/metrics_e2e_test.go b/test/e2e/metrics_e2e_test.go
index d88176d0c..ff10acbb6 100644
--- a/test/e2e/metrics_e2e_test.go
+++ b/test/e2e/metrics_e2e_test.go
@@ -109,8 +109,11 @@ var _ = Describe("Metrics", func() {
 				BeforeEach(func() {
 					updatedSubscription, err := crc.OperatorsV1alpha1().Subscriptions(subscription.GetNamespace()).Get(context.TODO(), subscription.GetName(), metav1.GetOptions{})
 					Expect(err).ToNot(HaveOccurred())
-					updatedSubscription.Spec.Channel = betaChannel
-					updateSubscription(GinkgoT(), crc, updatedSubscription)
+
+					Eventually(Apply(updatedSubscription, func(s *v1alpha1.Subscription) error {
+						s.Spec.Channel = betaChannel
+						return nil
+					})).Should(Succeed(), "could not update subscription")
 				})
 
 				It("deletes the old Subscription metric and emits the new metric", func() {
@@ -134,8 +137,12 @@ var _ = Describe("Metrics", func() {
 					BeforeEach(func() {
 						updatedSubscription, err := crc.OperatorsV1alpha1().Subscriptions(subscription.GetNamespace()).Get(context.TODO(), subscription.GetName(), metav1.GetOptions{})
 						Expect(err).ToNot(HaveOccurred())
-						updatedSubscription.Spec.Channel = alphaChannel
-						updateSubscription(GinkgoT(), crc, updatedSubscription)
+
+						Eventually(Apply(updatedSubscription, func(s *v1alpha1.Subscription) error {
+							s.Spec.Channel = alphaChannel
+							return nil
+						})).Should(Succeed(), "could not update subscription")
+
 					})
 
 					It("deletes the old subscription metric and emits the new metric(there is only one metric for the subscription)", func() {
diff --git a/test/e2e/util_test.go b/test/e2e/util_test.go
index fe4b15e04..187a3f468 100644
--- a/test/e2e/util_test.go
+++ b/test/e2e/util_test.go
@@ -39,6 +39,7 @@ import (
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/api/client/clientset/versioned"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/feature"
+	controllerclient "github.com/operator-framework/operator-lifecycle-manager/pkg/lib/controller-runtime/client"
 	"github.com/operator-framework/operator-lifecycle-manager/pkg/lib/operatorclient"
 	pmversioned "github.com/operator-framework/operator-lifecycle-manager/pkg/package-server/client/clientset/versioned"
 	"github.com/operator-framework/operator-lifecycle-manager/test/e2e/ctx"
@@ -882,3 +883,7 @@ func toggleFeatureGates(deployment *appsv1.Deployment, toToggle ...featuregate.F
 
 	awaitPredicates(deadline, w, deploymentReplicas(2), deploymentAvailable, deploymentReplicas(1))
 }
+
+func Apply(obj controllerclient.Object, changeFunc interface{}) func() error {
+	return ctx.Ctx().SSAClient().Apply(context.Background(), obj, changeFunc)
+}