Repository: bitnami-labs/sealed-secrets (language: go)

Issue:
Secrets are frequently unsealed
Hi. We have recently upgraded the sealed-secrets-controller to v0.8.1 from v0.7 and are experiencing that all sealed secrets are getting unsealed frequently without any modifications to the sealed secret entity.

![image](https://user-images.githubusercontent.com/124244/63360388-dbe36b00-c35d-11e9-8000-eb971ac4264e.png)

Is this an expected behavior?

Worth noting is that we are runnnig in an Openshift environment.



Discussion / hints:
before v0.8.x we hadn't events in the first place so you wouldn't have noticed it reported that way in the first place.
Thanks for the answer, that explains the behavior, and what I could read from the code is how it has worked for a long time, even though it has not been exposed in this way before 0.8. 

Is this how you want it to work or should the unsealed event only be triggered when the sealed secret object has actually changed since the last unseal? I can see arguments for both cases.
I'm not a fan of the noise so a fix would be welcome (although I don't think it's an urgent issue)
We're using ArgoCD to deploy our applications and sealed secrets. The noise of unseal events every 5 seconds triggers refreshes on argocd application controller, what leads to a very huge CPU load. The main problem is, that it raises with every additional sealed secret, which makes it quite unusable for us.
During my investigations about this topic I've seen, that when the status field is activated we get an incredible cpu load in sealed secret operator and argocd's application controller due to frequently unsealing sealed secrets (each secret is unsealed about every 5 seconds).

Sealed Secrets Operator with status field on:
![Status Field on](https://user-images.githubusercontent.com/6358461/105729597-b46ad780-5f2d-11eb-9709-6f926532784e.png)

Sealed Secrets Operator with status field off:
![Status Field off](https://user-images.githubusercontent.com/6358461/105729803-e2e8b280-5f2d-11eb-9268-b78817ba6289.png)

ArgoCD Application Controller load with Sealed Secrets Operator status field on
![Status Field on](https://user-images.githubusercontent.com/6358461/105732247-86d35d80-5f30-11eb-9038-700cf16f553c.png)

ArgoCD Application Controller load with Sealed Secrets Operator status field off
![Status Field off](https://user-images.githubusercontent.com/6358461/105732288-95217980-5f30-11eb-84d7-73550ce57709.png)

The Screenshots are showing how cpu usage is reduced by switching off the status field.

We are running on
`
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T20:55:23Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
`
Does this only happen if the status feature flag is enabled?
Yes it only happens when the status feature flag is enabled. Currently we have it disabled and all is running very smooth, with the caveat that we can't use the status in our argocd to detect if a sealed secret is healthy and ready.
@alfsch how many SealedSecrets do you have in the cluster? 

I also faced this issue and get around it by adding CPU limit to both SealedSecret and ArgoCD controller pods at the cost of lower performance. We need the health status update, but throttling it isn't a scalable solution for long term.

Will take a look at the code if I can submit a PR to fix it.
@akhy
We have about 40 sealed secrets in our cluster and the count is growing. As first solution we also used limits, but with the limits we had several crashes in argocd application controller. Currently it's easier for us to turn off the status update field, than changing all our throttling alerts....
I'd like to fix this and enable status by default
@mkmik Have you already worked on this? I came up with a very simple fix I which I could send a PR for.

On my test cluster with a two dozens of sealed secrets around, I could observe the following:

![image](https://user-images.githubusercontent.com/3942683/107854321-66a20a80-6e1b-11eb-9967-97eca337a070.png)

(1) is where I enabled status reporting

(2) is where I replaced controller image with one containing the fix

The phase between (2) and (3) is unsealing of secrets I believe. And at (3), everything returns to normal. I tested pushing new versions of some sealed secrets, and it works as expected.

Fix patch:
diff --git a/cmd/controller/controller.go b/cmd/controller/controller.go
index 45749dd579..affc437a52 100644
--- a/cmd/controller/controller.go
+++ b/cmd/controller/controller.go
@@ -305,6 +305,12 @@ func (c *Controller) updateSealedSecretStatus(ssecret *ssv1alpha1.SealedSecret,
 		ssecret.Status = &ssv1alpha1.SealedSecretStatus{}
 	}
 
+	// No need to update the status if we already have observed it from the
+	// current generation of the resource.
+	if ssecret.Status.ObservedGeneration == ssecret.ObjectMeta.Generation {
+		return nil
+	}
+
 	ssecret.Status.ObservedGeneration = ssecret.ObjectMeta.Generation
 	updateSealedSecretsStatusConditions(ssecret.Status, unsealError)
 


Test patch: