Repository: external-secrets/external-secrets (go)

Issue:
Non-existent AWS Secrets Manager secret not created when PushSecret is used with `updatePolicy: IfNotExists`
**Describe the bug**
When pushing secrets to AWS Secrets Manager with PushSecret that has `updatePolicy: IfNotExists`, an aws secret is not created at all if it does not exist. No error logs are generated. Without the `updatePolicy: IfNotExists` configuration the secret is created correctly.

**To Reproduce**
kubernetes version: 1.27.13, ESO version: 0.9.19

manifests:

```yaml
apiVersion: v1
kind: Secret
metadata: 
  name: test-secret-k8s
type: Opaque
data: 
  password: {{ "test-password" | b64enc | quote }}
```

```yaml
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: aws
spec:
  provider:
    aws:
      service: SecretsManager
      role: sample-role
      region: us-east-2

```

```yaml
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
  name: test-push-secret # Customisable
  namespace: test # Same of the SecretStores
spec:
  updatePolicy: IfNotExists
  refreshInterval: 10s # Refresh interval for which push secret will reconcile
  secretStoreRefs: # A list of secret stores to push secrets to
    - name: aws
      kind: SecretStore
  selector:
    secret:
      name: test-secret-k8s # Source Kubernetes secret to be pushed
  data:
    - conversionStrategy: None
      match: # Source Kubernetes secret key to be pushed
        secretKey: password
        remoteRef:
          remoteKey: test-secret # Remote reference (where the secret is going to be pushed)
          property: password
      metadata:
        secretPushFormat: string
```

**Expected behavior**
A secret should be created in AWS Secrets Manager if it does not exist there.

**Additional context**
Inspecting the code I believe the implementation of `SecretExists` for this provider [contains a bug here](https://github.com/external-secrets/external-secrets/blob/01a96d6fa4ee9a10d960a38082391a151054b06f/pkg/provider/aws/secretsmanager/secretsmanager.go#L233) -- It returns `true, nil` in case the aws secret was not found, resulting in no action. I believe it should return `false, nil`.


Pull request description:
fix: aws secretmanager's SecretExists returns true for non-existent secrets
## Problem Statement

The AWS Provider's SecretManager has a SecretExists function that currently returns `true` when a secret does not exist. Not sure why this was implemented in such a way. This PR fixes that to return `false` when a secret does not exist and alters the unit tests as well.

## Related Issue

Fixes #3682 

## Proposed Changes

How do you like to solve the issue and why?
Change the `handleSecretError` to return false in case the secret does not exist error pops up, hence returning false in the `SecretExists` function.

## Checklist

- [x] I have read the [contribution guidelines](https://external-secrets.io/latest/contributing/process/#submitting-a-pull-request)
- [x] All commits are signed with `git commit --signoff`
- [x] My changes have reasonable test coverage
- [ ] All tests pass with `make test`
- [x] I ensured my PR is ready for review with `make reviewable`


Patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 9a2dcb5ca..4c30307e1 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -49,7 +49,7 @@ jobs:
         uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
 
       - name: Setup Go
-        uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v5.0.1
+        uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
         id: setup-go
         with:
           go-version-file: "go.mod"
@@ -75,7 +75,7 @@ jobs:
         uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
 
       - name: Setup Go
-        uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v5.0.1
+        uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
         id: setup-go
         with:
           go-version-file: "go.mod"
@@ -106,7 +106,7 @@ jobs:
         run: git fetch --prune --unshallow
 
       - name: Setup Go
-        uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v5.0.1
+        uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
         id: setup-go
         with:
           go-version-file: "go.mod"
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
index 37126bbb8..d675f59f9 100644
--- a/.github/workflows/docs.yml
+++ b/.github/workflows/docs.yml
@@ -20,7 +20,7 @@ jobs:
           fetch-depth: 0
 
       - name: Setup Go
-        uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v5.0.1
+        uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
         with:
           go-version-file: "go.mod"
 
diff --git a/.github/workflows/helm.yml b/.github/workflows/helm.yml
index 469e8c305..183db79d8 100644
--- a/.github/workflows/helm.yml
+++ b/.github/workflows/helm.yml
@@ -34,7 +34,7 @@ jobs:
         with:
           version: v3.14.2 # remember to also update for the second job (release)
 
-      - uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
+      - uses: actions/setup-python@39cd14951b08e74b54015e9e001cdefcf80e669f # v5.1.1
         with:
           python-version: 3.7
 
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
index e96330be6..1e030e045 100644
--- a/.github/workflows/publish.yml
+++ b/.github/workflows/publish.yml
@@ -66,7 +66,7 @@ jobs:
           install: true
 
       - name: Setup Go
-        uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v5.0.1
+        uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
         id: setup-go
         with:
           go-version-file: "go.mod"
@@ -126,7 +126,7 @@ jobs:
         run: make docker.build
 
       - name: Run Trivy vulnerability scanner
-        uses: aquasecurity/trivy-action@7c2007bcb556501da015201bcba5aa14069b74e2 # master
+        uses: aquasecurity/trivy-action@6e7b7d1fd3e4fef0c5fa8cce1229c54b2c9bd0d8 # master
         with:
           image-ref: ${{ inputs.image-name }}:${{ steps.container_info.outputs.image-tag }}
           format: 'table'
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 9b7099a04..697a55d48 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -76,7 +76,7 @@ jobs:
           fetch-depth: 0
 
       - name: Setup Go
-        uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v5.0.1
+        uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
         id: setup-go
         with:
           go-version-file: "go.mod"
diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml
index 0ad2d323f..11e8f873b 100644
--- a/.github/workflows/scorecard.yml
+++ b/.github/workflows/scorecard.yml
@@ -33,6 +33,6 @@ jobs:
 
       # Upload the results to GitHub's code scanning dashboard.
       - name: "Upload to code-scanning"
-        uses: github/codeql-action/upload-sarif@b611370bb5703a7efb587f9d136a52ea24c5c38c # v3.25.11
+        uses: github/codeql-action/upload-sarif@4fa2a7953630fd2f3fb380f21be14ede0169dd4f # v3.25.12
         with:
           sarif_file: results.sarif
diff --git a/.github/workflows/update-deps.yml b/.github/workflows/update-deps.yml
index 2a2a7ab07..154da7f2c 100644
--- a/.github/workflows/update-deps.yml
+++ b/.github/workflows/update-deps.yml
@@ -40,7 +40,7 @@ jobs:
         branch: ${{ fromJson(needs.branches.outputs.branches) }}
     steps:
     - name: Setup Go
-      uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v5.0.1
+      uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v5.0.2
       with:
         go-version: "1.21"
 
diff --git a/apis/externalsecrets/v1beta1/secretstore_vault_types.go b/apis/externalsecrets/v1beta1/secretstore_vault_types.go
index d221001d0..73f9b28fa 100644
--- a/apis/externalsecrets/v1beta1/secretstore_vault_types.go
+++ b/apis/externalsecrets/v1beta1/secretstore_vault_types.go
@@ -86,6 +86,10 @@ type VaultProvider struct {
 	// https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header
 	// +optional
 	ForwardInconsistent bool `json:"forwardInconsistent,omitempty"`
+
+	// Headers to be added in Vault request
+	// +optional
+	Headers map[string]string `json:"headers,omitempty"`
 }
 
 // VaultClientTLS is the configuration used for client side related TLS communication,
diff --git a/apis/externalsecrets/v1beta1/zz_generated.deepcopy.go b/apis/externalsecrets/v1beta1/zz_generated.deepcopy.go
index 05a706316..d312b655b 100644
--- a/apis/externalsecrets/v1beta1/zz_generated.deepcopy.go
+++ b/apis/externalsecrets/v1beta1/zz_generated.deepcopy.go
@@ -3192,6 +3192,13 @@ func (in *VaultProvider) DeepCopyInto(out *VaultProvider) {
 		*out = new(CAProvider)
 		(*in).DeepCopyInto(*out)
 	}
+	if in.Headers != nil {
+		in, out := &in.Headers, &out.Headers
+		*out = make(map[string]string, len(*in))
+		for key, val := range *in {
+			(*out)[key] = val
+		}
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VaultProvider.
diff --git a/config/crds/bases/external-secrets.io_clustersecretstores.yaml b/config/crds/bases/external-secrets.io_clustersecretstores.yaml
index 310c1aa79..5703c216f 100644
--- a/config/crds/bases/external-secrets.io_clustersecretstores.yaml
+++ b/config/crds/bases/external-secrets.io_clustersecretstores.yaml
@@ -4401,6 +4401,11 @@ spec:
                           the option is enabled serverside.
                           https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header
                         type: boolean
+                      headers:
+                        additionalProperties:
+                          type: string
+                        description: Headers to be added in Vault request
+                        type: object
                       namespace:
                         description: |-
                           Name of the vault namespace. Namespaces is a set of features within Vault Enterprise that allows
diff --git a/config/crds/bases/external-secrets.io_secretstores.yaml b/config/crds/bases/external-secrets.io_secretstores.yaml
index b6a25e66e..7bef1c62b 100644
--- a/config/crds/bases/external-secrets.io_secretstores.yaml
+++ b/config/crds/bases/external-secrets.io_secretstores.yaml
@@ -4401,6 +4401,11 @@ spec:
                           the option is enabled serverside.
                           https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header
                         type: boolean
+                      headers:
+                        additionalProperties:
+                          type: string
+                        description: Headers to be added in Vault request
+                        type: object
                       namespace:
                         description: |-
                           Name of the vault namespace. Namespaces is a set of features within Vault Enterprise that allows
diff --git a/config/crds/bases/generators.external-secrets.io_vaultdynamicsecrets.yaml b/config/crds/bases/generators.external-secrets.io_vaultdynamicsecrets.yaml
index acf23bdc9..b7b4dc8e8 100644
--- a/config/crds/bases/generators.external-secrets.io_vaultdynamicsecrets.yaml
+++ b/config/crds/bases/generators.external-secrets.io_vaultdynamicsecrets.yaml
@@ -604,6 +604,11 @@ spec:
                       the option is enabled serverside.
                       https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header
                     type: boolean
+                  headers:
+                    additionalProperties:
+                      type: string
+                    description: Headers to be added in Vault request
+                    type: object
                   namespace:
                     description: |-
                       Name of the vault namespace. Namespaces is a set of features within Vault Enterprise that allows
diff --git a/deploy/crds/bundle.yaml b/deploy/crds/bundle.yaml
index d15190a08..c9845a103 100644
--- a/deploy/crds/bundle.yaml
+++ b/deploy/crds/bundle.yaml
@@ -4748,6 +4748,11 @@ spec:
                             the option is enabled serverside.
                             https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header
                           type: boolean
+                        headers:
+                          additionalProperties:
+                            type: string
+                          description: Headers to be added in Vault request
+                          type: object
                         namespace:
                           description: |-
                             Name of the vault namespace. Namespaces is a set of features within Vault Enterprise that allows
@@ -10374,6 +10379,11 @@ spec:
                             the option is enabled serverside.
                             https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header
                           type: boolean
+                        headers:
+                          additionalProperties:
+                            type: string
+                          description: Headers to be added in Vault request
+                          type: object
                         namespace:
                           description: |-
                             Name of the vault namespace. Namespaces is a set of features within Vault Enterprise that allows
@@ -12064,6 +12074,11 @@ spec:
                         the option is enabled serverside.
                         https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header
                       type: boolean
+                    headers:
+                      additionalProperties:
+                        type: string
+                      description: Headers to be added in Vault request
+                      type: object
                     namespace:
                       description: |-
                         Name of the vault namespace. Namespaces is a set of features within Vault Enterprise that allows
diff --git a/docs/api/spec.md b/docs/api/spec.md
index 554706bba..7ff62480a 100644
--- a/docs/api/spec.md
+++ b/docs/api/spec.md
@@ -8627,6 +8627,18 @@ the option is enabled serverside.
 <a href="https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header">https://www.vaultproject.io/docs/configuration/replication#allow_forwarding_via_header</a></p>
 </td>
 </tr>
+<tr>
+<td>
+<code>headers</code></br>
+<em>
+map[string]string
+</em>
+</td>
+<td>
+<em>(Optional)</em>
+<p>Headers to be added in Vault request</p>
+</td>
+</tr>
 </tbody>
 </table>
 <h3 id="external-secrets.io/v1beta1.VaultUserPassAuth">VaultUserPassAuth
diff --git a/hack/api-docs/requirements.txt b/hack/api-docs/requirements.txt
index b2c849bfb..7f7111444 100644
--- a/hack/api-docs/requirements.txt
+++ b/hack/api-docs/requirements.txt
@@ -18,7 +18,7 @@ mergedeep==1.3.4
 mike @ git+https://github.com/jimporter/mike@300593c338b18f61f604d18457c351e166318020
 mkdocs==1.6.0
 mkdocs-macros-plugin==1.0.5
-mkdocs-material==9.5.28
+mkdocs-material==9.5.29
 mkdocs-material-extensions==1.3.1
 mkdocs-minify-plugin==0.8.0
 packaging==24.1
diff --git a/pkg/provider/aws/secretsmanager/secretsmanager.go b/pkg/provider/aws/secretsmanager/secretsmanager.go
index 8c6b958ee..24452e1f5 100644
--- a/pkg/provider/aws/secretsmanager/secretsmanager.go
+++ b/pkg/provider/aws/secretsmanager/secretsmanager.go
@@ -230,7 +230,7 @@ func (sm *SecretsManager) handleSecretError(err error) (bool, error) {
 		return false, err
 	}
 	if aerr.Code() == awssm.ErrCodeResourceNotFoundException {
-		return true, nil
+		return false, nil
 	}
 	return false, err
 }
diff --git a/pkg/provider/oracle/oracle.go b/pkg/provider/oracle/oracle.go
index 41a01ecfb..2f4dfccbd 100644
--- a/pkg/provider/oracle/oracle.go
+++ b/pkg/provider/oracle/oracle.go
@@ -96,11 +96,22 @@ const (
 )
 
 func (vms *VaultManagementService) PushSecret(ctx context.Context, secret *corev1.Secret, data esv1beta1.PushSecretData) error {
+	if vms.encryptionKey == "" {
+		return fmt.Errorf("SecretStore must reference encryption key")
+	}
+	value := secret.Data[data.GetSecretKey()]
 	if data.GetSecretKey() == "" {
-		return fmt.Errorf("pushing the whole secret is not yet implemented")
+		secretData := map[string]string{}
+		for k, v := range secret.Data {
+			secretData[k] = string(v)
+		}
+		jsonSecret, err := json.Marshal(secretData)
+		if err != nil {
+			return fmt.Errorf("unable to create json %v from value: %v", value, secretData)
+		}
+		value = jsonSecret
 	}
 
-	value := secret.Data[data.GetSecretKey()]
 	secretName := data.GetRemoteKey()
 	encodedValue := base64.StdEncoding.EncodeToString(value)
 	sec, action, err := vms.getSecretBundleWithCode(ctx, secretName)
diff --git a/pkg/provider/vault/provider.go b/pkg/provider/vault/provider.go
index df97f6e06..597839c0a 100644
--- a/pkg/provider/vault/provider.go
+++ b/pkg/provider/vault/provider.go
@@ -149,9 +149,16 @@ func (p *Provider) initClient(ctx context.Context, c *client, client util.Client
 		client.SetNamespace(*vaultSpec.Namespace)
 	}
 
+	if vaultSpec.Headers != nil {
+		for hKey, hValue := range vaultSpec.Headers {
+			client.AddHeader(hKey, hValue)
+		}
+	}
+
 	if vaultSpec.ReadYourWrites && vaultSpec.ForwardInconsistent {
 		client.AddHeader("X-Vault-Inconsistent", "forward-active-node")
 	}
+
 	c.client = client
 	c.auth = client.Auth()
 	c.logical = client.Logical()
diff --git a/tilt.debug.dockerfile b/tilt.debug.dockerfile
index cbfabb3fa..5622fcefe 100644
--- a/tilt.debug.dockerfile
+++ b/tilt.debug.dockerfile
@@ -1,4 +1,4 @@
-FROM golang:1.22.5@sha256:fcae9e0e7313c6467a7c6632ebb5e5fab99bd39bd5eb6ee34a211353e647827a
+FROM golang:1.22.5@sha256:829eff99a4b2abffe68f6a3847337bf6455d69d17e49ec1a97dac78834754bd6
 WORKDIR /
 COPY ./bin/external-secrets /external-secrets
 


Test patch:
diff --git a/pkg/provider/aws/secretsmanager/secretsmanager_test.go b/pkg/provider/aws/secretsmanager/secretsmanager_test.go
index d0f514f36..d10c45031 100644
--- a/pkg/provider/aws/secretsmanager/secretsmanager_test.go
+++ b/pkg/provider/aws/secretsmanager/secretsmanager_test.go
@@ -1359,7 +1359,7 @@ func TestSecretExists(t *testing.T) {
 				wantError: true,
 			},
 		},
-		"SecretExistsReturnsTrueForNonExistingSecret": {
+		"SecretExistsReturnsFalseForNonExistingSecret": {
 			args: args{
 				store: makeValidSecretStore().Spec.Provider.AWS,
 				client: fakesm.Client{
@@ -1369,7 +1369,7 @@ func TestSecretExists(t *testing.T) {
 			},
 			want: want{
 				err:       nil,
-				wantError: true,
+				wantError: false,
 			},
 		},
 		"SecretExistsReturnsFalseForErroredSecret": {
diff --git a/pkg/provider/oracle/oracle_test.go b/pkg/provider/oracle/oracle_test.go
index 54284ab55..9885b551f 100644
--- a/pkg/provider/oracle/oracle_test.go
+++ b/pkg/provider/oracle/oracle_test.go
@@ -581,6 +581,7 @@ func TestOracleVaultGetAllSecrets(t *testing.T) {
 
 func TestOracleVaultPushSecret(t *testing.T) {
 	testSecretKey := "test-secret-key"
+	encryptionKey := "must-not-be-blank-for-push"
 	var testCases = map[string]struct {
 		vms       *VaultManagementService
 		data      testingfake.PushSecretData
@@ -589,6 +590,7 @@ func TestOracleVaultPushSecret(t *testing.T) {
 	}{
 		"create a secret if not exists": {
 			&VaultManagementService{
+				encryptionKey: encryptionKey,
 				Client: &fakeoracle.OracleMockClient{
 					SecretBundles: map[string]secrets.SecretBundle{
 						s2id: s2bundle,
@@ -605,8 +607,28 @@ func TestOracleVaultPushSecret(t *testing.T) {
 			},
 			"created",
 		},
+		"create a json secret if not exists": {
+			&VaultManagementService{
+				encryptionKey: encryptionKey,
+				Client: &fakeoracle.OracleMockClient{
+					SecretBundles: map[string]secrets.SecretBundle{
+						s2id: s2bundle,
+					},
+				},
+				VaultClient: &fakeoracle.OracleMockVaultClient{},
+			},
+			testingfake.PushSecretData{
+				SecretKey: testSecretKey,
+				RemoteKey: s1id,
+			},
+			func(vms *VaultManagementService) bool {
+				return vms.VaultClient.(*fakeoracle.OracleMockVaultClient).CreatedCount == 1
+			},
+			"{'key-a':'secret-a', 'key-b': 'secret-b'}",
+		},
 		"update a secret if exists": {
 			&VaultManagementService{
+				encryptionKey: encryptionKey,
 				Client: &fakeoracle.OracleMockClient{
 					SecretBundles: map[string]secrets.SecretBundle{
 						s1id: s1bundle,
@@ -626,6 +648,7 @@ func TestOracleVaultPushSecret(t *testing.T) {
 		},
 		"neither create nor update if secret content is unchanged": {
 			&VaultManagementService{
+				encryptionKey: encryptionKey,
 				Client: &fakeoracle.OracleMockClient{
 					SecretBundles: map[string]secrets.SecretBundle{
 						s1id: s1bundle,