Project: external-secrets (Provisioning / Security & Compliance)
File: 005-secret-generator-group.md

```yaml title: Secret Generators version: v1alpha1 authors: Christian Hnig, Jan Steffen, Moritz Johner creation-date: 2022-07-08 status: draft ``` <!-- toc --> // autogen please <!-- /toc --> There are use-cases where a secret could be generated by the operator instead of using pre-existing ones from the secret store. For example a MongoDB could be spun up using a password generated by the operator. This password is then used by an application in the cluster to access the database. No human ever needs to read that secret or have to specify it beforehand. A similar feature is available in a project called which tries to solve similar problems like this operator just for Hashicorp Vault as a secret store. The motivation is to boost adoption of zero-trust secrets management and to make it simple to create secure secrets for machine users of services running inside clusters (i.e. databases etc). enable zero-trust secret management for purely technical and internal users enable secure passwords by default reduce error rates when creating RSA or ECDSA private keys auto-rotation of secrets Implement several generators to support not only a number of data types but also generators that interface with external APIs. We add a new resource group `generators.external-secrets.io`. For each generator we add a new CRD, e.g. for password generation, VaultPKI, ECR, GCR etc. This custom resource contains all the configuration details to create a secure value. It can be referenced from a `Kind=ExternalSecret` or `Kind=PushSecret`. The controller generates the value and passes it onto the `Kind=Secret` with all the values that have been fetched from the provider. When used with a `Kind=PushSecret` it will generate the value and push it to the configured provider. A generator generates a known set of outputs, e.g. `username` and `password`, `private_key` and `certificate` or others. The following custom resource `Kind=VaultPKIBackend` contains all the parameters for issuing a certificate via HashiCorp Vault PKI engine. ```yaml apiVersion: generators.external-secrets.io/v1alpha1 kind: VaultPKIBackend metadata: name: my-vault-pki-creds spec: commonName: example.com altNames: example.org localhost auth: {} # service account etc. etc. ``` This resource can be referenced from an ExternalSecret or PushSecret. The ExternalSecret controller generates the secret value and it can be post-processed with existing features like template or key-rewrite. ```yaml apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: example spec: dataFrom: sourceRef: generatorRef: apiVersion: generators.external-secrets.io/v1alpha1 kind: VaultPKIBackend name: foobar ``` In order to accomodate the generator implementation we need to enhance the `spec.secretStoreRef` functionality and add `data[].sourceRef` and `dataFrom[].sourceRef` that allows a user to reference generator resources. This allows us to: improve composability and use multiple SecretStores within a single ExternalSecret use generators from both `data[]` and `dataFrom[]` nodes If `sourceRef` is not defined we fall back to

```yaml spec: data: secretKey: foo remoteRef: {} sourceRef: storeRef: name: foo dataFrom: find: { ... } sourceRef: storeRef: name: "foo" find: { ... } sourceRef: storeRef: name: "bar" ``` Specifying more than one attribute within a sourceRef is not supported and must be rejected. ``` sourceRef: storeRef: {} generatorRef: {} # ILLEGAL! ``` <details> <summary>PushSecret example</summary> ```yaml apiVersion: external-secrets.io/v1alpha1 kind: PushSecret metadata: name: "hello-world" spec: secretStoreRefs: [] # ... selector: generatorRef: apiVersion: generators.external-secrets.io/v1alpha1 kind: VaultPKIBackend name: foobar data: {} # ... ``` </details> <details> <summary>Password Generator Example</summary> ```yaml apiVersion: generators.external-secrets.io/v1alpha1 kind: Password metadata: name: foo spec: characterSet: "..." # TBD length: 42 apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: example spec: dataFrom: generatorRef: apiVersion: generators.external-secrets.io/v1alpha1 kind: Password name: foo ``` </details> <details> <summary>Example: multiple generators with key rewrite</summary> ```yaml apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: example spec: dataFrom: generatorRef: apiVersion: generators.external-secrets.io/v1alpha1 kind: ECR name: foo rewrite: regexp: source: .+ target: foo-$1 generatorRef: apiVersion: generators.external-secrets.io/v1alpha1 kind: ECR name: bar rewrite: regexp: source: .+ target: bar-$1 ``` </details> We can maintain a set of in-tree generators (e.g. for password, binary, hash, alphanumeric, VaultPKI, VaultAWS, ECR, GCR, ACR, Harbor...) but that doesn't provide an extensible interface for users. We can consider the following two approaches (possibly more) that provide extensibility: Note: This section is just an outlook and nothing that we aim to implement right away. We will re-iterate on this feature and see if we need to make the generator implementation more extensible. We can provide an in-tree generator that interfaces through a well-known protocol (e.g. HTTP webhook or gRPC). ```yaml apiVersion: generators.external-secrets.io/v1alpha1 kind: HTTPWebhook metadata: name: example spec: url: "http://httpbin.org/get?parameter={{ .remoteRef.key }}" result: jsonPath: "$.args.parameter" headers: Content-Type: application/json Authorization: Basic {{ print .auth.username ":" .auth.password | b64enc }} secrets: name: auth secretRef: name: webhook-credentials ``` Pros: simple to extend from a user perspective simple simple to add from external-secrets perspective Cons: prone to network errors high coupling of secret sync and generator This is very similar to that allows users to implement their own controller to issue certifiates. We maintain some core generator controllers and leave the implementation of others up to the user. ``` 1 2 ExternalSecret/ create Kind:GenerateRequest generate PushSecret generatorRef: FooGenerator Controller kind: Foo Controller result name: bar 3 result: username: foo 4 password: bar create Kind: Secret data: username: foo password: bar ``` Pros: loose coupling of generator and secret sync workflows Cons: very high complexity hard to extend from a user perspective Every time an ExternalSecret reconciles the generator will produce a new set of output values. Hence, the secrets are rotated. The frequency can be controlled with `refreshInterval`. With `refreshInterval=0` the Secret will be created exactly once and will not rotate. The generator produces an output once which is pushed to the provider. If a secret already exists in that place it can not be overridden.