Project: Teleport (Provisioning / Security & Compliance)
File: 0016-dynamic-configuration.md

authors: Andrej Tokark (andrej@goteleport.com) state: implemented This RFD presents several possible scenarios involving the interaction of explicitly-managed dynamic and static configuration. Design decisions are made as to which branches of these scenarios are to be supported by Teleport. Finally, the actual implementation is discussed. Some resources like `types.AuthPreference` or `services.ClusterConfig` are live representations of parts of the `auth_service` configuration section. These resources can be understood to constitute dynamic configuration of an auth server, as opposed to static configuration defined by the `teleport.yaml` file. The current behaviour is that Teleport creates dynamic configuration implicitly by deriving it from static configuration during auth server initialization. This RFD allows the dynamic configuration to be created/updated explicitly via `tctl`, independently of static configuration, for two main reasons: To facilitate automated/programmatic management of Teleport clusters. To add configuration capability to Teleport-as-a-service offerings that do not allow direct access to `teleport.yaml` files. We use the example of the `authentication` subsection of `auth_service` in `teleport.yaml` (as "the" static configuration) and the corresponding resource `types.AuthPreference` (as "the" dynamic configuration). The latter is assumed to be manipulable via `tctl` using the identifier `cap`. The `authentication` section is specified in `teleport.yaml`. The backend can have any `AuthPreference` data stored from the last auth server run. The auth server initialization procedure MUST commit data derived from the static configuration to the backend, overwriting any potential resource data already stored there. `tctl get cap` returns the resource data in line with `teleport.yaml`. Therefore, static configuration is always given precedence. This should also ensure backward compatibility with the already established workflows. The `authentication` section has never been specified in `teleport.yaml`. No user changes have been made to the dynamic configuration. The auth server initialization procedure MAY or MAY NOT commit the default settings to the backend (in the latter case a structure holding the default settings is returned in place of the missing resource when the resource is requested). In this option, dynamic-configuration resources are understood to exist only if they have been comitted as a result of having been specified in static configuration or via `tctl create`. The command `tctl get cap` would therefore return an error saying "authentication preference not found". To view the default state, `tctl get` would have to be invoked with a new CLI flag. `tctl get cap` returns the resource data corresponding to the default settings. The `authentication` section used to be specified in `teleport.yaml` but it has been removed and left unspecified since the last auth server init. No user changes have been made to the dynamic configuration. The auth server initialization procedure retains the last stored dynamic configuration which had been derived from the last specified static configuration. In particular, the backend MUST NOT be overwritten with the default settings. `tctl get cap` returns the resource data corresponding to the last specified static configuration. The auth server initialization procedure MUST NOT retain the last stored dynamic configuration. The resource data are either overwritten with the default settings or deleted (in the latter case a structure holding the default settings is returned in place of the missing resource when the resource is

`tctl get cap` then behaves as in Scenario 2. The `authentication` section has never been specified in `teleport.yaml`. No user changes have been made to the dynamic configuration. A user issues the command `tctl create authpref.yaml` with valid authentication preferences stored in the YAML file. (Issuing `tctl create -f authpref.yaml` would have the same effect.) The dynamic configuration is updated: the resource data read from `authpref.yaml` MUST be written to the backend (potentially overwriting the default data already stored there). The command returns with success. Upon the next auth server restart with `authentication` still unspecified in `teleport.yaml`, the auth server initialization procedure retains the last stored dynamic configuration, i.e. the one from `authpref.yaml`. `tctl get cap` returns the resource data corresponding to `authpref.yaml`. The `authentication` section has never been specified in `teleport.yaml`. A user has already made explicit changes to the dynamic configuration using `tctl create`. A user issues the command `tctl create authpref.yaml` with valid authentication preferences stored in the YAML file. The command is rejected with an "already exists" error, recommending to use the `-f`/`--force` flag to overwrite the already existing `AuthPreference` resource. The command is accepted and returns with success. The `authentication` section is specified in `teleport.yaml`. The auth server is running with static configuration as in Scenario 1. `tctl create authpref.yaml` returns with an error since the command cannot be used to overwrite non-default configuration. `tctl create -f authpref.yaml` allows the user to temporarily set the dynamic configuration until it gets replaced by the static configuration during a restart as in Scenario 1. In case the static configuration section is removed before the restart, the dynamic configuration corresponding to `authpref.yaml` is retained. `tctl create -f authpref.yaml` is rejected as it constitutes a security risk and may cause consistency issues in HA environments. The following table summarily depicts a key aspect of the scenarios elaborated above, namely the question of which configuration source is to be preferred by the auth server initialization procedure: | | dynamic updated by user | dynamic not updated by user | | :: | :: | :: | | static specified | static | static | | static unspecified | dynamic | defaults OR last static [?] | (The state marked [?] depends on the resolution of Scenario 3.) The resource label `teleport.dev/origin` (below shortened to `origin` for brevity) is used as the key indicator when determining the configuration sources and their precedence. The label can be associated with three values: `defaults`: for hard-coded resource objects consisting of default settings; `config-file`: for resource objects derived from static configuration; `dynamic`: for resource objects created "dynamically" (e.g. via `tctl`). The following table captures the ordinary means of performing the "fastest" transition between a pair of `origin` values. The leftmost column is the current/source `origin` value of a resource while the top row is the desired/target `origin` value: | from \ to | `defaults` | `config-file` | `dynamic` | | :: | :: | :: | :: | | `defaults` | n/a | specify in `teleport.yaml` & restart | `tctl create` | | `config-file` | remove from

| change in `teleport.yaml` & restart | `tctl create --force --confirm` | | `dynamic` | `tctl rm` | specify in `teleport.yaml` & restart | `tctl create --force` | The `teleport.dev/origin` label is reserved for system use. Resources derived from `teleport.yaml` are necessarily labelled either as `defaults` (when the relevant section is left unspecified) or as `config-file` (when the section is explicitly specified). Auth server API handlers automatically disregard the supplied `origin` value and reset it to `dynamic`. If `teleport.yaml` section is specified, store it as a resource in the backend with `origin: config-file`. If `teleport.yaml` section is not specified, attempt to fetch the resource currently stored in the backend. If the fetching attempt fails with a not-found error, store the default resource in the backend with `origin: defaults`. If the fetching attempt fails for another reason, return the error immediately. If the fetched resource has `origin: config-file` or `origin: defaults`, store the default resource in the backend with `origin: defaults`. If the fetched resource has another `origin` value, keep the backend as-is. This logic implies Choices 2.B and 3.B. Fetch the resource currently stored in the backend; on error return immediately. If called without `--force` and the fetched resource does not have `origin: defaults` then print an error: ``` non-default cluster auth preference already exists, use -f or --force flag to overwrite ``` If called with `--force` and the fetched resource does not have `origin: config-file` then store the resource specified in the provided YAML file in the backend. If called with `--force` and the fetched resource has `origin: config-file` then print an error: ``` This resource is managed by static configuration. We recommend removing configuration from teleport.yaml, restarting the servers and trying this command again. If you would still like to proceed, re-run the command with both --force and --confirm flags. ``` Invocation with both `--force` and `--confirm` will replace the stored resource with the YAML one until the auth server restart. This logic implies Choice 5.A and an adaptation of Choice 6.A. The `tctl rm` subcommand can be used to reset dynamic resources back to their defaults. Fetch the resource currently stored in the backend; on error return immediately. If the fetched resource does not have `origin: config-file`, replace the stored resource with the default one. If the fetched resource has `origin: config-file` then print an error: ``` This resource is managed by static configuration. We recommend removing configuration from teleport.yaml and restarting the servers in order to reset the resource to its default. ``` Reading any configuration resource requires the RBAC verb `read`. Performing any dynamic overwrite of a configuration resource requires the RBAC verb `update`. In addition, if the overwrite is to replace a resource with `origin: config-file`, the RBAC verb `create` is also required. | | `read` | `update` | `create` | | :: | :: | :: | :: | | `tctl get` | :heavycheckmark: | | | | `tctl create` | | :heavycheckmark: | | | `tctl create --force` | | :heavycheckmark: | | | `tctl create --force --confirm` | | :heavycheckmark: | :heavycheckmark: | | `tctl rm` | | :heavycheckmark: | |