Repository: gravitational/teleport
Pull request #52783: Machine ID: Document `kubernetes/v2` service
Author: login: timothyb89
Labels: name: documentation
name: size/md
name: no-changelog
description: Indicates that a PR does not require a changelog entry
name: backport/branch/v17
Created: 2025-03-05T02:54:36Z | Merged: 2025-03-06T01:06:28Z

This adds initial documentation for the new `kubernetes/v2` service.

This only includes reference documentation for now, since this service was added partway through v17; the existing Kubernetes guide will be updated in a separate PR for v18 and won't be backported to v17.

Updates #52412


Diff:
diff --git a/docs/pages/enroll-resources/machine-id/faq.mdx b/docs/pages/enroll-resources/machine-id/faq.mdx
index 25d2ea9914485..d20c5d431f418 100644
--- a/docs/pages/enroll-resources/machine-id/faq.mdx
+++ b/docs/pages/enroll-resources/machine-id/faq.mdx
@@ -93,3 +93,18 @@ If your use case absolutely requires long-lived certificates,
 [`tctl auth sign`](../../reference/cli/tctl.mdx#tctl-auth-sign) can
 alternatively be used, however this loses the security benefits of Machine ID's
 short-lived renewable certificates.
+
+## Can Machine ID be used to connect to multiple Kubernetes clusters?
+
+This is possible in Teleport v17.2.7 or higher, using the new `kubernetes/v2`
+output type in `tbot`. This service can expose many clusters at once via
+contexts in the generated `kubeconfig.yaml`, and if label selectors are used,
+will dynamically add contexts as clusters are added and removed in Teleport.
+
+Note that both `tbot` and the Teleport Proxy need to be running v17.2.7 to take
+advantage of this functionality.
+
+Refer to the
+[CLI reference](../../reference/cli/tbot.mdx#tbot-start-kubernetesv2) and
+[config reference](../../reference/machine-id/configuration.mdx#kubernetesv2)
+for more information.
diff --git a/docs/pages/reference/cli/tbot.mdx b/docs/pages/reference/cli/tbot.mdx
index 65f0a5e4635b3..f558a39cb8b64 100644
--- a/docs/pages/reference/cli/tbot.mdx
+++ b/docs/pages/reference/cli/tbot.mdx
@@ -427,6 +427,54 @@ command supports these additional flags:
 | `--reader-group`       | An additional group name or GID that should be allowed by ACLs to read this destination. Only valid for file destinations on Linux. |
 | `--kubernetes-cluster` | The name of the Kubernetes cluster in Teleport for which to fetch credentials |
 
+## tbot start kubernetes/v2
+
+Starts the Machine ID client `tbot` with a Kubernetes V2 output, fetching and
+writing Kubernetes credentials to a `kubeconfig.yaml` at a regular interval to
+the output specified with `--destination`.
+
+Unlike the `kubernetes` output, `kubernetes/v2` allows fetching many Kubernetes
+clusters at once, as multiple contexts within the generated `kubeconfig.yaml`.
+If label selectors are used and clusters are added or removed, the list of
+clusters will be updated on the bot's next renewal. At least one selector -
+either name or label - is required.
+
+Note that as with human users using `tsh kube ls`, only clusters the bot user
+has permission to access will be matched. Additionally, note that label
+selectors do not currently support wildcards.
+
+### Flags
+
+In addition to the [common `tbot start` flags](#common-start-flags), this
+command supports these additional flags:
+
+| Flag                    | Description |
+|-------------------------|-------------|
+| `--destination`         | A destination URI, such as `file:///foo/bar`. See [Destination URIs](#destination-uris) for more info. Required. |
+| `--reader-user`         | An additional user name or UID that should be allowed by ACLs to read this destination. Only valid for file destinations on Linux. |
+| `--reader-group`        | An additional group name or GID that should be allowed by ACLs to read this destination. Only valid for file destinations on Linux. |
+| `--name-selector`       | Selects a Kubernetes cluster by exact name match. Repeatable. |
+| `--label-selector`      | Selects many Kubernetes clusters by label match, e.g. `env=dev,role=ci`. Repeatable. |
+| `--disable-exec-plugin` | If set, disables the exec plugin. Allows credentials to be used without the `tbot` binary. |
+
+### Examples
+
+First, run `tbot` to generate credentials:
+
+```code
+$ tbot start kubernetes/v2 --proxy-server=example.teleport.sh:443 --destination=./example --label-selector env=dev --name-selector example --oneshot
+```
+
+This will fetch credentials for all clusters with the label `env: dev`, plus the
+cluster named `example`. You can then access the clusters using `kubectl`:
+
+```code
+$ kubectl --kubeconfig ./example/kubeconfig.yaml --context example.teleport.sh-example get pods
+```
+
+You can inspect the generated `kubeconfig.yaml` to see which clusters were
+matched by the label selector.
+
 ## tbot start application
 
 Starts the Machine ID client `tbot` with an application output, fetching and
diff --git a/docs/pages/reference/machine-id/configuration.mdx b/docs/pages/reference/machine-id/configuration.mdx
index 68ac72706eb7a..a3da43244dbb6 100644
--- a/docs/pages/reference/machine-id/configuration.mdx
+++ b/docs/pages/reference/machine-id/configuration.mdx
@@ -294,6 +294,58 @@ kubernetes_cluster: my-cluster
 (!docs/pages/includes/machine-id/common-output-config.yaml!)
 ```
 
+### `kubernetes/v2`
+
+The `kubernetes/v2` output type can be used to access many Kubernetes clusters
+as individual contexts within the same `kubeconfig.yaml`.
+
+```yaml
+type: kubernetes/v2
+
+# selectors include one or more matching Kubernetes clusters. Each match will be
+# included in the resulting `kubeconfig.yaml`, assuming the bot has permission
+# to access the cluster.
+selectors:
+  # name includes an exact match by name. Note that wildcards are not currently
+  # supported. Multiple name selectors can be specified if desired.
+  - name: foo
+  # labels include all clusters matching all of these labels. Multiple label
+  # selectors can be provided if needed.
+  - labels:
+      env: dev
+
+# The following configuration fields are available across most output types.
+# Note that `roles` are not supported for this output type.
+
+destination:
+  type: directory
+  path: /opt/machine-id
+
+# credential_ttl and renewal_interval override the credential TTL and renewal
+# interval for this specific output, so that you can make its certificates valid
+# for shorter than `tbot`'s internal certificates.
+#
+# This is particularly useful when using `tbot` in one-shot as part of a cron job
+# where you need `tbot`'s internal certificate to live long enough to be renewed
+# on the next invocation, but don't want long-lived workload certificates on-disk.
+credential_ttl: 30m
+renewal_interval: 15m
+```
+
+Each Kubernetes cluster matching a selector will result in a new context in the
+generated `kubeconfig.yaml`. This can be consumed like so:
+
+```code
+$ kubectl --kubeconfig /opt/machine-id/kubeconfig.yaml --context=example.teleport.sh-foo get pods
+```
+
+The context name is `[Teleport cluster name]-[Kubernetes cluster name]`, so the
+command above runs `kubectl get pods` on the `foo` cluster.
+
+If clusters are added or removed over time, the `kubeconfig.yaml` will be
+updated at the bot's normal renewal interval. You can trigger an early renewal
+by restarting `tbot`, or signaling it with `pkill -usr1 tbot`.
+
 ### `ssh_host`
 
 The `ssh_host` output is used to generate the artifacts required to configure


Reviews:
user: login: ptgott
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T14:57:01Z
user: login: zmb3
body: Note: if your PR description contains `fixes #xyz` (even if it is _partially fixes_) then the issue will close when the PR merges.

If you want to link this PR to the issue, but keep the issue open after the PR merges, you should use `Updates #xyz` instead of _partially fixes_.
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-05T16:44:36Z
user: login: timothyb89
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T17:33:59Z
user: login: timothyb89
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T00:25:25Z

Review comments:
diff_hunk: @@ -427,6 +427,54 @@ command supports these additional flags:
 | `--reader-group`       | An additional group name or GID that should be allowed by ACLs to read this destination. Only valid for file destinations on Linux. |
 | `--kubernetes-cluster` | The name of the Kubernetes cluster in Teleport for which to fetch credentials |
 
+## tbot start kubernetes/v2
+
+Starts the Machine ID client `tbot` with a Kubernetes V2 output, fetching and
+writing Kubernetes credentials to a `kubeconfig.yaml` at a regular interval to
+the output specified with `--destination`.
+
+Unlike the `kubernetes` output, `kubernetes/v2` allows fetching many Kubernetes
+clusters at once, as multiple contexts within the generated `kubeconfig.yaml`.
+If label selectors are used and clusters are added or removed, the list of
+clusters will be updated on the bot's next renewal. At least one selector -
+either name or label - is required.
+
+Note that as with human users using `tsh kube ls`, only clusters the bot user
+has permission to access will be matched. Additionally, note that label
+selectors do not currently support wildcards.
+
+### Flags
+
+In addition to the [common `tbot start` flags](#common-start-flags), this
+command supports these additional flags:
+
+| Flag                    | Description |
+|-------------------------|-------------|
+| `--destination`         | A destination URI, such as file:///foo/bar. See [Destination URIs](#destination-uris) for more info. Required. |
path: docs/pages/reference/cli/tbot.mdx
user: login: ptgott
body: ```suggestion
| `--destination`         | A destination URI, such as `file:///foo/bar`. See [Destination URIs](#destination-uris) for more info. Required. |
```
created_at: 2025-03-05T14:56:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -427,6 +427,54 @@ command supports these additional flags:
 | `--reader-group`       | An additional group name or GID that should be allowed by ACLs to read this destination. Only valid for file destinations on Linux. |
 | `--kubernetes-cluster` | The name of the Kubernetes cluster in Teleport for which to fetch credentials |
 
+## tbot start kubernetes/v2
+
+Starts the Machine ID client `tbot` with a Kubernetes V2 output, fetching and
+writing Kubernetes credentials to a `kubeconfig.yaml` at a regular interval to
+the output specified with `--destination`.
+
+Unlike the `kubernetes` output, `kubernetes/v2` allows fetching many Kubernetes
path: docs/pages/reference/cli/tbot.mdx
user: login: zmb3
body: Do we have an FAQ page for Machine ID where we could also add this info?
created_at: 2025-03-05T16:43:37Z
author_association: COLLABORATOR
diff_hunk: @@ -427,6 +427,54 @@ command supports these additional flags:
 | `--reader-group`       | An additional group name or GID that should be allowed by ACLs to read this destination. Only valid for file destinations on Linux. |
 | `--kubernetes-cluster` | The name of the Kubernetes cluster in Teleport for which to fetch credentials |
 
+## tbot start kubernetes/v2
+
+Starts the Machine ID client `tbot` with a Kubernetes V2 output, fetching and
+writing Kubernetes credentials to a `kubeconfig.yaml` at a regular interval to
+the output specified with `--destination`.
+
+Unlike the `kubernetes` output, `kubernetes/v2` allows fetching many Kubernetes
path: docs/pages/reference/cli/tbot.mdx
user: login: timothyb89
body: We do, and that's probably a more reasonable place to put the (for now) confusing step of "If running >= 17.2, then you can use this". I'll add a section for it.

I was wary of rewriting the existing guide for v17 knowing it only applies after some point release. I think we can make this the default for v18 and avoid future confusion, but in the mean time making the feature discoverable in the FAQ/reference will hopefully be enough.
created_at: 2025-03-05T17:33:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -427,6 +427,54 @@ command supports these additional flags:
 | `--reader-group`       | An additional group name or GID that should be allowed by ACLs to read this destination. Only valid for file destinations on Linux. |
 | `--kubernetes-cluster` | The name of the Kubernetes cluster in Teleport for which to fetch credentials |
 
+## tbot start kubernetes/v2
+
+Starts the Machine ID client `tbot` with a Kubernetes V2 output, fetching and
+writing Kubernetes credentials to a `kubeconfig.yaml` at a regular interval to
+the output specified with `--destination`.
+
+Unlike the `kubernetes` output, `kubernetes/v2` allows fetching many Kubernetes
path: docs/pages/reference/cli/tbot.mdx
user: login: timothyb89
body: I've added a short entry to the Machine ID FAQ about this, that also makes the versioning requirements more clear.
created_at: 2025-03-06T00:25:25Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-05T02:57:38Z
body: Amplify deployment status
Branch | Commit | Job ID | Status | Preview | Updated (UTC)
---------|---------|---------|---------|---------|---------
timothyb89/kubernetes-v2-docs | eea7105a41e5624a565d1c81ed138207c176630a | 4 | ✅SUCCEED | [timothyb89-kubernetes-v2-docs](https://timothyb89-kubernetes-v2-docs.d3b94eevwi10ji.amplifyapp.com) | 2025-03-06 00:53:20

author_association: CONTRIBUTOR
user: login: public-teleport-github-review-bot[bot]
created_at: 2025-03-06T01:07:41Z
body: 
@timothyb89 See the table below for backport results.

| Branch | Result |
|--------|--------|
| branch/v17 | [Create PR](https://github.com/gravitational/teleport/compare/branch/v17...bot/backport-52783-branch/v17?body=Backport+%2352783+to+branch%2Fv17&expand=1&labels=no-changelog&title=%5Bv17%5D+Machine+ID%3A+Document+%60kubernetes%2Fv2%60+service) |

author_association: NONE