Change:
Working: a pod with multiple containers, mounting a projected volume mount. Note the different UID/GID This works since 1.19+. On 1.18 and old we need as well. Not working: doing the same with ephemeral containers Then attach ephemeral container: When doing this, accessing the token from the ephemeral containers gives a permission denied. If we add to the pod spec, it does work. File permissions are the same on ephemeral containers and normal containers See above No response <details</details<detailsKind </details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig storage cc
cc does this need to be resolved before 1.23, since ephemeral containers is beta this release? /milestone v1.23
Thanks for pinging this, I missed it. Investigating... /assign
I'm having trouble reproducing this. The doc says that file mode should be set to the in the pod's (not container) . The owner is sometimes taken from the container's , though, I suspect when there's only one for the . If I create a pod with a container and projected volume: Then the token gets the mode (owner is correct, but mode is not the ): But if I try to reference the volume twice using different : Then the container gets the mode (in both containers): Now if I add an ephemeral container to : Then I see the same file ownership as before: So this differs, but is it the incorrect behavior for ephemeral containers? I'm not sure. I'm also not sure what behavior you're seeing for your first test case () because when I try it on my test cluster I get the same behavior as , regardless of whether there are 1 or 2 volumeMounts, except that the file gets mode 0644. Could you provide more succinct test cases with pods rather than deployments, and include the output of ? Any idea what's the intended pod-vs-container securityContext behavior for file ownership of projected volumes?
t2 reproducer: Testing we can access the token With ephemeral containers: attach ephemeral container: Testing access:
There is special logic to handle setting permissions for projected service account tokens. I wonder if the logic is not handling ephemeral containers properly.
The logic in (implemented in ) requires visibility to all containers at pod creation time. If all non-ephemeral containers run as a unified, non-root uid, the token will only be readable by that uid.
Thanks for the test cases and the links to the KEP, they made it much easier to figure out what's going on here. It sounds like everything is working as intended. The KEP even mentions that ephemeral containers are excluded from the heuristic. The only question is whether we should update the projected volume docs to mention the special handling of tokens. you mentioned that fsGroup works as you expected. Is that an acceptable work around for your use case?
Not really, since it impacts all volumes. Basically the use case is a sidecar wants access to a token. With setting fsGroup at the pod level, it impacts the main application's volumes which typically breaks everything. The changes in KEP 2451 fix this, but not with ephemeral containers. That being said, I do not personally have a real use case for this, unless the volume could also be dynamically as well, so if this is the intended behavior that is ok for me
is running the ephemeral container as a non-0 uid that is different than the non-0 uids of the other containers common?
I highly doubt it. For context on how I found this, I was experimenting with running a service mesh sidecar as an ephemeral container. That was almost entirely just an experiment and clearly an abuse of what ephemeral containers were design for though - I was mostly just playing around with things
since this is not a bug, switching type to documentation and removing from milestone
I recommend opening a documentation issue against that explains the docs change to make here. Ideally: be reasonably thorough, as the technical details here seem quite subtle.
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
/transfer website
Cool, the transfer worked. Here's a summary for k/website: Problem: Projected Volumes have specially handling for that causes them to disregard when all containers in a pod have the same . This undocumented behavior led to a bug report in kubernetes/kubernetes Proposed Solution: Update projected volumes doc to explain special behavior: When all containers specified at pod creation have the same , the will use this user as the file owner and set mode . Ephemeral containers are never present at pod creation, so ephemeralContainer aren't considered in this behavior. Page to Update:
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/sig docs
/triage accepted /lifecycle frozen

Discussion:
the projected files have the correct ownership set including container user ownership. <ins> When all containers in a pod have the same `runAsUser` set in their [`PodSecurityContext`](/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context) or container [`SecurityContext`](/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context-1), then the kubelet ensures that the contents of the `serviceAccountToken` volume are owned by that user, and the token file has its permission mode set to `0600`. {{< note >}} {{< glossary_tooltip text="Ephemeral containers" term_id="ephemeral-container" >}} added to a Pod after it is created do *not* change volume permissions that were set when the pod was created. If a Pod's `serviceAccountToken` volume permissions were set to `0600` because all other containers in the Pod have the same `runAsUser`, ephemeral containers must use the same `runAsUser` to be able to read the token. {{< /note >}} </ins> ### Windows In Windows pods that have a projected volume and `RunAsUsername` set in the