Repository: kubernetes
Issue: Automatic `fsGroup` handling
Labels: sig/node,kind/feature,needs-triage

### What would you like to be added?

> [!NOTE]  
> The template for feature requests says: _"Feature requests are unlikely to make progress as issues. Please consider engaging with SIGs on slack and mailing lists, instead."_
> I did start a [thread on Slack](https://kubernetes.slack.com/archives/C0BP8PW9G/p1732711766035909) and was asked to create an issue instead.

A feature to allow "automatic" handling of `fsGroup`. Currently users have to pick a number while the actual number often will not matter much. I'd like a feature asking Kubernetes to assign a `fsGroup` automatically if none has been specified explicitly or e.g. via a webhook/controller/etc.

This is part feature request part request for help in figuring out best practices, so I'd also be happy to just have a good discussion.
When researching this issue I would have _loved_ to find this exact issue to save me a lot of time.
And I do understand (more on that later) that this is partially due to OpenShift specifics and therefore not relevant to _this_ project.

/sig node

### Why is this needed?

We need to mount a few ephemeral volumes into a pod/container.
These mounts are accessible only to root, meaning the container user (non-root) cannot access them without setting `securityContext.fsGroup`, which is what we currently do in our operators (set a random `fsGroup` that is) but this has problems.

### Background

* Vanilla Kubernetes requires an `fsGroup` for non-root container users to access these volumes.
* OpenShift provides `SecurityContextConstraints` (SCC), which can automatically assign a valid fsGroup from a specified range if not explicitly set (depending on how exactly they are configured)
  * This helps us avoid specifying an fsGroup directly, which can simplify configuration
  * However, SCCs complicate things as they’re not available in vanilla Kubernetes
  * SCCs can put constraints on the allowed `fsGroup` values

#### Summary
* `Vanilla Kubernetes` requires an explicit fsGroup
  * If not set: Containers fail to access volumes
* `OpenShift` behavior will depend on the exact `SCC` active (it's not possible to find out upfront which is):
  * It might fail if a `fsGroup` is set and the group is out of the specified range -> Better solution here is to set no `fsGroup` so OpenShift can assign one automatically
  * It might fail if no `fsGroup` is set and OpenShift is set up to not assign one automatically

This leaves us with no good option.

### Current Proposed Solution/Idea

> [!NOTE]  
> This came out of a [longer discussion](https://kubernetes.slack.com/archives/CAW0GV7A5/p1730929137603739) in #kubernetes-operator on Slack.

* Detect OpenShift (e.g., by querying the SCC API)
* If OpenShift:
  * Set a special annotation requiring the `restricted-v2` SCC, allowing automatic fsGroup assignment.
  * Don’t set securityContext.fsGroup.
* Else (vanilla Kubernetes):
  * Hardcode fsGroup.

One could argue that this is an OpenShift-specific problem that we’ve addressed.
However, I wonder if there’s value in a generalized feature allowing for an automatically assigned fsGroup—without having to specify one manually—when the specific integer id to use is non-essential.

### Suggestion

* Leave the current functionality of `securityContext.fsGroup` as it is:
  * If set to an integer use that
  * If not set, don't set `fsGroup` at all
* Add a new value `auto` that will automatically pick a `fsGroup` value if none has been set
  * This is just _one idea_, this could be implemented multiple ways and I do understand that picking the "wrong" group Id might also have security implications