Change:
The e2e test should be relaxed. It currently checks 3 different aspects of the volume mount flag: with do not receive mounts from outside their own mount namespace, nor do mount namespaces outside the container receive mounts created within the container. with receive new mounts from the host OS, the container runtime, and other containers with , but no mount namespaces outside the container receive mounts created within the container. with receive new mounts identical to containers with , but additionally any mounts made by these containers are: propagated to the container runtime where they can be propagated to other containers (related to 2) also propagated to the Host OS The first 3 aspects, up to and including 3.i, are critical for container operation and should still be ensured by an e2e test. This last check, 3.ii., is the part that I believe is overly strict. The current e2e test is too opinionated on the underlying node OS configuration. The requirement of propagation to the host OS precludes administrators or solution providers from modifying the Kubernetes installation to hide these Kubernetes-specific mountpoints, which can be for a variety of reasons: While access to container-specific mounts in Kubernetes (like secrets) today are protected by filesystem permissions, the fact that the locations are visible to unprivileged users via the 'mount' command can provide information which could be used in conjunction with other vulnerabilities to decipher secrets or other protected container mount contents. Locking access to even listing the mountpoints behind the ability to enter the mount namespace (which in turn requires and ) removes one level of potential access to container-specific mounts. These container overlay mounts, secret and configmap mounts, and namespace mounts are all internal implementation details around the way the Kubernetes and the Container Runtime allocate resources to containers. For normal day-to-day operation, there is no need for these special-use mounts to be visible in the host OS's top-level 'mount' or 'findmnt' output. After much investigation and conversation on both the sig-node and sig-storage mailing lists, as well as within Red Hat's OpenShift communities, we have yet to turn up a use case that relies on this container-to-host propagation. In fact, it may be seen as an anti-pattern and run contrary to the container-native way of doing things. Relying on a host OS to interact with a container-owned mountpoint seems suspect. Today systemd's CPU usage is adversely affected by a large number of mountpoints, especially versions released prior to December 2020. This feature of systemd, which scans all /proc/1/mount to invoke udev events for every change suffers from two issues. The first was an inefficiency in systemd’s processing logic. This was fixed in December 2020, and this fix reportedly reduced the systemd mount-scanning-related overhead by 30-50%. The second is a lack of granularity in the events the kernel sends to systemd about mountpoint changes, which necessitates a full rescan of all mountpoints even on a minor change or single mountpoint addition. The kernel change to remedy this seems unlikely to move forward in the near term. (See for more details) For example, a fairly small OpenShift deployment today (which does not yet have the systemd December 2020 fix) has upwards of 400 mountpoints on every node owned by Kubelet, the Container Runtime, and infrastructure containers like OVN or CSI operators. Hiding the container mountpoints reduces the steady-state systemd CPU utilization from ~10% to ~0% on every worker node. On a more heavily loaded system, with 1000’s of container mountpoints, systemd’s usage will cap out at 100% of a single CPU core. Even with the 30-50% reduction in systemd usage expected from the December 2020 fix, having systemd take up as much as 50-70% of a single CPU in the worst-case scenario is a lot of unnecessary overhead. Administrators of other OSes with other host OS services that monitor or process mountpoints indiscriminately would be able to hide these container-specific mountpoints from the OS at large, at their option. With this requirement relaxed, it would be possible to run multiple instances of Kubernetes on a single host without the various mountpoints interfering with one another. There may be other inter-instance difficulties besides the mount namespace, but this does move in that direction without resorting to more complicated solutions like trying to run Kubernetes inside of containers.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig architecture storage /area conformance
I have a proof-of-concept of the relaxed test as well as some more and an OpenShift-based proof-of-concept of an actual implementation with the container mounts in their own namespace
/sig node
There's still a number of other issues, no? Is this a real motivation / use-case? There are other difficulties. Most of the difficulties with running Kubernetes inside containers are just the difficulties in sharing a kernel / host OS, which aren't related to this at all. The security and performance angles seem relevant, in any case.
Update: I mistakenly referred to this as a "conformance test" - Turns out it is not in fact a conformance test; just an e2e test :) Edited the original post to be more correct in this respect.
/remove-area conformance /area test
Fair point. I this (intentionally vague) section since some conversation on the sig-node mailing list with folks working on KIND suggested this may align with their goals in some way I'm not entirely clear on.
A good next step on this issue is to open a PR to the related e2e tests that documents why the tests are not good candidates for future inclusion in conformance. Is that something you can assist with ?
I have a branch that relaxes the test so it works in the current environment and a separated mount namespace environment... Can you explain a bit more what you mean about documenting this? Where would that kind of documentation go?

Discussion:
import ( "fmt" <ins> "strings" </ins> v1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

var _ = SIGDescribe("Mount propagation", func() { f := framework.NewDefaultFramework("mount-propagation") <del> ginkgo.It("should propagate mounts to the host", func() { </del> <ins> ginkgo.It("should propagate mounts within defined scopes", func() { </ins> // This test runs two pods: master and slave with respective mount // propagation on common /var/lib/kubelet/XXXX directory. Both mount a // tmpfs to a subdirectory there. We check that these mounts are

} } } <del> // Check that the mounts are/are not propagated to the host. // Host can see mount from master cmd = fmt.Sprintf("test `cat %q/master/file` = master", hostDir) err = hostExec.IssueCommand(cmd, node) framework.ExpectNoError(err, "host should see mount from master") // Host can't see mount from slave cmd = fmt.Sprintf("test ! -e %q/slave/file", hostDir) err = hostExec.IssueCommand(cmd, node) framework.ExpectNoError(err, "host shouldn't see mount from slave") </del> <ins> // Find the kubelet PID to ensure we're working with the kubelet's mount namespace cmd = "pidof kubelet" kubeletPid, err := hostExec.IssueCommandWithResult(cmd, node) framework.ExpectNoError(err, "Checking kubelet pid") kubeletPid = strings.TrimSuffix(kubeletPid, "n") framework.ExpectEqual(strings.Count(kubeletPid, " "), 0, "kubelet should only have a single PID in the system (pidof returned %q)", kubeletPid) enterKubeletMountNS := fmt.Sprintf("nsenter -t %s -m", kubeletPid) // Check that the master and host mounts are propagated to the container runtime's mount namespace for _, mountName := range []string{"host", master.Name} { cmd := fmt.Sprintf("%s cat "%s/%s/file"", enterKubeletMountNS, hostDir, mountName) output, err := hostExec.IssueCommandWithResult(cmd, node) framework.ExpectNoError(err, "host container namespace should see mount from %s: %s", mountName, output) output = strings.TrimSuffix(output, "n") framework.ExpectEqual(output, mountName, "host container namespace should see mount contents from %s", mountName) } // Check that the slave, private, and default mounts are not propagated to the container runtime's mount namespace for _, podName := range []string{slave.Name, private.Name, defaultPropagation.Name} { cmd := fmt.Sprintf("%s test ! -e "%s/%s/file"", enterKubeletMountNS, hostDir, podName) output, err := hostExec.IssueCommandWithResult(cmd, node) framework.ExpectNoError(err, "host container namespace shouldn't see mount from %s: %s", podName, output) } </ins> }) })