Change:
This issue was reported in the Description The isKernelPid function (Figure 1) checks if a given PID is a kernel PID by checking whether readlink of /proc/<pid/exe returns an error. This check is used to filter out kernel processes and move all other processes that were found in the root device’s cgroup to potentially less privileged manager’s cgroup (Figure 2). The check performed by isKernelPid is too broad. It is possible to create a process that will be filtered as a kernel PID and not moved into potentially less privileged device cgroup. A readlink of kernel process’ /proc/<pid/exe returns an ENOENT error (Figure 3). It is possible to make this operation return another error, for example, by putting the file in a too-long path (Figure 4). Despite the fact that the isKernelPid check can be bypassed, it is only invoked on the processes from root (“/”) devices cgroup and only in non-default kubelet configuration. This is when system cgroups name is set and the cgroup root is “/” (Figure 5), which can be set by passing: --system-cgroups=/something --cgroup-root=/ to kubelet arguments. Exploiting this issue requires the attacker to control a process in the root device cgroup and a privileged user with CAPSYSADMIN capability, which is present by default and must be explicitly dropped to modify the rules for device cgroups. Exploitation is, therefore, unlikely. Figure 35.1: The isKernelPid function in Figure 35.2 The ensureSystemCgroups calls isKernelPid to filter out kernel PIDs from processes from “/” devices cgroup (as the rootCgroupPath argument is hardcoded to “/” and cmutils.GetPids gets pids for given devices cgroup) and then moves those non-kernel PIDs to manager’s cgroup. Figure 35.3 Reading link of a kernel process results in ENOENT. Note that we read the link as root, if we did as unprivileged user, we would get EACESS error. Figure 35.4 Making readlink /proc/<pid/exe return a ENAMETOOLONG error via putting the binary in a too-long path. Figure 35.5 ensureSystemCgroups is called only if the systemCgroupsName (--system-cgroups) configuration parameter is not empty (which needs to be specified along with --cgroup-root parameter). Exploit Scenario An example of exploitation can be seen below, where a process spawned in a long path is not moved from the root device cgroup to another device cgroup. The process has been manually moved to the root cgroup via cgclassify, displayed in Figure 6. As a comparison, the standard and expected kubelet behavior is displayed in Figure 7, where the process is properly migrated to a different cgroup. Figure 35.6 Although kubelet found the attacker controlled process. It didn’t move it to another device cgroup since the process was put in a too-long path to trick the isKernelPid check. Figure 35.7 The standard behavior of kubelet moving the non-kernel system processes (the ones from root device cgroup) to the other cgroup. Recommendation isKernelPid should explicitly check the error returned from os.Readlink and return true only if the error value isENOENT. Anything else we need to know?: See for current status of all issues created from these findings. The vendor gave this issue an ID of TOB-K8S-027 and it was finding 36 of the report. The vendor considers this issue Informational Severity. To view the original finding, begin on page 85 of the Environment: Kubernetes version: 1.13.4
/sig node /area security /kind bug /remove-kind feature

Discussion:
func isKernelPid(pid int) bool { // Kernel threads have no associated executable. _, err := os.Readlink(fmt.Sprintf("/proc/%d/exe", pid)) <del> return err != nil </del> <ins> return err != nil && os.IsNotExist(err) </ins> } func (cm *containerManagerImpl) GetCapacity() v1.ResourceList {