Change:
What happened: After upgrading from kubelet 1.15 -1.16, volumes are no longer relabeled as containerfilet for selinux, causing most pods to fail. What you expected to happen: How to reproduce it (as minimally and precisely as possible): Run a kubernetes cluster on 1.15.[0-3]. Create a pod with a secret, configmap, or with provisioned storage (ebs-csi-driver). Find that file on disk of the node, and observe it is labeled with the file type containerfilet and the pod is able to read it. Upgrade the kubelet to 1.16 Delete the pod, and observe it fail on launch. Find the file on disk and see it labeled tmpfst or unlabeledt for configmap/secret and provisioned volumes, respectively. Anything else we need to know?: Environment: Kubernetes version (use ): 1.15 / 1.16 Cloud provider or hardware configuration: AWS OS (e.g: ): RHEL 7.7 Kernel (e.g. ): 3.10.0-1062.1.1.el7.x86_64 Install tools: kubeadm Network plugin and version (if this is a network-related bug): N/A Others:
/sig storage
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Is selinux enabled for docker? Can you paster output of : also
Also please paste YAML of one or more pods which are affected by this. Thank you!
Security Options - Perhaps of note here, we do not change the installed by docker, and confirms we don't make changes to that file. Docker version: Client and server blocks match. Pod yaml incoming
To clarify, do you want a runnable "minimum reproducible" , or is it more informational? If the latter, I have a pod that breaks and I'm working on the yaml for that, removing sensitive information, etc. If the former, I'll need to spend a few minutes getting a minimal reproduction with busybox or somesuch.
I need full pod's yaml that is not working. i.e - generated full YAML.
Hi, I'm sorry I haven't had a chance to follow up with this ticket. I will get some pod yaml attached early next week. Thank you for following up.
I think I'm seeing the same thing, to reproduce, use the following resource YAML: On k8s v1.15.5 (hyperkube) On k8s 1.16.2 (hyperkube) Are you in Enforcing mode () ?
Hey thanks much for following on with example YAML. We've been totally swamped with work so I haven't had time to address this issue. To answer your question: we were seeing all pods with storage (configmaps, secrets, persistent volumes) failing in Enforcing mode, and disabling Enforcement () allowed the same pods to successfully run, with no problem. There was not as much output in audit logs as we would have expected, so constructing custom policies to allow access was a bit of shooting in the dark. We were able to get configmaps and secrets to work passably with a custom policy allowing limited access to tmpfs labeled items, but we can't get away with allowing access to unlabeledt in our compliance environment. Thanks again for following up with more info and linking up this ticket, and let me know if I can provide any more helpful information. We've rolled back to 15.3 for the time being, so I'll be constructing knowledge somewhat from memory, but the upgrade was run recently, and I took reasonably detailed notes, so my info is be largely accurate.
cc
Can someone post pod's YAML after pod was created. I am looking for some generated fields in the Pod.
I finally had the chance today to dig a bit deeper why SELinux is broken in 1.16+ In the linked PR you can see the fix. See the details in there. If you want to give that a try, I have build a binaries here: A workaround I have used in the meantime was: My reproduction of the bug can be found there: If you run the packer build the pods fail because of the SELinux problem
nice work! looks like the patch we need is this one right?
if so, can you please propose a PR against master of kubernetes/kubernetes?
Reopening to track adding a test.
On second thought, spawned a separate issue to track adding test coverage. /close
Closing this issue. <detailsIn response to : On second thought, spawned a separate issue to track adding test coverage. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
we have just updated our cluster to 1.16.5, and even 1.16.6-beta.0, but the secret volume is still labeled as tmpfst:
I see the same result on the test above on v1.17.2
I can confirm the release binaries are not build correctly I have opened this issue in the release repo: /reopen Workaround is building the binaries yourself using bazel and the release tag
Reopened this issue. <detailsIn response to : I can confirm the release binaries are not build correctly I have opened this issue in the release repo: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
can someone confirm that 1.16.7 / 1.17.3 binaries are built with the selinux tags in make targets?
No they haven't as haven't been merged. Feel free to ask on those PRs again, I have not been successful, convincing that this should be merged with at least some priority
I am able to run this repro and get the desired outcome on 1.17.4. However, I am trying to attach Cinder volume whose context doesn't automatically get relabeled as containerfilet when I attempt to attach volumes with both in-tree and Cinder CSI plugin. I have filed this bug in the cloud-provider-openstack repo but not sure if this is a Kubernetes problem or a cinder-csi-plugin problem:
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten /assign are looking at adding selinux to CI using kops
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Discussion:
# Make /tmp hermetic build --sandbox_tmpfs_path=/tmp <ins> # This tag is required to build github.com/opencontainers/selinux correctly build --define gotags=selinux </ins> # Ensure that Bazel never runs as root, which can cause unit tests to fail. # This flag requires Bazel 0.5.0+ build --sandbox_fake_username

# gazelle:exclude _output # gazelle:exclude _tmp <ins> # This tag is required to build github.com/opencontainers/selinux correctly # gazelle:build_tags selinux </ins> # gazelle:prefix k8s.io/kubernetes # Disable proto rules, since the Go sources are currently generated by

go_library( name = "go_default_library", <del> srcs = ["selinux_stub.go"], </del> <ins> srcs = [ "selinux_linux.go", "xattrs.go", ], </ins> importmap = "k8s.io/kubernetes/vendor/github.com/opencontainers/selinux/go-selinux", importpath = "github.com/opencontainers/selinux/go-selinux", visibility = ["//visibility:public"], <ins> deps = select({ "@io_bazel_rules_go//go/platform:android": [ "//vendor/golang.org/x/sys/unix:go_default_library", ], "@io_bazel_rules_go//go/platform:linux": [ "//vendor/golang.org/x/sys/unix:go_default_library", ], "//conditions:default": [], }), </ins> ) filegroup(

go_library( name = "go_default_library", <del> srcs = ["label.go"], </del> <ins> srcs = [ "label.go", "label_selinux.go", ], </ins> importmap = "k8s.io/kubernetes/vendor/github.com/opencontainers/selinux/go-selinux/label", importpath = "github.com/opencontainers/selinux/go-selinux/label", visibility = ["//visibility:public"], <ins> deps = select({ "@io_bazel_rules_go//go/platform:android": [ "//vendor/github.com/opencontainers/selinux/go-selinux:go_default_library", ], "@io_bazel_rules_go//go/platform:linux": [ "//vendor/github.com/opencontainers/selinux/go-selinux:go_default_library", ], "//conditions:default": [], }), </ins> ) filegroup(