Repository: kubernetes-sigs/security-profiles-operator
Pull request #1156 (2022-09-08 20:11:26)

recording: Emit event when attempting to record a privileged container + add docs
<!--  Thanks for sending a pull request!  Here are some tips for you:

- If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
- Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
- If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
- If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?
/kind cleanup
/kind documentation

<!--
Add one of the following kinds:
/kind feature
/kind design

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

#### What this PR does / why we need it:
- docs: Document that the log based recorder can't record privileged pods - adds docs that make it clear that we can't record privileged containers using the log enricher
- recording: Issue an event if a privileged pod is recorded with a log-based recorder - when log enricher is used to record privileged containers, attach an event to the recording object

#### Which issue(s) this PR fixes:
None

<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.

Fixes #

or

None
-->

#### Does this PR have test?
No

<!--
If tests aren't applicable just write N/A.
-->

#### Special notes for your reviewer:
N/A

#### Does this PR introduce a user-facing change?

<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->

```release-note
If the log-based recorder is in use and the user attempts to either record a container which already had its SecurityContext set or attempts to record a privileged container (which ignores both seccomp profiles and selinux contexts), the profile recording webhook issues a warning event.
```


k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1156#" title="Author self-approved">jhrozek</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fsecurity-profiles-operator).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/security-profiles-operator/blob/main/OWNERS)~~ [jhrozek]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
k8s-ci-robot: @jhrozek: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-security-profiles-operator-verify | dcafcb18d605d28c004c1c05356d40abdadef93d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_security-profiles-operator/1156/pull-security-profiles-operator-verify/1567968880540782592) | true | `/test pull-security-profiles-operator-verify`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=security-profiles-operator&pr=1156). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Ajhrozek). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/security-profiles-operator/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @jhrozek: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-security-profiles-operator-verify | 05ba8cce125dd385426fb9b0ac7e7fc216969e8a | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_security-profiles-operator/1156/pull-security-profiles-operator-verify/1567970305312296960) | true | `/test pull-security-profiles-operator-verify`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=security-profiles-operator&pr=1156). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Ajhrozek). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/security-profiles-operator/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @jhrozek: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-security-profiles-operator-verify | 81ad4d788bc2eb7731d0ddab484460513df82f1d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_security-profiles-operator/1156/pull-security-profiles-operator-verify/1567982592005771264) | true | `/test pull-security-profiles-operator-verify`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=security-profiles-operator&pr=1156). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Ajhrozek). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/security-profiles-operator/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1156#pullrequestreview-1101724278" title="Approved">JAORMX</a>*, *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1156#" title="Author self-approved">jhrozek</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fsecurity-profiles-operator).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/security-profiles-operator/blob/main/OWNERS)~~ [JAORMX,jhrozek]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
JAORMX: /lgtm cancel
@jhrozek This is actually a user facing change, so this should have a relevant release note.
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1156#pullrequestreview-1101724278" title="Approved">JAORMX</a>*, *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1156#" title="Author self-approved">jhrozek</a>*, *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1156#pullrequestreview-1102077233" title="Approved">saschagrunert</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fsecurity-profiles-operator).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/security-profiles-operator/blob/main/OWNERS)~~ [JAORMX,jhrozek,saschagrunert]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->

Patch:
diff --git a/cmd/security-profiles-operator/main.go b/cmd/security-profiles-operator/main.go
index d43d816aa..d6d27e2e7 100644
--- a/cmd/security-profiles-operator/main.go
+++ b/cmd/security-profiles-operator/main.go
@@ -514,7 +514,7 @@ func runWebhook(ctx *cli.Context, info *version.Info) error {
 	setupLog.Info("registering webhooks")
 	hookserver := mgr.GetWebhookServer()
 	binding.RegisterWebhook(hookserver, mgr.GetClient())
-	recording.RegisterWebhook(hookserver, mgr.GetClient())
+	recording.RegisterWebhook(hookserver, mgr.GetEventRecorderFor("recording-webhook"), mgr.GetClient())
 
 	sigHandler := ctrl.SetupSignalHandler()
 	setupLog.Info("starting webhook")
diff --git a/installation-usage.md b/installation-usage.md
index 34aad82e5..854314cf3 100644
--- a/installation-usage.md
+++ b/installation-usage.md
@@ -453,6 +453,12 @@ test-recording-redis   Installed   15s
 Recording a SELinux profile would work the same, except you'd use `kind: SelinuxProfile`
 in the `ProfileRecording` object.
 
+Please note that log based recording does not have any effect if the recorded container
+is privileged, that is, the container's security context sets `privileged: true`. This
+is because privileged containers are not subject to SELinux or seccomp policies at all
+and the log based recording makes use of a special seccomp or SELinux profile respectively
+to record the syscalls or SELinux events.
+
 #### eBPF based recording
 
 The operator also supports an [eBPF](https://ebpf.io) based recorder. This
diff --git a/internal/pkg/webhooks/recording/recording.go b/internal/pkg/webhooks/recording/recording.go
index 679eba46a..6f8b8cef0 100644
--- a/internal/pkg/webhooks/recording/recording.go
+++ b/internal/pkg/webhooks/recording/recording.go
@@ -28,6 +28,7 @@ import (
 	corev1 "k8s.io/api/core/v1"
 	kerrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/labels"
+	"k8s.io/client-go/tools/record"
 	"sigs.k8s.io/controller-runtime/pkg/client"
 	"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
 	logf "sigs.k8s.io/controller-runtime/pkg/log"
@@ -44,16 +45,18 @@ const finalizer = "active-seccomp-profile-recording-lock"
 
 type podSeccompRecorder struct {
 	impl
-	log logr.Logger
+	log    logr.Logger
+	record *utils.SafeRecorder
 }
 
-func RegisterWebhook(server *webhook.Server, c client.Client) {
+func RegisterWebhook(server *webhook.Server, rec record.EventRecorder, c client.Client) {
 	server.Register(
 		"/mutate-v1-pod-recording",
 		&webhook.Admission{
 			Handler: &podSeccompRecorder{
-				impl: &defaultImpl{client: c},
-				log:  logf.Log.WithName("recording"),
+				impl:   &defaultImpl{client: c},
+				log:    logf.Log.WithName("recording"),
+				record: utils.NewSafeRecorder(rec),
 			},
 		},
 	)
@@ -194,6 +197,8 @@ func (p *podSeccompRecorder) updatePod(
 			return false, err
 		}
 
+		p.warnEventIfContainerPrivileged(profileRecording, ctr, pod)
+
 		p.updateSecurityContext(ctr, profileRecording)
 		existingValue, ok := pod.GetAnnotations()[key]
 		if !ok {
@@ -236,9 +241,9 @@ func (p *podSeccompRecorder) updateSecurityContext(
 
 	switch pr.Spec.Kind {
 	case profilerecordingv1alpha1.ProfileRecordingKindSeccompProfile:
-		p.updateSeccompSecurityContext(ctr)
+		p.updateSeccompSecurityContext(ctr, pr)
 	case profilerecordingv1alpha1.ProfileRecordingKindSelinuxProfile:
-		p.updateSelinuxSecurityContext(ctr)
+		p.updateSelinuxSecurityContext(ctr, pr)
 	}
 
 	p.log.Info(fmt.Sprintf(
@@ -249,6 +254,7 @@ func (p *podSeccompRecorder) updateSecurityContext(
 
 func (p *podSeccompRecorder) updateSeccompSecurityContext(
 	ctr *corev1.Container,
+	pr *profilerecordingv1alpha1.ProfileRecording,
 ) {
 	if ctr.SecurityContext == nil {
 		ctr.SecurityContext = &corev1.SecurityContext{}
@@ -256,6 +262,11 @@ func (p *podSeccompRecorder) updateSeccompSecurityContext(
 
 	if ctr.SecurityContext.SeccompProfile == nil {
 		ctr.SecurityContext.SeccompProfile = &corev1.SeccompProfile{}
+	} else {
+		p.record.Eventf(pr,
+			corev1.EventTypeWarning,
+			"SecurityContextAlreadySet",
+			"Container %s had SecurityContext already set, the profile recorder overwrote it", ctr.Name)
 	}
 
 	ctr.SecurityContext.SeccompProfile.Type = corev1.SeccompProfileTypeLocalhost
@@ -269,6 +280,7 @@ func (p *podSeccompRecorder) updateSeccompSecurityContext(
 
 func (p *podSeccompRecorder) updateSelinuxSecurityContext(
 	ctr *corev1.Container,
+	pr *profilerecordingv1alpha1.ProfileRecording,
 ) {
 	if ctr.SecurityContext == nil {
 		ctr.SecurityContext = &corev1.SecurityContext{}
@@ -276,6 +288,11 @@ func (p *podSeccompRecorder) updateSelinuxSecurityContext(
 
 	if ctr.SecurityContext.SELinuxOptions == nil {
 		ctr.SecurityContext.SELinuxOptions = &corev1.SELinuxOptions{}
+	} else {
+		p.record.Eventf(pr,
+			corev1.EventTypeWarning,
+			"SecurityContextAlreadySet",
+			"Container %s had SecurityContext already set, the profile recorder overwrote it", ctr.Name)
 	}
 
 	ctr.SecurityContext.SELinuxOptions.Type = config.SelinuxPermissiveProfile
@@ -391,3 +408,22 @@ func (p *podSeccompRecorder) InjectDecoder(decoder *admission.Decoder) error {
 	p.impl.SetDecoder(decoder)
 	return nil
 }
+
+func (p *podSeccompRecorder) warnEventIfContainerPrivileged(
+	profileRecording *profilerecordingv1alpha1.ProfileRecording,
+	ctr *corev1.Container,
+	pod *corev1.Pod,
+) {
+	if profileRecording.Spec.Recorder != profilerecordingv1alpha1.ProfileRecorderLogs {
+		return
+	}
+
+	if ctr.SecurityContext == nil || ctr.SecurityContext.Privileged == nil || !*ctr.SecurityContext.Privileged {
+		return
+	}
+
+	p.record.Eventf(profileRecording,
+		corev1.EventTypeWarning,
+		"PrivilegedContainer",
+		"Container %s in pod %s is privileged, cannot use log-based profile recording", ctr.Name, pod.Name)
+}
diff --git a/internal/pkg/webhooks/utils/safe_events.go b/internal/pkg/webhooks/utils/safe_events.go
new file mode 100644
index 000000000..bde3d564f
--- /dev/null
+++ b/internal/pkg/webhooks/utils/safe_events.go
@@ -0,0 +1,61 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package utils
+
+import (
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/client-go/tools/record"
+)
+
+type SafeRecorder struct {
+	recorder record.EventRecorder
+}
+
+func NewSafeRecorder(recorder record.EventRecorder) *SafeRecorder {
+	return &SafeRecorder{recorder: recorder}
+}
+
+func (sr *SafeRecorder) Event(object runtime.Object, eventtype, reason, message string) {
+	if sr.recorder == nil {
+		return
+	}
+
+	sr.recorder.Event(object, eventtype, reason, message)
+}
+
+// Eventf is just like Event, but with Sprintf for the message field.
+func (sr *SafeRecorder) Eventf(object runtime.Object, eventtype, reason, messageFmt string, args ...interface{}) {
+	if sr.recorder == nil {
+		return
+	}
+
+	sr.recorder.Eventf(object, eventtype, reason, messageFmt, args...)
+}
+
+// AnnotatedEventf is just like eventf, but with annotations attached.
+func (sr *SafeRecorder) AnnotatedEventf(
+	object runtime.Object,
+	annotations map[string]string,
+	eventtype, reason, messageFmt string,
+	args ...interface{},
+) {
+	if sr.recorder == nil {
+		return
+	}
+
+	sr.recorder.AnnotatedEventf(object, annotations, eventtype, reason, messageFmt, args...)
+}


Test patch:
diff --git a/internal/pkg/webhooks/recording/recording_test.go b/internal/pkg/webhooks/recording/recording_test.go
index 19dcb4f2c..c75835c50 100644
--- a/internal/pkg/webhooks/recording/recording_test.go
+++ b/internal/pkg/webhooks/recording/recording_test.go
@@ -36,6 +36,7 @@ import (
 
 	"sigs.k8s.io/security-profiles-operator/api/profilerecording/v1alpha1"
 	"sigs.k8s.io/security-profiles-operator/internal/pkg/webhooks/recording/recordingfakes"
+	"sigs.k8s.io/security-profiles-operator/internal/pkg/webhooks/utils"
 )
 
 var (
@@ -499,7 +500,7 @@ func TestHandle(t *testing.T) {
 		mock := &recordingfakes.FakeImpl{}
 		tc.prepare(mock)
 
-		recorder := podSeccompRecorder{impl: mock, log: logr.Discard()}
+		recorder := podSeccompRecorder{impl: mock, log: logr.Discard(), record: utils.NewSafeRecorder(nil)}
 		resp := recorder.Handle(context.Background(), tc.request)
 		tc.assert(resp)
 	}