Repository: kubernetes-sigs/security-profiles-operator
Pull request #1110 (2022-08-23 10:06:23)

Process bpf events in parallel
#### What type of PR is this?

/kind feature

#### What this PR does / why we need it:
We now spawn a go routine for each event to be able to process them in parallel.

#### Which issue(s) this PR fixes:

None

#### Does this PR have test?

None

#### Special notes for your reviewer:

None

#### Does this PR introduce a user-facing change?


```release-note
Updated BPF event processing to run in parallel.
```


k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1110#" title="Author self-approved">saschagrunert</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fsecurity-profiles-operator).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/security-profiles-operator/blob/main/OWNERS)~~ [saschagrunert]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
codecov-commenter: # [Codecov](https://codecov.io/gh/kubernetes-sigs/security-profiles-operator/pull/1110?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
> Merging [#1110](https://codecov.io/gh/kubernetes-sigs/security-profiles-operator/pull/1110?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (bc1ed8f) into [main](https://codecov.io/gh/kubernetes-sigs/security-profiles-operator/commit/c0fd9a489b852917726f72b4aac79aa7ec44332b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (c0fd9a4) will **increase** coverage by `0.01%`.
> The diff coverage is `100.00%`.

```diff
@@            Coverage Diff             @@
##             main    #1110      +/-   ##
==========================================
+ Coverage   50.68%   50.69%   +0.01%     
==========================================
  Files          42       42              
  Lines        4735     4736       +1     
==========================================
+ Hits         2400     2401       +1     
  Misses       2259     2259              
  Partials       76       76              
```

k8s-ci-robot: @saschagrunert: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-security-profiles-operator-verify | df6285690e72ec86a50b7d53e8fcd9fa0256b8aa | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_security-profiles-operator/1110/pull-security-profiles-operator-verify/1562033892041428992) | true | `/test pull-security-profiles-operator-verify`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=security-profiles-operator&pr=1110). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Asaschagrunert). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/security-profiles-operator/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
JAORMX: /approve

@pjbgf @jhrozek could you also take a look?
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1110#pullrequestreview-1082022097" title="Approved">JAORMX</a>*, *<a href="https://github.com/kubernetes-sigs/security-profiles-operator/pull/1110#" title="Author self-approved">saschagrunert</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fsecurity-profiles-operator).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/security-profiles-operator/blob/main/OWNERS)~~ [JAORMX,saschagrunert]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
JAORMX: /lgtm cancel

waiting for feedback from other maintainers

Patch:
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 945fafa08..ef45d9288 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -5,7 +5,7 @@ on:
       - main
   pull_request:
 env:
-  GO_VERSION: '1.18'
+  GO_VERSION: '1.19'
 permissions:
   actions: none
   checks: none
@@ -24,7 +24,7 @@ jobs:
     runs-on: macos-12
     steps:
       - uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b
-      - uses: actions/setup-go@84cbf8094393cdc5fe1fe1671ff2647332956b1a
+      - uses: actions/setup-go@268d8c0ca0432bb2cf416faae41297df9d262d7f
         with:
           go-version: ${{ env.GO_VERSION }}
       - run: make test-unit
@@ -45,7 +45,7 @@ jobs:
   bpf-btf:
     runs-on: ubuntu-20.04
     steps:
-      - uses: actions/setup-go@84cbf8094393cdc5fe1fe1671ff2647332956b1a
+      - uses: actions/setup-go@268d8c0ca0432bb2cf416faae41297df9d262d7f
         with:
           go-version: ${{ env.GO_VERSION }}
       - uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b
diff --git a/OWNERS b/OWNERS
index 06c1fa0a8..05016f01f 100644
--- a/OWNERS
+++ b/OWNERS
@@ -1,10 +1,10 @@
 # See the OWNERS docs at https://go.k8s.io/owners
 
 approvers:
-  - hasheddan
   - pjbgf
   - saschagrunert
   - JAORMX
   - jhrozek
-reviewers:
+emeritus_approvers:
   - cmurphy
+  - hasheddan
diff --git a/dependencies.yaml b/dependencies.yaml
index 3c7da41bb..7bc3227a9 100644
--- a/dependencies.yaml
+++ b/dependencies.yaml
@@ -1,6 +1,6 @@
 dependencies:
   - name: go
-    version: 1.18
+    version: 1.19
     refPaths:
     - path: .github/workflows/build.yml
       match: GO_VERSION
@@ -12,6 +12,8 @@ dependencies:
       match: GO_VERSION
     - path: hack/ci/Vagrantfile-ubuntu
       match: GO_VERSION
+    - path: hack/ci/Vagrantfile-flatcar
+      match: GO_VERSION
 
   - name: golangci-lint
     version: 1.48.0
@@ -176,5 +178,9 @@ dependencies:
       match: ^.*
     - path: bundle/manifests/security-profiles-operator.clusterserviceversion.yaml
       match: "^  name: security-profiles-operator"
+    - path: deploy/helm/Chart.yaml
+      match: "^appVersion: "
+    - path: deploy/helm/Chart.yaml
+      match: "^version: "
     - path: installation-usage.md
       match: ^helm install security-profiles-operator
diff --git a/deploy/helm/Chart.yaml b/deploy/helm/Chart.yaml
index 6fd1339b8..631029627 100644
--- a/deploy/helm/Chart.yaml
+++ b/deploy/helm/Chart.yaml
@@ -2,5 +2,5 @@ apiVersion: v2
 name: security-profiles-operator
 description: "The Kubernetes Security Profiles Operator."
 type: application
-version: "0.4.3"
-appVersion: "0.4.3"
+version: "0.4.4-dev"
+appVersion: "0.4.4-dev"
diff --git a/go.mod b/go.mod
index 13dc9da89..445fb7301 100644
--- a/go.mod
+++ b/go.mod
@@ -1,6 +1,6 @@
 module sigs.k8s.io/security-profiles-operator
 
-go 1.18
+go 1.19
 
 require (
 	github.com/acobaugh/osrelease v0.0.0-20181218015638-a93a0a55a249
diff --git a/hack/ci/Vagrantfile-fedora b/hack/ci/Vagrantfile-fedora
index 52e7c6400..eb7640ff5 100644
--- a/hack/ci/Vagrantfile-fedora
+++ b/hack/ci/Vagrantfile-fedora
@@ -24,7 +24,7 @@ Vagrant.configure("2") do |config|
       # Use a non-localhost DNS to avoid cluster DNS lookup loops
       echo "nameserver 8.8.8.8" > /etc/resolv.conf
 
-      GO_VERSION=1.18
+      GO_VERSION=1.19
       curl -sSfL -o- https://dl.google.com/go/go"$GO_VERSION".linux-amd64.tar.gz |
         tar xfz - -C /usr/local
 
diff --git a/hack/ci/Vagrantfile-flatcar b/hack/ci/Vagrantfile-flatcar
index ae2f99e9a..045a71e46 100644
--- a/hack/ci/Vagrantfile-flatcar
+++ b/hack/ci/Vagrantfile-flatcar
@@ -6,14 +6,14 @@ Vagrant.configure("2") do |config|
   config.vm.box_check_update = true
   config.vm.box_url = "https://lts.release.flatcar-linux.net/amd64-usr/current/flatcar_production_vagrant.json"
   memory = 8192
-  cpus = 4 
+  cpus = 4
 
   config.vm.provider :virtualbox do |v|
      v.customize [
       "modifyvm", :id,
       "--memory", memory,
       "--paravirtprovider", "default",
-      "--cpus", cpus 
+      "--cpus", cpus
     ]
     v.check_guest_additions = false
     v.functional_vboxsf = false
@@ -53,12 +53,12 @@ Vagrant.configure("2") do |config|
       systemctl enable --now containerd
       systemctl restart containerd.service
 
-      # Import the test image of security profiles operator 
+      # Import the test image of security profiles operator
       ctr -n=k8s.io image import /vagrant/image.tar
       crictl --runtime-endpoint="unix:///run/containerd/containerd.sock" images
 
       # Install Go
-      GO_VERSION=1.18
+      GO_VERSION=1.19
       mkdir /opt/go
       curl -sSfL -o- https://golang.org/dl/go$GO_VERSION.linux-amd64.tar.gz | tar xfz - -C /opt
 
@@ -68,7 +68,7 @@ Vagrant.configure("2") do |config|
       br_netfilter
       EOF
 
-      # Setup Networking 
+      # Setup Networking
       cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
       net.bridge.bridge-nf-call-iptables  = 1
       net.ipv4.ip_forward                 = 1
diff --git a/hack/ci/Vagrantfile-ubuntu b/hack/ci/Vagrantfile-ubuntu
index 6b5ed1b48..358d1d7b8 100644
--- a/hack/ci/Vagrantfile-ubuntu
+++ b/hack/ci/Vagrantfile-ubuntu
@@ -25,7 +25,7 @@ Vagrant.configure("2") do |config|
       # Use a non-localhost DNS to avoid cluster DNS lookup loops
       echo "nameserver 8.8.8.8" > /etc/resolv.conf
 
-      GO_VERSION=1.18
+      GO_VERSION=1.19
       curl -sSfL -o- https://dl.google.com/go/go"$GO_VERSION".linux-amd64.tar.gz |
         tar xfz - -C /usr/local
 
diff --git a/internal/pkg/daemon/bpfrecorder/bpfrecorder.go b/internal/pkg/daemon/bpfrecorder/bpfrecorder.go
index 3d4cfd7b9..e99199e84 100644
--- a/internal/pkg/daemon/bpfrecorder/bpfrecorder.go
+++ b/internal/pkg/daemon/bpfrecorder/bpfrecorder.go
@@ -39,6 +39,7 @@ import (
 	"github.com/go-logr/logr"
 	"github.com/jellydator/ttlcache/v3"
 	seccomp "github.com/seccomp/libseccomp-golang"
+	"golang.org/x/sync/semaphore"
 	"google.golang.org/grpc"
 	"google.golang.org/grpc/credentials/insecure"
 	"k8s.io/apimachinery/pkg/util/wait"
@@ -545,68 +546,83 @@ func toStringByte(array []byte) string {
 
 func (b *BpfRecorder) processEvents(events chan []byte) {
 	b.logger.Info("Processing events")
+
+	// Allow up to 1000 goroutines to run in parallel
+	const workers = 1000
+	sem := semaphore.NewWeighted(workers)
+
 	for event := range events {
-		func() {
-			// Newly arrived PIDs
-			const eventLen = 16
-			if len(event) != eventLen {
-				b.logger.Info("Invalid event length", "len", len(event))
-				return
-			}
+		if err := sem.Acquire(context.Background(), 1); err != nil {
+			b.logger.Error(err, "Unable to acquire semaphore, stopping event processor")
+			break
+		}
+		eventCopy := event
+		go func() {
+			b.handleEvent(eventCopy)
+			sem.Release(1)
+		}()
+	}
+}
 
-			pid := binary.LittleEndian.Uint32(event)
-			mntns := binary.LittleEndian.Uint64(event[8:])
+func (b *BpfRecorder) handleEvent(event []byte) {
+	// Newly arrived PIDs
+	const eventLen = 16
+	if len(event) != eventLen {
+		b.logger.Info("Invalid event length", "len", len(event))
+		return
+	}
 
-			// Filter out non-containers
-			if mntns == b.systemMountNamespace {
-				b.logger.V(config.VerboseLevel).Info(
-					"Skipping PID, because it's on the system mount namespace",
-					"pid", pid, "mntns", mntns,
-				)
-				return
-			}
+	pid := binary.LittleEndian.Uint32(event)
+	mntns := binary.LittleEndian.Uint64(event[8:])
 
-			// Blocking from syscall retrieval when PIDs are currently being analyzed
-			b.pidLock.Lock()
-			defer b.pidLock.Unlock()
+	// Filter out non-containers
+	if mntns == b.systemMountNamespace {
+		b.logger.V(config.VerboseLevel).Info(
+			"Skipping PID, because it's on the system mount namespace",
+			"pid", pid, "mntns", mntns,
+		)
+		return
+	}
 
-			// Short path via the mount namespace
-			if profile, exist := b.profileForMountNamespace.Load(mntns); exist {
-				b.logger.Info(
-					"Using short path via tracked mount namespace",
-					"pid", pid, "mntns", mntns, "profile", profile,
-				)
-				b.trackProfileForPid(pid, mntns, profile)
-				return
-			}
+	// Blocking from syscall retrieval when PIDs are currently being analyzed
+	b.pidLock.Lock()
+	defer b.pidLock.Unlock()
 
-			// Regular container ID retrieval via the cgroup
-			containerID, err := b.ContainerIDForPID(b.containerIDCache, int(pid))
-			if err != nil {
-				b.logger.V(config.VerboseLevel).Info(
-					"No container ID found for PID",
-					"pid", pid, "err", err.Error(),
-				)
-				return
-			}
+	// Short path via the mount namespace
+	if profile, exist := b.profileForMountNamespace.Load(mntns); exist {
+		b.logger.Info(
+			"Using short path via tracked mount namespace",
+			"pid", pid, "mntns", mntns, "profile", profile,
+		)
+		b.trackProfileForPid(pid, mntns, profile)
+		return
+	}
 
-			b.logger.V(config.VerboseLevel).Info(
-				"Found container for PID", "pid", pid, "containerID", containerID,
-			)
-			if err := b.findContainerID(containerID); err != nil {
-				b.logger.Error(err, "unable to find container ID in cluster")
-				return
-			}
+	// Regular container ID retrieval via the cgroup
+	containerID, err := b.ContainerIDForPID(b.containerIDCache, int(pid))
+	if err != nil {
+		b.logger.V(config.VerboseLevel).Info(
+			"No container ID found for PID",
+			"pid", pid, "err", err.Error(),
+		)
+		return
+	}
 
-			if profile, exist := b.profileForContainerIDs.LoadAndDelete(containerID); exist {
-				b.logger.Info(
-					"Saving PID for profile",
-					"pid", pid, "mntns", mntns, "profile", profile,
-				)
-				b.trackProfileForPid(pid, mntns, profile)
-				b.profileForMountNamespace.Store(mntns, profile)
-			}
-		}()
+	b.logger.V(config.VerboseLevel).Info(
+		"Found container for PID", "pid", pid, "containerID", containerID,
+	)
+	if err := b.findContainerID(containerID); err != nil {
+		b.logger.Error(err, "unable to find container ID in cluster")
+		return
+	}
+
+	if profile, exist := b.profileForContainerIDs.LoadAndDelete(containerID); exist {
+		b.logger.Info(
+			"Saving PID for profile",
+			"pid", pid, "mntns", mntns, "profile", profile,
+		)
+		b.trackProfileForPid(pid, mntns, profile)
+		b.profileForMountNamespace.Store(mntns, profile)
 	}
 }
 
diff --git a/nix/derivation-bpf.nix b/nix/derivation-bpf.nix
index f0c70a7a9..8febeb639 100644
--- a/nix/derivation-bpf.nix
+++ b/nix/derivation-bpf.nix
@@ -1,5 +1,5 @@
 { pkgs, buildGoModule, arch ? "x86" }:
-with pkgs; buildGo118Module rec {
+with pkgs; buildGo119Module rec {
   name = "security-profiles-operator";
   src = builtins.filterSource
     (path: type: !(type == "directory" && baseNameOf path == "build")) ./..;
@@ -15,7 +15,10 @@ with pkgs; buildGo118Module rec {
     which
   ];
   buildInputs = [
-    (libseccomp.overrideAttrs (x: { dontDisableStatic = true; }))
+    (libseccomp.overrideAttrs (x: {
+      doCheck = false;
+      dontDisableStatic = true;
+    }))
     glibc
     glibc.static
     libbpf
diff --git a/nix/derivation.nix b/nix/derivation.nix
index 5a300cc5b..e11ead52a 100644
--- a/nix/derivation.nix
+++ b/nix/derivation.nix
@@ -1,5 +1,5 @@
 { pkgs, buildGoModule }:
-with pkgs; buildGo118Module rec {
+with pkgs; buildGo119Module rec {
   name = "security-profiles-operator";
   src = builtins.filterSource
     (path: type: !(type == "directory" && baseNameOf path == "build")) ./..;
@@ -12,7 +12,10 @@ with pkgs; buildGo118Module rec {
     which
   ];
   buildInputs = [
-    (libseccomp.overrideAttrs (x: { dontDisableStatic = true; }))
+    (libseccomp.overrideAttrs (x: {
+      doCheck = false;
+      dontDisableStatic = true;
+    }))
     elfutils
     glibc
     glibc.static
diff --git a/nix/nixpkgs.json b/nix/nixpkgs.json
index 2003b66b5..caeff42ba 100644
--- a/nix/nixpkgs.json
+++ b/nix/nixpkgs.json
@@ -1,9 +1,9 @@
 {
   "url": "https://github.com/nixos/nixpkgs",
-  "rev": "df8f580137e9120ffc60b2cc85d62fcc928c2eab",
-  "date": "2022-07-12T06:23:42+00:00",
-  "path": "/nix/store/7ddkc6rqgnvvhxwi0dqzsga49jrhwqwx-nixpkgs",
-  "sha256": "1ih3xk0vmnr2ywga2fmisvyaz4nrqf647821r811zl7hv37qhv75",
+  "rev": "9d48848775e34286e67a283c0b2c5bcf0e3272e3",
+  "date": "2022-08-23T11:32:36+02:00",
+  "path": "/nix/store/vffk9jif083qi0f59l396i1g6nhhh05p-nixpkgs",
+  "sha256": "0v0mdnp9zlgavhc7ikfxaa2a5xw79x97f8fy5y8z332ayksnb433",
   "fetchLFS": false,
   "fetchSubmodules": false,
   "deepClone": false,


Test patch:
diff --git a/.github/workflows/olm_tests.yaml b/.github/workflows/olm_tests.yaml
index e7c072c7e..51d5fa95f 100644
--- a/.github/workflows/olm_tests.yaml
+++ b/.github/workflows/olm_tests.yaml
@@ -5,7 +5,7 @@ on:
       - main
   pull_request:
 env:
-  GO_VERSION: '1.18'
+  GO_VERSION: '1.19'
   KIND_IMG_TAG: v1.23.4
 permissions:
   actions: none
@@ -35,7 +35,7 @@ jobs:
         make operator-sdk
         make opm
 
-    - uses: actions/setup-go@84cbf8094393cdc5fe1fe1671ff2647332956b1a
+    - uses: actions/setup-go@268d8c0ca0432bb2cf416faae41297df9d262d7f
       with:
         go-version: ${{ env.GO_VERSION }}
     - name: Install dependencies
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 777ec1b63..e487e141a 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -5,7 +5,7 @@ on:
       - main
   pull_request:
 env:
-  GO_VERSION: '1.18'
+  GO_VERSION: '1.19'
 permissions:
   actions: none
   checks: none
@@ -39,7 +39,7 @@ jobs:
     runs-on: ubuntu-20.04
     steps:
       - uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b
-      - uses: actions/setup-go@84cbf8094393cdc5fe1fe1671ff2647332956b1a
+      - uses: actions/setup-go@268d8c0ca0432bb2cf416faae41297df9d262d7f
         with:
           go-version: ${{ env.GO_VERSION }}
       - name: Install dependencies