Repository: cilium/ebpf
Pull request #341 (2021-07-10 17:15:18)

Mark PerCPUCGroupStorage as per-cpu
Currently, using "some_map->Lookup" on a PerCPUCGroupStorage only allows retrieving the first value of the per-cpu storage area. This PR adds the missing declaration.

ti-mo: Thanks!

@lmb Do we need extra tests for this?
yadutaf: I pushed a proposed test in a dedicated test function. Adding per-cpu cgroup storage to the existing `TestPerCPUMarshaling` proved a bit difficult due to the way it works. According [to the documentation](https://www.kernel.org/doc/html/latest/bpf/map_cgroup_storage.html), the cgroup storage is only allocated when an eBPF program is attached to the target cgroup. Additionally, `MaxEntries` can not be set for this kind of storage.

Instead of stacking "ifs" in `TestPerCPUMarshaling`  and attaching a dummy program, I would propose to add a dedicated test which focuses on the unmarshaling step. It has the downside of not exercising the full `marshall->kernel->unmarshal` chain, but would it be acceptable to consider this covered by the existing tests ?
lmb: > I pushed a proposed test in a dedicated test function.

Yeah, good idea.

> I would propose to add a dedicated test which focuses on the unmarshaling step.

I think that's the part I'm least worried about: it's already being exercised by the other per-CPU maps. The important bit is to check that the kernel interface matches what we expect, so do a real lookup from the map. Would it be enough to reuse `mustCgroupFixtures` and attach an empty program to that cgroup to make the lookup from the map feasible?

yadutaf: > The important bit is to check that the kernel interface matches what we expect.

Totally makes sense.

> Would it be enough to reuse mustCgroupFixtures ?

I tried that, but the CGroup storage area is only allocated when attaching a program which uses this storage. Thus, the attached program must at least call `bpf_get_local_storage` with the allocated map.

I pushed a new revision with the tests moved to `map_test.go` to get access to the helpers without getting a circular dependency and a couple new helpers.
yadutaf: Tests failed on Linux 5.4. I'm not sure of the real reason. I suspect this is related to the changes in the way CGroup local storage is allocated in kernel commit 7d9c3427894fe70d1347b4820476bf37736d2ff0 in 5.9. Honestly, this is really a wild guess.

Would it be acceptable to raise the minimum Linux version for this test to 5.9 and thus skip the 5.4 tests ?
lmb: Please take a look at #348 I think you can keep all the map / prog creation bits in map_test.go
yadutaf: Thanks for your suggestion. Following it solves 2 out of 3 import cycles. However, there is still an issue because `map_test.go` imports `github.com/cilium/ebpf/link` which, in turn, imports again `github.com/cilium/ebpf`. This import of the 'link' module is necessary to attach the dummy program to the test cgroup while the dummy program is needed to trigger the cgroup local storage allocation on kernel side:

```
go test -exec sudo  ./...
# github.com/cilium/ebpf
package github.com/cilium/ebpf (test)
	imports github.com/cilium/ebpf/link
	imports github.com/cilium/ebpf: import cycle not allowed in test
FAIL	github.com/cilium/ebpf [setup failed]
ok  	github.com/cilium/ebpf/asm	0.093s
ok  	github.com/cilium/ebpf/cmd/bpf2go	1.572s
ok  	github.com/cilium/ebpf/internal	0.070s
ok  	github.com/cilium/ebpf/internal/btf	0.720s
?   	github.com/cilium/ebpf/internal/testutils	[no test files]
?   	github.com/cilium/ebpf/internal/unix	[no test files]
ok  	github.com/cilium/ebpf/link	1.994s
ok  	github.com/cilium/ebpf/perf	0.275s
FAIL
```

I see at least 2 possible ways forward:

1. We can accept to move this test to `link_test.go` where we do not have this import cycle. I understand this is not the ideal place to exercise the (un)marshaling, but in a sense it also exercises a great deal of the link as well, including the intended side effect of cgroup local storage allocation.
2. Alternatively, we could spend some time trying to directly call the low level `internal.BPFProgAttach` which is not part of the link module.
yadutaf: Well, moving to the internal calls proved to be much easier that I expected. This leaves us with the 5.4 test issue. I'm trying to dig in the kernel source code, but would clearly appreciate the help from a kernel side eBPF expert here...
lmb: > However, there is still an issue because map_test.go imports github.com/cilium/ebpf/link which, in turn, imports again github.com/cilium/ebpf.

Ah, I didn't realize what was happening. Apologies.

> This leaves us with the 5.4 test issue. I'm trying to dig in the kernel source code, but would clearly appreciate the help from a kernel side eBPF expert here...

I took another look at the code, it's fine to add a skip for kernels that don't have https://github.com/torvalds/linux/commit/7d9c3427894fe70d1347b4820476bf37736d2ff0 so basically only run the test on v5.9. The UAPI shouldn't change between kernel versions, and it's not worth spending the time to understand why 5.4 is weird, IMO.
yadutaf: Thanks! Test updated with the skip.

Patch:
diff --git a/link/kprobe.go b/link/kprobe.go
index 40a02f5..626a818 100644
--- a/link/kprobe.go
+++ b/link/kprobe.go
@@ -159,7 +159,7 @@ func kprobe(symbol string, prog *ebpf.Program, ret bool) (*perfEvent, error) {
 // pmuKprobe opens a perf event based on the kprobe PMU.
 // Returns os.ErrNotExist if the given symbol does not exist in the kernel.
 func pmuKprobe(symbol string, ret bool) (*perfEvent, error) {
-	return pmuProbe(kprobeType, symbol, "", 0, ret)
+	return pmuProbe(kprobeType, symbol, "", 0, perfAllThreads, ret)
 }
 
 // pmuProbe opens a perf event based on a Performance Monitoring Unit.
@@ -169,7 +169,7 @@ func pmuKprobe(symbol string, ret bool) (*perfEvent, error) {
 // 33ea4b24277b "perf/core: Implement the 'perf_uprobe' PMU"
 //
 // Returns ErrNotSupported if the kernel doesn't support perf_[k,u]probe PMU
-func pmuProbe(typ probeType, symbol, path string, offset uint64, ret bool) (*perfEvent, error) {
+func pmuProbe(typ probeType, symbol, path string, offset uint64, pid int, ret bool) (*perfEvent, error) {
 	// Getting the PMU type will fail if the kernel doesn't support
 	// the perf_[k,u]probe PMU.
 	et, err := getPMUEventType(typ)
@@ -222,7 +222,7 @@ func pmuProbe(typ probeType, symbol, path string, offset uint64, ret bool) (*per
 		}
 	}
 
-	fd, err := unix.PerfEventOpen(&attr, perfAllThreads, 0, -1, unix.PERF_FLAG_FD_CLOEXEC)
+	fd, err := unix.PerfEventOpen(&attr, pid, 0, -1, unix.PERF_FLAG_FD_CLOEXEC)
 
 	// Since commit 97c753e62e6c, ENOENT is correctly returned instead of EINVAL
 	// when trying to create a kretprobe for a missing symbol. Make sure ENOENT
@@ -253,7 +253,7 @@ func pmuProbe(typ probeType, symbol, path string, offset uint64, ret bool) (*per
 
 // tracefsKprobe creates a Kprobe tracefs entry.
 func tracefsKprobe(symbol string, ret bool) (*perfEvent, error) {
-	return tracefsProbe(kprobeType, symbol, "", 0, ret)
+	return tracefsProbe(kprobeType, symbol, "", 0, perfAllThreads, ret)
 }
 
 // tracefsProbe creates a trace event by writing an entry to <tracefs>/[k,u]probe_events.
@@ -262,7 +262,7 @@ func tracefsKprobe(symbol string, ret bool) (*perfEvent, error) {
 // Path and offset are only set in the case of uprobe(s) and are used to set
 // the executable/library path on the filesystem and the offset where the probe is inserted.
 // A perf event is then opened on the newly-created trace event and returned to the caller.
-func tracefsProbe(typ probeType, symbol, path string, offset uint64, ret bool) (*perfEvent, error) {
+func tracefsProbe(typ probeType, symbol, path string, offset uint64, pid int, ret bool) (*perfEvent, error) {
 	// Generate a random string for each trace event we attempt to create.
 	// This value is used as the 'group' token in tracefs to allow creating
 	// multiple kprobe trace events with the same name.
@@ -295,7 +295,7 @@ func tracefsProbe(typ probeType, symbol, path string, offset uint64, ret bool) (
 	}
 
 	// Kprobes are ephemeral tracepoints and share the same perf event type.
-	fd, err := openTracepointPerfEvent(tid)
+	fd, err := openTracepointPerfEvent(tid, pid)
 	if err != nil {
 		return nil, err
 	}
diff --git a/link/perf_event.go b/link/perf_event.go
index 5267a47..110da98 100644
--- a/link/perf_event.go
+++ b/link/perf_event.go
@@ -236,7 +236,7 @@ func getPMUEventType(typ probeType) (uint64, error) {
 // openTracepointPerfEvent opens a tracepoint-type perf event. System-wide
 // [k,u]probes created by writing to <tracefs>/[k,u]probe_events are tracepoints
 // behind the scenes, and can be attached to using these perf events.
-func openTracepointPerfEvent(tid uint64) (*internal.FD, error) {
+func openTracepointPerfEvent(tid uint64, pid int) (*internal.FD, error) {
 	attr := unix.PerfEventAttr{
 		Type:        unix.PERF_TYPE_TRACEPOINT,
 		Config:      tid,
@@ -245,7 +245,7 @@ func openTracepointPerfEvent(tid uint64) (*internal.FD, error) {
 		Wakeup:      1,
 	}
 
-	fd, err := unix.PerfEventOpen(&attr, perfAllThreads, 0, -1, unix.PERF_FLAG_FD_CLOEXEC)
+	fd, err := unix.PerfEventOpen(&attr, pid, 0, -1, unix.PERF_FLAG_FD_CLOEXEC)
 	if err != nil {
 		return nil, fmt.Errorf("opening tracepoint perf event: %w", err)
 	}
diff --git a/link/tracepoint.go b/link/tracepoint.go
index 9043045..7423df8 100644
--- a/link/tracepoint.go
+++ b/link/tracepoint.go
@@ -38,7 +38,7 @@ func Tracepoint(group, name string, prog *ebpf.Program) (Link, error) {
 		return nil, err
 	}
 
-	fd, err := openTracepointPerfEvent(tid)
+	fd, err := openTracepointPerfEvent(tid, perfAllThreads)
 	if err != nil {
 		return nil, err
 	}
diff --git a/link/uprobe.go b/link/uprobe.go
index b855ff5..e7fc5ac 100644
--- a/link/uprobe.go
+++ b/link/uprobe.go
@@ -41,6 +41,9 @@ type UprobeOptions struct {
 	// Symbol offset. Must be provided in case of external symbols (shared libs).
 	// If set, overrides the offset eventually parsed from the executable.
 	Offset uint64
+	// Only set the uprobe on the given process ID. Useful when tracing
+	// shared library calls or programs that have many running instances.
+	PID int
 }
 
 // To open a new Executable, use:
@@ -197,8 +200,13 @@ func (ex *Executable) uprobe(symbol string, prog *ebpf.Program, opts *UprobeOpti
 		offset = sym.Value
 	}
 
+	pid := perfAllThreads
+	if opts != nil && opts.PID != 0 {
+		pid = opts.PID
+	}
+
 	// Use uprobe PMU if the kernel has it available.
-	tp, err := pmuUprobe(symbol, ex.path, offset, ret)
+	tp, err := pmuUprobe(symbol, ex.path, offset, pid, ret)
 	if err == nil {
 		return tp, nil
 	}
@@ -207,7 +215,7 @@ func (ex *Executable) uprobe(symbol string, prog *ebpf.Program, opts *UprobeOpti
 	}
 
 	// Use tracefs if uprobe PMU is missing.
-	tp, err = tracefsUprobe(uprobeSanitizedSymbol(symbol), ex.path, offset, ret)
+	tp, err = tracefsUprobe(uprobeSanitizedSymbol(symbol), ex.path, offset, pid, ret)
 	if err != nil {
 		return nil, fmt.Errorf("creating trace event '%s:%s' in tracefs: %w", ex.path, symbol, err)
 	}
@@ -216,13 +224,13 @@ func (ex *Executable) uprobe(symbol string, prog *ebpf.Program, opts *UprobeOpti
 }
 
 // pmuUprobe opens a perf event based on the uprobe PMU.
-func pmuUprobe(symbol, path string, offset uint64, ret bool) (*perfEvent, error) {
-	return pmuProbe(uprobeType, symbol, path, offset, ret)
+func pmuUprobe(symbol, path string, offset uint64, pid int, ret bool) (*perfEvent, error) {
+	return pmuProbe(uprobeType, symbol, path, offset, pid, ret)
 }
 
 // tracefsUprobe creates a Uprobe tracefs entry.
-func tracefsUprobe(symbol, path string, offset uint64, ret bool) (*perfEvent, error) {
-	return tracefsProbe(uprobeType, symbol, path, offset, ret)
+func tracefsUprobe(symbol, path string, offset uint64, pid int, ret bool) (*perfEvent, error) {
+	return tracefsProbe(uprobeType, symbol, path, offset, pid, ret)
 }
 
 // uprobeSanitizedSymbol replaces every invalid characted for the tracefs api with an underscore.
diff --git a/types.go b/types.go
index c45dd64..c91d806 100644
--- a/types.go
+++ b/types.go
@@ -105,7 +105,7 @@ const (
 
 // hasPerCPUValue returns true if the Map stores a value per CPU.
 func (mt MapType) hasPerCPUValue() bool {
-	return mt == PerCPUHash || mt == PerCPUArray || mt == LRUCPUHash
+	return mt == PerCPUHash || mt == PerCPUArray || mt == LRUCPUHash || mt == PerCPUCGroupStorage
 }
 
 // canStoreMap returns true if the map type accepts a map fd


Test patch:
diff --git a/internal/testutils/cgroup.go b/internal/testutils/cgroup.go
new file mode 100644
index 0000000..6c3f33f
--- /dev/null
+++ b/internal/testutils/cgroup.go
@@ -0,0 +1,82 @@
+package testutils
+
+import (
+	"errors"
+	"io/ioutil"
+	"os"
+	"strings"
+	"sync"
+	"testing"
+
+	"golang.org/x/sys/unix"
+)
+
+var cgroup2 = struct {
+	once sync.Once
+	path string
+	err  error
+}{}
+
+func cgroup2Path() (string, error) {
+	cgroup2.once.Do(func() {
+		mounts, err := ioutil.ReadFile("/proc/mounts")
+		if err != nil {
+			cgroup2.err = err
+			return
+		}
+
+		for _, line := range strings.Split(string(mounts), "\n") {
+			mount := strings.SplitN(line, " ", 3)
+			if mount[0] == "cgroup2" {
+				cgroup2.path = mount[1]
+				return
+			}
+
+			continue
+		}
+
+		cgroup2.err = errors.New("cgroup2 not mounted")
+	})
+
+	if cgroup2.err != nil {
+		return "", cgroup2.err
+	}
+
+	return cgroup2.path, nil
+}
+
+func CreateCgroup(tb testing.TB) *os.File {
+	tb.Helper()
+
+	cg2, err := cgroup2Path()
+	if err != nil {
+		tb.Fatal("Can't locate cgroup2 mount:", err)
+	}
+
+	cgdir, err := ioutil.TempDir(cg2, "ebpf-link")
+	if err != nil {
+		tb.Fatal("Can't create cgroupv2:", err)
+	}
+
+	cgroup, err := os.Open(cgdir)
+	if err != nil {
+		os.Remove(cgdir)
+		tb.Fatal(err)
+	}
+	tb.Cleanup(func() {
+		cgroup.Close()
+		os.Remove(cgdir)
+	})
+
+	return cgroup
+}
+
+func GetCgroupIno(t *testing.T, cgroup *os.File) uint64 {
+	cgroupStat := unix.Stat_t{}
+	err := unix.Fstat(int(cgroup.Fd()), &cgroupStat)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	return cgroupStat.Ino
+}
diff --git a/link/link_test.go b/link/link_test.go
index 14343a3..48b0e00 100644
--- a/link/link_test.go
+++ b/link/link_test.go
@@ -7,8 +7,6 @@ import (
 	"os"
 	"path/filepath"
 	"reflect"
-	"strings"
-	"sync"
 	"testing"
 
 	"github.com/cilium/ebpf"
@@ -17,12 +15,6 @@ import (
 	"github.com/cilium/ebpf/internal/unix"
 )
 
-var cgroup2Mount = struct {
-	once sync.Once
-	path string
-	err  error
-}{}
-
 func TestRawLink(t *testing.T) {
 	cgroup, prog := mustCgroupFixtures(t)
 
@@ -98,26 +90,7 @@ func mustCgroupFixtures(t *testing.T) (*os.File, *ebpf.Program) {
 
 	testutils.SkipIfNotSupported(t, haveProgAttach())
 
-	prog := mustCgroupEgressProgram(t)
-	cgdir, err := ioutil.TempDir(cgroup2Path(t), "ebpf-link")
-	if err != nil {
-		prog.Close()
-		t.Fatal("Can't create cgroupv2:", err)
-	}
-
-	cgroup, err := os.Open(cgdir)
-	if err != nil {
-		prog.Close()
-		os.Remove(cgdir)
-		t.Fatal(err)
-	}
-	t.Cleanup(func() {
-		prog.Close()
-		cgroup.Close()
-		os.Remove(cgdir)
-	})
-
-	return cgroup, prog
+	return testutils.CreateCgroup(t), mustCgroupEgressProgram(t)
 }
 
 func mustCgroupEgressProgram(t *testing.T) *ebpf.Program {
@@ -135,34 +108,12 @@ func mustCgroupEgressProgram(t *testing.T) *ebpf.Program {
 	if err != nil {
 		t.Fatal(err)
 	}
-	return prog
-}
 
-func cgroup2Path(t *testing.T) string {
-	cgroup2Mount.once.Do(func() {
-		mounts, err := ioutil.ReadFile("/proc/mounts")
-		if err != nil {
-			cgroup2Mount.err = err
-			return
-		}
-		for _, line := range strings.Split(string(mounts), "\n") {
-			mount := strings.SplitN(line, " ", 3)
-			if mount[0] == "cgroup2" {
-				cgroup2Mount.path = mount[1]
-				return
-			}
-
-			continue
-		}
-
-		cgroup2Mount.err = errors.New("cgroup2 not mounted")
+	t.Cleanup(func() {
+		prog.Close()
 	})
 
-	if cgroup2Mount.err != nil {
-		t.Fatal(cgroup2Mount.err)
-	}
-
-	return cgroup2Mount.path
+	return prog
 }
 
 type testLinkOptions struct {
diff --git a/link/uprobe_test.go b/link/uprobe_test.go
index 0b5585b..51db785 100644
--- a/link/uprobe_test.go
+++ b/link/uprobe_test.go
@@ -11,6 +11,7 @@ import (
 
 	"github.com/cilium/ebpf"
 	"github.com/cilium/ebpf/internal/testutils"
+	"github.com/cilium/ebpf/internal/unix"
 )
 
 var bashEx, _ = OpenExecutable("/bin/bash")
@@ -88,6 +89,34 @@ func TestUprobeExtWithOpts(t *testing.T) {
 	defer up.Close()
 }
 
+func TestUprobeWithPID(t *testing.T) {
+	prog, err := ebpf.NewProgram(&kprobeSpec)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer prog.Close()
+
+	up, err := bashEx.Uprobe(bashSym, prog, &UprobeOptions{PID: os.Getpid()})
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer up.Close()
+}
+
+func TestUprobeWithNonExistentPID(t *testing.T) {
+	prog, err := ebpf.NewProgram(&kprobeSpec)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer prog.Close()
+
+	// trying to open a perf event on a non-existent PID will return ESRCH.
+	_, err = bashEx.Uprobe(bashSym, prog, &UprobeOptions{PID: -2})
+	if !errors.Is(err, unix.ESRCH) {
+		t.Fatalf("expected ESRCH, got %v", err)
+	}
+}
+
 func TestUretprobe(t *testing.T) {
 	c := qt.New(t)
 
@@ -118,14 +147,14 @@ func TestUprobeCreatePMU(t *testing.T) {
 	c.Assert(err, qt.IsNil)
 
 	// uprobe PMU
-	pu, err := pmuUprobe(sym.Name, bashEx.path, sym.Value, false)
+	pu, err := pmuUprobe(sym.Name, bashEx.path, sym.Value, perfAllThreads, false)
 	c.Assert(err, qt.IsNil)
 	defer pu.Close()
 
 	c.Assert(pu.typ, qt.Equals, uprobeEvent)
 
 	// uretprobe PMU
-	pr, err := pmuUprobe(sym.Name, bashEx.path, sym.Value, true)
+	pr, err := pmuUprobe(sym.Name, bashEx.path, sym.Value, perfAllThreads, true)
 	c.Assert(err, qt.IsNil)
 	defer pr.Close()
 
@@ -140,7 +169,7 @@ func TestUprobePMUUnavailable(t *testing.T) {
 	sym, err := bashEx.symbol(bashSym)
 	c.Assert(err, qt.IsNil)
 
-	pk, err := pmuUprobe(sym.Name, bashEx.path, sym.Value, false)
+	pk, err := pmuUprobe(sym.Name, bashEx.path, sym.Value, perfAllThreads, false)
 	if err == nil {
 		pk.Close()
 		t.Skipf("Kernel supports perf_uprobe PMU, not asserting error.")
@@ -162,23 +191,23 @@ func TestUprobeTraceFS(t *testing.T) {
 	ssym := uprobeSanitizedSymbol(sym.Name)
 
 	// Open and close tracefs u(ret)probes, checking all errors.
-	up, err := tracefsUprobe(ssym, bashEx.path, sym.Value, false)
+	up, err := tracefsUprobe(ssym, bashEx.path, sym.Value, perfAllThreads, false)
 	c.Assert(err, qt.IsNil)
 	c.Assert(up.Close(), qt.IsNil)
 	c.Assert(up.typ, qt.Equals, uprobeEvent)
 
-	up, err = tracefsUprobe(ssym, bashEx.path, sym.Value, true)
+	up, err = tracefsUprobe(ssym, bashEx.path, sym.Value, perfAllThreads, true)
 	c.Assert(err, qt.IsNil)
 	c.Assert(up.Close(), qt.IsNil)
 	c.Assert(up.typ, qt.Equals, uretprobeEvent)
 
 	// Create two identical trace events, ensure their IDs differ.
-	u1, err := tracefsUprobe(ssym, bashEx.path, sym.Value, false)
+	u1, err := tracefsUprobe(ssym, bashEx.path, sym.Value, perfAllThreads, false)
 	c.Assert(err, qt.IsNil)
 	defer u1.Close()
 	c.Assert(u1.tracefsID, qt.Not(qt.Equals), 0)
 
-	u2, err := tracefsUprobe(ssym, bashEx.path, sym.Value, false)
+	u2, err := tracefsUprobe(ssym, bashEx.path, sym.Value, perfAllThreads, false)
 	c.Assert(err, qt.IsNil)
 	defer u2.Close()
 	c.Assert(u2.tracefsID, qt.Not(qt.Equals), 0)
@@ -323,3 +352,32 @@ func TestUprobeProgramCall(t *testing.T) {
 	// Assert that this time the value has not been updated.
 	assertMapValue(t, m, 0, 0)
 }
+
+func TestUprobeProgramWrongPID(t *testing.T) {
+	m, p := newUpdaterMapProg(t, ebpf.Kprobe)
+
+	// Load the '/bin/bash' executable.
+	ex, err := OpenExecutable("/bin/bash")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Open Uprobe on '/bin/bash' for the symbol 'main'
+	// and attach it to the ebpf program created above.
+	// Create the perf-event with the current process' PID
+	// to make sure the event is not fired when we will try
+	// to trigger the program execution via exec.
+	u, err := ex.Uprobe("main", p, &UprobeOptions{PID: os.Getpid()})
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer u.Close()
+
+	// Trigger ebpf program call.
+	if err := exec.Command("/bin/bash", "--help").Run(); err != nil {
+		t.Fatal(err)
+	}
+
+	// Assert that the value at index 0 is still 0.
+	assertMapValue(t, m, 0, 0)
+}
diff --git a/map_test.go b/map_test.go
index f7973c9..069e759 100644
--- a/map_test.go
+++ b/map_test.go
@@ -12,6 +12,7 @@ import (
 	"testing"
 	"unsafe"
 
+	"github.com/cilium/ebpf/asm"
 	"github.com/cilium/ebpf/internal"
 	"github.com/cilium/ebpf/internal/btf"
 	"github.com/cilium/ebpf/internal/testutils"
@@ -1033,6 +1034,99 @@ func TestPerCPUMarshaling(t *testing.T) {
 	}
 }
 
+type bpfCgroupStorageKey struct {
+	CgroupInodeId uint64
+	AttachType    AttachType
+	_             [4]byte // Padding
+}
+
+func TestCgroupPerCPUStorageMarshaling(t *testing.T) {
+	numCPU, err := internal.PossibleCPUs()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if numCPU < 2 {
+		t.Skip("Test requires at least two CPUs")
+	}
+	testutils.SkipOnOldKernel(t, "5.9", "per-CPU CGoup storage with write from user space support")
+
+	cgroup := testutils.CreateCgroup(t)
+
+	arr, err := NewMap(&MapSpec{
+		Type:      PerCPUCGroupStorage,
+		KeySize:   uint32(unsafe.Sizeof(bpfCgroupStorageKey{})),
+		ValueSize: uint32(unsafe.Sizeof(uint64(0))),
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	t.Cleanup(func() {
+		arr.Close()
+	})
+
+	prog, err := NewProgram(&ProgramSpec{
+		Type:       CGroupSKB,
+		AttachType: AttachCGroupInetEgress,
+		License:    "MIT",
+		Instructions: asm.Instructions{
+			asm.LoadMapPtr(asm.R1, arr.FD()),
+			asm.Mov.Imm(asm.R2, 0),
+			asm.FnGetLocalStorage.Call(),
+			asm.Mov.Imm(asm.R0, 0),
+			asm.Return(),
+		},
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer prog.Close()
+
+	progAttachAttrs := internal.BPFProgAttachAttr{
+		TargetFd:     uint32(cgroup.Fd()),
+		AttachBpfFd:  uint32(prog.FD()),
+		AttachType:   uint32(AttachCGroupInetEgress),
+		AttachFlags:  0,
+		ReplaceBpfFd: 0,
+	}
+	err = internal.BPFProgAttach(&progAttachAttrs)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer func() {
+		attr := internal.BPFProgDetachAttr{
+			TargetFd:    uint32(cgroup.Fd()),
+			AttachBpfFd: uint32(prog.FD()),
+			AttachType:  uint32(AttachCGroupInetEgress),
+		}
+		if err := internal.BPFProgDetach(&attr); err != nil {
+			t.Fatal(err)
+		}
+	}()
+
+	var mapKey = &bpfCgroupStorageKey{
+		CgroupInodeId: testutils.GetCgroupIno(t, cgroup),
+		AttachType:    AttachCGroupInetEgress,
+	}
+
+	values := []uint64{1, 2}
+	if err := arr.Put(mapKey, values); err != nil {
+		t.Fatalf("Can't set cgroup %s storage: %s", cgroup.Name(), err)
+	}
+
+	var retrieved []uint64
+	if err := arr.Lookup(mapKey, &retrieved); err != nil {
+		t.Fatalf("Can't retrieve cgroup %s storage: %s", cgroup.Name(), err)
+	}
+
+	for i, want := range []uint64{1, 2} {
+		if retrieved[i] == 0 {
+			t.Errorf("Item %d is 0", i)
+		} else if have := retrieved[i]; have != want {
+			t.Errorf("PerCPUCGroupStorage map is not correctly unmarshaled, expected %d but got %d", want, have)
+		}
+	}
+}
+
 func TestMapMarshalUnsafe(t *testing.T) {
 	m, err := NewMap(&MapSpec{
 		Type:       Hash,