Enabling support for Python scripts for use with perf uprobes on Ubuntu 20.04
Published: 2022-01-26 08:48:16

Linux's `perf` subsystem, about which you can read a lot more here https://www.brendangregg.com/perf.html is an invaluable subsystem for tracing the performance of the kernel and user-space code.

It's a bit of the wild-west, it's a fast moving space with lots of moving pieces, and lots of unusual kernel- and user-space interop which makes it tricky to use.

What makes it trickier unfortunately is that Canonical saw fit to remove scripting support from the `perf` tool in the `linux-tools-generic` (and `linux-tools-$(name -r)`) package rendering it practically useless.

Confirm you have this problem with the following:


```
sudo perf script -s lang

Scripting language extensions (used in perf script -s [spec:]script.[spec]):

  Perl                                       [Perl]
  pl                                         [Perl]
  Python                                     [Python]
  py                                         [Python]
```

Great, seems like `perf script` has Python and Perl support, right? ... not quite:

The steps here can be also used to enable Perl scripting support, but I'm sticking with Python, since what little documentation there is covers this specifically.

```
sudo perf script -g py
Python scripting not supported.  Install libpython and rebuild perf to enable it.
For example:
  # apt-get install python-dev (ubuntu)
  # yum install python-devel (Fedora)
  etc.
```

Unfortunately the instructions won't help you and there's no way to rebuild the `linux-tools-*` packages easily even after adding the `{python,perl}-dev{,el}` packages as suggested.

Rather one is required to get the kernel sources, and patch them and then re-build the entire kernel package tree and _then_ and only then can a version of `linux-tools-*` with scripting support be installed:

The instructions ultimately are the normal instructions for building a new kernel; but I'll drop a block of shell commands below to save having to read a bunch of convolouted docs with multiple options:

- Follow to about half way: https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel
- Modify a file to remove `NO_LIBPYTHON=1` (note it must be REMOVED, not set to `0`)
- Compile the kernel from that source tree  (resume the build your own kernel guide)
- Install the `linux-tools` package with `dpkg -i ..` from the newly built packages.

So:

```
# 1. enable deb sources in /etc/sources.list
sed -i '/deb-src/s/^# //' /etc/apt/sources.list && apt update

# 2. install kernel sources for current kernel (installs to $PWD) e.g ./linux-5.4.0
apt-get source linux-image-unsigned-$(uname -r)

# 3. install python-dev package & general kernel build deps
apt install python-dev libncurses-dev gawk flex bison openssl libssl-dev dkms libelf-dev libudev-dev libpci-dev libiberty-dev autoconf default-jdk


# 4. change into the directory
cd ./linux-generic-5.4.0

# 5. apply this patch:
patch -p1 << EOPATCH
--- ./debian/rules.d/2-binary-arch.mkbak	2022-01-26 09:40:31.791079823 +0100
+++ ./debian/rules.d/2-binary-arch.mk	2022-01-26 09:40:46.671072005 +0100
@@ -702,7 +702,7 @@
 	cd $(builddirpa) && $(kmake) syncconfig
 	cd $(builddirpa) && $(kmake) prepare
 	cd $(builddirpa)/tools/perf && \
-		$(kmake) prefix=/usr HAVE_NO_LIBBFD=1 HAVE_CPLUS_DEMANGLE_SUPPORT=1 CROSS_COMPILE=$(CROSS_COMPILE) NO_LIBPYTHON=1 NO_LIBPERL=1
+		$(kmake) prefix=/usr HAVE_NO_LIBBFD=1 HAVE_CPLUS_DEMANGLE_SUPPORT=1 CROSS_COMPILE=$(CROSS_COMPILE) NO_LIBPERL=1
 endif
 ifeq ($(do_tools_bpftool),true)
 	$(kmake) CROSS_COMPILE=$(CROSS_COMPILE) -C $(builddirpa)/tools/bpf/bpftool
EOPATCH

# 6. Build the targets
#    if this fails and you need to install new deps, then
#    reset things with `fakeroot debian/rules clean`
fakeroot debian/rules binary-arch

# 7. Wait (about 15-20 minutes on a decent laptop, probably, YMMV with virtualization) 

# 8. Install the new package with dpkg -i 
#    note: filename here varies a lot by version and arch 
sudo dpkg -i linux-tools-*.deb
```

After all this then the command `sudo perf script -g py` should generate a pyton script with hooks for any perf recorders you might have enabled already.