Traceloop for systemd and Kubernetes + Inspektor Gadget
Channel: All Systems Go!
Date: 2019-09-21

https://media.ccc.de/v/ASG2019-159-traceloop-for-systemd-and-kubernetes-inspektor-gadget



Presenting [traceloop](https://github.com/kinvolk/traceloop), a “time travel” tracing tool to trace system calls in cgroups using BPF and overwritable ring buffers.

Many people use the “strace” tool to synchronously trace system calls using ptrace. [Traceloop](https://github.com/kinvolk/traceloop) similarly traces system calls but asynchronously in the background, using BPF and tracing per cgroup. I’ll show how it can be integrated with systemd and with Kubernetes via [Inspektor Gadget](https://github.com/kinvolk/inspektor-gadget).

Traceloop's traces are recorded in a fast, in-memory, overwritable ring buffer like a flight recorder. As opposed to “strace”, the tracing could be permanently enabled on systemd services or Kubernetes pods and inspected in case of a crash. This is like a always-on “strace in the past”.

Traceloop uses BPF through the gobpf library. Several new features have been added in gobpf for the needs of traceloop: support for overwritable ring buffers and swapping buffers when the userspace utility dumps the buffer.

https://github.com/kinvolk/traceloop
https://github.com/kinvolk/inspektor-gadget
https://github.com/iovisor/gobpf

Slides: https://docs.google.com/presentation/d/1zIZUrTrD7FkS9pHnWz87ZmoLTrO1g9-J_lDMD7E5kdo/edit

Alban Crequy

https://cfp.all-systems-go.io/ASG2019/talk/98A9LW/