ZFS Directory Scaling - Mateusz Piotrowski - EuroBSDcon 2023
Channel: EuroBSDcon
Date: 2023-10-02

Mateusz Piotrowski:  ZFS Directory Scaling 
It is not uncommon to store thousands of files in a single directory. Unfortunately, a growing directory size may result in unexpected bottlenecks slowing down the system. The performance hit can be observed even for simple operations like lookup, open, and create.

What are the data structures defining the performance of directories? What obscure tunables are worth taking a look at when storing thousands of files in thousands of directories? What are the recent developments in OpenZFS helping to address those performance problems?

The presentation walks the audience through the realm of OpenZFS performance engineering. First, it introduces the audience to the essential concepts in ZFS and provides the 101 of file system performance tuning. Afterwards, it takes a deep dive into the implementation of directories in OpenZFS. Finally, the presentation presents ways of improving ZFS directory scaling, discusses relevant tunables and how to use them, and comments on the in-progress work and recent developments in OpenZFS.

Mateusz Piotrowski:
Mateusz Piotrowski is a FreeBSD ports and documentation committer based in Berlin. He enjoys troubleshooting bugs, scripting automation, and designing robust software systems (always thoroughly documenting everything along the way). Recently, his interests have drifted toward tracing and performance engineering.