omniti-labs/zetaback — pull request #2 by lotheac (2013-11-14 13:51:49+00:00)
Title: use zfs list -o instead of multiple invocations

The current implementation of zetaback_agent runs 'zfs list' once and 'zfs get' twice for every snapshot, filesystem or volume present on the system. One of our fileservers currently has about 7k filesystems and 480k snapshots, so this is prohibitively slow. Instead of using zfs get we can just ask list to include the desired properties; this makes the listing much faster: about two minutes wall clock time. I didn't time how long it takes without the patch, but I ran it for 4 minutes, and judging from the command line of the zfs get process it was running when I interrupted it, it had gone through about 0.58% of the iterations of the loop. Extrapolating from there yields quite a large performance improvement :)

This does change the semantics a bit, since it takes into account the property values for the snapshots themselves instead of the parent filesystem. However the values are inherited by default, and I argue that this is the correct behavior anyway: if I set exclude on a single snapshot, but not its fs, then that snapshot should not be included (but others of the same fs should).