jborg/attic — pull request #119 by c4rlo (2014-10-05 18:03:12+00:00)
Title: Faster os.walk() replacement on Linux

Take advantage of the fact that on Linux, `readdir_r()` returns the file type with each directory entry. (The type may be "unknown" depending on the filesystem, so there's a fallback.)

Non-scientific data point: this reduces an `attic list` from ~500ms to ~240ms for a 87G repository on a USB NTFS drive that consists of ~17k individual segment files. An `attic extract` for a small file is similarly about 300ms faster (2.6 -> 2.3 sec).

Note: supposedly, the same method should work on FreeBSD also, but I've so far had no luck with testing attic on FreeBSD. If someone else wants to confirm if this works too, that would be great. In the meantime, this is only enabled on Linux.

In addition to the above, there are some small tweaks, mostly cosmetic, in the second commit (1e3a24a).