soft-matter/trackpy — pull request #100 by nkeim (2014-04-04 20:55:26+00:00)
Title: Storage class with frame index cacheing

This is rest of the solution to PR #98. `PandasHDFStoreBig` is a subclass of `PandasHDFStore` that stores a special DataFrame listing all of the frame numbers in the file. This makes it practical to have files containing tens of thousands of large frames (some of my HDF5 files are currently 60+ GB). The "format" is forwards- and backwards- compatible with the non-cached version.

I think the design is sound and the tests are sufficient, and I have already given the class plenty of exercise in production, but storage classes are both trusted and hard to change later, so take your time. I'd also like your thoughts on the class name — perhaps it could be better.

This PR also fixes a bug in storage class testing, and a bug in the two existing classes, whereby the `__del__()` method would fail if the file couldn't be opened, resulting in a warning.