timholy/HDF5.jl — pull request #102 by mbauman (2014-06-17 22:16:40+00:00)
Title: Re-use object references

Alright, this one is a bit more daring.

Use an ObjectIdDict to keep track of previously saved array objects, and reuse the their references if saved again in a different place. This can drastically reduce file size.

In my application, it reduced the size of the file from 491.1 MB to 74.4 MB. Which is awesome (and almost exactly the size of the data once loaded into Julia). But I'm not quite sure I'm making the references correctly. Theoretically, I think this could allow saving arrays that have circular references to themselves, but while all tests pass, that still doesn't work.