deal with unparseable lines
don't crash if gzipping is broken
change hash algorithm from DJB2 to FNV
use content knowledge to distribute files into hashed/ more evenly
don't crash on leftover temporary files
rebuild all hashed symlinks each time