Add support for bufferFully option to read entire block(s) at once before iterating
Add new PigDateLoader for filtering pig input by date!
ia-tools: FileDateFilter and LSRPigLoader added! Can filter any loader by date, also LSR loader for faster iteration over large input sets
ia-tools: Add DisablePathFilter, FirstPibJobOnly filter which DateFilter extends
add SequenceFileStorage from bacon
ia-tools: Aadded DateFilterLSRLoader for specifying datefilter directly!
archive-commons: New LocationUpdater system which will update the ALL.loc file, if changes are detected.
SeekableLineReaderFactory supports lastModified check (though http is untested)
ia-tools: CDXMapper exposed for SURT conversion via pig toSURT function (probably should change name)
ZipNumAllOutputFormat supports customizing shard name (default part-a), also using
-idx instead of .ALL.summary for index/summary files
archive-commons: Refactoring of ZipNumCluster for easier usage.
Refactored block loaders to a single, ZipNumBlockLoader that has all the settings for file and http io
Add LineBufferingIterator for buffering summary reads
archive-commons: make canonicalizer in WaybackURLKeyMaker settable
Merge pull request #3 from internetarchive/master

Merging in latest changes from internetarchive