Add support for bufferFully option to read entire block(s) at once before iterating
Add new PigDateLoader for filtering pig input by date!
ia-tools: FileDateFilter and LSRPigLoader added! Can filter any loader by date, also LSR loader for faster iteration over large input sets
ia-tools: Add DisablePathFilter, FirstPibJobOnly filter which DateFilter extends
add SequenceFileStorage from bacon
ia-tools: Aadded DateFilterLSRLoader for specifying datefilter directly!
Now builds cleanly, using public repos and updated dependencies.
Now builds cleanly, using public repos and updated dependencies.
archive-commons: New LocationUpdater system which will update the ALL.loc file, if changes are detected.
SeekableLineReaderFactory supports lastModified check (though http is untested)
ia-tools: CDXMapper exposed for SURT conversion via pig toSURT function (probably should change name)
ZipNumAllOutputFormat supports customizing shard name (default part-a), also using
-idx instead of .ALL.summary for index/summary files
archive-commons: Refactoring of ZipNumCluster for easier usage.
Refactored block loaders to a single, ZipNumBlockLoader that has all the settings for file and http io
Fixing up Java version.
Merge pull request #1 from ukwa/master

Stripping out Eclipse project files
Disabled building large jars-with-dependencies by default, as this prevents travis-ci from working.
Added heap size specification.
Upped heap size specification.
Switched to JDK6.
Switched back and set flags properly.
Attempted to modify ulimit.
Attempted to modify ulimit again.
Merge pull request #2 from ukwa/switchover-maven-build

Switchover maven build