Fixes to make the thing run. Droid test runs now. Newer version of hadoop to match the one I am using. Newer version of jwat to get sources
Fixed javadoc
The current jwat can identify compression, so do not replicate that work
Much more Garbage collection light use of buffers in arc migrator. Now the same buffer is reused for an entire Arc file migration, rather than being recreated for every record. The Buffering limit have been set to 4 MB so everything larger will still be written as a tempfile
Apparently the order in the droid return value collection is java version dependent. Now the thing is sorted
Minor unneeded code removed
Use jwat to determine if a arc file is compressed
Removed unused Gzip inputstream
Cleanup of IOUtils methods. Most are duplicate of apache IOUtils
Cleanup of Utils methods. Created a cache for the pattern utils to prevent continious compilation of the same patterns
Gitignore
Dedup migrator that works. Not it should be integrated to with the normal migrator so dedup migration happens seamlessly