Fix stream code. Refactor some code.
Fix launch script to support CommonCrawlServer base class.
Add support for setting Reduce failure percentage.
Optimize MergeSort performance by using priority queue instead of doing
lots of redundant, wasteful sorts. Add unit test to using Mockito.
Added configurable codec support to SplillWriter. Configure size of
backlog queue. Refactor IndexWriter interface and move it to uitls.
Add support for initialzing TextBytes from a byte buffer input stream.
Add utility split class.
Enhance comments for some constants and add preliminary support for
CrawlMaster.
Add HexDump utility class.
Add stub implementation (incomplete) of new S3N FileSystem
implementation.
Add support for min/max key generation (for root domain/super domain).
Refactor the merge code a bit.
Accomodate for some refactoring of dependent classes.
Remove some warnings.
Merge in Blekko URL List.
Add some utility code for searching CrawlDB database.
Add in hacked version crawler master server.
support launching test harness.