Initial commit of code to build term dictionary; much of the code has been copied from Ivory; not sure if it actually runs yet.
Added cmdline options to ComputeTermStatistics; Created AnalyzerUtil.
Skip documents that are too long.
Added program for merging term statistics; Dictionary appears to be working.
Building docvectors on ClueWeb12 segment00 seems to work fine, end-to-end.
Added API for access cf and df statistics.
Updated README with dictionary and document vector data.
Merge branch 'dictionary'