initial commit
gitignore
massive refactoring to factor out the parsing from the iteration
algorithm. Implements Iterable,Iterator interfaces. Iterator is now
tested. All green.
removed crc files
removed target files
new gitignore
Refactoring to use the hdfsCorpusLoader Iterable to load vocabulary as
well. Uses Generics. HdfsCorpusLoader is green.
updated package name for test of hdfsLoader
removed data/out from git
javadoc updated.
updated version to 1.0
refactored to use a BnsFactory to create the loaders.
updated to parse newest output from bns.pig
added tests to bnsLineParser all green!
added pig scripts which do first vectorizing step
add test data and test scripts
vectorizer hadoop reworked to use conf file. tested.
removed unneeded.
updated bns.pig to do a train-test split
configurations and driver updated to vectorize a train-test split using
a vocabulary generated from the result of bns.pig run on two folders
(postive vs. negative).
Tests have been updated to reflect the small change in format of the
line (random double [0,1] is added to do the line to split on). The
value is not culled in pig to avoid an unnecessary map phase.
bns.pig now outputs vocab index with bns value for features instead of
tokens.
The CorpusVectorizer now needs only the cardinality of the vectors to do
its job. 
I removed throwing an exception from ParseLine to (hopefully) make the
parsing more robust. this may not be a good idea.
The document's name is written in the sequence file instead of a
meaningless LongWritable.