Can use Lucene Analyzers to produce Mahout vectors
SparseVectorsFromBehemoth : generate labels in separate sequencefile
Tika processor does not override existing metadata
Mahout : optionally dump label file
BugFix SparseVectorsFromBehemoth : dump Labels
Added utility to generate a vector file at the libsvm format from Mahout vectors and label file
Mahout2LibSVM : increment attribute index and outputs int for labels
Minor changes to LanguageIdDriver and SparseVectorsFromBehemoth
DocumentFilter based on BehemothDocument metadata + used in GATEProcessor
Fixed GATE Corpus Generator
Bugfix : GATECorpusGenerator deletes the temporary GATE documents
Issue #23 Versioning BehemothDocument - NOT BACKWARD COMPATIBLE
Added BehemothReducer which uses the DocumentFilter. Refactored GATEDriver to use it and removed from GATEProcessor
LanguageIdDriver uses Behemoth reducer and can filter output
Small changes to README + added example of filtering for language module
Started DocumentSplitter + AnnotationsUtil (sorter + filter) and use in BehemothReducer
Added simple serialization test
Added more info to pom files
Added utility CorpusFilter + BehemothReducer used in TikaDriver and has methods for checking whether needed + DocumentFilter can be used on MimeTypes and / or URLs
Fixed typo on param names for filters