Added first version of bag-of-vectore feature generator which can extract sentence-internal bigrams and adjective-noun combinations. Unit tests broken
Updated unit tests so that they expect what the tokenizer now produces--- a corpus is a list of documents, which is a list of sentences, which is a list of tokens
started work on using more general n-gram composable features as well as unigrams. Mostly new code, but the format of  the features is now (AN, (big, cat)) instead of just a single string, which breaks a bunch of unit tests. still work in progress
Merge remote-tracking branch 'origin/bov-feature-extraction' into bov-feature-extraction

Conflicts:
	conf/exp0/exp0_base.conf
	thesisgenerator/plugins/bov.py
documented conf file options
- implemented add and multiply composer (with tests)
- enhanced tests for CompositeVectorSource (testing if unknown unigrams are discarded)
- linked composer's feature acceptance functionality to BovVectorizer's feature extraction
- moved things around in the class hierarchy to minimise code duplication
- linked composers to their respective features (the result is not used in any way right now)
More work on CompositeVectorSource, it can now compose all train-time document features and place them in a KD tree. It can then retrieve nearest neighbours of a given n-gram from the tree. Still thinking about the interface of all that. Some of the tests are not comprehensive, and some do not exist yet.
Moved things around in the vectorstore.py class hierarchy again; expanded unit tests a bit more
more work on implementation of get_nearest-neighbors
minor change to conf file for compatiblity with sklearn 0.14;  minor change to the semantics of UnigramVectorSource#get_vector, which now accepts a tuple as well as a string
(BROKEN) started moving to using vectorsource instead of thesaurus, but ran into issues with  migration to sklearn 0.14, needs extra work
(BROKEN) more of the previous, still broken because of incompatibility with sklear 0.14
mostly there
Minor changes towards not using explicit filtering, provided by the semantic space. This should happen by just returning no neighbours for entries that would have been filtered out or by not having to support the keep_only_IT switch at all
implementing feature filtering as a part of sklearn's feature selection
changed feature selector so that it removes features both at encode and decode time
removed all mentions of thesaurus in the tokenizer, this functionality belongs in the feature extraction module
fixed another unit test; optimised feature selector so it does not compute feature scores if there is no need for it
un-deleted call to stats_recorder.print_coverage_info, which got left out during refactoring
removed more references to obsolete parameter keep_only_IT