Added code for building observed window vectors for NPs from FET output
Optimised regex to speed things up
Fixed word order in NN compouns; Also removed feature type prefix, that's not needed
Fixed bug in reduce_dimensionality where the second matrix was not correctly made into the shape of the first
Ooops
Ooops
Merge branch 'master' of https://github.com/mbatchkarov/DiscoUtils
Updated io_utils.write_vectors_to_disk so that vectors_path can be a file name or a file-like object. Renamed several variables to improve legibility.
Not creating files if they are empty.
Added option to do_svd to output the just "training" matrix, just the "test" matrix or the concatenation of the two
Corrected path name (needs to contain -12 or -13 suffix)
Error message was outputting the wrong parameter name
Moved Bunch class to DiscoUtils
Modified collate_.... slightly in order to use it on APDT pre-composed vectors as provided by Julie. Added a sanity check (can't run APDT on window vectors)
reformat_entries now handling malformed lines better.
Removed unwanted parameter
Changed the semantics of the seed parameter of find_all_NPs- now operates on phrases instead on of phrase modifiers
reduced logging frequency in io_utils
Added a comment regarding a potential bug
Removed NER tag from Token's __eq__ method. This made the identity of a token dependent on the possibly wrong NER tagger. Sometimes, the same string, eg. New York, would result in different Token-s, because the NER tagger tagged one as O and the other as LOC