factorie/factorie — pull request #147 by jeevanshankar1991 (2014-04-22 05:46:18+00:00)
Title: WordEmbeddings - minor and useful updates

added few useful cmd options and its impl

a) ignore-stopwords - ignores stopwords from factorie's stopwords and removes the words whose length is 1. This way I could ignore punctuations as well. Default option is not to ignore stopwords.
b) binary - store embeddings, vocab in plain text or .gzip format. Default is plain text
c) encoding - read the corpus , save the vocab and store the embeddings all using the same encoding . Default is ISO-8859-15.
d) load-vocab - use previous used vocab . This could save lot of time for the same corpus.
e) max-vocab-size - cut-off for max vocab size user wants to use to learning the embedding. Reducing it will certainly increase the speed but may affect overall accuracy. Default is 2M.
added wordembedding as word2vec in /bin/fac

will be glad to hear comments.