piskvorky/gensim — pull request #175 by josephcc (2014-03-02 06:38:10+00:00)
Title: [Word2Vec] Support for reading calculated skip ngrams

This adds support for training from precompiled skip ngrams, instead of raw text. Each ``sentence" from the corpus reader now consists of the following format ([str:word1, str:word2], int:distance, int:count). This is useful if you are trying to train a word2vec model from a ngram model, for example the Google Web1T dataset. Both the Python and Cython implementations were updated.

New functionalities are controlled by the skip_ngram parameter, with the default value the behavior of existing functionalities should remain the same.

I wasn't sure how to change the matrix manipulation part, so each skip ngrams with counts will update the matrix multiple times. This is likely to be very inefficient, and should be improved in the future.