maciejkula/glove-python — pull request #9 by piskvorky (2014-11-05 21:16:41+00:00)
Title: Code for "analogy task" evaluation

Code for accuracy evaluations, as discussed.

When ran on 30k most common Wiki words (after `filter_extremes(keepn=30000)`), 100 dim, GloVe gets 33.3%, word2vec 45.9%.

The code is basically hacked out of gensim's word2vec, with an extra `fake_api` method, which monkey patches trained word2vec / glove objects, so that they share the same interface.

The accuracy is then a single function, the same for both models, taking this patched model object as parameter.

Use:
```python
logging.root.level = logging.INFO
model = Word2Vec.load('trainmodel') # or Glove.load('trainedmodel')
test_embed.fake_api(model) # monkey patch it
ok_words = model.word2id # or pick the top 30k most frequent words
sections = test_embed.accuracy(model, 'questions-words.txt', test_embed.most_similar, ok_words)
```

Code still a bit rough, and I removed the `__main__` part completely (didn't have time to finish that properly), but hopefully the idea's clear.