lisa-lab/pylearn2 — pull request #1018 by bartvm (2014-07-05 15:49:27+00:00)
Title: WIP: Word relationship extension, TextDatasetMixin

I've tried to implement the semantic-syntactic word relationship test by Google that is used in their word2vec paper. It basically tries to answer a set of questions "A is to B as C is to ..." using word embeddings. This PR contains:

* TextDatasetMixin: Mixin class that defines some attributes that are required for NLP datasets, namely the vocabulary (mapping words/characters to indices); the index and string to be used to represent words that are not in the vocabulary; methods to go from tokens to indices and back.
* WordRelationship: Train extension that adds monitoring channels

For now I'm running into two problems:

* How to retrieve layers by their name, given that layers can be nested
* The monitoring channel repeatedly calls channels for each monitoring batch, even if they don't require data