Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 3: Clear rejection
review_content: This paper introduces a method for aligning the representations of different word embedding models by leveraging synthetic data points. Even after reading the paper several times, and reading the authors' response to another reviewer's questions, I still struggle to say how exactly the authors achieve this alignment and precisely what problem the technique is supposed to resolve.

If I understand correctly, synthetic data points are generated by combining (with coefficient +/- 1) embeddings from within the same neighborhood, and if the result also falls within the neighborhood it is retained as a "latent word". These latent words help anchor the alignment process. Intuitively, having more points to anchor a neighborhood makes some sense, but I don't understand the details of how the alignment is actually being implemented, so it is hard to say anything more concrete.

It seems like the major novelty of the proposed approach is in the generation of the latent words. So, in addition to a more detailed explanation of the alignment process, I would have liked to see more analysis or discussion related to the methodology for choosing the latent words. In particular, an experiment in which the latent words are just random points within the neighborhood (rather than linear combinations of existing embeddings) would be very important.

Overall, I think this paper needs considerable work to improve the clarity of exposition, and could use some additional experiments to support the proposed method for choosing latent words.
review_title: Interesting idea, lacking in clarity

final_score: rating: 3: Clear rejection
review_content: This paper argues that word embedding algorithms are "unstable" and "inconsistent", although I am not sure what is meant by these terms, and proposes an alignment solution, although I'm not sure how this solution really works.  The authors state: 

"For a fixed local neighborhood size, we re-train the same model, using the same parameters, on the same training dataset. We then measure model stability as a function of neighborhood overlap across consecutive re-trained model instances."

I'm not sure what is meant by a "fixed local neighbourhood" (do you mean a fixed region in the induced embedding space? how is this chosen?). Nor do I understand what "neighborhood overlap" means (do you mean the fraction of overlapping words found in the same fixed regions in the two different embedding spaces?) 

My best guess is that the authors train two word embedding models initialized differently on the same data, and then measure the fraction of common words that are embedded in the same "local neighborhood", i.e. same volume of the induced embedding space.

That different models learn different embeddings which are not aligned is not at all a surprising finding. Furthermore, that these embedding spaces of different models can be aligned to transfer information from one space to the other is also not surprising. This has motivated a large amount of work on bilingual / multilingual word embeddings (see [1] for just one example), and semi-supervised word embeddings (see [2] for one example).

I was hoping that the mention of "latent words" meant that the authors had some kind of latent-variable approach to automatically learn the alignments, but I couldn't understand how the "latent words" are generated nor how they are used in the alignment process?

Overall, I am not sure what problem the authors are solving and I'm not sure what their proposed solution really involves, and lastly, I'm not sure in what way the intrinsic evaluation results shown are meant to be interpreted as a measure of success (does the proposed method actually improve results in a real task?). 

[1] "BilBOWA: Fast Bilingual Distributed Word Representations without Word Alignments", Gouws et al., ICML 2014, https://arxiv.org/pdf/1410.2455.pdf.
[2] "Retrofitting Word Vectors to Semantic Lexicons", Faruqui et al., 2014, https://arxiv.org/pdf/1411.4166.pdf.
review_title: Not sure what is being proposed

Decision: Reject