Venue: ICLR 2018 conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: The paper proposes a method to learn bilingual dictionaries without parallel data using an adversarial technique. The task is interesting and relevant, especially for in low-resource language pair settings.

The paper, however, misses comparison against important work from the literature that is very relevant to their task — decipherment (Ravi, 2013; Nuhn et al., 2012; Ravi & Knight, 2011) and other approaches like CCA. 

The former set of works, while focused on machine translation also learns a translation table in the process. Besides, the authors also claim that their approach is particularly suited for low-resource MT and list this as one of their contributions. Previous works have used non-parallel and comparable corpora to learn MT models and for bilingual lexicon induction. The authors seem aware of corpora used in previous works (Tiedemann, 2012) yet provide no comparison against any of these methods. While some of the bilingual lexicon extraction works are cited (Haghighi et al., 2008; Artetxe et al., 2017), they do not demonstrate how their approach performs against these baseline methods. Such a comparison, even on language pairs which share some similarities (e.g., orthography), is warranted to determine the effectiveness of the proposed approach.

The proposed methodology is not novel, it rehashes existing adversarial techniques instead of other probabilistic models used in earlier works. 

For the translation task, it would be useful to see performance of a supervised MT baseline (many tools available in open-source) that was trained on similar amount of parallel training data (60k pairs) and see the gap in performance with the proposed approach.

The paper mentions that the approach is “unsupervised”. However, it relies on bootstrapping from word embeddings learned on Wikipedia corpus, which is a comparable corpus even though individual sentences are not aligned across languages. How does the quality degrade if word embeddings had to be learned from scratch or initialized from a different source?
review_title: Review

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper presents a new method for obtaining a bilingual dictionary, without requiring any parallel data between the source and target languages. The method consists of an adversarial approach for aligning two monolingual word embedding spaces, followed by a refinement step using frequent aligned words (according to the adversarial mapping). The approach is evaluated on single word translation, cross-lingual word similarity, and sentence translation retrieval tasks.

The paper presents an interesting approach which achieves good performance. The work is presented clearly, the approach is well-motivated and related to previous studies, and a thorough evaluation is performed.

My one concern is that the supervised approach that the paper compares to is limited: it is trained on a small fixed number of anchor points, while the unsupervised method uses significantly more words. I think the paper's comparisons are valid, but the abstract and introduction make very strong claims about outperforming "state-of-the-art supervised approaches". I think either a stronger supervised baseline should be included (trained on comparable data as the unsupervised approach), or the language/claims in the paper should be softened. The same holds for statements like "... our method is a first step ...", which is very hard to justify. I also do not think it is necessary to over-sell, given the solid work in the paper.

Further comments, questions and suggestions:
- It might be useful to add more details of your actual approach in the Abstract, not just what it achieves.
- Given you use trained word embeddings, it is not a given that the monolingual word embedding spaces would be alignable in a linear way. The actual word embedding method, therefore, has a big influence on performance (as you show). Could you comment on how crucial it would be to train monolingual embedding spaces on similar domains/data with similar co-occurrence statistics, in order for your method to be appropriate?
- Would it be possible to add weights to the terms in eq. (6), or is this done implicitly?
- How were the 5k source words for Procrustes supervised baseline selected?
- Have you considered non-linear mappings, or jointly training the monolingual word embeddings while attempting the linear mapping between embedding spaces?
- Do you think your approach would benefit from having a few parallel training points?

Some minor grammatical mistakes/typos (nitpicking):
- "gives a good performance" -> "gives good performance"
- "Recent works", "several works", "most works", etc. -> "recent studies", "several studies", etc.
- "i.e, the improvements" -> "i.e., the improvements"

The paper is well-written, relevant and interesting. I therefore recommend that the paper be accepted.


review_title: Good and interesting work (a few issues on the paper's claims)

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: An unsupervised approach is proposed to build bilingual dictionaries without parallel corpora, by aligning the monolingual word embeddings spaces, i.a. via adversarial learning.

The paper is very well-written and makes for a rather pleasant read, save for some need for down-toning the claims to novelty as voiced in the comment re: Ravi & Knight (2011) or simply in general: it's a very nice paper, I enjoy reading it *in spite*, and not *because* of the text sales-pitching itself at times.

There are some gaps in the awareness of the related work in the sub-field of bilingual lexicon induction, e.g. the work by Vulic & Moens (2016).

The evaluation is for the most part intrinsic, and it would be nice to see the approach applied downstream beyond the simplistic task of English-Esperanto translation: plenty of outlets out there for applying multilingual word embeddings. Would be nice to see at least some instead of the plethora of intrinsic evaluations of limited general interest.

In my view, to conclude, this is still a very nice paper, so I vote clear accept, in hope to see these minor flaws filtered out in the revision.
review_title: Well-rounded contribution, nice read, incomplete related work

Meta-review: There is significant discussion on this paper and high variance between reviewers:  one reviewer gave the paper a low score.  However the committee feels that this paper should be accepted at the conference since it provides a better framework for reproducibility, performs more large scale experiments than prior work.  One small issue the lack of comparison in terms of empirical results between this work and Zhang et al's work, but the responses provided to both the reviewers and anonymous commenters seem to be satisfactory.
Decision: Accept (Poster)