Paper:
Recent work on bilingual lexicon induction (BLI) has frequently depended either on aligned bilingual lexicons or on distribution matching, often with an assumption about the isometry of the two spaces. We propose a technique to quantitatively estimate this assumption of the isometry between two embedding spaces and empirically show that this assumption weakens as the languages in question become increasingly etymologically distant. We then propose Bilingual Lexicon Induction with Semi-Supervision (BLISS) -a novel semi-supervised approach that relaxes the isometric assumption while leveraging both limited aligned bilingual lexicons and a larger set of unaligned word embeddings, as well as a novel hubness filtering technique. Our proposed method improves over strong baselines for 11 of 14 language pairs on the MUSE dataset, particularly for languages whose embedding spaces do not appear to be isometric. In addition, we also show that adding supervision stabilizes the learning procedure, and is effective even with minimal supervision.

Review:
This paper presents a new semi-supervised method to learn cross-lingual word embeddings mappings combining unsupervised distribution matching, alignment over a training dictionary, and a weak orthogonality constraint. The paper also shows that the underlying isometry assumption in orthogonal mappings weakens as the languages involved are more distant, proposes a new method to quantify the strength of the said assumption, and argues that the proposed semi-supervised mapping method is particularly suited for these more challenging cases.
I think that this is a solid work that explores an interesting research direction within cross-lingual embedding mappings. While the basic ingredients of the proposed method are not new, their combination is certainly original. In that regard, I think that the paper is rather incremental, but still has enough substance to make an interesting contribution. However, I think that some parts of the paper are too obscure, and I am not fully convinced by the experiments. I would appreciate if the authors could address my concerns below, and I would be happy to modify my score accordingly:
1) My understanding is that the proposed method (the one named BLISS in the experiments) only makes use of the proposed semi-supervised framework (Section 3.2) and is not followed by the iterative procrustes refinement (Section 3.3), but this is not clear at all from the paper. Could you clarify this?
2) It is well known that the retrieval method can have a big impact in bilingual dictionary induction due to the hubness problem. However, the paper does not detail which retrieval method is used in the experiments. I assume that MUSE uses CSLS and Vecmap uses nearest neighbor over cosine. Is this correct? What retrieval method does BLISS use?
3) I assume that when you talk about the "CSLS metric" in page 7 and 13 you refer to the unsupervised validation criterion of Lample et al. (Section 3.5 in their paper), and not to CSLS itself (Section 2.3 in their paper). In either case, this needs some clarification.
4) Unlike the "train" dictionaries, the "full" dictionaries from MUSE as provided at github also include the test set. Do you preprocess them to exclude the test set? If so, this should be clearly stated in the paper. If not, this would invalidate all these experiments.
5) The authors use different language pairs in their different result tables, which I find very confusing. For instance, none of the language pairs in table 5 (except for en-ru), are included in the main results (table 3), so we do not know how the different baselines and variants perform in them. Is there any reason for that?
6) Could you include all MUSE variants in Table 4?
7) While you compare your method to different versions of Vecmap (Artetxe et al., ACL 2017 & AAAI 2018), the last one (Artetxe et al., ACL 2018) (http://aclweb.org/anthology/P18-1073) is missing. That paper reports 48.1% and 48.2% accuracy for en-it and en-de in the unsupervised case, which is substantially better than your results for en-it (45.9%) and at par for en-de (48.3%). This goes against the main motivation of the paper (i.e. unsupervised distribution information and supervision from dictionaries can be combined for best results), as a completely unsupervised method seems to perform better than (or at least at par with) the proposed semi-supervised method. I think that the paper should include some discussion on this. In particular, I would like to know whether you have any argument to believe that both works are complementary.