Improving Word Translation via Semantic Cloning for Polysemous Words
Authors: Qiuyu Ding, Hailong Cao, Yongwei Zhou, Tiejun Zhao


Reviewer paper summaries: This paper addresses the problem of isomorphic imbalance in bilingual lexicon induction (BLI) by generating clones of polysemous words which are used for training cross-lingual word embeddings. It conducts experiments on 6 language pairs and derives small improvement in precision scores compared to baselines.
Summary of strengths: 1. The motivation of addressing isomorphic imbalance is valid and the corresponding methodological design is rational.

2. Small improvements in precision scores are observed against baselines.

3. Analyses on Hausdorff distance are interesting and informative.
Summary of weaknesses: 1. There is strong trend in the field of BLI towards purely prompting LLMs. [1] and [2] show that prompting off-the-shelf LLMs (no fine-tuning on BLI data needed) can already easily surpass all embedding-based BLI approaches on languages supported by the LLMs (LLAMA, GPT3.5, and so forth). The submission mentioned [1] but did not adopt the templates and prompting strategies recommended in [1] or [2]. On the contrary, the paper adopts a relatively quite poor-quality template (Appendix A.6) and only zero-shot prompting strategy, making the results reported in Table 1 even worse than embedding-based baselines. Although the submission only focuses on the embedding-based approach, the authors could have more discussions on how to link or extend the proposed method on polysemy to overall SOTA BLI approaches including [1] [2] (at least in a future work section) in addition to discussing only the embedding-based BLI methods which are already to some extent irrelevant in modern NLP research communities. Also, the claim made in footnote 1 seems also a bit outdated since embedding-based approaches already lag behind prompting LLMs on BLI benchmarks as reported in [1] and [2].

    [1] On Bilingual Lexicon Induction with Large Language Models. EMNLP 2023.

    [2] Self-Augmented In-Context Learning for Unsupervised Word Translation. Arxiv 2024.

2. The authors may enhance presenting the effectiveness of proposed approach. First, the overall improvement in BLI scores seem trivial (in this case, the authors may provide significance level on the precision scores; if the improvements are actually significant, then it is already convincing even though the absolute increase in scores is not large), and the analyses on robustness (randomness, how many random seed and runs are conducted, what is the standard deviation over runs) seem incomplete. Second, on the BLI test sets, there could also be polysemy phenomenon. I was wondering if the motivation of the work is only to derive better supervision signals from the training set or it also aims to improve the BLI performance on test set words with polysemy? If it is the latter, it would be beneficial to show the statistics of polysemy also in the BLI test set data and how much improvement can be achieved on the test set words with polysemy. In addition, when there is polysemy in the BLI test set, how are BLI scores calculated and if the calculation is the same as baselines?
Comments, suggestions and typos: 1. What does "data fusion" mean in footnote 4?

2. The paper's discussions of paper [1] in footnote 4 and line 548 seem inaccurate, highlighting the BLI fine-tuning. However, most of [1] experiments actually only leverage off-the-shelf LLMs (mT5, LLAMA, and so forth) and the best results are achieved via few-shot in-context prompting as a way to use the seed dictionary (I carefully read and double-checked [1]).

    [1] On Bilingual Lexicon Induction with Large Language Models. EMNLP 2023.

3. In line 871, the citation should be [1] instead of Li et al. 2022.

4. For the LLM prompting baselines described in Lines 381-387, what are their technical similarity and difference with [1]? More details should be included in the paragraph of Lines 381-387.

5. Recommend to add some references on using Hausdorff distance to measure the isometry between two embedding spaces. Concerning the formulation of Hausdorff distance in Eqs. 10-12, it seems a little confusing for me. For example, should Eq. 10 be any of the following?

    (1) $\underset{x}{max}(\underset{y}{min}(cos(x,y)))$

    (2) $\underset{y}{max}(\underset{x}{min}(cos(x,y)))$

    Another definition of the distance can be found on [https://en.wikipedia.org/wiki/Hausdorff_distance](https://en.wikipedia.org/wiki/Hausdorff_distance), and I found sth like $\underset{x}{sup}(\underset{y}{inf}(d(x,y)))$. However, in the BLI setup, larger cosine similarity corresponds to lower distance, so if the definition here should be sth like $min(max(cos(x,y)))$? It would also be helpful, if the authors could clarify if L2 normalization is conducted on cross-lingual word embeddings.

6. Recommend authors to add more details on the two stronger baselines BRSR and CLBLI. What is the cross-lingual word embedding model underlying BRSR, and for CLBLI, if its C1 variant (static embeddings) or C2 variant (static + dynamic embeddings) is used?

7. Is the proposed approach only applied on training set or is there any Semantic Clone operations also conducted on the test set words during BLI inference? If the former is true, wondering if it can cause some gap between BLI training and evaluation.

8. Recommend authors to proofread the whole paper carefully.

9. Typos: Appendix A.1 Spanis -> Spanish
Overall assessment scores: 3