Self-Augmented In-Context Learning for Unsupervised Word Translation (2024)

Abstract: Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of ‘traditional’ mapping-based approaches in the unsupervised scenario where no seed translation pairs are available, especially for lower-resource languages. To address this challenge with LLMs, we propose self-augmented in-context learning (SAIL) for unsupervised BLI: starting from a zero-shot prompt, SAIL iteratively induces a set of high-confidence word translation pairs for in-context learning (ICL) from an LLM, which it then reapplies to the same LLM in the ICL fashion. Our method shows substantial gains over zero-shot prompting of LLMs on two established BLI benchmarks spanning a wide range of language pairs, also outperforming mapping-based baselines across the board. In addition to achieving state-of-the-art unsupervised BLI performance, we also conduct comprehensive analyses on SAIL and discuss its limitations.

heading: 1 Introduction and Motivation
text: The task of word translation (WT), also known as bilingual lexicon induction (BLI), aims to automatically induce lexica of words with the same or similar meaning in different languages, thus bridging the lexical gap between languages. Even in the era of large language models (LLMs), BLI still has wide applications in machine translation and cross-lingual transfer learning (Sun et al., 2021; Zhou et al., 2021; Wang et al., 2022; Ghazvininejad et al., 2023; Jones et al., 2023). A particular BLI setup, termed (fully) unsupervised BLI, is especially compelling because it is not only more technically challenging but is also used as a pivotal component towards unsupervised machine translation (Lample et al., 2018; Artetxe et al., 2018b; Marchisio et al., 2020; Chronopoulou et al., 2021).
Until recently, BLI approaches have predominantly relied on learning cross-lingual word embedding (CLWE) mappings: these are known as MAPPING-BASED approaches and are developed based on static or decontextualised word embeddings (WEs) (Patra et al., 2019; Grave et al., 2019; Li et al., 2022a; Yu et al., 2023). Meanwhile, autoregressive LLMs have become the cornerstone of modern NLP techniques (Brown et al., 2020; Ouyang et al., 2022; Touvron et al., 2023a) with success in many real-world tasks (Kasneci et al., 2023; Wu et al., 2023; Thirunavukarasu et al., 2023; Li et al., 2024). Given this trend, recent BLI research has also started to shift towards exploring LLMs. Li et al. (2023) first show that prompting LLMs with gold-standard WT pairs as in-context examples (few-shot in-context learning: ICL) outperforms all existing BLI approaches in the supervised and semi-supervised BLI setups (where typically 1K∼5K gold-standard WT pairs are available for training or ICL), while zero-shot prompting still falls behind traditional MAPPING-BASED approaches in the fully unsupervised BLI setup, especially for lower-resource languages.
In this work, we thus aim at improving unsupervised BLI with LLMs. To this end, we analyze the limitations of zero-shot prompting and propose a novel self-augmented in-context learning (SAIL) method for unsupervised BLI with LLMs. The key idea is to first retrieve a set of high-confidence WT pairs by zero-shot prompting LLMs, then iteratively refine the high-confidence dictionary and finally use the gradually refined bilingual lexicon for BLI inference in an ICL fashion (§2). Our extensive experiments show that SAIL establishes new state-of-the-art unsupervised BLI performance on two standard BLI benchmarks. We also conduct thorough analyses on our approach, providing further insights into its inner workings (§3- §4). Our code is publicly available at https: //github.com/cambridgeltl/sail-bli.
743

heading: 2 Methodology
text: Unsupervised BLI: Task Preliminaries. We assume a pair of two languages: a source language Lx with its vocabulary X and a target language Ly with vocabulary Y . In a typical, standard BLI setup the vocabulary of each language contains the most frequent 200, 000 word types in the language (Glavaš et al., 2019; Li et al., 2022a). Given a source word wx ∈ X , the unsupervised BLI task then aims to infer its translation in Ly, without any word-level parallel data (i.e., seed translation pairs from a lexicon) available for training or ICL.1
Zero-Shot Prompting. Li et al. (2023) have proposed to prompt autoregressive LLMs for the BLI task, where the input word wx is embedded into a predefined text template. We adopt the pool of templates provided by Li et al. (2023) and conduct template search for each LLM on a randomly chosen language pair. As an example, the zero-shot template for LLAMA-27B is as follows:2
‘The Lx word wx in Ly is:’,
where Lx, Ly, and wx are placeholders for the source language, target language, and the query word in the source language (e.g., Lx = Hungarian, wx = macska, Ly = Catalan).
The deterministic beam search (with beam size of n as a hyper-parameter) is adopted to generate n output text pieces in the final beam, ranked by their sequence scores.3 For each of the n outputs, the first word in the generated output following the input sequence is extracted as a candidate answer. After filtering out those candidate answers not in Y , the candidate Ly word with the highest associated sequence score is returned as the final word translation prediction.
Limitations of Zero-Shot Prompting. The above zero-shot approach for unsupervised BLI, proposed by Li et al. (2023), comes with several limitations. First, the template does not stipulate the output format and thus parsing the output text may not be as straightforward as expected. Put simply, LLM’s prediction may not be the first word in the generated sequence. Second, the LLM may not fully ‘understand’ the input template and sometimes may
1Following prior work, when wx has multiple ground truth translations in Ly , a prediction is considered correct if it is any of the ground truth answers.
2The full list of templates used for other LLMs are presented in Appendix C.
3We use n = 5 following Li et al. (2023).
tend not to generate words in the target language especially for lower-resource languages. For the supervised BLI setup, where a dictionary of gold standard translation pairs is assumed and available, few-shot in-context learning can substantially improve final BLI performance (Li et al., 2023), since it not only provides examples of the desired output format but also helps LLMs ‘understand’ the BLI task. However, the availability of such a seed dictionary is not assumed in the unsupervised BLI task variant, and the key idea of this work is to derive and iteratively refine a seed dictionary by prompting LLMs.
SAIL: Self-Augmented In-Context Learning for Unsupervised BLI. We thus propose to facilitate and improve unsupervised BLI by S1) using zero-shot prompting to retrieve Dh, a set of highconfidence translation pairs, and then S2) leveraging these pairs as ‘self-augmented’ in-context examples for few-shot prompting to further iteratively refine Dh (across 0 to Nit−1 iterations, where Nit is a hyper-parameter denoting total times of Dh inference in S1 and S2), and finally S3) conducting few-shot learning with the final, Nit-th self-created seed lexicon Dh for BLI inference on the test set. Deriving High-Confidence Pairs. For both steps S1 and S2 outlined above, we start with the most frequent Nf words in Lx since representations of less frequent words are considered to be much noisier in general (Artetxe et al., 2018a). For each wx, we conduct Lx → Ly translation: we refer to this predicted word as ŵy.4 We then propose to conduct word back-translation, translating ŵy from Ly back into Lx. The word pair (wx, ŵy) is considered a high-confidence pair only if wx is also the output word of the back-translation step.5 We denote the set of all high-confidence pairs from the Lx words as Dxh. Likewise, we also start from the most frequent Nf words in Ly and symmetrically derive Dyh. Finally, we update the high-confidence dictionary with Dh = Dxh ∪ D y h. 6
Few-Shot Prompting with High-Confidence Pairs. Step S1 of SAIL relies on zero-shot prompting, but all the subsequent iterations in S2 and
4We do not require ŵy to be one of the most frequent Nf words in Ly .
5Earlier MAPPING-BASED approaches have retrieved highconfidence pairs through ranking cross-lingual word similarity scores (e.g., cosine similarity) to refine CLWE mappings (Artetxe et al., 2018a; Li et al., 2022a); in a sense, our work renovates and revitalises the idea with LLMs.
6Therefore, |Dxh| ≤ Nf , |Dyh| ≤ Nf , and |Dh| ≤ 2×Nf .
S3 apply few-shot prompting/ICL with the ‘selfaugmented’ high-confidence translation pairs Dh. Following Li et al. (2023), we adopt 5-shot prompting, and again conduct template search on the BLI task with a single, randomly selected language pair.7 The in-context examples, (wxi , w y i ) ∈ Dh, 1 ≤ i ≤ 5, are retrieved where the wxi words are the nearest neighbours of the input word wx in Lx’s static word embedding space. The few-shot template for LLAMA-27B is then as follows:
‘The Lx word wx1 in L y is wy1. The Lx word wx2 in L y is wy2. ... The L x word wx in Ly is’.

heading: 3 Experimental Setup
text: BLI Data and LLMs. We adopt two standard BLI benchmarks: 1) 5 languages from XLING (Glavaš et al., 2019) including German (DE), English (EN), French (FR), Italian (IT), and Russian (RU), their combinations resulting in 20 BLI directions; 2) 3 lower-resource languages including Bulgarian (BG), Catalan (CA), and Hungarian (HU) from PanLex-BLI (Vulić et al., 2019), which result in 6 BLI directions.8 For both benchmarks, a test set of 2K WT pairs is provided for each BLI direction. We experiment with four open-source LLMs: LLAMA 7B, LLAMA-27B, LLAMA 13B, and LLAMA-213B (Touvron et al., 2023a,b). Li et al. (2023) found that 4 other families of LLMs, including mT5, mT0, mGPT and XGLM, underperform LLAMA; we thus skip these LLMs in our work.
Implementation Details and BLI Evaluation. As mentioned in §2, our hyper-parameter and template search are conducted on a single, randomly selected language pair, which is DE-FR, following Li et al. (2023). Batch size is set to 1. We adopt Nit = 1, Nf = 5, 000 in our main experiments (§4.1) and then investigate their influence on BLI performance and the effectiveness of our proposed word back-translation in our further analyses (§4.2). Half-precision floating-point format (torch.float16) is adopted for all our SAIL and zero-shot experiments. Since our method does not imply any randomness, all results are from single runs. For evaluation, we adopt the standard top-1 accuracy as prior work.
7The decoding and output parsing strategy is the same as in zero-shot prompting.
8The two datasets are also used in many recent BLI works (Sachidananda et al., 2021; Aboagye et al., 2022; Li et al., 2022a,b; Vulić et al., 2020, 2023; Li et al., 2023).
Baselines. We adopt two established MAPPINGBASED baselines. 1) VECMAP is a representative unsupervised BLI approach and features a selflearning mechanism that refines linear maps for deriving CLWEs (Artetxe et al., 2018a). 2) CONTRASTIVEBLI learns CLWEs with a two-stage contrastive learning framework and is the strongest MAPPING-BASED approach for supervised and semisupervised BLI tasks on our two benchmarks (Li et al., 2022a); however, it does not support unsupervised setup. We extend CONTRASTIVEBLI to unsupervised BLI by initialising the initial map with the unsupervised VECMAP method. The CONTRASTIVEBLI C1 variant based on static WEs and its stronger C2 variant combining static and decontextualised WEs are both used as our baselines. We adopt Cross-domain Similarity Local Scaling (CSLS) retrieval (Lample et al., 2018) for all MAPPING-BASED approaches as recommended in the baselines. In addition, we report 3) ZERO-SHOT prompting with each of our LLMs as baselines following the previous findings of Li et al. (2023).

heading: 4 Results and Discussion

heading: 4.1 Main Results
text: Results on the Two BLI Benchmarks are summarised in Tables 1 and 2 respectively, with full BLI scores per each individual language pair in Tables 8 and 9 in Appendix F. As the main findings, 1) our SAIL shows consistent gains against ZERO-SHOT prompting for each of the 4 LLMs, showing the effectiveness of the proposed approach; 2) while ZERO-SHOT prompting still lags behind MAPPINGBASED approaches on PanLex-BLI’s lower-resource languages, applying SAIL outperforms MAPPINGBASED baselines across the board. The only exception is that CONTRASTIVEBLI (C2) still has a slight edge over SAIL with the weakest LLM overall, LLAMA 7B. 3) Among the 4 LLMs, LLAMA-213B presents the strongest BLI capability.
Variance and Statistical Significance. The whole SAIL method does not imply any variance due to randomness: it does not rely on any actual LLM fine-tuning; we adopt deterministic beam search; the deterministic nearest neighbour retrieval is used for deriving in-context examples. Here, we report the statistical significance with χ2 tests. When comparing SAIL and ZERO-SHOT prompting (both with LLAMA-213B), the p-value is 1.1e-251 on 20 XLING BLI directions and 2.7e-109 on 6 PanLex-BLI BLI directions. We then compare
SAIL (with LLAMA-213B) against CONTRASTIVEBLI (C2) which is our strongest MAPPING-BASED baseline: the p-values are 3.1e-300 and 7.8e-20 respectively. These show that our findings are strongly statistically significant.9

heading: 4.2 Further Analyses
text: Inspection of High-Confidence Dictionaries. To provide additional insight into our SAIL approach, we present statistics on the size of high-confidence dictionaries derived in our main experiments
9Usually p < 0.05 or p < 0.001 is considered to indicate statistical significance.
(Nit = 1, Nf = 5, 000, and with word backtranslation) over 20 XLING BLI directions and 6 PanLex-BLI BLI directions respectively for each of our four LLMs in Table 3. The values indicate that |Dh| of higher-resource languages (XLING) is typically greater than that of lower-resource languages (PanLex-BLI). In addition to the dictionary size, it is also worth investigating the quality of highconfidence dictionaries. However, to directly evaluate the quality of the ‘silver standard’ generated dictionaries is difficult since we do not have ground truth dictionaries for comparison. As a preliminary investigation, we randomly sample 50 translation pairs from the EN-DE LLAMA-213B-augmented dictionary and compare them with answers derived from Google Translate10 (EN→DE). We found that 40 out of the 50 pairs in our augmented dictionary are the same as the results from Google Translate. Although these results from Google Translate are also not ‘gold standard’ ground truth, it does point in the direction of reliability of extracted WT pairs.
Impact of Nit. Figure 1 shows the influence of the number of iterations Nit on the average BLI scores on XLING. When Nit = 1, where only step S1 is executed (see §2), SAIL already approaches (almost) its optimal performance. Further refining the Dh for more iterations (step S2) only leads to small fluctuations in BLI performance, which we deem not worth the increased computational cost. Figure 3 (Appendix B) with results on PanLex-BLI shows a similar trend.
Impact of Nf . We then study the impact of the frequency threshold Nf on the average BLI performance with a subset of XLING spanning DE-FR, EN-RU and RU-FR, each in both directions. The results in Figure 2 reveal that even with Nf = 1, 000, the BLI performance is boosted substantially when compared against the ZERO-SHOT baseline (i.e., when Nf = 0). When we further increase Nf , the
10https://translate.google.com/
accuracy score still increases slowly, and the gain seems negligible with Nf ≥ 5000: i.e., increasing Nf again may not be worth the extra computation.
Impact of Word Back-Translation. The backtranslation step aims to improve the quality of Dh. Here, we experiment with the ablated version of SAIL without back-translation on the same XLING subset (DE-FR, EN-RU and RU-FR) as before. The results in Table 4 clearly demonstrate the effectiveness of proposed word back-translation: the p-values (χ2 tests) are 8.8e-7 and 1.0e-10 respectively for LLAMA-27B and LLAMA-213B when comparing SAIL variants with and without the backtranslation mechanism.
CHATGPT for BLI? We additionally report GPT3.5 (OpenAI, 2022) and GPT-4 (Achiam et al., 2023) results on DE-FR, EN-RU and RU-FR with ZERO-SHOT prompting (see Appendix E for ex-
perimental details). Note that the procedure of instruction-tuning of LLMs usually covers largescale parallel data for machine translation. Therefore, leveraging CHATGPT models, even with ZEROSHOT prompting, is not in line with the motivation of unsupervised BLI and leads to unfair comparisons with the results of our main experiments and baselines.11 Here, we report CHATGPT results as an upper bound for ZERO-SHOT prompting. Our results in Table 5 show that 1) as expected, the instruction-tuned CHATGPT models outperform pretrained LLAMA-213B by a large margin in the ZEROSHOT setup, but 2) our SAIL method with the same pretrained LLAMA-213B outperforms both GPT-3.5 and the state-of-the-art GPT-412 in terms of the average performance, even for the selected higherresource languages, again demonstrating the effectiveness of the proposed SAIL approach.

heading: 5 Conclusion
text: We proposed Self-Augmented In-Context Learning (SAIL) to improve unsupervised BLI with LLMs. The key idea is to iteratively retrieve a set of highconfidence word translation pairs by prompting LLMs and then leverage the retrieved pairs as incontext examples for unsupervised BLI. Our experiments on two standard BLI benchmarks showed that the proposed SAIL method substantially outperforms established MAPPING-BASED and ZERO-SHOT BLI baselines. We also conducted a series of indepth analyses on the high-confidence dictionary, key hyper-parameters, and the back-translation mechanism, and we additionally show that our SAIL approach with LLAMA-213B can even outperform ZERO-SHOT prompting with the state-of-the-art GPT-4 model.
11The four LLAMA models used in our main experiments are pretrained LLMs without instruction-tuning (see Appendix D); our MAPPING-BASED baselines adopt static WEs derived from monolingual corpora of respective languages and our CONTRASTIVEBLI (C2) baseline additionally leverages pretrained mBERT (Devlin et al., 2019).
12We adopt the strong ‘gpt-4-turbo-2024-04-09’ model which ranked 1st on the LMSYS Chatbot Arena Leaderboard at the time of experimentation (May 12, 2024).
Limitations
The main limitation of this work, inherited from prior work as well (Li et al., 2023) is that the scope of our languages is constrained to the languages supported (or ‘seen’) by the underlying LLMs. For example, LLAMA-2 is reported to support only around 27 natural languages (Touvron et al., 2023b). This limitation could be mitigated if more advanced LLMs that support more languages are available in the future. It might also be feasible to adapt existing LLMs to more languages by fine-tuning on their monolingual corpora potentially combined with modern cross-lingual transfer learning techniques, whereas such adaptations of LLMs to unseen languages extend way beyond this work focused on the BLI task.
In addition, compared to the ZERO-SHOT baseline, our SAIL framework organically requires more computational time and budget, as reported in Table 7 of Appendix D.
Moreover, the SAIL framework is proposed and evaluated for the unsupervised BLI task. This work does not discuss if and how adapted variants of SAIL could also be applied to other NLP tasks beyond BLI. Further, the SAIL method should be equally applicable in weakly supervised BLI setups (Vulić et al., 2019) where a tiny set of available seed word translations (e.g., 50-500 word pairs) can be assumed to seed the iterative procedure. We leave this to future work.

heading: Acknowledgements
text: We thank the anonymous reviewers for their valuable feedback. Yaoyiran Li is supported by Grace & Thomas C. H. Chan Cambridge International Scholarship. Anna Korhonen is supported by the UK Research and Innovation (UKRI) Frontier Research Grant EP/Y031350/1 (the UK government’s funding guarantee for ERC Advanced Grants). Ivan Vulić is supported by a personal Royal Society University Research Fellowship Inclusive and Sustainable Language Technology for a Truly Multilingual World’ (no 221137).

heading: A Languages
text: B Impact of Nit with PanLex-BLI

heading: C Templates
text: Li et al. (2023) provide the suggested (carefully searched) templates for LLAMA 7B and LLAMA 13B, which we directly adopt in our work. For LLAMA27B and LLAMA-213B, we conduct template search following Li et al. (2023) on a single language pair DE-FR in both directions. For CHATGPT models used in §4.2, details about their templates are provided in Appendix E.
Zero-Shot Template. LLAMA 7B, LLAMA-27B and LLAMA-213B share the same zero-shot template as introduced in §2. LLAMA 13B’s zero-shot template is as follows:
‘Translate from Lx to Ly: wx=>’.
Few-Shot Template.. We have introduced the fewshot template of LLAMA-27B in §2. The remaining three LLMs happen to share the same few-shot template, given as follows:
‘The Lx word 'wx1' in L y is wy1. The Lx word 'wx2' in L y is wy2. ... The L x word 'wx' in Ly is’.

heading: D Reproducibility Checklist
text: • Source Code: our code is publicly available at https://github.com/cambridgeltl/sail-b li.
• Hyper-Parameter Search: Nit is selected from {1, 2, 3, 4} and Nf from {1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000}. • Software: Python 3.9.7, PyTorch 1.10.1, Transformers 4.28.1, OpenAI 1.28.1.
• Computing Infrastructure: we run our codes on Wilkes3, a GPU cluster hosted by the University of Cambridge. Each run makes use of a single Nvidia 80GB A100 GPU and 32× CPU cores. • Half-Precision Floating-Point Format: as introduced in §3, our BLI inference relies on torch.float16 for both our SAIL and the ZEROSHOT baseline. We have verified that fp16 can accelerate our computation with only negligible impact on the absolute BLI performance. Note that Li et al. (2023) did not specify torch.float16 in their ZERO-SHOT experiments with LLAMA 7B and LLAMA 13B, so the BLI scores reported are slightly different from ours.
• Data, WEs, LLMs: all the BLI data, WEs, LLMs (excluding CHATGPT models) and baseline codes are open-source and publicly available. The WEs for retrieving in-context examples are fastText WEs (Bojanowski et al., 2017) trained on monolingual corpora of respective languages: the version pretrained on Wikipedia13 is used for XLING and the version pretrained with Wikipedia plus Common Crawl14 is used for PanLex-BLI, as recommended by XLING and PanLex-BLI, respectively. The same WEs are used for our MAPPING-BASED baselines. The LLMs used in our main experiments (LLAMA models) are summarised in Table 7. Note that we only adopt pretrained versions of LLAMA (e.g., ‘meta-llama/Llama-2-7b-hf’) rather than the instruction-tuned models (e.g., ‘meta-llama/Llama-2-7b-chat-hf’). The details of CHATGPT models used in §4.2 are provided in Appendix E.
13https://fasttext.cc/docs/en/pretrained-vecto rs.html
14https://fasttext.cc/docs/en/crawl-vectors.h tml
• Baselines: for every baseline, we use its recommended setup for unsupervised BLI and make sure the recommended setup achieves its own (near)optimal performance. As introduced in §3, we extend CONTRASTIVEBLI to the unsupervised BLI setup. Specifically, we adopt the set of its hyperparameters recommended for the weakly supervised BLI setup, which we found can also achieve strong unsupervised BLI performance.
• Parameter Count and Runtime: we report the number of parameters of each LLM and the GPU runtime for BLI inference on a single BLI direction DE→FR, which contains circa 2K word pairs, in Table 7.
• Carbon Footprint: our work consumes about 750 A100 GPU hours in total. We estimate that our experiments causes the emission of circa 90kg CO2 equivalents according to a publicly available ‘machine learning emissions calculator’ (Luccioni et al., 2019)15.

heading: E Details of CHATGPT Experiments
text: We run our CHATGPT experiments introduced in §4.2 with the OpenAI API.16 The model ID for GPT-3.5 is ‘gpt-3.5-turbo-0125’. For GPT-4, we adopt the state-of-the-art ‘gpt-4-turbo-2024-04-09’ model which ranked 1st on the LMSYS Chatbot Arena Leaderboard at the time of experimentation (May 12, 2024).
Our input to CHATGPT consists of two types of input messages: a system message followed by a user message. For the user message, we adopt the following template for both GPT-3.5 and GPT-4 as recommended in Anonymous (2023):
‘Translate the Lx word wx into Ly:’,
which is also selected from the template pool of Li et al. (2023). We additionally adopt the following system message which is not used in Anonymous (2023) or Li et al. (2023):
‘Please complete the following sentence and only output the target word.’.
In our preliminary investigation, we find that our system message can considerably improve the BLI performance of both CHATGPT models.
15https://mlco2.github.io/impact/#compute 16https://platform.openai.com/docs/overview
There are two hyper-parameters used in our API calls: temperature = 0 and max_tokens = 5. Like our main experiments, we also extract the first word in the generated output sequence as the prediction for the target word. But different from our LLAMA experiments, we only derive a single output sequence from the CHATGPT API for each prompt. The code for our CHATGPT experiments is also provided in our GitHub repository.

heading: F Full BLI Results
text: Table 8 shows detailed BLI scores for each BLI direction in the XLING dataset. Similarly, individual per-direction results on PanLex-BLI are presented in Table 9.

heading: Future Work
text: This limitation could be mitigated if more advanced LLMs that support more languages are available in the future. It might also be feasible to adapt existing LLMs to more languages by fine-tuning on their monolingual corpora potentially combined with modern cross-lingual transfer learning techniques, whereas such adaptations of LLMs to unseen languages extend way beyond this work focused on the BLI task. In addition, compared to the ZERO-SHOT baseline, our SAIL framework organically requires more computational time and budget, as reported in Table 7 of Appendix D. Moreover, the SAIL framework is proposed and evaluated for the unsupervised BLI task. This work does not discuss if and how adapted variants of SAIL could also be applied to other NLP tasks beyond BLI. Further, the SAIL method should be equally applicable in weakly supervised BLI setups (Vulić et al. , 2019) where a tiny set of available seed word translations (e. g. , 50-500 word pairs) can be assumed to seed the iterative procedure. We leave this to future work