Improving Bilingual Lexicon Induction for Low Frequency Words (2020)

Abstract: This paper designs a Monolingual Lexicon Induction task and observes that two factors accompany the degraded accuracy of bilingual lexicon induction for rare words. First, a diminishing margin between similarities in low frequency regime, and secondly, exacerbated hubness at low frequency. Based on the observation, we further propose two methods to address these two factors, respectively. The larger issue is hubness. Addressing that improves induction accuracy significantly, especially for low-frequency words.

## introduction bilingual lexicon induction (bli) studies how to generate word-level translations from non-parallel corpora in two languages. recently, @xcite observe that rarer words are harder to translate than frequent ones. but their bli method is based on various "hand-crafted" features. we show that the same phenomenon occurs as well in bli methods that are based on word embeddings. this type of methods have become especially popular in recent years @xcite @xcite and achieved state-of-art accuracies @xcite . we briefly review bli methods that are based on word embeddings. without loss of generality, in this paper, we focus on "supervised" bli, which assumes that a seeding dictionary is available. unsupervised bli @xcite often alternates between inducing a seeding dictionary and using that to refine generated translations. therefore, to some extent, "supervised" bli is a key step in its "unsupervised" counterpart, and a more basic prototype to study. let the source space be x [x 1 , . . . , x m ], where x @xmath0. similarly, let the target space be y the typical supervised bli works by first learning a transformation w that minimizes the discrepancy between x s and y s , once the transformation w is learned, translation can be induced for a word x i , by retrieving the nearest neighbor (nn) of wx i in y. cosine distance is often adopted in the retrieval. solving the procrustes problem (eq. ( 1 )), followed by nn search is a representative framework. we use it as a baseline of this paper. despite the existing success in word embedding based bli, understanding of its performance against word frequency is still lacking. this paper observes that bli's accuracy significantly degrades for low-frequency words. then, two factors are identified that may explain the observation. motivated by them, we propose two methods that address each of the two factors, both improving bli's performance in low-frequency regime. ## lexicon induction at low frequency we study how induction accuracies vary for words of different frequencies. before we start, it should be emphasized that the frequency ranking of a source word and its translation(s) can differ a lot in their respective language. we term this fact as frequency mismatch, and the extent of mismatch also depends on the language pair. to simplify the problem, we design a "monolingual lexicon induction" (mli) task. ## two methods motivated by the two diagnostics in the last section, we introduce two methods, each individually improving the accuracy in low frequency regime. ## conclusion accuracy of bilingual lexicon induction decays for low-frequency words, as indicated by two factors: (1) diminishing margin between cosine similarities, and (2) exacerbated hubness. two methods are proposed to address each factor. experimental results validate their effectiveness.