Multi-Stage Framework with Refinement Based Point Set Registration for Unsupervised Bi-Lingual Word Alignment (2022)

Abstract: Cross-lingual alignment of word embeddings are important in knowledge transfer across languages, for improving machine translation and other multi-lingual applications. Current unsupervised approaches relying on learning structure-preserving transformations, using adversarial networks and refinement strategies, suffer from instability and convergence issues. This paper proposes BioSpere, a novel multi-stage framework for unsupervised mapping of bi-lingual word embeddings onto a shared vector space, by combining adversarial initialization, refinement procedure and point set registration. Experiments for parallel dictionary induction and word similarity demonstrate state-of-the-art unsupervised results for BioSpere on diverse languages – showcasing robustness against variable adversarial performance.

## introduction and background distributed word representations like word2vec @xcite , glove @xcite and fasttext @xcite capture rich semantic meaning, and is used for a range of nlp tasks. cross-lingual word embeddings (clwe) entails mapping vocabularies of different languages onto a single vector space for capturing semantic similarity across languages @xcite -for machine translation @xcite , pos tagging @xcite , & named entity recognition @xcite @xcite . linguistic correlation. this work is based on the observation that, monolingual representation spaces learnt independently for different languages tend to exhibit similarity in terms of geometric properties and orientations @xcite foot_0 . further, the frequency of words across languages have also been shown to follow the zipf's distribution foot_1 , with an overlap of nearly 70% for the most frequent words @xcite and 60% for synonyms @xcite across language pairs. existing techniques for extracting cross-lingual word correspondences rely on above inter-dependencies to learn transformations across monolingual embedding spaces. ## empirical evaluation we evaluate biospere on mapping semantically similar words across languages, for bi-lingual dictionary induction, word similarity and sentence translation retrieval tasks across diverse languages. dataset. we follow the setup of @xcite , and use fasttext monolingual vector embeddings (with 300 dimensions) @xcite for the top 200k most frequent words of each language as input vocabulary. we consider 8 different language pairs (including morphologically rich) -english (en), german (de), french (fr), spanish (es), italian (it), russian (ru), hebrew (he), finnish (fi), and romanian (ro) -a mix of isolating, fusional and agglutinative languages with dependent and mixed marking @xcite . evaluation. on word translation (dictionary induction), we use the gold dictionary with 1,500 source test words (and 200k target vocabulary) (github.com/facebookresearch/muse), while sentence translation retrieval uses europarl corpus containing 2,000 source and 200k target sentences. we report precision@1 (p@1) based on csls criteria @xcite . for word similarity on semeval 2017 data @xcite we report the pearson's correlation. baselines. the performance of biospere is compared against the following unsupervised methods: we also report the supervised approaches: (1) rcsls @xcite : optimizes csls criteria for learning @xcite ; ## conclusion this paper proposed biospere, a multi-stage unsupervised cross-lingual word embedding alignment framework -based on the novel coupling of generative adversarial training, refinement procedure and point set registration. experiments with diverse tasks on multiple languages demonstrate improved results over existing methods, and also depict robustness to hubness and adversarial performance.