A Generalized Method for Automated Multilingual Loanword Detection (2022)

Abstract: Loanwords are words incorporated from one language into another without translation. Suppose two words from distantly-related or unrelated languages sound similar and have a similar meaning. In that case, this is evidence of likely borrowing. This paper presents a method to automatically detect loanwords across various language pairs, accounting for differences in script, pronunciation and phonetic transformation by the borrowing language. We incorporate edit distance, semantic similarity measures, and phonetic alignment. We evaluate on 12 language pairs and achieve performance comparable to or exceeding state of the art methods on single-pair loanword detection tasks. We also demonstrate that multilingual models perform the same or often better than models trained on single language pairs and can potentially generalize to unseen language pairs with sufficient data, and that our method can exceed human performance on loanword detection.

## introduction throughout history, words and phrases have been exchanged between languages around the world @xcite . this can obscure genetic relations between languages (e.g., many people erroneously believe english and french are more closely related than they are) but may also increase comprehension of foreign languages by monoglots (e.g., written french is often partially comprehensible by english speakers). as @xcite observe, detecting that a word is loanword is conceptually straightforward: both similar sound and meaning suggests too great a coincidence for different words to have converged by chance 1 . detecting loanwords computationally has therefore relied on pairwise similarity measures based on transliteration detection and edit distance. however, foundational work in linguistic borrowing, e.g., by @xcite and @xcite , established that when borrowing words into a recipient language, speakers of that language will reproduce existing linguistic patterns when using new words, and the patterns that recipient speakers impose upon a borrowed word vary across time @xcite , and language pairs. some languages may adopt a word without much phonetic change due to already-similar phonotactics. others may fit imported words into a rigid sound pattern, with sometimes significant transformation. still others may change the meaning. changes are particular to the language pair, so automatically detecting loanwords between arbitrary languages is challenging. however, if successful, such capabilities would also provide benefits to many other nlp tasks such as machine translation, coreference, and named-entity recognition (ner), because common vocabulary, coreferents, or named entities across languages may often be loanwords. here, we present a novel method for automated loanword detection between arbitrary language pairs. we build upon existing edit distancebased approaches, incorporate semantic similarity metrics from multilingual language models mbert @xcite and xlm @xcite , and a method of assessing alignment of phonemes between donor words and loans to account for differences in phonotactics between the relevant languages. we also present and evaluate on the wiklow (wiktionary loanword) dataset, currently consisting of 13 language pairs with a high density of loanwords and 3 further language pairs with a lower density of loanwords. we also provide a methodology for expanding the dataset to new language pairs. we demonstrate that our method to detect loanwords across all language pairs in the dataset performs comparably to or better than existing methods on language-specific loanword detection tasks, that multilingual models can perform better than models trained on individual language pairs, even on data from that pair itself, and that our model can also exceed human performance. 2 our method supports both loanword detection and construction of parallel corpora of loanwords for other tasks. our conclusions suggest that there are some general principles of loanword detection that can be picked up by machine learning models independent of specific languages, and we propose follow-up challenges for nlp research in this area. ## related work prior approaches to detecting loanwords computationally follow the intuition mentioned above: that if two words in otherwise not closely related languages have similar meaning and sound similar, then this is likely evidence of borrowing. @xcite use a levenshtein-distance based approach to identify language groups and loanwords among languages of central asia. delz (2013)/köllner (2021) proposes theoretical approaches to loanword identification based on phylogenetic methods. @xcite also point out an issue we address herein: loanwords may be transformed to fit the borrowing language's phonology and phonotactics, so pronunciation similarity may be a weaker than ideal method. existing data resources relevant to loanwords include the the automated similarity judgment project (asjp) database @xcite and the world loanword database (wold) @xcite . our data source is wiktionary, which has previously been used in related etymological tasks by de melo (2014) and @xcite . one thing we should note is that much work in computational loanword detection and similar tasks is targeted at a specific language or group of languages, e.g., @xcite , japanese @xcite , uyghur @xcite @xcite , spanish @xcite , central asian languages @xcite , or turkic and indo-iranian @xcite . our approach attempts to address the problem at a multilingual level. we use and extend existing work in phonological processing by the nlp community, including the epitran @xcite and @xcite packages for representing phonetic and articulatory features. we incorporate semantic similarity measures from multilingual language models mbert and xlm, and develop a method of scoring the level of alignment of phonemes between a donor and a loanword to account for differences in language-specific phonology and phonotactics. our approach in principle supports loanword detection on any pair of languages supported by the upstream packages/models epitran, mbert, and xlm, but we discuss how we have (sec. 3) and can (sec. 8) also extend our approach to languages that are not at present covered by all of these. a work at a similar scale, albeit on the slightly different task of cognate classification, is jäger (2018), which evaluates pmi and svm-based methods over the asjp database. cognate detection work generally uses similar methods to those we use here, e.g., semantic and phonetic similarity @xcite , orthographic distance @xcite combined with semantic information @xcite , or global constraints @xcite . work in translation lexicons (e.g., @xcite ) is also relevant, for the hybrid approach to similarity metrics. loanword detection may be useful for phylogenetic reconstruction, like cognate detection @xcite . however, cognates are valid for reconstructing common ancestry; loanwords are not. for historical reconstruction, the two must be separated. many in the nlp community adopt a definition of "cognate" that subsumes loanwords (e.g., @xcite ). we do not adopt this definition, and use the linguistic definition that treats loanwords and cognates as distinct. ## data collection the wiklow dataset is collected using the process outlined in this section, which can be run for any pair of languages that have loans between them catalogued in wiktionary, making it easy to expand to new data. we begin by collecting data from wiktionary categories of the form [recipient]_terms_borrowed_from_ [donor] foot_1 . each link in the category is scraped for a loanword in the recipient language and the original form of that word in the donor language. we also collect all the available lemmas in the donor language, which we use later to calculate the closest phonetic neighbors for each loanword. we also collect homonyms for each loanword where available; homonyms are considered those words that have more than one etymology, where one is a loan from the relevant donor language foot_3 . using the epitran package (mortensen et al., 2018), we transliterate both loans and original words into the international phonetic alphabet (ipa). the epitran package can be extended to support new languages, as we did here in the case of finnish, using omniglot foot_4 as a resource. epitran is not a perfect mapping to real pronunciation, especially in the case of abjads such as arabic script, a point of relevance later (sec. 4.4, sec. 7.1). having gathered positive examples of loanwords, we need to gather sufficient negative examples to both train an algorithm, and to try and fool the trained algorithm. negative examples can be: to create the synonyms dataset, we take a list of 440 english words, each of which has multiple synonyms associated with it. with the google translate api, we translate the main word into one language from our current relevant pair, and each synonym into the other. we then construct word pairs in the donor and recipient language using the cartesian product of each word with each translated synonym. we remove any duplicates, and any pairs that also occur in the loanword dataset, as we do not want true positives labeled as negatives when training the loanword detection model. to create the hard negatives dataset, we use the panphon package @xcite to compute six edit distances (see sec. 4.2) between the ipa transcriptions of the gathered loanwords, and up to 20,000 candidate lemmas of the donor language, which are also transliterated into the ipa using epitran. the result here is that each loanword is paired with up to six candidates that have a low phonetic edit distance but are not the original word in the donor language. we remove duplicates where multiple distance metrics chose the same closest neighbor, and where pairs cooccur with the synonyms or loans datasets. finally in the randoms dataset, we pair each loan with a random word in the donor language. ## similarity metrics every word pair in the wiklow dataset has measures of textual, phonetic, semantic, and articulatory similarity associated with it. ## evaluation for evaluation, we create three data distributions for each language pair. one (the balanced distribution), contains half loanwords and half nonloans. this is a well-behaved distribution wellsuited for machine learning. the non-loans are drawn roughly 1 7 from the hard negatives, 4 7 from the synonyms, and 2 7 from the randoms, reflecting the notion that relatively few words in a language are likely to be very phonetically close to a loanword on average, while there are likely to be many more words of synonymous or similar meaning. another distribution attempts to approximate the actual proportion of loanwords from the donor language into the recipient language (the "realistic" distribution, or realdist). sometimes this proportion is well-documented, and at other times not. 8 . where a figure is provided in the linguistic literature, we use it. otherwise, we take the number of loanwords we collected from wiktionary and divide it by the total number of lemmas in the borrowing language, and impose a lower bound of 10%, to maintain enough loanwords in the testing set. the non-loans portion of the realdist set is drawn in the same proportions as in the balanced set. for all language pairs currently in the wiklow dataset, the realdist @xmath0, but for other language pairs, e.g., korean-chinese, >50% loanwords is certainly possible or likely @xcite . the final distribution (abbreviated alldata), takes all the data we collected from wiktionary, to purposely overweight the dataset against loanwords, to test our method in a difficult condition. to each distribution, we concatenate two one-hot vectors representing the scripts of the languages in the pair. this allows certain models to learn dependencies between the scripts and other variables, e.g., if the languages are written in different scripts, the textual levenshtein distance becomes nearly meaningless. each distribution was divided into a 90:10 train/test split, and then shuffled. we evaluate four different binary classifiers on all distributions: a logistic regressor (lr), a linear svm, a random forest (rf), and a deep neural network (nn). the neural network consists of 3 layers of 512, 256, and 128 hidden units respectively, all with relu activation and followed by 10% dropout, and a final sigmoid activation, and is trained for 5,000 epochs with adam optimization and bce loss. we perform the evaluations listed below. single multilingual model (smm) for each different data distribution, we train a single model on the data from every language pair listed in ## results our primary metrics are precision, recall, and f1score on positive loanword identification. distribution of the 4 classifiers we evaluated. the remaining tables and figures all focus on the results of the neural network, are sorted by decreasing number of loanwords in the language pair, and are discussed in sec. 7. ## discussion we can quantitatively compare our approach to that of @xcite , who report 75.35% average precision, 74.09% average recall, and 74.71% average f1 on loanword detection in uyghur on borrowings from russian, arabic, turkish, and chinese. our results are on different language pairs but are comparable to or exceed this, particularly if the testing set is balanced between loans and non-loans. in fig. 2 , we can see that in most cases, the multilingual model outperforms the single-pair models on the same language pair on loanword retrieval, though this effect is most pronounced in language pairs with a higher density of loanwords. the model trained on the smaller pruned realdist data sees an appreciable drop in precision, but an equal or greater increase in loanword recall, and this effect is especially pronounced in pairs with fewer loanwords in the data overall, suggesting that training on a more realistic distribution may be advantageous when prioritizing reducing false negatives. fig. 3 shows the correlation between test set size and performance of the smm (including unseen language pairs). there appears to be a strong correlation between the proportion of loanwords in a test set (as expected, a balanced set leads to optimal performance), but also the raw size of the test set itself. the model performs better on larger test sets, unseen or not, regardless of what data it was trained on. we speculate that this may be because when a borrowing language borrows a lot of words from a donor language, it does so at around the same time (e.g., english from norman french), meaning fa-ar hu-de de-it ca-ar p (+) 95 95 73 100 97 100 100 75 75 73 54 25 r (+) 75 36 33 20 97 93 92 30 64 30 29 10 f1 (+) 84 52 46 33 97 96 96 43 69 43 38 14 ## conclusions and future work automated loanword detection enables a number of downstream tasks. coreferents and named entities across languages may often be loanwords, and common vocabulary enables potential improvements in machine translation @xcite . parallel corpora of loanwords also afford learning cross-lingual contextual word embedding mappings-inspired by the success of pre-transformer embedding mappings @xcite , and the potential of post-transformer alignments @xcite . these can be incorporated into the transformer architecture to provide auxiliary signals to enhance translation in two ways: i) introducing another multi-head attention between the input language embeddings and their mappings in the target language space-similar to the second multi-head attention block in the original transformer architecture @xcite . we propose to map embeddings between a source language l x and target language l y by computing a transformation matrix between paired representations of semantically-equivalent words or sentences, then to compute attention weights between these mapped embeddings, and concatenate these auxiliary attention outputs with the attention between tokens from l x and already-generated tokens from l y . ii) unmasking identified loanwords in the target language in the decoder's input, which is expected to provide further context to the decoder in the target language. this would replicate a uniquely human linguistic capability: the ability to pick up context in an unfamiliar language by picking out known words (i.e., loans from a known language). fig. 4 shows a proposed architecture for these operations. mapping between embedding spaces also allows expanding our method and dataset to new languages not covered by mbert or xlm through resources like indicbert @xcite .