Caveats of Measuring Semantic Change of Cognates and Borrowings using Multilingual Word Embeddings (2022)

Abstract: Cognates and borrowings carry different aspects of etymological evolution. In this work, we study semantic change of such items using multilingual word embeddings, both static and contextualised. We underline caveats identified while building and evaluating these embeddings. We release both said embeddings and a newly-built historical words lexicon, containing typed relations between words of varied Romance languages.

## introduction languages are in constant evolution over time; words appear, disappear, and their syntactic form and semantic function evolve @xcite . however, languages evolutions can be closely inter-related, following phenomena of interactions and inheritance. cognates and borrowings, which are the targets of our study, are direct consequences of these phenomena. cognates are words which descend from the same ancestor word (their proto-form) belonging to a shared common direct parent language. for example, the french word chat 'cat' is cognate with spanish gatto and romanian cȃtus , ȃ, as they all descend from latin cattus 'cat', a direct ancestor of these three languages. when a word is an evolution of a form which does not come from a direct ancestor, it is called a borrowing. english cat also comes from cattus, foot_0 but as latin is not a direct ancestor of english, it is therefore a borrowing of english to latin. we consider the relation between cat and chat to be of 'borrowing' type by extension. borrowings mostly occur to designate 'realities that were unknown before the adopting speech community got in contact with the "giving" culture and its language' or to replace already existing meanings by the word of the related dominant culture @xcite . to study semantic variation, we look at our words' glosses, which are expressions of their meaning, here as their english translations or definitions. in our previous example, while the french and spanish cognates both retained the original sense 'cat', the romanian cognate went through a semantic change and is translated as 'handcuff'. semantic change studies historically relied on specific word relations, cognates and 'borrowings' @xcite , found through the comparative method (formalised by @xcite ). the last few years have seen the emergence of new tools such as contextualised embeddings to study semantic variation @xcite , enabling the comparison of word senses across domains, periods and languages. we join both approaches and expand on the work of @xcite , who use 'static' (non-contextualised) embeddings to study semantics of cognates and borrowings in contemporary romance languages and english. in this work, we use static as well as contextualised embedding to study the semantic evolution of cognates and borrowings, for both contemporary and older romance languages, as well as english. to this end, we first create a dataset of cognates and borrowings from the widely studied romance family (contemporary: spanish, french, italian, portuguese, romanian, old: latin, old spanish, middle french), to which we add english. 2 then, we compare several methods to tackle the issue of obtaining, for low-resource historical languages, embeddings spaces aligned with the ones of contemporary languages. both dataset and embeddings are released with the paper. 3 lastly, we use these embeddings to study semantic shift for both diachronic (between parent and child) and synchronic (between children) cognates or borrowing relations, and find that contextualised embeddings allow us to reach more accurate conclusions. at each step, we highlight the possible pitfalls. ## related works cognates and borrowings transcribe different aspects of their languages history, and are often studied through the lens of orthographic @xcite or phonetic combined with semantic variation @xcite . uban et al. (2021), which we extend, study semantic variation in modern romance languages between cognates and borrowings by considering their modern-day embeddings as a 'snapshot in time' of their meaning. as their dataset is not available, we can not use as a benchmark; however, like several public etymological databases, among which cognet @xcite , containing cognates and borrowings without differentiating between both relation types, and etymdb2 (fourrier and sagot, 2020), too small for our needs but which differentiate between both types, we build a dataset using the wiktionary 4 as etymological source. semantic change across languages is actively researched in the linguistic and sociology research communities @xcite , as it offers valuable information for sociological and historical analysis. in the nlp domain, many authors apply diachronic embeddings models to more than one language @xcite , but without considering their interactions. some work studies variations between languages or dialects, diachronically @xcite or synchronically @xcite . however, although several annotated datasets are available to evaluate diachronic semantic change detection methods @xcite , crosslingual semantic change does not have such resource and cognates and borrowings seem like a promising proxy for evaluating these methods. ## datasets and corpora construction we create a dataset of cognates and borrowings in all languages under study. to complement it, we need corpora in each language to train or extract word embeddings; such corpora are publicly available for highly studied languages. we use a sample of the oscar corpus (ortiz suarez et al., 2019; 4 the wiktionary is a user-built free multilingual dictionary, found at en.wiktionary.org @xcite for contemporary languages and latin. for middle french and old spanish, we use less well-known resources. ## cross-lingual embeddings we compare the semantic function of words in cognates and borrowings pairs. to this end, we explore various ways of obtaining aligned word embeddings in all languages (multilingual embeddings), using static and contextualised embeddings. the former are trained using fasttext @xcite and aligned a posteriori, while the latter are extracted using the multilingual language model mbert @xcite from corpora in all the languages under study. trained embeddings and language models can be found for all our contemporary languages. however, historical languages such as middle french and old spanish suffer from a scarcity of resources that we have to address. ## results our metric is cosine similarity, commonly used in semantic change detection @xcite . our results are summarised in ## conclusion in this work, we create a cognate and borrowing dataset for english and romance languages from different periods, as well as two aligned embeddings sets for all languages. when assessing embedding quality and alignment, we show that fast-text embeddings, even when already pre-trained and aligned, are poorer than the mbert ones on all respects. we therefore use the latter to study semantic change between cognates and borrowings: as expected, un-shifted word pairs are on average more similar than shifted ones. furthermore, we observe a trend between cognates and borrowings, the latter being seemingly more similar than the former. further analysis would be needed to determine whether this difference can be confirmed, by looking at chosen cognate and borrowings of similar histories in more languages. in summary, properly designed embeddings can be used to support historical lexicographic studies, while well-understood phenomena underlying cognates and borrowings can help design and evaluate cross-lingual word embeddings. piotr bojanowski, edouard grave, armand joulin, and tomas mikolov. 2016. enriching word vectors with subword information. arxiv preprint arxiv:1607.04606. tom bosc and pascal vincent. 2018. auto-encoding dictionary definitions into consistent word embeddings. in proceedings of the 2018 conference on empirical methods in natural language processing, pages 1522-1532, brussels, belgium. association for computational linguistics. alina maria ciobanu and liviu p. dinu. 2015. automatic discrimination between cognates and borrowings. in proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 2: short papers), pages 431-437, beijing, china. association for computational linguistics. alina maria ciobanu and liviu p. dinu. 2019. automatic identification and production of related words for historical linguistics. computational linguistics, 45(4):667-704. alexis conneau, guillaume lample, marc'aurelio ranzato, ludovic denoyer, and hervé jégou. 2017. word translation without parallel data. arxiv preprint arxiv:1710.04087. jacob devlin, ming-wei chang, kenton lee, and kristina toutanova. 2019. bert: pre-training of deep bidirectional transformers for language understanding. in proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171-4186, minneapolis, minnesota. association for computational linguistics. philip durkin. 2015. etymology. clémentine fourrier and benoît sagot. 2020. methodological aspects of developing and managing an etymological lexical resource: introducing etymdb-2.0. in proceedings of the 12th language resources and evaluation conference, pages 3207-3216, marseille, france. european language resources association. oana frunza and diana inkpen. 2009. identification and disambiguation of cognates, false friends, and partial cognates using machine learning techniques. international journal of linguistics, 1(1):1-37. edouard grave, piotr bojanowski, prakhar gupta, armand joulin, and tomas mikolov. 2018. learning word vectors for 157 languages. in proceedings of the eleventh international conference on language resources and evaluation (lrec 2018), miyazaki, japan. european language resources association (elra). william l. hamilton, jure leskovec, and dan jurafsky. 2016. diachronic word embeddings reveal statistical laws of semantic change. in proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1489-1501. association for computational linguistics. dirk hovy and christoph purschke. 2018. capturing regional variation with distributed place representations and geographic retrofitting. in proceedings of the 2018 conference on empirical methods in natural language processing, pages 4383-4394, brussels, belgium. association for computational linguistics. grzegorz kondrak. 2001. identifying cognates by phonetic and semantic similarity. in second meeting of the north american chapter of the association for computational linguistics. thomas krefeld. 2013. cognitive ease and lexical borrowing: the recategorization of body parts in romance. in cognitive ease and lexical borrowing: the recategorization of body parts in romance, pages 259-278. de gruyter mouton. andrey kutuzov, lilja øvrelid, terrence szymanski, and erik velldal. 2018. diachronic word embeddings and semantic shifts: a survey. in proceedings of the 27th international conference on computational linguistics, pages 1384-1397, santa fe, new mexico, usa. association for computational linguistics. guillaume lample, alexis conneau, ludovic denoyer, and marc'aurelio ranzato. 2017. unsupervised machine translation using monolingual corpora only. arxiv preprint arxiv:1711.00043. matej martinc, petra kralj novak, and senja pollak. 2020. leveraging contextual embeddings for detecting diachronic semantic shift. in proceedings of the 12th language resources and evaluation conference, pages 4811-4819, marseille, france. european language resources association. syrielle montariol and alexandre allauzen. 2021. measure and evaluation of semantic divergence across two languages. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 1247-1258, online. association for computational linguistics. benjamin muller, antonios anastasopoulos, benoît sagot, and djamé seddah. 2021. when being unseen from mbert is just the beginning: handling new languages with multilingual language models. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 448-462, online. association for computational linguistics. pedro javier ortiz suarez, benoit sagot, and laurent romary. 2019. asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. proceedings of the workshop on challenges in the management of large corpora (cmlc-7) 2019. cardiff, 22nd july 2019, pages 9 -16, mannheim. leibniz-institut f"ur deutsche sprache. hermann osthoff and karl brugmann. 1878. morphologische untersuchungen auf dem gebiete der indogermanischen sprachen. hirzel, leipzig, germany. dominik schlechtweg, barbara mcgillivray, simon hengchen, haim dubossarsky, and nina tahmasebi. 2020. semeval-2020 task 1: unsupervised lexical semantic change detection. in proceedings of the fourteenth workshop on semantic evaluation, pages 1-23, barcelona (online). international committee for computational linguistics. ana sabina uban, alina maria cristea, anca dinu, liviu p. dinu, simona georgescu, and laurentiu zoicas. 2021. tracking semantic change in cognate sets for english and romance languages. in proceedings of the 2nd international workshop on computational approaches to historical language change 2021, pages 64-74, online. association for computational linguistics. thomas wolf, lysandre debut, victor sanh, julien chaumond, clement delangue, anthony moi, pierric cistac, tim rault, remi louf, morgan funtowicz, joe davison, sam shleifer, patrick von platen, clara ma, yacine jernite, julien plu, canwen xu, teven le scao, sylvain gugger, mariama drame, quentin lhoest, and alexander rush. 2020. transformers: state-of-the-art natural language processing. in proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38-45, online. association for computational linguistics.