R o B o C o P : A Comprehensive RO mance BO rrowing CO gnate Package and Benchmark for Multilingual Cognate Identification (2023)

Abstract: The identification of cognates is a fundamental process in historical linguistics, on which any further research is based. Even though there are several cognate databases for Romance languages, they are rather scattered, incomplete, noisy, contain unreliable information, or have uncertain availability. In this paper we introduce a comprehensive database of Romance cognates and borrowings based on the etymological information provided by the dictionaries. We extract pairs of cognates between any two Romance languages by parsing electronic dictionaries of Romanian, Italian, Spanish, Portuguese and French. Based on this resource, we propose a strong benchmark for the automatic detection of cognates, by applying machine learning and deep learning based methods on any two pairs of Romance languages. We find that automatic identification of cognates is possible with accuracy averaging around 94% for the more difficult task formulations.

## introduction and related work cognates detection and discrimination, as both the foundation of historical linguistics @xcite and the starting point in historical investigation @xcite , open windows on numerous areas of social sciences. the immediate implications of the accurate identification of cognate chains can be found in linguistic phylogeny @xcite @xcite , allowing to trace back language relatedness @xcite as well as linguistic contact @xcite , and offering important clues concerning the geographical and chronological dimension of ancient communities @xcite . the cognate chains are the foundation of the "comparative grammar-reconstruction" method @xcite , and the etymological data thus obtained can be used as a source on human prehistory, corroborating the archaeological inventory @xcite , and providing the basis for 'linguistic paleontology' or 'socio-cultural reconstruction' @xcite ). an extensive perspective on cognate chains can serve as a basis in the detection of meaning divergence, especially when searching for common patterns that govern the cognitive mechanisms activated in semantic change @xcite . the lexicon still offers significant clues for building a 'universal' cognitive network derived from meaning shifts, easily observable in cognate sets, which would be essential in developing a comprehensive theory on cognition and neuropsychology @xcite . at the same time, an integrated view on the cognate pairs between any two related languages would allow taking steps forward in the study of language acquisition @xcite , as well as in the difficult task of eliminating false friends in automatic translation (uban and @xcite . training the machine towards an accurate detection of cognates becomes a necessity in today's large amount of linguistic data that still hasn't been processed from a historical point of view @xcite . since the foundation of the "comparative grammar-reconstruction" method over two centuries ago, linguistic phylogeny is still mainly investigated by means of manual comparison of cognate sets, which implies the extraction of systematic phonetic correspondences between words in language networks, and eventually allows the reconstruction of the protolanguage. although successfully used by classical linguists, this method is highly counter-economic, which has led to a sustained search for computational methods able to assist the process. the increasing interest in automatic methods for cognate detection calls for a directly proportional need for reliable databases of positive examples, consisting of lists of cognate sets as long as possible. these lists are not easily attainable, even if we deal with well-known languages, with well-developed electronic resources, like the romance languages 1 . in order to obtain longer lists of cognate sets, the definition of cognates has broadened its limits, including: a) words sharing a similar form and meaning, regardless of their etymology @xcite @xcite , also referred to as 'true friends' @xcite (e.g. eng. famine and fr. famine, although the first one is borrowed from the second); b) words etymologically related, regardless of the type of relation @xcite (e.g. eng. family, borrowed from middle french famile, in its turn borrowed from lat. familia, and rom. femeie 'woman', inherited from lat. familia); c) words that are similar in their orthographic or phonetic form and are possible translations of each other @xcite (e.g. eng. sprint and japanese supurinto; see above, @xcite ). besides these interpretations, there were also attempts to define cognates by establishing unnecessary limits: words that share a common origin and have the same english translation @xcite . following this acceptation, ro pleca 'leave' and es llegar 'arrive' are not to be identified as cognates, although they are both inherited from lat. plicare 'to fold'; such narrowing disregards the possible benefits of a comparative perspective on cognates in the analysis of semantic divergence, one of the most understudied and promising fields in historical linguistics. the simplest definitions of cognates, as "words sharing a common proto-word" @xcite or "words that share a common etymological origin" @xcite are likely to be the most effective in computational linguistics, although certain improvements can be made. in this paper, we use the following definition: two words are cognates if and only if the intersection of the sets of their etymons is not void. the urge for a wide (if not exhaustive) database of the romance lexicon derives both from internal and external needs. on the one hand, the high number of parallel chains of cognates would allow the romance linguists to revisit the issue of sound laws, which, although apparently well-known, still raises questions about features whose correspondence is not regular and, therefore, is not easily explainable (e.g. lat signum / signa, ro semn, es seña, vs lat sifflare, ro sufla, es chillar, where the latter form is considered to be irregular, although this phonetic evolution is not limited to an isolated number of instances). moreover, in certain cases where there aren't enough data, linguists still argue whether a phonetic change represents the rule or, on the contrary, it's the apparent exceptions that make the rule (e.g. lat @xmath0, but lat flore-> es flor: at the moment, both theories are based on an even number of examples). additionally, a complete diagram of the possible phonetic shifts would authorize etymologists to bring back into discussion long-standing etymological cruxes. as for the external needs, we postulate that the algorithms identified by training the machine on one of the best studied language families could be further successfully applied to other languages which are less known or have scarce resources. in terms of automatic approaches for cognate detection, the last decades bring a plethora of such methods @xcite @xcite @xcite . most methods proposed in previous studies include linguistic features and different orthographic and phonetic alignment methods in combination with shallow supervised machine learning models (such as svms) or clustering methods @xcite @xcite @xcite @xcite @xcite . a few studies employ deep learning for cognate detection or related tasks. rama (2016) use siamese convolutional neural networks (cnns) with character and phonetical features complemented with additional linguistic features in order to detect cognates in languages across three language families, with the majority of examples belonging to austronesian languages, with up to 85% accuracy. @xcite use language models including a recurrent neural network architecture for lexical borrowing detection. transformers were used in @xcite for predicting cognate reflexes. to the best of our knowledge, no previous studies have used transformer architectures specifically for cognate detection. previous results on romance cognate detection in particular are reported in (ciobanu and @xcite , in which cognates are automatically distinguished from non-cognate translation pairs, based on a smaller dataset of cognate pairs in the romance languages, with accuracies reaching 87% using an svm with alignment features. starting with these remarks, our main contributions are: the rest of the paper is organized as follows: in sections 2 and 3 we present the database which we have created and offer details about the processing steps involved, in section 4 we introduce our approach for the automatic detection of cognates. methodological details are discussed in section 4.1, and an extensive error and results analyses is presented in section 4.2. the last section is dedicated to final remarks. ## dataset even though there are several cognate databases for romance languages, they are incomplete (as the inventory of romance lexemes based on the swadesh list @xcite , cf. @xcite ), noisy (because of the lack of expert proofing, these being usually obtained with the help of volunteers, like wikipedia @xcite , built with automated translation methods @xcite , or are of uncertain availability (cf. @xcite ).to overcome as much as possible these weaknesses, we have decided to build from scratch a fully available database of romance cognates, for the main five romance languages (italian -it, spanish -es, french -fr, portuguese -pt and romanian -ro), starting with the available machine-readable reference dictionaries foot_2 , which contain etymological information. the process was semi-automated, guided and verified by human experts, to ensure the quality and coverage of the data. our strategy was to parse one by one all the dictionaries, to extract for each language exhaustive information related to every word and its relevant etymology features (namely its etymon(s), the source language(s), the part(s) of speech), and then to aggregate all this information in order to build a cognate database for all five romance languages (from now on called robocop -romance borrowings cognates package) (see section 2.2). since each of the five dictionaries had its own editorial choice of presenting the information, the preprocessing, the parsing and the postprocessing strategies had to be customized for each language, which implied a lot of expert and computational effort. the process was very specific to each dictionary and included a cyclical process similar to methodologies used in web scraping -running scripts implementing rule-based algorithms (such as regular expressions) to separate noise from the data for each dictionary and manual evaluation of each output with the assistance of linguists in our team, followed by potential refinement of the code to manage all exceptions. due to the lack of space, we cannot present in detail all the challenges of building the robocop database, but we only discuss some of the most common difficulties. addressing them all was a repetitive feedback process involving linguists and computer scientists. ## quantitative aspects of the database we list here some quantitative aspects of the database. the database comprises a total of 125,598 words across all languages and 90,853 cognate pairs. we find the following accuracies for extracting etymologies for each language (the average accuracy is 98.6%): spanish: 100%, romanian: 98%, portuguese 97%, italian 100%, french 98%. for cognates extraction we find the following accuracies for each language pair as seen in as reflected by the quantitative aspects provided in this section, we created both an effective resource for romance cognate pairs, and a comprehensive map of borrowings and etymologies for the romance languages. robocop is, to our best knowledge, one of the most high-coverage, reliable and complex databases of romance cognates. ## automatic cognate detection experiments during the last decades, several computational approaches to the automatic detection of cognate pairs reported fairly good results. the main problem when it comes to evaluating them is that, almost always, the results are not directly comparable across studies. this is not only due to the application of different methodologies, but, most of the time, due to the use of different databases. we address this issue by proposing a comprehensive, reliable database of romance cognates. in the following, we will present a series of experiments and results to further help the evaluation process of automatic cognate pairs detection, providing a benchmark for future approaches. ## conclusions and future work we introduced a comprehensive database (in graphic and phonetic form) and framework for the automatic analysis and detection of romance cognates (the largest database of this kind, in our best knowledge, with 125,598 words across all languages and 90,583 cognate pairs). our framework is the result of collaboration between computer scientists and linguists and includes: a linguistically informed and computationally usable definition of cognate words, a methodology for extracting cognate pairs automatically in a robust way, a comprehensive dataset of word etymologies for romance languages based on etymological information given by dictionaries, and a comprehensive database of cognate pairs, as well as benchmark results for automatic cognate detection, based on a series of machine learning experiments (using a variety of features and models: graphical and phonetical features, including prior feature engineering to obtain word alignment information, or alignment-agnostic, and several types of model architectures) for automatically detecting cognates. for the most difficult task (cognate detection for levenshtein-based negative sampling) we obtained an average accuracy around 94%. in future work we intend to distinguish virtual cognates in the database and to complement experiments with discrimination between virtual and true cognates. furthermore, we aim to investigate the discrimination between cognates and borrowings, also adding semantic features, as well as more phonetic features for each pair of romance languages.