Verba volant, scripta volant? Don’t worry! There are computational solutions for protoword reconstruction (2024)

Abstract: We introduce a new database of cognate words and etymons for the five main Romance languages, the most comprehensive one to date. We propose a strong benchmark for the automatic reconstruction of protowords for Romance languages, by applying a set of machine learning models and features on these data. The best results reach 90% accuracy in predicting the protoword of a given cognate set, surpassing existing state-of-the-art results for this task and showing that computational methods can be very useful in assisting linguists with protoword reconstruction.

## introduction and related work protoword reconstruction, consisting of recreating the words in a proto-language from their descendants in daughter languages, is central to the study of language evolution. as the foundation of historical linguistics @xcite and the basis for linguistic phylogeny @xcite @xcite , protoword reconstruction offers important pieces of information concerning the geographical and chronological dimensions of ancient communities @xcite , at the same time, allowing an insight into the cognitive and cultural world of our ancestors. the traditional process of reconstructing ancient languages consists of the "comparative grammar-reconstruction" method @xcite , and the etymological data thus obtained can be used as a source on human prehistory, corroborating the archaeological inventory @xcite , and providing the basis for 'linguistic paleontology' @xcite . the reconstruction of a word automatically implies a reconstruction of the surrounding realities, both natural and socio-cultural. for example, the presence in different indo-european languages of obviously related words for 'beech' or 'salmon' allowed the reconstruction of words from proto-indo-european and thus information about the elements of nature present in the immediate vicinity of the indo-europeans could be extracted. in the absence of any clear documentary or archaeological data, these lexical clues allowed the geographical identification of the indo-european homeland, also facilitating the chronology of successive waves of separation of indo-european languages from the common trunk. in the case of romance languages, although the mother tongue -latin -is attested, its presence in written texts is not an exhaustive source for linguistic, social, and historical analysis of the community that spoke it. it is now generally accepted that the spoken language represented a different diastatic, diaphasic, and diamesic variety from written language, used by the few educated people who decided to express themselves in writing @xcite . the latin language that we reconstruct from words inherited in romance languages is thus the only concrete and reliable living variety of the language from which romance languages originate, whether we call it oral/ vulgar latin or proto-romance. we will opt here for the name "proto-romance" when we refer to the language from which the romance languages originate, as this corresponds to the concept of protolanguage and protoword @xcite . furthermore, there are still numerous clearly cognate words present in several romance languages, whose etymon is not attested in latin (nor in any other language from which it might have been borrowed). for example, in the case of it. trovare 'find', fr. trouver, cat. trobar, etymologists have hotly debated over the decades whether one should reconstruct the protoform *tropare or *turbare @xcite . a series of cognates attested in all romance geograph-ical areas, like rom. încȃ 'moreover', it. anche, old fr. anc, cat. anc etc., has triggered over 15 etymological hypotheses over the last century, still without a generally accepted solution. although etymologists' interest in reconstructing the protolanguages has risen over the years, they still encounter numerous gaps when using exclusively the classical, manual methods @xcite . as the task of protoword reconstruction plays an important role in historical linguistics, studies have gone beyond the comparative method in an attempt to automate the process @xcite @xcite . however, the task has been recognized as difficult and challenging. computational protoword reconstruction is a fairly new direction of study, and consequently even state of the art approaches have limitations. complete automation of the reconstruction process is still a desideratum. @xcite proposed two systems (jakarta and prague) that, combined, cover the steps of the comparative method for protolanguage reconstruction, and several other approaches to reconstruct protowords computationally had been attempted previously @xcite @xcite . the work of computational biologists such as alexandre bouchard-côté, russell gray, robert mcmahon, and mark pagel, and coauthors took the protoword reconstruction one step further by applying methods from computational biology to the problem of the reconstruction of language history, often in collaboration with linguists @xcite @xcite . in recent years, researchers have introduced new methods for protoword reconstruction, based on modern computational techniques (for example, crf, transformers, rnn, deep learning) (ciobanu and @xcite @xcite @xcite @xcite . the computational methods are limited today by 1) the available data (sparse, inconsistent) and 2) by the insufficiency of linguistic knowledge embedded in the systems. the latest computational results on romance protoword reconstruction, in particular, are reported on the database of @xcite , which contains 8,799 cognates set in latin, italian, spanish, portuguese, french, and romanian (not all full cognates set). this is a revision of the dataset of @xcite (used with very good results in ( ciobanu and dinu, 2018) ) with the addition of cognates scraped from wiktionary. starting with these remarks, our main contributions are: the rest of the paper is organized as follows: in section 2 we present the database that we have created and offer details about the processing steps involved; in section 3 we introduce our approach for the automatic protoword reconstruction, along with methodological details; the results of our proposed experiments are fleshed out in section 4; and a comprehensive error analysis is described in section 5. the last section is dedicated to final remarks. ## data a major inconvenience in historical linguistics in general, and in computational approaches of protoword reconstruction in particular is the scarcity of available data. nonetheless, in the last few years, several initiatives have been undertaken in this direction. (ciobanu and dinu, 2018) developed a database of latin protowords, further expanded by @xcite with wiktionary data. recently, this dataset was extensively used for several studies @xcite @xcite . in 2023, @xcite published the most comprehensive database of romance related words, named robo-cop. it contains cognates and etymons in five romance languages: italian, spanish, portuguese, romanian, and french. it has already been used with good results on prominent historical linguistic tasks such as cognate identification @xcite , cognate-borrowings discrimination @xcite rection @xcite . ## results the previously described methods have been applied on both protorom and protorom-all5 datasets, using the orthographical form of the cognates and latin etymon, or alternatively the autogenerated phonemic representations (where the models were able to accommodate them). we also provide a comprehensive human evaluation of the results. linguists from our team manually analyzed the entire list of results, and we present the most significant observations regarding the models' successes and failures. the linguists did not correct the protoforms proposed by the models, but only evaluated and commented on them in relation to current knowledge in the field of historical linguistics. the metrics used include accuracy, (normalized) edit distance, and cov i , with @xmath0, 5, 10}, which stands for an extended version of the accuracy metric, where a correct prediction is one where the model found the correct etymon within the first i etymons predicted by our method (this metric is computed for models that are able to output a ranked list of predictions -flan-t5 and crf-based models). ## error analysis this section is dedicated to a deeper dive into qualitatively quantifying the errors produced by the previously proposed models. our objective is separating purely wrong predictions from "near misses", which may still provide value for linguists for the reasons discussed below. the error analysis was manually conducted by the linguists from our team, who specialize in romance languages. they did not modify the protoforms provided by the models in any way. their only intervention was to distinguish forms that were genuinely erroneous from those whose differences from the dictionary form were either insignificant or represented a correct adjustment to the reality of latin pronunciation. in the final quantitative analysis, forms in this category were therefore included in the list of correct predictions without any changes to their structure. through analyzing the errors, we have identified some patterns that typically reflect either an insufficient number of examples to support a particular phonetic change or the irregularity of the change itself. for example, the short tonic /u/ develops into spanish /o/ in half of the cases, while it remains /u/ in the other half. in such scenarios, the model may not know which phonetic treatment the cognates underwent and might choose the wrong variant. similarly, in cases of phonetic accidents, which are by nature irregular and unpredictable, the model cannot reconstruct the pre-accident form. instead, it reconstructs the intermediate form between the classical word and its romance descendants. identifying and systematizing these errors can help improve future results by broadening the input with information related to sound changes. before analysing the errors, a few preliminary points should be made. romance lexicography as a whole is graphocentric -it considers the written, classical latin (cl) lexical variants as the basis for the romance vocabulary, even though it goes without saying that vernacular languages, oral par excellence, developed from an oral language, in our case proto-romance (pr) @xcite . in the latest methodology used in romance etymology, developed within the dérom project @xcite , the etymological identification is based strictly on the comparative grammar -reconstruction method, starting from the lexical forms that were used uninterruptedly in romance languages. the lexemes attested in classical latin are only a written correlate, possibly further evidence of the existence of the form obtained by the methods of comparative historical linguistics. in the light of these considerations, we find that some of the reconstructed variants classified as errors should actually be considered as positive results and evidence that the machine could work at the same level as a linguist applying traditional methods. by positive results instead of errors we mean cases -not a few -where the machine reconstructed exactly the phonetic form valid for oral latin, at the expenses of the standard orthographical form as it is lemmatized in classical latin dictionaries. cases where the word obtained and the one given by the dictionary did not completely match were automatically considered as errors, although sometimes it was not a mistake as such. therefore, there are a number of protoforms which, although they appear in the list as inadvertences, are variants that should be taken into account with full attention by linguists. some are no more wrong than the form in the dictionary, some are closer to the actual oral form than those provided by lexicographers, while some are exactly the form that historical linguists would have reconstructed using traditional meth-ods based on the sound laws of each language (we discuss each case below). therefore, protoforms obtained by the automatic methods proposed here are sometimes preferable to the lemmatized ones, and this is the most important thing we can expect from the machine. below, we provide a list of situations categorized as errors, but where the the automatic protoword reconstruction is either comparable or better than the version proposed by the dictionary, as it represents exactly the linguistic variant we should consider as intermediate between classical latin and romance languages. the real errors in the experiments we developed stem primarily from lexicographic omissions or mistakes, as well as in the imprecise methodology employed by the ibero-romance dictionaries consulted, namely the lack of any distinction between inherited and borrowed latin words @xcite . this latter inaccuracy leads to a misinterpretation of the phonetic correspondences by the computer, given that only the inherited words, not the borrowed ones, underwent regular sound change. therefore, if we put together ro. roatȃ, sp. rueda, pt. roda, with ro. rotat , ie, sp. rotacion, pt. rotação, the computer will not be able to correctly infer the correspondence t/d/d and will confuse it with t/t/t, also assuming the series d/d/d. therefore, some reconstructions, especially in the case of words circumscribed only to ibero-romance languages, could not take this sound law into account (e.g., on the basis of sp. miedo, pt. medo, the computer could not reconstruct metus, but proposed medus, which is wrong). this kind of shortcomings will be easily overcome in the future, firstly by clearly establishing, in the pro-torom database, the inheritance-borrowing distinction, and secondly by extending the input provided to the computer with a number of basic phonetic laws. revised performance scores. looking at the best reported predictions, we can apply the linguistic observations stated in the previous section and count which wrong predictions can be actually considered acceptable errors. thus using these recovered predictions, the best models' scores would change as follows: ## conclusion in this paper, we built a new dataset for automatic protoword reconstruction, consisting of 19, 222 cognate sets from five romance languages (romanian, italian, spanish, portuguese, french). this is to date the largest database of its kind, surpassing its predecessor which totals 8, 799 cognate sets. we also proposed a series of comprehensive benchmarks ranging from deep-learning approaches, using llms and transformer-based architectures, to more classical algorithms such as crfs, some of which achieved performances of more than 85% accuracy when allowing multiple generated reconstructions. an in-depth linguistic analysis of the erroneous reconstructions was also performed using the predictions of the best performing models. this attempt shed some light on the various categories of mistakes, out of which several could be considered acceptable. when ignoring the aforementioned acceptable errors, we were able to surpass 90% accuracies. we consider this an important distinction, since in our view similar tools should aim at assisting linguists in their scientific endeavours. raw metrics are useful to compare computational methods, but, in order to assess their usability, a more qualitative inspection of the results should be performed. we hope through our research to incentivize further analysis. as for future work, we are looking into an additional refinement of the current cognate sets, but also extending the database with more examples, including properly validated monolingual latin reflexes that were excluded from our experiments for robustness sake. we also intend to expand past the proposed benchmarks with more novel approaches, relying on both the proposed dataset and the additional contents of its parent database, robocop.