Phonetic, Semantic, and Articulatory Features in A ssamese- B engali Cognate Detection (2022)

Abstract: In this paper, we propose a method to detect if words in two similar languages, Assamese and Bengali, are cognates. We mix phonetic, semantic, and articulatory features and use the cognate detection task to analyze the relative informational contribution of each type of feature to distinguish words in the two similar languages. In addition, since support for low-resourced languages like Assamese can be weak or nonexistent in some multilingual language models, we create a monolingual Assamese Transformer model and explore augmenting multilingual models with monolingual models using affine transformation techniques between vector spaces.

## introduction lexical cognates are words that are inherited by direct descent from a common etymological ancestor. due to sound change and semantic shift, cognates may or may not be easy to detect without rigorous application of the comparative method. for example, english "two" is cognate with armenian erku, as both are descended from proto-indo-european *dwóh 1 , with *dw->>tw-and *dw->>erkbeing regular, if non-intuitive, parallel sound changes. unlike loanwords, cognates are inherited and not borrowed, and are therefore necessarily subject to diachronic sound change. application of the comparative method to cognates can be used to discern the evolutionary paths of related languages, making them very useful for historical linguists, but first cognates must be distinguished from other classes of words like ordinary translations or words that simply sound similar. in this paper we focus on cognate detection between two closely-related languages: bengali (iso code bn) and assamese (iso code as). bengali (262 million speakers) and assamese (15 million speakers) are two languages of eastern india and bangladesh. they are both official languages of india (with most speakers located in the states of west bengal and assam, respectively), while bengali is also the national language of bangladesh. they share a common descent from early indo-aryan via magadhi prakrit, and are both typically written using bengali or eastern nagari script. the bengali-assamese languages (or gauda-kamarupa languages) is the subgrouping of eastern indo-aryan that contains both these languages and related dialects. they share certain grammatical features like classifying affixes (e.g., asm. -zon, beng. -dzon, referring to persons), as well as certain common phonetic innovations (such as the evolution of sanskrit /@/→/o/). despite the similarities, the two languages have some important differences, particularly in their sound patterns. assamese bengali s,s,z,z tc,tc h ,dý,dý h t,t h ,d,d h ú/t ",ú h /t " h ,ã/d ",ã h /d " h x,ô s,r therefore between these two languages, phonetic features, orthographic features, semantic features, or alignment of articulatory sequences may be more or less useful in determining cognate status, depending on the specific words in question. the word এক (/ek/), meaning "one" in both languages, is a clear case of common inheritance from sanskrit with the same sound changes applied; one need only look at the orthographic and phonetic forms to see this. but for assamese অকল (/okol/), meaning "only," the bengali cognate is actually একলা (/ekla/). in bengali, অকল is actually an arabic loan meaning "wisdom." in this paper we explore the contributions of phonetic, semantic, orthographic, and articulatory alignment features to the task of cognate detection between assamese and bengali. we use heuristic edit distance metrics, embedding vectors from various large multilingual language models (mlms), and neural networks to learn alignments between phonetic sequences. we also use an affine transformation technique to augment the embedding spaces of mlms with assamese-specific data. with combinations of features, we are able to achieve up to ∼94% f1 on cognate detection. our results also show that embeddings from a smaller monolingual bert variant can be mapped using affine transformations into the embedding space of larger multilingual models, which can improve both precision (up to 30%) and recall (up to 20%) in detecting assamese cognates in bengali. ## related work cognate detection has been approached from many angles in the nlp community. @xcite identifies cognates in algonquian using phonetic and semantic similarity. @xcite infer orthographic changes between cognates across languages. jäger (2018) evaluates pmi and svm-based methods in cognate detection over the automated similarity judgment project database @xcite . @xcite finds relationships between data size and genetic relatedness in automated cognate detection between english, german, dutch, and french. @xcite explore using global constraints in this task. @xcite explores sequence alignment and sound correspondence features in cognate detection in northern european languages; these are two of the feature types we also explore here. @xcite and @xcite explore the application of automated cognate detection methods to phylogenetic reconstruction and inference, and @xcite utilize wordnets to perform orthographic similarity-based cognate detection in various indian languages, but notably not assamese. @xcite and @xcite suggest that phonologically-aware articu-latory representations from panphon @xcite can either be used natively as embeddings or as features in attention-based neural models for downstream nlp tasks such as ner or entity linking for low-resource languages. labat and @xcite and @xcite suggest that adding semantic information to orthographic features works well for cognate detection in resource-rich languages like english and dutch (90% f1). similarly, @xcite suggests that adding large multilingual model embeddings to cognitive features like gaze improves cognate detection in low-resource languages like hindi and marathi (86% f1). work in translation lexicons (e.g., @xcite ) is also relevant here, for the hybrid approach to similarity metrics used. we combine multiple approaches which, to our knowledge, have never before been used all together. works such as @xcite and @xcite improve bilingual lexical induction using either linear or non-linear word embedding maps, but they use non-contextual embeddings like fasttext or word2vec. we extend such research to cognate detection using contextualized embeddings from transformer-based models to leverage additional monolingual representations in this task. ## datasets cognates in bengali and assamese must share a common descent from an ancestor language foot_1 ; the best-documented of these is sanskrit. however, many descendants of sanskrit make scholarly reborrowings from sanskrit (tatsama) that are fully reincorporated sanskrit forms adapted to fit the modern phonology. these exist alongside tadbhava words inherited from old indo-aryan with concomitant sound changes in the middle indo-aryan phase. for this data collection, we turned to wiktionary. namely, we scraped the categories of the form [descendant]_terms_derived_ from_sanskrit for each of the two descendants. 2 . we took the union of these two sets and then took the subset of the union where both the assamese and bengali forms had the same documented sanskrit ancestor. checking against com-mon ancestry filters out loanwords from the cognate datasets. ## methodology here we discuss the orthographic and phonetic features we extract from the data, our methods of assessing alignment between phonetic sequences, how we extract semantic similarity features from various language models, and how these different features combine in the cognate classification task. ## results and discussion we achieve 94% f1, 93% recall, and 95% precision when using all features. the alignment score feature provides the greatest single boost, and we find that adding semantic information to phonetic features provides as much additional performance as adding orthographic features, though specific false positives and negatives diverge significantly. fig. 5 shows positive precision, recall, and f1 for the neural network classifier using all features. we can also see that the classifier performs very slightly better using bengali as the baseline language than using assamese. similar results hold for other feature subsets: using the "bidirectional" all-languages model, feature sets ed-dl-m, ped-dl-ab-am, and ed-dl-b all show 94% f1(+) for bengali-assamese but 93% f1(+) for assamese-bengali. one possible reason for this is that bengali forms are on average somewhat more conservative, tending to preserve consonant clusters more than assamese, and in fact if we look at the false negatives for this result, we find many cases where one cognate has a consonant cluster and the other does not (see ## conclusion and future work we have presented here a method for detecting cognates between bengali and assamese that uses a mixture of phonetic, orthographic, articulatory alignment, and semantic features. the choice of these languages was motivated by their relatedness and the relative dearth of nlp work particularly on assamese, but we believe the methods presented herein are applicable to cognate detection and other types of heterogeneous similaritybased tasks on potentially any language pair. we found that our articulatory alignment score was by far the most informative feature. we also introduced a technique to map representations between embedding spaces and used it to introduce semantic features from a monolingual assamese model into four large multilingual models. adding semantic features to phonetic features alone is interesting on multiple levels-particularly using mapped instead of native embeddings. our ablation tests on different types of semantic representations suggest that i) linearly transforming vectors from one model's embedding space to another's carries certain semantic information with high fidelity, and ii) a model trained on a low-resource setting can be mapped to a richer model's space. if these hypotheses hold, transformed embeddings from a low-resourced lm can not only reduce the computational cost involved in training large multilingual language models but also improve downstream nlp tasks. nlp for minority languages may benefit from being able to detect cognates in better-resourced languages, both for computational historical linguistics, and for corpus building. for instance, other languages of assam (e.g., mishing, bodo) are not indo-aryan, but have loanwords cognate to indo-aryan words, alongside vocabulary cognate to other families, like sino-tibetan. our phonetic and alignment techniques may facilitate creating semantic models for these severely low-resourced languages unsupported by llms. collecting putative cognates is an essential step in most applications of computational historical linguistics, allowing finding regular sound correspondences (for which our alignment method could be adapted, e.g., by training individual attention weights over a sequence), identifying shared innovations, and reconstructing earlier word forms that could be used to reconstruct proto-languages a la @xcite and @xcite . the affine mapping technique we use warrants more exploration. not every affine map is a linear map, and other techniques like shear and rotation mapping may expose how simple a transformation can be used. other semantic techniques we wish to explore include pairwise scoring of cognate pair embeddings using a neural network. this has been shown to work well for coreference resolution and may be applicable for cognate detection. lastly, we would like to improve our monolingual assamese albert model and evaluate it on other downstream tasks like question answering.