Weakly-supervised Deep Cognate Detection Framework for Low-Resourced Languages Using Morphological Knowledge of Closely-Related Languages (2023)

Abstract: Exploiting cognates for transfer learning in under-resourced languages is an exciting opportunity for language understanding tasks, including unsupervised machine translation, named entity recognition and information retrieval. Previous approaches mainly focused on supervised cognate detection tasks based on orthographic, phonetic or state-of-the-art contextual language models, which under-perform for most under-resourced languages. This paper proposes a novel language-agnostic weakly-supervised deep cognate detection framework for under-resourced languages using morphological knowledge from closely related languages. We train an encoder to gain morphological knowledge of a language and transfer the knowledge to perform unsupervised and weakly-supervised cognate detection tasks with and without the pivot language for the closely-related languages. While unsupervised, it overcomes the need for hand-crafted annotation of cognates. We performed experiments on different published cognate detection datasets across language families and observed not only significant improvement over the state-of-the-art but also our method outperformed the state-of-the-art supervised and unsupervised methods. Our model can be extended to a wide range of languages from any language family as it overcomes the requirement of the annotation of the cognate pairs for training.

## introduction cognates are etymologically related word pairs across languages @xcite . however, cognates are defined in much broader terms in many different fields, including natural language processing (nlp) or psycholinguistics @xcite . in these areas, word pairs with similar meanings and spelling are also considered as cognates. in the recent development of automatic machine translation (amt), automatic cognate detection is found to be very effective for similar language translation @xcite . moreover, it helps to efficiently perform crosslingual information retrieval @xcite ) from different sources. very often, words that have similar spelling are recognised as cognates (example: the latin and the english word pair "cultūra" and "culture"). nevertheless, there are word pairs which are false friends or partial cognates @xcite . partial cognates are similar words across languages but carry different meanings in different contexts @xcite , thus making automatic cognate detection hard and challenging. identifying these cognates requires extensive linguistic knowledge across languages, which is quite hard and expensive to annotate. while cross-lingual automatic cognate detection systems exist, they have primarily been supervised methods requiring labelled data or language-specific linguistic rules. for under-resourced languages, finding annotators or linguists is a challenging task. this highlights the need for an efficient unsupervised languageagnostic cognate detection framework. we show that our weakly-supervised and unsupervised approaches can better exploit the available data than existing supervised methods and thus produce better results for under-resourced languages. the method transfers the morphological knowledge of a shared encoder in the unsupervised cognate detection framework into a siamese network setting, where the framework simultaneously learns word representation and cluster assignments in a selflearning setup. supervised cognate detection frameworks understand the relationship between word pairs by concentrating on their phonetic or lexical similar-ities based on their annotated positive or negative labels @xcite . recently, researchers tried to exploit contextual multilingual word embedding techniques to identify cognates which produced better results than only concentrating on phonetic transcriptions @xcite . although such methods had good results, annotating labels is quite expensive and tedious for many under-resourced languages. moreover, producing multilingual contextual word embeddings is challenging for these languages due to the need for more data sources on the web. @xcite highlighted that based on bilingual lexicon matching between two known languages, the similarity score produced by contextual word embeddings could differentiate between true or false cognate pairs. though this technique is label independent, the framework depends on the bilingual lexicon availability of the known language pairs. to alleviate the above challenges, in this paper, we propose a language-agnostic weakly-supervised cognate detection framework based on siamese architecture with an iterative clustering approach @xcite during back-propagation. our encoder design is inspired by @xcite , where they learn the n-gram character features of a sentence with attention. we introduce a positional encoder on n-gram features, which, in combination with the attention mechanism, learns sub-word representations of a word. we also depict the performance of our morphological knowledge-based weakly-supervised framework. this variant gives a better understanding of the grammar and structural analysis of the words of a language. thus, transferring this knowledge with the help of a shared encoder to closely-related languages enhances the understanding of structural and grammatical relatedness between cross-lingual word pairs. moreover, our word encoding method helps to produce better supervised cognate detection results, which outperform the state-of-the-art supervised results on various language pairs. in this paper, we have presented a complete set up of results for supervised, weakly-supervised and unsupervised setups. the extensive experiments (in section 6) on three different cognate detection datasets across language families have showcased the efficacy of our weakly-supervised and supervised cognate detection framework. for example, on six different indian language pairs, our weakly-supervised model (with morphological knowledge) has outperformed the state-of-the-art supervised model proposed by @xcite by an average of 9 points of f -score whereas, for celtic language pairs, it outperformed by 8.6 points of f -score. at the same time, our supervised framework has produced a state-of-the-art performance by outperforming the existing supervised model by an average of 16 points of f -score. thus, our model is robust across diverse language families for the supervised and weakly-supervised cognate detection task. interestingly, our experiments show that on indian language pairs like hindi-punjabi and hindi-marathi, an encoder with morphological knowledge of the hindi language performed better than an encoder with morphological knowledge of their ancestral language, sanskrit by an average of 1.5 points of f -score. however, the performance of the weakly-supervised framework for the language pair marathi-bengali has improved by 2 points of f -score. in a nutshell, our contributions are: ## related work algorithms for automatic cognate detection (acd) are mostly based on phonetic or orthographic similarity measures and often language-dependent or supervised approaches. @xcite developed an algorithm to align historical-comparative languages by their phonetic similarity. @xcite released a novel algorithm for aligning phonetically similar sequences. similarity-based al-gorithms follow these works to identify cognates between a language pair of different families. the orthographic similarity-based works calculate distances or string similarities between word pairs and define similarity scores to identify cognates @xcite @xcite . @xcite has released a convolutionbased model that also considers phonetic similaritybased scores into account to detect cognates for word pairs. some researchers have also taken parallel datasets into account to identify cognates. distance measurement based scores have become the feature set to identify cognates in these cases @xcite . @xcite performed a cognate detection task on indian languages, which includes a large amount of manual intervention during identification. @xcite introduced a character sequence-based recurrent neural network for identifying cognates between indian language pairs. the influence of classical machine learning and dynamic programming-based approaches defines automatic cognate detection tasks as semisupervised approaches. @xcite trained a linear svm based on word similarity and language-pair features to detect cognates. phonetic alignment based svm models performed quite efficiently on different language families while detecting cognates @xcite . some researchers designed orthographic substring similarity measures based svm models @xcite . another thread of research for cognate detection recently involved multilingual contextual knowledge injection methods. @xcite explored the effect of cross-lingual features on bilingual lexicon building, which was later implemented in indian language cognate detection methods. @xcite injected cross-lingual semantic features for cognate detection tasks using newly trained language models, which showed better results than state-of-the-art methods. recently, @xcite proposed incorporating gaze features in context aware cognate detection tasks, which improved results for hindi-marathi language pairs. these methods may produce better results; however, the main disadvantages lie in training these models as they are data-hungry models. thus, incorporating manually annotated gaze features and newly generated cross-lingual contextual embeddings becomes an impossible solution for many under-resourced languages. ## cognate detection framework in this section, we describe the components and working of the proposed language agnostic weakly-supervised cognate detection architecture using morphological knowledge, trained using an unsupervised loss function and iterative clustering method. the iterative clustering process enhances the understanding of word representation and cluster assignment. figure 1 depicts the shared word encoder based framework. it consists of two parts -(i) a morphology learner, which gathers the morphological knowledge of a language and (ii) an unsupervised cognate detector, which uses the morphological knowledge learnt using a shared word encoder to cluster the cognates between word pairs. ## training details we implemented our model using pytorch foot_0 . the learning rate for the indian, celtic and south african datasets was hand-tuned to 1e-4, 2e-3 and 4e-3, respectively, for the morphological training. at the same time, for the unsupervised cognate detection tasks the learning rates were 1e-2,1e-1,1e-2, respectively. to stabilize the learning of the model, we have implemented lambdalr foot_1 as the learning rate scheduler. for clustering, we have used k-means clustering (macqueen, 1967) with minibatch foot_2 . ## experimental evaluation we evaluated our framework on three different datasets in three different scenarios: (a) language pairs with pivot language and its morphological knowledge, (b) language pairs without the pivot language but with the shared encoder having morphological knowledge of the pivot language and (c) the effect of the historical language morphological knowledge transfer on the language pairs. we compare our models with the following supervised state-of-the-art cognate detection frameworks: (i) cnn based model siamese cnn based approach by @xcite , (ii) orthographic similarity based approach from labat and lefever ## ablation study effect of morphological training datasize. one of the challenges of morphological knowledge transfer is the efficient learning of word structure in the presence of a few morphological training datasets. as irish has few training datasets, we have evaluated the proposed framework on different samples of the irish-manx dataset (refer to section 6). we have down-sampled and up-sampled the morphology training set to 30% compared to the existing datasets. from figure 2 , we can observe that the best f -score 0.79 is achieved when the morphological learner model is trained with 30% more data size than the original size. this emphasizes our claim of the model's efficacy even with a slight increase in the morphological training datasets, which opens up the opportunity of implementing the weakly-supervised cognate detection framework on diverse under-resourced languages. ## conclusion this paper proposed a novel language agnostic weakly-supervised cognate detection framework based on siamese architecture. experiments on three different datasets consisting of indian languages, celtic languages and south-african languages showcase the efficacy of our framework in understanding the structural relations between cross-lingual words across languages. we also show that transferring morphological knowledge to closely-related word pairs with the help of a shared encoder improves the model's efficacy in different scenarios. our study on knowledge transfer from historical languages depicts changes in the word structures of modern languages. we demonstrate that our approach outperforms the existing supervised and semi-supervised frameworks and establishes state-of-the-art results for the cognate detection task. we also showcase the stability of learning morphology on a small number of training datasets, which opens up the possibility of deploying the system across language families. our future work will design and compare a semisupervised framework based on labelled and unlabelled training sets. we will study whether the semi-supervised framework improves efficiency while detecting the cognates.