Monolingual Adapter Networks for Efficient Cross-Lingual Alignment (2025)

Abstract: Multilingual alignment for low-resource languages is a challenge for embedding models. The scarcity of parallel datasets in addition to rich morphological diversity in languages adds to the complexity of training multilingual embedding models. To aid in the development of multilingual models for under-represented languages such as Sanskrit, we introduce GitaDB: a collection of 640 Sanskrit verses translated in 5 Indic languages and English. We benchmarked various state-of-the-art embedding models on our dataset in different bilingual and cross-lingual semantic retrieval tasks of increasing complexity and found a steep degradation in retrieval scores. We found a wide margin in the retrieval performance between English and Sanskrit targets. To bridge this gap, we introduce Monolingual Adapter Networks: a parameter-efficient method to bolster cross-lingual alignment of embedding models without the need for parallel corpora or full finetuning.

## introduction sanskrit is one of the oldest languages in human history, actively spoken by 25k people in india. the collection of scriptures, written in vedic and classic sanskrit, include vedas, bhramanas, arkanyas, upnishads, vedangas, upvedas, mahapurans, upapurans, darsanas, smritis, itihasa, and the bhagvada gita. these works have received so little attention that there is no consensus on the total verse count for brahmanas, aranyakas, upanishads, smritis, vedangas, upavedas, and darsanas. the rest (vedas, puranas, itihasas, and bhagvada gita) have an estimated total of 600,000 sanskrit verses. it is estimated that 30 million documents of sanskrit exist that are partly digitized @xcite . being silos of knowledge and wisdom, these are prominent works for cultural and historical studies. however, their accessibility is limited due to a lack of good quality translations and applications to search and analyze these works. figure 1 : bilingual retrieval accuracy of embedding models across indic languages and sanskrit (@xmath0. sota models that excel in indic-english retrieval, struggle with english-sanskrit retrieval. retrieval-augmented generation (rag) has emerged as the dominant paradigm for extending large language models' generative questionanswering capabilities to new domains @xcite @xcite . their multilingual and cross-lingual performance on question answering tasks have also been evaluated @xcite . the core challenge in creating a rag system is retrieval of high-quality documents to be passed as part of the context to a llm for generation. a standard retrieval pipeline uses a variation of semantic retrieval in addition to statistical methods such as bm-25 @xcite or graphs @xcite . semantic retrieval is based on similarity of vector embeddings of a given query and documents in the dataset. the quality of embeddings generated from an embedding model play a crucial role in the retrieval performance of this pipeline. a large proportion of the population interested in surveying sanskrit texts are non-native english speakers and often use their mother tongue, not english for most, as the preferred mode of communication and interaction with applications. the problem for sanskrit verse retrieval is trivial if both query and translation is available in english or indic languages (figure 1 ). the sanskrit verse can be retrieved based on semantic similarity of a translation (available in english or indic language) with the query. the non-trivial cases include: thus, multilingual embedding models capable of retrieving sanskrit documents for english and indic language queries will bolster the efforts in making accessible applications for the analysis of sanskrit texts. in our survey we found that existing corpora @xcite @xcite @xcite lacked multilingual parallel translations for sanskrit verses to benchmark multilingual and cross-lingual retrieval performance of embedding models. to aid in the development of models and applications, we introduce gitadb: a parallel aligned dataset of high quality translations of sanskrit verses in 5 indic languages and english to support the development of new models in the field of retrieval, embedding, and question answering. using our dataset we benchmark the performance of various multilingual embedding models in bilingual and cross-lingual retrieval. in our analysis, we found a wide gap in the retrieval performance of these models in retrieving sanskrit documents for english/indic queries. to bridge this gap we created monolingual adaptation networks, as a method to expand coverage of multilingual models to weakly represented languages. monolingual adaptation networks are dense feed-forward neural networks that learn to transform the embeddings for an under-represented language (sanskrit) to be closer to a pivot language (english) in a parameter and resource efficient manner. our main contributions in this paper include: ## related work in our survey we found various datasets for sanskrit translations. itihasa @xcite has a collection of 93,000 pairs of sanskrit and english translations created from two epics: ramayana and mahabharata. sahayaak @xcite is a collection of 1.5m pairs of sanskrit-hindi translations covering various domains such as daily conversations, sports, news, history, and ancient indian literature including the 700 verses from bhagvada gita. anveshana is a dataset of 3400 sanskrit document-english query pairs used to study the efficacy of translation based retrieval over direct retrieval for cross-lingual retrieval of ancient texts @xcite . samayik @xcite has a collection of 53,000 sanskrit-english pairs written in prose form, distinct from the poetic form of verses present in datasets like itihasa. other datasets such as indictrans2 @xcite and indicgen-bench @xcite cover modern sanskrit, distinct from the vedic and classic forms of sanskrit used in historic literature. most of the datasets we surveyed were either bilingual datasets for sanskrit or were multilingual datasets for low-resource indic languages excluding vedic and classic sanskrit. gitadb is the first dataset that contains multilingual verse aligned translations of 640 verses in 5 low-resource indic languages along with english. our primary objective is to identify embedding models' ability to retrieve similar verses for a given query, presented in different indic languages. @xcite introduced the concept of strong crosslingual alignment and its necessity in a multilin-gual embedding model's output. strong crosslingual alignment is achieved by maximizing intercluster distance and minimizing intra-cluster distance for multilingual embeddings of the same information. a model which exhibits low intra-verse distance and high inter-verse distance has strong cross-lingual alignment of translations which produces a high-quality retriever. thus, we use the concept of strong alignment in our study. multilingual alignment methods typically depend on parallel data or bilingual dictionaries, which are scarce for under-represented languages like sanskrit. more recent multilingual embedding models (e.g., labse @xcite , me5 @xcite ) aim to create shared representation spaces but still exhibit performance degradation on low-resource languages. parameterefficient adaptation methods such as adapter layers @xcite and mad-x @xcite have proven effective for cross-lingual transfer, yet they primarily target task adaptation rather than language alignment. in contrast, monolingual adapter networks focus specifically on resourceefficient language-space realignment, enabling embeddings of low-resource languages to be pushed closer to a pivot language without requiring parallel corpora in multiple languages or sacrificing performance on other languages. ## dataset our dataset is a collection of 640 verses taken from the bhagvada gita. the bhagvada gita is a subset of 700 verses from the mahabharata structured as a poetic discourse between arjuna and lord krishna, covering various parts of one's life: duty, knowledge, and devotion. it is also referred to as the summary of the vedas -the scriptures that form the roots of sanatan dharma. the bhagvada gita contains a total of 700 verses. after data cleanup, we were left with 640 verses with translations in 6 languages: hindi, english, gujarati, tamil, telugu, and odia for a total of 4480 sentences in our dataset. we sourced our translations from various online sources and align them at the verse level. for each verse, we store the sanskrit verse along with its translation in each language available: hindi, english, gujarati, tamil, telugu, and odia. each language uses a different script that adds a rich complexity in our dataset. after initial data collection, we found certain verses were fused together. these verses are translated in pairs/triplets as they provide necessary context for the pair/triplet of verses to be interpreted correctly. we translated each verse of the pair/triplet independently and found the meaning to be skewed without the appropriate context. thus, we decided to leave the fused verses as a single entity in our dataset. this brought our total verse count from down from 700 to 640. our dataset along with all our code for this paper can be found here foot_0 4 methods ## results overall, our experiments reveal a clear stratification in model performance across tasks and languages. while nearly all multilingual embedding models exhibited decent performance in bilingual retrieval from parallel corpora, their effectiveness dropped substantially when moving to tasks that required cross-script and cross-lingual alignment with sanskrit. the multilingual-e5 family consistently ranked at the top for bilingual scenarios, particularly the large variant, which demonstrated strong resilience to performance degradation. in the indic/english-to-sanskrit retrieval task ( there is a stark decline in performance for all models as compared to retrieval from english corpora in task 1 and for bilingual retrieval with sanskrit targets in task 2. the multilingual-e5 family also showcases a heavy language bias. languages, even with the lack of parallel corpora or full-finetuning of embedding models. the bitext retrieval task ( ## conclusion in this work, we introduced gitadb, a parallelaligned multilingual dataset of 640 bhagavad gita verses in sanskrit with translations in five indic languages and english. we benchmarked a range of multilingual embedding models on retrieval tasks of increasing complexity, revealing the strengths and limitations of current embedding models for cross-lingual and cross-script retrieval in a classical language setting. while state-of-the-art models such as the multilingual-e5 family demonstrated strong performance in parallel multilingual retrieval, their performance dropped substantially in bilingual sanskrit alignment and multilingual bitext retrieval scenarios. these results underscore the unique challenges of handling morphologically rich, low-resource languages with diverse scripts, even for models trained on extensive multilingual corpora. our method of creating resource efficient adapter networks proved effective in extending the capabilities of embedding models to an underrepresented languages without full finetuning or parallel multilingual corpora.