Machine Translation and Transliteration for I ndo- A ryan Languages: A Systematic Review (2025)

Abstract: This systematic review paper provides an overview of recent machine translation and transliteration developments for Indo-Aryan languages spoken by a large population across South Asia. The paper examines advancements in translation and transliteration systems for a few language pairs which appear in recently published papers. The review summarizes the current state of these technologies, providing a worthful resource for anyone who is doing research in these fields to understand and find existing systems and techniques for translation and transliteration.

## introduction the indo-aryan languages constitute a main branch of the indo-european language family, predominantly spoken in central and north india as well as in neighbouring countries such as sri lanka, pakistan, nepal, maldives, bangladesh and bhutan @xcite . the large linguistic varieties within the indo-aryan language family make it challenging to communicate both outside and within the region. machine translation and transliteration systems help to bridge language barrier,s enabling effective communication between different linguistic societies. the goal of this review paper is to provide an overview of the current state of machine translation and transliteration techniques for indo-aryan languages. the review discusses diverse techniques used in the recently published translation and transliteration systems which handle the various scripts and linguistic features of indo-aryan languages. the contribution of this study can be summarized as performing a systematic review of existing translation and transliteration techniques related to indo-aryan languages, highlighting the significant contributions and developments made by researchers in this constantly developing field. going forward, the review is structured to clearly look into the recent developments in machine translation and transliteration for indo-aryan languages. starting with the methodology explains how studies were selected based on their relevance. the following sections dive into various translation and transliteration approaches and outline the challenges faced in the field. ## methodology a systematic approach was adopted in this review to choose the relevant studies on machine translation and transliteration for indo-aryan languages. a comprehensive search was conducted across several major academic databases, including ieee xplore and google scholar. in addition to the academic database searches, several key papers were identified from references cited in already published research, ensuring a wide-ranging collection of studies relevant to the focus of the review. keywords such as "machine translation", "transliteration" and "romanized languages" were used to identify relevant literature. to avoid redundancy, duplicate publications across different databases were identified and removed. this review focused on papers published from 2018 to the available 2024 publications to ensure that the recent advancements were included. studies were chosen based on the relevance to machine translation and transliteration within the context of indo-aryan languages. this review also includes papers that have proposed and utilized relevant techniques as part of their work while not directly focused on translation or transliteration. specifically, the papers which proposed novel methodologies or made outstanding contributions to the field were prioritized. figure 1 illustrates the systematic flow of the paper selection process. ## machine translation (mt) and transliteration machine translation (mt) is the study of how to use machines to translate from a source language into another target language. this concept was first put forward by warren weaver in 1947 @xcite . from then on, mt has been one of the most challenging tasks in the natural language processing (nlp) field. figure 2 is an example of machine translation between sinhala and english. machine transliteration is the process of words transformation from one language into their phonetic equivalent of another. there are two types of machine transliteration: forward and backward transliteration. forward transliteration is the process of transliterating a word to a foreign language from the language from which it originated. on the other hand, when a word is converted back to the language of its origin from a foreign language, it is known as backward transliteration @xcite . figure 3 illustrates the difference between forward and backward transliteration using romanized sinhala. ## approaches in machine translation and transliteration many machine translation and transliteration systems have been implemented for indo-aryan languages. since transliteration is considered a form of translation, both translation and transliteration systems have used similar approaches. the following section will discuss the various machine translation and transliteration approaches found in the literature. here, the iso 15919 standard foot_0 for the transliteration of devanagari and related indic scripts into latin characters has not been the focus, but it may be relevant in rule-based approaches. ## current state of machine translation for indo-aryan languages this section provides an overview of the current state of mt approaches developed for diverse indo-aryan language pairs. ## current state of machine transliteration for indo-aryan languages the transliteration of indo-aryan languages has been a challenge of research for several decades. there are various models proposed to address the complexities of converting text from one script to another. over the years, the transliteration approaches have improved from traditional rule-based methods to modern neural and hybrid models, reflecting the increasing computational capabilities. from 2018 to 2024, there were more studies on transliteration systems for sinhala compared to other indo-aryan languages. in 2018, significant contributions were made to transliteration with the development of rule-based and modern machine-learning approaches. a rulebased transliteration system for romanized sinhala was proposed, using phonetic and transliteration rule bases to transliterate romanized text into native sinhala script. while effective, the system faced limitations in handling ambiguities, particularly with proper nouns @xcite . another study experimented with seq2seq and lstm models to develop a scalable transliteration pipeline for indian languages and evaluated different language transliterations. the results showed that the seq2seq models outperformed traditional lstm models, although they need large datasets for effective training @xcite . additionally, a character-level transliteration tool was created to improve tamil to sinhala nm,t demonstrating the utility of rule-based methods in translation tasks @xcite . according to their literature, that was the first tamil to english and sinhala to english transliteration tool that used a rule-based approach. in 2019, @xcite introduced a hybrid approach using smt and machine learning to transliterate personal names in the sri lankan context using moses smt toolkit for sinhala, tamil and english languages. this system showed the importance of incorporating ethnic origin classification for personal name transliteration to improve accuracy. another significant transliteration approach was the gurmukhi to roman transliteration, which used character mapping and handcrafted rules for the transliteration of punjabi to english with a good accuracy of 99.27% @xcite . there were further advancements in transliteration techniques during 2020 and 2021, particularly with the use of neural networks. a rule-based method which is proposed by ucsc is combined with a trigram model trained on social media text to improve the sinhala transliteration accuracy in the study by @xcite . another study in 2021 introduced a rule-based approach for singlish to sinhala transliteration with an error correction module to improve accuracy @xcite . @xcite exper-imented with various neural architectures for the transliteration of hindi and punjabi languages. out of those, a model with a character/grapheme level bidirectional encoder and auto-regressive decoder proved to be the best-performing architecture. in the same year, a systematic approach employing phrase-based statistical machine translation (pb-smt) to create an english-hindi parallel database for transliteration was introduced @xcite . another work in 2021 was the development of a python-based algorithm to transliterate between devanagari or roman scripts and brahmic scripts, and vice versa @xcite . additionally with the introduction of a method for normalizing and back-transliterating hindi-english code-switched text, this field saw further innovation @xcite . this system first normalized romanized hindi with the use of the seq2seq model based on an lstm encoder-decoder architecture and then syllabified the tokens to map them to the devanagari script. this approach could handle informal typing variations and phonetic discrepancies, improving the transliteration. moving into 2022, swa-bhasha (athukorala and sumanathilaka, 2022) proposed a novel approach using a combination of rule-based methods and fuzzy logic to transliterate singlish to sinhala even when vowels are omitted. this system has introduced a new numeric coding system to use with the romanized sinhala letters by matching with the recognized typing patterns. fuzzy logic-based implementation has been used for the mapping process. another back-transliteration system for romanized sinhala to sinhala was proposed by @xcite utilizing a transliteration unit (tu) based model and a bilstm encoder combined with an lstm decoder. moreover, in 2022, a bilingual rbmt system was developed for sanskrit-english. this system allowed users to type sanskrit using english orthography and transliterate sanskrit text into the english script @xcite . in 2023, @xcite introduced a generative adversarial networks (gans) based system using pix2pix gan architecture to transliterate ancient indian scripts (images) like nandinagari and sharda into modern devanagari script (images). yadav and kumar (2023) proposed a hybrid approach to transliterate hindi to english which includes image processing and a model trained with attention. the final phase of the proposed system, which is the transliteration phrase, used the python indicate transliteration library to transliterate hindi characters into the roman script. in the same year, swa-bhasha hybrid approach combining statistical methods with a trigram and rule-based model was proposed for singlish back transliteration @xcite . additionally, it incorporated a trie data structure to generate word suggestions. the work by @xcite has achieved 0.64-word level accuracy while liwera and ranathunga (2020) achieved 0.52-word level accuracy. this swa-bhasha system has performed much more accurately with 0.84-word level accuracy compared to the existing transliteration works for sinhala. by applying a similar hybrid approach, another back-transliteration system for romanized tamil, tamzhi, was proposed by @xcite . this system achieved 93% accuracy at the character level and 70% at the word level, further demonstrating the effectiveness of this method. in 2024, further advancement was made with the introduction of swa bhasha 2.0 @xcite , which is developed to address the ambiguities of romanized sinhala back transliteration using gru-based nmt. also, the study of swa-bhasha dataset @xcite introduced a rule-based transliteration tool which can annotate sinhala words into romanized sinhala. this system can accommodate the various ad hoc typing patterns used by the community. finally, in 2024, another model was proposed for accurate cross-script conversion, focusing on the hybrid model development for transliteration. this study compared two models: a hybrid of seq2seq with lstm and a hybrid of rule-based and nmt approaches. seq2seq with an lstm-based model demonstrated superior performance, especially in back-transliterating english text into different indic languages @xcite . ## gaps and challenges in machine translation and transliteration for indo-aryan languages despite significant advancements in the field of machine translation (mt) and transliteration for indo-aryan languages, there are still several challenges and gaps that can be identified. addressing these will be important to develop reliable systems for any language. this section describes some of the identified gaps and challenges in this field. ## conclusion the review highlights significant advancements in machine translation and transliteration for indo-aryan languages. translation systems have seen notable improvements in accuracy with the advancement of natural language processing. in transliteration, there has been progress in converting text between different scripts by managing the phonetic variations. notably, both translation and transliteration have seen significant enhancements with the advent of transformer architecture variations, which is marking a promising direction for future research in this field. these developments are important in improving effective communication and access to information across different indo-aryan language communities.