Team I ndi D ata M iner at I ndo NLP 2025: H indi Back Transliteration - R oman to D evanagari using LL a M a (2025)

Abstract: The increasing use of Romanized typing for Indo-Aryan languages on social media poses challenges due to its lack of standardization and loss of linguistic richness. To address this, we propose a sentence-level back-transliteration approach using the LLaMa 3.1 model for Hindi. Leveraging fine-tuning with the Dakshina dataset, our approach effectively resolves ambiguities in Romanized Hindi text, offering a robust solution for converting it into the native Devanagari script.

## introduction the widespread use of social media platforms and the prevalence of english keyboards have led to a significant rise in the use of romanized typing for indo-aryan languages, primarily for quick and informal communication. however, romanized text on social media often lacks consistency, with variations in spelling, phonetic representation, and vowel omission. this lack of standardization introduces ambiguity, as the same word can be written in multiple ways, such as (namaste) appearing as namste, nmst, or namastey. romanized text also involves one-to-many mappings based on context, such as romanized text sir can correspond to (english: head) or (english: sir) based on the context. such inconsistencies lead to misunderstandings in human communication and errors in nlp applications like machine translation. in addition to standardization issues, romanized scripts fail to preserve the linguistic richness and phonetic nuances of native scripts, often losing cultural and linguistic expression. certain sounds in hindi and other indo-aryan languages lack precise representation in roman script, resulting in phonetic ambiguities. for example, the hindi letters (retroflex t) and (dental t) are both commonly written as t or ta in romanized text, ignoring the critical distinction between retroflex and dental sounds in native pronunciation. similarly, english sounds do not always map neatly to hindi phonetics. for instance, the english sounds v and w are often transliterated as (v), which can be confused with b-like sounds such as or . such limitations underscore the challenges of relying solely on romanized text for meaningful communication and accurate linguistic representation. these challenges emphasize the need for robust back-transliteration systems to convert romanized indo-aryan text into native scripts. backtransliteration maps romanized text to its native script based on phonetic representation, addressing the absence of standardization and variability in typing habits. accurate back-transliteration enhances digital communication by promoting cultural preservation, improving readability, and reducing miscommunication. furthermore, it facilitates the integration of romanized content into automated systems such as machine translation, textto-speech, and text mining, significantly boosting their effectiveness and utility. transliteration can be approached at both the word level and the sentence level. word-level transliteration models often fall short due to their inability to account for contextual information, which is essential for accurately resolving ambiguities in romanized text. this study explores sentence-level transliteration for hindi, leveraging the llama 3.1(8b) @xcite model. the experiments include both zero-shot learning and fine-tuning approaches. for fine-tuning, the dakshina dataset @xcite is employed. the fine-tuned llama 3.1 model achieves significant improvements in transliteration accuracy, as demonstrated by the bleu scores on the hindi test dataset . on test set 1, the model achieves a bleu score of 0.8866 for character overlap and 0.6288 for word overlap. on test set 2, the bleu scores are 0.8176 for character overlap and 0.5105 for word overlap. these results underscore the effectiveness of fine-tuning in improving transliteration performance, providing a robust solution for the challenges associated with romanized hindi text conversion. ## related works in recent years, significant progress has been made in transliteration for indo-aryan languages. notable contributions include @xcite , who introduced brahmi-net, a statistical transliteration system capable of handling script conversion across 18 indo-aryan languages, resulting in 306 language pairs, including hindi. similarly, @xcite developed the dakshina dataset, supporting transliteration and language modeling tasks for 12 south asian languages written in roman script, providing a foundational resource for this domain. building on these efforts, @xcite explored multilingual neural machine transliteration for english and 10 indian languages, demonstrating the potential of multilingual systems. another significant milestone is the aksharantar dataset presented by @xcite , which covers 21 indian languages and achieved state-of-the-art results using the in-dicxlit model. additionally, @xcite evaluated sentence-level transliteration across 13 languages, including 12 from the dakshina dataset and amharic, using transfer learning models like mt5-base, byt5-base, and flanpalm-62b. transliteration for informal and social media text has also been addressed in shared tasks organized by the forum for information retrieval (fire). for instance, @xcite @xcite focused on transliterating hindi song lyrics written in roman script, shedding light on the challenges of informal text processing. transliteration of romanized assamese text on social media environment is explored in the study @xcite and recently back transliteration of romanized assamese social media text is explored by @xcite standard typing patterns, and ad-hoc transliterations. this highlights the need for further research tailored to the complexities of social media communication. ## approach in this experiment, we focus on training a backtransliteration model to convert romanized hindi text into devanagari script using a sentence-level model. the architecture used is the llama 3.1 model, which is fine-tuned using a pre-defined set of instructions and inputs. the model training process includes both zero-shot and fine-tuning techniques to enhance the model's transliteration capabilities. the code for training is available at this github repo foot_0 . ## results and discussion we evaluate the model's performance in two scenarios: a zero-shot setting, where responses are generated directly from the pre-trained model using prompts, and after the model fine for the bleu score, we compute two distinct types of overlap: character-level overlap and word-level (or token-level) overlap. the bleu score for character-level overlap evaluates the precision of individual characters in the generated output compared to the reference, making it particularly useful for fine-grained tasks such as transliteration. on the other hand, the bleu score for word-level overlap measures the precision of word-level tokens in the generated output, which is more suited for tasks emphasizing semantic accuracy and fluency. the bleu score is calculated by assigning equal weight to unigrams, bigrams, trigrams, and fourgrams to ensure a balanced evaluation across different n-gram levels. we compare our model against indicxlit (madhani et al., 2023), considering it as baseline. it is a transformer-based state-of-the-art multilingual transliteration model with 11 million parameters, supporting 21 indian languages for romanto-native and native-to-roman script conversions. using indicxlit, the romanized hindi text was converted into devanagari and compared with the outputs of our trained models. the fine-tuned model demonstrates the best performance overall. on test set 1, it achieves a wer of 18.92% and a cer of 6.84%. for bleu scores, the fine-tuned model achieves 88.66% for character-level overlap and 62.88% for word-level overlap, representing gains of 15.47% and 37.83%, respectively. similarly, on test set 2, the model significantly reduces the wer and cer by 41.37% and 11.3%, respectively, compared to the indicxlit baseline. furthermore, it achieves a bleu score of 81.76% on character-level overlap for test set 2, underscoring its effectiveness in transliteration tasks. additionally, we analyze the relationship between text length and model performance by plotting line graphs of the average bleu score and its variance against text length for both test set 1 and test set 2, as shown in fig. 2 . from the graphs, we observe that the model's performance remains relatively consistent for texts longer than 8 words across both test sets. however, a slightly higher variance in bleu scores for smaller text indicates that the model's performance is less stable on text of smaller length. ## conclusion and future work this paper addresses the challenges of backtransliteration of romanized hindi text, which often suffers from inconsistencies in spelling, phonetic representation, and the omission of vowels. we explore the use of the llama 3.1 (8b) model for back-transliteration, employing both prompting and fine-tuning methods. for fine-tuning, the dakshina dataset was utilized. our results demonstrate significant improvements in transliteration accuracy, as measured by word error rate (wer), character error rate (cer), and bleu score, providing an effective solution for handling the variability in romanized text and enhancing the performance of nlp applications such as machine translation and text mining. in future work, we plan to extend our approach to other indo-aryan languages, incorporating larger and more diverse datasets. we also aim to refine the model to handle even greater text variability and improve transliteration accuracy further. additionally, exploring domain-specific adaptations and integrating the model into real-time applications will be key directions for advancing back-transliteration systems in the future.