I ndo NLP 2025 Shared Task: R omanized S inhala to S inhala Reverse Transliteration Using BERT (2025)

Abstract: The Romanized text has become popular with the growth of digital communication platforms, largely due to the familiarity with English keyboards. In Sri Lanka, Romanized Sinhala, commonly referred to as “Singlish” is widely used in digital communications. This paper introduces a novel context-aware back-transliteration system designed to address the ad-hoc typing patterns and lexical ambiguity inherent in Singlish. The proposed system com bines dictionary-based mapping for Singlish words, a rule-based transliteration for out of-vocabulary words and a BERT-based language model for addressing lexical ambiguities. Evaluation results demonstrate the robustness of the proposed approach, achieving high BLEU scores along with low Word Error Rate (WER) and Character Error Rate (CER) across test datasets. This study provides an effective solution for Romanized Sinhala back-transliteration and establishes the foundation for improving NLP tools for similar low-resourced languages.

## introduction the rapid growth of digital communication platforms such as social media and messaging platforms has revolutionized communication with the use of informal, romanized representations of native scripts. sinhala is a morphologically rich language where approximately 17 million sri lankans (around 87% of the total population) use it as their main language for communication @xcite . many sinhala speakers use romanized sinhala, often referred to as "singlish", instead of the native script on digital communication platforms due to the convenience of using english keyboards. however, singlish is non-standardized, leading to variations in spelling and structure, which pose challenges for back-transliteration. the process of backtransliteration into native script has become crucial for nlp applications such as machine translation, information retrieval and sentiment analysis. however, the following challenges make this task complex: a system capable of handling the typing variations, ambiguity, and contextual dependencies inherent in singlish is required to address these challenges. back-transliteration is a greater challenge than forward-transliteration because it requires context awareness @xcite . this paper introduces a novel context-aware backtransliteration system for romanized sinhala leveraging a hybrid approach that combines: ## related works back-transliteration of romanized sinhala has been the focus of several studies exploring various approaches including rule-based, statistical, and neural approaches. below are some recent studies on singlish backward transliteration. in 2018, the sinhala language decoder by @xcite introduced a rule-based transliteration method as part of their work where romanized input text is processed using transliteration and phoneme rule bases. however, the system struggles with handling lexical ambiguity and some english proper nouns because of the static nature of its rule base. these limitations have restricted its ability to handle the informal typing patterns of romanized sinhala. in 2019, @xcite proposed a statistical machine translation (smt) approach to transliterate personal names across sinhala, tamil, and english. since the personal name transliteration depends on the ethnicity of the name, they employed ethnicity-specific models, achieving bleu scores of more than 89% for all language pairs. this was implemented with a classification followed by the naive bayes algorithm. the reason for selecting the smt approach instead of a neural approach is that nmt lacks robustness in translating rare words, and it requires a large amount of parallel data to train the model to achieve better results than smt. in 2020, a combination of trigram and rulebased models was proposed by @xcite . this hybrid approach integrated trigram models with rule-based methods to transliterate romanized sinhala. the trigram model was trained on singlish youtube comments and their corresponding sinhala transliteration. a rule-based approach was used to handle situations where the tri-gram model could not predict the sinhala transliteration of singlish words. however, the system occasionally fails to deliver the correct transliteration of a word due to ambiguities. silva and ahangama (2021) proposed another rulebased approach for romanized sinhala backward transliteration in 2021. the accuracy of the rulebased approach was further improved by using an error correction module which compares a news corpus from popular news sites. in 2022, a contextaware back-transliteration for romanized sinhala presented a neural machine translation approach (an encoder-decoder model) based on bidirectional lstm and lstm architectures @xcite . the study presented a transliteration unit approach considering the context of characters in a word. this system also failed to handle sentencelevel word disambiguation as it focuses on the context of the characters present in a word. a back transliteration system which can handle informal shorthand romanized sinhala was proposed by @xcite . a statistical trigram model combined with a rule-based approach for back transliteration and a knowledge base with trie data structure for word suggestions was used in the work. the proposed system achieved 0.84 word-level accuracy. this proposed architecture has been further extended for tamil by (mudiyanselage and sumanathilaka, 2024), showing the generalizability of the proposed model. however, lexical ambiguity correction (word sense disambiguation) and code-mixed romanized sinhala remain a persistent issue in these approaches. athukorala and sumanathilaka (2024) proposed a novel approach which combines rule-based methods and fuzzy logic to transliterate romanized sinhala to native script even when vowels are omitted. it introduced a new numeric coding system to use with the singlish letters by matching the identified typing patterns. for the mapping process, they have developed a fuzzy logic-based implementation. however, the system performs at the word level and does not handle lexical ambiguities. in 2024, @xcite proposed a gru-based nmt model for singlish backward transliteration. this system combined rulebased techniques with neural machine translation to address the complexities of romanized sinhala. a suggestion algorithm has eliminated word selection ambiguity by choosing word suggestions from a pool of predicted words. bleu scores reaching 0.8 indicate the high word-level transliteration accuracy of the proposed model. though many romonized sinhala to sinhala transliterators have been introduced, there still exists a gap in the availability of an effective reverse transliterator, which needs context awareness to handle ambiguity. ## methodology the proposed context-aware transliteration system is developed through a series of systematic steps to transliterate romanized sinhala text into native sinhala script, ensuring accurate and contextually appropriate output even while dealing with lexical ambiguity and ad hoc typing patterns. the methodology consists of five key steps, as described below. ## challenges and solutions the primary challenge of the proposed transliteration approach was the time consumption for processing long sentences containing highly ambiguous words. in the proposed transliteration approach, the major factor contributing to time consumption is the number of model inferences required for disambiguation. two key aspects that influence the number of model inferences: two strategies were developed to reduce the processing time while maintaining accuracy, as described in section 4.1 and 4.2. ## result evaluation and discussion for the baseline evaluation, a bert model trained on sinhala data sources for mask language mod-elling from hugging face (model 1 foot_1 ) was used to develop the proposed back transliteration system. then, it was further fine-tuned using native sinhala script data in the dakshina dataset @xcite . the training hyperparameters were used during fine tuning (model 2 foot_2 ): learning-rate: 5e-05, train-batch-size: 64, eval-batch-@xmath0, numepochs: 12. ## conclusion the proposed context-aware back-transliteration approach effectively converts romanized sinhala text into native sinhala script, addressing the challenges of ad-hoc typing patterns and lexical ambiguity inherent in romanized sinhala back-transliteration. evaluation results demonstrate the robustness of the proposed approach, achieving high bleu scores along with low word error rate (wer) and character error rate (cer) across test datasets. the codebase can be accessed through the link below for further research in this area. github link: https://github.com/sameera2001perera/ singlish-transliterator