Large Language Models as a Normalizer for Transliteration and Dialectal Translation (2025)

Abstract: NLP models trained on standardized language data often struggle with variations. We assess various Large Language Models (LLMs) for transliteration and dialectal normalization. Tuning open-source LLMs with as little as 10,000 parallel examples using LoRA can achieve results comparable to or better than closed-source LLMs. We perform dialectal normalization experiments for twelve South Asian languages and dialectal translation experiments for six language continua worldwide. The dialectal normalization task can also be a preliminary step for the downstream dialectal translation task. Among the six languages used in dialectal translation, our approach enables Italian and Swiss German to surpass the baseline model by 21.5 and 25.8 BLEU points, respectively.

## introduction language variation encompasses how language manifests across different regions, social groups, and individual speakers. one prominent form of this variation is dialects, distinct forms of a language spoken by particular groups, often defined by geographical or social boundaries. dialects include vocabulary, pronunciation, grammar, and usage variations, reflecting the rich tapestry of human experience and cultural identity. additionally, we encounter phenomena such as transliteration in language use, which involves converting text from one script to another while preserving its phonetic characteristics. transliteration, relying on mapping the pronunciation of words (their sounds) from one language into the orthography of another, is common practice in contexts where languages with different writing systems interact @xcite . translating language varieties presents a unique and complex challenge for linguists and translators. dialects, with their distinct vocabularies, pronunciations, and grammatical structures, reflect their speakers' cultural and regional identities. capturing these nuances in translation requires a deep understanding of both the source and target languages and the cultural contexts from which they arise. in the case of transliteration, unlike a few languages where the transliterated script serves as a standard means of input (as seen in systems like pinyin for chinese), most languages lack universally established transliteration systems. when individuals use scripts other than the formal script of the language to write, they do not always adhere to a specific standard @xcite . instead, they typically employ the informal script to offer a rough phonetic transcription of the intended word. this transcription can vary significantly from person to person due to various factors, including regional or dialectal variations in pronunciation, different transcription conventions, or individual idiosyncrasies. in the evolution of language and speech technology (lst) for a given language, varieties and dialects that have more data are initially prioritized. this results in a disparity in technology usage among speakers of different dialects of the same language. for example, despite the extensive work done in english, only a few studies focus on dialects or varieties such as african-american vernacular english compared to mainstream american english @xcite . historically, roman and related scripts have enjoyed widespread support across various platforms and devices for digital content creation. although native language keyboards in numerous languages are available, most users still prefer using the roman keyboard due to its comfort and familiarity. in this work, we try to address both of these shortcomings. we build models that can translate dialectal varieties through a normalization step. we also build models that will be greatly valued by users and involve the automatic transliteration normalization of romanized input into the native orthography. in summary, our contributions are: ## conclusion in this work, we show that it is possible to use the closed-sourced llm for the new tasks: transliteration normalization and dialectal normalization, even if we do not have data for training. we also show that if we have a small quantity of data for training (ten thousand), we can lora-tune open-sourced llms to be on par or even better in performance than the closed-source ones. these open-sourced models are significantly smaller and cheaper to run than closed-source ones. finally, one can use the dialectal normalization step as a prior step for the dialectal translation task. regarding the transliteration, we only use the romanized wikipedia data from the dakshina dataset. we do not use other data sources like native script wikipedia or the romanization lexicon. the aksharantar dataset also contains 26 million romanization lexicon pairs for 21 indic languages. in this work, we focused on sentence-level transliteration. in the future, we plan on using these vast data sources for model training.