R o M antra: Optimizing Neural Machine Translation for Low-Resource Languages through R omanization (2024)

Abstract: Neural Machine Translation (NMT) for low-resource language pairs with distinct scripts, such as Hindi-Chinese and Japanese-Hindi, poses significant challenges due to scriptural and linguistic differences. This paper investigates the efficacy of romanization as a preprocessing step to bridge these gaps. We compare baseline models trained on native scripts with models incorporating romanization in three configurations: both-side, source-side only, and target-side only. Additionally, we introduce a script restoration model that converts romanized output back to native scripts, ensuring accurate evaluation. Our experiments show that romanization, particularly when applied to both sides, improves translation quality across the studied language pairs. The script restoration model further enhances the practicality of this approach by enabling evaluation in native scripts with some performance loss. This work provides insights into leveraging romanization for NMT in low-resource, cross-script settings, presenting a promising direction for under-researched language combinations.

## introduction machine translation (mt) between languages with distinct writing systems, such as hindi, chinese, and japanese, poses significant challenges due to scriptural and linguistic differences. traditional mt methods, particularly statistical approaches, struggle with these complexities due to limited data availability, syntax, morphology variations, and script diversity @xcite . although nmt has brought advancements through deep learning models @xcite , translating between languages like hindi, chinese, and japanese remains largely underexplored. this challenge is especially acute for hindi-chinese translation, where no publicly available parallel datasets or nmt models exist, while japanese-hindi translation has more resources, though still limited. a core challenge in translating between these languages arises from their linguistic differences. chinese is an isolating language that relies heavily on word order (svo) to convey meaning, while hindi is a highly inflectional language that uses case endings and verb inflections, allowing for flexible word order (sov). for instance: मु झे पानी चािहए (mujhe pani chahiye) "i need water." the equivalent chinese sentence is: in chinese, strict word order is required, while in hindi, case inflections allow for flexibility. this disparity complicates direct translation, as the rigid structure of chinese contrasts with hindi's morphological richness. furthermore, while large corpora exist for chinese and japanese in other contexts, these languages are considered low-resource when paired with hindi due to the lack of publicly available parallel datasets. we hypothesize that romanization as a preprocessing step can enhance translation performance for low-resource language pairs with distinct scripts, such as hindi-chinese and hindi-japanese. romanization converts non-roman scripts into a unified roman script, mitigating scriptural differences and enabling more efficient model training. additionally, we propose a neural script restoration model to revert romanized text back to its original script, preserving crucial linguistic and semantic features. in this paper, we introduce a nmt framework that incorporates romanization for both source and target text, along with a neural script restoration model to ensure accurate evaluation in native scripts. our approach optimizes the handling of script diversity and improves translation quality for language pairs like hindi-chinese and hindi-japanese. our contributions are: the following sections elaborate on related work, our methodology, experimental setup, and results, demonstrating the effectiveness of our approach. ## related work machine translation (mt) has a long history, evolving from statistical approaches to the more recent neural-based methods. early machine translation systems, such as statistical machine translation (smt), heavily relied on rule-based methods and parallel corpora to achieve reasonable performance @xcite . however, smt systems faced significant challenges, particularly for low-resource language pairs, which often suffer from data scarcity and script disparity issues @xcite . the advent of neural machine translation (nmt) significantly improved translation quality, with architectures like sequenceto-sequence models @xcite and attention mechanisms @xcite . these advancements allowed nmt systems to handle complex linguistic patterns more effectively, especially in high-resource languages. yet, for low-resource language pairs with distinct scripts, nmt still struggles, particularly when translating between languages such as chinese and hindi, which feature vastly different writing systems and linguistic structures @xcite @xcite . romanization has emerged as a technique to bridge the gap between languages with disparate scripts, converting non-roman scripts into latin characters for more unified processing. romanization is particularly effective in low-resource settings, as it allows for leveraging existing tools designed for roman-script languages @xcite . this method facilitates transfer learning between different scripts, enabling nmt systems to share embeddings across languages @xcite . however, romanization can introduce issues, including loss of tonal and phonetic information, which is critical in languages like chinese and japanese @xcite . in the context of chinese and japanese, various methods have been explored to address these challenges. stro-kenet, for example, converts chinese characters into latinized stroke sequences, allowing subword learning in nmt systems @xcite . similarly, wubi encoding breaks chinese characters into component radicals, facilitating character-level translation in nmt models @xcite . these techniques have shown promising results but require significant adaptation for different language pairs, particularly for languages with highly distinct phonological structures, such as hindi. our work extends these efforts by exploring romanization on both source and target languages, focusing on translation tasks between japanese-hindi (ja-hi) and chinese-hindi (zh-hi) pairs. prior research often focused on source-side romanization @xcite , but our approach also incorporates target-side romanization, with the addition of a neural model for restoring romanized output to its original script post-translation. this ensures the retention of native linguistic characteristics and improves the overall quality of the translated output. ## methodology in this section, we describe the framework of our approach for neural machine translation (nmt) between low-resource languages with distinct scripts. our proposed framework involves three key components: romanization, translation, and script restoration, as illustrated in figure 1 . romanization is applied to simplify script processing, followed by translation using different combinations of romanized and native scripts. the final component involves script restoration, where romanized outputs are converted back to their original native script to ensure proper evaluation and usage. ## results & discussions in this section, we present the results of our experiments, comparing the performance of baseline models (w/o romanization) and models with romanization applied on both sides, source-side only, and target-side only. we also analyze the impact of different byte pair encoding (bpe) merge operations and evaluate the performance of our script restoration model. these results aim to answer our core hypothesis: can romanization improve translation performance for distant low-resource language pairs with distinct scripts? w/o romanization model metrics bleu chrf comet hi-zh 12.5 21.5 90.0 zh-hi 14.7 42.9 67.3 ja-hi 14.6 41.8 67.8 hi-ja 22.5 29.6 94.9 ## conclusion and future work this paper introduced an approach to neural machine translation (nmt) for low-resource language pairs like hindi-chinese (hi-zh) and hindi-japanese (hi-ja) by employing romanization as a preprocessing step, simplifying script processing and enhancing translation quality. integrating a neural script restoration model ensured accurate conversion of romanized outputs back to their original scripts, preserving semantic integrity. our experiments showed significant improvements in bleu and chrf scores, demonstrating the effectiveness of this approach for languages with distinct scripts and limited parallel resources. future work will focus on refining romanization to handle script-specific ambiguities, improving the script restoration model, expanding the framework to more language pairs, and incorporating advanced pre-trained language models to enhance translation accuracy and scalability.