Development of U rdu- E nglish Religious Domain Parallel Corpus (2023)

Abstract: Despite the abundance of monolingual corpora accessible online, there remains a scarcity of domain specific parallel corpora. This scarcity poses a challenge in the development of robust translation systems tailored for such specialized domains. Addressing this gap, we have developed a parallel religious domain corpus for Urdu-English. This corpus consists of 18,426 parallel sentences from Sunan Daud, carefully curated to capture the unique linguistic and contextual aspects of religious texts. The developed corpus is then used to train Urdu-English religious domain Neural Machine Translation (NMT) systems, the best system scored 27.9 BLEU points

## introduction neural machine translation @xcite has been a field of intense attention for researchers since its advent. it has shown explosive increase in research, introducing new paradigms, revealing new approaches, achieving new milestones and ultimately gaining far better accuracy levels than the previous statistical machine translation (smt) approaches. nmt research is not only focused on improving the translation quality of high-resource language pairs, but it also investigates techniques to train machines under different scenarios including monolingual @xcite , low-resource @xcite , multilingual @xcite , document level @xcite , and much more. these investigations open new hopes for nmt, but the availability of parallel corpus for training nmt systems is the bottle neck factor to improve translation quality. the more this factor is important the more it is difficult to obtain @xcite ; abdul-rauf and schwenk @xcite . after years of research on mt till today, only a few languages have huge parallel corpora available, some others have moderate parallel corpus whereas many languages still lack the availability of any parallel corpus for their training. training standard nmt systems is a real challenge in low resource settings. scarcity of available parallel corpus for low resource languages affect translation quality of nmt systems. same is the case for training domain specific nmt systems, which is subject to the availability of domain specific parallel corpus. hence, there is a need to investigate and analyse different nmt techniques for low resource settings including domain specific nmt training and adopt the possible ways to improve urdu-english machine translation which falls under the category of low resource language. the availability of massive monolingual religious translations in english and urdu motivated our research to develop a religious domain urdu-english parallel corpus. despite the abundant availability of religious corpora in multiple languages, parallel corpora are still limited. to our knowledge, u m c005 is the only religious domain parallel urdu-english corpus publicly available @xcite . the creation of such corpora for urdu, a low-resource language holds immense significance, as it enables the adaptation of machine translation systems tailored to this specialized domain. we have developed a bilingual urdu-english religious corpus of 18,426 sentences foot_0 . section 3 of this paper outlines the detailed steps and procedures taken for the development of this religious parallel corpus. we have also trained nmt models specialized for this domain where the best blue score is 27.9. our nmt experiments are described in section 4. ## related work we report the works related to publicly available religious domain parallel corpora specifically the hadith corpora. @xcite publish the first publicly available bilingual parallel corpus of islamic hadith extracted from the six canonical hadith books; using a domain-specific tool for hadith segmentation, resulting in bilingual english-arabic parallel corpus foot_1 of 39,038 annotated hadiths. however, sunan dawood is automatically aligned in their work where they report an accuracy of 92%, whereas our corpus is aligned and checked manually. abdul rauf et al. ( 2020 ) provide details about all the publicly available corpora for urdu-english language pair in biomedical, religious, technological, and general domain. we have used all the corpora mentioned in the study for our nmt experiments. ## methodology despite the fact that religious books and documents are available over the internet in massive amounts, along with their translations in many languages including english and urdu, the creation of a religious domain urdu-english parallel corpus is not easy as both languages have far different sentence segmentation and arrangement. the text of the available translations is coherent, as per the needs of language proficiency and flow. the difference in sentence structure of both the languages and the coherency of the text makes automatic sentence segmentation almost impossible. our corpus development cycle includes four different stages, collection of available translations, manual filtering of collected data, extraction of parallel translations, and sentence-level segmentation of parallel texts. ## urdu-english neural machine translation this section describes the results of nmt systems trained using our developed corpus and other publicly available urdu-english corpora. ## conclusion in this paper, we have successfully tackled the challenges of developing a parallel urdu-english corpus in the religious domain. the meticulous process of acquiring, processing, and aligning the data resulted in a corpus comprising 18,426 lines. the developed corpus underwent a thorough analysis to ensure the accuracy and integrity of data. it is then used to train urdu-english religious domain nmt systems, the best systems scored 27.9 bleu points. these findings underscore the effectiveness of the corpus in enabling accurate and meaningful translations within the religious context.