Leveraging Domain Adaptation and Data Augmentation to Improve Qur’anic IR in E nglish and A rabic (2023)

Abstract: In this work, we approach the problem of Qur’anic information retrieval (IR) in Arabic and English. Using the latest state-of-the-art methods in neural IR, we research what helps to tackle this task more efficiently. Training retrieval models requires a lot of data, which is difficult to obtain for training in-domain. Therefore, we commence with training on a large amount of general domain data and then continue training on in-domain data. To handle the lack of in-domain data, we employed a data augmentation technique, which considerably improved results in MRR@10 and NDCG@5 metrics, setting the state-of-the-art in Qur’anic IR for both English and Arabic. The absence of an Islamic corpus and domain-specific model for IR task in English motivated us to address this lack of resources and take preliminary steps of the Islamic corpus compilation and domain-specific language model (LM) pre-training, which helped to improve the performance of the retrieval models that use the domain-specific LM as the shared backbone. We examined several language models (LMs) in Arabic to select one that efficiently deals with the Qur’anic IR task. Besides transferring successful experiments from English to Arabic, we conducted additional experiments with retrieval task in Arabic to amortize the scarcity of general domain datasets used to train the retrieval models. Handling Qur’anic IR task combining English and Arabic allowed us to enhance the comparison and share valuable insights across models and languages.

## introduction recent advances in natural language processing (nlp) have helped to improve search relevance and retrieval quality. nevertheless, deep-learning techniques, specifically transformer-based approaches @xcite , are hardly employed in quran'ic nlp @xcite . in this work, figure 1 : data augmentation technique that leverages correlation of qur'anic verses for training retrieval models in-domain. we experimented with arabic and english languages. arabic, more precisely one of its variants, classical arabic (ca), is the language of the holy qur'an and is an integral component in tackling retrieval task using sacred scripture @xcite . english is another popular language used for search in various domains, including the islamic domain. addressing the problem using arabic and english allows for comparing the solutions and sharing insights across languages. english is a highresource language with a great choice of corpora and pre-trained lms for diverse domains. at the same time, depending on the domain, arabic can be considered a low-or medium-resource language @xcite . however, arabic is in more favorable conditions than english in the islamic domain; in the case of the arabic language, there are islamic corpora like openiti (romanov and seydi, 2019) and domain-specific lms @xcite . from this perspective, addressing qur'anic ir in english is more challenging as it requires a number of additional preparations, like preparing an islamic corpus and pre-training domain-specific lm. this state of affairs with the english language in the islamic domain necessitates addressing it alongside the arabic language. simultaneously, another advantage of handling the problem in english is the abundance of datasets to train for a general domain. training on general domain data can be a required step to prepare a retrieval model that needs a substantial amount of data for training, where in-domain data is scarce. experimenting with qur'anic ir in english will allow us to learn what works best and apply these approaches to arabic, where general domain data is insufficient. our main contributions are: the rest of the work is organized as follows: we start with addressing the problem of qur'anic ir in english. we prepare the islamic corpus and domain-specific lm (section 2). section 3 applies to both languages, english and arabic, including metrics choice, datasets for training and testing, experimental details, and training procedure of the retrieval models. section 4 is dedicated to qur'anic ir in arabic. apart from applying successful experiments that worked well with qur'anic ir in english, we executed more methods of preparing retrieval models for arabic language, including teacher-student distillation and employing machine translation. model comparison and final analysis is performed in section 5. the prior work done in the field is highlighted in section 6. 2 domain-specific language model as a backbone of in-domain ir ## preparing a retrieval model to extract relevant verses from the holy qur'an in arabic this section discusses how to address the same problem of designing an efficient neural ir model for extracting relevant verses from the holy qur'an in arabic. though the goal is essentially the same, the resources to achieve it are quite different in the case of the arabic language. the dataset for testing is the same as the one described in section 3.1. we use the queries as they were initially formulated in arabic by the authors of qrcd @xcite . for the choice of the metrics, refer to section 3.2. ## conclusion in this paper, we employed state-of-the-art approaches in ir to analyze and compare what works better to improve qur'anic ir in english and arabic. the results show that retrieval models in english outperform their arabic equivalents. the inherent linguistic complexity of the arabic language may explain this performance gap; nevertheless, transferring successful experiments from english to arabic, applying large machine-translated datasets, and using the proposed data-augmentation technique helped to enhance the results in qur'anic ir in arabic. one of the possible directions to take in the future is to extend this work to encompass more languages. this would broaden the scope of the semantic search for the holy qur'an, making it accessible to a larger audience. moreover, research conducted in a multilingual environment helps to exchange insights among languages and enhance the results in qur'anic ir. another essential step is to extensively evaluate real-world user queries to analyze models' performance in practice foot_3 .