HAQA and QUQA : Constructing Two A rabic Question-Answering Corpora for the Q uran and H adith (2023)

Abstract: It is neither possible nor fair to compare the performance of question-answering systems for the Holy Quran and Hadith Sharif in Arabic due to both the absence of a golden test dataset on the Hadith Sharif and the small size and easy questions of the newly created golden test dataset on the Holy Quran. This article presents two question–answer datasets: Hadith Question–Answer pairs (HAQA) and Quran Question–Answer pairs (QUQA). HAQA is the first Arabic Hadith question–answer dataset available to the research community, while the QUQA dataset is regarded as the more challenging and the most extensive collection of Arabic question–answer pairs on the Quran. HAQA was designed and its data collected from several expert sources, while QUQA went through several steps in the construction phase; that is, it was designed and then integrated with existing datasets in different formats, after which the datasets were enlarged with the addition of new data from books by experts. The HAQA corpus consists of 1598 question–answer pairs, and that of QUQA contains 3382. They may be useful as gold–standard datasets for the evaluation process, as training datasets for language models with question-answering tasks and for other uses in artificial intelligence.

## introduction natural language processing and artificial intelligence have been employed to computerize numerous tasks that require an expert in the field. one such task involves analyzing textual material to extract information that can be used to answer questions, including finding answers from islamic religious texts such as hadith sharif and the quran. the holy quran and the hadith sharif are the primary sources for millions of muslims worldwide. muslims draw from them for legislation, teachings, wisdom, knowledge, and a complete understanding of religion, making them important and fertile resources for answering their questions. consisting of 30 parts, 114 suras and 6236 verses, the text of the holy quran is the word of god in classical arabic (ca) @xcite . the quranic text has several characteristics, such as its series of verses of different lengths; one verse may cover various topics, and the same topic may be covered in many different verses. these characteristics lead to there being many challenges in processing and researching the quranic text. hadiths are the sayings and deeds of the prophet muhammad, may god bless him, that were handed down through a chain of narrators. they may consist of a short or long sentences about what the prophet, may god bless him and grant him peace, said, his conversations with someone else or what he narrated to his companions about his actions regarding a particular matter. the significance of the hadith lies in the quran's directive for muslims to follow the teachings of the prophet muhammad, since many of the topics that are touched upon in the quran are mentioned in greater detail in the hadiths. for example, god commanded muslims to pray according to the holy quran, but the method and mechanism for praying are mentioned in the hadith sharif. processing hadiths faces the same challenges as processing the quranic text. in addition, there are 33,359 hadiths in the al-sihah al-sittah books @xcite . much research effort has been devoted to developing a system that can respond to inquiries related to the quran @xcite , while only a few studies have focused on addressing questions from the hadiths. however, the primary difficulty of question-answering (qa) studies concerns the direct and easy questions and small size of the quranic question-answer collections and the absence of a hadith question-answer corpus. as a result, each study of building a qa system for hadiths has used its own dataset to evaluate the system, which has led to obstacles in comparing the results @xcite @xcite . in addition, the small size of quranic datasets in the training phase has affected the results of the language models in the quran qa task @xcite . therefore, we aim to enrich the arabic islamic language resources. the design objectives of our two question-answer datasets are as follows: (1) to use a variety of expert books, (2) to cover various types of questions and topics and different difficulty levels of the questions and (3) to collect as many questions as possible for use in training language models and systems evaluation. our contribution is threefold: (1) we present quran question-answer pairs (quqa), the most extensive reusable quran question-answer collection, by integrating the existing available datasets and enlarging them using different resources and challenging questions. this dataset covers a large number of questions and more verses, with the questions being in modern standard arabic (msa) and the answers from quran verses in ca. (2) we introduce hadith question-answer pairs (haqa), the first reusable arabic hadith question-answer corpus, by collecting the data from different expert resources. (3) we make these two datasets available foot_0 to the research community, which will reflect positively on research on islamic qa. they can be used as a golden test collection or as training and testing data in language model research. in the following section, we discuss the existing related collections. we then outline the methodology for designing, collecting, and building our two datasets. after that, we show the resulting datasets. finally, we present our conclusion. ## related work most of the existing studies of building qa systems for the holy quran have involved creating test sets to evaluate their systems, but these datasets are unavailable. for example, datasets containing 263 question-answer pairs have been developed, and a small part of the questions were collected from websites, with the vast majority generated manually from quranic text. these questions are solely about the ' @xcite . in addition, @xcite collected 1500 questions and answers from websites. alqahtani (2019) constructed the first available corpus of 1224 question-answer pairs called the annotated corpus of arabic al-quran question and answer (aqqac) that were gathered from the islam -quran & tafseer website foot_1 . studies have not used this dataset to evaluate qa systems for several reasons, such as (1) many of the answers only consist of interpretations and not evidence from the quran, and (2) some of the questions include complete verses from the quran written in ca, and the exact required answers are written in msa by the scholar. therefore, this dataset cannot be used directly since the exact answers do not use quranic words @xcite . nevertheless, after cleaning this dataset and excluding answers that only contain interpretations, we found that only 1232 verses are used to answer the questions, covering only 19% of the holy quran. in addition, the number of questions (611) is small, and they are simple and taken straight from the text. malhas and elsayed (2020) developed a dataset called ayatec, and the process of building this collection went through many stages. they began by collecting questions from different sources, then freelancers found the answers to these questions from the quran. finally, specialist religious scholars reviewed the datasets. in addition, they developed an extended version of ayatec called qrcd, which was intended to be an intensive machine reading comprehension (mrc) task. it has been used in several recent studies to train and test different language models to obtain a system for answering questions that performs well @xcite . nevertheless, the size of this dataset is relatively small, and the number of questions is very limited. after excluding indirect answers, there are 169 questions and 1166 records, since one question may have more than one answer. only 1247 verses are used to answer the questions, which means that this corpus only covers around 20% of the quran. not all correct answers are included @xcite . the use of this collection in measuring the system's performance does not measure the strength of the actual answering system, since the nature of the most questions is direct. based on the above, to address the shortage of datasets, we design and create the quqa by cleaning and integrating the existing datasets, enlarging them with more challenge questions from various sources and covering a more significant number of verses. on the other hand, other researchers have been interested in finding answers to questions from hadiths and tested their systems by building different test collections. neamah and saad (2017) collected hadiths and then asked university students to create questions from them, with the collection size reaching 12 questions. @xcite built a collection of 3825 question-answer pairs by reading the hadiths and extracting questions manually. @xcite constructed a corpus of 33 questions from native arabic speakers and online forums. nevertheless, all these datasets are unavailable, and to the best of our knowledge, no hadith sharif question-answer datasets are publicly available. therefore, we introduce the haqa dataset to the research community to fill this gap. the methodology for creating quqa and haqa went through several stages, consisting of designing the two datasets, identifying the data sources, and collecting and cleaning the data. figure 1 shows the development methodology of these two datasets, which we now go on to discuss in detail. ## evaluation of the corpora quqa is an arabic question-and-answer dataset on the holy quran consisting of 3382 records and over 301,000 tokens. since some questions may have more than one answer, there are 2189 questions. the answers in this corpus are extracted from 2930 verses of the holy quran. accordingly, this dataset covers almost 47% of the quran. we noticed that the questions in the new dataset are more diverse and challenging than those in the previous datasets, as shown in in addition, when we analyzed the arabic haqa dataset of hadith sharif answers, we found that there are 1598 records and 1359 questions. the hadiths in this collection were taken from various sources of basic hadith books; for example, there are hadiths from al-bukhari, muslim, al-tirmidhi, al-nasai, ibn majah, imam ahmad, ibn shaybah and others. the most important matters of religion, battles, biographies of men about the prophet muhammad, the foundations of faith, belief, the principles of jurisprudence, etiquette, dealings between people, the life of the prophet, worship and others are the main topics covered by this dataset. ## conclusion and future work this paper presents the process of building two islamic religious corpora in arabic. quqa and haqa are two datasets that contain questions and answers about the holy quran and the hadith sharif, respectively. since these corpora include more than 4900 records, they are considered to be the largest islamic corpora available foot_2 to the research community. these two datasets enrich the resources of the arabic language, which suffers from a shortage of datasets in comparison with english and other languages. they open the door to conduct much more research in the field of artificial intelligence and the task of studying the nature and understanding of classical arabic texts. in the future, we plan to enlarge these two corpora to cover a significant number of hadiths and quranic verses, including a greater variety of question types and challenging questions that will improve the dataset's quality. different languages, such as english, can be added to them. in addition, a question-answer corpus can be built for other islamic books using the same methodology, enhancing the state-of-the-art of islamic qa systems.