Quranic Verses Semantic Relatedness Using A ra BERT (2021)

Abstract: Bidirectional Encoder Representations from Transformers (BERT) has gained popularity in recent years producing state-of-the-art performances across Natural Language Processing tasks. In this paper, we used AraBERT language model to classify pairs of verses provided by the QurSim dataset to either be semantically related or not. We have pre-processed The QurSim dataset and formed three datasets for comparisons. Also, we have used both versions of AraBERT, which are AraBERTv02 and AraBERTv2, to recognise which version performs the best with the given datasets. The best results was AraBERTv02 with 92% accuracy score using a dataset comprised of label ‘2’ and label '-1’, the latter was generated outside of QurSim dataset.

## introduction in recent years, natural language processing (nlp) has been evolved with the introduction of transformer architecture by @xcite . bert, built on the transformer layer, has showed and produced state-of-the-art accuracy in a number of nlp tasks such as machine translation and text classification @xcite . there are two stages of bert: pre-training and fine-tuning. pre-training is used for masked language modeling and next sentence prediction. for fine-tuning, is to add one or more layers designed for specific task on top of the final encoder layer @xcite . google provides pre-trained models for english and other languages including arabic. several studies provide their own language model based on bert to perform better on specific tasks. arabert is recently published that contributes to arabic language model that is pre-trained to suit a wide range of arabic nlp related tasks. in this paper, we have used arabert language model to classify qursim dataset in semantic relatedness task. section 2 will outline the related work in semantic similarity and relatedness while section 3 will discuss arabert. section 4 will discuss qur-sim dataset and filtration process. finally, section 5 will showcase the results of using qursim datasets using arabert. ## arabert arabert is an arabic language model in which bert was trained on a large arabic corpus @xcite . the dataset includes arabic wikipedia, 1.5 billion words from arabic corpora @xcite and the open source international arabic news corpus @xcite . the corpus covers news articles from several arab news media with different topics and from different arab countries. the size of the pre-training dataset is 70 million sentences that amounts to approximately 24gb of text @xcite . arabert has produced better results on various arabic nlp tasks. in sentiment analysis, arabert performed better than mbert, which is a multilingual bert model developed by google, on most tested datasets @xcite . also, arabert outperformed mbert and tf-idf on the new twitter-based benchmark dataset for arabic sentiment analysis @xcite . in named entity recognition (ner), arabertv01 had better results over bi-lstm-crf model with macro-f1 score of 84.2, in which arabert new state of the art for ner on anercorp @xcite . arabert also outperformed other arabic ner tools such madamira and farasa in ner tasks using aqmar and news datasets @xcite . ## qursim qursim is a work of arabic text that pertains to the holy quran @xcite . the dataset showcases 7679 pairs of verses that are similar or related verses according to comments of ibn kathir's tafsir, which is highly respected for its interpretation of the holy quran. it also improves its dataset using lexical similarity approach such as term frequency-inverse document frequency (tf-idf). qursim dataset classifies pairs of quranic verses that are related into three classes. label '2' indicates that the two verses are strongly related and they share similar lexicon. label '1' means that the two verses are related based on the main topics that were mentioned in these verses but they share less similar lexicon. finally, label '0' is being classified as not obvious relation between verses as, for example, it draws analogies that are different from each other but they serve the same purpose. label '2', '1' and '0' comprises 40.09%, 48.41% and 11.49% respectively of the total qursim dataset. ## conclusion the paper presented experiments on the qursim dataset using fine-tuned arabert model to classify pairs of verses either to be semantically related or not. the paper applied data filtration to the qursim dataset to avoid redundancy and generate unrelated pairs of verses. also, the paper used both versions of arabert and the experiments suggested that arabertv0.2 has better results than arabertv2 across all three datasets. the best performance was achieved by arabertv0.2 with 92% accuracy score. however, by examining the results, arabert could not identify some of the classical arabic lexical synonyms and religious context which could be a result of classical arabic limitation in the arabert corpus. finally, this study has a lot of potential for improvement on both the datasets and fine-tuning the arabert model.