R o BERT 2 V ec TM : A Novel Approach for Topic Extraction in Islamic Studies (2024)

Abstract: Investigating “Hadith” texts, crucial for theological studies and Islamic jurisprudence, presents challenges due to the linguistic complexity of Arabic, such as its complex morphology. In this paper, we propose an innovative approach to address the challenges of topic modeling in Hadith studies by utilizing the Contextualized Topic Model (CTM). Our study introduces RoBERT2VecTM, a novel neural-based approach that combines the RoBERTa transformer model with Doc2Vec, specifically targeting the semantic analysis of “Matn” (the actual content). The methodology outperforms many traditional state-of-the-art NLP models by generating more coherent and diverse Arabic topics. The diversity of the generated topics allows for further categorization, deepening the understanding of discussed concepts. Notably, our research highlights the critical impact of lemmatization and stopwords in enhancing topic modeling. This breakthrough marks a significant stride in applying NLP to non-Latin languages and opens new avenues for the nuanced analysis of complex religious texts.

## introduction in islamic tradition, the hadith, which includes the recorded sayings and actions of prophet muhammad as documented by his companions, is fundamental not only to religious practices but also garners significant interest in linguistic and anthropological studies. these records, known collectively as "ahadith" (the plural form of hadith), are vital to islamic jurisprudence and serve as an important supplementary source to the qur'an. islamic scholarship traditionally emphasizes the authenticity of hadith transmission chains, often neglecting contents (matn) scrutiny for anachronisms or inconsistencies, characterizing the conventional methodology in hadith studies @xcite . islam is based on (the following) five (principles): to testify that none has the right to be worshipped but allah and muhammad is allah's messenger ‫.)ﷺ(‬ to offer the (compulsory congregational) prayers dutifully and perfectly. to pay zakat (i.e., obligatory charity). to perform hajj. (i.e., pilgrimage to mecca). to observe fast during the month of ramadan. ## robert2vectm topic modeling this section presents our semantic-based topic modeling for hadith text, as illustrated in figure 2 . ## experimental evaluation to evaluate our proposed model, we compared it to various models, including probabilistic and neural network models. all models were trained on the same pre-processed dataset for fairness. lda with tf-idf. latent dirichlet allocation (lda) @xcite effectively distributes topics across large corpora. we enhanced lda by replacing bow with term frequency-inverse document frequency (tf-idf) to improve accuracy, as discussed in @xcite . non-negative matrix factorization. nmf @xcite is widely used to identify latent topics in text. we used nmf with tf-idf for text analysis and fine-tuned the model, using it as a baseline. topic modeling in embedding spaces. etm, a generative model combining topic models and word embeddings @xcite , was trained on the hadith dataset using word2vec. it groups words into topics, revealing the dataset's themes. product of latent dirichlet allocation. prodlda @xcite , a neural variation of lda, produces more distinct and less overlapping topics. we trained it with our hadith dataset. neural variational document model. nvdm, an unsupervised topic modeling approach @xcite , uses an mlp encoder to learn semantic latent variables from documents' bag-ofwords representations. we adapted nvdm for arabic with arabert embeddings to enhance document modeling and semantic analysis of the hadith dataset, enabling performance comparison with our proposed model. bertopic. bertopic employs pre-trained transformer models along with a class-based tf-idf method to enhance the coherence and interpretability of identified topics. we have adopt this model as our baseline to compare with our proposed model on an arabic dataset, utilizing the xlm-r-bert-base-nli-stsb-meantokens (grootendorst, 2022) for its supports multiple languages. even though bertopic can automatically determine the number of topics, we set it to 20-70 for comparison with other models. cluwords. cluwords @xcite creates meta-words from pre-trained embeddings, improving document representation. we adapted this for our ahadith corpus using fasttext embeddings due to their effectiveness with arabic @xcite . the remaining steps are similar to those discussed above. robert2vectm settings. before initiating the embedding process, we conducted a comprehensive analysis of our dataset to identify and compile a list of arabic terms deemed irrelevant for our analysis 6 . this carefully selected list consists of arabic terms that were considered non-contributory to our analysis. the strategic removal of these words from each document is a crucial step that significantly enhances the quality of our subsequent topic modeling efforts. by reducing the noise in the data, we facilitate a more refined analysis, en6 the list of identified terms is available on our github repository: https://github.com/saniaaftar/ roberta2vectm . to generate hybrid embeddings, we configured doc2vec with 200-dimensional embeddings and set a minimum word count of 5 to filter out common terms. a context window of 10 was used to enhance our understanding of relationship between words. the training was conducted over 50 epochs to enhance the capture of semantic nuances, and a sampling rate of 1e-5 was used for optimized learning. for roberta, we used a version fine-tuned on arabic foot_4 , along with its corresponding tokenizer (learning rate: 5e-05, train batch size: 10). finally, we trigger ctm to generate a set of topics on a given dataset. for the best performance of ctm we have selected four parameters including activation function, number of neurons, optimization function and number of layers details is shown in ## conclusion this study address the challenge of analyzing and extracting meaningful information from the hadith. we developed a contextualized topic model with hybrid embeddings to identify significant topics in complex arabic and islamic contexts. experiments show that our model, outperforms many state-ofthe-art models in understanding thematic nuances in the hadith dataset. this advancement enables precise topic identification and categorization into islamic domains like "prayer and worship" and "islamic law and society". the model's proficiency with key islamic terms has been rigorously validated through comprehensive testing.