# A Transformer-Based Approach to Authorship Attribution in Classical Arabic Texts
## 1 Introduction
However, most of these studies have focused on various Arabic collections such as articles, poems, Tweets, and other text genres from Arabic books. Although Islamic law text, which includes Islamic legal rulings [23; 24] and Islamic doctrine affiliation [25], is considered to be among the Arabic literary genres, studies of AA related to this genre are limited. Numerous approaches and algorithms have been utilized for the task of AA. An early method was the rule-based approach, in which a set of rules were identified to help the system learn how to categorize and classify given data. Later, machine learning (ML) models were introduced [26] to support the development of solutions for various NLP tasks; when used in classification problems, ML needs to train on the data using known features [27]. However, this technique requires experts to extract those features, which is considered to be costly and time-consuming. Surveying and reviewing the literature [15; 16; 28] has revealed that most studies have used the traditional machine learning classifiers, such as naive Bayes, support vector machine (SVM), SMO-SVM, linear discriminant analysis (LDA), K-nearest-neighbors (KNN), logistic regression, and the Gaussian Bayes and decision trees classifiers. More recent ML approaches have shifted to using deep learning methods. Deep learning is a subfield of machine learning that works to stimulate the human brain for analytics learning [27]. There are many deep learning models used with NLP systems such as convolutional neural network (CNN) [29], recurrent neural network (RNN) [30], and sequence-to-sequence (Seq2 Seq) models [31]. Some AA studies, such as the work by [PERSON] and [PERSON] [32], have considered using DNN models in their experiments. Another study, conducted by [PERSON] et al. [33], proposed a regularized deep neural network (RDNN) method that enhances the accuracy of the AA for online text snippets, such as posts, called the PAA system. Although AA tasks have utilized deep learning models, the main challenge of using these models is that their training requires a large amount of labeled data, which are expensive to generate [26]. This problem has therefore led to a shift toward using transformer pretrained models to overcome the drawbacks of deep learning models [34]. Several pretrained transformers are available. The most popular is the bidirectional encoder representations from transformers (BERT) model, the first to be based on transformer decoder and encoder layers [26]. Another model, ELECTRA, is a pretrained model that trains two transformer models: a generator and a discriminator. Recently, the difference between the pretrained language representation models, such as BERT or ELECTRA, has either been based on replaced token detection (RTD) or masked language modeling (MLM) [35]. The BERT-based models are pretrained via MLM, in which the model is asked to find original tokens that have been hidden randomly in the input sequence [35]. ELECTRA methods are pretrained based on replaced token detection (RTD), which is considered to be more efficient than MLM [35]. Because BERT has achieved significant results in language understanding for English, the Arabic computing community has worked on Arabic variants of BERT (AraBERT [36], ARBERT [37], and MARBERT [37]) for the Arabic language. The Arabic computing community has also worked on Arabic variants of ELECTRA, i.e., AraELECTRA [35], which is an Arabic model that has been used for many NLP tasks. Recent works on NLP tasks in general, and AA in specific, have shown that pretrained transformer methods can achieve satisfactory results. Although several studies in the literature have focused on Arabic AA in general, the utilization of AA for Islamic text has not gained similar attention. This study investigates the recent state-of-the-art (SOTA) Arabic transformers in a rarely studied domain that has seen limited research contributions: the domain of Islamic legal texts.