# A Transformer-Based Approach to Authorship Attribution in Classical Arabic Texts
## 4 Results and Discussion  
We revisit the research question established in the introduction of this article and answer it based on the results and observations from the experiments. Our research question was: To what extent can Arabic pretrained transformer models provide satisfactory performance for the task of Arabic authorship attribution in Islamic legal texts? The conducted experiments utilized Google Colaboratory, with specifications illustrated in Table 3. Table 4 shows the results for the initial experiment, with default hyperparameters before any optimization. It is evident that ARBERT outperforms all other models on all metrics, achieving 60% in terms of accuracy and 58% in terms of F1-score. The nearest competitor was AraBERT, which achieved a 51% accuracy and a 49% F1-score. Moreover, all other models showed improvement on the baseline. The results also suggest that the performance is lower when the number of authors increases. This finding supports the results reported by [PERSON] [55]. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Value** \\ \hline Optimizer: adam\_epsilon & \(1\times 10^{-8}\) \\ learning\_rate & \(2\times 10^{-5}\) \\ num\_train\_epochs & 2 \\ per\_device\_train\_batch\_size & 16 \\ per\_device\_eval\_batch\_size & 128 \\ Max\_len (input text length) & 128 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Default AraELECTRA hyperparameters. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Dataset A**} & \multicolumn{2}{c}{**Dataset B**} & \multicolumn{2}{c}{**Dataset C**} & \multicolumn{2}{c}{**Dataset D**} \\ \cline{2-9}  & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** \\ \hline Baseline & 0.67 & 0.66 & 0.48 & 0.42 & 0.43 & 0.39 & 0.33 & 0.27 \\ (AraELECTRA) & & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance of AraELECTRA baseline. To answer this question, we present the results we obtained before optimization (Table 4) and after the three optimizations (Tables 5-7). For generalization, and because dataset D is the full dataset, we highlight in bold the best performance for this dataset. We started the experiments using the default hyperparameters in Table 1. Then, we increased the maximum length of the text from 128 to 512 in the first optimization. After that, we changed the number of epochs to five in the second optimization. Finally, we changed the number of epochs to 10 in the last optimization. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Dataset A**} & \multicolumn{2}{c}{**Dataset B**} & \multicolumn{2}{c}{**Dataset C**} & \multicolumn{2}{c}{**Dataset D**} \\ \cline{2-9}  & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** \\ \hline MARBERT & 0.95 & 0.95 & 0.94 & 0.94 & 0.94 & 0.94 & 0.93 & 0.93 \\ AraBERT & 0.97 & 0.97 & 0.98 & 0.97 & 0.93 & 0.93 & 0.93 \\ ARBERT & 0.96 & 0.96 & 0.96 & 0.94 & 0.94 & **0.94** & **0.94** \\ AraELECTRA & 0.97 & 0.97 & 0.96 & 0.96 & 0.93 & 0.93 & 0.90 & 0.90 \\ \hline Baseline & 0.67 & 0.66 & 0.48 & 0.42 & 0.43 & 0.39 & 0.33 & 0.27 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Performance results for the experiment in authorship attribution for Islamic texts with second optimization (no. of epochs = 5). \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Dataset A**} & \multicolumn{2}{c}{**Dataset B**} & \multicolumn{2}{c}{**Dataset C**} & \multicolumn{2}{c}{**Dataset D**} \\ \cline{2-9}  & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** \\ \hline MARBERT & 0.73 & 0.72 & 0.58 & 0.56 & 0.56 & 0.54 & 0.48 & 0.44 \\ AraBERT & 0.75 & 0.74 & 0.65 & 0.63 & 0.55 & 0.53 & 0.51 & 0.49 \\ ARBERT & 0.86 & 0.85 & 0.72 & 0.71 & 0.70 & 0.69 & **0.60** & **0.58** \\ \hline Baseline & 0.67 & 0.66 & 0.48 & 0.42 & 0.43 & 0.39 & 0.33 & 0.27 \\ \hline \hline \end{tabular}
\end{table}