# A Transformer-Based Approach to Authorship Attribution in Classical Arabic Texts
## 4 Results and Discussion
Table 7 shows the results of this final optimization; surprisingly, AraELECTRA achieved 97% in both accuracy and F1-score results, which are higher than previous experiments. In addition, MARBERT showed the lowest performance, with 95% in both accuracy and F1-scores. ARBERT and AraBERT obtained 96% in terms of accuracy and F1-scores, which is an increase compared to the previous experiment. To conclude, and based on our experiments, AraELECTRA (fine-tuned on 10 epochs with a maximum token length of 512) achieved the best performance for the AA of Islamic legal texts. By comparing our final best results with those reported in the literature for low-resource languages, as Table 8 shows, our best model is found to be comparable to the best model reported. This answers our research question, in that Arabic pretrained transformer models can provide satisfactory performances for the task of Arabic authorship attribution in Islamic legal texts. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Dataset A**} & \multicolumn{2}{c}{**Dataset B**} & \multicolumn{2}{c}{**Dataset C**} & \multicolumn{2}{c}{**Dataset D**} \\ \cline{2-9}  & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** & **Accuracy** & **F1-Score** \\ \hline MARBERT & 0.98 & 0.98 & 0.97 & 0.97 & 0.95 & 0.95 & 0.95 & 0.95 \\ AraBERT & 0.97 & 0.97 & 0.98 & 0.98 & 0.96 & 0.96 & 0.96 \\ ARBERT & 0.97 & 0.97 & 0.97 & 0.97 & 0.95 & 0.95 & 0.96 & 0.96 \\ AraELECTRA & 0.99 & 0.99 & 0.99 & 0.99 & 0.96 & 0.97 & **0.97** & **0.97** \\ \hline Baseline & 0.67 & 0.66 & 0.48 & 0.42 & 0.43 & 0.39 & 0.33 & 0.27 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Performance results for the experiment in authorship attribution for Islamic texts with third optimization (no. of epochs = 10). The results in Tables 4-7 answer the first research question, and highlight the most important hyperparameters that affect the AA for Islamic legal texts. Based on this evidence, we conclude that the best hyperparameters are 10 epochs and a maximum input length of 512 tokens. To further validate the performance and test for generalizability, we conducted an experiment using five-fold cross-validation with an ensemble of all models (from the K-folds) on dataset D, using the best optimization. Table 9 shows the results of this experiment. However, performance decreased by 1% for AraELECTRA and AraBERT. The five-fold cross-validation had no significant effect on the results for either MARBERT or ARBERT. Despite these lower results, AraELECTRA and ARBERT remain the best-performing models. To investigate our results further, we performed a model testing on an unseen independent dataset. We randomly selected 100 texts from a sample of 13 authors from Al-Maktaba Al-Shamela, and evaluated the predictions using the AraELECTRA and ARBERT base models, which are the two that specifically outperformed the other models after the cross-validation process. AraELECTRA and ARBERT gained 96% for the task of AA as Table 9 shows. However, there was a slight performance decrease when predicting and labeling unseen data: both models misclassified seven texts, and classified 93 texts correctly.