# Edge Convolutional Networks for Style Change Detection in Arabic Multi-Authored Text
## 5 ECNN-ASCD Evaluation
### Results and Discussion
#### 5.2.3 Performance Comparison  
A performance comparison of ECNN-ASCD against the other developed models is presented in Table 8. The results demonstrate that ECNN-ASCD achieves the top results across all three instance levels. The paired \(t\)-test was conducted on each pair of the five models on all types of instance as shown in Tables 9-11, where \"\(\Leftarrow\)\" indicates that the model in the row is significantly better compared to the model in the column, \"\(\Uparrow\)\" signifies a significant superiority of the model in the column, and \"\(\_\)\" indicates no significant difference in the results. Since the results of the paired t-test are symmetric, there is no need to calculate both directions of the comparison. Thus, the grey cells in Table 9 indicate redundant comparisons. The tables show that ECNN-ASCD performed statistically better than the others on all instances. These results suggest that GNNs, particularly EdgeConv, possess substantial potential for the SCD task.  
same encoder in ECNN-ASCD and BERT-MLP, deep learning in ECNN-ASCD helps edge representations to learn and update through layers.  
Furthermore, by comparing the GNN modules, ECNN-ASCD performs better than GCNN-ASCD, which is based on the GCN module. The EdgeConv module adopts multi-dimensional edge representations while passing its messages. GCN differs from EdgeConv by excluding edge representations across layers. Therefore, adopting learnable edge representations significantly contributes to the performance improvement in GNNs for SCD, distinguishing from the prevalent static edge representations in GCNs.  
Additionally, the nuances of style may vary according to different factors. Although designing AraSCD and PAN datasets aims to detect style changes, these datasets have not explicitly been annotated for integrating physiological or psychological variables. These variables may influence styles, such as the author's mood and environmental factors [104]. The writing style of the same author can also evolve over time [105]. Hence, identifying the stages of writing texts of the same author can also be considered another research direction [105; 106; 107]. In this paper, the research scope does not delve into studying the impact of these factors. Further studying is recommended to incorporate them and explore how they intricately shape writing style variations.  
Although the F1 score is the common evaluation metric in recent PAN editions [4; 5; 6; 7; 11], it is not well correlated to human perception and may not differentiate between writing style characteristics. Consequently, adopting other metrics in addition to the F1 score helps to analyze different perspectives of performance. The effectiveness of ECNN-ASCD can be evaluated from a human perspective. The human assessments can enhance the credibility and reliability of model predictions. It is one of the common metrics for evaluating NLP models, such as assessing chatbots [108]. In this research, time and resource constraints have posed significant challenges in conducting human evaluations, while they can provide valuable insights into style consistency. For a more comprehensive evaluation framework, future research could explore these aspects in greater detail.  
Moreover, ECNN-ASCD is one of the deep learning models that operates as a black box. The decision-making process of such models is challenging to interpret. Exploring techniques for interpretability can enable the understanding of predictions and ensure their fairness [109]. Applying these techniques may also facilitate model debugging and error analysis. While this paper has focused on developing ECNN-ASCD, the model's interpretability is not the primary focus of the current study. Further delving into interpretability is necessary to enhance the transparency and trustworthiness of ECNN-ASCD.