# Benchmarking Deep Learning Methods for Aspect Level Sentiment Classification
## 5 Experimental Results and Analysis
### Statistical Comparison of Deep Learning Methods in Aspect Level Sentiment Classification
Thus, alternate hypotheses H4 (corresponding to null hypothesis H3), H6 (corresponding to null hypothesis H5), and H8 (corresponding to null hypothesis H7) are accepted. This implies that there are non-random and significant differences in performance metrics of at least two out of the 35 deep learning methods for ALSC. Thus, post hoc tests are required for the relative comparison of deep learning methods. Figure 6: Nemenyi test Critical Distance diagram for 35 deep learning methods for Accuracy. Figure 7: Nemenyi test Critical Distance diagram for 35 deep learning methods for Macro-F1 Score. **Nemenyi Test Results. **  
The critical distance value for comparison of 35 deep learning methods over eight datasets is calculated using Equation (8). For \(q_{\text{a}}\) = 3.82, the value of critical distance turns out to be 19.59. In Figures 6-8, the deep learning methods are plotted against their mean ranks and are placed in ascending order of their ranks. As per Figures 6 and 7, the best performer in terms of Accuracy and Macro F1 score is GAT-BERT and the worst performer is the simple LSTM method. The lines falling inside the gray region in Figures 6 and 7 indicate methods that do not have significant performance differences in terms of Accuracy and Macro F1 score. Figures 6 and 7 reveal that top performer GAT-BERT could not outperform 18 deep learning methods in Accuracy and 19 deep learning methods in Macro F1 score. It is also observed from Figure 8 that GAT-BERT is amongst the worst three performers in terms of Time. Thus, it is difficult to conclude better performing methods across all three evaluation criteria based on the Nemenyi test. To deal with this problem Pareto approach [56] for finding non-dominated sets of deep learning methods is applied in the next section. **Selection of deep learning methods based on non-dominated sets. **  
For the selection of the best performing deep learning method across all the three evaluation criteria, the Pareto dominance concept is applied in this study. As per the Pareto dominance concept, one method \(m_{1}\) dominates the other method \(m_{2}\) if and only if \(m_{1}\) is strictly better than \(m_{2}\) in terms of at least one of the evaluation criteria and \(m_{1}\) performs no worse than \(m_{2}\) in terms of all the evaluation criteria. The methods that cannot be dominated by any other method using this approach are called non-dominated methods. Figure 9a,b illustrate the Pareto dominance approach as applied in this study. Figure 9a shows the plot of Accuracy vs. Time whereas Figure 9b shows the plot of Macro F1 score vs. Time. The objective is to maximize the Accuracy and Macro F1 score and to minimize the time. As per the figures, ASGCN dominates TNET, TD-LSTM, ASCNN, and ASTCN. GAT-BERT dominates AEN-BERT, BERT-SPC, RGAT, and CapsNet. However, ASGCN and GAT-BERT are dominated by none of the other methods. Thus, as per the Pareto dominance approach, ASGCN and GAT BERT are the two non-dominated methods. For statistical comparison of these two non-dominated methods, the Wilcoxon test is performed as discussed next. **Wilcoxon Test**  
As per the Pareto approach, ASGCN and GAT-BERT fall under the category of non-dominated methods. To further compare these two methods, a pairwise Wilcoxon test is  
Figure 8: Nemenyi test Critical Distance diagram for 35 deep learning methods for Time. performed. Significance level \(\alpha\) is set at 0.05. If the observed \(p\)-value is less than 0.05 the null hypothesis _HW0_ of Wilcoxon test is rejected. To compare two non-dominated methods across multiple evaluation criteria, the methods are tested one by one for significant differences on each criterion, in order of priorities assigned to each criterion. This is accomplished by conducting a Wilcoxon matched-pair test on each criterion in order of priority. In this study, we assign priority to Macro F1 score, second priority to Accuracy, and third to Time.