# Benchmarking Deep Learning Methods for Aspect Level Sentiment Classification
## 5 Experimental Results and Analysis
### Statistical Comparison of Deep Learning Methods in Aspect Level Sentiment Classification  
Statistical comparison of deep learning methods is carried out by applying the Friedman test as mentioned in Section 4. The null hypothesis of the Friedman Test is that all deep learning methods perform equally in ALSC. The first step of the test is that all deep learning methods are assigned ranks in ascending according to their performance scores for each dataset. The best method is assigned the lowest rank. Thus, the lower the rank of a model, the better the performance. Next, an average rank is calculated for each deep learning method which is the mean of ranks of methods on multiple datasets. The details of the Friedman Test have already been explained in Section 4 with Equations (6) and (7). If the null hypothesis of the Friedman Test is rejected, it is concluded that there is a statistical evidence of significant differences among at least two deep learning methods in ALSC. The rejection of the null hypothesis is followed by two post hoc tests: 1. Nemenyi Test 2: Wilcoxon Test for pair-wise comparison. The statistical tests are easily available in R ([[https://www.R-project.org/](https://www.R-project.org/)]([https://www.R-project.org/](https://www.R-project.org/)) accessed on 17 August 2021) statistical package. RQ. Is there any statistically significant difference in the performance of various deep learning methods proposed in ALSC literature from 2016 to 2021? As discussed in Section 3, this study compares the performance of 35 different deep learning methods proposed in ALSC literature. The experimentation is carried out on 8 benchmark datasets of ALSC literature that are Restaurant14, Laptop14, Restaurant 15, Restaurant 16, Twitter, Sentihood, Mitchell, and MAMS dataset. To answer the research question, it is imperative to perform statistical significance testing to investigate the evidence of the performance differences between deep learning methods. This testing is carried out by applying the Friedman test followed by post hoc tests. As per the statistical table of F-distribution [54], the critical value of F\({}_{\text{f}}\) for rejecting the null hypothesis of the [PERSON] test is 1.47. The critical value of F\({}_{\text{f}}\) is looked up in the statistical table of F-distribution. The degrees of freedom as per Equation (6) where \(m\) = 35 (number of deep learning methods) and \(n\) = 8 (number of datasets) are 34(\(m\) - 1) and 238(\(m\) - 1) (\(n\) - 1). **Hypothesis 3 (H3). **: _There is no significant difference in the performance of 35 different deep learning methods in terms of Accuracy._  
**Hypothesis 4 (H4). **: _At least two of the deep learning methods compared have a significant difference in their performance in terms of Accuracy._  
**Hypothesis 5 (H5). **: _There is no significant difference in the performance of 35 different deep learning methods in terms of Macro-F1 score._  
**Hypothesis 6 (H6). **: _At least two of the investigated deep learning methods compared have a significant difference in their performance in terms of Macro-F1 score._  
**Hypothesis 7 (H7). **: _There is no significant difference in the performance of 35 different deep learning methods in terms of Time._  
**Hypothesis 8 (H8). **: _At least two of the deep learning methods compared have a significant difference in their performance in terms of Time._Friedman Test is conducted using ranks as seen in Figures 6-8. The test statistics obtained from Equations (6) and (7) for various evaluation metrics are as under:  
**For Accuracy:**\(\chi_{f}^{2}=181.64\) and \(F_{f}=14.07\)  
**For Macro-F1 score:**\(\chi_{f}^{2}=162.79\) and \(F_{f}=10.43\)  
**For Time:**\(\chi_{f}^{2}=211.73\) and \(F_{f}=24.59\)  
For all three evaluation metrics, the experimentally observed value of \(F_{f}\) is greater than the critical value of 1.47 leading to the rejection of all three null hypotheses (H3, H5, and H7).