# Benchmarking Deep Learning Methods for Aspect Level Sentiment Classification
## 5 Experimental Results and Analysis
### Discussion of Results  
Tables 3-5 report the scores obtained by different deep learning methods with respect to Accuracy, Macro-F1 score, and Time. The top 10 best performing deep learning model for each dataset has been highlighted in boldface.  
Some observations from Tables 3 and 4 regarding Accuracy and Macro F1 score of deep learning models in ALSC are as follows:  
The best performing method across all datasets is GAT-BERT with average Accuracy and Macro F1 score of 0.8478 and 0.7334, respectively. The worst performer differs for every dataset while GRU is performing worst as per the average Accuracy and LSTM is the worst performer for Macro F1 score. The average Accuracy score range varies from 0.63 to 0.84 while the average range of Macro F1 score is from 0.43 to 0.73. The top 10 performing methods as per the average Accuracy and Macro F1 score are GAT-BERT, BERT-SPC, RGAT, CapsNet, AEN-BERT, ASGCN, ASTCN, ASCNN, TNET, and TD-LSTM. There are four BERT-based methods and six non-BERT methods among the top 10 performers. The performance of all top 10 deep learning methods is consistent on all eight datasets except TNET which has remarkably poor performance for Restaurant 15 dataset. ASGCN, ASTCN, and ASCNN form another cluster of similar performance across all datasets. TNET and TD-LSTM have similar performance for all datasets. There is an inconsistency in the range of scores for MAMS, being a difficult dataset. For the MAMS dataset, TD-LSTM and TC-LSTM have shown quite competitive scores with ASGCN, ASTCN, and ASCNN.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & \multicolumn{8}{c}{**Accuracy**} \\ \hline  & **Lap14** & **Rest14** & **Rest15** & **Rest16** & **Twitter** & **MAMS** & **Sentihood** & **Michell** & **Average** \\ \hline  
love** & 0.6052 & 0.688 & **0.742** & 0.766 & 0.569 & 0.624 & 0.882 & 0.7715 & 0.706 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Experimental results obtained for Accuracy in our study.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & & \multicolumn{8}{c}{**Macro F1 Score**} \\ \hline  & **Lap14** & **Rest14** & **Rest15** & **Rest16** & **Twitter** & **MAMS** & **Sentihood** & **Mitchell** & **Average** \\ \hline  
love** & 0.5423 & 0.4946 & 0.5536 & **0.655** & 0.4758 & 0.5732 & **0.5124** & 0.656 & 0.5579 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Experimental results for Macro-F1 score in our study.  
Though we have top performers based on the average Accuracy and Macro F1 score, still there are only small numerical differences in the scores of the top 10 deep learning methods. Thus, it is imperative to carry out statistical significance tests for empirical comparison of multiple deep learning models across various datasets on a scientific basis.  
Table 5 reports the time taken by different models.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & \multicolumn{8}{c}{**Time (in Seconds)**} \\ \hline  & **Lap14** & **Rest14** & **Rest15** & **Rest16** & **Twitter** & **MAMS** & **Sentihood** & **Mitchell** & **Average** \\ \hline  
love** & 8.667 & 14.108 & 11.210 & 15.411 & 23.567 & 52.708 & 12.224 & 18.456 & 19.544 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Experimental results for Time taken in our study.  
It is visible from Table 5 that the BERT-based methods are taking maximum time. The time range for non -BERT methods is almost similar (1 to 5 s) except RGAT. RGAT is the worst performer in terms of time with an outlier value of 260s per epoch.  
One interesting finding that can be derived from the results is that the top 10 performers in terms of Accuracy and Macro F1 score are taking maximum time with few exceptions. ASGCN, ASTCN and TD-LSTM are among the top 10 best performers but at the same time, they are not very time-consuming methods.