# Benchmarking Deep Learning Methods for Aspect Level Sentiment Classification
###### Abstract  
With the advancements in processing units and easy availability of cloud-based GPU servers, many deep learning-based methods have been proposed for Aspect Level Sentiment Classification (ALSC) literature. With this increase in the number of deep learning methods proposed in ALSC literature, it has become difficult to ascertain the performance difference of one method over the other. To this end, our study provides a statistical comparison of the performance of 35 recent deep learning methods with respect to three performance metrics-Accuracy, Macro F1 score, and Time. The methods are evaluated for eight benchmark datasets. In this study, the statistical comparison is based on [PERSON], Nemenyi, and Wilcoxon tests. As per the results of statistical tests, the top-ranking methods could not significantly outperform several other methods in terms of Accuracy and Macro F1 score and performed poorly on-time metric. However, the time taken by any method is crucial to analyze the overall performance. Thus, this study aids the selection of the Deep Learning method, which maximizes the accuracy and Macro F1 score and takes minimal time. Our study also establishes a framework for validating the performance of new and alternate methods in ALSC that can be helpful for researchers and practitioners working in this area. aspect based sentiment analysis (ABSA); aspect level sentiment classification  
is a time-consuming task and performing ABSA using traditional methods requires large domain-specific datasets and expert knowledge [5; 6]. As an alternative, Deep Learning (DL) based methods are competent to learn continuous features from data without any feature engineering. In addition, deep learning methods are also efficient in capturing the relatedness between context and target. The easy availability of GPU and cloud-based computational resources has made it feasible to train deep neural networks efficiently at a low cost. Thus, in recent years, many deep learning methods, mainly Convolution Neural Networks (CNN), Memory Networks, Recurrent Neural Networks (RNN), and their multiple variants, have been proposed for ALSC [7]. The difference between Traditional and Deep learning-based ALSC can be well understood with the help of Figure 1.  
With a surge in usage of deep learning methods to perform the ALSC task [8], it is imperative to ascertain whether the newly proposed methods improve over the previous models in a statistically significant way. Most of the existing deep learning methods are ignored by the researchers for comparison while proposing their new method. In addition, none of the comparative studies [7; 8] perform a statistical comparison of the newly proposed model with existing deep learning-based models for ALSC. Furthermore, existing studies do not compare the newly proposed deep learning methods with existing methods in terms of training time. The significant contribution of our study is that it provides a statistical comparison of state-of-the-art deep learning methods available in ALSC literature up to 2021 that include RNNs, Memory Networks, CNNs, the latest Hybrid Networks, and BERT-based methods. The statistical comparison is carried for three evaluation metrics that are Accuracy, Macro F1 score, and Time. To the best of our knowledge, this is the first study in which training time is considered for evaluating the performance of deep learning methods in ALSC. The statistical comparison performed in our paper looks out for statistical evidence for the enhanced performance of recently proposed advanced deep learning methods in ALSC. [PERSON] test [9] and post hoc tests viz. Nemenyi test [10] and Wilcoxon test [9; 11] are applied to experimental results of various deep learning methods across eight datasets of different domains. Figure 1: Traditional ALSC Vs. Deep Learning-based ALSC.