# Benchmarking Deep Learning Methods for Aspect Level Sentiment Classification
## 3 Deep Learning Methods for Aspect Level Sentiment Classification
### Recent Trends in Aspect Level Sentiment Classification  
ALSC can be performed using three main approaches: unsupervised, semi- supervised, and supervised. The unsupervised and semi supervised techniques mainly follow corpus based and lexicon-based approaches [14]. The corpus-based approach utilizes the domain specific large corpora for generating relevant information, which requires substantial manual effort, extensive training, and big data. The lexicon-based approach (also known as the knowledge-based approach) works with external sentiment knowledge bases known as lexicons. Such techniques are entirely dependent on the quality of the knowledge base and often suffers the problem of the limited word (or out of vocabulary) [15]. The supervised ALSC can be carried using conventional or deep learning methods. The conventional (also known as machine learning-based) techniques require extensive feature engineering, while deep learning methods can work efficiently without feature engineering. Thus, recently ALSC researchers are inclined more towards deep learning -based approaches. Deep Learning is a learning paradigm that utilizes artificial neural networks for the efficient learning process and has recently gained importance in many tasks related to NLP such as text summarization [16], machine translation [17], question answering [18], etc. Deep learning methods utilized for sentiment analysis have also proven to perform better than traditional methods. The ensemble of deep learning methods with symbolic models has also been leveraged to create a sentiment lexicon. Senticnet5 [19] and Senticnet5 [20] are the lexicons generated by combining symbolic and sub-symbolic paradigms of AI. The symbolic paradigms refer to the use of logic and semantic network, and sub-symbolic refers to the usage of deep learning methods to encode the meaning of words in the lexicon. In ALSC, the classification is reliant on the semantic structure of sentences, thus researchers have widely utilized LSTM in the field of ALSC. LSTM with the capability of handling non-linear data has proven to be successful for this task. [PERSON] et al. [21] used LSTM for the ALSC task for the first time. The authors also proposed two variants of LSTM called TD-LSTM and TC-LSTM. Later, [PERSON] et al. [22] leveraged the idea of combining aspect embedding with LSTM. The authors also proposed the attention-based variant of LSTM known as ATAE-LSTM. The integration of attention mechanisms in deep learning methods was initially proposed for computer vision tasks [23]. It has also been successfully applied on many NLP  
Figure 3: An example of implicit and explicit aspects. tasks [24; 25]. Attention-based methods have the capability of capturing the significance of the context words. This feature makes attention-based deep learning methods more promising for ALSC [22; 26; 27; 28]. Along with attention, memory networks have also been utilized for ALSC. [PERSON] et al. [29] proposed a deep memory network by using a pre-trained word vector as memory. The authors leveraged the attention mechanism for updating memory. [PERSON] et al. [30] proposed the Recurrent Attention Memory (RAM) method, which exploits attention mechanisms and uses hidden states generated by LSTM as memory. Simple attention networks can generate noisy features as well. This problem can be resolved using Capsule networks [31; 32]. The capsule network can dynamically route the spatial features from the lower layer to the upper layers. The hidden vectors generated from the lower layer are considered as one capsule, and the upper layer features are considered as another capsule. Another limitation of a simple attention network is the incapability to handle long-range dependencies. The syntactical structure of a sentence plays a crucial role in resolving the issue of long-range dependencies. However, most deep learning-based methods have not leveraged the syntactical structure of the sentence.