# Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification
## 4 Experiments
### Experimental Results
#### 4.4.1 Overall Performance  
As shown in Table 2, the experimental results present all discussed models in this work for text sentiment classification on MR, SST, IMDB, and Yelp2013, respectively. We can draw the observations from Table 2 as follows. For sentence-level sentiment classification, the experimental results are listed in the left half of Table 2. Firstly, we observe that the traditional machine learning-based representation model (i.e., SVM) performs poorly in sentence-level sentiment classification. The main reason is that statistical machine learning classifier only focuses on word frequency features without considering any textual semantics. Secondly, compared with SVM on two datasets, LSTM improves the accuracy \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multicolumn{2}{c}{**MR**} & \multicolumn{2}{c}{**SST**} & \multicolumn{2}{c}{**IMDB**} & \multicolumn{2}{c}{**Yelp2013**} \\ \cline{2-9}  & **Acc.** & **F1** & **Acc.** & **F1** & **Acc.** & **F1** & **Acc.** & **F1** \\ \hline SVM & 76.4 & 78.8 & 40.7 * & 42.4 & 79.8 & 81.7 & 57.6 & 59.4 \\ CNN & 81.5 * & 82.7 & 48.0 * & 49.3 & 87.0 * & 88.3 & 59.7 * & 61.4 \\ LSTM & 77.4 * & 79.6 & 46.4 * & 47.6 & 86.2 * & 88.7 & 62.0 & 63.6 \\ CNN-LSTM & - & - & - & - & 89.5 * & 90.2 & - & - \\ Tree-LSTM & 80.7 * & 82.2 & 50.1 * & 51.8 & 88.0 * & 89.3 & 64.6 & 66.9 \\ LR-Bi-LSTM & 82.1 * & 83.6 & 50.6 * & 52.3 & - & - & - & - \\ LSTM-GRNN & - & - & - & - & 88.2 & 89.5 & 65.1 * & 67.2 \\ LE-LSTM & 80.8 * & 82.4 & - & - & **89.6 * & **90.9** & 60.6 * & 62.7 \\ HAN & - & - & - & - & 88.6 & 89.8 & 68.2 * & 70.1 \\ Self-attention & 81.7 & 82.9 & 48.9 & 50.1 & - & - & - & - \\ H-LAN & - & - & - & - & 89.5 & 90.6 & **68.7** & **70.8** \\ LAN & **83.9** & **85.0** & **51.3** & **52.9** & - & - & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation results (%) of our models and state-of-the-art models on different datasets. The best result for each dataset is in bold. Results marked with # are retrieved from the references (i.e., [17; 29; 31; 35; 37; 44; 49]), while those unmarked with # are obtained either by our own implementation or with the same codes shared by the original authors. SVM: Support Vector Machine; CNN: convolutional neural network; Tree-LSTM: tree-structured long short-term memory; LR-Bi-LSTM: linguistically regularized-based bidirectional long short-term memory; LE-LSTM: lexicon-enhanced LSTM; LSTM-GRNN LSTM-based word encoder and a GRU-based sentence encoder; HAN: hierarchical attention network. by roughly 3.0%. Similarly, the accuracy and F1 of CNN on two datasets are increased by roughly 6.2% and 5.4%, respectively. The results show that the semantic structure learned by deep neural network is helpful to the performance of text sentiment analysis. Furthermore, when incorporating external knowledge into deep neural network, LR-Bi-LSTM improves upon the basic performance of LSTM by 4.5% on accuracy and 4.4% on F1 averagely. Tree-LSTM is slightly weaker than CNN on the MR dataset but shows a promising result on the fine-grained SST dataset. Our LAN model is the same as the spirit of LR-Bi-LSTM, but with a simple attention mechanism to capture salient sentiment-related words in sentences rather than relying on complex parsers like LR-Bi-LSTM. As can be seen from this table, the proposed LAN model outperforms LR-Bi-LSTM on two datasets by roughly 1.3% on accuracy and roughly 1.0% on F1. Our LAN model outperforms LE-LSTM on MR dataset by 3.1% on accuracy and 2.6% on F1, they all use the sentiment lexicon as external knowledge, which verifies that LAN combines the sentiment lexicon with attention mechanism to generate more efficient sentiment-based word embedding. Moreover, it is worth noting that deep neural network with attention mechanism (i.e., Self-attention) is comparable with CNN. Compared with Self-attention, our LAN model achieves a relatively better performance, which indicates that deep neural network with our proposed attention mechanisms can enrich the representation of sentences.