# Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification
## 4 Experiments
### Datasets and Evaluation Metrics  
In order to verify the effectiveness of our proposed models, we choose two sentence-level and document-level sentiment classification datasets to conduct our experiments. The detailed statistics of all the datasets are given in Table 1. The first two datasets are sentence-level datasets and the last two are document-level datasets.  
* Movie Review (MR) collected by Pang and Lee [43] is an authoritative movie review dataset for binary sentiment classification task, in which each review contains only one sentence. The corpus has 5331 negative samples and 5331 positive samples.
* Stanford Sentiment Treebank (SST) is an extended sentence-level version of the MR dataset proposed by Socher et al. [44], where each sentence is annotated as very positive, positive, neutral, negative, or very negative. To effectively avoid the poor generalization ability caused by the uneven distribution of samples, this dataset follows the same sample distribution of the MR dataset.
* Internet Movie Database (IMDB) proposed by Maas et al. [45] is a representative dataset for document-level sentiment classification task, which contains a large amount of movie reviews and recommendations with binary classes (i.e., positive and negative). It is made up of 50,000 movie reviews, and each movie review has several sentences.
* Yelp2013 obtained from the Yelp Dataset Challenge in 2013 [46], is another popular document-level sentiment classification dataset, which includes large-scale restaurant reviews labeled with five levels of ratings from 1 to 5 (i.e., very positive, positive, neutral, negative, and very negative).  
For the experimental evaluation, we adopt Accuracy and F-measure as the metrics to measure the performance of text sentiment classification. Specifically, the Accuracy metric is the standard competition criteria used in most NLP tasks at present, which reflects the difference between the measured result and the target value. The specific calculation process is as follows:  
\[\text{Accuracy }=\frac{\sum_{i=1}^{k}f(\textit{predict}(i),\textit{ ground\_truth}(i))}{k} \tag{33}\]  
where \(k\) is the total number of test samples, \(f(a,b)\) is a sign function (i.e., \(f(a,b)\ =\ 1\) when \(a\) is equal to \(b\); otherwise, \(f(a,b)\ =\ 0\)), \(\textit{predict}(i)\) represents the predicted sentiment label for sample \(i\) and \(\textit{ground\_truth}(i)\) corresponds to the true value of sample \(i\).  
In order to evaluate the experiment from a comprehensive perspective, the F-measure is introduced to measure the accuracy of the assessment analysis, which returns the weighted harmonic average value of Recall and Precision. The concrete calculation formula is as follows:  
\[F-\textit{measure }=\frac{2\ \times\ \textit{Recall}\ \times\ \textit{Precision}}{\textit{Recall}+\textit{Precision}} \tag{34}\]  
where _Recall_ is the number of samples correctly categorized divided by the total number of samples, and _Precision_ is the number of samples correctly categorized divided by the total number of samples predicted to be a certain category.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline
**Dataset** & **Type** & **C** & **L** & **Train** & **Dev** & **Test** & **V** \\ \hline MR & Sentence & 2 & 21 & 10,662 & - & CV & 20,191 \\ SST & Sentence & 5 & 18 & 8544 & 1101 & 2210 & 17,836 \\ IMDB & Document & 2 & 126 & 25,000 & - & 25,000 & 106,249 \\ Yelp2013 & Document & 5 & 179 & 62,522 & 7773 & 8671 & 73,691 \\ \hline \end{tabular}
\end{table}
Table 1: Summary statistics of the four mainstream datasets for text sentiment classification task. Type: level of text sentiment classification task, C: number of target classes, L: average sentence length, Train/Dev/Test: train/development/test set size, V: vocabulary size, CV: 10-fold cross validation, MR: Movie Review; SST: Stanford Sentiment Treebank; IMDB: Internet Movie Database; Yelp2013: Yelp Dataset Challenge in 2013.