## 4 Experiments and Results
### Sentiment Analysis Model: Training Setup
#### 4.2.1 Features  
The features used in our system are the following:  
* **Word n-grams**: appearances of \(1,2,...,n\) consecutive words.
* **Character n-grams**: appearances of \(3,4,...,n\) consecutive characters.
* **Part-Of-Speech tags**: appearances of each kind of POS.
* **Lexicon-Based features**: A sentiment lexicon associates to a word a number that represents its degree of positiveness/negativity. For each word \(w\) and polarity value \(score(w;l)\) in a given lexicon \(l\), we calculate the following features: (1) the total score \(\sum_{w}score(w;l)\); (2) the highest value \(\max_{w}score(w;l)\); (3) the number of positive words in the sentence; (4) the lowest value \(\min_{w}score(w;l)\); (5) the number of negative words  
\begin{table}
\begin{tabular}{l l l l} \hline \hline
\begin{tabular}{l} MAX \\ MAX \\ MAX \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 1: Domain and social criteria used in the case study.  
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multirow{2}{*}{**Aspect**} & \multicolumn{2}{c}{**Positive**} & \multicolumn{2}{c}{**Negative**} & \multicolumn{2}{c}{**Conflict**} & \multicolumn{2}{c}{**Neutral**} & \multicolumn{2}{c}{**Total**} \\ \cline{2-11}  & **Train** & **Test** & **Train** & **Test** & **Train** & **Test** & **Train** & **Test** \\ \hline FOOD & 867 & 302 & 209 & 69 & 66 & 16 & 90 & 31 & 1232 & 418 \\ \hline PRICE & 179 & 51 & 115 & 28 & 17 & 3 & 10 & 1 & 321 & 83 \\ \hline SERVICE & 324 & 101 & 218 & 63 & 35 & 5 & 20 & 3 & 597 & 172 \\ \hline AMBIENCE & 263 & 76 & 98 & 21 & 47 & 13 & 23 & 8 & 431 & 118 \\ \hline ANECD & 546 & 127 & 199 & 41 & 30 & 15 & 357 & 51 & 1132 & 234 \\ \hline Total & 2179 & 657 & 839 & 159 & 163 & 52 & 500 & 94 & 3713 & 1025 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Aspect categories distribution per sentiment class.  
in the sentence, and (6) the value of the last word in the sentence. The lexicons used in this work are the following:  
* Yelp Restaurant Sentiment Lexicon (Yelp-Res): this lexicon, which contains almost 40,000 unigrams, was created automatically from restaurant reviews contained in Yelp [41] ([[http://www.yelp.com/dataset_challenge](http://www.yelp.com/dataset_challenge)]([http://www.yelp.com/dataset_challenge](http://www.yelp.com/dataset_challenge))).
* NRC Hashtag Sentiment Lexicon (NRC-Hashtag): This lexicon was constructed from a pseudo-labelled corpus of tweets [41]. It contains almost 40,000 unigrams and 180,000 bigrams.
* NRC Yelp Word-Aspect Association (WA-Lexicon): It contains lexicons of words associated with the five aspect categories shown in Table 2, created from over 180,000 reviews in Yelp. Thus, unlike the other lexicons, this one is centred in the aspect categories. The value that represents the relationship between each word and each category was calculated using the PMI metric [24].
* **Word Clusters**: The CMU POS-daggerger ([[http://www.cs.cmu.edu/~ark/TweetNLP/clusters/50](http://www.cs.cmu.edu/~ark/TweetNLP/clusters/50) mpaths2]([http://www.cs.cmu.edu/~ark/TweetNLP/clusters/50](http://www.cs.cmu.edu/~ark/TweetNLP/clusters/50) mpaths2)) employed Brown clustering to divide the English words appearing in over 50 million tweets in 1000 clusters. We used the total occurrences of words from each of the clusters as features.