# Combining Post Sentiments and User Participation for Extracting Public Stances from Twitter
## 4 Experiments
### Experiment on Sentiment Classification
#### 4.1.1 The Effects of Word Embedding Models on Sentiment Classification  
In this section, we tested the performance of sentiment classification using LSTM for different word embedding models on various datasets in Table 1. First, the performance for our trained Word2 Vec models is shown in Figure 3.  
As shown in Figure 3, better performance can be achieved for datasets that have sentiment information or similar topics, such as Vec-All and Vec-SemEval. The larger the data size, the better performance. Also, datasets with sentiment information are consistently better than the general datasets since the topics in general datasets are too diverse to train relevant word contexts. To check the effect of training word embedding models on sentiment classification, we further compared the performance with Twitter-based pre-trained GloVe word embedding model as in Figure 4.  
As shown in Figure 4, the performance of Twitter-based pre-trained GloVe embedding models with embedding size from 25 to 200 have been tested. First, we can observe comparable performance for pretrained GloVe embedding of size 200 with Word2 Vec models Vec-Com-200. This is reasonable  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Dataset** & **Number of Posts** & **Number of Words** \\ \hline Vec-Hillary & [PERSON] & 20,000 & 4933 \\ Vec-All & [PERSON], [PERSON] & 40,000 & 8078 \\ Vec-SemEval & [PERSON], [PERSON], and & 55,983 & 11,351 \\ Vec-Com-150 & General dataset & 1,500,000 & 74,248 \\ Vec-Com-200 & General dataset & 2,000,000 & 90,412 \\ Vec-Com-250 & General dataset & 2,500,000 & 98,926 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Datasets for training word embedding models for sentiment classification.  
Figure 4: The performance of sentiment classification using Twitter-based pretrained GloVe embedding models.  
Figure 3: The performance of sentiment classification using different Word2 Vec models.  
since pre-trained embedding models are trained from general documents, which should be similar to our Word2 Vec model trained from general datasets. Also, better performance can be obtained when we increase the embedding size. However, since pre-trained GloVe embedding is general-purpose, their performance are inferior to that of Word2 Vec models trained from datasets with sentiment information.  
As shown in Figure 3, when training Word2 Vec models from general datasets, larger data size does not necessarily increase the performance since the word contextual information was diversified when there's more irrelevant data. However, since general datasets usually provide contextual information for more terms, they might help to improve the performance of sentiment classification for unknown posts which might contain many new words that are unseen in the training data. Thus, we combine the two Word2 Vec models using linear combination as in Equation (12):  
\[Vec(w)=\alpha*Vec_{SemEval}(w)+(1-\alpha)*Vec_{gen-200}(w) \tag{12}\]  
where \(\alpha\) denotes the weight of \(Vec_{SemEval}\) model. The effects of \(\alpha\) on classification performance is shown in Figure 5, where the best performance of 71.97% can be obtained when \(\alpha=1\), which means no general corpus included. However, general corpus include unknown words which could be helpful for more diverse topics.  
Thus, to strike the balance between classification performance and the ability to cover more diverse range of words, we decided the use the value of \(\alpha\) as 0.7 since it has comparable performance (a F-measure of 71.73%) as when \(\alpha=0.8\) (F-measure of 71.76%) and 0.9 (F-measure of 71.80%), but more diverse word coverage.