# Social Network Sentiment Analysis Using Hybrid Deep Learning Models
## 4 Experiments and Results
### Optimum Hyperparameter Setting  
For the choice of the hyperparameters, the neurons and kernel size of the convolutional layer and the neurons, dropout rate, and recurrent dropout rate of the LSTM layer were adjusted. According to the Keras documentation [42], LSTM layers have two different types of dropout rates, which are represented as a floating point number between 0 and 1. To evaluate the results, different tests were carried out, with the nine best combinations shown in Table 2 as an example of the tests performed. As shown in Table 2, the best values of the hyperparameters after the different tests to achieve the best accuracy (74.13%) are as follows:  
* Neurons in the Convolutional layer: 192;
* Neurons in the LSTM layer: 96;
* Dropout rate in the convolutional layer: 0.2;
* Dropout rate in the LSTM layer: 0.3;
* Kernel size: 8. However, there are two ways to load the embeddings in the process of creating deep neural network models for the word processing task. The first option is that the embeddings are learned as the network is trained with information from the corpus itself. The other option, which is the one employed in our model, is to use pre-trained word embeddings, that is, instead of the weights being adjusted with the corpus itself, they are matched \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{5}{c}{**Large Hybrid Model**} \\ \hline \hline \multicolumn{5}{c}{**Cross-Validation**} \\ \multicolumn{5}{c}{**Hybrid Model**} \\ \multicolumn{5}{c}{**Combination**} & **Optimiser** & **Loss Function** & **n = 33** & **n = 200** \\ \hline Convolutional + LSTM & Adam & Binary * & 73.14\% & 73.18\% \\ Convolutional + LSTM & Adam & Categorical \(\dagger\) & 73.24\% & 73.20\% \\ Convolutional + LSTM & Rmsprop & Binary & 72.87\% & 72.31\% \\ Convolutional + LSTM & Rmsprop & Categorical & 72.90\% & 71.68\% \\ Convolutional + GRU & Adam & Binary & 72.11\% & 60.03\% \\ Convolutional + GRU & Adam & Categorical & 73.12\% & 55.30\% \\ Convolutional + GRU & Rmsprop & Binary & 72.10\% & 69.79\% \\ Convolutional + GRU & Rmsprop & Categorical & 72.08\% & 64.63\% \\ \hline \hline \end{tabular}  
* Binary cross-entropy. \(\dagger\) Categorical cross-entropy.  
\end{table} 
Table 1: Summary of tests performed to evaluate the hybrid model with LSTM and GRU layer. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{2}{c}{**Layer Size**} & \multicolumn{2}{c}{**Dropout Parameter**} & **Kernel** & **Cross-Validation** \\ \hline \hline
**Convolutional** & **LSTM** & **Convolutional** & **LSTM** & **Size** & **Accuracy** \\ \hline
192 & 96 & 0.2 & 0.3 & 8 & 74.13\% \\
160 & 128 & 0.2 & 0.2 & 8 & 74.10\% \\
192 & 128 & 0.2 & 0.4 & 3 & 74.03\% \\
128 & 96 & 0.4 & 0.4 & 5 & 73.99\% \\
160 & 64 & 0.3 & 0.2 & 8 & 73.90\% \\
192 & 64 & 0.3 & 0.3 & 3 & 73.81\% \\
128 & 32 & 0.4 & 0.2 & 5 & 73.77\% \\
96 & 64 & 0.3 & 0.2 & 3 & 73.66\% \\
64 & 32 & 0.3 & 0.3 & 8 & 73.61\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Summary of tests performed to assess hyperparameters, layer size (number of neurons) for the convolutional and LSTM cases and kernel size. to a defined dictionary that has been created to solve this kind of problem. Thus, the cross-validation accuracy obtained using two well-known pre-trained embedded word dictionaries in Spanish are as follows:  
* GloVe dictionary: 66.55%. * Word2 vec dictionary: 67.79%. As can be seen, the results become worse, and this is due to the fact that a generic dictionary, being created in a general way and applicable to other corpora, depends, to a large extent, on the relationship between the words in our corpus and those in that dictionary, so it will always be better to learn from something more specific and closer to our corpus directly. However, it is also possible to create a vocabulary from a clean corpus by counting the occurrences of each word in the corpus. Therefore, for the selection of vocabulary features, the Information Gain (IG) method is applied to reduce and optimise the vocabulary and keep the terms with the highest frequency.