## 5 Model Compilation Using Deep Learning Methods
### Results and Discussion
#### 5.4.2 Impact of Hybrid Models  
We extended our experiment to build hybrid network structures, including CNN with LSTM, CNN with BiLSTM, and CNN with GRU. CNN was chosen due to its superiority over the other models. Moreover, hybrid models have widely used a feature extraction layer. The performance of the hybrid models is shown in the tables below for the two tested datasets, relating to the application of performance metrics for each model that utilized different embedding techniques where we marked the best results in bold.  
The results of the hybrid models for the Hajj dataset are presented in Table 10. As we can see that CNN+LSTM and CNN+BiLSTM performed similarly with respect to precision, recall, and the F-measure, with a slight difference in the performance of the two structures in terms of accuracy. Referring to Table 11, the accuracy of CNN+BiLSTM for the Hajj dataset was higher than for all the other models using random embedding and Word2 vec, whereas CNN+LSTM performed better with FastText.  
We observed that the performance of the three basic models was enhanced when combined with CNN. For instance, the accuracy of the three models improved from 79% to 94.4% for LSTM, from 88.8% to 93.54% for BiLSTM, and from 82.78% to 88.04% for GRU using the random embedding technique. Although the GRU performance improved when combined with CNN, all the metrics showed that the hybrid CNN+GRU yielded the lowest performance among the three hybrid models. It is worth mentioning that the performance of CNN+GRU improved significantly when using FastText embedding compared to the other embedding techniques. Furthermore, Word2 vec achieved the lowest performance  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Embedding**} & \multicolumn{3}{c}{**CNN**} & \multicolumn{3}{c}{**LSTM**} & \multicolumn{3}{c}{**BiLSTM**} & \multicolumn{3}{c}{**GRU**} \\ \cline{2-11}  & **P** & **R** & **F1** & **P** & **R** & **F1** & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline Random & 0.75 & 0.75 & **0.75** & 0.62 & 0.62 & 0.62 & 0.60 & 0.60 & 0.60 & 0.65 & 0.65 & 0.65 \\ Word2 vec & 0.58 & 0.58 & 0.58 & 0.55 & 0.55 & 0.55 & 0.64 & 0.64 & **0.64** & 0.48 & 0.48 & 0.48 \\ FastText & 0.77 & 0.77 & **0.77** & 0.65 & 0.65 & 0.65 & 0.64 & 0.64 & 0.64 & 0.64 & 0.64 & 0.64 \\ \hline \hline \end{tabular}
\end{table}
Table 7: The performance of the basic tested deep learning models using different embedding techniques for the Brexit dataset.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Embedding Model**} & \multicolumn{3}{c}{**Accuracy**} \\ \cline{2-5}  & **CNN** & **LSTM** & **BiLSTM** & **GRU** \\ \hline Random Embedding & **74.82\%** & 61.56\% & 59.86\% & 64.63\% \\ Word2 vec & 57.82\% & 55.10\% & **64.28\%** & 47.96\% \\ FastText & **76.87\%** & 64.62\% & 63.94\% & 63.61\% \\ \hline \hline \end{tabular}
\end{table}
Table 9: Accuracy of the basic tested deep learning models for the Brexit dataset.  
for all the experimental models with approximately 8-11% difference between random embedding and FastText.  
For the Brexit dataset, as shown in Table 12, the results confirmed that adding a CNN layer enhanced the performance scores achieved by the deep learning models. The accuracy score of LSTM+CNN increased by approximately 3%, 12%, and 11% using random embedding, Word2 vec, and FastText, respectively. Likewise, the accuracy of GRU+CNN increased by approximately 5%, 7%, and 3% for the three embedding techniques. On the other hand, BiLSTM+CNN achieved higher accuracy using both random embedding and FastText, but its accuracy decreased by 2% using Word2 vec embedding. Furthermore, as shown in Table 13, the accuracy of CNN+LSTM for the Brexit dataset was marginally better than CNN+BiLSTM using random embedding and FastText, while CNN+BiLSTM performed better with the pre-trained Word2 vec with an accuracy of 86.36% compared to 84.21% for CNN+LSTM.