# Fusion Text Representations to Enhance Contextual Meaning in Sentiment Classification
## 4 Experiments and Results
### Experimental Results and Discussions
The model we proposed achieved the best performance with a dropout rate of 0.4, maintaining accuracy and F1 score values identical to the previous ones at 89.96% and 90.21%, respectively. Another parameter we evaluated is the impact of L2 regularization, which is used here to address overfitting issues. We created several scenarios to examine our proposed model with different regularization values to determine the optimal performance. We conducted evaluations by setting the regularization values ranging from 0.1 to 0.0001. The best results were obtained with a regularization value of 0.001, which is the initial setting for our proposed model. The details of the results can be seen in Table 6.  
In addition to analyzing the impact of hyperparameters, we also evaluated the use of other deep learning models, namely CNN, LSTM, GRU, and biLSTM. We assessed the performance of the GloWord fusion embedding with these deep learning models to verify that our proposed model performs best. We can see the details in Table 7. After simulating the alternative deep learning models, we found that our proposed GloWord model performs best when using the biGRU model. This conclusion is further supported by evaluating the loss values of each model, where GloWord_biGRU achieved the lowest loss value of 0.2512 compared to the other models. In our proposed model, the biGRU model leverages the combined embeddings to analyze word relationships in both directions (left-to-right and vice versa). With a richer representation from both embeddings, the model can more effectively capture sentiment patterns within text sequences, supporting enhanced performance in sentiment classification compared to using one embedding type. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1** \\ \hline GloWord\_biGRU\_0.001 & **89.96** & **88.7** & **91.76** & **90.21** \\ GloWord\_biGRU\_0.01 & 89.46 & 87.28 & 92.58 & 89.85 \\ GloWord\_biGRU\_0.1 & 89.31 & 87.1 & 92.48 & 89.71 \\ GloWord\_biGRU\_0.0001 & 89.74 & 88.49 & 91.55 & 89.99 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Impact of different L2 regularization rates. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1** \\ \hline GloWord\_biGRU\_32 & 89.42 & 86.45 & 93.69 & 89.92 \\ GloWord\_biGRU\_64 & 89.8 & 88.49 & 91.68 & 90.06 \\ GloWord\_biGRU\_128 & **89.96** & **88.7** & **91.76** & **90.21** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Impact of different unit of biGRU. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1** \\ \hline GloWord\_biGRU\_0.1 & 89.58 & 92.15 & 86.7 & 89.35 \\ GloWord\_biGRU\_0.2 & 89.84 & 90.19 & 89.58 & 89.88 \\ GloWord\_biGRU\_0.3 & 89.65 & 87.09 & 92.29 & 90.08 \\ GloWord\_biGRU\_0.4 & **89.96** & **88.7** & **91.76** & **90.21** \\ GloWord\_biGRU\_0.5 & 88.41 & 93.42 & 82.83 & 87.81 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Impact of different dropout rates. In Figure 2, we present the performance history of the training and validation data throughout the training process, comparing our proposed model with other deep learning models. The graph showing accuracy performance during training data usage indicates that the accuracy of our proposed model (shown in the graph as biGRU training) competes closely with that of the biLSTM deep learning model. This is evident from Table 7, which shows that the accuracy difference between using biGRU and biLSTM is approximately 0.68%. However, this can be further examined through the F1 scores obtained by both models. When using the biLSTM deep learning model, the F1 score is 89.03%, whereas when using biGRU, the F1 score is 90.21%. Additionally, the loss value produced using biGRU is significantly lower at 0.2512. Moreover, biGRU has lower computational requirements compared to biLSTM, as the biGRU model uses fewer parameters.