# Fusion Text Representations to Enhance Contextual Meaning in Sentiment Classification
## 5 Conclusions  
In this study, we propose a fusion text representation model, GloWord_biGRU, to enhance the performance of sentiment classification on online reviews. Our proposed model aims to improve sentiment classification performance by using context-independent word embeddings, namely word2 vec and GloVe. The choice of this model can improve text representation quality with low computational cost. By combining these two pre-trained word embedding models, we can enhance the contextual meaning of the text, thereby improving the performance of sentiment classification. We used movie review data from the IMDB dataset to evaluate our proposed model. We employed biGRU as the model for sentiment classification. Various scenarios were tested to demonstrate that our proposed model performs better, including the influence of the number of units in the biGRU, GloVe dimensions, dense layer units, dropout rate, L2 regularization rate, different types of deep learning models, and comparisons with previous research. With these numerous evaluation scenarios, our results showed that the proposed model achieved the best performance among all, with an accuracy of 89.96% and an F1 score of 90.21%. Meanwhile, we used the cross-validation (CV) technique to reduce the occurrence of overfitting, and the average F1 score obtained after performing 5-fold cross-validation was 89.21%. This value is still higher compared to several sentiment classification models conducted previously. Although our proposed model achieved the best performance in this study, further improvements are necessary. For example, we could enhance text representation quality by combining context-independent embeddings with transformers while maintaining low computational cost. Additionally, exploring other models beyond deep learning, such as graph models or capsule networks, is essential. Furthermore, incorporating diverse datasets from multiple domains would provide a more comprehensive evaluation of model robustness and generalization capabilities, allowing us to analyze performance across various contexts. These considerations will guide our future research. Conceptualization, K.W.T. ; methodology, K.W.T. ; software, E.M.D. ; validation, J.H. ; formal analysis, K.W.T. ; investigation, H.L. ; resources, H.L. ; data curation, K.W.T. ; writing--original draft preparation, K.W.T. ; writing--review and editing, J.H. ; visualization, K.W.T. ; supervision, J.H. ; project administration, J.H. ; funding acquisition, K.W.T. All authors have read and agreed to the published version of the manuscript. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1** \\ \hline SVM [36] & 83.11 & 83.29 & 82.66 & 82.98 \\ Naive Bayes [36] & 78 & 89.87 & 72.30 & 80.14 \\ SentiMI [37] & 84 & 83.2 & 84.4 & 83.9 \\ LSTM-DNN [38] & 88.46 & 87.6 & 89.1 & 88.1 \\ FARNN-ATT [39] & 89.22 & - & - & - \\ WALE-LSTM [40] & 89.50 & - & - & - \\ Bert+biGRU [41] & 87.2 & 87.88 & 86.52 & 87.2 \\ Albert+biGRU [41] & 87.09 & 87.89 & 86.27 & 87.07 \\ GloWord\_biGRU & **89.96** & 88.7 & 91.76 & **90.21** \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Result comparison with previous models of sentiment classification. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Number of Fold** & **Accuracy** & **Precision** & **Recall** & **F1** \\ \hline Fold 1 & 87.83 & 94.17 & 80.85 & 87 \\ Fold 2 & 90 & 89.69 & 90.27 & 89.98 \\ Fold 3 & 89.9 & 88.29 & 92 & **90.11** \\ Fold 4 & 89.32 & 89.02 & 89.67 & 89.34 \\ Fold 5 & 89.79 & 90.97 & 88.33 & 89.63 \\ Avg. & 89.37 & 90.43 & 88.22 & **89.21** \\ \hline \hline \end{tabular}
\end{table} 
Table 11: The results of the 5-fold cross-validation. Funding: This research received no external funding. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable.