## 4 Experimental Results
### Balanced Accuracy  
Table 12 shows the mean and standard deviation of the models' balanced accuracy, obtained from 25 review data sets for the restaurants. The results show the same pattern as the F1 scores in Table 10. In terms of data imbalance processing algorithms, the models using the original data presented the best performance, with an average score of 0.961, while the performance of models with ADASYN was the worst, with an average of 0.932. In terms of the machine learning algorithms, the BERT model provided the best performance with an average of 0.962, while the LSTM model performed the worst, with an average of 0.912. Among the individual models, the LSTM and GRU models with the original data provided the highest accuracy, at 0.967.  
Table 13 compares the mean and standard deviation of the models' balanced accuracy, obtained from 33 review data sets for the tourist attractions. The results also show the same pattern as the F1 scores in Table 11. Among the data imbalance processing algorithms, the models using the original data presented the best performance, with an average score of 0.933, while the performance of models with ADASYN was the worst, with an average of 0.901. Among the machine learning algorithms, the BERT model provided the best performance, with an average of 0.938, while the RNN model performed the worst, with an average of 0.891. Among the individual models, the BERT model using SMOTE presented the highest F1 score, at 0.940.  
Figure 9 compares the balanced accuracy of the models in terms of data imbalance processing. The results show the same pattern as the F1 scores in Figure 8. The only difference is that the accuracy of a model is less than the F1 score of that model.  
\begin{table}
\begin{tabular}{l c c c c c} \hline  & **RNN** & **LSTM** & **GRU** & **BERT** & **Average** \\ \hline Original data & 0.916 \(\pm\) 0.028 & 0.940 \(\pm\) 0.027 & 0.936 \(\pm\) 0.029 & 0.938 \(\pm\) 0.026 & 0.933 \(\pm\) 0.027 \\ RandomOverSampler & 0.919 \(\pm\) 0.025 & 0.928 \(\pm\) 0.020 & 0.939 \(\pm\) 0.020 & 0.939 \(\pm\) 0.023 & 0.931 \(\pm\) 0.022 \\ SMOTE & 0.874 \(\pm\) 0.035 & 0.912 \(\pm\) 0.033 & 0.908 \(\pm\) 0.037 & 0.940 \(\pm\) 0.024 & 0.909 \(\pm\) 0.032 \\ Borderline-SMOTE & 0.876 \(\pm\) 0.029 & 0.911 \(\pm\) 0.036 & 0.905 \(\pm\) 0.036 & 0.937 \(\pm\) 0.027 & 0.907 \(\pm\) 0.032 \\ ADASYN & 0.871 \(\pm\) 0.037 & 0.895 \(\pm\) 0.043 & 0.901 \(\pm\) 0.033 & 0.937 \(\pm\) 0.025 & 0.901 \(\pm\) 0.034 \\ Average & 0.891 \(\pm\) 0.031 & 0.917 \(\pm\) 0.032 & 0.918 \(\pm\) 0.031 & 0.938 \(\pm\) 0.025 & 0.916 \(\pm\) 0.030 \\ \hline \end{tabular}
\end{table}
Table 13: Mean and standard deviation of the models’ balanced accuracy, obtained from 33 review data sets for the tourist attractions in Ulsan, where the ratio of irrelevant reviews is about 71.6%.  
\begin{table}
\begin{tabular}{l c c c c c} \hline  & **RNN** & **LSTM** & **GRU** & **BERT** & **Average** \\ \hline Original data & 0.950 \(\pm\) 0.030 & 0.967 \(\pm\) 0.019 & 0.967 \(\pm\) 0.019 & 0.961 \(\pm\) 0.025 & 0.961 \(\pm\) 0.023 \\ RandomOverSampler & 0.952 \(\pm\) 0.025 & 0.903 \(\pm\) 0.094 & 0.946 \(\pm\) 0.050 & 0.962 \(\pm\) 0.025 & 0.941 \(\pm\) 0.048 \\ SMOTE & 0.954 \(\pm\) 0.027 & 0.903 \(\pm\) 0.092 & 0.949 \(\pm\) 0.040 & 0.963 \(\pm\) 0.023 & 0.942 \(\pm\) 0.045 \\ Borderline-SMOTE & 0.951 \(\pm\) 0.027 & 0.907 \(\pm\) 0.079 & 0.942 \(\pm\) 0.054 & 0.963 \(\pm\) 0.023 & 0.941 \(\pm\) 0.046 \\ ADASYN & 0.947 \(\pm\) 0.030 & 0.880 \(\pm\) 0.113 & 0.941 \(\pm\) 0.048 & 0.959 \(\pm\) 0.029 & 0.932 \(\pm\) 0.055 \\ Average & 0.951 \(\pm\) 0.028 & 0.912 \(\pm\) 0.079 & 0.949 \(\pm\) 0.042 & 0.962 \(\pm\) 0.025 & 0.943 \(\pm\) 0.043 \\ \hline \end{tabular}
\end{table}
Table 12: Mean and standard deviation of the models’ balanced accuracy, obtained from 25 review data sets for the restaurants in Ulsan, where the ratio of irrelevant reviews is about 53.7%.