## 4 Experimental Results
### Precision  
We first checked the models' precision from the perspective of data imbalance processing algorithms and machine learning algorithms. Table 6 shows the mean and standard deviation of the models' precision, obtained from 25 review data sets for the restaurants in Ulsan, where the ratio of irrelevant reviews is about 53.7%. Each score in the table represents an average value for the 25 data sets, with five runs performed for each data set. This also applies to the experimental results shown in the below tables discussing the models' recall, F1 scores, and balanced accuracy, which were obtained from the same review data sets for the restaurants. The precision of the RandomUnderSampler was too low, less than 0.5 or even 0 depending on the models, so its scores were excluded. In terms of data imbalance processing algorithms, the models using the original data presented the best performance, with an average precision of 0.963, while the performance of the models with ADASYN was the worst, with an average of 0.940. In terms of machine learning algorithms, the BERT model provided the best performance, with an average of 0.963, while the LSTM model performed the worst, with an average of 0.919. Among the \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & **RNN** & **LSTM** & **GRU** & **BERT** & **Average** \\ \hline Original data & 0.965 \(\pm\) 0.026 & 0.966 \(\pm\) 0.018 & 0.967 \(\pm\) 0.017 & 0.962 \(\pm\) 0.020 & 0.963 \(\pm\) 0.020 \\ RandomOverSampler & 0.951 \(\pm\) 0.023 & 0.910 \(\pm\) 0.099 & 0.946 \(\pm\) 0.052 & 0.965 \(\pm\) 0.019 & 0.943 \(\pm\) 0.048 \\ SMOTE & 0.955 \(\pm\) 0.024 & 0.911 \(\pm\) 0.096 & 0.950 \(\pm\) 0.041 & 0.962 \(\pm\) 0.019 & 0.944 \(\pm\) 0.045 \\ Borderline-SMOTE & 0.953 \(\pm\) 0.024 & 0.912 \(\pm\) 0.087 & 0.941 \(\pm\) 0.059 & 0.963 \(\pm\) 0.019 & 0.942 \(\pm\) 0.047 \\ ADASYN & 0.954 \(\pm\) 0.026 & 0.897 \(\pm\) 0.099 & 0.943 \(\pm\) 0.053 & 0.966 \(\pm\) 0.022 & 0.940 \(\pm\) 0.050 \\ Average & 0.954 \(\pm\) 0.025 & 0.919 \(\pm\) 0.080 & 0.949 \(\pm\) 0.044 & 0.963 \(\pm\) 0.020 & 0.946 \(\pm\) 0.042 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Mean and standard deviation of the models’ precision, obtained from 25 review data sets for the restaurants in Ulsan, where the ratio of irrelevant reviews is about 53.7%. individual models, the GRU model with the original data provided the highest precision, at 0.967. Table 7 shows the mean and standard deviation of the models' precision, obtained from 33 review data sets for the tourist attractions in Ulsan, where the ratio of irrelevant reviews is about 71.6%. Each score in the table represents an average value for the 33 data sets, with five runs performed for each data set. This also applies to the experimental results, shown in the below tables discussing the models' recall, F1 scores, and balanced accuracy, which were obtained from the same review data sets for the tourist attractions. The precision of the RandomUnderSampler was too low, less than 0.5 or even 0 depending on the models, so its scores were excluded. In terms of data imbalance processing algorithms, the average precision of the algorithms was similar, ranging from 0.965 to 0.969. Among the machine learning algorithms, the GRU model provided the best performance, with an average of 0.973, while the RNN model performed the worst, with an average of 0.961. Among the individual models, the GRU model with Borderline-SMOTE provided the highest precision, at 0.976. Figure 2 compares the precision of the models in terms of data imbalance processing algorithms. The left shows the models' precision, obtained from 25 review data sets for the restaurants. The LSTM model was the most affected by oversampling, with its precision dropping from 0.966 to a maximum of 0.897. The precision of the GRU model also dropped from 0.967 to a maximum of 0.941 after oversampling. On the other hand, there was no significant difference in the precision of the models obtained from the 33 review data sets for the tourist attractions, as shown in the figure on the right.