## 4 Experimental Results
### Precision
To figure out why the performance of the LSTM and GRU models was degraded, we examined the changes in the TP and FP values of the models after oversampling was applied. Figure 3 compares the TP and FP values of the models obtained from the restaurant \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & **RNN** & **LSTM** & **GRU** & **BERT** & **Average** \\ \hline Original data & 0.960 \(\pm\) 0.017 & 0.972 \(\pm\) 0.012 & 0.969 \(\pm\) 0.014 & 0.968 \(\pm\) 0.012 & 0.967 \(\pm\) 0.014 \\ RandomOverSampler & 0.958 \(\pm\) 0.016 & 0.960 \(\pm\) 0.021 & 0.971 \(\pm\) 0.009 & 0.969 \(\pm\) 0.012 & 0.965 \(\pm\) 0.014 \\ SMOTE & 0.960 \(\pm\) 0.019 & 0.968 \(\pm\) 0.017 & 0.975 \(\pm\) 0.008 & 0.968 \(\pm\) 0.012 & 0.968 \(\pm\) 0.014 \\ Borderline-SMOTE & 0.964 \(\pm\) 0.012 & 0.970 \(\pm\) 0.014 & 0.976 \(\pm\) 0.009 & 0.968 \(\pm\) 0.015 & 0.969 \(\pm\) 0.013 \\ ADASYN & 0.962 \(\pm\) 0.022 & 0.960 \(\pm\) 0.039 & 0.974 \(\pm\) 0.013 & 0.967 \(\pm\) 0.013 & 0.966 \(\pm\) 0.022 \\ Average & 0.961 \(\pm\) 0.017 & 0.966 \(\pm\) 0.021 & 0.973 \(\pm\) 0.010 & 0.968 \(\pm\) 0.005 & 0.967 \(\pm\) 0.015 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Mean and standard deviation of the models’ precision obtained from 33 review data sets for the tourist attractions in Ulsan, where the ratio of irrelevant reviews is about 71.6%. Figure 2: Comparison of the models’ precision in terms of data imbalance processing algorithms obtained from (**a**) 25 review data sets for the restaurants and (**b**) 33 review data sets for the tourist attractions. reviews. In both the LSTM and GRU models, the TP values decreased, while the FP values increased after oversampling. This leads to the performance degradation of the two models. In general, when oversampling is applied, the ratio of a minority class increases. In our experimental data, the negative class with legitimate reviews is the minority. Thus, its ratio increases from 46.3% to 50% after oversampling. This increases the possibility that a given record in the test data set is classified as negative. As a result, the TN and FN values increase, while the TP and FP values decrease. On the other hand, in Figure 3, the FP values increased. Regarding this, more study is necessary to find out the reason in future. When data skewness is higher, both TP and FP decreased. Figure 4 compares TP and FP of the models obtained from the tourist attraction reviews. In the RNN, LSTM, and GRU models, the TP values decreased when RandomOverSampler, SMOTE, and Borderline-SMOTE were applied. In these cases, the FP values also decreased, which is in line with our expectation. The reduction rates of the TP and FP values were small, with a maximum of 0.09. As a result, no significant difference in precision was observed after oversampling. The above results regarding the models' precision can be summarized as follows:  
* When oversampling was applied for the nearly balanced data, such as the review data sets for the restaurants, the precision of the LSTM and GRU models was degraded, due to the fact that the TP values of the models decreased, while their FP values increased. * When oversampling was applied for the skewed data, such as the review data sets for the tourist attractions, there was no significant difference in the models' precision. Both the TP and FP decreased, and their reduction rates were so small. * The performance of the BERT model was not affected by oversampling algorithms. Figure 4: Comparison of TP and FP values of the models obtained from 33 review data sets for the tourist attractions where the ratio of irrelevant reviews is about 71.6%. Figure 3: Comparison of TP and FP values of the models obtained from 25 review data sets for the restaurants where the ratio of irrelevant reviews is about 53.7%.