applied sciences Article  
Efficient Detection of Irrelevant User Reviews Using Machine Learning  
[PERSON]\({}^{\copyright}\) and [PERSON],*()  
Footnote 11: footnot  
for Daewangam based on the reviews collected online. If irrelevant reviews similar to the above examples are included in the analysis, irrelevant expressions such as Chinese, restaurant, cafe and Haragum will be recommended as keywords for Daewangam, which will confuse users. The example shows that it is essential to detect and filter out irrelevant reviews prior to the data analysis. Irrelevant reviews can be viewed as a sub-type of spam reviews discussed in literature. [PERSON] and [PERSON] [4] introduced three types of spam reviews, as follows:  
* Type 1 (untrustful opinions): those that deliberately mislead readers by providing undeserving positive reviews or by giving malicious negative reviews. * Type 2 (reviews on brands only): those that do not provide any useful information on the target products in reviews, but only the brands or manufacturers of the products; these reviews are considered as spam because they are not targeted at specific products or places. * Type 3 (non-reviews): those that are non-reviews which have two main sub-types: advertisement and other irrelevant reviews containing no opinions. [PERSON] and [PERSON] discussed that Type 2 and 3 reviews are easier to identify compared to Type 1 reviews. From the discussion, most follow-up studies have focused on the detection of Type 1 reviews [5; 6; 7]. Regarding Type 2 and 3, studies have not been conducted sufficiently. On the other hand, irrelevant reviews can easily be seen nowadays, as data analysis and opinion mining have been actively conducted in various domains. Many applications, such as POI (Point-of-Interest) recommendation and sentiment analysis, require the analysis of a large amount of user reviews and use open search APIs to collect them [8; 9; 10]. Regarding this, [PERSON] and [PERSON] [11] recently discussed the detection of irrelevant reviews using supervised learning, but they did not consider various combinations of machine learning algorithms, which is essential to build an efficient model. In this paper, we discuss a method to detect irrelevant user reviews efficiently by combining various oversampling and machine learning algorithms. About 35,000 user reviews collected from 25 restaurants and 33 tourist attractions in Ulsan Metropolitan City, South Korea, were used for learning. The ratio of irrelevant reviews from the restaurants and tourist attractions was 53.7% and 71.6%, respectively. To deal with data skewness, Random Undersampler [12], Random Oversampler [13], SMOTE (Synthetic Minority Oversampling Technique) [14], Borderline-SMOTE [15], and ADASYN (Adaptive Synthetic Sampling Approach) [16] were used. To build a model for the detection of irrelevant reviews, machine learning algorithms, including RNN (Recurrent Neural Network) [17], LSTM (Long Short-Term Memory) [18], GRU (Gated Recurrent Unit) [19], and BERT (Bidirectional Encoder Representation from Transformer) [20] were adopted. These algorithms are known to provide high accuracy in text processing, which was discussed in [21]. By combining the data imbalance processing algorithms and machine learning algorithms together, a total of 24 models were implemented and compared for the experiments. The contributions of this paper can be summarized as follows:  
* Unlike existing studies focusing on the detection of Type 1 reviews, the proposed method discussed how to detect Type 3 reviews efficiently using machine learning algorithms. * To ensure that the detection model properly reflects the characteristics of real data, 35,000 user reviews, collected from 25 restaurants and 33 tourist attractions in Ulsan, South Korea, were used for learning. * Various combinations of oversampling and machine learning algorithms were considered in order to build an effective model to detect irrelevant reviews efficiently. A total of 24 models were implemented and compared to find out the best performing model.