# An Effective ELECTRA-Based Pipeline for Sentiment Analysis of Tourist Attraction Reviews
## 1 Introduction  
In recent years, review texts have provided a novel source of data for travel research. Review texts contain insightful feedback that is spontaneously provided by users. Online travel review texts are naturally considered as one of the most essential sources of user opinion. It is widely understood that reputation plays an important role for informing and affecting user decisions. User-generated Internet content, such as reviews, facilitates this process. Therefore, it is very important to analyze user opinions from data such as online reviews. The growing number of user reviews is the most readily available source of the opinions of crowds. However, the analysis of such opinions from reviews, especially from websites with a large number of users worldwide, is challenging.  
Sentiment analysis has been introduced to discover knowledge through user reviews. Sentiment analysis in review texts is a rapidly growing field of study and application, and has been applied in the domain of tourism aswell. However, compared to many other domains of sentiment analysis, online travel reviews are usually short texts, written informally--meaning the writer uses slang, misspelled words, and emoticons. Travel is a low-frequency activity, and travel frequency among users approximates [PERSON]'s law from a statistical perspective. The sparsity of noisy short texts, coupled with uneven sentiment distribution, makes it difficult to obtain ideal sentiment analysis results. Although several works have recognized these problems and treated the sentiment analysis of travel review texts as a domain specific classification problem [1; 2; 3; 4], they use either shallow machine learning models that lack the ability to extract deep semantic features or use non-pretrained deep learning models that heavily depend on a large amount of labeled text. It is still possible to improve the classification results of sentiment analysis of travel review texts.  
Our research objectives mainly focus on the following two aspects:  
First, we plan to carry out a series of pre-processing procedures to transform tourist attraction reviews into text that is more appropriate for learning emotional characteristics to ensure the performance gain of learning models. Second, we aim to use a pre-trained model, namely, ELECTRA, as a smart classifier to work on a sentiment analysis task based on tourist attraction reviews together with pre-processing procedures. This is due to ELECTRA's more efficient pre-training process and better performance on small data sets. To verify our approach, we build an expert-annotated review dataset in Chinese related to tourist attractions.  
Overall, we provide resources and benchmark model for those who use NLP technology to create sentiment analysis of tourist attraction review.  
In summary, this paper has the following contributions:  
* We propose a pre-trained pipeline model for the sentiment analysis of tourist attraction reviews, which can exploit the gains of pre-trained language models and which outperforms other baseline models.
* We develop annotation specifications and manually construct a Chinese tourist attraction review dataset to fill the research gap.
* We conduct a detailed comparison between our model and other baseline models in terms of performance evaluation and model evaluation, with an additional ablation experiment. The discoveries from these analyses can promote the development of sentiment analysis of other reviews.