# An Effective ELECTRA-Based Pipeline for Sentiment Analysis of Tourist Attraction Reviews
## 3 Methodology
### Pre-Processing Procedures  
Collecting raw travel reviews from the Ctrip Travel Website (the largest Online Travel Agency in China) using a Web crawler generally results in a very noisy dataset due to the spontaneity and creativity of the posted comments. Since tourist attraction reviews reflect  
Figure 1: Pipeline overview. M is short for mask operation, and G and D are the generator and discriminator, respectively.  
the satisfaction of tourists and their evaluations of the quality of services, perceptions of the image of attractions or information provision, they are filled with many modal words and emotional words, coupled with other noisy sources, such as phone numbers, amounts of money, times, dates, addresses and questions.  
Our pre-processing procedures include the following four sub-procedures:  
1. Stopword Removal  
Stopwords (abbreviated as Stop) are the most common words typically filtered out before the classification task. Thus, we removed all the stopwords. Here, we use the \"remove-stopwords\" method in the Gensim ([[https://radimrehurek.com/gensim/index.ht](https://radimrehurek.com/gensim/index.ht)]([https://radimrehurek.com/gensim/index.ht](https://radimrehurek.com/gensim/index.ht)) ml, accessed on 24 July 2022) Library with a stopwords dictionary that integrates Chinese words from the HIT (Harbin Institute of Technology), Baidu and SCU (Sichuan University) thesauruses. Given that the removal of stopwords has a significant impact on the set of feature vectors required for classification and the classification effect [39], an optimal selection of stopwords is required. It is not preferable to have a stopwords dictionary with as many stopwords as possible, but rather, it is better to have a targeted dictionary. Since the combination of words such as exclamations, onomatopoeia and pronouns with other words in emotional texts often has a strong emotional tendency, all words other than punctuation and non-semantic words are retained in the stopwords dictionary as much as possible. The final stopwords dictionary contains 1336 stopwords. We will make the final stopwords dictionary publicly accessible for the research community.  
2. Special Character Removal  
Special characters (abbreviated as Symbol) such as.,, should be removed in order to eliminate differences when assigning polarity. Here, we use the \"remove_stopwords\" method in the Gensim Library with a symbol dictionary that integrates special characters from the HIT thesaurus. Gensim is an open source software library that uses modern statistical machine learning and is designed to process large collections of text using data streams and incremental online algorithms, unlike most other machine learning packages that target only in-memory processing. The final symbol dictionary contains 263 special characters.  
3. Redundancy Deletion  
Repeated statements may disrupt polarity distribution, and phone numbers, amounts of money, times, dates and questions do not contribute to sentiment tendencies. Thus, we delete these statements and phrases, abbreviated as Deletion. We identify redundant statements from the text using the NLTK regexptokenizer and delete them. Several regular matching examples are illustrated in Table 11 in the Appendix.  
4. Negation Substitution  
Negation substitution (abbreviated as Negation) plays a critical role, as negation words would invert the word or sentence polarity in sentiment analysis. Thus, we substitute the negation and the negated word with its antonym. First, we identify the negation words in tokenized text using a negation dictionary (contains 243 negation words). Then, the antonym of the token following the negated word is looked up in the antonym dictionary (contains 18,797 pairs). If an antonym is found, the negation word and the negated word are replaced with the antonym. For example, we replace (not happy) with (depressed).