# Research on Sentiment Classification of Online Travel Review Text
## 3 Case Study
### Sentiment Classification Experiment  
(1) Acquisition of training set classification labels  
The machine learning classification method represented by SVM requires training data with sentiment classification results for model training. The sentiment classification results for these training data are also referred to as the training set classification labels. In this study, manual analysis and sentiment analysis software were used to generate the classification labels for the training set. SentiStrength [68] is a software package that estimates the strength of positive and negative emotions contained in text. It also has an artificial level of accuracy for short social network texts in English. We chose the nine-level sentiment classification results provided by SentiStrength. For negative emotions, the scores range from \(-1\) (not negative) to \(-4\) (extremely negative); for positive emotions, moreover, the scores range from \(1\) (not positive) to \(4\) (extremely positive); \(0\) represents neutral emotion. In this study, the SentiStrength results were again scored by humans, and the adjustment rate was about 24.7%. Finally, the sentiment analysis results of the dataset are presented in Figures 7-10. The abscissa represents the sentiment analysis results, which range from \(-4\) to \(4\) in a total of nine categories; the ordinate indicates the number of samples in each category. It can be seen that the number of samples of each sentiment value is extremely uneven. For example, in the review dataset for the Mutianyu Great Wall in Beijing, China, the number of texts in category \(2\) is \(1266\), while there is only \(1\) text in category \(-4\). Moreover, there is no category \(-4\) data in the review data of the Sydney Opera House in Australia. Figure 6: Vector representation of text. Figure 8: The sentiment distribution of the review text of the Harry Potter Wizarding World Theme Park in Orlando, USA. Figure 7: The sentiment distribution of the review text of Mutianyu Great Wall in Beijing, China. Figure 9: The sentiment distribution of the review text of Tower of London. (2) Sampling experiment of unbalanced data  
From the analysis presented in the previous section, we can see that the sentiment distribution of online travel review texts is very uneven. In fact, this is a typical unbalanced dataset. For unbalanced datasets, machine learning classifiers will tend to incorrectly divide new samples into categories with more samples, resulting in classification errors [69]. The methods used to process unbalanced datasets are mainly divided into undersampling, oversampling, and improved methods [70]. This study used Python to implement two types of sampling methods. Our experimental results demonstrate that, due to the extremely uneven sentiment distribution of the experimental dataset used, the undersampling dataset was so small that it was difficult to obtain more accurate classification results. Overall, Naive Random Over Sampler (ROS) [71] achieved the best sampling results. (3) Evaluation index  
The evaluation indicators of classification results that have been adopted by academia include Accuracy, Precision, Recall, and F1 score [72]. In binary classification, the sample categories are divided into positive and negative types. Let us suppose that TP represents the number of samples that are both actually positive and classified as positive, while FP denotes the number of samples that are actually negative but classified as positive; moreover, FN represents the number of samples that are in fact positive but are classified as negative, while TN indicates the number of samples that are both actually negative and classified as negative. In addition, the accuracy rate refers to the proportion of correct samples classified as positive to the samples classified as positive.