# A Sentiment Analysis Model Based on User Experiences of Dubrovnik on the Tripadvisor Platform
## 4 Results and Discussion  
### Model Effectiveness  
This section presents the results of the _robertaSentimentFT_Tripadvisor_ model evaluation with the test dataset containing 770 randomly selected reviews from the augmented dataset with 7969 reviews in total. In the test dataset, there are 98 negative reviews, 220 neutral reviews, and 452 positive reviews. Figure 7 shows the confusion matrix for the model using data augmentation and random oversampling methods. The confusion matrix visually illustrates the effectiveness of the selected model for this multi-classification problem. Reviews can be classified by the model into one of three distinct classes: negative, neutral, and positive, denoted in the confusion matrix by 0, 1, and 2, respectively. Therefore, the confusion matrix in Figure 7 contains three rows (actual or true classes) and three columns (predicted classes). For example, by checking the first row of the confusion matrix, it can be seen that the vast majority of actual negative reviews from the test dataset (92/98) were classified by the trained model correctly, five were classified wrongly as neutral, and only one actual negative review was classified wrongly as a positive review. Similarly, in the second and third rows, there were only a handful of misclassifications of actual neutral and positive reviews. By observing the confusion matrix columns, it can be seen that the model classified 95/770 reviews as negative, 228/770 reviews as neutral, and 443/770 as positive, which is close to the actual distribution of reviews. For class 0 (negative reviews), we calculated the values of \(TP_{0}\) = 92 (92 instances were correctly classified as class 0), \(FN_{0}\) = 6 (5 + 1 class 0 instances were wrongly classified as class 1 or class 2), \(FP_{0}\) = 3 (3 + 0 class 1 and class 2 instances were wrongly classified as class 0), and \(TN_{0}\) = 669 (209 + 8 + 18 + 434 class 1 or class 2 instances were correctly classified as class 1 or class 2), which gave a false-negative rate of \(FNR_{0}\) = 6.1% and a false-positive rate of \(FPR_{0}\) = 0.44%. On the other hand, for class 1, we found the values of \(TP_{1}\) = 209, \(FN_{1}\) = 11, \(FP_{1}\) = 23, and \(TN_{1}\) = 527, resulting in the following values: \(FNR_{1}\) = 5% and \(FPR_{1}\) = 4.18%. Lastly, class 2 has \(TP_{2}\) = 434, \(FN_{2}\) = 18, \(FP_{2}\) = 9, and \(TN_{2}\) = 309, as well as \(FNR_{2}\) = 3.98% and \(FPR_{2}\) = 2.83%. Therefore, the false-negative rate was similar for all three classes, but the false-positive rate for class 1 and class 2 were much higher than that of class 0. Because of the nuances and ambiguities between the neutral and positive reviews, this model sometimes decided to classify an actual class 1 review into class 2 and vice versa.  
Table 5 provides detailed performance metrics of the model, including the accuracy, precision, recall, and \(F1\) measure. Because of the multi-classification problem, we first calculated the precision, recall, and \(F1\) score for each of the three classes, and the values presented in Table 5 are their average values as arithmetic means. The performance of the model is given for both raw and preprocessed data using data augmentation and/or random oversampling methods. An analysis of the obtained results showed that the model performs excellently in recognizing sentiments in reviews when both random oversampling and data augmentation were applied. In this case, all four performance metrics had a high value of 95% (highlighted in bold in Table 5), which can be also verified from the confusion matrix presented in Figure 7. However, it must be noted that the individual precision values for classes 0, 1, and 2 vary between 90.09% for class 1, 96.84% for class 0, and 97.97% for class 2. On the other hand, the individual recall values for classes 0, 1, and 2 are quite similar--93.88% for class 0, 95% for class 1, and 96.02% for class 2.