## 3 Experiment
### Experiment Results
#### 3.3.4 Performance Validation  
To validate the effectiveness of the proposed method in refining a user's opinion, the recommendation results of the test set were investigated in detail. First, we observed the changes in frequencies of like and dislike between the original and the estimated feedback ratings. Then, we conducted sentiment analysis on the review text of the test set and examined the sentiment scores of review texts according to the original and estimated feedback ratings.  
Figure 9a,b show the frequency distributions of user feedback logs of two binary rating categories--like and dislike--in the test set according to the original and estimated feedback ratings, respectively. It was observed that the frequencies of dislike (filled box) whose ratings were 1 and 2 increased in Figure 9b compared to the original feedback ratings in Figure 9a, while the portion of dislike (filled box) whose ratings were 4 and 5 decreased in Figure 9b compared to the original feedback ratings in Figure 9a. For more precise analysis, the percentages of user feedback logs with respect to the ordinal (feedback) ratings and the binary ratings were shown in Table 5. The percentage, where the ordinal ratings were 1 or 2 and the binary ratings were dislike, increased from 39% (original ordinal ratings) to 76% (estimated feedback ratings), while the percentage, where the ordinal ratings were 4 or 5 and the binary ratings were dislike, decreased from 37% to 13%. This implies that the proposed method was successful at refining user opinions, especially for negative ones, by substituting ordinal ratings with binary ratings when the binary rating was dislike for building a feedback rating matrix.  
We assumed that if the proposed method performed successfully, the estimated feedback ratings would be positively correlated with the sentiment of user opinions contained in review texts. In other words, the sentiment score should be high (or low) when the feedback rating is high (or low). Thus, the sentiment analysis mentioned in Section 3.2 was conducted on the review text of the user feedback logs  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{5}{c}{**Binary Rating**} \\ \cline{2-6}  & \multicolumn{3}{c}{**Like**} & \multicolumn{3}{c}{**Dislike**} \\ \cline{2-6}  & \multicolumn{3}{c}{**Original**} & **Estimated** & **Original** & **Estimated** \\ \hline Ordinal (feedback) & 1, 2 & 7.69\% & 10.65\% & 39.53\% & 76.00\% \\ rating & 4, 5 & 80.77\% & 83.56\% & 37.21\% & 12.50\% \\ \hline \hline \end{tabular}
\end{table}
Table 5: Percentage of user feedback logs of two binary rating categories, like and dislike, when the original ordinal ratings and estimated feedback ratings were 1 or 2 and 4 or 5.  
Figure 9: Frequency distributions of user feedback logs with respect to the two categories of binary ratings, like (empty) and dislike (filled), according to the **(a)** original ordinal ratings and **(b)** estimated feedback ratings.  
in the test set, and compared the results between original and estimated feedback ratings. Figure 10 shows the surface plots illustrating the frequency distributions of user feedback logs according to the sentiment score (\(x\)-axis) and the original and estimated feedback ratings (\(y\)-axis).  
By the assumption, a surface plot having an uplifted surface near the diagonal from \((-2,1)\) to \((2,5)\) and caved-in near the vertices, \((2,1)\) and \((-1,5)\), represents the optimal estimation of feedback ratings. In Figure 9(a), it can be observed that the rear surface of the diagonal protruded and the front of the diagonal caved in, and Pearson correlation coefficient between the original ordinal ratings and the sentiment score was 0.46. On the other hand, there was a noticeable uplift near the diagonal in the case of the estimated feedback ratings as shown in Figure 9(b), and the correlation coefficient was 0.67. The increased correlation indicates that the proposed method appropriately refined the user's opinion by using binary ratings.