## 3 Experiment
### Settings  
Four types of experiments were conducted to observe the performances and characteristics of the proposed method. First, the collected user feedback logs were explored to identify the data sparsity problem. Second, the impact of the proposed method's parameters on the recommendation performances were investigated to determine the optimal values of the parameters. Third, we evaluated the performances of the proposed method and compared them with the state-of-the-art method. Lastly, recommendation results were closely observed to validate the effectiveness of the proposed method in refining user opinions. Specifically, we performed sentiment analysis, the task of identifying user's opinion inherent in a text [27] to compare the results with those of the proposed method. The collected user feedback logs were partitioned into training, validation, and test set, where the training set and validation set were respectively utilized for building a feedback rating matrix and for selecting the optimal parameters of the proposed method and the test set was used for the performance evaluation of the proposed method using the feedback rating matrix. Among two hundred of the most recent logs, the oldest one hundred of the logs were selected as the validation set and the most recent one hundred of the logs were selected as the test set. Note that the feedback ratings of the test set generated by performing the proposed method were indicated as 'estimated feedback ratings' while those before performing as 'original feedback ratings'. All experiments were repeated thirty times, and the results were averaged to minimize the randomness. As evaluation measures, we adopted root mean squared error (RMSE) and mean absolute percentage error (MAPE), which are two of the most widely utilized measures for rating estimation [28; 29]. RMSE is defined as the square root of the average squared difference between the actual and the estimated feedback ratings as shown in Equation (13). \[RMSE=\sqrt{\frac{\sum_{(i,j)\in T}(f_{i,j}-\hat{f}_{i,j})^{2}}{|T|}}, \tag{13}\]  
where \(T\) is a set of index pairs \((i,j)\) in the test set and \(f_{i,j}\) and \(\hat{f}_{i,j}\) respectively indicate the original and the estimated feedback ratings from \(u_{i}\) to \(e_{j}\). In addition, MAPE is defined as the average percentage \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Type** & **Category** & **Frequency** & \begin{tabular}{l} **Combination Frequency** \\ **(Intersection, Union)** \\ \end{tabular} \\ \hline \multirow{4}{*}{\
\begin{tabular}{l} Ordinal \\ rating \\ \end{tabular} } & 1 & 32 & \\  & 2 & 46 & \\  & 3 & 77 & \\  & 4 & 293 & \\  & 5 & 878 & \((180,1348)\) & \\ \cline{2-5}  & \multicolumn{1}{l}{Total} & 1326 & \\ \hline \multirow{2}{*}{
\begin{tabular}{l} Binary \\ rating \\ \end{tabular} } & Like & 164 & \\  & Dislike & 38 & \\  & Total & 202 & \\ \hline \multicolumn{2}{l}{Review text} & 179 & - \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Summary of user feedback logs utilized in the experiments. of the absolute difference between the original and the estimated feedback ratings divided by the absolute value of the original feedback ratings, and it is calculated by Equation (14). \[MAPE=\frac{100}{|T|}\sum_{(i,j)\in T}\frac{|f_{i,j}-\hat{f}_{i,j}|}{|f_{i,j}|}. \tag{14}\]  
The smaller values of RMSE and MAPE indicate better recommendation performances. We compared the recommendation performances of the proposed method with TMF [22], which exploits ordinal and binary ratings simultaneously. Specifically, TMF estimates the unknown feedback ratings by utilizing additional user latent matrix generated by analyzing binary ratings. When calculating the additional matrix, two parameters--\(w_{l}\) and \(w_{d}\)--were used to control the weights of like and dislike information in binary ratings. The comparison between the proposed method and TMF can provide an opportunity to demonstrate the effectiveness of substituting ordinal ratings with binary ratings compared to the simultaneous utilization of the two ratings.