## 3 Experiment
### Experiment Results
#### 3.3.3 Performance Evaluation
\begin{table}
\begin{tabular}{l l l l l} \hline \hline \multirow{2}{*}{**Type**} & \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Performance**} \\ \cline{3-5}  & & **RMSE** & **MAPE (\%)** \\ \hline \multicolumn{5}{l}{Ordinal rating only} & \(O\) & 0.2573 \(\pm\) 0.0006 & 6.4974 \(\pm\) 0.0067 \\ \hline \multicolumn{5}{l}{Binary rating only} & \(B\) & 0.4263 \(\pm\) 0.0027 & 10.7651 \(\pm\) 0.0039 \\ \hline \multirow{5}{*}{Ordinal and binary rating} & \multirow{2}{*}{Mixed} & \(\text{TMF}(1,1)\) & 0.1381 \(\pm\) 0.0009 & 3.4873 \(\pm\) 0.0055 \\  & & \(\text{TMF}(1,0)\) & 0.1744 \(\pm\) 0.0014 & 4.4040 \(\pm\) 0.0058 \\ \cline{1-1} \cline{2-5}  & & \(\text{TMF}(0,1)\) & 0.1317 \(\pm\) 0.0008 & 3.3257 \(\pm\) 0.0053 \\ \cline{1-1} \cline{2-5}  & & \(F(c_{o})\) & 0.1677 \(\pm\) 0.0008 & 4.2348 \(\pm\) 0.0057 \\ \cline{1-1} \cline{2-5}  & & \(F(c_{b})\) & 0.1201 \(\pm\) 0.0005 & 3.0328 \(\pm\) 0.0083 \\ \cline{1-1} \cline{2-5}  & & \(F(c_{l})\) & 0.1149 \(\pm\) 0.0006 & 2.9015 \(\pm\) 0.0064 \\ \cline{1-1} \cline{2-5}  & & \(F(c_{d})\) & 0.0999 \(\pm\) 0.0012 & 2.5227 \(\pm\) 0.0062 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Recommendation performances of the proposed method and compared methods with respect to the type of the utilized user feedback in terms of RMSE and MAPE. Figure 7: Heatmap representation of the feedback rating matrices, where \(x\) and \(y\)-axis respectively represent the users and editing experts belong to the test set. ordinal ratings with binary ratings when a user disliked an editing expert outperformed the rest of the methods with an RMSE of 0.0999, and \(F(c_{o})\), which used ordinal ratings preferentially, showed the worst performance among \(F\)s with an RMSE of 0.1677. In addition, \(F(c_{d})\) showed better performances than \(F(c_{l})\) with an RMSE of 0.1149, where the \(p\)-value of 0.0023 smaller than 0.05 indicates that there is significant performance improvement for \(F(c_{d})\). A similar phenomenon was observed in the performance results of TMFs as well. The performances of \(\text{TMF}(0,1)\), where dislike information was adopted, were superior to \(\text{TMF}(1,0)\), where like information was adopted, and the performances of \(\text{TMF}(1,1)\), where both like and dislike information was adopted, were better than those of \(\text{TMF}(1,0)\). They were statistically significant since the \(p\)-value between \(\text{TMF}(0,1)\) and \(\text{TMF}(1,0)\) was 0.0039, and between \(\text{TMF}(1,1)\) and \(\text{TMF}(1,0)\) was 0.0035. Moreover, it was noticeable that when both ordinal and binary ratings were given, substituting ordinal ratings with binary ratings enhances the performances more than incorporating the two ratings together. This can be supported by the fact that the performances of \(F(c_{d})\) were superior to \(\text{TMF}(0,1)\), and it is statistically approved with a \(p\)-value of 0.0012. Both methods utilized ordinal and binary ratings, but \(F(c_{d})\) only utilized binary ratings when the binary rating was dislike while \(\text{TMF}(0,1)\) incorporated the dislike information of binary ratings with ordinal ratings. The conclusion that utilizing ordinal ratings degrades the opinion inference performance especially for negative ones conforms with the problem of ordinal ratings raised in Section 1, that negative opinions are not correctly expressed in ordinal ratings. Further analyses were conducted to show the trends in the recommendation performances with diverse \(k\)s as illustrated in Figure 7(a,b). The performances of all nine methods enhanced as \(k\) increased, and the performance ranks remained the same except for \(F(c_{b})\) and \(F(c_{d})\). \(F(c_{b})\) showed the best performances when \(k\) was smaller than 40, but \(F(c_{d})\) overtook \(F(c_{b})\) with \(k\) larger than 50.