## 6 Results
Finally, in Table 3, we also compare \(\text{COA}[\hat{w}_{\Theta}]\) to the COA implementation using the surrogate function \(\hat{w}^{\prime}_{\Theta}\) introduced in Reference [16], henceforth referred to as \(\text{COA}[\hat{w}^{\prime}_{\Theta}]\). While \(\hat{w}^{\prime}_{\Theta}\) is also based on a matrix factorization model, this model does not take into account that user data is not missing at random. Each table cell shows the average optimality gaps of solutions to the respective instance set at the specified interaction level \(\psi\). The better results among \(\text{COA}[\hat{w}_{\Theta}]\) and \(\text{COA}[\hat{w}^{\prime}_{\Theta}]\) are printed bold. A one-sided Wilcoxon signed-rank test was used to determine for each instance set whether the difference in optimality gaps is significant or not. Entries for which the Wilcoxon test has assessed at a 95% confidence level that either \(\text{COA}[\hat{w}_{\Theta}]\) or \(\text{COA}[\hat{w}^{\prime}_{\Theta}]\) has produced better optimality gaps are marked with an asterisk. It can be observed that, for lower levels of user interaction, solutions generated by \(\text{COA}[\hat{w}^{\prime}_{\Theta}]\) exhibit extremely high optimality gaps. However, the higher the number of user interactions, the more \(\text{COA}[\hat{w}^{\prime}_{\Theta}]\) can catch up with \(\text{COA}[\hat{w}_{\Theta}]\). However, most of the time, \(\text{COA}[\hat{w}_{\Theta}]\) still dominates \(\text{COA}[\hat{w}^{\prime}_{\Theta}]\). Summarizing, it can be concluded that the new matrix factorization model is a significant improvement over our previous model \(\hat{w}^{\prime}_{\Theta^{\prime}}\), resulting in \(\text{COA}[\hat{w}_{\Theta}]\) generating better solutions with fewer user interactions than \(\text{COA}[\hat{w}^{\prime}_{\Theta}]\) most of the time. \begin{table} \begin{tabular}{c c c c c c c c c c c c} \hline \hline  & & & & & & & & & & & & \\ \hline \((n,m)\) & \multicolumn{3}{c}{**(100, 500)**} & \multicolumn{3}{c}{**(100, 1000)**} & \multicolumn{3}{c}{**(200, 1000)**} & \multicolumn{3}{c}{**(200, 2000)**} & \multicolumn{3}{c}{**(300, 1500)**} & \multicolumn{3}{c}{**(300, 3000)**} \\ \hline \(\psi\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) & \(\text{COA}[\hat{w}_{\Theta}]\) \\ \hline
30\% & 4.31 & **4.03** & - & - & **0.99** & 0.99 & - & - & 0.30 & 0.32 & **1.93** & 1.94 \\
40\% & **1.71** & 2.00 & 4.42 & 4.95 & **0.57** & 0.62 & **1.84** & 2.06 & **0.23** & 0.23 & **1.17** * & 1.31 \\
50\% & **0.73** * & 1.11 & **2.72** & 3.03 & **0.41** & 0.46 & **1.01** * & 1.44 & **0.16** & 0.16 & **0.68** * & 0.86 \\
60\% & **0.31** * & 0.69 & **1.27** * & 1.66 & **0.23** * & 0.29 & **0.58** * & 0.91 & 0.10 & **0.09** & **0.36** * & 0.49 \\
70\% & **0.12** * & 0.42 & **0.53** * & 0.90 & **0.11** & 0.14 & 0.33 * & 0.50 & **0.04** * & 0.06 & **0.16** * & 0.30 \\
80\% & **0.11** * & 0.18 & **0.22** * & 0.53 & **0.04** * & 0.08 & **0.11** * & 0.21 & **0.02** & 0.04 & **0.05** * & 0.10 \\
90\% & **0.05** & 0.11 & **0.03** * & 0.13 & **0.01** & 0.03 & **0.02** * & 0.06 & **0.00** * & 0.01 & **0.01** * & 0.02 \\ \hline \hline \end{tabular}