## 3 Results  
Table 1 shows the mean (M) and standard deviation (SD) of the overall relevance and quality scores of the OER recommendations calculated by means of the normalized R-Score metric based on the results of the user study for each of the four recommendation approaches analyzed: the hybrid approach combining content-based and quality-based recommendation techniques (labeled as CB + Q in the tables), the traditional content-based recommendation technique (labeled as CB), the quality-based  
non-personalized recommendation technique (labeled as Q), and the random recommendations (labeled as R). The Kruskal-Wallis and Mann-Whitney U tests were used to determine statistical significance. Firstly, a Kruskal-Wallis H test was used to determine if there were statistically significant differences between the overall relevance and quality scores of the different recommendation approaches. The results of this test can be seen in Table 1. Secondly, a series of Mann-Whitney U tests were conducted to compare the overall relevance and quality scores between each pair of recommendation approaches (i.e., CB + Q vs. CB, CB + Q vs. Q, CB + Q vs. R, CB vs. Q, CB vs. R, and Q vs. R). Table 2 shows the results of these tests. As shown in Table 1, the OER recommendations generated by the hybrid approach were the most relevant for the participants (M = 0.64, SD = 0.25), followed very closely by those generated by the traditional content-based recommendation technique (M = 0.60, SD = 0.29). In fact, this difference was not found to be statistically significant. The relevance of the OER recommendations generated according to the other recommendation approaches were found to be much less relevant for the participants, with those generated based on OER pedagogical quality being more relevant (M = 0.25, SD = 0.21) than the random recommendations (M = 0.17, SD = 0.21). With the exception of the hybrid and content-based recommender approaches, the differences between the overall relevance scores for the other pairs of recommendation approaches were found to be statistically significant. Large effect sizes (r > 0.5) were found in all comparisons except between Q and R, where the effect size was small to medium, and between CB + Q and CB, where the effect size was insignificant and, as mentioned before, the difference between overall relevance scores was non-statistically significant. Regarding overall quality scores, the recommendation approaches that recommended OER with highest quality were the quality-based approach (M = 0.78, SD = 0.04) and the hybrid approach (M = 0.77, SD = 0.10). The difference between the quality scores of these two approaches was small and non-statistically significant. Both of these approaches outperformed, in terms of overall quality scores, the traditional content-based recommendation technique (M = 0.60, SD = 0.17), as well as the random recommendations (M = 0.53, SD = 0.10). In all these cases (CB + Q vs. CB, CB + Q vs. R, Q vs. CB, and Q vs. R), the difference between overall quality scores was found to be statistically significant with a large effect size. Finally, the traditional content-based recommendation technique outperformed randomly generated recommendations in terms of overall quality scores, being the difference between these scores statistically significant with a small to medium effect size. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Factor**} & \multicolumn{3}{c}{**CB + Q**} & \multicolumn{3}{c}{**CB**} & \multicolumn{3}{c}{**Q**} & \multicolumn{3}{c}{**R**} & \multicolumn{3}{c}{**Kruskal-Wallis H Test**} \\ \cline{2-10}  & **M** & **SD** & **M** & **SD** & **M** & **SD** & **M** & **SD** & **H** & \(p\)**-Value** \\ \hline Relevance & 0.64 & 0.25 & 0.60 & 0.29 & 0.25 & 0.21 & 0.17 & 0.21 & 91.6 & \textless{}0.001 \\ Quality & 0.77 & 0.10 & 0.60 & 0.17 & 0.78 & 0.04 & 0.53 & 0.10 & 115.1 & \textless{}0.001 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Relevance and quality scores of the open educational resources (OER) recommendations.