## 4 Discussion  
The results of the conducted user study show that learning object pedagogical quality scores can be used to improve traditional content-based OER recommender systems, allowing these systems to recommend OER with more quality and equally relevant for the users. Therefore, this study proves that OER repositories can make use of OER quality measures in order to help their users to find OER of sufficiently high quality and related to a specific topic, which is a challenge often faced by both teachers and learners when searching for OER [12]. The results of this research show that the hybrid recommendation approach statistically significantly outperformed the traditional content-based recommendation approach in terms of quality, the quality-based recommendation approach in terms of relevance, and the random recommendations in terms of both quality and relevance. Therefore, these results are consistent with those of [53], which also compared the relevance (by means of a user study) and quality (by means of A/B testing) of random OER recommendations and OER recommendations generated by a hybrid recommender system using quality scores. This study has also examined the usefulness of non-personalized OER recommendations generated by using solely pedagogical quality scores. On the basis of the obtained results, it can be concluded that these types of recommendations, although they could be useful to provide users of OER repositories with features such as \"the 2020 top OER selections\", they are not very useful for helping teachers and learners to find educational resources of their specific area of interest. In summary, measures of OER pedagogical quality have limited utility by themselves to suggest OER to users, but they can be successfully used for enhancing existing OER recommender systems in order to recommend OER with higher pedagogical quality. Most of the results obtained in the user study were expected. Firstly, the hybrid recommender system was expected to outperform the quality-based approach in terms of relevance because this latter approach is not able to generate personalized recommendations, and hence it recommended OER of high quality but that were not found relevant by the users. Secondly, the fact that the hybrid recommender system outperformed the traditional content-based recommender system in terms of quality was also expected, because this latter system recommended the most similar OER to the one the participant was viewing, without taking into account data related to the pedagogical quality of these OER. An interesting and unanticipated finding of this study was that the hybrid recommender system was able to recommend OER as relevant as the ones suggested by the content-based approach and of similar quality to those recommended by the quality-based approach. This means that this hybrid recommender system was able to improve the pedagogical quality of the OER it recommended without having a cost in terms of relevance. A reasonable hypothesis would have been that this system was capable of enhancing OER quality but with some detriment to relevance. However, the results obtained in this study indicate that this enhancement occurred at no cost. This fact confirms that hybrid OER recommender systems, by combining multiple techniques together, are able to overcome the shortcomings of some of these techniques by using the strengths of the other ones. Lastly, regarding the comparison of the random recommendations against the other approaches, these recommendations were, as expected, the least useful ones in terms of both relevance and quality. By taking into account that a high recommender accuracy does not always correlate with user satisfaction or perceived usefulness [64], in order to evaluate the real value of OER recommendations it is necessary to not only evaluate their relevance or accuracy, but also the pedagogical quality of the recommended OER.