## 5 Results and Discussion
### Recommendation of Reinforcement Activities  
We next evaluate the recommendation of reinforcement activities. Given a query made by a student in the discussion forum that reflects a doubt or difficulty in the resolution of a problem or in the understanding of a concept, the recommender proposes to the student one or more exercises/activities from tests of past academic years that allow them to practice on the same and similar concepts than the one that is being addressed in their query.  
For this second set of experiments, we have selected a subset of 20 queries from the test set because most of the topics that are dealt with in the forum are not present in the exam dataset. Again, we only show the results obtained for the strategy that weights the similarity between the text of the activities and the title and body sections of the query. We also compare the three textual representations: bags of words, bags of key phrases and word embeddings.  
Table 2 summarizes the evaluation results for the recommendation of reinforcement activities. Moreover, Figures 5-7 show the detailed results. As in the case of the recommendation of posts, the result of the relaxed evaluation are better than those of the strict evaluation. The best results are obtained when the text is represented as a set of key phrases. When using the \"relaxed criterion\" the precision at 5 is above 80%. When using the \"strict criterion\", the precision at 5 is 65%. For the bags of words approach, precision at 5 is 75% for the strict evaluation and 64% for the relaxed evaluation. These are very promising results: given that the database of test questions is much smaller than that of student conversations, we hypothesize that increasing the number of activities would lead to better recommendation results. However, it is difficult in that the database of activities reflects the diversity of questions raised by students in the forum.  
In contrast, the results of the embeddings approach are worse than those obtained when recommending posts and also worse than those of the bags of words approach. We can see in the figures that, while for the bags of words and key phrase approaches all the queries receive, at least, two relevant recommendations, in the case of the word embeddings approach there are a number of queries for which no relevant recommendations are suggested. We have observed that this occurs, particularly, when the queries address topics that use a vocabulary that is shared between different algorithmic schemes and data structures, as it is the case, for instance, of the branch and bound scheme (that frequently mentions the use of heaps), or the Dijkstra, Prim or Kruskal algorithms (that mention the use of graph structures).  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Similarity** & **Criterion** & **P@5** \\ \hline \multirow{2}{*}{Bags of words} & Problem (strict) & 0,64 \\  & Scheme (relaxed) & 0,75 \\ \hline \multirow{2}{*}{Bag of key phrases} & Problem (strict) & **0,65** \\  & Scheme (relaxed) & **0,80** \\ \hline \multirow{2}{*}{Embeddings} & Problem (strict) & 0,45 \\  & Scheme (relaxed) & 0,63 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Precision at 5 for the different strategies for the recommendation of reinforcement activities. Best results are highlighted in bold.