# Deep Learning Architecture for Collaborative Filtering Recommender Systems
## 3 Results
### Recommendation Results  
In this section, the first set of experiments tests the proposed RNCF architecture on the _Netflix_* dataset. \(F1\) results are shown in the top graph of Figure 7. As it can be seen, recommendation quality is improved for all the tested values of \(N\) (number of recommendations). It is remarkable that the most improved values are the most relevant ones: around 11 recommendations for a single user. The bottom graph in Figure 7 shows the details of the \(F1\) quality measure: we can separately see their _Precision_ and _Recall_ components. In this dataset, _Precision_ does not significantly change from the baseline to the proposed RNCF, and the complete \(F1\) improvement is due to the _Recall_ behavior. Additionally, low numbers of recommendations (\(N\)) produce better recommendation quality improvements. As we will see, the same type of results has been found on the performed experiments using the _MovieLens_ and the _FilmTrust_ datasets. Regarding numerical improvement values, if we take \(N=11\) as a reasonable number of recommendations, we approximately obtain the following improvement percentages: _Netflix_ 34%, _MovieLens_ 20%, _FilmTrust_ 12%. These improvements decrease when a higher number of recommendations is selected. Figure 6: _Netflix._ Mean Absolute Error (MAE) obtained by training both the NCF baseline (top chart) and the proposed RNCF neural architecture (chart below). Training and testing evolutions for the 60 first epochs (\(x\)-axis). We have conducted some additional research to understand why the _Recall_ quality measure explains most of the \(F1\) improvement, as well as the decreasing quality improvement when the number of recommendations raises. Finally, we have found the justification: there is an inverse correlation between prediction values and reliability values. There is a tendency for higher predictions to present lower reliabilities. This finding has a deep influence on the evolution of both the _Precision_ and the _Recall_ quality measures: usually, recommendations are made by selecting the highest predicted values; instead, the proposed architecture 'discards' the non-reliable recommendations (the ones with a high predicted error). Since the proposed architecture can discard non-reliable predictions from the set of high predicted items, it will do a better job if there is a sufficient number of predictions to select and to discard, particularly due to the narrow margin that provides the low reliabilities related to high predictions. Here is where the _Recall_ quality measure can show its behavior. _Recall_ is a relative measure, relative to the number of relevant items: highly valued items that usually are the highly predicted ones. On the other hand, _Precision_ is an absolute measure that depends on the constant \(N\) (number of recommendations). The _Recall_ difference between the baseline method and the proposed one will increase as the number of relevant items increases, since the proposed method will be able to better choose the most reliable and highly predicted ones. Instead, _Precision_ does not incorporate the number of relevant items in its equation: it just incorporates the constant \(N\) value. Finally, the higher the number of recommendations, the lower the RNCF recommendation improvement: simply, the higher the value of \(N\), the faster the reliable items are finished, and the _Recall_ improvement decreases. Figures 8 and 9, corresponding to the _MovieLens_ and the _FilmTrust_ experiments, show the same main behavior of the discussed Figure 7 (_Netflix_*). It is remarkable that the bigger the dataset, the higher the proposed neural architecture improvement. This is the expected result, since big datasets offer better opportunities to find items that are, at the same time, highly predicted and highly reliable. By selecting these items, our architecture can improve recommendations. This way, we can claim that the proposed RNCF deep learning architecture is scalable in the sense that it works better when the dataset size increases.