# Deep Learning Architecture for Collaborative Filtering Recommender Systems
## 3 Results
### Prediction Results  
As explained in the previous section, the proposed RNCF architecture contains a Multilayer Neural Network (Figure 3, abstraction level 2) that learns from the errors made in the MF predictions. Once the network has learned, it can predict \"errors in predictions\" (reliabilities). The smaller the predicted error, the greater its reliability. Figure 4 shows the quality of the obtained reliabilities on the _MovieLens_ dataset. As it can be seen, the neural network is not overfitted and its testing Mean Absolute Error (MAE) is close to the 0.01 value when 15 epochs have been run. The reliability values can be considered accurate enough to feed the neural network on top of the proposed architecture (Figure 3, abstraction level 3). The _Netflix_ and _FilmTrust_ results are very similar to the _MovieLens_ one; they have been omitted to keep this subsection as short as possible.  
Once reliabilities can be predicted on the abstraction layer 2 of the RNCF architecture, they can feed the abstraction layer 3 (on Figure 3). Figures 5 and 6 show the obtained MAE by using both the NCF baseline and the proposed RNCF. Figure 5 shows the _MovieLens_ results, whereas Figure 6 shows the _Netflix_ ones. _FilmTrust_ prediction results have been omitted to keep the size of this subsection short; they show the same behavior as the _MovieLens_ and the _Netflix_ results.  
\begin{table}
\begin{tabular}{c c} \hline
**Parameter** & **Value** \\ \hline \% training & 80 \\ \% testing & 20 \\ \(N\) factors & \{1, \(\dots,96\}\) step 5 \\ Recommendation threshold (\(\theta\)) & 4 \\ \hline \end{tabular}
\end{table}
Table 2: Cross-validation values.  
Figure 5 shows that the neural network is not overfitted both for the NCF baseline (top graph on Figure 5) and for the proposed RNCF (bottom graph on Figure 5). Comparing the testing MAE values at the end of the training (40 epochs in the baseline and 60 epochs in RNCF), we can claim the superiority of the proposed RNCF when applied to the _MovieLens_ dataset: an error of 0.47 versus 0.62. RNCF achieves an improvement of 24% in the quality of the predictions. From the analysis of Figure 6 (results obtained using the _Netflix_ dataset), the same conclusions can be extracted: there is not overfitting, and RNCF achieves an improvement of approximately 22% in the quality of the predictions. Overall, the proposed method achieves a strong improvement in the quality of the RS predictions.  
Figure 4: _MovieLens_. Mean Absolute Error (MAE) quality in the prediction of the reliabilities.  
Figure 5: _MovieLens_. Mean Absolute Error (MAE) obtained by training both the NCF baseline (top chart) and the proposed RNCF neural architecture (chart below). Training and testing evolutions for the 60 first epochs (\(x\)-axis).  
Since the processed version of _Netflix_ is much bigger than the _Movielens_ one, we have also used a bigger value for the neural network _batch_ size hyperparameter in order to avoid a significant increase in runtime. High _batch_ sizes tend to produce fluctuations in trends results, as shown in Figure 6 (_Test MAE_). These fluctuations can be smoothed by processing the average of a series of separate runs of the same neural network. Fluctuations appear on the _TestMAE_ curve, and not on the _TrainingMAE_ curve, due to the smaller size of the test set.