# Towards Hyper-Relevance in Marketing: Development of a Hybrid Cold-Start Recommender System
## 5 Results and Discussion  
The final RS is defined through the evaluation and comparison of a series of experiments, considering various combinations of hyper-parameters, as presented in Section 4. The defined evaluation metrics are the following: Hit Rate, Portfolio Coverage, Average Price and Standard Deviation of Price. Regarding the first step of the training phase, based on the input data, a variation in the content-based inputs and the number of days to consider from the transactional dataset is performed, presented in Table 3. It is concluded that, regarding the 1 CR, the full combination of inputs provided the best performance based on the hit rate, 1.8%, with the highest impact resulting from the brand category, the lack of this variable resulting in a hit rate of 0.5%. The 2 CR, using as input the correlation matrix extracted from descriptive keywords, exhibited low predictive power, with a hit rate of 0.4%, indicating a possible user preference for a higher variety of products. The CR demonstrated a slight increase of hit rate through the increase of the transactional dataset's size, from 2.2% to 2.8%, although the computational cost is significant, increasing from 37 s to 806 s. The MR demonstrated low sensitivity versus the size of the transitional dataset, with the highest hit rate, 3.0%, being associated with three days worth of transactional data. The second step involves the exploration of higher similarity coefficient matrix size, considering each model individually, represented in Table 4. It can be concluded that an increase in the matrix size causes only small improvements in the hit rate for the 1 CR, from 1.9% to 2.0%, at a high computational cost, from 60 s to 1058 s. Regarding the 2 CR, a considerable increase in the hit rate is observed, from 0.9% to 2.3%, although the computational time also increases substantially, from 60 s to 974 s. For the CR and MR, a significant increase in hit rate and portfolio coverage was observed when using the full intermediate matrices, reaching 5.1% and 4.6% in hit rate and 9.4% and 6.3% in portfolio coverage, with a relatively small a trade-off in computational time, reaching 543 s and 88 s. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Inputs** & **Hit Rate** & **Portfolio Coverage** & **Avg. Price** & **Std. Dev. Price** & **Comp. Time** \\ \hline
1 CR & Price segments, recommended age, brand & 1.8\% & 8.4\% & EUR 33.05 & EUR 38.59 & 60 s \\ \hline
1 CR & Recommended age, brand & 1.5\% & 9.1\% & EUR 42.82 & EUR 39.49 & 65 s \\ \hline
1 CR & Price segments, brand & 1.8\% & 9.4\% & EUR 28.77 & EUR 21.58 & 61 s \\ \hline
1 CR & Price segments, recommended age & 0.5\% & 11.0\% & EUR 49.19 & EUR 14.94 & 55 s \\ \hline
2 CR & Keyword segments & 0.4\% & 4.9\% & EUR 77.74 & EUR 82.75 & 67 s \\ \hline CR & Transactional dataset size: 1 day & 2.2\% & 4.1\% & EUR 47.87 & EUR 39.28 & 37 s \\ \hline CR & Transactional dataset size: 3 days & 2.3\% & 8.6\% & EUR 66.42 & EUR 65.69 & 104 s \\ \hline CR & Transactional dataset size: 7 days & 2.4\% & 9.0\% & EUR 43.41 & EUR 45.53 & 195 s \\ \hline CR & Transactional dataset size: 30 days & 2.8\% & 9.9\% & EUR 39.70 & EUR 40.44 & 806 s \\ \hline MR & Transactional dataset size: 1 day & 1.2\% & 2.6\% & EUR 39.08 & EUR 50.80 & 31 s \\ \hline MR & Transactional dataset size: 3 days & 3.0\% & 6.3\% & EUR 25.56 & EUR 26.72 & 80 s \\ \hline MR & Transactional dataset size: 7 days & 2.5\% & 1.8\% & EUR 25.05 & EUR 30.51 & 42 s \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Evaluation of input combinations. When exploring the hybridisation step, an initial computational study was performed to predict the impact of each individual RS in the final hybrid system. Figure 7 reproduces the results of the study, where the relative weight of each individual model varied from 0 to 4, evaluating the impact of this variation in the model hit rate. Expressing each weight in the corresponding percentage, 0 is equivalent to 0%, 1 to 25%, 2 to 40%, 3 to 50%, and 4 to 57%.