# Diffusion Model as a Base for Cold Item Recommendation
## 5 Result Analysis
### Embedding Quality (RQ3)  
**Generated embedding quality to the Refiner's performance:** The Refiner takes the collaborative embeddings calculated by DM-based generator to calculate refined collaborative embeddings for the final recommendation. Therefore, the relationship analysis between input and the final recommendation quality is important. To this end, we compared the performance of the DMCIR model with two different configurations and the performance of the two DMCIR+ models whose input collaborative embeddings are the output of each DMCIR model. Table 7 shows the result. DMCIR\({}_{0}\) and DMCIR\({}_{opt}\) indicate that DMCIR  
Figure 6: Performance with regard to the number of experts \(|E|\). with the number of experts is zero and with the best number of experts (three for CiteULike, five for ML25M), respectively. DMCIR\({}_{0}\)+ and DMCIR\({}_{opt}\)+ use the outputs of DMCIR\({}_{0}\) and DMCIR\({}_{opt}\) as inputs, respectively. The performance of DMCIR+ is proportional to its input item collaborative embeddings. This indicates that the qualitative first prediction of item collaborative embeddings is essential to calculate more accurate refined embeddings. An interesting finding is that while the performance difference between DMCIR\({}_{0}\) and DMCIR\({}_{opt}\) and the difference between DMCIR\({}_{0}\)+ and DMCIR\({}_{opt}\)+ in CiteULike are similar, in ML25M the difference between DMCIR\({}_{0}\)+ and DMCIR\({}_{opt}\)+ is much smaller compared to the difference between DMCIR\({}_{0}\) and DMCIR\({}_{opt}\). That indicates that, in the ML25M dataset, some information captured by the MoE architecture in DMCIR from item collaborative embeddings can be compensated by the refining process, which is re-considering user embeddings and user-item interactions. Thus, further research is required to extract and integrate information using integrated architecture. However, calculating high-quality collaborative embeddings upstream is still important to achieving top recommendation accuracy, especially at the top part of the recommendation list that users most frequently examine. **Side information quality to model performance:** Since the proposed method relies on side information as a pivotal input, the quality of the side information is crucial. To investigate the relationship between the quality of side information and model performance, we evaluated our method using side information embeddings of varying quality. Specifically, we calculated degraded side information by randomly dropping \(p\) % of the original content prior to embedding calculation. Figure 7 shows the relative R@20 scores calculated with side information of different qualities. Across all datasets and model types, performance consistently degrades with lower-quality side information, demonstrating that side information quality is a critical factor in the effectiveness of the proposed method. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Metric** & **\(\text{\bf{DMCIR}}_{0}\)** & **\(\text{\bf{DMCIR}}_{opt}\)** & **\(\text{\bf{DMCIR}}_{0}\)+** & **\(\text{\bf{DMCIR}}_{opt}\)+** \\ \hline \multirow{6}{*}{CiteULike} & P@10 & 0.119 & 0.120 & 0.125 & **0.127** \\  & P@20 & 0.089 & 0.090 & **0.095** & **0.095** \\  & P@50 & 0.054 & 0.054 & **0.058** & **0.058** \\ \cline{2-6}  & R@10 & 0.259 &  
**Embedding Transformation Analysis:** Here, we analyze how the Refiner transforms its input collaborative embeddings--specifically, the ground-truth user embeddings and the generated item embeddings. Since the Refiner projects these inputs into a different space, it is not straightforward to directly compare the input and output embeddings of the Refiner. To address this, we evaluate the transformation by examining the relative angles between embeddings. We compute the pairwise cosine similarity and construct similarity matrices for each set of vectors.