# Diffusion Model as a Base for Cold Item Recommendation
## 3 Cold Item Recommender with a Diffusion Model Base
### Preliminary: Diffusion Model  
DM generates realistic data that satisfy a given condition (requirement, e.g., generate an image from its description represented by text embedding) from pure Gaussian noise. DM consists of a forward process that adds noise to the original data and a backward process that generates the original data from noise. **Forward Process:** In the forward process, Gaussian noises are added \(T\) times to given original data \(x_{0}\) to obtain a noised data sequence \(x_{1}\), \(x_{2}\), \(\ldots\), \(x_{T}\). Noised data \(x_{t}\) at timestep \(t\) are sampled from \(x_{t-1}\) using Equation (1). \[q(x_{t}|x_{t-1})=\mathcal{N}(x_{t};\sqrt{1-\beta_{t}}x_{t-1},\beta_{t}t) \tag{1}\]  
where \(\mathcal{N}\), \(\mathrm{I}\), and \(\beta_{t}\) indicate Gaussian distribution, the identity matrix, and noise scale at timestep \(t\), respectively. \(\beta_{t}\) increases as timestep \(t\) increases. It can be learned from the data, \begin{table}
\begin{tabular}{c c} \hline
**Notation** & **Explanation** \\ \hline \(u,i\) & A user, an item \\ \(U,I,I_{w},I_{c}\) & The set of users, items, warm items, cold items \\ \(e_{u},e_{i}\) & The ground-truth collaborative embeddings of \(u\), and \(i\) \\ \(c_{i}\) & Side information embedding of \(i\) \\ \(te_{i}\) & Time step embedding of timestep \(t\) \\ \(R^{tr},R^{te}\) & training and test data \\ \(L_{u}^{k}\) & Top-\(k\) cold item recommendation list for \(u\) \\ \(x_{0}\) & The original dataset of DM. It indicates \(e_{i}\) in this paper \\ \(d_{cond}\,d_{cf}\) & The dimension of side information (\(c_{i}\)), collaborative embeddings (\(e_{u},e_{i}\)) \\ \(d_{time}\) & The dimension of time step embedding (\(te_{i}\)) \\ \(r_{ui}\) & The interaction indicator between user \(u\) and item \(i\) \\ \(p_{ui}\) & The predicted interaction indicator between \(u\) and \(i\) \\ \(E\) & The set of experts in Mixture-of-Experts architecture \\ \hline \end{tabular}
\end{table} 
Table 1: Notations. but generally a predefined fixed scheduler is used such as the linear scheduler [27], or cosine scheduler [55], or sigmoid scheduler [56]. We adopted the linear scheduler represented in Equation (2), due to its empirical stability [33; 38] and implementation simplicity. \[\beta_{t}=\beta_{s}\cdot(1-\frac{t-1}{T-1})+\beta_{e}\cdot\frac{t-1}{T-1} \tag{2}\]  
where \(\beta_{s}\), \(\beta_{e}\), and the maximum timestep \(T\) are hyperparameters. Following the proposal by [PERSON] et al. [27], we set \(\beta_{s}=0.0001\). The optimal values for \(\beta_{e}\) and \(T\) will be evaluated in Section 5. As shown in Equation (1), sampling \(x_{t}\) depends on its previous \(x_{t-1}\) only, and when Gaussian noises are applied, we can assume \(x_{t}\) is sampled from \(x_{0}\) by the single noise addition represented in Equations (3) and (4) [46]. \[q(x_{t}|x_{0})=\mathcal{N}(x_{t};\sqrt{\bar{x}_{t}}x_{0},1-\bar{x}_{t}\text{I}) \tag{3}\]  
\[\bar{x}_{t}=\Pi_{i=1}^{t}\alpha_{i},\qquad\alpha_{t}=1-\beta_{t} \tag{4}\]  
Since \(\beta_{t}\) increases as \(t\) increases, \(\alpha_{t}\) and \(\bar{x}_{t}\) decrease. As a result, the information in \(x_{0}\) decreases, but the noise increases as \(t\) increases. At the last time step \(T\), \(x_{T}\) converges to pure Gaussian noise. **Backward Process:** The backward process of DM is a denoising process. From \(x_{T}\), the fully noised data at the final timestep, the noise is removed step by step to recover \(x_{0}\). The ground-truth denoising step, which is derived by using Bayes' rule, is represented in Equations (5)-(7). \[q(x_{t-1}|x_{t},x_{0})=\mathcal{N}(x_{t-1};\mu_{q}(x_{t},x_{0},t),\sigma_{q}^{ 2}(t)\text{I}) \tag{5}\]  
\[\mu_{q}(x_{t},x_{0},t)=\frac{\sqrt{\bar{\alpha}_{t}}(1-\bar{x}_{t-1})x_{t}+ \sqrt{\bar{\alpha}_{t-1}}(1-\alpha_{t})x_{0}}{1-\bar{\alpha}_{t}} \tag{6}\]  
\[\sigma_{q}^{2}(t)=\frac{(1-\bar{x}_{t-1})(1-\alpha_{t})}{(1-\bar{\alpha}_{t})} \tag{7}\]  
Unfortunately, we do not know the original data \(x_{0}\) at the inference time.