# Diffusion Model as a Base for Cold Item Recommendation
## 3 Cold Item Recommender with a Diffusion Model Base
### Preliminary: Diffusion Model
#### 3.2.2 DM-Based Generator  
The DM-based generator is a DM that generates collaborative embeddings \(e_{i}\) of cold items from their side information \(c_{i}\) as its generation condition. Therefore, in the training, we assigned \(e_{i}\) and \(c_{i}\) to the original data \(x_{0}\) and \(c\) in Algorithm 1, respectively. **Network architecture:** For the \(x_{0}\)-predictor \(\hat{x}_{\theta}(x_{t},t,c_{i})\) in Equation (8), instead of simple MLPs used in previous works [33; 38], we added a guideline predictor consisting of a Mixture-of-Experts network (MoE) [58] to handle complex collaborative embedding distributions more effectively and adapt to diverse patterns with limited information on items' side information. As shown in Figure 2, side information \(c_{i}\) is given to the MoE to generate the first prediction of collaborative embedding \(\hat{x}_{0}^{(1)}\); then, with the inputs of DM, \(\hat{x}_{0}^{(1)}\) is fed into the MLP as additional information to predict \(x_{0}\), the collaborative embedding of cold item \(e_{i}\). The MoE network consists of the gate network represented in Equation (11) and multiple experts \(e\in E\) that have the same network structure described in Equation (12), but different parameters are learned from each other. \[g=sigmoid(W_{g}\times c_{i}+b_{g}) \tag{11}\]  
Figure 1: Overview of the proposed two-phase cold item recommender. where \(g\in\mathbb{R}^{|E|}\) indicates the gate vector whose elements represent the weights assigned to individual experts' output. \(W_{g}\in\mathbb{R}^{|E|\times d_{\textit{end}}}\), \(b_{g}\in\mathbb{R}^{|E|}\), and _sigmoid_ indicate the gate network's weight matrix, bias, and the sigmoid activation function, respectively. \[\hat{x}_{0,e}=W_{e}\times c_{i}+b_{e} \tag{12}\]  
where \(W_{e}\in\mathbb{R}^{d_{\textit{cf}}\times d_{\textit{end}}}\) and \(b_{e}\in\mathbb{R}^{d_{\textit{cf}}}\) represent the weight matrix and bias of individual expert \(e\). We chose a single-layer linear transformation without an activation function as the expert network because our pre-experiments showed that the configuration mentioned above shows better performance than more complicated non-linear networks, which is also supported by other works [20; 59]. The first prediction \(\hat{x}_{0}^{(1)}\) for \(x_{0}\) is calculated by the weighted sum of the experts' outputs, as represented in Equation (13). \[\hat{x}_{0}^{(1)}=tanh(\Sigma_{e=1}^{|E|}g_{i}\cdot\hat{x}_{0,e}) \tag{13}\]  
where \(g_{i}\) and \(tanh\) represent the value of the \(i\)-th element of \(g\) and the hyperbolic tangent activation function, respectively. With \(\hat{x}_{0}^{(1)}\), \(x_{t}\), \(c_{i}\), and the time step embedding [60]\(te_{t}\in\mathbb{R}^{d_{\textit{time}}}\), which is related to time step \(t\), are concatenated and passed to the three-layer MLP to calculate the final prediction \(\hat{x}_{0}^{(2)}\) for \(x_{0}\). The network structure of the MLP is represented in Equations (14)-(16). \[mlp^{(1)}=\textit{swish}(W_{mlp}^{(1)}\times(x_{t}\parallel\hat{x}_{0}^{(1)} \parallel c_{i}\parallel te_{t})+b_{mlp}^{(1)}) \tag{14}\]  
where \(W_{mlp}^{(1)}\in\mathbb{R}^{d_{\textit{cf}}\times(2\times d_{\textit{cf}}+d_{ \textit{end}}+d_{\textit{time}})}\) and \(b_{mlp}^{(1)}\in\mathbb{R}^{d_{\textit{cf}}}\) represent the weight matrix and bias of the first layer of the MLP. _swish_ indicates the swish activation function [61]. \[mlp^{(2)}=\textit{swish}(W_{mlp}^{(2)}\times mlp^{(1)}+b_{mlp}^{(2)}) \tag{15}\]  
\[\hat{x}_{0}^{(2)}=W_{mlp}^{(3)}\times mlp^{(2)}+b_{mlp}^{(3)} \tag{16}\]  
where \(W_{mlp}^{(l)}\in\mathbb{R}^{d_{\textit{cf}}\times d_{\textit{cf}}}\) and \(b_{mlp}^{(l)}\in\mathbb{R}^{d_{\textit{cf}}}\) represent the weight matrix and bias of the \(l\)-th layer of the MLP. The only difference in the network structure between the second and third layers is the existence of the activation function.