# Novel Paintings from the Latent Diffusion Model through Transfer Learning
## 3 Materials and Methods  
### Denoising Diffusion Probabilistic Model  
[PERSON] et al. [4] proposed the denoising diffusion probabilistic model. There are two processes, the diffusion process and denoising process, in this model, which is shown in Figure 4. In the diffusion process, given the sample data \(x_{0}\) from the data distribution \(q(x)\), the Gaussian noise is appended to the sample data \(x_{0}\) to acquire a sequence of disturbed samples \(x_{0},x_{1},\cdots,x_{T}\):  
\[ \begin{split} q(x_{t}\mid x_{t-1})&=\mathcal{N} \Big{(}x_{t};\sqrt{1-\beta_{t}}x_{t-1},\beta_{t}\mathbf{I}\Big{)},\\ q(x_{1:T}\mid x_{0})&=\prod\limits_{t=1}^{T}q(x_{t }\mid x_{t-1}),\end{split}  \tag{1}\]  
where \(T\) means the diffusion steps, \(\beta_{t}\) is the hyper-parameter with range from 0 to 1, \(\mathcal{N}(\mu,\Sigma)\) is the Gaussian distribution function with the mean \(\mu\) and variance \(\Sigma\), \(\mathbf{I}\) is the identity matrix. We can obtain the arbitrary sample \(x_{t}\) in the diffusion process based on Equation (1) through the reparameterization trick:  
\[ \begin{split} x_{t}&=\sqrt{\alpha_{t}}x_{t-1}+ \sqrt{1-\alpha_{t}}\epsilon_{1}\\ &=\sqrt{\alpha_{t}\alpha_{t-1}}x_{t-2}+\sqrt{1-\alpha_{t}} \epsilon_{1}+\sqrt{\alpha_{t}(1-\alpha_{t-1})}\epsilon_{2}\\ &=\sqrt{\alpha_{t}\alpha_{t-1}}x_{t-2}+\sqrt{1-\alpha_{t}\alpha _{t-1}}\epsilon_{2}\\ &=\ldots\\ &=\sqrt{\alpha_{t}}x_{0}+\sqrt{1-\bar{x}_{t}}\epsilon_{t},\end{split}  \tag{2}\]  
where \(\alpha_{t}=1-\beta_{t}\), \(\bar{x}_{t}=\prod\limits_{i=1}^{t}\alpha_{i}\), and \(\epsilon_{1},\epsilon_{2},\bar{\epsilon}_{2},\epsilon_{t}\sim\mathcal{N}(0, \mathbf{I})\). Usually, \(\beta_{1}<\beta_{2}<\cdots<\beta_{T}\) and \(\bar{x}_{1}>\cdots>\bar{x}_{T}\). \(x_{T}\) will slide into the isotropic Gaussian distribution when \(T\to\infty\). Equation (2) can also be formulated as  
\[q(x_{t}\mid x_{0})=\mathcal{N}\big{(}x_{t};\sqrt{\bar{x}_{t}}x_{0},(1-\bar{x}_ {t})\mathbf{I}\big{)}. \tag{3}\]In the denoising process, to reconstruct the image from Gaussian noise \(x_{T}\), we need to reverse the diffusion process and learn the model \(p_{\theta}\) to estimate these conditional probabilities:  
\[ \begin{split} p_{\theta}(x_{t-1}\mid x_{t})=\mathcal{N}(x_{t-1}; \boldsymbol{\mu}_{\theta}(x_{t},t),\boldsymbol{\Sigma}_{\theta}(x_{t},t)),\\ p_{\theta}(x_{0:T})=p(x_{T})\prod_{t=1}^{T}p_{\theta}(x_{t-1} \mid x_{t}).\end{split}  \tag{4}\]  
The reverse conditional probability \(q(x_{t-1}\mid x_{t},x_{0})\) can be regarded as the reference to learn \(p_{\theta}(x_{t-1}\mid x_{t})\), which is tractable and can be computed by [PERSON] rule:  
\[ \begin{split} q(x_{t-1}\mid x_{t},x_{0})&=\mathcal{ N}\big{(}x_{t-1};\tilde{\boldsymbol{\mu}}_{t}(x_{t},x_{0}),\tilde{\beta}_{t} \mathbf{I}\big{)}\\ &=q(x_{t}\mid x_{t-1},x_{0})\frac{q(x_{t-1}\mid x_{0})}{q(x_{t} \mid x_{0})}\\ &\propto\exp\left(-\frac{1}{2}\left(\frac{(x_{t}-\sqrt{\alpha_{t }}x_{t-1})^{2}}{\beta_{t}}+\frac{(x_{t-1}-\sqrt{\alpha_{t-1}}x_{0})^{2}}{1- \tilde{\alpha}_{t-1}}-\frac{\big{(}x_{t}-\sqrt{\alpha_{t}}x_{0}\big{)}^{2}}{1 -\tilde{\alpha}_{t}}\right)\right)\\ &=\exp\left(-\frac{1}{2}\left(\left(\frac{\alpha_{t}}{\beta_{t} }+\frac{1}{1-\tilde{\alpha}_{t-1}}\right)x_{t-1}^{2}-\left(\frac{2\sqrt{ \alpha_{t}}}{\beta_{t}}x_{t}+\frac{2\sqrt{\alpha_{t-1}}}{1-\tilde{\alpha}_{t- 1}}x_{0}\right)x_{t-1}+C\right)\right)\\ &=\exp\left(-\frac{1}{2\tilde{\beta}_{t}}(x_{t-1}-\tilde{ \boldsymbol{\mu}}_{t})^{2}\right),\end{split}  \tag{5}\]  
where \(C\) is the function not relating \(x_{t-1}\), \(\tilde{\boldsymbol{\mu}}_{t}\) relies on the \(x_{t}\) and \(\epsilon_{t}\), \(\tilde{\beta}_{t}\) is the scalar:  
\[ \begin{split}\tilde{\boldsymbol{\mu}}_{t}&=\frac{1 }{\sqrt{\alpha_{t}}}\bigg{(}x_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\tilde{\alpha}_{ t}}}\epsilon_{t}\bigg{)},\\ \tilde{\beta}_{t}&=\frac{1-\tilde{\alpha}_{t-1}}{1- \tilde{\alpha}_{t}}\beta_{t}.\end{split}  \tag{6}\]  
The diffusion model is the likelihood model, which learns to denoise the variable with a sequence of denoising autoencoders.