The sampling procedure of DDIM is also reminiscent of neural networks with continuous depth, like those proposed by Chen and Grathwohl, since the samples it produces from the same latent variable have similar high-level visual features, regardless of the specific sample trajectory. 7. Discussion We have presented DDIMs—an implicit generative model trained with denoising auto-encoding or score matching objectives—from a purely variational perspective. DDIM is able to generate high-quality samples much more efficiently than existing DDPMs and NCSNs, with the ability to perform meaningful interpolations from the latent space. The non-Markovian forward process presented here seems to suggest continuous forward processes other than Gaussian, which cannot be done in the original diffusion framework, since Gaussian is the only stable distribution with finite variance. We also demonstrated a discrete case with a multinomial forward process in the appendix, and it would be interesting to investigate similar alternatives for other combinatorial structures. Moreover, since the sampling procedure of DDIMs is similar to that of a neural ODE, it would be interesting to see if methods that decrease the discretization error in ODEs, including multi-step methods such as Adams-Bashforth, could be helpful for further improving sample quality in fewer steps, an idea also explored by Queiruga and colleagues. It is also relevant to investigate whether DDIMs exhibit other properties of existing implicit models, such as those seen by Bau and colleagues. Acknowledgements The authors would like to thank Yang Song and Shengjia Zhao for helpful discussions over the ideas, Kuno Kim for reviewing an earlier draft of the paper, and Sharvil Nanavati and Sophie Liu for identifying typos. This research was supported by NSF, ONR, AFOSR, and Amazon AWS. Appendix  Experimental Details  Datasets and architectures We consider 4 image datasets with various resolutions: CIFAR10 (32 by 32, unconditional), CelebA (64 by 64), LSUN Bedroom (256 by 256) and LSUN Church (256 by 256). For all datasets, we set the hyperparameters for alpha according to the heuristic in the work by Ho and colleagues to make the results directly comparable. We use the same model for each dataset, and only compare the performance of different generative processes. For CIFAR10, Bedroom and Church, we obtain the pretrained checkpoints from the original DDPM implementation; for CelebA, we trained our own model using the simplified denoising objective. Our architecture for the noise-prediction model follows that in the work by Ho and colleagues, which is a U-Net based on a Wide ResNet. We use the pretrained models for CIFAR10, Bedroom and Church, and train our own model for the CelebA 64 by 64 model. Our CelebA model has five feature map resolutions from 64 by 64 down to 4 by 4. We use the original CelebA dataset, not CelebA-HQ, using the pre-processing technique from the StyleGAN repository. For the LSUN Bedroom and Church datasets, our results also show that DDIM outperforms DDPM with fewer sampling steps. For example, with 50 steps on the Bedroom dataset, DDIM achieves an FID of 6.75, which is already better than the 100-step DDPM's FID of 6.81 and close to the 1000-step DDPM's FID of 6.36. Reverse process sub-sequence selection We consider two types of selection procedures for choosing the timesteps for accelerated sampling: First, a "Linear" method, where we select timesteps that are evenly spaced. Second, a "Quadratic" method, where the spacing between selected timesteps increases. We used the quadratic method for CIFAR10 and the linear method for the remaining datasets, as these choices achieved slightly better FID scores. Closed form equations for each sampling step The general sampling update rule for a step depends on the previous image, the model's noise prediction, and a random noise term. The amount of randomness is controlled by the parameter eta. For the standard DDPM, eta is 1. For DDIM, eta is 0. For the DDPM variant with a larger variance, the update equation uses a different coefficient for the random noise term, making the update more stochastic. This explains why it performs worse when the number of sampling steps is small. Samples and Consistency Additional generated samples for CIFAR10, CelebA, and Church confirm the quantitative results. The samples from DDIM with 100 steps are visually very similar to those from the 1000-step models, while the 100-step DDPM samples are of noticeably lower quality. Additional images also demonstrate the consistency property of DDIM on the CelebA dataset, showing that different sampling trajectories from the same starting noise produce images with the same high-level features. Interpolation To generate interpolations, we randomly sample two initial noise vectors from a standard Gaussian distribution. We then use spherical linear interpolation, or slerp, to create a path between these two vectors in the latent space. Finally, we use the DDIM to generate an image for each point along this path. To generate interpolations on a grid, we sample four latent variables and apply the same interpolation technique in two dimensions. Additional images of grid interpolations for CelebA, Bedroom, and Church datasets show smooth and meaningful transitions, for example, changing a person's expression, rotating a room's viewpoint, or morphing the architecture of a church.