Score SDE VE
Overview
ScoreBased Generative Modeling through Stochastic Differential Equations (Score SDE) by Yang Song, Jascha SohlDickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon and Ben Poole.
The abstract of the paper is the following:
Creating noise from data is easy; creating data from noise is generative modeling. We present a stochastic differential equation (SDE) that smoothly transforms a complex data distribution to a known prior distribution by slowly injecting noise, and a corresponding reversetime SDE that transforms the prior distribution back into the data distribution by slowly removing the noise. Crucially, the reversetime SDE depends only on the timedependent gradient field (\aka, score) of the perturbed data distribution. By leveraging advances in scorebased generative modeling, we can accurately estimate these scores with neural networks, and use numerical SDE solvers to generate samples. We show that this framework encapsulates previous approaches in scorebased generative modeling and diffusion probabilistic modeling, allowing for new sampling procedures and new modeling capabilities. In particular, we introduce a predictorcorrector framework to correct errors in the evolution of the discretized reversetime SDE. We also derive an equivalent neural ODE that samples from the same distribution as the SDE, but additionally enables exact likelihood computation, and improved sampling efficiency. In addition, we provide a new way to solve inverse problems with scorebased models, as demonstrated with experiments on classconditional generation, image inpainting, and colorization. Combined with multiple architectural improvements, we achieve recordbreaking performance for unconditional image generation on CIFAR10 with an Inception score of 9.89 and FID of 2.20, a competitive likelihood of 2.99 bits/dim, and demonstrate high fidelity generation of 1024 x 1024 images for the first time from a scorebased generative model.
The original codebase can be found here.
This pipeline implements the Variance Expanding (VE) variant of the method.
Available Pipelines:
Pipeline  Tasks  Colab 

pipeline_score_sde_ve.py  Unconditional Image Generation   
ScoreSdeVePipeline
class diffusers.ScoreSdeVePipeline
< source >( unet: UNet2DModel scheduler: DiffusionPipeline )
Parameters
 This model inherits from DiffusionPipeline. Check the superclass documentation for the generic methods the —

library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) —
unet (UNet2DModel): UNet architecture to denoise the encoded image. scheduler (SchedulerMixin):
The ScoreSdeVeScheduler scheduler to be used in combination with
unet
to denoise the encoded image.
__call__
< source >(
batch_size: int = 1
num_inference_steps: int = 2000
generator: typing.Optional[torch._C.Generator] = None
output_type: typing.Optional[str] = 'pil'
return_dict: bool = True
**kwargs
)
→
ImagePipelineOutput or tuple
Parameters

batch_size (
int
, optional, defaults to 1) — The number of images to generate. 
generator (
torch.Generator
, optional) — A torch generator to make generation deterministic. 
output_type (
str
, optional, defaults to"pil"
) — The output format of the generate image. Choose between PIL:PIL.Image.Image
ornd.array
. 
return_dict (
bool
, optional, defaults toTrue
) — Whether or not to return a ImagePipelineOutput instead of a plain tuple.
Returns
ImagePipelineOutput or tuple
ImagePipelineOutput
if
return_dict
is True, otherwise a `tuple. When returning a tuple, the first element is a list with the
generated images.