patrickvonplaten commited on
Commit
98254b9
1 Parent(s): 6adc5ba

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -1
README.md CHANGED
@@ -4,4 +4,52 @@ tags:
4
  - pytorch
5
  - diffusers
6
  - unconditional-image-generation
7
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  - pytorch
5
  - diffusers
6
  - unconditional-image-generation
7
+ ---
8
+
9
+ # Score-Based Generative Modeling through Stochastic Differential Equations (SDE)
10
+
11
+ **Paper**: [Score-Based Generative Modeling through Stochastic Differential Equations](https://arxiv.org/abs/2011.13456)
12
+
13
+ **Authors**: Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole
14
+
15
+ **Abstract**:
16
+
17
+ *Creating noise from data is easy; creating data from noise is generative modeling. We present a stochastic differential equation (SDE) that smoothly transforms a complex data distribution to a known prior distribution by slowly injecting noise, and a corresponding reverse-time SDE that transforms the prior distribution back into the data distribution by slowly removing the noise. Crucially, the reverse-time SDE depends only on the time-dependent gradient field (\aka, score) of the perturbed data distribution. By leveraging advances in score-based generative modeling, we can accurately estimate these scores with neural networks, and use numerical SDE solvers to generate samples. We show that this framework encapsulates previous approaches in score-based generative modeling and diffusion probabilistic modeling, allowing for new sampling procedures and new modeling capabilities. In particular, we introduce a predictor-corrector framework to correct errors in the evolution of the discretized reverse-time SDE. We also derive an equivalent neural ODE that samples from the same distribution as the SDE, but additionally enables exact likelihood computation, and improved sampling efficiency. In addition, we provide a new way to solve inverse problems with score-based models, as demonstrated with experiments on class-conditional generation, image inpainting, and colorization. Combined with multiple architectural improvements, we achieve record-breaking performance for unconditional image generation on CIFAR-10 with an Inception score of 9.89 and FID of 2.20, a competitive likelihood of 2.99 bits/dim, and demonstrate high fidelity generation of 1024 x 1024 images for the first time from a score-based generative model.*
18
+
19
+ ## Inference
20
+
21
+ *SDE* models can use **continous** noise schedulers such as:
22
+
23
+ - [scheduling_sde_ve](https://github.com/huggingface/diffusers/blob/main/src/diffusers/schedulers/scheduling_sde_ve.py)
24
+
25
+ for inference.
26
+
27
+ See the following code:
28
+
29
+ ```python
30
+ # !pip install diffusers
31
+ from diffusers import ScoreSdeVePipeline
32
+
33
+ model_id = "google/ncsnpp-ffhq-1024"
34
+
35
+ # load model and scheduler
36
+ sde_ve = ScoreSdeVePipeline.from_pretrained(model_id)
37
+
38
+ # run pipeline in inference (sample random noise and denoise)
39
+ image = sde_ve()["sample"]
40
+
41
+
42
+ # save image
43
+ image[0].save("sde_ve_generated_image.png")
44
+ ```
45
+
46
+ Please take a look at [pipeline_score_sde_ve](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/score_sde_ve/pipeline_score_sde_ve.py)
47
+ for more details on how to write your own denoising loop.
48
+ For more information generally on how to use `diffusers` for inference, please have a look at the [official inference example](_) # <- TODO(PVP) add link
49
+
50
+ ## Samples
51
+
52
+ 1. ![sample_1](https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/hf/ddpm-cifar10/image_0.png)
53
+ 2. ![sample_1](https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/hf/ddpm-cifar10/image_1.png)
54
+ 3. ![sample_1](https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/hf/ddpm-cifar10/image_2.png)
55
+ 4. ![sample_1](https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/hf/ddpm-cifar10/image_3.png)