library_name: diffusers | |
tags: | |
- stable-diffusion | |
license: mit | |
## Consistency Decoder | |
This is a decoder that can be used to improve decoding for Stable Diffusion VAEs. To know more, refer to the [DALL-E 3 technical report](https://cdn.openai.com/papers/dall-e-3.pdf). | |
To original code repository can be found [here](https://github.com/openai/consistencydecoder). | |
## Usage in 🧨 diffusers | |
```python | |
import torch | |
from diffusers import DiffusionPipeline, ConsistencyDecoderVAE | |
vae = ConsistencyDecoderVAE.from_pretrained("openai/consistency-decoder", torch_dtype=torch.float16) | |
pipe = StableDiffusionPipeline.from_pretrained( | |
"runwayml/stable-diffusion-v1-5", vae=vae, torch_dtype=vae.dtype | |
).to("cuda") | |
pipe("horse", generator=torch.manual_seed(0)).images | |
``` | |
## Results | |
_(Taken from the original [code repository](https://github.com/openai/consistencydecoder))_ | |
## Examples | |
Original Image | GAN Decoder | Consistency Decoder | | |
:---:|:---:|:---:| | |
![Original Image](https://github.com/openai/consistencydecoder/blob/main/assets/gt1.png?raw=true) | ![GAN Image](https://github.com/openai/consistencydecoder/blob/main/assets/gan1.png?raw=true) | ![VAE Image](https://github.com/openai/consistencydecoder/blob/main/assets/con1.png?raw=true) | | |
![Original Image](https://github.com/openai/consistencydecoder/blob/main/assets/gt2.png?raw=true) | ![GAN Image](https://github.com/openai/consistencydecoder/blob/main/assets/gan2.png?raw=true) | ![VAE Image](https://github.com/openai/consistencydecoder/blob/main/assets/con2.png?raw=true) | | |
![Original Image](https://github.com/openai/consistencydecoder/blob/main/assets/gt3.png?raw=true) | ![GAN Image](https://github.com/openai/consistencydecoder/blob/main/assets/gan3.png?raw=true) | ![VAE Image](https://github.com/openai/consistencydecoder/blob/main/assets/con3.png?raw=true) | | |