Update README.md

README.md

@@ -6,8 +6,13 @@ tags:
 inference: false
 ---
 
-# Stable Diffusion v1 Model Card
-This model card focuses on the model associated with the Stable Diffusion model, available [here](https://github.com/CompVis/stable-diffusion).
+# Stable Diffusion v1-1 Model Card
+
+Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input.
+For more information about how Stable Diffusion functions, please have a look at [🤗's Stable Diffusion with D🧨iffusers blog](hf.co/blog/stable_diffusion).
+
+The **Stable-Diffusion-v1-1** was trained on 237,000 steps at resolution `256x256` on [laion2B-en](https://huggingface.co/datasets/laion/laion2B-en), followed by
+194,000 steps at resolution `512x512` on [laion-high-resolution](https://huggingface.co/datasets/laion/laion-high-resolution) (170M examples from LAION-5B with resolution `>= 1024x1024`). For more information, please refer to [Training](#training).
 
 ## Model Details
 - **Developed by:** Robin Rombach, Patrick Esser
@@ -27,28 +32,36 @@ This model card focuses on the model associated with the Stable Diffusion model,
           pages     = {10684-10695}
       }
 
-## Usage examples
+## Examples
+
+We recommend using [🤗's Diffusers library](https://github.com/huggingface/diffusers) to run Stable Diffusion.
 
 ```bash
 pip install --upgrade diffusers transformers scipy
 ```
 
 Run this command to log in with your HF Hub token if you haven't before:
+
 ```bash
 huggingface-cli login
 ```
 
 Running the pipeline with the default PLMS scheduler:
 ```python
+import torch
 from torch import autocast
 from diffusers import StableDiffusionPipeline
 
-model_id = "CompVis/stable-diffusion-v1-1-diffusers"
-pipe = StableDiffusionPipeline.from_pretrained(model_id, use_auth_token=True).to("cuda")
+model_id = "CompVis/stable-diffusion-v1-1"
+device = "cuda"
+
+generator = torch.Generator(device=device).manual_seed(0)
+pipe = StableDiffusionPipeline.from_pretrained(model_id, use_auth_token=True)
+pipe = pipe.to(device)
 
 prompt = "a photograph of an astronaut riding a horse"
 with autocast("cuda"):
-    image = pipe(prompt, guidance_scale=7)["sample"][0]  # image here is in PIL format
+    image = pipe(prompt)["sample"][0]  # image here is in PIL format
     
 image.save(f"astronaut_rides_horse.png")
 ```
@@ -58,10 +71,11 @@ To swap out the noise scheduler, pass it to `from_pretrained`:
 ```python
 from diffusers import StableDiffusionPipeline, LMSDiscreteScheduler
 
-model_id = "CompVis/stable-diffusion-v1-1-diffusers"
+model_id = "CompVis/stable-diffusion-v1-1"
 # Use the K-LMS scheduler here instead
 scheduler = LMSDiscreteScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", num_train_timesteps=1000)
-pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, use_auth_token=True).to("cuda")
+pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, use_auth_token=True)
+pipe = pipe.to("cuda")
 ```
 
 # Uses
@@ -83,8 +97,10 @@ _Note: This section is taken from the [DALLE-MINI model card](https://huggingfac
 
 
 The model should not be used to intentionally create or disseminate images that create hostile or alienating environments for people. This includes generating images that people would foreseeably find disturbing, distressing, or offensive; or content that propagates historical or current stereotypes.
+
 #### Out-of-Scope Use
 The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
+
 #### Misuse and Malicious Use
 Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to:
 
@@ -113,6 +129,7 @@ Using the model to generate content that is cruel to individuals is a misuse of
   considerations.
 
 ### Bias
+
 While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases. 
 Stable Diffusion v1 was trained on subsets of [LAION-2B(en)](https://laion.ai/blog/laion-5b/), 
 which consists of images that are primarily limited to English descriptions. 
@@ -129,23 +146,27 @@ The model developers used the following dataset for training the model:
 - LAION-2B (en) and subsets thereof (see next section)
 
 **Training Procedure**
-Stable Diffusion v1 is a latent diffusion model which combines an autoencoder with a diffusion model that is trained in the latent space of the autoencoder. During training, 
+Stable Diffusion v1-4 is a latent diffusion model which combines an autoencoder with a diffusion model that is trained in the latent space of the autoencoder. During training, 
 
 - Images are encoded through an encoder, which turns images into latent representations. The autoencoder uses a relative downsampling factor of 8 and maps images of shape H x W x 3 to latents of shape H/f x W/f x 4
 - Text prompts are encoded through a ViT-L/14 text-encoder.
 - The non-pooled output of the text encoder is fed into the UNet backbone of the latent diffusion model via cross-attention.
 - The loss is a reconstruction objective between the noise that was added to the latent and the prediction made by the UNet.
 
-We currently provide three checkpoints, `sd-v1-1.ckpt`, `sd-v1-2.ckpt` and `sd-v1-3.ckpt`,
-which were trained as follows,
-
-- `sd-v1-1.ckpt`: 237k steps at resolution `256x256` on [laion2B-en](https://huggingface.co/datasets/laion/laion2B-en).
-  194k steps at resolution `512x512` on [laion-high-resolution](https://huggingface.co/datasets/laion/laion-high-resolution) (170M examples from LAION-5B with resolution `>= 1024x1024`).
-- `sd-v1-2.ckpt`: Resumed from `sd-v1-1.ckpt`.
-  515k steps at resolution `512x512` on "laion-improved-aesthetics" (a subset of laion2B-en,
+We currently provide four checkpoints, 
+- [`stable-diffusion-v1-1`](https://huggingface.co/CompVis/stable-diffusion-v1-1), 
+- [`stable-diffusion-v1-2`](https://huggingface.co/CompVis/stable-diffusion-v1-2), 
+- [`stable-diffusion-v1-3`](https://huggingface.co/CompVis/stable-diffusion-v1-3), and 
+- [`stable-diffusion-v1-4`](https://huggingface.co/CompVis/stable-diffusion-v1-4).
+
+The checkpoints were trained as follows:
+- `stable-diffusion-v1-1`: 237,000 steps at resolution `256x256` on [laion2B-en](https://huggingface.co/datasets/laion/laion2B-en).
+  194,000 steps at resolution `512x512` on [laion-high-resolution](https://huggingface.co/datasets/laion/laion-high-resolution) (170M examples from LAION-5B with resolution `>= 1024x1024`).
+- `stable-diffusion-v1-2`: Resumed from `stable-diffusion-v1-1`.
+  515,000 steps at resolution `512x512` on "laion-improved-aesthetics" (a subset of laion2B-en,
 filtered to images with an original size `>= 512x512`, estimated aesthetics score `> 5.0`, and an estimated watermark probability `< 0.5`. The watermark estimate is from the LAION-5B metadata, the aesthetics score is estimated using an [improved aesthetics estimator](https://github.com/christophschuhmann/improved-aesthetic-predictor)).
-- `sd-v1-3.ckpt`: Resumed from `sd-v1-2.ckpt`. 195k steps at resolution `512x512` on "laion-improved-aesthetics" and 10\% dropping of the text-conditioning to improve [classifier-free guidance sampling](https://arxiv.org/abs/2207.12598).
-
+- `stable-diffusion-v1-3`: Resumed from `stable-diffusion-v1-2`. 195,000 steps at resolution `512x512` on "laion-improved-aesthetics" and 10 % dropping of the text-conditioning to improve [classifier-free guidance sampling](https://arxiv.org/abs/2207.12598)
+- *`stable-diffusion-v1-4`*: ...
 
 - **Hardware:** 32 x 8 x A100 GPUs
 - **Optimizer:** AdamW
@@ -172,33 +193,6 @@ Based on that information, we estimate the following CO2 emissions using the [Ma
 - **Compute Region:** US-east
 - **Carbon Emitted (Power consumption x Time x Carbon produced based on location of power grid):** 11250 kg CO2 eq.
 
-## Usage
-
-### Setup
-
-- Install `diffusers` with
-
-`pip install -U git+https://github.com/huggingface/diffusers.git`
-- Install `transformers` with
-
-`pip install transformers`
-
-```python
-import torch
-from diffusers import StableDiffusionPipeline
-
-pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-1-diffusers")
-
-prompt  = "19th Century wooden engraving of Elon musk"
-
-seed = torch.manual_seed(1024)
-images = pipe([prompt], num_inference_steps=50, guidance_scale=7.5, generator=seed)["sample"]
-
-# save images
-for idx, image in enumerate(images):
-    image.save(f"image-{idx}.png")
-```
-
 
 ## Citation
 
@@ -213,4 +207,4 @@ for idx, image in enumerate(images):
     }
 ```
 
-*This model card was written by: Robin Rombach and Patrick Esser and is based on the [DALL-E Mini model card](https://huggingface.co/dalle-mini/dalle-mini).*
+*This model card was written by: Robin Rombach and Patrick Esser and is based on the [DALL-E Mini model card](https://huggingface.co/dalle-mini/dalle-mini).*