How to Generate AI Images with Stable Diffusion XL in 5 Minutes
Tags: beginners, generativeai, python, tutorial
Published: 2024-03-12 00:40:13

---
title: How to Generate AI Images with Stable Diffusion XL in 5 Minutes
published: true
description: An easy guide generate images with SDXL on your own machine, quickly
tags: beginners,generativeai,python,tutorial
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v4jirmvxcdd792eu4cfv.png
# Use a ratio of 100:42 for best results.
# published_at: 2024-03-12 00:32 +0000
---

Want to generate awesome AI images from your machine? 

Stable Diffusion XL is one of the best local image generators out there, and here's how you can set it up in minutes. 

Note: you will need a good graphics card for this. A minimum of 4G of VRAM is needed; you'll be much better off with 8G or more. 

We will use [Stable Diffusion XL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) on a Linux system. The instructions are the same for Mac or Windows if you use WSL. 

## Step 1: Create a Python Virtual Environment

Let's set up a Python Virtual Environment. This helps us manage dependencies and keep our project clean. 

```
python -m venv stablediff
```

Activate the virtual environment with this command

```
source stablediff/bin/activate
```

You should see the name of your environment in parens before your prompt to know it works:

!["Install Stable Diffusion"][0]


Now, we'll install our dependencies!

## Step 2: Install Dependencies

Next, we need to install our dependencies:

```
pip install invisible_watermark transformers accelerate safetensors xformers
```

Lastly, install diffusers, since it will downgrade packages to make everything work together:

```
pip install diffusers
```

Now you're ready to create your Python file and generate some images!

## Step 3: Create a Simple Generator

Create a file named `app.py` or whatever you want. 

In that file, let's import our libraries:

```python
from diffusers import DiffusionPipeline
import torch
```

Next, we want to initialize the pipeline to generate images. We're going to use the `stable-diffusion-xl-base-1.0` pretrained model. 

We'll set our datatype to float16 for memory efficiency and enable the use of safetensors:

```python
pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
```

Next, we'll send the pipeline to the GPU:

```python
pipe.to("cuda")
```

And next, we'll have the text prompt to send to the model. This can be anything you want. 

```python
prompt = "A anthropomorphic poodle riding a dirt bike through the forest"
```

One thing you may want to add is enabling xformers for memory efficiency:

```python
pipe.enable_xformers_memory_efficient_attention()
```

Now, we can generate our image! 

```python
images = pipe(prompt=prompt).images[0]
```

Once the image is generated, we can save it: 

```python
images.save("output.png")
```

## Step 4: Run it! 

Now we run the file and get a cool image! 

You should see something like this at the prompt:

!["Install Stable Diffusion"][1]

and now we'll have an image (output.png) generated! 

!["Install Stable Diffusion"][2]

Pretty awesome, right? 

## Using Base + Refiner

You can get even better quality images using an "ensemble of experts" pattern with a base and a refiner:

```python
from diffusers import DiffusionPipeline
import torch

# load both base & refiner
base = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True
)
base.to("cuda")
refiner = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-refiner-1.0",
    text_encoder_2=base.text_encoder_2,
    vae=base.vae,
    torch_dtype=torch.float16,
    use_safetensors=True,
    variant="fp16",
)
refiner.to("cuda")

# Define how many steps and what % of steps to be run on each experts (80/20) here
n_steps = 40
high_noise_frac = 0.8

prompt = "A anthropomorphic poodle riding a dirt bike through the forest"

# run both experts
image = base(
    prompt=prompt,
    num_inference_steps=n_steps,
    denoising_end=high_noise_frac,
    output_type="latent",
).images
image = refiner(
    prompt=prompt,
    num_inference_steps=n_steps,
    denoising_start=high_noise_frac,
    image=image,
).images[0]
```

This produces some nice results, with settings you can tweak. 

!["Install Stable Diffusion"][3]

Awesome!

### Conclusion

This is the easiest, most low-overhead way I know to run Stable Diffusion XL. If you prefer, you can also use the Stable Diffusion Web UI, which gives you easy access to lots of controls and allows you to swap models and refiners easily. 

[Follow me](https://dev.to/jeremycmorgan) for more cool stuff like this! 

[0]: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ie90koa0y30j3932rzv9.png
[1]: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6igzht0wivdzqhp56jij.png
[2]: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5j6eqf1cgrfjayoixllc.png
[3]: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f4uohhby4pboo9uorrse.png