Setting up Stable Diffusion for MacOS
Tags: python, ai
Published: 2022-08-26 13:48:00

> With the landscape quickly changing, this article is fast becoming outdated!
>
> If you face issues with the tutorial below I recommend you checkout the [latest advice here](https://github.com/invoke-ai/InvokeAI/blob/main/docs/installation/INSTALL_MAC.md).

---

[Stable Diffusion](https://github.com/CompVis/stable-diffusion) is a latent text-to-image diffusion model that was [recently made open source](https://stability.ai/blog/stable-diffusion-public-release).

For Linux users with dedicated NVDIA GPUs the instructions for setup and usage are relatively straight forward. However for MacOS users you can't use the project "out of the box". Not to worry! There are some steps to getting it working nevertheless!

## Environment Setup

To begin you need [Python](https://www.python.org/), [Conda](https://docs.conda.io/en/latest/), and a few other libraries set up:

```console
# Install Python, Cmake, Git, and Protobuf
brew install python \
  cmake \
  git \
  protobuf

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install Conda:
## Either use this for older "pre-M1" Macs:
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
bash Miniconda3-latest-MacOSX-x86_64.sh

## Or use this for older M1 Macs:
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
bash Miniconda3-latest-MacOSX-arm64.sh
```

You may need to restart your terminal at this point for the changes for the new libraries to be picked up.

Clone this fork of the project and checkout the apple patch branch:

```console
git clone https://github.com/magnusviri/stable-diffusion
cd stable-diffusion
git checkout apple-silicon-mps-support
```

At this point you will need to make sure you're using Python 3, [check out this article for different ways to make Python 3 the default version on your Mac](https://opensource.com/article/19/5/python-3-default-mac).

Set up the Conda environment:

```console
conda env create -f environment-mac.yaml
conda activate ldm
```

And finally set the following environment variable:

```console
export PYTORCH_ENABLE_MPS_FALLBACK=1
```

## Code Changes

Our environment is now set up, but we have a few tweaks that we need to allow the code to gracefully fallback to using the CPU (if required!).

Append `.contiguous()` at [ldm/models/diffusion/plms.py#L27](https://github.com/magnusviri/stable-diffusion/blob/6be63b5bbe95f04ad480e172d40994ecbe242b21/ldm/models/diffusion/plms.py#L27) resulting in:

```diff
-        attr = attr.to(torch.float32).to(torch.device(self.device_available))
+        attr = attr.to(torch.float32).to(torch.device(self.device_available)).contiguous()
```

Similarly append a new line `x = x.contiguous()` after [ldm/modules/attention.py#L211](https://github.com/magnusviri/stable-diffusion/blob/6be63b5bbe95f04ad480e172d40994ecbe242b21/ldm/modules/attention.py#L211) so it looks something like:

```diff
def _forward(self, x, context=None):
+       x = x.contiguous()
        x = self.attn1(self.norm1(x)) + x
```

## Download Stable Diffusion Weights

Let's install our diffusion weights

```console
curl "https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media" > sd-v1-4.ckpt
```

## Create Images :rocket:

You should now be ready to generate images on your MacOS device using Stable Diffusion! :tada: :tada:

```console
python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckpt sd-v1-4.ckpt --skip_grid --n_samples 1
```

![A drawing of web developers](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ynrp56ixjay83azo72ke.jpg)

---

Tricks and hacks gleamed from <https://github.com/CompVis/stable-diffusion/issues/25> - credit to all the folks in that thread for figuring out how to get things working!