Repository: huggingface/diffusers

Issue:
StableDiffusionPipeline producing unexpected output with MPS device using diffusers==0.4.0
### Describe the bug

I tried testing the potential speed updates of diffusers 0.4.0 on my M1 mac using an existing StableDiffusionPipeline-based script, and I found that a large image that would take ~3 min to generate in diffusers 0.3.0 was estimated to take more than 10x as long.

Since my existing script had a lot going on (e.g. large resolutions, attention slicing), I tried to diagnose the problem with a minimal script (see below), running in two identical environments, with the only difference being the diffusers version. 

In diffusers 0.3.0, it takes ~35 seconds to generate a reasonable result like this:
![test_0 3 0](https://user-images.githubusercontent.com/31020859/194449869-a3d8d0ef-0408-4fac-bb5c-5dc6813435de.png)

In diffusers 0.4.0, it takes ~50 seconds (which is slower than 0.3.0, but better that the 10x performance hit I was getting before), but each attempt (with varying seeds) triggered the NSFW filter. Disabling the filter, the results appear to be just noise:
![test_0 4 0](https://user-images.githubusercontent.com/31020859/194450034-4b4b67d8-8076-4391-9c00-2f8a3cce0aff.png)

I'm not sure if the initial 10x performance hit I initially observed in my original script would be fixed by fixing this bug, but this certainly seems to be at least part of it. 



### Reproduction

import torch
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4")
pipe.to("mps")

result = pipe("dogs playing poker", generator=torch.manual_seed(1))

result.images[0].save("test.png")

### Logs

```shell
Under 0.4.0 there's also this warning:

/opt/homebrew/Caskroom/miniforge/base/envs/sd/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py:222: UserWarning: The operator 'aten::repeat_interleave.self_int' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/mps/MPSFallback.mm:11.)
  text_embeddings = text_embeddings.repeat_interleave(num_images_per_prompt, dim=0)
```


### System Info

- `diffusers` version: 0.4.0
- Platform: macOS-12.6-arm64-arm-64bit
- Python version: 3.10.6
- PyTorch version (GPU?): 1.13.0.dev20220911 (False)
- Huggingface_hub version: 0.10.0
- Transformers version: 4.21.3
- Using GPU in script?: MPS
- Using distributed or parallel set-up in script?: no


Discussion:
Hi @jncasey, thanks for reporting this!

From the details you describe it looks like one of the last-minute optimizations we did for the release is not compatible with `mps`, so it's  falling back to the CPU and that's why it's slower. It's probably the reason why it fails to generate images, I'll test and work on a fix today.

I do apologize for the trouble, I tested on my Mac a few hours before the release, but not after all the changes were merged together. Thanks again for letting us know!
(For reference, I believe this is what's affecting `mps`: https://github.com/huggingface/diffusers/pull/741#discussion_r988907920)

Resolving patch:
diff --git a/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py b/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
--- a/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
+++ b/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
@@ -218,8 +218,10 @@ def __call__(
             text_input_ids = text_input_ids[:, : self.tokenizer.model_max_length]
         text_embeddings = self.text_encoder(text_input_ids.to(self.device))[0]
 
-        # duplicate text embeddings for each generation per prompt
-        text_embeddings = text_embeddings.repeat_interleave(num_images_per_prompt, dim=0)
+        # duplicate text embeddings for each generation per prompt, using mps friendly method
+        bs_embed, seq_len, _ = text_embeddings.shape
+        text_embeddings = text_embeddings.repeat(1, num_images_per_prompt, 1)
+        text_embeddings = text_embeddings.view(bs_embed * num_images_per_prompt, seq_len, -1)
 
         # here `guidance_scale` is defined analog to the guidance weight `w` of equation (2)
         # of the Imagen paper: https://arxiv.org/pdf/2205.11487.pdf . `guidance_scale = 1`
@@ -256,8 +258,10 @@ def __call__(
             )
             uncond_embeddings = self.text_encoder(uncond_input.input_ids.to(self.device))[0]
 
-            # duplicate unconditional embeddings for each generation per prompt
-            uncond_embeddings = uncond_embeddings.repeat_interleave(batch_size * num_images_per_prompt, dim=0)
+            # duplicate unconditional embeddings for each generation per prompt, using mps friendly method
+            seq_len = uncond_embeddings.shape[1]
+            uncond_embeddings = uncond_embeddings.repeat(batch_size, num_images_per_prompt, 1)
+            uncond_embeddings = uncond_embeddings.view(batch_size * num_images_per_prompt, seq_len, -1)
 
             # For classifier free guidance, we need to do two forward passes.
             # Here we concatenate the unconditional and text embeddings into a single batch


Test patch: