Repository: huggingface/diffusers

Issue:
`num_images_per_prompt` > 1 still broken with `UnCLIPPipeline`
### Describe the bug

If I use the setting `num_images_per_prompt` with `UnCLIPPipeline` I get the following error:
```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[<ipython-input-12-a06ca1ef5374>](https://localhost:8080/#) in <module>
      6 pipe = pipe.to("cuda")
      7 
----> 8 images = pipe(prompt, num_images_per_prompt=9).images

2 frames
[/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py](https://localhost:8080/#) in decorate_context(*args, **kwargs)
     25         def decorate_context(*args, **kwargs):
     26             with self.clone():
---> 27                 return func(*args, **kwargs)
     28         return cast(F, decorate_context)
     29 

[/usr/local/lib/python3.8/dist-packages/diffusers/pipelines/unclip/pipeline_unclip.py](https://localhost:8080/#) in __call__(self, prompt, num_images_per_prompt, prior_num_inference_steps, decoder_num_inference_steps, super_res_num_inference_steps, generator, prior_latents, decoder_latents, super_res_latents, prior_guidance_scale, decoder_guidance_scale, output_type, return_dict)
    304         do_classifier_free_guidance = prior_guidance_scale > 1.0 or decoder_guidance_scale > 1.0
    305 
--> 306         text_embeddings, text_encoder_hidden_states, text_mask = self._encode_prompt(
    307             prompt, device, num_images_per_prompt, do_classifier_free_guidance
    308         )

[/usr/local/lib/python3.8/dist-packages/diffusers/pipelines/unclip/pipeline_unclip.py](https://localhost:8080/#) in _encode_prompt(self, prompt, device, num_images_per_prompt, do_classifier_free_guidance)
    184             text_encoder_hidden_states = torch.cat([uncond_text_encoder_hidden_states, text_encoder_hidden_states])
    185 
--> 186             text_mask = torch.cat([uncond_text_mask, text_mask])
    187 
    188         return text_embeddings, text_encoder_hidden_states, text_mask

RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 693 but got size 77 for tensor number 1 in the list.
``` 

This happens both on `0.11.0` and `main` to me. I understand https://github.com/huggingface/diffusers/pull/1762 made it into the release, but I'm afraid it may not have fixed it fully? 

PS: I can work around it by doing a
```py
images = pipe([prompt]*9).images
``` 


### Reproduction

```py
import torch
from diffusers import UnCLIPPipeline

prompt="The spirit of a tamagotchi wandering in the city of Vienna"
pipe = UnCLIPPipeline.from_pretrained("kakaobrain/karlo-v1-alpha", torch_dtype=torch.float16)
pipe = pipe.to("cuda")

images = pipe(prompt, num_images_per_prompt=9).images #Doing [prompt] changes the # of tensors mismatch in the error, but does not fix it
```

Gives me this error both locally and on [Google Colab](https://colab.research.google.com/drive/1iMBevEQCLELpdfEPvw4PQckq6yPzzoK7)

### Logs

_No response_

### System Info

diffusers==0.11.0 & main
transformers==4.25.1
safetensors==0.2.6
accelerate==0.15.0


Discussion:

Resolving patch:
diff --git a/src/diffusers/pipelines/unclip/pipeline_unclip.py b/src/diffusers/pipelines/unclip/pipeline_unclip.py
--- a/src/diffusers/pipelines/unclip/pipeline_unclip.py
+++ b/src/diffusers/pipelines/unclip/pipeline_unclip.py
@@ -173,7 +173,7 @@ def _encode_prompt(self, prompt, device, num_images_per_prompt, do_classifier_fr
             uncond_text_encoder_hidden_states = uncond_text_encoder_hidden_states.view(
                 batch_size * num_images_per_prompt, seq_len, -1
             )
-            uncond_text_mask = uncond_text_mask.repeat(1, num_images_per_prompt)
+            uncond_text_mask = uncond_text_mask.repeat_interleave(num_images_per_prompt, dim=0)
 
             # done duplicates
 


Test patch:
diff --git a/.github/workflows/nightly_tests.yml b/.github/workflows/nightly_tests.yml
--- a/.github/workflows/nightly_tests.yml
+++ b/.github/workflows/nightly_tests.yml
@@ -61,7 +61,6 @@ jobs:
       - name: Install dependencies
         run: |
           python -m pip install -e .[quality,test]
-          python -m pip install git+https://github.com/huggingface/accelerate
           python -m pip install -U git+https://github.com/huggingface/transformers
 
       - name: Environment
@@ -135,7 +134,6 @@ jobs:
           ${CONDA_RUN} python -m pip install --upgrade pip
           ${CONDA_RUN} python -m pip install -e .[quality,test]
           ${CONDA_RUN} python -m pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu
-          ${CONDA_RUN} python -m pip install git+https://github.com/huggingface/accelerate
 
       - name: Environment
         shell: arch -arch arm64 bash {0}
diff --git a/.github/workflows/pr_tests.yml b/.github/workflows/pr_tests.yml
--- a/.github/workflows/pr_tests.yml
+++ b/.github/workflows/pr_tests.yml
@@ -59,7 +59,6 @@ jobs:
       run: |
         apt-get update && apt-get install libsndfile1-dev -y
         python -m pip install -e .[quality,test]
-        python -m pip install git+https://github.com/huggingface/accelerate
         python -m pip install -U git+https://github.com/huggingface/transformers
 
     - name: Environment
@@ -127,7 +126,6 @@ jobs:
         ${CONDA_RUN} python -m pip install --upgrade pip
         ${CONDA_RUN} python -m pip install -e .[quality,test]
         ${CONDA_RUN} python -m pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu
-        ${CONDA_RUN} python -m pip install git+https://github.com/huggingface/accelerate
         ${CONDA_RUN} python -m pip install -U git+https://github.com/huggingface/transformers
 
     - name: Environment
diff --git a/.github/workflows/push_tests.yml b/.github/workflows/push_tests.yml
--- a/.github/workflows/push_tests.yml
+++ b/.github/workflows/push_tests.yml
@@ -61,7 +61,6 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install -e .[quality,test]
-        python -m pip install git+https://github.com/huggingface/accelerate
         python -m pip install -U git+https://github.com/huggingface/transformers
 
     - name: Environment
@@ -131,7 +130,6 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install -e .[quality,test,training]
-        python -m pip install git+https://github.com/huggingface/accelerate
         python -m pip install -U git+https://github.com/huggingface/transformers
 
     - name: Environment