internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: huggingface/diffusers

Issue:
ValueError on FlaxStableDiffusionPipeline
### Describe the bug

`FlaxStableDiffusionPipeline.__call__()` raises `ValueError: assignment destination is read-only`.

Probably NSFW filters are involved.

### Reproduction

- FlaxStableDiffusionPipeline
- 'CompVis/stable-diffusion-v1-4'
- revision='bf16'
- dtype=jnp.bfloat16
- prompt 'pretty asian girl' (no negative prompt)

### Logs

```shell
File "diffusers-api/diffusers_tasks.py", line 27, in generate
    images = app.pipeline(prompt_ids, p_params, rng, jit=True)[0]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".anyenv/envs/pyenv/versions/3.11.6/envs/diffusion/lib/python3.11/site-packages/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py", line 419, in __call__
    images[i] = np.asarray(images_uint8_casted[i])
    ~~~~~~^^^
ValueError: assignment destination is read-only
```


### System Info

jax/flax cuda environment
diffusers 0.21.4

### Who can help?

_No response_


Discussion:
Hey @YusukeSuzuki,

This error has nothing to do with `diffusers` per-se but instead with how Flax/JAX works.

Resolving patch:
diff --git a/src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py b/src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py
--- a/src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py
+++ b/src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py
@@ -410,13 +410,13 @@ def __call__(
 
             images_uint8_casted = np.asarray(images_uint8_casted).reshape(num_devices * batch_size, height, width, 3)
             images_uint8_casted, has_nsfw_concept = self._run_safety_checker(images_uint8_casted, safety_params, jit)
-            images = np.asarray(images)
+            images = np.asarray(images).copy()
 
             # block images
             if any(has_nsfw_concept):
                 for i, is_nsfw in enumerate(has_nsfw_concept):
                     if is_nsfw:
-                        images[i] = np.asarray(images_uint8_casted[i])
+                        images[i, 0] = np.asarray(images_uint8_casted[i])
 
             images = images.reshape(num_devices, batch_size, height, width, 3)
         else:


Test patch:
Repository: huggingface/diffusers

Issue:
Using pretrained MusicLDM pipeline
### Describe the bug

Hello, in diffusers docs, MusicLDM is used as follow:
```python
from diffusers import MusicLDMPipeline
import torch
import scipy

repo_id = "cvssp/audioldm-s-full-v2"
pipe = MusicLDMPipeline.from_pretrained(repo_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "Techno music with a strong, upbeat tempo and high melodic riffs"
audio = pipe(prompt, num_inference_steps=10, audio_length_in_s=5.0).audios[0]

# save the audio sample as a .wav file
scipy.io.wavfile.write("techno.wav", rate=16000, data=audio)
```

But the pretrained path  `cvssp/audioldm-s-full-v2` is seem only compatible with AudioLDM, and can not be used with MusicLDM.
Can you fix the usage of MusicLDM

### Reproduction

```python
from diffusers import MusicLDMPipeline
import torch
import scipy

repo_id = "cvssp/audioldm-s-full-v2"
pipe = MusicLDMPipeline.from_pretrained(repo_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "Techno music with a strong, upbeat tempo and high melodic riffs"
audio = pipe(prompt, num_inference_steps=10, audio_length_in_s=5.0).audios[0]

# save the audio sample as a .wav file
scipy.io.wavfile.write("techno.wav", rate=16000, data=audio)
```

### Logs

_No response_

### System Info

diffusers==0.23.0

### Who can help?

_No response_


Discussion:

Resolving patch:
diff --git a/src/diffusers/pipelines/musicldm/pipeline_musicldm.py b/src/diffusers/pipelines/musicldm/pipeline_musicldm.py
--- a/src/diffusers/pipelines/musicldm/pipeline_musicldm.py
+++ b/src/diffusers/pipelines/musicldm/pipeline_musicldm.py
@@ -51,7 +51,7 @@
         >>> import torch
         >>> import scipy
 
-        >>> repo_id = "cvssp/audioldm-s-full-v2"
+        >>> repo_id = "ucsd-reach/musicldm"
         >>> pipe = MusicLDMPipeline.from_pretrained(repo_id, torch_dtype=torch.float16)
         >>> pipe = pipe.to("cuda")
 


Test patch:
Repository: huggingface/diffusers

Issue:
`AutoPipeline.from_pipe` fails when passed a ControlNet pipeline
### Describe the bug

The implementation of `AutoPipeline` does a `replace` of `"Pipeline"` with `"ControlNetPipeline"` when the `controlnet` argument is passed.

When `from_pipe` is called with an existing `ControlNet` pipeline, the following transformation occurs.

```python
StableDiffusionControlNetPipeline --> StableDiffusionControlNetControlNetPipeline
```

This results in an error, as the `StableDiffusionControlNetControlNetPipeline` class does not exist.

### Reproduction

```python
from diffusers import AutoPipelineForText2Image, ControlNetModel

canny = ControlNetModel.from_pretrained("lllyasviel/control_v11p_sd15_canny")

# automatically chooses `StableDiffusionControlNetPipeline`
controlnet_pipe1 = AutoPipelineForText2Image.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=canny)

# replaces `Pipeline` with `ControlNetPipeline` -> `StableDiffusionControlNetControlNetPipeline`
controlnet_pipe2 = AutoPipelineForText2Image.from_pipe(controlnet_pipe1, controlnet=canny)
# ValueError: AutoPipeline can't find a pipeline linked to StableDiffusionControlNetControlNetPipeline for None
```

### Logs

```shell
ValueError: AutoPipeline can't find a pipeline linked to StableDiffusionControlNetControlNetPipeline for None
```


### System Info

- `diffusers` version: 0.21.4
- Platform: macOS-14.1-arm64-arm-64bit
- Python version: 3.10.6
- PyTorch version (GPU?): 2.0.1 (False)
- Huggingface_hub version: 0.16.4
- Transformers version: 4.30.2
- Accelerate version: 0.18.0
- xFormers version: 0.0.15.dev+71205ec.d20221126
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@yiyixuxu 


Discussion:
@yiyixuxu could you take a look here? 
Working great now, thanks!

Resolving patch:
diff --git a/src/diffusers/pipelines/auto_pipeline.py b/src/diffusers/pipelines/auto_pipeline.py
--- a/src/diffusers/pipelines/auto_pipeline.py
+++ b/src/diffusers/pipelines/auto_pipeline.py
@@ -24,6 +24,7 @@
     StableDiffusionControlNetInpaintPipeline,
     StableDiffusionControlNetPipeline,
     StableDiffusionXLControlNetImg2ImgPipeline,
+    StableDiffusionXLControlNetInpaintPipeline,
     StableDiffusionXLControlNetPipeline,
 )
 from .deepfloyd_if import IFImg2ImgPipeline, IFInpaintingPipeline, IFPipeline
@@ -97,6 +98,7 @@
         ("kandinsky", KandinskyInpaintCombinedPipeline),
         ("kandinsky22", KandinskyV22InpaintCombinedPipeline),
         ("stable-diffusion-controlnet", StableDiffusionControlNetInpaintPipeline),
+        ("stable-diffusion-xl-controlnet", StableDiffusionXLControlNetInpaintPipeline),
     ]
 )
 


Test patch:
Repository: huggingface/diffusers

Issue:
TensorRTStableDiffusionImg2ImgPipeline: can't set attribute 'guidance_scale'
### Describe the bug

`guidance_scale` became a not setable prop. So this example will report an error. 

### Reproduction

I'm using the example code:

https://github.com/huggingface/diffusers/blob/main/examples/community/README.md#tensorrt-image2image-stable-diffusion-pipeline

### Logs

```shell
Traceback (most recent call last):
  File "C:\Code\diffusers-playground\img2img\sdt.py", line 41, in <module>
    image = pipe(
  File "C:\Users\Jannchie\anaconda3\envs\streamdiffusion\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Code\diffusers-playground\img2img\utils\__init__.py", line 1007, in __call__
    self.guidance_scale = guidance_scale
  File "C:\Users\Jannchie\anaconda3\envs\streamdiffusion\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 607, in __setattr__
    super().__setattr__(name, value)
AttributeError: can't set attribute 'guidance_scale'
```
```


### System Info

- `diffusers` version: 0.24.0
- Platform: Windows-10-10.0.22631-SP0
- Python version: 3.10.13
- PyTorch version (GPU?): 2.1.0+cu121 (True)
- Huggingface_hub version: 0.20.1
- Transformers version: 4.36.2
- Accelerate version: 0.25.0
- xFormers version: 0.0.22.post7
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_


Discussion:

Resolving patch:
diff --git a/examples/community/stable_diffusion_tensorrt_img2img.py b/examples/community/stable_diffusion_tensorrt_img2img.py
--- a/examples/community/stable_diffusion_tensorrt_img2img.py
+++ b/examples/community/stable_diffusion_tensorrt_img2img.py
@@ -1004,7 +1004,7 @@ def __call__(
         """
         self.generator = generator
         self.denoising_steps = num_inference_steps
-        self.guidance_scale = guidance_scale
+        self._guidance_scale = guidance_scale
 
         # Pre-compute latent input scales and linear multistep coefficients
         self.scheduler.set_timesteps(self.denoising_steps, device=self.torch_device)


Test patch:
Repository: huggingface/diffusers

Issue:
gradient-checkpointing ignored in SDXL+LoRA training
There is `--gradient-checkpointing` option, but never handled. Is this intended?

https://github.com/huggingface/diffusers/blob/9f283b01d20aa8d9e5251dbaad19432bfb2ffe11/examples/text_to_image/train_text_to_image_lora_sdxl.py#L302-306


Discussion:
Ah, yeah seems like a bug. Would you be open to fix this with a PR? 

Resolving patch:
diff --git a/examples/text_to_image/train_text_to_image_lora.py b/examples/text_to_image/train_text_to_image_lora.py
--- a/examples/text_to_image/train_text_to_image_lora.py
+++ b/examples/text_to_image/train_text_to_image_lora.py
@@ -486,6 +486,9 @@ def main():
 
     lora_layers = filter(lambda p: p.requires_grad, unet.parameters())
 
+    if args.gradient_checkpointing:
+        unet.enable_gradient_checkpointing()
+
     # Enable TF32 for faster training on Ampere GPUs,
     # cf https://pytorch.org/docs/stable/notes/cuda.html#tensorfloat-32-tf32-on-ampere-devices
     if args.allow_tf32:
diff --git a/examples/text_to_image/train_text_to_image_lora_sdxl.py b/examples/text_to_image/train_text_to_image_lora_sdxl.py
--- a/examples/text_to_image/train_text_to_image_lora_sdxl.py
+++ b/examples/text_to_image/train_text_to_image_lora_sdxl.py
@@ -706,6 +706,12 @@ def load_model_hook(models, input_dir):
     accelerator.register_save_state_pre_hook(save_model_hook)
     accelerator.register_load_state_pre_hook(load_model_hook)
 
+    if args.gradient_checkpointing:
+        unet.enable_gradient_checkpointing()
+        if args.train_text_encoder:
+            text_encoder_one.gradient_checkpointing_enable()
+            text_encoder_two.gradient_checkpointing_enable()
+
     # Enable TF32 for faster training on Ampere GPUs,
     # cf https://pytorch.org/docs/stable/notes/cuda.html#tensorfloat-32-tf32-on-ampere-devices
     if args.allow_tf32:


Test patch:
Repository: huggingface/diffusers

Issue:
ValueError when `to('cuda')` on Value-guided planning pipeline
### Describe the bug

When I run the Value-guided planning pipeline colab <https://colab.research.google.com/drive/1rXm8CX4ZdN5qivjJ2lhwhkOmt_m0CvU0> with `to('cuda')` to speed up the diffusion process. It gave me a ValueError.

### Reproduction

<https://colab.research.google.com/drive/1SFl7daLQxd8QyHJP6ndSPznSFX2rppQ2> Code block 24 `pipeline.to(DEVICE)`

### Logs

```shell
ValueError: ValueGuidedRLPipeline {
  "_class_name": "ValueGuidedRLPipeline",
  "_diffusers_version": "0.25.0.dev0",
  "_name_or_path": "bglick13/hopper-medium-v2-value-function-hor32"
}
 has been incorrectly initialized or <class 'diffusers.experimental.rl.value_guided_sampling.ValueGuidedRLPipeline'> is incorrectly implemented. Expected {'env', 'unet', 'value_function', 'scheduler'} to be defined, but dict_keys([]) are defined.
```

### System Info

colab

### Who can help?

@yiyixuxu
ValueError when `to('cuda')` on Value-guided planning pipeline
### Describe the bug

When I run the Value-guided planning pipeline colab <https://colab.research.google.com/drive/1rXm8CX4ZdN5qivjJ2lhwhkOmt_m0CvU0> with `to('cuda')` to speed up the diffusion process. It gave me a ValueError.

### Reproduction

<https://colab.research.google.com/drive/1SFl7daLQxd8QyHJP6ndSPznSFX2rppQ2> Code block 24 `pipeline.to(DEVICE)`

### Logs

```shell
ValueError: ValueGuidedRLPipeline {
  "_class_name": "ValueGuidedRLPipeline",
  "_diffusers_version": "0.25.0.dev0",
  "_name_or_path": "bglick13/hopper-medium-v2-value-function-hor32"
}
 has been incorrectly initialized or <class 'diffusers.experimental.rl.value_guided_sampling.ValueGuidedRLPipeline'> is incorrectly implemented. Expected {'env', 'unet', 'value_function', 'scheduler'} to be defined, but dict_keys([]) are defined.
```

### System Info

colab

### Who can help?

@yiyixuxu


Discussion:

Resolving patch:
diff --git a/src/diffusers/experimental/rl/value_guided_sampling.py b/src/diffusers/experimental/rl/value_guided_sampling.py
--- a/src/diffusers/experimental/rl/value_guided_sampling.py
+++ b/src/diffusers/experimental/rl/value_guided_sampling.py
@@ -49,10 +49,9 @@ def __init__(
         env,
     ):
         super().__init__()
-        self.value_function = value_function
-        self.unet = unet
-        self.scheduler = scheduler
-        self.env = env
+
+        self.register_modules(value_function=value_function, unet=unet, scheduler=scheduler, env=env)
+
         self.data = env.get_dataset()
         self.means = {}
         for key in self.data.keys():


Test patch:
Repository: huggingface/diffusers

Issue:
[Tracker] change to posix for better Windows support
In https://github.com/huggingface/diffusers/pull/6564, @fabiorigano introduced the use of Posix to better support Windows compatibility. 

It'd be nice to change the instances of `os.path.join()` to `Path(...).as_posix()`. 

Feel free to open PRs for this and tag me. 

While opening PRs, please target only ONE script at a time. 

Let's go 🚀 



Discussion:
Hi @sayakpaul, I checked the mentioned PR and also understood the issue. This is a documentation change right? We need to change every instance of `os.path.join()` to `Path(...).as_posix()`?
> Hi @sayakpaul, I checked the mentioned PR and also understood the issue. This is a documentation change right? We need to change every instance of `os.path.join()` to `Path(...).as_posix()`?

Hey @Bhavay-2001 this is not a documentation change. Try going through this issue #6561 you'll understand why we are doing this.  
Hi @charchit7, yes I understood that this is to provide windows support for the adapter.py. I previously thought that it was a documentation change issue, however I have changed the file and opened the corresponding PR.
Thanks 
Hi, @sayakpaul! Does this issue request to change all os.path.join() instances in the whole Diffusers repo? Can I just take any script containing os.path.join() and change it?
I think so. 

Resolving patch:
diff --git a/src/diffusers/loaders/unet.py b/src/diffusers/loaders/unet.py
--- a/src/diffusers/loaders/unet.py
+++ b/src/diffusers/loaders/unet.py
@@ -16,6 +16,7 @@
 from collections import defaultdict
 from contextlib import nullcontext
 from functools import partial
+from pathlib import Path
 from typing import Callable, Dict, List, Optional, Union
 
 import safetensors
@@ -503,8 +504,9 @@ def save_function(weights, filename):
                 weight_name = CUSTOM_DIFFUSION_WEIGHT_NAME if is_custom_diffusion else LORA_WEIGHT_NAME
 
         # Save the model
-        save_function(state_dict, os.path.join(save_directory, weight_name))
-        logger.info(f"Model weights saved in {os.path.join(save_directory, weight_name)}")
+        save_path = Path(save_directory, weight_name).as_posix()
+        save_function(state_dict, save_path)
+        logger.info(f"Model weights saved in {save_path}")
 
     def fuse_lora(self, lora_scale=1.0, safe_fusing=False, adapter_names=None):
         self.lora_scale = lora_scale


Test patch:
Repository: huggingface/diffusers

Issue:
trained_betas param is not documented in DDPMScheduler class
### Describe the bug

trained_betas param is not documented in DDPMScheduler. Docstring should be updated here:

https://github.com/huggingface/diffusers/blob/main/src/diffusers/schedulers/scheduling_ddpm.py#L145

### Reproduction

see https://github.com/huggingface/diffusers/blob/main/src/diffusers/schedulers/scheduling_ddpm.py#L145

### Logs

_No response_

### System Info

any

### Who can help?

@stevhliu 


Discussion:
Good catch! Would you like to open a PR to add `trained_betas` to the docstring? 🤗 
I don't work on the dev version now. I'll let you beat me to it. Happy to
review though ;)

> Message ID: ***@***.***>
>


Resolving patch:
diff --git a/src/diffusers/schedulers/scheduling_ddpm.py b/src/diffusers/schedulers/scheduling_ddpm.py
--- a/src/diffusers/schedulers/scheduling_ddpm.py
+++ b/src/diffusers/schedulers/scheduling_ddpm.py
@@ -143,6 +143,8 @@ class DDPMScheduler(SchedulerMixin, ConfigMixin):
         beta_schedule (`str`, defaults to `"linear"`):
             The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from
             `linear`, `scaled_linear`, or `squaredcos_cap_v2`.
+        trained_betas (`np.ndarray`, *optional*):
+            An array of betas to pass directly to the constructor without using `beta_start` and `beta_end`.
         variance_type (`str`, defaults to `"fixed_small"`):
             Clip the variance when adding noise to the denoised sample. Choose from `fixed_small`, `fixed_small_log`,
             `fixed_large`, `fixed_large_log`, `learned` or `learned_range`.


Test patch:
Repository: huggingface/diffusers

Issue:
IP-Adapter support for StableDiffusionXLControlNetInpaintPipeline
Hello 👋 

**Is your feature request related to a problem? Please describe.**
`StableDiffusionXLControlNetPipeline` and `StableDiffusionXLControlNetImg2ImgPipeline` support IP-Adapter thanks to https://github.com/huggingface/diffusers/pull/6573. However, it is not added for `StableDiffusionXLControlNetInpaintPipeline` pipeline as I see. It would be very useful to have ControlNet inpainting pipeline support IP-Adapters.

**Describe the solution you'd like.**
IP-Adapter support for `StableDiffusionXLControlNetInpaintPipeline` as in `StableDiffusionXLControlNetPipeline` and `StableDiffusionXLControlNetImg2ImgPipeline`


Discussion:
hi @enesmsahin 
sure! do you want to open an PR for this? 
Hi @yiyixuxu I would like to work on this issue

Resolving patch:
diff --git a/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint_sd_xl.py b/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint_sd_xl.py
--- a/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint_sd_xl.py
+++ b/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint_sd_xl.py
@@ -28,7 +28,12 @@
 )
 
 from ...image_processor import PipelineImageInput, VaeImageProcessor
-from ...loaders import FromSingleFileMixin, StableDiffusionXLLoraLoaderMixin, TextualInversionLoaderMixin
+from ...loaders import (
+    FromSingleFileMixin,
+    IPAdapterMixin,
+    StableDiffusionXLLoraLoaderMixin,
+    TextualInversionLoaderMixin,
+)
 from ...models import AutoencoderKL, ControlNetModel, ImageProjection, UNet2DConditionModel
 from ...models.attention_processor import (
     AttnProcessor2_0,
@@ -146,7 +151,7 @@ def rescale_noise_cfg(noise_cfg, noise_pred_text, guidance_rescale=0.0):
 
 
 class StableDiffusionXLControlNetInpaintPipeline(
-    DiffusionPipeline, StableDiffusionXLLoraLoaderMixin, FromSingleFileMixin
+    DiffusionPipeline, StableDiffusionXLLoraLoaderMixin, FromSingleFileMixin, IPAdapterMixin
 ):
     r"""
     Pipeline for text-to-image generation using Stable Diffusion XL.
@@ -158,6 +163,7 @@ class StableDiffusionXLControlNetInpaintPipeline(
         - [`~loaders.StableDiffusionXLLoraLoaderMixin.load_lora_weights`] for loading LoRA weights
         - [`~loaders.StableDiffusionXLLoraLoaderMixin.save_lora_weights`] for saving LoRA weights
         - [`~loaders.FromSingleFileMixin.from_single_file`] for loading `.ckpt` files
+        - [`~loaders.IPAdapterMixin.load_ip_adapter`] for loading IP Adapters
 
     Args:
         vae ([`AutoencoderKL`]):


Test patch:
Repository: huggingface/diffusers

Issue:
use_linear_projection not in mid_block in unet_motion_model
### Describe the bug

https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/unets/unet_motion_model.py#L299

use_linear_projection not passed

### Reproduction

https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/unets/unet_motion_model.py#L299

use_linear_projection not passed

### Logs

_No response_

### System Info

https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/unets/unet_motion_model.py#L299

use_linear_projection not passed

### Who can help?

https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/unets/unet_motion_model.py#L299

use_linear_projection not passed


Discussion:
@jiarenyf Would you like to open a PR to fix? 
I'll go for this!

Resolving patch:
diff --git a/src/diffusers/models/unets/unet_motion_model.py b/src/diffusers/models/unets/unet_motion_model.py
--- a/src/diffusers/models/unets/unet_motion_model.py
+++ b/src/diffusers/models/unets/unet_motion_model.py
@@ -217,6 +217,7 @@ def __init__(
         use_motion_mid_block: int = True,
         encoder_hid_dim: Optional[int] = None,
         encoder_hid_dim_type: Optional[str] = None,
+        time_cond_proj_dim: Optional[int] = None,
     ):
         super().__init__()
 
@@ -252,9 +253,7 @@ def __init__(
         timestep_input_dim = block_out_channels[0]
 
         self.time_embedding = TimestepEmbedding(
-            timestep_input_dim,
-            time_embed_dim,
-            act_fn=act_fn,
+            timestep_input_dim, time_embed_dim, act_fn=act_fn, cond_proj_dim=time_cond_proj_dim
         )
 
         if encoder_hid_dim_type is None:
@@ -306,6 +305,7 @@ def __init__(
                 num_attention_heads=num_attention_heads[-1],
                 resnet_groups=norm_num_groups,
                 dual_cross_attention=False,
+                use_linear_projection=use_linear_projection,
                 temporal_num_attention_heads=motion_num_attention_heads,
                 temporal_max_seq_length=motion_max_seq_length,
             )
@@ -321,6 +321,7 @@ def __init__(
                 num_attention_heads=num_attention_heads[-1],
                 resnet_groups=norm_num_groups,
                 dual_cross_attention=False,
+                use_linear_projection=use_linear_projection,
             )
 
         # count how many layers upsample the images


Test patch: