Repository: stable-diffusion-webui
Issue: [Bug]: Certain specific resolutions triggering CUDA out of memory
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

I have been able to generate images up to 1900x1900 on an RTX 4080 run locally. I've been running experiments at larger sizes for printing purposes generally at 8:10 for framing. Something I've just noticed recently is that I can generate images that are 1632x2048 or 1648x2048 pixels, however if I try to generate at a proper 8x10 ratio of 1632x2040 I trigger the CUDA out of memory error (even though it is a smaller resolution). Since the time I was first generating 1900x1900 and 1632x2040 images, I have downloaded many extensions and had to edit the requirements_versions.txt in order to get Dreambooth working and I'm wondering if this might have something to do with it. Has anyone noticed anything similar? I realize most people don't bother with larger resolutions because it mostly spits out garbage at that size, but you'd be surprised at what is possible if you're patient.

### Steps to reproduce the problem

Try generating 1632x2048 and 1632x2040 and see what happens.

### What should have happened?

CUDA should not be out of memory as I'm generating larger images perfectly fine.
![IMG_2089](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/49330152/af24bf3e-e2d6-4c35-9912-effbd04a4943)


### Commit where the problem happens

A1111 version 1.3.1 and SD version 2.1-768

### What Python version are you running on ?

Python 3.10.x

### What platforms do you use to access the UI ?

Windows

### What device are you running WebUI on?

Nvidia GPUs (RTX 20 above)

### What browsers do you use to access the UI ?

Google Chrome

### Command Line Arguments

```Shell
No but I edited the requirements_versions.txt:

blendmodes==2022
transformers==4.25.1
accelerate==0.18.0
basicsr==1.4.2
gfpgan==1.3.8
gradio==3.31.0
numpy==1.23.5
Pillow==9.5.0
realesrgan==0.3.0
torch
omegaconf==2.2.3
pytorch_lightning==1.9.4
scikit-image==0.20.0
timm==0.6.7
piexif==1.1.3
einops==0.4.1
jsonmerge==1.8.0
clean-fid==0.1.35
resize-right==0.0.2
torchdiffeq==0.2.3
kornia==0.6.7
lark==1.1.2
inflection==0.5.1
GitPython==3.1.30
torchsde==0.2.5
safetensors==0.3.1
httpcore<=0.15
fastapi==0.94.0
tomesd==0.1.2
```


### List of extensions

Deforum, txt2video, Dreambooth, Prompt Generator, Wildcards Manager

### Console logs

```Shell
Arguments: ('task(9wsubu1qpoq7kdg)', 'a __trev/color/type__ __jumbo/medium/photography/filmtypes__ film of __trev/locations/destinations__ by __artists/Photography/by_country/canada__', 'text, watermark, soft, out of focus, blurry', [], 60, 4, False, False, 100, 1, 5.5, -1.0, -1.0, 0, 0, 0, False, 2040, 1632, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], 0, True, False, 1, False, False, False, 1.1, 1.5, 100, 0.7, False, False, True, False, False, 0, 'Gustavosta/MagicPrompt-Stable-Diffusion', '', <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x000002A5A2739AB0>, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, None, None, False, 50) {}
Traceback (most recent call last):
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\call_queue.py", line 57, in f
    res = list(func(*args, **kwargs))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\txt2img.py", line 57, in txt2img
    processed = processing.process_images(p)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 610, in process_images
    res = process_images_inner(p)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
    return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 728, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 976, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 383, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 257, in launch_sampling
    return func()
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 383, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\sampling.py", line 198, in sample_dpm_2
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 137, in forward
    x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict([cond_in], image_cond_in))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\external.py", line 167, in forward
    return self.get_v(input * c_in, self.sigma_to_t(sigma), **kwargs) * c_out + input * c_skip
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\external.py", line 177, in get_v
    return self.inner_model.apply_model(x, t, cond)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_utils.py", line 28, in __call__
    return self.__orig_func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
    x_recon = self.model(x_noisy, t, **cond)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
    out = self.diffusion_model(x, t, context=cc)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
    h = module(h, emb, context)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
    x = layer(x, context)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 334, in forward
    x = block(x, context=context[i])
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 269, in forward
    return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 121, in checkpoint
    return CheckpointFunction.apply(func, len(inputs), *args)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 136, in forward
    output_tensors = ctx.run_function(*ctx.input_tensors)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 272, in _forward
    x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_optimizations.py", line 247, in split_cross_attention_forward
    s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\functional.py", line 378, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 100.81 GiB (GPU 0; 15.99 GiB total capacity; 3.28 GiB already allocated; 10.13 GiB free; 3.49 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

INFO:sd_dynamic_prompts.dynamic_prompting:Prompt matrix will create 100 images in a total of 100 batches.
  0%|                                                                                           | 0/60 [00:03<?, ?it/s]
Error completing request
Arguments: ('task(m2jbdkx91dn2fi7)', 'a __trev/color/type__ __jumbo/medium/photography/filmtypes__ film of __trev/locations/destinations__ by __artists/Photography/by_country/canada__', 'text, watermark, soft, out of focus, blurry', [], 60, 4, False, False, 100, 1, 5.5, -1.0, -1.0, 0, 0, 0, False, 2040, 1632, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], 0, True, False, 1, False, False, False, 1.1, 1.5, 100, 0.7, False, False, True, False, False, 0, 'Gustavosta/MagicPrompt-Stable-Diffusion', '', <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x000002A852757730>, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, None, None, False, 50) {}
Traceback (most recent call last):
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\call_queue.py", line 57, in f
    res = list(func(*args, **kwargs))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\txt2img.py", line 57, in txt2img
    processed = processing.process_images(p)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 610, in process_images
    res = process_images_inner(p)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
    return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 728, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 976, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 383, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 257, in launch_sampling
    return func()
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 383, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\sampling.py", line 198, in sample_dpm_2
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 137, in forward
    x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict([cond_in], image_cond_in))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\external.py", line 167, in forward
    return self.get_v(input * c_in, self.sigma_to_t(sigma), **kwargs) * c_out + input * c_skip
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\external.py", line 177, in get_v
    return self.inner_model.apply_model(x, t, cond)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_utils.py", line 28, in __call__
    return self.__orig_func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
    x_recon = self.model(x_noisy, t, **cond)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
    out = self.diffusion_model(x, t, context=cc)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
    h = module(h, emb, context)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
    x = layer(x, context)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 334, in forward
    x = block(x, context=context[i])
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 269, in forward
    return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 121, in checkpoint
    return CheckpointFunction.apply(func, len(inputs), *args)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 136, in forward
    output_tensors = ctx.run_function(*ctx.input_tensors)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 272, in _forward
    x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_optimizations.py", line 247, in split_cross_attention_forward
    s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\functional.py", line 378, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 50.41 GiB (GPU 0; 15.99 GiB total capacity; 3.03 GiB already allocated; 10.38 GiB free; 3.25 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

INFO:sd_dynamic_prompts.dynamic_prompting:Prompt matrix will create 100 images in a total of 100 batches.
  0%|                                                                                           | 0/60 [00:01<?, ?it/s]
Error completing request
Arguments: ('task(m7c3yglnauks31s)', 'a __trev/color/type__ __jumbo/medium/photography/filmtypes__ film of __trev/locations/destinations__ by __artists/Photography/by_country/canada__', 'text, watermark, soft, out of focus, blurry', [], 60, 4, False, False, 100, 1, 5.5, -1.0, -1.0, 0, 0, 0, False, 2040, 1632, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], 0, True, False, 1, False, False, False, 1.1, 1.5, 100, 0.7, False, False, True, False, False, 0, 'Gustavosta/MagicPrompt-Stable-Diffusion', '', <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x000002A852755ED0>, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, None, None, False, 50) {}
Traceback (most recent call last):
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\call_queue.py", line 57, in f
    res = list(func(*args, **kwargs))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\txt2img.py", line 57, in txt2img
    processed = processing.process_images(p)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 610, in process_images
    res = process_images_inner(p)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
    return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 728, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\processing.py", line 976, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 383, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 257, in launch_sampling
    return func()
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 383, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\sampling.py", line 198, in sample_dpm_2
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_samplers_kdiffusion.py", line 137, in forward
    x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict([cond_in], image_cond_in))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\external.py", line 167, in forward
    return self.get_v(input * c_in, self.sigma_to_t(sigma), **kwargs) * c_out + input * c_skip
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\k-diffusion\k_diffusion\external.py", line 177, in get_v
    return self.inner_model.apply_model(x, t, cond)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_utils.py", line 28, in __call__
    return self.__orig_func(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
    x_recon = self.model(x_noisy, t, **cond)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
    out = self.diffusion_model(x, t, context=cc)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
    h = module(h, emb, context)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
    x = layer(x, context)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 334, in forward
    x = block(x, context=context[i])
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 269, in forward
    return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 121, in checkpoint
    return CheckpointFunction.apply(func, len(inputs), *args)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 136, in forward
    output_tensors = ctx.run_function(*ctx.input_tensors)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 272, in _forward
    x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\modules\sd_hijack_optimizations.py", line 247, in split_cross_attention_forward
    s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k)
  File "D:\Stable Diffusion\stable-diffusion-webui-1.3.1\venv\lib\site-packages\torch\functional.py", line 378, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 50.41 GiB (GPU 0; 15.99 GiB total capacity; 3.03 GiB already allocated; 10.38 GiB free; 3.25 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
```


### Additional information

_No response_