Repository: stable-diffusion-webui
Issue: [Bug]: CUDA out of memory on "odd" resolutions in high res fix
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

When using high res fix I've noticed that "odd" resolutions or upscale factors completely throw VRAM allocation overboard trying to allocate ridiculous numbers like 50 GB of VRAM where higher upscale factors which end up in a larger higher res image work fine.

Normally I generate at 640x960, then upscale by x2.6 which results in 1664x2496 (4'153'344 pixels) on my 12GB 3080 Ti which is the max I can handle without getting this exception (maybe more would be possible if this bug didn't exist).

Now if I want to upscale by factor 2.55 (=1632x2448), I get:
CUDA out of memory. Tried to allocate 58.07 GiB (GPU 0; 12.00 GiB total capacity; 2.56 GiB already allocated; 6.11 GiB free; 3.60 GiB reserved in total by PyTorch)

This has always been like this for as long as I've been using A1111.

### Steps to reproduce the problem

- Generate image at 640x960
- high res fix at factor 2.55
- error

### What should have happened?

- no error
- correcto memory allocation/calculation

### Commit where the problem happens

20ae71f or earlier

### What Python version are you running on ?

Python 3.10.x

### What platforms do you use to access the UI ?

Windows

### What device are you running WebUI on?

Nvidia GPUs (RTX 20 above)

### What browsers do you use to access the UI ?

Mozilla Firefox

### Command Line Arguments

```Shell
default
```


### List of extensions

Config-Presets
a1111-sd-webui-tagcomplete
sd-dynamic-prompts
sd-dynamic-thresholding
sd-webui-additional-networks
sd-webui-controlnet
stable-diffusion-webui
ultimate-upscale-for-automatic1111

### Console logs

```Shell
2023-05-31 20:23:44,940 - sd_dynamic_prompts.dynamic_prompting - INFO - Prompt matrix will create 1 images in a total of 1 batches.
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00,  4.11it/s]
  0%|                                                                                                                                                                                                                                                                                                | 0/30 [00:11<?, ?it/s]
Error completing request
Arguments: ('task(4cw5tsivkkpftce)', 'masterpiece, best quality,a woman with long __hair-color-natural__ hair\nBREAK jeans and tee, lush garden', 'EasyNegative, drawn by bad-artist, sketch by bad-artist-anime, (bad_prompt:0.8), (artist name, signature, watermark:1.4), (ugly:1.2), (worst quality, poor details:1.4), bad-hands-5, badhandv4, blurry,', [], 30, 16, False, False, 1, 1, 6, -1.0, -1.0, 0, 0, 0, False, 960, 640, True, 0.34, 2.55, '4x-UltraSharp', 0, 0, 0, 0, '', '', [], 0, True, False, 1, False, False, False, 1.1, 1.5, 100, 0.7, False, False, True, False, False, 0, 'Gustavosta/MagicPrompt-Stable-Diffusion', '', False, 7, 100, 'Constant', 0, 'Constant', 0, 4, False, False, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, None, 'Refresh models', <controlnet.py.UiControlNetUnit object at 0x0000020DFFCED660>, <controlnet.py.UiControlNetUnit object at 0x0000020DFFCEE2F0>, <controlnet.py.UiControlNetUnit object at 0x0000020DFFCEC7F0>, <controlnet.py.UiControlNetUnit object at 0x0000020DFFCECB50>, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, None, None, False, None, None, False, None, None, False, None, None, False, 50) {}
Traceback (most recent call last):
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\call_queue.py", line 57, in f
    res = list(func(*args, **kwargs))
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\txt2img.py", line 57, in txt2img
    processed = processing.process_images(p)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\processing.py", line 611, in process_images
    res = process_images_inner(p)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
    return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\processing.py", line 729, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\processing.py", line 1059, in sample
    samples = self.sampler.sample_img2img(self, samples, noise, self.hr_c, self.hr_uc, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 356, in sample_img2img
    samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 257, in launch_sampling
    return func()
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 356, in <lambda>
    samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 594, in sample_dpmpp_2m
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 156, in forward
    x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict(c_crossattn, image_cond_in[a:b]))
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\external.py", line 112, in forward
    eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\external.py", line 138, in get_eps
    return self.inner_model.apply_model(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\sd_hijack_utils.py", line 28, in __call__
    return self.__orig_func(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
    x_recon = self.model(x_noisy, t, **cond)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
    out = self.diffusion_model(x, t, context=cc)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
    h = module(h, emb, context)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
    x = layer(x, context)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 334, in forward
    x = block(x, context=context[i])
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 269, in forward
    return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 121, in checkpoint
    return CheckpointFunction.apply(func, len(inputs), *args)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 136, in forward
    output_tensors = ctx.run_function(*ctx.input_tensors)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 272, in _forward
    x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\modules\sd_hijack_optimizations.py", line 247, in split_cross_attention_forward
    s1 = einsum('b i d, b j d -> b i j', q[:, i:end], k)
  File "E:\StableDiffusion\auto1111\stable-diffusion-webui\venv\lib\site-packages\torch\functional.py", line 378, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 58.07 GiB (GPU 0; 12.00 GiB total capacity; 2.49 GiB already allocated; 6.21 GiB free; 3.53 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
```


### Additional information

_No response_