Repository: stable-diffusion-webui
Issue: [Bug]: getting a lot of memory errors lately after just a few generations
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

since SD came out until yesterday I was able to generate nearly infinite times on a single session, but after the last couple updates it can give me an out of memory error in as little as 4 to 5 generations.. sometimes if i wait a few minutes it will be good to go, but other times I have to restart the program.

could be an updated extension or the main program, not sure

### Steps to reproduce the problem

1. Go to .... 
2. Press ....
3. ...


### What should have happened?

not run out of memory

### Commit where the problem happens

2c1bb46c7ad5b4536f6587d327a03f0ff7811c5d

### What platforms do you use to access the UI ?

Windows

### What browsers do you use to access the UI ?

_No response_

### Command Line Arguments

```Shell
--api --cors-allow-origins http://localhost:5173 --deepdanbooru --administrator --no-half-vae --no-half --disable-safe-unpickle --xformers
```


### List of extensions

 ABG_extension/
 Config-Presets/
 DreamArtist-sd-webui-extension/
 Hypernetwork-MonkeyPatch-Extension/
 PromptGallery-stable-diffusion-webui/
 SD-latent-mirroring/
 StylePile/
 a1111-sd-webui-haku-img/
 a1111-sd-webui-tagcomplete/
 animator_extension/
 asymmetric-tiling-sd-webui/
 auto-sd-paint-ext/
 batch-face-swap/
 booru2prompt/
 custom-diffusion-webui/
 ddetailer/
 deforum-for-automatic1111-webui/
 depth-image-io-for-SDWebui/
 depthmap2mask/
 embedding-inspector/
 enhanced-img2img/
 extensions.py
 model-keyword/
 multi-subject-render/
 novelai-2-local-prompt/
 openOutpaint-webUI-extension/
 prompt-fusion-extension/
 prompt_gallery_name.json
'put extensions here.txt'
 sd-dynamic-prompts/
 sd-extension-aesthetic-scorer/
 sd-extension-steps-animation/
 sd-extension-system-info/
 sd-infinity-grid-generator-script/
 sd-web-ui-quickcss/
 sd-webui-additional-networks/
 sd-webui-gelbooru-prompt/
 sd-webui-model-converter/
 sd-webui-multiple-hypernetworks/
 sd-webui-riffusion/
 sd_dreambooth_extension/
 sd_grid_add_image_number/
 sd_web_ui_preset_utils/
 sdweb-merge-block-weighted-gui/
 sdweb-merge-board/
 seed_travel/
 shift-attention/
 stable-diffusion-webui-Prompt_Generator/
 stable-diffusion-webui-aesthetic-gradients/
 stable-diffusion-webui-artists-to-study/
 stable-diffusion-webui-auto-tls-https/
 stable-diffusion-webui-cafe-aesthetic/
 stable-diffusion-webui-conditioning-highres-fix/
 stable-diffusion-webui-daam/
 stable-diffusion-webui-dataset-tag-editor/
 stable-diffusion-webui-depthmap-script/
 stable-diffusion-webui-embedding-editor/
 stable-diffusion-webui-images-browser/
 stable-diffusion-webui-inspiration/
 stable-diffusion-webui-instruct-pix2pix/
 stable-diffusion-webui-pixelization/
 stable-diffusion-webui-promptgen/
 stable-diffusion-webui-tokenizer/
 stable-diffusion-webui-visualize-cross-attention-extension/
 stable-diffusion-webui-wd14-tagger/
 stable-diffusion-webui-wildcards/
 ultimate-upscale-for-automatic1111/
 unprompted/


### Console logs

```Shell
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:14<00:00,  1.39it/s]
Error completing request███████████████████████████████████████████████████████████████| 20/20 [00:13<00:00,  1.56it/s]
Arguments: ('task(ei4jtntmekaygim)', 'chemistry     Volumetric flask,\n    Erlenmeyer flasks,\npilling colorful bubbling chemicals\nboiling steam, calcinator, alembic', '', [], 20, 0, False, False, 1, 1, 7, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, [], 0, 0, 0, 0, 0, 0.25, False, True, False, 0, -1, True, 'keyword prompt', 'keyword1, keyword2', 'None', 'textual inversion first', True, False, 1, False, False, False, 1.1, 1.5, 100, 0.7, False, False, True, False, False, 0, 'Gustavosta/MagicPrompt-Stable-Diffusion', '', False, 'x264', 'mci', 10, 0, False, True, True, True, 'intermediate', 'animation', False, False, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'Refresh models', None, '', 'Get Tags', '', '', 0.9, 5, '0.0001', False, 'None', '', 0.1, False, 0, True, -1.0, False, '', True, False, 0, 384, 0, True, True, True, 1, '\n            <h3><strong>Combinations</strong></h3>\n            Choose a number of terms from a list, in this case we choose two artists\n            <code>{2$$artist1|artist2|artist3}</code>\n            If $$ is not provided, then 1$$ is assumed.\n            <br/><br/>\n\n            <h3><strong>Wildcards</strong></h3>\n            <p>Available wildcards</p>\n            <ul>\n        </ul>\n            <br/>\n            <code>WILDCARD_DIR: scripts/wildcards</code><br/>\n            <small>You can add more wildcards by creating a text file with one term per line and name is mywildcards.txt. Place it in scripts/wildcards. <code>__mywildcards__</code> will then become available.</small>\n        ', None, '', 'outputs', 1, '', 0, '', True, False, False, False, False, False, 1, 1, False, False, '', 1, True, 100, '', '', 8, True, 16, 'Median cut', 8, True, True, 16, 'PNN', True, False, None, None, '', '', '', '', 'Auto rename', {'label': 'Upload avatars config'}, 'Open outputs directory', 'Export to WebUI style', True, {'label': 'Presets'}, {'label': 'QC preview'}, '', [], 'Select', 'QC scan', 'Show pics', None, False, False, '', 25, True, 5.0, False, False, '', '', '', 'Positive', 0, ', ', True, 32, 0, 'Median cut', 'luminance', False, 'svg', True, True, False, 0.5, 1, '', 0, '', 0, '', True, False, False, False, False, 'Not set', True, True, '', '', '', '', '', 1.3, 'Not set', 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', False, 'None', '', 'None', 30, 4, 0, 0, False, 'None', '<br>', 'None', 30, 4, 0, 0, 4, 0.4, True, 32, None, True, '', 5, 24, 12.5, 1000, 'DDIM', 0, 64, 64, '', 64, 7.5, 0.42, 'DDIM', 64, 64, 1, 0, 92, True, True, True, False, False, False, 'midas_v21_small', False, True, False, True, True, [], False, '', True, False, 'D:\\stable-diffusion-webui\\extensions\\sd-webui-riffusion\\outputs', 'Refresh Inline Audio (Last Batch)', None, None, None, None, None, None, None, None, False, 4.0, '', 10.0, False, False, True, 30.0, True, False, False, 0, 0.0, 'Lanczos', 1, True, 10.0, True, 30.0, True, '', False, False, False, False, 'Auto', 0.5, 1, 0, 0, 512, 512, False, False, True, True, True, False, True, 1, False, False, 2.5, 4, 0, False, 0, 1, False, False, 'u2net', False, False, False, False, '{inspiration}', None) {}
Traceback (most recent call last):
  File "D:\stable-diffusion-webui\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "D:\stable-diffusion-webui\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "D:\stable-diffusion-webui\modules\txt2img.py", line 56, in txt2img
    processed = process_images(p)
  File "D:\stable-diffusion-webui\modules\processing.py", line 484, in process_images
    res = process_images_inner(p)
  File "D:\stable-diffusion-webui\modules\processing.py", line 628, in process_images_inner
    x_samples_ddim = [decode_first_stage(p.sd_model, samples_ddim[i:i+1].to(dtype=devices.dtype_vae))[0].cpu() for i in range(samples_ddim.size(0))]
  File "D:\stable-diffusion-webui\modules\processing.py", line 628, in <listcomp>
    x_samples_ddim = [decode_first_stage(p.sd_model, samples_ddim[i:i+1].to(dtype=devices.dtype_vae))[0].cpu() for i in range(samples_ddim.size(0))]
  File "D:\stable-diffusion-webui\modules\processing.py", line 422, in decode_first_stage
    x = model.decode_first_stage(x)
  File "D:\stable-diffusion-webui\modules\sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "D:\stable-diffusion-webui\modules\sd_hijack_utils.py", line 28, in __call__
    return self.__orig_func(*args, **kwargs)
  File "D:\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "D:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 826, in decode_first_stage
    return self.first_stage_model.decode(z)
  File "D:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\autoencoder.py", line 90, in decode
    dec = self.decoder(z)
  File "D:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "D:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 637, in forward
    h = self.up[i_level].block[i_block](h, temb)
  File "D:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "D:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 132, in forward
    h = nonlinearity(h)
  File "D:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 2059, in silu
    return torch._C._nn.silu(input)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 256.00 MiB (GPU 0; 6.00 GiB total capacity; 4.88 GiB already allocated; 0 bytes free; 5.24 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
```


### Additional information

if i wait a few minutes sometimes it works again, like it's taking a while to give back the vram

_No response_