Repository: stable-diffusion-webui
Issue: [Bug]: Generation can hang randomly when running with whole network no GPU
Labels: bug-report

### Checklist

- [X] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

When running generation on txt2img the generation can freeze at any point of generation. It does often happen on the last step. GPU is maxed out

### Steps to reproduce the problem

1. start Automatic1111 with SDXL
2. set prewiev to update with each pass
4. set preview quality to full
5. generate copule of 1024x1024 images (almost never happens on first one)

### What should have happened?

Images should generate correctly

### What browsers do you use to access the UI ?

Google Chrome

### Sysinfo

[sysinfo-2023-12-26-21-44.json](https://github.com/AUTOMATIC1111/stable-diffusion-webui/files/13773802/sysinfo-2023-12-26-21-44.json)


### Console logs

```Shell
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.7.0
Commit hash: cf2772fab0af5573da775e7437e6acdca424f26e
#######################################################################################################
Initializing Civitai Link
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Civitai Link revision: 136983e89859fd0477b4a437ed333142a6aa29a4
SD-WebUI revision: cf2772fab0af5573da775e7437e6acdca424f26e

Checking Civitai Link requirements...
[!] python-socketio[client] version 5.7.2 NOT installed.

#######################################################################################################
Launching Web UI with arguments: --xformers --no-half-vae
Style database not found: G:\sd\webui\styles.csv
CivitAI Browser+: Aria2 RPC started
ControlNet preprocessor location: G:\sd\webui\extensions\sd-webui-controlnet\annotator\downloads
2023-12-26 22:24:08,604 - ControlNet - INFO - ControlNet v1.1.422
2023-12-26 22:24:09,077 - ControlNet - INFO - ControlNet v1.1.422
Civitai: API loaded
[ControlNet-Travel] extension Mikubill/sd-webui-controlnet found, ControlNet-Travel loaded :)
Loading weights [31e35c80fc] from G:\sd\webui\models\Stable-diffusion\sd_xl_base_1.0.safetensors
WARNING:py.warnings:G:\sd\webui\extensions\stable-diffusion-webui-two-shot\scripts\two_shot.py:397: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  canvas_image = gr.Image(source='upload', mirror_webcam=False, type='numpy', tool='color-sketch',

WARNING:py.warnings:G:\sd\webui\extensions\stable-diffusion-webui-two-shot\scripts\two_shot.py:471: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  visual_regions = gr.Gallery(label="Regions").style(grid=(4, 4, 4, 8), height="auto")

WARNING:py.warnings:G:\sd\webui\extensions\stable-diffusion-webui-two-shot\scripts\two_shot.py:471: GradioDeprecationWarning: The 'grid' parameter will be deprecated. Please use 'columns' in the constructor instead.
  visual_regions = gr.Gallery(label="Regions").style(grid=(4, 4, 4, 8), height="auto")

WARNING:py.warnings:G:\sd\webui\extensions\weight_gradient\scripts\weight_gradient.py:269: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  gr.HighlightedText(label="Form",

Creating model from config: G:\sd\webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
WARNING:py.warnings:G:\sd\webui\extensions\stable-diffusion-webui-two-shot\scripts\two_shot.py:397: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  canvas_image = gr.Image(source='upload', mirror_webcam=False, type='numpy', tool='color-sketch',

WARNING:py.warnings:G:\sd\webui\extensions\stable-diffusion-webui-two-shot\scripts\two_shot.py:471: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  visual_regions = gr.Gallery(label="Regions").style(grid=(4, 4, 4, 8), height="auto")

WARNING:py.warnings:G:\sd\webui\extensions\stable-diffusion-webui-two-shot\scripts\two_shot.py:471: GradioDeprecationWarning: The 'grid' parameter will be deprecated. Please use 'columns' in the constructor instead.
  visual_regions = gr.Gallery(label="Regions").style(grid=(4, 4, 4, 8), height="auto")

WARNING:py.warnings:G:\sd\webui\extensions\weight_gradient\scripts\weight_gradient.py:269: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  gr.HighlightedText(label="Form",

WARNING:py.warnings:G:\sd\webui\extensions\infinite-zoom-automatic1111-webui\iz_helpers\ui.py:253: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  output_video = gr.Video(label="Output").style(width=512, height=512)

WARNING:py.warnings:G:\sd\webui\extensions\stable-diffusion-webui-Prompt_Generator\scripts\prompt_generator.py:229: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  row.style(equal_height=True)

Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Civitai: Check resources for missing info files
Civitai: Check resources for missing preview images
Startup time: 218.1s (initial startup: 0.5s, prepare environment: 56.2s, import torch: 36.0s, import gradio: 15.6s, setup paths: 17.8s, import ldm: 0.2s, initialize shared: 1.4s, other imports: 14.6s, setup codeformer: 2.5s, setup gfpgan: 0.7s, list SD models: 0.1s, load scripts: 14.0s, reload hypernetworks: 0.1s, initialize extra networks: 0.5s, scripts before_ui_callback: 0.2s, create ui: 57.6s, gradio launch: 1.2s).
Civitai: Found 0 resources missing info files
Civitai: No info found on Civitai
Civitai: Found 6 resources missing preview images
Civitai: No preview images found on Civitai
1001
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "G:\sd\system\python\lib\site-packages\uvicorn\protocols\websockets\websockets_impl.py", line 247, in run_asgi
    result = await self.app(self.scope, self.asgi_receive, self.asgi_send)
  File "G:\sd\system\python\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\starlette\middleware\errors.py", line 149, in __call__
    await self.app(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\starlette\middleware\cors.py", line 76, in __call__
    await self.app(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\starlette\middleware\gzip.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "G:\sd\system\python\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "G:\sd\system\python\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "G:\sd\system\python\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\starlette\routing.py", line 341, in handle
    await self.app(scope, receive, send)
  File "G:\sd\system\python\lib\site-packages\starlette\routing.py", line 82, in app
    await func(session)
  File "G:\sd\system\python\lib\site-packages\fastapi\routing.py", line 289, in app
    await dependant.call(**values)
  File "G:\sd\system\python\lib\site-packages\gradio\routes.py", line 604, in join_queue
    session_info = await asyncio.wait_for(
  File "asyncio\tasks.py", line 445, in wait_for
  File "G:\sd\system\python\lib\site-packages\starlette\websockets.py", line 133, in receive_json
    self._raise_on_disconnect(message)
  File "G:\sd\system\python\lib\site-packages\starlette\websockets.py", line 105, in _raise_on_disconnect
    raise WebSocketDisconnect(message["code"])
starlette.websockets.WebSocketDisconnect: 1001
1001
1001
1001
1001
Applying attention optimization: xformers... done.
Model loaded in 178.6s (load weights from disk: 11.4s, create model: 2.3s, apply weights to model: 159.9s, apply half(): 0.1s, apply dtype to VAE: 0.4s, move model to device: 0.2s, hijack: 0.1s, load textual inversion embeddings: 0.8s, calculate empty prompt: 3.3s).
 58%|███████████████████████████████████████████████▌                                  | 29/50 [05:51<04:00, 11.44s/it]
Total progress:  60%|███████████████████████████████████████▌                          | 30/50 [05:51<03:48, 11.43s/it]
```


### Additional information

I am sure it happened without extensions and even in fresh install (some time I did the test which might have been in faaf? version) (generation way faster when using VAE tiling extension or no extensions at all). (Maybe becouse some of them access the frame data way more?). Some data are in shared memory with CPU. GPU is at 100% when nothing appears to be happening.