Repository: stable-diffusion-webui
Issue: [Bug]: Generation frequently gets stuck 
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

Generations getting stuck randomly on arbitrary percentages of the progress bar, both on txt2img and img2img. This happens both for batch sizes or batch counts equato to 1 and greater than 1. All tested on 512x512 images with default checkpoint and several different samplers. Not using Restore Faces or High Res Fix, scripts, textual inversion or loras, just vanilla Automatic1111. The WebUI is still responsive after the generation gets stuck, but clicking Interrupt does nothing. 

Frequency of the issue is probably about 30% of generations, and requires full restart after it happens, so the application is pretty much unusable for me as of now.

Tried updating my Nvidia driver to Nvidia Studio Driver 536.40. I use an RTX 3060. Will try to downgrade the driver next and see if it fixes the issue.

### Steps to reproduce the problem

1. Go to txt2img or im2img tab and type prompts.
2. Press the Generate button.
3. Either individual image or batch total progress gets stuck both on the WebUI and on the command prompt.

### What should have happened?

Image should have been generated with progress bar completed.

### Version or Commit where the problem happens

1.4.0

### What Python version are you running on ?

Python 3.10.x

### What platforms do you use to access the UI ?

Windows

### What device are you running WebUI on?

Nvidia GPUs (RTX 20 above)

### Cross attention optimization

Automatic

### What browsers do you use to access the UI ?

Google Chrome

### Command Line Arguments

```Shell
--api
```


### List of extensions

No.

### Console logs

```Shell
Already up to date.
venv "E:\AI Generation\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Version: v1.4.0
Commit hash: 394ffa7b0a7fff3ec484bcd084e673a8b301ccc8
Installing requirements
Launching Web UI with arguments: --api
No module 'xformers'. Proceeding without it.
*** "Disable all extensions" option was set, will not load any extensions ***
Loading weights [6ce0161689] from E:\AI Generation\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
Creating model from config: E:\AI Generation\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode

Thanks for being a Gradio user! If you have questions or feedback, please join our Discord server and chat with us: https://discord.gg/feTf9x3ZSB
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
DiffusionWrapper has 859.52 M params.
Startup time: 5.0s (import torch: 1.3s, import gradio: 1.0s, import ldm: 0.4s, other imports: 0.7s, scripts list_optimizers: 0.4s, create ui: 0.7s, gradio launch: 0.2s).
preload_extensions_git_metadata for 17 extensions took 2.14s
Applying attention optimization: Doggettx... done.
Textual inversion embeddings loaded(0):
Model loaded in 3.7s (load weights from disk: 0.6s, find config: 0.1s, create model: 0.8s, apply weights to model: 0.7s, apply half(): 0.8s, move model to device: 0.6s).
100%|███████████████████████████████████████| 20/20 [00:10<00:00,  1.89it/s]
100%|███████████████████████████████████████| 20/20 [00:09<00:00,  2.21it/s]
100%|███████████████████████████████████████| 20/20 [00:04<00:00,  4.60it/s]
100%|███████████████████████████████████████| 20/20 [00:04<00:00,  4.52it/s]
100%|███████████████████████████████████████| 20/20 [00:04<00:00,  4.55it/s]
100%|███████████████████████████████████████| 20/20 [00:04<00:00,  4.37it/s]
100%|███████████████████████████████████████| 20/20 [00:04<00:00,  4.39it/s]
100%|███████████████████████████████████████| 20/20 [00:04<00:00,  4.38it/s]
100%|███████████████████████████████████████| 20/20 [00:04<00:00,  4.55it/s]
100%|███████████████████████████████████████| 20/20 [00:04<00:00,  4.39it/s]
Total progress: 100%|█████████████████████| 200/200 [01:00<00:00,  3.32it/s]
100%|███████████████████████████████████████| 20/20 [00:10<00:00,  1.89it/s]
100%|███████████████████████████████████████| 20/20 [00:11<00:00,  1.71it/s]
100%|███████████████████████████████████████| 20/20 [00:11<00:00,  1.74it/s]
Total progress:  30%|██████▌               | 60/200 [00:35<01:17,  1.80it/s]
```


### Additional information

![IssueStuk02](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/115922286/9debf44a-6e89-4561-93fa-634ec3216c75)
![IssueStuk01](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/115922286/43b1e8f1-3398-40f7-8d41-191aebea3fea)