Repository: stable-diffusion-webui
Issue: [Bug]: RuntimeWarning: Invalid Value encountered in cast x_sample = x_sample.astype(np.uint8)
Labels: bug-report

### Checklist

- [ ] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [ ] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

After trying to run txt2img I am getting this error and at the end I endup with just a black image "C:\Users\Nikola\stable-diffusion-webui\modules\processing.py:966: RuntimeWarning: invalid value encountered in cast
  x_sample = x_sample.astype(np.uint8)" I have tried to remove my VAE, because at first I thought it was the problem, but than it happened again without it too. What more, when I tried it, it sometimes worked without errors, and sometimes with them. Sometimes this line with "processing.py:966" is "py:68" if it helps 

### Steps to reproduce the problem

1. I start the webui-user.bat file
2. Put my input: score_9, score_8_up, score_7_up BREAK handsomize, 2boys, son goku, vegeta, dragon ball z, outdoors <lora:handsomize_x_pdxl_v1:1.2>
3. Set sampling steps to 34
4. Change image size to anything higher than 768x1024(w,h)
5. VA I am using is: sharpspectrumvae_v10.ckpt (SharpSpectrumVAE this is what is called on CivitAI, it fixes faces on my art there)
6. Run
7. Gets error after first or second try, it's about my luck

### What should have happened?

I should get an image of something similar to what I wanted, but instead I get black image with errors.

### What browsers do you use to access the UI ?

Google Chrome

### Sysinfo

[sysinfo-2024-03-30-13-52.json](https://github.com/AUTOMATIC1111/stable-diffusion-webui/files/14812162/sysinfo-2024-03-30-13-52.json)


### Console logs

```Shell
Already up to date.
venv "C:\Users\Nikola\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.8.0
Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5
Launching Web UI with arguments: --disable-nan-check
No module 'xformers'. Proceeding without it.
Loading weights [fbcf965a62] from C:\Users\Nikola\stable-diffusion-webui\models\Stable-diffusion\anythingelseV4_v45.ckpt
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 13.7s (prepare environment: 3.0s, import torch: 4.9s, import gradio: 1.4s, setup paths: 1.5s, initialize shared: 0.3s, other imports: 0.7s, load scripts: 1.0s, create ui: 0.5s, gradio launch: 0.2s).
Creating model from config: C:\Users\Nikola\stable-diffusion-webui\configs\v1-inference.yaml
Loading VAE weights specified in settings: C:\Users\Nikola\stable-diffusion-webui\models\VAE\sharpspectrumvae_v10.ckpt
Applying attention optimization: Doggettx... done.
Model loaded in 24.7s (load weights from disk: 21.2s, create model: 0.4s, apply weights to model: 1.5s, load VAE: 1.0s, calculate empty prompt: 0.5s).
100%|██████████████████████████████████████████████████████████████████████████████████| 34/34 [04:50<00:00,  8.54s/it]
C:\Users\Nikola\stable-diffusion-webui\modules\processing.py:966: RuntimeWarning: invalid value encountered in cast/it]
  x_sample = x_sample.astype(np.uint8)
Total progress: 100%|██████████████████████████████████████████████████████████████████| 34/34 [05:43<00:00, 10.10s/it]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 34/34 [05:43<00:00,  8.08s/it]
```


### Additional information

I have updated my GPU Drivers ad my GPU is GTX 1660Ti