Repository: stable-diffusion-webui
Issue: [Bug]: PLMS Sampler distortion at the last step
Labels: bug-report

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

After updating SD from v1.2.1 > 1.6.1 the sampler PLMS started to malfunction. First it looks like a VAE issue, but i tried about 3 different VAEs but no one changed anything at all to solve this issue. The whole Picture breaks with distortion.

### Steps to reproduce the problem

1. Update to v1.6.1/ Use a newer version as v1.2.1
2. Load your old prompt and seed you used into the txt2img from png info
3. Generate
4. Look how ugly it is
![00013-3287328300](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/77841885/50f6c52a-880e-490b-9223-bb44a107a458)

### What should have happened?

1. Update to v1.6.1
2. Load your old prompt and seed you used, into the txt2img from png info
3. Generate
4. Enjoy generating
![00035-3287328300](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/77841885/804714e3-948e-4838-89a2-6506c2705dbe)


### Sysinfo

[sysinfo-2023-12-06-21-11.txt](https://github.com/AUTOMATIC1111/stable-diffusion-webui/files/13591641/sysinfo-2023-12-06-21-11.txt)


### What browsers do you use to access the UI ?

Mozilla Firefox

### Console logs

```Shell
################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye)
################################################################

################################################################
Running on thefrieber user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
Create and activate python venv
################################################################

################################################################
Launching launch.py...
################################################################
Using TCMalloc: libtcmalloc_minimal.so.4
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Version: v1.6.1
Commit hash: 4afaaf8a020c1df457bcf7250cb1c7f609699fa7
Installing requirements
If submitting an issue on github, please provide the full startup log for debugging purposes.

Initializing Dreambooth
Dreambooth revision: c2a5617c587b812b5a408143ddfb18fc49234edf
Successfully installed accelerate-0.19.0 fastapi-0.94.1
[+] xformers version 0.0.20 installed.
[+] torch version 2.0.1+cu118 installed.
[+] torchvision version 0.15.2+cu118 installed.
[+] accelerate version 0.19.0 installed.
[+] diffusers version 0.16.1 installed.
[+] transformers version 4.30.2 installed.
[+] bitsandbytes version 0.35.4 installed.
Launching Web UI with arguments: --xformers --no-half-vae
Loading weights [3eede11bf8] from /home/thefrieber/Desktop/AUTO1111/stable-diffusion-webui/models/Stable-diffusion/fakePvcStyle_11.safetensors
Creating model from config: /home/thefrieber/Desktop/AUTO1111/stable-diffusion-webui/configs/v1-inference.yaml
Couldn't find VAE named anything-v4.0.vae.pt; using None instead
Model loaded in 1.3s (load weights from disk: 0.1s, create model: 0.5s, apply weights to model: 0.5s).
Applying attention optimization: xformers... done.
/home/thefrieber/Desktop/AUTO1111/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/main.py:257: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  with gr.Row().style(equal_height=False):
/home/thefrieber/Desktop/AUTO1111/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/main.py:1047: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  db_gallery = gr.Gallery(
/home/thefrieber/Desktop/AUTO1111/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/main.py:1047: GradioDeprecationWarning: The 'grid' parameter will be deprecated. Please use 'columns' in the constructor instead.
  db_gallery = gr.Gallery(
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 24.2s (prepare environment: 14.5s, import torch: 2.2s, import gradio: 0.8s, setup paths: 1.8s, initialize shared: 0.2s, other imports: 0.4s, load scripts: 2.3s, create ui: 0.7s, gradio launch: 1.2s).
100%|███████████████████████████████████████████| 24/24 [00:01<00:00, 13.51it/s]
100%|███████████████████████████████████████████| 18/18 [00:02<00:00,  7.38it/s]
Total progress:  93%|█████████████████████████▏ | 42/45 [00:05<00:00,  7.96it/s]
100%|███████████████████████████████████████████| 24/24 [00:01<00:00, 15.63it/s]
100%|███████████████████████████████████████████| 18/18 [00:02<00:00,  7.81it/s]
Total progress:  93%|█████████████████████████▏ | 42/45 [00:04<00:00,  9.18it/s]
100%|███████████████████████████████████████████| 24/24 [00:01<00:00, 15.54it/s]
100%|███████████████████████████████████████████| 18/18 [00:02<00:00,  7.77it/s]
Total progress:  93%|█████████████████████████▏ | 42/45 [00:04<00:00,  9.19it/s]
^CInterrupted with signal 2 in <frame at 0x564c6c76fd50, file '/usr/lib/python3.10/threading.py', line 324, code wait>
```


### Additional information

-Used with and without "Use old emphasis implementation. Can be useful to reproduce old seeds."
-Used different VAEs
-Generated with a rx580 8gb gpu(old, ROCm) now generating with rtx 4070 12gb edition(CUDA, still the same, new version malfunctions)
-Tried to play around with clip skip 1 and 2
-Looked if Face restoration was the issue