Repository: stable-diffusion-webui
Issue: [Bug]: 1.6.0 Processing negative embeddings wrong/different?
Labels: bug

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

Upgraded from 1.5.1 to 1.6.0 and can no longer reproduce images from 1.5.1

I am aware that some changes happened with the DDIM sampler, however I do not believe that they should be causing this problem. (See below)

The following images are made with the exact same prompts, negative prompts, settings, embeddings, etc.

| The first is from version 1.5.1| The second is with 1.6.0 |
| - | - |
| ![1 5 1-with-embeddings](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130285881/c9ce74b9-fafe-41b3-bb05-83fca38a4da2) | ![1 6 0-with-embeddings](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130285881/fb12d6b1-f77a-456e-853d-67564120eae1) |

The differences are quite noticeable. After some searching around other issues trying to see if others were having similar problems and tested various compatibility settings and other samplers. This is the result from both 1.5.1 and 1.6.0 with all embeddings removed from the negative prompts. Nothing else was changed.

| 1.5.1 without-embeddings | 1.6.0 without-embeddings |
| - | - |
![1 5 1-without-embeddings](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130285881/da4f97e8-fc09-4935-a891-55eb9aa538f6) | ![1 6 0-without-embeddings](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130285881/6a917267-bef5-4728-91be-0eeb325609a4) |

My initial findings seemed to point to the changes to the DDIM sampler in the update so also tested with various other samplers. Results are similar although not quite as drastic. These images are using Euler a using embeddings and appear very similar but are still different.
| 1.5.1 euler with embeddings | 1.6.0 euler with embeddings |
| - | - |
| ![1 5 1-euler-a-with-embeddings](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130285881/77744170-3041-419a-b2b9-2c00aa7557a9) | ![1 6 0-euler-a-with-embeddings](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/130285881/d63ca2d4-a8fe-4edd-a099-b01fc0e8e3d4) |


### Steps to reproduce the problem

1. Generate image in 1.5.1 with negative embedding 
2. Generate image in 1.6.0 with exact same values as the previous image
3. Notice they're different
4. Redo steps 1-2 with no negative embeddings
5. Notice they are now identical.


### What should have happened?

It should produce the exact same image.

### Sysinfo

[sysinfo#13150.txt](https://github.com/AUTOMATIC1111/stable-diffusion-webui/files/12564059/sysinfo.13150.2.txt)


### What browsers do you use to access the UI ?

Google Chrome

### Console logs

```Shell
venv "D:\AI\Auto1111\stable-diffusion-webui-1.6.0\venv\Scripts\Python.exe"
Python 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Launching Web UI with arguments: --xformers --opt-channelslast --ckpt-dir D:\AI\Models\checkpoints --hypernetwork-dir D:\AI\Models\hypernetworks --embeddings-dir D:\AI\Models\embeddings --lora-dir D:\AI\Models\loras --vae-dir D:\AI\Models\vae --realesrgan-models-path D:\AI\Models\upscalers --esrgan-models-path D:\AI\Models\upscalers
Loading weights [953a5db015] from D:\AI\Models\checkpoints\AlinaDarkScienceMix-VAE-f16.safetensors
Running on local URL:  http://127.0.0.1:7861

To create a public link, set `share=True` in `launch()`.
Creating model from config: D:\AI\Models\checkpoints\AlinaDarkScienceMix-VAE-f16.yaml
Startup time: 8.4s (prepare environment: 1.4s, import torch: 2.2s, import gradio: 0.6s, setup paths: 0.5s, initialize shared: 0.2s, other imports: 2.2s, load scripts: 0.5s, create ui: 0.3s, gradio launch: 0.3s).
Applying attention optimization: xformers... done.
Model loaded in 4.1s (load weights from disk: 0.6s, create model: 0.2s, apply weights to model: 1.7s, apply channels_last: 0.8s, calculate empty prompt: 0.7s).
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00,  9.89it/s]
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:03<00:00,  2.10it/s]
Total progress:  95%|██████████████████████████████████████████████████████████████▋   | 38/40 [00:32<00:01,  1.17it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 12.05it/s]
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:03<00:00,  2.20it/s]
Total progress:  95%|██████████████████████████████████████████████████████████████▋   | 38/40 [00:33<00:01,  1.13it/s]
Total progress:  95%|██████████████████████████████████████████████████████████████▋   | 38/40 [00:33<00:01,  1.29it/s]
```


### Additional information

Both versions tested are clean installs for all tests.