Repository: AUTOMATIC1111/stable-diffusion-webui

Issue:
[Bug]: x/y Checkpoint plot giving wrong results - wrong ckpt weight's loaded from cache if checkpoint cache is enabled in settings (>2)
### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

Using x/y plot with multiple checkpoints and with the same seed, does not generate consistent results.
It seems that the checkpoint order gets swapped around with each button press.
And the currently selected checkpoints also plays a role, it seems the weight from the currently selected checkpoint is mixed into the plot.

The result changes depending on which checkpoint is currently selected, even though, the selected checkpoint should not matter in this case, since it should only use the weights from the x/y plot checkpoints

### Steps to reproduce the problem

1. in the settings, set _Checkpoints to cache in RAM_ to a value >= 2

2. use two checkpoints in an x/y plot:
![image](https://user-images.githubusercontent.com/1590330/200419537-ce03242b-0c67-4422-bd17-b7328cc7856e.png)

3. use a different checkpoint as the currently selected in the main drop down
![image](https://user-images.githubusercontent.com/1590330/200421634-f5334e06-ad12-4d9c-87b8-c98c1cca7563.png)

4. generate an image multiple times, the labels stay in the same order, but the images swap with each new button press
![image](https://user-images.githubusercontent.com/1590330/200419751-516a3c3e-c926-4d2b-8f94-7b10eaaf2d86.png)
![image](https://user-images.githubusercontent.com/1590330/200419809-3e4c8eec-bd15-4c7c-b3a4-f688cf8f71a5.png)
![image](https://user-images.githubusercontent.com/1590330/200419865-9cfa62d8-9c0f-41c5-b472-46881a912e9b.png)


### What should have happened?

Result should be consistent.
Currently selected checkpoint should not matter.

### Commit where the problem happens

804d9fb83d0c63ca3acd36378707ce47b8f12599

### What platforms do you use to access UI ?

Windows

### What browsers do you use to access the UI ?

Mozilla Firefox

### Command Line Arguments

```Shell
Launching Web UI with arguments: --xformers --force-enable-xformers
```


### Additional information, context and logs
Checkpoints to cache in RAM needs to be set to >= 2





Discussion:
The problem only happens, if the setting **Checkpoints to cache in RAM**  is set to > 0.

Resolving patch:
diff --git a/modules/sd_models.py b/modules/sd_models.py
--- a/modules/sd_models.py
+++ b/modules/sd_models.py
@@ -163,13 +163,21 @@ def load_model_weights(model, checkpoint_info, vae_file="auto"):
     checkpoint_file = checkpoint_info.filename
     sd_model_hash = checkpoint_info.hash
 
-    if shared.opts.sd_checkpoint_cache > 0 and hasattr(model, "sd_checkpoint_info"):
+    cache_enabled = shared.opts.sd_checkpoint_cache > 0
+
+    if cache_enabled:
         sd_vae.restore_base_vae(model)
-        checkpoints_loaded[model.sd_checkpoint_info] = model.state_dict().copy()
 
     vae_file = sd_vae.resolve_vae(checkpoint_file, vae_file=vae_file)
 
-    if checkpoint_info not in checkpoints_loaded:
+    if cache_enabled and checkpoint_info in checkpoints_loaded:
+        # use checkpoint cache
+        vae_name = sd_vae.get_filename(vae_file) if vae_file else None
+        vae_message = f" with {vae_name} VAE" if vae_name else ""
+        print(f"Loading weights [{sd_model_hash}]{vae_message} from cache")
+        model.load_state_dict(checkpoints_loaded[checkpoint_info])
+    else:
+        # load from file
         print(f"Loading weights [{sd_model_hash}] from {checkpoint_file}")
 
         pl_sd = torch.load(checkpoint_file, map_location=shared.weight_load_location)
@@ -180,6 +188,10 @@ def load_model_weights(model, checkpoint_info, vae_file="auto"):
         del pl_sd
         model.load_state_dict(sd, strict=False)
         del sd
+        
+        if cache_enabled:
+            # cache newly loaded model
+            checkpoints_loaded[checkpoint_info] = model.state_dict().copy()
 
         if shared.cmd_opts.opt_channelslast:
             model.to(memory_format=torch.channels_last)
@@ -199,14 +211,9 @@ def load_model_weights(model, checkpoint_info, vae_file="auto"):
 
         model.first_stage_model.to(devices.dtype_vae)
 
-    else:
-        vae_name = sd_vae.get_filename(vae_file) if vae_file else None
-        vae_message = f" with {vae_name} VAE" if vae_name else ""
-        print(f"Loading weights [{sd_model_hash}]{vae_message} from cache")
-        model.load_state_dict(checkpoints_loaded[checkpoint_info])
-
-    if shared.opts.sd_checkpoint_cache > 0:
-        while len(checkpoints_loaded) > shared.opts.sd_checkpoint_cache:
+    # clean up cache if limit is reached
+    if cache_enabled:
+        while len(checkpoints_loaded) > shared.opts.sd_checkpoint_cache + 1: # we need to count the current model
             checkpoints_loaded.popitem(last=False)  # LRU
 
     model.sd_model_hash = sd_model_hash


Test patch: