Repository: AUTOMATIC1111/stable-diffusion-webui

Issue:
[Bug]: "1" checkpoint caching is useless
### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What happened?

"1" checkpoint caching is useless. When you set the webui to keep n caches, only n-1 is actually useful. This happens because model is cached after loading it. When you set caching to 1, what's in the cache is really just your currently loaded checkpoint. When you load another one, the other checkpoint can't possibly be in the cache, so it's loaded and inserted into cache dict. Then, the cache dict now has 2 entries, and so the oldest one will be discarded. Leaving you with 1 cache of the current model, again. So when caching is set to 1, there's practically no caching. There's always the current model taking up 1 cache "space" (it's not duplicated in memory though). 

### Steps to reproduce the problem

1. Set caching to 1
2. Load a model
3. Load another model
4. Load the first model

### What should have happened?

Step 4 should be loading the first model from cache, but it doesn't. Because the current model in cache is from step 3.

### Commit where the problem happens

17a2076f72562b428052ee3fc8c43d19c03ecd1e

### What platforms do you use to access UI ?

Other/Cloud

### What browsers do you use to access the UI ?

Google Chrome

### Command Line Arguments

```Shell
--autolaunch --ckpt {workspace3}/novelai/final_pruned.ckpt --vae-path {workspace3}/novelai/animevae.pt --deepdanbooru --disable-safe-unpickle  --no-half-vae --xformers
```


### Additional information, context and logs

Caching should keep cache of previous models, not current model. 

<details>
<summary>my log</summary>

```
To create a public link, set `share=True` in `launch()`.
(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), '/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/animevae.pt')
{(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), '/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/animevae.pt')}
Loading weights [925997e9] with animevae VAE from cache
Applying xformers cross attention optimization.
Weights loaded.
(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), None)
{(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), '/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/animevae.pt')}
Loading weights [925997e9] from /content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt
{(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), None)}
Applying xformers cross attention optimization.
Weights loaded.
(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), '/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/animevae.pt')
{(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), None)}
Loading weights [925997e9] from /content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt
Loading VAE weights from: /content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/animevae.pt
{(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), '/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/animevae.pt')}
Applying xformers cross attention optimization.
Weights loaded.
(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), None)
{(CheckpointInfo(filename='/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt', title='novelai/final_pruned.ckpt [925997e9]', hash='925997e9', model_name='novelai_final_pruned', config='/content/nai/stable-diffusion-webui/repositories/stable-diffusion/configs/stable-diffusion/v1-inference.yaml'), '/content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/animevae.pt')}
Loading weights [925997e9] from /content/nai/stable-diffusion-webui/models/Stable-diffusion/novelai/final_pruned.ckpt
2022-10-31T08:19:07Z INF Initiating graceful shutdown due to signal interrupt ...
```
</details>


Discussion:

Resolving patch:
diff --git a/modules/sd_models.py b/modules/sd_models.py
--- a/modules/sd_models.py
+++ b/modules/sd_models.py
@@ -163,11 +163,11 @@ def load_model_weights(model, checkpoint_info, vae_file="auto"):
     checkpoint_file = checkpoint_info.filename
     sd_model_hash = checkpoint_info.hash
 
-    vae_file = sd_vae.resolve_vae(checkpoint_file, vae_file=vae_file)
+    if shared.opts.sd_checkpoint_cache > 0 and hasattr(model, "sd_checkpoint_info"):
+        sd_vae.restore_base_vae(model)
+        checkpoints_loaded[model.sd_checkpoint_info] = model.state_dict().copy()
 
-    checkpoint_key = checkpoint_info
-
-    if checkpoint_key not in checkpoints_loaded:
+    if checkpoint_info not in checkpoints_loaded:
         print(f"Loading weights [{sd_model_hash}] from {checkpoint_file}")
 
         pl_sd = torch.load(checkpoint_file, map_location=shared.weight_load_location)
@@ -197,18 +197,15 @@ def load_model_weights(model, checkpoint_info, vae_file="auto"):
 
         model.first_stage_model.to(devices.dtype_vae)
 
-        if shared.opts.sd_checkpoint_cache > 0:
-            # if PR #4035 were to get merged, restore base VAE first before caching
-            checkpoints_loaded[checkpoint_key] = model.state_dict().copy()
-            while len(checkpoints_loaded) > shared.opts.sd_checkpoint_cache:
-                checkpoints_loaded.popitem(last=False)  # LRU
-
     else:
         vae_name = sd_vae.get_filename(vae_file) if vae_file else None
         vae_message = f" with {vae_name} VAE" if vae_name else ""
         print(f"Loading weights [{sd_model_hash}]{vae_message} from cache")
-        checkpoints_loaded.move_to_end(checkpoint_key)
-        model.load_state_dict(checkpoints_loaded[checkpoint_key])
+        model.load_state_dict(checkpoints_loaded[checkpoint_info])
+
+    if shared.opts.sd_checkpoint_cache > 0:
+        while len(checkpoints_loaded) > shared.opts.sd_checkpoint_cache:
+            checkpoints_loaded.popitem(last=False)  # LRU
 
     model.sd_model_hash = sd_model_hash
     model.sd_model_checkpoint = checkpoint_file


Test patch: