How to use vae files in stable-diffusion-webui?

#6
by songjoe - opened

Here are error logs. Some key configurations can be found in the logs. File diffusion_pytorch_model.bin was renamed to MODEL_NAME.vae.pt. It was then placed in the same folder as the model file.

Loading weights [0ff127093f] from F:\Data\stable-diffusion-webui\models\Stable-diffusion\Basil_mix_fixed.safetensors
Loading VAE weights found near the checkpoint: F:\Data\stable-diffusion-webui\models\Stable-diffusion\Basil_mix_fixed.vae.pt
Failed to load checkpoint, restoring previous
Loading weights [bbf07e3a1c] from F:\Data\stable-diffusion-webui\models\Stable-diffusion\basil_mix.ckpt
Applying xformers cross attention optimization.
changing setting sd_model_checkpoint to Basil_mix_fixed.safetensors: RuntimeError
Traceback (most recent call last):
  File "F:\Data\stable-diffusion-webui\modules\shared.py", line 534, in set
    self.data_labels[key].onchange()
  File "F:\Data\stable-diffusion-webui\modules\call_queue.py", line 15, in f
    res = func(*args, **kwargs)
  File "F:\Data\stable-diffusion-webui\webui.py", line 84, in <lambda>
    shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights()))
  File "F:\Data\stable-diffusion-webui\modules\sd_models.py", line 441, in reload_model_weights
    load_model_weights(sd_model, checkpoint_info)
  File "F:\Data\stable-diffusion-webui\modules\sd_models.py", line 281, in load_model_weights
    sd_vae.load_vae(model, vae_file, vae_source)
  File "F:\Data\stable-diffusion-webui\modules\sd_vae.py", line 135, in load_vae
    _load_vae_dict(model, vae_dict_1)
  File "F:\Data\stable-diffusion-webui\modules\sd_vae.py", line 160, in _load_vae_dict
    model.first_stage_model.load_state_dict(vae_dict_1, strict=False)
  File "F:\Data\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for AutoencoderKL:
    Missing key(s) in state_dict: "encoder.down.0.block.0.norm1.weight", "encoder.down.0.block.0.norm1.bias", "encoder.down.0.block.0.conv1.weight", "encoder.down.0.block.0.conv1.bias", "encoder.down.0.block.0.norm2.weight", "encoder.down.0.block.0.norm2.bias", "encoder.down.0.block.0.conv2.weight", "encoder.down.0.block.0.conv2.bias", "encoder.down.0.block.1.norm1.weight", "encoder.down.0.block.1.norm1.bias", "encoder.down.0.block.1.conv1.weight", "encoder.down.0.block.1.conv1.bias", "encoder.down.0.block.1.norm2.weight", "encoder.down.0.block.1.norm2.bias", "encoder.down.0.block.1.conv2.weight", "encoder.down.0.block.1.conv2.bias", "encoder.down.0.downsample.conv.weight", "encoder.down.0.downsample.conv.bias", "encoder.down.1.block.0.norm1.weight", "encoder.down.1.block.0.norm1.bias", "encoder.down.1.block.0.conv1.weight", "encoder.down.1.block.0.conv1.bias", "encoder.down.1.block.0.norm2.weight", "encoder.down.1.block.0.norm2.bias", "encoder.down.1.block.0.conv2.weight", "encoder.down.1.block.0.conv2.bias", "encoder.down.1.block.0.nin_shortcut.weight", "encoder.down.1.block.0.nin_shortcut.bias", "encoder.down.1.block.1.norm1.weight", "encoder.down.1.block.1.norm1.bias", "encoder.down.1.block.1.conv1.weight", "encoder.down.1.block.1.conv1.bias", "encoder.down.1.block.1.norm2.weight", "encoder.down.1.block.1.norm2.bias", "encoder.down.1.block.1.conv2.weight", "encoder.down.1.block.1.conv2.bias", "encoder.down.1.downsample.conv.weight", "encoder.down.1.downsample.conv.bias", "encoder.down.2.block.0.norm1.weight", "encoder.down.2.block.0.norm1.bias", "encoder.down.2.block.0.conv1.weight", "encoder.down.2.block.0.conv1.bias", "encoder.down.2.block.0.norm2.weight", "encoder.down.2.block.0.norm2.bias", "encoder.down.2.block.0.conv2.weight", "encoder.down.2.block.0.conv2.bias", "encoder.down.2.block.0.nin_shortcut.weight", "encoder.down.2.block.0.nin_shortcut.bias", "encoder.down.2.block.1.norm1.weight", "encoder.down.2.block.1.norm1.bias", "encoder.down.2.block.1.conv1.weight", "encoder.down.2.block.1.conv1.bias", "encoder.down.2.block.1.norm2.weight", "encoder.down.2.block.1.norm2.bias", "encoder.down.2.block.1.conv2.weight", "encoder.down.2.block.1.conv2.bias", 

In my limited experience, this may be because

  1. The bin file is not adapted.
  2. It seems that config.json is not being loaded correctly to initialize the network.

I would be grateful if I could get your advice.

I do not provide customer service. Please resolve it yourself.

Thank you for your reply.
I researched further and found that it was due to inconsistent naming of the internal network structure of the bin file and the ckpt file.
This results from the inconsistent naming convention of a HF Diffusers saved pipeline and a Stable Diffusion checkpoint.

File diffusion_pytorch_model.bin was renamed to MODEL_NAME.vae.pt.

So the renaming will not take effect.

This script can help with the conversion. However, it had a little compatibility issue, and I made a small modification to it. The new script can be found here.

Anyway, thank you for reading this discussion.

Sign up or log in to comment