fp16 weights of VAE is not saved correctly
#160
by
aviadm
- opened
Try to delete the .bin files and all the other .safetensors of the VAE, then load the .fp16.safetensors file.
It throws inside 'load_sub_module':
ValueError: Cannot load <class 'diffusers.models.autoencoder_kl.AutoencoderKL'> from /fsx/cld-models/stable-diffusion/1/vae because the following keys are missing:
decoder.mid_block.attentions.0.proj_attn.weight, encoder.mid_block.attentions.0.key.weight, encoder.mid_block.attentions.0.proj_attn.bias, decoder.mid_block.attentions.0.value.weight, decoder.mid_block.attentions.0.key.bias, decoder.mid_block.attentions.0.query.weight, encoder.mid_block.attentions.0.query.bias, encoder.mid_block.attentions.0.proj_attn.weight, decoder.mid_block.attentions.0.proj_attn.bias, encoder.mid_block.attentions.0.key.bias, decoder.mid_block.attentions.0.key.weight, encoder.mid_block.attentions.0.query.weight, decoder.mid_block.attentions.0.value.bias, decoder.mid_block.attentions.0.query.bias, encoder.mid_block.attentions.0.value.weight, encoder.mid_block.attentions.0.value.bias.
Please make sure to pass `low_cpu_mem_usage=False` and `device_map=None` if you want to randomly initialize those weights or else make sure your checkpoint file is correct.
And then attempts to load the .bin
I confirm this comes from the .fp16.bin model and not related to the safetensors
aviadm
changed discussion title from
fp16.safetensors of VAE is not saved correctly
to fp16 weights of VAE is not saved correctly
Indeed, fp16 was the matter with the 5700. Deleting torch_dtype=torch.float16 solves this, but introduces other challenges to VRAM conservation which I'm giving the simplest, dumbest, possible solution for here:
from diffusers import StableDiffusionPipeline
import torch
model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(model_id)
pipe = pipe.to("cuda")
prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt, width=400, height=400).images[0]
image.save("astronaut_rides_horse.png")
Sorry, I was replying to myself, in the wrong thread, and now can't delete this.