OSError: Error no file named config.json found

by Elucipath - opened

Looks like the repo doesn't have a config.json like previous vae that were release.

even I have converted the ckpt to diffusers it gives black images

For reference, I got this VAE to run by

  1. Using the VAE module implementation from https://github.com/Stability-AI/generative-models/tree/main (using this VAE in diffusers requires a conversion step)
  2. Using fp32 precision (fp16 gives NaNs / black images - you have to use fp32 or bfloat16)

Quoting the config here https://github.com/Stability-AI/generative-models/blob/main/configs/inference/sd_xl_base.yaml#L4:

scale_factor: 0.13025
disable_first_stage_autocast: True


is it supposed to be backwards compatible? i managed to convert it to diffusers format and vae.encode/decode works but diffusion returns distorted images.

@knoopx No - they retrained the VAE from scratch, so the SDXL VAE latents look totally different from the original SD1/2 VAE latents, and the SDXL VAE is only going to work with the SDXL UNet. This is what latents from the original VAE look like:


Sign up or log in to comment