Use in Diffusers

#12
by rafationgson - opened

Hello! Can these models be used with diffusers? I tried with ControlNetModel.from_single_file and encountered an error. I would like to use kohya_controllllite_xl_scribble_anime.safetensors.

I would like to know the answer to this as well, direct usage in ControlNet pipeline is not possible and converting the controlnet using diffusers util methods doesn't work either as it will crash with KeyError: 'time_embed.0.weight'

@rafationgson @shtif please refer to this github issue (https://github.com/huggingface/diffusers/issues/5577). from_single_file cannot be used here, you should use from_pretrained instead. As a result, simply downloading the safetensors here also does not work, you also need to download the config.json from the original repo, and put them in the same folder.

@rafationgson @shtif please refer to this github issue (https://github.com/huggingface/diffusers/issues/5577). from_single_file cannot be used here, you should use from_pretrained instead. As a result, simply downloading the safetensors here also does not work, you also need to download the config.json from the original repo, and put them in the same folder.

@aaroncisa how do you use in diffusers specifically? Is there a step by step guide?

I would like to know the answer to this as well, direct usage in ControlNet pipeline is not possible and converting the controlnet using diffusers util methods doesn't work either as it will crash with KeyError: 'time_embed.0.weight'

i have the same problem. have you sovled it, thanks

@rafationgson @shtif please refer to this github issue (https://github.com/huggingface/diffusers/issues/5577). from_single_file cannot be used here, you should use from_pretrained instead. As a result, simply downloading the safetensors here also does not work, you also need to download the config.json from the original repo, and put them in the same folder.

from the worker, https://huggingface.co/kohya-ss/controlnet-lllite, could not find any config.json. anybody have others idea?
i have try to convert the sd/sdxl.safetensors to diffusers's, but i failed.
and , i also have try this https://huggingface.co/spaces/diffusers/sdxl-to-diffusers
failed agin.

@rafationgson @shtif please refer to this github issue (https://github.com/huggingface/diffusers/issues/5577). from_single_file cannot be used here, you should use from_pretrained instead. As a result, simply downloading the safetensors here also does not work, you also need to download the config.json from the original repo, and put them in the same folder.

from the worker, https://huggingface.co/kohya-ss/controlnet-lllite, could not find any config.json. anybody have others idea?
i have try to convert the sd/sdxl.safetensors to diffusers's, but i failed.
and , i also have try this https://huggingface.co/spaces/diffusers/sdxl-to-diffusers
failed agin.

anybody solved this problem?

Sign up or log in to comment