mhdang

/

dpo-sdxl-text2image-v1

like 216

diffusers IS a normal format, LDM format is 💩

Yes. please upload in ckpt or normal .safetensor format

I'm not sure what do you mean by "normal .safetensor format", can you elaborate it more?

It's uploaded in .safetensors format. See here.

Try load it using unet = UNet2DConditionModel.from_pretrained("mhdang/dpo-sdxl-text2image-v1", subfolder="unet", torch_dtype=torch.float16) or follow the README.md.

I'm not sure what do you mean by "normal .safetensor format", can you elaborate it more?

It's uploaded in .safetensors format. See here.

Try load it using unet = UNet2DConditionModel.from_pretrained("mhdang/dpo-sdxl-text2image-v1", subfolder="unet", torch_dtype=torch.float16) or follow the README.md.

i mean like a model that u can load straight into auto1111, this one seems like i need to put it into diffusers sdxl1.0 unet folder.

I found this on a reddit comment by ComfyAnon on how to convert it using comfyui: Those are diffusers format UNET files so to use it in ComfyUI you can take this file for DPO SDXL: https://huggingface.co/mhdang/dpo-sdxl-text2image-v1/blob/main/unet/diffusion_pytorch_model.safetensors

Download it, rename it then put it in the ComfyUI/models/unet folder and then use the advanced->loaders->UNETLoader node. For the CLIP and VAE you can use a Checkpoint Loader node with SDXL selected (or SD1.5 if you use the DPO 1.5 model).

To convert it to a regular checkpoint you can use the CheckpointSave node.

The paper author may be not a fan of AI painting and is not familiar with trends of AI painting community. Diffusers is really a great library for researcher to create new stuffs such as the DPO which make diffusion model output image truly aligned with human's preference and aesthetics rather than using a lot of mysterious negative "spellings". Although using diffusers is a bit hardcore for those who do not want get into deep learning and python, the oringal LDM weight format "normal .safetensor format" used by most popular AI painting software A1111 is really poorly designed and put all stuffs (CLIP, UNET, VAE) into one file. Using script https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_to_original_sdxl.py or other AI painting software that supports diffusers weight. People from huggingface may help convert the diffusers weight to LDM weight. I make simple try and think this DPO finetuned model has great potential to get popular. So just wait a few days and the model will spread over the community then various format weight will be easy to get.