ByteDance

/

SDXL-Lightning

like 1.92k

Follow

ByteDance 673

My suggestion:

1. Train regular SDXL model on your dataset. Then apply SDXL-Lightning LoRA on top for acceleration.
2. Preferably you train SDXL as LoRA as well, This ensures minimum model changes for best compatibility.

More advanced:

1. If quality is not good, you can merge SDXL-Lightning LoRA onto your model then train on top. However, if you use MSE loss, it may dilute the acceleration.
2. The most advanced is to merge SDXL-Lightning LoRA, then use adversarial objective, like the SDXL-Lightning paper does.

How do we create the SDXL lightning Lora on custom datasets? Any best practises and code?

My suggestion:

1. Train regular SDXL model on your dataset. Then apply SDXL-Lightning LoRA on top for acceleration.
2. Preferably you train SDXL as LoRA as well, This ensures minimum model changes for best compatibility.

More advanced:

1. If quality is not good, you can merge SDXL-Lightning LoRA onto your model then train on top. However, if you use MSE loss, it may dilute the acceleration.
2. The most advanced is to merge SDXL-Lightning LoRA, then use adversarial objective, like the SDXL-Lightning paper does.

Do you publish a full SDXL checkpoint (not just unet) in safetensor format ? That makes it easier to fine tune your model with standard tools such as kohya_ss or onetrainer.

@HansHelga
Full SDXL checkpoint is available here: https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0

@HansHelga
Full SDXL checkpoint is available here: https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0

Sorry I wasn't clear. What I meant is that would it be possible to package SDXL Lightning into a single safetensor file, so that it can be used without needing the original SDXL model?