How to fine tune it?

#35
by AeroDEmi - opened

I was wondering if it is possible to fine-tune the base model, or how to create similar Loras. Can we use the same diffusers text2img examples ?

ByteDance org

Hi, @AeroDEmi
Of course. You can directly download and load our models into training.
Try merge it into UNet and create a new LoRA, or just fine-tune our LoRA. Both are worthy a try.
Our models are fully publicly available.

Awesome! Any details on how you tuned them? Should I use the LCM guide?

ByteDance org

Hi, @AeroDEmi
We haven't try further fine-tuning them. You could give it a try.
It should be working just by merging the LoRA into UNet and treat it as a new accelerated base model.

AeroDEmi changed discussion status to closed

@AeroDEmi
Hello. Thank you for your valuable discussion. Did you make it happen? If so, could you share some example code with me?

how hyper sd/flux trained? Could you please provide a fine-tune script? thanks

Hi, @AeroDEmi
We haven't try further fine-tuning them. You could give it a try.
It should be working just by merging the LoRA into UNet and treat it as a new accelerated base model.

Can we further finetune the LoRA as a normal one with diffusion loss, or should we train it on Hyper related losses? Otherwise it will lose the few-step generation capacity?

Sign up or log in to comment