finetuning

#16
by hollowsense - opened

is fine-tuning possible?

This is my question as well.
The simple answer is "yes", if you check CivitAI, you'll find out people already have done a lot of stuff with this model. I don't know why most of them don't bother to share the details...

Update:
Apparently most of those models are combinations of an already available XL model with the Turbo model. I'm on it, if I get acceptable results, I will let you know.

Great, will look forward to your findings!

may be LoRA weights can work out of the box with this model? anyone?

@hollowsense Yes, SDXL 1.0 LoRa's are apparently fine with this model.

@Muhammadreza So if I have to fine-tune a pre-trained model on around 10,000 image-caption pairs (domain specific), with a very very low budget, what do you recommend. This model or something else? I presume I will be using a combination of LoRa and Dreambooth? Am I on the right track.

@kushbhuwalka44 To be honest, I'd suggest you use 1.5 base then, because fine tuning 1.5 on a budget machine (Google Colab Pro or Pro+, RunPod, even a local 3090/4090 GPU) is much easier and faster. But the results will not be this good.
I suggest you go EMU (Meta's method), keep a small dataset on very good quality images for more epochs/steps in fine tuning.

@Muhammadreza Thanks for your response. Do you have any tips on fine-tuning?

how much would it cost to fine tune this on 100,000 image pairs?

@kushbhuwalka44 To be honest, I'd suggest you use 1.5 base then, because fine tuning 1.5 on a budget machine (Google Colab Pro or Pro+, RunPod, even a local 3090/4090 GPU) is much easier and faster. But the results will not be this good.
I suggest you go EMU (Meta's method), keep a small dataset on very good quality images for more epochs/steps in fine tuning.

I also interested in EMU model, but it seems meta haven't open-source the EMU model. Is there any released version of EMU model?

Sign up or log in to comment