Training inpainting discussion
#33
by
minhtri171997
- opened
As I know, may be it not correct yet. I currently having an issue as:
- I want to train some specific style of image ( my custom style ) using DreamBooth on stable-diffusion model runwayml/stable-diffusion-v1-5, it worked, but the issue here is i want to use this model "stabilityai/stable-diffusion-2-inpainting" to train custom styles as stable-diffusion-v1-5. But I could not find the way to fine-tune that inpainting model. I have try to merged these models for my use case but there was difference in the param shape of text_model from inpainting is different from normal stable diffusion model.
Is there any solution to train new styles using inpainting model or is there a method to merge these two model for my purpose ?
If there is anything not clear yet, please feel free to leave a message i will clarify that
Thank you,
I made a quick LORA inpainting finetuning script for this https://github.com/sshh12/terrain-diffusion/blob/main/scripts/train_text_to_image_lora_sd2_inpaint.py, hope its helpful!