is this because this model is using sd1.2 weights? I find that it absolutely can't draw anatomically correct human face (only tried for inpainting)
It can. You need to use negative prompts extensively though.
I've tried - ((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))
seems maybe something off with the model? 1.4 doesn't have the same issue, it just can't inpaint so well.
Does the Pipeline even understand prompt weights? Maybe it gives better results without?
This code is from the link above:
from diffusers import StableDiffusionInpaintPipeline pipe = StableDiffusionInpaintPipeline.from_pretrained( "runwayml/stable-diffusion-inpainting", revision="fp16", torch_dtype=torch.float16, ) prompt = "Face of a yellow cat, high resolution, sitting on a park bench" #image and mask_image should be PIL images. #The mask structure is white for inpainting and black for keeping as is image = pipe(prompt=prompt, image=image, mask_image=mask_image).images image.save("./yellow_cat_on_park_bench.png")
But I am also having hard times generating proper faces which img2img can do easily.