Lykon/dreamshaper-8-inpainting

Dreamshaper 8 inpainting

lykon-models/dreamshaper-8-inpainting is a Stable Diffusion Inpainting model that has been fine-tuned on runwayml/stable-diffusion-inpainting.

Please consider supporting me:

• on Patreon
• or buy me a coffee

Diffusers

For more general information on how to run inpainting models with 🧨 Diffusers, see the docs.

1. Installation

pip install diffusers transformers accelerate

2. Run

from diffusers import AutoPipelineForInpainting, DEISMultistepScheduler
import torch
from diffusers.utils import load_image

pipe = AutoPipelineForInpainting.from_pretrained('lykon-models/dreamshaper-8-inpainting', torch_dtype=torch.float16, variant="fp16")
pipe.scheduler = DEISMultistepScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to("cuda")

img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png"
mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png"

image = load_image(img_url)
mask_image = load_image(mask_url)


prompt = "a majestic tiger sitting on a park bench"

generator = torch.manual_seed(33)
image = pipe(prompt, image=image, mask_image=mask_image, generator=generator, num_inference_steps=25).images[0]  
image.save("./image.png")

Notes

• Version 8 focuses on improving what V7 started. Might be harder to do photorealism compared to realism focused models, as it might be hard to do anime compared to anime focused models, but it can do both pretty well if you're skilled enough. Check the examples!
• Version 7 improves lora support, NSFW and realism. If you're interested in "absolute" realism, try AbsoluteReality.
• Version 6 adds more lora support and more style in general. It should also be better at generating directly at 1024 height (but be careful with it). 6.x are all improvements.
• Version 5 is the best at photorealism and has noise offset.
• Version 4 is much better with anime (can do them with no LoRA) and booru tags. It might be harder to control if you're used to caption style, so you might still want to use version 3.31. V4 is also better with eyes at lower resolutions. Overall is like a "fix" of V3 and shouldn't be too much different.