radames commited on
Commit
5d3ef06
1 Parent(s): 1e4037a

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -0
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: diffusers
3
+ pipeline_tag: text-to-image
4
+ inference: true
5
+ base_model: stabilityai/sd-turbo
6
+ ---
7
+ # DPO LoRA Stable Diffusion v2-1
8
+ Model trained with LoRA implementation of Diffusion DPO Read more [here](https://github.com/huggingface/diffusers/tree/main/examples/research_projects/diffusion_dpo)
9
+
10
+
11
+ Base Model: https://huggingface.co/stabilityai/stable-diffusion-2-1
12
+
13
+ ## Running with [🧨 diffusers library](https://github.com/huggingface/diffusers)
14
+
15
+
16
+ ```python
17
+
18
+ from diffusers import DiffusionPipeline
19
+ from diffusers.utils import make_image_grid
20
+ import torch
21
+
22
+ pipe = DiffusionPipeline.from_pretrained(
23
+ "stabilityai/sd-turbo", # SD Turbo is a destilled version of Stable Diffusion 2.1
24
+ # "stabilityai/stable-diffusion-2-1", # for the original stable diffusion 2.1 model
25
+ torch_dtype=torch.float16, variant="fp16"
26
+ )
27
+ pipe.to("cuda")
28
+ pipe.load_lora_weights("radames/stable-diffusion-2-1-DPO-LoRA", adapter_name="dpo-lora-sd21")
29
+ pipe.set_adapters(["dpo-lora-sd21"], adapter_weights=[1.0]) # you can play with adapter_weights to increase the effect of the LoRA model
30
+ seed = 123123
31
+ prompt = "portrait headshot professional of elon musk"
32
+ negative_prompt = "3d render, cartoon, drawing, art, low light"
33
+ generator = torch.Generator().manual_seed(seed)
34
+ images = pipe(
35
+ prompt=prompt,
36
+ negative_prompt=negative_prompt,
37
+ width=512,
38
+ height=512,
39
+ num_inference_steps=2,
40
+ generator=generator,
41
+ guidance_scale=1.0,
42
+ num_images_per_prompt=4
43
+ ).images
44
+ make_image_grid(images, 1, 4)
45
+ ```
46
+
47
+ ## Examples
48
+
49
+ Left Withoud DPO right with DPO LoRA
50
+
51
+ <img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/R8E0hRpWIE6OhhtvgJeEU.png style="max-width: 60rem;">
52
+ <img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/Eg4LbyxCfhmsk2INzqODw.png style="max-width: 60rem;">
53
+ <img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/GD7KumSCNweBWMJ1TArI-.png style="max-width: 60rem;">
54
+ <img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/SO7QoA9lZJY9hI0U4fBLy.png style="max-width: 60rem;">
55
+ <img src=https://cdn-uploads.huggingface.co/production/uploads/6064e095abd8d3692e3e2ed6/ZWbQwIQ5OklEgF9RW581R.png style="max-width: 60rem;">
56
+