yichaodu commited on
Commit
0e7e92f
1 Parent(s): 32fe188

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +44 -0
README.md ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - stable-diffusion
4
+ - stable-diffusion-diffusers
5
+ - text-to-image
6
+ inference: true
7
+ ---
8
+
9
+ # Aligned Diffusion Model via DPO
10
+
11
+ Diffusion Model Aligned with thef following reward model and DPO algorithm
12
+ ```
13
+ close-sourced vlm: claude3-opus gemini-1.5 gpt-4o gpt-4v
14
+ open-sourced vlm: internvl-1.5
15
+ score model: hps-2.1
16
+ ```
17
+
18
+ ## How to Use
19
+
20
+ You can load the model and perform inference as follows:
21
+ ```python
22
+ from diffusers import StableDiffusionPipeline, UNet2DConditionModel
23
+
24
+ pretrained_model_name = "runwayml/stable-diffusion-v1-5"
25
+
26
+ dpo_unet = UNet2DConditionModel.from_pretrained(
27
+ "path/to/checkpoint",
28
+ subfolder='unet',
29
+ torch_dtype=torch.float16
30
+ ).to('cuda')
31
+
32
+ pipeline = StableDiffusionPipeline.from_pretrained(pretrained_model_name, torch_dtype=torch.float16)
33
+ pipeline = pipeline.to('cuda')
34
+ pipeline.safety_checker = None
35
+ pipeline.unet = dpo_unet
36
+
37
+ generator = torch.Generator(device='cuda')
38
+ generator = generator.manual_seed(1)
39
+
40
+ prompt = "a pink flower"
41
+
42
+ image = pipeline(prompt=prompt, generator=generator, guidance_scale=gs).images[0]
43
+
44
+ ```