pcuenq HF staff commited on
Commit
c96cd60
1 Parent(s): bc3cb52
Files changed (1) hide show
  1. README.md +24 -1
README.md CHANGED
@@ -11,10 +11,33 @@ inference: true
11
  ---
12
 
13
  # LoRA text2image fine-tuning - https://huggingface.co/pcuenq/pokemon-lora
14
- These are LoRA adaption weights for https://huggingface.co/pcuenq/pokemon-lora. The weights were fine-tuned on the lambdalabs/pokemon-blip-captions dataset. You can find some example images in the following.
15
 
16
  ![img_0](./image_0.png)
17
  ![img_1](./image_1.png)
18
  ![img_2](./image_2.png)
19
  ![img_3](./image_3.png)
20
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  ---
12
 
13
  # LoRA text2image fine-tuning - https://huggingface.co/pcuenq/pokemon-lora
14
+ These are LoRA adaption weights trained on base model https://huggingface.co/runwayml/stable-diffusion-v1-5. The weights were fine-tuned on the lambdalabs/pokemon-blip-captions dataset. You can find some example images in the following.
15
 
16
  ![img_0](./image_0.png)
17
  ![img_1](./image_1.png)
18
  ![img_2](./image_2.png)
19
  ![img_3](./image_3.png)
20
 
21
+ ## How to Use
22
+
23
+ The script below loads the base model, then applies the LoRA weights and performs inference:
24
+
25
+ ```Python
26
+ import torch
27
+ from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler
28
+ from huggingface_hub import model_info
29
+
30
+ # LoRA weights ~3 MB
31
+ model_path = "pcuenq/pokemon-lora"
32
+
33
+ info = model_info(model_path)
34
+ model_base = info.cardData["base_model"]
35
+ pipe = StableDiffusionPipeline.from_pretrained(model_base, torch_dtype=torch.float16)
36
+ pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
37
+
38
+ pipe.unet.load_attn_procs(model_path)
39
+ pipe.to("cuda")
40
+
41
+ image = pipe("Green pokemon with menacing face", num_inference_steps=25).images[0]
42
+ image.save("green_pokemon.png")
43
+ ```