ritwikraha commited on
Commit
70d5e1f
1 Parent(s): 9c398d5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +73 -1
README.md CHANGED
@@ -1,3 +1,75 @@
1
  ---
2
- license: mit
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: cc-by-2.0
3
+ library_name: diffusers
4
+ tags:
5
+ - art
6
+ - code
7
+ - stable-diffusion-xl
8
+ - stable-diffusion-xl-diffusers
9
  ---
10
+ # SDXL LoRA DreamBooth - comic_style_LoRA
11
+
12
+ <Gallery />
13
+
14
+ | Image 1 | Image 2 |
15
+ |---|---|
16
+ | ![example 1](./1.png) | ![example 2](./2.png) |
17
+ | Image 3 | Image 4 |
18
+ |---|---|
19
+ | ![example 3](./3.png) | ![sample 4](./4.png) |
20
+
21
+
22
+ ## Model description
23
+
24
+ These are comic_style LoRA adaption weights for `stabilityai/stable-diffusion-xl-base-1.0`.
25
+
26
+ The weights were trained using [DreamBooth](https://dreambooth.github.io/).
27
+
28
+ LoRA for the text encoder was enabled.
29
+
30
+ Special VAE used for training: `madebyollin/sdxl-vae-fp16-fix`.
31
+
32
+ DataSet: custom hand-drawn sketches by [ritwikraha](https://www.ritwikraha.com/)
33
+
34
+ ## Trigger words
35
+
36
+ You should use a photo in the style of TOK comics to trigger the image generation.
37
+
38
+ ## Usage
39
+
40
+ ```
41
+ !pip install diffusers accelerate -q
42
+ import torch
43
+ from PIL import Image
44
+ from diffusers import DiffusionPipeline, AutoencoderKL
45
+
46
+ vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
47
+ pipe = DiffusionPipeline.from_pretrained(
48
+ "stabilityai/stable-diffusion-xl-base-1.0",
49
+ vae=vae,
50
+ torch_dtype=torch.float16,
51
+ variant="fp16",
52
+ use_safetensors=True
53
+ )
54
+ pipe.load_lora_weights('ritwikraha/comics_style_LoRA')
55
+ _ = pipe.to("cuda")
56
+
57
+ prompt = "a photo of 18th century London in the style of TOK comics, 8k"
58
+ negative_prompt ="ugly face, multiple bodies, bad anatomy, disfigured, extra fingers"
59
+ image = pipe(prompt=prompt,
60
+ negative_prompt=negative_prompt,
61
+ guidance_scale=3,
62
+ num_inference_steps=50).images[0]
63
+ image
64
+ ```
65
+
66
+ ## Download model
67
+
68
+ Weights for this model are available in Safetensors format.
69
+
70
+ [Download](ritwikraha/comics_style_LoRA/tree/main) them in the Files & versions tab.
71
+
72
+
73
+
74
+
75
+ ---