Add diffusers code loading example

#1
by multimodalart HF staff - opened
Files changed (1) hide show
  1. README.md +43 -9
README.md CHANGED
@@ -3,31 +3,65 @@ license: mit
3
  language:
4
  - en
5
  library_name: diffusers
 
 
 
 
6
  ---
7
 
8
  # Mann-E Turbo
9
 
10
  This is a part of _Mann-E Community Edition_ models. This model is a basic implementation of [Mann-E](https://mann-e.com)'s original model (which is not Stable Diffusion anymore) as a LoRa for SDXL. Since the whole business of Mann-E started around SD ecosystem, we've decided to release our LoRa for SD users and people who enjoy using models locally!
11
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ## Considerations
13
 
14
  - Model __IS__ capable of making accidental and intentional _NSFW_ material. So be careful while using the LoRa in presence of people who might be sensitive to this material.
15
  - The model has tested with SDXL 1.0, SDXL Turbo and even SDXL Lightning models and it works perfectly.
16
- - We've tested the model with A1111, so for `diffusers` and ComfyUI, we may need contributions.
17
  - DreamShaperXL, TurboVisioXL and RealVis were models with the best results. You may consider using them or merges/fine-tunes based on those!
18
  - This LoRa is compatible with other LoRa's as well. Be careful to weight it exactly 1 to get results like what we're presenting here.
19
  - DPM SDE++ Karras is the best scheduler. 6-10 steps for turbo models work perfectly, also CFG of 2.5 to 4.0 works perfectly!
20
 
21
- ## Sample Generations
 
 
 
22
 
23
- <div style="text-align:center;">
24
-
25
- <img src="./hf-1.png" width=512 height=512>
 
26
 
27
- <img src="./hf-2.png" width=512 height=512>
 
 
 
28
 
29
- <img src="./hf-3.png" width=512 height=512>
 
30
 
31
- <img src="./hf-4.png" width=512 height=512>
 
 
 
 
 
 
 
32
 
33
- </div>
 
 
3
  language:
4
  - en
5
  library_name: diffusers
6
+ tags:
7
+ - lora
8
+ - text-to-image
9
+ base_model: "stabilityai/stable-diffusion-xl-base-1.0"
10
  ---
11
 
12
  # Mann-E Turbo
13
 
14
  This is a part of _Mann-E Community Edition_ models. This model is a basic implementation of [Mann-E](https://mann-e.com)'s original model (which is not Stable Diffusion anymore) as a LoRa for SDXL. Since the whole business of Mann-E started around SD ecosystem, we've decided to release our LoRa for SD users and people who enjoy using models locally!
15
 
16
+ ## Sample Generations
17
+
18
+ <div style="text-align:center;">
19
+
20
+ <img src="./hf-1.png" width=512 height=512>
21
+
22
+ <img src="./hf-2.png" width=512 height=512>
23
+
24
+ <img src="./hf-3.png" width=512 height=512>
25
+
26
+ <img src="./hf-4.png" width=512 height=512>
27
+
28
+ </div>
29
+
30
  ## Considerations
31
 
32
  - Model __IS__ capable of making accidental and intentional _NSFW_ material. So be careful while using the LoRa in presence of people who might be sensitive to this material.
33
  - The model has tested with SDXL 1.0, SDXL Turbo and even SDXL Lightning models and it works perfectly.
34
+ - We've tested the model with A111 and `diffusers`. For ComfyUI, we may need contributions if it doesn't work.
35
  - DreamShaperXL, TurboVisioXL and RealVis were models with the best results. You may consider using them or merges/fine-tunes based on those!
36
  - This LoRa is compatible with other LoRa's as well. Be careful to weight it exactly 1 to get results like what we're presenting here.
37
  - DPM SDE++ Karras is the best scheduler. 6-10 steps for turbo models work perfectly, also CFG of 2.5 to 4.0 works perfectly!
38
 
39
+ ## Load with 🧨 diffusers
40
+ ```py
41
+ from diffusers import DiffusionPipeline, DPMSolverSinglestepScheduler
42
+ import torch
43
 
44
+ pipe = DiffusionPipeline.from_pretrained(
45
+ "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16
46
+ ).to("cuda")
47
+ #You can use other base models such as `Lykon/dreamshaper-xl-1-0`, `0x4f1f/TurboVisionXL-v3.2` or `SG161222/RealVisXL_V4.0`
48
 
49
+ pipe.load_lora_weights(
50
+ "mann-e/Mann-E_Turbo",
51
+ weight_name="manne_turbo.safetensors",
52
+ )
53
 
54
+ #This is equivalent to DPM++ SDE Karras, as noted in https://huggingface.co/docs/diffusers/main/en/api/schedulers/overview
55
+ pipe.scheduler = DPMSolverSinglestepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True)
56
 
57
+ image = pipe(
58
+ prompt="a cat in a bustling middle eastern city",
59
+ num_inference_steps=8,
60
+ guidance_scale=4,
61
+ width=768,
62
+ height=768,
63
+ clip_skip=1
64
+ ).images[0]
65
 
66
+ image.save("a_cat.png")
67
+ ```