multimodalart HF staff commited on
Commit
3f92697
1 Parent(s): 0db5342

Add diffusers code loading example

Browse files

it already works out of the box with diffusers, just requires proper tagging and code examples

Files changed (1) hide show
  1. README.md +43 -9
README.md CHANGED
@@ -3,31 +3,65 @@ license: mit
3
  language:
4
  - en
5
  library_name: diffusers
 
 
 
 
6
  ---
7
 
8
  # Mann-E Turbo
9
 
10
  This is a part of _Mann-E Community Edition_ models. This model is a basic implementation of [Mann-E](https://mann-e.com)'s original model (which is not Stable Diffusion anymore) as a LoRa for SDXL. Since the whole business of Mann-E started around SD ecosystem, we've decided to release our LoRa for SD users and people who enjoy using models locally!
11
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ## Considerations
13
 
14
  - Model __IS__ capable of making accidental and intentional _NSFW_ material. So be careful while using the LoRa in presence of people who might be sensitive to this material.
15
  - The model has tested with SDXL 1.0, SDXL Turbo and even SDXL Lightning models and it works perfectly.
16
- - We've tested the model with A1111, so for `diffusers` and ComfyUI, we may need contributions.
17
  - DreamShaperXL, TurboVisioXL and RealVis were models with the best results. You may consider using them or merges/fine-tunes based on those!
18
  - This LoRa is compatible with other LoRa's as well. Be careful to weight it exactly 1 to get results like what we're presenting here.
19
  - DPM SDE++ Karras is the best scheduler. 6-10 steps for turbo models work perfectly, also CFG of 2.5 to 4.0 works perfectly!
20
 
21
- ## Sample Generations
 
 
 
22
 
23
- <div style="text-align:center;">
24
-
25
- <img src="./hf-1.png" width=512 height=512>
 
26
 
27
- <img src="./hf-2.png" width=512 height=512>
 
 
 
28
 
29
- <img src="./hf-3.png" width=512 height=512>
 
30
 
31
- <img src="./hf-4.png" width=512 height=512>
 
 
 
 
 
 
 
32
 
33
- </div>
 
 
3
  language:
4
  - en
5
  library_name: diffusers
6
+ tags:
7
+ - lora
8
+ - text-to-image
9
+ base_model: "stabilityai/stable-diffusion-xl-base-1.0"
10
  ---
11
 
12
  # Mann-E Turbo
13
 
14
  This is a part of _Mann-E Community Edition_ models. This model is a basic implementation of [Mann-E](https://mann-e.com)'s original model (which is not Stable Diffusion anymore) as a LoRa for SDXL. Since the whole business of Mann-E started around SD ecosystem, we've decided to release our LoRa for SD users and people who enjoy using models locally!
15
 
16
+ ## Sample Generations
17
+
18
+ <div style="text-align:center;">
19
+
20
+ <img src="./hf-1.png" width=512 height=512>
21
+
22
+ <img src="./hf-2.png" width=512 height=512>
23
+
24
+ <img src="./hf-3.png" width=512 height=512>
25
+
26
+ <img src="./hf-4.png" width=512 height=512>
27
+
28
+ </div>
29
+
30
  ## Considerations
31
 
32
  - Model __IS__ capable of making accidental and intentional _NSFW_ material. So be careful while using the LoRa in presence of people who might be sensitive to this material.
33
  - The model has tested with SDXL 1.0, SDXL Turbo and even SDXL Lightning models and it works perfectly.
34
+ - We've tested the model with A111 and `diffusers`. For ComfyUI, we may need contributions if it doesn't work.
35
  - DreamShaperXL, TurboVisioXL and RealVis were models with the best results. You may consider using them or merges/fine-tunes based on those!
36
  - This LoRa is compatible with other LoRa's as well. Be careful to weight it exactly 1 to get results like what we're presenting here.
37
  - DPM SDE++ Karras is the best scheduler. 6-10 steps for turbo models work perfectly, also CFG of 2.5 to 4.0 works perfectly!
38
 
39
+ ## Load with 🧨 diffusers
40
+ ```py
41
+ from diffusers import DiffusionPipeline, DPMSolverSinglestepScheduler
42
+ import torch
43
 
44
+ pipe = DiffusionPipeline.from_pretrained(
45
+ "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16
46
+ ).to("cuda")
47
+ #You can use other base models such as `Lykon/dreamshaper-xl-1-0`, `0x4f1f/TurboVisionXL-v3.2` or `SG161222/RealVisXL_V4.0`
48
 
49
+ pipe.load_lora_weights(
50
+ "mann-e/Mann-E_Turbo",
51
+ weight_name="manne_turbo.safetensors",
52
+ )
53
 
54
+ #This is equivalent to DPM++ SDE Karras, as noted in https://huggingface.co/docs/diffusers/main/en/api/schedulers/overview
55
+ pipe.scheduler = DPMSolverSinglestepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True)
56
 
57
+ image = pipe(
58
+ prompt="a cat in a bustling middle eastern city",
59
+ num_inference_steps=8,
60
+ guidance_scale=4,
61
+ width=768,
62
+ height=768,
63
+ clip_skip=1
64
+ ).images[0]
65
 
66
+ image.save("a_cat.png")
67
+ ```