StrugglerXYH commited on
Commit
58fae0a
1 Parent(s): 3c75413

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +82 -3
README.md CHANGED
@@ -1,3 +1,82 @@
1
- ---
2
- license: openrail++
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: openrail++
3
+ tags:
4
+ - art
5
+ - stable diffusion
6
+ - ControlNet
7
+ - SDXL
8
+ - Diffusion-XL
9
+ pipeline_tag: text-to-image
10
+ ---
11
+ # MistoLine
12
+
13
+ ## Control Every Line!
14
+ ![Intro Image](assets/intro.png)
15
+
16
+ MistoLine: A Versatile and Robust SDXL-ControlNet Model for Adaptable Line Art Conditioning.
17
+
18
+ MistoLine is an SDXL-ControlNet model that can adapt to any type of line art input, demonstrating high accuracy and excellent stability. It can generate high-quality images (with a short side greater than 1024px) based on user-provided line art of various types, including hand-drawn sketches, different ControlNet line preprocessors, and model-generated outlines. MistoLine eliminates the need to select different ControlNet models for different line preprocessors, as it exhibits strong generalization capabilities across diverse line art conditions.
19
+ We developed MistoLine by employing a novel line preprocessing algorithm (Anyline) and retraining the ControlNet model based on the Unet of stabilityai/ stable-diffusion-xl-base-1.0, along with innovations in large model training engineering. MistoLine showcases superior performance across different types of
20
+ line art inputs, surpassing existing ControlNet models in terms of detail restoration, prompt alignment, and stability, particularly in more complex scenarios.
21
+
22
+ MistoLine maintains consistency with the ControlNet architecture released by lllyasviel, as illustrated in the following schematic diagram:
23
+ ![ControlNet architecture](assets/controlnet_1.png)
24
+ ![ControlNet architecture](assets/controlnet_2.png)
25
+ *reference:https://github.com/lllyasviel/ControlNet*
26
+
27
+ More information about ControlNet can be found in the following references:
28
+ https://github.com/lllyasviel/ControlNet
29
+ https://huggingface.co/docs/diffusers/main/en/api/pipelines/controlnet_sdxl
30
+
31
+ The model is compatible with most SDXL models, except for PlaygroundV2.5 and CosXL. It can be used in conjunction with LCM and other ControlNet models.
32
+ We have open-sourced the corresponding model weight files for non-commercial use by individual users.
33
+
34
+ ## Apply with Different Line Preprocessors
35
+ ![preprocessors](assets/preprocessors.png)
36
+
37
+ ## Compere with Other Controlnets
38
+ ![comparison](assets/comparison.png)
39
+
40
+ ## Application Examples
41
+
42
+ ### Sketch Rendering
43
+ ![Sketch Rendering](assets/sketch_rendering.png)
44
+
45
+ ### Model Rendering
46
+ ![Model Rendering](assets/model_rendering.png)
47
+
48
+ ## ComfyUI Recommended Parameters
49
+ ```
50
+ sampler steps:30
51
+ CFG:7.0
52
+ sampler_name:dpmpp_2m_sde
53
+ scheduler:karras
54
+ denoise:0.93
55
+ controlnet_strength:1.0
56
+ stargt_percent:0.0
57
+ end_percent:0.9
58
+ ```
59
+
60
+ ## Checkpoints
61
+ * mistoLine_rank256.safetensors : General usage version, for ComfyUI and AUTOMATIC1111-WebUI.
62
+ * mistoLine_fp16.safetensors : FP16 weights, for ComfyUI and AUTOMATIC1111-WebUI.
63
+
64
+ ## ComfyUI Usage
65
+ ![ComfyUI](assets/comfyui.png)
66
+
67
+ ## 中国(大陆地区)便捷下载地址:
68
+ 链接:https://pan.baidu.com/s/1DbZWmGJ40Uzr3Iz9RNBG_w?pwd=8mzs
69
+ 提取码:8mzs
70
+
71
+ ## Citation
72
+ ```
73
+ @misc{
74
+ title={Adding Conditional Control to Text-to-Image Diffusion Models},
75
+ author={Lvmin Zhang, Anyi Rao, Maneesh Agrawala},
76
+ year={2023},
77
+ eprint={2302.05543},
78
+ archivePrefix={arXiv},
79
+ primaryClass={cs.CV}
80
+ }
81
+ ```
82
+