Transformers
English
controlnet
Inference Endpoints
GeroldMeisinger commited on
Commit
46b6a5b
1 Parent(s): 1099b93

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +64 -9
README.md CHANGED
@@ -1,12 +1,46 @@
1
  ---
2
  license: openrail
3
  datasets:
4
- - laion/laion2B-en-aesthetic
5
  language:
6
  - en
7
  ---
8
 
9
- Based on https://github.com/lllyasviel/ControlNet/discussions/318
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
  ```
12
  accelerate launch train_controlnet.py ^
@@ -25,6 +59,25 @@ accelerate launch train_controlnet.py ^
25
  --seed=0
26
  ```
27
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  Images converted with https://github.com/shaojunluo/EDLinePython
29
 
30
  Default settings are:
@@ -40,14 +93,16 @@ Default settings are:
40
  }
41
  ```
42
 
43
- **control-edgedrawing-default-drop50-fp16-checkpoint-40000**
44
-
45
- additional argument: `--proportion_empty_prompts=0.5`
46
  Trained for 40000 steps with default settings
47
- => empty prompts were probably too excessive
48
 
49
- **control-edgedrawing-default-noisy-drop0-fp16-checkpoint-40000**
50
 
51
- `smoothed=True`, but no empty prompts
 
 
 
 
52
  Trained for 40000 steps with default settings
53
- => conditioning images are too noisy
 
1
  ---
2
  license: openrail
3
  datasets:
4
+ - ChristophSchuhmann/improved_aesthetics_6.5plus
5
  language:
6
  - en
7
  ---
8
 
9
+ Based on my GitHub monologs at [Edge Drawing - a Canny alternative](https://github.com/lllyasviel/ControlNet/discussions/318)
10
+
11
+ Controls image generation by edge maps generated with [EdgeDrawing Parameter-Free](https://github.com/CihanTopal/ED_Lib).
12
+
13
+ For usage see the model page on [Civitai.com]().
14
+
15
+ # Image dataset
16
+
17
+ * [laion2B-en aesthetics>=6.5 dataset](https://huggingface.co/datasets/ChristophSchuhmann/improved_aesthetics_6.5plus)
18
+ * `--min_image_size 512 --max_aspect_ratio 2 --resize_mode="center_crop" --image_size 512`
19
+ * resulting in 180k images
20
+
21
+ Original
22
+
23
+
24
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/RgXLEFqwSOzYQXvrs6pOr.png)
25
+
26
+ EdgeDrawing Parameter-Free
27
+
28
+
29
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/jmdCGeMJx4dKFGo44cuEq.png)
30
+
31
+ Canndy Edge Detection (default in Automatic1111)
32
+
33
+
34
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/JZTpa-HZfw0NUYnxZ52Iu.png)
35
+
36
+ Example
37
+ sampler=UniPC steps=20 cfg=7.5 seed=0 batch=9 model: v1-5-pruned-emaonly.safetensors cherry-picked: 1/9
38
+ prompt: _a detailed high-quality professional photo of swedish woman standing in front of a mirror, dark brown hair, white hat with purple feather_
39
+
40
+
41
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/2PSWsmzLdHeVG-i67S7jF.png)
42
+
43
+ # Training
44
 
45
  ```
46
  accelerate launch train_controlnet.py ^
 
59
  --seed=0
60
  ```
61
 
62
+ # Versions
63
+
64
+ **v3 - control-edgedrawing-cv480edpf-drop0-fp16-checkpoint-45000**
65
+
66
+ Conditioning images generated with [edpf.py](https://gitlab.com/-/snippets/3601881) using [opencv-contrib-python::ximgproc::EdgeDrawing](https://docs.opencv.org/4.8.0/d1/d1c/classcv_1_1ximgproc_1_1EdgeDrawing.html).
67
+
68
+ ```
69
+ ed = cv2.ximgproc.createEdgeDrawing()
70
+ params = cv2.ximgproc.EdgeDrawing.Params()
71
+ params.PFmode = True
72
+ ed.setParams(params)
73
+ edges = ed.detectEdges(image)
74
+ edge_map = ed.getEdgeImage(edges)
75
+ ```
76
+
77
+ 45000 steps
78
+
79
+ **v2 - control-edgedrawing-default-noisy-drop0-fp16-checkpoint-40000**
80
+
81
  Images converted with https://github.com/shaojunluo/EDLinePython
82
 
83
  Default settings are:
 
93
  }
94
  ```
95
 
96
+ `smoothed=True`, but no empty prompts
 
 
97
  Trained for 40000 steps with default settings
98
+ => conditioning images are too noisy
99
 
100
+ **v1 - control-edgedrawing-default-drop50-fp16-checkpoint-40000**
101
 
102
+ Same as v1
103
+
104
+ Update: bug in algorithm produces too sparse images on default, see https://github.com/shaojunluo/EDLinePython/issues/4
105
+
106
+ additional arguments: `--proportion_empty_prompts=0.5`
107
  Trained for 40000 steps with default settings
108
+ => empty prompts were probably too excessive