Transformers
English
controlnet
Inference Endpoints
GeroldMeisinger commited on
Commit
1099b93
1 Parent(s): a43243f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -11
README.md CHANGED
@@ -11,10 +11,9 @@ Based on https://github.com/lllyasviel/ControlNet/discussions/318
11
  ```
12
  accelerate launch train_controlnet.py ^
13
  --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" ^
14
- --output_dir="control-edgedrawing-default-drop50-fp16/" ^
15
  --dataset_name="mydataset" ^
16
  --mixed_precision="fp16" ^
17
- --proportion_empty_prompts=0.5 ^
18
  --resolution=512 ^
19
  --learning_rate=1e-5 ^
20
  --train_batch_size=1 ^
@@ -26,7 +25,11 @@ accelerate launch train_controlnet.py ^
26
  --seed=0
27
  ```
28
 
29
- Trained for 40000 steps on images converted with https://github.com/shaojunluo/EDLinePython using `smoothed = False` and default settings:
 
 
 
 
30
 
31
  ```
32
  { 'ksize' : 5
@@ -37,13 +40,14 @@ Trained for 40000 steps on images converted with https://github.com/shaojunluo/E
37
  }
38
  ```
39
 
40
- **TODO**
 
 
 
 
41
 
42
- Results are not good so far:
43
 
44
- * `--proportion_empty_prompts=0.5` me be too excessive for 40000 steps
45
- * Use `smoothed = True` next time, maybe control net doesn't pick up on single pixels
46
- * Find better parameter spread instead of default values, most images are very sparse
47
- * Train on more steps
48
- * Train on more diverse dataset
49
- * Train on higher-precision
 
11
  ```
12
  accelerate launch train_controlnet.py ^
13
  --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" ^
14
+ --output_dir="control-edgedrawing-[version]-fp16/" ^
15
  --dataset_name="mydataset" ^
16
  --mixed_precision="fp16" ^
 
17
  --resolution=512 ^
18
  --learning_rate=1e-5 ^
19
  --train_batch_size=1 ^
 
25
  --seed=0
26
  ```
27
 
28
+ Images converted with https://github.com/shaojunluo/EDLinePython
29
+
30
+ Default settings are:
31
+
32
+ `smoothed=False`
33
 
34
  ```
35
  { 'ksize' : 5
 
40
  }
41
  ```
42
 
43
+ **control-edgedrawing-default-drop50-fp16-checkpoint-40000**
44
+
45
+ additional argument: `--proportion_empty_prompts=0.5`
46
+ Trained for 40000 steps with default settings
47
+ => empty prompts were probably too excessive
48
 
49
+ **control-edgedrawing-default-noisy-drop0-fp16-checkpoint-40000**
50
 
51
+ `smoothed=True`, but no empty prompts
52
+ Trained for 40000 steps with default settings
53
+ => conditioning images are too noisy