mfidabel commited on
Commit
5566ba7
1 Parent(s): 292dece

Added training info

Browse files
Files changed (1) hide show
  1. README.md +33 -3
README.md CHANGED
@@ -18,9 +18,9 @@ language:
18
  library_name: diffusers
19
  ---
20
 
21
- # controlnet- mfidabel/controlnet-segment-anything
22
 
23
- These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with new type of conditioning. You can find some example images in the following.
24
 
25
  **prompt**: contemporary living room of a house
26
 
@@ -35,4 +35,34 @@ These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with new
35
  **prompt**: contemporary living room, high quality, 4k, realistic
36
 
37
  **negative prompt**: low quality, monochrome, low res
38
- ![images_2)](./images_2.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  library_name: diffusers
19
  ---
20
 
21
+ # ControlNet - mfidabel/controlnet-segment-anything
22
 
23
+ These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with a new type of conditioning. You can find some example images in the following.
24
 
25
  **prompt**: contemporary living room of a house
26
 
 
35
  **prompt**: contemporary living room, high quality, 4k, realistic
36
 
37
  **negative prompt**: low quality, monochrome, low res
38
+ ![images_2)](./images_2.png)
39
+
40
+ ## Training
41
+
42
+ **Training Data** This model was trained using a Segmented dataset based on the [COYO-700M Dataset](https://huggingface.co/datasets/kakaobrain/coyo-700m).
43
+ [Stable Diffusion v1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5) checkpoint was used as the base model for the controlnet.
44
+
45
+
46
+ The model was trained as follows:
47
+
48
+ - 25k steps with the [SAM-COYO-2k](https://huggingface.co/datasets/mfidabel/sam-coyo-2k) dataset
49
+ - 28k steps with the [SAM-COYO-2.5k](https://huggingface.co/datasets/mfidabel/sam-coyo-2.5k) dataset
50
+ - 38k steps with the [SAM-COYO-3k](https://huggingface.co/datasets/mfidabel/sam-coyo-3k) dataset
51
+
52
+ In that particular order.
53
+
54
+ - **Hardware**: Google Cloud TPUv4-8 VM
55
+
56
+ - **Optimizer**: AdamW
57
+
58
+ - **Train Batch Size**: 2 x 4 = 8
59
+
60
+ - **Learning rate**: 0.00001 constant
61
+
62
+ - **Gradient Accumulation Steps**: 1
63
+
64
+ - **Resolution**: 512
65
+
66
+ Learning rate: warmup to 0.0001 for 10,000 steps and then kept constant
67
+
68
+