dimitribarbot
commited on
Commit
•
55978e9
1
Parent(s):
e3473c0
Add thanks section in README file
Browse files
README.md
CHANGED
@@ -12,9 +12,9 @@ tags:
|
|
12 |
|
13 |
# SDXL ControlNet: DWPose
|
14 |
|
15 |
-
Here are
|
16 |
|
17 |
-
|
18 |
|
19 |
First, install all the libraries:
|
20 |
|
@@ -23,7 +23,7 @@ pip install -q easy-dwpose transformers accelerate
|
|
23 |
pip install -q git+https://github.com/huggingface/diffusers
|
24 |
```
|
25 |
|
26 |
-
|
27 |
|
28 |
To generate a realistic DJ with the following image driving the pose:
|
29 |
|
@@ -87,7 +87,7 @@ Image generated by SDXL is:
|
|
87 |
|
88 |
![Pose 1](./images/dwpose_image_1.png)
|
89 |
|
90 |
-
|
91 |
|
92 |
To generate a anime version of a woman sitting on a bench with the following image driving the pose:
|
93 |
|
@@ -153,7 +153,7 @@ Image generated by SDXL is:
|
|
153 |
|
154 |
![Pose 2](./images/dwpose_image_2.png)
|
155 |
|
156 |
-
|
157 |
|
158 |
The [training script](https://github.com/huggingface/diffusers/blob/main/examples/controlnet/README_sdxl.md) by HF🤗 was used.
|
159 |
|
@@ -170,4 +170,14 @@ Data parallel with a single GPU batch size of 2 with gradient accumulation 8.
|
|
170 |
Constant learning rate of 8e-5
|
171 |
|
172 |
#### Mixed precision
|
173 |
-
fp16
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
|
13 |
# SDXL ControlNet: DWPose
|
14 |
|
15 |
+
Here are ControlNet weights trained on stabilityai/stable-diffusion-xl-base-1.0 with [DWPose](https://github.com/IDEA-Research/DWPose) conditioning.
|
16 |
|
17 |
+
## Using in 🧨 diffusers
|
18 |
|
19 |
First, install all the libraries:
|
20 |
|
|
|
23 |
pip install -q git+https://github.com/huggingface/diffusers
|
24 |
```
|
25 |
|
26 |
+
### Example 1
|
27 |
|
28 |
To generate a realistic DJ with the following image driving the pose:
|
29 |
|
|
|
87 |
|
88 |
![Pose 1](./images/dwpose_image_1.png)
|
89 |
|
90 |
+
### Example 2
|
91 |
|
92 |
To generate a anime version of a woman sitting on a bench with the following image driving the pose:
|
93 |
|
|
|
153 |
|
154 |
![Pose 2](./images/dwpose_image_2.png)
|
155 |
|
156 |
+
## Training
|
157 |
|
158 |
The [training script](https://github.com/huggingface/diffusers/blob/main/examples/controlnet/README_sdxl.md) by HF🤗 was used.
|
159 |
|
|
|
170 |
Constant learning rate of 8e-5
|
171 |
|
172 |
#### Mixed precision
|
173 |
+
fp16
|
174 |
+
|
175 |
+
## Thanks
|
176 |
+
|
177 |
+
- [raulc0399](https://huggingface.co/raulc0399): for highly inspiring me with the creation of the [DWpose dataset](https://huggingface.co/datasets/dimitribarbot/dw_pose_controlnet) based on the [Openpose one](https://huggingface.co/datasets/raulc0399/open_pose_controlnet).
|
178 |
+
|
179 |
+
- [thibaud](https://huggingface.co/thibaud): for highly inspiring me with the hyper parameters of the HF training script, based on the [Openpose ControlNet](https://huggingface.co/thibaud/controlnet-openpose-sdxl-1.0).
|
180 |
+
|
181 |
+
- [RedHash](https://huggingface.co/RedHash): for the [easy_dwpose](https://github.com/reallyigor/easy_dwpose) module, which highly simplifies the DWPose inference and which I used in the examples above.
|
182 |
+
|
183 |
+
- [Hugging Face](https://huggingface.co/): for the ControlNet training script 🤗.
|