Update README.md
Browse files
README.md
CHANGED
@@ -6,6 +6,7 @@ language:
|
|
6 |
- en
|
7 |
---
|
8 |
|
|
|
9 |
## Using HunyuanDiT ControlNet
|
10 |
|
11 |
|
@@ -20,7 +21,9 @@ language:
|
|
20 |
```bash
|
21 |
cd HunyuanDiT
|
22 |
# Use the huggingface-cli tool to download the model.
|
|
|
23 |
huggingface-cli download Tencent-Hunyuan/HYDiT-ControlNet --local-dir ./ckpts/t2i/controlnet
|
|
|
24 |
|
25 |
# Quick start
|
26 |
python3 sample_controlnet.py --no-enhance --load-key distill --infer-steps 50 --control_type canny --prompt "在夜晚的酒店门前,一座古老的中国风格的狮子雕像矗立着,它的眼睛闪烁着光芒,仿佛在守护着这座建筑。背景是夜晚的酒店前,构图方式是特写,平视,居中构图。这张照片呈现了真实摄影风格,蕴含了中国雕塑文化,同时展现了神秘氛围" --condition_image_path controlnet/asset/input/canny.jpg --control_weight 1.0
|
@@ -67,10 +70,17 @@ Examples of condition input and ControlNet results are as follows:
|
|
67 |
|
68 |
### Training
|
69 |
|
70 |
-
We utilize [**DWPose**](https://github.com/IDEA-Research/DWPose) for pose extraction. Please follow their guidelines to download the checkpoints and save them to `hydit/annotator/ckpts` directory.
|
|
|
|
|
|
|
|
|
|
|
|
|
71 |
```bash
|
72 |
-
pip install matplotlib
|
73 |
-
pip install onnxruntime_gpu
|
|
|
74 |
```
|
75 |
|
76 |
|
@@ -162,3 +172,4 @@ c. Using depth ControlNet during inference
|
|
162 |
python3 sample_controlnet.py --no-enhance --load-key distill --infer-steps 50 --control_type pose --prompt "一位亚洲女性,身穿绿色上衣,戴着紫色头巾和紫色围巾,站在黑板前。背景是黑板。照片采用近景、平视和居中构图的方式呈现真实摄影风格" --condition_image_path controlnet/asset/input/pose.jpg --control_weight 1.0
|
163 |
```
|
164 |
|
|
|
|
6 |
- en
|
7 |
---
|
8 |
|
9 |
+
|
10 |
## Using HunyuanDiT ControlNet
|
11 |
|
12 |
|
|
|
21 |
```bash
|
22 |
cd HunyuanDiT
|
23 |
# Use the huggingface-cli tool to download the model.
|
24 |
+
# We recommend using distilled weights as the base model for ControlNet inference, as our provided pretrained weights are trained on them.
|
25 |
huggingface-cli download Tencent-Hunyuan/HYDiT-ControlNet --local-dir ./ckpts/t2i/controlnet
|
26 |
+
huggingface-cli download Tencent-Hunyuan/Distillation-v1.1 ./pytorch_model_distill.pt --local-dir ./ckpts/t2i/model
|
27 |
|
28 |
# Quick start
|
29 |
python3 sample_controlnet.py --no-enhance --load-key distill --infer-steps 50 --control_type canny --prompt "在夜晚的酒店门前,一座古老的中国风格的狮子雕像矗立着,它的眼睛闪烁着光芒,仿佛在守护着这座建筑。背景是夜晚的酒店前,构图方式是特写,平视,居中构图。这张照片呈现了真实摄影风格,蕴含了中国雕塑文化,同时展现了神秘氛围" --condition_image_path controlnet/asset/input/canny.jpg --control_weight 1.0
|
|
|
70 |
|
71 |
### Training
|
72 |
|
73 |
+
We utilize [**DWPose**](https://github.com/IDEA-Research/DWPose) for pose extraction. Please follow their guidelines to download the checkpoints and save them to `hydit/annotator/ckpts` directory. We provide serveral commands to quick install:
|
74 |
+
```bash
|
75 |
+
mkdir ./hydit/annotator/ckpts
|
76 |
+
wget -O ./hydit/annotator/ckpts/dwpose.zip https://dit.hunyuan.tencent.com/download/HunyuanDiT/dwpose.zip
|
77 |
+
unzip ./hydit/annotator/ckpts/dwpose.zip -d ./hydit/annotator/ckpts/
|
78 |
+
```
|
79 |
+
Additionally, ensure that you install the related dependencies.
|
80 |
```bash
|
81 |
+
pip install matplotlib==3.7.5
|
82 |
+
pip install onnxruntime_gpu==1.16.3
|
83 |
+
pip install opencv-python==4.8.1.78
|
84 |
```
|
85 |
|
86 |
|
|
|
172 |
python3 sample_controlnet.py --no-enhance --load-key distill --infer-steps 50 --control_type pose --prompt "一位亚洲女性,身穿绿色上衣,戴着紫色头巾和紫色围巾,站在黑板前。背景是黑板。照片采用近景、平视和居中构图的方式呈现真实摄影风格" --condition_image_path controlnet/asset/input/pose.jpg --control_weight 1.0
|
173 |
```
|
174 |
|
175 |
+
|