Edit model card

HunyuanDiT Distillation Acceleration

Language: English | 中文

We provide a distillation version of HunyuanDiT for your inference acceleration.

Based on progressive distillation method, we accelerate Hunyuan-Dit two times without any performance drop. With the use of distillation model, It achieves the effect of halving the time consumption based on any inference mode.

The following table shows the requirements for running the distillation model and the acceleration performance of our distillation model (batch size = 1). We evaluate the accelaration on various GPU (like H800,A100, 3090, 4090) as well as different inference mode.

GPU CUDA version model inference mode inference steps GPU Peak Memory inference time
H800 12.1 HunyuanDiT PyTorch 100 13G 28s
H800 12.1 HunyuanDiT TensorRT 100 12G 10s
H800 12.1 HunyuanDiT Distill+PyTorch 50 13G 14s
H800 12.1 HunyuanDiT Distill+TensorRT 50 12G 5s
A100 11.7 HunyuanDiT PyTorch 100 13GB 54s
A100 11.7 HunyuanDiT TensorRT 100 11GB 20s
A100 11.7 HunyuanDiT Distill+PyTorch 50 13GB 25s
A100 11.7 HunyuanDiT Distill+TensorRT 50 11GB 10s
3090 11.8 HunyuanDiT PyTorch 100 14G 98s
3090 11.8 HunyuanDiT TensorRT 100 14G 40s
3090 11.8 HunyuanDiT Distill+PyTorch 50 14G 49s
3090 11.8 HunyuanDiT Distill+TensorRT 50 14G 20s
4090 11.8 HunyuanDiT PyTorch 100 14G 54s
4090 11.8 HunyuanDiT TensorRT 100 14G 20s
4090 11.8 HunyuanDiT Distill+PyTorch 50 14G 27s
4090 11.8 HunyuanDiT Distill+TensorRT 50 14G 10s

Basically, the requirements for running the models is the same as the original model.

Instructions

The dependencies and installation are basically the same as the original model.

Then download the model using the following commands:

cd HunyuanDiT
# Use the huggingface-cli tool to download the model.
huggingface-cli download Tencent-Hunyuan/Distillation-v1.1 ./pytorch_model_distill.pt --local-dir ./ckpts/t2i/model

Inference

Using Gradio

Make sure you have activated the conda environment before running the following command.

# By default, we start a Chinese UI.
python app/hydit_app.py  --load-key distill 

# Using Flash Attention for acceleration.
python app/hydit_app.py --infer-mode fa --load-key distill   

# You can disable the enhancement model if the GPU memory is insufficient.
# The enhancement will be unavailable until you restart the app without the `--no-enhance` flag. 
python app/hydit_app.py --no-enhance ---load-key distill  

# Start with English UI
python app/hydit_app.py --lang en --load-key distill  

Using Command Line

We provide several commands to quick start:

# Prompt Enhancement + Text-to-Image. Torch mode
python sample_t2i.py --prompt "渔舟唱晚" --load-key distill  --infer-steps 50

# Only Text-to-Image. Torch mode
python sample_t2i.py --prompt "渔舟唱晚" --no-enhance --load-key distill  --infer-steps 50

# Only Text-to-Image. Flash Attention mode
python sample_t2i.py --infer-mode fa --prompt "渔舟唱晚" --load-key distill --infer-steps 50

# Generate an image with other image sizes.
python sample_t2i.py --prompt "渔舟唱晚" --image-size 1280 768 --load-key distill  --infer-steps 50

More example prompts can be found in example_prompts.txt

Downloads last month
495
Unable to determine this model’s pipeline type. Check the docs .