HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models
Paper โข 2608.13205 โข Published โข 1
HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models
Jiazi Bu* |
Pengyang Ling*ยง |
Yujie Zhou* |
Yibin Wang |
Yuhang Zang |
Xuanlang Dai |
Shengyuan Ding | Tianyi Wei | Xiaohang Zhan | Jiaqi Wang | Tong Wu | Dahua Lin | Xingang Panโ
Shengyuan Ding | Tianyi Wei | Xiaohang Zhan | Jiaqi Wang | Tong Wu | Dahua Lin | Xingang Panโ
Shanghai Jiao Tong University, Nanyang Technological University, Shanghai AI Laboratory,
University of Science and Technology of China, Fudan University, Shanghai Innovation Institute
The Chinese University of Hong Kong, CPII under InnoHK, JD.com, Adobe Research
(*Equal Contribution)(ยงProject Leader)(โ Corresponding Author)
University of Science and Technology of China, Fudan University, Shanghai Innovation Institute
The Chinese University of Hong Kong, CPII under InnoHK, JD.com, Adobe Research
This model is presented in the paper HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models. Project page: https://bujiazi.github.io/hpsd.github.io/
Abstract
Text-Image-to-Video (TI2V) models are an emerging unified architecture, where a single model simultaneously supports text-to-video (T2V) and image-to-video (I2V) generation. Given a high-quality first frame or a detailed textual prompt, TI2V models unlock substantially better visual quality than their T2V mode, raising a natural question: can the capability elicited by such privileged conditions be internalized into the model's own base generation ability? A common approach toward this goal is model self-distillation. However, the most straightforward solution, supervised fine-tuning, follows an off-policy strategy: its supervision is confined to teacher-generated endpoints from a fixed offline distribution rather than student-visited states, lacking precise correction tailored to the evolving policy. Recent on-policy distillation methods instead suffer from condition-state mismatch, where supervision is steered toward the given first frame instead of the student's actual content, misleading the correction. To achieve self-distillation that absorbs the teacher's privileged prior while retaining precise policy correction, in this work, we propose Hybrid-Policy Self-Distillation (HPSD), a novel self-distillation framework where a single TI2V model acts as both teacher and student under different conditions: the teacher operates in TI2V mode with a high-quality first frame and an enhanced prompt, while the student runs in the base T2V mode with only the vanilla prompt. Specifically, the student inherits off-policy teacher trajectory points as anchors, locally refines them toward its own policy, and finally receives velocity-level supervision on these self-generated roll-outs. Extensive experiments demonstrate that HPSD significantly improves T2V performance while also delivering notable TI2V gains, effectively strengthening the model's base generation ability.
๐ Model
The adapter_model.safetensors is based on WAN-2.2-TI2V with our HPSD training.
๐ง Github Link
https://github.com/Bujiazi/HPSD
๐ Inference
import torch
from huggingface_hub import snapshot_download
from diffusers import WanPipeline, AutoencoderKLWan
from diffusers.utils import export_to_video
from peft import PeftModel
dtype = torch.bfloat16
device = "cuda"
model_id = "Wan-AI/Wan2.2-TI2V-5B-Diffusers"
checkpoint = snapshot_download(repo_id="Bujiazi/HPSD", repo_type="model")
vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
pipe = WanPipeline.from_pretrained(model_id, vae=vae, torch_dtype=dtype)
pipe.to(device)
pipe.transformer = PeftModel.from_pretrained(pipe.transformer, checkpoint, torch_dtype=dtype).to(device)
height = 704
width = 1280
num_frames = 81
num_inference_steps = 50
guidance_scale = 5.0
base_seed = 42
prompt = "A sports car speeding along a winding coastal road at sunset. Dynamic low-angle tracking shot, realistic reflections, dramatic clouds, subtle motion blur, cinematic composition."
negative_prompt = "่ฒ่ฐ่ณไธฝ๏ผ่ฟๆ๏ผ้ๆ๏ผ็ป่ๆจก็ณไธๆธ
๏ผๅญๅน๏ผ้ฃๆ ผ๏ผไฝๅ๏ผ็ปไฝ๏ผ็ป้ข๏ผ้ๆญข๏ผๆดไฝๅ็ฐ๏ผๆๅทฎ่ดจ้๏ผไฝ่ดจ้๏ผJPEGๅ็ผฉๆฎ็๏ผไธ้็๏ผๆฎ็ผบ็๏ผๅคไฝ็ๆๆ๏ผ็ปๅพไธๅฅฝ็ๆ้จ๏ผ็ปๅพไธๅฅฝ็่ธ้จ๏ผ็ธๅฝข็๏ผๆฏๅฎน็๏ผๅฝขๆ็ธๅฝข็่ขไฝ๏ผๆๆ่ๅ๏ผ้ๆญขไธๅจ็็ป้ข๏ผๆไนฑ็่ๆฏ๏ผไธๆก่
ฟ๏ผ่ๆฏไบบๅพๅค๏ผๅ็่ตฐ"
generator = torch.Generator(device=device).manual_seed(base_seed)
output = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
height=height,
width=width,
num_frames=num_frames,
guidance_scale=guidance_scale,
num_inference_steps=num_inference_steps,
generator=generator,
).frames[0]
export_to_video(output, "hpsd_test.mp4", fps=16)
๐ Citation
If you find our work helpful, please consider giving a star โญ and citation ๐
@article{bu2026hpsd,
title={HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models},
author={Bu, Jiazi and Ling, Pengyang and Zhou, Yujie and Wang, Yibin and Zang, Yuhang and Dai, Xuanlang and Ding, Shengyuan and Wei, Tianyi and Zhan, Xiaohang and Wang, Jiaqi and others},
journal={arXiv preprint arXiv:2608.13205},
year={2026}
}
๐ Acknowledgements
The code is built upon the below repositories, we thank all the contributors for open-sourcing.
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support