YuanzeLin/Illumicraft-checkpoints

This repository contains the pretrained checkpoints for:

IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation (NeurIPS 2025)

Yuanze Lin, Yi-Wen Chen, Yi-Hsuan Tsai, Ronald Clark, Ming-Hsuan Yang

🔗 Links

• 📄 Paper: https://arxiv.org/abs/2506.03150
• 🌐 Project Page: https://yuanze-lin.me/IllumiCraft_page/
• 💻 GitHub: https://github.com/yuanze-lin/IllumiCraft
• 🎥 YouTube: https://youtu.be/qAV58sADEzo
• 🤗 Dataset: https://huggingface.co/datasets/YuanzeLin/IllumiCraft

✨ Overview

IllumiCraft is a controllable video generation framework that jointly models scene geometry and illumination dynamics. By leveraging foreground appearance, HDR lighting maps, and 3D point tracks, IllumiCraft enables temporally consistent video generation with controllable motion and relighting effects.

📥 Download Checkpoints

Download the pretrained checkpoints:

python download_weights.py

The downloaded checkpoints will be organized as:

checkpoints/
├── Wan2.1-Fun-1.3B-Control/
└── illumicraft_pretrained_weights/

🚀 Usage

Please refer to the official GitHub repository for installation, dataset preparation, training, and inference instructions:

https://github.com/yuanze-lin/IllumiCraft

📖 Citation

If you find IllumiCraft useful for your research, please consider citing:

@article{lin2026illumicraft,
  title={IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation},
  author={Lin, Yuanze and Chen, Yi-Wen and Tsai, Yi-Hsuan and Clark, Ronald and Yang, Ming-Hsuan},
  journal={Advances in Neural Information Processing Systems},
  volume={38},
  pages={27798--27829},
  year={2026}
}