nickchiu/Refocusing-Any-Video

Refocusing Any Video: Temporally Consistent Video Refocusing for Dynamic Focal Planes

SIGGRAPH Asia 2026

Chee Kim Gan, Pin-Yen Chiu, I-Sheng Fang, Jun-Cheng Chen
Research Center for Information Technology Innovation, Academia Sinica

Inference checkpoints for Refocusing Any Video (RAV). Video Defocus Deblurring (VDD) recovers an all-in-focus video; Video Bokeh Rendering (VBR) refocuses it using depth and a chosen focal plane. Optional identity adapters guide face restoration.

Source code and inference instructions

Release status

• Inference code and pretrained checkpoints
• Training code — being prepared for a future release

Checkpoints

Folder	Contents
vdd_weights/checkpoint-90000/	VDD LoRA weights
vbr_weights/checkpoint-100000/	VBR LoRA weights and required input convolution
identity_adapter/	Identity Adapter
identity_adapter_track4gen/	Identity Adapter with Track4Gen Refiner
facenet/	MTCNN face detector weights

Download and inference

After setting up the code environment, run from the cloned code repository root:

python tools/checkpoints/transfer_checkpoints.py download \
  --repo-id nickchiu/Refocusing-Any-Video --local-dir ./checkpoints

Download SVD, Video Depth Anything, and the auxiliary face models separately using the checkpoint guide, then follow the inference example. These adapters require the code and SVD base model. Generated details may differ from the original scene.

Models and licenses

RAV uses Stable Video Diffusion (SVD), Video Depth Anything, FaRL, ArcFace, OpenAI CLIP, and facenet-pytorch.

SVD and applicable derivatives are subject to the SVD license and NOTICE. Powered by Stability AI (attribution required by the SVD license). MTCNN retains its MIT license; other models follow their providers' terms. An additional project-specific license has not yet been specified.

Citation

If you use this work, please cite:

@misc{gan2026refocusinganyvideo,
  title  = {Refocusing Any Video: Temporally Consistent Video Refocusing for Dynamic Focal Planes},
  author = {Gan, Chee Kim and Chiu, Pin-Yen and Fang, I-Sheng and Chen, Jun-Cheng},
  year   = {2026},
  note   = {SIGGRAPH Asia 2026},
  url    = {https://github.com/aiiu-lab/Refocusing-Any-Video}
}