UGG

UGG is a video diffusion model that generates RGB, per-frame camera rays and depth together. It is built on Wan2.2-TI2V-5B, extended to a mixture of transformers with RGB, ray and depth experts (12.6B parameters in total). Whatever subset of the three modalities you supply, the model fills in the rest. That makes one checkpoint cover six tasks:

task	given	generated
video → 4D	all frames	depth + cameras
image + camera → 4D	first frame + trajectory	video + depth
first/last image + camera → 4D	first and last frames + trajectory	video + depth
text → 4D	prompt	video + depth + cameras
camera → 4D	trajectory (+ prompt)	video + depth
depth → 4D	depth (+ prompt, optional frames)	video + cameras

Code, samples and the training recipe: https://github.com/amanikiruga/UGG

Files

file	contents
ugg_release.safetensors	the full UGG transformer (bf16) plus VAE latent normalisation statistics

The Wan2.2 VAE and UMT5-XXL text encoder are not duplicated here. The code fetches them from Wan-AI/Wan2.2-TI2V-5B.

Usage

git clone https://github.com/amanikiruga/UGG && cd UGG
# environment: see the repo README
python infer.py task=video input=assets/bonn_balloon2/frames.mp4

from ugg import UGG

model = UGG.from_pretrained()          # downloads ugg_release.safetensors on first use
pred = model.video_to_4d("clip.mp4")    # pred.rgb, pred.depth, pred.c2w, pred.intrinsics
pred.save("outputs/clip")

Output resolution is 288×384. Clip length must satisfy num_frames % 4 == 2 (for example 10, 26 or 50). Inference takes one 80 GB GPU.

License

The UGG weights are released under the MIT license. The Wan2.2 base components they rely on are distributed by Wan-AI under their own license.