noctuashap/TrackWan-1.3B-I2V-synth-stage2

TrackWan 1.3B I2V — Motion-Track Teacher (synth stage-2)

A motion-track-conditioned image-to-video model based on Wan2.1 1.3B (Fun-InP). On top of the standard I2V conditioning it takes a set of sparse point tracks (a small number of pixel trajectories with per-frame visibility) and generates a video whose motion follows those tracks. It is the "teacher" model in a MotionStream-style track-conditioning recipe.

This repository is a complete, self-contained diffusers pipeline — transformer, VAE, T5 text encoder, CLIP image encoder, tokenizer, scheduler — so it loads directly with no external weights.

Checkpoint

• Final checkpoint (step 600) of the synth_stage2_paperLR run.
• Stage-2 fine-tune (lr 1e-6, global batch size 128, 600 steps) on a combined synthetic track dataset, with stochastic track/motion dropping, starting from the stage-1 OpenVid-1M teacher.
• 480×832, 121 frames, 24 fps, flow_shift = 6.

Architecture

Standard Wan2.1 1.3B DiT (30 layers, hidden 1536) with a WanTrack head:

field	value
in_channels	52 (16 latent + 20 I2V/mask + 16 track channels)
out_channels	16
track_config.id_dim	64 (sinusoidal track-id embedding)
track_config.track_channels	16
track_config.vae_spatial_compression	8
track_config.vae_temporal_compression	4

The track pathway is track_encoder.{temporal_conv, proj} plus the extra patch_embedding input channels [:, 36:] (the "track slot").

Usage (FastVideo)

The model is a diffusers-format directory and loads directly by path in FastVideo's WanTrack inference / training stack. VAE, T5, and CLIP are the standard Wan2.1 components and are bundled here.

Notes

• VAE / T5 / CLIP are identical to the Wan2.1 base components; they are included so the pipeline is directly loadable.
• Base model: Wan2.1 1.3B (Fun-InP variant).