memset0/vsqa-preview-1.3b

VSQA preview 1.3B — DMD step 1000

Preview checkpoint. This is the student of a completed 1000-step DMD run, exported at its final step (1000). It is published for early inspection.

What it is

A few-step (3-step DMD) student of Wan2.1-T2V-1.3B trained with video sparse attention (VSA) and NVFP4 quantization-aware training (QAT) of both attention and linear layers (VSQA project).

Training lineage (all at 61 frames, 896×448, global batch 16, NVFP4 attention and linear QAT):

1. Dense NVFP4 per-layer reconstruction (groups of 5 blocks, teacher inputs), from the original Wan2.1-T2V-1.3B weights, LR 1e-6; the step-500 checkpoint is used.
2. Sparse ODE initialization with an auxiliary reconstruction loss (endpoint weight 0.5, attention and MLP reconstruction weights 0.25 each), VSA C128 cube (4, 4, 8) at sparsity 0.9, 3-step schedule, LR 1e-5; the step-500 checkpoint is used.
3. DMD distillation, 1000 steps (this checkpoint), student LR 2e-6, critic LR 4e-7, generator updated every 5 iterations, no EMA, real-score guidance 3.5. Teacher and critic are the original dense Wan2.1-T2V-1.3B.

Sampling contract

• 3 denoising steps at timesteps [1000.0, 941.1763916015625, 800.0], no additional timestep warp.
• flow_shift = 8.0, guidance scale 1.0 (distilled), 61 frames at 896×448.
• Attention backend used in training: VSA_QAT_TRAIN_C128 (sparsity 0.9); linear quantization nvfp4_qat_train. Reproducing the trained numerics needs the FastVideo VSA/QAT kernels; running the weights with plain dense BF16 attention and linears is a different (untested) configuration.

Files

Diffusers layout. transformer/ holds the exported student; text_encoder/, tokenizer/, vae/ and scheduler/ are unchanged from the base model. metadata.json (normalized training contract), artifact_manifest.json, resolved.yaml and source.yaml record provenance; the export passed a strict reload and its payload digests match artifact_manifest.json.

License follows the base model (Apache-2.0).