BlueSourceJY/shotpilot-stage1-recipe-b

ShotPilot Stage 1 generator — Recipe B training checkpoints

These are raw training checkpoints of ShotPilot Stage 1, a chunk-level autoregressive, multi-shot text-to-video generator fine-tuned from Wan2.1-T2V-1.3B. Generation is conditioned on:

• an enriched global prompt, injected by cross-attention;
• a per-chunk action token (BOS_VIDEO / CUT / CONTINUE);
• a per-chunk episode-time token.

A chunk is 3 Wan latent frames. History follows the ShotStream "RGB6" policy: full clean history for the current shot (and the previous shot at a first CUT), and 6 sampled RGB reference frames for earlier shots.

Recipe B forks the original run at step 1600. Only these settings change:

Setting	Original run	Recipe B
Learning rate	1e-5 (constant after a 50-step warmup)	2e-6 constant from step 1601
EMA	none	0.999, fp32, updated every optimizer step
History noise	none (clean history, timestep 0)	σ ~ U[0, 0.2) per target on current/previous-shot history chunks; RGB6 references stay clean
Weights used for evaluation	raw	EMA

Everything else is shared with the original run:

• full fine-tune of the DiT plus action and time embeddings; frozen VAE and UMT5;
• 8 clips per update, all chunks supervised, 10% text dropout;
• AdamW (0.9, 0.999), weight decay 0.01, grad clip 1.0;
• bf16 FSDP on 8 × AMD MI210.

The config is in configs/stage1_generator_recipe_b.yaml.

Contents

Folder	Step	Files
checkpoint-0001800/	1800	rank-00000.pt … rank-00007.pt, metadata.json, _SUCCESS.json
checkpoint-0002000/	2000	same
checkpoint-0002200/	2200	same
checkpoint-0002400/	2400	same
checkpoint-0002600/	2600	same

Each checkpoint is about 22.8 GB of FSDP-sharded training state for 8 ranks. It holds the model, AdamW moments, the fp32 EMA copy, and the sampler, data-loader and RNG state. metadata.json stores the resolved config, the step and data fingerprints. _SUCCESS.json marks a completely written checkpoint.

These are resumable training checkpoints, not a single-file inference model. They are PyTorch pickles: load them only with torch.load(..., weights_only=False) from a source you trust. To get inference weights, the ShotPilot code exports either the EMA or the raw weights (scripts/export_stage1_model.py --weights ema) on 8 ranks. The training code is not public yet.

Checkpoint 1600, the fork source, belongs to the original run and is not included. Training was still running when this snapshot was uploaded on 2026-10-12.

Evaluation

All 22 original-run evaluations and these 5 Recipe B evaluations are in the dataset BlueSourceJY/shotpilot-stage1-evaluations. It includes every generated video and the per-clip scores.

The evaluation suite is fixed: 64 validation clips plus 32 paired CUT/CONTINUE interventions, 50-step UniPC, CFG 5, and fixed seeds. Five VBench-derived dimensions are scored on a custom subset; this is not the official VBench total. The temporal metrics are computed within shots. Steps are matched across the two runs:

Step	Run	Aesthetic	Imaging	Subject cons.	Motion smooth.	Dynamic deg.	CUT/CONT latent RMSE
1800	Recipe B (EMA)	0.5158	0.6383	0.9652	0.9892	0.3555	0.9814
1800	Original	0.5165	0.5858	0.9487	0.9868	0.5312	0.9001
2000	Recipe B (EMA)	0.5118	0.6402	0.9619	0.9884	0.4062	0.9982
2000	Original	0.5101	0.5727	0.9571	0.9892	0.3008	0.9868
2200	Recipe B (EMA)	0.5117	0.6253	0.9591	0.9881	0.3932	1.0166
2200	Original	0.4988	0.6338	0.9557	0.9872	0.4648	1.0312
2400	Recipe B (EMA)	0.5135	0.6281	0.9608	0.9871	0.4115	1.0511
2400	Original	0.4985	0.5882	0.9484	0.9855	0.6562	1.0117
2600	Recipe B (EMA)	0.5174	0.6237	0.9598	0.9869	0.4115	1.1194
2600	Original	0.5020	0.5932	0.9522	0.9845	0.5182	1.0156

Motion smoothness and dynamic degree mostly track how much motion there is. Compare them at matched motion amplitude, not by their raw maxima.

License and data

The weights are released under CC BY-NC 4.0 for non-commercial research. The fine-tuning data comes from non-commercial research video datasets. The base model, Wan2.1-T2V-1.3B, is Apache-2.0.