MiniDance
A 119.5M-parameter text-to-video DiT trained from scratch on nature scenery for about ¥12 (≈ $1.7) — roughly 11 hours on a single Colab L4. Give it an English or Chinese prompt and it generates a 33-frame, 256×256 clip (12 fps). Inference runs on an 8 GB GPU.
一个从零训练的 1.2 亿参数文生视频模型，专注自然风光，在一张 Colab L4 上训练约 11 小时、成本约 12 元。支持中英文提示词，生成 256×256、33 帧的短视频，8 GB 显卡即可推理。
 |
 |
| Ocean waves rolling onto a sandy beach at sunset, golden light on the water. | A waterfall cascading down mossy rocks into a clear pool. |
 |
 |
| Aerial view of a turquoise river winding through autumn forest. | 海上日落，金色余晖映在水面。 |
Files
| File | Description |
|---|---|
minidance-base-sft.safetensors |
pretrain + SFT, recommended |
minidance-base.safetensors |
pretrain only (64k steps) |
Both are the EMA weights in bf16 (239 MB); the model config is stored in the safetensors metadata.
Usage
git clone https://github.com/JuniverseLi/minidance && cd minidance
uv sync # or: pip install -r requirements.txt (after installing CUDA PyTorch)
hf download JuniverseLi/MiniDance --local-dir ./out/MiniDance
uv run python scripts/sample.py --ckpt out/MiniDance/minidance-base-sft.safetensors \
--prompt "A waterfall cascading down mossy rocks into a clear pool." "海上日落，金色余晖映在水面。" --seed 0
The first run also downloads the frozen Wan2.2 VAE (2.7 GB) and Qwen3-0.6B (1.5 GB).
Model
| Component | Model | Params | |
|---|---|---|---|
| Video VAE | Wan2.2 VAE (16× spatial, 4× temporal, 48 channels) | — | frozen |
| Text encoder | Qwen3-0.6B (last hidden state, 1024-d) | 0.6B | frozen |
| Denoiser | MiniDance DiT: dim 768, 12 blocks, 12 heads, 3D RoPE, QK-Norm, adaLN-single, text cross-attention | 119.5M | from scratch |
Trained with rectified flow (logit-normal timesteps, shift 3, 10% caption dropout); sampled with 30 Euler steps and CFG 5.
Training
| Stage | Data | Steps | LR | Time (L4) |
|---|---|---|---|---|
| Pretrain | 40,429 scenery clips, long + short captions | 64,000 × batch 32 | 2e-4 → 2e-5 | ≈ 10.5 h |
| SFT | 11,496 high-aesthetic clips with motion, short captions | 3,000 × batch 32 | 5e-5 → 5e-6 | ≈ 35 min |
Data: JuniverseLi/minidance-scenery, filtered from OpenVid-1M.
Limitations
A "live wallpaper" model: good at forests, waves, rivers and lakes, sunsets, aerial shots, clouds and mountains; weaker on rare scenes (glaciers, lightning, fire, aurora); cannot generate people, animals, vehicles, cities or text, which were excluded from the training data. Motion is mostly gentle (flowing water, drifting clouds, slow camera moves).
License
Apache 2.0. The frozen Wan2.2 VAE and Qwen3-0.6B are Apache 2.0; the training data derives from OpenVid-1M (CC BY 4.0).