jboone100/FastWan2.2-TI2V-5B-MLX-q8

FastWan2.2-TI2V-5B — MLX, 8-bit text encoder

lBroth/FastWan2.2-TI2V-5B-MLX with one change: the umT5-XXL text encoder is quantised to 8 bit, and the tokenizer is included so the repo is self-contained. The transformer and VAE are that conversion's files, byte for byte.

Made for LocalLab, which runs it on Apple Silicon.

Why

The encoder reads the prompt once per clip and is then released, but it was the largest thing in the pipeline: 11.4 GB on disk, and 22.7 GB resident in loaders that upcast it to float32. At 8 bit it is 6.0 GB, and the embedding it produces stays within a cosine of 0.995 of the float32 one — as close as bfloat16's 0.997.

Measured on an Apple M2 Max (30-core GPU, 32 GB), encoding one prompt:

Encoder	On disk	Resident	Encode	Cosine vs float32
float32 (as mlx-video loads bf16)	11.4 GB	22.7 GB	11.7 s	—
bfloat16	11.4 GB	11.4 GB	1.5 s	0.997
8-bit (this repo)	6.0 GB	6.0 GB	1.1 s	0.995
4-bit (not shipped)	3.2 GB	3.2 GB	1.0 s	0.911 — breaks the prompt

4-bit is included in that table because it looks tempting and isn't: a 44% mean error in the embedding is a different prompt, not a cheaper one.

Files

File	Size
model.safetensors	10.0 GB	transformer (bf16), unchanged
t5_encoder_q8.safetensors	6.0 GB	umT5-XXL, 8-bit, group size 64
vae.safetensors	2.8 GB	Wan 2.2 VAE (z_dim 48), unchanged
tokenizer/	17 MB	umT5 tokenizer, from google/umt5-xxl
config.json	—	the original, plus a t5_quantization marker

Total ≈ 18.8 GB, against 24.2 GB for the source repo.

Loading it

The encoder file is named t5_encoder_q8.safetensors, not t5_encoder.safetensors, on purpose: a loader that doesn't expect quantised weights fails immediately on a missing file rather than cryptically on mismatched shapes. Quantise the encoder's Linear layers at 8 bit, group size 64, before loading the weights — config.json states this under t5_quantization.

Using mlx-video directly? Point it at the source repo instead; it expects the bfloat16 encoder.

Generation, for reference

Three denoise steps (sigmas 1.0 / 0.757 / 0.522, re-noised between steps), no classifier-free guidance, 24 fps. Measured end to end on an Apple M2 Max (30-core GPU, 32 GB) with LocalLab's engine, at 832×480 — each figure a complete generation from prompt to MP4, from a normal install of this repo:

Clip	Prompt	3 steps	Decode	Total	Peak
1 s (25 frames)	1.7 s	16.3 s	25.3 s	43 s	17.0 GB
5 s (121 frames)	1.9 s	87.7 s	221.3 s	311 s	15.0 GB

The peak is one component at a time — the encoder, then the transformer, then the VAE — so it stays near the largest single part rather than the 18.8 GB total. The decoder dominates, and its tile size is chosen from the memory available, which is why the 5-second clip peaks lower than the 1-second one.

Licence and attribution

Apache 2.0 throughout; see LICENSE and NOTICE. Credit belongs to Wan-AI for the model, FastVideo for the distillation, lBroth for the MLX conversion, and Google for umT5.