Mage-Flow-Turbo-NVFP4-Balanced-AJH

A standalone, four-step quantized derivative of microsoft/Mage-Flow-Turbo. It packages the complete transformer, scaled-FP8 text encoder, VAE, scheduler, pinned Mage inference source, and native SM120 runtime.

Policy

• Transformer: 44 native NVFP4 MLP projections + 4 BF16 projections.
• Text encoder: 252 scaled FP8 E4M3 projections.
• NVFP4 activation search: amax.
• Up activation multiplier: 1.0.
• Down activation multiplier: 0.75.
• Turbo defaults: 4 steps, CFG 1, static shift 6.

BF16 transformer projections:

• transformer_blocks.9.img_mlp.net.2
• transformer_blocks.10.img_mlp.net.2
• transformer_blocks.11.img_mlp.net.0.proj
• transformer_blocks.11.img_mlp.net.2

Showcase

Generated directly at 1024×1024 with this package, four steps, CFG 1, static shift 6, and seed 3334072683, without upscaling or post-processing.

A 4K resolution high detail photo realistic image of the top half of a cyborg woman with dark black hair, striking blue eyes that have a very subtle glow in the iris, standing side profile, head tilted up towards the sky with a questioning expression, she has subtle gaps in her skin that hint at a robotic nature, outdoor forest night setting, sky filled with bright brilliant stars that glow against the dark setting, nebula visible

Matched BF16 comparison

Both images use the same Turbo weights, scaled-FP8 text encoder, prompt, seed 3334072683, 1024×1024 resolution, four steps, CFG 1, and static shift 6. Only the transformer implementation changes.

BF16 Turbo transformer	Balanced NVFP4 Turbo transformer
4.89 s denoise, 8.52 GiB peak	3.51 s denoise, 6.29 GiB peak

Matched transformer tradeoff

In the original matched two-branch harness on the project SM120 test GPU at 1024×1024, the Balanced transformer denoised in 3.51 seconds with 6.29 GiB peak allocated, versus 4.89 seconds and 8.52 GiB for the BF16 Turbo transformer. This is about a 1.39× denoising speedup and 2.22 GiB lower transformer-stage VRAM for this exact four-step case; results are hardware and prompt dependent.

The scaled-FP8 encoder used about 4.95 GiB standalone versus 8.36 GiB for BF16. Direct prompt-conditioning cosine against BF16 ranged from 0.9932 to 0.9971 in the frozen four-case suite.

CFG-1 single-branch execution

Turbo defaults to CFG 1. Following the upstream Mage pipeline, this runtime does not encode a negative prompt or execute an unconditional transformer branch unless CFG is greater than 1 and the negative prompt is nonempty.

At 1024×1024 in resident mode, removing that redundant branch reduced warm denoising from 2.78 to 1.43 seconds and warm end-to-end latency from 4.90 to 3.56 seconds, a further 1.38× speedup. CFG values above 1 continue to use the normal conditional and unconditional branches.

Persistent server mode

For repeated generation on a 16 GiB GPU, the packaged runtime can retain the text encoder, transformer, and VAE on CUDA instead of transferring each component for every stage:

pipe, load_report = runtime.load_pipeline(model=model_id, torch=torch)
placement = runtime.place_pipeline_resident(pipe, torch)
image, stages, screening = runtime.generate_staged(
    pipe=pipe,
    prompt=prompt,
    negative_prompt="",
    height=1024,
    width=1024,
    steps=4,
    cfg=1.0,
    seed=1,
    static_shift=6.0,
    torch=torch,
    keep_resident=True,
)

With the CFG-1 single-branch path active, warm 1024×1024 latency decreased from 5.90 seconds staged to 3.56 seconds resident: a 1.66× end-to-end speedup. Peak allocated VRAM increased from 5.89 GiB to 11.39 GiB. The staged and resident decoded-pixel hashes were identical. Resident mode is intended for a long-running WebUI, API server, or ComfyUI process; the default remains staged for lower VRAM use.

Together, CFG-1 single-branch execution and resident placement reduced warm latency from the original 7.21-second staged path to 3.56 seconds, a 2.02× combined speedup on the tested GPU.

Requirements

Tested on Linux x86-64, NVIDIA Blackwell SM120, CUDA 13.1, Python 3.11, PyTorch 2.13.0+cu130, comfy-kitchen==0.2.22, and flash-attn==2.8.3. Use a virtual environment.

python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The packaged binaries target the tested stack. Run build_native.sh after changing PyTorch, CUDA, or the C++ ABI.

Generate

CUDA_VISIBLE_DEVICES=0 .venv/bin/python generate.py \
  --model ajh-code/Mage-Flow-Turbo-NVFP4-Balanced-AJH \
  --prompt 'A detailed watercolor fox reading under an old oak tree' \
  --output fox.png --height 1024 --width 1024 \
  --steps 4 --cfg 1 --seed 1

Stock Diffusers does not understand the custom native NVFP4 modules; use the packaged entry point or a compatible integration. SM120 is required by this runtime.

Validate

python validate_release.py

Attribution

The scaled-FP8 Qwen3-VL artifact comes from starsfriday/Mage-Flow-Edit-FP8 at revision 5a64f75aa6a962cbc6c9f0bc42ed0f819b739890. See THIRD_PARTY_NOTICES.md and the included license files.