oakmindai/GLM-5.3-Flash-NVFP4-FTW

GLM-5.3 Flash NVFP4 — Spark Lab FTW

This is a ready-to-run Spark Lab checkpoint for GLM-5.3 Flash, optimized for the NVIDIA DGX Spark and its Grace Blackwell GB10 Superchip.

Spark Lab focuses on making frontier open-weight models practical on one DGX Spark. It provides tested model recipes, GB10 readiness checks, artifact preparation, unified-memory planning, NVMe-backed MoE execution, and OpenAI- and Anthropic-compatible APIs.

Spark Lab GitHub: link coming soon.

This deployment is validated for text input and text output. The vision components in the upstream checkpoint are not enabled by this Spark Lab path.

Why Spark Lab

GLM-5.3 Flash contains 320B total parameters with 18B active parameters. Its complete checkpoint is larger than the memory available to applications on one 128 GB GB10. Spark Lab uses bounded unified memory for active tensors and caches routed expert rows from local NVMe, allowing the complete model to run without dropping layers or substituting weights.

FreeToken Weight (FTW) performs model-layout work ahead of launch. Routed MoE tensors are stored in independently addressable expert banks, while resident tensors are aligned and sharded for the native loader. This revision also stores the four bandwidth-dominant projections in each KDA layer as per-output-row FP8 W8A16. The recurrent gates remain BF16, and the routed experts retain Red Hat AI's NVFP4 values. This reduces resident weight storage by about 4.25 GiB and leaves more unified memory available for the expert cache.

Provenance

1. Z.ai developed and published the original GLM-5.3 Flash model.
2. Red Hat AI published the NVFP4 source checkpoint. Its routed-expert quantization was produced with LLM Compressor.
3. FreeToken provides the native inference backend, FTW format and converter, KDA/DSA kernels, FP8 W8A16 projection kernel, and NVMe-MoE expert-bank runtime.
4. Spark Lab provides the DGX Spark product experience: GB10 checks, model recipes, capacity planning, artifact lifecycle, deployment policy, and serving workflow.
5. OakMind AI converted, measured, documented, and published this FTW artifact.

The exact Red Hat AI source revision is 9eaeadaf026871a90640e32c0604f6ab0b2d641d. No retraining was performed. In addition to the publisher's routed-expert NVFP4 quantization, Spark Lab applies the KDA FP8 conversion described above. Embeddings, output head, recurrent KDA gates, shared experts, and DSA projections retain their source precision.

Artifact identity:

• FTW fingerprint: 4c021651a1e61802
• Tensor payload: 172.03 GiB across 23 shards
• Entries: 1,382 resident weights and 252 expert-bank tensors

Install and run on NVIDIA DGX Spark

Install the package, which currently ships the sparklab command under the FreeToken distribution:

uv venv && source .venv/bin/activate
uv pip install "freetoken[accel]"
sparklab --version

The Spark Lab catalog automatically downloads this pinned FTW artifact when one is available, so the normal workflow does not require a manual hf download step:

sparklab doctor --storage-path /path/to/models
sparklab plan glm-5.3-flash --root /path/to/models --prepare
sparklab pull glm-5.3-flash --root /path/to/models --prepare
sparklab run glm-5.3-flash --root /path/to/models

Use a fast local NVMe path for /path/to/models. Review plan before downloading; the source checkpoint plus prepared artifact require substantial temporary capacity when you explicitly choose a from-source conversion.

For a direct server launch from a manually downloaded artifact, use the measured GB10 profile:

FREETOKEN_DISK_READ_WORKERS=16 sparklab serve \
  --model /path/to/GLM-5.3-Flash-NVFP4-FTW \
  --moe-backend offload \
  --moe-storage disk \
  --attention-backend dsa \
  --nvfp4-backend triton \
  --moe-host-cache-gb 0 \
  --memory-ratio 0.96 \
  --moe-cache-auto \
  --moe-prefill-sparse-max-tokens 512 \
  --disable-moe-prefill-overlap \
  --moe-prefill-hit-d2d \
  --cuda-graph-max-bs 0 \
  --page-size 1 \
  --cache-type naive \
  --max-running-requests 1 \
  --host 127.0.0.1 \
  --port 8000

After the readiness message appears:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "GLM-5.3-Flash-NVFP4-FTW",
    "messages": [{"role": "user", "content": "Explain unified memory."}],
    "max_tokens": 256,
    "stream": true
  }'

GB10 validation

The complete artifact passed FTW index/shard validation and an end-to-end streaming generation probe on one NVIDIA DGX Spark. The following fixed batch-one result uses the configuration above and a 54-token AIME prompt followed by a 255-token greedy response:

Metric	Measured value
Decode throughput	4.98 tok/s
Warm time to first token	5.379 s
First cold-cache request TTFT	10.745 s
Inter-token p50 / p99	196.75 / 266.95 ms
Expert-cache miss rate	7.44%
Expert-cache slots	6,979
Physical expert reads	116.65 GiB
Reported device allocation	104.98 GiB

Compared with the same-machine all-BF16-resident control, the optimized artifact reduced warm TTFT by 4.0% and improved decode throughput by 18.4%. TTFT is still mostly NVMe I/O: a cold prompt must load routed-expert rows that are not resident. Repeating a prompt while its expert working set remains cached can be much faster.

This is a performance probe, not a certification or a general capacity promise. Broad quality parity, long-context recall, full coding-agent completion, multimodal serving, and 60-minute endurance gates remain outstanding. The measured greedy reasoning path was coherent, but the fixed 255-token response ended before its final answer.

Credits, references, and license

• Base model and technical report: zai-org/GLM-5.3-Flash
• Publisher-quantized source checkpoint and evaluations: RedHatAI/GLM-5.3-Flash-NVFP4
• Quantization tooling used for the source NVFP4 checkpoint: LLM Compressor
• DGX Spark product, orchestration, model workflow, and deployment guidance: Spark Lab (GitHub link coming soon)
• Native inference engine, FTW format, conversion, and kernels: FreeToken
• FTW conversion, GB10 validation, and publishing: OakMind AI

Review the base model and NVFP4 source checkpoint for license terms, acceptable-use requirements, limitations, and intended-use guidance.

Citation

For GLM-5.3 Flash, cite the official GLM technical report using the citation in the base model card.

For Spark Lab:

@software{sixteenmileslabs2026sparklab,
  title={Spark Lab: Frontier Open-Weight Model Inference on NVIDIA DGX Spark},
  author={{Sixteen Miles Labs}},
  year={2026},
  url={https://github.com/sixteen-miles-labs/freetoken},
  license={Apache-2.0}
}

For the underlying inference research:

@article{yang2026freetoken,
  title={FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution},
  author={Yang, Shuo and Fan, Xiaoze and Pan, Melissa and Xi, Haocheng and Wang, Zhe and Sun, Shanlin and Keutzer, Kurt and Han, Song and Zaharia, Matei and Xu, Chenfeng and Stoica, Ion},
  journal={arXiv preprint arXiv:2608.16157},
  year={2026},
  url={https://arxiv.org/abs/2608.16157}
}