oakmindai/GLM-5.3-NVFP4-FTW

GLM-5.3 NVFP4 — SparkLab FTW

This is a ready-to-run SparkLab checkpoint for GLM-5.3, prepared for one NVIDIA DGX Spark with its Grace Blackwell GB10 Superchip.

SparkLab packages tested model recipes, hardware checks, unified-memory planning, NVMe-backed MoE execution, and OpenAI-compatible serving for one DGX Spark.

Validated scope: text input and text output on one 128 GB NVIDIA DGX Spark. Vision and other modalities are out of scope.

What this repository contains

This repository does not introduce a new model. It converts the immutable Inferact NVFP4 checkpoint to SparkLab's sharded FTW execution format:

1. Z.AI developed GLM-5.3.
2. Inferact produced the NVFP4 source checkpoint using NVIDIA Model Optimizer.
3. SparkLab provides the native runtime, FTW format, conversion tooling, kernels, model recipe, and serving flow.
4. OakMind AI performed, validated, documented, and published this conversion.

The exact source revision is ce67b36f3669192b5bb233819f0fda6c8a9837f8. Routed experts retain their NVFP4 representation and are laid out as independently addressable expert banks. To fit the complete model on GB10, SparkLab converts large resident BF16 attention, dense, and output projections to per-row W8A16 FP8. No training was performed. The FTW payload is 428,713,099,264 bytes across 77 shards; its manifest fingerprint is a0e799b03bceb4bf.

Run with SparkLab on NVIDIA DGX Spark

Install SparkLab and download the checkpoint:

git clone https://github.com/sixteen-miles-labs/sparklab
cd sparklab
./install.sh

hf download oakmindai/GLM-5.3-NVFP4-FTW \
  --local-dir ~/models/GLM-5.3-NVFP4-FTW

Verify the host and storage path:

sparklab doctor --storage-path ~/models/GLM-5.3-NVFP4-FTW

Start the OpenAI-compatible server with the measured configuration:

SPARKLAB_DISK_READ_WORKERS=20 sparklab serve \
  --model ~/models/GLM-5.3-NVFP4-FTW \
  --moe-backend offload \
  --moe-storage disk \
  --nvfp4-backend flashinfer \
  --moe-host-cache-gb 0 \
  --moe-cache-size 675 \
  --moe-cache-policy layer_lru \
  --memory-ratio 0.90 \
  --num-tokens 2048 \
  --disable-moe-prefill-overlap \
  --moe-prefill-sparse-max-tokens 256 \
  --moe-shared-expert-overlap \
  --cuda-graph-max-bs 0 \
  --max-running-requests 1 \
  --host 127.0.0.1 \
  --port 8000

Send a request:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "GLM-5.3-NVFP4-FTW",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

GB10 validation

The complete checkpoint was measured with a 54-token prompt and a 256-token output cap on one DGX Spark:

Metric	Result
Decode throughput	0.813 tok/s
Warm TTFT	2.530 s
OOM events	0
Swap-out growth	0

The run began with pre-existing swap use and read 7,882 swap-in pages. Its output reached the 256-token cap before stating the expected final answer, so these numbers are bounded performance evidence, not a correctness, quality, context-length, concurrency, endurance, parser, or production certification.

Credits and license

• Base model and architecture: Z.AI
• Quantization: Inferact, using NVIDIA Model Optimizer
• Runtime, FTW format, conversion, kernels, and deployment: SparkLab
• Research ancestry: FreeToken
• FTW conversion and publishing: OakMind AI

The upstream model is distributed under the custom Z.AI license included in this repository. Review the base model card and quantized source card for complete terms, limitations, evaluation details, and intended-use guidance.

Citation

@software{sparklab2026,
  title  = {SparkLab: Frontier Open-Weight Inference on NVIDIA DGX Spark},
  author = {{SixteenMiles Labs}},
  year   = {2026},
  url    = {https://github.com/sixteen-miles-labs/sparklab}
}

For GLM-5.3 research, also use the citation provided by the upstream model authors.