0xTank/SuperGLM-5.2-v2-W4W8-Overlay

SuperGLM-5.2 v2 W4/W8 overlay — TP4/K5/200K on 4× DGX Spark

This repository distributes our 62 derived W8 overlay shards, not a full checkpoint. Users obtain the pinned upstream packed base separately and assemble the checkpoint locally. The overlay package is approximately 7.13 GB; it avoids duplicating more than 400 GB of unchanged base weights.

Identity

• SuperGLM v2 source: Jiunsong/SuperGLM-5.2-abliterated-NVFP4@076582b8a58d3f924a68af550a630edffada5e95
• Packed base: QuantTrio/GLM-5.2-Int4-Int8Mix@1d3bcfe5ec549ecd000fd80b37f191183842e983
• Our derived changes: 124 source tensors converted to 372 W8A16 group-128 tensor entries in 62 overlay shards.
• Unchanged routed experts remain W4A16; unchanged ordinary linears and the matching 256-expert native MTP remain W8A16.
• The final checkpoint is a SuperGLM v2 W4/W8 derivative. QuantTrio is the upstream packed base, not the model identity.

No full base checkpoint is included.

Install and verify

Agent-directed path

An infrastructure agent can start from this page. It must read AGENT_INSTALL.md, then the linked GitHub AGENTS.md. The packaged agent-bootstrap.sh clones the pinned installation repository and enters its guarded prepare/launch workflow.

bash agent-bootstrap.sh --prepare

The script deliberately stops for real cluster discovery when no cluster configuration exists; it never copies private topology values from the reference deployment.

The complete scripts, runtime recipe, benchmarks, and independent verifier are at:

https://github.com/0xTank/SuperGLM-5.2-v2-W4W8-200K-4x-DGX-Spark

git clone https://github.com/0xTank/SuperGLM-5.2-v2-W4W8-200K-4x-DGX-Spark
cd SuperGLM-5.2-v2-W4W8-200K-4x-DGX-Spark
python3 -m pip install 'huggingface_hub[cli]' torch safetensors
./scripts/install-hf-overlay.sh

The installer downloads the pinned base directly from QuantTrio, creates a hard-linked local output tree, applies this repository’s overlays and index, and runs the independent verifier. The base and output directories must share a filesystem.

Users can instead reconstruct the overlays directly from the pinned SuperGLM v2 source with ./scripts/build-model.sh.

Verified serving profile

Setting	Value
Hardware	4× DGX Spark / GB10
Parallelism	TP4, DCP1
Native MTP	K5 probabilistic
Maximum context	200,000
KV cache	nvfp4_ds_mla, 7,000,000,000 bytes/node
max_num_seqs	4
max_num_batched_tokens	8,192
CUDA graphs	[6,12,18,24]
Prefix cache	enabled
Thinking	disabled by default

Natural-language results

Warm C1, thinking off, temperature 0.7, top-p 1.0, top-k 40, 256-token cap.

Workload	Median decode	Median TTFT	MTP acceptance
Explanation	27.383 tok/s	—	—
Diagnosis	32.401 tok/s	—	—
Coding	31.965 tok/s	—	—
Planning	25.714 tok/s	—	—
Mixed-chat median	29.674 tok/s	0.470 s	59.962%
Prose	25.882 tok/s	0.494 s	48.919%
Essay	24.726 tok/s	0.499 s	46.494%
Reading	28.612 tok/s	0.590 s	58.361%
Math	32.640 tok/s	0.573 s	71.071%

Cold server-side prefill measured 663.18 tok/s at 5K and 622.43 tok/s at 50K. Identical warm repeats reached 97.73% and 99.812% prefix-cache hits; cached values are not presented as cold prefill throughput.

Checkpoint verification

The archived measured build passed with:

• 62 W8 overlay files
• 372 redirected index entries
• 2,343 native 256-expert MTP tensors
• maximum W8 round-trip relative L2 0.008788
• pinned chat-template SHA-256 172dc74a35e1752df75ecfb2b2cf9326d2852bb1379868ebeec9571654489679

The package includes provenance and source-evidence hashes. Run the verifier after assembly; do not serve an unverified tree.

Behavior measurement

An internal 32-prompt semantic suite recorded 32 substantive assists, zero refusals, zero garbled outputs, and zero errors through both the direct API and a minimal Hermes profile with thinking off. Raw high-risk prompts and generated harmful text are intentionally not distributed. Results are measurements, not warranties.

Security and use

The reference vLLM server has no application authentication. Keep it on a trusted private network and add authentication, TLS, rate limits, ACLs, and tool sandboxing before sharing it. Generated text is untrusted data.

This is an experimental research and systems-engineering release. Operators are responsible for lawful use, access control, outputs, and compliance with upstream licenses. Read TERMS_OF_USE.md, RESPONSIBLE_USE.md, and LEGAL_NOTICE.md.

Credits

Built from Z.ai GLM-5.2, Jiunsong SuperGLM v2, and QuantTrio’s packed base, with runtime work based on Tony D’Angelo’s DGX Spark recipe, eugr’s build harness, vLLM, and the sparse-MLA contributors credited in the GitHub repository. This project is unofficial and is not endorsed by those parties.