GLM-5.2-Legume ("Legume The First") — a SIQ frankenmodel

SIQ = SparkInfer Quantization (yes, it's sick): rank-sliced Trellis/EXL3 expert payloads with runtime-dynamic K3/K4 tier mixes, served by the vLLM+SparkInfer ("gilded-gnosis") stack. This model exists to torture-test exactly that machinery.

In Québécois French, when someone takes a bad enough hit, the cliché says they'll think no more than a légume — a vegetable. This model took the hit on purpose: 16 of 79 layers, 64 of 256 experts per layer. It is a vegetable. It has one talent left: babbling open-source licenses.

This is a torture-test artifact, not a language model you should use. Outputs are not meaningful outside (approximate) license recitation — and mostly not even there. It exists to exercise serving-stack machinery on a single consumer GPU.

What it is

A depth-pruned + expert-pruned ("REAP"-style) SIQ frankenmodel of willfalco/GLM-5.2-EXL3-TR3-3.36bpw (a rank-sliced Trellis/EXL3 "SIQ" quant of zai-org/GLM-5.2):

• Layers: old layers 0–11 and 74–77 renumbered 0–15 (num_hidden_layers=16, MoE layers 3–15, MTP layer dropped).
• Experts: 64 of 256 per MoE layer, selected by tournament (below).
• Quantization: parent's Trellis TR3 payloads dequantized from TP4, concatenated, re-encoded at TP1 through the reference encoder's uncalibrated branch, each expert at its parent tier (mixed K3/K4).
• Size: ~23.6 GiB — fits a 32 GiB RTX 5090 with the full GLM-5.2 serving tuple (DSA sparse indexer, fp8_ds_mla KV, CUDA graphs).

Why it exists

The maintainers of the vLLM+SparkInfer ("gilded-gnosis") stack serve the real GLM-5.2 SIQ in production on 4×RTX PRO 6000. Stack development needs a model with the same architecture, loader path, and attention tuple that boots on one consumer card, so engine changes can be torture-tested without touching production. Legume covers, unpatched, on one sm_120 GPU:

• glm_moe_dsa architecture with the DSA sparse indexer active
• fp8_ds_mla KV cache (production's real KV dtype) with CUDA graphs
• the rank-sliced SIQ Trellis loader with runtime-dynamic tier counts (arbitrary K3/K4 mixes per layer — the SparkInfer #117 case)
• MoE + MLA + sparse-attention interaction under real routing

The tournament ("license-babbling master" selection)

v1 of Legume picked experts by lowest quantization round-trip MSE. This public build picked them by what the router actually uses on a narrow domain — a corpus of open-source license texts (recited by the full production GLM-5.2; Apache-2.0 held out):

1. The 256 experts of each MoE layer were split into 4 disjoint 64-expert cohorts (cohort A = the v1 selection; B/C/D dealt round-robin by MSE).
2. Each cohort was assembled into a bootable 16L/64E model and the license corpus (18 licenses, 13,872 tokens, recited by the full production GLM-5.2) was run through it with enable_return_routed_experts — real engine routing (sigmoid noaux_tc
   • e_score_correction_bias, actual top-8), captured per token per layer.
3. Per layer, the 64 experts with the highest routed counts across all four rotations won their seats. Per-layer cohort win counts are in tournament/selection.json. Headline: seats split almost evenly across cohorts (14–20 each per layer) — v1's lowest-quantization-MSE experts kept only ~26% of their seats. Quantization fidelity and domain routing mass are nearly orthogonal.

Selection, not training — no gradient ever flowed. The router bias and gate rows were subset to the winners; every expert payload is byte-identical to its (re-encoded) parent tier.

Held-out eval (canonical Apache-2.0 text, never in the corpus)

build	expert rule	mean logprob	ppl
v1 (private)	lowest quant MSE	−13.4155	670,329
v2 (this repo)	license tournament	−12.4721	260,962

+0.94 nats/token, a 2.57× perplexity reduction — from expert selection alone, zero training. Both are still far beyond broken (uniform-random over this vocab would score ≈ −11.9; a 16-of-79-layer model has no continuity of thought), but the tournament measurably pulled the wreckage toward the license domain.

First words at temperature 0, same eval config, from "Licensed under the Apache License, Version 2.0 (the":

• v1: agyipiilogilogilogilog… loops (and, in its classic serving config, the immortal losacylosloslosloslosloslos)
• v2: emits <arg_value> or simply stops — statistically closer to the target text, expressed as reverent silence

Measured serving numbers (RTX 5090, gilded-gnosis r25)

• Boots unpatched; DSA indexer active; dynamic tiers accepted on every layer
• fp8_ds_mla KV + CUDA graphs: full engine init, clean 1/2/5/8/9-token small-prompt battery
• Decode 116.6 tok/s single-stream (measured on v1 in the graphs-on serving config; v2 is byte-identical in geometry and kernel paths)
• Known edge (documented, not fixed): the offline in-process engine's long-prefill MHA fallback (forward_mha) is unimplemented for this tuple — prefills ≤512 tokens per request in offline mode; the API server path is unaffected in production configs

Reproduction

Everything under tools/: glm_franken.py (plan/dequant-gate/re-encode/ assemble), legume_tournament.py (select/seal/merge), legume_routed_capture.py, legume_apache_eval.py, gen_license_corpus.py, legume_v2_campaign.sh (the full orchestration), plus the byte-pinned base encoder. The sealed plan (fingerprinted) ships in the checkpoint as franken_plan.json; per-expert provenance in tier_bitmap.json (franken_src_experts).

Credits

• zai-org — GLM-5.2, the parent of parents (MIT)
• willfalco — the GLM-5.2 EXL3-TR3 3.36bpw SIQ quant this was carved from
• turboderp — exllamav3 / EXL3 Trellis quantization
• brandonmusic — the TR3 calibration encoder toolchain
• b12x / SparkInfer — the gilded-gnosis vLLM serving stack
• malaiwah — the lobotomy, the tournament, and the name

Disclaimer

No benchmark was harmed in the making of this model, because none could be run. Do not deploy. Do not fine-tune. Do not ask it for legal advice — * especially* not about licenses.