aday777/glm5_next_tiny_fixture

GLM-5.3-Flash tiny architecture fixture (glm5_next)

A ~0.27M-parameter random-init TEXT checkpoint plus a reduced, nested config that reproduces the GLM-5.3-Flash glm5_next schema, so loaders, quant planners, and CI jobs can exercise the new multimodal-MoE config parsing, expert-table sizing, and safetensors load path without the real weights.

Support this work — if this saved you time, donate BTC: bc1q5ayht3fxhj0v95fk0z8l2f6900g3awdsw5842p

What this is

• Base architecture: zai-org/GLM-5.3-Flash (released 2026-08-25), model_type: glm5_next, architectures: ["Glm5NextForConditionalGeneration"]. The base is a large multimodal MoE that cannot be instantiated in a unit test, in CI, or on a laptop.
• What this fixture is: a byte-reproducible random-init text checkpoint plus a reduced config that keeps the real top-level wrapper fields (model_type, architectures, image_token_id, language_model_only, text_config, vision_config) and the GLM MoE field names inside text_config, including the dense-to-MoE layer schedule (first_k_dense_replace, routed + shared experts).
• What it is not: not trained, not distilled, not a quality or benchmark claim, and not a quantization of anything.
• Why it is useful: it lets you test config parsing, weight-name mapping, expert-table sizing, router/top-k bookkeeping, and safetensors load paths in milliseconds, and it exercises the glm5_next wrapper shape that the plain glm_moe_dsa text fixture does not.

Fixture geometry (what this checkpoint actually contains)

Field	Value
num_hidden_layers	4
hidden_size	64
num_attention_heads / num_key_value_heads	4 / 4
head_dim	16
n_routed_experts	8
num_experts_per_tok	2
n_shared_experts	1
n_group	1
first_k_dense_replace	1
moe_intermediate_size	32
intermediate_size (dense)	128
num_nextn_predict_layers	0
dtype	float32
vocab_size	256

Total: 113 tensors, 1,097,984 data bytes = 274,496 float32 parameters.

Intentional omissions (documented, not silent)

• Text-only. No vision encoder or projector tensors; vision_config is a placeholder. A full multimodal loader must supply vision/projector tensors and remap names.
• No MTP / next-n-predictor head (num_nextn_predict_layers: 0).
• No lm_head tensor; a loader must tie to model.embed_tokens.weight or supply its own head.
• Tokenizer metadata files are placeholders (no vocab file); use your own tokenizer.
• Weight names are a reduced text-only convention; they are not guaranteed to match every tensor a full Glm5NextForConditionalGeneration loader expects.

Verification actually performed (stdlib only, no torch in this environment)

• safetensors header parses: 113 tensors, contiguous data_offsets, header padded to 8-byte alignment; __metadata__ records the generator string.
• checksums.txt records the SHA-256 of every tensor blob.
• Deterministic regeneration: SplitMix64 seed 20260902, Box-Muller normals, scale 0.02, float32 row-major, consumed in sorted-name order.
• Generator script is included as build_fixture.py so the folder can be rebuilt and diffed.

Not yet verified: loading under a specific transformers version (no torch/transformers in the build environment), and whether Glm5NextForConditionalGeneration accepts this reduced text-only geometry without extra fields. Treat those as open until run against a real install.

How to use

Read the tensors with the standard library (no torch needed, matching how this was built):

import json, struct
with open("model.safetensors", "rb") as f:
    n = struct.unpack("<Q", f.read(8))[0]
    header = json.loads(f.read(n))
    # header[name] = {"dtype", "shape", "data_offsets"}; data starts at byte 8+n

Or with the safetensors package:

from safetensors.torch import load_file
tensors = load_file("model.safetensors")   # {name: tensor}

To exercise a real loader, build a config from config.json (the glm5_next model type; use AutoConfig.from_pretrained(..., trust_remote_code=True) where needed) and feed these weights in. There is no lm_head tensor, no vision stack, and the tokenizer files are placeholders, so supply your own head/tokenizer/vision.

License

The generated fixture content (random weights, config, scripts) is released under MIT (see LICENSE). The glm5_next architecture and config schema belong to the base model zai-org/GLM-5.3-Flash under its own terms, which were not independently re-verified this cycle — check the base repository before redistribution.

Citation

Zhipu AI / Z.ai, GLM-5.3-Flash, 2026.

Support this work

If this saved you time or money, consider a donation: Bitcoin: bc1q5ayht3fxhj0v95fk0z8l2f6900g3awdsw5842p