aday777/qwen3_5_tiny_fixture

Qwen3.8-27B tiny architecture fixture (qwen3_5)

A ~0.17M-parameter random-init TEXT checkpoint plus a reduced config that keeps the real Qwen3.8-27B qwen3_5 field names, so loaders, quant planners, and CI jobs can exercise the hybrid linear+full-attention schema and the safetensors load path without the real 27B weights.

What this is

• Base model: Qwen/Qwen3.8-27B (released 2026-08-13), model_type: qwen3_5, architectures: ["Qwen3_5ForConditionalGeneration"], reported Apache 2.0. The base is a large multimodal dense model with a hybrid linear+full attention stack and an MTP/next-n head, so it cannot be instantiated in a unit test, in CI, or on a laptop.
• What this fixture is: a byte-reproducible random-init text checkpoint plus a reduced config that preserves the real top-level wrapper fields (model_type, architectures, image_token_id, language_model_only, text_config, vision_config) and the real qwen3_5 field names inside text_config — including layer_types (the linear/full attention schedule), linear_*, attn_output_gate, full_attention_interval, and num_nextn_predict_layers.
• How it is changed from the base: same schema, tiny geometry, float32, and a reduced standard-attention tensor set. It is a schema/load fixture, not a quantization and not a distilled model.
• What it is not: not trained, not distilled, not a quality or benchmark claim.
• Why it is useful: it lets you test config parsing, weight-name mapping, and safetensors load paths in milliseconds, and it exercises the qwen3_5 multimodal wrapper shape that a plain text-only fixture does not.

Fixture geometry (what this checkpoint actually contains)

Field	Value
num_hidden_layers	4
hidden_size	64
num_attention_heads / num_key_value_heads	4 / 2
head_dim	16
layer_types	linear, linear, linear, full
full_attention_interval	4
intermediate_size	128
num_nextn_predict_layers	1
dtype	float32
vocab_size	256

Total: 40 tensors, 690,944 data bytes = 172,736 float32 parameters.

MTP / next-n head (documented value-add)

This fixture includes a random-init mtp.pre_mtp_fc_norm.weight and mtp.fc.weight pair and sets num_nextn_predict_layers: 1, so a loader can exercise the MTP/next-n path. The weights are random-init (not trained) and the base repository's shipped MTP head presence was not primary-source confirmed this cycle — treat the head as a schema placeholder, not a trained drafter.

Intentional omissions (documented, not silent)

• Text-only. No vision encoder or projector tensors; vision_config is a placeholder. A full multimodal loader must supply vision/projector tensors.
• The tensor set is a reduced standard-attention convention. The real model's linear-attention (conv/ssm) tensors are NOT included.
• No lm_head tensor; a loader must tie to model.embed_tokens.weight or supply its own head.
• Tokenizer metadata files are placeholders (no vocab file); use your own tokenizer.

Verification actually performed (stdlib only, no torch in this environment)

• Generator build_qwen3_5_fixture.py executed via the standard library and printed 40 tensors with a SHA-256 per blob; checksums.txt records those hashes.
• config.json written with model_type: qwen3_5 and the real field names above.
• Deterministic regeneration: SplitMix64 seed 20260904, Box-Muller normals, scale 0.02, float32 row-major, consumed in sorted-name order.

Verified this cycle (stdlib header re-parse, validate_fixtures.py): the safetensors header re-parses cleanly — 40 tensors, contiguous data_offsets starting at 0, final offset equals file size minus header, and the tensor count matches checksums.txt (40 lines).

Not yet verified (open): loading under a specific transformers version, whether Qwen3_5ForConditionalGeneration accepts this reduced text-only geometry without the linear-attention tensors, and the base LICENSE file terms (reported Apache 2.0 via search, not independently re-read). Treat these as open until run against a real install.

How to use

Read the tensors with the standard library (no torch needed, matching how this was built):

import json, struct
with open("model.safetensors", "rb") as f:
    n = struct.unpack("<Q", f.read(8))[0]
    header = json.loads(f.read(n))
    # header[name] = {"dtype", "shape", "data_offsets"}; data starts at byte 8+n

Or with the safetensors package:

from safetensors.torch import load_file
tensors = load_file("model.safetensors")   # {name: tensor}

To exercise a real loader, build a config from config.json (the qwen3_5 model type; use AutoConfig.from_pretrained(..., trust_remote_code=True) where needed) and feed these weights in. There is no lm_head tensor, no vision stack, and the tokenizer files are placeholders, so supply your own head/tokenizer/vision.

License

The qwen3_5 architecture and config schema belong to the base model Qwen/Qwen3.8-27B, reported Apache 2.0. The base LICENSE file was not independently re-read this cycle — check the base repository before redistribution.

Citation

Qwen Team, Qwen3.8-27B, 2026.

---

Buy me a beer: BTC bc1q5ayht3fxhj0v95fk0z8l2f6900g3awdsw5842p