midium-ai/d3-turbo-8bit

d3-turbo — MLX 8-bit

An 8-bit MLX quantization of vllm-sr/d3-turbo (revision a0f3e9e59ebd1202fe1d4de1c4c0f9a986758cbb): vLLM Semantic Router's 27B multimodal decision model (Qwen3.5-27B + vision, d3-turbo v1.1). Give it a state and typed questions (choice, yes/no, score) and it returns a probability for every option without generating text. Built and served by Midium's System One runtime (POST /v1/systemone).

What was quantized

• Language model: affine 8-bit, group size 64, round-to-nearest (no calibration data).
• Kept as in the source: the vision tower and the 255-way answer-code readout (readout.safetensors).
• Tokenizer, chat template and the decision configuration are unchanged.

Weights: 28.2 GB (bf16 source: 52.2 GB).

Evaluation against the bf16 model

Same runtime, same requests, question by question, on a seeded sample of up to 150 questions per set.

Suite	Questions	Top answer agrees with bf16	Accuracy change per set	Answers lost / gained
14 decision sets (routing, reply checks, tool preselection, model routing)	2,058	99.3–100%	-0.7 to +0.7 pts	2 / 1
Public sets, 150-way intent choice, 0–9 scores, option-shuffle and typo robustness, synthetic images	1,860	98.3–100%	-1.7 to +1.3 pts	2 / 2

All 140 synthetic image questions (colour, shape, count, position, printed word, tallest bar, presence) give the same answer as bf16.

Mean total-variation distance between the 8-bit and bf16 answer distributions: at most 0.007 per set.

Usage

This checkpoint keeps the source's decision protocol — the d3 code-readout protocol: the 255-code readout at the last token (Box: at each edge marker), with the state block attending to itself both ways and the question block to the state and itself (prefix_bidir). It needs a runtime that implements that protocol: Midium's System One runtime loads it directly; generic MLX text-generation loaders do not.

License

Apache-2.0, as the source model.