mammoth-1 — MLX 8-bit
An 8-bit MLX quantization of baselabs/mammoth-1 (revision 61cec44c281c693992e3d6ebeb44dd3402dfa186): baselabs' 27B typed decision model (a full fine-tune of Qwen3.5-27B).
Give it a state and typed questions (choice, yes/no, score) and it returns a probability for every option without
generating text. Built and served by Midium's System One runtime (POST /v1/systemone).
What was quantized
- Language model: affine 8-bit, group size 64, round-to-nearest (no calibration data).
- Kept as in the source: nothing else: the vision tower is not part of this text-only build.
- Tokenizer, chat template and the decision configuration are unchanged.
Weights: 28.6 GB (bf16 source: 54.7 GB).
Evaluation against the bf16 model
Same runtime, same requests, question by question, on a seeded sample of up to 150 questions per set.
| Suite | Questions | Top answer agrees with bf16 | Accuracy change per set | Answers lost / gained |
|---|---|---|---|---|
| 14 decision sets (routing, reply checks, tool preselection, model routing) | 2,058 | 100.0–100% | +0.0 to +0.0 pts | 0 / 0 |
| Public sets, 150-way intent choice, 0–9 scores, option-shuffle and typo robustness | 1,720 | 97.5–100% | +0.0 to +1.3 pts | 1 / 4 |
Mean total-variation distance between the 8-bit and bf16 answer distributions: at most 0.008 per set.
Usage
This checkpoint keeps the source's decision protocol — OneJev's qev protocol: the answer read from the LM-head logits of the slot labels (A.., 256 slots), with block-bidirectional full attention (modeling_overrides in config.json, readout.json). It needs a runtime that implements that
protocol: Midium's System One runtime loads it directly; generic MLX text-generation loaders do not.
License
MIT, as the source model.
- Downloads last month
- 9
8-bit