negativevoid/Scram-0.8B-6bit

Scram-0.8B (6-bit): a 603 MB tool-calling model that knows when not to call

(internal designation: corium-v5.4; the training recipe and ledger use that name)

Qwen3.5-0.8B-Base, distilled from verified Qwen3.5-4B/9B reasoning traces, trained and evaluated end-to-end on a single MacBook (M4, 64 GB, no CUDA).

Training data is public: negativevoid/scram-curriculum (17,837 verified traces, the complete SFT set).

What it's for: multi-step procedural reasoning that ends in a well-formed tool call, with refusal discipline: declining out-of-scope requests and asking for missing parameters instead of fabricating them.

What it is not: a chat assistant, and not a knowledge base. World knowledge is deliberately not in the weights (closed-book ≈ 0.20; ≈ 0.92 with retrieved notes). Single-turn. English only.

Numbers

Every number below is co-reported with the ones that look bad. That is the point.

Official BFCL harness (their generator + evaluator, v4 data, prompting mode)

category	run-1	run-2
irrelevance (n=240)	96.67	94.17
live_irrelevance (n=884)	92.42	94.46
live_relevance (n=16)	18.75	31.25
simple AST (n=400)	37.75	45.75
multiple / parallel / par-mult	44.5 / 25.0 / 23.0	39.0 / 28.0 / 24.0

Irrelevance ~94–97 is high for the size class (board fetched 2026-07-20: Gemini-2.5-Flash 93.67, ToolACE-2-8B 90.79, Claude-Sonnet-4.5 86.61, Qwen3-8B 79.07 note the board moves, re-verify before quoting). AST is the weak axis: 46 vs Hammer2.1-0.5b's 66. Diagnosed cause is delegate-vs-solve arbitration: the model answers computable questions directly instead of calling a tool, because the training curriculum never demonstrated "solvable, but call anyway." Outright wrong-function selection is rare (29/2311 scored items in run-1).

Reason-to-act gauntlet (n=300 + 60 traps, identical scoring for all models)

model	derive→valid-call	missing-param	out-of-scope
corium-v5.4 (603 MB)	0.687	30/30	30/30
Qwen3.5-4B (its own teacher)	0.510 (McNemar p=7e-6)	1/30	11/30
Qwen3.5-0.8B-instruct	0.410	8/30	7/30
Qwen3.5-2B-instruct	0.200	11/30	18/30
Hammer2.1-0.5b	0.047	1/30	0/30

Traps: 60/60 in this run; 120/120 cumulative across two independent runs.

On beating its own teacher: students outperforming teachers after CoT distillation is a known result (Ho et al., ACL 2023), not a discovery here. Our contribution is a mechanism hypothesis: 100% of the teacher's misses contain the correct derived value in prose, so the differential looks like derive→valid-call emission reliability rather than reasoning quality. This is prose-inspection evidence, not a causal test. The constrained-decoding ablation that would settle it has not been run.

Limitations, confounds, and things that failed

• GSM8K was trained on (deliberate scope change). Any GSM8K number here measures trained capability, not transfer. It is not a held-out benchmark for this model.
• Our reasoning battery is our own instrument. Effect sizes are not comparable to anything outside this project.
• A null result: adding an explicit "Extraction:" parse step to the traces (v6) produced 100% format adoption and zero capability gain (0.607 vs 0.610 fresh GSM8K), plus a 4pp reasoning-breadth cost. Making a latent step explicit did not create the skill. Underpowered (n=300, ±5pp CI), so treat as suggestive.
• When2Call MC: 0.343 under a generative letter-choice protocol (the official one is loglikelihood, so this is not comparable to their paper). The model refuses well procedurally but does poorly at selecting the description of the right action. Note the likely confound: Ross et al. §5 already report that Qwen-family models avoid the "unable to answer" option, so this may be inherited family behaviour rather than anything about our curriculum.
• Single-seed arms. The teacher-quality comparison at 0.8B (9B-taught 0.917 vs 4B-taught 0.870) is one seed. At 0.6B, three seeds pooled to −1.5pp (p=0.185, null).
• Refusal classes do not cross-transfer. Wrong-tool, missing-param, and out-of-scope had to each be taught explicitly; adding one fixed only that one.
• The tool schemas it learned on are synthetic. It has never seen a real-world function inventory. That is the honest explanation for the AST gap, and it predicts where it will be weakest on your tools: unfamiliar naming conventions and large menus of similar functions. Expect to add domain schemas if you fine-tune.

Use

from mlx_lm import load, generate
model, tokenizer = load("negativevoid/Scram-0.8B-6bit")
print(generate(model, tokenizer, prompt="...", max_tokens=256))

Note: the model was trained with enable_thinking=False templating; matching that at inference matters. Give it generous token budgets, because it was trained to deliberate before emitting, and tight caps (e.g. 32) truncate it mid-answer and read as failure.

Reproduction

Every experiment is a seed-pinned YAML; nulls and failed predictions are logged as prominently as successes in DECISIONS.md.

License and attribution

Apache-2.0. Derivative of Qwen/Qwen3.5-0.8B-Base (Apache-2.0, © Alibaba/Qwen), modified by supervised fine-tuning on synthetic distilled traces. Quantized to 6 bits with MLX.