flint-weight-edited

A training-free, weight-edited derivative of RecursiveSI/flint. No gradient updates and no new training data were used. The weights were edited directly: find a behaviour direction or component with teacher-forced reads, edit the matrices that write it, then check with dose-response, random controls and held-out generation. The edits target two problems users reported in the Flint discussions:

1. Tool-schema drift. The model writes the argument and tool names it learned in pre-training instead of the names in the supplied schema. Examples: path instead of Claude Code's file_path or OpenCode's filePath, and bash instead of Bash (discussions #9, #11, #12).
2. Overthinking and runaway reasoning. <think> blocks that keep going after the answer is known, and that hit the token budget on math and code (discussions #13, #14, #18, #26).

Current version: v3 (schema_x2.5_neg0+close_b1.0+rowp9.2, 42 edits).

Across 1,774 paired held-out generations, v3 scores 0.780 against 0.722 for the original (p < 10⁻⁶). It uses 19% fewer tokens. GSM8K goes from 62.7% to 79.7%.

Architecture, tokenizer, chat template, config and the sglang kit are identical to the original. Only the values of 2,326 bf16 tensors in shards 9, 10, 11 and 15 changed:

• 22 attention-output, shared-expert or lm_head tensors;
• the down_proj of all 256 routed experts in each of layers 37–45.

Shards 1–8 and 12–14 are byte-identical to the original, and the safetensors index is unchanged.

What was edited

Edit	Tensors	Operation
Schema-copy heads ×2.5	self_attn.o_proj input columns of 11 heads: L45H28, L46H31, L37H8, L37H29, L33H10, L41H28, L41H14, L37H11, L43H6, L46H23, L30H15	columns × 2.5
Prior heads ablated	self_attn.o_proj columns of L44H4, L33H21, L45H26	columns × 0
Thinking-closure amplification	layers 37–45: self_attn.o_proj, mlp.shared_expert.down_proj, and every routed expert's down_proj (256 × 9)	W ← W + 1.0 · u_l u_lᵀ W
Answer-mode </think> suppression	lm_head.weight row 150007 (</think>)	row ← row + 9.2 · r

Closure direction. u_l is the unit "ready to close </think>" direction at layer l. It is the mean residual difference between positions where the model's own next token is </think> and other sentence-boundary positions in the same reasoning trace. On held-out traces it gives d′ ≈ 3.5 and 100% leave-one-out sign consistency in layers 24–45. The writers in layers 37–45 put out +u mainly when reasoning is about to end. Doubling their u-component (β = 1) makes the model close once it has reached a stopping state, without a constant push. The routed-expert part carries most of the effect: dense-only closure at the same β gave about half the GSM8K gain (v2, below).

lm_head row edit. At closure strength 1.0 the model sometimes emitted a second </think> inside its answer. The fix is one rank-1 change to the </think> output row: r = w/|w|, with |w| = 9.2.

w is a ridge fit on the edited model's final normed residual, using practice data only:

• target −8 logit at answer positions (after </think>);
• target 0 at the model's own closing position;
• constrained orthogonal to the mean thinking state, so the logit is not shifted while the model is still reasoning.

The exact spec is in edit/edit_spec.json and the directions are in edit/directions.pt.

The schema heads were found by direct logit attribution of each of the 49 × 32 attention heads to the margin logit(schema name) − logit(prior name). It was measured at 80 decision points across Claude Code, OpenCode, Codex and pi/omp tool schemas.

Results

Held-out generation, 887 items × 2 decodes = 1,774 paired items

Each item was run once greedily and once sampled (T = 1.0, top-p 0.95, top-k 20, seed 11), with at most 1,024 new tokens. The original and the edited model ran in the same harness on 2× T4. Routed experts were int4 (g128) and dense tensors bf16, so the edited experts were evaluated as their int4 quantization (edit/bake_int4.py). No item was used to select edits.

The table reports accuracy, then items only the original got right / items only v3 got right, then an exact McNemar p-value, then the share of outputs that hit the token cap.

Kind	n	Original	v3	orig-only / v3-only	p	Hit cap (orig → v3)
GSM8K (held-out rows)	400	0.627	0.797	10 / 78	<10⁻⁶	0.36 → 0.15
HumanEval (held-out)	268	0.541	0.619	26 / 47	0.019	0.44 → 0.33
Agent-harness tool calls (Claude Code, Codex, pi, omp, OpenCode)	350	0.551	0.591	32 / 46	0.14	—
↳ schema-valid call	350	0.79	0.82
Agent-harness answer after a tool result	100	0.760	0.750	12 / 11	1.0	0.00 → 0.04
Custom, user-invented tool schemas	96	0.875	0.833	8 / 4	0.39	—
Generic tool selection	240	0.904	0.912	4 / 6	0.75	—
No-tool questions	120	0.983	0.975	1 / 0	1.0	—
Easy chat	200	0.980	1.000	0 / 4	0.13	—
All	1,774	0.722	0.780	93 / 196	<10⁻⁶

Total generated tokens fell from 595.8k to 483.1k (−18.9%). Mean thinking length fell on every kind:

Kind	Original	v3
Easy chat	685 chars	134 chars
GSM8K	2,372 chars	1,643 chars
HumanEval	2,559 chars	2,107 chars

The two decodes agree. Greedy goes from 0.729 to 0.777 and sampled from 0.714 to 0.782.

Honest reading:

• Overall. v3 is clearly better than the original on reasoning-heavy items. Most of the gain is GSM8K and HumanEval answers that previously ran out of budget while still thinking. Tool calling is equal or slightly better.

• Custom tool schemas. These lose 4 points (8 vs 4 items, not significant). Of the 5 greedy losses we inspected, 4 are changes in style rather than broken schemas:

  • two parallel calls instead of one;
  • an exploratory first call (SHOW TABLES) before the requested one.

  The fifth is a stray </think> inside a URL argument.

• Second </think>. 89 of the 1,774 outputs (5%) still contain a second </think> inside the answer, against 2 for the original. Most are on answer-after-tool-result, easy and GSM8K items. On those items v3 is correct as often as the original. With a reasoning parser that splits at the first </think>, the stray tag ends up in the visible content. A refit that targets these states is in progress.

• Answer after a tool result. A few of these answers now run long (4% hit the cap, against 0%).

Teacher-forced probes

These were run on CPU, with v3 in its evaluated int4 form, on the original model's own held-out reasoning traces and on reference solutions.

Metric	Original	v1	v3
log P(</think>) where the model actually stops (GSM8K traces)	−0.136	−0.009	−0.002
P(</think>) at sentence ends in the last 40% of reasoning (GSM8K)	0.007	0.047	0.118
Max P(second </think>) inside answers	0	0	0.0001
NLL of GSM8K reference solutions	1.675	1.700	1.718 (+2.6%)
NLL of HumanEval canonical solutions	0.382	0.392	0.406 (+6.3%)

Schema-preference probes cover 190 sequences: Claude Code, OpenCode, Codex, pi and computer-use style schemas, plus 48 invented custom tools. The margins are logit(schema name) − logit(prior name) on held-out decision points.

Probe	Original	v1	v3
Decisions where the schema name wins (all targets)	34.7%	76.4%	73.6%
Invented custom tools: schema name wins	—	100%	100% (margin +15.3)
Claude Code Read → file_path vs path	−10.8	−2.2	−2.2
Claude Code Write	−3.9	+1.4	+0.3
OpenCode read → filePath	−2.1	+3.5	+3.5
OpenCode write	−1.1	+5.1	+5.8
Claude Code Bash vs bash	+0.7	+3.3	+3.1
Control: pi read → path (schema agrees with prior)	+19.8	+20.3	+19.8
Control: Codex shell vs bash	+12.9	+12.9	+12.4

Claude Code Read (file_path) remains the hardest case: the margin is still negative on most decision points.

Version history (same 1,774-item evaluation)

Version	Edits	All	GSM8K	HumanEval	Tokens
Original	—	0.722	0.627	0.541	595.8k
v2	heads + dense-only closure β = 1 + lm_head row	0.741	0.688	0.567	539.6k
v3	heads + full closure β = 1 (incl. routed experts) + orthogonal lm_head row	0.780	0.797	0.619	483.1k

v1 (heads + closure β = 0.5) was published first. Its 1,774-item run was later found to have the routed-expert part missing, because of an evaluation-harness bug that has since been fixed. Without that part it scored 0.721.

Method and compute

1. Probing (practice split). Directions and heads come from teacher-forced reads on the model's own traces.
2. Dose sweeps. Doses were swept with random-head and random-direction controls.
3. Selection. Versions were selected by generation on a separate 887-item held-out set. That set covers GSM8K and HumanEval, harness tool calls with multi-turn error recovery, invented custom tool schemas, generic tools, no-tool and easy prompts.

Generation ran on Kaggle T4s and the probes ran on a 4-core CPU box with an int4 working copy. The final edits were applied to the original bf16 tensors. The uploaded shards were verified byte-for-byte against the spec (edit/verify_upload.py). The ledger of logged evaluation passes is in edit/passes.jsonl.

License and credit

Apache-2.0, as for the original. All credit for the model goes to RecursiveSI. This repository only changes weight values as described above. See LICENSE and sglang/NOTICE. Serve it exactly like the original; see sglang/README.md.