OpenGrad — Qwen3.5-2B, joint xLAM + CALL_PREDICTION removal (fixed-compute)

Full-parameter SFT of Qwen/Qwen3.5-2B on Canonical-v2 with xLAM removed, training for the reference's 2,400-step budget. Produced by OpenGrad.

Research artifact, not a production model. Published because the experiment's whole point is a negative result that is only checkable if the weights behind it are available.

What this experiment actually removed

Canonical-v2 maps xLAM to every CALL_PREDICTION record and the other three sources to COMPLETE_TRAJECTORY. Removing xLAM therefore removed a source and the corpus's entire call-prediction supervision channel at the same time:

this model measures the effect of removing xLAM together with the CALL_PREDICTION channel.

It is not a pure xLAM-content ablation, and no xLAM-specific causal claim can be based on it. Source identity and supervision type are perfectly aligned in this corpus. Separating them would need another evidence-backed CALL_PREDICTION source.

Results (confirmatory partition, 1,277 examples, scored once)

run	`call_f1`	precision	recall	over_call	clarify	unsupp
B0 (untrained)	0.6264	0.4618	0.9735	0.6238	0.1186	0.0177
reference (full corpus) @1800	0.7470	0.7350	0.7594	0.1505	0.7682	0.5430
this arm (fixed-compute) @1200	0.6030	0.7893	0.4879	0.0716	0.8059	0.6026
matched-exposure arm @1060	0.5557	0.8067	0.4238	0.0558	0.8194	0.6203

Removing the joint source/channel drops recall far below both the reference and B0 while precision rises and over-calling falls to a small fraction. The models become conservative but stop recalling the calls they should make — the direction the missing supervision predicts.

The arm that trains more (this one: 2,400 steps, ~1.53× the reference's passes over its corpus) does better than the matched-exposure arm. That is consistent with the recall loss tracking the missing supervision, but it does not rule out exposure. Batches are token-budgeted, so steps do not hold supervised tokens fixed: the matched-exposure arm logged 6,743,788 supervised tokens against the reference's 5,678,531 (1.19×, not ~1.0×), and this arm 7,683,301. The metrics above come from checkpoints that saw fewer supervised tokens than the reference's selected checkpoint: 3.80M here at 1200 and 3.36M for the matched arm at 1060, against 4.27M for the reference at 1800. "Exposure does not explain the result" is therefore not supported as stated. The two arms are one experiment and must be read together.

Honest limits

Not a promotion. Every checkpoint of both arms is REJECTED on regression.call_recall against B0 (every matched-arm DEV checkpoint and fixed-arm 1800/2400 also fail call_f1_retention). Every checkpoint of the reference run is rejected too, its checkpoint 600 on over-call alone. The gate was left exactly as written.

Single seed. A small between-arm difference is a finding to replicate, not a settled result.

Provenance. The matched-exposure arm was launched at 12:49 UTC on 2026-09-11 from commit 500cb4e with uncommitted changes (git_dirty: true in experiment.json and events.jsonl). 0807fa3 is the 13:26 UTC commit that recorded the finished run, not the launch commit. This arm's own run_start event records commit 95a8225 with git_dirty: true, while its experiment.json records git_dirty: false.

Some behaviours are unmeasured. Tool-selection accuracy, argument validity and schema validity are not computed; their absence is not a zero. The evaluation population has no direct-answer examples.

Checkpoints

All four retained checkpoints are published under their step: checkpoint-600, checkpoint-1200 (the DEV-selected one), checkpoint-1800, checkpoint-2400. Selection used the 2,373-example DEV partition and the rule frozen before the run; the confirmatory partition was scored once, on checkpoint-1200 only.

Intended use

Research artifact. Not safety-tuned, not aligned, not for autonomous tool use. It inherits the limitations of its public training sources, including synthetic data and unverified tool invocations, and it will behave poorly outside the decision-boundary behaviour it was trained for.