samuel500/graft-repro-adapters

GRAFT Countdown repro adapters

Adapters from our reproduction of the Countdown-Code experiments (Fig 2) of the GRAFT paper, plus follow-up arms (no-intervention anchor x learning rate, GRAFT and GRAFT w/o routing at lr 5e-4). All runs train adapters on the frozen SFT base samuel500/graft-qwen3-8b-countdown-sft (revision a2d54fdf) for 200 steps on 1x H200; the base model's card lists its SFT data source and recipe. The experiments, rerun commands and caveats are documented in rl/REPRO.md on branch repro of samuel500/GRAFT, a fork of the paper's code (jnward/GRAFT); rl/repro_data/runs.csv there is the same index as below.

Layout

<class>/<run_name>/: model.safetensors + dual_lora_config.json (checkpoint-200, adapter weights only), run_config.yaml, eval/fseval.json (n=256 fseval), eval/probe.json (expr-only pass@1, k=1, n=256), meta.json (code commit, sweep, base model, lr, seed, deploy scale, sha256).

• graft/: GRAFT: dual MLP adapters (64 RETAIN + 64 FORGET neurons per layer), detector-flagged hacks routed to FORGET. Deploy with FORGET ablated (scale 0.0).
• graft-no-routing/: The paper's GRAFT w/o routing control: same dual adapters and anchor as GRAFT, detector recall 0 so nothing is routed. Reported at scale 0.0 like GRAFT; scale 1.0 = both adapters.
• no-intervention/: Standard RL, nothing ablated (scale 1.0). Paper config: dual m64 adapters with routing off. The +anchor arms train ONE 128-neuron adapter (forget_neurons 0) on the GRAFT recipe's LeetCode anchor.

Which scale to deploy

Scales are (retain, forget) multipliers on the two adapters. deploy_scale is the forget scale with retain at 1: 0.0 = RETAIN only (GRAFT's deployed model; also how the paper reports GRAFT w/o routing), 1.0 = both adapters (No intervention, which has no FORGET adapter to ablate).

Loading

With the code repo's rl/ on sys.path (HF transformers path; the adapter shape is read from dual_lora_config.json, including the single 128-neuron adapter of the +anchor no-intervention arms):

from huggingface_hub import snapshot_download
from eval_utils import load_gradient_routing_model
from gradient_routing import set_scales

base = snapshot_download("samuel500/graft-qwen3-8b-countdown-sft", revision="a2d54fdf0aa9fa6ee8e487777ee7d63adaf8ec48")
ckpt = snapshot_download("samuel500/graft-repro-adapters",
                          allow_patterns="graft/cdhf100_gr_lccoh64_lr3_s9/*")
model = load_gradient_routing_model(f"{ckpt}/graft/cdhf100_gr_lccoh64_lr3_s9", base_model=base)
set_scales(model, retain_scale=1.0, forget_scale=0.0)  # deploy: FORGET ablated

For vLLM serving (vllm_utils.py), pass the run's mlp_config from run_config.yaml: m64 for GRAFT, GRAFT w/o routing and the paper-config no-intervention runs, m128_retain_only for the +anchor no-intervention runs.

Runs

hack = ground-truth hack rate, pass@1 = expr-only pass@1, both at the deploy scale (n=256, from eval/). lr/3 = 5e-4/3, the paper's GRAFT lr.

class	run	arm	seed	lr	anchor	deploy scale	hack	pass@1	code commit
graft	cdhf100_gr_lccoh64_lr3_s9	GRAFT (lr/3)	9	1.667e-4	leetcode	0.0	0.000	0.773	52f034c
graft	cdhf100_gr_lccoh64_lr3_s15	GRAFT (lr/3)	15	1.667e-4	leetcode	0.0	0.020	0.773	52f034c
graft	cdhf100_gr_lccoh64_lr3_s16	GRAFT (lr/3)	16	1.667e-4	leetcode	0.0	0.008	0.801	52f034c
graft	cdhf100_gr_lccoh64_lr3_s1	GRAFT (lr/3)	1	1.667e-4	leetcode	0.0	0.016	0.797	52f034c
no-intervention	countdown_code_hack_reward_penalty_amountmissing_s9	No intervention	9	5e-4	none	1.0	0.941	0.250	52f034c
no-intervention	countdown_code_hack_reward_penalty_amountmissing_s15	No intervention	15	5e-4	none	1.0	1.000	0.000	52f034c
no-intervention	countdown_code_hack_reward_penalty_amountmissing_s16	No intervention	16	5e-4	none	1.0	0.480	0.543	52f034c
no-intervention	countdown_code_hack_reward_penalty_amountmissing_s1	No intervention	1	5e-4	none	1.0	0.727	0.293	c459f01
no-intervention	cdhf100_dn_lr3_s9	No intervention (lr/3)	9	1.667e-4	none	1.0	0.535	0.512	c459f01
no-intervention	cdhf100_dn_lr3_s15	No intervention (lr/3)	15	1.667e-4	none	1.0	0.328	0.758	c459f01
no-intervention	cdhf100_dn_lr3_s16	No intervention (lr/3)	16	1.667e-4	none	1.0	0.297	0.734	c459f01
no-intervention	cdhf100_dn_lr3_s1	No intervention (lr/3)	1	1.667e-4	none	1.0	0.305	0.754	c459f01
no-intervention	cdhf100_dn_lccoh64_lr1_s9	No intervention + anchor	9	5e-4	leetcode	1.0	0.473	0.535	c459f01
no-intervention	cdhf100_dn_lccoh64_lr1_s15	No intervention + anchor	15	5e-4	leetcode	1.0	0.895	0.109	c459f01
no-intervention	cdhf100_dn_lccoh64_lr1_s16	No intervention + anchor	16	5e-4	leetcode	1.0	0.547	0.527	c459f01
no-intervention	cdhf100_dn_lccoh64_lr1_s1	No intervention + anchor	1	5e-4	leetcode	1.0	0.891	0.152	c459f01
no-intervention	cdhf100_dn_lccoh64_lr3_s9	No intervention + anchor (lr/3)	9	1.667e-4	leetcode	1.0	0.355	0.703	6f52095
no-intervention	cdhf100_dn_lccoh64_lr3_s15	No intervention + anchor (lr/3)	15	1.667e-4	leetcode	1.0	0.738	0.305	6f52095
no-intervention	cdhf100_dn_lccoh64_lr3_s16	No intervention + anchor (lr/3)	16	1.667e-4	leetcode	1.0	0.297	0.723	6f52095
no-intervention	cdhf100_dn_lccoh64_lr3_s1	No intervention + anchor (lr/3)	1	1.667e-4	leetcode	1.0	0.285	0.746	6f52095
graft	cdhf100_gr_lccoh64_lr1_s9	GRAFT (lr 5e-4)	9	5e-4	leetcode	0.0	0.094	0.230	f3ab693
graft	cdhf100_gr_lccoh64_lr1_s15	GRAFT (lr 5e-4)	15	5e-4	leetcode	0.0	0.035	0.766	f3ab693
graft	cdhf100_gr_lccoh64_lr1_s16	GRAFT (lr 5e-4)	16	5e-4	leetcode	0.0	0.000	0.555	f3ab693
graft	cdhf100_gr_lccoh64_lr1_s1	GRAFT (lr 5e-4)	1	5e-4	leetcode	0.0	0.180	0.484	f3ab693
graft-no-routing	cdhf100_noroute_anchor_lr1_s9	GRAFT w/o routing (lr 5e-4)	9	5e-4	leetcode	0.0	0.281	0.668	f3ab693
graft-no-routing	cdhf100_noroute_anchor_lr1_s15	GRAFT w/o routing (lr 5e-4)	15	5e-4	leetcode	0.0	0.352	0.633	f3ab693
graft-no-routing	cdhf100_noroute_anchor_lr1_s16	GRAFT w/o routing (lr 5e-4)	16	5e-4	leetcode	0.0	0.305	0.707	f3ab693
graft-no-routing	cdhf100_noroute_anchor_lr1_s1	GRAFT w/o routing (lr 5e-4)	1	5e-4	leetcode	0.0	0.707	0.359	f3ab693