phi-monster/Galahad-object-unified

Galahad C3-jfs (rebuild) — dual-branch grounded VLA

Seven-axis release weight · Project page · Code + battery

Paper: Instruction Blindness in Vision–Language–Action Policies: Diagnosis and a Low-Rank Data Cure — PDF · Project page · Code + battery

Single-weight MolmoAct2 policy: LoRA folded into the base, foresight head (_gf.*, 13 tensors) carried in the same model.safetensors. One forward, dual output (action + predicted object region). Not a base+adapter bundle.

Status: RECIPE ORIGINAL · CODE PART-RECONSTRUCTED · NUMBERS RE-MEASURED · BATTERY PENDING

Rebuilds the weight published as galahad-release-v2, deleted 2026-07-21.

• Recipe is original, not a reconstruction. Trained on the intended fork pin 80633827176a0203064cb141383664fba024e050 (exposes enable_lora_vlm/enable_lora_action_expert natively; no compat patch). The pin is on no branch of allenai/lerobot (rebased past) — it survives on refs/pull/2/head.
• Reconstructed: galahad_foresight.py — a provably pre-rename snapshot; other silent drifts possible.
• 🔴 Prior numbers do NOT transfer. TASK 94.0% n=200, SWAP2 200/200, C4-i 58.3%, C6 0.3-0.5 cm were on the DELETED weight. Not repeated here.

Recipe (frozen)

base	allenai/MolmoAct2-Think-LIBERO
fork pin	80633827176a0203064cb141383664fba024e050 (old API, native)
LoRA	VLM + action_expert, r32 / α16 / dropout0.05; trainable 109069184 (expert LoRA'd, not full-trained)
schedule	joint-from-start (GALAHAD_FS_OBS_FIRST=0), 3000 steps, DDP-8, batch 1/GPU
foresight	M=8 tokens, G=16, α W=0.05 (frozen — original never rebalanced), BCE pos_weight 15
data	phi-monster/arena-deconf-c1v2-jul2026 (900 ep / 95,658 frames, baked 16×16 region GT)
merge	peft merge_and_unload; self-check A/B/C PASS (fold Δ 0.012, tol 0.05); lora=0 _gf=13 base_prefix=0
repo	phi-monster/research @ d09e4df

Contract gates (this run)

gate	result
token append (train)	seq 472 -> 480 (+8 foresight tokens)
region GT	binary min0/max1/n_unique2 (identity stats; quantile normalizer did not touch it)
region GT positive-fraction	0.0508 — compact object blob, not the ~0.5 arm-sweep
action expert	LoRA'd, not full-trained
foresight regionIoU (train)	0.572 (median of last 40 per-rank samples, n=954; per-rank single-sample stat)

Evaluation — un-fakeable battery on libero_object_task (LIBERO-PRO, the B63 bench)

Re-measured on THIS weight (n=40/face = 8 non-round-bottle scenes x 5; round bottles dropped via --drop, a known-0%-oracle-grasp motor wall, not a grounding wall). Suite: zhouxueyang/LIBERO-Pro bddl+init (paired; derangement + satisfiability re-verified), assets from an on-box LIBERO-PRO checkout. Every cell gated on the inference token-append (seq 470 -> 478 [contract MATCHED]), 0 tripwires.

face	result	reading
TASK (perturbed instruction, full vision)	38/40 = 95.0%	follows the substituted instruction
SWAP2 (in-scene decoy)	true-target 0/40 = 0.0%, OBEYED_NAME 38/40	EEF goes to the named decoy (d_named<<d_true), not the memorized target
OCC (both cameras blanked)	0/40 = 0.0%	collapses -> vision is required
NONSENSE ("xxx" instruction)	8/40 = 20.0%	mostly collapses (5/8 scenes 0%); 72.5pp below TASK -> instruction required; residual per-scene visual prior on ~3 scenes
UNPERTURBED_COMPLETION (stock libero_object)	34/40 = 85.0%	positive control: the weight is capable

Cold-verify — honest read (NOT all controls collapse):

• OCC collapses to 0% -> vision is required (rules out solving from proprio/memory).
• SWAP2 is the un-fakeable core: true-target ~0 WITH OBEYED_NAME high (EEF goes to the named decoy, d_named << d_true). A visual-prior / instruction-ignoring policy would grab the CANONICAL object, not the decoy, so SWAP2 cannot be a visual-prior artifact -> instruction-following is real.
• NONSENSE mostly collapses (8/40 = 20.0%): 5 of 8 scenes go to 0% under a meaningless instruction; 3 scenes retain a per-scene visual-prior success. The instruction-grounding excess is large (TASK 38/40 = 95.0% vs NONSENSE 8/40 = 20.0% = ~72pp), so the instruction is required; a small residual visual prior remains on a few scenes. Reported as-is.
• Merge action-equivalence 1.19e-2 (cf. B63 1.17e-2) = faithful fold.

🔴 These are the rebuilt weight's OWN numbers. B63's TASK 94.0% / SWAP2 200/200 were on the deleted weight.

Evaluation — un-fakeable battery on libero_object_task (LIBERO-PRO, the B63 bench)

Re-measured on THIS weight (n=40/face = 8 non-round-bottle scenes x 5; round bottles dropped via --drop, a known-0%-oracle-grasp motor wall, not a grounding wall). Suite: zhouxueyang/LIBERO-Pro bddl+init (paired; derangement + satisfiability re-verified), assets from an on-box LIBERO-PRO checkout. Every cell gated on the inference token-append (seq 470 -> 478 [contract MATCHED]), 0 tripwires.

face	result	reading
TASK (perturbed instruction, full vision)	38/40 = 95.0%	follows the substituted instruction
SWAP2 (in-scene decoy)	true-target 0/40 = 0.0%, OBEYED_NAME 38/40	EEF goes to the named decoy (d_named<<d_true), not the memorized target
OCC (both cameras blanked)	0/40 = 0.0%	collapses -> vision is required
NONSENSE ("xxx" instruction)	8/40 = 20.0%	mostly collapses (5/8 scenes 0%); 72.5pp below TASK -> instruction required; residual per-scene visual prior on ~3 scenes
UNPERTURBED_COMPLETION (stock libero_object)	34/40 = 85.0%	positive control: the weight is capable

Cold-verify — honest read (NOT all controls collapse):

• OCC collapses to 0% -> vision is required (rules out solving from proprio/memory).
• SWAP2 is the un-fakeable core: true-target ~0 WITH OBEYED_NAME high (EEF goes to the named decoy, d_named << d_true). A visual-prior / instruction-ignoring policy would grab the CANONICAL object, not the decoy, so SWAP2 cannot be a visual-prior artifact -> instruction-following is real.
• NONSENSE mostly collapses (8/40 = 20.0%): 5 of 8 scenes go to 0% under a meaningless instruction; 3 scenes retain a per-scene visual-prior success. The instruction-grounding excess is large (TASK 38/40 = 95.0% vs NONSENSE 8/40 = 20.0% = ~72pp), so the instruction is required; a small residual visual prior remains on a few scenes. Reported as-is.
• Merge action-equivalence 1.19e-2 (cf. B63 1.17e-2) = faithful fold.

🔴 These are the rebuilt weight's OWN numbers. B63's TASK 94.0% / SWAP2 200/200 were on the deleted weight.

Evaluation — un-fakeable battery on libero_object_task (LIBERO-PRO, the B63 bench)

Re-measured on THIS weight (n=40/face = 8 non-round-bottle scenes x 5; round bottles dropped via --drop, a known-0%-oracle-grasp motor wall, not a grounding wall). Suite: zhouxueyang/LIBERO-Pro bddl+init (paired; derangement + satisfiability re-verified), assets from an on-box LIBERO-PRO checkout. Every cell gated on the inference token-append (seq 470 -> 478 [contract MATCHED]), 0 tripwires.

face	result	reading
TASK (perturbed instruction, full vision)	38/40 = 95.0%	follows the substituted instruction
SWAP2 (in-scene decoy)	true-target 0/40 = 0.0%, OBEYED_NAME 38/40	EEF goes to the named decoy (d_named<<d_true), not the memorized target
OCC (both cameras blanked)	0/40 = 0.0%	collapses -> vision is required
NONSENSE ("xxx" instruction)	9/40 = 22.5%	mostly collapses (5/8 scenes 0%); 72.5pp below TASK -> instruction required; residual per-scene visual prior on ~3 scenes
UNPERTURBED_COMPLETION (stock libero_object)	34/40 = 85.0%	positive control: the weight is capable

Cold-verify — honest read (NOT all controls collapse):

• OCC collapses to 0% -> vision is required (rules out solving from proprio/memory).
• SWAP2 is the un-fakeable core: true-target ~0 WITH OBEYED_NAME high (EEF goes to the named decoy, d_named << d_true). A visual-prior / instruction-ignoring policy would grab the CANONICAL object, not the decoy, so SWAP2 cannot be a visual-prior artifact -> instruction-following is real.
• NONSENSE mostly collapses (9/40 = 22.5%): 5 of 8 scenes go to 0% under a meaningless instruction; 3 scenes retain a per-scene visual-prior success. The instruction-grounding excess is large (TASK 38/40 = 95.0% vs NONSENSE 9/40 = 22.5% = ~72pp), so the instruction is required; a small residual visual prior remains on a few scenes. 🔴 9/40 = 22.5% is HIGHER than B63's 6.0% (12/200) — at n=40 vs B63's n=200 this is likely small-n, but the garbage-instruction control is a bit less clean on this reconstruction. Stated, not smoothed. (A second independent n=40 nonsense run gave 20.0%, consistent.)
• Merge action-equivalence 1.19e-2 (cf. B63 1.17e-2) = faithful fold.

Reproduction verdict: these n=40 numbers REPRODUCE the deleted weight's B63 grounding pattern (TASK ~94% -> 95.0%; SWAP2 OBEYED high 38/40 + true-target 0; OCC 0). The n=40 re-measurement is the release target for the reconstructed weight; we do NOT extend to n=200.

🔴 The n=200 main table was measured on the now-DELETED weight and is NOT transferred here (TASK 94.0%, SWAP2 200/200). This release ships the reconstructed weight with its own n=40 re-measurement above + the verdict that it reproduces B63. The recipe is original (real fork pin 80633827, native LoRA API), but galahad_foresight.py is a recovered pre-rename snapshot => RECONSTRUCTION, not a byte reproduction. Bench: libero_object_task from zhouxueyang/LIBERO-Pro.

C6 chain-demo figure: pending — handed to the writing team; not rendered on this weight here (so its absence is intentional, not missing).