Quantizability Gate — A′ students for GR00T-N1.5
A VLA policy emits a 16-step chunk of end-effector deltas. Most of that chunk is gross motion that can be executed in half the environment steps by summing adjacent deltas (K2 temporal quantization), but some of it — grasp/release transitions, precise insertion, loaded door pulls — breaks when compressed.
A quantizability gate decides this per chunk. A VLM judge (with an evolved guidance prompt) produces the labels; A′ is a small CNN student distilled from those labels, cheap enough to run inline at every chunk.
The gate emits a calibrated confidence in [0,1], not just a decision. Ï„ (the
compression threshold) is a per-(domain × teacher × architecture) operating
point, not a constant — different judges calibrate differently, so the same
Ï„=0.5 means different things across checkpoints. Treat Ï„ as trainable: an RL
head on top of confidence (and the exposed 128-d features) is the intended
extension point.
Checkpoints
| file | domain | teacher | notes |
|---|---|---|---|
A_robocasa_cosmos3.pt |
RoboCasa Kitchen (24 tasks) | Cosmos3-Nano (local VLM) | full stride-8 labels |
A_robocasa_gemma4.pt |
RoboCasa Kitchen | Gemma-4 (local VLM) | best closed-loop to date |
A_robocasa_frontier.pt |
RoboCasa Kitchen | GPT-5.6 (API) + action numbers | |
A_robocasa_cosmos9k.pt |
RoboCasa Kitchen | Cosmos3-Nano | 9k-frame calibration subset |
A_real_droid_pnp_cosmos3.pt |
real robot (MoSS DROID pick-and-place) | Cosmos3-Nano | |
robocasa_task_embeddings.npz |
— | — | 334 instructions × 384-d MiniLM |
Each is ~1.3 MB (0.32 M params): 4-block CNN over 3 stacked camera views (9×128×128) concatenated with a frozen MiniLM instruction embedding.
Closed-loop reference (RoboCasa, 24 tasks × 50 episodes each)
| policy | success | avg steps |
|---|---|---|
| uncompressed baseline | 0.657 | 327 |
| naive K2 (compress everything) | 0.598 | 221 |
| A′ gemma4, τ=0.5 | 0.667 | 258 |
| A′ cosmos3, τ=0.5 | 0.659 | 299 |
| gate token in DiT (architecture C) | 0.638–0.647 | 289 |
Reproducibility noise on this benchmark is ±1.5 pp, so A′ recovers baseline success while cutting ~21 % of environment steps; naive K2 cuts more steps but loses 6 pp of success.
Usage
from gate_api import QuantizabilityGate
from merge_k2 import merge_k2
gate = QuantizabilityGate("checkpoints/A_robocasa_gemma4.pt",
"checkpoints/robocasa_task_embeddings.npz")
out = gate(views, instruction) # views: 3 RGB arrays [left, right, wrist]
if out["confidence"] >= tau: # tau is yours to choose or to learn
chunk = merge_k2(chunk, delta_dims=range(5, 11), state_dims=[11])
out["features"] is the 128-d penultimate activation — the input to attach an
RL head to if you want to learn Ï„ (or a richer policy over K) rather than fix it.
views must be ordered as gate.views
(left_view, right_view, wrist_view); the checkpoint carries the order.
Drop-in server
module_gate_server.py serves the same POST /judge contract as the heavy VLM
judge, so an existing gated eval stack swaps to the student with no code change:
python module_gate_server.py --ckpt checkpoints/A_robocasa_gemma4.pt \
--task-emb checkpoints/robocasa_task_embeddings.npz --port 8130
Action space (RoboCasa, 12 dims)
0–4 unused (always zero) · 5–7 EE delta xyz · 8–10 rotation · 11 gripper
(0/1). Under K2, dims 5–10 sum across the merged pair and dim 11 takes the
last value. Summed deltas are clipped to the controller limit; raising that
limit does not by itself recover success — the gate does.
Scale reference (calibration statistics over the dataset): |Δ| median 0.343, 10th pct 0.124, 90th pct 0.734; consecutive-step turn angle median 11°, with only 1.1 % of steps turning more than 90°.
Running the benchmark
Environment setup, the three-process runtime layout, the full task list, how to sweep tau, and the pitfalls that have actually cost us runs are written up here: RoboCasa evaluation setup guide
Related
- Gate-token VLA (architecture C):
prehj/groot-n15-quantizability-gate-C-robocasa - Code, prompt-evolution loop, and label elicitation: https://github.com/rakybond007/GR00T-action-quantization/tree/action-quantization-gate-v2
evolved_guidance_robocasa.txt— the evolved guidance the teacher used, and what the students were distilled from.
LIBERO is not covered: it was only used for policy-side K2/varK experiments, so no gate was trained there.