prehj/faster-vla-speed-module

Faster-VLA speed module (per-chunk speed-up predictor for frozen GR00T-N1.5)

A small MLP that predicts, for each action chunk of a frozen VLA policy, how much faster the chunk can be executed (speed-up K ∈ [1, 3]). The policy is never retrained. At run time the policy's predicted chunk is compressed post hoc by K (consecutive delta actions merged, exact block rule) and the gripper is time-scaled by the same factor.

Code, label toolchain and evaluation: https://github.com/rakybond007/faster_vla_harness (see docs/TESTING.md).

Inputs / output

input	shape	source at run time
image features	V × 768	frozen DINOv3 ViT-S/16 (facebook/dinov3-vits16-pretrain-lvd1689m, gated on HF) of every camera the policy sees (LIBERO: front + wrist, RoboCasa: left + right + wrist)
state	LIBERO 8, RoboCasa 9	proprioception (LIBERO: eef pos, axis-angle, finger joints; RoboCasa: relative eef pos / quat, finger joints)
predicted chunk	16 × 7	the frozen policy's own predicted action chunk
output	scalar	K = 1 + 2·sigmoid(z)

No task identity is given to the module.

Training labels (no rollouts used)

K_max per demonstration chunk from measured, real-robot-applicable factors:

• tolerance = min(demonstration spread, geometric margin)
  • geometric margin: grasp / place margin near a contact, wrist-camera clearance elsewhere
• the largest K whose calibrated extra controller tracking error (position + camera lever arm × orientation) stays within the tolerance
• capped by the Franka joint velocity / acceleration / torque limits
• v17b: careful before every contact; with an open hand, the tolerance is also bounded by the side clearance (wrist-camera distance to everything except the object about to be grasped)
• final transform: labels below 2 are lifted toward 1.5 (K' = 1.5 + 0.5 (K − 1), order kept) and capped at 2.5

Simulator ground truth was used only to validate the tools. Hyper-parameters were chosen on held-out label error (pre-registered grid), never on rollouts.

Files

path	base policy	labels	notes
libero/v17b/ + ensemble/seed{1..4}/	prehj/GR00T-N1.5-libero-baseline-bs32-60k	v17b	final; K = min over the five members, no floor
robocasa/v17b/ + ensemble/seed{1..4}/	prehj/GR00T-N1.5-robocasa-baseline	v17b	final
robocasa365/v17b/ + ensemble/seed{1..4}/	robocasa/robocasa365_checkpoints gr00t_n1-5/multitask_learning/checkpoint-120000	v17b	final; also compresses mobile-base chunks
*/v16b (+ ensembles)		v16b	previous version (labels as measured, no lift / cap)
libero/v13, robocasa/v13 (+ ensembles), libero/v11, libero/v10, robocasa/v12, robocasa/v11			earlier versions

Each folder: model.pt, norm.npz, metrics.json (+ val_pred.parquet for the main model). SHA256SUMS lists the model.pt hashes.

Results (closed loop, final v17b, no speed floor)

Paired on the same initial states, task-stratified bootstrap. Speed-up = env steps of the reference / env steps of the arm, on episodes both solve.

benchmark	1x	uniform 1.5x	ours (v17b)	speed-up vs 1x	Δ vs uniform 1.5x [95% CI]	speed-up vs uniform 1.5x [95% CI]
LIBERO (40 tasks x 50)	96.2%	94.4%	93.0%	1.64	−1.4 [−2.7, +0.0]	1.21 [1.19, 1.22]
RoboCasa (24 tasks x 50)	66.4%	66.4%	65.3%	1.67	−1.1 [−3.8, +1.6]	1.20 [1.13, 1.27]
RoboCasa365 (18 tasks x 50)	51.8%	53.4%	54.6%	1.52	+1.1 [−2.2, +4.6]	1.15 [1.06, 1.24]

At the success rate of uniform 1.5x, the per-chunk module is 1.15–1.21x faster than uniform 1.5x.

Usage

pip install huggingface_hub
huggingface-cli download prehj/faster-vla-speed-module --local-dir speed_module
# K server (from the git repo): one model, or a comma-separated list for the ensemble
M=speed_module/libero/v17b; ENS=$M/model.pt; for s in 1 2 3 4; do ENS=$ENS,$M/ensemble/seed$s/model.pt; done
python tools/qmodule/server.py --bench libero --model $ENS --k-min 1.0 --port 9800   # no floor (final setting)
curl -X POST localhost:9800/k -d '{"images": [...], "shape": [...], "state": [...], "pred": [[...]]}'

The full closed-loop protocol (policy server + K server + LIBERO / RoboCasa clients) is in docs/TESTING.md of the git repo.

Limitations

• Policies: trained for GR00T-N1.5 fine-tuned on LIBERO / RoboCasa. The module conditions on that policy's predicted chunks.
• LIBERO: the remaining loss is concentrated in stove + moka pot (libero_10_2, libero_10_8).
• RoboCasa: the wrist-clearance tool still under-detects near drawer / door fronts.
• An execution floor (K >= 1.5) used in earlier versions overrode 40-56% of the labels and was dropped; the final lift keeps the order of the tool labels instead of overriding them.
• The side clearance uses monocular depth (Depth Anything 3); at close range it reads obstacles too close (median 6 mm vs 11 mm on simulator depth), so some chunks near a grasp are slower than necessary.