xlerobot-coffee-model-real-smolvla-baseline

SmolVLA, vision only. The architecture control for the Coffee Automata force study.

Methods A, B and D are all ACT. If the force-aware variants win, that result is only as general as ACT itself. This set trains a different policy family — a 450M vision-language model with an action expert — on the same data, the same rungs and the same step budget, with no force channel at all. It is the reference point for "how much of the difference is the method, and how much is the backbone".

Trained on the real 17-DoF XLeRobot, two cameras (head + right_wrist).

Contents

folder	run	chunk	t1 episodes
g135_shared_t1_t3_t5_2cam_17dof	sv_g135m	50	mixed (see below)
g35_shared_t3_t5_2cam_17dof	sv_g35	50	success1 / standard
t1_place_cup_2cam_17dof	sv_t1m	50	mixed (see below)
t3_cup_to_tray_2cam_17dof	sv_t3	50	success1 / standard
t5_tray_to_table_2cam_17dof	sv_t5	50	success1 / standard

Each folder is a complete pretrained_model (final 100k checkpoint at the root) with a checkpoints/{025000,050000,075000}/ step sweep alongside — the same 25k spacing as every other model in the study, so checkpoints compare step-for-step.

⚠ The two t1-containing rungs use different data from every other model in the study

t1_place_cup_2cam_17dof and g135_shared_t1_t3_t5_2cam_17dof were retrained on a mixed t1 set. Every other model in this repo, and every ACT model in the A / B / D repos, uses the original t1.

	t1 episodes	source
original t1 (ACT A/B/D, and this repo's t3/t5/g35)	50	t1_place_cup_redcup_success1 only
mixed t1 (these two folders)	50	30 of success1 + all 20 of success2 + success3 + success4

success1 is a single session in which the cup starts in much the same place every time. success2/3/4 vary the placement far more. The mix trades 20 near-duplicate episodes for 20 varied ones, so the rung covers a wider start distribution. The episode count stays at 50, deliberately: it is what keeps the rung comparable in size to t3, t5 and the other tasks. The 30 kept from success1 are evenly spaced across its 50 rather than the first 30, since episodes recorded back-to-back drift and a prefix would sample one end of that drift.

What this means for the comparison. SmolVLA-vs-ACT stays clean on t3, t5 and g35. On t1 and g135 the two differ in both architecture and t1 data, so a difference there cannot be attributed to architecture alone. Read those two cells as "SmolVLA on the better t1 data", not as a controlled architecture comparison.

Recipe

Fine-tuned from lerobot/smolvla_base at its default hyper-parameters — deliberately untuned, so it is a baseline and not a competitor that got extra attention:

SmolVLM2-500M-Video-Instruct backbone · chunk_size 50 · n_action_steps 50 · batch 8 · 100k steps · seed 1000 · AdamW lr 1e-4 · frozen vision encoder · action-expert-only training · images resized to 512×512 with padding.

The head and wrist cameras are renamed to the camera1/camera2 slots the base checkpoint expects, and the unused third slot is padded (empty_cameras=1). State and action are the 17-DoF vectors, projected into SmolVLA's 32-D slots.

Data: IntelligentDecisionLab/xlerobot-coffee-real-2cam.

Comparing against the ACT methods

method	repo	force
A — vision + position	…-model-real-a-vision-pos	none
B — HPI token	…-model-real-b-force	input, no loss
D — closed force loop	…-model-real-d-force-closed-loop	input + future-force loss
SmolVLA (this repo)	—	none

Same rungs (t1, t3, t5, g35, g135) and the same 25k step sweep. Same data on t3 / t5 / g35; different t1 data on t1 and g135 — see the warning above.

Caveats

• No on-robot evaluation yet. Training loss is not task success, and SmolVLA's loss is not comparable to ACT's — different objectives, different action parameterisation. Only rollout success rates will settle the comparison.
• Default hyper-parameters mean this is an untuned baseline. A poor showing is evidence about default SmolVLA on this data, not about the architecture's ceiling.
• t2_push_button has no rung — it is absent from the 2-camera dataset.

Part of the X-Lerobot Coffee Automata project. AS-CITI Intelligent Decision Lab.