SleepMastger/srlwam-lilawam-stack-v2

srlwam: LiLa-WAM stack_v2 policies

LiLa-WAM policies (a flow-matching action expert on a frozen DINOv3 ViT-L/16) for two-cube stacking on a Franka Panda, from the stack_v2 experiment (four training runs). Three runs use the stack_v2 simulation corpus. The fourth, real_only, uses no simulation data: it is a re-run of the stack_v1 real_only baseline (see Normalization). This repo uses the same layout and deploy code as the stack_v1 LiLa-WAM release in aabyaneh/srlwam, so the same deploy/check_deployment.py gates apply unchanged.

Training data: aabyaneh/srlwam dataset, revision cffcf593 (stack/sim/stack_v2), with real recordings from stack/real/stack_v0. Experiment: srlwam/model/experiments/stack_v2_lilawam, Slurm jobs 4878–4881 (2026-10-01).

Checkpoints

Each directory contains model.pt (selected: lowest validation action L1 over a 16-step horizon, or lowest video loss for the video run), model_final.pt (last epoch), config.yaml, norm_stats.json, split.json, deploy.json (obs frame, action units, training state range, weight sha256), fixture_reference.npz and eval_real_holdout.json.

checkpoint	trained on	obs frame	action gain	model.pt epoch	dir. cos (step 0)	gripper acc (step 0)	net dz h16 (truth −0.0593)	deploy self-test
real_only	16 real episodes only, 600 ep, real normalization, no sim; re-run of stack_v1 real_only	raw_robot	0.1861	510	+0.328	0.968	−0.0710	8/8
real_finetune_from_sim_v2	sim_v2_pretrain → 16 real episodes, 200 ep, sim_v2 normalization (badly scaled for real data, see below)	raw_robot	0.1861	110	+0.308	0.956	−0.0816	7/8
sim_v2_pretrain	stack_v2 sim, 750 train demos (30 appearance variants, whole-variant split), 120 ep, 8×H200	sim_ee_frame	0.1975	80	+0.093	0.576	+0.0350	5/8
real_video_from_sim_v2	sim_v2_pretrain, visual adapter adapted on real camera video only (no real state/actions) plus labeled sim replay, 50 ep	sim_ee_frame	0.1975	37	+0.091	0.573	+0.0319	5/8

The held-out columns are open-loop agreement with the operator's next 16 actions on real episodes 0/1/15/17 (906 windows) in the checkpoint's own obs frame, for model.pt. They are not closed-loop success rates. Each directory's eval_real_holdout.json has the model_final.pt numbers and the wrong-frame control. The self-test column is deploy/check_deployment.py run on the training machine against deploy/fixture_observations.npz. The log is at tools/logs/hf_release_refs_v2_4921.log.

Read before deploying

• Start with real_only. It is the only checkpoint here that passes all 8 gates. It contains no stack_v2 data, so it behaves like the stack_v1 real_only baseline, not like a stack_v2 result.
• real_finetune_from_sim_v2 fails the "actions in training action range" gate (37–44% of steps are far outside it). Its real training data was normalized with the sim_v2 ranges, which do not fit real data. See Normalization.
• sim_v2_pretrain and real_video_from_sim_v2 do not transfer zero-shot. Real robot state falls about 21 training ranges outside the stack_v2 sim state range (gate 4). Gripper agreement on real data is at chance, and the predicted net dz is positive, so the policy rises instead of descending. Treat them as sim baselines and parents, not as real-robot policies.
• Action units differ. The two sim-unit checkpoints were trained at a stack_v2 gain of 0.1975 m/unit. The shared adapter's command.real_controller_delta divides by the stack_v1 real-pack scale 4.8436, which is about 6% wrong for these weights. Use command.delta, or divide by deploy.json → action.real_controller_scale (4.564). The real-trained checkpoints use real-pack units (0.1861), the same as every stack_v1 real checkpoint.
• All four checkpoints were trained with the same action conventions: chunk 32, replan every 16, 10 Hz, +1 = gripper open, and rotation outputs not applied (the adapter holds orientation).

Normalization

Each checkpoint's norm_stats.json is byte-identical to the stats file its training job loaded, so the release is consistent between training and deployment. The model maps each action and state channel from its [min, max] range to [-1, 1] and does not clip.

The sim_v2 ranges differ a lot from stack_v1. stack_v2 sim takes about 3.5× smaller translation steps (±0.016 vs v1's ±0.057). It also holds the orientation almost fixed, so each quaternion channel spans only about 1e-4 (v1: about 1e-2).

That matters only when real data is normalized with sim ranges. Here is the real training data's range (its 0.5–99.5 percentile stats) after normalization:

checkpoint (stats used)	real xyz actions	real quaternion state
stack_v1 real_finetune_from_sim_v1 (sim_v1 stats)	within [-1, 1]	−32 to −0.6
real_finetune_from_sim_v2 (sim_v2 stats)	−3.9 to +4.5	−1298 to +15
real_only (real stats)	≈ [-1, 1]	≈ [-1, 1]

As a result, real_finetune_from_sim_v2 is not a clean measure of what stack_v2 sim pretraining adds. A fair comparison needs a fine-tune with real-data (or combined sim + real) normalization. Cosmos job 4885 has the same issue on the action side.

real_only sits in this experiment only because it was launched with it. It uses the same real recordings (data/stack_v2/packed/real is a symlink to the stack_v1 real pack), a byte-identical norm_stats.json and the same config as the stack_v1 real_only (apart from the run name). It is a second seed-matched run of that baseline.

sim_v2_pretrain and real_video_from_sim_v2 are normalized consistently on sim_v2 data. They fail on the robot because the real state lies far outside the sim_v2 state range, which is a sim-to-real gap, not a normalization error.

Usage

hf download SleepMastger/srlwam-lilawam-stack-v2 --local-dir srlwam-lilawam-stack-v2 && cd srlwam-lilawam-stack-v2
pip install torch torchvision transformers omegaconf numpy opencv-python scipy matplotlib pyyaml
python deploy/check_deployment.py --checkpoint lilawam/stack_v2/real_only
python deploy/check_deployment.py --checkpoint lilawam/stack_v2/real_only --weights model_final.pt

import sys; sys.path.insert(0, "deploy")
from lilawam_deploy import load_policy
policy, spec = load_policy("lilawam/stack_v2/real_only")   # reads deploy.json: obs frame, gain, weights

Then follow deploy/DEPLOYMENT.md gate by gate. The DINOv3 encoder is bundled at deploy/dinov3-vitl16-pretrain-lvd1689m, and each config.yaml points to it by relative path.

Layout

lilawam/stack_v2/<checkpoint>/   weights + config + norm stats + split + deploy.json + eval
deploy/                          shared LiLa-WAM deploy adapter, gates, fixture, model code, DINOv3 ViT-L/16
training/                        stack_v2 experiment configs, plan, launch scripts, stats, splits
tools/                           bundle build / reference generation scripts and the self-test log
SHA256SUMS.txt                   sha256 of every file in this release

Trained by Amin Abyaneh. This repo mirrors the bundles staged in srlwam/model/hf_release/lilawam/stack_v2 (built 2026-10-02 by tools/build_bundles_v2.py).