JacksonHYD/pusht-repro-checkpoints

Checkpoints — "One Config Value, Thirty Points"

Model weights from the reproduction study at github.com/JacksonHYD/vla-pusht-repro-artifact.

All policies were trained from scratch on lerobot/pusht (206 demonstrations, 25650 frames, 96×96 images) on a single RTX 4060 Laptop (8 GiB).

Contents

Folder	Configuration	Steps	Success
dp_repro_200000	Diffusion Policy, official recipe (crop 84², GroupNorm, random backbone)	200k	64.0 % (n=50)
dp_repro_120000	same recipe, shorter training — the checkpoint used in the paper's 2×2	120k	63.5 % (n=200)
dp_official_200000_default_arch	Diffusion Policy with the library's shipped defaults	200k	33.3 % (n=30)
act_pusht_100k_100000	ACT, n_action_steps=10	100k	27.0 % (n=200)
smolvla_expert_016000	SmolVLA action expert (VLM frozen), n_action_steps=10	16k	24 % (n=50)
smolvla_lora_032000	SmolVLA with LoRA, n_action_steps=10	32k	28 % (n=50)

Only pretrained_model/ is uploaded; optimizer states (training_state/) are not included, so these checkpoints can be evaluated but not resumed.

Reproducing

The reported numbers require four source patches against LeRobot 7e241bd6 (deterministic convolution, per-batch RNG seeding, state-less ACT, per-episode step recording). Without them, evaluation is not bit-reproducible.

See patches/ in the GitHub repository for the patch file and a per-change explanation, and results/eval/ for the raw per-episode outcomes behind every number above.

Citation

@misc{huang2026configvalue,
  title  = {One Config Value, Thirty Points: Configuration Defaults and the
            Reproducibility of a Robot Learning Benchmark},
  author = {Huang, Yandong},
  year   = {2026},
  note   = {arXiv preprint}
}