Diffusion Policy on Push-T: dataset-size study
Four Diffusion Policy checkpoints (CNN/UNet, low-dim keypoint observations) trained on the 2D Push-T task with 50, 100, 150 and 200 self-collected teleoperated demonstrations, to measure how performance scales with dataset size.
Code, configs and evaluation fork: https://github.com/manhasv/pusht200 Dataset: https://huggingface.co/datasets/manhasv/pusht-200-data
Usage
N=50 # one of: 50, 100, 150, 200
hf download manhasv/diffusion-policy-pusht200 demos${N}/latest.ckpt --local-dir checkpoints/
python pusht200_eval.py -c checkpoints/demos${N}/latest.ckpt -o eval_out/demos${N}
Requires the conda environment from the GitHub repo (conda_environment_pusht.yaml).
Files
| File | Training demos | SHA-256 |
|---|---|---|
demos50/latest.ckpt |
50 | de9db8eb1beac50b44aca53dbf14adda0fa604b7b5139522447775adddfb98ff |
demos100/latest.ckpt |
100 | 7d370af26ab36f46ca07d5194f5ff168cc27d162adffc78652bc933847d4b4d5 |
demos150/latest.ckpt |
150 | 5fdc94263d3775fc56aef5ab4ee5ef23e0bbcb67cff46a08cacf30ffade410ca |
demos200/latest.ckpt |
200 | 862a972d50419ba541aa402224fdcb65952a1b1e1e85f59dbac3f9fe5ce6f452 |
Each file is a full training checkpoint (model, EMA model, optimizer state, and the exact training config).
Training
- Base config: upstream
train_diffusion_unet_lowdim_workspace.yaml+task/pusht_lowdim.yaml; only dataset path, subset size, validation split and step budget changed (pusht_200/train_pusht_datasize.yaml). - Data: 200 self-collected demos (env seeds 0–199); each model uses a random subset of size N (subset seed 42).
- Same number of gradient steps for every model (~100k, batch 256)
- Training seed 42; final checkpoint used (no checkpoint selection on test seeds).
- Hardware: NVIDIA RTX A5000 (Brown Oscar cluster).
Results
Evaluated on the same 50 fixed test seeds (100000–100049) for every model, sampling seed 0, EMA weights. Reward = max fraction of the target area covered by the T; success = coverage ≥ 0.95.
| Demos | Mean reward ± std | Success rate |
|---|---|---|
| 50 | 0.57 ± 0.45 | 44%(22/50) |
| 100 | 0.73 ± 0.42 | 66%(33/50) |
| 150 | 0.84 ± 0.34 | 74%(37/50) |
| 200 | 0.88 ± 0.32 | 84%(42/50) |
License
MIT, following the upstream Diffusion Policy repository.