2cmCubeInBox_Grid — ACT Policy (chunk_size=100)

An ACT (Action Chunking Transformer) policy trained via LeRobot to pick up a small cube with an SO-101 robotic arm and place it into a cardboard box.

Dataset

Trained on mrtonyy/2cmCubeInBox_Grid — 260 episodes / 141,948 frames.

The workspace was marked with a physical 2cm×2cm tape grid (matching the cube's own footprint), with one cell tagged as a fixed origin reference. Cube-pick demonstrations cover 23 grid positions, ~10 repetitions each (230 episodes), forming a single dense, contiguous block directly above the destination box — every cell inside that block was demonstrated, with no deliberately-withheld interior cells. The dataset also includes ~30 "hold position, no cube present" idle episodes, teaching the policy to stay still rather than act when no cube is in view. Any test of generalization to positions outside the demonstrated block is done separately at rollout time, not via a held-out split baked into the training data.

Two cameras (overhead, wrist), 640×480 @ 30fps. Box position was held fixed throughout, so this dataset isolates cube-pick position as the sole varying factor.

Architecture

Standard ACT config from LeRobot:

• Vision backbone: resnet18, ImageNet-pretrained
• Transformer: dim_model=512, n_heads=8, dim_feedforward=3200, n_encoder_layers=4, n_decoder_layers=1
• CVAE objective: use_vae=true, latent_dim=32, n_vae_encoder_layers=4, kl_weight=10.0
• chunk_size=100, n_action_steps=100 — the policy commits to a 100-step (~3.3s at 30fps) predicted action chunk before re-observing and re-planning

A companion checkpoint was trained in parallel with chunk_size=50 / n_action_steps=50 (otherwise identical setup) as an A/B comparison of shorter vs. longer open-loop commitment windows.

Training

• 180,000 steps, batch size 8, AdamW (lr=1e-5, weight_decay=1e-4)
• No image augmentation
• No held-out validation split — trained on 100% of recorded frames (eval_split=0.0)
• Trained from scratch (not fine-tuned from a prior checkpoint or dataset)
• Hardware: Apple Silicon Mac mini, MPS backend

Robot

SO-101 follower/leader arm pair, teleoperated for data collection.