ACT_sim_handle_basket (RoboSynChallenge submission checkpoint)

ACT (Action Chunking with Transformers) checkpoint for the RoboSynChallenge handle_basket simulation task, re-packaged in the LeRobot 0.4.4 layout so that it loads without any legacy code path.

This checkpoint is the imitation-learning component of our preliminary-round submission. It is used both

• as a stand-alone ACT baseline for handle_basket, and
• as the action-chunk generator that the submitted hybrid policy hands over to after its analytic grasp/place stage.

Provenance

Item	Value
Upstream weights	RoboSynChallenge/ACT_sim_handle_basket
Upstream format	legacy LeRobot b883328 (normalization buffers stored inside model.safetensors)
This repository	same weights, migrated to the LeRobot 0.4.4 external pre/post-processor layout
Migration tool	lerobot.processor.migrate_policy_normalization with a key-restoration pass (see below)

The migration is format-only: no weight is retrained, re-initialised or fine-tuned. model.safetensors is byte-for-byte the upstream tensor payload.

Why a key-restoration pass was necessary

LeRobot's official migrator renames feature keys with ".".join(parts[:-1]).replace("_", "."), which turns observation_images_cam_high into observation.images.cam.high instead of observation.images.cam_high. Those keys never match the policy features, so the image branch would silently stay un-normalised (a mid-grey image stays 0.5 instead of becoming roughly 0.066 after ImageNet normalisation).

The migrator was therefore followed by a pass that rewrites the state-file keys back to the real feature names declared in config.json, and by an assertion that normalisation actually applies:

image 0.5 -> ~0.066   (must NOT be ~0.5)

If you re-derive these files yourself, please re-run that assertion — a checkpoint that looks fine can score 0 % purely because of this key mismatch.

Files

File	Size	Role
model.safetensors	206,560,504 B	ACT weights (upstream payload)
config.json	2,117 B	ACT policy config, chunk_size=50, n_action_steps=50
policy_preprocessor.json	1,780 B	LeRobot 0.4.4 preprocessing pipeline
policy_preprocessor_step_3_normalizer_processor.safetensors	1,704 B	normalisation statistics
policy_postprocessor.json	661 B	LeRobot 0.4.4 postprocessing pipeline
policy_postprocessor_step_0_unnormalizer_processor.safetensors	1,704 B	de-normalisation statistics

Interface

Observation state	observation.state, shape [14] (obs["robot"]["qpos"])
Observation images	observation.images.cam_high, observation.images.cam_right_wrist, observation.images.cam_left_wrist, each [3, 480, 640], float32 in [0, 1]
Action	action, shape [14]: [left_arm(0-5), left_gripper(6), right_arm(7-12), right_gripper(13)]
Action type	absolute joint positions
Chunk size	50 predicted, 50 executed per inference (n_action_steps)

Usage

from lerobot.policies.act.modeling_act import ACTPolicy
from lerobot.policies.factory import make_pre_post_processors

policy = ACTPolicy.from_pretrained("<this-repo>")
pre, post = make_pre_post_processors(
    policy_cfg=policy.config,
    pretrained_path="<this-repo>",
    preprocessor_overrides={"device_processor": {"device": policy.config.device}},
    postprocessor_overrides={"device_processor": {"device": "cpu"}},
)
batch = {
    "observation.state": state,                 # [1, 14] float32
    "observation.images.cam_high": img_high,    # [1, 3, 480, 640] float32 in [0, 1]
    "observation.images.cam_right_wrist": img_rw,
    "observation.images.cam_left_wrist": img_lw,
}
action = post(policy.select_action(pre(batch)))  # [1, 14]

A ready-made adapter that handles both this 0.4.4 layout and the legacy b883328 layout is shipped in the submission repository alongside this card.

Evaluation protocol

• Task: handle_basket, setting random (domain randomisation on)
• Episode limit: 500 environment steps (configs/handle_basket/random/gym_config.json)
• Success criterion (official): the milk carton is inside the basket, the basket is lifted and displaced more than 0.15 m along +y, and all of that holds for 55 consecutive steps.

Limitations

• Simulation only. This checkpoint was trained on the released simulated handle_basket demonstrations; it is not a real-robot policy.
• It is not a stand-alone solution for the whole task. On its own it performs the grasp/place/carry sequence at the base rate of the released ACT baseline; the submitted hybrid policy adds an analytic, RGB-only grasp-and-place stage in front of it.
• Training data, seeds and hyper-parameters are those of the upstream released checkpoint; only the on-disk format was changed here.