leejaehot/aiworker-towel-act-v1

AI Worker towel — ACT, final step 100000

Final checkpoint from the AI Worker towel sorting experiment. This repository contains full model weights, the matching pre/postprocessors and normalization tensors, and a hardware-free inference example.

Task: Place the white towel in the white box on the left and the blue towel in the blue box on the right.

Training uses episodes 0–89; episodes 90–100 are held out. Seed 1000, batch 8, action chunk 100. W&B run. These are final checkpoints, not checkpoints selected for best real-world success.

Compatibility

• Recorded robot type: ffw_bg2_rev4.
• 19 state values and 19 native action values, in the order in robot_spec.json.
• Four RGB views: left/right head (376×672) and left/right wrist (640×480), CHW RGB.
• Recorded rate: 30 Hz. Actual inference/control timing must be measured on the target computer.
• Keep the saved normalization tensors with the model; do not recompute them on the target robot.
• Same-model hardware requires matching camera viewpoints, joint order, signs, units, calibration, and action semantics.
• A robot of a different embodiment requires a compatible observation/action adapter and may require new demonstrations and fine-tuning. Uploading a checkpoint alone does not establish cross-robot performance.
• No robot driver, motor commands, closed-loop deployment, or physical success evaluation is included.

The complete ACT weights include the ResNet18 backbone; no separate ImageNet weight download is required.

Runtime

Validated training stack: LeRobot 0.6.0, source commit 30da8e687a6dfc617fcd94afc367ac7071c376ce. Exact package versions are in runtime_versions.json. Use the corresponding LeRobot checkout rather than assuming the latest release is compatible.

git clone https://github.com/huggingface/lerobot.git
cd lerobot
git checkout 30da8e687a6dfc617fcd94afc367ac7071c376ce
# Install LeRobot and its policy dependencies in a dedicated environment.
python -m pip install -e '.[pi,dataset]'
hf auth login
hf download leejaehot/aiworker-towel-act-v1 --repo-type model --local-dir ./model

Predict on a recorded observation

With the model files downloaded and a local copy of the towel dataset available:

python ./model/infer_checkpoint.py \
  --model ./model \
  --dataset-root /path/to/aiworker_towel \
  --frame-index 0 \
  --device cuda \
  --output ./predicted_actions.json

The example loads weights strictly, restores the saved processors, reads a recorded observation, and returns an unnormalized action chunk. It never connects to a robot. The included inference_validation.json records the successful strict-load and finite-action check. This is a recorded-observation check, not a physical task-success evaluation.

Using the same AI Worker

The official Cyclo inference workflow supports ACT and Pi0.5. Download this entire repository into a local model directory visible inside the policy backend container. In Cyclo's inference page, choose the policy family and set Policy Path to that directory containing config.json, model.safetensors, and the saved processor files. For Pi0.5, enter the task instruction given above. Use 3D Sim Deploy to inspect the first action chunk before physical deployment.

The official ffw_bg2_rev4 robot configuration matches this dataset's 19 state/action names and their order, and all four camera names. Retain the recording setup's image orientation, calibration, units, and action semantics. The official configuration includes 270-degree wrist-camera rotation; avoid applying that rotation twice.

Prefer a downloaded local folder over entering a remote model ID: the inspected Cyclo loader determines policy type from the local config.json. A compatible Cyclo backend provides the robot integration; a new custom adapter is not inherently required for this same robot. The actual robot computer's installed Cyclo/LeRobot versions have not been checked, and must support the checkpoint and saved processors. The validated standalone runtime is recorded above.

The two model families optimize different losses; their raw loss values are not directly comparable. Use matching deployment conditions and task success criteria for behavioral comparisons.