ZibinDong/ActionCodec2-1st-order

ActionCodec2 First-Order Action Tokenizer

This repository contains a pretrained first-order ActionCodec2 tokenizer for continuous robot actions.

It maps an action trajectory to integer tokens and decodes tokens back to an approximate trajectory. It is intended for training or running autoregressive robot policies and VLA models.

This repository contains a complete tokenizer artifact, not a neural-network checkpoint. Keep every file and directory together when downloading or uploading it.

Hugging Face repository: ZibinDong/ActionCodec2-1st-order

Install

pip install numpy scipy torch "transformers>=4.57,<5" huggingface-hub pyyaml

The artifact includes its own runtime, so you do not need to import the source repository when loading it with trust_remote_code=True.

Quick start

Load this artifact from the Hub:

import numpy as np
from transformers import AutoProcessor

codec = AutoProcessor.from_pretrained(
    "ZibinDong/ActionCodec2-1st-order",
    trust_remote_code=True,
)

# Synthetic episode for a copy-paste check. Replace with your own (T, 7) data.
actions = np.zeros((12, 7), dtype=np.float32)
actions[:, 6] = 1.0  # open gripper

tokens = codec.encode(actions, fps=15)
decoded_actions = codec.decode(tokens, fps=15)

For Transformers training code:

features = codec(actions, fps=15)
tokens = features["input_ids"]

actions may also be a regular batch with shape (B, T, D). A batch must have one common T; encode variable-length episodes separately. decode returns a CPU torch.float32 tensor with shape (B, T_out, D).

Default action layout

The saved artifact is bound to single_eef_delta:

Columns	Meaning	Units
0:3	End-effector position delta (x, y, z)	metres per step
3:6	End-effector rotation increment as a rotation vector	radians
6	Gripper command	open when >= 0.8, otherwise closed

The rotation increment is body-frame and follows R_next = R_previous @ Exp(rotvec). The first six columns are per-step increments, not velocities. The gripper column is an absolute open/closed command.

Use physical values in the registered units. Do not pass generic normalized [-1, 1] features unless they have first been converted to the action-space contract.

Other registered layouts

The artifact also contains joint, dual-arm, absolute, and delta layouts. Inspect the candidates for your action dimension:

codec.print_action_spaces(action_dim=7)

Select another compatible layout explicitly:

codec = codec.for_action_space("single_joint6_delta")
tokens = codec.encode(joint_actions, fps=15)

The action shape alone is not enough to select a layout. Confirm column order, units, rotation convention, gripper convention, and recording rate from your controller or dataset.

For absolute layouts, pass the physical state immediately before the first action to both calls:

tokens = codec.encode(actions, fps=15, current_state=current_state)
decoded = codec.decode(tokens, fps=15, current_state=current_state)

current_state has shape (B, D), including when actions is a single (T, D) episode.

Sampling rate and reconstruction

The fitted artifact has a 15 Hz target clock. Pass the actual source rate with fps=...; for example, fps=30 for data recorded at 30 Hz. The codec resamples to its target clock.

Physical quantization and time resampling are lossy. The decoded trajectory is an approximation, and its number of steps can differ from the input.

Fit your own tokenizer

For a different physical action representation, install the ActionCodec2 source repository and fit a new codec:

from actioncodec2 import ActionCodec2

codec = ActionCodec2(action_space="single_eef_delta")
codec.fit(episodes, fps=15, backend="auto")
codec.save_pretrained("./my-actioncodec2")

The saved directory is another complete artifact that can be loaded with AutoProcessor.from_pretrained(..., trust_remote_code=True).

Artifact contents

• router_config.yaml: fitted profiles, token offsets, budgets, and action-space presets
• profiles/joint/: fitted joint profile and vocabulary
• profiles/eef/: fitted end-effector profile and vocabulary
• runtime/: private runtime used by the Hugging Face loader
• processing_actioncodec2.py: lightweight Hugging Face entry point
• config.json and processor_config.json: Transformers metadata

Model details

• Primitive order: 1
• Selected action space: single_eef_delta
• Codec rate: 15 Hz
• Fitted profiles: joint, eef
• Token budget: 4096 per profile
• Registered layouts: 13