roboterax/Era0-5B-Preview-RoboDojo

Era0-Preview

Era0-Preview is a vision-language-action policy for RoboDojo, using a Qwen3.5 visual-language backbone and a prefix-KV action expert. It supports three RGB cameras, Cartesian end-effector actions and vectorized simulation with arx_x5.

Evaluation code: XPolicyLab / Era0_Preview · Detailed guide: policy README.

Use the Git repository directly. Evaluation code is not distributed through a model-repository archive.

Model weights, tokenizer/processor files and the exact norm_stats.pkl are provided in this repository. Evaluation code is maintained in the XPolicyLab repository linked above.

Local reference results

The table below reproduces the Overview of a local RoboDojo evaluation across seeds 0, 1 and 2, using native episode targets for 54 task variants / 42 merged tasks. Score is the primary metric; values are mean ± standard deviation across seeds, in percentage points.

Capability	Score	Success rate (%)
Generalization	34.99 ± 1.31	28.89 ± 1.29
Precision	26.50 ± 0.31	19.67 ± 0.42
Long-Horizon	48.93 ± 0.35	37.17 ± 0.96
Memory	42.66 ± 1.25	42.22 ± 1.23
Open	12.70 ± 0.20	11.83 ± 0.31
Overall	33.16 ± 0.35	27.96 ± 0.47

Coverage and aggregation. The local report contains 6,225 of 6,300 planned episodes (98.81%), with 1,716 successful episodes. Missing episodes are make_toast_random (10 for seed 0 and 25 each for seeds 1 and 2) and store_laptop_and_headphones_random (15 for seed 2). Overview metrics use all available episodes without a minimum per-task count and apply the report's task/capability averaging; overall success rate is therefore not the pooled ratio of successes to episodes. These are incomplete local reference results, rather than an official leaderboard evaluation.

Evaluation settings. The local campaign used arx_x5, Cartesian end-effector actions, image history 8/1/1 at interval 40, 10 denoising steps, and 10 executed actions from each 25-action chunk starting at index 1. Local IK protection was enabled. It combines reused results from an older SDK with additional episodes run using legacy single-render timing on a newer SDK; the additional runs kept that SDK's physics unchanged. Consequently, this table does not establish performance for the default policy-side image-delay / official-execution commands below, or for a uniform SDK revision. Official evaluation should report its own results under its selected SDK and execution protocol.

1. Install RoboDojo and the policy

Follow the official RoboDojo installation guide for Isaac Sim, the simulator environment and assets. Keep the simulator and policy Python environments separate.

Place the linked XPolicyLab fork at RoboDojo/XPolicyLab, using branch feature/era0_preview. For a fresh XPolicyLab checkout inside an installed RoboDojo workspace:

cd /path/to/RoboDojo
git clone --branch feature/era0_preview https://github.com/Zheng-Liming/XPolicyLab.git XPolicyLab
cd XPolicyLab/policy/Era0_Preview
bash install.sh

If XPolicyLab already exists, use the linked branch in that checkout while preserving any local changes. The policy installation creates .venv; it requires Python 3.11 and the CUDA/FlashAttention dependencies listed in the policy directory. It does not install the simulator.

2. Model files

The adapter requires a checkpoint directory containing at least:

checkpoints/era0-preview/
  config.json
  hy_xvla_adapter.json
  model.safetensors
  norm_stats.pkl
  tokenizer.json
  tokenizer_config.json
  processor_config.json
  chat_template.jinja

Keep all tokenizer/processor assets supplied with the checkpoint. The exact normalization file must accompany the weights. Expected normalization SHA256:

f0d1a21fc63e778ef5d75799073a719470b507f96f3822a905b4195ed755b285

Download model assets from this policy directory using:

bash download_checkpoint.sh

The downloader uses this repository's main revision and excludes code archives. Alternatively, pass an existing compatible checkpoint directory directly to the evaluator. This custom VLA checkpoint is loaded through the XPolicyLab adapter, rather than generic text-generation AutoModel loading.

3. Default image timing

Evaluation defaults to policy-side image delay by one observation update (HY_OBS_LAG=1, driver --obs-lag 1) and SDK rendering (ROBODOJO_EVAL_LEGACY_RENDER=0, driver --render-mode sdk).

• All three camera images use the previous observation update.
• State, language and the relative-action reference pose remain current.
• Queues are independent for each simulator environment and cleared on reset; the first image is repeated until an earlier update exists.
• This changes policy inputs only. It does not inject legacy rendering or change SDK physics, rewards, task layouts, normalization or action encoding.

To disable image delay, set HY_OBS_LAG=0 on the policy server or pass --obs-lag 0 to the three-seed driver. SDK rendering remains the default. A positive image lag cannot be combined with legacy rendering.

Execution and image timing are separate: the adapter's default execution profile retains local IK protection. The commands below explicitly select official, which uses the audited SDK execution path and disables that local protection. Both profiles record their settings and leave installed SDK files unchanged. See SDK compatibility for supported SDK versions and execution details.

4. One task / one seed

Run from RoboDojo/XPolicyLab/policy/Era0_Preview. This example uses GPU 0 and ten trajectories:

EVAL_NUM=10 ROBODOJO_NUM_ENVS=10 ERA0_EXECUTION_PROFILE=official \
  bash eval.sh RoboDojo press_by_number \
  era0-preview arx_x5 ee 0 0 0 .venv RoboDojo

The last two arguments are the policy environment and simulator environment; replace RoboDojo with your simulator environment name or Python executable. Image lag 1 and SDK rendering are already the defaults.

5. Complete native evaluation: three seeds

Preview the evaluation plan without starting simulation:

POLICY_ENV=.venv bash eval_three_seeds.sh \
  --checkpoint "$(pwd)/checkpoints/era0-preview" \
  --sim-env RoboDojo --gpu-ids 0,1,2,3,4,5,6,7 \
  --execution-profile official \
  --output /path/to/fresh/era0-evaluation

Append --run to execute the plan. The evaluator uses all runnable tasks from the installed SDK, seeds 0, 1 and 2, and each task's native episode count. No static task list is required. Counts and task availability are resolved from the selected SDK rather than a code archive. --output must be a fresh directory for launch logs and protocol metadata only. Result JSON and videos remain in the SDK-native eval_result/RoboDojo/<task>/Era0_Preview/arx_x5/<seed>_ckpt_name=<name>,action_type=ee/<run_id>/ directory. Use an actual XPolicyLab checkout inside the SDK workspace, rather than an external-checkout symlink.

The plan records checkpoint/normalization fingerprints, task inventory, image lag, rendering mode and execution profile. Check the driver's evaluation_summary.json for execution completeness. Score aggregation uses the installed SDK's scripts/internal/summarize_result.py; Overview / Avg Score in eval_result/RoboDojo/_summary.md is the primary aggregate metric. From the SDK root, run bash scripts/robodojo.sh summarize to regenerate that official report. Video paths and filenames stay identical to other policies; no policy-local score summarizer is used. A single-task check does not establish full-benchmark performance.

For the official CLI and split-machine deployment, see the policy evaluation guide and RoboDojo evaluation documentation.