flex-pi/flexpi-robodojo-real

Flex-π · RoboDojo_real (ARX X5, Piper, Piper X)

Three Flex-π checkpoints for the RoboDojo_real robots, one per robot. Each is fine-tuned from flex-pi/flexpi-pretrain-preview on that robot's RoboDojo_real demonstrations and is served through XPolicyLab's FlexPi policy adapter. The model reads three RGB cameras plus the joint and end-effector state of both arms, and returns 32-step action chunks. It trains on RGB + DINO modalities and reads no depth.

Checkpoints

folder (env_cfg_type)	weights	tasks	demonstrations (frames)	control rate
arx_x5/	step_103840.pt	cover_blocks, insert_tubes, make_bread, make_food, pack_and_pour_fruit, store_in_safe	599 (664,412)	30 Hz
piper/	step_083600.pt	fill_pen_holder, insert_charger, put_objects_into_basket, stack_and_cover_blocks, stack_bowls, stand_up_bottles	595 (534,792)	30 Hz
piper_x/	step_063520.pt	cap_pen, classify_objects, disassemble_LEGO, hang_mugs, pack_objects_into_backpack, sweep_blocks	600 (406,433)	25 Hz

Each folder holds three files:

• step_N.pt: the weights, bf16, 12,082,213,237 bytes.
• config.yaml: the architecture and processor, which the adapter reads at boot. The file is the same in all three folders.
• dataset_stats.json: that robot's normalization statistics.

folder	sha256 of step_N.pt	sha256 of dataset_stats.json
arx_x5/	76d550d0e2c6dca77943f2d463dfb7a0fde8f31be44a31b344fb6e9a165e6d78	b485348493742015e3df25abcf9e93e41d956c045bad10e8b94e0fc4e8cb1647
piper/	ede12798d845cbbf526c4ed8af344bf3e08fc2b92f0665e35e0f062d75a9119d	7a769693984869eafc911e331b93cd8663036c07da38e3eef4eac81fd12a3398
piper_x/	ac986f0aa590f31a596f37e6cc98184bc9667b37da522df27a52d38330ca95d0	e82bedf0de999b025b2b9fe188af8a97b2364fb9442bde13d57f67630a9367c5

Model

Architecture	Mixture-of-Transformers: Wan2.2-TI2V-5B video expert (5.00 B) + ActionDiT action expert (1.02 B), 30 layers
Base	flex-pi/flexpi-pretrain-preview, step 190000
Modalities	RGB + DINO; the pointmap stream is off
Cameras	head camera at 256×320 and two wrist cameras at 224×224, resized from 480×640 frames; composited at 384×320, 33 frames
Actions	32-step chunks. Each arm has an end-effector pose relative to the chunk's first state (in the tool frame below), joint deltas and an absolute gripper value. The adapter returns absolute targets.
Normalization	per robot, frame-pooled q01/q99 (dataset_stats.json)
Fine-tuning	the robot's own demonstrations; global batch 256; 40 epochs (two 20-epoch cosine cycles); colour jitter (brightness 0.3, contrast 0.4, saturation 0.5)

Serving

The serving code is XPolicyLab's FlexPi adapter, from its pull request: branch flexpi-robodojo-real of github.com/YOLO12138/XPolicyLab until it is merged (checked at commit 1f0eb990: with these folders its answers are bit-identical to the offline checks below). The adapter is policy/FlexPi; its README.md section RoboDojo_real checkpoints is the full deploy reference.

Requirements

• Code: that XPolicyLab as the XPolicyLab/ directory of a RoboDojo checkout; the server reads env_cfg/ from the parent of XPolicyLab/. bash XPolicyLab/policy/FlexPi/install.sh flexpi creates the conda environment (python 3.10, torch 2.7.1+cu128).

• GPU: about 16 GB of memory per server, with the T5 text encoder on the CPU.

• Base files under policy/FlexPi/FlexPi/checkpoints/, the default of DIFFSYNTH_MODEL_BASE_PATH: the Wan2.2 VAE, the umt5-xxl text encoder and its tokenizer (about 12.8 GB, from ModelScope). config.yaml sets skip_dit_load_from_pretrain: true, and the checkpoint carries every fine-tuned weight, so the server does not read the Wan2.2-TI2V-5B DiT or the ActionDiT init file.

  cd <RoboDojo root>/XPolicyLab/policy/FlexPi && conda activate flexpi
  export DIFFSYNTH_MODEL_BASE_PATH="$(pwd)/FlexPi/checkpoints"
  python - <<'EOF'
  from flexpi.models.helpers.io import ModelConfig
  for repo, pattern in [
      ("DiffSynth-Studio/Wan-Series-Converted-Safetensors", "Wan2.2_VAE.safetensors"),
      ("DiffSynth-Studio/Wan-Series-Converted-Safetensors", "models_t5_umt5-xxl-enc-bf16.safetensors"),
      ("Wan-AI/Wan2.1-T2V-1.3B", "google/umt5-xxl/"),
  ]:
      ModelConfig(model_id=repo, origin_file_pattern=pattern).download_if_necessary()
  EOF
  conda deactivate
  

• DINOv3 timm/vit_base_patch16_dinov3.lvd1689m, fetched from the Hugging Face hub at boot. An offline host needs it in the local HF cache.

• env_cfg/<env_cfg_type>.yml in the RoboDojo root. Its config.robot names an entry of env_cfg/robot/_robot_info.json with arm_dim [6, 6] and ee_dim [1, 1]. RoboDojo's open-source env_cfg has arx_x5.yml only; without a piper.yml or piper_x.yml, the server stops at init with FileNotFoundError.

Download

cd <RoboDojo root>
git -C XPolicyLab fetch https://github.com/YOLO12138/XPolicyLab flexpi-robodojo-real && git -C XPolicyLab checkout 1f0eb990
cd XPolicyLab/policy/FlexPi
hf download flex-pi/flexpi-robodojo-real --include "arx_x5/*" --local-dir checkpoints/flexpi-robodojo-real

hf comes with huggingface_hub 0.34 or later; inside the flexpi environment (huggingface_hub 0.29.2) the same command is huggingface-cli download. --local-dir writes regular files. The symlinked layout of the HF cache does not serve these checkpoints; the launcher refuses it at start.

Launch

One server per robot, each from a shell at the conda base environment:

cd <RoboDojo root>/XPolicyLab/policy/FlexPi
bash setup_eval_policy_server.sh RoboDojo_real <task_name> checkpoints/flexpi-robodojo-real/arx_x5 \
    arx_x5 joint 0 <gpu_id> flexpi <port> 0.0.0.0 2>&1 | tee flexpi_server_arx_x5.log

The positional arguments are bench_name task_name ckpt_name env_cfg_type action_type seed gpu_id conda_env port host; task_name only labels the run, and env_cfg_type names the folder's robot. The launcher activates conda_env itself, under set -u; in a shell where that environment is already active, a conda deactivate hook that reads an unset variable can stop it right after its [SERVER] lines.

The PR's deploy.yml ships the settings these checkpoints need:

setting	value
eef_frame	relabeled
use_depth, infer_present_pointmap, infer_joint_pointmap	false
num_inference_steps, replan_steps	10, 32
action_dim_mask	false
use_per_cam, obs_train_parity, offload_text_encoder	true
speed switches (torch_compile, compile_encoders, glue_cache, encoder_cuda_graph, joint_loop_cuda_graph)	false

The server stops at boot when the folder's statistics name another robot than env_cfg_type, or when eef_frame is not relabeled. A correct start logs these lines:

• [SERVER] trained config: .../flexpi-robodojo-real/arx_x5/config.yaml
• [FlexPi] stats coverage robodojo_realworld_34d_relabeled_arx_x5: eef_frame=relabeled env_cfg_type=arx_x5 match
• [FlexPi] Loaded model + processor from trained config: .../arx_x5/config.yaml
• [FlexPi] ready: ... needs_depth=False state_bridge=yam32 eef_frame=relabeled

The station's client also needs policy/FlexPi: it reads its deploy.yml and runs its episode loop, deploy.py.

Robot side

Observation, every control tick

key	shape	content
state.left_arm_joint_state, state.right_arm_joint_state	(6,)	joint angles in rad, in the recorded order and sign
state.left_ee_joint_state, state.right_ee_joint_state	(1,)	gripper reading on the recorded scale: about 0.9 open, about 0 closed
state.left_ee_pose, state.right_ee_pose	(7,)	[x, y, z, qw, qx, qy, qz]: metres, scalar-first quaternion, in the recorded convention (below)
vision.cam_head.color, vision.cam_left_wrist.color, vision.cam_right_wrist.color	(480, 640, 3) uint8, or one encoded JPEG	RGB
instruction	str	the task's training instruction (below)

• Left and right are the robot's own left and right; the left arm appears in the left half of the head image.
• The EE pose is each arm's link6 (flange) point in that arm's own frame, as the RoboDojo_real recordings store it. There is no shared world frame.
  • arx_x5: the origin is that arm's link6 position at zero joints, and the axes are parallel to the arm base (x forward, y left, z up).
  • piper, piper_x: the arm's base frame (x forward, y left, z up).
• The adapter turns the recorded orientation into the model's tool frame itself (eef_frame: relabeled), so the pose is sent unconverted.
• The quaternion is normalized and turned into a rotation matrix, so its sign does not matter.
• The state's positions, joints and grippers are clamped to their trained q01..q99 range before the model sees them, so an out-of-range reading raises no error.
• The recordings start with all joints near 0 and both grippers open (97–100 % of episodes). At that pose a robot whose conventions match the recordings reads about these values; the quaternion's overall sign is free:

robot	arm	ee_pose position (m)	ee_pose quaternion (w, x, y, z)	gripper
arx_x5	left	(0.000, 0.000, 0.001)	(1.000, -0.008, 0.000, -0.001)	0.909
arx_x5	right	(0.000, 0.000, 0.001)	(1.000, 0.010, 0.001, 0.000)	0.891
piper	left	(0.056, 0.000, 0.213)	(0.733, 0.000, 0.680, 0.000)	0.927
piper	right	(0.057, 0.000, 0.212)	(0.731, -0.002, 0.682, 0.002)	0.906
piper_x	left	(0.098, 0.000, 0.211)	(0.516, -0.483, 0.484, -0.516)	0.912
piper_x	right	(0.098, 0.000, 0.212)	(0.516, -0.483, 0.482, -0.517)	0.913

Output

Each call returns 32 targets, one per control tick. Element k is the target for tick t+k+1 after the observation at tick t. XPolicyLab's client loop executes all 32 before the next call.

• joint:
  • left_arm_joint_state, right_arm_joint_state (6): absolute joint angles in rad, the current joints plus the predicted deltas. The adapter applies no joint-limit or velocity clipping.
  • left_ee_joint_state, right_ee_joint_state (1): the grippers.
• ee:
  • left_ee_pose, right_ee_pose (7): [x, y, z, qw, qx, qy, qz] in the same recorded per-arm frame as the observation, with qw ≥ 0. The point is the link6 flange, not a fingertip.
  • The same two gripper keys.
• Both action types decode one network output.
• The gripper target is the predicted gripper reading of the next tick, on the recorded scale, clipped to [0, 1] (the Pipers' recorded readings reach -0.04). The recordings carry no separate command: each recorded action equals the next recorded state.

Frames

The model's tool frame has z along the approach axis (wrist to fingers), y along the finger-opening axis and x pointing away from the wrist camera. At the start pose this is z forward, y left, x down on all three robots, and wrist roll turns about z. The adapter maps the recorded orientation R onto it with one constant rotation P per robot, R_model = R P; ee outputs go back through R P^T. Positions, joints and grippers pass through unchanged.

robot	recorded tool axes at the start pose	P
arx_x5	x forward (approach), y left, z up	Ry(+90°) = [[0,0,1],[0,1,0],[-1,0,0]]
piper	x down, y left, z forward (approach)	identity
piper_x	x right, y down, z forward (approach)	Rz(+90°) = [[0,-1,0],[1,0,0],[0,0,1]]

XPolicyLab's _EEF_REFRAME_P_ROBODOJO_REAL, used by other policies, is a different table: P followed by a 180° roll. A pose that arrives with P already applied gets it twice: arx_x5 and piper_x then sit 90° off (piper's P is the identity), and nothing raises.

Cameras

• cam_head, cam_left_wrist and cam_right_wrist are all required, as RGB uint8.
• The recordings are 480×640 JPEGs in XPolicyLab's legacy channel-reversed byte format. decode_image_bit returns them as RGB, which is what the model trained on.
• A non-uint8 array is cast to uint8, so a float image in [0, 1] arrives black.
• The adapter resizes as training did, 480×640 to 256×320, then to 224×224 for the wrist cameras. Another aspect ratio is distorted differently from training.
• In the recordings each image trails the state by about 2 frames: about 1.5–1.7 frames for the wrist cameras, and about 1.9 (arx_x5) or 2.6–2.8 (Pipers) for the head camera. The model trained on that pairing.

Instructions

The instruction goes in alone. The adapter wraps it in the training template. A missing instruction becomes complete the task, and a bytes value becomes the text b'...'; neither prompt was trained.

robot	task	instruction
arx_x5	cover_blocks	Cover the three blocks from left to right, then uncover them again in the order of red, green, and blue.
arx_x5	insert_tubes	Pick up the test tubes on the table and insert them into the test tube rack.
arx_x5	make_bread	Pick up the two slices of bread from the bowl and place them into the toaster, then place the two small bowls on the plate.
arx_x5	make_food	Take the cutting board out and place it on the table. Then place the steak and the knife on the cutting board, put the vegetables and shrimp into the pot, place the pot on the stove, and finally cover it with the lid.
arx_x5	pack_and_pour_fruit	Place all the fruits into the blue bowl, then pour the fruits from the blue bowl into the large white bowl.
arx_x5	store_in_safe	Place all the objects into the safe, then close the safe.
piper	fill_pen_holder	Pick up the pen holder and place all the pens into it.
piper	insert_charger	Insert the charger plug into the power strip, then connect the charging cable to the plug.
piper	put_objects_into_basket	Place all the objects on the table into the basket.
piper	stack_and_cover_blocks	Stack the blocks on the table, then cover them with the cup.
piper	stack_bowls	Stack the bowls on the table.
piper	stand_up_bottles	Stand the bottle upright.
piper_x	cap_pen	Pick up the pen and the pen cap, then put the cap on the pen.
piper_x	classify_objects	Classify the two types of objects on the table and place them into the two baskets.
piper_x	disassemble_LEGO	Take apart the assembled LEGO structure and place all the pieces into the bowl.
piper_x	hang_mugs	Hang the mugs on the mug rack.
piper_x	pack_objects_into_backpack	Place all the objects on the table into the backpack.
piper_x	sweep_blocks	Pick up the broom, hand it over to the right hand, then use the dustpan to sweep the blocks.

Timing

• Control rate: 30 Hz (arx_x5), 30 Hz (piper), 25 Hz (piper_x), the recorded rates. A 32-step chunk spans 1.07 / 1.07 / 1.28 s. The model has no rate input, so at another rate the learned motions play back faster or slower.
• Latency on one A100 (bf16, 10 denoising steps): about 1.0 s per chunk. The first call after a reset or an instruction change takes about 25 s more, for the umt5-xxl encode on the CPU; the encoded prompt is cached until the next reset.
• Boot: about 2–3 minutes.

Validation

All checks ran offline through the FlexPi adapter with the settings above; the PR at 1f0eb990 reproduces them bit for bit.

• Inputs: the adapter's state built from the raw recorded values equals the training state exactly, on every training frame. Its image inputs are identical to the training inputs.
• Output decoding: the joint and ee paths turn the recorded actions back into the recorded targets within 1.2e-7 rad and 3.2e-8 m.
• Weights: every transformer weight loads bit-exactly. With only the VAE, T5 and tokenizer as base files, these folders give outputs bit-identical to the training run's own config served with every base file (one window per robot, CPU, fp32).
• Serving: downloaded with the command above and started with the launch command, the server's answer over XPolicyLab's websocket client equals the direct replay bit for bit (arx_x5, CPU, fp32).
• Replay: 24 recorded windows per robot (4 per task) on an A100 in bf16, each 32-step chunk compared with the recorded future. "Hold" repeats the current state.

robot	joint MAE (rad)	hold	gripper MAE	hold	EE position (mm)	hold	EE rotation (°)	hold
arx_x5	0.0068	0.0735	0.0020	0.0171	4.9	37.6	0.59	8.5
piper	0.0066	0.0790	0.0034	0.0510	4.1	36.5	0.59	9.1
piper_x	0.0156	0.0849	0.0066	0.0689	7.7	45.7	1.59	10.2

The replayed windows come from the training demonstrations, since every recording was used for training. Behaviour on the real robots is untested.

License

The weights are released under the MIT license. The fine-tuning data, RoboDojo, is Apache-2.0; the base model's training data carries the terms listed on the flexpi-pretrain-preview card. At inference the model downloads DINOv3 timm/vit_base_patch16_dinov3.lvd1689m, which is under the DINOv3 License, and Wan2.2 components, which are Apache-2.0.

Citation

@article{yan2026flexpi,
  title   = {Flex-$\pi$: A Multi-Stream World-Action Model with Compute Flexibility},
  author  = {Yan, Ge and Liu, Jinghao and Fan, Yuzhi and Cai, Lei and Liao, Minwen
             and Zhang, Jesse and Fox, Dieter},
  journal = {arXiv preprint arXiv:2608.10860},
  year    = {2026},
  url     = {https://arxiv.org/abs/2608.10860}
}