MINT-SJTU/Evo-DexBench_model_access_example

Evo-DexBench Model Access Examples

Six inference checkpoint examples for Evo-DexBench, developed by MINT-SJTU. Use them to connect a policy server to the benchmark's simulator and reproduce the model integration workflow.

✨ Overview

• Three policies: DP-T, DP-C, and KineDex.
• Two checkpoints per policy: Single2 Core and Double3 Core.
• Four tasks per checkpoint: separate single-arm and bimanual policies.
• Independent model and simulation environments, connected through the benchmark's WebSocket + MessagePack Policy API.

These are task-family specialists, not universal policies for all 30 benchmark tasks. This repository contains model checkpoints, not simulation assets, demonstrations, or training code.

📦 Contents

The six checkpoints total approximately 5.80 GB (5.40 GiB).

Evo-DexBench_model_access_example/
├── DP-T/
│   ├── single2-core.ckpt
│   └── double3-core.ckpt
├── DP-C/
│   ├── single2-core.ckpt
│   └── double3-core.ckpt
├── KineDex/
│   ├── single2-core.ckpt
│   └── double3-core.ckpt
├── manifest.json
└── README.md

Policy	Architecture	Inputs	Single2 / Double3 size
DP-T	Official Diffusion Policy Transformer	RGB + robot state	0.171 / 0.171 GB
DP-C	Official Diffusion Policy convolutional U-Net	RGB + robot state	1.177 / 1.185 GB
KineDex	Official KineDex visual/tactile diffusion policy with the Evo-DexBench adapter	RGB + robot state + tactile	1.544 / 1.552 GB

Each checkpoint was trained jointly on its four Core tasks. The exported payload retains the original inference EMA weights, normalizers, model configuration, and policy metadata without changing tensor precision or values. EMA weights are stored under the inference model entry; duplicate weights, optimizer state, and training-progress state have been removed. These files are inference-only, not full training-resume checkpoints. They use the native policy-workspace .ckpt format, not a Transformers or Diffusers from_pretrained model layout.

The manifest records file sizes, SHA256 checksums, and observation contracts.

Supported tasks

Checkpoint group	Task ID	Task name
single2-core	m06_v1	FaucetHandle
	m07_v1	MouseWheel
	m08_v1	CoffeeKnob
	m09_v1	KettleLidTab
double3-core	b11_v1	LaptopClosing
	b12_v1	StaplerPaperMark
	b13_v1	ScissorsOpening
	b14_v1	BoxLidOpening

Observation and action contract

All checkpoints use 30 Hz control and two observation frames. Benchmark simulation runs at 120 Hz. Default receding-horizon execution applies eight actions before requesting the next chunk.

Group	RGB views, in order	State	DP motion action	KineDex action
Single2	context, interaction, wrist_right	31	30	30 motion + 5 force
Double3	interaction, wrist_left, wrist_right	62	60	60 motion + 10 force

RGB views are supplied as 224×224 model inputs. KineDex uses full tactile observations with five finger arrays per hand, padded to 29×1 each. Its client converts predicted finger forces into bounded joint-control residuals. DP-T and DP-C do not request tactile observations. These examples do not use depth inputs.

Double3 uses the interaction camera, not the default dual-arm context view. Select the camera options shown below when evaluating its checkpoints. DP retains the original 100-step DDPM inference; KineDex retains its checkpoint's DDIM settings. Do not change action timing or sensor contracts to compensate for a slower GPU.

🚀 Quick Start

Run setup and evaluation from an Evo-DexBench code checkout. DP checkpoints require a code revision containing examples/policies/diffusion_policy/ and the policy-dp setup profile. If these are absent, obtain a DP-enabled release before following the DP commands.

1. Prepare the simulator and assets

tools/setup_environment.sh --profile sim --name evodex-sim \
  --with-dp-client --with-kinedex-client
conda activate evodex-sim
hf auth login
dexbench assets download --asset-dir "$PWD/assets"
source assets/activate_evodex_assets.sh

Accept the terms and obtain access to the official PartNet-Mobility repository before configuring assets. See the asset repository for asset distribution and licensing details.

2. Download checkpoints

In the simulator environment:

hf download MINT-SJTU/Evo-DexBench_model_access_example \
  --repo-type model --local-dir checkpoints/model-access-example

To download only one policy, add, for example, --include 'DP-C/*.ckpt'. Only load checkpoints from trusted sources: these .ckpt files use Python pickle and must be loaded through the provided model server.

3. Run DP-C or DP-T

Create the independent model environment:

tools/setup_environment.sh --profile policy-dp --name evodex-dp

Terminal 1 — start a DP-C server:

conda activate evodex-dp
python -m dp_policy.server \
  --checkpoint checkpoints/model-access-example/DP-C/single2-core.ckpt \
  --upstream-root "$PWD/.third_party/DiffusionPolicy-official" \
  --host 127.0.0.1 --port 8100 --device cuda:0

Terminal 2 — run ten diagnostic episodes:

conda activate evodex-sim
source assets/activate_evodex_assets.sh
dexbench evaluate --policy dp_c --task-id m09_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --policy-kwargs '{"host":"127.0.0.1","port":8100,"embodiment":"single","execute_steps":8}' \
  --log results/dp_c_m09.jsonl

For DP-T, serve DP-T/single2-core.ckpt and select --policy dp_t in the client. Either single-arm checkpoint supports all four Single2 Core tasks.

For Double3, restart the server with the matching double3-core.ckpt, then use a supported bimanual task and the interaction view:

dexbench evaluate --policy dp_c --task-id b11_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --policy-kwargs '{"host":"127.0.0.1","port":8100,"embodiment":"dual","execute_steps":8,"camera_roles":["interaction","wrist_left","wrist_right"]}' \
  --log results/dp_c_b11.jsonl

For a DP-T Double3 checkpoint, use the same camera contract and --policy dp_t.

4. Run KineDex

Create its independent model environment:

tools/setup_environment.sh --profile policy-kinedex --name evodex-kinedex

Terminal 1 — start the model server:

conda activate evodex-kinedex
python -m kinedex_policy.server \
  --checkpoint checkpoints/model-access-example/KineDex/single2-core.ckpt \
  --upstream-root "$PWD/.third_party/KineDex-official" \
  --host 127.0.0.1 --port 8400 --device cuda:0

Terminal 2 — evaluate with full tactile inputs:

conda activate evodex-sim
source assets/activate_evodex_assets.sh
dexbench evaluate --policy kinedex --task-id m07_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --policy-kwargs '{"host":"127.0.0.1","port":8400,"embodiment":"single","execute_steps":8,"tactile_mode":"full"}' \
  --log results/kinedex_m07.jsonl

For Double3, restart the server with KineDex/double3-core.ckpt, then run:

dexbench evaluate --policy kinedex --task-id b11_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --policy-kwargs '{"host":"127.0.0.1","port":8400,"embodiment":"dual","execute_steps":8,"tactile_mode":"full","camera_profile":"interaction"}' \
  --log results/kinedex_b11.jsonl

The ten-episode commands are integration checks, not standard benchmark results. To run the standard 100-episode, four-worker protocol, omit --episodes and --num-envs. Use a new log filename for every invocation. GPU memory use includes both model inference and simulation rendering; a single-worker check is the starting point for memory-constrained machines.

Training and additional policy options are documented in the code repository's DP and KineDex examples. This model repository does not duplicate their code.

⚖️ License

The released checkpoints are provided under Apache License 2.0 by MINT-SJTU. Upstream policy implementations, simulation software, datasets, and assets retain their respective licenses. This repository does not redistribute PartNet-Mobility geometry.

🙏 Acknowledgement

We thank the authors of Diffusion Policy, KineDex, LeRobot, ManiSkill, and SAPIEN for their open-source tools. The adapters pin Diffusion Policy to 5ba07ac6661db573af695b419a7947ecb704690f and KineDex to 9246f77d7b969b181b0f1b600601b9a9c8083011.

📄 Citation

If you find Evo-DexBench useful in your research, please cite:

@misc{cheng2026evodexbenchbenchmarkinggeneralizationrelated,
      title={Evo-DexBench: Benchmarking Generalization across Related Tasks for Dexterous Manipulation},
      author={Bing Cheng and Yiran Mao and Zewei Ye and Yifan Jin and Xinyi Chen and Ruifeng Tu and Shuhuai Wang and Tao Lin and Zhimu Zhou and Yuxin Du and Fangwei Zhong and Junchi Yan and Bo Zhao},
      year={2026},
      eprint={2610.21180},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2610.21180},
}