N0-VTLA - UniVTAC, eight tasks, one policy
Task policy for N0-VTLA, a vision-tactile-language-action model that conditions a flow-matching action expert on predicted latent tactile tokens.
This is a task policy, not a pretrained base. For post-training on your own robot start from n0-vtla-base.
| Config | sim_single_arm_tactile |
| Tactile pathway | enabled, n_latent=5, views (tactile_a, tactile_b) |
| Action space | 8-dim joint |
A single joint policy covering all eight UniVTAC tasks.
| Task | First attempt | One retry | Two retries |
|---|---|---|---|
| Grasp Classify | 100% | 100% | 100% |
| Insert Hole | 100% | 100% | 100% |
| Insert Tube | 95% | 95% | 95% |
| Pull-out Key | 95% | 95% | 100% |
| Lift Bottle | 75% | 100% | 100% |
| Lift Can | 75% | 85% | 90% |
| Put Bottle in Shelf | 65% | 95% | 95% |
| Insert HDMI | 55% | 65% | 65% |
| Mean | 82.5% | 91.9% | 93.1% |
Inference noise is unseeded, so retries are genuinely independent draws rather than replays.
Serving
This checkpoint requires the zero-contact tactile baseline. Three of the eight tasks close
the gripper inside pre_move, so episode frame 0 already carries the object's imprint and the
tactile difference measures zero for the whole episode. The reference images ship in this repo
under assets/tactile_baseline/.
VTLA_ASSET_ID=univtac_single8_joint_norm \
VTLA_BLANK_BASELINE_DIR=assets/tactile_baseline \
python scripts/serve_zmq.py --config sim_single_arm_tactile \
--ckpt <this-dir> --addr "tcp://127.0.0.1:5557"
Confirm it took effect: the serving log must print
using FIXED blank tactile baseline from <dir>. Setting it only on the training side is not
enough.
action_horizon is 50; set exec_horizon: 50 in the deploy YAML.
Prompts
| Task | Prompt |
|---|---|
insert_hole |
insert hole |
insert_HDMI |
insert HDMI |
insert_tube |
Insert the tube into the slot |
grasp_classify |
grasp classify |
lift_can |
lift can |
lift_bottle |
Lift the bottle |
pull_out_key |
Rotate and pull out the key |
put_bottle_in_shelf |
put bottle in shelf |
Copy them exactly. The capitalisation is inconsistent because the strings come from the datasets; a prompt that merely reads correctly to a human has moved a score by up to 35 points.
Evaluation protocol
Measured on the UniVTAC simulator at commit 695a22d
(branch NeoSim of anlorla/UniVTAC), on held-out seeds
starting at 100. Success criteria on three tasks were tightened after these numbers were
measured, so a success rate on this benchmark is not comparable without the simulator commit
beside it; see
docs/EVAL.md for the details and for
the full evaluation procedure.
Caveat on the tactile pathway
This checkpoint carries the tactile pathway, but this benchmark cannot demonstrate that touch
contributes to the score. Object randomisation is +/-2-5 mm with no domain randomisation, so a
policy that ignores its cameras and its tactile sensors entirely can still score well. Use
scripts/probe_z_tactile_dependence.py to measure the causal contribution yourself.
License
CC BY-SA 4.0, as the parent repository.