SimpleTouch

Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?

Project Page · Paper · arXiv · Code

SimpleTouch adds a tactile expert to a pretrained π0.5 policy and learns contact-rich manipulation from task demonstrations, without additional tactile policy pretraining. A frozen T3 encoder represents two RGB tactile images, and the action expert reads the tactile expert's layerwise context. Future-touch prediction provides auxiliary supervision during training and is not run at inference.

Checkpoints

This repository provides task-specific 20k-step inference checkpoints for six UniVTAC tasks.

Task	Directory	Actions executed per policy query
Lift Bottle	lift_bottle	50
Pull-out Key	pull_out_key	50
Lift Can	lift_can	50
Put Bottle	put_bottle_in_shelf	50
Insert Hole	insert_hole	25
Insert Tube	insert_tube	25

These are the default H50/H25 execution horizons, not the number of flow-matching inference steps. Temporal ensembling is off. All six checkpoints use the Fast inference path.

Each task directory contains:

<task>/global_step_20000/
├── actor/model_state_dict/full_weights.pt
├── config.yaml
├── norm_stats.json
└── _SUCCESS

Use the SimpleTouch loader, which reads the configuration and normalization statistics and validates the inference bundle. Optimizer state and distributed training-resume files are not included.

Quick Evaluation

Follow the setup instructions for the policy environment, UniVTAC, T3, and tokenizer first. Run the following commands from the code repository root, with the policy environment active and those paths configured.

Single GPU:

export TASK=insert_hole
GPU_ID=0 bash scripts/simpletouch/evaluate_univtac.sh

The launcher automatically downloads and validates the selected checkpoint on first use. Change TASK to any directory name in the table above.

Multiple GPUs: split the same 100 evaluation episodes across GPUs and merge the results into one directory.

export TASK=insert_hole
SIMPLETOUCH_EVAL_GPUS=0,1,2,3 bash scripts/simpletouch/evaluate_univtac.sh

For a short single-GPU smoke test, prefix the command with TOTAL_NUM=3. To use an existing local download, also set CKPT_PATH=/path/to/insert_hole/global_step_20000.

Results are saved under results/UniVTAC/SimpleTac/<task>/<run-id> in the code repository. Set SIMPLETOUCH_EVAL_ROOT to use another output root.

Dependencies and Scope

• Obtain the T3-large Mini encoder and PaliGemma tokenizer using the instructions in the code repository.
• Install the UniVTAC simulation environment and assets separately. They are not included in these weight bundles.
• These checkpoints are intended for their corresponding UniVTAC tasks, not direct deployment on physical robots. Third-party models, datasets, and dependencies retain their respective licenses and access requirements.

Citation

@misc{yang2026simpletouchvisionlanguageactionmodelsmaster,
  title={SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?},
  author={Chen Yang and Linzhe Shi and Changjie Wu and Hang Zhang and Ronghan Chen and Lingjun Zhang and Xu Hu and Mu Xu and Jiansheng Fan and Chen Wang},
  year={2026},
  eprint={2610.02784},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2610.02784},
}