Do You Need Proprioceptive States in Visuomotor Policies?
Authors: name: Juntu Zhao, name: Wenbo Lu, name: Di Zhang, name: Yufeng Liu, name: Yushen Liang, name: Tianluo Zhang, name: Yifeng Cao, name: Junyuan Xie, name: Yingdong Hu, name: Shengjie Wang, name: Junliang Guo, name: Dequan Wang, name: Yang Gao

Imitation-learning-based visuomotor policies have been widely used in robot
manipulation, where both visual observations and proprioceptive states are
typically adopted together for precise control. However, in this study, we find
that this common practice makes the policy overly reliant on the proprioceptive
state input, which causes overfitting to the training trajectories and results
in poor spatial generalization. On the contrary, we propose the State-free
Policy, removing the proprioceptive state input and predicting actions only
conditioned on visual observations. The State-free Policy is built in the
relative end-effector action space, and should ensure the full task-relevant
visual observations, here provided by dual wide-angle wrist cameras. Empirical
results demonstrate that the State-free policy achieves significantly stronger
spatial generalization than the state-based policy: in real-world tasks such as
pick-and-place, challenging shirt-folding, and complex whole-body manipulation,
spanning multiple robot embodiments, the average success rate improves from 0\%
to 85\% in height generalization and from 6\% to 64\% in horizontal
generalization. Furthermore, they also show advantages in data efficiency and
cross-embodiment adaptation, enhancing their practicality for real-world
deployment.