Compose Your Policies! Improving Diffusion-based or Flow-based Robot
  Policies via Test-time Distribution-level Composition
Authors: name: Jiahang Cao; name: Yize Huang; name: Hanzhong Guo; name: Rui Zhang; name: Mu Nan; name: Weijian Mai; name: Jiaxu Wang; name: Hao Cheng; name: Jingkai Sun; name: Gang Han; name: Wen Zhao; name: Qiang Zhang; name: Yijie Guo; name: Qihao Zheng; name: Chunfeng Song; name: Xiao Li; name: Ping Luo; name: Andrew F. Luo
Published: 2025-10-01 16:05:53

Abstract: Diffusion-based models for robotic control, including vision-language-action
(VLA) and vision-action (VA) policies, have demonstrated significant
capabilities. Yet their advancement is constrained by the high cost of
acquiring large-scale interaction datasets. This work introduces an alternative
paradigm for enhancing policy performance without additional model training.
Perhaps surprisingly, we demonstrate that the composed policies can exceed the
performance of either parent policy. Our contribution is threefold. First, we
establish a theoretical foundation showing that the convex composition of
distributional scores from multiple diffusion models can yield a superior
one-step functional objective compared to any individual score. A
Gr\"onwall-type bound is then used to show that this single-step improvement
propagates through entire generation trajectories, leading to systemic
performance gains. Second, motivated by these results, we propose General
Policy Composition (GPC), a training-free method that enhances performance by
combining the distributional scores of multiple pre-trained policies via a
convex combination and test-time search. GPC is versatile, allowing for the
plug-and-play composition of heterogeneous policies, including VA and VLA
models, as well as those based on diffusion or flow-matching, irrespective of
their input visual modalities. Third, we provide extensive empirical
validation. Experiments on Robomimic, PushT, and RoboTwin benchmarks, alongside
real-world robotic evaluations, confirm that GPC consistently improves
performance and adaptability across a diverse set of tasks. Further analysis of
alternative composition operators and weighting strategies offers insights into
the mechanisms underlying the success of GPC. These results establish GPC as a
simple yet effective method for improving control performance by leveraging
existing policies.