MUSAR: Exploring Multi-Subject Customization from Single-Subject Dataset
  via Attention Routing
Authors: name: Zinan Guo; name: Pengze Zhang; name: Yanze Wu; name: Chong Mou; name: Songtao Zhao; name: Qian He
Published: 2025-05-05 17:50:24

Abstract: Current multi-subject customization approaches encounter two critical
challenges: the difficulty in acquiring diverse multi-subject training data,
and attribute entanglement across different subjects. To bridge these gaps, we
propose MUSAR - a simple yet effective framework to achieve robust
multi-subject customization while requiring only single-subject training data.
Firstly, to break the data limitation, we introduce debiased diptych learning.
It constructs diptych training pairs from single-subject images to facilitate
multi-subject learning, while actively correcting the distribution bias
introduced by diptych construction via static attention routing and dual-branch
LoRA. Secondly, to eliminate cross-subject entanglement, we introduce dynamic
attention routing mechanism, which adaptively establishes bijective mappings
between generated images and conditional subjects. This design not only
achieves decoupling of multi-subject representations but also maintains
scalable generalization performance with increasing reference subjects.
Comprehensive experiments demonstrate that our MUSAR outperforms existing
methods - even those trained on multi-subject dataset - in image quality,
subject consistency, and interaction naturalness, despite requiring only
single-subject dataset.