FashionComposer: Compositional Fashion Image Generation
Authors: name: Sihui Ji; name: Yiyang Wang; name: Xi Chen; name: Xiaogang Xu; name: Hao Luo; name: Hengshuang Zhao
Published: 2024-12-18 18:59:50

Abstract: We present FashionComposer for compositional fashion image generation. Unlike
previous methods, FashionComposer is highly flexible. It takes multi-modal
input (i.e., text prompt, parametric human model, garment image, and face
image) and supports personalizing the appearance, pose, and figure of the human
and assigning multiple garments in one pass. To achieve this, we first develop
a universal framework capable of handling diverse input modalities. We
construct scaled training data to enhance the model's robust compositional
capabilities. To accommodate multiple reference images (garments and faces)
seamlessly, we organize these references in a single image as an "asset
library" and employ a reference UNet to extract appearance features. To inject
the appearance features into the correct pixels in the generated result, we
propose subject-binding attention. It binds the appearance features from
different "assets" with the corresponding text features. In this way, the model
could understand each asset according to their semantics, supporting arbitrary
numbers and types of reference images. As a comprehensive solution,
FashionComposer also supports many other applications like human album
generation, diverse virtual try-on tasks, etc.