FaceChain-SuDe: Building Derived Class to Inherit Category Attributes
  for One-shot Subject-Driven Generation
Authors: name: Pengchong Qiao; name: Lei Shang; name: Chang Liu; name: Baigui Sun; name: Xiangyang Ji; name: Jie Chen
Published: 2024-03-11 14:43:40

Abstract: Subject-driven generation has garnered significant interest recently due to
its ability to personalize text-to-image generation. Typical works focus on
learning the new subject's private attributes. However, an important fact has
not been taken seriously that a subject is not an isolated new concept but
should be a specialization of a certain category in the pre-trained model. This
results in the subject failing to comprehensively inherit the attributes in its
category, causing poor attribute-related generations. In this paper, motivated
by object-oriented programming, we model the subject as a derived class whose
base class is its semantic category. This modeling enables the subject to
inherit public attributes from its category while learning its private
attributes from the user-provided example. Specifically, we propose a
plug-and-play method, Subject-Derived regularization (SuDe). It constructs the
base-derived class modeling by constraining the subject-driven generated images
to semantically belong to the subject's category. Extensive experiments under
three baselines and two backbones on various subjects show that our SuDe
enables imaginative attribute-related generations while maintaining subject
fidelity. Codes will be open sourced soon at FaceChain
(https://github.com/modelscope/facechain).