RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance
Authors: name: Zhicheng Sun; name: Zhenhao Yang; name: Yang Jin; name: Haozhe Chi; name: Kun Xu; name: Kun Xu; name: Liwei Chen; name: Hao Jiang; name: Di Zhang; name: Yang Song; name: Kun Gai; name: Yadong Mu
Published: 2024-05-23 15:12:15

Abstract: Customizing diffusion models to generate identity-preserving images from
user-provided reference images is an intriguing new problem. The prevalent
approaches typically require training on extensive domain-specific images to
achieve identity preservation, which lacks flexibility across different use
cases. To address this issue, we exploit classifier guidance, a training-free
technique that steers diffusion models using an existing classifier, for
personalized image generation. Our study shows that based on a recent rectified
flow framework, the major limitation of vanilla classifier guidance in
requiring a special classifier can be resolved with a simple fixed-point
solution, allowing flexible personalization with off-the-shelf image
discriminators. Moreover, its solving procedure proves to be stable when
anchored to a reference flow trajectory, with a convergence guarantee. The
derived method is implemented on rectified flow with different off-the-shelf
image discriminators, delivering advantageous personalization results for human
faces, live subjects, and certain objects. Code is available at
https://github.com/feifeiobama/RectifID.