IDAdapter: Learning Mixed Features for Tuning-Free Personalization of
  Text-to-Image Models
Authors: name: Siying Cui; name: Jiankang Deng; name: Jia Guo; name: Xiang An; name: Yongle Zhao; name: Xinyu Wei; name: Ziyong Feng
Published: 2024-03-20 12:13:04

Abstract: Leveraging Stable Diffusion for the generation of personalized portraits has
emerged as a powerful and noteworthy tool, enabling users to create
high-fidelity, custom character avatars based on their specific prompts.
However, existing personalization methods face challenges, including test-time
fine-tuning, the requirement of multiple input images, low preservation of
identity, and limited diversity in generated outcomes. To overcome these
challenges, we introduce IDAdapter, a tuning-free approach that enhances the
diversity and identity preservation in personalized image generation from a
single face image. IDAdapter integrates a personalized concept into the
generation process through a combination of textual and visual injections and a
face identity loss. During the training phase, we incorporate mixed features
from multiple reference images of a specific identity to enrich
identity-related content details, guiding the model to generate images with
more diverse styles, expressions, and angles compared to previous works.
Extensive evaluations demonstrate the effectiveness of our method, achieving
both diversity and identity fidelity in generated images.