Personalized Text-to-Image Generation with Auto-Regressive Models
Authors: name: Kaiyue Sun, name: Xian Liu, name: Yao Teng, name: Xihui Liu

Personalized image synthesis has emerged as a pivotal application in
text-to-image generation, enabling the creation of images featuring specific
subjects in diverse contexts. While diffusion models have dominated this
domain, auto-regressive models, with their unified architecture for text and
image modeling, remain underexplored for personalized image generation. This
paper investigates the potential of optimizing auto-regressive models for
personalized image synthesis, leveraging their inherent multimodal capabilities
to perform this task. We propose a two-stage training strategy that combines
optimization of text embeddings and fine-tuning of transformer layers. Our
experiments on the auto-regressive model demonstrate that this method achieves
comparable subject fidelity and prompt following to the leading diffusion-based
personalization methods. The results highlight the effectiveness of
auto-regressive models in personalized image generation, offering a new
direction for future research in this area.