DreamCache: Finetuning-Free Lightweight Personalized Image Generation
  via Feature Caching
Authors: name: Emanuele Aiello; name: Umberto Michieli; name: Diego Valsesia; name: Mete Ozay; name: Enrico Magli
Published: 2024-11-26 15:03:14

Abstract: Personalized image generation requires text-to-image generative models that
capture the core features of a reference subject to allow for controlled
generation across different contexts. Existing methods face challenges due to
complex training requirements, high inference costs, limited flexibility, or a
combination of these issues. In this paper, we introduce DreamCache, a scalable
approach for efficient and high-quality personalized image generation. By
caching a small number of reference image features from a subset of layers and
a single timestep of the pretrained diffusion denoiser, DreamCache enables
dynamic modulation of the generated image features through lightweight, trained
conditioning adapters. DreamCache achieves state-of-the-art image and text
alignment, utilizing an order of magnitude fewer extra parameters, and is both
more computationally effective and versatile than existing models.