StyleDrop: Text-to-Image Generation in Any Style
Authors: name: Kihyuk Sohn; name: Nataniel Ruiz; name: Kimin Lee; name: Daniel Castro Chin; name: Irina Blok; name: Huiwen Chang; name: Jarred Barber; name: Lu Jiang; name: Glenn Entis; name: Yuanzhen Li; name: Yuan Hao; name: Irfan Essa; name: Michael Rubinstein; name: Dilip Krishnan
Published: 2023-06-01 17:59:51

Abstract: Pre-trained large text-to-image models synthesize impressive images with an
appropriate use of text prompts. However, ambiguities inherent in natural
language and out-of-distribution effects make it hard to synthesize image
styles, that leverage a specific design pattern, texture or material. In this
paper, we introduce StyleDrop, a method that enables the synthesis of images
that faithfully follow a specific style using a text-to-image model. The
proposed method is extremely versatile and captures nuances and details of a
user-provided style, such as color schemes, shading, design patterns, and local
and global effects. It efficiently learns a new style by fine-tuning very few
trainable parameters (less than 1% of total model parameters) and improving
the quality via iterative training with either human or automated feedback.
Better yet, StyleDrop is able to deliver impressive results even when the user
supplies only a single image that specifies the desired style. An extensive
study shows that, for the task of style tuning text-to-image models, StyleDrop
implemented on Muse convincingly outperforms other methods, including
DreamBooth and textual inversion on Imagen or Stable Diffusion. More results
are available at our project website: https://styledrop.github.io