FiVA: Fine-grained Visual Attribute Dataset for Text-to-Image Diffusion
  Models
Authors: name: Tong Wu; name: Yinghao Xu; name: Ryan Po; name: Mengchen Zhang; name: Guandao Yang; name: Jiaqi Wang; name: Ziwei Liu; name: Dahua Lin; name: Gordon Wetzstein
Published: 2024-12-10 17:02:58

Abstract: Recent advances in text-to-image generation have enabled the creation of
high-quality images with diverse applications. However, accurately describing
desired visual attributes can be challenging, especially for non-experts in art
and photography. An intuitive solution involves adopting favorable attributes
from the source images. Current methods attempt to distill identity and style
from source images. However, "style" is a broad concept that includes texture,
color, and artistic elements, but does not cover other important attributes
such as lighting and dynamics. Additionally, a simplified "style" adaptation
prevents combining multiple attributes from different sources into one
generated image. In this work, we formulate a more effective approach to
decompose the aesthetics of a picture into specific visual attributes, allowing
users to apply characteristics such as lighting, texture, and dynamics from
different images. To achieve this goal, we constructed the first fine-grained
visual attributes dataset (FiVA) to the best of our knowledge. This FiVA
dataset features a well-organized taxonomy for visual attributes and includes
around 1 M high-quality generated images with visual attribute annotations.
Leveraging this dataset, we propose a fine-grained visual attribute adaptation
framework (FiVA-Adapter), which decouples and adapts visual attributes from one
or more source images into a generated one. This approach enhances
user-friendly customization, allowing users to selectively apply desired
attributes to create images that meet their unique preferences and specific
content requirements.