Artist: Aesthetically Controllable Text-Driven Stylization without
  Training
Authors: name: Ruixiang Jiang; name: Changwen Chen
Published: 2024-07-22 17:58:05

Abstract: Diffusion models entangle content and style generation during the denoising
process, leading to undesired content modification when directly applied to
stylization tasks. Existing methods struggle to effectively control the
diffusion model to meet the aesthetic-level requirements for stylization. In
this paper, we introduce Artist, a training-free approach that
aesthetically controls the content and style generation of a pretrained
diffusion model for text-driven stylization. Our key insight is to disentangle
the denoising of content and style into separate diffusion processes while
sharing information between them. We propose simple yet effective content and
style control methods that suppress style-irrelevant content generation,
resulting in harmonious stylization results. Extensive experiments demonstrate
that our method excels at achieving aesthetic-level stylization requirements,
preserving intricate details in the content image and aligning well with the
style prompt. Furthermore, we showcase the highly controllability of the
stylization strength from various perspectives. Code will be released, project
home page: https://DiffusionArtist.github.io