The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in
  Text-to-Image Models
Authors: name: Alfio Ferrara, name: Sergio Picascia, name: Elisabetta Rocchetti

Text-to-image diffusion models have demonstrated remarkable capabilities in
generating artistic content by learning from billions of images, including
popular artworks. However, the fundamental question of how these models
internally represent concepts, such as content and style in paintings, remains
unexplored. Traditional computer vision assumes content and style are
orthogonal, but diffusion models receive no explicit guidance about this
distinction during training. In this work, we investigate how transformer-based
text-to-image diffusion models encode content and style concepts when
generating artworks. We leverage cross-attention heatmaps to attribute pixels
in generated images to specific prompt tokens, enabling us to isolate image
regions influenced by content-describing versus style-describing tokens. Our
findings reveal that diffusion models demonstrate varying degrees of
content-style separation depending on the specific artistic prompt and style
requested. In many cases, content tokens primarily influence object-related
regions while style tokens affect background and texture areas, suggesting an
emergent understanding of the content-style distinction. These insights
contribute to our understanding of how large-scale generative models internally
represent complex artistic concepts without explicit supervision. We share the
code and dataset, together with an exploratory tool for visualizing attention
maps at https://github.com/umilISLab/artistic-prompt-interpretation.