ComposeAnything: Composite Object Priors for Text-to-Image Generation
Authors: name: Zeeshan Khan, name: Shizhe Chen, name: Cordelia Schmid

Generating images from text involving complex and novel object arrangements
remains a significant challenge for current text-to-image (T2I) models.
Although prior layout-based methods improve object arrangements using spatial
constraints with 2D layouts, they often struggle to capture 3D positioning and
sacrifice quality and coherence. In this work, we introduce ComposeAnything, a
novel framework for improving compositional image generation without retraining
existing T2I models. Our approach first leverages the chain-of-thought
reasoning abilities of LLMs to produce 2.5D semantic layouts from text,
consisting of 2D object bounding boxes enriched with depth information and
detailed captions. Based on this layout, we generate a spatial and depth aware
coarse composite of objects that captures the intended composition, serving as
a strong and interpretable prior that replaces stochastic noise initialization
in diffusion-based T2I models. This prior guides the denoising process through
object prior reinforcement and spatial-controlled denoising, enabling seamless
generation of compositional objects and coherent backgrounds, while allowing
refinement of inaccurate priors. ComposeAnything outperforms state-of-the-art
methods on the T2I-CompBench and NSR-1K benchmarks for prompts with 2D/3D
spatial arrangements, high object counts, and surreal compositions. Human
evaluations further demonstrate that our model generates high-quality images
with compositions that faithfully reflect the text.