MultiRef: Controllable Image Generation with Multiple Visual References
Authors: name: Ruoxi Chen; name: Dongping Chen; name: Siyuan Wu; name: Sinan Wang; name: Shiyun Lang; name: Petr Sushko; name: Gaoyang Jiang; name: Yao Wan; name: Ranjay Krishna
Published: 2025-08-09 09:36:21

Abstract: Visual designers naturally draw inspiration from multiple visual references,
combining diverse elements and aesthetic principles to create artwork. However,
current image generative frameworks predominantly rely on single-source inputs
-- either text prompts or individual reference images. In this paper, we focus
on the task of controllable image generation using multiple visual references.
We introduce MultiRef-bench, a rigorous evaluation framework comprising 990
synthetic and 1,000 real-world samples that require incorporating visual
content from multiple reference images. The synthetic samples are synthetically
generated through our data engine RefBlend, with 10 reference types and 33
reference combinations. Based on RefBlend, we further construct a dataset
MultiRef containing 38k high-quality images to facilitate further research. Our
experiments across three interleaved image-text models (i.e., OmniGen, ACE, and
Show-o) and six agentic frameworks (e.g., ChatDiT and LLM + SD) reveal that
even state-of-the-art systems struggle with multi-reference conditioning, with
the best model OmniGen achieving only 66.6% in synthetic samples and 79.0% in
real-world cases on average compared to the golden answer. These findings
provide valuable directions for developing more flexible and human-like
creative tools that can effectively integrate multiple sources of visual
inspiration. The dataset is publicly available at: https://multiref.github.io/.