A High-Quality Dataset and Reliable Evaluation for Interleaved
  Image-Text Generation
Authors: name: Yukang Feng; name: Jianwen Sun; name: Chuanhao Li; name: Zizhen Li; name: Jiaxin Ai; name: Fanrui Zhang; name: Yifan Chang; name: Sizhuo Zhou; name: Shenglin Zhang; name: Yu Dai; name: Kaipeng Zhang
Published: 2025-06-11 06:21:20

Abstract: Recent advancements in Large Multimodal Models (LMMs) have significantly
improved multimodal understanding and generation. However, these models still
struggle to generate tightly interleaved image-text outputs, primarily due to
the limited scale, quality and instructional richness of current training
datasets. To address this, we introduce InterSyn, a large-scale multimodal
dataset constructed using our Self-Evaluation with Iterative Refinement (SEIR)
method. InterSyn features multi-turn, instruction-driven dialogues with tightly
interleaved imagetext responses, providing rich object diversity and rigorous
automated quality refinement, making it well-suited for training
next-generation instruction-following LMMs. Furthermore, to address the lack of
reliable evaluation tools capable of assessing interleaved multimodal outputs,
we introduce SynJudge, an automatic evaluation model designed to quantitatively
assess multimodal outputs along four dimensions: text content, image content,
image quality, and image-text synergy.
  Experimental studies show that the SEIR method leads to substantially higher
dataset quality compared to an otherwise identical process without refinement.
  Moreover, LMMs trained on InterSyn achieve uniform performance gains across
all evaluation metrics, confirming InterSyn's utility for advancing multimodal
systems.