Unified Text-to-Image Generation and Retrieval
Authors: Leigang Qu, Haochuan Li, Tan Wang, Wenjie Wang, Yongqi Li, Liqiang Nie, Tat-Seng Chua


Reviewer paper summaries: This paper presents a unified framework for text-to-image generation and retrieval within the context of Multimodal Large Language Models (MLLMs). The authors identify the limitations of traditional text-to-image retrieval, which relies on a fixed database, and text-to-image generation, which struggles with knowledge-intensive images. To address these issues, the paper proposes a method that combines generative and retrieval capabilities, utilizing the intrinsic discriminative abilities of MLLMs. The framework includes a generative retrieval method that operates in a training-free manner and an autonomous decision module to choose the best-matched image between generated and retrieved options. Additionally, the authors introduce TIGeR-Bench, a benchmark designed to evaluate the performance of unified text-to-image generation and retrieval systems across creative and knowledge-intensive domains. Extensive experiments on the TIGeR-Bench and other benchmarks demonstrate the superiority of the proposed method.
Summary of strengths: S1: The proposed unified framework effectively combines text-to-image generation and retrieval, addressing the limitations of each approach when used independently.

S2: The creation of TIGeR-Bench provides a standardized evaluation tool for future research in this area, covering both creative and knowledge-intensive domains.

S3: The introduction of a training-free generative retrieval method is innovative, allowing for efficient retrieval without compromising the discriminative power of MLLMs.

S4: The experimental performance is satisfactory.
Summary of weaknesses: W1: The general idea of the generative cross-modal retrieval is not a new story, like in [1]. The contribution is a little bit overclaimed in the study.

[1] Li et al. Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond.
Comments, suggestions and typos: N/A
Overall assessment scores: 4