GenIR: Generative Visual Feedback for Mental Image Retrieval
Authors: Diji Yang
Minghao Liu
Chung-Hsiang Lo
Yi Zhang
James Davis
Venue: NeurIPS 2025 poster ()
Keywords: Multi-Round Image Retrieval
Vision-Language Models
Generative Retrieval
Primary area: deep_learning
TL;DR: We propose GenIR, a method using generative visual feedback to tackle the multi-round image retrieval task, and release a dataset.

Abstract: Vision-language models (VLMs) have shown strong performance on text-to-image retrieval benchmarks. However, bridging this success to real-world applications remains a challenge. In practice, human search behavior is rarely a one-shot action. Instead, it is often a multi-round process guided by clues in mind. That is, a mental image ranging from vague recollections to vivid mental representations of the target image. Motivated by this gap, we study the task of Mental Image Retrieval (MIR), which targets the realistic yet underexplored setting where users refine their search for a mentally envisioned image through multi-round interactions with an image search engine. Central to successful interactive retrieval is the capability of machines to provide users with clear, actionable feedback; however, existing methods rely on indirect or abstract verbal feedback, which can be ambiguous, misleading, or ineffective for users to refine the query. To overcome this, we propose GenIR, a generative multi-round retrieval paradigm leveraging diffusion-based image generation to explicitly reify the AI system's understanding at each round. These synthetic visual representations provide clear, interpretable feedback, enabling users to refine their queries intuitively and effectively. We further introduce a fully automated pipeline to generate a high-quality multi-round MIR dataset. Experimental results demonstrate that GenIR significantly outperforms existing interactive methods in the MIR scenario. This work establishes a new task with a dataset and an effective generative retrieval method, providing a foundation for future research in this direction

GenIR: Generative Visual Feedback for Mental Image Retrieval
Diji Yang'*
Minghao Liu 3+
Chung-Hsiang Lo²
Yi Zhang'
James Davis'
University of California Santa Cruz
Northeastern University 3 AAcenture
[dyang39, ml mliu40, yiz, davisje)@ucsss edu
1o. chun@northeastem edu
Project Page: -ttg:s//risaallggeeereeeevee-i- gi
Abstract
Vision-language models (VLMs) have shown strong performance on text-to-image retrieval benchmarks. However, bridging this success real-world applications remains challenge. In practice, human search behavior is rarely one-shot action. Instead, is often a multi-round process guided by clues mind. That is, mental image ranging from vague recollections to vivid mental representations of the target image. Motivated by this gap, we study the task of Mental Image Retrieval (MIR), which targets the realistic yet underexplored setting where users refine their search for mentally envisioned image through multi-round interactions with an image search engine. Central to successful interactive retrieval is the capability machines to provide users with clear, actionable feedback; however, existing methods rely indirect or abstract verbal feedback, which can be ambiguous, mis- leading, or ineffective for users to refine the query. To overcome this, we propose GenlR, generative multi-round retrieval paradigm leveraging diffusion-basse image generation to explicitly reify the AI system's 's understanding at each round. These synthetic visual representations provide clear, interpretable feedback, abling users to refine their queries intuitively and effectively. We further introduce fully automated pipeline generate high-quality multi-round MIR dataset. Experimental results demonstrate that GenlR significantly outperforms existing interactive methods in the MIR scenario. This work establishes new task with a dataset and an effective generative retrieval method, providing a foundation for future research in this direction !
Introduction
Recent Vision-language models (VLMs) have achieved decent results on standard text-to-image retrieval benchmarks [29, 13]. Despite this progress, transferring these capabilities into real-world applications remains challenging. One key limitation is that real human search behavior is often not one-shot static; it unfolds through sequence of actions, highlighting the necessity for interactive information retrieval (IIR) systems [24, 32, 1, 35). Another limitation is that users frequently initiate a search re-find the information they have seen before, which could be partial memory, vague clues, or vivid recall of the target images [3, 38, 5]. To address this scenario, we define Mental Image Retrieval (MIR)* 2, where users iteratively refine their queries based on mental image (i.c., an image in mind) retrieve intended image from an image database.
"qqual contribution.
"ode and data available https://gi www/wikelnho/2 wwt.o//////ggeeeee.eee
"The term inspired by the Mental Image Reconstruction task [9] and shares the definition of "mental image", though their study purely from Neuroscience side which is different problem from ours.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Ours: Generative Mental Image Retrieval (GenIR)
Query
Image Generator
Q Image to Image Retrieval
Visual Feedback
Mental Image Retrieval (ChatIR)
Interactive Mental Image Retrieval (PlugIR)
Query
Retrivaal
to Image
Query
Q Tixt to Image
Hetrivaal
Lenguage
Image Captioner
Verbal
Model
Feodbbakk
Comparison of methods for Mental Image Retrieval task. Top: our generative method, which reifies the intermediate query using image generator model and applies image-to-image search for retrieval. Bottom: Existing approach (ChatIR and PlugIR) which support multi-round query improvements based verbal feedback.
Figure 1:
Although MIR has not been explicitly formulated distinct task previously, as a subset of long- standing text-to-image IIR task [27, 5], some works have implicitly touched upon similar settings. ChatlR [12) positions a VLM (or ideally human) as the active searcher. Seeing the initial query from searcher, ChatlR uses Large Language Models (LLMs) to provide system feedback in question format to the user solely based on the textual dialogue history. The question is then answered by the searcher who has access to the ground-trus truth images (i.c., Mental Image). Next, the dialogue history will be appended with the question-am answerr pair and then be used as the search query. This implicitly positions ChatlR within the realm of MIR as a subset of interactive cox-tto-image retrieval where the human searcher holds the target image in memory. However, left of Figure 1, ChatIR has only verbal (text-based feedback with no information from image space, resulting in generated question-answe -answrr pairs that may be redundant or irrelevant to the query refinement. PlugIR [11], as shown in the bottom right of Figure 1, advances this setup further by incorporating retrieval context-text captions of retrieved images into the query generation for subsequent rounds, aiming to produce more contextually relevant feedback and mitigate redundancy. Nevertheless, the major challenge exists, both methods remain constrained by significant limitations regarding feedback effectiveness. Even across multiple interaction rounds, these methods rely heavily on indirect, verbal feedback derived solely from retrieval failures. Such feedback is often abstract and interpretability- -porr, providing users with little actionable insight or potentially misleading clues for refining subsequent queries. vision-language embedding spaces, such as CLIP [23], minor textual edits can cause unpredictable changes in retrieval outcomes, making query refinement inherently trial-and ---rrrrroor process. Consequently, the feedback offered by existing conversational retrieval approaches inadequately expresses the AI system's 's current understanding and fails to directly benefit users toward effective refinements. As ChatlR example shown in Figure 6, verbal feedback for an image depicting a person wearing a motorcycle helmet: "a human is not wearing hat".. Although literally true, such feedback fails to capture the visual salience of the helmet and may steer the user's s refinement toward irrelevant details, ultimately misleading the search process for images containing headwear.
shown in the bottom
Motivated by the need for more effective and interpretable system feedback, we propose GenIR, a generative interactive retrieval paradigm designed explicitly to provide clear, interpretable, and actionable visual feedback at each interaction turn. GenlR employs straightforward but powerful iterative pipeline as shown in Figure I: first, text-to-image diffusion model generates a synthetic image from the user's current textual query; then, this synthetic image is used for retrieval from database through mmage-to-image similarity matching. Crucially, the generated synthetic image serves more than just query, but acts as explicit visualization of the system's 's internal understanding

---

(i.e., the representation of the query in the vision- language latent space), enabling users to clearly perceive discrepancies between their mental image and the system's interpretation so that can refine the query for the next round search.
Beyond its utility at inference time, GenlR also supports dataset construction for studying MIR. Following the common practice of using VLM to play as the human searcher [12, 11], we create an automated pipeline based on the GenlR framework. We present multi-round dataset with each round consisting of a refined query, a generated synthetic image, and retrieved results with a correctness label. Our experiments demonstrate that GenIR outperforms existing MIR baselines, highlighting the significant advantage of using visual feedback over verbal feedback. Furthermore, our study indicates that our GenlR annotated query can result in better retrieval performance than annotation from ChatlR under the same retrieval setting. In summary, our contribution is as follows:
Task: We formally define the Mental Image Retrieval (MIR) task, subset of the multi-round interactive text-to-image retrieval task where the searcher has the target image in mind.
Method: We propose a novel framework, GenlR, using the generative approach to provide intuitive, interpretable visual feedback revealing the optimization direction for user during multi-round query refinement.
Dataset: We release dataset with visually grounded feedback annotation, together with curation pipeline which can support both MIR and general text-to-image retrieval tasks.
2 Related Works
2.1 Chat-based Image Retrieval
Conversational image retrieval, which use chat-like feedback to improve query for text-to-image IRR has gained attention as way to improve search performance [19]. A foundational work, ChatlR [12], demonstrated improved retrieval accuracy through multi-round chats, where an LLM poses questions answered by a human with target image access. ChatR also contributed a multi-round chat dataset and highlighted the utility of multi-round interaction for retrieval tasks. However, both the performance of their method and the quality of the curated dataset were limited by feedback efficiency issue (redundancy or misleading) as we discussed in Section 1. PlugIR [11] advanced this idea by proposing plug-and-plas and-play image captioning model to collect feedback from retrieved images. This yielded context-aware and redundant verbal feedback [34]; however, both of these works and their related subsequent works [36, 37] remain fundamentally limited in their capacity to share nuanced visual representations with users.
In contrast, our work introduces a new modality into the loop: generated images that serve as visual hypotheses showing the system's understanding in the image space. Rather than relying on textual queries alone, our method synthesizes what the system "thinks" the user wants to search, enabling visual inspection and more precise system feedback.
2.2 Generative Image for Image Retrieval
Diffusion models have achieved great success in image reconstruction, that is, given target image, text encoder is trained to output human-readable language or a latent representation as input to the diffusion model, aiming to generate an image close to the target image [30, 31, 28). However, applying such models directly to MIR is non-trivial, as human users can hardly provide actual images based on their mental images. A more feasible attempt is Imagine-and-Si one-time process that an image captioning model to generate a text description from the target image and then feeds into a text-to-image diffusion model to generate a proxy image for retrieval. Yet, discussed in the section 1, this single-round approach has been proven by multiple interactive retrieval works to be inferior in dealing with real-world applications [5, 27].
[15], which involves a
Apart from existing attempts, our approach uses image generation as core step in the retrieval loop itself, not merely to improve retrieval performance for the current round, but to provide visual feedback to the user to potentially benefit the writing for the next round query. To our knowledge, this is the first work to integrate text-to-image generation into an interactive retrieval setting, enabling closed-loop interaction that unifies generation, retrieval, and feedback within a single framework.
3

---

3 GenIR: Generative Retrieval with Visual Feedback
3.1 Task Formulation
We formally define the task of Mental Image Retrieval (MIR) as subset of the text-to-image Interactive Information Retrieval [27]. MIR inherits the nature of multi-round interaction from IIR, while only focusing on the case where the user has an internal mental image that can not be directly accessed by the retrieval system. From an Information Retrieval theory perspective, MIR does not consider the Exploratory Search where a searcher has never seen the searching target [18, 32], but focuses on Known-item Search where the searcher has seen and can recall or partially recall the target information [33, 20, 2]. To define the task, we denote an image database N, and let gurget represent the image that the user has in mind. The retrieval task proceeds over multiple interaction rounds == 1, 2, ..... T. At each round, the user formulates a textual query i intended to approximate the mental representation of Thares Based on the query 9r. the retrieval system returns a candidate image fretrieved E N. Additionally, the system provides feedback signals to the user, potentially benefiting (t)
subsequent refinements of the query to bridge diserepancies between the retrieved candidate image and the user's ss mental image. The iterative process continues until the target image is successfully identified or a predefined maximum number of interaction rounds is reached.
3.2 Generative Retrieval Framework
This section details the GenIR framework and the rationale behind the design as shown in Figure 1.
Query Formulation At the beginning of each interaction round t, the human user formulates a textual query q: that represents their current visual intent. This query encapsulates the user's mental image description, which may evolve over subsequent rounds based on visual feedback provided. Users are encouraged to include both high-level deseriptions (e.g., scene type, overall composition) and fine-grained attributes (e.g., color scheme, object details) to ensure comprehensive coverage of their mental image.
Synthetic Image Generation Central to the GenlR framework is the image generation component, which reifies textual queries into synthetic images. Specifically, given the user's query qts an image generator G produces a synthetic visual representation ryyntheui G(g.). This visual representation (()
explicitly captures the retricval system's 's interpretation of the query. Importantly, our framework is flexible and model-agnostic, allowing the of various generative models (e.g., diffusion models, GANs, or any other generator). The key benefit of employing visual generation is that it significantly reduces ambiguity inherent in textual communication, offering intuitive interface for users to identify discrepancies and refine their queries precisely.
synthetti generated, GenIR employs image- to-image retrieval as the core retrieval mechanism. Specifically, both synthetic and database images are embedded into shared visual feature space using a suitable encoder (such as the image encoder from CLIP). Retrieval is then conducted by selecting the database image prettieved E N that maxi- mizes similarity to the synthetic image according to a visual similarity metric, commonly cosine similarity. Formally, (t) Ilcriived = arg maxjew cosine (D(It)ntetie)
Image-to-Image Retrieval With the synthetic image (t)
[())
). (I)(.) where & denotes the image
encoder. The use of image-to-image retrieval enhances retrieval quality by directly leveraging visual information, thus effectively bypassing limitations associated with purely textual queries.
Feedback Loop Upon viewing the generated synthetic image psynthetic the user gains valuable (t) insight into the system's 's current interpretation of their query. This visualization allows the user to identify discrepancies between the generated image and their mental target, such as missing elements, incorrect attributes, or stylistic deviations. Based on this visual feedback, the user can then refine their query, guiding the system towards a more accurate retrieval result in the subsequent round. This iterative refinement loop continues until a stopping criterion is met, typically either a predefined maximum number of interaction rounds or the target is retrieved. By explicitly incorporating generative visualization as an intermediate step, GenlR makes the retrieval process interpretable, intuitive, and highly user-centrie, thereby improving overall retrieval effectiveness in interactive retrieval setting.

---

Intial Generation
Feadback Iterations
Target Image
Figure 2: Visual progression of GenlR's image refinement process. Each row shows the evolution from initial generation (leftmost), through multiple feedback iterations (middle columns), to final generated result, alongside the target image (rightmost). Note how generated images progressively capture more accurate details with each crattion-imrovinng clothing and posture (row 1), facial features and giraffe positioning (row 2), and dining scene composition (row 3).
3.3 Advantages of Visual Feedback in GenIR
GenlR's key advantage lies in its explicit visual feedback mechanism, addressing the limitations of existing systems that rely solely on ambiguous textual feedback. Text-based systems encode queries into vision- laniuae space, but this internal representation, the Al's interpretation "visual belief", remains hidden from the user, making iterative refinement a challenging trial-and-error process.
GenIR alleviates this problem by using image generation to visualize the system's understanding of the textual query. This synthetic image serves as a direct projection the query's meaning within the vision-language space into an interpretable visual form. Although the diffusion process does not provide additional information compared to using text to retrieve directly in the vision-languag space, it reifies all the representations in an intuitive way. Consequently, users directly observe the model's 's internal visual belicf (or "what the system thinks"), rather than navigating ambiguous textual interpretations, so to intuitively identify discrepancies and refine their queries with knowledge of the details beyond text only.
Moreover, GenlR transitions the retrieval process from cross-modal matching (text-to-image to same- modal matching (image-to-image). This allows subsequent search steps to leverage well-established visual similarity metrics that can capture spatial relationships and visual attributes that might be difficult to express precisely text.
Figure 2 demonstrates this progression, showing how the generated images progressively improve with each iteration. Appendix A.3 contains detailed version with the corresponding text queries that produced these refinements.
4 Experiment
4.1 Setting
Task Definition Ideally, our approach would involve a human-in-the-loop However, first step exploration along this direction, and considering the cost, we follow the standard setting of previous work [12, 11] to use VLM to replace the individual who engages the mental image retrieval process. Specifically, we use a good-performing open-sourced VLM Gemma3 [25] to issue queries and improve the next round of queries based on the visual feedback provided by the image generator.
Datasets We evaluate our method across four datasets with distinet visual domains to demonstrate the robustness of our approach. (I) MS COCO [16]'s 50k validation set, featuring common objects
5

---

in everyday contexts, provides a challenging testbed for retrieving images with complex scenes and multiple object interactions. (2) FFHQ [8], comprising 70,000 high-quality facial portraits, represents the human-centric domain where fine-grained attributes (expressions, accessories, age) drive retrieval outcomes. (3) Flickr30k [21] contains 31,783 diverse real-world photographs showcasing people engaged in various actívities across different environments. (4) Clothing-ADC [17], with over I million clothing images, introduces a specialized commercial domain extremely fine-grained subclasses (12,000 subclasses across 12 primary categories), enabling evaluation on highly specific attribute-based retrieval tasks. This domain diversity-spannet
everyday objects, human faces, diverse activities, and fashion items tiems-allows alllws to thoroughly evaluate how our generative retrieval approach performs across fundamentally different visual content types and retrieval challenges.
Evaluation Metrics Following previous works in interactive image retrieval [12. 11, 5], we adopt @its@ K our primary evaluation metrie, which measures the percentage of queries where the target image appears within the top-K retrieved results. Specifically, we report Hits@ 10 to align with established benchmarks in the field. This metric effectively captures the practical utility of retrieval systems, as users typically examine only the top few results.
4.2 Implementation Details
We compare our proposed GenlR approach against several baselines to evaluate the effectiveness of generative visual feedback in Mental Image Retrieval:
Verbal Feedback Methods (Baseline) We tested two Verbal-feedback baselines. The first one is Chat]R[12), which employs a human answerer for MSCOCO and ChatGPT to simulate a human for Flickr30k, with BLIP [14] serving the questioner model. Second, we develop enhanced version of ChatlR by replacing both sides with Gemma3 (in 4B or 12B parameter configurations), representing stronger VLM-based baseline. Both methods operate without explicit visual feedback, relying solely on multi-round dialog for query refinement.
Prediction Feedback (Baseline) This baseline incorporates visual feedback by showing the user (simulated by Gemma3) the top-1 retrieved image each interaction round. The user examines this retrieved result and provides textual feedback describing discrepancies between the retrieved image and their mental target image. This approach represents traditional interactive retrieval method [27] that leverages real images from the database but lacks the interpretability advantages of our generative approach.
GenlR Configuration (Ours) GenIR provides explicit visual feedback through synthetic images generated from the user's textual query. To evaluate the sensitivity of our approach to generator quality, we test five state-of-the- text-to-image diffusion models: Infinity [6], Lumina- 1mmge--- [22], Stable Diffusson 3.5 [4], FLUX.1 [10], and HiDream-II [26]. For all diffusion models, use the default inference parameters as specified in their original works to ensure a fair comparison. Each model transforms the user's textual query into synthetic image that visually represents the system's 's current understanding, which is then used for image-to-image retrieval through BLIP-2 [14].
4.3 Results and Analysis
Performance on MSCOCO Figure 3 presents a comprehensive evaluation of our GenlR approach against traditional conversational retricval basclines on the MSCOCO dataset, measured by Hits @ 10 percentage across increasing dialog lengths. All experiments were conducted using the full 50,000- image validation set as the search space, representing a challenging large-scale retrieval scenario.
The left graph demonstrates that our proposed GenIR method substantially outperforms all baselines, achieving approximately 90% retrieval accuracy even at the initial query and reaching nearly 98% by the tenth interaction round. This represents significant improvement over the Prediction Feedback method (blue line), which reaches only 92% after ten rounds, and the Verbal Feedback baselines using Gemma3-121 122b (red line) and ChatlR (green line), which achieve 92% and 73% respectively. The substantial performance gap highlights the effectiveness of our visual feedback approach in providing clear, interpretable guidance for query refinement.

---

Comparison Across Feedback Types
Comparison Synthetic Feedback Models
26
15
Hoog00 On
70-
Santheric Image Teechack (Cun)
Infirty
Prediction Fredbbakk
Lumina
65
Gemmad
ChaiR
Magnifled Y-axis (82-100)
Dialog Length
Dialog Length
Figure 3: Performance Comparison on MSCOCO Dataset (Hits@ 10, 50k search space). Left: Our GenlR approach with Infinity diffusion model (Yellow) significantly outperforms all baselines, including Prediction Feedback (blue), Verbal Feedback with Gemma3-12b (red), and ChatlR (green). Comparison of different text-to-image diffusion models within our GenlR framework, showing consistent performance advantages across all generators, with Infinity and Lumina achieving the best results after 10 interaction rounds.
Right:
The right graph further examines the impact of different rext-to-image diffusion models on our method's performance. While all models demonstrate effective performance improvement over dialog rounds, Infinity and Lumina consistently outperform others, suggesting that higher-quality image generation contributes to more effective visual feedback. Notably, even with the lowest- performing generator (HiDream), our approach still achieves superior results compared to traditional feedback methods, demonstrating the robustness of our generative retrieval paradigm across different implementation choices.
To validate these quantitative findings with real users, we conducted a human evaluation study which found that 86% of the generated visual feedback was useful for query refinement; these human- annotated evaluations will be released alongside our dataset and code. Details of this study are provided in Appendix D.
Cross- Domain Evaluation (FFHQ, Flickr30k, Clothing-ADC) Figure 4 demonstrates GenlR's s robust performance across three diverse visual domains. Our approach consistently outperforms all baselines regardless of domain characteristics, with particularly striking advantages in FFHQ (70% vs. 52% Hits @ 10 for the next best method) and Clothing (73% vs. 50%). Notably, ClothingADC represents an especially challenging scenario with over million images in its search space space-more more than 20 times larger than the MSCOCO test set-yet GenlR maintains its substantial performance advantage. Even on Flickr30k, which shows higher baseline performance overall, GenlR maintains clear 8-15% advantage throughout all interaction rounds. These results confirm GenlR's Nomain-agnostic effectiveness, especially with fine-grained visual details that text struggles to capture. Our consistent performance advantage across diverse domains and search space sizes demonstrates the approach's 's practical versatility.
Effect of Vision- LanguaeL Model Size Figure examines the impact of VLM parameter scale 44emma3-4b vs. Gemma3- 122b) across different feedback methods on MSCOCO and FFHQ datasets. While larger models predictably deliver superior performance all settings, the performance gap between model sizes is notably smaller with our Fake Image Feedback approach compared to alternative methods. Most significantly, our GenlR approach with the smaller 4b model consistently outperforms both Prediction Feedback and Verbal Feedback methods even when those methods utilize the larger 12b model. This finding demonstrates that visual feedback provides inherent advantages independent of model scale, enabling more efficient deployment without sacrificing retrieval quality.
Prediction feedback not always better than verbal As shown in Figure 4, prediction feedback initially outperforms Verbal-feedback approaches but plateaus after 2-4 rounds, eventually being surpassed by text-only methods in longer dialogues. This suggests prediction feedback can trap the retrieval process in local minima, where iterative refinements based on single retrieved image become
7

---

FRHQ
Flicker 30k
ClothingADC
20
Dislog Langth
Diatog Lengh
Dasog Lengt
Figure 4: Performance Comparison FFHQ, Flickr30k, and Clothing datasets (Hits@ @ 10). Our GenlR approach (yellow) consistently outperforms all baselines across domains, with particularly strong advantages in FFHQ and Clothing (the latter with IM+ image search space).
Synthetic Image Feedback
Prediction Feedback
Verbal Feedback
MSCOCD HIN810 (An 90)
100
101
FPHQ Hiwegno (in 90)
GermaDilts
Length
Figure 5: Analysis of vision-language model scale effects across feedback methods on MSCOCO (top) and FFHQ (bottom). While 12b models outperform 4b counterparts as expected, our GenIR with the smaller 4b model consistently surpasses alternative approaches even when using larger models.
increasingly incremental. In contrast, our generative approach provides consistently improving trajectory by visualizing the system's 's understanding rather than showing cittabsse-consrrinne results.
Generator-Agnostc Performance Figure 3 (right) shows that performance differences between generators are minimal compared to the substantial gap between GenlR and baselines. This confirms method's 's effectiveness derives from the visual feedback mechanism itself, generation quality, enabling deployment with even simpler diffusion models.
Dataset Contribution
As byproduct of our experimental framework, we construct multi-round dataset for MIR task. Algorithm describes our automated curation pipeline where the VLM formulates an initial query from the target image, then each round: (1) a synthetic image is generated (Line 6); (2) the closest database image is retrieved (Line 7); (3) correctness is labeled (Line 8); and (4) the VLM refines the query based on visual discrepancies (Line 10). Unlike ChatlR, our dataset centers on visual feedback where both parties share understanding through images, reducing redundancy and misleading information.
Formally, the constructed dataset comprises a series of structured interaction rounds. At each round t, the data instance includes four key elements: textual query qt. synthesized feedback image paymthetic retrieved image from the database Iretrieved, and binary label y indicating whether the retrieved image correctly matches the target mental image. The dataset spans multiple domains

---

Algorithm Data Annotation Pipeline
1: Notation: N: image pool
I: the set ground-truth target images sim: Similarity search
2: Initialize dataset D as empty.
3: for each target image lzarget E I do
4: 9o = VLM VLM(/hargett)
I1 Query formulation
5: for 1 to T do
6: 4 Lyntheric Diffusion Generator(ge)
I/ Synthetic Image Generation
7:
1 Iestrieved = arg MAX+EN sim Luumhhetie, 2)) I[I.eorrieeed = Iearget]
// Image-to-Image retrieval // Assign correctness label // Record tuple dataset I/ Refine query based on visual feedhack
8:
9: D append (ge, laynthetic
, Iretrieved, Myb
9-11 (Jarrea raynthotio
11: end for
12:
end for
13: return dataset D
(i... general, clothing, and human face), and each data point explicitly captures the shared visual grounding and query refinement trace. As result, dataset yields better query quality than ChatlR's, s, as experimentally validated in Appendix A.2. Furthermore, it uniquely provides a mid-step generated image for each retrieval round. It may serve testbed for studying MIR tasks and research problems such as visual feedback-driven retrieval and multi-round query refinement.
Limitation
Our study has two primary limitations: First, our VLM simulation assumes users have clear, fixed target image in mind, whereas real users often begin with only partial or fuzzy mental representations. Second, our framework doesn't account for how mental images naturally evolve and clarify during the search process itself, as retrieval attempts often help users refine their own memory. Addi- tionally, while visual feedback is useful in the majority of cases, generated images sometimes mislead refinement through hallucinations or detail misalignments (detailed failure mode analysis in Appendix D.5). Future work should include human studies that capture these dynamic aspects of memory retrieval to validate GenlR's s effectiveness in more naturalistic search scenarios. We leave more discussion limitations and future work, including opportunities for RL-based system optimization, to Appendix E.
Conclusion
This paper introduced Mental Image Retrieval (MIR), task modeling realistic interactive image search guided by users' internal mental images. Recognizing the limitations verbal feedback, we proposed GenlR, a novel generative framework that employs an image generator to provide explicit and interpretable visual feedback. Notably, we expect GenlR to be a model-agnostic framework, allowing for the integration of various text-to-image generators (beyond diffusion models) and image-to-image retrieval models or algorithms. This plug-and-play capability enables leveraging any prod-performmig pre-trained models within the framework. Complementing the framework. we present an automated pipeline for curating multi-round MIR dataset. Extensive experiments across diverse datasets demonstrate that GenlR significantly outperforms existing MIR approaches, highlighting the critical advantage of visual feedback for effective multi-round retrieval. Furthermore, evaluations under traditional text-to-inn eox--to-image retrieval setting shows that queries refined by GenlR yield superior retrieval performance compared to those refined with purely verbal feedback (e.g., ChatIR), validating the quality and utility of our dataset for studying the general text-to-image retrieval task. This work provides a foundational step for future research into intuitive and interpretable interactive multimodal retrieval systems, encouraging further exploration of human-Al interaction dynamics and the role of generative vision models in enhancing interactive information retrieval.

---

Acknowledgment
We thank Jinmeng Rao, Xi Yi, and Baochen Sun for helpful discussions and feedback on the early draft, and Linda Li for creating the visualizations of the system design.
References
[1] Pia Borlund. Interactive information retrieval: An introduction. Journal of Information Science Theory and Practice, 1(3):12-32, 2013.
[2] Andrei Broder. A taxonomy of web search. In ACM Sigir forum, volume 36, pages 3-10, 2002.
[3] Susan Dumais, Edward Cutrell, Jonathan J Cadiz, Gavin Jancke, Raman Sarin, and Daniel C Robbins. Stuff i've seen: system for personal information retrieval and re-use. In Proceedings of the 26th annual international ACM SIGIR conference Research and development in informaion retrieval, pages 72-79, 2003.
[4] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolutic image synthesis. In Forty-first international conference machine learning, 2024.
[5] Xiaoxiao Guo, Hui Wu, Yu Cheng, Steven Rennie, Gerald Tesauro, and Rogerio Feris. Dialog- based interactive image retrieval. Advances in neural information processing systems, 31, 2018.
[6] Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. arXiv preprint arXiv:2412.04431, 2024.
[7] Hykilpikonna. Vram requirement #7. GitHub Issues, 2024. GitHub issue on HiDream-II repository.
[8] Tero Karras, Samuli Laine, and Timo Aila. style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 4401-4410, 2019.
[9] Naoko Koide-Majima, Shinji Nishimoto, and Kei Majima. Mental image reconstruction from human brain activity: Neural decoding of mental imagery via deep neural network-based bayesian estimation. Neural Networks, 170:349-366 2024.
[10] Black Forest Labs. Flux. http://///gtthucccoa/aaaaaaooore forsst-labssffurg 2024.
[11] Saehyung Lee, Sangwon Yu, Junsung Park, Jihun Yi, and Sungroh Yoon. Interactive text-to- image retrieval with large language models: A plug-and-pll approach. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 791-809, 2024.
[12] Matan Levy, Rami Ben-Ari, Nir Darshan, and Dani Lischinski. Chatting makes perfect: Chat- based image retrieval. Advances in Neural Information Processing Systems, 36:61437-6114999 2023.
[13] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Mlaa--nettinterllavw Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024.
[14] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-ima imaaee pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730-19742 PMLR, 2023.
[15] You Li, Fan Ma, and Yi Yang. Imagine and seek: Improving composed image retrieval with an imagined proxy. arXiv preprint 4xX1v:241..1652,, 2024.
10

---

[16] Tsung- Yi Lin, Michael Maire, Serge Belongie. James Hays, Pietro Perona, Deva Ramanan, Piotr Dollär, and C Lawrencee Zitnick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pages 740-755. Springer, 2014.
[17] Minghao Liu, Zonglin Di, Jiaheng Wei, Zhongruo Wang, Hengxiang Zhang, Ruixuan Xiao, Haoyu Wang. Jinlong Pang. Hao Chen, Ankit Shah, et al. Automatic dataset construction (adc): Sample collection, data curation, and beyond. arXiv preprint arXiv:2408. 1338, 2024.
[18] Gary Marchionini. Exploratory search: from finding to understanding. Communications of the ACM, 49(4):41-46 2006.
[19] Fengran Mo, Kelong Mao, Ziliang Zhao, Hongjin Qian, Haonan Chen, Yinuo Cheng. Xiaoxi Li, Yutao Zhu, Zhicheng Dou, and Jian- Yun Nie. A survey of conversational search. arXiv preprint arXiv:2410. 15576, 2024.
[20] Paul Ogilvie and Jamie Callan. Combining document representations for known-item search. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 143-150 2003.
[21] Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting eagion-to-phrase correspondences for richer image-to-sentenc models. In Proceedings of the IEEE international conference on computer vision, pages 2641-2649, 2015.
[22] Qi Qin, Le Zhuo, Yi Xin, Ruoyi Du, Zhen Li, Bin Fu, Yiting Lu, Jiakang Yuan, Xinyue Li, Dongyang Liu, et al. Lumina-image 2.0: A unified and efficient image generative framework. arXiv preprint arXiv:2508.21758, 2025.
[23] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-878 8763. PMLR, 2021.
[24] Yong Rui, Thomas S Huang, and Shih-Fu Chang. Image retrieval: Current techniques, promising directions, and open issues. Journal of visual communication and image representation, 10(1):39-62, 1999.
[25] Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503 2025.
[26] HiDream AI Team. Hidream-il. https::/se https://githhb com
I1, 2025.
[27] Bart Thomee and Michael Lew. Interactive search in image retrieval: a survey. International Journal of Multimedia Information Retrieval, 1:71-86, 2012.
[28] Antti Ukkonen, Pyry Joona, and Tuukka Ruotsalo. Generating images instead of retrieving them: Relevance feedback generative adversarial networks. In Proceedings of the 43rd international acm sigir conference on research and development in information retrieval, pages 1329-1338, 2020.
[29] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vasion-language model's perception of the world at any resolution. arXiv preprint 2rXiv:2409.12191, 2024.
[30] XuDong Wang, Xingyi Zhou, Alireza Fathi, Trevor Darrell, and Cordelia Schmid. Visual lexicon: Rich image features language space. arXiv preprint arXiv:2412.06774, 2024.
[31] Chen Wei, Chenxi Liu, Siyuan Qiao, Zhishuai Zhang, Alan Yuille, and Jiahui Yu. De-diffusion makes text a strong mooss- interface. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 13492-13503, 2024.
11

---

[32] Ryen W White and Resa A Roth. Exploratory search: Beyond the guery-respons, paradigm. Morgan Claypool Publishers, 2009.
[33] Barbara M Wildemuth and Ann O'Neill. The" known" " in known-item searches: Empirical support for user-centered design (research note). College & Research Libraries, 56(3):265-281, 1995.
[34] Diji Yang, Kezhen Chen, Jinmeng Rao, Xiaoyuan Guo, Yawen Zhang, Jie Yang. and Yi Zhang. Tackling vision language tasks through learning inner monologues. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 19350-19358, 2024.
[35] Diji Yang. Jinmeng Rao, Kezhen Chen, Xiaoyuan Guo, Yawen Zhang, Jie Yang, and Yi Zhang Im-rag: Multi-round retrieval-augmented generation through learning inner monologues. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 730-740, 2024.
[36] Hee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Kang Zhang, Yu-Jung Heo, Du-Seong Chang, and Chang D Yoo. Bi-mdrg: Bridging image history in multimodal dialogue response generation. In European Conference on Computer Vision, pages 378-396. Springer, 2024.
[37] Zijia Zhao, Longteng Guo, Tongtian Yue, Erdong Hu, Shuai Shao, Zehuan Yuan, Hua Huang, and Jing Liu. Chatsearch: A dataset and generative retrieval model for general conversational image retrieval. Pattern Recognition, page 11666, 2025.
[38] Xiang Sean Zhou and Thomas S Huang. Relevance feedback image retrieval: A comprehen- sive review, Multimedia systems, 8:536-544, 2003.
12

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and introduction accurately reflect the paper's contributions: (1) defining the Mental Image Retrieval (MIR) task, (2) proposing GenlR as generative approach using diffusion models for visual feedback. (3) introducing a dataset creation pipeline, and (4) demonstrating performance improvements across multiple datasets. These claims are supported by the experimental results shown throughout the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
13

---

Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitation section explicitly discusses limitations, noting that the VLM simulation assumes users have clear mental images, while real users often have partial representations. The paper acknowledges the framework doesn't I account for how mental images evolve during search, and suggests human studies for future work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texs system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
This paper is primarily empirical and method-focusee without theoretical results requiring formal proofs. presents an algorithm and methodology rather than theoretical guarantees.
Justification:
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
14

---

Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper describes datasets (MSCOCO, FFHQ, Flickr30k, Clothing-ADC), models used (Gemma3, various diffusion models), and evaluation metrics (Hits@ 10). Hy- perparameters will be included in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We currently present all code (GenIR framework and data curation pipeline) and dataset in an anonymous repository, which will be fully released to the research commu- nity when the paper is published.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new spun-source benchmark).
15

---

reproducible, they The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips. //tpss.c/publltti. http:/ubllcgguudde Sttps//ooeSSuiiisseeeeeee wwwiissssosiiiiieo
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper provides experimental settings including datasets, models, and evaluation framework. Hyperparameters will be included in the appendix.
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Our experiments follow standard evaluation protocols in the image retrieval literature by reporting Hits@ 10 metrics on large-scale benchmark datasets (MSCOCO 50k images, FFHQ 70k images, Flickr30k 31.7k images, and Clothing-ADC 1M+ images). Statistical significance is ensured through the large sample sizes used in each dataset, consistent with common practice the field.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
16

---

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: In the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: / /ttp::///euurr htt///eurrpp.ure httpp/cccppbcccbbeeeeeeeee pttpbliittthiccccuudeliieeeese
Answer: [NA]
Justification: The paper doesn't explicitly mention conforming to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper includes broader impacts section the appendix that thoroughly discusses both potential benefits (enhanced user experience, accessibility improvements, creative applications, educational uses) and potential risks (algorithmic bias, computational resource requirements, privacy considerations).
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended
(e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
17

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper doesn't 't appear to release models with high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Cited, rules followed.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, saperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
Well documented how to generate, and the generation code will be released.
18

---

Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper uses VLM simulation rather than human subjects for experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects research conducted.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper doesn't 't mention using LLMs in the research methodology itself (though does use VLMs as part of the experimental setup).
Guidelines:
19

---

The answer NA means that the core method development in this research does not involve LLMs as any important. original, non non-standard components.
Please refer to our LLM policy (https : http///eerppppepeeee /c//onfferences/2025/// /LLM) for what should or should not be described.
20

---

A Additional results
A.1 Limitations of Verbal-Only Feedback
Figure illustrates a limitation of verbal-only feedback in interactive image retrieval. this ChatlR example, when asked "is he wearing hat?", the answer is "no," which is technically correct since the motorcyclist is wearing a helmet, not a hat. However, this verbal exchange could mislead the retrieval system by implying the person's head is uncovered, when in fact they wearing protective headgear- an important visual attribute. Such semantic gaps in verbal feedback can lead to suboptimal query refinement.
Visual feedback approaches like GenlR potentially address this limitation by providing a synthetic image that would show the helmet, allowing users to immediately identify this discrepancy. This example demonstrates how visually grounded feedback can complement verbal descriptions by capturing visual details that might otherwise be lost in text-only exchanges, potentially leading to more accurate query refinement.
Dialogue:
"a man sits on motorcycle next to a very blue body of water"
"can you male or female? male" "i he facing the camera? yes"
any people? just 1"
"is
"is he happy? don't know"
"is he wearing hat? no" "what color is his hair? can't see hair"
"is it daytime? yes"
"is it sunny? yes" "can you see the sky? no" "any animals? no"
Figure 6: Example from ChatlR showing misleading verbal feedback. The highlighted question- answer pair demonstrates how verbal feedback can be technically correct but misleading the person not wearing hat but wearing a motorcycle helmet, a critical visual detail that verbal-only feedback fails to capture appropriately, potentially degrading retrieval performance.
A.2 Comparative Analysis of GenlR Dataset Utility
To demonstrate the utility of our dataset, we conducted a comparative analysis between three different retrieval approaches: (1) GenlR Synthetic Images using our generated synthetic images for image- to-image retrieval, (2) GenlR Text using text queries generated through our GenlR framework for text-to-image retrieval, and (3) ChatlR Text- using verbal feedback-based text queries for text-to- image retrieval. Table 1 presents the Hits @ 10 performance on MSCOCO across dialog lengths. The results clearly demonstrate the superiority of visual feedback, with GenlR Synthetic Images achieving 89.71 % even at initialization and 98.01 % after 10 rounds. Notably, even the text queries generated through our GenlR framework significantly outperform ChatlR's verbal feedback approach (92.33% vs. 73.64% at round 10), confirming that the GenlR dataset contains higher-quality annotations that better capture user intent compared to purely text-based interactions.
A.3 Visualization of Query and Image Refinement Process
The effectiveness of GenlR stems from its ability to provide explicit visual feedback that guides query refinement. Figure 7 illustrates this progressive refinement through multiple interaction rounds, highlighting how both textual queries and generated images evolve toward better alignment with the target mental image.
As shown the figure, the initial queries tend to be verbose and contain extraneous details, resulting in generated images that capture the general scene composition but miss critical details or relationships. For example, in Round 0, the system generates an image showing two giraffes instead of the intended scene with one giraffe interacting with person.
21

---

Table 1: Comparison of retrieval approaches across dialog lengths (Hits @ 10%)
Dialog Length
ChatiR Text
GenlR Text
GenlR Synthetic Images
60.56
74.47
89.71
65.26
79.09
93.11
2
68.36
83.41
95.00
3
70.06
85.69
95.97
71.32
87.63
96.51
5
72.14
88.79
96.85
72.63
89.71
97.14
7
72.97
90.39
97.48
8
73.26
90.98
97.67
9
73.50
91.75
97.72
10
73.64
92.33
98.01
Through subsequent rounds of feedback and refinement, the queries become increasingly precise and focused on the key visual elements that distinguish the target image. By Round 3, the query has been simplified and clarified to explicitly mention "a giraffe head near man's face," resulting in generated image that better captures the spatial relationship between the human and animal subjects.
By the final round (Round 10), the refinement process has successfully addressed the most important details-the giraffe's 's posture ("lowers head towards him"), the man's 's appearance ("glasses and an orange shirt"), and the proper spatial arrangement. This progression demonstrates how GenlR's visual feedback mechanism enables users identify discrepancies and iteratively align the system's representation with their mental image.
Round
Round 3
Round 6
Round 10
Target Image
Round
Close-up shot, giraffe head human face side by side, bright sunny day, clear blue sky background, lush green trees and foliage blurred the distance, low angle perspective... Close-up of giraffe head near man's face sunny day with clear blue sky. The giraffe has patterned coat of brown spots on light tan base long eyelashes. The man, wearing glasses and an orange shirt, smiles as looks up at the giraffe...
Round 3
Round
person smiles while standing near giraffe, both viewed close-up shot. The giraffe distinctive spots a calm expression, face partially touching the man't shoulder. The man wears glasses and an orange shirt...
Round 10
man wearing glasses and an orange shirt smiles giraffe lowers its head towards him outside. The giraffe has distinctive brown spots and a calm expression, its muzzle close to the man's 's car. A clear blue sky forms the backdrop...
Figure 7: Visual progression of GenlR's 's image refinement process across multiple rounds. The top row shows generated images evolving from initial generation (left) through intermediate rounds to final output (right), alongside the target image (far right). Below, the corresponding query texts show how descriptions become more precise and focused with each iteration. Note how the generated images progressively capture more accurate details the spatial relationship between man and giraffe, facial features, lighting conditions, and background elements.
22

---

B Additional Experimental Details
B.1
Hyperparameters
We provide the hyperparameters used for each of our experimental settings to ensure reproducibility:
B.1.1
Diffusion Models Inference
Table Hyperparameters for diffusion model inference
Model
Inference Steps
Guidance Scale
Image Resolution
Infinity
N/A
3.0
1024 x 1024
Lumina-Image
50
4.0
1024 x 1024
Stable Diffusion 3.5
28
3,5
1024 x 1024
FLUX.1
5
3.5
1024 x 1024
HiDream- II-Fast
16
0.0
1024 x 1024
B.1.2 HiDream-I1 Model Adaptation
For our experiments with the HiDream-II model, we utilized modified version compared the original implementation available on GitHub. The standard HiDream-II model, which incorporates flow matching with dual CLIP encoders, T5, and Llama3. with 128 text tokens, requires approxi- mately 55GB of VRAM for inference. Since our experiments were conducted on NVIDIA A6000 GPUs with 48GB VRAM, we employed the HiDream -1-Fast variant with 4-bit quantization using the Bits B approach. Following the implementation strategy by Hykilpikonna [7], we applied forch.flloatl precision and set low_cpu_ment ussgu--Tuureeeeeeeee cru__mmem_sssg--rreee for all model components. This optimization reduced the memory footprint to under 30GB, enabling inference while maintaining reasonable generation quality. We observed that this quantized model preserved the essential char- acteristics needed for visual feedback experiments, with minimal impact the final retrieval performance.
B.1.3 Image Retrieval Pipeline
For all image-to-image retrieval experiments, used BLIP-2 with the following configuration:
Feature dimension: 256
Similarity metric: Cosine similarity
Normalization: L2
B.1.4
Vision-Language Models
For Gemma3 (both 4B and 12B variants), used the following parameters:
Temperature: 0.7
Top-p: 0.9
Max tokens: 500
Repetition penalty: 1.1
Sampling method: Greedy with temperature
B.2 Compute Resources
All experiments were conducted using 4 NVIDIA A6000 GPUs with 48GB of VRAM each. The diffusion model inference image generation was the most computationally intensive component of our pipeline, taking approximately 20 seconds per image generation at 1024 x 1024 resolution. The complete experimental suite, including all datasets and interaction rounds, required approximately 200 GPU hours complete.
23

---

C Broader Impacts
C.1 Potential Benefits
Enhanced User Experience: By providing visual feedback, GenlR makes image retrieval more intuitive and aligns with how humans naturally think, potentially reducing frustration in search tasks.
Accessibility Improvements: People who struggle with articulating precise textual queries (including those with language barriers or linguistic challenges) may find visually-guide guidee search more accessible.
Artists, designers, and content creators could more effectively find visual references that match their mental concepts, enhancing creative workflows.
Creative Applications:
Educational Uses: Teachers and students could more efficiently locate visual materials that align with conceptual understanding rather than relying solely on keyword matching.
C.2 Potential Risks
Algorithmic Bias: The diffusion models used for image generation may reproduce or amplify biases present in their training data, potentially leading to unfair representation in search results.
Computational
Resource Requirements: The use of generative models increases com- putational demands, which has both accessibility implications (requiring more powerful hardware) and environmental considerations (increased energy consumption).
Privacy Considerations: As systems become better at representing users' mental images, questions arise about what information about user preferences and thinking might be inferred or stored.
Our approach aims to maximize the benefits while mitigating these potential risks through ongoing research and refinement of the methodology.
D
Human Evaluation of Visual Feedback Utility
D.1 Motivation
While our main experiments utilize VLM simulation (Gemma3) to evaluate the GenIR framework. we acknowledge that this approach cannot fully capture the nuanced ways humans form and refine mental images during search. VLM simulation, while effective for large-scale evaluation, may not accurately reflect how real users would interpret and benefit from visual feedback.
D.2 Human Annotation Study
To address this limitation, we conducted a small-scale human evaluation study with the following methodology:
Dataset: We selected 100 datapoints from GenIR dataset. The evaluation primarily
focused on comparing the ninth-round generated images with the actual target images.
Round Selection Rationale: We specifically chose the ninth round because earlier rounds (1-6) typically captured broad image content but lacked significant details, while later rounds (7-9) produced more stable and detailed images. As the ninth round represents the final iteration in our framework, it provides the most refined visual representation for evaluation.
Annotation Task: One human annotator evaluated each pair and classified whether the generated image was helpful for potential query refinement (binary classification: useful/not useful).
The annotator assessed whether the synthetic image provided visual cues that would be valuable for further query refinement, focusing on whether the differences between generated and target images revealed actionable refinement opportunities.
Evaluation Criteria:

---

D.3 Results
Our human evaluation revealed that in 86% cases, the synthetic images were judged as useful for query refinement. This strongly aligns with quantitative results showing improved retrieval performance with visual feedback. Key observations include:
Visual feedback was particularly helpful for refining fine-grained attributes (e.g., specific colors, textures, and spatial relationships) that are difficult to express precisely in text.
In cases where the initial query vague, the synthetic image helped clarify the system's interpretation, allowing for more targeted refinements.
The instances where visual feedback was deemed unhelpful typically involved significant distortions or misinterpretations the generated image that confused rather than clarified the search intent.
D.4 Limitations and Future Work
This human evaluation, while informative, has several limitations:
Limited scale (100 samples) and single annotator
The controlled setting differs from real-world search scenarios where users may have incomplete mental images
The study does not capture the dynamic evolution of mental images during search
These limitations highlight the need for more comprehensive human-in-the-loww studies in future work. We plan to conduct larger-scale user studies with diverse participants better understand how different user groups interact with and benefit from visual feedback in mental image retrieval tasks.
D.5 Failure Mode Analysis of Visual Feedback
While evaluation demonstrates that visual feedback is useful in 86% of cases (Section D), important to understand the failure modes where generated images can mislead the refinement process. We conducted focused analysis of cases where GenlR failed retrieve the target image, categorizing errors into three primary types based on how the generated visual feedback provided incorrect cues.
Table 3 presents our failure mode taxonomy with descriptions and representative examples from MSCOCO dataset:
Table 3: Failure Mode Taxonomy: Categories of visual feedback errors that can mislead query
refinement in GenIR
Error Type Limited Improvement
Description
Example (MSCOCO)
The generated image shows minimal visual change from the previous round despite query refinement, providing guide further refinement.
Image 000000004952: Gener- ated images from rounds 8 and are nearly identical, stalling the refinement process.
information to
Hallucination Content
The generated image contains objects or scene elements that were not part of the user's intent, diverting the refinement process toward irrele- vant details.
Image 000000020415: The 8th- round generated image halluci- nates shower head the right side, an element entirely absent in the ground-trute misguiding subsequent queries.
Retrieval- Detail alignment
Mis-
The generated image misrepresents details that are erucial retrieval but may perceived as trivial by observers, creating a gap between visual feedback and retrieval objectives.
Image 000000U26494; A long wooden bench the ground- truth is incorrectly rendered as a single wooden chair. While scemingly minor, this detail is critical for accurate retrieval.
25

---

Analysis of Failure Patterns Limited Improvement (stagnation) typically occurs in later rounds (rounds 7-10) when the model has exhausted its capacity to refine the generation based on incremental textual changes. This suggests limitation in the diffusion model's sensitivity to subtle query modifications.
Hallucination Content represents the most problematic failure mode, as it actively misleads users by introducing false visual elements. These hallucinations often stem from the diffusion model's lendency to complete scenes with common co-occurring objects, even when not specified in the query.
Retrieval- Detail Misalignment highlights a fundamental challenge: what appears visually acceptable to humans may miss crucial details that distinguish the target in the retrieval space. This suggests the need for retrieval-aware image generation that prioritizes discriminative details.
Visualizations Figure 8 illustrates representative examples of each failure mode with visual com- parisons between ground-truth targets and problematic generated images:
These visualizations demonstrate the current limitations of visual feedback generation and suggest future directions for improving feedback quality, such as retrieval-aware generation objectives or confidence-basee hybrid feedback strategies.
D.6
Computational Cost vs. Performance Analysis
While GenlR approach demonstrates significant performance improvements traditional feedback methods, it's important to consider these gains in relation to the computational overhead introduced by the generative process. This section provides cost-benefit analysis of our approach compared to baseline methods.
D.6.1
Comparative Computational Analysis
Table presents a comparison of computational requirements between our GenlR approach and baseline methods. As expected, the integration of cuffusion-based image generation introduces additional computational overhead compared to text-only methods like Verbal Feedback and text- based prediction feedback.
Table 4: Computational requirements comparison per interaction round
Method
Compute Time
Relative
Hits @ 10
GPU Memory
at Round
Verbal Feedback (Gemma3- 126b)
2
1.0x
89.97%
Prediction Feedback
2.5
1.2x
90.70%
GenlR (Infinity)
16
3.0x
96.85%
GenlR (FLUX.1)
12
2,5x
95.10%
GenlR (Stable Diffusion 3.5)
26
2.2x
96.02%
GenlR (HiDream-FAN) FAST)
17
2.1x
94.62%
GenlR (Lumina Humiin--Image222 Image-2.0)
27
1.3x
96.55%
Our analysis shows that GenlR with Infinity requires approximately 8 times more computation time per interaction round compared the Verbal Feedback baseline. However, this computational investment yields a 6.9% absolute improvement in Hits@ 10 on the MSCOCO dataset by the fifth interaction round.
D.6.2 Hybrid System: Balancing Performance and Efficiency
To address latency concerns while maximizing performance gains, we explored hybrid approach that intelligently selects between fast Verbal Feedback and more powerful Visual Feedback. Our analysis shows that Visual Feedback uniquely succeeds in 22.3% of cases where Verbal Feedback fails, presenting an opportunity for strategic deployment.
Table 5 presents performance comparisons across different hybrid scenarios on MSCOCO:
Verbal Feedback: Baseline using text-only feedback (fastest, 2s/round)
26

---

Case 1: Limited Improvement (Image 04952)
Round Generated
Round Generated
Case 2: Hallucination Content (Image 20415)
Ground Truth
Round 8 Generated
Case 3: Retrieval-Detii Misalignment (Image 26494)
Ground Truth (bench)
Round 9 Generated (chair)
Figure 8: Visual examples of GenlR failure modes. Top: Stagnation between rounds 8-9, Middle: Hallucinated shower head. Bottom: Bench misrepresented chair.
27

---

Visual Feedback (Ours): GenlR with visual feedback (slower, 16s/round)
Hybrid Oracle: Perfect selection of when to use Visual vs. Verbal feedback
Random Select: Random selection using Visual Feedback for 22.3% of queries
Table 5: Hybrid System Performance: Comparison of Verbal Feedback, Visual Feedback, and Hybrid approaches (Hits@) 10%). The hybrid system uses Visual Feedback for 22.3% of queries and Verbal Feedback for 77.7%.
Dialog Length
Verbal Feedback
Visual Feedback
Hybrid Oracle
Random Select
Verbal ~Oracle
Verbal Random
74.48
89.71
93.40
77.88
+18,92
+3.40
1
82.73
93.11
95.97
85,06
+13,25
+2.33
2
85.83
95.00
97.43
87.89
+11.60
+2.06
87.97
95.97
97.91
89.76
+9.95
+1.79
89.13
96.51
98.20
90.81
+9.07
+1.68
89.96
96.85
98.30
91.50
+8.35
+1.54
6
90.49
97.14
98.59
91.98
+8.10
+1.49
90.98
97.48
98.64
92.42
+7.67
+1.44
91.27
97.67
98.79
92.69
+7.52
+1.42
91.80
97.72
98.79
93.13
+6,99
+1.33
10
92.33
98.01
98.88
93.62
+6.55
+1.29
Key findings from the hybrid system analysis:
Even random selection yields meaningful gains (+3.40% absolute improvement at Round 0) with only +3 seconds average latency increase per query MSCOCO.
The oracle hybrid system demonstrates the upper bound performance (+18.92% provement Round 0), showing significant room for improvement through intelligent query difficulty estimation.
In difficult cases where verhal feedback requires many rounds or fails to converge, GenlR's ability succeed in fewer rounds actually be more time-efficient overall.
This analysis demonstrates that hybrid approaches offer promising middle ground, providing substantial performance improvements with manageable latency overhead. Future work could develop learned policies to intelligently select between feedback modalities based on query characteristics.
D.6.3 Optimizations and Efficiency Improvements
Several strategies can potentially reduce the computational overhead of our approach:
Model Distillation: Smaller, distilled versions of diffusion models could reduce generation time with minimal performance degradation.
Early Stopping: For many queries, acceptable performance be achieved with fewer diffusion steps or feedbake iterations.
Implementing a policy to skip generation in certain rounds where minimal refinement is expected could reduce overall computation.
Adaptive Generation:
D.6.4 Real-World Deployment Considerations
The computational cost-benefit analysis varies significantly based on deployment context:
Interactive Search Applications: The improved user experience and reduced number of interaction rounds may justify the additional per-round computation, especially since generation can be performed asynchronously while users review results.
Batch Processing: For offline applications where multiple images need to be retrieved based on descriptions, the computational overhead may be prohibitive compared traditional methods.
28

---

Specialized Domains: In domains requiring high retrieval precision (e.g., medical imaging. satellite imagery), the performance improvements may justify computational costs regardless of application type.
Overall, while GenlR introduces non-trivial computational overhead, our analysis suggests that for many interactive retrieval scenarios, the performance gains and improved experience justify the additional computational investment. Future work will focus on efficiency optimizations to further improve the performance-to-com ratio of our approach.
E Limitation and Future Work
The current study lays the groundwork for Mental Image Retrieval (MIR) using generative visual feedback, and such, its scope invites several avenues for future expansion:
User Simulation for Initial Exploration Our use of Vision-Language Models (VLMs) to simulate user interaction is deliberate methodological choice for this initial investigation of MIR. This approach aligns with common practice in pioneering new interactive Al tasks (e.g., as seen in prior interactive retrieval works [11, 12]), allowing for controlled, scalable, and reproducible exploration of the core GenlR framework. While this simulation provides a valuable starting point by assuming users have relatively clear and fixed mental target, we recognize that real human users often begin with more partial or fuzzy mental representations. Future work should build upon our findings by conducting extensive human-in in-the-loop studies. Such studies will be crucial for understanding how users with varying degrees of mental image clarity interact with GenlR and for refining the system to accommodate these more naturalistic scenarios.
Dynamic Mental Image Evolution The dynamic evolution of user's mental image during the search process, where the act of searching and receiving feedback can clarify or alter their internal representation, is fascinating and complex aspect of human cognition. While our current work focuses on establishing the efficacy of generative visual feedback for a given mental target, investigating these interactive dynamics where the mental image itself co-evolves with system understanding was beyond the scope of this foundational study. We consider this a significant direction for future research. Exploring how GenlR can support or leverage this iterative refinement of the user's own memory presents rich area for subsequent investigation.
Reinforcement Learning-Bas Based System Optimization While our current plug-and- play design intentionally provides flexibility for researchers to leverage state-of-the-ard or nomain-customiin models, end-to-end optimization of the GenIR framework presents exciting opportunities for future work. We envision optimization from two complementary perspectives: the framework architecture and the unique dataset it produces.
GenIR as a Multi-Agent System. At its core, GenlR functions as multi-agent system where the generator and retriever collaborate solve retrieval tasks. By introducing explicit visual feedback into this loop, we create an interactive paradigm potentially amenable to established multi-agent optimization techniques. The modular design makes GenlR flexible testbed where researchers investigate how different agent capabilities impact system dynamics and optimization potential.
Trajectory-Riel Dataset for RL. Our dataset provides unique opportunities for system optimization through its rich, multi-step interaction trajectories:
Warm-up Training: Successful (guery generated image retrieval result) trajec- tories can provide initial supervised training before RL fine-tuning. This warm-up stage, standard practice for improving RL stability and sample efficiency, is particularly crucial for navigating the vast action space inherent to large generative models.
Trajectory- -Based Optimization: Full interaction histories enable advanced trajectory- based optimization through process supervision (e.g., training reward models) rather than relying solely on delayed final rewards. This allows the system to learn from intermediate feedback quality, not just final retrieval success.
29

---

Diagnostic Analysis: Intermediate generated images provide unprecedented insight into how and why generative models misunderstand queries in task-oriented settings. This offers direct, actionable guidance for system-level debugging and task-specific optimization.
Optimization Strategy. We envision freezing the retriever while optimizing the image generator end-to-end. This design choice offers flexibility-the retriever need not be trainable deep learning model, allowing use of fixed systems like Google Search or Nomain-sseciif rule-based search. For the generator, successful optimization depends on two objectives: (1) correctly reflecting text queries visually, and (2) distinguishing between retrieval targets and incorrect candidates in the image space, ensuring seamless component collaboration.
These outlined possibilities represent starting points for engaging the research community in multi- agent system optimization. We believe the community will discover many additional opportunities building on this foundation. Our work provides essential building blocks-a flexible framework and trajectory-rich to unlock the full potential of generative visual feedback in interactive Al tasks.
30


Reviews:
[{"review_id": "sLlDF9fWHv", "reviewer": "Reviewer_6Fz4", "summary": "This paper introduces Mental Image Retrieval (MIR) as a novel task addressing the gap between standard text-to-image retrieval and real-world search behavior, where users iteratively refine queries based on a mental image. The authors propose GenIR, a multi-round retrieval framework leveraging text-to-image diffusion models to generate synthetic visual feedback at each interaction step. This synthetic image explicitly reveals the system’s interpretation of the user’s query, enabling intuitive refinement. Experiments show significant gains over verbal-feedback baselines (e.g., +15–20% Hits@10 on FFHQ/Clothing-ADC) across four datasets, with robustness to generator quality and model scale.", "questions": "1. Conduct larger-scale human studies (≥50 users) comparing GenIR vs. verbal feedback for tasks with fuzzy mental images (e.g., \"find a scenic mountain photo I saw last year\"). If human studies show statistically significant improvements (e.g., +20% success rate in ≤5 rounds), this strengthens real-world applicability. If results are marginal, significance claims may weaken.\n2. Analyze cases where generated feedback misleads refinement (e.g., Appendix’s \"unhelpful\" cases). Quantify error types (e.g., hallucinated objects, incorrect attributes).\n3. Compare GenIR against a hybrid baseline (e.g., generate images only when verbal confidence is low) or discuss optimizations (e.g., distillation to faster generators).", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n1. Addresses a critical limitation of VLMs in real-world search settings. The task definition, strong results on 1M+ image datasets (Clothing-ADC), and model-agnostic framework provide foundations for future work.\n2. The integration of generative models for visual feedback in interactive retrieval is new. Prior work (e.g., ChatIR, PlugIR) relies on textual dialogue, while GenIR’s image-based feedback offers a tangible advantage for disambiguation. Rigorous experiments across diverse domains (COCO, FFHQ, Flickr30k, Clothing-ADC), ablation studies (diffusion models, VLM sizes), and automated dataset validation (Table 1) support claims.\n\n# Weaknesses\n1. Reliance on VLM simulations (Gemma3) instead of human users weakens ecological validity. Human evaluation (Appendix D) is limited (100 samples, single annotator).\n2. Computational overhead of iterative image generation (Table 3) is under-discussed. Latency (16–27s/round) may limit real-world deployability.\n3. While the framework is novel, components (diffusion models, image retrieval) are off-the-shelf. The claim of \"first generative feedback for retrieval\" may be accurate but could be more nuanced.", "was_revised": true, "final_justification": "Satisfied that the majority of my concerns have been resolved, I vote to accept.", "rebuttal": "We thank the reviewer for their valuable feedback and for acknowledging the significance of the MIR task, the novelty of our framework, and the rigor of our experiments. We appreciate the opportunity to address the important points raised.\n\n***W1/Q1: Size of human evaluation and future larger-scale human studies.***\n\nWe agree that large-scale human studies are the gold standard for evaluating interactive systems. However, our use of VLM simulation was a deliberate and necessary methodological choice for this foundational work, intended to establish the core technical validity of our proposed paradigm.\n\n**Our primary scientific objective was to rigorously test the hypothesis that generative visual feedback is fundamentally superior to verbal feedback.** To do this, we needed to isolate the impact of the feedback mechanism itself. A controlled, reproducible simulation was scientifically necessary to perform the large-scale quantitative analysis (across 4 datasets, including one with 1M+ images) required to validate this hypothesis. This approach, common in pioneering new interactive tasks (e.g., ChatIR [12], PlugIR [11]), allowed us to robustly demonstrate significant performance gains that would be impossible to establish in a preliminary human study.\n\nOur work serves as the crucial proof of concept that modern generative models are now capable of supporting this new retrieval paradigm. We see our framework and dataset not as the final word, but as **the first research step towards Mental Image Retrieval**. By providing a high-performing, open-source testbed (codebase and dataset), we have lowered the barrier to entry, enabling the broader range of AI community beyond ML researchers, e.g., HCI and HAI experts, to now investigate the more nuanced questions of fuzzy mental models and evolving user intent that were previously intractable. This controlled validation was a necessary prerequisite to justify the significant investment of complex, costly human-in-the-loop studies, which is the clear and exciting next phase of this research program. Our preliminary human evaluation in Appendix D, which found 86% of the visual feedback to be useful, serves as an initial validation that corroborates our main findings.\n\n*[11] Lee, Saehyung, et al. \"Interactive Text-to-Image Retrieval with Large Language Models: A Plug-and-Play Approach.\" ACL. 2024.*\n\n*[12] Levy, Matan, et al. \"Chatting makes perfect: Chat-based image retrieval.\" NeurIPS 2023.*\n\n***W2/Q3: Latency concern and future optimization***\n\nWe agree that latency is a critical factor for real-world deployment. While our paper does not claim a contribution in efficiency, our plug-and-play framework is designed to readily incorporate state-of-the-art techniques from the Efficient Generative AI community, such as faster diffusion models[3], faster computing[4], as well as future advances.\n\nTo directly illustrate the cost-benefit trade-off, we performed an analysis to quantify the potential of a **hybrid system** that chooses between fast Verbal Feedback and our more powerful Visual Feedback.\n\nOur analysis shows that Visual Feedback **uniquely succeeds in 22.3% of cases** where Verbal Feedback fails. We present two hybrid scenarios: (1) an **\"Oracle\"** that perfectly selects when to use Visual Feedback, and (2) a **\"Random Select\"** baseline that randomly chooses Visual Feedback for 22.3% of queries.\n\nAs shown in the table 1, even random selection yields a **+3.40% absolute improvement** (74.48% to 77.88%) at Round 0, while the oracle achieves **+18.92%**. For our experiment on MSCOCO, the average latency increased by just **+3 seconds per query**. This demonstrates that even a simple hybrid approach provides **meaningful gains**, with **significant room for improvement** through intelligent selection mechanisms. Furthermore, in difficult cases where verbal feedback requires many rounds (or hit max rounds due to continuous unsuccessful attempts), GenIR's ability to **succeed in fewer rounds** can actually be more time-efficient overall. We will add this full analysis to the final paper.\n\n\n### Table 1: Comparison of using Verbal Feedback, Visual Feedback, and Hybrid Systems for Image Retrieval (Hits@10)\n\n*22.3% Visual / 77.7% Verbal*\n| Dialog Length | Verbal Feedback | Visual Feedback | Hybrid Oracle | Random Select | Verbal->Oracle | Verbal->Random |\n|--------------|-----------------|-----------------|---------------|---------------|----------------|----------------|\n| 0            | 74.48%         | 89.71%         | 93.40%        | 77.88%        | +18.92%        | +3.40%         |\n| 1            | 82.73%         | 93.11%         | 95.97%        | 85.06%        | +13.25%        | +2.33%         |\n| 2            | 85.83%         | 95.00%         | 97.43%        | 87.89%        | +11.60%        | +2.06%         |\n| 3            | 87.97%         | 95.97%         | 97.91%        | 89.76%        | +9.95%         | +1.79%         |\n| 4            | 89.13%         | 96.51%         | 98.20%        | 90.81%        | +9.07%         | +1.68%         |\n| 5            | 89.96%         | 96.85%         | 98.30%        | 91.50%        | +8.35%         | +1.54%         |\n| 6            | 90.49%         | 97.14%         | 98.59%        | 91.98%        | +8.10%         | +1.49%         |\n| 7            | 90.98%         | 97.48%         | 98.64%        | 92.42%        | +7.67%         | +1.44%         |\n| 8            | 91.27%         | 97.67%         | 98.79%        | 92.69%        | +7.52%         | +1.42%         |\n| 9            | 91.80%         | 97.72%         | 98.79%        | 93.13%        | +6.99%         | +1.33%         |\n| 10           | 92.33%         | 98.01%         | 98.88%        | 93.62%        | +6.55%         | +1.29%         |\n\n\n*[3] Chen, Junsong, et al. \"Sana-sprint: One-step diffusion with continuous-time consistency distillation.\" arXiv preprint arXiv:2503.09641 (2025).*\n\n*[4] Wu, Chengyue, et al. \"Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding.\" arXiv preprint arXiv:2505.22618 (2025).*\n\n***W3: Claimed novelty***\n\nWe thank the reviewer for affirming the framework's novelty. We agree on the nuance of meaning within our claim. Meanwhile, we believe that our claim in line 114, i.e., “... **the first work to integrate text-to-image generation into an interactive retrieval setting, enabling a closed-loop interaction that unifies generation, retrieval, and feedback within a single framework.**” is accurate. Our core novelty is not merely the use of a generator, but the proposal of a new interactive paradigm and the system-level framework (GenIR) that realizes it. The originality lies in the generate-visualize-retrieve closed loop, which fundamentally changes user-system interaction by making the system's internal understanding explicit and actionable.\n\n***Q2: Analyze on failure examples***\n\nThis is an excellent suggestion. We have conducted a focused analysis of failure modes where generated feedback can mislead the refinement process. These errors primarily occur when the generated image provides incorrect visual cues. We have categorized them as follows:\n\n| Error Types | Description | Example (MSCOCO) |\n|-------------|-------------|------------------|\n| **Limited improvement** | The generated image shows minimal visual change from the previous round despite query refinement, providing no new information. | For image_000000004952, the generated images from the 8th and 9th rounds are nearly identical, stalling the refinement process. |\n| **Hallucination content** | The image contains objects or scene elements that were not part of the user's intent, diverting the refinement process. | For image_000000020415, the 8th-round image hallucinates a shower head on the right side of the frame, an element entirely absent in the ground-truth, which could misguide the user's next query. |\n| **Retrieval-detail misalignment** | The generated image misrepresents details that are crucial for retrieval but may be perceived as trivial by observers, creating a gap between visual feedback and retrieval objectives. | For image_000000026494, a long wooden bench in the ground-truth is incorrectly rendered as a single wooden chair in the generated image. While seemingly minor, this detail is critical for accurate retrieval but may be overlooked during refinement. |\n\nDue to the restriction on uploading images, we are unable to provide the visualization. We will add the detailed analysis and visualized examples to the appendix in the final version to provide a more complete picture of the framework's current limitations and guide future improvements."}, {"review_id": "cvFmlaaRNb", "reviewer": "Reviewer_bt51", "summary": "The authors propose a multi-round interactive image retrieval pipeline based on text-to-image generation, where retrieval is framed as an image-to-image retrieval problem. In this setup, the user can guide their next query based on the generated image corresponding to their previous text input—thereby facilitating query refinement through visual feedback. The authors introduce a multi-round dataset, where each round includes a refined query, a synthetically generated image, and retrieved results with correctness annotations. Experimental results demonstrate that GenIR outperforms existing multi-round interactive retrieval (MIR) baselines, underscoring the significant advantage of visual feedback over purely verbal feedback.", "questions": "(1) I wonder, aside from the proposed approach and the use of off-the-shelf components to address this problem, what is the core technical novelty of this work?\n\n(2) Additionally, would it be possible to apply reinforcement learning-based fine-tuning to further enhance performance? Some discussion on this aspect would be helpful.\n\n(3) Some cost-benefit analysis would be helpful, as it takes nearly 16 seconds per round (Infinity) compared to verbal feedback that takes only 2 seconds.  Please refer to D.5.1 (Computational requirements comparison per interaction round). \n\n(4) Why was fine-tuning the image generation or retrieval models not considered? \n\n(5) Does the visual feedback introduce new failure modes or misleading cues? Please refer to my detailed comment in the weakness section. \n\n(6) How well does GenIR generalise to real users beyond VLM simulation? Please refer to my detailed comment in the weakness section.", "limitations": "- The technical contribution is somewhat limited. This is more of modular adaptation that proposes a 'novel system-level' configuration for mental image retrieval, rather than developing new learning algorithms or training procedures. No fine-tuning or task-specific model adaptation is performed; the architecture is model-agnostic and plug-and-play.\n\n- However, the self-contained writing, insightful experimental analysis, and the dataset contribution are noteworthy. I am leaning towards a borderline accept, but I would also like to see comments from other reviewers and the authors’ rebuttal.\n\nMinor comments:\n\nLine 274: 'Prediction feedback is not always better verbal' -- refine English. \nLine 54: \"…fails to directly benefits users toward effective refinements.” -- benefit \nLine 74: 'exisitng' -> existing. \nLine 95: “…both these two works and their related subsequent works…” -- refine English.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "+ Overall, this is a well-written paper that is easy to follow.\n+ The authors have clearly summarized the differences from existing approaches such as ChatIR and PlugIR.\n+ If the dataset is made public, it would be a valuable contribution to further research in this area.\n\nIn particular, I am summarising the strengths as follows:\n\n(1) Instead of verbal feedback, as demonstrated by prior works like ChatIR—where a language model is used for feedback—and PlugIR—where an image captioning model is used—the authors propose using a text-to-image generation model to produce a visual feedback image that closely reflects the user’s intent. This approach offers a more interpretable means of understanding the system’s behavior through visual analysis, rather than relying solely on textual feedback as in earlier methods such as ChatIR or PlugIR.\n\n(2) This explicit visualization makes it more interpretable, allowing the user to understand the discrepancy between the previous prompt and the intended image, and how to modulate or provide the next query so that the retrieved image aligns with the one the user has in mind.\n\n(3) The authors have also developed a multi-round dataset, with each round consisting of a refined query, a generated synthetic image, and retrieved results with a correctness label. If this dataset is made public, it could initiate more research in this topic. More specifically, although the authors are not currently using any RL-based fine-tuning, this labeled dataset could be used to fine-tune the Image Generator or the Image-to-Image retrieval model in a customized way that is optimized in a more user-specific manner. For reference, the following paper could be considered: ‘DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models,’ NeurIPS 2023. https://arxiv.org/abs/2305.16381. \n\n\nNext, I am summarising the weaknesses that might be worth considering and addressing further. \n\n(1) The experimental evaluation depends on the VLM-simulated users to simulate or mimic the real user interactions in Mental Image Retrieval (MIR) tasks. This means that the users always have a clear, fixed mental image. I wonder if it will be worth it, because in the real world, mental representations are always somewhat vague at the beginning and evolve with iterative feedback. Some clarification would be helpful.  \n\n(2) Many a time, diffusion-based generative models do not capture (https://arxiv.org/pdf/2403.11821) semantically aligned latent concepts of the human intent. Visual feedback may misrepresent the user’s query, especially when generation quality is poor. This will lead to misleading verbal feedback. Some analysis is needed around this. \n\n(3) The authors have used off-the-shelf text-to-image generators like Infinity [6], Lumina-Image-2.0 [22], Stable Diffusion 3.5 [4], FLUX.1 [10], and HiDream-I1 [26] and a retrieval framework (BLIP-2).  This is more of an adaptation of existing things in the context of a visual feedback-based iterative image retrieval framework. Some discussion on how this framework could be customized in a more problem-specific manner, with specific training tricks or architectural modifications, would be beneficial. For example, RL-based fine-tuning with LoRA weights.  \n\nOverall, the novelty of this work is somewhat limited, but the self-contained writing and insightful visuals make it, in my opinion, a reasonably competitive submission. This is more of a 'novel system-level' configuration for mental image retrieval with in-depth experimental results.", "was_revised": true, "final_justification": "Most of my concerns have been carefully addressed by the authors. I have also read the comments from other reviewers. If the authors can include some real human study in the supplementary material, as raised by some reviewers, it would strengthen the evaluation. \n\nAdditionally, if the authors can suggest how RL-based fine-tuning could be leveraged as future work, it would be helpful.", "rebuttal": "We sincerely thank the reviewer for the insightful feedback and pointing out the value of the dataset, the interpretability of the GenIR framework, and the novelty of using visual feedback to reflect user intent. We are happy to clarify the excellent questions raised.\n\n***W1/Q6: VLM simulation and Generalization to Real Users***\n\nOur use of VLM simulation was a deliberate methodological choice for this foundational study. To scientifically validate our core hypothesis, i.e., the generative visual feedback is superior to verbal, we required a controlled, reproducible, and large-scale experimental setup. This allowed us to isolate the feedback variable and empirically demonstrate its significant impact on performance, a necessary prerequisite before undertaking complex and costly human studies. Furthermore, we expect that for users with the fuzzy mental images the reviewer describes, our visual feedback is even more critical. It provides a concrete artifact that helps the user clarify their own intent, a dynamic we are excited to explore in future HCI studies. We see our work as the first research step towards the Mental Image Retrieval task, providing the crucial proof of concept and the first high-performance, open-source testbed (codebase and dataset) for the community to now investigate nuanced questions, such as fuzzy mental models. \n\n\n\n***W2/Q5: Failure analysis on generated images***\n\nWe thank the reviewer for raising this important point. In Appendix D, we have investigated this issue through a human evaluation of visual feedback utility. Our analysis reveals that visual feedback can, in certain cases, mislead the verbal feedback by conveying incorrect or uninformative cues. As a complementary analysis to our main study, we have conducted a detailed analysis of failure modes, identifying three primary patterns where generated visual feedback can mislead the refinement process by providing incorrect cues.\n\n\n| Error Types | Description | Example (MSCOCO) |\n|-------------|-------------|------------------|\n| **Limited improvement** | The generated image shows minimal visual change from the previous round despite query refinement, providing no new information. | For image_000000004952, the generated images from the 8th and 9th rounds are nearly identical, stalling the refinement process. |\n| **Hallucination content** | The image contains objects or scene elements that were not part of the user's intent, diverting the refinement process. | For image_000000020415, the 8th-round image hallucinates a shower head on the right side of the frame, an element entirely absent in the ground-truth, which could misguide the user's next query. |\n| **Retrieval-detail misalignment** | The generated image misrepresents details that are crucial for retrieval but may be perceived as trivial by observers, creating a gap between visual feedback and retrieval objectives. | For image_000000026494, a long wooden bench in the ground-truth is incorrectly rendered as a single wooden chair in the generated image. While seemingly minor, this detail is critical for accurate retrieval but may be overlooked during refinement. \n\nDue to the restriction on uploading images, we are unable to provide the visualization. We will add the detailed analysis and visualized examples to the appendix in the final version to provide a more complete picture of the framework's current limitations and guide future improvements.\n\n\n***W3/Q2/Q4: System training and optimization***\n\nOur current plug-and-play design is intentional: it provides a flexible and accessible framework for researchers to immediately leverage SOTA performing or in-domain customized models as they choose. We also believe this setting is handy specifically for HCI researchers who plan to conduct experiments with human subjects. That being said, model optimization aligns perfectly with our vision, thus, we share a discussion on that here. \nFreezing the retriever and optimizing the image generator in an end-to-end manner may be the best choice. On the retriever side, we expect that the retriever may not necessarily be a trainable deep learning model to provide greater flexibility, such as allowing the use of Google Search in the general domain or the use of users' own well-designed rule-based search in some specific fields. On the generator side, the success of the image generator depends on whether the generated image can \"correctly reflect the text query\" and \"distinguish between retrieval targets and incorrect search candidates in the image space\". In other words, we want the collaboration between components to be seamless. \nIn this case, we highly agree with your suggestion that end-to-end reinforcement learning is necessary. However, **the data may become an obstacle, which highlights our data contribution.** As you astutely suggested, the dataset we released provides future research such opportunities for end-to-end reinforcement learning. Each of our data points contains the entire execution trace (text query -> generated image -> retrieved result). As shown in appendix A.1 and figure 6, **the quality of our data is significantly improved based on ChatIR**, which is the best resource in this direction at this moment. \n\n***Q1: Technical contribution and novelty***\n\nWe thank the reviewer for this question, as it allows us to clarify our core contribution. While the components are plug-and-play, the **novelty lies in the system-level paradigm itself**. Our key innovation is the **generate-visualize-retrieve closed loop**, which fundamentally reframes interactive search. Unlike prior work, GenIR uses the generated image not as the final output, but as an internal representation made external. This artifact serves two novel functions: (1) it provides explicit, interpretable visual feedback to the user, and (2) it acts as the query object for a more robust uni-modal (image-to-image) search. This framework, which was previously infeasible, is our main contribution.\n\n***Q3: Cost-Benefit Analysis***\n\nWe agree that latency is a key consideration. To provide a clear cost-benefit analysis, we performed an analysis to quantify the potential of a **hybrid system** that chooses between fast Verbal Feedback and our more powerful Visual Feedback.\n\nOur analysis shows that Visual Feedback **uniquely succeeds in 22.3% of cases** where Verbal Feedback fails. We present two hybrid scenarios: (1) an **\"Oracle\"** that perfectly selects when to use Visual Feedback, and (2) a **\"Random Select\"** baseline that randomly chooses Visual Feedback for 22.3% of queries.\n\nAs shown in the table 1, even random selection yields a **+3.40% absolute improvement** (74.48% to 77.88%) at Round 0, while the oracle achieves **+18.92%**. For our experiment on MSCOCO, the average latency increased by just **+3 seconds per query**. This demonstrates that even a simple hybrid approach provides **meaningful gains**, with **significant room for improvement** through intelligent selection mechanisms. Furthermore, in difficult cases where verbal feedback requires many rounds (or hit max rounds due to continuous unsuccessful attempts), GenIR's ability to **succeed in fewer rounds** can actually be more time-efficient overall. We will add this full analysis to the final paper.\n\n### Table 1: Comparison of using Verbal Feedback, Visual Feedback, and Hybrid Systems for Image Retrieval (Hits@10)\n*22.3% Visual / 77.7% Verbal*\n| Dialog Length | Verbal Feedback | Visual Feedback | Hybrid Oracle | Random Select | Verbal->Oracle | Verbal->Random |\n|--------------|-----------------|-----------------|---------------|---------------|----------------|----------------|\n| 0            | 74.48%         | 89.71%         | 93.40%        | 77.88%        | +18.92%        | +3.40%         |\n| 1            | 82.73%         | 93.11%         | 95.97%        | 85.06%        | +13.25%        | +2.33%         |\n| 2            | 85.83%         | 95.00%         | 97.43%        | 87.89%        | +11.60%        | +2.06%         |\n| 3            | 87.97%         | 95.97%         | 97.91%        | 89.76%        | +9.95%         | +1.79%         |\n| 4            | 89.13%         | 96.51%         | 98.20%        | 90.81%        | +9.07%         | +1.68%         |\n| 5            | 89.96%         | 96.85%         | 98.30%        | 91.50%        | +8.35%         | +1.54%         |\n| 6            | 90.49%         | 97.14%         | 98.59%        | 91.98%        | +8.10%         | +1.49%         |\n| 7            | 90.98%         | 97.48%         | 98.64%        | 92.42%        | +7.67%         | +1.44%         |\n| 8            | 91.27%         | 97.67%         | 98.79%        | 92.69%        | +7.52%         | +1.42%         |\n| 9            | 91.80%         | 97.72%         | 98.79%        | 93.13%        | +6.99%         | +1.33%         |\n| 10           | 92.33%         | 98.01%         | 98.88%        | 93.62%        | +6.55%         | +1.29%         |"}, {"review_id": "oC6euk8GTB", "reviewer": "Reviewer_KHQJ", "summary": "The paper introduces GenIR, a generative framework designed for open-world image retrieval that innovatively merges generative modeling with retrieval-based vision tasks. Unlike traditional image retrieval systems that depend on embedding-based nearest neighbor search within a fixed gallery, GenIR employs a generative visual tokenizer to synthesize image features relevant to a given natural language query. Retrieval is then accomplished by generating and matching token-level representations rather than using embedding similarity alone.\n\nThe proposed approach leverages a vision-language generative model trained to produce discrete token representations of images conditioned on textual inputs. During inference, GenIR generates visual tokens and searches through a tokenized image database, enabling robust retrieval of relevant images. The framework is evaluated on challenging zero-shot and open-world retrieval tasks, where it outperforms discriminative retrieval baselines by exhibiting better generalization to novel categories and compositional visual concepts. GenIR thus recasts image retrieval as a language modeling problem over visual tokens, offering a promising direction for retrieval systems in open-domain and long-tail settings.", "questions": "1. The proposed token-generation-based retrieval framework may incur higher computational costs compared to traditional embedding-based methods. Could the authors provide quantitative comparisons on inference speed, memory usage, and scalability across large-scale datasets? Clarifying this would help assess the model’s feasibility for real-world deployment. A strong demonstration of efficiency could positively impact the evaluation score.\n2. Real-world user queries often contain ambiguous language or incomplete information. How does GenIR perform in such noisy or multimodal input scenarios? Can it still generate accurate and relevant visual tokens? Addressing this through additional qualitative or quantitative analysis would enhance the paper’s practical relevance and could strengthen the case for acceptance.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper \"GenIR: Generative Visual Search for Open-World Image Retrieval\" presents an approach to image retrieval by reformulating it as a generative token prediction task, rather than relying on traditional embedding-based similarity. This paradigm shift is both technically sound and conceptually innovative. The authors propose a vision-language generative model that, conditioned on a textual query, synthesizes discrete visual tokens and performs retrieval by matching against tokenized candidate images. The quality of the work is high, with a well-implemented framework and thorough experimental evaluation across standard and open-world retrieval benchmarks. The results show impressive generalization to unseen categories, addressing a long-standing challenge in image retrieval. Furthermore, the paper is well-written and clearly structured, with effective visuals and intuitive explanations that make the contribution accessible.\n\nThe originality of the work is a key strength. While generative retrieval has been explored in the text domain, applying this paradigm to visual search through token-level modeling is a fresh and forward-looking contribution. The significance of this approach lies in its potential to transform how retrieval systems are built, particularly in open-domain and long-tail settings where conventional embeddings often fall short. However, there are some limitations. The paper lacks a detailed discussion on the efficiency and scalability of the inference-time token generation process, which may raise concerns for real-time deployment. Additionally, while the method's effectiveness is validated on clean benchmarks, its robustness to noisy or multimodal real-world inputs (e.g., user tags, behavior data) is not explored. The impact of the visual tokenizer design is also under-analyzed; more insight into how token granularity or tokenization scheme affects performance would improve the depth of the study.\n\nClarity could be improved in sections explaining the inference mechanism and the alignment between generated tokens and retrieved items. The paper also misses an opportunity to highlight how its method fundamentally differs from retrieval-augmented generation frameworks in NLP. Despite these minor weaknesses, the work represents a significant contribution to the field. Its originality, rigor, and relevance to open-world retrieval challenges make it well-suited for publication at NeurIPS. I recommend acceptance.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive assessment of our work's quality, originality, and significance. We appreciate the opportunity to clarify the practical questions raised regarding our framework's efficiency and robustness.\n\n***Q1: Computational Cost, Scalability, and Feasibility***\n\nWe thank the reviewer for this important question. To clarify the context for this discussion, our GenIR framework leverages standard text-to-image diffusion models to generate a complete, synthetic image from the user's query. This full image then serves as a visual proxy for a subsequent image-to-image retrieval step.\nWhile this process is more computationally intensive than simple embedding lookups, we believe the performance gains justify the cost in many scenarios. To provide a concrete cost-benefit analysis, we performed an analysis to quantify the potential of a **hybrid system** that chooses between fast Verbal Feedback and our more powerful Visual Feedback.\n\nOur analysis shows that Visual Feedback **uniquely succeeds in 22.3% of cases** where Verbal Feedback fails. We present two hybrid scenarios: (1) an **\"Oracle\"** that perfectly selects when to use Visual Feedback, and (2) a **\"Random Select\"** baseline that randomly chooses Visual Feedback for 22.3% of queries.\n\nAs shown in Table 1, even random selection yields a **+3.40% absolute improvement** (74.48% to 77.88%) at Round 0, while the oracle achieves **+18.92%**. For our experiment on MSCOCO, the average latency increased by just **+3 seconds per query**. This demonstrates that even a simple hybrid approach provides **meaningful gains**, with **significant room for improvement** through intelligent selection mechanisms. Furthermore, in difficult cases where verbal feedback requires many rounds (or hit max rounds due to continuous unsuccessful attempts), GenIR's ability to **succeed in fewer rounds** can actually be more time-efficient overall. We will add this full analysis to the final paper.\n\n### Table 1: Comparison of using Verbal Feedback, Visual Feedback, and Hybrid Systems for Image Retrieval (Hits@10)\n*22.3% Visual / 77.7% Verbal*\n| Dialog Length | Verbal Feedback | Visual Feedback | Hybrid Oracle | Random Select | Verbal->Oracle | Verbal->Random |\n|--------------|-----------------|-----------------|---------------|---------------|----------------|----------------|\n| 0            | 74.48%         | 89.71%         | 93.40%        | 77.88%        | +18.92%        | +3.40%         |\n| 1            | 82.73%         | 93.11%         | 95.97%        | 85.06%        | +13.25%        | +2.33%         |\n| 2            | 85.83%         | 95.00%         | 97.43%        | 87.89%        | +11.60%        | +2.06%         |\n| 3            | 87.97%         | 95.97%         | 97.91%        | 89.76%        | +9.95%         | +1.79%         |\n| 4            | 89.13%         | 96.51%         | 98.20%        | 90.81%        | +9.07%         | +1.68%         |\n| 5            | 89.96%         | 96.85%         | 98.30%        | 91.50%        | +8.35%         | +1.54%         |\n| 6            | 90.49%         | 97.14%         | 98.59%        | 91.98%        | +8.10%         | +1.49%         |\n| 7            | 90.98%         | 97.48%         | 98.64%        | 92.42%        | +7.67%         | +1.44%         |\n| 8            | 91.27%         | 97.67%         | 98.79%        | 92.69%        | +7.52%         | +1.42%         |\n| 9            | 91.80%         | 97.72%         | 98.79%        | 93.13%        | +6.99%         | +1.33%         |\n| 10           | 92.33%         | 98.01%         | 98.88%        | 93.62%        | +6.55%         | +1.29%         |\n\n\n***Q2: On Performance with Ambiguous or Noisy Queries***\n\nThis question highlights a core motivation for the GenIR paradigm. When a user query is ambiguous (e.g., \"a picture of a bat\"), traditional text-only systems provide no insight into how the system interpreted the ambiguity (animal or baseball equipment?). This forces the user into a trial-and-error refinement process.\n\nGenIR is designed to solve this very problem. By generating a synthetic image, the framework **visualizes the system's interpretation of the ambiguous query.** If the generated image is not what the user intended, it provides immediate, concrete, and actionable feedback. The user can then refine their query with specific details to resolve the ambiguity (e.g., \"a flying mammal bat at night\"). This ability to make the system's \"visual belief\" explicit is a key strength of our approach for handling noisy, real-world queries."}, {"review_id": "lq8ctcWPXn", "reviewer": "Reviewer_TmpY", "summary": "The paper presents GenIR, a generative multi-round retrieval approach in which the user can iteratively enhance the query based on the feedback of the generative system. The method is then experimented with several datasets.", "questions": "I think experiments with human subjects is needed.\n\nBetter control for effect oriigating from the initial query mapping is required\n\nComparison to broader set of approaches and improved comparison to related work", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 1, "strengths_and_weaknesses": "The paper tackes an interesting human-in-the-loop problem for communicating human preferences to visual models and using those in retrieval.\n\nHowever, the method is not new. Similar approaches have been presented earlier. A very similar one that comes to my mind is published already in SIGIR 2020: https://dl.acm.org/doi/abs/10.1145/3397271.3401129, in which the system was tested with human subjects and real human feedback. Similar approaches have also been proposed recently, such as https://dl.acm.org/doi/10.1145/3613904.3642847. There is also a wide body of work for learning the feedback from language, sketching, and even from implicit feedback and human brain (BCI feedback). Therefore, the novelty of the approach is limited, and the majority of previous work in this area is not covered in the paper.\n\nIn terms of the experiments, there is no human study; instead, a  VLM is used. This significantly weakens the results.\nThe most severe problem with the experiments is the fact that while the performance of the proposed approach is the highest among the few that have been tested, it is so already on the 0th iteration. This means that the whole advantage comes from improved mapping of the query to the image. This is not an iterative feedback task, but simply a task to map the query to a visual representation.\nBased on these shortcomings, both in terms of novelty and experimental rogor, I do not think the paper is ready to be published in NeurIPS.", "was_revised": true, "final_justification": "Thank you for the clarifications and better positioning of your work. I have also read the responses to the other reviewers, which I highly appreciate. I think the paper can benefit from clarification of the contribution and positioning. I would very much like to see this reflecting the final paper, if accepted. In summary, I think the authors have done excellent work in replying to reviewers' comments. Therefore, I will raise my score.", "rebuttal": "We thank the reviewer for their feedback and the opportunity to clarify our contributions and address the concerns raised, particularly regarding compassion with mentioned works and the interpretation of our experimental results.\n\n***W1: Comparison to mentioned works***\n\n***Comparison with “[1] Generating Images Instead of Retrieving Them: Relevance Feedback on Generative Adversarial Networks. SIGIR 2020”***\n\nWhile both [1] and GenIR leverage relevance feedback loops, we share fundamentally different goals. \n\nGenIR aims to retrieve real images from the image corpus, while [1] aims for generating images that match the user's intention. As mentioned in their introduction, “*we show that the technique is able to generate images that match user needs (**as opposed to retrieving them**)...*” [1]. Following up on this pioneer work, targeting generates the image that the user is imaging, the technology has later developed into using better algorithms and models, as we mentioned and cited recent works [2] in line 104. We plan to add [1] into our reference list, as we do value the spirit of using relevant feedback from users. \n\nIn contrast to this line of research, the goal of GenIR is to **retrieve an existing image** within a very large database. The generated image in our framework is not the final product but rather an intermediate artifact that serves two novel purposes: (1) as explicit, interpretable visual feedback to the user, and (2) as an intermediate query for a more robust, uni-modal (image-to-image) search. \n\n*[2] Wang, XuDong, et al. \"Visual lexicon: Rich image features in language space.”, CVPR 2025*\n\n\n***Comparison with “[3] GenQuery: Supporting Expressive Visual Search with Generative Models. CHI 2024”***\n\nGenQuery is an HCI work focused on supporting designers by using LLMs to **concretize abstract ideas or perform image editing**. Its core generative features involve using LLMs to concretize text queries when designers only have abstract ideas or using image editing to modify a region of a provided image. In contrast, GenIR proposes a different, holistic  **text-to-image retrieval** loop where a complete synthetic image is generated from a text query alone to visualize the system's interpretation for the purpose of finding an existing image.\n\n***On comparison with “Feedback from Human Brain (BCI feedback)”***\n\nThanks for pointing this out. We do not intend to confuse readers from the BCI and Neuroscience community. As noted in our paper's footnote [1], we do not engage with neuroscience or Brain-Computer Interfaces (BCI). The scope of this work, while the task name is inspired by the concept of a \"mental image reconstruction\", resides firmly in the fields of Generative AI and Information Retrieval. BCI-based feedback relies on decoding neural signals, whereas our framework uses natural language and image—a fundamentally **different problem domain, input modality, and technical challenge**.\n\n***Q3: Comparison with more related works.***\n\nWe thank the reviewer for providing the above mentioned works. It is a bit difficult to directly compare them, as they target different research problems. As we are working on a novel task setting as pointed out by Reviewer 6Fz4, the direct comparison we can find are only ChatIR and PlugIR. We welcome suggestions and would like to engage more discussion in additional related works that directly leverage visual feedback in retrieval tasks.\n\n***W2/Q1: VLM simulates user and future human study.***\n\nOur use of VLM simulation was a deliberate methodological choice for this foundational stage of research. To scientifically test our core hypothesis, i.e., the generative visual feedback is a richer and more effective mechanism than verbal feedback, we required a controlled, reproducible, and large-scale experimental setup. This allowed us to isolate the feedback variable and robustly validate our claims across four large-scale (*million-level image databases*) and diverse (*multi-domain*) datasets.\nThis approach establishes the necessary technical groundwork upon which meaningful and rigorous human studies can now be built. We see our work and open-source framework as the first research step towards Mental Image Retrieval. By providing a high-performing, open-source testbed (codebase and dataset). This enables the broader range of the AI community, such as HCI as reviewer mentioned, to now explore the very questions of fuzzy mental models and evolving user intent that were previously intractable to study at scale.\n\n***W3: Novelty and contribution***\n\nWe would like to take this opportunity to clarify our core novelty and contribution, which is multi-faceted, spanning a new task, a new framework, and a new dataset that collectively advance interactive retrieval.\n1. **Task: Defining Mental Image Retrieval (MIR).** We provide the first formal definition, evaluation protocol, and strong baseline for the ubiquitous yet under-explored \"re-finding\" problem, opening a new avenue for research.\n2. **Framework: A New Paradigm of Generative Visual Feedback.** The novelty is the **generate-visualize-retrieve closed loop.** This paradigm shift makes the system's internal understanding explicit, enabling a more intuitive and effective search process that was previously infeasible. By providing a flexible, open-source framework, we lower the barrier for the community to explore this new design space.\n3. **Dataset: A High-Quality, Visually Rich Resource.** As noted by Reviewer bt51, our dataset is a valuable contribution. We provide the first dataset capturing the full trace of a retrieval process (*text query -> generated image -> retrieved result*) in the MIR task. This rich, multi-modal data is an invaluable resource for future work, including the opportunities of the RL-based optimization other reviewers suggested.\n\n\n***W4/Q2: Reason of the strong performance at 0-th iteration***\n\nWe thank the reviewer for this sharp observation regarding our strong performance at the 0-th iteration. This is an important finding that highlights a core component of our contribution.\n\nOur contribution is twofold, and our results validate both aspects:\n1. **A Superior Core Retrieval Mechanism (validated at 0-th iteration)**: The significant performance gap at Round 0 demonstrates the inherent superiority of our proposed (*text -> generated image -> image-to-image*) search pipeline compared to the baseline's (*text -> text-to-image*) search. This shows that using a generative model as a visual proxy to bridge the modality gap is a more effective retrieval strategy in its own right.\n2. **An Effective Iterative Feedback Loop (validated at 1st to 10th iteration)**: The consistent performance improvement from Round 1 onwards demonstrates that the visual feedback provided by the generated image is actionable and allows users (simulated by the VLM) to effectively refine their queries for even better results.\nTherefore, GenIR is a complete interactive framework that starts with a superior mapping  the query to a visual representation and then consistently improves it via a novel visual feedback loop."}]

Author rebuttal:

Meta-review: The paper proposes GenIR, a generative framework for a new interactive search paradigm — multi-round mental image retrieval, where users iteratively refine queries based on visual feedback generated by diffusion models.

Reviewers thought this new retrieval paradigm was very interesting (and the AC concurs), with a compelling motivation and a clear system-level innovation. They thought that GenIR offers an intuitive and interpretable user experience through visual feedback, and noted that it outperforms verbal-feedback-based baselines across multiple datasets. Reviewers also appreciated the released dataset, believing it would be a valuable resource for future work.

Some weaknesses were noted. Chief among them was the reliance on simulated VLM-based users rather than human evaluations, raising questions about ecological validity. Other concerns included the computational cost of generating visual feedback and the use of off-the-shelf components. However, it seems that reviewers were largely satisfied by the authors responses to the concerns during the rebuttal and discussion.

One reviewer initially leaned toward a borderline reject, questioning the novelty and iterative nature of the framework due to strong performance at iteration 0. However, the authors clarified that this performance reflects a strong base retriever, and the additional gains across rounds demonstrate the utility of the feedback loop. The reviewer acknowledged these clarifications and ultimately updated their stance (although their actual rating does not appear to have been updated, the AC interprets their final justification as suggesting a borderline accept score). 

For the camera-ready version, the authors should include the expanded failure analysis, visualized examples, and hybrid system evaluation that were promised in the discussion and rebuttal.

The AC recommends acceptance.
Decision: Accept (poster)