ImageRAG: Dynamic Image Retrieval for Reference-Guided Image Generation
Authors: Rotem Shalev Arkushin
Rinon Gal
Amit H. Bermano
Ohad Fried
Venue: ICLR 2026 Poster (2026)
Keywords: RAG
image generation
rare-concept generation
Primary area: generative models
TL;DR: Rare concept image generation using dynamically retrieved image references.

Abstract: While recent generative models synthesize high-quality visual content, they still struggle with generating rare or fine-grained concepts.
To address this challenge, we explore the usage of Retrieval-Augmented Generation (RAG) for image generation, and introduce ImageRAG, a training-free method for rare concept generation. 
Using a Vision Language Model (VLM), ImageRAG identifies generation gaps between an input prompt and a generated image dynamically, retrieves relevant images, and uses them as context to guide the generation process. 
Prior approaches that use retrieved images require training models specifically for retrieval-based generation. In contrast, ImageRAG leverages existing image conditioning models, and does not require RAG-specific training.
We demonstrate our approach is highly adaptable through evaluation over different backbones, including models trained to receive image inputs and models augmented with a post-training image-prompt adapter. 
Through extensive quantitative, qualitative, and subjective evaluation, we show that incorporating retrieved references consistently improves the generation abilities of rare and fine-grained concepts across three datasets and three generative models.

Published as a conference paper at ICLR 2026
IMAGERAG: DYNAMIC IMAGE RETRIEVAL FOR EAFERENCE-GUIDER IMAGE GENERATION
Rotem Shalev-Arkushin
Rinon Gal
Amit H. Bermano
Ohad Fried
Tel-Aviv University NVIDIA
Reichman University
Model agnostic
Multi-image
Personalization
"Aa Aneevons fish
"A yolden retriever cradta."
"A plocto of a rnd ponda playing Bondergonton
"bukarrig
Sal-ooveen costuese.""
FLUX
SDXL
T21
T21
Outpot
Irittal Oulput
Invicl Oulpur
Invita Oulput
Retrissed Re!
Final culput
Retrisued
Final culpur
Ratriaued Rañi.
Final output
Retreved Ref.
Final curput
Using image references broadens the generation capabilities of image generation models. Given a text prompt, our method, ImageRAG, dynamically retrieves relevant images and provides them to text-to-image model references (Retrieved Ref.). ImageRAG works with different models, e.g. FLUX and SDXL (left), and OmniGen (right), and with different controls, e.g. text (left, middle), and personalization (right).
Figure 1:
ABSTRACT
While recent generative models synthesize high-quality visual content, they still struggle with generating rare or fine-grained concepts. To address this challenge, we explore the usage of Retrieval-Augmee
Generation (RAG) for image generation, and introduce ImageRAG, training-fiee free method for rare concept generation. Using a Vision I Language Model (VLM), ImageRAG identifies generation gaps between input prompt and generated image dynamically, retrieves relevant images, and uses them as context to guide the generation process. Prior approaches that use retrieved images require training models specifically for retrieval-based generation. In contrast, ImageRAG leverages existing image conditioning models, and does not require RAG-specific training. We demonstrate our approach is highly adaptable through evaluation over different backbones, including models trained to receive image inputs and models augmented with post-training image-prompt adapter. Through extensive quantitative, qualitative, and subjective evaluation, we show that incorporating retrieved references consistently improves the generation abilities and fine-grained concepts across three datasets and three generative models. Our project page is available at: https: pobo://rotemts shalev. github. 1o/ ImagoRAG
INTRODUCTION
Deep generative models (Ho al. 2020 Rombach al.) 2022; Dhariwal & Nichol 2021 Labs 2023) have revolutionized image generation, offering high-quality, diverse, and realistic visual content synthesis. They enable text-to-image generation well as wide range of tasks, from layout-based

---

Published as a conference paper at ICLR 2026
synthesis to image editing and style transfer (Avrahami al. 2022 Hertz et al. 2022 Mokady et al. 2023; Avrahami et al., 2202bb Zhang et al.] 2023] Brooks et al. 2023 Nitzan et al. 2024). These
large models require great amounts of training data, substantial training durations, and extensive computational resources. As a result, contemporary text-to-image (T21) models, that are limited to the data they
trained on, struggle with generating user-specific concepts or updated content. Specifically, they have difficulty with generating rare concepts, stylized content, or fine-grained categories (e.g., specific bird species, as in Fig. 2 left), even if they were trained on images containing them (Samuel et al., [024b) Haviv et al. 2024). In these cases, diffusion models tend to
"hallucinate", and potentially generate content unrelated to the textual prompt (see Fig. right). To
Prompt
Generation
Retrieval
Prompt
Base model
+Reference
ImageRAG
2
"A photo of rhinoceros aukler"
"Cradle" (SDXL)
"Chime" " (OmniGen)
"Electric fan"
Figure 2: Left: generation vs. retrieval. Although some generative models, such SDXL, use CLIP as a text encoder, they sometimes fail to generate concepts ("Generation' column) that are retrieved successfully using CLIP ('Retrieval' column). Right: Hallucinations. When models do not know the meaning of a prompt, they may "hallucinate" and generate unrelated images ('Base model' column). By applying method retrieve and utilize relevant references (*+Reference' column), the base models can generate appropriate images ('ImageRAG' column).
tackle these problems, several approaches have been proposed for personalized (Gal et al. 2023 Ruiz Voynov et al. 2023 Arar et al. 2024), stylized (Hu et al. [021] Li et al. [222a]] Zhang
et al. 2023 al.
2024). or rare concept (Samuel et al. 20246 Pan al.) 2025) generation. Most approaches,
however, require training or specialized optimization techniques for each new concept.
We observe that similar problems exist with text generation using Large Language Models (LLMs). LLMs struggle with generating text based on real-world facts, proprietary or updated data, and tend to hallucinate when lacking sufficient knowledge (Brown 2020 Ji et al.
2023). To solve these problems
with LLMs, Retrieval Augmented Generation (RAG) (Lewis et al.
2020) has been proposed. RAG dynamically retrieves the most relevant information from external data sources given a query, and supplies it to LLM as context input, enabling contextually accurate and task-specific responses. Investigating this idea for images, we note that previous works employing image retrieval for better image generation (Chen et al. 2022 Sheynin et al. 2022 Blattmann al. 2022, Hu al. 2024). train models specifically for the task, hindering wide applicability. A recent work (Lyu et al. 2025)
propose requires training of specific retrieval module per base generative model. In contrast, ImageRAG, a method that dynamically retrieves and provides images as references to pretrained models, enhance their generation capabilities, and does not require any additional training. Instead, we use existing generative models in the same vein as the common use of LLMs, and retrieve reference images during sampling for guided generation, by leveraging existing Vision-Language Models (VLMs). We note that in the visual domain, the context is more limited compared to the language case. Hence, to apply RAG for image generation, we cannot simply provide references for all the concepts in prompt. Therefore, we need to decide which images to use as context, how to retrieve them, and how to use the retrieved images, to successfully generate a required prompt.
In this work, we address these questions by offering novel method that dynamically identifies relevant and useful examples given a prompt, and uses them as references to guide the model toward generating the required result. Leveraging the abilities of T2I models to produce many concepts, we only pass concepts that the models struggle to generate, focusing on generation gaps. To understand what the challenging concepts are, we propose novel method that applies guided multimodal ohain-of-thought (CoT) process using a VLM. By encouraging step-by-step reasoning, CoT (Wei) et al. 2022a Yang et al. 2023 helps VLMS reach more reliable and consistent conclusions by
2

---

Published as a conference paper at ICLR 2026
decomposing complex tasks into explicit intermediate steps that reduce errors. Specifically, we first generate an initial image and then iteratively prompt a VLM to assess the aligmment between the image and prompt, identify missing visual components, and suggest complementary concepts. We then use these concepts to retrieve reference images that guide subsequent generation.
Our approach is not related specific T2I model, and can be applied different base models. To demonstrate it, we apply ImageRAG to three models: Omnigen (Xiao et al.) 2024), SDXL (Podell al. -PPaa-ePr (Ye al. 2023), and FLUX (Labs) (023)+ #0minnCoooooooo (Tan et al. 2024). We perform quantitative, qualitative, and human evaluations of our method with these models, and show that ImageRAG enhances their rare and fine-grained concept generation capabilities. These results indicate that the image generation community could benefit from adopting RAG for class or task-specific generation during sampling time.
2 RELATED WORK
In-context learning (ICL) has emerged as a powerful paradigm in which large language models (LLMs) are capable of performing new tasks without additional fine-tuning (Brown (020). By providing a few examples or relevant context directly in the input prompt, ICI. enables models to infer the desired task and generate appropriate outputs. Despite its flexibility, ICL. limited by the finite context window of the model, making the selection of relevant and concise context critical
for optimal performance. Recently, visual ICL presented promising results (Gu et al. 2024; Wang
ct al. 2023: Xiao et al. 2024 Najdenkoska et al. 2024 Sun et al. (024). Visual ICL has mostly
been explored in the context of learning from analogies (Gu et al. 2024, Wang et al. 2023 Xiao
et al.) 2024; Nguyen al. (0244). However, the ability of learning from single examples has also
been researched with multimodal generative models that allow images as input (Xiao et al.) 2024
Sun al.
2024 Wang et al., 2024b). Such models allow image prompting and facilitate exploring RAG for image generation.
Retrieval Augmented Generation (RAG) Lewis et al. (2220) is method for improving the generation abilities of a pretrained model without additional training, by dynamically retrieving and supplying information as context through text prompts. For each given query, relevant information retrieved from an external database, and supplied to the model for improved generation that relies on it as context. While RAG has been greatly explored for text generation tasks and applied over multiple
pretrained LLMs (Lewis et al. 2020 Gao et al. 2023: Ram et al. 2023 Borgeaud et al.) 2022, Li
et al. 2024b Zhang et al. 2025). has yet to explored for enhancing pretrained image generation models' capabilities. Some previous work used nearest-neighhor image retrieval to improve image
generation (Sheynin et al.] 2022; Blattmann et al.] 2022 Chen et al.) 2022] Lyu ct al. 2025) or for editing-guidance (Hu et al. 2024 Sanguigni
al.) 2025), however they either train models specifically for retrieval-aided generation or require a retrieval-module training per-model. Unlike them, our method leverages pretrained models and does not require additional training. A recent work, Yuan et al. (2025). ), utilizes an LLM for prompt decomposition to concepts and retrieves references representing each of them. Then, they generate layout with all the concepts to assert all of them are generated. Unlike our method, they retrieve all the concepts in prompt, ignoring prior knowledge of the model. Moreover, while using a layout ensures all concepts are present the result, this strategy may constrain the generation abilities to isolated concepts without interaction between them. For example, their method may struggle to generate concept in a style or interacting concepts.
Text-to-image generation advanced greatly with the introduction of diffusion models (Ho et al.) 2020), which produce high-quality and diverse images of wide range of concepts (Dhariwal & Nichol [202T] Rombach et al. 2022 Podell et al. 2024 Xiao et al. 2024). However, they struggle with rare concepts and cannot generate user-specific concepts without additional training or optimization.
Personalization works generate images of user-specific concept. However, they often require an optimization process for learning each new concept (Nitzan et al.,[2022 Gal ct al. 2023 Ruiz et al.
2023
Arar et al. 2024 Alaluf et al. 2023 VVynov et al. [203]] Avrahami al. 2023a Kumari et al.
2023
To mitigate this challenge, recent works train image-encoders that allow prompting existing
pretrained generative models with images (Ye et al. 2023 Wang et al. 2004a).
Rare Concept Generation focuses on image generation of uncommon concepts. Samuel et al.
2024a b) explored generating rare concepts using few examples of each concept to optimize
3

---

Published as a conference paper at ICLR 2026
seeds that produce images similar to the references. However, in addition to the requirement of an optimization process per new concept, these works do not address the questions of how to find and choose the reference images. Pan et al. (2025) suggests parameter--effcient fine-tuning approach over full datasets.
3 METHOD
"Pecording to chese comples of SeLd: s11,, S227 <22?, georaat <p>.
<p2: photo of confoood prissly boor in cotoutos ohoss."
retrieval
T21
Guided CoT
T21
11
Guided CoT
Concepts
Caption
$: image caption describlng nissing concepts.
"A grizzly bear wits "A classroon with calculus "quatices
Eowe this Siage watch
Nrtch COnCOptS are
confased
calcates 21004
Figure 3: Top: high-level overview of method. Given text prompt <p>, generate initial image using a text-to-image (T2I) model. Then, using guided CoT process, we generate retrieval-captiion <cj>,, retrieve images from external database for each caption <ij>, and use them as references to the model for better generation. Bottom: the guided Chain-of- -hoouh (CoT) process. We use a VLM decide if the initial image matches the given prompt. If not, ask it to list the missing concepts, and to create caption that could be used to retrieve appropriate examples for each of the missing concepts.
Our goal is increase the robustness of T2I models, particularly with concepts and fine- fineggrained categories, which they struggle to generate. To do so, we investigate retrieval- setieval-augmented generation (RAG) approach, through which dynamically select images that can provide the model with missing visual cues. Importantly, we focus models that were not trained for RAG, and show that existing image conditioning tools can be leveraged support RAG post-hoc. As depicted in Fig. 3 given text prompt and a T2I generative model, we start by generating an image with the given prompt. Then, we employ a guided chain-of-thought (CoT) reasoning process, through which VLM decides whether the image aligns with the prompt, and identifies gaps in it if not. If gaps were identified, we aim to retrieve images representing missing concepts that would help fill these gaps. These images are then used as context to guide the T2I model toward better alignment with the prompt. In the following section, we provide a detailed description our method.
3.1 GUIDED CHAIN-OF-THOUGHT (CoT)
To identify missing concepts in an image and retrieve relevant references, we employ a guided CoT process with VLM (depicted in Fig. bottom). Since currently the amount of images we can pass to an image generation model is limited, we cannot pass images representing each of the concepts in prompt. However, T2I models are capable of generating many concepts successfully, strategy would be passing only concepts they struggle to generate. To identify challenging concepts, we generate an initial image with T2I model. Then, we query a VLM with the initial prompt and image, asking it to decide if they match. If not, we guide the VLM to generate image captions for retrieval, by asking
efficient
identify missing concepts the image, focusing on content and style, since these are easy to convey through visual cues. We use example so it knows to only return short generic concepts as an answer. These concepts are the ones we should retrieve and use as references.
However, as demonstrated in Tab. 3 image retrieval from brief, generic concept descriptions yields worse results than retrieval from detailed image captions. Therefore, after identifying the missing concepts, query the VLM to generate detailed image captions for images describing each of the identified missing concepts, which will be used to retrieve exemplar images. By constraining the CoT answers to concise answers, avoid overthinking and achieve accurate retrievals.
4

---

Published as a conference paper at ICLR 2026
Beyond performance gains and fine-grained concepts, the guided CoT adds interpretability. as we can inspect which concepts were identified as missing. The approach does rely on the diagnostic accuracy of the VLM; therefore, we performed robustness experiments over various VLMs and explored which ones could be trusted for this task (see Appendix E.3 for more details). All the used prompts can be found in Appendix C
3.2 RETRIEVE AND USE REFERENCE IMAGES
We aim to retrieve images that could be described by the generated image captions from external dataset. To retrieve images matching a given caption, we compare the caption to all the images in the retrieval-dataset using a text-image similarity metric and retrieve the most similar images. Text-to-image retrieval is an active research field (Radford et al. 2021 Zhai et al. 2023 Ray et al. 2024) Vendrow al., 2024), where no single method is perfect. Retrieval is especially hard when the dataset does not contain an exact match to the query (Biswas & Ramnath (2024) or when the task is fine-grained retrieval, which depends on subtle details (Wei et al. 2022b). Therefore, we experimented with multiple retrieval strategies (see Tab. 4). First, we tried cosine similarity between CLIP (Radford et al., 202I) and SigLIP (Zhai et al., 2023) text and image embeddings. Next, following common retrieval workflow, in which first image candidates are being retrieved using pre-computed embeddings, and then the retrieved candidates are being re-ranked using a different, often more expensive but accurate, method (Vendrow et al., 2024), we additionally experimented with some re-ranking methods of reference candidates. Although re-ranking sometimes yields better results compared to using cosine similarity between CLIP embeddings (see Tab. 4. the difference was not significant in most of our experiments. Therefore, for simplicity, in our experiments we used cosine similarity between CLIP embeddings as our similarity metric. The retrieval of good candidates using CLIP raises question. Some of the base models, e.g. SDXL (Podell et al. 2024). use CLIP as text-encoder, so how can they benefit from references retrieved by the same model? Our empirical experiments, exemplified in Fig. 2] show that while SDXL struggles to generate some concepts, they are retrieved successfully using CLIP. We hypothesize that retrieval an easier task than generation. Hence, even though the model cannot generate some concepts, it can still benefit and learn from images representing them, which can be retrieved using CLIP.
After relevant images are retrieved, we can use image conditioning methods to condition T21 models using our reference images by augmenting the input prompt to contain the retrieved im- ages as examples. Formally, given prompt p. concepts, and compatible image for each concept, we augment the prompt with the following template: "According to these examples of <a>:<imgı>,... <e>>:<ingn>> generate <p>", where e; is compatible image caption of the image <img,>, for i E [1, ra]. This prompt allows models to learn missing concepts from the images, guiding them to generate the required result. We experimented with three models and conditioning methods (see Sec. 4. demonstrating our method is model- and conditioning method-agnostic.
The design of our method allows using a threshold as assurance of reference quality. This way, when retrieving reference, a threshold could be applied, so only good references, i... ones that pass the threshold, would be used. Additionally, we can optionally iterate the generation loop until the VLM indicates alignment between the text prompt and the generated image.
Validation options:
4 EXPERIMENTS
To evaluate the effectiveness of our method, we performed automatic and human evaluations. To assess its adaptability to different models, we experimented with applying ImageRAG to Omni- Gen (Xiao et al.) 2024), SDXL (Podell et al.] 2024) through IP-Adapter (Ye et al.) 2023), and FLUX (Labs) 2023) through OminiControl (Tan et al. 2024). In most experiments, we used GPT as our VLM and CLIP as our retrieval method, as they performed best overall, and 350K images subset of LAION (Schuhmann et al. 2022) as our retrieval-datasee for generic usage. We additionally tested the abilities of different VLMs to identify rare concepts (see Appendix (E.5) and experimented with different retrieval strategies (see Tab, 4. and with specialized datasets instead of a generic one (see Appendix E.3). Appendix contains additional implementation details.
5

---

Published as a conference paper at ICLR 2026
Table 1: GPTScore comparisons of fine-grainet image generation with text-to-image models. First- part columns feature OmniGen- -asee models, middle part columns feature FLUX-based models, and last-part columns feature SDXL-based models. In each part, best results are bolded.
Dataset
OmniGen
GraPE-O
ImageRAG-O
FLUUX
ImageRAG-F
SDXI.
ImageRAG-SDD
ImageNet 0.75
0.68
0.88
0.84
0.9
0.86
0.92
iNaturalist 0.07
0.06
0,56
0.07
0.31
0.51
0.70
CUB
0.57
0.45
0.73
0.79
0.85
0.94
0.97
Table 2:
Comparisons on fine-grain fine-grained image generation with T2I models. For each set, we report mean (# standard error) CLIP, SigLIP text-to-image similarities, and DINO feature similarity between real and generated images, Second-part rows feature OmniGen-basse models, third-part feature FLUX- models, and bottom feature SDXL -bsed models. In each part, best results are bolded.
ImageNet
iNaturalist
CUB
CLIP f
1
DINO
CLIPT
SigLIP 1
DINOT
CLIP 1
SigLIP 7
DINO
Pixart-E
D:20g400000
0.12
0.601 #0.000
0.16g
0,22750.022
0.611
0,232+20044
0,101=0005
0.738=0000
OmniKion
0:247
0.12240..01
1.602-6.00
0.014 0.165446.02
0.595-6-02
0.231 18005
0.100
0.747 10005
GraPE-O
0:251 0.25129.000 20.000
0.12400.001
0.09240.00u
0,15730.002
0.016
0.601
0.210=8000
0.115=9.0s
0.7475
ImageRAG--)
0.264-000m
0.134
10.000
0.708 10.00m
0.197 L0.00
0.095 10:003
0.701 16.000
0.253 120000
0.125
10000
0.760 100000
FLUX
0.271 0.271 110.00
0. 1.1..00.00
0.898 0.698-A0.0om
10.000
0,065 0,222
0.654
0,267
0.135
0.746
ImageRAG-F
0.27750000
0.140-5..00
0000
0.70g=00000 0002
0.238-00.00
0,083+0.000 002
0.691 40
0,277
0,144
002
0,767
2:0 00s
SDXL
0 0.207:90000
0. 0.135299.00
0. 0,700
0.259
0, 0,096
0.696
0
0.172=
0. 0.782-0000
ImageRAG-SS
0.274-0000
10.000
0.141 100.000
0.709 L0.000
0.243
0.118
111810.000
0.724 1.0. 1.0000
0.314
0.174 L.0.
0.784 0.00001
4.1 QUANTITATIVE COMPARISONS
We evaluate the ability of ImageRAG to improve T2I generation of rare and fine-grained concepts by comparing the results of different base models with their results when applying ImageRAG to them. As additional baselines, we compare with Pixart-E (Chen et al.] 2025). 5) and the OmniGen-batt version of GraPE (Goswami al. 2024). The last is an iterative LLM--ased image generation method which employs editing tools to insert missing objects. Following rare concept generation works (Samuel et al. [224bb Pan al. 2025). we use the ane-ggrined datasets ImageNet (Deng it al. 2009), iNaturalist ( (Van Horn et al., 2018). and CUB (Wah et al.] (011]) for evaluation. [Samuel al. 2024b) reported that 25% of Ietge classes are in the tail of L.AION (Schuhmann et al.
2022). and most of the classes in CUB and iNaturalist are in its tail. For i iNaturalist, we use the
first 1000 classes. Additional experimental results over the Flowers (Nilsback & Zisserman, (0008), Dogs (Khosla et al., 2011). and Cars (Krause al.) 2013) datasets are reported in Appendix E.I Following prior work (Pang et al., 2024; Ruiz et al. 2023; Zhang et al. 2023). we evaluate all methods with the commonly used similarity metries CLIP (Radford et al. (021)) and DINO (Zhang ct al. 2022). in Tab.2 For fairness, we use open-CLIP for evaluation, and OpenAl CLIP for retrieval. We additionally report SigL.P (Zhai et al. 2023) embedding similarity, which outperforms CLIP on several classification and retrieval tasks. When examining these scores alone, improvement seems mild. However, these metrics gauge only coarse semantic similarity: for instance, two bird species appear similar in the embedding space despite possessing meaningful distinctions required for concept-specific generation. Therefore, we additionally report GPTScores (Peng et al. (025). which has been shown to correlate with human judgment for personalized image generation evaluation, where concepts are inherently unknown to the model. Moreover, we conduct user studies (see Sec. 1. and supply visual examples (Fig. and Appendix [B). where improvements are much clearer, as they capture the intended evaluation signal more faithfully. As Tabs. demonsttaaee all base models results improve when using ImageRAG for rare concept and fine- fine-grained generation.
4.1.1 USER STUDIES
To evaluate our results further, conducted user study with 67 participants, including two types of comparative studies (977 comparisons) and an absolute study (231 answers).
Comparative studies: 1) comparing OmniGen, SDXL, and FLUX with and without our method, and 2) comparing our results to retrieval-based models trained for the task of image generation using retrieved images: RDM (Blattmann et al. 2022). knn-diffusion (Sheynin et al. 2022). and

---

Published as a conference paper at ICLR 2026
SDXL
OmniGen
PLUX
Not
Th4t
Too
Qual by:
Quality
Quality
60-
Overal
Ownakl
Overall
Cocooe pop.eaeee0 pwrustt--pprrtaaa paroetage
0%
M/% Relmagen
0%
200%
50%
100%
600
Whh
KNik-DDifaation
RDM
Tor
m
Teat
Text
-0-
Quel ty
Quality
Qualtyy
Overall
Overt
71%
Overall
0%
50%
100%
0%
55%
300%
0%
50h
100%
0-
Ornikes Imageei6-
0%
Imagei/G
200 IvageAnG-dD
Figure 4: User study results. Left: Users preference percentage of our method compared to other methods in terms of text alignment, visual quality, and overall preference, Right: Percentage of rare concept generation per model by users' rating of whether concepts are contained in generated images.
Table 3: Ablation studies over OmniGen. "Rephrased prompt": only prompt rephrasing without image references. "Retrieve concepts": using the missing concepts for retrieval instead of using detailed image captions, "Retrieve prompt": using the prompt for retrieval. Best results are bolded.
ImageNet
CUB
CLIP f
SigLIP f
DINO 1
CLIP 1
SigLIP 1
DINO
OmniGen
0.247 + 0.002
0.122 + 0.001
0.692 + 0.003
0.231 + 0.005
0.109 0.000
0.003
0.747 = 0.005
Rephrased prompt-O
0.248 + 0.002
0.124 0.042
0.696
+ 0.003
0.230 0.005
0.107 0.004
0.750 : 0.005
Retrieve concepts-O
0.258 + 0.002
0.130
0.001
0.694 1: 0.003
0.210
0.113
0.003
0.719 : 0.006
Retrieve prompt-O
0.258 + 0.002
0.130
+ 0.001
0.691
+ 0.003
0.246 - 0.004
0.120 = 0.003
0.736 + 0.005
ImageRAG-OO
0.264 + 0.001
0.134
0.001
0.708 1 0.002
0.253 4 0.003
0.002 0.125
0.760 + 0.003
Relmagen (Chen et al., 2022). Since these are largely proprietary models with no API, we compared images and prompts published in their papers. In each comparison, participants chose between an ImageRAG result and baseline image based text alignment, visual quality, and overall preference. Since some prompts contain uncommon concepts, we supply a real image of the least familiar concept in each prompt (not taken from dataset). As demonstrated in Fig. (left), participants favored ImageRAG over all other methods in all three criteria of text alignment, visual quality, and overall preference.
bssoute study: to assert our method not only improves the current results but also generates the rare concepts, we performed absolute study. We asked participants which of the results of all baseline models with and without our method contain a reference object. Note: we only used images where our method ran, meaning where the VLM decided the initial images did not match the prompt. As shown in Fig. 4 indeed our results contain the reference object in most cases; 92% (OmniGen). 90% (SDXL), and 84% (FLUX), while the baseline models originally did not contain in most cases (indicating that the VLM was able to identify the missing concepts accurately). Appendix [F) supplies more information about the studies, including questions and visual examples comparisons presented in it for each retrieval-based generation model (Fig. 6). OmniGen (Fig. 7). SDXL. (Fig. 8). and FLUX (Fig. with and without ImageRAG.
4.2 QUALITATIVE EXAMPLES
Fig. 5 shows rare concept generation examples by OmniGen, FLUX, and SDXL, with and without ImageRAG. In all examples, the base models did not generate the required concepts and guessed an animal based on the prompt. Moreover, even if they were able to deduce the required animal from the context, e.g., bird in the left-most prompt, they did not generate the exact bird species requested. When supplied with relevant references using ImageRAG, all methods succeeded in the generation tasks. Fig. presents additional visual results with more complex and creative prompts, and Fig. [4] shows personalized generation examples with rare concepts. In Appendix E.6 we discuss diversity and present diverse generation results using each model with different seeds.
7

---

Published as a conference paper at ICLR 2026
Zalophus californianus on rocks by the sea.
An Anas platyrhynchos in river.
A photo boston bull in field.
A Cyanocitta
A Geococcyx in the desert.
Prompt
cristata
Retrieved Reference
OmniGen
ImageRAG (OmniGen)
SDXI.
ImageRAG (SDXL IPA)
FLUX
ImageRAG (FLUX OminiControl)
Figure 5: Qualitative comparisons: rare concept generation. Examples from ImageNet, CUB, and iNaturalist. The top image column is the retrieved reference using ImageRAG for each prompt. OmniGen, SDXL, and FLUX all struggle to generate the uncommon concepts, however when using ImageRAG, all models successfully generate the required prompts.
8

---

Published as a conference paper at ICLR 2026
Table 4: Similarity metric ablation study over OmniGen. Results of our method using different similarity metrics for image retrieval. Best results are bolded.
ImageNet
CUB
CLIP
SigLIP 1
DINO
CLIP
SigI.IP
DINO f
GPT Re-rank
0.265 0.001
0.135 = 0.001
0.707 = 0.002
0.255 + 0.004
0.125 0.003
0.762 0.004
BM25 Re-rank
0.264
0.001
0.134 1 0.001
0.707 1 0.002
0.253 0.003
0.123 0.003
0.763 0.004
SigL.IP
0.259
0.006
0.133
0.001
0.704 + 0.002
0.243
0.0003
0.761 0.004
CLIP
0.264 0.001
0.134
0.001
0.708 + 0.002
0.253 + 0.003
0.125 0.002
0.760 0.003
4.3 ABLATION STUDIES
To evaluate the contribution of each part of our method, we conducted ablation studies and reported our results in Tabs. and First, to ensure the performance gap is not based on an LLM interpreting rare words, we evaluated the base models over rephrased text prompts, without providing reference images. To do so, we asked GPT to rephrase the prompts, to make them easier for a T2I generative model, explicitly asking it to change found in Appendix[[] As shown, rephrasing was not enough for a meaningful improvement in the results ("Rephrased prompt" in Tabs. 3 and (). Next, investigate the importance of using detailed image captions for retrieval, rather than using the original prompt or the missing concepts. We do so by evaluating ImageRAG when retrieving images similar to the prompt directly ("Retrieve prompt" in the tables), as done by previous works (Blattmann et al. [022]] Shhynnin et al. [022]] Chen et al. 2022), or images similar missing concepts, without generating compatible image captions for each concept ("Retrieve concepts" in Tabs. a While retrieval with each of them improves the initial results, retrieving detailed captions improves the results even further. Fig. T5 presents examples.
words to their description if necessary. Full prompt can be
We additionally investigate the effect of different similarity metrics for retrieval (Tab. 4. We tried CLIP (Radford et al. 202Tj, SigL.IP (Zhai et al. 2023), re-ranking of retrieved candidates by BM25 (Robertson et al.] 2009) over image captions generated by GPT, and by GPT (Hurst et al., 2024). Re-ranking was performed over the top candidates retrieved from each of CLIP and SigLIP Although re-ranking with GPT produced slightly better results, the improvement was not significant enough to justify the cost of applying this complex strategy vs. a more straightforward CLIP metric. Hence, our other experiments use CLIP. Nevertheless, all retrieval strategies improved the generation abilities of the base model by providing helpful references. Next, as we rely on a VLM to identify rare concepts in images, we performed a VLM robustness experiment explained in Appendix [E.5 with various VLMs. GPT performed best and can successfully identify rare concepts, hence we can rely on it for our method. However, Gemini and Qwen also performed well, and could be potentially used instead of GPT. We further investigate the effect of the retrieval-dataset size (discussed in Appendix E.4) Typically, the larger the dataset, the better the results. However, even a relatively small dataset can improve results. Moreover, we experiment with using specific proprietary retrieval dataset and show that the more relevant the dataset is, the better the results are (Appendix E.3)
LIMITATIONS
The capabilities of ImageRAG depend on the retrieval data and method, and on the base model. Retrieval data: our ability aid generation depends on the retrieval dataset. E.g., our method will not help when generating specific dog breed from a dataset of birds, as in Fig. 12 left. Specifically, we noticed that if the retrieved image is completely unrelated to the text (as in a bird and dog). the models tend to rely almost exclusively on the text. This is likely because semantically unrelated concepts do not attend to each other. As presented in Tab. G the more relevant the retrieval dataset is, the more accurate the generation will be.
Retrieval method: performance is tied to the quality of the retriever. CLIP-based retrieval inherits its weaknesses, such poor counting (Paiss al. 2023). Additionally, we rely on the used VLM to decide whether we should apply our method and identify gaps. Although VLMs are powerful and usually answer correctly, even for rare concepts, evaluated in Appendix(E.5) sometimes they misidentify an initial image as a match to the original prompt. leading to not applying our method. This happens especially if the prompt could be interpreted in multiple ways. For example, for the

---

Published as a conference paper at ICLR 2026
prompt "A photo of love in the mist", OmniGen generated an initial image of couple in the mist. This does match the prompt, hence the VLM identified it as correct; however, our intention was the flower "love in the mist". In that case, indicate the mismatch in the query so the VLM knows it should refer to the other meaning.
either add "the flower" to the prompt explicitly
concepts remain difficult for the generator itself; e.g., both OmniGen and SDXL struggle to reproduce text even when provided with text references. Fig. 12 shows visual limitations examples.
Underlying model:
6 CONCLUSION
In this work, we propose simple yet effective approach for applying RAG to pretrained T2I models. We demonstrate that incorporating relevant image references improves the rare concept generation abilities of T2I models. By leveraging a VLM, we enable dynamic retrieval of suitable reference images. Our experiments span three distinct models, illustrating the adaptability of our method across different model types. Our findings highlight that image references can enhance image generation with minimal modifications to existing models, thereby broadening their practical applicability.
7
ETHICS STATEMENT
The development of ImageRAG introduces enhancement possibilities for image generation models, enabling rare or fine-grained concept generation. While these advancements hold promise for creative industries, personalized content creation, and visualization, they also raise ethical concerns, including potential misuse for harmful content such as deepfakes and the use of private data. Therefore, transparency in data usage and adherence to privacy regulations are essential. We condemn any misuse of the proposed technology, and actively research tools to identify and prevent malicious usage of generative models. Moreover, unlike pre-trained models, which encode all knowledge in their weights, our method uses an external dataset. This allows the users to filter out unwanted images. This way, a possible solution to avoid copyright issues to only use images with appropriate licenses in the retrieval dataset, while following relevant restrictions if they apply. In the retrieval scenario, it also easier to directly attribute the source of the image. This should allow users to, c.g., credit the original creators, ensure images follow any license restrictions required by the publishers of the original image.
REPRODUCIBILITY STATEMENT
We provide detailed implementation details in Sec. 4and Appendix A including retrieval-dataset details, models used, hyperparameters, etc. The used prompts are also provided in Appendix C
To further facilitate reproducibility, we will release full code for applying ImageRAG over FLUX, SDXL, and OmniGen upon publication.
ACKNOWLEDGMENTS
We thank Gal Chechik and Daniel Arkushin for helpful discussions. This research was partially funded by ISF grant numbers 1337/22, 1574/21.
REFERENCES
Yuval Alaluf, Elad Richardson, Gal Metzer, and Daniel Cohen-Or. A neural space-time representation for text-to-image personalization. ACM Transactions on Graphics (TOG), 42(6):1-10, 2023.
Moab Arar, Andrey Voynov, Amir Hertz, Omri Avrahami, Shlomi Fruchter, Yael Pritch, Daniel Cohen-Or, and Ariel Shamir. Palp: Prompt aligned personalization text-to-image models. arXiv preprint arXiv:2401 400.v:2401.06105, 2024.
10

---

Published as a conference paper at ICLR 2026
Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18208-18218, 2022.
Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen-Or, and Dani Lischinski. Break-a-scene: Extracting multiple concepts from single image. In SIGGRAPH Asia 2023 Conference Papers, pp. 1-12, 2023a.
Omri Avrahami, Thomas Hayes, Oran Gafni, Sonal Gupta, Yaniv Taigman, Devi Parikh, Dani Lischinski, Ohad Fried, and Xi Yin. Spatext: Spatio-textual representation for controllable image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18370- 18380, 2023b.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar arXiv: 139.2333,.11 2025.
Biplob Biswas and Rajiv Ramnath. Efficient and interpretable information retrieval for product question answering with heterogencous data. In Proceedings of the Seventh Workshop Commerce and NLP@ LREC-COLING 2024, pp. 19-28, 2024.
Andreas Blattmann, Robin Rombach, Kaan Oktay, Jonas Müller, and Bjöm Ommer. Retrieval- augmented diffusion models. Advances in Neural Information Processing Systems, 35:15309- 15324, 2022.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Bapiln nann-Batiiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. In International conference machine learning, pp. 2206-2240. PMLR, 2022.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 18392-18402, 2023.
Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005. 14165, 2020.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo. Huchuan Lu, and Zhenguo Li. Pixart-sigma: for 4k text-to-image generation. In European Conference on Computer Vision, pp. 74-91. Springer, 2025.
training of diffusion transformer
Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W Cohen. Re-imagen: Retrieval-augmenti text-to-image generator. In The Eleventh International Conference on Learning Representations, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. leee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794 2021.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In The Eleventh International Conference on Learning Representations, 2023.
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Retrieval aerievaa--agmeented generation for large language models: A survey. arXiv preprint arXiv:2312. 10997, 2023.
11

---

Published as a conference paper at ICLR 2026
Ashish Goswami, Satyam Kumar Modi, Santhosh Rishi Deshineni, Harman Singh, Parag Singla, et al. Grape: A generate- pla---aaann--eeee framework for compositional t2i synthesis. arXiv preprint arXhv:2412.06089, 2024.
Zheng Gu, Shiyuan Yang, Jing Liao, Jing Huo, and Yang Gao. Analogist: Out-of-the-boo visual in-context learning with image diffusion model. ACM Transactions on Graphics (TOG), 43(4): 1-15, 2024.
Adi Haviv, Shahar Sarfaty, Uri Hacohen, Niva Elkin-Koren, Roi Livni, and Amit Bermano. Not every image is worth a thousand words: Quantifying originality in stable diffusion. arXiv preprint arXiv:2408. 08184, 2024.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kñrr Aberman, Yael Pritch, and Daniel Cohen-or. Prompt- to-prompt image editing with cross-attention control. In The Eleventh International Conference on Learning Representations, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021.
Hexiang Hu, Kelvin CK Chan, Yu-Chuan Su, Wenhu Chen, Yandong Li, Kihyuk Sohn, Yang Zhao, Xue Ben, Boqing Gong, William Cohen, et al. Instruct-imagen: Image generation with multi- modal instruction. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 4754-4763, 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-4o system card. arXiv preprint arXiv:2410.21276. 2024.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
Aditya Khosla, Nityananda Jayadevaprakash. Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine-grained image categorization: Stanford dogs. In Proc. CVPR workshop on fine- one-grained visual categorization (FGVC), volume 2, 2011.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pp. 554 561, 2013.
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun- Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 1931-1941, 2023.
Black Forest Labs. Flux. http://///....aaoooo////oa/eaaaa
2023.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmente genera- tion for knowledge-intensive nlp tasks. Advances Neural Information Processing Systems, 33: 9459-9474, 2020.
Likun Li, Haoqi Zeng, Changpeng Yang, Haozhe Jia, and Di Xu. Block-wise lora: Revisiting fine-grained lora for effective personalization and stylization in text-to-image generation. arXiv preprint 20t.::200000000000 2024a.
Yuankai Li, Jia-Chen Gu, Di Wu, Kai-Wei Chang, and Nanyun Peng. Brief: Bridging retrieval and inference for multi-hop reasoning via compression. arXiv preprint arXiv:2410. /5277, 2024b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
12

---

Published as a conference paper at ICLR 2026
Yuanhuiyi Lyu, Xu Zheng, Lutao Jiang, Yibo Yan, Xin Zou, Huiyu Zhou, Linfeng Zhang, and Xuming Hu. Realrag: Retrieval-augmmened realistic image generation via self-reflective contrastive learning. In Forty- International Conference on Machine Learning, 2025.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. 2013.
Ron Mokady, Amir Hertz, Kfir Aberman, Yacl Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattemn Recognition, pp.
2023.
Ivona Najdenkoska, Animesh Sinha, Abhimanyu Dubey, Dhruv Mahajan, Vignesh Ramanathan, and Filip Radenovic. Context diffusion: In-context aware image generation. In European Conference on Computer Vision, pp. 375-391. Springer, 2024.
Thao Nguyen, Yuheng Li, Utkarsh Ojha, and Yong Jae Lee. Visual instruction inversion: Image editing via image prompting. Advances in Neural Information Processing Systems, 36, 2024.
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp. IEEE, 2008.
Yotam Nitzan, Kfir Aberman, Qiurui Hc, Orly Liba, Michal Yarom, Yossi Gandelsman, Inbar Mosseri, Yael Pritch, and Daniel Cohen-Or. Mystyle: A personalized generative prior. ACM Transactions on Graphics (TOG), -1) 2022.
Yotam Nitzan, Zongze Wu, Richard Zhang, Eli Shechtman, Daniel Cohen-Or. Taesung Park, and Michaël Gharbi Lazy diffusion transformer for interactive image editing. arXiv preprint arXiv:2404. /2382, 2024.
Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, and Tali Dekel. Teaching clip to count to ten. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 31 3170-3180, 2023.
Ziying Pan, Kun Wang, Gang Li, Feihong He, and Yongxuan Lai. Finediffusion: scaling up diffusion models for fine-grained image generation with 10,000 classes. Applied Intelligence, 55(4):309, 2025.
Lianyu Pang, Jian Yin, Baoquan Zhao, Feize Wu, Fu Lee Wang, Qing Li, and Xudong Mao. Attndreambooth: Towards text-aligned personalized text-to-image generation. Advances in Neural Information Processing Systems, 37:39869-3999 -99900, 2024.
Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong. Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. Dreambench++: A human-aligned benchmark for personalized image generation. In The Thirteenth International Conference on Learning Representations, 2025. URL
net ntt/ 4GSOEEJJJKK
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763. PMLR, 2021.
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton Leyton-Brown, and Yoav Shoham. In-context retrieval-augmmented language models. Transactions of the Association for Computational Linguistics, 1 11:1316-1331. 2023.
Arijit Ray, Filip Radenovic, Abhimanyu Dubey, Bryan Plummer, Ranjay Krishna, and Kate Saenko. Cola: A benchmark for compositional text-to-image retrieval. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Published as a conference paper at ICLR 2026
Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends@ in Information Retrieval, 3(4):333-389 2009.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 22500-22510, 2023.
Dvir Samuel, Rami Ben-Ari, Nir Darshan, Haggai Maron, and Gal Chechik. Norm-guided latent space exploration for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024a.
Dvir Samuel, Rami Ben-Ari, Simon Raviv, Nir Darshan, and Gal Chechik. Generating images of rare concepts using pre-trained diffusion models. In Proceedings of the Thirty-Fighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence, pp. 4695-4703, 2024b. 2024b.
Fulvio Sanguigni, Davide Morelli, Marcella Cornia, Rita Cucchiara, et al. Fashion-rag: Multimodal fashion image editing via retrieval-augmented generation. In Proceedings of the International Joint Conference on Neural Networks, 2025.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 15:25278--52944 2022.
Shelly Sheynin, Oron Ashual, Adam Polyak, Uriel Singer, Oran Gafni, Eliya Nachmani, and Yaniv Taigman. knn-diffusion: Image generation via large-scale retrieval. In The Eleventh International Conference on Learning Representations, 2022.
Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-contex: learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 14398-14409, 2024.
Zhenxiong Tan, Songhua Liu, Xingyi Yang. Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint arXiv:2411.15098, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint 2xXv:23322.1800 2023.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8769-8777 2018.
Edward Vendrow, Omiros Pantazis, Alexander Shepard, Gabriel Brostow, Kate E Jones, Oisin Mae Aodha, Sara Beery, and Grant Van Horn. Inquire: natural world text-to-image retrieval benchmark. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman, p+: Extended textual condition- ing in text-to-image generation. arXiv preprint arXiv:2303.09522. 2023.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
14

---

Published as a conference paper at ICLR 2026
Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, and Yao Hu. Instantid: Zero-shot edentity-preservine generation in seconds. arXiv preprint arXiv:2401.07519, 2024a.
Zhendong Wang, Yifan Jiang, Yadong Lu, Pengcheng He, Weizhu Chen, Zhangyang Wang, Mingyuan Zhou, et al. In-contexi learning unlocked for diffusion models. Advances in Neural Information Processing Systems, 36:8542- 118542-8662... 2023.
Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. Genartist: Multimodal Ilm as an agent for unified image generation and editing, Advances in Neural Information Processing Systems, 37: 128374-128395, 2024b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of-thouugh prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022a.
XS Wei, YZ Song, OM Aodha, J Wu, Y Peng. J Tang, J Yang, and S Belongie. Fine-grained image analysis with deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):8927-88)) 8948, 2022b.
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. arXiv preprint arXiv:2409. 1340, 2024.
Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303. 11381, 2023.
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.0000 06721), 2023.
Huaying Yuan, Ziliang Zhao, Shuting Wang, Shitao Xiao, Minheng Ni, Zheng Liu, and Zhicheng Dou. Finerag: Fine- tine-graieed retrieval augmented text-to-image generation. In Proceedings of the 31st International Conference Computational Linguistics, pp. 11 11196-15 1205, 2025.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11975-1 1986, 2023.
Gong Zhang, Kihyuk Sohn, Meera Hahn, Humphrey Shi, and Irfan Essa Finestyle: Fine-grained controllable style personalization for text-to-image models. Advances in Neural Information Processing Systems, 37::2937-552611 2024.
Han Zhang, Rotem Shalev-Arkushin, Vasileios Baltatzis, Connor Gillis, Gierad Laput, Raja Kushal- nagar, Lorna Quandt, Leah Findlater, Abdelkareem Bedri, and Colin Lea. Towards ai-driven sign language generation with non-manual markers. arXiv preprint 2502:22502.0661,, 2025.
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. In The Eleventh International Conference Learning Representations, 2022.
Lvmin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023.
15

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
We use a random subset of LAION (Schuhmann 2022) containing 350K images the dataset from which we retrieve images. As retrieval similarity metric, we use CLIP "ViT-B/32". We do not use retrieval threshold; however, we notice that all retrieved images in our experiments have similarity greater than 0.26. For VLM we use GPT-40-2024-08-- (Hurst et al. 2024) with
temperature of for higher consistency (unless GPT fails to find concepts, see Appendix D). Full GPT prompts supplied in Appendix [ As our T2I generation base models use SDXL (Podell
ct al.
2024) with the ViT-H aPadppperr YYe al. 2023) plus version ("ip-adapter-pluc sdxl vit-h
using 0.5 for the ip. ciapter_scalee FLUX (Labs 2023) schnell with OminiControl (Tan et al. 2024
using 50 steps, and OmniGen (Xiao et al., 2024) with the default parameters (2.5 guidance scale, 1.6
image guidance scale). As OmniGen only supports images as context, we up 3 concepts for each prompt and image per concept. For SDXL and FLUX, as the encoders we used are limited image, we use 1 concept and image.
The time and resources required to run our method depend on the used baseline model and the used retrieval-dataset. However, CLIP embeddings for the retrieval-dataset are pre-computed once, so at inference finding compatible reference images operates in sub-second time. Our experiments show that adding reference images the base generation models does not meaningfully increase the latency of the models. Therefore, the only additional time our method requires is the time required to perform the iterative sorii--ffthoughh with the VLM through 3 API calls with an image, for decision, identification, and image -aptionggenerationn parts, and generating the second image, which depends on the base generative model and could optimized per model, as described in Sec. With GPT-4o and Qwen-2.5, the CoT process takes 10-30 seconds in total, where for GPT it depends network latency and API queue delays. If using GPT, the cost the 3 calls together approximately 0.01-0.05S.
B
VISUAL EXAMPLES
ROM
IajaRA
Re- Ba-lmagen
ImageRAAG
KWNN-Dffusion
Ima
"A Gorrl cotida stanating frant diurch with steepte."
"A Bessonn Seor
Shosta playing booketloud
reseling
nesupaper.
"A agborg Eoodo wearing onwar.
"A Souss made banonas Londrord styyo"
"A corgi biving sucbi."
of
"Vestor iobustrstion of red tiger haod."
Berganaszzo dhackard dog catebhing fridhee."
"Atien cart.on
"Ructy fire bystreet is to the
"heed Kids Brekpeek Warter.""
"weture"
zdga
painteed greas.
Figure 6: Comparisons between ImageRAG and different methods using retrieval for generation. Prompts and results of all other methods taken from their papers. The methods compared to are RDM Blattmann et al. (2022). Re-Imagen Chen et al. (2022). and KNN-Diffusion (Sheynin et al. (2022).
Pair examples rare and fine-grained concept generation with and without ImageRAG are presented in Fig. (OmniGen examples), Fig. (SDXL examples), and Fig. (FLUX examples).
16

---

Published as a conference paper at ICLR 2026
OmniGen
ImageRAG
Real example
"A summerr tanager on a tree."
"A Tursiops truncatus swinmming in the ocean."
"Cyanocitta cristata on a tree with red fruit on it."
"An Struthio camelus in the savanna.
"A chow wearing a sombrero."
Figure 7: Examples of rare concept generation using ImageRAG with OmniGen.
17

---

Published as a conference paper at ICLR 2026
SOXL
ImageRAG
Real example
"Vanessa kershawi smelling a flower."
"A Pracyon lotor on tree."
"A close-up image of Diadon holocanthus."
"A whip poor will on tree."
"An Struthio camelus in the savanna."
Figure Examples rare concept generation using ImageRAG with SDXL.
18

---

Published as a conference paper at ICLR 2026
FLUX
ImageRAG
Real example
"A summer tanager on tree."
"A corgi catching a frisbee."
"Great Pyrenees with mountains on the background." "
"An Anemone fish in the ocean."
"A bowl with corn.
Figure Examples of rare concept generation using ImageRAG with FLUX.
19

---

Published as a conference paper at ICLR 2026
OmniGen
ImageRAG
OmniGen
ImigeRRGG
"A rhino working in a construction site." "
"Origami birds tlying over New York."
"A green namba in a pink room.
"An African grey underwater."
"The Taj Mohal made of sand in the middle of desert."
"A Lego of raccoon in a child bedroom."
"A wooden duck swinmming in a lake."
"Dolphins swimming in the iguazu.
"An illustration of happy blue tang in the sea.
"A whale sculpture from recycled plastic on a beach."
Figure 10: More creative generation examples.
"A tiger in lab coat with 1 1980s Miami vibe, turning well oiled science content machine, digital art."
"Photo of an athlete cat exploining it's latest scandal at a press conference to jouraliste..
"A raccoon wearing formal clothes, wearing a tophat and holding cane.
Text prompt
The raccoon is holding garbage bag. Oil painting in the style of.."
abstract cubism
pointilism
OmniGen
ImageRAG
MAsDnnE FEALCC
Figure 11: More long and complex generation examples.
20

---

Published as a conference paper at ICLR 2026
More creative examples are presented in Fig. 10 Longer and more complex results are presented in Fig. 11
Retrieval data
Retrieval method
Underlying model
"The New York Skybine with "Depp Learning" In foreworks."
"A neon sipn With the tect 'Neur
"A photo of a boston bulll in field." "
"A photo of pembroke dog on garden.
"Five dags on the street."
"Four cars on the street."
Prompt
No ImageRAG
Deep LeARiNG
nUVLS
DDEEP
UNLEP
ImageRAG
LEARNING
Figure 12: Limitations of our method. Left: retrieval data. If the retrieval data does not contain relevant examples, our method cannot help. the examples above we used CUB as the retrieval dataset, which only contains images of birds, thus it does not help for generation of other concepts such as dog breeds. Middle: retrieval method. Our method relies on the quality of the retrieval method. e.g. when using CLIP - we cannot help with counting Paiss et al. (003).. Right: underlying model. Some concepts are not well learned from images by the base models, such as text. these cases, our ability to help is limited.
Visual examples of limitations of our method are presented in Fig. 12
C RETRIEVAL-CAPTION GENERATION PROMPTS
Full prompts used for querying GPT the retrieval-caption generation part of our method:
Decision: 'Does this image match the prompt " "(prompt)"" Consider both content and style aspects. Only answer yes or no.'
'What are the differences between this image and the required prompt? In your answer only provide missing concepts in terms of content and style, each in separate line. For example, if the prompt "An oil painting of sheep and car" and the image is painting of a car but not an oil painting, the missing concepts will be: oil painting style a sheep'
Missing Concepts Identification:
'For each concept you suggested above, please suggest an image caption describing an image that explains this concept only. The captions should be stand-alone description of the images, assuming no knowledge of the given images and prompt, that can use to lookup images with automatically. In your answer only provide the image captions, each in new line with nothing else other than the caption."
Caption Generation:
Rephrase request prompt: prompt used for the rephrasing ablation experiment: 'Please rephrase the following prompt to make easier and clearer for the text-lo-image generation model that generated the above image for this prompt. The goal is to generate image that matches the given text prompt. the prompt is already clear, return as it is. Simplify and shorten long descriptions of known objects/entities but DO NOT change the original meaning of the text prompt. If the prompt contains rare words, change those words to description of their meaning. In your answer only provide the prompt and nothing else. The prompt be rephrased: "(prompt)".
21

---

Published as a conference paper at ICLR 2026
Table 5: Comparisons on fine-grained image generation with T2I models. For each set, we report CLIP-T and CLIP-I similarity scores. First-part rows feature OmniGen-brrd based models, second-part feature S..UX-basee models, and bottom feature SDXL-based models. In each part, best results are bolded.
Dogs
Flowers
Cars
CLIP-T
CLIP-I
CLIP-T1
CLIP-I
CLIP-T 1
CLIP-1
OmniGen ImageRAG-O
0.27
0.52
0.28
0.58
0.24
0.47
0.30
0.57
0.31
0.66
0.29
0.55
FLUX
0.30
0.62
0.30
0.66
0.31
0.60
ImageRAG-F
0.31
0.63
0.31
0.68
0.31
0.60
SDXL
0.34
0.61
0.34
0.68
0.34
0.62
ImageRAG-SD
0.34
0.62
0.34
0.69
0.34
0.62
D VLM ERROR HANDLING
The VLM may sometimes fail identify the missing concepts in an image, and will respond that is "unable respond". In these rare cases, allow up to 3 query repetitions, while increasing the query temperature each repetition. Increasing temperature allows for more diverse responses by encouraging the model to sample less probable words. In most cases, using our suggested step-by-step method yields better results than retrieving images directly from the given prompt (see Tabs, 3 and S). However, if the VLM still fails to identify the missing concepts after multiple attempts, we fall back to retrieving images directly from the prompt, as usually means the VLM does not know what is the meaning of the prompt.
ADDITIONAL. EXPERIMENTS
E.1
ADDITIONAL DATASETS
OmriGen
ImageRAG O
SDXL
ImageRAG SD
FLUX
ImagaRAG-F
Retrioued Referonce
"A photo of Pembrokk.""
"A photo of blvetick""
"A photo of tree poppy."
"Chevroket Morte Corko Coupe 70 7
Figure 13: Visual examples from the Dogs, Flowers, and Cars datasets.
Following RealRAG Lyu et al. 2025) we evaluate our method over three additional datasets: Oxford Flowers Nilssbck & Zisserman 2008). Stanford Dogs (Khosla al. (201)), and Stanford Cars (Krause et al. (013)) with the metrics CLIP-T (text-image similarity), and CLIP-I (image-image similarity, between generated and ground-truth images). Similarly RealRAG, we included the dogs,
22

---

Published as a conference paper at ICLR 2026
flowers, and cars datasets in the retrieval dataset, as in our "proprietary data generation" experiment (Appendix[Es) (E.3). Results are presented Tab. [5] Visual examples from generations of these sets are presented in Fig. 13
E.2 PERSONALIZED GENERATION
For models that support multiple input images, we can combine our method with personalized generation, to generate concept combinations with personal concepts. In this case, image for personal content, and I+ other reference images for missing concepts. For example, given an image of specific cat, we can generate diverse images of it, ranging from mug featuring the to lego of it or atypical situations like the cat writing code or teaching a classroom of dogs (Fig. 14].
one
<p> My cat is the cot in thiis image:
"My cat in ctossroom teaching dogs."
"My cat on cart food commercial."
"A Eego of my cat on a kids room.."
"My cat writingg code in computer.
"My cat on a ary.."
OmnniGan
ImageRAG
Figure 14: Personalized generation example. ImageRAG can work in parallel with personalization methods and enhance their capabilities. For example, although OmniGen can generate images of subject based on an image, struggles to generate some concepts. Using references retrieved by our method, it can generate the required result.
E.3 PROPRIETARY DATA GENERATION
Table 6: Proprietary data usage experiment. Results for using each dataset the retrieval-dataset ("Proprietary-<mokke
vs. using our subset from LAION as the retrieval-datiae ("LAION- <model>"). Here, "O" indicates OmniGen based models, "F" indicates FLUX based models, and "SD" indicates SDXL based models, Best results for each model are bolded.
ImageNet SigL.P t
iNaturalist
CLIP t
DINO
CL.IP
SigL.IP +
DINO
LAION-O
0.264
0.001
0.134 + 0.001
0.708 + 0.002
0.197
0.002
0.095 0.002
0.701 0.002
Proprietary-O
0.266
0.001
0.136
0.001
0.710
0.002
0.212
0.002
0.114 0.001
0.732 0.002
LAION-F
0.271
0.001
0.137 + 0.001
0.698 + 0.002
0.222
0.002
0.065
0.002
0.654 0.002
Proprietary-F
0.001 0.275
0.140 0.001
0.002
0.227
0.002
0.080
0.002
0.682 0.002
LAION-SD
0.274 0.001
0.141 + 0.001
0.709 + 0.002
0.243
0.002
0.118
0.001
0.724 0.002
Proprietary-SD
0.288 0.001
0.142 1 0.001
0.736 = 0.003
0.251
0.002
0.118
0.002
0.737 0.002
common use for RAG in NLP is generation based on proprictary data (Lewis et al. 2020), where the retrieval-dataset is proprietary. A similar application in image generation generating images
23

---

Published as a conference paper at ICLR 2026
based proprietary gallery of images; e.g., for personalization, where the gallery is of a personal concept or a company brand, or a private collection of images that could broaden the knowledge of a model. Our LAION-based experiments explored the scenario where user has access to general, large-scale set. Here, we further evaluate the performance of ImageRAG when we have access to a potentially smaller, specialized dataset. We repeat the experiments with the datasets used in Tab. 2 but this time retrieve samples from within each dataset rather than from the LAION subset. Results are reported in Tabs. F 7 We observe that although applying our method with the generic dataset of LAION subset already improves the results, they improve even further when using proprietary petrivval-datasee
E.4 ADDITIONAL ABLATIONS
OPT repirase
Retrieve Concapte
Retrisse Caphonna
GPT rechrane
Ratrieue
Ratrieue
OrriiBnn
SDXL
Canoapta
Capliona
"Cetlador
Coltuter telxphone"
tebedhora"
1199
"Graany Smith
"Elactrie fon.
"Chave"
"Chos""
"Mrizon yr2""
"Laori"
Figure 15: Visual examples of the ablation studies. Left: Omnigen, right: SDXL.
Ablations studies over SDXL., as explained in Sec. 4 Junder ablations, are reported Tab. presents visual examples of the ablations over Omnigen and SDXL.
Fig. 15
Retrieval-dataset size: we investigate the effect of the retrieval-dataset size. We tested our method over ImageNet (Deng al., 2009) and Aireraft (Maji al. (013) when using 1000, 10,000, 100,000, and 350,000 examples from LAION Schuhmann et al. (022). Fig. T6 shows that increasing the dataset size typically leads better results. However, even using relatively small dataset can already lead to improvements. For OmniGen, 1000 examples were enough to see improvement over the baseline model. SDXI. has stronger baseline, hence more examples were needed for improvement.
Table 7: Additional proprictary data usage experiments. Results for using each dataset as the retrieval-dataset ("Proprietayy<<modeb> ("LAION-<model>> Here, "O" indicates OmniGen based models, "SD" indicates SDXL based models. Best results for each model are bolded.
vs. using subset from LAION as the retrieval-dataset
CUB
Airerafi SigLIP
CLIP f
SigLIP f
DINO 1
CLIP f
DINO f
LAION-O
0.253 + 0.003
0.525 + 0.002
0.760 + 0.003
0.228 + 0.006
0.103 + 0.005
0.747 + 0.010
Proprictary-O
0.269 + 0.003
0.136 + 0.002
0.773 = 0.004
0.244 = 0.007
0.109 + 0.005
0.786 =: 0.010
LAION-F
0.267 +: 0.003
0.135 1. 0.002
0.746 1: 0.004
0.266 :: 0.006
0.128 +: 0.005
0.738 1: 0.002
Proprictary-F
0,291 + 0.002
0.153 1 0.002
0.770 + 0.002
0.269 = 0.006
0.137 + 0.004
0.753 +: 0.087
SD
0.314 T 0.001
0.174 1. 0.002
0.784 1 0.001
0.272 1 0.005
0.141 1 0.005
0.756 1 0.011
Proprietary-SD
0.314 + 0.002
0.175 + 0.001
0.786 + 0.003
0.280 + 0.005
0.152 + 0.003
0.785 0.009
24

---

Published as a conference paper at ICLR 2026
Table 8: Ablation studies over SDXL. "Rephrased prompt" stands for only rephrasing the text prompt without giving additional images. "Retrieve concepts" stands for using the missing concepts directly instead of using more detailed image captions for retrieval, and "Retrieve prompt" stands for using the prompt directly for retrieval. Best results are bolded.
ImageNet
CUB
CLIP f
SigLIP f
DINO
CLIP
SigLIP
DINO
SDXL
0.267 + 0.002
0.136 + 0.001
0.700
0.003
0.315 : 0.001
0.172 1 0.003
0.782 + 0.002
Rephrused prompt-SD
0.266 0.002
0.000
0.705
0.003
0.309 1 0.003
0.170 : 0.002
0.781 4: 0.400.0000
Retrieve cOncepis-SD
0.274 0.001
0.141
0.001
0.702
0.003
0.312 0.002
0.173 + 0.002
0.777 +: 0.004
Retrieve prD
0.274 + 0.001
0.140
+: 0.001
0.702
0.003
0.314 0.001
0.174 = 0.001
0.778 + 0.004
ImageR AG-SD)
0.274 0.001
0.141
0.001
0,709
0.002
0.001 0.314
0.174
0.002
0.784
0.001
ImageNet
Aircraft
0.28
0.275
w/o RAG
0.270
0.26
0.265
CLIP Score
CLIP Score
0.24
0.260
0.22
0.255
OmniGen SDXL+IPA
OmniGen
IDXL+IPA
0.250
0.20
w/a RAG
0.245
0.18
10a
10a
10a
10a
10a
10a
# Samples
# Samples
Figure 16: Retrieval dataset size vs. CLIP score on ImageNet (left) and Aircraft (right). Dashed lines represent the scores of the base models. Even relatively small, unspecialized retrieval sets can already improve results. More data leads to further increased scores. However, small sets may not contain relevant retrieval examples, and their use may harm results, particularly for stronger models.
25

---

Published as a conference paper at ICLR 2026
E.5 VLM ROBUSTNESS
We performed a VLM robustness experiment to choose which VLM we should use and make sure VLM can accurately identify missing rare concepts in images. We randomly sampled 20 classes from the fine-grained dataset iNaturalist (Van Horn et al. 2018), and for each class, we generated types of prompts: "A photo of a <class name>" " 2. "A photo of a <class name> and <other>" 3. "A photo of <other>". In total, we obtained 780 prompts; 20 prompts of the first type (1 per class), and 380 prompts for each of the second and third types (19 for each class, for every class other than the <class name>>. Finally, for each prompt, we asked the VLM if the prompt matches an image of that class, and if not, what are the missing concepts, as in our method. Note that each photo actually contains <class name> but not <other>. This way, were able to evaluate the ability of the VLM to identify missing rare concepts in images. The results of this experiment were a success; GPT-4o (Hurst et al.] (2024), which is the VLM used in experiments, achieved 100% correct answers for the first and third prompt types and 99.7% correct answers for the second prompt type (1 wrong answer). We repeated this experiment with Gemini (Team al., 2023) which achieved 95% correct first-type answers (1 wrong), 98.9% correct second-type, and 90% correct third-type, and with Dwen2.5-VV-77--Isttuee answers, respectively. This indicates that while GPT identifies rare concepts best, both Gemini and Qwen also perform well and could potentially be used instead of GPT in the pipeline. We performed this experiment with LLa VA (Liu et al. (223) as well, but it did not succeed at all (Simply answered 'No' on all queries).
(Bai al. 2025) which achieved 100%, 100%, and 92% correct
E.6 DIVERSITY
Real example
ImageRAG-FLUX different seeds
"A 2alophus californianus on rocks by the sea."
"An Anerone fish in the ocean."
"An Anos platgrlynchos in a river."
"A Cyanocitta cristata on a tree."
"Great Pyrenees with mountains on the background"
Figure 17: ImageRAG-F (mLnX+Omin(Controo) generations with different seeds. Left-most column presents real example of the rare concept in the prompt, other columns present diverse generations the same prompt by ImageRAG-F.
Generation examples with different seeds are presented in Fig. 18 (OmniGen), Fig. 19 SDXL+II. Adapter), and Fig. 17 (FLUX+OminiControl)
The diversity of generated results varies with the conditioning method, though all models can produce diverse outputs aligned with the text prompt. We observe some trade-off between diversity and
26

---

Published as a conference paper at ICLR 2026
Real example
ImageRAG-Omnilen different seeds
"A Zalophus californianie on rocks by the cea."
"A Procyon Rotor perched on tree."
"A Cyanacitta oristata on a tree."
"An Anas platyrhynchhs in river."
"A photo of a boston bull in field."
Figure 18: ImageRAG-O (OmniGen) generations with different seeds. Left-most column presents real example of the rare concept in the prompt, other columns present diverse generations of the same prompt by ImageRAG-O.
textual faithfulness that depends the chosen model. Since our approach is compatible with various architectures, the model can be selected based on the priorities Adapter yields outputs that are less diverse but closely match the reference image, OmniGen favors higher diversity at the cost of slightly reduced faithfulness, and PLUX+OminiControl provide trade-off between the two.
user. For instance, SDXL+IP.
F USER STUDY QUESTIONS
In the user study, asked users to compare pairs of images at time, by asking which adheres better to the prompt and has better visual quality. We supplied real references (not from our dataset) asked were: For each criteria, choose the better
for rare concepts with each pair. The questions image out of A and B given the following text prompt: <prompt>, The less familiar concept "<rare_com pcrare_concep>> is presented on the left of the image options.
Better text alignment (choose A B)
Better visual quality (choose A or B)
Overall preference (choose A or B)
Pair examples of using our method vs. other retrieval-based generation approaches can be found in Fig. 6 Due to lack of access to the models, all prompts and results of the other methods were taken from their papers.
G LLM USAGE
have used LLM (specifically, ChatGPT), for proofreading and rephrasing during paper writing.
27

---

Published as a conference paper at ICLR 2026
Real example
ImageRAG-SDXX different seeds
"A Proojon lotor perched on tree."
"Vonessa kershowi in field."
"A Gyanocitta cristata on tree."
"An oil pointing of Pracyos Rotor."
"A crodle and golden retriever."
Figure 19: ImageRAG-SD (SDXL=IP-Adapte)) generations with different seeds. Left-most column presents real example of the rare concept in the prompt, other columns present diverse generations of the same prompt by ImageRAG-SD.
28


Reviews:
[{"review_id": "DQUXVPPXtu", "reviewer": "Reviewer_bsqR", "summary": "This paper proposes ImageRAG, a training-free retrieval-augmented generation method that enhances text-to-image models by dynamically identifying generation gaps using a vision-language model and retrieving relevant reference images to guide the synthesis of rare or fine-grained concepts. The approach is model-agnostic, demonstrated through evaluations on diverse backbones like OmniGen, SDXL, and FLUX, showing improved alignment with prompts without requiring additional training.", "questions": "See weakness", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method's adaptability is a key strength, as it seamlessly integrates with existing image-conditioning models like IP-Adapter and OminiControl, enabling broad applicability across different architectures. Extensive experiments, including quantitative metrics, qualitative examples, and human studies, robustly validate that ImageRAG consistently improves rare concept generation, with users preferring it over baselines in text alignment and visual quality.", "weaknesses": "1. The method's efficacy is highly dependent on the retrieval dataset; if it lacks relevant images (e.g., specializing in birds when generating dogs), performance may not improve, as illustrated in the retrieval data limitations.\n\n2. It relies on the VLM's accuracy for gap identification; errors in concept detection (e.g., false positives in alignment checks) could lead to missed enhancements, though the paper notes robustness issues with some VLMs.\n\n3. While the paper notes VLM API calls add 10–30 seconds, it lacks comparisons to baseline T2I inference latency and does not explore optimizations (e.g., cached embeddings for small datasets) for real-time use cases.\n\n4. Although diversity is mentioned, the paper does not quantify how ImageRAG impacts generation diversity across models (e.g., SDXL’s low diversity vs. OmniGen’s high diversity) or whether retrieval introduces bias toward reference image styles.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5y6bMHyzAq", "reviewer": "Reviewer_fN6i", "summary": "This paper introduces RAG into text-to-image generation process for some rare or fine-grained concepts. This approach is training-free and model agnostic, and can operate with off-the-shelf image-conditioning interfaces. Image RAG contains four steps: 1/ rough image generation from a text prompt; 2/ use VLM to identify the missing concepts in a CoT process,  3/ retrieve reference images corresponding to the missing concepts from an external dataset; 4/ regenerate the image using these reference images by a generative model. The authors evaluated the method using three different base models, including SDXL, FLUX, and OmniGen, on three benchmarks.", "questions": "- Since the limitations are listed in this paper, is there any solution or strategy to improve them?\n\n- Is there any failure cases that VLM misidentify the gaps between text prompts and rough images?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The adaptation of RAG to text-to-image generation is inspiring. It focuses on \"polish\" the generated image at inference time rather than retraining.\n\n- The formulation of CoT-based retrieval is promising, which is effective for identifying the missing visual elements.\n\n- The evaluation is comprehensive, including qualitative comparisons, user study, and ablation studies. It is good to see that failure cases are included in this paper, which help the reader to better understand the limitation of this paper.", "weaknesses": "- The applicable scenario is limited. ImageRAG suits most for the scenario where rare or even weird concepts exist, such as \"a boston bull\", but the successfulness greatly depends on the coverage of the retrieval dataset (e.g. LAION-350K subset). It would be better to see that this method can handle well with the complex and lengthy prompts that are usually more customer-driven. \n\n- No report or measurement of the latency and computational cost of this multi-step pipeline, which is even more important than the metrics measuring image quality. We should always consider the latency for real-time or interactive use cases.\n\n- The technical novelty is limited. Even though the concept of ImageRAG is good, each component in this framework is individually normal. And the success of ImageRAG heavily rely on the performance of the submodule such as CLIP, GPT-4o and T2I, etc. \n\n- The performance improvements in CLIP or DINO metrics are in general very limited (<0.01 absolute).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dNBRW8xQEu", "reviewer": "Reviewer_1GfW", "summary": "The paper propose a image Retrieval-Augmented Generation (RAG) framework for image generation named ImageRAG. ImageRAG using VLM to retrieves relevant images to enhance custom image generation. Through extensive quantitative, qualitative, and subjective evaluation, paper show that incorporating retrieved references consistently improves the generation abilities of rare and fine-grained concepts.", "questions": "SDXL+IPA have different image and text input interface, and cannot use the prompt template in Figure 3. How dose the ImageRAG handle models like this? Are the images correctly associated with the corresponding text snippets?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. The paper proposes a novel pipeline designed for image retrieval-augmented generation (RAG), which can retrieve relevant images to enhance the generative performance of diffusion models.\n2. The proposed framework automates the common engineering practice of improving generation quality through reference images, demonstrating strong practical value.\n3. The paper introduces an efficient approach for leveraging image knowledge database, which offers valuable insights for the development of multimodal large models.\n4. The paper is clear and easy to follow.", "weaknesses": "1. If the target concept to be generated does not exist in the retrieval dataset, is there a possibility that irrelevant images may be selected, thereby potentially affecting the generation quality adversely? The paper should include corresponding quantitative analyses.  \n2. [Retrieval-Augmented Diffusion Models] has proposed a similar concept. The paper should provide a detailed comparison with this work with same base model and dataset, and analyze advantages of ImageRAG.  \n3. Will the diversity of generated images be constrained by the retrieval dataset, leading to visually similar outputs for same prompts? The paper should include relevant analyses to clarify this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Aph5lgkIPx", "reviewer": "Reviewer_vzAE", "summary": "This paper aims to address the recent generative models problems in generating rare or fine-grained concepts. ImageRAG framework is introduced as a training-free method for rare concept generation. ImageRAG identifies generation gaps with the guidance from a Vision Language Model. The VLM retrieves relevant images while ImageRAG uses them as context to guide the generation process.", "questions": "### **1. On Related Work and Comparison with RealRAG**\n\n**a. Conceptual Differences**\n\n* In this paper, *RealRAG* (Lyu et al., ICML 2024) is mentioned only briefly in the related work section but not discussed elsewhere.\n  Could the author elaborate on how the method differs conceptually from RealRAG, especially given that RealRAG also employs retrieval-augmented generation but does **not** require task-specific model retraining?\n* What are the main methodological distinctions between ImageRAG and RealRAG in terms of:\n\n  * Retrieval mechanisms and\n  * Integration with the generation process?\n\n**b. Experimental Comparison**\n\n* In the experiments, only a few baseline models are included. Could the authors clarify why RealRAG and other recent retrieval-based image generation methods were excluded from the comparison?\n* Have the authors evaluated the framework under the same experimental conditions as RealRAG (e.g., dataset, retrieval backbone, evaluation metrics) to ensure a fair comparison?\n\n---\n\n### **2. On Chain-of-Thought (CoT) Reasoning for Concept Identification**\n\n* The authors mention that CoT reasoning is used to identify challenging or ambiguous concepts. Could the authors expand on how the CoT process interacts with the retrieval or generation stages?\n* Is there a trade-off between **CoT length** and **generation quality** (e.g., longer CoTs improving reasoning but reducing visual fidelity or inference speed)?\n* Have the authors observed any “overthinking” effects — cases where extended reasoning sequences introduce redundancy or semantic drift — and if so, how do the authors mitigate them?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The idea of applying RAG in image generation is up-to-date.\n\n2. The method illustration is clear.\n\n3. Writing is easy to follow.", "weaknesses": "1. As the most relevant work, RealRAG[A] is only mentioned in the related work section, while ignoring in the other sections, especially for the introduction and experiments.  \n\na. Line-090: \"previous works employing image retrieval for better image generation, train models specifically for the task, hindering wide applicability.\" The reviewer believe RealRAG does not train models specifically for task. \n\nb. In the experiments, only several base models are included for comparison, more methods in this area should be compared, especially for RealRAG[A].\n\n[A] Lyu Y, Zheng X, Jiang L, et al. RealRAG: Retrieval-augmented Realistic Image Generation via Self-reflective Contrastive Learning[C]//Forty-second International Conference on Machine Learning.\n\n2. In the method, CoT is used for identifying challenging concepts. Could the author provide more discussion regarding this part? Is there a  trade-off between the CoT length and the generation performance? Is there any over thinking problem in this scenarios?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)