Wiki-R1: Incentivizing Multimodal Reasoning for Knowledge-based VQA via Data and Sampling Curriculum
Authors: Shan Ning
Longtian Qiu
Xuming He
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Learning
Visual Question Answering
Reinforcement Learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A data-generation-based curriculum RL framework for knowledge-based visual question answering

Abstract: Knowledge-Based Visual Question Answering (KB-VQA) requires models to answer questions about an image by integrating external knowledge, posing significant challenges due to noisy retrieval and the structured, encyclopedic nature of the knowledge base. These characteristics create a distributional gap from pretrained multimodal large language models (MLLMs), making effective reasoning and domain adaptation difficult in the post-training stage. In this work, we propose \textit{Wiki-R1}, a data-generation-based curriculum reinforcement learning framework that systematically incentivizes reasoning in MLLMs for KB-VQA. Wiki-R1 constructs a sequence of training distributions aligned with the model’s evolving capability, bridging the gap from pretraining to the KB-VQA target distribution. We introduce \textit{controllable curriculum data generation}, which manipulates the retriever to produce samples at desired difficulty levels, and a \textit{curriculum sampling strategy} that selects informative samples likely to yield non-zero advantages during RL updates. Sample difficulty is estimated using observed rewards and propagated to unobserved samples to guide learning. Experiments on two KB-VQA benchmarks, Encyclopedic VQA and InfoSeek, demonstrate that Wiki-R1 achieves new state-of-the-art results, improving accuracy from 35.5\% to 37.1\% on Encyclopedic VQA and from 40.1\% to 44.1\% on InfoSeek. The project page is available at https://artanic30.github.io/project_pages/WikiR1/.

Published as a conference paper at ICLR 2026
WIKI-RI: INCENTIVIZING MULTIMODAL REASONING FOR KNOWLEDGE-BASED VQA VIA DATA AND SAM- PLING CURRICULUM
Shan Ning!., Longtian Qiu', Xuming Hell
Shanghai Tech University, Shanghai, China
Shanghai Engincering Research Center of Intelligent Vision and Imaging
9 LLingang Laboratory, Shanghai, China
(ningshan2022, qiult, hexm/@@harghaaitehhhc edu. cn
ABSTRACT
Knowledge-Baud Based Visual Question Answering (KB-VQA) requires models to swer questions about an image by integrating external knowledge, posing signif- icant challenges due noisy retrieval and the structured, encyclopedic nature of the knowledge base. These characteristics create a distributional gap from pre- trained multimodal large language models (MLLMs), making effective reasoning and domain adaptation difficult in the post-training stage. In this work, we propose Wiki-RI, data-generationn-aaae
curriculum reinforcement learning framework that systematically incentivizes reasoning in MLLMs for KB. Wiki-R1 con- structs a sequence of training distributions aligned with the model's evolving capa- bility, bridging the gap from pretraining to the KB-VQA target distribution. We in- troduce controllable curriculum data generation, which manipulates the retriever to produce samples at desired difficulty levels, and curriculum sampling strategy that selects informative samples likely to yield non-zero advantages during RL updates. Sample difficulty is estimated using observed rewards and propagated to unobserved samples to guide learning. Experiments on two KB-V VQA bench- marks, Encyclopedic VQA and InfoSeek, demonstrate that Wiki-R1 achieves new state-of-the-art results, improving accuracy from 35.5% to 37. % on Encyclope- dic VQA and from 40.1 % to 44. 14.1% % InfoSeek. The project page is available /iikii
INTRODUCTION
Knowledge-Band -ased Visual Question Answering (KB-VQA) is a challenging multimodal task that requires answering questions about image by integrating external knowledge. A widely adopted approach is the Retrieval-Augmented Generation (RAG) framework, which leverages pretrained models and is further adapted to the task: retriever first fetches relevant knowledge passages, and a generator then produces an answer conditioned on this context. However, the noise in the retrieval system is inherent, and the knowledge base (Vrandečić & Krötzsch 2014) typically consists of structured, encyclopedic information. Consequently, the model must not only reason over noisy and imperfect external evidence but also comprehend retrieved information presented in a structured, encyclopedic form largely unseen during pretraining. These characteristics position KB-VQA as challenging downstream task for pretrained MLLMs, one that demands robust reasoning ability and effective domain transfer, and is typically addressed in the post-training stage.
Prior work has pursued two main directions. One line aims to improve retrieval quality Lerner et al., 2024) Yan & Xie 2024, Yang et al.] 2025 Deng et al.) 2025), but retrieval remains inher- ently noisy and cannot guarantee full coverage necessary evidence. Another line of work focuses on enhancing reasoning to handle imperfect retrieval. Specifically, models must understand ency- clopedic passages and selectively extract relevant information while filtering out irrelevant content. Early efforts primarily relied supervised fine-tuning (Caffagni et al. 2024| Qi et al. 2024 Coc- chi et al.) 2024), which enables models to reason over retrieved knowledge for specific training instances. However, empirical results indicate that such approaches may have limited reasoning

---

Published as a conference paper at ICLR 2026
Training Progress- Training Accuracy.
Training Progress Ratio
0.22
fatiovel
A NW
Distributton 6ep
Palicy Learing
0.200
GT Retreed
De Dumplyr Sam
0.50
0.75
0.25
0.03
Pretnaining buttiisttinn
Torget Distributton on
0330
Wis Nestica
0.32
100
300 100
466
lieration
Iteration
(a)
(b)
(c)
Figure 1: (1a and (Ib): Training dynamics of DAPO KB-VQA. RL optimization suffers from
a high proportion of zero-advantage samples and low training accuracy. highlighting the distribution gap between pretraining and the KB-VQA target domain. ( [c):: Motivation of Wiki-R1. To mitigate this gap, Wiki-RI generates a sequence of training distributions with progressively reduced discrepancies and employs curriculum sampling strategy to select informative samples.
robustness (Section 4.4 More recent reinforcement learning methods, including GRPO (Shao al., 2024). have demonstrated promising reasoning capabilities in general retrieval-augmented genera- tion (RAG) settings (Jin et al., 2025 Wu al. 2025). Despite these advances, the effectiveness of RL-based approaches in tasks that require both multimodal reasoning and cross-domain adaptation, such as KB-VQA, remains largely unexplored.
To investigate this, we conduct preliminary experiments applying the popular RL. algorithm DAPO (Yu et al., (025) to incentivize the reasoning ability of MILMs on KB-VQA. We observe that over 80% of the samples exhibit zero advantages(Figure [ab during training, and the overall training accuracy remains low, around (Figure [b]. These observations indicate that reinforce- ment learning on KB-VQA suffers from a severe sparse reward problem, which is exacerbated by the distributional gap between the model's pretraining data and the KB-VQA target domain. To further investigate the source of this distributional gap, conduct experiments using the ground- truth retrieval, which corresponds to a setting with substantially reduced retrieval noise. As shown in Figure [1] both the prevalence of zero gradients and the low training accuracy alleviated. This observation indicates that retrieval noise is a significant contributing factor to the sparse reward and ineffective training in RL for KB-VQA.
To address this challenge, we propose a data-generation-basde curriculum reinforcement learning framework, Wiki-RI, designed incentivize the reasoning ability of MLLMs on the challenging KB-VQA task. Wiki-RI constructs a sequence of training distributions adaptively aligned with the model's evolving capability, gradually bridging the gap from pretraining the KB- VQA target dis- tribution, as illustrated in Figure [1 Unlike conventional curriculum learning, we generate training data with controllable difficulty rather than selecting from a fixed dataset. Specifically, we introduce controllable curriculum data generation, which manipulates the retriever to produce samples at the desired difficulty level, adaptively adjusted based on the model's observed training accuracy during RL optimization. Since generated data may not always match the intended difficulty, we further pro- pose a curriculum sampling strategy that selects samples likely to yield RL updates. To estimate sample difficulty, we use observed rewards proxy and propagate this information to unobserved samples. Together, controllable data generation and curriculum sampling form principled framework that systematically guides the model through progressively harder amples, ensuring meaningful learning signals and stable reinforcement learning on KB-VQA.
advantages during
We evaluate our proposed framework on two standard knowledge-based visual question answering benchmarks: Encyclopedic- -VQA (Mensink et al. (023) and InfoSeek (Chen et al., 2023). Our method, Wiki-R1, achieves new sfute-of-the-ar art performance on both datasets, with an accuracy of 37.1% on Encyclopedic-VQU (surpassing the previous best of 35.5%) and 44.1% % on InfoSeek (improving upon the prior state-of-the-art of 40. I 1%). Notably, on the challenging Unseen--uestion split of InfoSeek, our model attains an accuracy of 47.8%. This performance not only exceeds the previous benchmark but also surpasses our model's overall accuracy, underscoring its strong generalization capability to novel queries.
Our main contributions are as follows:
We propose Wiki-RI, a data-generation based curriculum RL framework that incentivizes the reasoning ability of MI.LMs on KB-VQA with data and sampling curriculum.
2

---

Published as a conference paper at ICLR 2026
Wiki-RI constructs curriculum of training distributions by manipulating the retrieval sys- tem and adaptively adjusting difficulty based on the model's performance. Curriculum sampling complements this process by selecting informative samples using propagated re- ward signals, ensuring the curriculum effectively guides learning.
Experimental results demonstrate that Wiki-RI consistently surpasses prior state-of-the methods on two challenging knowledge-baad VQA benchmarks, with particularly pro- nounced improvements in unseen settings.
RELATED WORK
2.1 KNOWLEDGE-BASED VISUAL QUESTION ANSWERING
The KB-VQA task addresses questions whose answers require external or domain-specine knowl- edge beyond what is present in the image itself. Early datasets such as OK-VQA (Marino et al., 2019 [cchwen al. 2022). FVQA (Wang et al.) 2016). KVQA (Shah et al., 2019). S3VQA (Jain) al. ) and ViQuAE (Lerner et al. 2022a) posed questions requiring commonsense knowledge.
Building on these datasets, a line of early methods (Gui et al. 2021; Marino et al. 2021 Hu et al.)
2022 Ding al.) 2022 Lin & Byrne 2022 Wu et al. [0022] Xenos et al. 2023) explored how to utilize external knowledge in VQA through structured knowledge graphs, multi-modal reasoning, or evidence retrieval. With the emergence of LLM-based MLLMs, these early datasets provide only limited coverage for evaluating KB-VQA in more realistic scenarios, as they often lack fine-grained knowledge, require minimal visual understanding, and cover only restricted range of visual en- tity categories, as noted by (Chen et al. (002)). To address these limitations, recent benchmarks such Encyclopedic- -erppedi--QAAAaaa (Mensink al. 2023) and InfoSeek (Chen et al. (023) present greater challenges by targeting highly specific, Wikipedia-scall knowledge. They require models to capture detailed information about particular entities and nuanced encyclopedic facts.
To tackle this task, Retrieval Retrieval-Augmented Generation (RAG) has emerged as a widely adopted paradigm, where models retrieve relevant content from external knowledge bases such as Wikipedia to support question answering. Recent studies can be broadly categorized into two directions. The first focuses on improving the retrieval system itself, for instance, by training contrastive image-
text encoders to achieve more accurate retrieval results (Xu et al.] 20244 Radford et al. 2021 Sun
ct al. 2023 Wei et al. 2023] Xiao et al. 2024 Caffagni et al. 2024| Ning et al. 2023
Qiu et al. 2024 However, due to the large scale of knowledge bases and the inherent long tail distribution
of training data, retrieval noise is often unavoidable. The second line of work, therefore, aims to adapt models noisy retrieval outputs. For example, Wiki-LLaVA (Caffagni et al.) (024) integrates external multimodal knowledge via a hierarchical retrieval pipeline within a contrastive embedding space (Radford et al.. (02T). RoRAVLM (Qi et al. 2024) instead introduces visual token refine- ment module to filter out query-irrelevnt visual information from both retrieved and query images. More recently, ReflectiVA VA (Cocchi et al. 2024) employs reflective tokens to dynamically determine the reliability of retrieved content, thereby mitigating the impact of noisy retrieval results. In this work, we propose leverage reinforcement learning to enhance the model's 's ability to reason under noisy retrieval conditions, enabling it to derive correct answers even when the retrieved content is imperfect.
2.2 CURRICULUM LEARNING FOR RL
Curriculum learning (Bengio et al. 2009 Graves et al. 2017) structures the training process by gradually moving from easier more difficult examples. In reinforcement learning, curricula are typically based on task complexity (Justesen et al., 2018 Wang et al.) 2019 Li et al. (201)), or al- ternatively learned through bacher-student frameworks formulated as partially observable Markov decision processes (Matiisen et al. 2017 Portelas et al.) 2019). With the success of R1epSeek-R1, recent studies have explored incorporating curriculum learning into value-free RL frameworks such as GRPO (Shao et al.) 2024 Yu et al.) 2025 Qiu al. 2025 2026). For instance, ADARFT (Shi et al. 2025a) dynamically prioritizes samples with higher learning potential based on recent reward signals, while DUMP (Wang et al. 2025b) adopts the Upper Confidence Bound principle to adap- tively adjust sampling probabilities across different data distributions. In the context of multimodal
RAG, several works (Ji et al. 2025 Wang et al.
2025a
Zhang et al. 2025) apply fixed curric-
3

---

Published as a conference paper at ICLR 2026
Retrieval System
maiiny
Garolen Ieyg
Quidon harge
Policy
Modil
Taggl
Pramgpont couervet
Passage
Syctelcc Data
Sampling Schedule
Pulky klodd
Ke ve-V Data
Serilin
Sen dlog
So Sempra
Diribtutee
Abrt Bevtton
Une Irpectte
Pretroined MLLM Duttrbution
Lavel
Poct-troined Distribttio
Currculum Data Generation
Curricutum Sampling
Figure 2: Left: Controllable curriculum data generation. We manipulate the retriever generate training samples with gradually increasing difficulty, adaptively aligned with the model's evolving capability, bridging the gap from pretraining the KB-VQU target distribution. Right: Curricu- lum sampling with observation propagation. We adaptively select informative samples likely to produce non-zero advantage during RL updates, with sample difficulty estimated from observed re- wards and propagated to unobserved examples.
ula, training policies progressively from easy to hard samples. More advanced approaches, such as VL-Cogito (Yuan et al. 2025), estimate sample difficulty using current reward signals and dynami- cally adjust sample weights accordingly. In this work, go beyond relection-based curricula and instead generate controllable training distributions, enabling principled, difficulty-aware data con- struction that bridges the gap between pretraining and target distribution. We further introduce observation-propage propagation mechanism that propagates sparse on-policy reward signals to unobserved examples, yielding reliable difficulty estimates drive curriculum sampling.
WIKI-RI
In this section, present Wiki-RI, curriculum reinforcement learning framework that enhances the reasoning ability of multimodal large language models (MLLMs) for the challenging knowledge- based visual question answering (KB-VQA) task. We first formulate the KB-VQA task to establish the problem setting in Section 3.1 and then introduce the training objective under our post-training design two tightly coupled components: (i) curriculum data generation (Section 3.3 which constructs a progressive training sequence from easy to hard by manipulating the retrieval system, and (ii) curriculum sam- pling (Section (.4). which dynamically selects informative samples via observation propagation. The overall pipeline is illustrated in Figure with detailed pseudo-code provided in the appendix.
reinforcement learning setting Section 3.2 Building on this objective,
3.1
TASK DEFINITION
The goal of Knowledge-basee Visual Question Answering (KB-VQA is to generate answer to a textual query q given an image I", by jointly reasoning over the input and relevant knowledge retrieved from an external knowledge base (KB). Formally, we define large-scale multimodal KB, such as Wikipedia /Vrandežić & Krötzsch 2014), as
B (P,, 1
(1)
where P; denotes textual article and T is the corresponding visual content associated with entity i. To incorporate external knowledge, a retriever is employed to select a subset relevant multimodal documents, and non- parameerri,, rule-based retrieval modification function is also incorporated to adjust the retrieval results:
Se Retriever(q, 19, C B.
where denotes the retrieval modification function and S contains the knowledge passages most relevant to the query (q, 19). The retrieved set Sg serves as additional context for answer generation. Formally, the KB- BB-VOA objective to model the conditional distribution of the answer y given the query g, the image 1", and the retrieved knowledge Sz:
max E E(r [logpo(y po (y [9 (7)
(3)
where & denotes the learnable parameters and D the KB-VQA dataset.
4

---

Published as a conference paper at ICLR 2026
3.2 TRAINING OBJECTIVE
To address the challenging KB- -QQ task that requires reasoning ability, we consider post-training setting which a pretrained MLLM is further adapted to the KB-VQA task via reinforcement learning. In this stage, the model is optimized to maximize the expected reward over KB-VQA data, conditioned on the -eeee- pair (g. 1 and the retrieved knowledge Sg. A key challenge is the sparsity of reward signals, which can hinder stable optimization. To address this, we leverage the retrieval modification function e and further introduce a sampling schedule μ., which together shape the learning signal and training distribution. Formally, the gradient under (14, (6) for the model policy ne is given by
(TT p4, 0) ( (eg Vo log (o 19,11,,55 (()
(4)
where ÿ is the sampled answer from the policy Ta, and denotes the ground -ruth answer.
Unlike traditional policy gradient methods that rely random sampling for μ and a fixed retrieval strategy for Φ, our framework Wiki-RI explicitly incorporates curriculum-awan sampling and con- trollable retrieval modifications. This design provides principled way to align data generation with optimization, thereby mitigating the gap between pretraining and target distributions. Further details presented in the following sections.
3.3 CURRICULUM DATA GENERATION
To systematically bridge the gap from pretraining to the KB-VQA target distribution, we manipulate the retriever to generate a sequence of training samples with con- trollable difficulty. Intuitively, retrieving more candidates increases the likelihood of including use- ful query-relevant information, but also introduces additional noise. Motivated by this property, we design controllable data generation method that adjusts both the number of retrieved candidates and whether the ground-truth article is explicitly included in the retrieval results, which is illustrated in Figure 2
Controllable Data Generation
Specifically, we define a discrete gap level E 1..1...., G}. which represents the degree of distri- bution shift between the generated training samples and the true KB-VQA target distribution. For each level we instantiate a retrieval modification function o, (k: 7) that specifies the number of retrieved candidates & and whether the ground-truth snippet 7 enforced.
Easiest level (g 0): we set l: 1 and y which retrieve only the ground-truth snippet.
< G): le is set to g while keeping 1, introducing noisy candidates alongside the ground truth.
Intermediate levels (1
Hardest level (g = G): set 7 and k: = G 1, the retrieval system no longer guarantees inclusion of the ground truth, fully aligning with the inference-time distribution.
This design produces a controllable hierarchy of training distributions, beginning with g= which closely resembles the pretraining distribution, and gradually converging to the target KB-VQA dis- tribution at g = G.
Gap-Level Schedule To dynamically adjust the gap level during training, design schedule based on the model's observed training accuracy. Concretely, we maintain sliding window of the most recent samples and compute the average training accuracy. Once this moving average ceeds an upgrade threshold т, we promote the gap level g + g + and reset the stored observations. This mechanism ensures that the model is gradually exposed more challenging training distri- butions only after has sufficiently mastered the current level, enabling a smooth transition from pretraining-like data the target KB-VQA distribution.
3.4 CURRICULUM SAMPLING WITH OBSERVATION PROPAGATION
Sampling Schedule The training data generated by our controllable curriculum may not fully sat- isfy the desired difficulty. To address this, we introduce a curriculum sampling strategy μs. Prior work (Shi et al.) 2025b) has shown that samples with training accuracy near 0.5 provide the strongest gradient signal for reinforcement learning. Accordingly, during training, we sample data using a Gaussian distribution centered the historical mean training accuracy of 0.5.
5

---

Published as a conference paper at ICLR 2026
Formally, denote by j a sampling schedule represented as a distribution over D:
(g. 19.9% ~ pl)), / E A(D)
(5)
where (g. 19, 19,y) y) denotes a sampled training data. This ensures that the model primarily trains on samples that are challenging yet solvable, maximizing learning efficiency and stabilizing the RL optimization process.
Difficulty Estimation via Observation Propagation A key challenge in the sampling schedule is the sample difficulty estimation. Though observed reward provides a direct evaluation of data, it is extremely sparse, which can undermine the effectiveness of curriculum sampling. To address this, we introduce observation propagation mechanism to estimate the difficulty of unobserved samples, which is illustrated in Figure We leverage the insight that the correlation between differ- ent VQA samples is related to the model's understanding of their associated knowledge base article. Concretely, we construct a label propagation graph over VQA samples, where the edge weights be- tween two samples reflect the similarity of their associated knowledge base articles. We then apply label propagation to propagate observed accuracies from the training set to unobserved samples. This allows to approximate sample-wise expected accuracies, ensuring that curriculum sampling remains effective even under sparse observations. We provide the details of label propagation in the appendix.
4 EXPERIMENTS
In this section, we present the experimental validation of our method on two challenging bench- marks, along with the implementation details. Moreover, we conduct comprehensive ablation stud- ies to demonstrate the effectiveness of each key component of our method.
4.1 EVALUATION BENCHMARKS
Encyclopedic VQA. To evaluate the performance of multi-modal large language models (MILMs) on visual questions requiring extensive external knowledge, we utilize the recently pro- posed Encyclopedic VQA (EVQA) (Mensink et al.] (203)) dataset. This dataset contains visual questions about detailed properties fine-grained categories and is primarily constructed using an- notations from iNaturalist 2021 (Horn et al. 2021) and the Google Landmarks Dataset V2 (Weyand) ct al. 2020). The Encyclopedic VQA dataset comprises approximately 22 Ik question-answer pairs associated with 16.7k different fine-graie fine-grained entities, each represented by up to five images. The dataset is divided into training, validation, and test splits, containing 1M, 13.6k, and 5.4k samples, respectively. For the knowledge base, Encyclopedic VQA filters out non-English Wikipedia pages from the WIT dataset (Srinivasan et al. 2021 and compiles a total of 2M Wikipedia pages. We report the BEM (Bulian et al. 2022) score of the test set using official scripts.
InfoSeek. The InfoSeek (Chen et al. 2023) benchmark is tailored for ietemmatiionseekinng ques- tions that require expert knowledge. It consists of 1.3 million visual information-secte seeking questions, encompassing more than 11,000 visual entities from OVEN (Hu et al. 2023). The dataset comprises 934k training, 73k validation, and 348k test samples. Due to computational resource restrictions, we sample class-balanced 1k validation set report the final performance with official scripts and select another Ik subset for hyperparameter selection. For the knowledge base, we follow previous works (Yan & Xie 2024] Coochii al. 2024) and utilize knowledge base with 100,000 Wikipedia articles accompanied by images.
4.2 BASELINES
To evaluate the effectiveness of our method, we consider two categories of basclines. (1) Zero-shot MILMMs The first category consists of zero-shot multimodal large language models (MLLMss). We evaluate models of different scales, including BLIP-2 (Li et al.) 2023). InstructBLIP (Dai et al.) 2023), LLaVA 1.5 (Liu et al. 2023). Qwen2.5-VL (Bai et al. 2025), and GPT-4V (OpenAl 2023). These models are directly applied to KB-VQA without retrieval augmentation, which highlights the inherent difficulty of the task. (2) aetrieval-augmenne
Generation. The second category corre- sponds methods under the retrieval-auue -augmented generation (RAG) setting. In this setting, models enhance answer accuracy by retrieving relevant snippets from an external knowledge base. Since our focus is KB-VQA with noisy retrieval system, we primarily compare with methods that do

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison on Encyclopedic VQA and InfoSeek. All results of retrieval- augmented generation methods are reported without applying any re-ranking stage to reorder re- trieved documents. Retrieval Mode spans two columns: the first specifies the retrieval model, while the second indicates the type of knowledge source utilized. The V. and T. indicate the visual and textual retrieval mode. The Con, and Col. indicate textual retrieval model, Contriver (Izacard al, 202T and Colbert V2 (Santhanam et al. 202T) respectively.
Retrieval Mode
EVQA
InfoSeek Unseen-E
Method
Avg.
Single-hop
All Unseen-Q
All
Zero-shot MLLMS
BLIP-2
12.6
12.4
12.7
12.3
12.5
12.5
InstructBLIP
11.9
12.0
8,9
7.4
8.1
10.1
LLaVA-1.5 7B
16.0
16.9
8.3
8.9
7.8
12.4
Qwen-2.5-VI.
18.6
18.8
26.3
16.1
19.6
19.2
Qwen-2.5-VL 7B
26.6
26.3
25.3
17.2
19.9
23.1
GPT-4V
26.9
28.1
15.0
14.3
14.6
21.4
Retrieval Augereeled- Generation
+TRv+T
CLIP ViT-B/32
+ T.
12.4
RORA-VLM
CLip+Google Search
V. + T.
20.3
25.1
27.3
Wiki-LLaVA
CLIP VTT-L/14+Con.
T.
18.3
19.6
28.6
25.7
27.1
23.4
EchoSight
EVA-CLIP
T.
22.4
21.7
30.0
30.7
30.4
26.1
EchoSight
EVA-CLIP
26.4
18.0
19.8
18.8
21.9
ReflectiVA
CLIP ViT-L/14
24.9
26.7
34.5
32.9
33.7
30.2
ReflectiVA
EVA-CLIP-8B
T.
28.0
29.2
40.4
39.8
40.1
34.7
ReflectiVA
EVA-CLIP-8B
V.
35.5
35.5
28.6
28.1
28.3
31.9
Wiki-RI 3B
EVA-CLIP-SB + Col. V++
40.4
35.9
46.0
40.3
42.2
39.1
Wiki-R1 R1 7B
EVA-CLIP-SB Col. V.+ T.
41.0
37.1
47.8
42.3
44.1
40.6
perform dedicated retriever training, including DPR (1 (Lerner al.
2024
RORA-VLM @te
2024
Wiki-LLaVA (Caffagni et al. 2024). EchoSight Yan & Xie 2024).
and ReflectiVA (Cocchi
al.
2024).
Table 2: Performance comparison on the ViQuAE benchmark under zero-shot transfer setting. We compare against RC baselines (Lemer al. 2022a) and MLLM methods. Wiki-RI consistently outperforms all prior methods, even surpassing the RC semi-orach configuration.
Type
Model
Exact Match
RC Baseline
Zero-shot
20.96
18.06
Few-shot
25.43
22.07
Few-shot (semi-oracle)
44.10
40.32
Few-shot (full-oracle)
63.17
57.55
MLLM Baseline
L.L.A-v1.55
15.1
26.6
Wiki-LLaVA (E-VQA)
10.5
16.7
Wiki-LLaVA (InfoSeek)
12.7
21.8
ReflectiVA
23.2
38.1
Ours
Wiki-RI 3B
53.8
48.6
Wiki-RI 7B
55.6
50.3
4.3
IMPLEMENTATION DETAILS
Training Data. To implement reinforcement learning under the KB-VQA setting, construct balanced training set by sampling examples according to their ground-trute -tuth entities. Specifically, construct antit-hhllanced subsets by sampling 20k examples from Encyclopedic VQA (Mensink) jtt al. 2023) and 20k examples from InfoSeek (Chen et al. 2023), ensuring that each entity is propor- tionally represented within the subsets. The resulting training set contains a total of 40k examples. Notably, the scale of our training data is far smaller compared with baselines, and we provide data scale comparison in the appendix.

---

Published as a conference paper at ICLR 2026
Table 3: Results under the oracle Wikipedia entity setting. KB Article denotes providing the entire ground-truth Wikipedia article to the MLLM, while KB Passage denotes using model-specinf strategies to retrieve relevant passages within the article. The LLM column specifies the large lan- guage model backbone used in each method.
EVQA Single-hop
InfoSeek Unseen-E
Method
LLM
Unseen Q
Overall
KB Article
Vicuna-7B
42.9
14.2
13.4
13.8
LLVVA-v1.5
1LaMA-331188
54.1
20.1
17.7
18.8
KB Pasxage
Wiki-LLaVA
LLaMA-3.1-33
46.8
51.2
50.6
50.9
ReflectiVA
LLaMA-3.1-88
75.2
57.8
57.4
57.6
Wiki-RI(Our)
Qwen-2.5-3B
68.5
64.0
65.9
65.3
Wiki-RI(OUrs)
Qwen-2.5-7B
69.2
65.5
69.5
68.2
framework and implement our proposed design based on the DAPO (Yu et al.) 2025) algorithm. The reward function for reinforcement learning is defined the generated answer exactly matches the ground-trute truth answer, and 0 otherwise. The learning rate for both variants is set to 1e-6, and we set the number of rollouts for each sample to 4. For other hyperparameters, we follow the official scripts. For base models, we employ the recently released Qwen2.5-VI (Bai et al. 2025) models (3B and 7B), which represent the state-of-the-e among open-source multimodal language models. For curriculum data generation, the window size w is set to 300, the gap threshold 0.55, and the maximum gap G' is set to 6. The training takes hours for the 3B variant and 12 hours for the 7B variant on 4 A 100 GPUs.
Training Details. We adopt the widely used VERL (Volcengine 2025)
rule-based binary signal: the model receives a reward of 1 if
Retrieval System We follow previous works Yaan & Xie, 2024) that utilize EVA-CLIP 8B to compute the visual similarity score and utilize COlBERT v2 (Santhanam al. 202 l) to extract the relevant text chunks and compute the question relevance score. We use weighted sum to combine these scores. The score weight is selected based on the recall on the training set of Encyclopedic VQA and InfoSeek, respectively. More details are provided in the appendix.
4.4 PERFORMANCE ANALYSIS
Comparison with State Art. We evaluate our model on the two benchmarks described above, comparing against zero-shot multimodal LLMs (MLLMs), and strrieal--augment baselines. As shown in Table [ our proposed method with the 3B variant surpasses previous state-of oof-the-rrtt ap- proaches. Moreover, our framework consistently achieves strong performance across both bench- marks using a single retrieval system, in contrast to prior methods such as EchoSight and ReflectiVA whose performance is highly sensitive the retrieval mode. For instance, ReflectiVA (Cocchi al.) 2024) attains 35.5 on EVQA under visual retrieval, but its accuracy on InfoSeek drops to 28.3 com- pared to 40.1 with textual retrieval. These results demonstrate that our framework is not only more robust across benchmarks but also achieves superior overall performance.
Inference with Oracle Documents. To comprehensively evaluate our model, we further conduct experiments under an oracle setting, where the ground-truth entity (i.e., the Wikipedia page asso- ciated with the query) is directly provided. In this configuration, Wiki-R1 is only given retrieval results from the ground-truth entity, while the passages within the article may still contain noise. Thus, this setting can be regarded as the upper bound of our approach by eliminating entity-level retrieval noise. As shown in Table 3. Wiki-RI shows strong performance on both benchmarks, demonstrating its strong ability to effectively leverage correct retrieval results.
Generalization to ViQuAE To further assess the generalization capability of Wiki-RI, we con- duct zero-shot transfer evaluation the ViQuAE benchmark (Lemner et al. 2022b). Following the official evaluation protocol, we report F1 and Exact Match scores using the provided 1.4M Wikipedia article knowledge base. As shown in Table 2 Wiki-RI achieves 53.8 Fi and 48.6 EM with the 3B model, and 55.6 Fl and 50.3 EM with the 7B model. Our method substantially out- performs existing MILMM baselines such ReflectiVA and even surpasses the RC semi-oracle set-
8

---

Published as a conference paper at ICLR 2026
Table Ablation study of framework design on Encyclopedic VQA and InfoSeek. We conduct experiments on Qwen-2.5-VL 3B model. Each row progressively adds components, and we mark enabled modules with v. The Samp. Cur, Data Cur, Obs. Prop. indicate the sampling curriculum, data curriculum generation, and observation propagation strategies.
Modules
EVQA
InfoSeek Unseen-E
Method
Data Cur.
Samp. Cur.
Obs. Prop.
Single-hop Overall
Unseen Q
Overall
Zero-shot
18.6
18.8
26.3
16.1
19.6
SFT
21.6
25.1
38.7
24.9
29.5
SFT
34.4
32.1
DAPO
X
35.9
31.4
44.9
39.8
41.5
39.4
34.5
46.9
41.1
43.0
36.4
32.1
45.2
37.3
40.0
40.4
35,9
46.0
40.3
42.2
Congerisen of zere Aenritgg in Traiig
Accuracy over Training Iterations
Accuracy (%)
Namter
Vaniiia DAFO (EVOA)
(EVQA)
Will-R1 (itoSSook)
300
200 200
500
600
700
Training Iteration
Figure 3: Left: Number of ignored trajectories. Trajectories are ignored when they provide zero advantage and no training signal; a larger number indicates lower training efficiency, Right: Accuracy over training iterations. Performance is reported the EVQA test set and the InfoSeek validation set. The star denotes an increase in curriculum difficulty during Wiki-R1 training.
ting. These results demonstrate that Wiki-RI exhibits strong cross-dataset generalization to unseen knowledge sources and question distributions.
4.5 ABLATION STUDY
Effectiveness of Curriculum Data Generation To assess the contribution of each component in our framework, we conduct a detailed ablation study. We start from supervised fine-tuning (SFT) baseline, and then incorporate the strong reinforcement learning algorithm DAPO (Yu al. (025). Building upon DAPO, we further introduce curriculum data generation strategy, which adapts the retrieval policy construct training data from easier to challenging instances.
As shown in Table naive SFT yields only limited improvements, while DAPO, as a powerful RI. algorithm, achieves substantial gains. Our proposed data curriculum further enhances the effective- ness of DAPO, particularly on the more challenging EVQA benchmark, highlighting the importance of curriculum-guides data generation noisy retrieval settings.
Effectiveness of Curriculum Sampling We further analyze the proposed sampling strategy by introducing curriculum sampling top of data curriculum, and then augmenting with observation propagation. As shown in Table 4 directly applying curriculum sampling alone leads to degraded performance. We attribute this to the sparsity of observations: selecting the next training stage solely based on the accuracy of observed samples tends to either repeatedly select small subset of seen samples randomly sample from entirely unobserved instances. This highlights the necessity of our observation propagation module, which alleviates the sparsity issue and enables curriculum sampling function as intended. thereby improving both training efficiency and effecctiveness.
Efficiency of Observation Propagation Our proposed observation propagation module addresses the sparsity of observations by efficiently identifying samples required for constructing the sampling curriculum. This reduces the number skipped trajectories that contain no reward signal. To illus- trate this effect, compare three settings: (i) Vanilla DAPO, (ii) DAPO with curriculum sampling

---

Published as a conference paper at ICLR 2026
schedule, denoted Curriculum Sampling Only, and (iii) DAPO with curriculum sampling plus observation propagation, resulting in the Wiki-RI. As shown in Figure 3 observation propagation significantly decreases the number of skipped traeectories during training, thereby improving the efficiency of RI optimization. Moreover, by reducing wasted samples, simultaneously enhances the overall training effectiveness.
Visualization of Training Dynamics. To gain deeper insights into the behavior of our framework, we track the performance of DAPO and Wiki-R1 across training iterations. As shown Figure DAPO exhibits rapid improvement in the early stage (e.g., within the first 100 iterations), but its per- formance on EVQA degrades training progresses. We attribute this overfitting on the relatively easier InfoSeek dataset: compared to InfoSeek, EVQA involves noisier retrieval results (Table 6 which deviate further from the MLLM's pretrained distribution. contrast, Wiki-RI with curricu- lum training achieves stable improvements both benchmarks, and its best performance emerges when training reaches the highest curriculum difficulty level aivelclooelyy matching the challenges in real inference scenarios.
Similarity Measures on Label Propagation To assess the impact of the similarity measure used in label propagation, compare TF-IDF with semantically-aware text embedding model (Sen- tence Transformer) in Table 5] The results demonstrate that the proposed framework generalizes across different similarity metrics and benefits from semantically enriched representations.
Table 5: Different similarity measures for label propagation.
EVQA
InfoSeck Unseen-E
Method
Avg.
Single-hop Overall
Unseen-Q
Overall
TF-IDF
40.4
35.9
46.0
40.3
42.2
39.1
Sentence Transformer
41.1
36.3
46.2
41.3
43.0
40.7
LIMITATION
While proposed Wiki-RI effectively incentivizes the reasoning ability of MLLMs KB-VQA, it also has certain limitations. In particular, manipulating the retrieval system provides only partial means of controlling the gap between the pretraining and target distributions, rather than fully controllable data generation process. We view this as promising direction for future research, where advances in controllable data generation could enable more principled curriculum design for KB-VO BB-VQQ and related tasks.
6 CONCLUSION
In this work, we introduce Wiki-RI, lata-generattonn-asse curriculum reinforcement learning framework that incentivizes the reasoning ability of multimodal large language models on challeng- ing KB-VQA tasks. By constructing a sequence of training distributions aligned with the model's evolving capability, and combining controllable curriculum data generation with adaptive curricu- lum sampling, Wiki-R1 effectively mitigates sparse reward issues and guides the model through progressively harder examples. Extensive experiments Encyclopedic VQA and InfoSeek demon- strate significant improvements over state-of-the-ar methods, including strong generalization to un- seen questions. Our framework provides principled approach for integrating retrieval and rein- forcement learning in downstream tasks with distributional gaps, offering insights for future research on domain-adaptive adaptive reasoning in serieval-augmenter multimodal settings.
7 ACKNOWLEDGMENTS
This work was supported by NSFC 62350610269, Shanghai Frontiers Science Center Human- centered Artificial Intelligence, and MoE Key Lab of Intelligent Perception and Collaboration (ShanghaiTech University). This work was also supported by HPC platform of Shang- haiTech University.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. ArXiv, abs/2502. 139233, 2025.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In International Conference on Machine Learning, 2009.
Jannis Bulian, Christian Buck, Wojeiech Gajewski, Benjamin Boerschinger, and Tal Schuster.
Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation. In Conference Empirical Methods in Natural Language Processing, 2022.
Davide Caffagni, Federico Cocchi. Nicholas Moratelli, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. Wiki-llava: Hierarchical retrieval-augmented generation for mul- timodal llms. 2024.
Yang Chen, Hexiang Hu, Yi Luan, Haitian Sun, Soravit Changpinyo, Alan Ritter, and Ming-Wor Wei Chang. Can pre-trained vision and language models answer visual informntion-sseeinng questions? ArXiv, abs/2302.117133 2023.
Federico Cocchi, Nicholas Moratelli, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. Aug- menting multimodal Iims with self-reflective tokens for knowledge- visual question answer- ing. 2025 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 9199- 9209, 2024.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Albert Li, Pascale Fung, and Steven C. H. Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. ArXiv, abs/2305.06500, 2023.
Lianghao Deng, Yuchong Sun, Shizhe Chen, Ning Yang, Yunfeng Wang, and Ruihua Song. Muka: Multimodal knowledge augmented visual information-sseking. In International Conference on Computational Linguistics, 2025.
Yang Ding, Jing Yu, Bang Liu, Yue Hu, Mingxin Cui, and Qi Wu. Mukea: Multimodal knowledge extraction and accumulation for knowledge-based visual question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5089-5098, 2022.
Alex Graves, Marc G. Bellemare, Jacob Menick, Rémi Munos, and Koray Kavukcuoglu. Automated curriculum learning for neural networks. ArXiv, abs/1 704.03003, 2017.
Liangke Gui, Borui Wang, Qiuyuan Huang, Alexander G. Hauptmann, Yonatan Bisk, and Jianfeng Gao. Kat: A knowledge augmented transformer for vision-and-languuae. ArXiv, abs/21 12.08614, 2021.
Grant Van Horn, Elijah Cole, Sara Beery, Kimberly Wilber, Serge J. Belongie, and Oisin Mac Aodha. Benchmarking representation learning for natural world image collections. 202/ IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 12879-12888, 2021.
Hexiang Hu, Yi Luan, Yang Chen, Urvashi Khandelwal, Mandar Joshi, Kenton Lee, Kristina Toutanova, and Ming-Wei Chang. Open-domain visual entity recognition: Towards recogniz- ing millions of wikipedia entities. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 22031-1204 2023.
Ziniu Hu, Ahmet Iscen, Chen Sun, Zirui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A. Ross, and Alireza Fathi. Reveal: Retrieval-augmente visual-language pre-training with solte- multimodal knowledge memory. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 23369-23377 2022.

---

Published as a conference paper at ICLR 2026
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learn- ing. Trans. Mach. Learn. Res., 2022, 2021. http////tps//pppp..eearrrttttttt orrg/ Corpus DD:
Aman Jain, Mayank Kothyari, Vishwajeet Kumar, Preethi Jyothi, Ganesh Ramakrishnan, and Soumen Chakrabarti. Select, substitute, search: A new benchmark for knowledge-auge -uggeee visual question answering- Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.
Yuelyu Ji, Rui Meng, Zhuochun Li, and Daqing He. Curriculum guided reinforcement learning for efficient multi hop retrieval augmented generation. ArXiv, abs/2505. 17391, 2025.
Bowen Jin, Hansi Zeng. Zhenrui Yue, Dong Wang. Hamed Zamani, and Jiawei Han. Search- rl: Training Ilms to reason and leverage search engines with reinforcement learning. ArXiv, 0as/2500.095111 2025.
Niels Justesen, Ruben Rodriguez Torrado, Philip Bontrager, Ahmed Khalifa, Julian Togelius, and Sebastian Risi. Illuminating generalization in deep reinforcement learning through procedural level generation. arXiv: Learning, 2018.
Paul Lerner, Olivier Ferret, Camille Guinaudeau, Hervé Le Borgne, Romaric Besançon, José G. Moreno, and Jesús Lovón-Melgarejo. Viquae, a dataset for knowledge-based visual question answering about named entities. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022a.
Paul Lerner, Olivier Ferret, Camille Guinaudeau, Hervé Le Borgne, Romaric Besançon, José G Moreno, and Jesús Lovón Melgarejo. Viquae, dataset for knowledge- visual question answering about named entities. In Proceedings of the 45th international ACM SIGIR conference on research and development information retrieval, pp. 3108-3120, 2022b.
Paul Lerner, Olivier Ferret, and Camille Guinaudeau. Cross-modal retrieval for knowledge-bart visual question answering. In European Conference on Information Retrieval, 2024.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301. 12597, 2023.
Rich Gang Li, A. Jabri, Trevor Darrell, and Pulkit Agrawal. Towards practical multi-object manipu- lation using relational reinforcement learning. 2020 IEEE International Conference Robotics and Automation (ICRA), pp. 4051 4058, 2019.
Weizhe Lin and Bill Byrne Retrieval augmented visual question answering with outside knowledge. arXiv preprint arXiv:2210.03809 2022.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee, Improved basclines with visual instruction tuning. ArXiv, abs/2310.03744, 2023.
Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: visual question answering benchmark requiring external knowledge. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3190-3199, 2019.
Kenneth Marino, Xinlei Chen, Devi Parikh, Abhinav Gupta, and Marcus Rohrbach. Krisp: Integrat- ing implicit and symbolic knowledge for open-domain knowledge-basse basedd vqa. In Proceedings of the JEEF/CV conference on computer vision and pattern recognition, pp. 14111-14121, 2021.
Tambet Matiisnn, Avital Oliver, Taco Cohen, and John Schulman. Seacher-student curriculum learn- ing. IEEE Transactions on Neural Networks and Learning Systems, 31:3732-3740. 2017.
Thomas Mensink, Jasper R. R. Uijlings, Lluís Castrejón, Arushi Goel, Felipe Cadar, Howard Zhou, Fei Sha, Andre F. de Araújo, and Vittorio Ferrari. Encyclopedic vqa: Visual questions about de- tailed properties of fine- grained categories. 2023 IEEE/CVF International Conference on Com- puter Vision (ICCV) pp. 3090-3101, 2023.
12

---

Published as a conference paper at ICLR 2026
Sha Ning, Longtian Qiu, Yongfei Liu, and Xuming He. Hoiclip: Efficient knowledge transfer for hoi detection with lasoonlanguage models. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.
OpenAl. Vision httppp/p//..e.eetooooooooo/ooooa 2023.
Rémy Portelas, Cédric Colas, Katja Hofmann, and Pierre- Oudeyer. Teacher algorithhs for cur- riculum learning of deep rl in continuously parameterized environments. ArXiv, abs/1910.07224,
2019.
Jingyuan Qi, Zhiyang Xu, Rulin Shao, Yang Chen, dingnan jin, Yu Cheng, Qifan Wang, and Lifu Huang. Rora-vlm: Robust retrieval-augmented vision language models. ArXiv, abs/2410.08876, 2024.
Longtian Qiu, Shan Ning, and Xuming He. Mining fine- fine-grained image-text alignment for zero-shot captioning via text-only training. ArXiv, 0///2401102244/ 2024.
Longtian Qiu, Shan Ning, Jiaxuan Sun, and Xuming He. Noisygrpo: Incentivizing multimodal cot
reasoning via noise injection and bayesian estimation. arXiv preprint arXiv:2510.21122, 2025.
Longtian Qiu, Shan Ning, Chuyu Zhang, Jiaxuan Sun, and Xuming He. Da-dpo: Cost-efficient difficulty-aware preference optimization for reducing milm hallucinations. arXiv preprint arXiv:2601.00623. 2026.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PMLR, 2021.
Keshav Santhanam, O. Khattab, Jon Saad-Falcon, Christopher Potts, and Matei A. Zaharia. Col- bertv2: Effective and efficient retrieval via lightweight late interaction. In North American Chap- ter of the Association for Computational Linguistics, 2021.
Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi A-okvqa: A benchmark for visual question answering using world knowledge. In European Conference on Computer Vision, 2022.
Sanket Shah, Anand Mishra, Naganand Yadati, and Partha Pratim Talukdar. Kvqa: Knowledge- aware visual question amswering. In AAAI Conference on Artificial Intelligence, 2019.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Jun-Mei Song, Mingchuan Zhang, Y. K. Li, Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. ArXiv, abs/2402.03300. 2024.
Taiwei Shi, Yiyang Wu, Linxin Song, Tianyi Zhou, and Jieyu Zhao. Efficient reinforcement finetun- ing via adaptive curriculum learning. ArXiv, 0bb/22544.05500, 2025a.
Taiwei Shi, Yiyang Wu, Linxin Song, Tianyi Zhou, and Jicyu Zhao. Efficient reinforcement finetun- ing via adaptive curriculum learning. ArXiv, abs/2504.05520, 2025b.
Krishna Srinivasan, Karthik Raman, Jiecao Chen, Michael Bendersky, and Marc Najork. Wit: Wikipedia-based image text dataset for multimodal multilingual machine learning. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.
Quan Sun, Yuxin Fang, Ledell Yu Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. ArXiv, abs/2303.113381 2023.
Volcengine. verl: Volcano engine reinforcement learning for llms. httpp:://iithb. com/ vvolcengine/ver] 2025. Accessed: 2025-09-22.
Denny Vrandečić and Markus Krötzsch. Wikidata: A free collaborative knowledgebase. Commu- nications of the ACM, pp. 88, Sep 2014. doi: 10.1145/262948.. URL http: p doi.
arg/10.1145/22
13

---

Published as a conference paper at ICLR 2026
Peng Wang, Qi Wu, Chunhua Shen, Anthony R. Dick, and Anton van den Hengel. Fvqa: Fact-based visual question answering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40: 2413-2427, 2016.
Rui Wang, Joel Lehman, Jeff Clune, and Kenneth O. Stanley. Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions. ArXiv, abs/1901.01755 00770111111.....1000010011 2019.
Shaohan Wang. L. Zhang, Zheren Fu, and Zhendong Mao. Cl-rag: Bridging the gap retrieval- augmented generation with curriculum learning. ArXiv, abs/2505.1049) 2025a.
Zhenting Wang. Guofeng Cui, Kun Wan, and Wentian Zhao. Dump: Automated distribution-level
curriculum learning for rl-based IIm post-training. ArXiv, abs/2504.09710, 2025b.
Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. Uniir Training and benchmarking universal multimodal information retrievers. ArXiv, abs/2311.17136, 2023.
Tobias Weyand, Andre F. de Araújo, Bingyi Cao, and Jack Sim. Google landmarks dataset v2 - large-scale benchmark for instance-level recognition and retrieval. 2020 IEFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 2572-2581, 2020.
Jialin Wu, Jiasen Lu, Ashish Sabharwal, and Roozbeh Mottaghi. Multi-modal answer validation for knowledge- baaed vqa. In Proceedings of the AAAI conference on artificial intelligence, vol- ume 36, pp. 2712-2721, 2022.
Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, and Ziwei Liu. Mmsearch- rl: Incentivizing Imms to search. ArXiv, abs/2506.20670, 2025.
Alexandros Xenos, Themos Stafylakis, Ioannis Patras, and Georgios Tzimiropoulos. A simple base- line for knowledge-based visual question answering. arXiv preprint 1xXi::2310.13570, 2023.
Zilin Xiao, Ming Gong, Paola Cascante- Xingyao Zhang, Jie Wu, and Vicente Ordonez. Grounding language models for visual entity recognition. In European Conference on Computer Vision, 2024.
Ning Xu, Yifci Gao, An-An Liu, Hongshuo Tian, and Yongdong Zhang. Multi-modal validation and domain interaction learning for knowledge-basee visual question answering. JEEE Transactions on Knowledge and Data Engineering. 36:6628-666 -6448, 2024.
Yibin Yan and Weidi Xie. Echosight: Advancing visual-language models with wiki knowledge. ArXiv, abs/2407.11735 2024.
Wei Yang, Jingjing Fu, Rui Wang, Jinyu Wang, Lei Song, and Jiang Bian. Omgm: Orchestrate multiple granularities and modalities for efficient multimodal retrieval. In Annual Meeting of the Association for Computational Linguistics, 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Honglin Yu. Weinan Dai, Yuxuan Song, Xiang Wei, Haodong Zhou, Jingjing Liu, Wei Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yong-Xu Xu Wu, and Mingxuan Wang. Dapo: An open-source Ilm reinforcement learning system at scale. ArXiv, abs/2503. 14476, 2025.
Ruifeng Yuan, Chenghao Xiao, Sicong Leng, Jianyu Wang, Long Li, Weiwen Xu, Hou Pong Chan, Deli Zhao, Tingyang Xu, Zhongyu Wei, Hao Zhang, and Yu Rong. Vl-cogito: Progressive curricu- lum reinforcement learning for advanced multimodal reasoning. ArXiv, abs/2507.22607, 2025.
Chenliang Zhang, Lin Wang, Yuanyuan Lu, Yusheng Qi, Kexin Wang, Peixu Hou, and Wenshi Chen. curriculum learning approach to reinforcement learning: Leveraging rag for multimodal question answering. ArXiv, ab332508.10337, 2025.
14

---

Published as a conference paper at ICLR 2026
Table 6: Retrieval results on EVQA test and InfoSeek validation sets. We report Recall @ for
K {1, 5, 10, 20}. The CLIP 1-I is the retrieval with the visual similarity score from EVQA-CLIP 8B only.
Methods
Retrieval Mode
EVQA
Test
InfoSeek Val
R @@
R@5
R@ 10
@@20
R@D
R@5
R@
@@20
CLIP I-I
Visual
11.0
26.2
33.8
41.0
45.7
65.7
71.6
76.2
ReflectiVA
Textual
10.1
20.5
29.4
56.1
77.6
86.4
ReflectiVA
Visual
15.6
36.1
49.8
29.6
41.4
46.6
Wiki-RI
Visual+Textus
16.7
41.0
44.8
47.5
46.9
67.1
72.9
77.2
A APPENDIX
A.1 DETAILS OF RETRIEVAL SYSTEM
In this section, we provide detailed design of our retrieval system, which consists of two main modules: visual-based retrieval module and texfual-based retrieval module. To combine the outputs of these two modules, we employ score fusion strategy that weights and merges the visual and textual retrieval scores produce final ranking of candidate knowledge snippets for each query. The performance is shown in Table 6
Visual-based Retrieval We first perform coarse-level retrieval using a visual-based approach. Following previous work (Yan & Xie 2024 Cocchi et al. 20024) we employ EVA-CLIP 8B (Sun ) to extract global visual features from the query image and the images in the knowledge base B. The similarity between the query and candidate images is then computed using cosine similarity of their corresponding feature vectors. This provides an initial ranking of candidate knowledge items based on visual relevance.
Textual-based Retrieval In the textual-based retrieval stage, we aim achieve two objectives: (i) extract query-relevant textual passages from each knowledge base article, and (ii) assess the rel- evance the article to the query using text retrieval model. Specifically, we employ the ColBERT V2 (Santhanam et al. 2021) model and split each article into chunks of size 256. The relevance score of an article to a given query q is determined by the highest relevance score among its re- trieved passages.
Retrieval Score Fusing After obtaining the visual similarity score and textual relevance score T for each knowledge base article, we fuse the two scores using weighted sum:
s, X V (1-
(6)
where À E [0, 1] is tunable hyperparameter controlling the relative importance of visual and textual cues. We select À based on the training set: À 0.985 for EVQA and À 0.997 for InfoSeek. The values are close to 1 because V is normalized to V [0, while T is unnormalized and can take values E [0, +00).
A.2 PSEUDO CODE FOR WIKI-RI
To provide clearer overview of the training process, we present the pseudo code of Wiki-RI in Algorithm
A.3 TRAINING DATA SCALE COMPARISON
In this section, we provide a comparison the training data scale between our proposed framework and baseline methods. As shown in T method requires substantially fewer training samples while achieving superior performance. This highlights the efficiency of Wiki-R and demonstrates its applicability in scenarios with limited computational data resources.
15

---

Published as a conference paper at ICLR 2026
Algorithm Wiki-RI Data and Sampling Curriculum Reinforcement Learning
Require: KB-VQA Training dataset D, knowledge base B, policy model Ra, gap threshold re-
ward function r( Propagation Graph K.
1: Select RL algorithm A (e.g.. PPO,GRPO.DAPO)
2: Initialize retrieval function By where =
3: Initialize estimated sample reward H. < {0]
4: Initialize sliding window reward W
5: while training is not finished do
6: Select sample with target difficulty (4, "")"") from D according H
Curriculum
sampling with predicted sample reward
7:
S Retriever (g _", B: 0g)
Controllable Data Generation
8:
Construct Batch Data X x- (g. 19
9:
Generate responses G n0 o X)
10:
Compute reward: Rt 4- r(X, G)
11:
Update Policy + A(Ta, X, G,
12:
Maintain sliding window W 4- (WU
>> Only preserve last elements
13:
if
W Σw,ew WA 2 r and g < then
Upgrade retrieval modification function
14:
- 9 9+++
15:
Reset W
16:
end if
H < Propagate KC)
Difficulty estimation via observation propagation
for each index with H|] do
19: H|i 4 H[i + H|]
20:
end for end while
21:
Table 7: Comparison of training data scale and performance across different methods.
Method
FT Retrieval
FT Generation
EVQA
InfoSeek
Wiki-LLaVA
916.385
902,509
Echosight
X
916,385
902,509
ReflectiVA VA
X
V
2,900,000
2,500,000
Wiki-RI
20,000
20,000
A.4 DETAILS OF OBSERVATION PROPAGATION
In this section, we provide a detailed design of the observation propagation mechanism used in curriculum sampling. The goal of this component ing samples by propagating the limited reward signals observed during RL training. By leveraging correlations among VQA samples that share the same knowledge base article, we can predict the expected reward for unobserved samples, enabling more effective curriculum-bast difficulty esti- mation and sample selection.
estimate the difficulty unobserved train-
To implement observation propagation, we first model the correlations be- tween VQA samples label propagation graph K. Specifically, the correlation between samples is derived from the associated ground-truth knowledge base articles. To quantify the relatedness between different articles, we adopt simple rule-based textual similarity approach using TF-IDF To reduce noise from weakly related articles, we retain only the top 100 edges for each node in K, ensuring that the propagation graph focuses on the most relevant article connections.
Graph Construction
Label Propagation After constructing the label propagation graph, we apply a non paramertttticc label propagation algorithm propagate observed reward signals to unobserved samples (Algo- rithm 2). This yields estimated rewards for all training samples, enabling effective curriculum sam- pling even under sparse observations.
16

---

Published as a conference paper at ICLR 2026
Algorithm 2 Non-Parametric Label Propagation
Require:
Label propagation graph K, observed reward vector A, smoothing factor max itera- tions T. convergence criterion e
1: Normalize each row of K so that E, Kij =
2: Initialize propagated reward A pred f- A
3: for = to do
4: Anew f ak Apred + (1 a)
5: if | - Aprell < c then break
6:
end if
7: A Apred <- new
8:
end for
9: return pred
Algorithm Details balancing the contribution between propagated information and initial ob- served rewards. and the maximum number of iterations is set to 7 10, which found sufficient for convergence in practice. The convergence criterion defined as 10-1 These parameters are used consistently across all experiments.
A.5 HYPERPARAMETER SENSITIVITY ANALYSIS
To assess the robustness of our method, we conduct sensitivity analysis on two key hyperparame- ters: the curriculum gap threshold and the observation-propaaation smoothing factor a.
Curriculum threshold 7 The sensitivity analysis for the curriculum gap threshold is conducted within empirically determined interval that supports meaningful currieulum progression under our KB-VQA training setup (20k InfoSeek + 20k EVQA). In preliminary diagnostics, we observe that 5 values below this interval (e.g., T 0.5) cause the model escalate through difficulty levels too rapidly, reaching the maximum level around step 238, whereas values above it (e.g., = 0.6) lead to stagnation, with only single upgrade occurring step 327, These behaviors indicate that E (0.5, 0.6) forms the region in which the curriculum operates as intended, and we therefore perform the sensitivity study within this range. shown in the left part of Figure performance remains largely stable, suggesting that the method is robust to variations of 7 within its effective operating regime.
Smoothing factor For the smoothing factor a, the sensitivity analysis is performed around the commonly adopted default configuration used in standard aaoo-ppropngation implementations. Specifically, examine the default choice. We evaluate the robustness of the method within this local neighborhood. As presented in the right part of Figure 4 the model maintains comparable final accuracy across all tested a values, indicating that the method is insensitive to moderate variations of a around its typical operating region.
[0.7,0.9], 0.9], which forms meaningful local neighborhood around
Across both hyperparameters, the model converges to similar final accuracy within the explored intervals. Although different settings may slightly affect the rate of convergence, the eventual per- formance remains largely stable, suggesting that the approach is robust to hyperparameter variations and does not rely on extensive hyperparameter optimization.
A.6 TRAINING COST COMPARISON
We compare the training cost of our method against representative baselines in Table S The re- sults show that Wiki-R1 achieves competitive lower total training costs compared to existing approaches.
Given that both Wiki-LLaVA and ReflectiVA derived from the LLaVA-115 architecture and did not report their training times, we estimated their training costs based on LLaVA-1.5, using the formula: Training Time x Baseline Time (Data Ratio / Batch Size Ratio) x Model Size Factor.
17

---

Published as a conference paper at ICLR 2026
Threchold Infosark
Gap Threshald Infedsel
Smmothing Fecter EVGA
Smoothhig Factor InfeSeok
Training Beration
Training Iteration
Training Iteration
Traiing Iteratioon
Figure 4: Comparison across gap thresholds and smoothing factors. We report EVQA test and InfoSeek validation performance across training iterations under different hyperparameter settings. For the left two figures, the star denotes an increase in curriculum difficulty during Wiki-R1 training. The chosen hyperparameter is 0.55 and the is 0.8.
Table 8: Training cost comparison across different methods. FT Retrieval indicates whether the retriever is fine-tuned; FT Generation indicates whether the method fine-tunes the generation model. Training time is measured in A100 GPU hours.
Method FT Retrieval
FT Generation
#Training Samples InfoSeek
Training Time
EVQA
Wiki-LLaVA
916,385
902,509
Echosight
X
916,385
902,509
40
ReflectiVA
X
2,900,000
2,500,000
~1,688
Wiki-RI (3B)
X
20,000
20,000
36
Wiki-RI (7B)
X
20,000
20,000
48
B EXPERIMENTAL STABILITY AND RELIABILITY OF RESULTS
To assess the reliability of our reported results, we conducted each experiment three times with independent runs under the same training settings. For each run, we recorded the performance on EVQA and InfoSeck multiple training iterations. Figure 5 shows the performance curves for all three runs. We observe that while the convergence speed varies slightly across runs, the final performance levels after convergence highly consistent. This indicates that method is stable and the reported improvements are robust to random initialization and training stochasticity.
C THE USE OF LARGE LANGUAGE MODELS
In this work, large language models (LLMs) were used solely as an assistive tool for refining the writing of text authored by the researchers. Specifically, LLMs were employed improve the read- ability, clarity, and conciseness of sentences drafted by the authors. All research ideas, experimental designs, analyses, and scientific claims were conceived and developed by the authors without the involvement of LLMs.
18

---

Published as a conference paper at ICLR 2026
EVQA Different Seeds
InfoSeek Different Seeds
36
Seed Seed Seed
Seed
Seed
Accuracy (%)
Accuracy (%)
20
30
26
600
Training Iteration
Training Iteration
Figure 5: Performance over training iterations for three independent runs on EVQA and InfoSeek.
19


Reviews:
[{"review_id": "eOgOXoY5Np", "reviewer": "Reviewer_kWfL", "summary": "This paper addresses the task of enhancing multimodal large language models (MLLMs) for Knowledge-Based Visual Question Answering (KB-VQA), where models need to integrate visual and textual information to answer questions. The authors propose Wiki-R1, a reinforcement learning framework with two main contributions. First, Curriculum Data Generation manipulates the retriever to create training samples at different difficulty levels, enabling the model to learn progressively from easy (accurate retrieval) to hard (noisy retrieval) examples. Second, Curriculum Sampling with Observation Propagation implements a sampling strategy that prioritizes the most informative samples (those with approximately 50% accuracy) and propagates difficulty estimates to unseen samples. These components work together with RL optimization to provide denser reward signals and improve reasoning capabilities in KB-VQA scenarios. The method is evaluated on two KB-VQA benchmarks.", "questions": "What exact data configuration was used for the SFT baseline in Table 3? Does it use the same entity-balanced samples as the RL experiments?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper clearly identifies why RL fails in KB-VQA (retrieval noise leads to sparse rewards) and proposes a reasonable data-centric solution to increase reward density and improve downstream reasoning performance.\n- The core idea is elegant: generating progressive difficulty data through controllable parameters and selecting the most valuable samples using accuracy-based Gaussian sampling to balance between \"already learned\" and \"not yet learned\" examples. I particularly appreciate the Observation Propagation mechanism for finding similar samples to stabilize training.\n- The method shows consistent performance gains and potential better data efficiency compared to baselines.", "weaknesses": "- Some key parameters are not specified, including the exact definition of the reward function in the RL objective, and the observation propagation parameters.\n- The reliance on TF-IDF similarity may restrict the method to surface-level lexical matching, potentially missing semantically similar samples that use different vocabulary.\n- The paper lacks details needed to assess the true benefit of RL. Specifically: What retrieval configuration was used for the SFT baseline? Does it use the same samples as the RL experiments? Was the SFT training data also curriculum-generated? Without a fair comparison where both SFT and RL use identical curriculum data, it's difficult to isolate whether the gains come from the curriculum design itself or from the RL optimization.\n- Results are reported as single values without standard deviations or significance tests, making it difficult to assess the reliability of the comparisons.\n- Table 5's comparison may be misleading because it only lists training sample counts without reporting actual training time, GPU costs, or convergence speed. While Wiki-R1 achieves better results with less data, it employs computationally intensive RL training (multiple rollouts per sample, dynamic sampling, observation propagation), whereas baselines use simpler SFT. The paper reports Wiki-R1's training time but provides no training time data for baselines, making it hard to determine which method has lower total computational cost.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dDOOzxYOBw", "reviewer": "Reviewer_Wc3T", "summary": "This paper identifies the distribution gap with KB-VQA data which leads to sparse rewards in RL approaches and proposes Wiki-R1, a curriculum RL framework with two core components: Controllable Curriculum Data Generation and Curriculum Sampling with Observation Propagation. Experiments on Encyclopedic VQA (EVQA) and InfoSeek benchmarks show state-of-the-art (SOTA) results.", "questions": "See Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Targeted Problem: Addresses a critical pain point of distribution gap and sparse rewards in RL-based KB-VQA.\n2. Generalization: Excels on unseen question splits e.g., InfoSeek Unseen-Q:47.8% vs. prior 40.4, indicating robust reasoning.\n3. Component Validity: Ablations clearly show that data curriculum improves DAPO performance, and propagation is necessary for sampling curriculum to work).", "weaknesses": "1. Limited Retrieval Control: The data generation relies on adjusting retrieval noise (number of candidates, ground-truth inclusion) but does not fully control the type of noise (e.g., irrelevant vs. slightly relevant candidates).\n2. Hyperparameter Transparency: No sensitivity analysis for key hyperparameters (e.g., curriculum gap threshold τ, observation propagation smoothing factor α).\n3. RL Algorithm Scope: Only uses DAPO as the base RL algorithm—no comparison with other RL methods (e.g., PPO, GRPO) to validate generalizability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YkrpCr9tdE", "reviewer": "Reviewer_zx7D", "summary": "This paper proposes Wiki-R1, a data-generation-based curriculum reinforcement learning framework designed to enhance multimodal large language models (MLLMs) on Knowledge-Based Visual Question Answering (KB-VQA) tasks.\nThe key idea is to systematically bridge the distribution gap between pretraining data and the KB-VQA domain by Controllable Curriculum Data Generation and Curriculum Sampling with Observation Propagation.\nThrough these mechanisms, the model learns progressively harder reasoning examples, mitigating sparse rewards during RL fine-tuning.\nExperiments on Encyclopedic-VQA and InfoSeek show new state-of-the-art performance with strong generalization to unseen questions.", "questions": "The paper defines an upgrade threshold τ = 0.55 for moving to the next difficulty level. How sensitive is performance to this threshold? Would too-rapid or too-slow progression harm training stability?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The framework introduces an elegant combination of data-level and sampling-level curricula. The idea of controlling retrieval difficulty rather than merely selecting data is innovative and well-motivated by the sparse reward challenge in KB-VQA.\n2. The approach yields notable accuracy gains with only ~40k training samples — far less than prior methods requiring millions — highlighting efficiency and scalability.", "weaknesses": "1. While the combination of controllable retrieval and curriculum sampling is well-engineered, the theoretical novelty may be seen as incremental over prior curriculum RL works. The core mechanism (progressively harder data and adaptive sampling) is conceptually similar.\n2. The performance gains are mostly demonstrated on EVQA and InfoSeek. It’s unclear whether the framework generalizes to other KB-VQA settings (e.g., OK-VQA) or to different retrieval model architectures.\n3. The literature review seems to focus on works on EVQA/InfoSeek. A large collection of works in other KB-VQA datasets (e.g. OK-VQA) are missed in the discussion.\n4. The proposed model outperforms many existing works, however, it remains unclear to me whether the gain is from RL training since many existing works are training-free. The authors should make it clear in the table (whether the approach is training-free) and compare the approach with training methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8ZuxnSti2t", "reviewer": "Reviewer_YGJ1", "summary": "The paper proposes Wiki-R1, a curriculum reinforcement learning framework based on data generation, which specifically includes three methods: (1) controlling the difficulty of samples during training, and (2) a novel sampling strategy that selects samples likely to yield significant advantages in reinforcement learning updates based on their reward values.", "questions": "See Weakness", "limitations": "", "rating": 2, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "1) The paper’s motivation is very clear.\n2) The main figure in the paper is drawn clearly.", "weaknesses": "1) The paper provides limited explanation of the proposed method, making it somewhat unclear.\n2) In Table 2, Qwen2.5 is used, while in Table 3, Qwen2.5-VL is employed; however, the main table does not explicitly specify which LLM is used, leading to some confusion regarding the model configurations.\n3) In Table 3, the inclusion of the sampling curriculum and observation propagation strategies yields inferior performance compared to using data curriculum generation alone, which undermines the effectiveness of the sampling curriculum and observation propagation strategies.\n4) Table 3 does not clarify whether the SFT (Supervised Fine-Tuning) data includes ground-truth (GT) annotations, raising the possibility of an unfair comparison.\n5) The method needs to be evaluated on more datasets and base models to better validate its effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)