RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generation
Authors: Pengcheng Jiang
Lang Cao
Ruike Zhu
Minhao Jiang
Yunyi Zhang
Jiaming Shen
Jimeng Sun
Jiawei Han
Venue: ICLR 2026 Poster (2026)
Keywords: RAG
Large Language Models
Question Answering
Knowledge Graphs
Graph LLM
Primary area: foundation or frontier models, including LLMs

Abstract: Large language models (LLMs) have achieved impressive performance on knowledge-intensive tasks, yet they often struggle with multi-step reasoning due to the unstructured nature of retrieved context. While retrieval-augmented generation (RAG) methods provide external information, the lack of explicit organization among retrieved passages limits their effectiveness, leading to brittle reasoning pathways. Recent interpretability studies highlighting the importance of structured intermediate reasoning further align with this perspective. 
We propose Retrieval-And-Structuring (RAS), a framework that dynamically constructs question-specific knowledge graphs through iterative retrieval and structured knowledge building. RAS interleaves targeted retrieval planning with incremental graph construction, enabling models to assemble and reason over evolving knowledge structures tailored to each query. On seven knowledge-intensive benchmarks, RAS consistently outperforms strong baselines, achieving up to 8.7\% and 7.0\% gains with proprietary and open-source LLMs, respectively. Our results demonstrate that dynamic, question-specific knowledge structuring offers a robust path to improving reasoning accuracy and robustness in language model generation.

Published as a conference paper at ICLR 2026
RAS: KNOWLEDGE-INTENSIV LLM GENERATION
FOR
Pengcheng Jiang
Lang Cao Jimeng Sun
Ruike Zhu
Minhao Jiang
Yunyi Zhang
Jiaming Shen 9
Jiawei Han
University of Illinois Urbana-Champaign
Google DeepMind
ABSTRACT
Large language models (LLMs) have achieved impressive performance on knowledge-intensive tasks, yet they often struggle with multi-step reasoning due to the unstructured nature of retrieved context. While retrivaal-augmeette (RAG) methods provide external information, the lack of explicit organization among retrieved passages limits their effectiveness, leading to brittle reasoning pathways. Recent interpretability studies highlighting the importance of structured intermediate reasoning further align with this perspective. We propose Retrieval- And-Structuring (RAS), framework that dynamically constructs question-specific knowledge graphs through iterative retrieval and structured knowledge building. RAS interleaves targeted retrieval planning with incremental graph construction, enabling models to assemble and reason over evolving knowledge structures tai- lored each query. On seven knowledge-intensivy benchmarks, RAS consistently outperforms strong baselines, achieving up to 8.7% and 7.0% gains with propri- etary and open-source L.Mss, respectively. Our results demonstrate that dynamic, question-specifee knowledge structuring offers robust path to improving reasoning accuracy and robustness in language model generation.
generation
INTRODUCTION
Complex reasoning tasks such as scientific analysis or multi-hop question answering demand both comprehensive knowledge and structured logical thinking ( (Yang et al., 2018). While large language models (LLMs) have achieved remarkable performance across a wide range of natural language processing tasks (Devlin et al., 2018; Brown et al., 2020), they often struggle with knowledge- intensive reasoning due to the absence of precise, logically organized information (Rae et al., 2021; Ling et al., 2024). This limitation has motivated growing research into augmenting LLMs with structured knowledge to enhance their reasoning capabilities (Wang et al., 2021).
Retrieval-augmeente generation (RAG) approaches provide L.MSs with additional context from retrieved passages (Guu et al., 2020; Lewis et al., 2020; Izacard & Grave, 2021; He et al., 2024), but often face hallucination challenges (Maynez al., 2020; Zhang al., 2023b), where generated content deviates from retrieved information. This issue stems from the unstructured nature of passages, which forces the model implicitly bridge logical gaps. Briefly, interpretability analyses have suggested that LLMs attempt to chain facts across context internally, and failures in these implicit reasoning chains correlate with hallucinations (Lindsey al., 2025). These findings reinforce the need for explicitly structured intermediate knowledge to guide reasoning.
Recent efforts have integrated knowledge graphs (KGs) with LLMS (Sun et al., 2019; Yu et al., 2022; He et al., 2024; Edge et al., 2024), providing compact relational representations that support more interpretable reasoning (Hogan et al., 2021; Jiang et al., 2024: Sun et al., 2023). However, existing approaches typically rely static, corpus-wide graphs. This design introduces two limitations. First, global KGs are costly to build and maintain: indexing a corpus like Wikipedia 2018 can require millions of LLM calls and cost tens of thousands to millions of USD (see Appendix G). Second, global graphs often blend evidence from many documents, leading to ambiguous or even contradictory relations. For example, global KG might simultaneously encode that Geoffrey Hinton is linked

---

Published as a conference paper at ICLR 2026
to deep learning, to cognitive neuroscience, and to critiques of large which aspect is relevant to user query's 's focus. Similarly, biomedical KGs may contain both positive and negative associations between drug and disease, reflecting conflicting studies. In contrast, question-specific KG built from targeted documents resolves these conflicts by grounding relations in coherent, query-relevant context.
clarifying
These limitations highlight the need for knowledge graphs that are constructed on demand, tailored to the query, and structured to support reasoning. To this end, we propose Setrivval-And-Strrcturrii (RAS), a framework that dynamically constructs and reasons over question-specific knowledge graphs through iterative retrieval and structured knowledge building. The RAS process unfolds in three steps: (1) planning step that identifies knowledge gaps and generates targeted sub-queries, (2) a sarrieval-and-struuttuiir step that extracts factual triples from retrieved passages and incrementally answering step that produces
builds a -epestio--scccciicccc graph, and (3) a final outputs conditioned on the accumulated structured knowledge.
RAS addresses several limitations of prior methods. Unlike traditional RAG, which performs single- pass retrieval (Guu et al., 2020; Lewis et al., 2020; Izacard & Grave, 2021), RAS iteratively plans and fills knowledge gaps at inference. In contrast to static KG-based approaches (He et al., 2024; Edge et al., 2024), RAS dynamically constructs capturing only task-relevant information. This design avoids both the inefficiency of offline indexing and the noise of global graphs, enabling precise and robust reasoning.
graphs tailored to each question,
Through extensive evaluations across seven benchmarks spanning open-domain QA, closed-set QA, and long-form generation, RAS consistently outperforms strong baselines by 7.0% with open-source LLMs and 8.7% with proprietary models. Our main contributions are:
We propose RAS, a framework that dynamically builds question-specific knowledge graphs through iterative retrieval and structuring.
We design a unified graph structure-awand aware model that jointly plans retrieval and generates answers over evolving knowledge graphs.
We show consistent gains across seven benchmarks, with up to 8.7% improvement over strong RAG baselines, while maintaining efficiency and scalability.
2 RELATED WORK
Retrieval-Augmennte Generation (RAG). RAG enhances language model performance on knowledge-intensing tasks by incorporating retrieved passages into the model input (Guu al., 2020; Lewis et al., 2020), improving factual accuracy and grounding. Early approaches retrieved a fixed number of passages once before generation (Shao et al., 2023; Es al., 2024; Lyu et al., 2024a), while later methods explored adaptive retrieval (Jiang et al., 2023) or retrieval evaluation (Kim et al., 2024b) to improve relevance. Iterative retrieval-generation approaches (Shao et al., 2023; Guan et al., 2024) and targeted subquery strategies (Khattab al., 2023; Yao et al., 2023; Press et al., 2022; Trivedi et al., 2023) progressively enrich the evidence. Self-RAG (Asai et al., 2023) introduced seff-reflective retrieval, and RPG (Lyu al., 2024b) extracted fine-grained paragraphs. More recent work (Jiang et al., 2025c; Mei et al., 2025) further improves RAG by reinforcement learning over search behaviors. Despite these advances, retrieved context often contains redundancy or misses critical facts. Our work departs from these by converting retrieved content into a structured, evolving graph aligned with the query.
Graph as Context for LLMs. Graphs offer explicit, relational structures that help models go beyond flat text by making multi-hop relationships more tractable (Yasunaga et al., 2021; 2022; Yu, 2022: Ju et al., 2022; Zhu et al., 2024; Gutiérrez et al., 2024). GraphToken (Perozzi et al., 2024) shows LLMs can process serialized graphs directly (Liu al., 2021). G-Retriever (He et al., 2024) leverages global KGs for entity-centric subgraph retrieval, while GraphRAA-styy methods (Edge et al., 2024; Jiang et al., 2025b) construct large corpus-level graphs with community summarization. These methods rely static graphs, which are costly to construct (Appendix G) and often introduce irrelevant noise. By contrast, RAS builds question--spectte offline costs and providing denser, tesk-relevant context tailored to each reasoning trajectory. This design aligns with findings that many LLM errors stem from failed implicit reasoning chains (Lindsey al., 2025), which explicit, query-focused structuring can mitigate.
knowledge graphs dynamically, eliminating prohibitive
2

---

Published as a conference paper at ICLR 2026
RAS Framework
Plan
[NO_RETRIEVAL]
Question
M
[SUFFICIENT]
19 Bein
M
Answer A
Subquery [SUBQ]
Retrieval
[Hu:8a...]
Merge
Structuring
Triples gt
Text t
Question-Specifi KG Go
Figure 1: Overview of the Siniecaa--And-SStuuttrirnnn (RAS) framework. RAS operates through three stages: (1) Planning ($3.1): the model strategically determines retrieval needs and generates focused sub-queries based the current knowledge state; (2) Text Retrieval and Structuring ($3.2): the system retrieves passages based on sub-queries, extracts factual triples, and merges them into evolving question- expands iteratively with reasoning needs; and (3) Answering ($3.3): the accumulated structured knowledge leveraged to generate the final output. We provide step-by-step running example in Figure 25.
knowledge graph that
BEFRIEVAL-NDD-STRUCTURNN (RAS) FRAMEWORK
Effective knowledge-intensivy language generation requires not only retrieving relevant information, but also structuring and reasoning over it systematically. We introduce Saarivval-And-Struuttrrr (RAS), framework that interleaves iterative retrieval planning with dynamic question-speciie knowledge graph construction, enabling large language models (LLMs) to reason over progressively organized knowledge tailored each query. Figure illustrates the overall workflow.
Key Definitions. We define the core concepts used in the RAS framework as follows. The Main question (Q) denotes the original task input. A subquery (q:) is focused retrieval query generated at iteration i to obtain supporting evidence. At the first iteration, = Q. Retrieved text (t,) set of the top-k documents retrieved by q. A text-to-triple model fiz converts retrieved text into triples (g:), structured as subject-predicate- predicate-object facts. These triples are incrementally accumulated into an evolving question- question-specific knowledge graph (Gq). representing organized evidence related to Q. The model M produces an plan (pc) at each step, determining whether to continue retrieval ( ([SUBQ) ), stop retrieval ([SUFFICTENT)) ) or, initially, answer directly without retrieval (INN__ETTREVAL) ]).
3.1 KNOWLEDGE-AWARE PLANNING
The planning step initiates and controls the rerrieval-nnd-sttuuuurnng process by dynamically assessing the current knowledge state.
Initial Planning. Formally, given an input query Q, the model M generates an initial plan po:
Po + M(0; INSTp1an: 0; Q)
where INSTpian is the planning instruction (as shown in Figure 15). Po can take one of two forms:
[SUBQ] 40 Q: If . assesses that the query cannot be satisfactorily answered with its own knowledge, we start the iteration with the main question Q as the initial subquery, and move to the next stage ($3.2).
[NO RETRIEVAL]: If M determines that Q can be answered directly without requiring any additional knowledge, the planning process terminates, and the framework proceeds directly to the final Answering stage ($3.3).

---

Published as a conference paper at ICLR 2026
Iterative Planning. At iteration i 0, given the accumulated knowledge G; and the subquery- history Igu. 9o...... dis gil, the model updates the plan:
Pi+1 + MMGNN((..): INSTP1ani (Vo. .o....... Pi gil; Q)
(2)
where GNN is a graph neural network for encoding and projecting the evolving KG G Oh is the subquery at iteration k, and Ok is the extracted graph information (a list of triples) from the retrieved context fx.
The output Pi+1 at each iteration can be either:
[SUBQ] @++1: The model generates a new subquery 4++1 to guide the retrieval of additional relevant knowledge. The subquery is designed to fill specific gaps in the current knowledge state with respect to answering Q. The framework proceeds to the next stage ($3.2).
[SUFFICIENT]] The accumulated knowledge G2 deemed sufficient to comprehensively address the main question Q. The iterative retrieval process terminates, and the framework proceeds the Answering stage ($3.3).
Planning serves as a key driver of the RAS framework's 's iterative retrieval and refinement process. By dynamically assessing the adequacy of the retrieved knowledge and generating targeted sub-queries, it enables the efficient acquisition of query-relevant information.
3.2 TEXT RETRIEVAL AND STRUCTURING
Once [SUBQ] is detected, we use the subquery g: to retrieve the text 4; and transform it into structured knowledge ds, which is progressively merged to the question-specifer graph Gq-
Text Retrieval. We use text retriever to retrieve the top-k semantically relevant passages 4, from the corpus C' for each subquery q;:
4 - Retrieval (qi, C,k) k)
We use standard dense retriever by default but note that RAS is compatible with more advanced information retrieval methods (Chaudhary et al., 2023; Kang et al., 2024; Jiang et al., 2025a).
Text-to- Text-to-Triples Conversion. To extract essential factual information from the retrieved passages 4, we employ a text-to-triples model f12e- This model is trained on the full WikiOfGraph dataset (Kim et al., 2024a), which is high-quality, LLM-curatee text-to- to-triples corpus. Details of the training process are provided in Appendix D The model generates structured triples in the following format:
gi < feze (t)) = [[s0, Fo, o0): (3)g.) "19.1> 919.))]
(4)
where each triple (sj. structured representation enables efficient downstream reasoning and facilitates integration with external knowledge graphs. Although feze is a lightweight LLM capable of fast inference using techniques such as quantization (Dettmers et al., 2022) and optimized inference frameworks like vLLM (Kwon et al., 2023), the text-to-triples conversion can be precomputed offline as well when maximal efficiency is required.
T, oy) represents
fact extracted from the text. This
Iterative Knowledge Enrichment to Question-Specific KG. The extracted triples di are then converted into graph structure gf (Vi, Ex), where Vú and E; denote the sets of nodes and edges, respectively. Each node e E V, corresponds to a unique subject or object entity in gi while each edge E E; represents predicate connecting the attributes of nodes and edges are obtained through Centence-BER' (Reimers, 2019):
entities. To enrich the graph with semantic information,
cmb(v) + (n), Ve Vù; emb(e) + encode(e), Ve E E; These semantic embeddings enable the model to capture the nuanced relationships between entities and facilitate reasoning over the KG.
(5)
To progressively enrich the question-related knowledge in response to the evolving sub- sub-queries, the structured graph of at each iteration i is merged into an evolving KG Go = (Vq. Eq) specific to the main question Q:
(6) After enriching Go with the new knowledge, we plan ($3.1) for the next step. Based on Go and the chain of previous subqueries and their associated graph information, the model decides whether to generate another focused subquery for additional retrieval or to proceed with answering ($3.3) if the accumulated knowledge sufficient.
Gq + GQ Ugk.
4

---

Published as a conference paper at ICLR 2026
3.3 KNOWLEDGE -UGMMETEED ANSWERING
When answering is triggered, the model M generates an answer A to the main question Q either conditioned knowledge graph Go and subquery chain (go. (o..9)),.., (gi,9: when retrieval-and- structuring was processed, or directly when no retrieval is needed.
If no retrieval is needed (po- = INO_RETRREVALL ]), the answer generated directly:
A + (M; INSTAasi 0; Q)
Otherwise, after iterative knowledge enrichment concludes with (SUFFICIENT) plan the maxi- mum iteration is reached, the answer is generated using encoded KG (Gq) and subquery chain:
A M(GNNNGQ)) INSTAasi [go. 90: .... qs (.lgg]:4
(8)
where INST is the answering instruction (as shown in Figure 16). M attends to knowledge in Gq and subquery chain to generate accurate, coherent answers. This structured conditioning enables systematic reasoning grounded in the assembled knowledge.
3.4 STURCTURE-AWARY MULTITASK LEARNING
The RAS framework is trained through a multitask setup that unifies knowledge-awaz planning and wewwledge-augmentie answering under a standard next-token prediction objective.
Planning
[NO_ RETRIEVAL] [SUFFICIENT] / [SUBQ] 4:11
Graph LLM
Each training instance corresponds to either a plan- ning task an answering task, selected randomly:
Graph Encodier &
Projector f
Text Embeerider
Planning: Input:
current encoded question- specific KG Go. planning instruction INSTpian (shown in Figure 15), subquery-trije -tripees history ((go. 90.
GO
INST INST
plan
I0..
shared parameters
... qi, 9:), main question Q. Output:
Answering
next plan Pi+1.
Answer A
Answering: Input: final encoded question-
specific KG Go. answering instruction INST (shown in Figure 16), subquery- triples history (Igo, 9o..... qi, 9: main question Q. Output:
Graph LLM
Groph Encoder
Embaadder
final answer A.
Projector
GO
INST Ans
No Bo Q
The model .M used in RAS is based Graph LLM, an architecture adapted from prior work (Perozzi al., 2024; He et al., 2024). Note: This multi- task training setup is applied to open-source setting, while we test RAS under closed-source setting (see Appendix F.1) in Section 4 as well.
Figure 2: Structure-Aware Multitask Learning for the Graph LLM in RAS. single LLM trained with both planning and answering tasks in parameter-efficient way (fine-tuning the graph components with LoRA (Hu et al., 2022)).
4
EXPERIMENTS
4.1 SETTINGS
Training Data & Setting. We develop HotporQA-SUBQ. dataset constructed based on HotpotQA (Yang et al., 2018), to train our model's action planning and answering capabilities. Our dataset creation begins with document filtering: using Claude-3.5-Sonme (Anthropic, 2024), we identify and retain only the supporting documents necessary for answering the main question, removing irrelevant content. For each supporting document d,, we then iteratively generate a subquery ęj. considering the main question, previous subqueries, and supporting documents. During iteration j. when more supporting documents remain, we create training samples with input (qu- 9u.... qj. 9j. Q} and output label "[SUBQ] 4/+1", where Is represents triples extracted from document dj. and Q is the main question. For the final supporting document, we label the sample as " "SUFFF ICIENT]" To identify queries that can be answered directly, test base L.LM (LLaMA-2-7BB on HotpotQA's main
5

---

Published as a conference paper at ICLR 2026
queries without context. For correctly answered queries, we create training samples with the main question Q as input and " [NO RETRIEVAL]" as the output label. Additionally, to ensure fair comparison with existing approaches, we incorporate the subset of Arc-Easy (2,147 samples) and ASQA (3,897 samples) from Self-RAG's training data, resulting in 208k training samples in total. We place detailed training data processing, dataset statistics, and data samples in Appendix C. For efficient inference, we train a text-to-triples model fr2 on the WikiOFGraph dataset (Kim et al., 2024a) using LLaMA-3.2-3B-Instte for optimized runtime performance. We present our hyperparameter study of each component in Appendix Knowledge Sources. We employ faiss (Douze al., 2024) for efficient vector searching over the dense index. Following the Self-RAG (Asai et al., 2023), we utilize the Wikipedia 2018 8 (Izacard et al., 2023) by default, while specifically using the Wikipedia 2020 for PopQA to access more recent information. To optimize retrieval efficiency, we partition the index into five segments.
as the base model (see Appendix D), and deploy it using vLLM
Test Datasets Metrics & Compared Baselines. We conduct comprehensive evaluations on diverse Knowledge-intensive tasks following previous studies (Asai et al., 2023; Lyu et al., 2024b). The evaluation encompasses three categories of datasets: (1) open-domain short-form generation datasets: TriviaQA (Joshi et al., 2017), PopQA (Mallen al., 2022), and 2 2WikiMultihop@A (Ho et al., 2020); (2) closed-set task datasets: PubHealth (Zhang et al., 2023a) and ARC-Challenge (Clark al., 201 8); and (3) long-form generation datasets: ALCE-ASQA (Gao al., 2023; Stelmakh et al., 2022) and EL.5 (Fan et al., 2019). For evaluation metrics, we maintain consistency with prior work (Asai al., 2023; Mallen et al., 2022: Lyu et al., 2024b), employing "golden match" accuracy for PopQA and TriviaQA, token-level Fl score for 2WikiMultihoQAA, accuracy for PubHealth and ARC-Challenge and ROUGE-LSum alongside MAUVE score (Pillutla al., 2021) for ASQA and ELI5. Our comparative analysis includes three baseline categories: models without retrieval augmentation, incorporating Claude 3.5 Sonnet as a sfute-of-the--oee closed-source baseline; models with single retrieval over top-5 documents, including Claude 3.5 Sonnet, and SuRe (Kim et al., method; and models with self-reflective retrieval, including
2024b), a leading gerrive-and-summarre leading approaches Self-RAG (Asai et al., 2023), RPG (Lyu et al., 2024b). We place more details of datasets and metrics in Appendices E. and E.2, respectively.
Inference Setting. We evaluate RAS using both trained open-source models (RAS /SB based on LLaMA-2-7B/LLLMM--- the closed-souree 3.5dde--5--onneee & Claude-4.5-Sonne' (Anthropic, 2025) under varied inference settings. For open-source models (RAS7R/8B)) we follow prior work (Asai et al., 2023; Lyu et al., 2024b) and adopt zero-shot inference across all datasets. For the closed-source model (RAS RAS we apply few-shot inference with two exemplars for ASQA and EL15, while using zero-shot inference for all other datasets. More details are provided in Appendices F.1 and F.2.
and a Graph Transformer encoder (Shi et al., 2020)) and
For PopQA and TriviaQA evaluation, we follow established settings (Asai et al., 2023; Luo et al., 2023a), incorporating top-five web search engine results as initial retrieved context to. For ASQA and ELI5, we maintain methodological consistency with prior work (Lyu et al., 2024b; Asai et al., 2023; Gao et al., 2023), utilizing their predetermined five-document context. Under these conditions, we omit plan generation and text retrieval phases, implementing static inference (Asai et al., 2023) with five fixed iterations. For remaining datasets, we establish a maximum iteration count of five. Following previous studies, we employ Contriever-MS MARCO (Izacard et al., 2021) as the primary dense retriever, with BM25 (Robertson et al., 2009) serving as the retrieval mechanism for 2WikiMultihopQA. Across all retrieval processes, maintain consistent top-k document selection of five. We present more details of inference settings in Appendix F.
4.2 RESULTS
Main Result. Our performance evaluation, presented in Table 1, demonstrates that the LLaMA-2- 7B/LLLMMA-3-8 model fine-tuned with RAS outperforms existing SFT-based open-source solutions, including Self-RAG (Asai et al., 2023) and RPG (Lyu et al., 2024b). Notably, compared to the earlier SOTA models, RAS/B shows a 9.7% improvement in short-form question-anssering and a 7.9% gain in long-form generation tasks. Additionally, when applied to Claude-3.5-Somee RAS consistently achieves superior results compared to single retrieval RAG approaches, including retrieve- and-summarize approach SuRe (Kim et al., 2024b). We find that sometimes (e.g., on TriviaQA and PubHealth) single-hop retrieval could not boost LLM's performance, and even makes it worse, which demonstrates the necessity of on-demand retrieval, aligning with previous findings.

---

Published as a conference paper at ICLR 2026
Short-form
Closed-set
Long-form Generation
Model/Method
TQA
2WQA (F1)
PopQA (acc)
Pub (acc)
ARC (acc)
ASQA (mauve)
ELI5
(ucc)
(rouge)
(rouge)
(mauve)
w/o Retrieval ChatGPT
74.3
24.8
29.3
70.1
75.3
36,2
68.8
22.8
32.6
Sonnet-3.5
40.0
30.2
21.8
26.5
Single Retrieval (#docs=5)
Closed-souree
Sonnet-3.5 g.deeet
69.1
41.9
51.5
49.1
88.6
n/a
n/a
I/a
n/a
gonnct-3.5
72.5
53.7
57.3
53.9
87.1
38.8
61.6
20.2
32.3
SuRearr=<(Kim 2024b)
72.3
38.1
53.6
57.2
79.6
36.0
74.2
19.2
51.6
SuRessmmi35
76.8
37.6
41.2
62.8
91.6
30.2
69.9
15.4
27.2
w/ Self-Reffeectiv Retrieval
ReAcIsema--1 (Yao 2023)
73.4
53.7
55.0
62.2
38.8
61.6
20.2
32.3
IRCoT Scman-3 (Trivedi et al., 2023)
IRCoT
74.7
54.9
53.2
59.4
92.0
38.8
61.6
20.2
32.3
Nelrieval-nnd-Sttruttrii (ours)
RAS
77.6
57.7
62.3
70.5
23.3
w/o Retrieval
Lluma27y
30.5
18.9
14.7
34.2
21.8
15.3
19.0
18.3
32.4
Llama21u
38.5
20.2
14.7
29.4
18.2
Llama3sa
56.1
21.2
26.7
33.2
42.2
17.6
25.0
18.2
39.7
w/ Single Retrieval (#docs=5)
Lluma27p
42.5
21.0
38.2
30.0
22.1
18.6
Llama2yus
47.0
31.2
45.7
30.2
26.0
20.5
24.7
18.6
42.3
Open-source
Llama3
60.4
33.4
48.6
36.5
40.1
23.9
$2.1
18.8
40.7
SuRenn (Kim al., 2024b)
$1.2
20.6
39.0
36.2
35.8
16.1
w/ Self-Reflective Retrieval
Self RA--RAGG (Asai al., 2023)
66.4
25.1
54.9
72.4
67.3
35.7
74.3
17.9
35.6
Self-RAG1s
26.9
55.8
RPG (Lyu et al., 2024b)
65.1
33.6
56.0
73.4
65.4
37.6
84.4
19.1
46.4
ReActsa (Yao et al., 2023)
64.0
25.0
42.7
52.4
59.0
22.1
32.0
18.6
35.3
I (Trivedi 2023)
61.5
27.6
44.3
22.1
32.0
18.6
35.3
Wetrieval-And--Sracture (ours)
RAST RASSa
72.7
42.1
58.3
74.7
68.5
37.2
95.2
19.7
47.8
73.8
44.2
20.1
Table 1: Performance Comparison. We highlight the top-2 closed-source models and top-2 open-source 7B models, with the best model each category underlined for each dataset. RAS is designed for general open-domane question answering rather than Knowledge Graph Question Answering (KGQA) tasks (Perevalov et al., 2022); we therefore exclude comparisons with -GQAA-ppcific methods (Sun et al., 2023; Luo et al., 2023b; Ma et al., 2024). Graph encoding and projection are omitted for R.S.some-
Although RAS integrates planning and answering in unified framework, these components be decoupled for greater flexibility. Our "role-swapping" study Figure 3 demonstrates that performance is primarily constrained by answering capability rather than planning. When Sonnet-3.5 handles planning while RAS7B performs answering, the system achieves 62.4% accuracy on ARC-C, compared 93.9% when Sonnet-3.5 performs both tasks. Despite having 60x fewer parameters, RASTB achieves planning performance comparable to (and sometimes exceeding) Sonnet-3.5.
We analyze the impact of each component of RAS in Table 2 and provide detailed discussion below.
Effect of Iterative Planning and Retrieval. Comparing the base model with "No Planning" variant shows that iterative planning provides consistent improvements across all metrics (e.g., +8.8% on TQA, +9.0% on 2WQA). This demonstrates the importance of dynamically determining retrieval needs and generating focused sub-queries. Without planning, the model relies on single-pass retrieval, which may miss crucial information needed for complex reasoning. Also, when turning off retrieval, performance degradation more obvious, due the knowledge-intensive nature of those datasets.
Effect of Graph Construction and Encoding, The impact of structured knowledge representation evident from two ablations. First, "No Text-to-Triple" degrades performance significantly on all metrics (e.g.. -9.0% on 2WQA, -22.2% MAUVE on ASQA), showing the value of essential information extraction via converting retrieved text into structured triples. Second, removing the graph encoder (oNo GraphToken") during training inference consistently hurts performance across datasets, with particularly large drops on PubHealth (- 1.1%% and -1000 respectively). This suggests that the graph structure encoding helps the model better leverage the knowledge relationships.

---

Published as a conference paper at ICLR 2026
TQA
2WQA
Pub
ASQA
(ace)
(F1)
(acc) (rg)
(mv)
RAST
72.7
42.1
74.7
37.2
95.2
Training Phave
40
w/o GraphEncode
70.2
38.4
66.4
33.1
85.0
w/o LoRA
71.5
37.8
54.8
32.8
84,8
w/o Text-to-Triple Triple
70.4
38.2
71.4
36.2
73.8
w/o Multi-Task
68.6
39.2
65.5
36.7
88.9
TrAADA
2W
PODOA
PubHoolth
ARC
Plan: RASsame
Ans: RASttrnet
Plan: RASssmet
Ans: RASra Anss RAS
Inference Phase
Plan: RAS
Ans: RASssma
Plan: RASys
w/o Retrieval
56.9
27.4 4
69.0
31.3
70.6
w/o GruphEncode
68.8
38.7
67.3
36.5
93.6
Figure 3: Role-Swapping Study. We alternate Sonnet- 3.5 and RASH on the planning and answering tasks, and evaluate the overall performance.
w/o Planning
66.7
37.8
71.5
37.2
95.2
Table 2: Ablations in Training and Inference (with RASTB). Training: "No GraphEncode" re- moves the graph encoder during training, using only LoRA-based LLM fin-ttning, "No LoRA" uses graph token optimization without low-rank adapta- tion. "No Text-to- Triple" keeps the original retrieved texts instead of converting them into triples. "No Multi-Task trains two models separately handling planning and answering. Inference: "No Retrieval" tests direct query answering without any context. "No GraphEncode" removes graph encoding and pro- jection during inference, using only textual context. "No Planning" removes the planning module and runs single-pass
TQA
ASQA-MV
75
70
7B
7B
65
8B
8B
10% 30%
50% 70% 100%
10% 30% 50%
70%
100%
Figure 4: Impact Graph Information Abundance. For each sample, we randomly shuffle its associated triples five times and take different ratios (10%-100%% of the shuffled data. The performance scores aver- aged across these five shuffling runs.
answering pipeline.
Effect of LoRA and Multi-task Learning. Our experiments reveal that parameter-efficient training strategies significantly impact model performance. Using only graph token optimization without LoRA leads to substantial degradation (-11.8% on average). A similar observation can be made for "No Multi-Task" (e.g.. 12.3% accuracy degradation on PubHealth), indicating the significance of jointly training the model on both action planning and answer generation tasks rather than optimizing for each task separately, supporting findings from prior work (Lyu et al., 2024b). The complementary effects suggest that while graph-basee knowledge representation is valuable, it needs to be combined with careful parameter tuning and multi-task learning to achieve optimal performance.
Impact of Graph Information bbuddacc.. Figure 4 shows that increasing the amount of structured graph information consistently improves RAS's 's performance across tasks. For TQA, both RAS7B and RASgB exhibit approximately linear gains as more triples are retained, with no clear saturation even at 100% information. For ASQA-MV, RAS7B benefits from sharp improvement between 10% and 30% graph information, followed by steady increases, while RAS&B maintains a smoother and more stable growth pattern throughout. These results confirm that RAS effectively leverages structured knowledge at all levels of availability, and that more complete knowledge graphs consistently translate to stronger reasoning and generation quality. Additionally, the larger 8B model consistently outperforms the 7B variant under all conditions, suggesting that scaling the model size further enhances RAS's A ability to utilize structured knowledge. Interestingly, even partial graphs containing only 30% 650 of triples already deliver substantial gains over low-information baselines, highlighting RAS's s robustness to incomplete or partially retrieved knowledge.
Impact of Training Data Volume. Figure 5 demonstrates how training dataset size influences model performance across different tasks. Considering the computational efficiency, we sampled 2,000 instances each from TQA and 2WQA for evaluation, while maintaining the original sizes of other datasets. The results indicate that model performance generally improves with increased training data volume. Notably, our model achieves competitive performance even with limited training data using only 5% (10K instances) of our full dataset, it surpasses previous state-of- the-art models on TQA, 2WQA, and ELI5. These results suggest both the robustness of our ar-
8

---

Published as a conference paper at ICLR 2026
TQA
ZWQA
PubHeaathh
ASQA-RG
ASQA-MV
ELIS-RG
ELIS-MV
45
80
D00
20
RAS..
72
80
40
30
70
RASu
RASu
RASyj
25
RAS
60
RA5
18
RASyu
15
10k
7dk
200k
19k
7dk
2008
10k
70k
200k
10k
70k
200k
10K
70k
200k
10k
70k
200k
10k
70kc
2006
74
43
80
35
20
RAS
72
40
30
68
RA5aa
RA5
RA5
60
40
RA5
18
RA5
102
73k
20ck
70k
202k
70c
200k
70c
200k
10k
7dk
20ck
10k
70k
200k
1.kk
70kc
203k
Figure 5: Impact of Training Data Volume Model Performance. Results for RAS (top) and RASua (bottom) illustrate how performance scales with increasing training data.
chitectural design and the effectiveness of our data curation methodology. However, we observed an exception with the ELJ5 dataset, where performance were inconsistent. This irregularity can be attributed to the inclusion of ASQA training data in our training set, following established setting from previous research (Asai et al., 2023; Lyu al., 2024b). Among test datasets, ASQA and ELI5 are unique in requiring long-form response generation. The periodic decline in ELI5 performance suggests that the model's response generation began align more closely with ASQA's training data distribution, potentially at the expense of EL15's distinct characteristics.
Impact of Triple Extractor Se- lection We examine how triple extraction quality affects RAS performance using three mod- els: Flan-T--Large, LLaMA- 3.2-3B, and Claude-3.5-Somee As shown in Table 3, higher- quality extraction consistently improves results across datasets, with Staude-3.5-Sonee achicv- ing the best accuracy but sig- nificantly lower efficiency (1.5e- 2 sec/token vs 2.0e-4 for LLLaMA-3.2-3B- These results demonstrate the accuracy-effiiienc tradeoff in triple extraction, with LLaMA-3.2-30 providing the optimal balance for our experiments.
Performance 2WQA
Efficiency (tokens/s)
Triple Extractor fru
TQA
PopQA
Flan T5-Laage LLaMA-3.2-2 Claude-3.5-Somme
70.3
40.7
56.7
166.0
72.7
42.1
58.3
4,885.3
73.8
44.5
60.1
68.2
Table 3: Impact of Triple Extractor Selection. Higher-quality text- to-triples models lead to better answer generation in RAS. Claude-3.5- Sonnet achieves the best performance on TriviaQA, 2WikiMulihopQA. and PopQA. ILLMA-3.2-33 (deployed w/ vLLM) used in our experi- ments, selected its strong balance between accuracy and efficiency.
5 CONCLUSION
We presented RAS, framework that dynamically constructs question-specific knowledge graphs through iterative retrieval and structured reasoning. Experiments across seven benchmarks show consistent improvements of up 6.4% and 7.0% with open-source and proprietary LLMs, respectively. The modular architecture enables transparent reasoning and seamless integration with external knowledge sources. Limitations and future work discussed in Appendix A. The statement of the LLM usage is placed in Appendix B. Broader Impacts, safeguards, and used assets are discussed in Appendix K. Our data and code can be found at: https: // ///i/ttin gighub..... com/ ppattaat--- jj //AAS.
ETHICS STATEMENT
This work does not involve human subjects, personal data, or sensitive user information. Our research focuses on algorithmic improvements for knowledge-intensive language model reasoning. We are mindful of potential ethical concerns, such as risks misinformation or biased outputs when deploying retrieval-augmented LLMs. To mitigate these risks, provide transparent descriptions of our methods, ablation studies on failure cases, and open-sourcing code and models to facilitate community scrutiny. We also discuss broader impacts and safeguards in Appendix K.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
Research was supported in part by a research gift from Google Inc.; the AI Institute for Molecular Discovery, Synthetic Strategy, and Manufacturing: Molecule Maker Lab Institute, funded by U.S. National Science Foundation under Awards No. 2019897 and 2505932; NSF IIS 25-37827; and the Institute for Geospatial Understanding through Integrative Discovery Environment (I-GUIDE) by NSF under Award No. 21 2118329. The research has used the Delta/DeltaAl advanced computing and data resource, supported in part by the University of Illinois Urbana- Chammpaiin and through allocation #250851 1 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which supported by National Science Foundation grants OAC #2320345, #2 #2138286, #2 #2137603, and #2138296.
REFERENCES
Anthropic. Introducing claude 3.5 sonnet.
claude-3e 3-5 Sonontt,, 2024. Accessed: 2025-01-06.
Anthropic. Claude sonnet 4.5 system card. Technical report, Anthropic. 2025.
URL https: /assets anthropic com /ttpmmm112ff 4e fcc wa..4455//rr/// Aeriginall Sestt-------ystee---aaa Card. pdf. System card for the Claude Sonnet 4.5 large language model.
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Aditi Chaudhary, Karthik Raman, Krishna Srinivasan, Kazuma Hashimoto, Mike Bendersky, and Mare Najork. Exploring the viability of synthetic query generation for relevance prediction. arXiv preprint ar Xrviv:2305. 1944, 2023.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetunee language models. Journal of Machine Learning Research, 25(70):1-53, 2024.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv preprint arXiv: 1rX3::8803.05557 2018.
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3, int8 Q: 8-bit matrix multiplication for transformers at scale. Advances in neural information processing systems, 35: 2022.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint /.10:0//00.04000 2018.
Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. arXiv preprint arXiv:2401.0828. 2024.
Darren Edge. Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. From local to global: graph rag approach to query-focused summarization. arXiv preprint arXiv:2404. 16130, 2024.
Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. RAGAs: Automated evaluation of retrieval augmented generation. In Nikolaos Aletras and Orphee De Clereq (eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pp. 150-158, St. Julians, Malta, March 2024. Association for Computa- tional Linguistics. URLhhttss / aowhlllnthooogo
2004 eacl demo 1 6/.
10

---

Published as a conference paper at ICLR 2026
Angela Fan, Yacine Jernite, Ethan Perez, David Grangier. Jason Weston, and Michael Auli. Eli5: Long form question answering. arXiv preprint 1xXiv:997.0099900 2019.
Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen, Enabling large language models to generate text with citations. arXiv preprint arXiv:2305. /4627, 2023.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama herd of models. arXiv preprint arXiv:2407.3 2 2/733, 2024.
Jian Guan, Wei Wu, Zujie Wen, Peng Xu, Hongning Wang, and Minlie Huang. Amor: A recipe for building adaptable modular knowledge agents through process feedback. arXiv preprint arXiv:2402.011669 2024.
Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. Hipporag: Neurobio- logically inspired long-term memory for large language models. arXiv preprint arXiv:2405. 14831, 2024.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In International conference on machine learning, pp. 3929-3938. PMLR, 2020.
Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V. Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. G-retriever: augmeete generation for textual graph understanding and question answering. arXiv preprint arXiv:2402.07630, 2024.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:201 1.000000..000. 2020.
Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d' Amato, Gerard De Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, et al. Knowledge graphs. ACM Computing Surveys (Csur), 54(4):1-37, 2021.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 874-880, Online, April 2021. Association for Computational Linguistics. doi: 10. 10.18653/v120221.eam wac.:2ma...lllmm...... URL https: //aes//aclllaii http//aaaaaatlooooooooooo org/ 2021 eacl manminn 74.
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning. arXiv preprint 12.00921 18, 2021.
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-YY Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251):1-43, 2023.
Pengcheng Jiang, Lang Cao. Cao Xiao, Parminder Bhatia, Jimeng Sun, and Jiawei Han. Kg- fit: Knowledge graph fine-tuning upon open-world knowledge. The Thirty-Eighh Annual Conference Neural Information Processing Systems, 2024.
Pengcheng Jiang, Jiacheng Lin, Lang Cao, R. Tian, S. Kang, Z. Wang, Jimeng Sun, and Jiawei Han. Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning. arXiv preprint arXiv: 2503.00 3503,00223, 2025a.
Pengcheng Jiang, Cao Xiao, Minhao Jiang, Parminder Bhatia, Taha Kass-Hout Hout, Jimeng Sun, and Jiawei Han. Seasoning-enhaneee healthcare predictions with knowledge graph community retrieval. In The Thirteenth International Conference on Learning Representations, 2025b.
11

---

Published as a conference paper at ICLR 2026
Pengcheng Jiang, Xueqiang Xu, Jiacheng Lin, Jinfeng Xiao, Zifeng Wang, Jimeng Sun, and Jiawei Han. s3: You don't t need that much data to train a search agent via via In Proceedings of the 2025 Conference Empirical Methods in Natural Language Processing, 2025c.
Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yu, Yiming Yang, Jamie Callan, and Graham Neubig. Active retrieval augmented generation. arXiv preprint arXiv:2305.06988; 2023.
Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint 1eXiv:1705.0355 2017.
Mingxuan Ju, Wenhao Yu, Tong Zhao, Chuxu Zhang, and Yanfang Ye. Grape: Knowledge graph enhanced passage reader for open-domain question answering. arXiv preprint vrXiv22200.029::3 2022.
SeongKu Kang. Shivam Agarwal, Bowen Jin, Dongha Lee, Hwanjo Yu, and Jiawei Han. Improving retrieval in theme-specific applications using corpus topical taxonomy. In Proceedings of the ACM Web Conference 2024, pp. 1497-1508, 2024.
Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia. Demonstrate-scarchh praedctd Composing retrieval and language models for knowledge-intensive nlp. arXiv preprint arXiv:2212. 14024, 2023.
Daehee Kim, Deokhyung Kang, Sangwon Ryu, and Gary Geunbae Lee. Ontology-free general- domain knowledge graph-to-text generation dataset synthesis using large language model. arXiv preprint 2.Xi::2409.00088, 2024a.
Jachyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Woo Lee, Minjoon Seo, Jung- Woo Ha, and Jinwoo Shin. Sure: Summarizing retrievals using answer candidates for open-domain qa of Ilms. arXiv preprint arXiv:2404.13881, 2024b.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium Operating Systems Principles, 2023.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, al. Retrieval-augmented genera- tion for Enowledge-ineensive nip tasks. Advances in Neural Information Processing Systems, 33: 1459-9474, 2020.
Chin Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches oul, pp. 74-81, 2004.
Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. On the biology of large language model. Transformer Circuits Thread, 2025. URL https:////// http:///tansfoomeee circuitsss. pub/2025 htt/s//ttributt. wwwprrtuttiiinoognnooooaaaoaa graphs/ /biosogiboo....ooooooo .tmll
Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su. Deductive verification of chain-of-thouse reasoning. Advances in Neural Information Processing Systems, 36, 2024.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du Zhilin Yang, and Jie Tang. P-wuing v2: Prompt tuning can be comparable fine-tuning universally across scales and tasks. arXiv preprint arXiv:2 10.07602, 2021.
Hongyin Luo, Yung-Sung Chuang, Yuan Gong, Tianhua Zhang, Yoon Kim, Xixin Wu, Danny Fox, Helen Meng. and James Glass. Sail: Search-augment instruction learning. arXiv preprint arXiv:2305. 15225, 2023a.
12

---

Published as a conference paper at ICLR 2026
Linhao Luo. Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. Reasoning on graphs: Faithful and interpretable large language model reasoning. arXiv preprint arXiv:2310.01061, 2023b.
Yuanjie Lyu, Zhiyu Li, Simin Niu, Feiyu Xiong, Bo Tang, Wenjin Wang, Hao Wu, Huanyong Liu, Tong Xu, and Enhong Chen. Crud-rag: A comprehensive chinese benchmark for retrieval- augmented generation of large language models, 2024a. URL https: //te:////ara http/arxivv.o//// org abs/ 2401. 17043.
Yuanjie Lyu, Zihan Niu, Zheyong Xie, Chao Zhang, Tong Xu, Yang Wang. and Enhong Chen. serrivve-planggnerraee An iterative planning and answering framework for knowledge-intensinn Ilm generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 4683-4702, 2024b.
Shengjie Ma, Chengjin Xu, Xuhui Jiang, Muzhi Li, Huaren Qu, Cehao Yang, Jiaxin Mao, and Jian Guo. think-on-graph 2.0: Deep and faithful large language model reasoning with knowledge- guided retrieval augmented generation. arXiv preprint 0rXiv:2407.10800 2024.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi.
When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. arXiv preprint arXiv:2212 105 2022.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.). Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1906-1919, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/ 2020.acl- main. 173. URL https: http////aaaantognr/e/eeee org/ 22///222.c22 acl acl-main. maiin.13.
Lang Mei, Zhihan Yang, Xiaohan Yu, Huanyao Zhang, and Chong Chen. Ai-searchplanner, Mod- ular agentic search via pareto-optimal multi-objective reinforcement learning. arXXiv preprint arXiv:2508. 20368, 2025.
Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. A discrete hard em approach for weakly supervised question answering. arXiv preprint arXiv: 190904499, 2019.
Aleksandr Perevalov, Xi Yan, Liubov Kovriguina, Longquan Jiang, Andreas Both, and Ricardo Usbeck. Knowledge graph question answering leaderboard: A community resource prevent replication crisis. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, and Stelios Piperidis (eds.). Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 2998-3007, Marseille, France, June 2022. European Language Resources Association. URL https: htt///aaaaaattoooo...oo.oo wwt. 1rec-1. 1. 321/-
Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Mehran Kazemi, Rami Al-Rfou, and Jonathan Halcrow. Let your graph do the talking: Encoding structured data for llms. arXiv preprint arXiv:2402.05862, 2024.
Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and 7aid Harchaoui. Mauve: Measuring the gap between neural text and human text using divergence frontiers. Advances Neural Information Processing Systems, 34:4816 48288 2021.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350, 2022.
Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112..11446 2021.
N Reimers. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv: 10084, 2019.
Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333-389 2009.
13

---

Published as a conference paper at ICLR 2026
Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. Enhancing strieval-augmentt large language models with iterative retrieval-generation synergy. arXiv preprint arXiv:2305.15294 2023.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification. arXiv preprint 1xXiv:2009003599 2020.
Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming- Wei Chang. Asqa: Factoid questions meet long-form answers. arXiv preprint arXiv:2204.06095 2022.
Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M Ni, Heung-Yeue Yeung Shum, and Jian Guo. Think-on-graph Deep and responsible reasoning of large language model knowledge graph. arXiv preprint 2xXiv:2307.06697, 2023.
Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. Ernie: Enhanced representation through knowledge integration. arXiv preprint /904 0et.:9994.0923,, 2019.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-though reasoning for knowledge-intense multi-step questions. In Proceedings of the 6l Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10014-10037, 2023.
Xuezhi Wang, Haohan Wang, and Diyi Yang. Measure and improve robustness in nlp models: A survey, arXiv preprint arXiv:2112.0833 2021.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio. William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv: 1809009000, 2018.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. arXiv:2210.03629, 2023.
Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. Qa-gnn: Reasoning with language models and knowledge graphs for question answering. arXiv preprint arXiv:2104.06378, 2021.
Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D Manning, Percy S Liang, and Jure Leskovec. Deep bidirectional language-knowleddg graph pretraining. Advances in Neural Information Processing Systems, 35:37309-37323, 2022.
Wenhao Yu. Retrieval-augmented generation across heterogeneous knowledge. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, pp. 52-58, 2022.
Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang. Heng Ji, and Meng Jiang, survey of knowledge-enhancee text generation. ACM Computing Survey (CSUR), 2022.
Tianhua Zhang, Hongyin Luo, Yung-Sung Chuang, Wei Fang, Lue Gaitskell, Thomas Hartvigsen, Xixin Wu, Danny Fox, Helen Meng, and James Glass. Interpretable unified language checking. arXiv preprint arXiv:2304.03728 2023a.
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren's 's song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219, 2023b.
Yinghao Zhu, Changyu Ren, Shiyun Xie, Shukai Liu, Hangyuan Ji. Zixiang Wang, Tao Sun, Long He, Zhoujun Li, Xi Zhu, and Chengwei Pan. REALM: Rag-driven enhancement of multimodal electronic health records analysis via large language models. arX1v:2402.07016, 2024.
14

---

Published as a conference paper at ICLR 2026
Contents of Appendix
A Limitations & Future Work
16
B The Use of Large Language Models
16
HotpotQA-SUBQ Dataset
17
C.I Supporting Document Filtering
17
C.2 Sub-query Generation
18
C,3 Sample Labeling
18
C,4 Dataset Statistics
20
C.5 Data Examples
20
Training Details
22
D.I Text-to-Triples Model Training
22
D.2 Multitask Training of Planning and Answering Capabilities
................................
23
Evaluation Datasets & Metries
24
E.1 Test Datasets
24
E.2
Evaluation Metrics
25
F Inference & Evaluation Details
26
F.1
Closed-Soune Model Settings
F2
Open-Source Model Settings
29
F3
Instructions
29
G Cost & Efficiency Comparison with Other Graph-based RAG Methods
29
H Supplementary Results and Analysis
Qualitative Examples
32
Hyperparameter Study
36
K Broader Impacts, Safeguards, and Assets
37
15

---

Published as a conference paper at ICLR 2026
A LIMITATIONS & FUTURE WORK
While RAS demonstrates strong performance, several key limitations warrant consideration for future improvements:
Knowledge Extraction Quality: The effectiveness of both planning and answer generation depends on the quality of text-to-triple extraction. Despite using an end-to-end rext-to-triples model for triple extraction, implementing a more sophisticated approach such as dedicated Named Entity Recognition and Relation Extraction (NER-RE) pipeline could potentially enhance RAS's reasoning capabilities and knowledge representation.
Graph Evolution Strategy: The current approach to evolving knowledge graphs could be enhanced by incorporating more sophisticated graph pruning and merging strategies. Future work could explore dynamic graph summarization techniques to maintain the most relevant information while preventing excessive graph growth during iterations.
Adaptability: While RAS performs well on the evaluated tasks, its effectiveness across highly specialized domains or multilingual settings remains be investigated. Future research could focus on developing domain- adapttaiono techniques and multilingual knowledge structuring approaches.
Interactive Refinement: The current implementation lacks mechanisms for interactive refine- ment of the constructed knowledge graphs. Future versions could incorporate human feedback loops to improve the quality of knowledge structuring and reasoning paths.
These limitations suggest several promising directions for future research, including improved knowledge extraction techniques and enhanced graph evolution strategies. A particularly promising direction the integration of external knowledge graphs into RAS's s iterative loop, as its graph-based architecture naturally accommodates the incorporation of established KGs like Wikidata or domain- specific knowledge bases. This could significantly enhance the system's 's reasoning capabilities by combining dynamically constructed knowledge with curated structured information.
B THE USE OF LARGE LANGUAGE MODELS
In this work, large language models (LLMs) used solely for drafting and polishing the manuscript. Their role was limited to grammar correction, phrasing refinement, and enhancing readability. All research ideas, methodological innovations, theoretical analyses, and experimen- tal work were entirely conceived, designed, and carried out by the authors. The authors take full responsibility for the scientific content of the paper.
16

---

Published as a conference paper at ICLR 2026
C HOTPOTQA-SUBQ DATASET
The source of our dataset is HotpotQA (Yang et al., 2018).' Our goal is to create dataset that can let the model learn the capabilities of subquery generation and answering with intensive knowledge. We name the dataset as HolpoiQA-SUBQ.
The construction of HotporQA-SUBQ includes three steps: filter out irrelevan/unnccessary "supporting does" (SC.1), (2) generate one subquery for each supporting document ($C.2), and (3) sample labeling ($C.3).
C.1 SUPPORTING DOCUMENT FILTERING
To identify and filter out irrelevant supporting documents from HotpotQA, employ an instruction- based approach using Claude 3.5 Sonnet, as shown in Figure
Identify which documents are HELPFUL to answer the question. Output only the document numbers separated by commas.
Examples:
Example (Some documents are not helpful)::
Question: What nationality was James Henry Miller's wife? Supporting docs:
1. Margaret "Peggy" Seeger (born June 17, 1935) is an American folksinger. She is also well known in Britain, where she has lived for more than 30 years, and was married to the singer and songwriter Ewan MacColl until his death in 1989. 2. Seeger's father was Charles Seeger (1886-1979), an important folklorist and musicologist; her mother was Seeger's second wife, Ruth Porter Crawford.
James Henry Miller, better known by his stage name Ewan MacColl, was an English folk singer and songwriter.
Output: 1,3
Explanation: Only docs 1 and 3 are helpful doc shows Peggy Seeger (who is American) was married Ewan MacColl, and doc confirms Ewan MacColl is James Henry Miller. Doc about Seeger's parents is not helpful.
Example (All documents are helpful):
Question: The Oberoi family part of hotel company that has head office in what city? Supporting docs:
1. The Oberoi family is an Indian family that is famous for its involvement in hotels, namely
through The Oberoi Group.
2. The Oberoi Group is hotel company with its head office in Delhi.
Output: 1,2
Explanation: Both docs are helpful doc links the Oberoi family to The Oberoi Group, and doc 2 provides the head office location.
Question:
[question]
Supporting docs:
[enumerated dicpreeesddcmccs
Output only the helpful document numbers separated by commas:
Figure 6: Prompt used for filtering supporting documents in HotpotQA. The prompt includes examples demonstrate the difference between helpful and irrelevant documents. The input parts to the prompt are highlighted.
The filtering prompt designed with clear examples that illustrate the criteria for document relevance. For each HotpotQA sample, we enumerate all supporting documents and use Claude to identify only
We the version hotpot_traing v1. from https: //gt thub. cowbbbboo//htttt.gpp /hotpot.
17

---

Published as a conference paper at ICLR 2026
those that contribute directly answering the question. After filtering out irrelevant documents, filter the question with no supporting documents. This filtering step reduces noise in the training data and helps focus the model truly relevant information during sub-query generation.
C.2 SUB-QUERY GENERATION
For sub-query generation, the template follows to generate subquery per (filtered) document:
Given this main question and supporting document, generate a simple sub-query question) that will help retrieve information from the document to answer the main question.
Main Question: [main [uieii_neetoo]
Current Document ([topcc]):
coctuumeet__onttene
[If previous queries exist:]
Previously generated sub-queries:
[sub_query_]] _ueey_uerrr_____ [sub query_uerrr_.. 2]
Write ONE clear and specific question that:
1. Can be answered using ONLY this document
2. Helps retrieve information needed for the main question
3. Is direct and focused on key information from this document
Write only the question, without any explanations or formatting.
Figure 7: Prompt used for subquery generation from HotpotQA. The input parts to the prompt are highlighted.
The prompt template enforces Dacument-speciffcce goal-orientation, and conciseness in sub-query generation. For iterative querying, we maintain a list of previously generated sub-queries to avoid redundancy and encourage progressive information gathering.
C.3 SAMPLE LABELING
The sample labeling process transforms the filtered and augmented HotpotQA examples into training instances for both the Planning and Answering components. We formalize this process in Algorithm 1.
The algorithm takes as input the filtered HotpotQA dataset DD base language model M, and text- to-triple conversion model frze. For each example, we first attempt direct answer generation using the base LLM (Line 3). If successful, we create [NO_ [NO_RETRIEVAL) training instances for both the planning and answering (Lines 4-6).
For examples requiring retrieval, we process each supporting document sequentially:
1. Convert document text to structured triples using frze (Line 12)
2. For intermediate documents (i n):
Construct input by concatenating previous subquery-graph pairs
18

---

Published as a conference paper at ICLR 2026
Algorithm 1 HotporQA-SUBQ Sample Labeling
Require: D: Filtered HotpotQA dataset
Require: M: Base LLM (LLaMA-2-7B)
Require: Seze: Text-to-triple conversion model
Ensure: Tplan: Training data for Planning
Ensure: Tans: Training data for Answering
1: Initialize Tplan, Tans < (,, 0
2: for all (Q, (do: .... d , A) do
3: At (QQ
Direct answer attempt
4: if' A = A then
5:
Tplan + Tplan U ((Q, (O__RETREVVAI)>
6:
Tans t- Tans A))
7:
continue
8:
end if
9:
subgo, ... subqn GenerateSubqqueries(( -...))
10:
Go. Gn + 0
Initialize graph contexts
for to do
12:
- f1z((d) if then
Convert text to triples
14:
input +- Formatinput( )z) Q)
15:
Tplan Tptan U {(inpul, [SURQ] subg:+1)}
else
17:
imput
Q) Talan +- Tptan U (input, (SUFFICIENT]] }
18:
19:
Tans + Tans U (inputt, A)}
20:
end if
21:
G; 4 G Ugi
Accumulate graph context
22:
end for
23:
end for return Tplan: Tans
Label with [SUBQ] and next subquery
3. For final document n)):
Include accumulated context
Label planning sample as [SUFFICIENT]
Create answering training instance with ground truth answer
The input formatting function (Lines 14 and 17) follows the template:
[SUBQ]
Retrieved Graph Information: go [SUBQ] in
Retrieved Graph Information: gn
Question: Q
This process generates two datasets:
Trains the planning ability to determine retrieval needs and generate targeted sub- queries
Tptan:
Tans:
Trains the answering ability to synthesize final responses from accumulated graph context
Table illustrates the distribution of our generated datasets. The planning dataset demonstrates balanced distribution across three label types: 35% [SUFFICIENT], 55% [SURQ], and 10%
19

---

Published as a conference paper at ICLR 2026
[NO_ RETRIEVAL]. The answering dataset incorporates both no-retrieval and sufficient cases from multiple sources. The size of the answering dataset exceeds the combined total of no-retrieval and sufficient cases from the planning dataset for two reasons: first, incorporated additional data from ASQA and Arc-Easy datasets; second, we included no-retrieval cases that were initially filtered out in step of process. This comprehensive approach ensures robust training set for the answering component.
Our labeling approach ensures that models learn both the iterative nature of complex question answering and the importance of structured knowledge representation. The complementary training objectives help develop robust reasoning capabilities while maintaining retrievability of supporting evidence.
C.4 DATASET STATISTICS
Planning Data
Answering Data
Queries
129,902
78,164 (w/ 3,897 ASQA & 2,147 Arc-Fasy) (475, 466. 2,214) (13,4, 2,332)
Input Tokens
(Mean, Median. Max)
(338. 301. 1,910)
Output Tokens
(Mean, Median, Max)
(13, 12, 62)
Subqueries
(Min, Mcan, Max)
(0, 0.8,
(0, 1.2, 6)
[SUFFICH ICIENT)
45,722
[SUBQ]
71,676
[NO_ RETRIEVAL]
12,504
[SUBQ] Tokens
(Min, Mean, Max)
(6, 18.3,
Nodes
(Mean, Median, Max)
(11.6, 11, 56)
(11.7, 11, 84) (21.3, 20, 160)
Edges
(Mean. Median. Max)
(21.2, 20, 116)
Table 4: Statistics of our constructed HotDotQA-SUBQ dataset.
C.5
DATA EXAMPLES
"Input":
You are a planner to determine if the question can be answered with current information and output the appropriate label as well as the subquery needed.
question can directly answered Outpui (SUBQ) with an subquery for retrieval if still needs a subquery.
Outpui (ND RETRIEVAL]
question ilself without any retrieval.
Output [SUFFICIENT] the question can be enswered with the provided information.
Question: Given a chat history sepersted by new lines, generates an informaivve, knowledgeabble and engaging response, Inputtrinl love pizza. Whie it's basically just cheese and bread you can top pizza with vegelables, meat etc. You can even make it without cheesefYnPizza is the greatest food everl I like the New York style. Inl so toppings can be folded hantsunnblll am not that big fan Chicago deep disn though.".
like that
is only thick crisp the edge, but soft and thin middle
"label": "
Figure 8: Training Data Example (Planning [NO_ RETRIEVALL)
"Input":
planner
question can
information output appropriate
subquery needed,
Output (NO RETRIEVAL] if the question can be directly answered with the question itself without any retrieval. subquery for retrioval if still needs subquery.
Output [SUBQ]
Cutput [SUFFICIENT]
question can answered the provided information.
[SUBQ] Where The Pick Motor Company Limited located?
Retrieved Graph Information: [(S>
Company Limited] Alias| New Companyy, Company Limited!
Ps Location] Oa Stamford, Lincalnshirej, (SS Pick Motor Company Limited] Ps Operational period] O+ 1889-1925), (S> Pick Metor Company Limited] P> industryl O> Motor vehicle manufacturing]]
Question: The Pick Motor Company Limited is located in town on which river ?*.
'labe!" "[SUBQ] Which river Stamford located on?"
Figure 9: Training Data Example (Planning [SUBQ))
20

---

Published as a conference paper at ICLR 2026
"input""
You are planner to determine if the question can be answered with current information and oulpui the appropriate label as well as the
subquery if needed.
Ouiput (NO RETRIEVAL] the question
directly answered with the questoon itself without any retrieval.
subquery for retrivaal needs subquery. Output [SUFFICIENT] if the question can be answered with the provided information.
Output [SUBQ]
[SUBQ] gaming control
member
Retrieved Graph Information: [(S> Martin Hake| Ps Former posiiion Co Member of the United States House of Representatives", '(S>
Martin R. Hoke) P> Birth date) O> May 18, 19521, (Sp Martin R. Hake) P> Occupationi O> Politiciany, '(S> Martin R. Hokej Ps State) O> 'SS> Martin Hokel Nationality| O> American),, '(S> Martin Hoke] Party, Repubican), '(S> Martin Hokel Member O+ Ohio Casino Control Commission), (S> Martin R. Hoke] Ps Born] Oz 1952)]
[SUBQ] What
centrol board provides oversight of Ohio's casines?
Retrieved Graph Information: (*1S> Ohio Casino Control Commission| Function] Provides oversight the state's casinos)", YS> Ohio
Gasino Control Commission P> Location) O= Onioj. '(S> Ohio Casina Control Commission] Ps Type] Oo Gaming control boardy, (S> Ohia
Casino Control Commission P> Abbreviation| O= OCCC)]
Question: Martin R. Hoke, is an American Republican poltician, member of which gamnigg control board Ohio that provides oversight of the state's 's casinos?
"label": "[SUFFICIENT]"
Figure 10: Training Data Example (Planning (SUFFICIENT)
'Input":"
You are answerer given question and retrieved graph information.
[SUBQ] is subquery generated through reasening question.
graph information (SUBQ]
graph information we retfieved to answer the subquery.
(NO RETRIEVAL] means the question can be answered with the question itself without any retrieval.
The main question starts ("Question: V.. Please
question, subqueries retrieved graph information they helpful.
[NO]RETRIEST RETRIVVLL]
Question: Which won the Nobel Prize in Litersture in 1961, Ivo Andri Nicholas Pilegg??".
"label": "Ivo Andri'
Figure 11: Training Data Example (Answering [NO RETRIEVAL))
"Input":
You are a answerer given a question and retrieved graph information.
Each [SUBQ] is subquery generated through reasoning for the question. The retrieved graph information follows each [SUBQ] is relevant graph information we refrieved answer subquery.
[NO RETRIEVAL] means the question can be answered with the question itself without any retrieval.
The main question starts with l"Question: V.. Please answer the question, with subqueries and retrieved graph information if they are helpful.
[SUBQ] What type of athiete is Darold Williamso??
Retrieved Graph Information: [(S> Darold Willimmson] P> Nationality O> American)", (S> Darold Wiliamson|| P> Birth datel O+ February 18, 1963y, YS> Darald Wiliamson| Occupation) Track athlete]]
[SUBQ] What specific skills are included track and field events?
Retrieved Graph Information: [[S> Athietics] Ps Includes) 0= Road nunningy. '(S> Track and field] P> Includes] O+ Throwing]: ((5 Track and end field P> Categorised under Athleticy), (SS Track and field Includes] Oa Rurning). YS= Track and field] Venuel O= Stadium with an oval running track enclosing a grass fieldy, (S> Athletics] P> Includee] O Track and fieldy, (S5 Athletica P> includes) O+ Race walking', (S> Track and field] P> Based on skills of) O> Running): (S= Track and field Ps Based on skills Jumping], '(S= Track and field] Ps Based skills Throwing)]
P> Includes]
Jumpingy,
Athletics) Indudes]
Cross country running). (S>
Question: Darold Willimson is an athiste in what running and jumping sport?".
'label": "Track field'
Figure 12: Training Data Example (Answering - (SUFFICCENT)))
21

---

Published as a conference paper at ICLR 2026
D TRAINING DETAILS
D.1 TEXT-TO-TRIPLES MODEL TRAINING
For our text-to-triple conversion model, we fine-tuned a Flan-T5-Large model (Chung et al., 2024) and a MAMAA..--3B-Instiie model (Grattafiori et al., 2024) to transform raw text passages into structured knowledge triples. The model processes input text sequences up to 512 tokens in length and generates structured triples in standardized format "(S > subject] P> predicatel O> object)".
Our training dataset WikiOFGraph (Kim al., 2024a), curated by ChatGPT, comprises 5,851,776 text-triple pairs, with an additional 5,000 samples reserved for validation. Each training instance consists of a natural language text passage paired with its corresponding set of comma- separated triples. For example:
Text:
"William Gerald Standridge (November 27, 1953 April 12, 2014) was American stock car racing driver. He was competitor in the NASCAR Winston Cup Series and Busch Series."
Triples:
gerald standridge] Nationalityl American))
(S> William gerald standridgel P> Occupation| O> Stock car racing driver).
Willimm gerald standridge] P> Competitori O> Busch series).
(S> William gerald standridge] P> Compestori
Nascar winston cup series),
(S- Willam gorald standridge] Ps Birth date) O= November 27, 1953).
gerald standridge) P>
2014)
Figure Example of WikiOFGraph data
We implemented the training using the AdamW optimizer with learning rate of 2e-5, incorporating linear warmup and decay schedules. The training process ran for 500,000 steps with batch size of 32 per GPU and gradient accumulation over 4 steps. To optimize training efficiency and memory usage, we employed mixed-precision training using bfloat16 and applied weight decay at 0.01. The maximum source and target sequence lengths were set 512 and 256 tokens respectively.
For evaluation, we primarily relied on ROUGE-1, ROUGE-2, and ROUGE-L scores to assess the quality of triple generation. We supplemented these metries with custom triple matching accuracy measures that consider subject matching, predicate normalization, and object entity alignment. Validation metries were computed 5,000-step intervals throughout training.
The training curves for various metrics are shown in Figure 14, demonstrating steady improvement in the model's ability to extract structured knowledge from text. The training of this model was conducted on eight NVIDIA RTX 6000 with 48GB memory.
ROUGE-1
ROUGE-2
ROUGE-L
6.9000
0.7009
0.7000
0.6000
0.5000
03300
2,2000
6.1009
0.0000
0.1000
0.0000
0,0000
59k
100k
139k
2002
2014
3018
2014
4000
4558k
Smik
50k
1004
150k
450kk
100k
150k
2010
2010
2004
4000
2000
Step
Step
Step
Figure 14: Text-to- Triple Model (base model: Flan- T5-Lage) Training Curves.
22

---

Published as a conference paper at ICLR 2026
D.2 MULTITASK TRAINING OF PLANNING AND ANSWERING CAPABILITIES
We employ Graph LLM (Perozzi et al., 2024; He et al., 2024) as our foundational model architecture and utilize our constructed HotpotQA-SUUs -UBBQ dataset train a unified model capable of performing both graph-conditioned action planning and answer generation through multitask learning.
Since both tasks share an identical architectural foundation, as illustrated in Figure ??, we differentiate their functionality through specialized instruction sets. For Planning, we employ the following instruction template:
You are a planner to determine if the question can be answered with current information and output the appropriate label well as the subquery if needed. Output [NO_RETRA] RETRIEVAL] if the question can be directly answered with the question itself without any retrieval.
Output [SUBQ] with an subquery for retrieval still needs subquery. Output [SUFFICIENT] if the question can be answered with the provided information.
Figure 15: Instruction aN for Planning.
For the Answering, we utilize this distinct instruction set:
You are an answerer given a question and retrieved graph information. Each [SUBQ] is subquery we generated through reasoning for the question. The retrieved
graph information follows each [SUBQ] is relevant graph information we retrieved to answer the subquery.
[NO] [NO_RETRIEVAL] means the question can be answered with the question itself without any retrieval.
The main question starts with "Question: Please answer the question, with subqueries and retrieved graph information if they are helpful.
Figure 16: Instruction INSTans for Answering.
smoothed_LP
continusl
1.5
0.5
Time
15
Figure 17: Training loss comparison over one epoch. The plot compares RAS-7B (green) RAS-8B (orange) training trajectories. Two additional RAS-7B variants are shown: "continual" (yellow): continual learning approach where Answering training precedes Planning training, and "ptune" (purple): parameter-efficcee variant that only tunes graph tokens without LoRA. The lower and more stable loss curves of the standard RAS variants demonstrate the effectiveness of joint training with LoRA. We the last 100 steps for loss smoothing.
For graph encoding, implement Graph Transformer (Shi et al., 2020), selected for its robust capability in handling Our base language models comprise I.LMAA2-7BB and L.AMMA--8BB, chosen both maintain
connected graphs- a common characteristic of text-extracted triples.
23

---

Published as a conference paper at ICLR 2026
consistency with previous research (Asai et al., 2023; Lyu al., 2024b) and to investigate framework's performance scaling across different model capacities.
Our implementation of GraphLLM differs from G-Retriever (He et al., 2024) primarily due to the distinct nature of our graph structures. While G-Retriever operates on single interconnected graphs, framework processes multiple potentially disconnected subgraphs, each corresponding to different subqueries. To address this architectural difference, we adopt sequential encoding strategy: rather than encoding the entire graph at once, we process each subgraph individually using Graph Transformer, followed by mean pooling across all subgraph embeddings to produce the final encoded representation.
The training process utilizes 4 NVIDIA RTX 6000 Ada Generation GPUs, each with 48GB memory. We train all models for 2 epochs using a batch size of 2 and gradient accumulation steps of implementing peak learning rate of le-5 with 0.15 warmup ratio and 0.01 decay rate. The maximum sequence lengths are set to 300 tokens for generation and 2,500 tokens for input, with training conducted in BF precision.
E
EVALUATION DATASETS & METRICS
E.1 TEST DATASETS
We evaluate RAS on diverse benchmark datasets spanning short-form QA, closed-set tasks, and forgffrmm generation in the zero-shot setting, aligning with (Asai et al., 2023; Lyu et al., 2024b). Below we describe each dataset:
Short-form Generation Datasets:
-unffitered (TQA) (Joshi et al., 2017): A large-scale QA dataset containing 11,313 dresstion-aswwr pairs in our test set. The questions are sourced from trivia enthusiasts and cover diverse topics.
2WikiMultiHopQA (2WQA) (Ho et al., 2020): A multi-hop question answering dataset (with 12,576 samples in test set) that requires models combine information from multiple Wikipedia articles to answer questions.
PopQA (Mallen et al., 2022): A dataset focusing on questions about long-tail entities, containing 1,399 queries where the monthly Wikipedia page views less than 100. These questions test models' ability to handle queries about less common entities.
Closed-set Tasks:
PubHealth (Pub) (Zhang et al., 2023a): dataset for verifying health-related claims. Models must classify statements "true" "false" based scientific evidence. The dataset contains 987 test samples.
ARC-Challenge (ARC) (Clark et al., 2018): A multiple-choice science question dataset designed to be challenging for models, requiring multi-hop reasoning and background knowledge. The dataset contains 1,172 test samples.
Long-form Generation Datasets:
ASQA (Stelmakh et al., 2022): A long-form question answering dataset that requires models generate comprehensive answers with proper citation of sources. Models must balance information completeness with factual accuracy. The dataset contains 948 test samples.
ELIS (Fan et al., 2019): A dataset derived from the "Explain Like I'm Five" subreddit, containing questions seeking straightforward explanations of complex topics. The dataset contains 1,000 test samples.
24

---

Published as a conference paper at ICLR 2026
E.2 EVALUATION METRICS
We employ different evaluation metrics appropriate for each task category:
Short-form Generation:
For PopQA and TriviaQA, we use the "golden match" metric (Asai et al., 2023; Min et al., 2019; Guu et al., 2020)", where a prediction p considered correct if it contains any normalized version of the ground truth answers G {gi..... Jn
match(p, -0)=11
{
if 3g E G: norm(g) S norm(p)
(9)
otherwise
where norm(.) normalizes text by lowercasing, removing articles and punctuation.
For 2Wik,MuliHoopA, we follow RPG (Lyu al., 2024b)² to use token-level Fl score between prediction p and ground truth g:
(p. = 2. precision recall
(10)
precision + recall
where precision and recall are computed over normalized token overlap:
[tokens(p) n tokens(g)
precision
(11)
[tokes(
(token((p) n tokens(g)
recall
(12)
[token((g)
Closed-set Tasks: For both PubHealth and ARC-Challenge, accuracy (Asai al., 2023), computed as:
accuracy norm(p.) = norm(g:)] 100
(13)
N
where N is the total number of examples.
Long-form Generation: For ASQA and ELI5, we use multiple metrics
ROUGE-L score to measure the longest common subsequence between prediction and reference (Lin, 2004)
MAUVE score (Pillutla et al., 2021) to test the generation fluency by comparing the distribu- tion of generated text against human references
All scores are reported as percentages, multiplied by 100.
IImplementation aligned with http:///gttttu.oomkaaaaaaaaaaaaa ret r 1_11mmmmmmrrrrmme "Implementation aligned with http:://ggtttoo.ccc///rrrrraedes retri retriever wwwriiveesrr//eee ly 'Implementation aligned with http:////whhaaa....o..ooeocoooee eval.py
25

---

Published as a conference paper at ICLR 2026
F INFERENCE & EVALUATION DETAILS
We evaluated RAS using both closed-source and open-source language models. For closed-source evaluation, we used Claude-3.5-Sonnet (Sonnet-3.5) our base model. For open-source evaluation, we tested with both LLaMA-2-7B and LLaMA-3-8B, as shown in the performance table (Table 1).
For the open-source models, we first fine-tuned the GraphLLM architecture on our HotporQA-SUBQ dataset (see Section D.2). We then conducted zero-shot wwwllede-intensiin generation tests. With Claude-3.5-Sond 3.5-nnntt, we used few-shot prompting for both action planning and answer generation phases. Below, we detail our approach for each model type.
F.1 CLOSED-SOURCE MODEL SETTINGS
For text-to-Triple Conversion (Figure 18), we instruct the model to extract structured knowl- edge triples following specific formatting rules. The prompt precisely defines the triple format as (S > subject >> predicate O > object) and provides comprehensive guidelines to ensure consistent knowledge representation. Key rules include extracting maximal meaningful relationships, maintaining original entity casing, avoiding pronoun usage, and ensuring clear predicate specification. The example demonstrates the conversion of a biographical complex information about an individual's ' life, career, and temporal details can be systematically decomposed into atomic facts. This standardized format enables reliable knowledge accumulation and reasoning in subsequent stages of the RAS framework.
into structured triples, showing how
relationship triples given Each triple should have exactly one subject (S+), one predicate (P>), and one object (O>).
1. Extract as many meaningful triples as possible
Each triple must be in format: (S- subject| P> predicate| O> object)
triples should
Avoid using pronouns (Uhe/she) - always use actual names
Keep all entries their original case (uppercase/lvercasse)
Make predicales clear specific
input only entity, output ENTITY| is] ENTITY) 8. [IMPORTANT] Do not include any other text the output, only the triples or the entity (for the Rule case).
entiy input.
Example Input:
"William Gerald Standhidge (November 27, 1953 - April 12, 2014) was an American slock car racing driver. He was competitor in the NASCAR Winston Cup Series and Busch Series."
Example Output:
(S> William gerald standridge| P> Nationalityl D> American).
William gerald standridge) P> Occupation) O= Stock racing driver),
William gerald standridge] P> Competior]
Busch series). William gerald standridge) P> Compettori O- Naszar winsion cup saries).
(S>
(S> Willaam gerald standridge) P> Birth date) November 27, 1953), 2014)
Wiliamm gerald standridge)
date]
Input Text:
[text]
Figure 18: Few-shot prompt for text-to-triples transformation with closed-source LLM.
For Planning (Figure 19), we design a comprehensive prompt that guides the model in making strategic decisions about information retrieval needs. The prompt instructs the model to output one of three labels based on careful analysis of the current knowledge state: (1) [NO_ RETRIEVAL) when the question can be answered directly, either due to the model's 's inherent knowledge or the question's 's nature, (2) [SUBQ] accompanied by a focused subquery when additional information needed, or (3) DSUFFICCEET when the accumulated knowledge is adequate to answer the question. The prompt includes diverse examples demonstrating different scenarios:
(I) Generating follow-up queries based on partial information (2) Creating queries when relevant information is missing (3) Recognizing when accumulated information sufficient (4) Identifying questions that don't require external knowledge (5) Handling common knowledge questions
key feature the prompt is its emphasis on query efficiency it explicitly prohibits generating redundant subqueries that might retrieve already-knows information. This design helps maintain the system's is efficiency while ensuring comprehensive knowledge gathering.
26

---

Published as a conference paper at ICLR 2026
are planner determine the question can be answered with current information (Subquery PPREE_SBBQ retrieved graph information (PREV GRAPH INFOJ) and output the appropriate label as well as the subquery neaded.
Output [NO RE RE the question can be directly answered with the question Itself without any retrivval. You expected to output knowledgeable enough answer the question, you believe question type is
INO RETRIEVAL]
you believe an
suilable for retrivval
Outgut [SUBQ] with an subquery for retrieval if still needs subquery. Do not make an simiar subquery that has been made before information.
(PREV_ SUBQ)),
Output [SUFFICIENT] very likey retrieve question can be answered with the provided information. The main question starts with "Queslion:" :
Examples:
Example (Use information (PREV_ GRAPH INFO) further gemeraae new subquery):
Input:
[PREV SUBQ] Where The Pick Motor Company Limited located?
[PREV_ GRAPH_ING [IFO] [TS> Motor Company Limitad P> Allsi O= New Motor Company", Pick Metor Company Limited]
Location Stamford, Lincolnshirej, (S= Moter Company Limited] Operational period Os 19255,, '(S> Company
Limited] P> Industry) O> Motor vehicle manufacturing)]
Question: The Pick Motor Company Limited is located in town which river ?
Output:
[SUBQ] Which river is Stamford located on?
Example (No relevant information found [PREV_ __PAPHGGR____ GRAPH_INFO]]:
Input:
[PREV_ SUBQ] What madals did Michael Johnson
the 1996 Olympics?
[PREV GRAPH INFO] [(S> Michael Johnson] F> Nationality O= American",, '(S> Micheel Johnson| Ps Birth datej O September 13, 1967),
(St Michael Johnson] P> Sport] O= Track and field', (S> Micharl Jahnson] Ps Teamn] O= United Stales Olympic leam)] Question: What was Michael Johnson's winning
400m
Olympics?
Output:
[SUBC]
times
Johnson the the Olympic Games?
Example (The current information is sufficient
the question):
Input:
[PREV SUBQ) What gaming control board in Ohio is Martin R. Hoke member of?
[PREV GRAHH GRAPHLINF]
Martin R, Hoke P> Former position) O> Member of the United States House of Rapresentatieel" 'SS Martin R. datel May 18, 1962y, (S- Martin Martin R. Hake] Ps Nationaltyy O= omerrcann. (S= Martin R. Hoke) P> Partyl O= Republican), '(S> Martin R. Hake| P> Member ofl O+ Ohio
Occupation| O= Polilician..
Hokel Ps State] O= Ohioj, '(S>
Casino Control Commission), YS- Martin R, Hokej P> Born] O> 1862y]
[PREV SUBQ) What gamnng control board provides oversight Ohio's casinos?
|PREV GRAPH INFO] (MS Ohio Casino Control Commission] P> Function] O> Provides overeight of the state's casinos]"" '(S> Ohio Casino
Control Commissionn| P> Location) O= Ohio), YSS Ohio Casino Contral Commission] P> Type] O Gaming control boardy. (S> Ohio Casino
Control Commission| P> Abbreviation) O> OCCCJ]
Question: Martin R. Hoke, is an American Republican politccian, member of which garring control boerd in Ohio that provides oversight of the state's la casince?
Output:
[SUFFICIENT]
Example (The question is not suitable for retrieval)::
Input:
chat history separated by new lines, generates informative, knowledgeabllc and engaging response. ##input:
love While it's basically just cheese and bread style.
with vegetables, meat You make without cheese!
greatest food everl like
do too. Iike that the crust is only thick and crisp at the adge, but soft and thin the middle so its toppings can be folded half. Absolutely! am not that big of fan of Chicago deep dish though
Oulput:
[NO_RETTRE RETRIEVAL]
Example (You are knowledgeable enough to answer the question):
Input:
Whatis capital the United
Output:
IND RETRIEVAL]
[VERY IMPORTANT] Please only ether output [NO] RETRIEVAL] (2) [SUBQ] with concrete subquery retrieval, (3) [SUFFICIENT] the question can be answered with the provided information.
[VERY IMPORTANT] Do not outpui any other text. DO NOT make an identical subquery [SUBC] that has been made before (PRREV_UBQ]j!
Now, your
Input:
(planner_ [
Output:
Figure 19: Few-shot prompt for planning with closed- source L.M..
27

---

Published as a conference paper at ICLR 2026
For Answering (Figure 20), design prompt that focuses synthesizing precise answers from structured knowledge graphs. The prompt emphasizes selective use of retrieved information instructing the model to utilize subqueries and graph information only when relevant to the question at hand. A distinctive feature of this prompt is its requirement for definitive answers even under uncertainty incomplete information, ensuring the model always provides response.
The prompt includes carefully selected examples demonstrating two key scenarios: direct fact retrieval (Nobel Prize winner) and complex reasoning across multiple knowledge pieces (athlete's sport classification). These examples illustrate how to effectively combine information from multiple subqueries and their associated graph information to construct accurate answers. The prompt strictly enforces concise answer generation, requiring only the essential information without additional explanation or commentary.
You are answerer given quesiion and retreved graph information. Each [SUBQ] subquery ganerated through reasoning question. retrieved graph information [SUBQ] graph information we retneved answer subquery.
The main question starts with "Question:" Please anewer the question, with subqueries and retrieved graph information if they are helpful (do them they helpful). must enewer question,
enough
answer question, you are enswer.
Examples:
Example 1:
Input:
Question: Which person won the Literature 1961, or Nicholas Pllegg??
Output:
Exampie 2:
[SUBQ] What type athiete is Darold Willimson?
Retrieved Graph Information: [(S> Darold Wiliamsonn P> Nationality O+ Americanj), ((S Daroid Williamson| Ps Birth datej O= February 19,
1983y, (St Derold Willimmonn P> Occupation O+ Track sthlete]]
(SUBQ)
field
Retrieved Graph Information: [(S> Athletics P> Includes] Oh Road running). (S> Track and field Ps Includes) O+ Throwing), (S> Track and field] P> Includes] O> Jumpingy, (S= Athletics P> Indludes O> Cross country running): (S> Track and field] Ps Categorised under] O= Amlaticay, '(S> Track fiel Includes) Oz Running), YS>
Vanuel Stadium gress field', (S* Athletics] P> Includes/ O> Track and fieldy, (Sa Athietics) Includee) O= Race walkingy, 'SS Track and fiedd P> Based on skills off O= Running): (S> Track and field] Ps Based on skills of O> Jumping). ((S> Track and field] Ps Based on skills off O> Throwing]]] and jumping sport?".
running track enclosing
Question: Darold Williamon is athlete what
Output:
Track and field
[VERY IMPORTANT] Please only output the answer the question. IMPORTANT] output any other
Now. your tum:
Input:
(answerer Input)
Output:
Figure 20: Few-shot prompt for answering with closed-source LLM.
In addition, for ASQA and ELIS with closed-source models (RAS and all the other base- lines (e.g., SuRe) using Sonnet-3.5), we conduct few-shot inference with two in-context learning demonstrations, aligning with previous study's 's (Gao et al., 2023) implementation." 56
'https: //gi http://ww.o........oooo////////o 'Exemplars he found at https: prompts
28

---

Published as a conference paper at ICLR 2026
F.2 OPEN-SOURGE MODEL SETTINGS
For inference with open-source model, we use our GraphLLM trained by Hotpot-SUBQ (see Ap- pendix D.2), we use 8 NVIDIA RTX 6000 Ada Generation with 48GB memory and CUDA version 12.4. We use Python 3.10, PyTorch 2.1.2, and torch-geometric 2.6.1 I throughout all experiments.
For more efficient text retrieval, we split the dense index of the corpus into eight splits, and load them on eight GPUs with faiss-gpu 1.7.2.
We maximum new tokens 100 for PopQA, TriviaQA, and 2WikiMultihopAA, as 50 for PubHealth and ARC-Challenge, and as 300 for ASQA and ELIS, aligning with previous study (Asai et al., 2023; Lyu et al., 2024b). All the generation are configured with batch size of
As we always set the main question as the first subquery, we pre-extract the triples from the context the first iteration using our text-to-triple model for batch triple extraction.
We use exact same instructions for planning and answering as we used in training phase, as shown in Figures 15 and 16.
F.3 INSTRUCTIONS
We use the following task-specific instructions for zero-shot inference:
Dataset
Instruction
ARC-C (baselinc)
Given four answor candidates, A. C Cand D, choose the answer choice. Please answer with the capitalized alphahet only, without adding any extra phrase or period.
ARC-C
Which true? Output B, C, D.
PubHealth (baseline)
Is the following statement correct or not? Say true if it's correct, otherwise, say false. Don't 't capitalize or add periods, just say "true" or "false".
PubHealth
Is statement 'truc' or 'false' ? Only octput 'truc' or 'false'.
ASQA (baseline)
Instruction:
ACCURATE, ENGAGING, CONCISE answer for given question using retrieved graph information (some of w which might be irrelevant). Use an unbiased and journalistic tone.
ASQA
Answer the following question. question may anbiguous and muliiple correct amswers, and in that case, you have to provide a long-form answer incloding all correct answers. [ong Form]
(huseline)
Instruction:
given question using the retrieved graph information (sorme of which might be irrelevant). Use an unbiased and journalistic tone.
ACCURATE, ENGAGING,
CONCISE answer
ELIS
Provide a paragraph- lengghh response using simple words to answer the following question. [Long Form]
Table 5: Task-specific instructions. For short-form QA, we do not use instructions and use the original questions only.
G COST & EFFICIENCY COMPARISON WITH OTHER GRAPH-BASED RAG METHODS
Static vs. Dynamic Structuring. Static corpus-level graph methods such as GraphR (Edge et al., 2024) and HippoRAG (Gutiérrez et al., 2024) require building and maintaining large knowledge graphs offline. This involves millions of LLM calls for entity/relation extraction and summarization, plus heavy clustering and embedding computation. At the scale of Wikipedia 2018 (roughly 5M articles, ~3B tokens), these pipelines ineur substantial costs, days of GPU/CPU compute, and 250 GB storage overhead. In contrast, our RAS constructs question-specifie knowledge graphs dynamically at inference, eliminating expensive offline indexing. The only extra step is lightweight triple extraction from retrieved passages, which we implement with a small LLaMA-3B model accelerated by vLLM (Kwon et al., 2023), achieving throughput of nearly 5K tokens/s (Table 3).
Cost Estimation Methodology. We estimate costs based on empirical data from Microsoft's s GraphRAG analysis 8 and scale Wikipedia 2018. Cost estimates assume current API pricing and may vary significantly based on implementation details, model selection, and optimization strategies.
AAl the Wikipedia dumps (2018, 2020) we used downloaded from https: / / homb.com/ fa facebook reeeookkeeearcc//tt...
$https: / httt://///ccccccctt.....co www.oot..roooooooomoommmooooo.. com wttp://////oaaaaeeaaaaaaoooooooa
graphrag costtrag-coostsoooo costs- wnd -o
29

---

Published as a conference paper at ICLR 2026
Computation of Indexing Cost. We estimate costs using the following assumptions:
Corpus size: Wikipedia 2018 = 3B tokens, =5M articles with ~600 tokens/article.
GraphRAG pipeline: Based on Microsoft's empirical analysis showing $0.000011 per word for GPT-4o-mini.
For 3B tokens (=2.25B words), base processing cost: $25k-$30K
Entity/relation extraction with few-shot prompting increases token usage by 3-5x: S75k- $150k
Community summarization for ~10k communities at $0.02 per community: $200-$500
Additional overhead (clustering, embeddings, storage): 310k-225k
Total GraphRAG: $85k-$175k for GPT-4o-mini; $250k-$500k for GPT-4o
HippoRAG pipeline: More efficient extraction but still corpus-wide processing.
Entity extraction: 3B tokens at reduced complexity = 315k-$30k
Synonym linking and embedding overhead: $5k-S15k
Graph construction and indexing: $5k-$10k
Total HippoRAG: $25k-$55k
RAS pipeline: No offline corpus-wide wide indexing. Triple extraction uses local LLaMA-3B model.
Offline indexing cost: $0 (no corpus preprocessing required)
Runtime triple extraction: Local model inference (negligible marginal cost)
Total RAS: $0 offline costs
Cost Sensitivity Analysis.
Costs vary significantly based
Model selection: GPT-4o costs =3x more than GPT-4o-mini
Chunk size: Smaller chunks (300 1200 tokens) can reduce costs by 30-50%
Prompt optimization: Well-tuned prompts can reduce token usage by 20-40%
Processing parameters: Conservative estimates assume non-optimized settings
Quantitative Comparison. Table summarizes estimated costs with uncertainty ranges.
Method
Offline LLM Cost
Compute Time
Storage
Ready for RAG?
GraphRAG
155k-$500k (model-deppndent))
>7 days (distributed)
50-70 GB
Yes (community reports)
HippoRAG
$25k 225k-S55k (optimized extraction)
>3 days (cluster)
30-50 GB
Yes (node-passes paode-passage matrix)
RAS (ours)
Negligible
Inference-time only
Small (triple eache)
Yes (dynamic graph)
Table Estimated cost comparison for building knowledge graph indexes at Wikipedia 2018 scale. Ranges reflect model choice, optimization level, and implementation variations.
Efficiency Visualization. We visualize the contrast in offline indexing cost ranges. RAS requires no precomputation, while GraphRAG and HippoRAG incur substantial upfront costs.
Indexing (USD)
Takeaway. RAS achieves the structured reason-
ing beneñits of graphs without any offline index-
106
10s
ing costs, compared to GraphRAG ($85k-$500k)
104
and HippoRAG ($25K-$55k). By eliminating up- front investment requirements and enabling pay-per- query scaling through local model inference, RAS
103
102
makes graph-enhanced RAG accessible for scenar-
GraphR HippoRAG
RAS
ios where large-scale preprocessing is impractical or cost-prohibitive.
Figure 21: Estimated offline indexing cost ranges at Wikipedia scale (log scale).
30

---

Published as a conference paper at ICLR 2026
H SUPPLEMENTARY RESULTS AND ANALYSIS
Higher Performance with More Contexts?
We further examine the performance of Sonnet-3.5 when increasing the number of retrieved docu- ments. Table 7 compares Sonnet-3.5 under different retrieval depths with RAS-Sonnet.
Method
2WikiMulahopOA (FI)
PopQA (Acc)
PubHealth (Acc)
ARC (Acc)
Sonnet-3. 3.5 (#docs=5)
53.7
57.3
53.9
87.1
Sonnet-3.5 (#docc=11)
55.1
58,5
56.6
87.9
Sonnet-3.5 (5docs=15)
55.4
58.0
86.9 56.9
RAS (Sonnet)
57.7
62.3
71.4
93.8
Table 7: Performance comparison between Sonnet-3.5 under varying context sizes and LAS-Sonnet.
Increasing retrieval depth yields diminishing returns for Sonnet-3.5, while RAS-Sonnet achieves significantly higher performance with structured, targeted context. This highlights the limitations of passive context scaling and the benefits of RAS's organized retrieval strategy.
Error Analysis on Graph Construction
We present two representative cases where the quality of retrieved structured knowledge significantly impacts final answer generation. These examples are distinct from the qualitative examples shown in Section and highlight both successful and failed reasoning paths.
Case 1: Accurate Reasoning with Well-Structured Graph
Question: Who directed the film adaptation of the novel "Tinker Tailor Soldier Spy"? Subqueries and Graphs:
[SBBQ] Who di rected the film "Tinker Tailor Soldier Spy"? (S> Tinker Tailor Soldier Spy P>> Film adaptation director O> Tomas Alfredson),
(S> Tinker Tailor Soldier Spy P> Release year O> 2011)
[SUBQ] Who is Tomas Alfredson?
(S> Tomas Alfredson P> Occupation O> Film director)
Answer: Tomas Alfredson (Correct)
Analysis: The subquery identifies the correct adaptation and its director. The structured triple precisely encodes this relation, allowing the model to reason accurately.
Case 2: Reasoning Failure Due to Spurious or Missing Triples Question: What invention is Nikola Tesla best known for? Subqueries and Graphs:
[SUBQ] What are Nikola Tesla's major inventions?
(S> Nikola Tesla P> Known for O> Farthquake machine),
(S> Nikola Tesla P> Proposed O> Death ray).
(S> Nikola Tesla P> Created O> Tesla coil)
[SUBQ] What is a Tesla coil used for?
(S> Tesla coil P> Used in O> High-frequency experiments)
Answer: Earthquake machine or death ray (Incorrect)
Analysis: The graph omits the key triple (Nikola Tesla Invented Alternating current system), instead surfacing fringe inventions. This misleads the model into emphasizing less impactful or speculative work, leading to an incorrect answer.
These examples reveal that RAS makes reasoning pathways more interpretable by structuring retrieved knowledge explicitly. When failures occur, they can often be traced to identifiable issues in the extracted triples, such missing key relations overemphasis fringe facts, rather than opaque
31

---

Published as a conference paper at ICLR 2026
model behavior. This transparency not only facilitates analysis but also suggests concrete avenues for improving upstream components like triple extraction or subquery generation, making RAS flexible foundation for future refinement.
I QUALITATIVE EXAMPLES
In this section, show some qualitative examples and running examples of RAS.
Example 1 (PopQA)
[SUBQ] Who the screenwriter for The Fake?
Retrieved Graph Information:
(S> The Fake
P> Release year O> 1953),
(S> The Fake P> Genre O> British crime film),
(S> The Fake
P> Director O> Godfrey Grayson),
(S> The Fake
P> Stars | O> Dennis O'Keefe),
(S> The Fake
P> Stars O> Coleen Gray).
(S> The Fake P> Stars O> Hugh Williams),
(S> The Fake P> Plot concems O> American detective solving theft of priceless painting),
(S> Priceless painting P> Stolen from O> Tate Gallery),
(S> Tate Gallery| P> Located in O> London)
[SUBQ] Who wrote the screenplay for the 1953 British crime film The Fake?
Retrieved Graph Information:
(S> Antony Carr P> Occupation O> Author),
(S>
Antony Carr P> Genre O> Crime novels).
Antony Carr P> Birth year 1916).
S> Antony Carr P> Death year O> 1995).
S
Antony Carr P> Nationality O> English),
Antony Carr P> Father Robert Carr),
(S> Robert Carr P> Occupation O> Singer),
(S> Antony Carr P> Niece O> Susannah Carr).
Susannah Carr P> Occupation O> Newsreader),
(S> Susannah Carr P> Nationality O> Australian)
[SUBQ] wrote script or screenplay the 1953 British crime The Fake directed by
Godfrey Grayson?
Retrieved Graph Information:
(S> The Fake P> a O> British silent drama film),
(S> The Fake P> Release year O> 1927),
(S> The Fake P> Directed by O> Georg Jacoby),
(S> The Fake P> Stars O> Henry Edwards).
(S> The Fake P> Stars | O> Elga Brink).
(S> The Fake P> Stars O> Juliette Compton),
(S> The Fake P> Based
O> Play of the same title),
(S>
Play of the same title P> Written by O> Frederick Lonsdale),
(S>
Play of the same title P> Release year O> 1924),
The Fake P> Made at O> Twickenham Studios),
Twickenham Studios Located O> London)
Question:
Who the screenwriter for The Fake?
Answer: Frederick Lonsdale
Figure 22: An inference example in PopQA dataset.
32

---

Published as a conference paper at ICLR 2026
Example 2 (TriviaQA)
[SUBQ] Who is the most successful UK solo artist in the USA?
Retrieved Graph Information:
Most successful male UK urban artist US chart history). (S> Jay Sean P> Song reached number one O> Down),
(S> Jay Sean P> Achievement
(S> Jay Sean P> Chart topped O> Billboard Hot 100).
(S> Taio Cruz P> Topped chart O> US Billboard Hot 100).
(S> Cliff Richard P> Claimed worldwide sales O> 260 million).
(S> Cliff Richard P> Ranking O> Second highest selling British male solo artist worldwide).
(S> Elton John P> Ranking O> Highest selling British male solo artist worldwide),
(S> Cliff Richard P> Backing band O> The Shadows).
(S> Robbie Williams P> Chart topper O> Rock DJ),
(S> Robbie Williams
P> Collaboration with O> Kylie Minogue)
[SUBQ] What are the US chart records or album sales figures for Elton John in the United States? Retrieved Graph Information:
(S> Elton John P> Plays O> Piano),
(S> Elton John P> Performs O> Vocals),
(S> Dee Murray P> Plays O> Bass),
(S> Nigel Olsson P> Plays O> Drums),
(S> Davey Johnstone
P> Plays O> Electric guitar),
(S> Paul Buckmaster P> Arranges O> Orchestral arrangement).
(S> Bob Birch P> Plays O> Bass guitar),
S Charlie Morgan P> Plays O> Drums),
SS Charlie Morgan P> Plays O> Percussion).
Guy Babylon P> Plays
Keyboards).
(S> Matt Springer P> Ranked O> Elton John's studio albums).
(S> Ultimate Classic Rock P> Employs O> Matt Springer),
(S> Daryl Hall & John Oates P> Nationality O> American),
SS> Daryl Hall & John Oates P> Sold albums in US O> 13 million),
(S> Daryl Hall & John Oates
P> Sold singles in US O> 6 million)
Question: Who is the most successful UK solo artist in the USA?
Answer: Elton John
Figure 23: An inference example in TriviaQA dataset.
33

---

Published as a conference paper at ICLR 2026
Example 3 (ASQA)
[SUBQ] Who seng the song I'm corning out?
Retrieved Graph Information:
(S> Diana Ross Sang O> Coming Out),
(S> Bernard Edwards [P> Wrote O= I'm Coming Out).
Nile Rodgers P> Wrote Oz Coming Out),
(S> Bernard Edwards P> Produced O> I'm Coming Out).
(St Nile Rodgers Pb Produced Oz I'm Coming Out),
Coming Out Released | August 1980).
(S> I'm Coming Out Ps Second single from album O= Diana),
Diana Ross P> Commissioned O> Chic).
(S> Stevie J P> Sampled OO I'm Coming Out).
The Notorious B.I.G. P> Used sample
O= Money Mo Problems),
Hines P> Covered
Coming Out)
Keyshia Cole P> Remade O I'm Coming Cut).
lggy Azslea
O> I'm Coming Out),
(S>
Arlana Grande P> Sampled in song O> Break Your Heart Right Back),
Hella Sinnen P> Sampled Oo I'm Coming Out),
(S>
Ameria Ps Covered O> Coming Out)
[Long Form]
seng song coming out?
recorded by American singer Diana Ross. written and produced Chic members Berarr Edwards and Rodgers, and released on August 22, 1980, as the second single from Ross' self-tted terth album Diana (1980). 2003, American singer Amerie covered "I'm Coming Out for the soundtrack the film "Maid in Manhattan", Her version single select European
Ground Truth: "I'm Coming Out"
and
"Imm Coming Out song by American singer Diana Ross. It written Bernard Edwards and Nile Rodgers and produced by Edwards and Rodgers. The song was released on August 22, 1980 as the second single from Roes's album Diana. The song was sampled by Stevie in his 2005 song "U Don't Know and by The Natonous B.I.G. in his 1997 song "Mo Money Mo song
Generation:
album You. In 2014, lagy Azales featured on remake of the song. Ariana Grande sampled the song in her 2015 song "Break Your Heart Right Back".
Problems". Marcia Hines covered song
1981
Keyshia Cole
Example (ELI5)
[SUBQ] What stock 'pu'' and how does it work?
Retrieved Graph Information:
(S> Put option P>> ls| Financial contract),
Put option Gives buyer O+ Right to sell underlying asset),
(S* Put option P> Has O+ Predeteminedd price).
option
Expiration date),
(S= Put option P> Can be associated with Q= Stocks),
(S> Put option P>> Can be associated with O= Commodities),
Put option buyer P>
obligated to
contract price).
(St Put option| P> Used for Q+ Hedging)
Put option buyer P> Bets O> Stock decine)
(So Put option P> Has O= Defined risk)
[Long Form) Question:
stock 'put' how
price. effectively means of protecting stock you own, or for belling against a stock. Let's say you buy an contract for August 140 puls on Apple. That that the date August options expire. you would be able sell 100 shares $140 you choose. So if Apple drops to $125 by that time, you'd protect 31 15/hhare gains. This might only cost you $1-2 per share buy the contract. stock will fall, the put will increase in value as the strike price approaches "in the money." So that $1-2/sh, spend $10/share the there is set expiration date, the values have relationship to both the underlying stock and the time until the expiration. So a month the option that's $10 the money about the difference between option value and stock itseit.
a type stock option, grants you option
specified
you simply think the the
option might
dropping there's greater likellhood
the
put, stock trading at $130) might $20 while the day of expiration it'd be
contract gives buyer right to underlying expiration date. The put option buyer is not obligated to seil the asset the contract price, but they can bet on the stock price decinee.
Generation: put option
predetermined price
Figure 24: Examples of form generation ASQA and ELI5.
34

---

Published as a conference paper at ICLR 2026
What religion did Marie Curie practice after marniage Plerre Curie?
Iteration
for frrst tevotinn Kreoiieee' nocded, we use the main query itett as the subouery) Subquery En
None
INSTT
None
Planning
[SUBQ] raligion Mario Curle practice after hor marriage to Plarre Curie?
Retrieval
Retrieved Documents fo
Dect
mit shuo vou Flos Pnserrmnntinn ui coloodd
Graph ge
mnlancenn Senverr science la's wilicch ab mibrsiy agristic
(Meee Fheme wxcding 1995
Douse wici rouiin wailaa aceribic acis."
Moria Pormi sttoddeekat) agoune spectii
Dec
feet Corter whethheeee
(Wais curis noriec Ho chiceseity jncaes noterialan)
Autlownn feo demoou ohence doriigh core
Evolving KG Gq
hvardveww thenong
dece Aurarity (i shneggiee hern the parci buseeeete stulled
(F-porr Casa jassadetto
Enrich
Jouns tamily locua ao snalic wort
Structuring
jOure family sriccal loundation pursone
paas sollatonn Damea Youris
INST
1oo gel Q
Subquery
Planning
[SUBQ] Did Marie and Pierre Curie participate any religicus
Retrieved Documents
fh oomm hurtes wetenee werh Paeer mnter Centaato pwe rour sanes wasl
Extracted Graph Ø1
(Dari deughtes inapiinn drnal tore. paens coneron steent intuutor
absenstion Spaiis
cww.mnattaametttooo to Drack
recoids.
Evolving Gg
Oweeleen Pheaa bost on rear Cuse homel
Dec
"Th Cultrde unditur
aolern the duales
Palis in ci vochcoeaccionn canenani cunaan)) Care tinss
opplicc
Enrich
(urbooss Cheers inhessasiioi jemouded solotios)
(Fee
Structuring
motort aenumassal recll ding Seile Feets"
Iteration
Answer
P
After her monilge Pioree Qutlc in 1595, Muric
Curee precited no nelgion, This concurieeyy demonationed by two koy ploces of evidenco Fiise, Cathees Church records deda
Gg
Go
INST
[qp: d:)
druggheess nwaet baprlowd, which highly unsouss their sccial shanding time.
Planning
[SUFFICIENT]
aap
Answering
Sccond, the home life ceplic bhy sccular 2iay helr a the wedding canemany and maints mect thoroughty socular householld spinncing Bundays tabonatary by rither than church services.
Figure 25: A Running Example of RAS.
35

---

Published as a conference paper at ICLR 2026
J HYPERPARAMETER STUDY
Table 8: study. We highlight the setting used experiments.
Hyper-parameter
Studied Values
Text-to-Triple Triples Model
Batch size
{2, 4, 8, 16, 32)
Learning rate
(1e-5, 2e-5, le-4, 2e-4)
GraphLLM (Action Planner Answerer)
GNN Setting (for Graph Token]
GNN architecture
[GCN, GAT, Graph Transformer ]
Hidden dimension
(512, 768, 1024, 2048)
Number of layers
[2, 4, (4, 6, 8, 12)
Number of heads
Dropout
{0.05, 0.1, 0.2, 0.3)
Projector intermediate dimension
10024, 2048, 4096]
Projector output dimension
4096
LoRA Setting
LoRA rank (r)
[4. 8, 16. 32) {8. 16, 32) (0.01. 0.05, 0.1. 0.15)
LoRA alpha
LoRA dropout
General Setting
Learning rate
(1e-6, 2c-6, 5e-5, lc-4) (2, 8, 16, 32)
Batch size (training)
Batch size (inference)
Weight decay
(0.001, 0.01, 0.05, 0.1)
Gradient accumulation steps
[2, 4, 8, 16] [0.1, 0.3, 0.5, 1.0)
Gradient clipping
Warmup ratio
[0.05, 0.1, 0.15, 0.2)
Max text length
2500
Max new tokens
Task-specific (see Appendix F.2)
Others
Dense retrieval top-k
5
36

---

Published as a conference paper at ICLR 2026
K BROADER IMPACTS, SAFEGUARDS, AND ASSETS
Broader Impacts. The RAS framework presents several promising impacts for both research and practical applications. By transforming unstructured retrieved content into structured, question- specific knowledge graphs, RAS improves reasoning transparency and factual reliabilit-qqulitie that are especially valuable education, scientific research, and technical writing. The ability to explicitly organize retrieved knowledge into interpretable structures also opens up new possibilities for human-in ioitaa--i-th---oop Al systems, enabling users trace and verify model reasoning paths.
However, several ethical considerations remain. RAS's s reliance on structured knowledge representa- tions means its effectiveness may be limited in low-resource languages or domains with sparse data. Additionally, biases present in training data can be preserved or even amplified in structured forms. Addressing these concerns will require rigorous evaluation across diverse user groups and careful curation of source corpora. Promoting faimess, transparency, and inclusivity should be core design goals for future RAS- RAS-based systems.
Safeguards. To mitigate potential harms, propose several safeguards during the development and deployment of RAS-based systems. First, developers should routinely audit training and retrieval corpora for biases and factual inconsistencies, and apply data augmentation counterfactual genera- tion techniques where appropriate. Second, regular evaluations should be conducted across a diverse range of tasks, languages, and user demographics assess any unintended performance disparities. Third, when applied in sensitive domains, human-in-the-loog oversight should be incorporated to validate the structured knowledge outputs and final generations. Lastly, transparency measures, such as providing users with access to intermediate structured knowledge (e.g., the constructed knowledge graph), can help foster trust and enable error analysis.
Assets Used. To support the development and evaluation of the RAS framework, we utilized wide range of publicly available datasets, open-source tools, and pretrained models. All datasets were selected based on relevance to knowledge-intenn intensiee tasks, including question answering and long-form generation, and were licensed for academic research or distributed under permissive open-source terms such as MIT, Apache 2.0, and CC BY. Our training and evaluation relied on benchmark datasets such as HotpotQA, ASQA, TriviaQA, and ARC, while structured knowledge was derived using models trained WikiOfGraph. We used open-source infrastructure including FAISS for retrieval, vLLM for inference optimization, and Graph LLM as our model backbone. For baseline comparisons, incorporated implementations from Self-RAG, RPG, and ALCE. Closed-source models like Claude-3.5, 3.n-ooeee were used for performance comparison and triple extraction under API constraints. Full licensing and usage details are provided Table
37

---

Published as a conference paper at ICLR 2026
Asset
Source / URR
License
Usage
HotporQA
https://hotpons thub.
MIT
Base QA dataset
io/
HotporQA-SURQ
We constructed from HotpotQA https://github, com/ google-tessaacch/Laagggas tree//masterrllaguugg//ssss http:://allonai.org//aal arc
MIT (inherited) Apache 2.0
Subquery training dataset Training and evaluation
ASQA
Arc-Easy
A12 Research License
Training (subset)
TriviaQA
https://nlp.cs.
CC BY-SA 4.0
Evaluation
wasbingtoon.cchh/zziivlggg
PopQA
https:/hhuggingfaccecco
CC BY 4.0
Evaluation
Gatasstss/akariiss//Pedo
2WikiMultihopQA
https://gitthbb.com/
Apache 2.0
Evaluation
Alab-WEt//wikkiuut.hhhp
PubHealth
http://buggiingface.co/
MIT
Evaluation
attssets//blgplo/ppbbeelttt http:://allenaaloon//atat arc
ARC-Challenge
A12 Research License
Evaluation
EL15
mttp:://huggingfacc.coo co/ datasets/eli5
Apache 2.0
Evaluation
Wikipedia
httpa://githubb.com
Creative Com- mons Public Li- censes CC BY 4.0
Knowledge source for retrieval
2018/2020
facebockressaarch/ttlas
WikiOfGraph
https://github.cco
Triple model training
GachuikLI//iLkkOOrraah
Claude-3.5-Somne
https:://wwwanttrron/ com https://ai. .meta.ccmmm cwm.commmmmmmmmm ama
Proprictary
Default closed source model Default open source model
ILaMA-2/3
Meta LLaMA
Community
License
LLaMA-3.2-3B
Derived from Meta LLaMA
Meta LLaMA
Triple extraction
Community
License
Llange
https://huggiiggfac.. co/
Apache 2.0
Triple extractor
soogle/flan-tSS-lagge
Graph LLM
Modified from https://githalt
Research use
Model backbone
Faiss
https://gittuub.com/
MIT
Dense retrieval
facebockresearch/lasss
VLLM
https://gittubb.com
Apache
Optimized inference
vLim-project/yllm
Sentence-BERT
https://ww..bertt.u net
Apache 2.0 MIT
Graph node/edge embedding Baseline eval
Self-RAG Code
https://gichub.. com/
Akariasai/sel.-raa
RPG Code
https://github. com/
Research use
Baseline eval
haruhi-sudo/RRG
ALCE Code
https://github.com com/
MIT
Eval metrics
princeteen-nlp/ALcc
Table 9: Assets used in RAS. All assets are licensed or cited for research use or open-source compatibility.
38


Reviews:
[{"review_id": "TLqwCF42wR", "reviewer": "Reviewer_pjgt", "summary": "LLMs perform well on knowledge-intensive tasks but struggle with multi-step reasoning due to unstructured retrieved context, a limitation RAG shares. Aligning with structured intermediate reasoning insights, the authors propose RAS—a framework that dynamically builds question-specific knowledge graphs via iterative retrieval and incremental graph construction. Evaluated on seven benchmarks, RAS outperforms baselines by up to 8.7% (proprietary LLMs) and 7.0% (open-source LLMs), validating dynamic, query-tailored knowledge structuring as effective for enhancing reasoning accuracy/robustness. Data and code are publicly available.", "questions": "Please refer to the weakenesses.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is easy to follow.\n2. The studied problem is important.\n3. The proposed method is evaluated on an extensive set of datasets.", "weaknesses": "1. Important related work is neglected. Constructing dynamic knowlege graph for sovling complex reasoning tasks in RAG is a well-studied area. Representative work, such as SG-Prompt, ERA-CoT, and KnowTrace, is not analyzed in the paper, especially KnowTrace. Compared with these existing work, the novelty and technical contribution are limited.\n2. Important baselines are missing in the experiments. It is not clear the true performance of the proposed method among existing work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dBUrlMDsVw", "reviewer": "Reviewer_X9ud", "summary": "The paper proposes Retrieval-And-Structuring (RAS), a framework designed to enhances reasoning in large language models for knowledge-intensive tasks. Unlike traditional RAG approaches that retrieve but fail to organize context, RAS dynamically builds question-specific knowledge graphs through iterative retrieval and structured knowledge synthesis. The framework consists of three key stages: planning (to identify knowledge gaps), retrieval and structuring (to form factual triples), and graph-based answering. Experiments across seven diverse benchmarks demonstrate gains across multiple datasets.", "questions": "Since graph extraction can be viewed as a form of information organization and denoising, complex sentences with multiple conditions might lose contextual nuances when converted to triples.\nWhy not combine graph representations and original retrieved text during answering, to preserve both structure and completeness?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+ The paper introduces RAS, a dynamic, query-specific knowledge graph construction framework that avoids inefficiencies of global KG indexing and eliminates costly offline graph building, achieving pay-per-query scalability.\n+ The method is evaluated comprehensively on multiple datasets across both open-source and closed-source LLMs, demonstrating consistent and generalizable effectiveness.\n+ The framework improves reasoning transparency and provides a structured way to bridge retrieval and reasoning in a unified pipeline.", "weaknesses": "- The paper lacks concrete examples demonstrating how graph representations outperform plain text in enhancing reasoning accuracy. It remains unclear which aspects of reasoning (e.g., factual grounding, compositional reasoning, or logical chaining) benefit most from graph structuring.\n- Moreover, since RAS and RPG are trained on different datasets of different scales, and the appendix shows that training set size directly affects performance, the gains of RAS over RPG on open-source settings (insignificant on half of the datasets) might stem from data scale rather than the graph mechanism itself.\n- In principle, clearer graph structures should lead to larger improvements on more complex reasoning tasks, yet the paper does not analyze this correlation or provide corresponding comparisons to validate the claim that graph structuring enhances reasoning.\n- There are several presentation issues, such as missing “top-2” formatting in Table 1 (best/second-best not consistently marked) and missing figure references in the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UP7D0G1tRM", "reviewer": "Reviewer_pGQB", "summary": "This paper proposes a RAG framework for structuring the context in a graph form for improving knowledge-intensive tasks. In detail, this framework is composed of sub-question generation, retrieval-and-structuring, and terminal-and-answering. Additionally, this paper also proposes a training framework to enhance the model to be compatible with this framework. Then the comprehensive evaluation over five tasks shows the advantages of new framework over closed and open source models.", "questions": "See weakness!", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. I appreciate this paper for utilizing the graph form representing knowledge and an elegant workflow to address complex and knowledge-intensive tasks. \n2. The experiments are solid across many classic knowledge-intensive tasks, and the improvement is good.\n3. This framework is general for closed and open-source LLMs, where the code is open-source.", "weaknesses": "1. This paper lacks the motivation for why organizing context in a graph form improves the performance of RAG. \n\n2. The involved benchmarks are outdated. There are many new graph-rag datasets for reasoning on domain knowledge. The results on such datasets are necessary to improve the quality.\n\n[1] GraphRAG-Bench: Challenging Domain-Specific Reasoning for Evaluating Graph Retrieval-Augmented Generation\n[2] When to use Graphs in RAG: A Comprehensive Benchmark and Analysis for Graph Retrieval-Augmented Generation", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JCk4JgX9IF", "reviewer": "Reviewer_Tuj9", "summary": "This paper proposes a planning-based iterative RAG method for knowledge-intensive LLM generation. It conducts experiments on several datasets to verify its effectiveness.", "questions": "See weaknesses above. Please respond to these cons.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is clear and well-written, with a reasonable motivation and considerable insight.\n\n2. The experiments are sufficient and provide a comprehensive analysis.\n\n3. The reproduction is excellent, with code and extensive reproduction details provided.", "weaknesses": "1. The backbone used in the experiments appears somewhat outdated. We recommend using a more recent backbone, such as Claude-4 and LLama-3.1. Alternatively, consider the GPT-4/5 series or the Qwen3 series.\n\n2. Please provide a more detailed analysis of the core differences between this paper and other iterative RAG methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)