Log-Augmented Generation: Scaling Test-Time Reasoning with Reusable Computation
Authors: Peter Baile Chen
Yi Zhang
Dan Roth
Samuel Madden
Jacob Andreas
Mike Cafarella
Venue: ICLR 2026 Poster (2026)
Keywords: log
KV cache
generation
Primary area: foundation or frontier models, including LLMs

Abstract: While humans naturally learn and adapt from past experiences, large language models (LLMs) and their agentic counterparts often fail to retain reasoning from previous tasks and apply it to future contexts.
We introduce **L**og-**A**ugmented **G**eneration (LAG), a novel framework that *directly reuses* prior computation and reasoning from past logs at test time, enabling models to learn from previous tasks to perform better on new, unseen challenges, without sacrificing efficiency or scalability.
Our approach represents task logs as key-value (KV) caches that encode the reasoning context of prior tasks, while storing KV values for only a selected subset of tokens. When a new task arises, LAG retrieves KV values from relevant logs to augment generation.
Unlike reflection-based memory mechanisms, which require additional extraction or distillation steps, LAG reuses prior reasoning verbatim.
Moreover, it extends beyond existing KV caching techniques, primarily designed for efficiency, by explicitly improving accuracy through log reuse.
Experiments on knowledge- and reasoning-intensive datasets demonstrate that our method significantly outperforms standard agentic systems without log utilization, as well as existing approaches based on reflection and KV cache techniques.

Published as a conference paper at ICLR 2026
LOG-AUGMENTEE GENERATION: SCALING TEST-
TIME REASONING WITH REUSABLE COMPUTATION
Peter Baile Chen'
Yi Zhang² Dan Roth
Samuel Madden
Jacob Andreas Michael Cafarella*
MIT Greenshoe, Inc. Oracle Al and the University of Pennsylvania
Correspondence: pete cesterremiiteed
ABSTRACT
While humans naturally learn and adapt from past experiences, large language models (LLMs) and their agentic counterparts often fail to retain reasoning from previous tasks and apply to future contexts. We introduce Log-Augmented Generation (I (LAG), novel framework that directly reuses prior computation and reasoning from past logs at test time, enabling models to learn from previous tasks perform better on new, unseen challenges, without sacrificing efficiency scal- ability. Our approach represents task logs as key-value (KV) caches that encode the reasoning context of prior tasks, while storing KV values for only selected subset of tokens. When new task arises, LAG retrieves KV values from relevant logs augment generation. Unlike iiflection-based memory mechanisms, which require additional extraction or distillation steps, LAG G reuses prior reasoning ver- batim. Moreover, extends beyond existing KV caching techniques, primarily designed for efficiency, by explicitly improving accuracy through log reuse. Ex- periments on knowledge- and reasoning-inted -intonninn datasets demonstrate that our method significantly outperforms standard agentic systems without log utilization, as well as existing approaches based on reflection and KV cache techniques !
INTRODUCTION
The ability to learn from past experience is highly valuable, a skill that humans naturally possess but large language models (L.LMs) do not have by default. Consider the two multi-hop questions shown in Figure D which share two intermediate reasoning steps. When humans tackle the first question, they naturally break it down into four steps, generate intermediate results, and remember them. Then, when faced with the second question, they reuse the relevant computations from the first task, reducing the reasoning required from four steps to two steps. In contrast, when models process the two questions sequentially, they handle them independently without retaining memory of the previous task, which prevents them from recognizing the connection and reusing reasoning.
In this paper, we propose Log-Augmented Generation (LAG), framework that directly reuses prior computation and reasoning from past logs at inference time to close this gap. To implement this framework, natural idea is store all model interactions retrieval system and, when a new task arises, retrieve relevant logs to include in the model's 's con- text for generation. While full textual logs can retain rich contextual details, they often introduce substantial noise and may exceed context length limitations. We instead represent logs using KV values corresponding to subset of tokens past reasoning traces to represent the full reasoning context- centext-reducing size while enabling context-dependent interpretation. Our method is inspired by the insight that the KV representations of individual tokens encapsulate far more meaning than the token alone: a token's 's KV value is weighted aggregation of embeddings from the entire surround- ing context, enabling to capture the semantics of the broader sequence. This means it is possible to store only subset of tokens along with their KV values, while still retaining the essence of the full reasoning context.
textual logs in database or
Our approach differs from prior work that relies on log extraction and reflection. Recent studies (Madaan et al., 2022 Shinn et al. 2023 Suzgun et al. 2025) propose methods that distill logs into
Data and code are available wttpttttt//// pettpp//ppteeraatt we..ie.iiiii thub. 1o/1ag/

---

Published as a conference paper at ICLR 2026
Humana
Existing: Agentic Retrieval--uggmened generation (RAG)
Quary 1: When voe the most recem ti tiigell innenlyy Inanddly Community
(e.g., RoAct)
Award given to the city where the headourners the orly group leger than the company BMG perthe
2005-077
Query
Ajenticc
Step
Thought process:
Agentic
Stnyy
Group only group larger Saryy
Repeaated end exccunal reazo
headiuua
Musto Group locaned
Ours: Log-augmented goneration (LAG)
Santa Mon
The most scea eieylle Friendly Community Awaar voea
grven to Santa Moric 2013.
Quary 1
Apentia RAG
Step
Stop
$
shrarkquarteeed)
Retrieve relevent
Encoce and store reascrinng trooes
Thought process:
logo
store
Sturty in Biownts record lobel la Emidary Records.
2.
Emsrrcy Recorda is part of Unieersst Music Grous
Teok Solve Queey with Enoluding Tosk
The headewurior a of Unkeersal Husic Group locsted in
The must recent Bicycle Frienda Communtly Aoaard
Generation tion with retrisssed of encooninng timons
glenn
Repertecaandreen macsssing pincesss
Repeated
Figure 1:
While humans naturally possess the ability to learn from past experiences, LLMs lack this capability, resulting in redundant reasoning. Our log-augmented generation framework allows LLMs to directly reuse prior thought/ reasoning processes. Algorithm ] includes details of LAG.
reusable components like thoughts (Feng al. 2024), problem-solving strategies (Yang et al. 2024), or domain- specciii knowledge (Suzgun et al. 2025). However, such distilled content often lacks the full reasoning context and may become overly abstract, which can limit its practical usefulness and reusability. Instead, our framework directly encodes and reuses past reasoning and computation, avoiding the need for additional knowledge extraction and distillation steps. Our framework further utilizes KV representation of logs to implement this encoding for more effective reuse.
Our approach also stands apart from existing KV cache methods that focus caching KV values for prompt instructions (Gim et al. 2024) or document corpora in retricval- -ugiiveee-- generation (RAG) systems (Lu al. 2024 Chan et al.; 2024 Sun et al. 2024). In comparison, our system aims to cache and reuse the reasoning trace of previous executions. Moreover, the primary goal of these methods is reduce redundant computation when identical content reappears. In contrast, we emphasize using KV caching not just for computational efficiency, but as a means to retain contextual information, enhancing not just efficiency but accuracy.
In summary, this paper introduces log- eggguugeented generation, new framework designed to enable the direct reuse of prior reasoning. LAG leverages KV values to represent past reasoning context to support effective reuse. We evaluated LAG on both knowledge-intensin tasks, including math and science
tasks, such as open-
domain multi-hop question answering: and reasoning-intensii questions. LAG significantly outperforms both standard agentic systems without log usage and existing reflection and KV caching techniques, achieving superior effectiveness and efficiency.
2 METHODOLOGY
We begin by outlining the general framework of log-augmented generation, which enables the direct reuse of prior reasoning, in Section 2.1 Next, Section 2.2 we detail our specific implementation of the framework, focusing on how logs are represented using KV values.
2.1 LOG-AUGMENTED GENERATION
Our framework aims to enable LLMs to directly leverage previous executions when performing generative tasks in order to boost their performance on future tasks. This requires three additional components on top of the base model: (1) an encoding and storage module that represents and stores past executions in form that be casily and effectively reused; (2) a retrieval module that, given new task, identifies the most relevant and useful prior history; (3) generation module that integrates the retrieved prior history into the context of the L.M to enhance its generation.
2

---

Published as a conference paper at ICLR 2026
Algorithm| | outlines the general roggameeeeeed generation framework applied to agentic systems, which involves multiple sequential LLM generations for multi-step reasoning. At the end of each step, the model chooses between two actions: either providing answer to the task generating a follow-up query or sub-task to progress toward a solution. This cycle continues until the task is resolved or, more realistically, until set iteration limit is reached. Notably, our approach inte- grates seamlessly with such sequential generation workflows, requiring only minimal modifications: adding a storage step, retrieval step, and slight adjustments to the prompting and model generation processes. Furthermore, while our framework is demonstrated within the context of agentic systems, can be readily adapted other generative paradigms, as the core loop of storing, retrieving, and augmented generation can be seamlessly integrated into any LLM-based generation process.
Algorithm Overview of our log-augmented generation framework (LAG) applied to agentic sys- tems involving multiple sequential generations, with our framework generalizable to any generative tasks. Gray text indicates steps representing interaction with external tools (e.g., document retrivval) by the agentic system. Blue text shows additional steps introduced for LAG.
Require: Generator LM M, number of maximum reasoning steps allowed C, Document retriever Ro, Docu- ment store D, Log encoder E, Log retriever Rg, Log store L
1: Input: input task x, Output: an answer
2: Initialize: model response y to None, number of reasoning steps to 0 retrieved documents D 0.
retrieved logs L {) conversation history H to
3: while does not include an answer and с C do
4: if y is None then
5:
Next action is
6: else
Extract next action from y
8:
Retrieve relevant documents d from D using RD given and update D with dl Retrieve relevant logs I from L using Re given and update L with
10:
Formulate user-defined prompt p based on x, a, y, and D. Append L to the beginning of
11:
M generates given p and L
12: Append and H
13: Update L with H using 3
Log retrieval. The retriever, Rr, selects relevant logs from L by performing standard semantic similarity ranking between stored logs and the next action a (typically follow-up query sub- task), Specifically, R2 retrieves the top-k logs with the highest cosine similarity to a based on their embeddings, which are generated using standard text embedding models. Log embeddings are precomputed and stored offline in L, so at test time, only the embedding for needs to be computed.
The remaining two components-storree and peerration-dppenn on the chosen log representation, which we will detail further in Section 2.2
2.2 LOGS REPRESENTED AS KV VALUES
As outlined Section T. our goal is to represent logs a way that retains the full reasoning context while keeping the amount of logs stored small, in order to reduce noise and avoid surpassing context length limits. While textual logs are a natural choice, capturing the entire reasoning context this way requires storing full reasoning traces, which can be inefficient and unwieldy. To address this, we propose using KV values as the log representation, leveraging the inherent property that the KV value of any token attends the entire context. We first describe what information is encoded and stored in the log store L, followed by description of how the stored KV values are used to enhance model generation. The retrieval component Re was described above.
KV values for context representation. We achieve the effect of retaining the full reasoning con- text while storing a small number of tokens by differentiating between the content used as context for encoding the KV cache and the actual tokens stored within it. Crucially, the KV representation of a single token encapsulates more than just the token's 's meaning it reflects the semantics of the entire context. KV values are produced via the attention mechanism. As shown in Figure 2 the attention weights allow each token to attend to all previous tokens, making a token's 's KV represen- tation weighted combination of the embeddings of all prior tokens in the sequence. Because of
3

---

Published as a conference paper at ICLR 2026
Existing methods
Round model responso
Altention weight token 1o tolion jia., how much importance rnswn shoul give to token
tokens the lost mund model
Round model responso
Tokens of loundound model response
rosponsss
Last round model response
1. Encode only the isat round model response values tokens from last round model resporise
full reesoning
Store the KV valulus of tokens from the last round model resporsse
Figure 2: A typical attention weight, after applying the lower-triangular causal mask, enables each token's s KV values to attend to all preceding context with different levels of importance. Leveraging this property, LAG selectively stores the KV values of subset of tokens from the model's most recent response, while still preserving the complete reasoning context. This approach differs from existing KV caching methods, which do not differentiate between content for encoding and storage.
this property, storing only subset of tokens and their associated KV values is sufficient to retain the essential semantics of the entire reasoning context.
As shown in Algorithm [ task includes multiple turns of user and assistant messages (we "model response" and "assistant message" interchangeably), where the total number of turns de- pends the specific generative framework, but should be least instructions, task details, and optional inputs such as retrieved content, while assistant messages represent the model's reasoning steps. To capture the full reasoning context, as illustrated in Fig- ure 2 we encode a sequence of all model responses across all turns into KV values. For storage, however, we retain only the KV values associated with the last model response, treating it as a con- densed summary of the entire reasoning trace. This last response is chosen because it likely reflects the model's most refined understanding the task, allowing the corresponding KV values attend to different parts of the overall reasoning context. This contrasts with existing KV cache methods, which do not distinguish between what gets encoded and what is stored (Lu al,, 2024 Sun et al.) 2024]. For example, when storing the KV cache of document, these approaches typically encode and store the document's KV cache directly. A straightforward adaptation in our setting would be to treat the last model response ceappendentl-eencodiee and storing it KV cache without considering the reasoning from earlier responses, as illustrated in Figure
User messages convey
Token selection for KV values. Although KV values encode richer semantics than plain text tokens, they come with higher storage cost since each token's KV representation dimensional vector. To reduce storage costs, we may opt retain the KV values for only small subset of important tokens, such as those representing the final answer, or in agentic settings, the next action a from Algorithm[[ when the answer cannot be determined since these encapsulate the result of the reasoning process. Nonetheless, as we will discuss in Section 4.2 our analysis shows that storing KV values for greater number of tokens generally improves performance, though at the expense of increased storage. Overall, we identify that storing KV values for the tokens in the last model response, our selected storage strategy, offers the best performance-storage tradeoff.
high-
Generation with KV values. Once the relevant logs are identified via the retriever Re, their cor- responding KV values are retrieved and provided to the LLM to augment generation. However, simply concatenating them-analogous to concatenating multiple textual logs-and passing them to the LLM for generation is problematic, due to the dependence of KV values on positional infor- mation. The KV values the log store L were generated based on the positional IDs within their original encoding context, which differ from those in the current context. Inspired by (Lu et al. 2024 (un et al. 2024), we address this issue by first stripping the stored KV caches of their original positional embeddings, and then reapplying positional embeddings appropriate for their new
4

---

Published as a conference paper at ICLR 2026
text. We present the basic concept below, which is simplified representation of the actual matrix computations used practice.
Current LLMs Rotary Positional Embeddings (RoPE) to encode positional information by ap- plying a position-dependee rotation matrix to each 2D subvector æ within the key vectors of the KV values, where the rotation matrix is determined by the rotation angle & that is function of the input token's 's positional ID.
(0)
sin(0)) cos (0)
RoPE(z, 0)
sin
To remove the influence of positional encoding, we first multiply RoPE(x, 0) by the inverse of the rotary matrix to obtain x.
cos(O) (0)
sin(0) (0) cos [) RoPE(x.
cos -in(( () cos
sin(0)
cos(()
sin(0)
(2)
sin
cos (0)
Next, we compute a new rotary matrix using 0', which derived from the updated positional IDs, allowing generate RoPE(x, 0*) aligned with the new context.
cos(( (8))
sin (8*) cos( (0")
RoPE(1r 0') = sin(O) (0)
x
(3)
With the updated KV values for each log, we concatenate them and use them to augment LLM generation.
EXPERIMENTS
3.1 EXPERIMENTAL SETUP
Datasets. We selected both representative wwwlldge-intensivee and reasoning -ittiini- datasets to assess the effectiveness of different methods. For the knowledge-intensive setting, we used open- domain multi-hop QA datasets: Musique (Trivedi Specifically, we used the development split of Musique, which includes 2,417 questions, and ran- domly sampled subset of 4,000 questions from the 2WikiMultiHop dev set. For reneoving-intensvve intensive tasks, we focused on math and science QA datasets: GPQA (Rein et al.] 2024) and MMLU-Pr Wang et al. 2024),, using 448 questions from the GPQA development split and random sample of 1,000 questions from the MMLU-Pro dev set.
2022 and 2WikiMultiHop (Ho et al.) 2020).
Metrics. We use standard evaluation metrics assess end-to-end performance the different QA datasets. For short-answer tasks, such as those in multi-hop QA datasets, we report Exact Match (EM) and Fl scores by comparing the predicted and gold answers. For multiple-choice questions in the math and science QA datasets, we evaluate using exact match between the predicted and correct choices. Additionally, we report the average number of reasoning iterations required solve each task as measure of efficiency.
Baselines. We evaluated our approach by comparing it with problem-solving frameworks that do not utilize logs, existing methods based reflection and KV cache techniques, and ablated version of our log-augmented generation method where logs are represented as plain text.
Standard agentic: We employed ReAct-style agentic reasoning framework (Yao et al., (023) as a strong problem-solving framework, given its retry capability and adaptability across diverse task types. As discussed in Section 2.I] the agentic system operates itera- tively, deciding at each step whether to provide an answer or initiate a new reasoning action, which may include interactions with external tools.
Reflection: As representative baseline for reflection- based systems, we included Dynamic Cheatsheet (Suzgun et al. 2025). This method enhances generation through online reflec- tion, retrieving the top-k most relevant logs and extracting key insights from them that applicable to the current user task. The insights are subsequently incorporated into the model's context enhance its generation.
5

---

Published as a conference paper at ICLR 2026
KV cache: We adopt Block Attention (Sun et al. 2024 as representative baseline for existing KV cache methods. As discussed in Section 2.2 this approach does not distinguish between the content being encoded and the content stored as KV values. Specifically, we implement the version that encodes the last model response and stores the KV values corresponding the tokens in that response. During generation, it retrieves the top-k most relevant logs in their KV format, which are then used to enhance the model's s generation process.
LAG To assess the impact of different log representations, we introduce variant of our log-augmented generation framework that is identical in all aspects except for the log rep- resentation-it uses texts instead of KV values. We implemented two versions of LA Grext- LAG all refers to storing and augmenting models with reasoning traces from all rounds. LeG -lat refers storing and augmenting models with reasoning traces from the last round only.
All baselines that utilize logs are implemented on top of the standard agentic framework for gen- eration. For retrieving relevant logs, LA stores the full reasoning trace, so the retriever compares the input query with the entire trace. In contrast, all other baselines store only the final round of reasoning, so the retriever uses just that last round compute semantic similarity. In this section, we denote the default version of our log-augmented generation framework that represents logs using KV values as LAGgv, to distinguish it from the variant of our framework that utilizes textual representations.
Environment. We used several top-performing lightweight embedding models from the MTEB leaderboard (Muennighoff et al. (022), including Snowflake arctic smmeed---v 0 (Yu jet al.] 2024) and Qwen3 Mereen-EEmeddiiig Embedding-------e O. 6B (Zhang et al., [025). For method execu- tion, we used open-source LLMs that allow direct access and manipulation of KV val- ues, specifically Llama-3.1127 1-mma--31-70B-Instreeee
Owen3-30B-A322 ---------------sssttt-----------
and
The main paper reports results using 1.ama-3.1--2 with Snowflake ke. while additional evaluations with other embedding models and L.MMs are presented in Appendix [na Appendix[]] respectively. The prompts used for ReAct-style agentic systems are provided in Appendix[A] Since agentic systems perform iterative reasoning until an answer is found, and this process generally lacks a convergence guarantee, is necessary to set a maximum number of reasoning steps. For multi-hop question answering, cap the number of reasoning steps at 8, as these tasks usually require up to 4 hops. For sussoning-intensin tasks, we limit the number of steps to 3. In all settings that have access to logs, we retrieve top-3 most relevant logs, and in RAG setups, we additionally retrieve top-2 most relevant documents.
We evaluate methods under two log store settings, static and dynamic, which differ in when logs are indexed. In the static log store setting, the store is constructed offline: we assume a predefined set of tasks has been executed in advance generate logs, which are then indexed into the log store. During online usage, this fixed store can be queried for relevant logs, but remains unchanged until the next offline update. In contrast, dynamic log store continuously evolves: after each task completes. its newly generated logs are immediately added and indexed. The main paper reports results for the static log store setting, while results for the dynamic log store setting are included in Appendix G. Under the static log store setting, we split each dataset into X% for constructing the log store and Y% for evaluating on "unseen" test questions. The X% portion used to populate the log store is referred to as the "seen" questions. To reflect realistic deployment conditions where ground truth answers are typically unavailable, we do not use gold answers to filter out incorrect logs during the log storage phase. However, developers who have access ground truth data may choose to apply such filtering to improve log quality and overall system performance. We evaluated the robustness of our approach across multiple data splits (X/Y 70/30, 50/50, and 30/70). The main paper reports results with X = 70 and Y = 30, while results for the other splits are shown in Appendix H
3.2 RESULTS
The accuracy (measured exact match and F1 scores) and efficiency (number of reasoning itera- tions) numbers are reported in Tables [ and In general, LAGgv outperforms all baselines across both knowledge- and reasoning-intensive datasets in terms of both metries. Notably, the variant of

---

Published as a conference paper at ICLR 2026
Table 1: Exact Match (EM) and F1 scores knowledge-intensing datasets Musique and 2WikiMul- tiHop, along with the number of iterations required to complete each task. Seen or unseen refers to whether the questions were used to construct log stores. Bolded numbers indicate the best perfor- mance among all methods, and Underlined numbers indicate the performance of LAG or LAGgy when they rank second.
Musique (seen)
Musique (unseen)
2WikiMulrHHap (seen)
2WikiMuliHHpp (unseen)
EM
F1
flter.
EM
F1
Hlter.
EM
Rlter.
EM
F1
H
Standard agentic Yao et al
2023
26.3
36.4
3.98
27.0
37.3
3.90
50.2
58.9
2.53
51.6
60.2
2.49
(uzziir
37.5
48.3
2.17
KV cache
Sun et 1 20 2124
28.5
38.5
3.36
28.7
40.7
3.37
52.9
61.1
1,93
48.3
57.1
1.89
LAGue-ll
2.52
LAG last
29.6
40.0
3.17
30.7
42.1
3.20
55.8
64.1
1.77
55.0
63.6
2,00
LAUkv
32.3
43.2
57.5
Table 2: Performance on reasoning-intensive datasets GPQA and MMLU-Pro.
GPQA (seen)
GPQA (unseen)
MMLU-Pro (scen)
MMLU-Pro (unseen)
EM
Alter.
EM
Mlter.
EM
Mlter.
EM
Wlter.
Standard agentic Yao et al. 20233
22.4
1.96
18.5
1.87
39.7
1.68
41.3
Reflection Suzgun ot al. 2025
21.4
1.81
20.0
1.84
40.0
1.54
41.0
KV cache
el al [2024]
19.5
19.3
1.68
1.61
LAGaaal
22.4
17.0
1.92
39.7
1.62
39.7
1.60
LAG,arlas LAGgy
21.7
1.89
18.5
1.93
41.6
1.53
42.0
1.50
24.6
1.58
30.4
1.62
44.3
1.33
42.3
1.36
our log-augmented generation framework that uses plain text (LAGGex) to represent logs generally
ranks the most effective approach.
Benefits of log-augmented generation. Both LAGky and LAG surpass standard agentic sys- tems that do not leverage logs, underscoring the value of logs in improving both the accuracy and methods
efficiency of these systems. Furthermore, their superior performance over reflection- suggests that directly reusing past reasoning and computation steps is more effective than approaches that rely on knowledge extraction and abstraction, which might produce overly abstract representa- tions that lose the richness of the original reasoning context.
Benefits KV values for log representation. The superior performance of LAGkV over LAG demonstrates that KV representations are more effective than plain text for capturing and utilizing logs. When LAG include all rounds of reasoning matching what is encoded in the KV values LAGgy= perform worse, likely because the full reasoning trace contains noise, whereas the KV values produced by the attention mechanism may emphasize more pertinent aspects. On other hand, when LAG include only the last round-matching what LAGgy stores-they also underperform, likely because they lose the broader reasoning context, which KV representations are still able capture through encoding the full reasoning process.
Moreover, the fact that LAGky outperforms existing KV cache methods-which do not differentiate between what is encoded and what is stored indicatess that the encoding strategy used in these methods fails to capture the full reasoning context. In contrast, LAGgy effectively preserves this broader context by encoding the entire reasoning context. We also note that adding more content for encoding does not affect the resulting KV size, as the size is determined solely by the number of stored tokens, the number of attention heads, and the dimension of each attention head-the latter two being fixed by the model architecture. Therefore, our more effective encoding strategy offers performance gain without incurring additional storage cost.
4 ANALYSIS
Section 3.2 demonstrated that LAG outperforms existing approaches. In this section, we present deeper analysis yis to better understand the sources of this improvement (Section[4.I), We also examine the tradeoff between performance and storage when representing logs with different token subsets

---

Published as a conference paper at ICLR 2026
Musique
ZWINiMutlHop
MMLU-Pro
48.0
42.5
EAM moth
LAG
2ekcnccyercck. Lends:
Accoriss
17.3
10
15
10
10
Number
Number
Figure 3: Exact match score of standard agentic systems and LAG varying the maximum number of reasoning steps allowed to solve task. Logs show efficiency benefits when the performance of LAG in on par with agentic systems but requires fewer reasoning steps. They show performance benefits when LAG surpasses the agentic systems at points where their performance has plateaued.
Table 3: Breakdown of accuracy improvements at iteration 20 (25 for Musique), the point where both agentic systems and LAG reach performance plateau. I, C, and U represent incorrect, correct, and unsolvable, respectively. X + refers to the number of questions that were initially X but became Y after incorporating logs.
#Correct ans, w/o logs
I-+ C
C-I
U-C C
C U
Total improvement
Musique 2WikiMultHHop
220
+20
626
+138
-111
+12
-0
+39
GPQA MMLU-Pro
28
+18
-10
+15
-1
+22
127
+35
-39
+12
-6
+2
(Section 4.2
and evaluate how performance varies with the number of retrieved logs (Section 4.3 All analyses were conducted on unseen questions across all datasets.
4.1 ACCURACY AND FFFICIENCY BENEFITS OF LOGS
highlights how LAG enhances both the accuracy and efficiency of standard agentic systems. In this section, we take closer look to better understand why logs are effective. Logs of past reasoning trace, intuitively, should support both knowledge reuse and insight reuse. Knowledge reuse allows partial solutions from past tasks to be applied to new ones, enabling the model to solve tasks using fewer reasoning steps and thus improving efficiency, which is illustrated in Figure [ Insight reuse, the other hand, helps the model identify prior mistakes, leading to correct answers where it previously failed, and recognize successful problem-solving strategies, allowing it to answer previously unsolvable questions, ultimately enhancing accuracy.
Section 3.2
We define "unsolvable questions" as those that agentic systems are unable to answer within a rea- sonable number of reasoning steps. As discussed in Section 3.1. although agentic systems the- oretically continue reasoning indefinitely until a solution is found, this is not practical. To address this, we set an upper limit on the number of iterations. We extend this by increasing the maximum allowed iterations until the accuracy of standard agentic systems levels off. At this plateau, any questions that remain unanswered are considered unsolvable. As shown in Figure 3, our experi- ments show that performance flattens at around 20-25 iterations across all datasets. The figures also include the accuracy of LAG across different maximum iteration limits.
As shown in Figure we observe clear efficiency and accuracy gains by LAG. For example, in the Musique dataset, LAG reaches an exact match score at iteration that slightly higher than that of standard agentic systems at iteration 7, yielding 3.5x efficiency improvement. Additionally, even at iteration 25, where both systems reach their peak performance, LAG still outperforms the standard agentic system, demonstrating superior accuracy. To further illustrate this, Table 3 breaks down the sources of this accuracy improvement across all datasets. The results indicate that while the model may be misled by relevant log information and therefore turs some previously correct questions into incorrect or unsolvable ones, they more often convert previously incorrect or unsolvable questions into correct ones, highlighting their overall effectiveness in improving accuracy.
8

---

Published as a conference paper at ICLR 2026
Table 4: Performance of LAGgV (using KV representation) and the size of the stored KV values (in GB) varying the content stored as KV values, along with the performance of standard agentic systems and the variant of our log- augmentedd generation framework that represents logs as texts (all reasoning traces stored and provided), LeG -ax.
Musique
2WikiMuliHopp
GPUA
MMLU-PPo
EM
Fi
Aber.
KV size (GB)
EM
F1
Vhen.
KV size
EM
Fi
Mler
KV size
EM
F1
Aberr
KV size
Standard apenic
27.0
37.3
3,90
51.6
60.2
2,49
185
1.77
41.3
1,58
LAGa=sll
569
2.29
LAGgy
Last 3 rounds
33.2
45.1
3.501
93
57.2
66.0
1.98
20.7
2.08
34.3
1.57
83
Last 2 rounds
33.6
45.4
1.22
70
56.4
66.4
1.94
76
22.2
1.81
64
38.3
1.51
70
Last round
22.2
45.0
2.68
38
55.2
64.7
1.84
44
30.4
1.22
27
42.3
1.36
46
East oction
29.3
40.3
3.55
611
56.2
64.0
2.34
5.0
25.9
1.84
7.0
42.0
1.60
73
Qualitative examples. We manually reviewed questions and selected representative examples that illustrate the reuse of knowledge and insights, which contributes to improving the accuracy and efficiency of standard agentic systems. Additional details are provided in Appendix B
Lightweight safeguards. Table shows that logs can sometimes introduce errors, motivating the of two lightweight and easy-to-apple safeguards: (I) similarity thresholding, which filters out low-relevance logs during retrieval, and (2) a brief reflection step, which prompts the model to assess fallback
the trustworthiness of the retrieved information before using These mechanisms strategies that enhance the robustness of LAG. Detailed results are provided in Appendix !
4.2 KV VALUES FOR CONTEXT REPRESENTATION
As mentioned Section 2.2 although KV values are more effective representation than text, they incur higher storage costs due to each token's KV being high-dimensional vector. To address this, we explore how varying the subset of tokens whose KV values are stored impacts perfor- mance- -while still encoding the full reasoning trace. Our goal is to understand how different levels of stored content affect the tradeoff' between model performance and storage efficiency in LAG. Storing fewer tokens leads a higher "compression" rate, lowering storage requirements but poten- tially reducing the fidelity of the contextual representation.
Table presents the performance and storage footprint associated with different token storage strategies, alongside baselines that either provide no logs or represent all past reasoning traces as plain text. The "last action" strategy refers to storing only the tokens tied to the model's final ac- tion-eithen an answer or follow-up up question. Overall, we find that storing the last model response yields the best balance, achieving superior accuracy and efficiency while maintaining a reasonable storage cost. Additionally, we observe that storing only the KV values corresponding to the model's last action performs comparably to storing the entire last model response. This approach still out- performs standard agentic systems without logs and the textual log representation that includes all model responses. Importantly, it achieves this while significantly reducing the average KV stor- age size across all datasets, practical and efficient tradeoff for developers who aim to optimize for storage.
factor of 6.30, bringing the average down 6.35GB, making it
We observe that selecting tokens corresponding to the last action may be effective because the final action represents the outcome of the reasoning process. While it may consist of only a few tokens, the corresponding KV values likely encapsulate and attend broader span of the earlier reasoning steps. These findings highlight the strength KV as a representation, capable of encoding richer se- mantic information than plain text tokens while requiring significantly fewer tokens, thereby keeping storage costs relatively reasonable.
4.3 IMPACT OF THE NUMBER OF LOGS RETRIEVED ON PERFORMANCE
We also investigate how the number of retrieved logs affects the overall performance. Table presents the results of LAG the number of retrieved logs (k) varies from 0 to 3, where k represents the baseline standard agentic system. Overall, we find that increasing the number of retrieved logs generally leads to improved performance.

---

Published as a conference paper at ICLR 2026
Table 5: Exact Match (EM) and Fl scores of LAG varying the number of logs kt retrieved each reasoning step. k = 0 refers to standard agentic systems.
k =
k =
=
EM
F1
#lter.
EM
F1
#lter.
EM
F1
#lter.
EM
Fl
#lter.
Musique
27.0
37.3
3.90
29.5
41.3
2.84
32.2
44.5
2.76
32.2
45.0
2.68
WiikiMultiloo
2.49
2.05
1.91
1.84
GPQA
18.5
1.87
23.0
1.73
30.4
1.61
30.4
1.62
MMLU-Pro
41.3
1.58
41.0
1.42
39.0
1.40
42.3
1.36
5
CONCLUSION
This work presents LAG, new framework that equips LLMs with the ability to directly reuse reasoning traces from previous tasks. Central to this framework is the use of KV values to represent logs, preserving essential reasoning context more effectively than plain text, while keeping storage requirements manageable. Experiments across both knowledge- and reasoning-intensii datasets demonstrate that LAG substantially outperforms standard agentic systems and surpasses existing reflection-bast approaches and KV caching techniques.
ACKNOWLEDGMENTS
We thank Yongjoo Park and Tim Kraska for their helpful discussions. We gratefully acknowledge support from the DARPA ASKEM Award HR001 12220042, the ARPA-H Biomedical Data Fabric project, Liberty Mutual, Jane Street, the MIT MGAIC Consortium, the MIT Everest Projects, the MIT-IBM Watson AI Lab, ONR-LiRD collaboration with MIT. We also thank Amazon, Google, and Intel for their contributions through the MIT Data Systems and Lab (DSAIL.).
100014-23-1-2364 and the Google PhD Fellowship in
REFERENCES
Brian J Chan, Chao-Ting Chen, Jui-Hung Cheng, and Hen-Hsen Huang. Don't do rag: When cache- augmented generation is all you need for knowledge tasks. arXiv preprint arXiv:24/2. 2024.
Tao Feng, Pengrui Han, Guanyu Lin, Ge Liu, and Jiaxuan You. Thought-retriever: Don't jus't retrieve raw data, retrieve thoughts. 2024.
In Gim, Guojun Chen, Scung-seob Lee, Nikhil Sarda, Anurag Khandelwal, and Lin Zhong. Prompt cache: Modular attention reuse for low-latency inference. Proceedings of Machine Learning and Systems, 6:325-338, 2024.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint 10Xix:201100000 2020.
Songshuo Lu, Hua Wang, Yutian Rong. Zhi Chen, and Yaohua Tang. Turborag: Accelerating retrieval-uugmented generation with precomputed kv caches for chunked text. arXiv preprint arXiv:2410.07590. 2024.
Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. Memprompt: Memory-assisted prompt editing with user feedback, 2022.
Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. Mteb: Massive text embed- ding benchmark. arXiv preprint 1700022200.07316. 2022. doi: 10.48500AR31222200071666 URL pettps://arxx. org/ http://b//22100/ 07316
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-level google-prood q&a bench- mark. In Fürst Conference Language Modeling, 2024.
10

---

Published as a conference paper at ICLR 2026
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634-86535 8652, 2023.
East Sun, Yan Wang. and Lan Tian. arXiv:2409.15333 15355, 2024.
Block-attention for efficient
rag.
arXiv preprint
Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. Dynamic cheat- sheet: Test-time learning with adaptive memory. arXiv preprint arXiv:2504.0790 07952, 2025.
Harsh Trivedi, Niranjan Balasubramanian. Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539-555 554, 2022.
Yubo Wang. Xueguang Ma. Ge Zhang. Yuansheng Ni, Abhranil Chandra, Shiguang Guo. Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi- task language understanding benchmark. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
Ling Yang. Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E Gonzalez, and Bin Cui. Buffer of thoughts: Thought-augmmented reasoning with large language models. Advances in Neural Information Processing Systems, 17:113519-113544 2024.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference Learning Representations (ICLR), 2023.
Puxuan Yu, Luke Merrick, Gaurav Nuti, and Daniel Campos. Aretic-embed 2.0: Multilingual re- trieval without compromise. arXiv preprint arXiv:241..04506, 2024.
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang. Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 embedding: Advanc- ing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05177 2025.

---

Published as a conference paper at ICLR 2026
A PROMPTS
The prompts used the agentic system are shown in Tables and 7
Table 6: Prompt used for knowledge-intensive datasets: Musique and 2WikiMultiHop. Blue texts indicate changes introduced for LAG.
(retrieved logs)
Do not use your general knowledge. Do not assume the existence of external knowledge. Do not
make any guesses.
You are provided with a user question, and information that might relevant to the user question.
Your task consists of the following steps:
1. From the provided information, extract facts that is relevant to the user question
2. Based on the provided information only, determine if you have sufficient information to answer the question
If you can determine the answer, output a short answer (in a few words) to the user question. The short answer must be wrapped in <ans></ans>.
If you cannot determine the answer, output some keywords that help you retrieve infor- mation. The keywords must be wrapped in <keywords><<eywords>.
Here is the information: (retrieved documents)
Here is the user question:
[Question] }
12

---

Published as a conference paper at ICLR 2026
Table Prompt used for reasoning- intensive datasets: GPQA and MMLU-Pro. Blue texts indicate changes introduced for LAG.
(retrieved logs}
You are provided with
question. Your task consists of the following steps: 1. From the provided information, extracts the key insights helpful for solving the user question
2. Break down and solve the question step by step, without relying the provided answer choices
3. Based on your analysis, determine you have sufficient information to identify the single most probable answer
If you can identify the answer, output the answer as the letter corresponding to the answer choice, placed inside parentheses and wrapped in <ans></ans> (e.g., <ans>(A)</ans>>.
If you cannot identify the answer, output sub-questions that, if solved, can lead new information. The sub-questions must be wrapped in <subgpestion><<ssqqpeettinnn
Here is the information: (previous round model response)
Here is the user question: (Question)
Here are the multiple-choice answers: (Multiple choices)
13

---

Published as a conference paper at ICLR 2026
B EXAMPLES WHERE LAG ENHANCES STANDARD AGENTIC SYSTEMS
Tables and present an example from the multi-hop QA dataset Musique, illustrating how LAG retrieves logs of previous questions containing partial solutions that be directly applied to new questions. Our method reduces the number of reasoning rounds needed-from to just l-and corrects an originally incorrect answer.
Tables 10 and I1 present an example from the science QA dataset GPQA, where LAG retrieves logs from prior questions that contain reusable insights. These insights help the model recognize previously overlooked aspect of the problem, allowing it to correct answer that was initially incorrect.
Table 8: At round 9, the standard agentic system produced an incorrect response. It failed to extract the "role" information (highlighted in bold) from the document, and returned the producer's name without identifying the role he plays.
User message
(...omitted instructions and passages...
Document title: The Godfather Part II
Document content: The Godfather Part is a 1974 American crime film produced and directed by Francis Ford Coppola from screenplay co-written with Mario Puzo, starring AI Pacino and Robert De Niro. Partially based on Puzo's 1969 novel The Godfather, the film both sequel and prequel to The Godfather, presenting parallel dramas: one picks up the 1958 story of Michael Corleone (Pacino), the new Don of the Corlcone crime family, protecting the family business the aftermath of an attempt on his life; the prequel covers the journey of his father, Vito Corleone (De Niro), from his Sicilian childhood to the founding of his family enterprise in New York City.
Document title: Mistress (1992 film)
Document content: Mistress is 1992 comedy film starring Robert De Niro, Danny Aiello, Eli Wallach, Robert Wuhl and Martin Landau. The picture was written by Barry Primus and J.F. Lawton and directed by Primus.
(...8 documents omitted...)
Here the user question: The Godfather Part II, who did the producer of Mistress play?
Assistant message (Round 9)
(..oomiteed reasoning steps...)
Since Robert De Niro played role in The Godfather Part II, and he is also star in the film Mistress, we can determine that the answer is related to Robert De Niro. However, cannot determine which specific role Robert De Niro played in The God- father Part II from the provided information. Therefore, the answer is: <ans>Robert De Niro</ans>
14

---

Published as a conference paper at ICLR 2026
In contrast to the standard agentic system's response shown in Table 8 LAG enables models to directly reuse knowledge embedded in prior reasoning traces, eliminating the need to retrieve and analyze relevant documents. This results in an 8-round reduction in problem-solving and yields correct answer. The reusable knowledge is bolded.
Table 9:
User message (retrieved logs included beginning)
(...omitted logs...)
(past reasoning trace corresponding to the question Who was played by the director of The Good Shepherd in The Godfather Part II?)
(...omitted reasoning trace...)
The director of The Good Shepherd is Robert De Niro.
In The Godfather Part II, Vito Corleone was played by Robert De Niro.
(...omitted reasoning trace...)
(..oomiteed logs...)
(User prompt)
(..omitted instructions and passages...)
Document title: Mistress (1992 film)
Document content: Mistress is a 1992 comedy-drama film starring Robert De Niro, Danny Aiello, Eli Wallach, Robert Wuhl and Martin Landau. The picture was written by Barry Primus and J.F. Lawton and directed by Primus.
Here is the user question: In The Godfather Part II, who did the producer of Mistress play?
Assistant message (Round
(...omitted reasoning steps...)
We have the name of the actor who played in The Godfather Part II (Robert De Niro).
We have the name the film he played in (The Godfather Part II).
We have the name of the film he played in (Mistress).
We have the name of the producer Mistress (Barry Primus).
Since we have the name of the actor, the film he played in, and the name of the producer of Mistress, we can determine the answer. <ans> Vito Corlconec/ans>
15

---

Published as a conference paper at ICLR 2026
Table 10: An incorrect response from the standard agentic system, which ignored F/7-configurable double bonds, which also contributes to stereoisomer count.
User message
(...omitted instructions...)
How many stereoisomers are there for the compound bechooo--eatyy222mtttlldddd--- dien-5-ol?
(..oomitted instructions...)
Assistant message (Round) 1)
To solve this problem, we need to understand the concept of stereoisomers and how to calculate them for given compound.
1. **Key Insights:***
Stereoisomers are molecules that have the same molecular formula and sequence of bonded atoms but differ in the three-dimensional orientations of their atoms in space.
The number stereoisomers can be calculated based on number of chiral centers (asymmetric carbon atoms) in the molecule.
Each chiral center can have two possible configurations (R or S), leading to 2" possible stereoisomers, where is the number of chiral centers.
2. ** **Brakking Down the Problem:*
- The given compound is 6-chloro-9-ethyl-
To find the number of stercoisomers, we need identify the chiral centers in the molecule.
A chiral center is an asymmetric carbon atom with four different groups attached to it.
We will analyze the molecule identify the chiral centers.
** Analyzing the Molecule:#*
The molecule has the following structure: 2aaboon---ttyy---mmmtylnnaea-a/
We can break down the molecule into its components:
The main chain has 11 carbon atoms.
There are two double bonds at positions and 7.
There is chlorine atom position 6.
There is an ethyl group at position 9.
There is methyl group at position 2.
There is hydroxyl group at position 5.
Based on the structure, we can identify the chiral centers:
The carbon atom at position 2 has methyl group, a hydrogen atom, a double bond, and hydroxyl group attached to it, making it chiral center.
The carbon atom at position 5 has hydroxyl group, hydrogen atom, double bond, and a chlorine atom attached to it, making it chiral center.
The carbon atom position has an ethyl group, hydrogen atom, a double bond, and chlorine atom attached to it, making a chiral center.
*Cacculating the Number of Stereoisomers:
We have identified three chiral centers in the molecule. Each chiral center can have two possible configurations (R or S).
Therefore, the total number of stereoisomers 2 8.
Based on our analysis, we can conclude that the number of stereoisomers for the compound 6-chloro-9 4-chloro-9-ethyle methl-munntlltlyaaa3... 3,n.aadddeaa...i--------- -,7- is 8. <ans>(D)</ans>
Gold reasoning from human expert
Considering each double bond stereocenter, have total of four stereocenters, and thus it will have 16 stereoisomers (using the formula 2n, where n is the number of stereocenters).
16

---

Published as a conference paper at ICLR 2026
Unlike the standard agentic system's response in Table T0 LAG enables the model to uncover insights (bolded) embedded in prior reasoning heps--articcularlyy the relationship between E/7-configurable double bonds and chiral centers. Although this insight doesn't directly solve the problem, it helps guide the model use double bonds basis for determining the number of stereoisomers, detail that was completely missed in the earlier assistant response in Table I0 This enables the model arrive at the correct answer.
Table 11:
User message (retrieved logs included at the beginning)
(past reasoning trace corresponding to the question How many of the following compounds will exhibit optical activity?)
To solve this question, we need to understand what optical activity and which compounds property
exhibit it. Optical activity light. This property is typically associated with chiral molecules, which have non- superimposable mirror image.
compound that can rotate plane-polarize
Let's break down the given compounds and identify their chirality: 1. (Z)-I-chloro-2 meroo---mmhhluuu----eeeee This compound an alkene with a (Z) configuration, which means it has specific arrangement of atoms but is not chiral...
(..omiited logs...)
(User prompt)
(...omitted instructions..) How many stereoisomers are there for the compound dien-5-ol? (..oomitted instructions...)
Assistant message (Round 1)
To solve this question, let's break it down step by step.
1. The compound with multiple double bonds and hydroxyl group. This suggests that it may have multiple stereocenters.
-ethll---metyylundeer
has a complex structure
2. The compound has three double bonds (3, 7, and 5), which means it has the potential for multiple stereoisomers due the possibility of E/Z isomerism.
3. The compound also has a hydroxyl group (5-ol), which means it has stereocenter at the 5-position.
4. To determine the number of stereoisomers, need consider the number of stereocen- ters and the possible configurations at each stereocenter.
5. The compound has 3 double bonds, which result in 2 8 possible E/Z configura- tions.
The compound has stereocenter (5-ol), which can have 2 possible configurations and S).
To calculate the total number of stereoisomers, we need to multiply the number of E/Z configurations by the number of possible configurations at the stereocenter. 8. The total number of stereoisomers is (E/Z confiuuratii)) (possible configurations at the stereo- center) = 16.
Based on the analysis, we have sufficient information to identify the single most probable answer. <ans><\)</an>>>>>>
Gold reasoning from human expert
Considering each double bond stereocenter, have total of four stereocenters, and thus it will have 16 stereoisomers (using the formula 2n, where n is the number of stereocenters).
17

---

Published as a conference paper at ICLR 2026
C
LIMITATIONS
Although the general log-augmented generation framework described in Algorithm I is compatible with any model during generation, the specific key-value (KV) log representation detailed in Section 2.2 and used in LAG limited to open-source models. This is because accessing and modifying internal KV states is only possible with open models.
Furthermore, each task model encounters is vastly different from those it has previously han- dled-whether in domain, required knowledge, or other aspects-there may be little reusable rea- soning. However, we believe this scenario is unlikely practice, as it is common for users pose similar questions repeatedly, either from the same individual or across different users.
D IMPLEMENTATION DETAILS
The experiments conducted Linux cluster equipped with V100 and A100 GPUs. For model generation, we utilized Hugging Face (HF)'s formers library and Llama-3.1-8B hosted HF. We also used the Snowflake embedding hosted on HE, with embeddings computed using the sentence_transformeee
package.
The concatenated KV values of logs passed L.LM using past_key_val ves parameter in the HuggingFace API: model. generate (..., past_key_vvaluesm..
E RESULTS ON DIFFERENT EMBEDDING MODELS
To assess the gemeralizability of LAG across different embedding models, additionally evaluated it using the Qwen3 Embedding 0 6B retriever. As shown in Table [2 our method continues to deliver consistent performance gains.
Table 12: Performance using -wen3-Ember -er-eebbeednngggg 6B. Musique
MMLU-Pro
EM
#lterations J
EM
#Iterations
Best baseline
27.1
3.99
40.0
1.69
LAG
28.2
3.23
44.3
1.59
F RESULTS ON DIFFERENT LLMS
To evaluate the generalizability of LAG across different LLMs, we further tested it with As shown Ta-
-aann--------- A3B- 1.B-Inntruuct-22 -207 and Llama-3,
ble B
LAG continues to outperform the strongest baseline.
Table 13:
Performance
using
-2507
and
Llama-2 3..1aa33.-BBB.. Instrut..
Musique
MMLU-Pro
EM #iterations 4
EM
#Iterations
Owen3-30B -ann3-330--ABB-In--- Inst
Best bascline
43.3
3.57
83.6 86.5
1.13
LAG
50.0
3.20
1.00
L Llama 3. 3.1--00---- Inst ruct
Best bascline
35.4
3.93
64.4
1.21
LAG
38.4
3.34
65.7
1.14
18

---

Published as a conference paper at ICLR 2026
G RESULTS ON DYNAMIC LOG STORE
Table 14 presents results under the dynamic log store setting, showing that our method continues to deliver consistent performance gains.
Table 14: Performance under the dynamic log store setting.
Musique
MMLU-Pro
EM
#lterations J
EM
#lterations
Best baseline
24.3
3.67
40.0
1.69
LAG
25.6
2.88
42.2
1.54
H RESULTS ON DIFFERENT SPLITS
Table 15 reports results when X% of data used build the log store and Y% as unseen queries, under (X, Y)= (50, 50) and (30, 70). As in Section 3.2 LAG consistently outperforms the baselines on both knowledge- and reasoning-intentn intensive datasets in terms of accuracy and efficiency.
Table 15: Exact Match (EM) and Fl scores on knowledge-intensiee dataset Musique and reasoning- intensive dataset GPQA, along with the number of iterations required to complete each task. Seen or unseen refers to whether the questions were used to construct log stores. Bolded numbers indicate the best performance among all methods.
Musique (seen)
Musique (unseen)
GPQA (seen)
GPQA (unseen)
EM
F1
#lter. 4
EM
F1
#lter.
EM
#lter.
EM
#lter.
X 30, Y=70
Standard agentic LAG
26.9
37.3
4.0
26.4
36.3
3,9
22.4
2.0
20.7
1.9
31.2
43.8
2.8
30.8
41.9
2.9
24.3
1.6
22.0
1.7
X 50, Y=50
Standard agentic
25.9
36.4
3,9
27.1
37.0
4.0
21.9
2.0
20.6
1.9
LAG
30.8
42.5
2.7
31.5
42.7
2.8
24.1
1.6
25.0
1.6
19

---

Published as a conference paper at ICLR 2026
I RESULTS WITH LIGHTWEIGHT SAFEGUARDS
Table T6 reports performance with the lightweight safeguards. Both thresholding and reflection substantially reduce negative transitions from correct incorrect (C I) and correct unsolvable (C + U), and combining them yields the strongest robustness.
Table 16: Performance with lightweight safeguards. I, C, and U represent incorrect, correct, and unsolvable, respectively. X refers to the number of questions that were initially X but became Y after incorporating logs.
Total improvement
I-C CHI
U U-C C-U
Musique
LAG
+49
-47
+31
-6
+27
LAG with reflection
+56
-25
+36
-6
+61
LAG with threshold
+48
-19
+31
+56
LAG with reflection and threshold
+57
-20
+37
-6
+68
MMLU-Pro
LAG
+28
-30
+15
-11
+2
LAG with reflection
+28
-11
+13
-6
+24
LAG with threshold
+32
+15
-5
+27
LAG with reflection and threshold
+32
-11
+16
-5
+32
J THE USE OF LARGE LANGUAGE MODELS (LLMs)
LLM was used only aid writing quality (proofreading and polishing grammar). No ideas, claims, methods, results, or references are generated by LLMs. All content decisions and revisions are made by the authors.
20


Reviews:
[{"review_id": "Ebbmwer2nc", "reviewer": "Reviewer_QLNm", "summary": "This paper presents \"LAG\", Log-Augmented generation, which modifies standard RAG procedures to use logs/generated reasoning sequences. Furthermore, it uses KV-cache intermediate values as the retrieved representations. The paper presents strong empirical evidences for the method proposed.", "questions": "1. What is the context length for the retrieval model? (Sorry If I missed this) What is the length of the logs retrieved?\n2. Given the logs are likely OOD for the retrieval model, why do you think you see such nice retrieval performance? Is it the quality of snowflake in general? Or a characteristic of the data?\n3. What exactly is the query of the retrieval step? Formatting, context, instruction, etc?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* Use of retrieval for reasoning is very nice and the idea of using intermediate KV cache representations. This is a very nice idea and I believe an important area of study as it helps to decouple knowledge representation and reasoning representations.\n* The empirical successes of the paper especially using the KV cache values rather than full text representation of logs motivates future exploration of a wide variety of topics in retrieval augmented methods.", "weaknesses": "* **Scaling with Context Length**:  The KV cache vs text result is most interesting I believe, but also under explored. It is of course related to context length limitations, but it is hard to know exactly how the context size of the model changes this (e.g., Llama 8B's performance: https://arxiv.org/pdf/2504.06214v1). \n* **Depth of contribution**: It's not clear to me how to evaluate the novelty of the contribution here. I understand the novelty of retrieving logs, but it is not a sea-change from past work that retrieved data (e.g., https://arxiv.org/pdf/2203.08773 or https://arxiv.org/pdf/2311.07850). \n* **Overall Metric Scores**: It's perhaps hard to evaluate the quality of the method, when as I understand the metrics for GPQA are pretty far off of SOTA?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5SChnlnkVY", "reviewer": "Reviewer_TQuH", "summary": "This paper introduces Log-Augmented Generation (LAG), a well-motivated and novel framework for enabling LLMs to reuse past computations by storing reasoning traces as KV caches. The key technical insight is to decouple the encoding context (the full reasoning trace) from the stored content (the KV values of the last response), which is clearly differentiated from prior work. This approach yields strong and consistent empirical improvements in both accuracy and efficiency over well-chosen baselines across multiple datasets. The experimental model in this paper is singular, lacking a formal analysis of the framework's reliability, generalization ability, or potential error propagation issues. Furthermore, the default method's storage costs are substantial, and the paper lacks an analysis of retrieval failures that lead to performance degradation.", "questions": "- **Cross-context** **KV** **Cache Reuse:** When reapplying RoPE to “stripped” KV values from logs, does the accumulated error remain negligible when these caches are reused repeatedly across highly diverse contexts or multiple retrieval rounds? Is there a scenario in which retrieved KV values harm rather than help due to drift or misalignment?\n- **Task Drift and Out-of-domain Generalization:** How robust is LAG to highly dissimilar or out-of-distribution new tasks given log stores with only weakly relevant or misleading past traces? Are there any fallback mechanisms or metrics to assess when not to use retrieved logs?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Clear and Motivated Problem Setting:** The paper identifies a practical and under-explored challenge—enabling LLMs to effectively reuse prior computation at test-time—mirroring a natural aspect of human reasoning. The distinction between reusing prior reasoning and simply increasing context length is well argued, particularly in the early discussion and Figure 1.\n2. **Novel Use of** **KV** **Cache Representations:** Unlike existing KV cache approaches that target efficiency, LAG innovatively leverages the richer context encoded in KV values, advocating for selective storage (particularly from the last model response) to balance context richness and storage constraints. The distinction in Figure 2 between existing and proposed methods clearly demonstrates this key conceptual difference.\n3. **Mathematical Clarity and Soundness:** The paper provides concrete mathematical details for handling positional encodings when reusing KV caches across different contexts, specifically for rotary embeddings. The approach to “stripping” and “reapplying” positional information is well formulated and critical to correctly reusing stored KV values.\n4. **Comprehensive Experimental Evaluation:** Results in Tables 1 and 2 and subsequent ablations provide compelling evidence that LAG outperforms standard agentic, reflection, and existing KV cache methods in both accuracy (EM/F1) and efficiency (iterations), across diverse benchmarks. Qualitative examples and analyses further illuminate the mechanism by which LAG improves performance, especially via knowledge and insight reuse.\n5. **Insightful Analysis:** The work dives into ablation over stored token count, the number of retrieved logs, and the tradeoff between accuracy and storage costs—which is rarely discussed rigorously in similar works.", "weaknesses": "1. **Related Work Positioning—Missing Direct Recent Papers:** The paper omits discussion of several key recent works that closely align with LAG's aims of scaling test-time computation and effective reuse for retrieval-augmented generation. Relevant missing works include:\n\n   1. *Yue et al. (2025): Inference Scaling for Long-Context Retrieval Augmented Generation*. \n\n   2. *Geiping et al. (2025): Scaling up Test-Time Compute with Latent Reasoning\n\n       The absence of discussion or comparison with these is non-trivial: some address scaling and dynamic retrieval or propose closely related “augmented” or logic-based test-time reasoning, which could either complement or reveal subtle differences with LAG.\n\n2. **Selection/Relevance Filtering Is Rudimentary:** Selection of logs uses standard semantic similarity on embeddings, but the method lacks adaptive or feedback-based techniques to mitigate harmful interference from irrelevant or misleading prior logs (see swings noted in **Table 3**, where some previously correct or unsolved problems become incorrect upon log incorporation). No mechanisms for detecting or counteracting negative transfer are proposed beyond manual thresholding.\n\n3. **Lack of End-to-End Theoretical Analysis or Guarantees:** The framework is empirically strong, and equations for RoPE handling are rigorously presented, but the broader questions of reliability, generalization, or even error propagation with retrieved KV caches across very long log histories are not analyzed. There's no formal proof or error bound on when reused reasoning might be counterproductive.\n\n4. The entire experimental validation is conducted on a **single model** (Llama-3.1-8B-Instruct). It is unknown how these findings generalize to other model architectures or, more importantly, to much larger models (e.g., 70B+). Larger models might be less reliant on logs, or their KV representations might behave differently.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6nqQdNGjNg", "reviewer": "Reviewer_JAu2", "summary": "This paper introduces Log-Augmented Generation (LAG), a framework designed to enable LLMs to reuse prior computations and reasoning traces at test time. The core technical innovation is to represent past execution logs not as text, but as key-value (KV) caches. The authors leverage the insight that a token's KV representation encapsulates the semantics of the entire preceding context. Their method encodes the full reasoning trace of a completed task into KV values but only stores the KV values corresponding to the final model response, creating a compact yet contextually rich representation. For new tasks, relevant logs are retrieved and their KV caches are injected into the model's generation process after adjusting for positional information. The paper evaluates LAG on both knowledge-intensive and reasoning-intensive QA datasets, demonstrating significant improvements in accuracy and efficiency over baselines, including standard agentic frameworks and reflection-based memory systems.", "questions": "1. The experiments use a static log store. What new challenges would arise in a fully dynamic setting where new logs are added after every task? Specifically, how would the system manage retrieval efficiency and the risk of \"polluting\" the log store with noisy or incorrect reasoning over a long period?\n\n2. Table 3 shows that logs can cause previously correct answers to become incorrect. Could you provide a qualitative example of this failure mode? What types of retrieved logs are most likely to mislead the model, and does this suggest a need for a more advanced mechanism to filter or weight retrieved logs based on some confidence metric?\n\n3. The paper argues against abstraction because it can be lossy. However, abstraction also enables generalization. Your method seems optimized for tasks with significant sub-problem overlap. How do you expect LAG to perform on new tasks that are only distantly or conceptually related to problems in the log store, where direct computational reuse is not possible? Would an abstraction-based method potentially perform better in such a scenario?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "**Novel Use of KV Cache for Reasoning:** The paper's primary contribution is the repurposing of KV caching from a tool for computational efficiency to a mechanism for improving reasoning and accuracy by reusing past computations. This presents a compelling and conceptually distinct alternative to reflection-based memory systems.\n\n*Effective Encoding Mechanism:** The technical approach of encoding the full reasoning history while only storing the KV values for the last model response is a clever solution to the trade-off between context richness and storage cost .\n\n**Strong Empirical Performance:** The paper's claims are well-supported by a rigorous set of experiments with comprehensive baselines, including a no-memory agent, a text-based version of LAG, and prominent reflection-based systems. It demonstrates consistent and significant improvements across four challenging datasets spanning both knowledge- and reasoning-intensive tasks.", "weaknesses": "**Static Log Store Evaluation:** The experiments are conducted using a static log store that is built offline. This is a simplified setting that sidesteps critical challenges of a true lifelong learning system, such as the computational cost of retrieval in a massive and ever-growing log store, and the long-term effects of noise accumulation.\n\n**Insufficient Analysis of Error Propagation:** The log store is intentionally populated without filtering for correctness to simulate a realistic scenario. While Table 3 shows a net positive impact, it also confirms that logs can mislead the model ($C \\rightarrow I$) 7. The paper lacks a deeper qualitative or quantitative analysis of these failure cases and does not propose mechanisms to mitigate the risk of negative transfer from reusing flawed reasoning traces.\n\n**Simple Retrieval Mechanism:** Retrieval is based on standard top-k semantic similarity 8. While effective in these experiments, this simple approach may become a bottleneck as the log store grows and the diversity of reasoning traces increases, potentially failing to retrieve the most relevant computational context.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)