Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning
Authors: Jiwon Song
Dongwon Jo
Yulhwa Kim
Jae-Joon Kim
Venue: NeurIPS 2025 poster ()
Keywords: large language model
reasoning
test-time compute scaling
KV cache
Primary area: deep_learning
TL;DR: Reasoning Path Compression accelerates inference of reasoning LLMs by periodically compressing KV cache of generated tokens exploiting their semantic sparsity.

Abstract: Recent reasoning-focused language models achieve high accuracy by generating lengthy intermediate reasoning paths before producing final answers.
While this approach is effective in solving problems that require logical thinking, long reasoning paths significantly increase memory usage and reduce throughput of token generation, limiting the practical deployment of such models.
We propose Reasoning Path Compression (RPC), a training-free method that accelerates inference by leveraging the semantic sparsity of reasoning paths.
RPC periodically compresses the KV cache by retaining cache entries that receive high importance score, which are computed using a selector window composed of recently generated queries.
Experiments show that RPC improves generation throughput of QwQ-32B by up to 1.60$\times$ compared to the inference with full KV cache, with an accuracy drop of 1.2\% on the AIME 2024 benchmark.
Our findings demonstrate that semantic sparsity in reasoning traces can be effectively exploited for compression, offering a practical path toward efficient deployment of reasoning LLMs. Our code is available at https://github.com/jiwonsong-dev/ReasoningPathCompression.

Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning
Jiwon Song Dongwon Jo Yulhwa Kim Jae-Joon Kim Seoul National University SSungkyunkwan University [jiwon. song, dongwonjo, kimjae joon @snu. ac (yulhwakim) @skku. edu
Abstract
Recent reasuee language models achieve high accuracy by generating lengthy intermediate reasoning paths before producing final answers. While this approach is effective in solving problems that require logical thinking, long rea- soning paths significantly increase memory usage and reduce throughput of token generation, limiting the practical deployment of such models. We propose Reason- ing Path Compression (RPC), training-free method that accelerates inference by leveraging the semantic sparsity of reasoning paths. RPC periodically compresses the KV cache by retaining cache entries that receive high importance score, which are computed using selector window composed of recently generated queries. Experiments show that RPC improves generation throughput of QwQ-32B by up 1.60x compared to the inference with full KV cache, with an accuracy drop 1.2% the AIME 2024 benchmark. Our findings demonstrate that semantic sparsity in reasoning traces can be effectively exploited for compression, offering a practical path toward efficient deployment of reasoning LLMs. Our code is available at com/ /tttps://iite wtwpaauw..o/.jww//snee.e.eeeeeee hwwp://wwnnooooooeeooo dev htt.ggg//wweeeeoooaaaaaaaooeeeee
Introduction
Large language models (LLMs) equipped with reasoning capabilities have expanded the application of LLMs beyond simple natural language processing tasks complex moblem-solviigg tasks such as Science, Technology, Engineering, and Mathematics (STEM) reasoning and code generation. Early reasoning approaches primarily focused on guiding LLMs through explicit step-by-step logic to facilitate more interpretable and accurate outcomes M. Recently, advanced reasoning LLMs,, such as OpenAI ol [2] DeepSeek-RI [3], adopted the concept test-time compute scaling 5 This method involves generating longer, iterative reasoning outputs, which significantly enhance accuracy. Such iterative generation allows models to carefully evaluate intermediate reasoning steps, refine outputs through internal reflection, and ultimately handle tasks requiring complex reasoning.
Though reasoning LLMs have been widely adopted due their ability handle complex tasks through complicated reasoning processes, reasoning LLMs face challenges in inference efficiency due their tendency generate long reasoning sequences. The long token sequences required for detailed reasoning processes substantially increase the KV cache overhead during inference. For example, the reasoning path of OpenAl's 03-mini-high can exceed 50K tokens (6. and Claude 3.7 Sonnet [7] supports reasoning sequences of up to 64K tokens. Such a long token generation imposes critical memory and computational overhead, significantly slowing down inference. Consequently. is crucial to develop KV cache compression techniques to mitigate these inference efficiency issues and support practical deployment of reasoning L.MMs.
*Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Question: Four identical spin. 1/2 particles are placed one one-dimensional infinite potential longth L. Calculate the energies of the ground state, first excited statc, and sccond excited statc of the system in terms enorgy E. (Horo C= m*h1/2mL²)
Short Input (-200 Tokens)
Resszring LLM
Long Output (10,000+ Tokens)
Reasoning Path Example from QwQ--32
[--]] Since there are four particles, the ground state should have the first two energy levels each filled with two particles. Wait, no, aduullly, lovel can hold two particies, so four partickes, that would be n=1 and have two, n=2 can hava so total four. So the ground state would have particles in n=1 and two in n=2?
filled
particles? Wait, no,
Waif a second, buf I need to be careful. Lef me think again [--] So for four particles, the lowest energy configuration is to fill up n-1 (two particles) and n=2 (another two particles). Therefore, the total energy would be 2E, + 2Eg Since E, = n"E, so that would be 2(1"E) 2(2"E) = 8E 10E. Wait, but hold on, that seems lkk the total energy. Hmm, but let me confirm.
Wait, actually, each particle n=1 lovel contributes each [--]] So total energy is 8E= 10E. So ground state energy 10E?
But wait, hold on. Lef me check again. The energy lovals for cach particio are F, n"E, so n is 1-2 is 4E, n- 3 is 9E, etc. So for two particles in n=1, their tobal contribution is 2E. Two particles in n=2, thoir contribution =E.
So total 10E. That seems right. [--]
Example of a reasoning path of reasoning LLM. Redundant reasoning steps (e.g., repeated checks and re-derivations) are visually highlighted, illustrating the semantic sparsity that motivates our compression method. The parts highlighted in the same color semantically identical.
Figure I:
Although there are several existing works on compressing KV cache for long sequences these works primarily focus on efficient handling of long input prompts. In contrast, the problem of efficiently managing the KV cache for long generated sequences has received limited attention. Unlike input prompts, whose importance can be easily assessed at prefill stage [8) generated tokens pose challenge because their future relevance is often unpredictable. As token seemingly insignificant at one point might become crucial later, naively discarding such tokens can substantially degrade model accuracy.
However, as illustrated in Figure I we observe that sequences generated during reasoning processes exhibit distinct properties compared sequences generated in conventional LLM decoding. Specifi- cally, reasoning sequences frequently revisit previous cases repeat similar logic, so they have low information density relative to their length. We refer to this phenomenon as the semantic sparsity of reasoning paths. This sparsity highlights the inefficiency of retaining all KV entries and the possibility to selectively remove KV cache corresponding to less important tokens without disrupting the overall reasoning process.
Motivated by this observation, we propose Reasoning Path Compression (RPC), method for accel- erating inference in reasoning LLMs by compressing the KV cache associated with explicit thinking tokens. RPC compresses KV cache periodically during decoding. significantly reducing overhead compared to previous step-wise compression techniques which compress KV cache at each decoding step. At each compression interval, it estimates token importance based on attention allocation over a recent window and retains only the top-ranked entries according to fixed compression ratio. This design preserves recent context while discarding low-impact KV entries, mitigating performance degradation. By applying RPC to QwQ-32B [12]. we reduce the KV cache size of generated tokens by up to 75%, and improve decoding throughput by up to 1.60x, while keeping the pass@) I drop on the AIME 2024 [3] dataset within 1.2% compared to the inference with full KV cache.
2 Background
2.1 Reasoning LLMs
Reasoning LLMs solve problems by generating explicit intermediate steps, known as reasoning paths, instead of directly producing an answer 3 12 [4]. This behavior is reinforced by the way such models are trained: reasoning LLMs are typically fine-tuned with reinforcement learning objectives that reward correct answers after multi-step inference, thereby encouraging longer generations. Consequently, the lengths of generated sequences increase as training progresses [55. Allocating up to 32K tokens for explicit reasoning has yielded steady accuracy gains across complex reasoning

---

(a)
(b)
Throughpur (tokemsis)
250
242.5
2331
213,8
Peak Peak Memory (GB)
200
174.0
204.2
200
135.5
101.1
100
75.4
84.0
00M
1024
2048
4096
16384
1024
4096
16384
32768
Generation Length
Generation Length
Figure 2: (a) Token generation throughput and (b) peak memory of QwQ-32B different generation lengths. The results are evaluated on 4x H100 GPUs with batch size 16.
benchmarks without showing signs of plateau, raising expectations of further improvements with even larger thinking budgets [16]. Such extensive token generation significantly enlarges the KV cache, increasing memory usage and reducing inference throughput. As shown in Figure 2 generating sequences of 16K to 32K tokens dramatically reduces throughput while sharply increasing peak memory usage.
To mitigate the overhead of generating long reasoning paths, large body of work has fo- cused on training based approaches that encourage reasoning LLMs to produce shorter se- 21 [2].. relatively few attempts have been made to improve reasoning efficiency at inference time (23 These approaches utilize length-aware training objectives: either en- couraging the generation of short sequences introducing mechanisms to compress tokens into latent representations [24]. However, their effectiveness typically remains limited when applied to complex reasoning benchmarks widely used to evaluate modern reasoning LLMs (c.g.. LiveCodeBench (25)). For example, although LightThinker [22] achieves competitive accuracy with shortened reasoning paths relatively simpler reasoning tasks like MMLU (26) and BBH (27), our experimental results in Section 4.3 indicate significant performance degradation when evaluated on more complex reasoning benchmarks. This discrepancy arises primarily due the conflicting training objectives. The sewooiing-orinnter objectives aim to promote detailed reasoning steps, whereas the length- objectives encourage shorter outputs. Thus, effectively training reasoning LLMSs consistently produce shorter reasoning paths remains challenging.
quences [17 18 20
2.2 KV Cache Compression
The degradation of throughput and the increase in memory usage observed when processing long sequences with LLMs primarily result from growth KV cache size. Thus, there are many attempts to directly compress the KV cache, but these works primarily focus efficient handling of long input prompts. For example, SnapKV (8] and HeadKV [9| are specifically designed to compress KV cache associated with long input contexts. These methods do not address the compression of generated tokens produced in reasoning paths.
Other techniques like H2O 10 and TOVA [1] attempt to extend KV cache compression mechanisms to support basic levels of compression during generation. These methods maintain the KV cache within predefined budget by evicting tokens whenever the cache reaches this size limit during decoding. However, their designs predominantly target scenarios involving long input sequences and relatively short outputs, they are effective when identifying and evicting less relevant input tokens is critical for efficient output generation. Hence, H2O and TOVA struggle to preserve accuracy when applied to reasoning LLMS (see Section 4.3 Moreover, while setting a fixed KV cache budget straightforward in input-dominated scenarios, is challenging to predefine cache budgets for reasoning LLMs, they inherently produce long output sequences of varying lengths. Overall, there are currently no KV cache compression methods tailored to reasoning LLMs.
Reasoning Path Compression
3.1 Motivation: Semantic Sparsity of Reasoning Paths
Reasoning L.LMs do not directly generate the final answer. Instead, they produce reasoning paths, which often contain redundant segments offering little new information, such as repeated logical

---

Reasaning
Non- Reasoning
12
11.59
11.28
11
10:67
Entropy
10.33
10.08
10
9.82
9.47
1024
Leigut
Figure 3: 3-gram Shannon entropy comparison between reasoning LLM and non-reasoning L.M.
steps or re-evaluations of previous generated reasoning. As previously presented in Figure 1 such redundancy frequently observed in model-generated reasoning paths. Additional examples are provided in Appendix AA] We refer to this phenomenon, the presence of extended spans of generated tokens that are semantically redundant, as semantic sparsity. To quantify semantic sparsity, we compute the n-gram Shannon entropy using base-2 logarithm, defined as:
Hn =
Σ gEGn.
p(g) loga p(g)
where Gn denotes the set of all unique n-grams of length and p(g) is the empirical probability of each n-gram g.
To analyze semantic sparsity of reasoning paths, compare the redundancy in sequences generated by conventional LLMs and reasoning LLMs. For this comparison, we use 3-gram entropy to measure phrase-level repetition and evaluate two models with identical architecture (LLaMA-3.1-
8B-Instruct (8
3 a reasoning-orienter model, is tested on [29]. tuned for long-form writing, is tested on subset of HelloBench 30 consisting of prompts that require generating outputs exceeding 8192 tokens.
13 and DaiSSee--R--istill---laaa
AIME 2024
As shown in Figure 3 DeepSeck-Re RI-Distill-Llam-
consistently exhibits lower 3-gram entropy than Long Writer-8B across output lengths from 1024 8192 tokens. This indicates higher phrase- level repetition in reasoning paths compared to general long-form writing. These results provide quantitative evidence of semantic sparsity, suggesting that large portions the reasoning trace can be compressed with minimal impact overall coherence.
3.2 Overview of Reasoning Path Compression
We introduce Reasoning Path Compression (RPC), KV cache compression framework tailored for reasoning LLMs (Figure4) RPC leverages the semantic sparsity inherent in reasoning paths to efficiently eliminate KV entries. The key insight motivating RPC is that reasoning LLMs generate explicit reasoning steps, and many of these reasoning steps lose relevance as the reasoning process progresses. Exploiting this observation, RPC periodically compresses redundant KV entries during
RPC 4x (P=4, R=2)
Ingul Taken
Oulpul Taken
Evicted
Retained (Selected or Selector)
Evaluated Tokens
Selector Window Tokens
Step P+R=6
1st Compression
O1
02
O3
X
D4
X
OS
O6
Attention Score
C2
2d Compression
Step 2P+R=10
010
Figure 4: Illustration of RPC with compression interval P 4, selector window R = 2, and compression ratio c At each compression step, recent tokens are used to evaluate the importance of previously generated tokens.

---

token generation. Moreover, since recently generated tokens inherently rely on the context provided by preceding tokens, these recent tokens serve as essential indicators of contextual importance. Thus, RPC assesses the relevance of previously generated tokens by analyzing how strongly they are attended to by the most recent tokens, referred to as selector window. All compression decisions are made dynamically during inference based solely on attention-derived statistics. Hence, RPC does not require any model modification or additional training, and it is straightforward to integrate RPC into existing inference pipelines of reasoning LLMs.
3.3 Periodic KV Cache Compression Dynamics RPC
One of the unique features of RPC compared to other KV cache compression methods is its periodic approach to KV cache compression. The KV cache compression dynamics of RPC is controlled by two critical hyperparameters: the compression interval P. which represents how frequently KV cache compression is triggered, and the size of the selector window R. which denotes the number of recent tokens used assess importance.
As illustrated Figure 4 RPC waits for P + R tokens to be generated to start the first compression eycle. At this point, the importance of the initial set of P tokens is evaluated using the selector window composed of the most recent It tokens. Given a target compression ratio c, RPC retains only the top P tokens based on their importance scores. Subsequent compression cycles are triggered с each time an additional P tokens have been generated.
It is important to note that during each periodic compres- sion cycle, RPC evaluates a combined set comprising both tokens retained from the previous cycle and newly generated tokens, rather than compressing only newly generated ones. By jointly reassessing all these tokens at each cycle, RPC nat- urally allows outdated tokens to fade out as the reasoning path progresses. As result, the reasoning context remains prop- erly updated and relevant throughout the inference process, even after multiple cycles of KV cache compression.
# KV Cache Entries
w/o RPC 1
RPC 1/c
Specifically, at the second compression cycle, the selector window, now updated include the latest R tokens, evalu- ates the importance of the previously retained & tokens and e the newly generated set of P tokens. Among these P é + P tokens, RPC retains only the top 2P tokens with the highest importance scores and discards the rest. Generalizing this procedure, the N-th compression cycle, the total number of tokens evaluated with the selector window is (N-1)f' + P.
KV Pruning
# Decoding Steps
Figure 5: KV cache size with and without RPC.
RPC retains only the top NT' tokens with the highest importance scores from this set. As selector tokens are always preserved, the total number of KV entries remaining after the N-th compression cycle is NP + R. As shown in Figure this periodic compression effectively regulates the size of KV cache over time.
To fully leverage the advantages of periodic compression, the compression interval must be carefully selected. A small P value may lead to accuracy degradation after compression, as the semantic context is too limited. On the other hand, a large P provides a broader semantic context for effective compression, while it introduces computational inefficiency and higher peak memory usage by delaying the compression. Given the significance of the compression interval P, an ablation study analyzing its impact and recommendations derived from the analysis discussed in Section 4.5
3.4 Important Token Selection with Selection Window
Another unique feature of RPC is the concept of the selector window used for selecting important tokens. Previous KV cache compression methods employ various strategies for calculating token importance. For example, SnapKV V computes attention scores relative to the final tokens in the input prompt, based on the observation that the last segment of the input shows similar attention allocation pattern to the generation stage. H20 averages attention scores across all preceding tokens, and TOVA mimics RNN operations by reusing the attention scores calculated during token generation gating
5

---

Algorithm 1: Important token selection algorithm of RPC
Input: generation step query of step qt, KV cache CRV, selector query cache Co Output: updated Ckv, updated C
// Cache selector queries
if (t R) 2 and (t R)) mod P R then
L Append Qi to Co
// Compress KV cache every P steps
if (t. R) 2 0 and (t - R) mod P= then
8 Importance of tokens in CKV:
// Compute importance score Cump t KV cache with top- N-P. importance scores; ; // Retain important KV cache
OKV f- Cemp UCKvI-R:]s
// Retain KV cache of selector window // Reset selector query cache
Ce +1;
return CKV. Co
scores for the KV cache eviction. In contrast, RPC leverages the observation that recently generated tokens in reasoning paths represent logical outputs derived from preceding contexts. Therefore, attention scores relative these recent tokens can effectively indicate the relevance of previously generated tokens.
The algorithm for selecting important tokens in RPC is presented in Algorithm RPC evaluates token importance using attention scores aggregated across a selector window of the R most recent tokens and all attention heads. Then, promote coherent token selection and reduce token-level noise, RPC applies local average pooling. Formally, the importance of each past token t at each layer defined as:
Importance()) 2w
Aun (yr,kktt4
R. H
Here, Attn/, (gr. ke+z) denotes the attention weight from the r-th selector token token generated at t i-th generation step at head h of layer é. The pooling window size controls the smoothing level, encouraging contiguous retention of semantically related tokens. To eliminate redundant computations and efficiently compute these importance scores, RPC caches the query vectors of selector tokens.
The selector window size It determines how many recent tokens RPC uses to assess the importance of previously generated tokens. A smaller Rt may lead unstable or noisy importance estimations, as scoring can be dominated by limited number of tokens. In contrast, larger values of It increase memory overhead by requiring additional caching of query vectors. Thus, choosing an appropriate value for R involves balancing the robustness in token scoring with computational overhead. A detailed ablation study and recommendation for optimal R values are provided in Section 4.5
Experiments
4.1 Experimental Setup
Models and Datasets. We evaluate RPC using two open-source reasoning LLMs with different model sizes:
with 7B parameters [3] and QwQ-32B with 32B parameters 122. All outputs are generated using nucleus sampling with temperature = 0.6 and top-p 0.95. For QwQ-32B, additionally set top-k 40 following the model's recommended decoding configuration. The maximum number generated tokens capped at 32768, following the default settings of tested models.
Our evaluation covers three reasoning-intensive benchmarks: American Invitational Mathematics Examination (AIME) 2024 for mathematical reasoning, LiveCodeBench 25 for coding tasks, and IFEval [31] for instruction following. We sample completions per instance to compute pass@ @ass@1 1, where A 8 for AIME 2024, & = 4 for LiveCodeBench, and & 1 for IFEval, respectively.
Datasets.

---

DeepSeek Owen-7B
QwQ
65. 55.5%
60%
66.0%
APC
Redurdancy
45%
45%
48.0%
30%
33.8%
30%
20.2%
15%
15.4%
15%
13.3%
5.3%
Nw1
N-2
N-4
N=1
N-2
N-4
Figure 6: Redundancy rate comparison between full KV and RPC.
Table
Accuracy (%) comparison between baselines and RPC.
ReepSeek-RI--issttll-eeeee
QwQ-32B
Method
AIME 2024 LiveCodeBench
IFEval (pass@)
AIME 2024
LiveCodeBench (pass@)
IFEval
(pass@11
(pass@)
(pass@1)
(pass
Full KV Cache
55.5
37.6
55.1
79.5
63.4
83.9
H20
42.5
22.5
51.8
75.0
54.2
74.3
TOVA LightThinker
42.5
21.5
48.8
70.0
43.8
50.6
6.7
0.7
25.1
RPC (P 4096)
52.9
35.9
56.6
78.3
62.2
82.6
RPC (P 1024)
50.4
33.5
78.3
61.2
81.7
Implementation Details. Our implementation uses FlashAttention-2 32 as the attention kernel for
all decoding layers and built on top of HuggingFace Transformers v4.45 33 Unless otherwise specified, we use the following default RPC hyperparameters: We set the selector window size R to 32 and apply local pooling with window size = 3 for importance smoothing. The compression interval P is set to 1024 or 4096. The target compression ratio is set to 4x by default.
Baselines. We compare our proposed RPC with training-based reasoning path compression method, LighTThinker (22), and previous KV cache compression techniques, H2O [0] and TOVA M.. To ensure fair comparison with H2O and TOVA, we set their KV cache budgets to match the overall compression ratio (4x) x of RPC. For each of the evaluation datasets, we profile the average generation length of the original reasoning LLMs with full KV caches, and allocate 25% of this average length as fixed KV cache budget for all prompts within that dataset. Meanwhile, LighTThinker does not offer direct control over the compression ratio, so we measure its effective compression ratio after inference.
4.2 Redundancy Reduction
We quantitatively evaluate the redundancy-reducci effect of RPC using an embedding- bbased simi- larity analysis. For each model, we generate outputs on the AIME 2024 dataset using both the full KV baseline and RPC. The generated outputs are segmented into sentences and pairwise cosine similarities are computed between all sentence embeddings within the same output. Two sentences are considered semantically similar if their cosine similarity exceeds 0.75. We define the redundancy rate as the proportion of sentences that have more than N semantically similar counterparts within the same output, where N E 21,
As shown in Figure 66 the redundancy rate significantly decreases after applying RPC, across all models. Specifically, the proportion semantically repetitive sentences (i.e., -=1) is reduced by nearly half, and the gap widens for higher redundancy thresholds (N=2, 4). This indicates that RPC not only removes verbatim repetitions but also suppresses subtle paraphrased duplications that frequently appear in reasoning trajectories. These results provide strong evidence that RPC effectively leverages semantic sparsity maintain concise yet coherent reasoning sequences.
Additional details of the embedding redundancy analysis and visualized examples of RPC's
token selection provided in Appendix B
4.3 Accuracy Evaluation
Table compares the accuracy between RPC and the baseline methods. LighTThinker, a training- based reasoning path compression approach, shows the lowest accuracy across all benchmarks despite

---

(a)
DeepSeek-R1 -wt.-R11---tt-eemmmee -weill-Qwen-78
750
Throughpue (tolenas)
RPC
Throughput, (tolersi))
200
600
653.0
709.0
671.4
575.2
120
174.0
207.3
188.0
147.3
504.8
117.5
342.3
OOM
150
8192
16384
32768
8192
16384
32768
Output Length
Output Length
(b)
DeepSenk-AR A1 HL-Distll-Qwenn.
QwQ-328
Full KV RPC
300
75
Peak Memoory (GB)
Pesk Merrory (GO)
240
60-
180
65
75.7
30-
120
204.2
36.2
135.5
126.9
163
15-
30.6
15.0
2004
81.92
16384
32768
16384
32768
Output Length
Output Length
Figure 7: (a) Throughput (tokens/s) and (b) peak memory usage (GB) comparison between RPC (P 4096) and full KV cache inference.
operating with mild compression ratio (1.49x, 1.41 1.57x for AIME 2024, LiveCodeBench. IFEval, respectively). This result highlights the limited effectiveness of length-aware training approaches for reasoning LLMs.
Other KV cache compression baselines, such as H2O and TOVA, achieve higher accuracy than Ligh(Thinker but still exhibit significant gap compared to full KV inference. Moreover, their requirement of a predefined KV cache budget limits applicability in real-world scenarios where the total generation length cannot be known in advance.
In contrast, RPC achieves accuracy comparable to full KV inference without any additional training or prior knowledge of the output length. With compression interval of P == 4096, RPC successfully limits the accuracy drop to within 2.6% for DeepSeek- R RI-Distill-Qwee -well--Qeee-BBB and 1.2% for QwQ-32B on AIME 2024. A shorter interval (P 1024) slightly reduces accuracy while providing stronger compression and efficiency. Therefore, careful selection of / is important, and we provide an ablation study analyzing its impact in Section 4.5
4.4 Efficiency Evaluation
We evaluate the efficiency of RPC in terms ooken-generation throughput and peak memory usage. All experiments conducted using input prompt with 128 tokens and measure throughput for generating sequences of 8192, 16384, 32768 tokens, with a batch size of 16. The compression interval P is set to 4096. Throughput and memory measurements for DeepSeek-RR 1-Distill-Qwen-7BB obtained on a single NVIDIA H100 SXM GPU, while QwQ-32B evaluations are conducted on four S presents the throughput and peak memory improvements achieved by
H100 SXM GPUs. Figure RPC relative to the original models with full KV cache. Additional analyses on the efficiency are also provided in Appendix
Throughput. As shown in Figure 7(a). RPC consistently improves token generation throughput with particularly large gains observed for long generation length (c.g.. 32768 tokens), scenario commonly encountered with reasoning LLMs. RPC achieves 1.68 x throughput improvement when generating 32768 tokens with DeepSeek-PP -I when generating 16384 tokens with QQQQ32B. Notably, QwQ-32B with full KV cache cannot handle reasoning tasks with generation lengths of 32768 tokens runs out of memory. However, RPC successfully enables token generation at this length.
and 1.60 throughput improvement
Memory Consumption. As shown in Figure bb), RPC effectively reduces peak memory usage by periodically compressing the KV cache. Since peak memory usage includes contributions from model parameters, intermediate activations, and the KV cache, the reduction in peak memory is not

---

Throughput (tokens/s)
Peak Memory
AIME 2024 (pass@1)
800
45
Through hput (tokensis)
52.9
54.2
53.8
50.4
600
33
Peak Memory
500
30
400
25
64
256
4096
8192
16384
Pruning Interval P
Figure 8: Effect of compression interval P on accuracy, throughput, and peak memory.
directly proportional the KV cache compression ratio. Nevertheless, as the KV cache becomes the dominant factor in peak memory usage for longer generation lengths, RPC provides increasingly -1--sei---------------
substantial memory savings as generation length grows. For reduces peak memory usage from 75.7GB to 36.2GB when generating 32768 tokens, thereby RPC achieves over 50% memory reduction. Similarly, for QwQ-32B, RPC reduces the overall memory requirement by over 50%, thereby resolving the out-of-memory issue for the generation of 32768 tokens. These results demonstrate that RPC effectively mitigates the memory bottleneck inherent in long-sequence generation of reasoning LLMs by compressing KV cache.
RPC
4.5
Ablation Studies
To better understand the effect of key hyperparameters in RPC, perform ablation studies using the AIME 2024 dataset. We analyze two critical components: the compression interval P. which determines how often KV-cache compression is applied, and the selector window size R, which controls the number of recent tokens used for attention-based importance scoring.
Compression Interval. We evaluate compression interval from 4 to 16384 to examine the trade-off between compression interval, reasoning accuracy, and inference efficiency (throughput and peak memory). As shown Figure g reasoning accuracy improves as P increases. It indicates that overly frequent compression disrupt the reasoning process by prematurely evicting tokens critical for subsequent reasoning steps. However, when P becomes excessively large (e.g.. P = 8192), throughput declines and peak memory usage rises significantly, compression. Therefore, selecting an appropriate P value is essential to balance accuracy preservation and efficiency gains. Here, the configurations p 4096 and P 1024 represent practical choices that offer strong balance between performance and efficiency in reasoning-intensive scenarios.
large P delays the KV cache
Table 2: Effect of selector window size R.
P
Metric
R
32
128
AIME 2024 (pass@) 1)
49.2
48.3
52.9
49.2
4096
Throughput (tokens/s)
662.54
662.84
671.38
673.26
Peak Memory (GB)
28.72
28.72
28.72
29.38
AIME 2024 (pass@] 1)
45,8
49.2
50.4
50.0
1024
Throughput (tokens/s)
746.21
745.08
751.69
742.46
Peak Memory (GB)
24.62
24.62
25.15
27.37
Selector Window Size. We evaluate the impact of the selector window size R on the RPC algorithm by evaluating R (1. 8, 32, 128}. As shown in Table 2 small R values such as 1 and 8 yield relatively low accuracy (e.g., below 50% with P 4096), because small values can result in unstable selection of semantically critical tokens. This effect is more pronounced for P = 1024 than P 4096, as tokens are evicted more frequently with smaller P. Therefore, I must be sufficiently large to ensure robust importance estimation. However, excessively large R (e.g., 128) can negatively impact accuracy, as older selector tokens may not reflect the current reasoning context effectively. Because varying R has only marginal effects on throughput and peak memory usage, accuracy is

---

the primary consideration when selecting R. Based on our results, R = 32 is the best choice as it provides the highest accuracy.
Conclusion
We introduce Reasoning Path Compression (RPC) for compressing KV cache of reasoning L.MMs. We observe that reasoning paths often contain redundant segments and inherent semantic sparsity. RPC leverages this characteristic by periodically compressing the KV cache and employs an importance scoring mechanism based a selector window composed of recent queries. As RPC does not require any additional training or model modifications, it can be applied to broad range of reasoning LLMs. Experimental results demonstrate that RPC compresses the KV cache by 4x with accuracy degradation limited to 1.2%. This aggressive KV cache compression results in up to 1.60 throughput improvement. Moreover, RPC successfully resolves the out-of-memony issue encountered by large reasoning models with 32B parameters when generating long reasoning paths of up to 32K tokens, by achieving 50% reduction of overall memory requirement.
Acknowledgments and Disclosure of Funding
This work was supported part by Institute of Information & communications Technology Plan- ning & Evaluation (IITP) grant funded by the Korea government (MSIT) No.RS-20025-027315555 Development of Low Power Training/Inference Technologies based on AI Semiconductors, RS-2025- 10692981: AI Semiconductor Innovation Research Center, 55--0023-0025600 semiconductor support program nurture the best talents, No.2021 2202100-02068 Artificial Intelligence Innovation Hub), Samsung Research Funding Center under Project SRFC-TC1603-533 and BK21 FOUR program Seoul National University.
artificial intelligence
References
[1] Jason Wei. Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of thouuht prompting elicits reasoning in large language models. Advances in neural information procesing systems, 35:24824-24837 2022.
Aaron Jacch, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Hel- yar, Aleksander Madry, Alex Beutel, Alex Camey, et al. Openai system card, arXiv preprint arXiv:2412. 16720, 2024.
[3] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. /2948, 2025.
[4] Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. Recursive introspection: Teaching language model agents how to self-improve. Advances in Neural Information Processing Systems, 17:55249-55228 2024.
[5] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.033)4 2024.
[6] Marthe Ballon, Andres Algaba, and Vincent Ginis. The relationship between reasoning and performance in large language models o3 (mini) thinks harder, not longer. arXiv preprint arXiv:2502.15631 2025.
[7] Claude 3.7 Sonnet anthropic.com.
[Accessed
03-05-2025).
[8] Yuhong Li, Yingbing Huang, Bowen Yang. Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. Snapkv: Llm knows what you looking for before generation. Advances Neural Information Processing Systems, 37:22947 -2970, 2024.
[9] Yu Fu, Zefan Cai, Abedelkadir Asi, Wayne Xiong, Yue Dong, and Wen Xiao. Not all heads matter: A level kv cache compression method with integrated retrieval and reasoning. arXiv preprint a1012:2410.19258, 2024.
10

---

[10] Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng. Ruisi Cai, Zhao Song. Yuandong
Tian, Christopher Ré, Clark Barrett, et al. H22: Hlaav-hhtterr oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems, 36:34661-34710, 2023.
[11] Matanel Oren, Michael Hassid, Nir Yarden, Yossi Adi, and Roy Schwartz. Transformers are multi-stale state mns. arXiv preprint
2024.
[12] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning. March 2025.
[13] Mathematical Association America. American invitational mathematics examination. https://mas /maa org/maa- invitatioonala www.iiiiiinala-aoooooiooooo compeeooooooccoootttttttttoooo cttomttttiiins////ooo [Accessed 15-05- 2-005.025J.
[14] LG Al Research, Exaone deep: Reasoning enhanced language models, arXiv preprint Ariiv:2003 12524,
2025.
[15] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with Ilms. arXiv preprint arX11:2501.12599 2025.
[16] An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, al. Qwen3 technical report. arXiv preprint wxxv:25055.03388 2025.
[17] Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable thain-of-thought compression llms. arXiv preprint arXiv:2502. 12067, 2025.
[18] Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se- Young Yun. Self-training clicits concise reasoning in large language models. arXiv preprint arXiv:2502.20122, 2025.
[19] Haotian Luo, Shen, Haiying Hc, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: sength-harmonizing fine-tuning for ol- reasoning pruning. arXiv preprint arXiv:2501. /2570, 2025.
[20] Daman Arora and Andrea Zanette. Training language models to reason efficiently. arXiv preprint MiXiv:2502.0446: 2025.
[21] Yi Shen, Jian Zhang. Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang. and Shiguo Lian. Dast: bifficulty-ddaptivv slow-thinking for large reasoning models. arXiv preprint 0444 2025.
[22] Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. Lightthinker: Thinking step-by-step compression. arXiv preprint arXiv:250.11588) 2025.
[23] Chenxu Yang, Qingyi Si, Yongjic Duan, Zheliang Zhu, Chenyu Zhu, Qiaowci Li, Minghui Chen, Zheng Lin, and Weiping Wang. Dynamic carly exit in reasoning models, 2025.
[24] Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey efficient reasoning for large language models. arXiv preprint arXiv:2503. 1 2025.
[25] Naman Jain, King Han, Alex Gu, Wen-Ding Li. Fanjia Yan, Tianjun Zhang. Sida Wang, Armando Solar- Lezama, Koushik Sen, and lon Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.077 07974, 2024.
[26] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Mcasuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
[27] Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung. Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, and Jason Wei. Challenging big- big-bench tasks and whether chain-of ofohhhghht can solve them. arXiv preprint arXiv:2210.09261. 2022.
[28] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten. Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21 2783, 2024. 2024.
[29] Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng. Siqi Zhu, Lei Hou, Yuxiao Dong Jie Tang, and Juanzi Li. Longwriter: Unleashing 10,000+ word generation from long context llss. arXiv preprint arXiv:2408.00 07055. 2024.

---

[30] Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou. Jiaheng Liu, Wenge Rong. Zekun Moore Wang, Jian Yang. Ge Zhang, al. Hellobench: Evaluating long text generation capa- bilities large language models. arXiv preprint arXiv:2409. 16191, 2024.
[31] Jeffrey Zhou, Tianjian Lu, Swaroop Mishra. Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Hou. Instruction -oollwiion-followinng evaluation for large language models. arXiv preprint arXiv:231..09111 2023.
[32] Tri Dao. Flashattention-22 Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.0866 2023.
[33] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer. Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-arr natural language processing In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online, October 2020, Association for Computational Linguistics.
[34] Karl Cohbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, arXiv preprint arXiv:2 10. 14168, 2021.
[35] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024.
[36] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019.
[37] Joshua Ainslic, James Lee-Thorp, Michiel De Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. arXiv preprint Xrx:: 13245, 2023.
12

---

A Examples of Redundant Reasoning Paths
In this section, provide additional redundant reasoning examples from the models that used for evaluations. These examples clearly illustrate the semantic sparsity present in the generation outputs of reasoning LLMs. Ellipses (l...) indicate omitted portions of the lengthy original output.
In Example A. QwQ-32B [2] identifies the correct answer early but redundantly re-derives multiple times, leading to excessive token generation. Each redundant derivation is highlighted with a box illustrate the inefficiency. The problem sampled from AIME 2024 dataset [3].
Example A.1
Problem. There exist real numbers and y, both greater than 1, such that
log, (1/") = log,, (x=10 = 10.
Find zy.
QwQ-32B Output. <think> Okay, so have this problem here: there exist real
numbers z and y. both greater than 1, such that log,, (yr" ) = 10, and log, (2**) = 10. need
to find the product zy. Hmm, logarithms with variables in the base and the argument. Let me try to unpack this step by step.
First, let's recall some logarithm properties. [...] So now, I have two equations:
1. x a 10
2. 4y 10
So maybe can solve for from one equation and substitute into the other? From equation 10
Plugging into equation 2:
4y 10/z
4ry
10 4y
= 10
10
4ry 100 zy 25
10
Wait, hold on, that seems too straightforward. But let me check. Wait, step by step:
Equation 2 after substitution:
4y divided by (12 is equal to 10. So dividing by (10) is multiplying by fo. Therefore,
4ry
4y 10
10
10
4ry 100 zy 25
10
Hmm, so is that the answer? Wait, but let me make sure didn't skip any steps where assumptions might be wrong. Let's check if this holds. [...]
Let's check the original equations.
First equation: log; (55). Wait, log, (53) is 5, but the problem says it should be 10. So that's not good. Hmm. So 5 and y = is not solution. Hmm, maybe my assumption that xy = 25 is correct but æ and y individually aren't integers? Alternttvely. maybe made an error in the substitution. Let check again. [...]
Let denote Equation I as 2- 10 and Equation 2 Let me multiply these two equations together:
1y-u
10.
4y
10
10 + 4ry
100
Ty 25
Therefore, despite the complication with substitution, the product zy is indeed 25 However, the problem states that such real numbers æ and y exist, so even if we can't find exact values
for vr and y, the product is 25|
Wait, but maybe we need to confirm that there actually exist such æ and y? [...] Therefore, even though we can't get exact values for æ and y, the product ry = 25 is indeed correct, and such numbers exist. Therefore, the answer is 25
13

---

Wait, but the problem says "Find zy", maybe 25 is the answer. and the existence is given, so even without finding a and y. through the algebra we can get that zy = 25. So maybe the answer is logy(4) log_ (v)
25 [...]
10/2
Cross-multiplyigg: 5 10 2y 50 2zy
ry 25
Oh! Wait, this is much simpler way! I didn't need to go through all the substitutions and transcendental equations. Since log, (x) is (w) substituting the expressions from both equations gives directly the relation. So indeed,
5
x
50
2zy
ry = 25
2y
10
So that's 's straightforward. The earlier complication was because I tried to substitute variables and ended up in loop, but this reciprocal relationship is key here.
Therefore, the answer is 25 So I think that's solid.
* **Final *
25
</think> [...]
similar pattern is observed in Example A.2 where Baabbee--RR-iisill--eeeaaa re-computes the correct solution despite having already reached Each redundant derivation is highlighted with box illustrate the inefficiency. The problem is sampled from GSM8K dataset (34
repeatedly
Example A.2
Problem. Alice has 20 quarters. She wants to exchange them for nickels and so she goes to the bank. After getting back from the bank, she discovers that 20% of the nickels are iron nickels worth $3 each. What is the total value of her money now?
DeepSeek- -i-t--Ril---i--eeeeeeee
Output. <think> Alright, so Alice has 20 quarters, and she wants to exchange them for nickels. Hmm, okay, let me think about this. First, I need figure how much money Alice has initially. Quarters
worth 25 cents each, right? So, if she has 20 quarters, can calculate the total value by multiplying 20 by 25 cents. Let me write that down: 20 quarters $0.25 per quarter. So, 20 0.25 equals... hmm, 20 divided by 4 is 5, so 20 0.25 is $5.00. Okay, so Alice has $5.00 in quarters. [...]
So, now I need to calculate the total value of her money now. That would be the value of the iron nickels plus the value of the regular nickels.
First, the iron nickels: 20 nickels $3 each. Let me compute that: 20 * 3 $60.00. Then, the regular nickels: there are 100 total nickels, minus iron nickels, so 80 regular nickels. Each regular nickel is worth $0.05, 80 0.05 $4.00. So, adding those together,
$60 + $4 $64.00
Wait, that seems like big jump from $5.00 to $64.00. Let me double-check my calculations.
She started with 20 quarters, which is S0.25 = $5.00. Exchanged them all for nickels, so 100 nickels. 20% of 100 is 20. so 20 iron nickels S3 each is 20 * $60. The remaining 80 nickels are regular, so 80 0.05 = $4. So, total $60 + $4 $64 That seems correct. But let me think again about the exchange. [...]
So, 20% of her nickels iron nickels worth $3 each, she effectively has 20 coins worth $3 each and 80 coins worth S0.05 each. So, total value indeed 203 800.05 60 + 4 $64
Wait, but in reality, exchanging quarters for nickels should just be different form
14

---

of the same money. So, why is the total value increasing? [...] Wait, maybe I'm overcomplicating. The key is that she exchanges all her quarters into nickels, which would give her 100 nickels, but 20% of them iron nickels worth $3 each, so 20 nickels at $3 and 80 at $0.05. So, the total value is 203 + 800.05 60 + + 964 So, that's the total value.
But let me think again, is this possible? Because exchanging 20 quarters for nick- els should just be different form of the same money, so why would the total value increase? [..]] But in this case, 20 of them are worth $3 each, so total value is 203 800.05 = 60 + $64.00 So, in effect, she's getting where some are worth more. So, this seems to be the case. So, maybe the answer is $64.00
money by exchanging her quarters into nickels
But just to make sure, let me recast the problem. Total initial value: quarters * $0.25 $5.00.
After exchanging, she has 100 nickels. But 20% of those are iron nickels worth $3. So, 20
nickels * $60, and 80 nickels * $0.05 $4. So, total is $60 $4 $64
Yes, that seems correct. Alternatively, maybe the question trying to trick me into thinking that the total value is still $5, but no, since 20% of the nickels are worth more, the total value increases.
Therefore, the total value of her money now is $64.00
Final Answer The total value of her money now is 64 dollars. </think> [...]
In Example A.3
repeatedly stating previously mentioned priors revisiting earlier examined cases during the problem-solving process. The problem is sampled from
QwQ-32B exhibits a similar pattern to Figure 1
dataset [35].
Example A.3
You perform high-throughput experiment white lupine find genes contribut- ing to resistance to the fungal disease anthracnose. As result, you receive three candidate genes of unknown function GI, G2, and G3. You create three knock-out mutants, gl, g2, and g3, and a set of double-mutants, glg2, glg3, and g2g3. You know that at least one of these genes test those mutant plants: do they have a higher sensitivity to anthracnose than the wild-type because they cannot produce certain gene products? After tests with the pathogen, you receive the following results where 100% is the level of resistance to the pathogen in control; 50% is half of the control's resistance; 25% a quarter of the control's resistance; 0% all plants
Problem.
transcription factor acting upstream of (an)other gene(s). You start to
show signs of infection:
resistance of gl: 75% of control
resistance of g2: 0% from control
resistance of g3: 50% from control
resistance of glg3: 10% from control
resistance of g2g3: 0% from control
resistance of glg2: 0% from control
Which conclusion regarding those genes' interaction can you draw from this experiment?
QwQ-32B Output, <think Okay, so have this problem about white lupine genes and their resistance anthracnose. Let try to figure this out step by step.
First, the setup: There are three genes, GI, G2, G3. They made knockout mutants for each (gl, g2, g3), and all the double mutants (glg2, g2, glg3, g2g3). The question is about their interactions, especially since at least one is a transcription factor upstream of others. The resistance levels given for each mutant and double mutant. The resistance percentages as follows:
15

---

gl: 75%
g2: 0%
g3: 50%
glg3: 10%
g2g3: 0%
glg:: 0%
Hmm. Let me think about how these genes might interact. Since a transcription factor is upstream, it probably controls the expression of another gene. If gene is a transcription factor, knocking it out would prevent the downstream gene from being expressed, leading to loss of function. Looking the single mutants first:
g2 has 0% resistance. That's the lowest, so losing G2 completely breaks resistance.
gl is 75%, so it's still somewhat resistant.
g3 is 50%, so halfway.
Now, the double mutants:
glg2 is 0%: same g2 alone.
g2g3 is also 0%: same g2.
glg3 is 10%: which is worse than either gl (75%) or g3 (50%).
Wait, so when you knock out GI and G3 together, the resistance drops further to 10%, which worse than either single knockout. That suggests that GI and G3 might be acting in parallel pathways? Because their combined loss worse than either alone.
But then looking at g2g3 and glg2, both are 0%, same as g2. So maybe G2 is epistatic the others? Because when you have g2, it's already 0%, adding another mutation doesn't t make it worse. That might mean that G2 is the main gene, and the others are downstream or in a different pathway that doesn't matter once G2 gone.
Alternatively, maybe G2 is the transcription factor, and it's required for the expres- sion of G1 and/or G3. If G2 is transcription factor, then knocking it out would prevent expression of the downstream genes. So G2 upstream of G1 and G3, then in the g2 mutant, both GI and G3 wouldn't it be expressed either. But then why does the gl mutant still have 75% resistance? That seems contradictory.
Wait, maybe G2 is not the transcription factor here. Let me think again. Alternatively, maybe G3 is the transcription factor. Let's see. If G3 is a transcription factor that controls G1 and/or G2. Then, the g3 mutant, both GI and G2 might be downregulated.
But then the g3 single mutant has 50% resistance. The g2 mutant has 0 Alternatively, GI might be a transcription factor. Let's consider that. G1 is a transcription factor, and it's controlling G2 or G3.
Looking the double mutants:
glg3 has 10%. Since GI alone is 75%, and G3 alone is 50%, combining them gives 10%, which is significant drop. So maybe G1 and G3 in parallel pathways, and their combined loss is more severe. [...]
Alternatively, maybe G2 is transcription factor that's needed to activate G1 and G3. Let think through that. If G2 is a transcription factor, then in the g2 mutant, neither GI nor G3 would be expressed. But then why does the gl mutant still have 75% resistance? If G1 is expressed in the g2 mutant, but the gl mutant (which still has G2) has 75%, that suggests that Gl can function even if G2 present. Wait, this getting confusing.
Alternatively, perhaps G3 transcription factor that activates G2. Let's see: If G3 is a transcription factor for G2, then in the g3 mutant, G2 might not be expressed. Then,
16

---

the g3 single would have no G2, so resistance would be 0%, but in reality g3 is 50%, so that can't be right. Hmm. Let's try another approach. [...]
17

---

B Redundancy Reduction
B.1 Details of Embedding- Redundancy Measurements
This section provides details of the embedding-basee analysis described in Section 4.2 used to demonstrate the redundancy-reducing effect of RPC.
For this analysis, we generated outputs for each sample in the AIME 2024 dataset using both the full KV baseline and RPC. Generation stopped after the 4128 decoding step, where the first compression RPC is completed, given = 4096 and R = 32. The generated outputs are separated into individual sentences, and each sentence is embedded using the all-MiniLM-LLGn M- model with Sentence Transformers library 36.. The model maps each token sequence to a 384-dimensional vector.
Pairwise cosine similarities were computed between all sentence embeddings within the same generated output. Two sentences with cosine similarity above 0.75 were considered semantically similar. Finally, we defined the redundancy rate as the proportion sentences that have more than N semantically similar counterparts within the same output, where N 1,2,4.
B.2 Visualized Examples of Token Selection with RPC
In this section, we provide qualitative examples illustrating how RPC selects tokens to retained in the KV cache.
In the following examples, token colors indicate how frequently each token is retained across layers: tokens preserved by a larger proportion of layers appear in warmer colors, following the scale defined in the color legend below.
Legend:
Black: Unselected or low selection (<40%)
Yellow: Medium selection (40-60%)
Orange: High selection (60-80%)
Red: Very high selection (280%)
Examples B. and B.2 are sampled from outputs of DeepSea--Rl-DistillQeeer. 3 and QwQ 32B 112
respectively, the AIME 2024 dataset.
While natural that recent tokens tend to exhibit higher retention frequencies due to LLMs' tendency to assign larger attention weights to local tokens, the selection pattern from semantic perspective can broadly categorized into two types:
Tokens containing critical information that advances the reasoning process, such as conclusions of intermediate steps.
Information-bearinn tokens:
(2) Transition tokens:
Tokens indicating shifts or boundaries between reasoning stages, which, although not semantically rich their own, play important contextual role through the attention mechanism by aligning and integrating information across reasoning steps.
This qualitative analysis demonstrates that RPC effectively preserves essential reasoning content while filtering out redundant or outdated derivations, thereby maintaining the continuity and integrity of the reasoning trajectory.
Example B.1
Problem. Let AABC have cireumcenter and incenter 7 with TA 1. OT, circumradius R = 13, and inradius 6. Find AB ACC.
DeepSeek-RI-Qwen-TB Output. <think> Okay, so need to find the product AB times AC in triangle ABC, given that the circumradius is 13. the inradius 6, and the segment IA is perpendicular to OI, where O is the circumcenter and is the incenter. Hmm,
18

---

that seems bit involved, but let me try to break it down step by step.
First, let me recall some properties and formulas related to triangle centers, circum- radius, inradius, and maybe some coordinate geometry. Since the problem involves perpendicularity between IA and OI, maybe coordinate geometry could be good approach here. Alternatively, there might be some formula that connects these elements directly, Let think.
I know that in any triangle, the distance between the incenter and circumcenter can be expressed by the formula: OP = R(R - 2r) where R is the circumradius and is the inradius.
Wait, is that correct? Let me verify. Hmm, actually, I think the formula is OF R² 2Rr. Yes, that's 's right. So OI squared equals R squared minus 2 times R times r. So substituting the given values, R is 13 and r 6, OP 13 2* 13*6 169 156 = 13. Therefore, Of is sqrt(13). Hmm, but the problem states that IA is perpendicular to OI. Maybe this can help relate IA and OI? [...]
The inradius related the area and semiperimeter: r area 8, where s (a + b c)/2. The circumradius R is related to the sides by the formula and similar for other sides. Also, the distance between O and is given by OF Ra 2Rr, which we already computed 13. So OI sqri(13). But we also have that So maybe we can use vectors or coordinate geometry to relate these.
2R sin A,
IA perpendicular
Alternatively, perhaps using the formula for the distance from the incenter to ver- tex. The length IA can be expressed in terms of the angles and inradius. Let me recall that in a triangle, the distance from the incenter to vertex A is given by: IA = (()qrt((12 M (elef((s M) (rrigh))2))
Wait, is that correct? Let me see. The formula for the distance from the incenter to vertex A: yes, in triangle ABC, the distance from to A is sqrt(r² + (s a)?), where s is the semiperimeter and the side BC. Similarly for other vertices. [...]
But since is vertex of the triangle on the circumradius of 13, so OA 13, so x y 13 169.
Hmm, this might get complicated with too many variables. Maybe there is better way.
Alternatively, since the cireumcenter, coordinates can be set such that triangle ABC is inscribed in circle of radius 13. Let me perhaps place point A at (13, 0) for simplicity, so coordinates A are (13, 0). Then at (0, O). Then points and are somewhere the circle x2 + +2 169. [...]
Expressed vectors, Ol is (I O) and IA is (A D). Since O is the origin, Ol is just the coordinates and IA is (A I). So their dot product is (A I)
Therefore,
|1==0 0.
Therefore. a A = II? Hmm, that's an equation relating the coordinates of I and A. Since on the circumcircle, IAI R 13. [...]
Therefore, aA 169 13 156, IA sqrt(156) 23*sqr(33)) Hmm, that seems useful.
So IA is sqrt(156). But we also have IA sin(a/2). Since so:sqrt(150 6/ /ini((//)
19

---

Therefore, sin(a/2) 6 sqrt(156) 6 (2*str(39))) sqri(39) sqrl(39)133
Wait, let's compute that again:
Wait 6 divided by (Qrt(156) equal to divided by (sqrt(4*39)) = 6/(2()))(()) = 3/sqrt(39). Rationalizing the denominator: 3/sqrt(39) 3 (3r9(339)333 sqrt(39)/33 So sin(a/2) sqrt(39)/33.
Therefore, sin(a/2) sqrt(39)/13. Let me note that sqrt(39)/13 is equal to sqrt(39)/13, which approximately 6.245/13 0.48, which is plausible.
Then, cos(a/2) would be sqri(1 (39/169)) sqri( 130/169) sqrt(130)11
Therefore, cos(a/2)
Then, sin a 2 sin(a/2) cos(a/2) = 2 2 = ==sqrt(3
Compute 39*130: 390100 3900, 39*30 1170, so total 3900 1170 5070. So sqrt(5070).
Hmm. 5070 factors: 5070 so 507 13*39 133
10 10#3 Wait, 507 divided by 13 is 39,
Wait, 39 is 3*13, so 5070 10*3*13*13 Therefore sqrt(5070) = sqr(( (ou((00))1))) 13*tq)t(3)). Because 10 30. So sqrt(5070) 13*sqrt(30)
Therefore, sin (((3 (2 sqrt 30)
Thus sin (2 (0)t(3))/133
Hmm, so angle A has sine equal to /3/013..
Therefore, the side BC can be found via the Law of Sines: a= BCBC= = 2R sin a = 22 2230113)))1133 = 4 /30. [...]
Example B.2
x² y²
Problem. Let B, C, and D points on the hyperbola
such that ABCD is
20 24
rhombus whose diagonals intersect at the origin. Find the greatest real number that is less than BD2 for such rhombi.
QwQ-32B Output. <think> Okay, so I have this problem here about a hyperbola and a rhombus inscribed in it. The hyperbola equation is given x3/20 y2224 1. The rhombus ABCD has its diagonals intersecting at the origin, and need to find the greatest real number that less than BDa for all such rhombi. Hmm, okay, me break this down step by step. [...]
But first, let me think about coordinates. Let me denote points A and C as (xl, yl) and (-x1, -yl), and points B and (x2, y2) and (-x2, -y2). Since four points lie on the hyperbola, they must satisfy the equation x3/20 y2224 1. So for points A and C, 1220 (y1) and similarly for points and D, (x2)2/22 (y2)9/24 =
Let me consider parametrizing the hyperbola. The hyperbola equation is x 9 =1. For hyperbola parametrization. we can use hyperbolic functions. Let me recall that standard hyperbola x²/22 2/bb =1 can be parametrized as (a sec8, b tano), or using hyperbola parameters. Alternatively using coordinates:
20

---

Let me set X sqr)(20) seco, y sqrt(24) tang. That would satisfy the hyperbola equation. Alternatively, perhaps using coordinates in terms of parameters and for points A and B.
Alternatively, since the hyperbola is symmetric, maybe we can set point A as (x, y) and point B as (u, v), such that x3/20 y9/24 =1 and u²/20 v4/24 =1. Then, the condition is x y v =0 (from x1x2 + yly2 =0), Then, BD² would be the squared distance between points B and D. Since D is (-u, -v), BD is the distance between (u, v) and (-u, -v), which is sqrl((uu)22 (2v) so BD squared (2u) (2v) =4(u + v2). [...]
Alternatielly, maybe use Lagrange multipliers to minimize u? + subject to the constraints:
u//20 =1, 2. y
and also x4/20 y3/24 =1.
But this might get complicated with multiple variables. Let me see if can express variables in terms of each other.
Let me suppose that OA is (x.y) and OB is (u,v). Since OA and OB are perpendic- ular, then u= u= y, v -kx for some scalar Wait, because if (x.y) and (u,v) are perpendicular, then (u,v) is scalar multiple of (-y, x) or (y, -x). Let me check:
If OA is (x,y), then perpendicular vector would be (-y, x) or (y, -x). So. (u.v) must be a scalar multiple that. Let me suppose (u,v) = x+y-, x), Then, substituting into the hyperbola equation for B:
u2/20- v2/24 (22 y2)20) ( (2 x2)24) X2 (y2/20 x2/24 =1.
Wait, but the hyperbola equation for point A is x²/20 y9/24 =1, y2224 = (x3/20 -1). Hmm, so let me rearrange that.
Wait, let me see:
Given (u.v) = X* x). Then, substituting into the hyperbola equation:
u2/20 v2/24 = (A3 yA2 (X2 x2)/24 X2 (y2/20 x2/24) ) =1.
But from point A's equation, x3/20 y2/24 =1, which can be rearranged to y (x+/20 -1). Therefore, y 224 -1). Let me plug that into the expression above.
So, = (24
x2/24.
Wait, me compute that step by step.
First, compute y2/20:
/20 = (24*(x2/00 -1] /20 24/(20) -1) (24/20)
Wait, perhaps this is getting too convoluted. Let me think differently.
Let me denote S = u² + v2, Since BD squared is 4S, we need to find the minimal value of S given that (u,v) lies on the hyperbola and (u,v) perpendicular to (x.y) which is also on the hyperbola.
Alternatively, since (u,v) is along the direction perpendicular to (x.y), let me parametrise
21

---

(u.v) scalar multiple of (-y, x). So let me write -, v x Then, substituting into the hyperbola equation for B:
u//20- v9/24 = (A2 y (2 x2)/2 X? (y9/20xx2)) ) =l.
From point A's hyperbola equation, x3/20 y3/24 =1, so rearranged: y9/24 = x²/20 =x -1). Let substitute that into the expression above.
First, compute y420 x3/24:
y4/20 (24* 1)] /20 (24/20)) -1) (6/5)
(6)))) -1)
x3/24 is just x3/24.
Therefore, y400 - x3/24 (6/5)xx220 -1) x3/24
Let me compute this term:
First, (x/5)x -1) = (6x3)/(100) 6/5 (3x3)/50 6/5
Then subtract x3/24:
(3x3/50 - 6/5) x3/24 (3x2/50 - x2/24) -6/5 [...]
22

---

C Extra Experimental Results
C.1
Output Length Statistics across Evaluation Datasets
Table 3 summarizes the output length statistics of DeepSee--R--Dssill-owaaaar
[ and QwQ 32B 12]. measured under the full KV cache setting. For both models, AIME 2024 [13] and LiveCodeBench [25] exhibit long reasoning outputs (over 10K tokens on average), whereas IFEval produces much shorter outputs. Accordingly, compression RPC is triggered multiple times for long-contex, benchmarks but rarely for short ones, which justifies using consistent P value (1024 or 4096) across all datasets to prevent excessive KV cache growth.
Table 3: Output length distribution under full KV cache setting.
DeepSeek- RI---stiil---------- Dist-iilll--wwee-7
QwQ-32B
Dataset
Mean
Min
Max
Std
Mean
Min
Max
Std
AIME 2024
13668.6
2413
32768
9356.4
13834.6
2747
32768
7365.5
LiveCodeBench
11889.1
809
32768
7066.2
13454.6
32768
9692.1
IFEval
1778.4
190
32768
5073.3
1336.9
144
32768
1844.3
C.2 Granularity of Attention Score Aggregation for Token Selection
To examine how the granularity of attention score aggregation for token selection impacts accuracy, we compare three aggregation schemes: layer-wise, group-wise (key-value group), and head-wise (no aggregation). In our approach, attention scores are aggregated at the layer level, meaning that saliency is averaged across all heads within each layer.
Table 4 presents results for DeepSeek-R1 -aet--i--iiiil--eeineeee
on AIME 2024 dataset under different aggregation granularities. The results show that layer-wise aggregation consistently yields the highest accuracy, indicating that averaging attention scores across heads helps stabilize the estimation of token importance and preserve overall performance after compression.
Table 4: AIME 2024 (pass@) 1) results for Drn-Secck-R--DDsilll----M-M with different attention score aggregation granularities.
P
Layer Aggregation
Group Aggregation
Head (No Aggregation)
4096
52.9
50.8
49.6
1024
50.4
50.4
47.5
Layer-wise aggregation offers a coarser yet more reliable estimation token saliency than head-level aggregation, which often suffers from high variance and instability across heads. This observation is consistent with previous work. TOVA (IT] reported that layer-level token selection outperformed head-level selection in terms of perplexity.
Moreover, in models using grouped-query attention (GQA) multiple attention heads share a single KV head. Performing token selection separately for each head in such architectures would require maintaining distinct KV caches per head, introducing significant memory overhead and defeating the purpose of compression. Therefore, even aside from accuracy, head-level selection is impractical for GQA-based models.
Overall, the layer-level token selection strategy adopted in RPC offers a practical and stable solution for compressing the KV cache of reasoning LLMs.
C.3 Efficiency Evaluation
We evaluate the efficiency gains of RPC by comparing decoding throughput and peak memory usage against the inference with full KV cache. Specifically, we report results for the default 4x compression setting of RPC with two compression intervals, P = 1024 and P = 4096. Throughput measured in tokens per second, and peak memory reflects the maximum GPU memory consumption during generation.
23

---

Measurements were conducted using two reasoning models: ----Seek-RI-iisillll QwQ-32B. For DeepSeek RI-Distill-Qwen
and evaluations were performed on single NVIDIA H100 SXM GPU, while QwQ-32B was tested using 4 NVIDIA H100 SXM GPUs in parallel. We fix the input length to 128 tokens and vary the generation length across 4096, 8192, 16384, and 32768 tokens. Batch size is varied across 8, 16, and 32 to assess scalability under different workloads.
Results for CrepSee-RR--istilll ---isttil-QQen--BB are shown Table 5 QwQ-32B are reported in Table 6
and the corresponding results for
Table 5: De--Seck-R--istiil----e QB's throughput and peak memory usage by batch size and
generation length.
Metric
Batch Size
4096
8192
16384
32768
Full KV Cache
8
401.50
368.72
330.41
256.92
Throughput (tokens/s)
16
669.53
653.04
504.50
342.88
32
1328.58
1031.51
671.83
OOM
19.20
22.95
30.47
45.50
Peak Memory (GB)
16
23.09
30.60
45.63
75.70
32
30.86
45.89
75.96
OOM
RPC (P = 1024)
8
448.19 19
428.31
407.00
385.00
Throughput (tokens/s)
16
848.75
794.62
751.69
650.20
32
1504,40
1499.80
1288.51
977.11
17.08
18.02
20.27
24.75
Peak Memory (GB)
16
18.86
20.74
25.15
34.20
32
22.40
26.16
35.00
53.08
RPC (P = 4096)
8
406.43
420.62
385.75
362.75
Throughput (tokens/s)
16
753.11
708.95
671.38
575.21
1318.33
1247.44
1064.05
883.43
19.20
20.14
22.02
25.77
Peak Memory (GB)
16
23.09
24.96
28.72
36.24
32
30.86
34.62
42.13
57.16
The results show that RPC consistently improves decoding efficiency over full KV cache inference various batch sizes and generation lengths. As the batch size increases, both the throughput gains and peak memory reductions become more pronounced. This because larger batches amplify the memory bottleneck imposed by the growing KV cache, allowing RPC's compression to better utilize available GPU compute resources. Notably, full KV cache inference results in oft-of-memory (OOM) errors for DeepSeek-R CeapSeek-R--Distil---ee ---Distill-Qeen--7 when the batch size 32 and the generation length reaches 32768, and for QwQ-32B when the batch size is 16 at 32768 tokens or 32 at 16384 tokens or longer. In contrast, RPC enables successful generation under all of these settings.
When comparing compression intervals, peak memory than P = 4096 across both models. While p 1024 offers stronger compression. may come at modest accuracy cost, as shown in Section 4.3 Therefore, P 1024 and P = 4096 can be considered complementary settings: the former prioritizes efficiency, and the latter provides more balanced trade-off between performance and accuracy.
1024 achieves slightly higher throughput and lower
24

---

Table 6: QwQ-32B's throughput and peak memory usage by batch size and generation length.
Metric
Batch Size
4096
8192
16384
32768
Full KV Cache
8
128.79 79
109.80
93.28
64.85
Throughpul (tokens/s)
16
213.75
173.99
117.51
OOM
32
351.34
228.59
OOM
OOM
8
83.40
100.58
134.94
203.66
Peak Memory (GB)
16
101.14
135.50
204.22
OOM
32
136.61
205.33
OOM
OOM
RPC (P" = 1024)
8
135.79
131.97
124.45
111.84
Throughput (tokens/s)
16
238.76
229.22
176.73
178.06
32
411.42
392.04
328.56
246.81
8
73.75
78.42
89.19
111.84
Peak Memory (GB)
16
81.81
91.24
112.77
155.81
32
97.95
116.70
159.78
245.94
RPC (P = 4096)
8
126.59
113.32
115.75
102.57
Throughput (tokens/s)
16
214.27
207.28
187.97
147.26
32
345.02
314.67
279.34
208.48
8
83.40
87.70
96.28
114.53
Peak Memory (GB)
16
101.14
109.73
126.90
163.40
32
136.61
153.79
188.14 14
261.13
C.4 Effect of Aggressive Compression
To assess the robustness of RPC under extreme compression, we evaluate its performance with target compression ratio of 8x. This setting represents highly aggressive compression scenario where only one-eighth of the generated tokens' KV entries are retained over time. Table 7 shows the resulting performance across the three benchmark datasets.
Table 7: Accuracy (%) of RPC 8x compared to RPC 4x and full KV cache.
QwQ-32B
Method
AIME 2024 (pass@11)
LiveCodeBench (pass@11)
IFEval (pass
AIME 2024
LiveCodeBench
1FEval
(pass
(pass@e1)
(pass@)
Full Cache
55.5
37.6
55.1
79.5
63.4
83.9
RPC Best
$2.9
35.9
57.3
78.3
62.2
82.6
RPC 8x (P 4096)
47.5
32.8
55.
72.1
57.2
84.3
RPC 8x (P = 1024)
37.5
27.2
58.4
72.1
57.4
82.8
Both models exhibit notable performance drop on AIME 2024 and LiveCodeBench under 8x compression, compared to the default 4x setting, indicating the difficulty of preserving reasoning fidelity under extreme compression. Nevertheless, the stronger reasoning model QwQ-32B demon- strates greater robustness, maintaining pass @ scores close to the results of RPC 4x across both benchmarks. In contrast, IFEval, benchmark characterized by lower reasoning difficulty, the performance remains stable or even improves slightly for both models, suggesting that lightt instruction instruction-follooinng tasks are less sensitive to aggressive KV cache compression.
25

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The main claims in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
26

---

Justification:
Section
4.3 discusses modest accuracy decline when RPC is applied.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specifiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results that require formal proofs. The paper focuses on the empirical evaluations of the proposed method.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The model, dataset and information on generation settings are provided in Section
Guidelines:
27

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The paper provides comprehensive details on the experimental settings, in- cluding models used, datasets and evaluation metrics. The steps to reproduce the results are clear.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/tipps.cc/public/guuddee hotpa//iii.i/iiide///eeeebbiiiii
/Ct////oeeeeeeaeeaaaeee ssi onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
28

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 4.J provides dataset and hyperparameter informations. Section 4.5 provides how the parameters are chosen.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The paper does not report error bars and information about the statistical significane of the experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information about the compute resources used for the experiments, mentioning the use of NVIDIA H100 SXM GPUs.
29

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't Iidn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https::/t//////ttt httttp::/nnerrips.oo httpg////.ooorg...co/o/acl/laaae
Answer: [Yes]
Justification: The paper targets to improve inference efficiency of reasoning LLMs and does not raise ethical concerns; no privacy-sensitiv data is used.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: The paper briefly discusses benefits in LLM inference efficiency, while knowledging trade-offs in accuracy due to aggressive compression.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
30

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The paper does not pose high risks for misuse that would require specific safeguards.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper properly credits existing assets, such as datasets and models, and follows appropriate licenses and terms of use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi sesrsswithhcee thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets including datasets or pretrained models. Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
31

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
The LLM was only used for writing assistance, editing, and formatting purposes. It not involved in the core methodology, scientific rigorousness, or originality of the research.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non- standardd components.
Please refer to LLM policy (https://neurips http://ppp.../oooeeeeeeee/////// for what should or should not be described.
32


Reviews:
[{"review_id": "uNw57aCi0F", "reviewer": "Reviewer_SCoz", "summary": "This paper introduces Reasoning Path Compression (RPC), a training-free method to accelerate inference in reasoning-focused language models that generate long intermediate reasoning paths. The key insight is that reasoning sequences exhibit \"semantic sparsity\" - they often contain redundant steps, repeated logic, and low information density relative to their length. RPC periodically compresses the KV cache by retaining only high-importance tokens (determined by attention scores), achieving up to 1.60× throughput improvement on QwQ-32B with only 1.2% accuracy drop, making reasoning LLMs more practical for deployment.", "questions": "What is the distribution of the response length in the reasoning model? How often will the compression be triggered under the large interval P=4096?", "limitations": "Authors have discussed their limitations and social impact in the paper", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n* This paper first investigates the semantic sparsity in the reasoning models: the long chain of thoughts often contains redundant messages with little information. It defines the semantic sparsity as the n-gram Shannon entropy and finds that the entropy is much lower in the reasoning models than the generic purpose LLMs in various response lengths. This motivates the compression of the response in reasoning models. \n* Compared with other baselines, RPC can keep comparable performance with a 4.0x compression ratio on the KV Cache in DeepSeek-R1-Distill-Qwen-7B and QwQ-32B. It results in a higher throughput and lower peak memory during the inference. \n* It provides detailed analyses on the important hyperparameters such as the selector window size R and the compression interval P to investigate their impact on the throughput and peak memory. \n\nWeaknesses:\n* From Figure 7, we can find that the accuracy significantly drops when the compression interval P is small. To achieve comparable results, it requires a large P such as 1024. Therefore, responses that are shorter than 2048 tokens can only trigger at most one compression when P=1024. It is important to demonstrate the distribution of the DeepSeek-R1-Distill-Qwen-7B's and QwQ-32B's response length in the AIME 2024, LiveCodeBench, and IFEval datasets. If the response lengths are significantly smaller than P=4096 or 1024, the comparable accuracy in Table 2 may be because most of the test cases do not trigger compression at all, rather than good performance of RPC. \n* As discussed in Figures 1 and 3, there is some redundant information in the long reasoning steps, and the goal of the compression is to remove the redundancy. So it is important to demonstrate what kind of tokens have been compressed based on the proposed Important Token Selection. The newly generated tokens may have a higher attention score than the previous similar tokens. \n\nOthers:\nFigure 6 is a little vague when zoomed in", "was_revised": true, "final_justification": "Thank the authors for the detailed responses and additional experiments. They have solved my concerns, and I will raise my score accordingly.", "rebuttal": "**[Response SCoz-1]  Weakness 1 and Question 1:  Output length distribution of reasoning LLMs**\n\nThank you for raising the important point about the relationship between output length and the frequency of compression triggers in RPC.\n\nIn **Tables SCoz-1 and SCoz-2**, we provide detailed statistics on the output lengths of R1-Distill-Qwen-7B and QwQ-32B for all three benchmarks used in our evaluation (AIME 2024, LiveCodeBench, and IFEval).\n\nFor AIME 2024 and LiveCodeBench, the mean output length ranges from 12K to 14K tokens, with a maximum of 32K tokens. In such cases, the proposed RPC with P=1024 or 4096 effectively mitigates rapid KV cache growth.\n\nFor IFEval, while the maximum still reaches 32K tokens, the mean output length is below 2K tokens. As you correctly pointed out, compression is rarely triggered when P=1024 or 4096 in this setting. \n\nHowever, as shown in **Table SCoz-3**, because the average output length of IFEval is much shorter than that of AIME 2024 and LiveCodeBench, using a smaller P value (such as 256 or 512) can still preserve accuracy effectively. Nonetheless, we chose to use a consistent P value of 1024 or 4096 across all benchmarks, as our primary objective is to prevent excessive KV cache expansion caused by unexpectedly long reasoning paths. \n\n\n**Table SCoz-1 Output length distribution (R1-Distill-Qwen-7B)**\n\n\n| R1-Distil-Qwen-7B | Mean     | Min   | Max    | Std     |\n|----------------------------|----------|-------|--------|---------|\n| AIME 2024                  | 13668.6  | 2413  | 32768  | 9356.4  |\n| LiveCodeBench              | 11889.1  | 809   | 32768  | 7066.2  |\n| IFEval                     | 1778.4   | 190   | 32768  | 5073.3  |\n\n**Table SCoz-2 Output length distribution (QwQ-32B)**\n| QwQ-32B         | Mean     | Min   | Max    | Std     |\n|-----------------|----------|-------|--------|---------|\n| AIME 2024       | 13834.6  | 2747  | 32768  | 7365.5  |\n| LiveCodeBench   | 13454.6  | 491   | 32768  | 9692.1  |\n| IFEval          | 1336.9   | 144   | 32768  | 1844.3  |\n\n**Table SCoz-3 IFEval pass@1**\n| Model                        | Full KV | P=256 | P=512 | P=1024 | P=4096 |\n|------------------------------|---------|-------|-------|--------|--------|\n| R1-Distill-Qwen-7B  | 55.1    | 57.1  | 56.7  | 57.3   | 56.6   |\n\n\n**[Response SCoz-2] Weakness 2: Analysis on evicted tokens with an example**\n\nThank you for your valuable comment. To evaluate whether the proposed RPC effectively removes redundancy from long reasoning sequences, we provide both a concrete example of token selection and quantitative evidence supporting the removal of redundant tokens.\n\nFirst, to illustrate how RPC selects important tokens, we visualized the tokens retained by RPC using **QwQ-32B** on a sample from the **AIME 2024** dataset. Since the set of selected tokens can vary across layers in RPC, we highlighted in bold the tokens that were selected by more than 50% of the layers in **Example SCoz-1 (Shown at the bottom of this rebuttal)**. (Given the long sequence lengths produced by reasoning models, it presents a representative subset of the output.)\n\nFrom this example, we observe that the tokens most frequently retained by RPC fall into two primary categories:\n\n- **Tokens containing critical information that advances the reasoning process**, such as conclusions of intermediate steps.\n\n\n- **Tokens indicating transitions between reasoning steps**, which, although not semantically rich on their own, play an important contextual role when processed through the attention mechanism, as they help align and integrate information across reasoning stages.\n\n\nThis qualitative analysis demonstrates that RPC effectively preserves essential reasoning content while removing redundant or outdated derivations.\n\nSecond, to further validate RPC's ability to reduce redundancy, we performed an additional quantitative analysis using embedding-based sentence similarity.\n\nSpecifically, we applied sentence-level vector embeddings to the generated outputs and computed pairwise cosine similarity between sentence embeddings. Two sentences were considered “semantically similar” if their cosine similarity exceeded 0.75. Based on this, we defined the redundancy rate as the proportion of sentences that have more than N(N=1,2,4) semantically similar sentences within the same output.\nThis analysis was conducted on outputs from three reasoning LLMs (R1-Distill-Qwen-7B, R1-Distill-Llama-8B, and QwQ-32B) using the AIME 2024 dataset.\n\nAs shown in **Table SCoz-4**, our analysis reveals that the redundancy rate dropped significantly after applying RPC. This result provides strong evidence that RPC successfully leverages **semantic sparsity** to reduce redundancy in reasoning trajectories.\n\n**Table SCoz-4 Embedding-based redundancy rate (ratios of sentences with more than N similar sentences during generation) reduction with RPC**\n|                  |   N=1   |        |   N=2   |        |   N=4   |        |\n|------------------|---------|--------|---------|--------|---------|--------|\n|                  | Full KV |  RPC   | Full KV |  RPC   | Full KV |  RPC   |\n| R1-Distill-Qwen-7B      | 65.5%   | 33.8%  | 48.9%   | 15.4%  | 29.8%   | 4.1%   |\n| R1-Distill-Llama-8B     | 69.7%   | 33.8%  | 52.7%   | 15.4%  | 35.0%   | 4.1%   |\n| QwQ-32B                | 66.0%   | 28.2%  | 48.0%   | 13.3%  | 29.6%   | 5.3%   |\n\n\nIn the revised version of our paper, we will include both comprehensive token selection visualizations and the full embedding-based redundancy analysis in the appendix. We believe this will help readers more clearly appreciate the effectiveness of RPC in eliminating redundancy while preserving critical reasoning information.\n\nOnce again, we sincerely thank the reviewer for this helpful suggestion.\n\n\n**[Response SCoz-3] Weakness 3: Vague Figure 6**\n\nThank you for bringing this to our attention. In the revised version of the paper, we will update Figure 6 with a higher-resolution version to enhance readability.\n\n\n**Example Scoz-1 Redundant reasoning path and selected tokens in bold**\n\n\n**[BEGINNING OF EXAMPLE]** **Okay, so I have this problem here about a hyperbola and**  a rhombus inscribed in it. The hyperbola equation is **given as x²/20 - y²/24 = 1. The**  rhombus ABCD has its diagonals intersecting at the origin, and I need to find the greatest real number that is less than BD² for **all**  such rh **ombi. Hmm, okay,**  let me break this down step **by step.**\n \n…\n**Therefore, the problem reduces to points A** (x1,y1) and B(x2,y2) on the **hyperb** ola such that x1x2 + y1y2 =0, and we need to find the minimal BD² (Wait, no, the question says \"the greatest real number that is less than BD² for all such rhombi\". So actually, we need to find the infimum of BD², but the question is asking for the greatest real number less than all BD². Hmm, maybe the minimal BD²? Or maybe it's the supremum? Wait, no. Wait, \"the greatest real number that is less than BD² for all such rhombi\". So, in other words, find the infimum of BD², but such that this number is less than all possible BD². Therefore, the infimum is the greatest lower bound, so the answer would be the infimum. But maybe BD² can be **made arbitrarily small? Probably not, because the hyperbola is fixed. Wait,**  but perhaps BD² has a minimum value? Let me think.\n \nAlternatively, maybe BD² can be as small as possible, but maybe there is a lower bound. Wait, but since the hyper **bola is symmetric, maybe BD** ² can be made **as small as approaching some value. Hmm,**  perhaps I **need to parametrize the points.**\n \n…\n  **Let me suppose that OA is (x** ,y) and OB is (u,v). Since OA and OB are perpendicular, **then u = k y, v =**  -k **x for some scalar k. Wait, because if**  (x,y) and (u,v) are perpendicular, then (u,v) is a scalar multiple of (-y, x) or (y, -x). **Let me check:**\n \n  **If OA**  is (x,y), then a perpendicular vector would be (-y, x) or (y, -x). So, (u,v) must be a scalar multiple of that **.**  Let me suppose (u,v) = λ*(-y, x). Then, substituting into the hyperbola equation for B:\n \nu²/20 - v²/24 = ( (λ² y²)/20 ) - ( (λ² x²)/24 ) = λ² ( y²/20 - x²/24 ) **=1.**\n \n  **Wait, but**  the hyperbola equation for point A is x²/20 - y²/ **24 =1, so y²** /24 = (x²/ **20 -1). Hmm, so let me rearrange that.**\n \n  **Wait, let me see:**\n \n  **Given (u** ,v) = λ*(-y, x). Then, substituting into the hyperbola equation:\n \nu²/20 - v²/24 = (λ² y²)/20 - (λ² x²)/24 = λ² ( y²/20 - x²/24 ) **=1.**\n \n  **But from point A's equation, x²/** 20 - y²/2 **4 =1, which can be rearranged to**  y²/24 = (x²/20 -1). Therefore **, y² =24*(** x²/2 **0 -1). Let me plug**  that into the expression above.\n \nSo, y²/20 - x²/24 = (24*(x²/20 -1))/20 - x²/24.\n \nWait, let me compute that **step by step.**\n \n  **First, compute**  y²/20:\n \ny²/20 = [24*(x²/20 -1)] /20 = 24/(20)*(x²/20 -1) = (24/20)*(x²/20 -1  **)**\n \n  **Wait, perhaps this is getting too convoluted. Let me think differently.**\n \n  **Let me denote S = u² + v². Since BD squared is 4** S, **we**  need **to find**  the minimal value **of**  S given that (u,v) lies on the hyperbola and (u,v) is perpendicular to (x,y) which is also on the **hyperbola.**\n \n  **Alternatively, since**  (u,v) is along the direction perpendicular to (x,y), let me parametrise (u,v) as a scalar multiple of (- **y, x).**  So **let me**  write u = **-** λ y, v = λ x **. Then, substit** uting into the hyperbola equation for **B** :\n \nu²/20 - v²/24 = (λ² y²)/20 - (λ² x²)/24 = λ² ( y²/20 - x²/24 ) =1  **.**\n \n  **From point A's hyperbola equation,**  x²/20 - y²/ **24 =1, so rearranged: y²/** 24 = x²/ **20 -1, so y² =24*(** x²/2 **0 -1). Let me substitute that into the expression above.**\n \n  **First, compute**  y²/20 - x²/24:\n \ny²/20 = [24*(x²/20 -1)] /20 = (24/20)*(x²/20 -1) = (6/5)*(x²/20 -1)\n \nx²/24 **is just x²/24.**\n \n  **Therefore, y** ²/20 - x²/24 = (6/5)(x²/20 -1) - x²/24\n \n **Let me compute this term:** ... **[END OF EXAMPLE]**"}, {"review_id": "lswuiCBS1g", "reviewer": "Reviewer_jmzr", "summary": "Based on the redundant segments and inherent semantic sparsity in reasoning LLMs, this paper introduces a reasoning  compression method RPC.  RPC leverages this characteristic by periodically compressing the KV cache and employs an importance scoring mechanism based on a selector window composed of recent queries. Experimental results validate the effectiveness of the proposed method, e.g., demonstrating that RPC compress KV cache 4$\\times$ with accuracy degradation limited to 1.2\\%.", "questions": "Please see more details in Weaknesses. BTW, the proposed algorithm involves many hyperparameters. So, could these hyperparameters be learned from an optimization objective?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This paper is well-written and easy to follow.\n2. The proposed method has engineering impact. For example,  KV‐cache is significantly reduced without fine-tuning model weights.\n3. The experimental evaluations are relatively sufficient, although more baseline methods are encouraged to be involved. \n\nWeaknesses:\n1. The proposed algorithm involves many hyperparameters, and in practical applications you need to perform a fine-grained grid search to achieve good results. So, this method seems not flexible enough. \n2. In practice, different heads produce different attention scores. Some heads play a critical role at specific reasoning stages, but from this method, their peaks can be drowned out by moderate scores from other heads, causing them to fall out of the Top-P after aggregation. This  can potentially lead to loss of fine-grained information.\n3. Does it make sense to rely on attention weights as the core metric?\n4. I feel the paper includes too few baselines, considering only the full CV compression method. Could more baseline methods be evaluated? (I’m not a KV-cache specialist myself. Sorry I can’t suggest specific references.)", "was_revised": true, "final_justification": "I'd like to raise the score from 3 to 4.", "rebuttal": "**[Response jmzr-1] Weakness 1 and Question 1: Selecting hyperparameters**\n\nThank you for your valuable comment. As you pointed out, the proposed RPC method involves setting two hyperparameters, compression interval (P) and selector window size (R), to minimize accuracy degradation after compression. In our experiments, we used grid search to identify appropriate values.\n\nHowever, we would like to clarify that **the sensitivity of these hyperparameters is relatively low, and fine-grained tuning is not necessary**. This is why our grid search was conducted over a coarse set of values, as shown in Figure 7 and Table 3. For example, in Table 3, we evaluated R using only four values (1, 8, 32, 128), and the best accuracy was achieved at R=32. Slight variations (e.g., R=40) do not significantly impact accuracy, so we limited our evaluation to powers of two.\n\nMoreover, **both P and R exhibit consistent and predictable trends**, which further simplifies the tuning process. Since P controls how frequently compression occurs, it is expected to saturate once enough tokens are accumulated for effective compression. Similarly, R determines the size of the context window for importance estimation: too small an R may miss relevant information, while too large an R may include outdated context. As a result, accuracy tends to improve initially with increasing R, reaches a peak, and then degrades as R becomes excessively large. These intuitive trends allow users to tune P and R without needing an exhaustive search.\nImportantly, the hyperparameter setup exhibits good cross-task transferability, as the values selected for AIME2024 generalize well to other benchmarks. (In our experiments, we used P=1024 or 4096 and R=32 across all benchmarks.) While some tuning may be required for tasks with significantly different output length distributions, the process remains straightforward and does not involve burdensome fine-grained search.\n\nBecause the effects of hyperparameters (P and R) are easy to interpret and manage, we believe that additional learning-based optimization is unnecessary. However, we see potential to eliminate the need for the compression interval P. Specifically, RPC could be further improved by incorporating a mechanism that dynamically detects when a sufficient number of tokens have been generated to trigger compression, rather than relying on a fixed interval. We consider this a promising direction for future work.\nIn summary, only P and R function as true hyperparameters in RPC, and both can be set in a simple and intuitive manner for effective and robust performance.\n\n**[Response jmzr-2] Weakness 2: Granularity of token selection and attention score aggregation**\n\nAs you pointed out, different attention heads often play different roles. However, from the perspective of selecting important tokens for KV cache compression, a **coarser-grained token selection strategy**, such as **layer-wise selection** as adopted in RPC, offers clear advantages in preserving accuracy.\n\nTo support this, **Table jmzr-1** compares accuracy after KV cache compression using different granularities of token selection: **layer-wise**, **key-value-group-wise**, and **attention head-wise**. The results show that layer-wise token selection, which aggregates attention scores across all heads within a layer, consistently yields the highest accuracy.\n\n**Table jmzr-1 AIME 2024 pass@1 comparison between attention score aggregation granularities**\n| AIME 2024 (pass@1)      | Layer Aggregation | Group Aggregation | Head (No Aggregation) |\n|-------------------------|-------------------|-------------------|----------------------|\n| R1-Distill-Qwen-7B (P=4096)          | 52.9              | 50.8              | 49.6                 |\n| R1-Distill-Qwen-7B (P=1024)          | 50.4              | 50.4              | 47.5                 |\n| QwQ-32B (P=4096)        | 78.3              | 75.0              | -                    |\n| QwQ-32B (P=1024)        | 78.3              | 77.5              | -                    |\n\nThese findings indicate that finer-grained selection at the head level does not necessarily lead to better accuracy in KV compression. One possible reason is that **attention scores from a single head may not contain sufficient semantic information** to identify truly important tokens. A token strongly attended by one head may not correspond to meaningful context and could even reflect noise. In contrast, aggregating attention scores across all heads in a layer allows for a more semantically informed estimate of token importance.\n\nThis observation, advantage of coarse-grained token selection, is also supported by previous work. **AdaKV** [1] updated its method to use group-level selection rather than head-level selection. Their experiments showed that group-level and head-level selection yielded nearly identical accuracy on **LongBench**. Additionally, **TOVA** [2] reported that for both TOVA and **H2O** [3], **layer-level token selection** outperformed head-level selection in terms of perplexity.\n\nMoreover, in models using **grouped-query attention (GQA)**, multiple attention heads share a single KV head. Performing token selection separately for each attention head in such models would require maintaining separate KV caches per head. This introduces additional memory overhead, potentially defeating the purpose of compression. (Due to this overhead, it was not feasible to fully evaluate the accuracy of **QwQ** under head-wise token selection, and thus the corresponding entry in **Table jmzr-1** is left blank.) Therefore, **even aside from accuracy**, head-level selection is not a practical option for GQA-based models.\n\nOverall, the **layer-level token selection strategy** used in RPC is a practical approach for compressing the KV cache of reasoning LLMs.\n\n\n**[Response jmzr-3] Weakness 3: Why we use attention weights as our core metric**\n\nBecause attention weights are directly used to merge token information (specifically, the value vectors in the KV pairs), they reflect the relative influence of each token on subsequent outputs. Therefore, attention weights can serve as a reasonable proxy for token importance.\n\nBuilding on this intuition, many prior KV cache compression techniques (e.g., SnapKV [4], H2O[3]) have also leveraged attention weights to identify and retain important tokens.\nIn summary, leveraging attention weights for token screening is not only intuitive but also a well-established practice in KV compression research, and RPC extends this paradigm to the unique characteristics of reasoning LLMs.\n\n**[Response to jmzr-4] Weakness 4: Additional evaluation results of baselines** \n\nAs you noted, while **Table 1** in the main paper compares RPC with the full KV cache as well as baseline methods (H2O, TOVA, and LightThinker), Table 2 only compares RPC with full KV Cache.\n\nWe agree that a more comprehensive comparison is more desirable. To address this, we have evaluated the baselines across all three benchmarks (AIME 2024, LiveCodeBench, and IFEval) using **R1-Distill-Qwen-7B**, and the results are now provided in **Table jmzr-2**.\n\n**Table jmzr-2 Additional evaluation results of baselines and comparison with RPC (R1-Distill-Qwen-7B)**\n|                  | AIME 2024 | LiveCodeBench | IFEval |\n|------------------|-----------|---------------|--------|\n| Full KV          | 55.5      | 37.6          | 55.1   |\n| H2O (avg)        | 42.5      | 22.5          | 51.8   |\n| TOVA (avg)       | 42.5      | 21.5          | 48.8   |\n| LightThinker     | 6.7       | 0.7           | 25.1   |\n| RPC (P=4096)     | 52.9      | 35.9          | 56.6   |\n| RPC (P=1024)     | 50.4      | 33.5          | 57.3   |\n\nFor this extended evaluation, we allocated 25% of the average output length (measured from full KV cache setup) as a fixed cache size for H2O(avg) and TOVA(avg). Please note that this approach produces slightly different results (yet with same trend) from the per-sample cache budgeting strategy used in our origianal manuscript, which would require first measuring the generation lengths of the reasoning LLMs under full KV cache settings and then determining the KV cache budget as 25% of each individual length.\nRunning H2O and TOVA with such a strategy is extremely slow due to their inefficiencies and token-by-token compression mechanisms, so it was impractical to adopt the per-sample cache budgeting strategy for this extended evaluation.\n\nEven under this more comprehensive evaluation, RPC maintains its advantage by consistently preserving accuracy better than previous KV compression techniques.\n\nWe will also update **Table 2** in the revised version of the paper to reflect the new results.\n\n[1] Feng, Yuan, et al. \"Ada-kv: Optimizing kv cache eviction by adaptive budget allocation for efficient llm inference.\" arXiv preprint arXiv:2407.11550 (2024).\n\n[2] Oren, Matanel, et al. \"Transformers are multi-state rnns.\" arXiv preprint arXiv:2401.06104 (2024).\n\n[3] Zhang, Zhenyu, et al. \"H2o: Heavy-hitter oracle for efficient generative inference of large language models.\" Advances in Neural Information Processing Systems 36 (2023): 34661-34710.\n\n[4] Li, Yuhong, et al. \"Snapkv: Llm knows what you are looking for before generation.\" Advances in Neural Information Processing Systems 37 (2024): 22947-22970."}, {"review_id": "Nm9f6CYUFb", "reviewer": "Reviewer_r668", "summary": "This paper proposes Reasoning Path Compression (RPC), a training-free KV cache compression method aimed at improving the inference efficiency of reasoning-focused LLMs that generate long intermediate reasoning paths. The authors observe that such reasoning paths often exhibit semantic sparsity (e.g., redundant or repeated logical steps) and leverage this property by periodically compressing the KV cache, retaining only tokens that receive high importance scores computed using recent queries as a selector window. The method is evaluated on the DeepSeek-R1-Distill-Qwen-7B and QwQ-32B models across benchmarks including AIME 2024, LiveCodeBench, and IFEval, and is compared against prior methods such as LightThinker, H2O, and TOVA.", "questions": "### Questions and Suggestions for the Authors\n\n\n1. **Further empirical investigation of semantic sparsity**  \n   The motivation around semantic sparsity currently relies on n-gram Shannon entropy measurements using a single model. Have the authors conducted further empirical investigations of semantic sparsity on other reasoning LLMs and with other metrics, such as embedding-based similarity between reasoning steps, to measure redundancy beyond exact n-gram repetition? Additionally, do similar redundancy patterns appear on coding benchmarks such as LiveCodeBench?\n\n2. **Ablation and justification for the selector window**  \n   The paper highlights the selector window as a key component, asserting that attention scores relative to recent tokens effectively indicate the relevance of previous tokens. While an ablation on different selector window sizes is provided, it would be helpful to clarify how this empirically supports the claim that using attention scores over recent tokens is an effective method for importance estimation, versus alternatives like average attention across all tokens.\n\n3. **Clarification on comparison with SnapKV and HeadKV**  \n   The paper does not include SnapKV and HeadKV in the experimental comparisons. Since LLM generation is autoregressive and both the prompt and generated tokens contribute to the autoregressive input for the next token, could the authors clarify why these methods are not included and how the differences between compressing prompts and compressing generated reasoning paths practically manifest in the context of these methods?\n\n4. **Clarification on the accuracy drop with larger R**  \n   The paper mentions that using a larger selector window size (R) can lead to an accuracy decrease, but the explanation provided is not fully clear. Could the authors elaborate on why evaluating importance over a larger set of recent tokens would decrease performance?\n\n---\n\n**Criteria for potential score increase**:  \nA response that includes a deeper investgation of semantic sparsity across models and metrics, and clarifications on the observed behaviors (such as the accuracy drop with larger R) would strengthen the paper’s motivation and technical justification. This would positively impact my evaluation of the paper’s quality and clarity.", "limitations": "The paper could benefit from noting that RPC is primarily tested on math and coding tasks using DeepSeek and Qwen models, and its generalization to other tasks is unclear. Additionally, while RPC is training-free, tuning hyperparameters (e.g., compression interval, selector window) may require task- or model-specific calibration to maintain accuracy.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The motivation of the paper is clear: reasoning chains in LLMs are long and often contain semantically redundant steps, which increases inference cost.\n- The proposed RPC method is training-free and shows good empirical performance gains, improving throughput and reducing memory usage with only modest accuracy drops.\n- The paper evaluates the method on well-chosen benchmrks (AIME 2024, LiveCodeBench, IFEval) using two models (DeepSeek-R1-Distill-Qwen-7B and QwQ-32B), which supports the validity of the findings.\n- The ablation studies are informative, illustrating how compression interval and selector window size affect the accuracy-efficiency trade-off, which is useful for practitioners.\n\n### Weaknesses\n\n- The claim that prior KV compression methods are primarily designed for long input/short output settings (e.g., line 93) would benefit from additional justification or empirical validation to clarify the limits of prior methods.\n- While the empirical results are strong, the motivation regarding semantic sparsity could be investigated more deeply. The current analysis uses n-gram Shannon entropy to measure redundancy, which is appropriate for detecting exact or near-exact repetition, but may miss paraphrased or logically equivalent content. Incorporating embedding-based similarity measures between reasoning steps, commonly used for semantic redundancy detection, would strengthen the analysis.\n- The redundancy analysis appears to primarily use DeepSeek-R1-Distill-Llama-8B outputs. Given that outputs from reasoning models can vary significantly, especially on reasoning-heavy tasks, additional analysis using a broader set of models would strengthen the motivation and generality of the semantic sparsity observation.\n- A stronger empirical investigation of semantic sparsity would help the reader understand the problem space more thoroughly and clarify why the proposed method works as well as it does.\n- Minor clarity issue: there is a typo on line 264 (“comperssing”).", "was_revised": true, "final_justification": "My main concern was the simplicity of the sparsity analysis in the paper, which used simple n-gram Shannon entropy to measure redundancy, which is appropriate for detecting exact or near-exact repetition, but may miss paraphrased or logically equivalent content.\n\nGiven that the authors have conducted some more appropriate sparsity analysis using semantics in the rebuttal, I now recommend accept. However, I would strongly suggest that the authors include the new semantic sparsity analysis in their manuscript.", "rebuttal": "**[Response r668-1] Weakness 1 and Question 3: Justification for the unsuitability of existing methods in long generation scenario**\n\nAs discussed in **Section 2.2** of our manuscript, existing KV compression techniques can be categorized into two types:\n1. those that compress only the input prompt\n2. those that also compress the generated tokens\n\nWe now explain why both categories fail to address the challenges posed by reasoning models.\n\n**1. Methods that compress only the input prompt (e.g., SnapKV, HeadKV):**\nThese methods are designed to compress long input prompts only, so they do not apply any compression to the KV cache associated with newly generated tokens.\nHowever, reasoning tasks feature relatively short inputs but very long outputs. As shown in **Table r668-1**, the average input length is approximately 115 tokens, while the average output length exceeds 13K tokens. In this setting, compressing the input tokens brings negligible benefit, as they make up only a small fraction of the total KV cache. (Further discussion on the difference between compressing input/output tokens: **Response r688-6**)\n\n**Table r668-1 Input/output length statistics of AIME 2024 with QwQ-32B**\n| | Mean | Std  |\n|-----------|----------|--------|\n| Input     | 115.5    | 38.1   |\n| Output    | 13834.6  | 7365.5 |\n\n**2. Methods that compress generated tokens (e.g., H2O, TOVA):**\nTo address generative compression, H2O and TOVA maintain a fixed-size cache during generation and evict tokens based on attention scores. While effective for standard language modeling tasks with predictable and relatively short outputs, they struggle with reasoning tasks for the following reasons:\n- Output lengths in reasoning tasks are not only much longer but also highly variable across samples. As a result, choosing an appropriate cache budget becomes difficult.\n- These methods also evict tokens one-by-one at each decoding step. This incurs substantial computational overhead and may disrupt long-term semantic coherence.\n\nOn the other hand, the proposed RPC addresses these limitations in two key ways:\n RPC defines a compression ratio rather than a fixed KV cache size, enabling adaptive budget allocation based on the actual number of generated tokens.\nSecond, RPC performs periodic compression at fixed intervals, reducing computational overhead while preserving semantic consistency across long reasoning paths.\n\nWe provide accuracy comparison in **Table 1** of our manuscript. To further support our claim, we extended the evaluation to additional benchmarks and models in **Table r668-2**. Here, previous works consistently demonstrate degraded performance, while RPC remains robust.\n\n**Table r668-2 Accuracy evaluation with R1-Distill-Qwen-7B**\n|   | AIME 2024 | LiveCodeBench | IFEval |\n|------------------|-----------|---------------|--------|\n| Full KV   | 55.5  | 37.6  | 55.1   |\n| H2O (avg)    | 42.5   | 22.5    | 51.8   |\n| TOVA (avg)  | 42.5   | 21.5    | 48.8   |\n| LightThinker | 6.7 | 0.7  | 25.1   |\n| RPC (P=4096) | 52.9    | 35.9   | 56.6   |\n| RPC (P=1024) | 50.4    | 33.5  | 57.3   |\n\n**[Response r668-2] Weakness 2/3/4 and Question 1: Additional investigation on semantic sparsity**\n\n**Comprehensive Analysis of Semantic Sparsity**\n\nWe conducted a deeper investigation of semantic sparsity using embedding-based similarity analysis. Specifically, we applied sentence-level vector embeddings to the generated outputs and computed pairwise cosine similarity between sentence embeddings. Two sentences were considered “semantically similar” if their cosine similarity exceeded 0.75. Based on this, we defined the **redundancy rate** as the proportion of sentences that have more than **N(N=1,2,4) semantically similar sentences** within the same output.\n\nIn addition, as you recommended, we expanded the analysis to a broader set of models (three reasoning LLMs and two non-reasoning LLMs). The analysis of reasoning and non-reasoning LLMs used AIME 2024 and  HelloBench dataset, respectively.\n\nAs shown in **Table r668-3**, reasoning LLMs exhibit higher redundancy rates compared to non-reasoning LLMs. This indicates that reasoning LLMs generate more semantically similar output than non-reasoning LLMs.\n\n**Table r668-3 Embedding-based redundancy rate comparison between reasoning and non-reasoning LLMs**\n|   |     | N=1   | N=2   | N=4   |\n|---------------|---------------|-------|-------|-------|\n| Reasoning     | R1-Distill-Qwen-7B  | 65.5% | 48.9% | 29.8% |\n|  | R1-Distill-Llama-8B | 69.7% | 52.7% | 35.0% |\n|   | QwQ-32B     | 66.0% | 48.0% | 29.6% |\n| Non-Reasoning | LongWriter-8B      | 44.5% | 32.5% | 22.2% |\n|   | LongWriter-9B     | 33.8% | 19.8% | 12.0% |\n\n**Impact of RPC in reducing semantic redundancy**\n\nWe examined whether semantic redundancy is effectively reduced by RPC.\n\nAs shown in **Table r668-4**, our analysis reveals that the redundancy rate dropped significantly after applying RPC. This demonstrates that RPC successfully reduces redundancy in reasoning trajectories.\nWe sincerely appreciate your suggestion to include a more quantitative analysis of semantic sparsity. These new results and discussions will be updated in the revised version of the manuscript.\n\n**Table r668-4 Embedding-based redundancy rate reduction with RPC**\n|  |   N=1  |    |   N=2  |  |  N=4   |    |\n|-----------------------|---------------|---------|---------------|---------|---------------|---------|\n|  |Full KV| RPC| Full KV| RPC|Full KV |RPC|\n| R1-Distill-Qwen-7B| 65.5%| 33.8%| 48.9%| 15.4%| 29.8%| 4.1%|\n| R1-Distill-Llama-8B| 69.7%| 33.8%| 52.7%| 15.4%| 35.0%| 4.1%|\n| QwQ-32B| 66.0%| 28.2%| 48.0%| 13.3%| 29.6%| 5.3%|\n\nWe will also update these new results and discussions in our manuscript.\n\n\n**[Response  r668-3] Weakness 5: Typo**\n\nThank you for catching this typo. We will correct it.\n\n**[Response r668-4] Question 1: Redundancy pattern on coding benchmark**\n\nWe observe that redundancy patterns similar to those found in mathematical reasoning tasks also frequently appear in code generation. When reasoning LLMs are applied to coding problems, the generated outputs often include repeated explanations of core algorithmic concepts, multiple reconfirmations of solution steps, and restatements of reasoning strategies.\n\nTo quantify this redundancy, we applied the same embedding-based similarity analysis described in **Response r668-2**. As shown in **Table r668-5**, the redundancy level observed in LiveCodeBench is even higher than that measured in AIME 2024. This observation reinforces our claim that semantic sparsity is a general property of reasoning LLM outputs.\n\n**Table r668-5 Embedding-based redundancy rate of R1-Distill-Qwen-7B**\n\n| | N=1   | N=2   | N=4   |\n|---------------|-------|-------|-------|\n| AIME 2024| 65.5% | 48.9% | 29.8% |\n| LiveCodeBench | 70.9% | 54.3% | 35.8% |\n\n**[Response r668-5] Question 2: Justification for the selector window**\n\nAs shown in **Table 3** of our manuscript, using a large selector window size of R=128 results in lower accuracy compared to R=32. This result indicates that when the selector window includes outdated tokens, these tokens may interfere identifying truly relevant tokens for the current reasoning step. Consequently, we can expect that using attention from all previously generated tokens is likely to degrade accuracy. More importantly, accumulating attention scores from all tokens is not a practical solution in reasoning tasks with long output sequences, as it introduces substantial computational overhead.\n\nTo explore alternative strategies, we experimented with using a selector window composed of the last few tokens of the input prompt. This alternative led to a slight decrease in accuracy (see **Table r668-6**). While using the prompt's final tokens as selectors can be advantageous in capturing prompt-relevant context, our results suggest that leveraging recently generated tokens—those most pertinent to the current reasoning step—yields more robust and accurate token selection for KV retention.\n\n**Table R668-6 Accuracy of R1-Distill-Qwen-7B after applying RPC with recent or prompt tokens as selector window**\n| | P=4096 | P=1024|\n|------------------|---------------------------|----------------------------|\n|Recent Tokens|52.9|50.4|\n|Prompt Tokens|42.5|45.0|\n\n\n**[Response r668-6] Question 3: Difference of compressing input tokens and output tokens**\n\nFor input tokens, the entire sequence is available from the beginning of inference and remains fixed, making it possible to compress the input context in a single step without dynamic updates.\n\nIn contrast, output tokens are generated autoregressively and their final length is unknown in advance. The importance of each generated token evolves as reasoning progresses, so token selection must be performed dynamically during generation.\n\nIt is important to note that methods such as SnapKV and HeadKV are designed specifically for compressing input tokens and do not provide mechanisms for compressing dynamically generated output tokens. Since our focus is on efficient long-form reasoning generation, where output token compression is essential, we did not include SnapKV and HeadKV in our experimental comparisons.\n\n**[Response r668-7] Question 4: Clarification on the accuracy drop with larger R**\n\nAs mentioned earlier, the motivation for using recently generated tokens as the selector window is to ensure that token retention decisions reflect the current stage of reasoning. By focusing on attention from the most recent tokens, RPC can better identify which past tokens remain relevant. \n\nHowever, if the selector window size R becomes too large (e.g., 128), it starts to include many older tokens whose attention patterns may no longer match the current reasoning context. Involving these outdated tokens in importance estimation can distort the selection process, leading to the retention of non-critical tokens and possibly discarding those essential for the present reasoning step."}, {"review_id": "q6cTJxuIZv", "reviewer": "Reviewer_JM3v", "summary": "This work proposes a novel technique for inference acceleration, termed Reasoning Path Compression (RPC), which requires no model training. The method is designed to enhance generation efficiency by exploiting the inherent semantic sparsity of reasoning paths. Specifically, RPC evaluates and quantifies the importance of KV cache entries through a selector window comprising the latest queries. Based on this scoring, the method performs periodic compression of the KV cache, preserving only high-importance entries. Experimental results demonstrate that applying RPC increases the generation throughput of the Qwen-32B model by up to 1.60× over the baseline (full KV cache), with the accuracy drop on the AIME 2024 benchmark contained to 1.2%.", "questions": "It appears the throughput experiments were conducted with fixed prompt and decode lengths. I suggest adding batch throughput experiments using realistic reasoning benchmarks to strengthen the evaluation.\n\nI suggest providing a comparative analysis between the selected kv cache and the actual generated reasoning tokens to substantiate the claim that the algorithm effectively eliminates redundant reasoning logic.\n\nI recommend revising the figure 4 to explicitly illustrate how to select tokens to evict.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe paper proposes a novel method for KV cache selection that demonstrates a minimal loss in accuracy on benchmarks such as AIME, LiveCodeBench, and IFEval. Notably, the method significantly reduces memory usage while simultaneously improving throughput. This algorithm is easy to implement and provides decent results.\n\nWeaknesses:\nThe connection between the proposed motivation and the algorithm appears to be weak. \nThe decision to permanently discard tokens from the cache, making them unavailable for subsequent decoding steps, seems problematic. This approach could lead to a degradation in accuracy if tokens that are initially deemed unimportant become critical later in the generation process.\nRegarding Figure 4, I found it difficult to understand the compression step.", "was_revised": true, "final_justification": "I thank the authors for their detailed rebuttal. Their responses have addressed the questions I raised regarding the clarity of the methodology and the experimental setup. The new explanations have clarified my previous misunderstandings.\n\nResolved Issues: My primary concerns about the clarity of the core methodology have been fully resolved by the authors' clarifications.\n\nRemaining Issues: None.\n\nJustification for Score Change: The authors' clear and effective response has directly improved my understanding of the paper's contributions. Therefore, I have raised the score for \"Clarity\" by one point and will increase my overall score by one point. I believe the paper is now in a stronger position for acceptance.", "rebuttal": "**[Response JM3v-1] Weakness 1: Concerns on permanent KV cache eviction**\n\nThank you for the valuable comment. First, we would like to clarify that there are two major research directions for addressing the **long KV cache problem** in LLM inference:\n1. **Memory Offloading Approaches** (e.g., QUEST [1]): These methods aim to distribute memory requirement by offloading KV cache entries to host memory and selectively retrieving them during decoding, rather than discarding them permanently. While this strategy has the advantage of fully preserving all KV information, it does **not reduce total memory usage**, and the repeated indexing and retrieval introduce **non-trivial overhead** at each decoding step.\n2. **Permanent Eviction Approaches** (e.g., TOVA [2], H2O [3]): This direction focuses on permanently evicting KV cache entries, **directly reducing memory usage and avoiding the runtime overhead**. Due to these efficiency benefits, there has been growing interest in designing eviction-based compression methods [2, 3]. As you rightly pointed out, naive eviction strategies can lead to significant accuracy degradation. Therefore, the key challenge in this direction is to design effective compression mechanisms that accurately identify and retain the most critical KV entries to preserve model performance.\n\nOur proposed RPC falls into the second category and we proposed a novel KV cache compression mechanism specialized for reasoning LLMs. The proposed RPC addresses the redundancy in generated reasoning paths through periodic, attention-based eviction of KV cache entries for generated tokens. As shown in Table 1 of our manuscript, RPC achieves better accuracy preservation compared to prior works. This experimental result validates its effectiveness in compressing long reasoning paths.\nWe will revise the manuscript to better highlight these distinctions and clarify the rationale behind the permanent eviction strategy in RPC.\n\nThank you again for your thoughtful feedback.\n\n\n**[Response JM3v-2] Weakness 2 and Question 3: Clarification of Figure 4**\n\nThank you for your comment. We will revise **Figure 4** to more clearly illustrate the token eviction process in the proposed RPC method. As figure updates are not allowed during the rebuttal period, we outline below our planned revisions to be included in the updated manuscript.\n\nWe will modify the figure to visually depict the following key components of RPC:\n1. **selector window** and its role in evaluating token importance\n2. **computation of importance scores**\n3. **selection of tokens to retain or evict** at each compression step.\n\nMore specifically, we will clarify the eviction mechanism as follows:\n\n At each compression interval, RPC identifies which tokens to retain based on attention scores computed between the most recent R token (i.e., the selector window) and all previously generated tokens in the KV cache. To support efficient attention score computation, the query vectors of the selector tokens are cached during generation. Using these cached queries, RPC computes aggregated attention scores across attention heads in each layer to estimate the importance of tokens in the current KV cache.\n\nImportantly, eviction is not limited to recently generated tokens. Instead, RPC considers the entire KV cache, which includes both tokens retained from previous compression steps and newly generated tokens. Tokens are ranked by their aggregated attention scores, and RPC retains only the top-k tokens, along with the R selector tokens. For the N-th compression cycle, k is defined as NP/c, where P is the compression interval and c is the compression ratio. This design allows RPC to naturally phase out outdated and less relevant tokens as reasoning progresses.\n\n**[Response JM3v-3] Question 1: Throughput measurement on real reasoning dataset**\n\nTo evaluate throughput under realistic conditions, we measured the throughput of reasoning LLMs on the AIME 2024 benchmark across various batch sizes. As in our original manuscript, R1-Distill-Qwen-7B was evaluated on a single NVIDIA H100 GPU, while QwQ-32B was run on four H100 GPUs. Please note that, since we used a real-world benchmark, the output lengths of samples within a batch vary.\n\nAs shown in **Table JM3v-1  JM3v-2**, RPC consistently achieved significant throughput improvements compared to Full KV cache. For example, at batch size 16, RPC achieved up to 2.58× higher throughput than Full KV. At batch size 32, Full KV encountered out-of-memory errors, while RPC successfully completed generation with high throughput and no memory issues.\n\n**Table JM3v-1 Throughput (tokens/sec) measurement on AIME 2024  (R1-Distill-Qwen-7B)**\n| Batch Size | 8      | 16      | 32      |\n|------------|--------|---------|---------|\n| Full KV | 152.92 | 252.12 | OOM|\n| P=4096  | 257.66 | 474.65  | 706.01  |\n| P=1024   | 348.82 | 591.28  | 938.85  |\n\n**Table JM3v-2 Throughput (tokens/sec) measurement on AIME 2024 (QwQ-32B)**\n| Batch Size | 8  | 16 | 32 |\n|------------|-------|--------|--------|\n| Full KV    | 30.10 | OOM    | OOM    |\n| P=4096     | 70.51 | 98.97  | 152.40 |\n| P=1024     | 81.76 | 120.83 | 205.60 |\n\nThese results confirm that the efficiency gains of RPC generalize to realistic batched inference on actual reasoning benchmarks, not just under fixed-length scenarios.\n\n**[Response JM3v– 4] Question 2: Analysis on selected KV cache and elimination of redundancy**\n\nThank you for highlighting the importance of an analysis on the selected KV cache. We fully agree that such analysis is essential to substantiate our claim.\n\nThrough observations of the selected KV cache on real examples, we find that the tokens most frequently retained by RPC generally fall into two key categories:\n\n- Tokens containing critical information that advances the reasoning process, such as conclusions of intermediate steps.\n- Tokens indicating transitions between reasoning steps, which, although not semantically rich on their own, play an important contextual role when processed through the attention mechanism, as they help align and integrate information across reasoning stages.\n\nHowever, due to space constraints, it is difficult to present a full example of the selected KV cache from the long generated outputs. Therefore, we instead conducted a quantitative analysis using embedding-based sentence similarity.\n\nSpecifically, we applied sentence-level vector embeddings to the generated outputs and computed pairwise cosine similarity between sentence embeddings. Two sentences were considered “semantically similar” if their cosine similarity exceeded 0.75. Based on this, we defined the redundancy rate as the proportion of sentences that have more than N(N=1,2,4) semantically similar sentences within the same output.\nThis analysis was conducted on outputs from three reasoning LLMs (R1-Distill-Qwen-7B, R1-Distill-Llama-8B, and QwQ-32B) using the AIME 2024 dataset.\n\nAs shown in **Table JM3v-3**, our analysis reveals that the redundancy rate dropped significantly after applying RPC. This result provides strong evidence that RPC successfully leverages semantic sparsity to reduce redundancy in reasoning trajectories.\n\n**Table JM3v-3: Embedding-based redundancy rate (ratios of sentences with more than N similar sentences during generation) reduction with RPC**\n\n| |       N=1       |    |       N=2       | |       N=4       |   |\n|-----------------------|-----------------|---------|-----------------|---------|-----------------|---------|\n|  | Full KV | RPC     | Full KV  | RPC  | Full KV  | RPC     |\n| R1-Distill-Qwen-7B | 65.5%   | 33.8%   | 48.9%   | 15.4%   | 29.8%    | 4.1%    |\n| R1-Distill-Llama-8B  | 69.7%     | 33.8%   | 52.7%   | 15.4%   | 35.0%  | 4.1%    |\n| QwQ-32B   | 66.0%   | 28.2%   | 48.0%| 13.3%   | 29.6% | 5.3%    |\n\nIn the revised version of our paper, we will include both comprehensive token selection visualizations and the full embedding-based redundancy analysis in the appendix. We believe this will help readers more clearly appreciate the effectiveness of RPC in eliminating redundancy while preserving critical reasoning information.\n\nOnce again, we sincerely thank the reviewer for this helpful suggestion.\n\n[1] Jiaming Tang, , Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, Song Han. \"QUEST: Query-Aware Sparsity for Efficient Long-Context LLM Inference.\" ICML. 2024.\n\n[2] Oren, Matanel, et al. \"Transformers are multi-state rnns.\" arXiv preprint arXiv:2401.06104 (2024).\n\n[3] Zhang, Zhenyu, et al. \"H2o: Heavy-hitter oracle for efficient generative inference of large language models.\" Advances in Neural Information Processing Systems 36 (2023): 34661-34710."}]

Author rebuttal:

Meta-review: The paper focuses on making LLM reasoning more practical and introduces a KV cache compression approach (RPC) that takes advantage of the redundancy and semantic sparsity of reasoning paths. The proposed method is shown to achieve significant compression with minimal impact on accuracy. 

The approach is nicely motivated and intuitive. The various experiments and ablation studies convincingly demonstrate the relevance of the method. In particular the reviewers and AC all appreciate the additional results to address the concerns on output length vs frequency of compression, hyperparameter selection, additional analysis to showcase how RPC effectively preserves valuable content while removing redundant information, additional analysis in support of the the use of attention-based metric. 

We strongly urge the authors to follow up on their promise to incorporate these very valuable materials.
Decision: Accept (poster)