The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
Authors: Milad Aghajohari
Kamran Chitsaz
Amirhossein Kazemnejad
Sarath Chandar
Alessandro Sordoni
Aaron Courville
Siva Reddy
Venue: ICLR 2026 Poster (2026)
Keywords: LLM Reasoning
RL for LLMs
Reasoning Models
Scalable Reasoning
Test-Time Scaling
Primary area: foundation or frontier models, including LLMs
TL;DR: Delethink enables reasoning LLMs to scale linearly in compute with constant memory by chunking traces instead of carrying quadratic context. It delivers up to 40% faster speed and 70% lower memory with no performance loss.

Abstract: Reasoning LLMs suffer from quadratic compute growth as their context length increases, making reinforcement learning with verifiable rewards (RLVR) and test-time scaling prohibitively expensive. Prior work has tried to lighten the computational burden by shortening reasoning traces through pruning, summarization, or multi-stage training, but these methods remain bound to quadratic costs. We introduce Delethink, a thinking algorithm that realizes the Markovian Thinking Paradigm. Instead of producing one long monolithic reasoning trace,  Delethink thinks in a sequence of chunks, the Delethink trace. Each chunk continues reasoning by referring only to a fixed number of prior tokens, which functions as a Markovian state sufficient for progressing reasoning, while deleting the rest. This preserves continuity without carrying the quadratic baggage. As a result, compute scales linearly and peak memory remains constant. In experiments, we show that Delethink can be applied directly to off-the-shelf reasoning models ranging from $1.5\textnormal{B}$ to $30\textnormal{B}$ parameters, with no loss in performance. Extended reasoning becomes possible under fixed memory and linear compute, while enabling efficient RL training on new tasks. On the DeepScaleR dataset, Delethink trains R1DistillQwen1.5B to the same benchmark performance as a standard long chain-of-thought (LongCoT) approach, where both models generate up to $24\textnormal{k}$ thinking tokens. The difference is efficiency. Delethink reasons $40\%$ faster with $70\%$ less memory footprint. By decoupling reasoning length from context length, the Markovian Thinking paradigm opens the door to next-generation reasoning LLMs that can scale to millions of tokens with linear compute and constant memory.

Published as a conference paper at ICLR 2026
THE MARKOVIAN THINKER: AGNOSTIC LINEAR SCALING OF REASONING
ARCHITECTURE-
Milad Aghajohari Kamran Chitsaz 16.,, Amirhossein Kazemnejad "
Sarath 6,,6,7 Alessandro Sordoni Aaron Courville
Siva Reddy 133,4,5,
Mila Miicosooft Research
McGill University
*ServiceNow Research
CCanada CIFAR AI Chair
"Chandar Research Lab
"Polytechnique Montréal
8 Université de Montréal 'Equal contribution 'quual advising
ABSTRACT
Reasoning LLMs suffer from quadratic compute growth their context length in- creases, making reinforcement learning with verifiable rewards (RLVR) and test- time scaling prohibitively expensive. Prior work has tried to lighten the compu- tational burden by shortening reasoning traces through pruning, summarization, or multi- muuli training. but these methods remain bound to quadratic costs. We introduce Delethink, thinking algorithm that realizes the Markovian Thinking Paradigm. Instead producing one long monolithic reasoning trace, Delethink thinks in sequence of chunks, the Delethink trace. Each chunk continues rea- soning by referring only to a fixed number of prior tokens, which functions as a Markovian state sufficient for progressing reasoning, while deleting the rest. This preserves continuity without carrying the quadratic baggage. As a result, com- pute scales linearly and peak memory remains constant. In experiments, that Delethink can be applied directly to off-the-shelf reasoning models ranging from 1.5B to 30B parameters, with no loss in performance. Extended reason- ing becomes possible under fixed memory and linear compute, while enabling efficient RL training on new tasks. On the DeepScaleR dataset, Delethink trains R RIDistillQwenl .5B to the same benchmark performance as standard long chain- of-thought (LongCo'T) approach, where both models generate up to 24k think- ing tokens. The difference is efficiency. Delethink reasons 40% faster with 70% less memory footprint. By decoupling reasoning length from context length, the Markovian Thinking paradigm opens the door to next next-generation reasoning LLMs that can scale to millions of tokens with linear compute and constant memory.
show
INTRODUCTION
Reasoning L.MSs answer questions by generating long chain of thought (LongCoT)) before giving a final answer, a method that has significantly advanced performance on many tasks and benchmarks (Jaech et al., 2024; Guo al., 2025; OpenAI, 2025). Scaling these "thinking tokens" continues to push capability (Agarwal et al., 2025). When thinking tokens grow linearly the training and infer- ence cost grows quadratically due to the quadratic complexity of self-attention, making LongCoT prohibitively expensive. For instance, forward pass through a 1M-token context requires over thousand times more compute than one with a 32K-token context. Moreover, throughput is inversely proportional to context length, reducing token/s at MM context by over 30x vs. 32k (Ao et al., 2025).
Prior work focused on cutting reasoning length to reduce the computational costs. During inference (Yan et al., 2025) iteratively summarizes past reasoning, (Lin et al., 2025) drops irrelevant or redun- dant tokens, and others distill shorter thinking styles with skipped steps (Xia et al., 2025; Liu al., 2024a; Cheng & Van Durme, 2024; Han et al., 2024; Luo et al., 2025a). During RL training, Luo et al. (2025b) explored multi-stage training which limits the fraction of training that involves long reasoning traces and others implicitly reward shorter lengths among correct solutions (Aggarwal & Welleck, 2025; Shen et al., 2025; Li et al., 2025; Hou et al., 2025). Instead cutting quadratic cost by cutting thinking, want algorithm to think more with self-attention with linear compute.

---

Published as a conference paper at ICLR 2026
LongCoT trace


LongCoT reasoning (cornett grows inearly with more thinking takens)
Delethink trace (markovian thinking)
</8>
Chunk
Chunk 2
Chunk K
Chunk K
(context
throughout)
prompt
response
Figure Delethink Inference: LongCoT generates a single long chain-of ofain-of-thought. Delethink reasons in sequence of short chunks, where each chunk conditions only on the original prompt and a small suffix of the previous chunk.
In this paper, we investigate LLMs can reason effectively while retaining only a fixed number of recent thinking tokens and ignoring the rest. We call such model, Markovian Thinker, since the next reasoning steps depend only on the last chunk of thought, rather than the entire thought history. This avoids the quadratic cost while enabling the model to scale its thoughts with linear compute.
Current practice for training and scaling test-time compute in reasoning LLMs relies on long chain- of-thought (LongCoT) training, which incurs quadratic cost in context length. We introduce simple novel algorithm, Delethink, which can be readily applied to off-the-stee tfe-hhe-sheff reasoning LLMs to make them Markovian thinkers, thereby scaling compute linearly with the number of thinking tokens. As shown in Figure 1, Delethink produces a sequence of short chunks; each chunk conditions on (i) the original prompt and (ii) small suffix of the previous chunk, which together act sufficient (Markovian) state.
Surprisingly, Delethink is highly effective. We show that only carrying the last few thousand think- ing tokens across chunks enables LLMs to continue their reasoning while matching or surpassing their LongCoTT performance, even if the latter mirrors their training regime. For instance, Delethink inference boosts the performance of R RIDistillQwen1. thinking 128k tokens and simultancously reduces costs terms of both FLOPs and KV cache requirements. This happens while standard LongCo'T inference rarely escapes the train-time think- ing length. This is done in a zero-shot fashion, with neither prompting nor training. We observe the same pattern across open-source' reasoning models from 1.5B 30B parameters and across benchmarks spanning PhD-level questions, coding tasks, and math competitions (Section 3.
on AIME'24 24 by 9% while extending its
We show that Delethink not only enables LLM to function effectively as Markovian thinker inference time, but can also used to train native Markovian thinkers, thereby making the en- tire reinforcement learning process significantly R RIDistillQwen.. (Guo et al., 2025) on the DeepScaleR dataset (Luo et al., 2025b), where both methods could think up 24k. While the Delethink-traine model performs competitively to the LongCoT model, generates tokens 40% faster and reduces KV cache footprint by 70%. We em- pirically validate that training a model with an average thinking length of 96K tokens would require 27 H100-months using LongCoT and only 7 H100-months using Delethink (Figure 7).
compute-efficient than LongCoT. We trained
We summarize contributions:
this work, we show a simple method build Markovian thinker, that scales compute linearly and keeps memory constant during both training and inference.
'Evaluating this effect requires access the model's 's thinking tokens, which precludes tests closed-source models.
2

---

Published as a conference paper at ICLR 2026
(a) Test-time Scaling
(b) RL Training Cost
(c) Inference Throughput
Dulttlens SI, 24C
LongCo RL
Calutions inde LongCo 8L, 24K
R3 Duta 1.58
Delethink
LongCoT
0.35
0.33
AIME25 (avgh1 28)
Tokens Spcond
0.30
H100 Hours
16000
Palky Uplate
0:30
12000
46000
3000
0.23
Thisking Budget (log scale)
Thinking Tokers
Thinking
Figure 2: (a) Delethink-traincl checkpoint matches LongCoT's accuracy while scaling better at test time and using less compute. Delethink Inference also enables the LongCoT checkpoint and the original model scale. (b) Training cost vs. average sequence length: quadratic for LongCoT and linear for Delethink, as predicted. (c) Generation throughput in sglang (Zheng et al., 2023) falls with context length; Delethink keeps context fixed, whereas LongCoT grows it linearly.
We introduce Delethink Inference that enables off-the-shelff LLMs to be zero-shot Markovian thinkers.
We introduce Delethink Training, that trains these Markovian thinkers through RL. We show it matches the performance of the standard long chain-of-thought in much less compute.
2 RELATED WORKS
Efficient Thinking Some works seek efficiency by shortening reasoning traces. Some distill traces with skipped steps or tokens (Liu et al., 2024a; Xia et al., 2025). Others control length by early exits (Ding et al., 2024), adjusting the budget to problem complexity (Han al., 2024), or steering activations toward brevity (Zhao et al., 2025b). Structured prompting and collaboration help reduce tokens: CoThinking outlines then reasons (Fan et al., 2025). (Cheng & Van Durme, 2024) thinks in shorter trace of contemplative tokens. RI. approaches include stabilizing GRPO to avoid wasted length (Liu et al., 2025), and training models to prune or exit early while maintaining accuracy (Luo et al., 2025a; Dai et al., 2025; Zhao al., 2025a). Delethink is orthogonal, seeking not fewer tokens for the same performance but longer traces for better performance.
Making Room for Extra Thinking Prior work has tried to shorten reasoning traces at inference time. Lin et al. (2025) drops irrelevant tokens and redundant steps using judge LLM. Xiao et al. (2025) prunes low-value segments by perplexity. Yan et al. (2025) iteratively summarizes current reasoning so later steps continue depending only on the summary. InftyThink's 's style fixed by design, distilled from hand-crafted dataset that locks the model into one pattern. Delethink, in contrast, learns through RL.
Linear Architectures Sliding or sparse attention (Beltagy et al., 2020; Zaheer al., 2020) and kernel-based linear attention and low-rank linear attention (Katharopoulos et al., 2020; Choroman- ski al., 2021; Wang et al., 2020) aim to avoid all-pairs interactions, the quadratic component of attention, by approximation or sparsity. Mamba architecture replaces self-attention with state-space models (Gu et al., 2021: Gu & Dao, 2023; Dao & Gu, 2024), achieving constant- constant-memory. linear-time generation via recurrent state updates. Hybrid systems interleave attention with alternatives which scale asymptotically quadratic (Lieher et al., 2024; A121 Labs, 2024). Our approach orthogonal to these efforts that linearize attention or replace it. We keep the underlying model unchanged without relying on attention approximations or architecture swaps.
KV Eviction and Compression To cut inference memory without retraining, eviction policies keep only the most useful past tokens. Zhang et al. (2023); Yang et al. (2024) select important tokens based their estimated contribution to attention, while Xiao et al. (2024) preserves small set of iitention-sink tokens to stabilize quality under sliding windows. Compression approaches shrink each retained token's footprint via quantization to sub-4-bit while maintaining accuracy (Hooper
3

---

Published as a conference paper at ICLR 2026
2aketinke2kk
Delttink64k
Ddethinkäkk
AIME24
AIME25
GPQA Diamond
LiveCodeBench
0:384
0.302
0:353
0.00
R1 Disill Qwen 1.B
(aagR64)
0.336
Accurty (negh6r)
0.200
Accrozy (avg8122
0.290
Accurry (meg)33))
0.177
0.243
2.255
0.229
0.152
0.173
0.172
0.164
0.226
0.111
0.29
0.099
0.000
Thinking Tokens
Thinking Tokers
Thinking Tokens
Thinking Tokers
0.682
0.444
0.515
0.400
RI Discill Qwen 7B
(xx4884)
0.542
Azzunry (vsy814)
442
Acararc (av2812)
0:443
Acannte (ovg01))
0:423
2.1
0.364
202
0:304
1.224
0.185
8190
0c
53x
12
53k
Edk
128k
53c
88
12
53k
18c
128k
Thinking Tokans
Thinking
Tokera
Thinking Tokorss
Thinking Tokers
Figure 3: Delethink Inference on RIDistill Models: We ablate among three variants of Delethink where we change the context length. That is, for Delethink @ 8k we keep 4k tokens and generate 4k tokens, etc. All variants improve performance with more thinking. However, the 4k, test-time scales the performance above the original model's.
et al., 2024; Liu et al., 2024b). These techniques are orthogonal to Delethink: they reduce the cost of long chain-of-thougle by selecting compressing the KV cache inference time, whereas we restructure and learn a reasoning process so only a short context is needed in the first place. Recent approaches introduce additional gates learn how evict tokens during training (Eańcucki al., 2025) but usually require introducing additional learnable weights to modify the model architecture. Our approach can be readily applicable to any generation model.
3 DELETHINK INFERENCE
In standard LongCoT, sampling trace y for given prompt æ (i.e., y <(-[x)) the model's context grows linearly with the number of thinking tokens: generating the last token uses |ql + +ly|| - tokens. In contrast, Delethink reasons in sequence of chunks with fixed per-chunk thinking context size C (e.g.. 8K).
Let denote the query, typically containing system prompt and question. In the first chunk, the model generates a response up to C tokens as usual with the input query as prompt: y1 ~ T-| = q). where 1, and yı represent the first chunk's prompt and response, respectively. If the response y1 ends with [EOS], the trace is complete. Otherwise, Delethink advances by constructing the next prompt from the original query and a markovian state consisting of the last m < tokens of the previous chunk output?:
y(--))-m 1 12
where denotes concatenation and x; and yı is the prompt and response for chunk respectively. In effect, preceding reasoning tokens are deleted when forming the next prompt (hence the name Delethink). Given æy, the model generates up to C m new thinking tokens for chunk l: y (-(aa)). This procedure repeats until [8OS] is produced or the iteration cap I is reached. We refer to the resulting sequence as a Delethink trace T [ y1)..... (æ } where is number of chunks in the Delethink trace We illustrate this process in Figure 1.
In total, the LLM may think up to C + (I - 1) (C- - m) tokens, while each chunk's context remains bounded by |ql C. Since query remains constant through the Delethink process and lq] < C in practice, the maximum chr-chunk model context is OCC) = O(1) with respect to the total number of thinking tokens. In the paper, fix C/2; Appendix B.5 ablates the markovian state size.
In practice, we fold the first hundred tokens of the initial chunk into q may contain planning tokens.
4

---

Published as a conference paper at ICLR 2026
AIME24
AIME25
GPQA-Diamond
LiveCodeBench
Accuracy (ava032)
Artaray (aug832)
Accuracy (awg03)
Accuracy (sug)32)
0.75
0.
0.6
0.50
0.4
0.25
02-
0.4
2
2
2
2
215
2
2
2
2
2
2
2
Thinking Budget
Thinking Budget
Thinking Budget
Thinking Budget
Qwen3 308-A38
Qwen3 30B-A3B Delethink Infer. (ctx=16K)
Qwen3 308-A3B
Delethink Infer. (ctx=8K)
Figure 4: Delethink Inference on Qwen3. Delethink with 16k context length matches the perfor- mance of original model that has an 256k context length. Delethink with 8k still scales but does not show the same scaling profile. The reasoning of Qwen3 requires 16k context to become markovian.
Qwen3 088-A3
Qwen3 30B A3B Delethink Inference (ctx=16K)
AIME25
AIME24
GPQA-Diamond
9.322
Warde Hurd-ccmmat Frat
0.312
28
18
121
0.197
CrowtNerdBarch CoatNerBBecch
Figure 5: (Left) Problem-solving overlap of Delethink vs. LongCoT: on AIME '24 and '25 they solve nearly the same set of questions. On GPQA, each solves an equal number that the other misses. (Right) CrossWordBench stress-tests Delethink: deleting previous tokens removes access to already found words. Delethink remains competitive, but its zero-shot limits are evident.
3.1 ZERO-SHOOT MARKOVIAN THINKING FOR OFE-TUE-SSIELE LLMS
We test whether Delethink Inference can induce markovian thinking without additional training or prompting, by applying it to off-the-shelf reasoning LLMs and comparing against standard Long- COT. We evaluate Qwen3-30B-A31 Thinking (Yang et al., 2025)² and the R1-Distill family (Guo et al., 2025) (1.5B-14B), whose native context windows are 256K and 128K, respectively. We evalu- ate tasks that require extended reasoning: AIME'2 24 and AIME 25 (MAA, 2025; competition math- ematics), -Diamond (Rein et al., 2024; PhD-level questions), LiveCodeBench' (Jain et al., 2025; competition-leve programming puzzles). For controlled comparison, budget from 8K to 256K tokens (up to 128K for R1 R1-Distill). For Delethink, we additionally vary the per-chunk context C from 8K to 16K (4K to 8K for RI-Distill since it has smaller context window) to measure sensitivity to chunk size.
vary the thinking
Results and Test-time scaling Delethink Inference matches or exceeds LongCoT across models and tasks (Figures 3 and 17), despite deviating from the training-time regime of the base model. With 8k, RI-Distill (1.5B-14B) attains performance and scaling comparable to (and sometimes better than) LongCoT. Awen3-30B-A3B shows the same pattern with C = 16k. These results indicate that current reasoning LLMS exhibit latent markovian behavior even without explicit training for it. On R1-Distill Delethink uses additional thinking tokens more effectively: accuracy continues to improve up to 128K tokens, solving instances that LongCoT fails to solve. In contrast, LongCoT plateaus around 40K tokens across benchmarks.5 For Qwen3-30B-Ar the improvement with larger budgets is present but less pronounced.
'tttps: / hwg.iiggggggggg www.cc...................... ct. wwwp/////////oo/tocco/.e/..ee.ee --enn//ween330mm -2024-08-01 to 2025-02-01, following Guo al. (2025).
'We omit budget-forcing test-time scaling methods such as S1 (Guo al., 2025) because they hurt perfor- mance on reasoning M.I,Ms; see Appendix B.6.2.
5

---

Published as a conference paper at ICLR 2026
Algorithm Delethink Training Step
Inputs: query q, reasoning LLM no; thinking context size C; markovian state size m; Delethink iterations cap I: group size G; reward function R; learning rate 7/. T, R < [], []
D Delethink traces, rewards
Generate Delethink Traces:
for i +- 1 to G do
a +
> prompt generate up to C tokens b concatenate first few tokens of y trace for group
To C)
+ ( 11:100
T 6 [(x; y)l
for / <- 1 to I - do
if last(y)) = [EOS] then break
a < ② y-m: y - (2, C m) append(T, (z. y))
D keep last m thinking tokens
generate up C m tokens D appending chunk to trace
append( R, R(T))
trace reward
ap r)
Estimate Advantages:
{A[i]] + ComputeAdvantagge ((R())))
> off-the-shelf advantage estimator
Updating Parameters:
J r Er (Ta
El U(1,
Compute the loss according to Equation (1)
0 V
Effeet of per-chunk context size C Smaller C (e.g., 4K for RI-Distill, 8K for Owen3-30B-A3BD) underperforms larger C and the LongCo'T baseline, as the markovian state carried between chunks becomes too small and the model can lose the thread of reasoning. Interestingly, smaller (e-g.. 4K for RI-Distill, 8K for Qwen3-30B-A33 of the inference procedure, appears to perceive more headroom, emits [EOS] less frequently, and thus uses the full thinking budget more consistently.
often yields better scaling: because the model is unaware
Problem Coverage We compare problem coverage on AIME'24, 24, AIME'25, and GPQA-Diamont (Fig 5), counting problem as solved if the majority vote is correct. On both AIME's, both methods solve essentially the same problems (except on AIME'25), 25). On GPQA-Diamonee 77% of problems are solved by both, but each method uniquely solves about 1% that the other misses.
Limits of zero-shot Delethink To probe failure modes, we evaluate Qwen3-30B-Ae WordBench (Leng et al., 2025), which contains crossword puzzles at varying difficulty levels. Rea- soning in this tasks requires maintaining live grid plus filled entries state that can exceed the capacity of m. As shown in Figure 5, Delethink matches LongCoT on 7x7 puzzles but lags on 14x 14, especially with C 8k. This suggests that fully native markovian thinkers needed when the task-state cannot be compressed into the carried context, and that zero-shot Delethink reaches its limits in such settings.
Cross-
4 DELETHINK TRAINING
The RL objective for LLMs is to maximize the expected reward:
O(0) Bg-D,rsxo(ig) [R(T)] - 33 KL ([[(Te||Ted)
where is a dataset queries, Ty is the policy, nie is the reference policy, 3 is KL coefficient, т is generated trace using Delethink algorithm (Section 3), and R(7) is the trace-level reward. To optimize this objective, we derive the policy gradient under Delethink dynamics in Appendix B. The resulting objective closely mirrors standard RL training for LLMs (Lambert et al., 2024).
Majority voting does apply code, LiveCodeBench is excluded.

---

Published as a conference paper at ICLR 2026
Delethink
LongCoT
tall
Metric
Base
LongCoT
Delethink
Think Tokens
nS
n$
FLOP
O(n²)
O(n2s)
TFLOPs
Memory
((n)
O(nS)
O(n)
Backward Time
Ta
O(TaS²)
O(TaS)
Generation Time
To
O(TaS*)
O(TaS)
0
500K
1M
Sequance longth
Figure 6: Computational profiles of LongCo'T and Delethink scaling from n to n.S tokens.
Concretely, Delethink Training iterates three steps: (1) sample Delethink traces for queries, (2) estimate advantages for those traces, and (3) update the model to upweight advantageous tokens and downweight non-advantageous ones, Intuitively, If trace yields a correct answer, a gradient step increases the probability of all chunks in that trace, since each contributes to the reward.
Given a query q, we sample a group G of Delethink traces TI...., TG from the current policy Tha where each trace is a sequence of chunks т = {(ı,, y1)) Optimizing the expected return proceeds by taking gradients with respect to the following loss function:
1
((0) E T (110)
E
((x
æt yı; 0):
G
(((T))
per-Delethink trace loss
where ((T) = = |yn| is the total number of response tokens in Delethink trace т. This similar to GRPO, where per-trace term is normalized by the total number of thinking tokens. U(x y; represents the per-chunk (x, y) objective which closely follow that of PPO in LLMs. Specifically,
To( Å,, clip Te (3t)
To(UE) (To.a (yx)
1--1-c,,, 1,1+, A. -BKL [m|]
U(x, v,
min
where T(y) is the probability of predicting token Y1 of the chunk's response y. KL is Kull- back-Leibler loss (Shao et al., 2024) controlled by coefficent B. The advantage Å, can be estimated with any off-the-shelf estimator (Kazemnejad et al., 2025; Yu et al., 2025). For simplicity we use the GRPO formulation: Ar. = Ar, = (R(T,) - -1)/o, where R((T)) is the reward for g-th trace (e.g. whether model reaches the correct answer the end of last chunk). fu and are the mean and standard deviation of rewards across the trace group {T₂}. Pseudo-code is shown in Algorithm 1, illustrating training on single query (we optimize over batches in practice).
4.1 COMPUTATIONAL COST OF SCALING THINKING
An RL step has two parts: generation and backward for updating the policy. We study how both scale when an LLM's thinking length grows from n tokens to nS tokens under LongCoT and Delethink. As Table shows, both stages scale linearly in Delethink but quadratically in LongCoT.
Total FLOPs and Backward Time Suppose training LLM think for n tokens costs C' FLOPs. With LongCoT, scaling by S costs O(C)2) because self-attention grows quadratically with length. Delethink instead runs in 28 chunks. Each chunk carries forward tokens and generates y new 8 The result is O(4n linear in S. Backward time tracks total FLOPs.
Peak memory KV Cache entries have fixed size per token. In LongCo'T, the KV cache grows linearly with thinking length, limiting parallel requests on the GPU. For example, the KV cache of
We omit conditioning on the context æ. y.c for brevity.
Assume m C/2

---

Published as a conference paper at ICLR 2026
Delethink
LongCoT
LongCoT
Delethink
LongCoT
350
300
AIME25 (avg0122)
0.30
250
0.28
200
0.21
150
17
100
0.22
0.20
100
200
300
400
500
600
700
800
900
1000
14
5k
8k
12k
16k
20k
23x
Step
Thinking Lengths Positive Examples
Figure 7: (Left) Delethink Training and LongCoT both train RIDistillQwenl.3 to think for 24k tokens. Delethink learns effectively as LongCoT even though it uses 70% less memory and generates tokens 40% faster. (Right) Delethink and LongCoT use their thinking budgets well. At longer lengths, Delethink produces correct answers, showing it spends its budget effectively.
a IM-token trace on RI-Distill 1.5B, small model, alone fills an entire H100. Going beyond re- quires sharding the sequence across GPUs with requence-paralleine adding heavy communication overhead. In contrast, Delethink keeps only the current chunk's KV cache, usage stays constant.
Generation Time Assume an infinite stream of requests saturating the GPU, each with maximum length L. The optimal throughput satisfies T x under simplifying assumptions (Ao et al., 2025; see B.2). This can be intuited from the fact that generation must access the KV cache, which grows linearly with sequence length, limiting parallel requests. LongCoT uses a growing KV cache; Delethink keeps it fixed per chunk. Both must generate n.S tokens, but LongCoT's throughput falls by factor of S, scaling time by S2. Delethink's throughput is constant, its time scales only S.
RL TRAINING EXPERIMENTAL SETUP
Model and Dataset. We train RIDistillQwenl.5 1.5B (Guo al., 2025) the DeepScaleR dataset (Luo et al., 2025b), which contains 40k verifiable math questions. We benchmark our models on A4ME'24, AIME'25 (MAA, 2025), and HMMT.
Baselines. Our main baseline trains with a maximum of 24k thinking tokens. We also train a baseline with 8k thinking tokens show the benefit of allowing more thinking tokens. We refer to the first as LongCoT-24k and the latter as LongCoT-8k
Training Setup. We set Delethink cap iteration, I 5, and thinking context size,C 8k tokens. With this setup, the model is able to think up 24k tokens which equals LongCoT's thinking budget. We train all models for 1000 RL steps with learning rate of 1e-6. In each step we sample 128 questions and generate 8 responses per question. Delethink is implemented in the verl framework (Sheng al., 2024) and uses sglang (Zheng et al., 2023) for generating responses. Training runs on 8 H100 GPUs without sequence parallelism. See B.4 for full details.
DELETHINK TRAINING RESULTS
Delethink and LongCoT-244k share 24k-token thinking budget and start at the same pre-training performance (Figure 7), showing that Delethink turns RIDistillQwenl.5 into a Markovian thinker on par with its LongCoT counterpart (Section 3). Despite each Delethink RL step costing less than a LongCoT-24k step. Delethink matches and at equal RL steps, surpasses LongCoT-24k during training (Figure 7). Thus, Delethink learns effectively as LongCoT while using fewer compute resources. The extended budget size is critical. With only 8k tokens, LongCoT underperforms Delethink by 5.5% (Figure 7), confirming that the additional 16k thinking tokens necessary for
8

---

Published as a conference paper at ICLR 2026
the stronger results and that Delethink learns to leverage them. Figure 8 supports this as it shows that both LongCoT-24k and Delethink increase their accuracy while spending their thinking budget.
Training also induces budget awareness. As discussed in Section 3, zero-shot application of Dele- think Inference might over-iterate or under-iterate when the maximum iteration is unknown. Our RL scheme penalizes traces that exceed the maximum iterations and rewards those that respect it. Over training, successful responses lengthen as the thinker exploits its extra thinking budget (Figure 7), while negatively rewarded, overthought traces shorten.
6.1 EMPIRICAL COMPUTE
We evaluate the compute efficiency of Delethink against LongCoT-24k. Delethink completes each RL step in 215s, faster than LongCoT-24K at 248.5s. Its token generation is also more efficient, reaching 8,500 tokens per second per H100 versus 6,000 for LongCoT-24k. As a result, Delethink finishes batch response generation in 130s compared to LongCoT's 170s, in line with the predic- tions of Section 4.1. The backward pass takes 80s for Delethink while 70s for LongCoT-24k. At first glance, this appears to contradict theory. However, the difference arises from constant and lower-order terms that dominate at shorter sequence lengths. Figure 11 shows the crossover for RIDistill@wen1.55. below 32k tokens, Delethink can be slower since attentionn blocks domi- nate, but beyond that point, the quadratic scaling of attention makes Delethink increasingly faster. At one million tokens, Delethink achieves 17x reduction in FLOPs. In Figure we empirically measure the time of single RL step assuming a target average thinking length. As theory predicts, time grows quadratically for LongCoT but linearly for Delethink. Additionally, We also measure the throughputs of inference engines under both methods. Consistent with the theory, Delethink main- tains the same throughput regardless of thinking length, while LongCoT shows a steady decline.
6.2 DELETHINK TEST-TIME SCALING
Delethink Inference vs. LongCoT We investigate the test-time scaling behavior of Delethink and LongCoT. As shown in Figure 2, both LongCoT-244 and LongCoT-8k quickly plateau once they reach their training-time limits, indicating that LongCoT's test-time scaling is largely constrained by its training budget. In contrast, Delethink continues improve even when reasoning with 100k more tokens than it encountered during training. For example, some AIME'SS 25 problems are only solved after 140k tokens B.7, despite the model being trained just 24k. This demonstrates that Delethink enables genuine test-time scaling of thinking tokens far beyond its training-tinn -tme limits.
Delethink Inference on LongCoT Checkpoints As discussed in Section 3.1, off-the-shelf LLMs behave as Markovian thinkers under Delethink. This lets us generate traces from models not trained with it. That suggests Delethink Inference may improve the LongCoT-24k checkpoint though it was not trained with Delethink. We test this in Figure 15. Delethink Inference lets the LongCoT-24K checkpoint scale past its training limit at test time. Performance rises by nearly 4%, almost matching the gain from RL training, with no extra training cost. We hypothesize that keeping queries under 8k tokens makes the 24k baseline behave as if it has room to continue. But when queries stay under 16k, the effect disappears, breaking the infinite budget illusion. Delethink Inference might improve conventionally trained checkpoints, though they cost quadratically more to train than Delethink.
6.3 CONTEXT SIZE ABLATION
We ablate context size to 4K and 2K, set the state size m to half the context size, and adjust the number of iterations to keep the maximum thinking length at 24K tokens. We call these runs 8K, 4K, and 2K. As Figure 16 shows they successfully train with smaller state for Markovian Thinking.
6.4 SCALING DELETHINK TO 96K
We scale the thinking budget from 24K to 96K tokens to test Delethink's scalability. We fix the context at C = 8K and raise the iteration cap from I = 5 to I = 23, yielding a 96K total budget. We train OpenMath (Moshkov et al., 2025), harder benchmark than DeepScaleR that demands longer reasoning. Starting from the 24K checkpoint, we continue training for 150 steps with un- changed hyperparameters. Figure 10 reports results AIME'24 and AIME'25. Delethink 96K

---

Published as a conference paper at ICLR 2026
Dekttink
24K
LangCT BK
E22
AIMEZ4 (sug(l 28)
AIME25 (avyee 28
0.1
0.24
6:8
6:18
0.8
41
96
12k
15k
18k
21
24t
41
66
96.
15k Thisking Budgett
12k
13a
21k
Thinking Budget
Figure 8: Cumulative accuracy on AIME 24: LongCoT-8k plateaus once it exceeds its 8k training- time limit. In contrast, Delethink and LongCoT-24k learn use their thinking budget effectively.
Dekettik (C+6K)
Deletink
Deketbis
0.70
844
ANNE2 (6c))32)
0.40
Toin Reward
0.38
055
022
300 400
500
600
700
100
200
300
400
500
500
Step
RL Step
Figure 9: We vary C while fixing the thinking budget at 24K tokens. (Left) Smoothed training reward vs. RL step. (Right) Validation accuracy (AIME'2 (4M vs. RL step. Delethink 24K with = 8K and 4K performs similarly. The 2K variant lags but improves steadily over the base model via training.
surpasses the 24K checkpoint and matches or exceeds 24K with test-time scaling to 128K tokens. The average reasoning length reaches 36K tokens AIME'24 and 42K AIME 25, indicating active use of the larger budget. These results show that Delethink scales to long reasoning traces.
Delethink 96K
Delethink 96K + Extented Thinking Budget
Delethink 24K
Thinking Budget of 128K
AIME24 (Tasce Lee.)
AIME25 (sugh120)
AIME25 [Tince Len.)
AIME34 (sug81120)
30K
644
30K
30K
28K
2MK
Stop
Stop
Stop
Stop
Figure 10: Scaling Delethink 96K. AIME'24/255 accuracy and average trace length vs. RL step for Delethink 96K; dashed curves show extended thinking budget. Despite only 150 RL steps, 96K surpasses both baseline and its extended thinking variant, with mean trace lengths up to 42K.
DISCUSSION
Extending thinking length has advanced reasoning LLMSs, but quadratic compute cost. We propose the Markovian Thinking Paradigm, where each step retains only the minimal state needed to continue, yielding linear compute and constant memory for both training and inference. We show it is practical: off-the-shelf LLMs become Markovian via Delethink Inference and can be further refined into native Markovian thinkers with RL through Delethink Training. Experiments match LongCoT performance while using linear compute and fixed memory, and Delethink Infer- ence enables test-time scaling of thinking length beyond static LongCoT budgets. Thus, Markovian Thinking moves from concept practice, opening a path reasoning tens of millions of tokens.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
SR is supported by CIFAR AI Chair and Mila Salasang grant. AC is supported by Canada CIFAR Al Chair, the Canada Research Chair, the NSERC Discovery Grant, and funding from Mi- crosoft Research. SC is supported by the Canada CIFAR AI Chair, the Canada Research Chair in Lifelong Machine Learning, and the NSERC Discovery Grant. We thank the Mila IDT team and Digital Research Alliance of Canada for providing the compute resources used in our experiments.
REFERENCES
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applehaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, al. gpt-oss-120b- & gpt-oss-20b model card. arXiv preprint arXiv:2508. 10922,, 2025.
Pranjal Aggarwal and Sean Welleck. LI: Controlling how long reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.0460 2025.
A121 Labs. Jamba-1.5: Hybrid transformer-mamba models at scale. arXiv preprint arXiv:2408. 12570, 2024. URL UtL.htppss//arrrvvvorr//// 1252422700
Ruicheng Ao, Gan Luo, David Simehi-Levi, and Xinshang Wang. Optimizing 1lm inference: Fluid- guided online scheduling with memory constraints. arXiv preprint arXiv:2504.11320 2025.
Iz Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.051550 2020. URL
05150.
Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.13171, 2024.
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamás Sarlós, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, and Adrian Weller. Rethinking attention with performers. In International Conference on Learning Representations (ICLR), 2021. URL https: https:
2009 14794.
Muzhi Dai, Chenxu Yang, and Qingyi Si. S-grpo: Early exit via reinforcement learning in reasoning models. arXiv preprini arXiv:2505.07666 2025.
Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprini urXiv:2405.2 2024. URL https: //arxiv.. org/abs 2405. 21060.
Mengru Ding, Hanmeng Liu, Zhizhang Fu, Jian Song, Wenbo Xie, and Yue Zhang. Break the chain: Large language models can be shortcut reasoners. arXiv preprint arXiv:2406.06580 2024.
Siqi Fan, Peng Han, Shuo Shang, Yequan Wang, and Aixin Sun. Cothink: Token-efficient reasoning via instruct models guiding reasoning models. arXiv preprint arXiv:2505.22017, 2025.
Albert Gu and Tri Dao. Lincar-time sequence modeling with selective state spaces, arXiv preprint 007522 2023. URL https: /tt///aavvvv///////////////aaev/ http//aaww..rraaaa///o//aaaaaaaa wttp///a///....1/..00..3
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces, In Advances in Neural Information Processing Systems (NeurIPS), 2021. URL https://arxiv org/ abs//bb//////oo
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint 2.Xi::2501.124:: 2025.
Tingxu Han, Zhenting Wang, Chunrong Fang. Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token- budget-aware Ilm reasoning. arXiv preprint arXiv:24/2.18544 2024.

---

Published as a conference paper at ICLR 2026
Coleman Richard Charles Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. KVQuant: Towards 10 million context length LLM inference with KV cache quantization. In Advances in Neural Information Processing Systems, 2024. URL 2024 (poster).
forum? /ttpss///ppeeeveeeeee net forrruuummmme///om (tt.rrmmmwmdttttttteeeee0 NeurIPS
Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. Thinkprune: Pruning long Shain-of-thoughe of Ilms via reinforcement learning. arXiv preprint arXiv:2504.01296. 2025.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412. 16720, 2024.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamina- tion free evaluation of large language models for code. In The Thirteenth International Confer- ence on Learning Represenlations, 2025. URL https:/ /openrevieee net forum? iid= chfJJYC3iL.
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are mms: Fast autoregressive transformers with linear attention. In NeurIPS, 2020. URL https:
16236.
Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux. VinePPO: Refining credit assignment in RL train- ing of LLMs. In Forty-second International Conference Machine Learning, 2025. URL https: opwnreview. net / (orummidd=My2223FAA
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brah- man, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15/1 2024.
Jixuan Leng. Chengsong Huang, Langlin Huang, Bill Yuchen Lin, William W. Cohen, Haohan Wang, and Jiaxin Huang. Crosswordbench: Evaluating the reasoning capabilities of Ilms and Ivlms with controllable puzzle generation, 2025. URL https: /ttpiv.iiii.///aa///.a//aaaaa..a org/ abs/2504 00043.
Chen Li, Nazhou Liu, and Kai Yang. Adaptive group policy optimization: Towards stable training and token-efficient reasoning. arXiv preprint arXiv:2503. 15952, 2025.
Omer Lieber, Benjamin Lenz, Ido Szpektor, Kevin Leyton- Brown, et al. Jamba: A hybrid transformer-mamba language model. arXiv preprint arXiv:4403. 2024. URL https:
Weizhe Lin, Xing Li, Zhiyuan Yang. Xiaojin Fu, Hui-Ling Zhen, Yaoyuan Wang, Xianzhi Yu, Wulong Liu, Xiaosong Li, and Mingxuan Yuan. Trimr: Verifier-based training-free thinking compression for efficient test-time scaling, 2025. URL https: /ttp:///aivvvv/vi org....oooooo..ooooooooooooooo.. org/ 17155.
Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. Can language models learn skip steps? Advances in Neural Information Processing Systems, 37:45359-45388 2024a.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl-zero-like training: A critical perspective. arXiv preprint arXiv:2503.2072 2025.
Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. KIVI: A tuning- free asymmetric 2bit quantization for KV cache. In Proceedings of the 41st International Conference Machine Learning, volume 235 of Pro- ceedings of Machine Learning Research, pp. 32332-32344. PMLR, 2024b. URL https: / //proceed ngs. mlr. .presp v235/ 1 ii24bb html.
12

---

Published as a conference paper at ICLR 2026
Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-prumer: Length- hoagth-harmoizzing fine-tuning for ol-like reasoning pruning. arXiv preprint arXh::250.125570, 2025a.
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing ol- preview with a 1.5b model by scaling rl. https://pretty-ttteeeearpr notioooooooooooonnoooooooo 2025b, Notion Blog.
notion. site/
MAA. American invitational mathematics examination (aime) 2024: Aime i and aime ii, https:
/ maa. org/ maa- iiiiiiitttss wnw.tatioonllooooooooooo www.ttiinnnlppppp compiiiinncccmmttiiiiio/s///so/t 2025. Official competition prob- lems provided by the MAA.
Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. Aimo-2 winning solution: Building state-of oate-of-theeart math- ematical reasoning models with openmathreasoning dataset. arXiv preprint arXiv:2504.1689) 2025.
OpenAl. Gpt-5 system card. https: //dn. openai.n........... www.cccm///./.o 5- 3-system-- card. aard August 2025, Accessed: 2025-09-17.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a bench- mark. In First Conference on Language Modeling, 2024.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Yi Shen, Jian Zhang. Jieyun Huang. Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, and Shiguo Lian. Dast: ing models. Xiv preprint
slow-thinking for large reason-
25:::2203.0447.. 2025.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 19256, 2024.
Sinong Wang, Belinda 7.. Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04766; 2020. URL http:://://ar abs abs/ 04768,
Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. Tokenskip: Controllable thain-of-thought compression lms.. arXiv preprint arXiv:2502. 12067, 2025.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming
language models with attention sinks. In International Conference on Learning Representattonss 2024. URL https::////..........tt http///ppeeeeeeeeeeeeeeee net forum? wtwrummmmi-GNGsssoo d=NG7
zVF. ICLR 2024.
Yang Xiao, Jiashuo Wang. Ruifeng Yuan, Chunpu Xu, Kaishuai Xu, Wenjie Li, and Pengfei Liu. Limopro: Reasoning refinement for efficient and effective test-time scaling. arXiv preprint arXiv:2505.19187 2025.
Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yueting Zhuang. Inftythink: Breaking the length limits of long-context reasoning in large language models. ArXiv. 0as/2503.066977 2025. URL https: /appi.pp..../ www.nticscco......... orr//
CorpusID:276903:
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin. Kai Dang. Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui
13

---

http://isssp.ppppppeeeee////pppp Published as a conference paper at ICLR 2026
Men, Ruize Gao, Shixuan Liu, Shuang Luo. Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXiv preprint arXiv:2505.088 09388, 2025.
Shuo Yang, Ying Sheng, Joseph E Gonzalez, Ion Stoica, and Lianmin Zheng, Post-training sparse attention with double spansity. arXiv preprint arXiv:2408. 070022 2024.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv: 14476, 2025.
Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang. and Amr Ahmed. Big bird: Transformers for longer sequences. arXiv preprint arXiv:2007. 14062, 2020. URL https:tt///// happ:///arxt http///xxxvvorrr/ abs/2007.2 14062.
Zhenyu Zhang, Ying Sheng. Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen. H20: Heavy-hitter oracle for efficient generative inference of large lan- guage models. In Advances in Neural Information Processing Systems, volume 36,
2023. URL
Haoran Zhao, Yuchen Yan, Yongliang Shen, Haolei Xu, Wenqi Zhang, Kaitao Song, Jian Shao, Weiming Lu, Jun Xiao, and Yueting Zhuang. Let llms break free from overthinking via self- braking tuning. arXiv preprint arXivv2505.14604, 2025a.
Weixiang Zhao, Jiahe Guo, Yang Deng. Xingyu Sui, Yulin Hu, Yanyan Zhao, Wanxiang Che, Bing Qin, Tat-Seng Chua, and Ting Liu. Exploring and exploiting the inherent efficiency within large reasoning models for self-guided efficiency enhancement. arXiv preprint arXiv:2506.1560 2025b.
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. Sglang: Efficient execution of structured language model programs. arXiv preprint urXiv:2312.07111 104, 2023. doi: 00448500arXi.22112.077
URL https: /ttttp:::/rrxxv.rrtte wwtp:vv.orgg/b///111111 07104.
Adrian Lańcucki, Konrad Staniszewski, Piotr Nawrot, and Edoardo M. Ponti. Inference-time hyper- scaling with kv cache compression, 2025. URL.htts:///are wwwps//w.rgg///////ee//ooosse//e 2506. 05345.
A
APPENDIX
B
LLM USAGE
We used ChatGPT to refine the English writing of the paper.
B. DERIVING DELETHINK LOSs
In this section, show that the Delethink Training loss is basically the policy gradient. Note that the Policy gradient without the PPO clipping is the following:
V (
=B TTu
Σ vo log (a 8t) A
Here, T = -Spp ag. To...., S.) denotes a trajectory sampled from the policy, and A, is the advantage time In Delethink, the actions are generating the response segment of each chunk given the
14

---

Published as a conference paper at ICLR 2026
query segment. Note that we need to sum this across the Delethink trace. Plugging in the policy gradient would be,
lyf Ae log Tolue |x() ,x2
VaJ(0)
By
(3)
If we add the PPO Clipping and the ratios to this objective it results in:
|r| ly U(T,0)
min
clip
(rá(y()) xx - 2,1+c) A.]
(4)
where
To (0( (6)
x
y
(0))
(5)
(6)
Tom
Ui
The only extra term that is left is the KL term which is added only constrain the policy not to deviate from the base model. With that addition, we will have:
Irl (
(O) = By
min () )A, clip (r (0), 10,,1-,,10 c. An -3KL Red
t=1
(6)
That concludes our proof.
B.2 DERIVING THROUGHPUT RELATION
We follow Ao et al. (2025). Consider a GPU with maximum memory M, serving an infinite stream of incoming requests and equally unbounded stream of completed outputs. Let n² denote the equilibrium number concurrent requests. Each request has prefill tokens and l' decode tokens. In steady state, the KV-cache memory required is
M = n" 01++ (1+
According to Ao et al. (2025), the equilibrium throughput of an attention bassdd LLM under mem- ory constraint is
(8)
do + di n- 1
where do the fixed per-batch overhead and di the time cost per unit of memory. Thus, throughput exhibits an inverse dependence the total effective context length. In regimes where di n ( { > do. we obtain the approximation
71
Throughput"
(9)
di n (1
di (
Consequently, for the same GPU, the ratio of throughputs when decoding lengths are In and 1/ satisfies
Tr
1
1+
R3
Tr
+
di
(14
In the long-thinking regime where 1' > I, this simplifies to
Tr K
T
P
establishing that throughput is (approximately) inversely proportional to the decoding length this regime.
15

---

Published as a conference paper at ICLR 2026
Delethink
LongCoT
le9
1.5
TFLOPs
1.0
0.5
0.0
20K
40K
Sequence length
When training RI RiDtsiillQeenl. with Delethink with 8k thinking context, for thinking lengths ~ 30k the number of FLOPs for both is equal. This is because the non-attention components like dense layers dominate here. However, after 30k quadratic cost of attention dominates.
Figure 11:
Intuition. Decoding repeatedly accesses the KV cache, whose size, and therefore memory-time cost, grows linearly with the number of decoded tokens This linear growth drives the inverse relationship between throughput and (' in the long-thinking regime.
B.3 DELETHINK FLOPs CROSSOVER vs. LONGCOT24K
We can compute, in closed form, the FLOPs required for reasoning of a certain length under Dele- think and LongCoT. We calculated this for main experiment setup, where Delethink runs with = 8K, m 4K, single step with 1000 episodes, and results are shown in Figure 6. We high- light the crossover in Figure 11 at short sequences. Up to roughly 30K tokens, Delethink spends slightly more FLOPs than LongCoT. That seems counterintuitive: Delethink scales linearly, Long- CoT quadratically. This is because in Delethink each token is generated once and then reprocessed as input to the next chunk. While the quadratic compute of attention quickly overtakes this double- counting cost in longer sequences, it matters short lengths. This is clear when considering attention layers. FLOPs in non non-attention layers scale linearly, and in Delethink each token is gener- ated once and then reprocessed in the next chunk as prompt. Therefore, the cost of non-attention layers doubles. The crossover occurs around 30K tokens in our setting. Beyond this point, Dele- think is linearly more efficient in FLOPs, generation time at these response lengths is still slower for LongCoT. This is because gen- eration throughput is limited by memory-access and memory-footprint bottlenecks, not just FLOPs. As explained in Section 6.1, our training still runs faster even in this regime due to faster genera- tion. Our next step, Delethink 96K, significantly faster than its counterpart LongCoT-RL. 96K in both backward pass and generation, to the point that we could not train the latter under our compute resources.
of FLOPs. Note that while total backward time tracks total
B.4 DETAILED RL TRAINING SETUP
In this section describe training setup and hyperparameters meticulously.
B.4.1
HYPERPARAMETERS
Table represents the key hyperparamters used in our RL training experiments.
B.5 PREFILI ABLATION OF QWEN3
In Figure 12 we showcase what necessary covry-over size to make the thinking process markovian for different models.
16

---

Published as a conference paper at ICLR 2026
Hyperparameter
Delethink Training
RL LongCoT
Rollout (inference)
Sampling (T' top-p cop-k/ni n)
0.6/ 1.0/ -1/8 2,048 8,192
0.6/ 1.0/ -1/8 2,048 / 24,576
Prompt / response length (tokens)
Algorithm
Advantage estimator
GRPO N/A
GRPO N/A
KL penalty
Policy optimization
PPO epochs/ / number of grad updates
1 1/2 0.20
1/ 0.20 0.226
Clip ratio (low / high)
Grad clip (global-norm)
1.0 Trace Length- Saatth-Batch Mean
1.0 Seq-mean-RRe -acch Mcan
Loss aggregation
Actor optimizer
Learning rate
1 x 10 0
x 10-6
Weight decay
0
Warmup
constant; steps -1 (disabled) 1,000
constant; steps = (disabled)
Total training steps (optim)
1,000
Data
Train / val batch size
128 / 16 (per step, logical)
128 / (per step, logical)
Rewarding crities
Reward function
HFF-Math-Vers Verify
HF-Math- Verity
Table I: Key hyperparameters for RL LongCoT vs. Delethink Training.
Rl-distillqqwen-1558
R1-distill-qwen--7
qwen33-00--33b-hiiiinn-2---
0.70
0.7
GPoA-Diamond Avg@e
AIME24 Avg@32
0.60
0.6
0.55
0.5
0.45
0.4
0.40
prefill
prefill
Figure 12: Ablating the necessary state size to make the process markovian. 1B and 7B R R1Distill models become markovian even with short context lengthes. Qwen3 which has an original 256k context, requires larger state size for becoming markovian thinker.
B.6 ZERO-SHOT MARKOVIAN THINKING FOR R RDDISILL
B.6.1 WORKING MEMORY ABLATION
In Figure 13 we ablate the necessary context size and its affect on zero-shot Delethink performance.
B.6.2 BUDGET FORCE ABLATION: DOES SI WORK RI?
We evaluate Budget-Force decoding scheme inspired by the SI "simple test-time scaling" proto- col (Guo et al., 2025), We run decoding and, whenever the model finalizes early before using the allotted token budget, we cut the response at the first finalization marker and append short contin- uation before regenerating from the entire prompt. Concretely, for each query first sample with temperature 0.6, top-p = 0.95, n = 1 under a 32k token budget. If the model emits any of </think>, **Final l Answer**, \boxed (or EOS) before the budget consumed,
17

---

Published as a conference paper at ICLR 2026
Delethink (C=BK)
Delethink (C=4K)
Delethink (C=2K)
LongCoT
AIME24
AIME25
GPQA-Diamond Diamond
LiveCode Bench
Accuory (2w9864)
Arzunry (wa)064)
Aczuncy (awge32)
Arznncy (22))22)
0.2
Qwnn
8:00
0.88
0.13
800
822
0.8
6:10
52k
Thisking
Thisking
Thinking
Thinking
Acurary (awp644)
Accrncy (megl64))
Accunayy (2ee8aa))
8.51
0.0
0.33
Qwen
835
0.8
6.44
8.2
0.8
0.15
128
328
Thinking Tokens
Thinking Takers
Thinking Tokers
Thinking Takers
Accuraty (oe8b4)
Rccuracy (ney0644)
Accuracy (oay)32)
0.5
Accuracy (oag)33)
6.56
6.57
Quen
120
13K
120K
13C
12:K
10KK
120K
12K
128K
Thinking Tokens
Thinking Tokees
Thinking Tokers
Thinking Tokess
Figure 13: Thinking Context Size Ablation. Scaling behavior of RI-Distill models across AIME'24, AIME' 25, GPQA-Diamond, and LiveCodeBench under varying per-hunk contexts (2k, 4k, 8k. Accuracy is plotted against total thinking tokens, with smaller contexts requiring proportionally more iterations.
Table 2: Accuracy (%) across benchmarks under identical compute/decoding budgets.
Model
Method
AIME24
AIME25
GPQA-Diamond
LiveCodeBench
Long CoT 32k
28.2
24.2
33.7
18.8
RI RI-distill-qwen-I.3
Budget Force
24.6
19.9
33.4
17.7
Delethink
37.6
29.7
34.6
19.9
Long CoT 32k
53.1
39.3
49.5
38.2
RI -i-dns
Budget Force
50.1
35.1
47.8
36.6
Delethink
64.8
47.5
50.1
40.1
Long CoT 32k
68.9
52.9
59.1
51.71
RI-distill-qpeen-14
Budget Force
70.2
52.6
59.1
51.9
Delethink
72.6
52.5
57.5
trim at the earliest occurrence, append the literal cue Wait \n, and re-issue a generation from query respoose. We repeat this micro-forcing until the round budget is exhausted continuation cap (20) is reached. To avoid distributional drift, we do not alter model decoding settings mid-run; the only intervention is the prompt-level cut-and-continut At the end, the model still has not emit- ted EOS after fully using the budget, we add a compact finalization hint (**Final Answer** \n) and request short natural continuation. All models identical sampling parameters across condi- tions. We compare three settings: (i) Long-CoT 32k (single pass, no forcing). (ii) Budget-Force as above, and (iii) Delethink. Unless noted, we use cont math tasks are verified with the HF checker using < thinkk>>>> as the thinking delimiter. Scores and LiveCodeBench under
20, poun_samples= 8,
ported in Table 2 are accuracy on AIME 24/A2ME 25, the same compute budgets. In almost all the cases, Budget Force peforms worse than the LongCoT baseline (normal sampling), demonstrating that such methods are not applicable to reasoning models like R1-distii disiiil family.
18

---

Published as a conference paper at ICLR 2026
AIME25
LongCoT Delethink
0.8
Accuracy
0.6
18
0.4
0.2
218
11
28:
23
12425
27
400
4k
24k
45k
66k
86k
107k
128k
Thinking Tokens
Figure 14:
The average thinking length per question and its corresponding accuracy. Delethink Inference truly test-time scales RIDistillQwen1 1wen1. ..5B performance on AIME'25.
B.6.3 HYPERPARAMETERS
B.7 AIME'25 DETAILED LENGTH VS. ACCURACY
In Figure 14 show detailed lengths of Delethink traces solving AIMR'25 to showcase that Delethink solves questions via genuine test-time scaling.
19

---

Published as a conference paper at ICLR 2026
C DELETHINK INFERENCE ON LONGCOT CHECKPOINTS
LongCoT-RR T-RL 24K
LongCoT RL + Delethink (C=8K)
LangCoT RL 24K + Delethink (C=16K)
bulning
t1aining
AIME25 (avgi0 120)
0.33
HMMT25 (ava(128))
0.18
0.17
0.30
0.16
0.29
0.27
16K
12K
64K
123K
16K
32K Thinking Budget
64K
128K
Thinking Budget
Figure 15: Delethink Inference on a LongCoT-RL. 24K checkpoint (AIME'25, avg 128). Long- CoT plateaus its trained budget, while Delethink (C 8k, C 16k) keeps scaling; C 8k yields the larger gain. The shaded region marks the trained budget.
D CONTEXT SIZE ABLATION RESULTS
Delethink 24K (C=BK)
Delethink 24K (C=4K)
Delethink 24K (C=2K)
0.44
0.65
AIME24 (2vg032)
0.40
Train Roward
0.60
0.36
0.55
0.32
0.50
0.28
0.24
100
200
400
500
600
700
200
300
400
500
700
RL Step
Step
Figure 16: We vary while fixing the thinking budget at R3 24K tokens; smaller C yields a smaller Markov state. (Left) Smoothed training reward vs. RL step. (Right) Validation accuracy (AIME'24 24) vs. RL. Delethink 24K with C 8K and 4K performs similarly in training reward and validation accuracy. The 2K variant lags but improves steadily over the base model during Delethink training.
E ZERO-SHOT DELETHINK INFERENCE ON STATE-OF-THE-ARY LLMS
In Figure 17 show zero-shot application of Delethink Inference off-the-shelf state-of-the-art LLMss.
F DETAILED TASK PERFORMANCE
In Figure 18 we show detailed performance of Delethink checkpoints on both in-distribution and oft-of-distribution tasks.
G DELETHINK EXAMPLES
In Figure 19 and Figure 20 show Delethink Inference in action.
20

---

Published as a conference paper at ICLR 2026
Delethink (C=16K)
Delethink (C=8K)
LongCoT
AIME24
AIME25
GPQA-Dlamond
Live CodeBench
Acunsyy (2w9064)
Rccunyy [1g)644)
Rccuncy (2sg833)
Accuracy (owe933))
GPT-OSS 120B
6:76
0.00
8.4
852
0.90
0:4
8331
033
0.90
0.3
Thinking Budget
Thinking Bulget
Thinking Budget
Thinking Budget
Accursy (av)864)
Accunsyy (wa))44)
Ricuracy (2wge322)
Accurscy (2uet32)
Qwen3 30B-A38
0.3
03
0.50
0.44
8.5
0.8
0.51
0.8
Thinking Budget
Thinking Budget
Thinking Budget
Thinking Budget
Figure 17: State-of-the-art reasoning LLMs, GPT-OSS- and Qwen3-30B-A3B, are capable of Markovian Thinking zero-shot, providing strong initialization for training, signaling scalability. Delethink closely tracks LongCoT and recovers most of final accuracy (the curves nearly coincide on Qwen3)
Delethink
LonyCoT-RL
LongCoT-RL,
R1-Disill
0.48
Perfurmance
Accuracy
0.32
0.24
L.M
2.11
0.16
0.08
AIME2A
AIMEZ5
HMMT725
GPOA Demand
UneCndeBench
Figure 18: On III math tasks (AIME'24/25, 25M Delethink outperforms LongCoT-RL. 24K. Shaded regions show gains from test-time scaling (through sequential sampling). where Dele- think improves the performance even more; on OOD tasks (GPQA-Diamond, LiveCodeBench) gains are modest, yet Delethink still matches slightly beats LongCoT-RL 24K.
21

---

Published as a conference paper at ICLR 2026
Prompt
Given that O is any point in space, and A,, B3, C. D are four points such that no three of them are collinear, but they coplanar, and OÅ 2x BO 3y CO + 1s DO, find the value of 2x 3y 1s.
Response
nik Okay, got vector prohlem
again
heing asked.
We have a point space, and four points C, such that three them are collincar. they are coplanar. That all four points lie the plane, but none of them lie the straight line. Interesting. So, they 're coplanar, hut not colinear.
Then, we re given that the vector OA equal to 2x times vector BO plus by times vector CO plus 42 times vector DO. We by
I...I
Alternatively, pethaps if take another vector product, say the scalar product with A. but that might not be helptul.
Wait, pethaps I should consider that the equation must bold for any position of O. which would roquire that the coefficients multiplying equation be Wail, see:
From:
7 (2x + 39 + 4=)0 (2x
d + 4D
So.
x 2 3y 1
2x.
Since O is arbitrary. the only way this equation holds is numerator is proportional to but since free, perhups the coeflicients must satisfy:
The numerator must zero, but that conflicting.
Wait, maybe Altematively, perhaps if take different points, say itself, set up equations.
Wait, getting tangled. Let's stop back.
We have the equation:
O 2n(BO) + 3yy(O) 4o(00)
I...I
Expressed
7- 2x - 4
Expanding:
7-7 =20 - 35 + 1:D
Bring all tenns to the left:
A = (2x + 3y
1)ō
-2x B 3gC 1:D
Wait, that similar before. Then, rearrang::
7 B + B 4= + By 1)
But since O is arhitrary, the vector on the right can be any vector in space because can be any vector, but the left side is
fixed 7 2rr B C. + 12. So the only way this holdd
[..] </taink>
Figure 19: Example of Chunk on boundaries in Delethink-RL The horizontal line specifies the chunk boundary.
22

---

Published as a conference paper at ICLR 2026
Prompt
Suppose that and posilive inlegers with such that the interval [mn, contains multiples of 2021 than multiples of 2000. Compute the maximum possible value of n mn..
Response
<think> have this problem here: need lind the maximum possible value where and positive integers with m < n. and interval [m,, n)) contains more multippes of 2021 than multiples 2000. Hmm, intoresting, Let
break this down step by step.
First, understand
n, including Within inserval, the number of multiples of 2021 should be greater than the number multiples of 2000. need to find the largest possible difference between n and that satisfies condition.
exactly heing
an
[...]
Wait, let's take 2001, 4000. But wait, needs be than m. 2001, 4001. Then, in (2001,4001), multiples of 2000: 2000 is less than 2001, so only 4000. So, Multiples of 2021: 2021" 1=2021, 2021 12-4042, which beyond 4001. So, Still equal.
Wait, perhaps when is just multiple 2000,
too close.
Wait, let's try a different m. Suppose m 2021, then the interval would he [20211 n). The number multiples of 2021 interval is (if beyond 4042) and number of multiples 2000 would be the number multiples of Let's say
2000 2021
Then, multiples 2000: 2000 less than 2021, so not included. 4000 is less than 4042, so within (042). Wait. no,
4000 less than 4042, it's still less than 4=402 Wait, but 4000 less than 4042, (2021,4042) multiples of 2000 are 4000 only 4000 the interval. Wait, 4000 less than 4042, so yes, included. Wait, no, 4000 less than 4042, [2021,40 (4042) includes 4000? Wait, 4000 less than 4042, if m=2021, n=4042, the interval (2021, (2021,4042) includes 4000? No, because 4000 less than 2021. So, from 2021 to 4042, the multiples of 2000 are 4000 is not included, next is 6000 way abuve. Wail, actually.
multiple
2021
more than 2000, so the next multiple would he 4000 + 2000 6000.
[...]
m=1, 2=2000" +1. m=2000% + something. Wait, perhaps it's better think in terms of how many multiples of 2000 and 2021 included.
interval includes many multiples as possible including multiples of 2021. Alternatvelly, the other way around.
Wait, perhaps
occurs
Wait. but the previous example. m=l, nm4043, (1,4043,, which includes 2000 and 4000 multiples of 2000, A=2, and includes 2021 and 4042 multiples of 2021, so B=2, Thus. A=B=2, so not satillynng AJB.
Rut when mm4042, [1,4042), multiples of 2000 are 2000 and 4000, which are hoth included, so A=2. Multiples of 2021 are 2021 4042, which is excluded, only 2021 included, so Bel. Thus, A=2 ; B=1, which satisfics condition.
So, n m 4042 4040)
Wait is that correct? Wait, 4042 -1:4041. So, n=4042, m=1. Then, (1,4042) includes 2000,4000 (A=2) and 2021,4041 excluded), actually in [1,4042), multiples 2021 2021, excluded. So,
L..]
Wait, perhaps something clse.
Wait, let m=2021,
How many multippes are there? included? 2021 by
20or(2000/000: floor(n- (/20000)))) 2?
Wail. if 4042
2021,4042) includes multiples 2000: 2000 loss than 2011 lirst multiple 4000, which in (202) (042), and 6000 excluded.
So, A=1.
Multiples of 2021: 2021,4042, 4042 is excluded, only 2021. Thus, B=I.
[..]]
</thirks
Figure 20: Example of Chunk on boundaries in Delethink-RL. The horizontal line specifies the chunk boundary.
23


Reviews:
[{"review_id": "7jdtylIhQZ", "reviewer": "Reviewer_iAzf", "summary": "The paper introduces Delethink, an algorithm aimed to mitigate the quadratic computational cost of long chain-of-thought (LongCoT) reasoning in LLMs. The proposed method, replaces a single monolithic reasoning trace with a sequence of shorter chunks. Each chunk is generated by conditioning only on the original prompt and a fixed-size suffix of the preceding chunk, allowing the model to theoretically scale its reasoning length with linear compute and constant memory. The paper presents two primary use cases: a zero-shot inference strategy for off-the-shelf models and an RL training framework (Delethink Training). Empirical results show that Delethink can match the performance of standard LongCoT on reasoning benchmarks while claiming to be more efficient in terms of speed and memory usage.", "questions": "Please see Weaknesses Above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses the critical and timely problem of controlling increasing complexity in LLM reasoning because of long chain-of-thought.\n2. The proposed Delethink algorithm is simple and can be applied as a zero-shot inference strategy to existing models without requiring any kind of architectural modifications.\n3. The results are generally comprehensive, showing stronger performance at lower costs for both inference and training of reasoning models.", "weaknesses": "1. Identical compute budgets: Major figures in the paper are compared based on ``thinking tokens'' but Delethink also has to recompute kv cache and q values after every chunk. A more fair comparison would have been if wall-clock and optionally TFLOPs across methods would have been reported.\n2. From results, it seems that the performance of Deletethink is heavily dependent on the choice of length of chunk context. For instance, 1.5 and 7B models, it performs well with 8K on GPQA, but for AIME it performs better with 4K. Similarly, for Qwen3-30B even 8K performs worse than 16K. Given strong dependence it is unclear how can the optimal length (or even the one that performs better than longCoT) of chunk context be determined.\n3. ``Summarization-based reasoning'' (e.g., iterative summarization, InftyThink) enablelonger effective thinking at lower cost, including at least one of these baselines is essential (e.g., iterative summary every K tokens).\n4. From figure 2a, it seems that for certain token lengths (at lower token lengths), Deletethink performs worse than LongCoT. Further, since we don't results with wall time on x-axis are not shown, it is unclear whether the same trend holds for other model/dataset combinations as well. If this is generally true, then this should be acknowledged in the limitations section.\n\nThe above four points are the primary weaknesses of the paper.\n\nIn addition:\n\n## Minor Weaknesses/Suggestions:\n\n5. Ablate the ``fold first 100 tokens of initial chunk into q.'' This is a useful practical tweak but undermines the purely markovian nature claim. Can authors show how much of the zero-shot benefit depends on this?\n6. Other types of baselines: Comparison of LongCoT with sliding window (if architecturally possible) or KV eviction, i.e., do not re-prefill q between chunks could strengthen the claims.\n7. Suggestions for writing and presentation: \n    - The section 4.1 could be simplified in formulation. For instance, variable C is introduced but not used again. Similarly, the need for extending length from n to nS seems unnecessary?\n    - There are several empty ablation sections such as A.5, A.6.1, A.6.2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FgP6RLJdtx", "reviewer": "Reviewer_ntQj", "summary": "This paper introduces Delethink, a novel algorithm designed to overcome the quadratic computational cost associated with long chain-of-thought (LongCoT) reasoning in Large Language Models (LLMs). The core idea is the \"Markovian Thinking Paradigm,\" where the model reasons in a sequence of short, fixed-context chunks. Each new chunk conditions only on the original prompt and a small suffix (the \"Markovian state\") of the previous chunk, effectively \"deleting\" the rest of the reasoning history. This approach decouples the total reasoning length from the computational context, leading to linear scaling of compute and constant memory usage.", "questions": "See weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The Markovian Thinking Paradigm: A new formulation for LLM reasoning that avoids the quadratic cost of self-attention over long sequences.\nAn RL training framework to create native Markovian thinkers, which is shown to be significantly more compute- and memory-efficient than standard LongCoT training while achieving competitive or superior performance.\nComprehensive experiments demonstrating that Delethink matches or surpasses LongCoT performance on complex reasoning benchmarks (e.g., AIME, GPQA) while being 40% faster and using 70% less memory, and enabling effective test-time scaling beyond training-time limits.", "weaknesses": "The authors should analyze the content of the reasoning traces in successful vs. failing cases. What information is being lost when tokens are deleted? \nThe claims of orthogonality to methods like KV cache eviction and quantization are valid, but a combined evaluation is missing.\nIf deleting already generated tokens is incompatible with KV cache technology, how does the inference speed compare to LongCoT?\nIn lines 247-248, \"we vary the thinking budget from 8K to 256K tokens.\" How is the thinking budget controlled?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SAWhOzRd8n", "reviewer": "Reviewer_YiPw", "summary": "The paper deals with efficiency in reasoning language models, particularly the compute cost of long chain-of-thought. The paper proposes Delethink, which generates reasoning in chunks, with each chunk conditioned only on the prompt and a fixed-size subset of tokens from the previous chunk. This achieves more favorable compute scaling and constant memory, while maintaining or modestly decreasing accuracy in the settings tested.", "questions": "Please address the statements and questions described in the Weaknesses above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- Simple, intuitive, and creative method based on feeding in subsets of the reasoning tokens-so-far.\n- The zero-shot finding is surprising: feeding the truncated reasoning trace-so-far can match performance of using the full reasoning trace, without additional training, in some settings (Figure 4).\n- Leads to good gains in efficiency (training cost, inference throughput, computational cost of scaling the token budget).\n- Enables improved performance at large token budgets in the tested settings.\n- Clear formalization of the method and clearly written paper.", "weaknesses": "- Most analysis is done on AIME 24 and AIME 25. It's unclear whether the results generalize to other settings, which is particularly noteworthy due to the assumptions present in the method (discussed in the next two points). \n- In the zero-shot setting, the method relies on the Markov assumption; concretely, that the relevant context is contained in the beginning and ending of the chunk of the reasoning trace. It is indeed surprising that this assumption held for Qwen3 30B-A3B on the math reasoning datasets in Figure 4. However, I am skeptical that this would hold in many tasks. If so, the zero-shot setting may be of limited use. \n- When the model is trained, the learning algorithm should ideally put the relevant context into the beginning/end part of the chunk so that it is available at the next step. I have two concerns:\n    - The zero-shot experiments seem to suggest that standard LongCoT may naturally place tokens in positions that are amenable to Delethink. Do you have evidence that the model learns to place tokens much differently than with standard Long CoT training?\n     - Can you provide experiments on a broader range of tasks beyond AIME to demonstrate generalization? The CrossWordBench results in Figure 5 suggest potential limitations when task state cannot be compressed into the carried context. \n- Delethink underperforms standard long CoT at reasonable thinking budget levels (4k-21k, Figure 8).\n- Delethink Training appears to be a standard RL algorithm applied to the MDP defined by the Delethink setting. I'm fine with this if it works well and the paper isn't overclaiming; however I'm listing it as a minor weakness to denote that the training is not particularly surprising or innovative.\n- The method is referred to as a \"thinking algorithm\"; it is unclear what this means.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vTNZfYPtZz", "reviewer": "Reviewer_pDsN", "summary": "It proposes a novel reasoning algorithm that enables large language models (LLMs) to perform extended reasoning with linear compute complexity and constant memory usage, in contrast to the standard Long Chain-of-Thought (LongCoT) approach, which incurs quadratic computational costs due to the self-attention mechanism’s dependence on full context length.\n\nA zero-shot method that applies to off-the-shelf reasoning LLMs (1.5B–30B parameters), enabling them to reason over hundreds of thousands of tokens with no performance loss—and often performance gains—compared to LongCoT.\n\nAn RL-based training framework that trains LLMs as native Markovian thinkers. It matches LongCoT’s task performance while using 70% less memory and achieving 40% faster token generation.", "questions": "1. How to select the per-chunk context size C of different size of model and tasks?\n2. the different between this work and mem-agent[1].\n3. Does the different rl algorithm have different effects on preformance such ppo/reinforce++ ?\n\n\n\n[1] MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  It introduces a genuinely novel paradigm—Markovian Thinking—for long-horizon reasoning in large language models (LLMs). While prior work has attempted to compress, prune, or summarize reasoning traces to mitigate the quadratic cost of attention, Delethink reframes the problem entirely: it assumes that only a fixed-size suffix of prior reasoning is necessary to continue coherent thought. This is a conceptual shift from “how to shorten thinking” to “how to make thinking Markovian.”\n2. Comprehensive empirical evaluation across multiple model scales (1.5B–30B), benchmarks (AIME, GPQA-Diamond, LiveCodeBench), and reasoning lengths (up to 128K tokens).\n3. Efficient test-time scaling: Models can reason far beyond their training-time token budgets without retraining.\n4. Feasible RL training on long-horizon tasks (e.g., 96K-token reasoning would require ~27 H100-months with LongCoT vs. ~7 with Delethink).", "weaknesses": "1. lacks of comparision between mem-agent[1] on long-context-generation task.\n2. It carries the last few thousand thinking tokens across chunks enables LLMs to continue their reasoning while matching or surpassing\ntheir LongCoT performance. Did you compare this to summarization of the chunks.\n\n\n\n[1] MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)