Next-ToBE: Probabilistic Next Token-Bag Exploitation for Activating Anticipatory Capacity in LLMs
Authors: Yihe Liu
Huibin Wang
Xianming Hu
Pinyi Zhang
Jiahao Xiong
Chenglin Wang
Nuoyi Chen
Hongbo Zhao
Jie Zhang
Kai Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Large language models
anticipatory capacity
Primary area: foundation or frontier models, including LLMs
TL;DR: Next-ToBE introduces a soft-target distribution that activates and refines anticipatory capacity in LLMs, improving reasoning performance by looking ahead beyond the immediate next token.

Abstract: Auto-regressive large language models (LLMs) exhibit a non-trivial capacity to "anticipate'' long-range future tokens despite being trained to predict only one token at a time. Nevertheless, how to systematically profile, enhance and leverage such capacity to practically improve LLM reasoning performance remains unclear. In this paper, we propose **Next Token-Bag Exploitation (Next-ToBE)** to tackle this challenge. Next-ToBE quantifies LLM’s anticipatory capacity by measuring how well tokens in the future window are pre-captured by the model’s current softmax probabilities. This capacity is  strongly correlated with LLM generative quality but often suppressed by the rigid one-hot objective in next-token prediction. To address this, we replace the {one-hot target vector} in next-token prediction  with a soft target distribution 
spanning additional future tokens. Specifically, the immediate next token retains the highest importance, while more distant ``look-ahead tokens'' are also included to enrich supervision, with their importance  dynamically determined by temporal and semantic relevance patterns to   inject forward-looking pressure. 
Besides, the fitting process emphasizes the model’s intrinsic anticipatory tendency, thus preserving the confidence and fidelity of the pre-trained model to improve training stability. 
Overall, Next-ToBE not only effectively activates LLM  anticipatory capacity through fine-tuning, yielding notable gains in  reasoning performance with higher memory and computational efficiency against the MTP baselines, but also shows great potential in pretraining setting by successfully cultivating  this capacity from scratch. These  highlight its value as an effective strategy to extend the prediction horizon of LLMs, enabling them to see further, and reason better.

Published as a conference paper at ICLR 2026
NEXT-TOBE: PROBABILISTIC NEXT TOKEN-BAG EXPLOITATION FOR ACTIVATING ANTICIPATORY CAPACITY IN LLMS
Yihe Liu Huibin Wang"", Xianming Hu Pinyi Zhang', Jiahao Xiong', Chenglin Wang',
Nuoyi Chen², Hongho Zhao', Jie Zhang?, Kai Zhangl
1 School of Computer Science and Technology, East China Normal University
2 Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University
ABSTRACT
Auto-regressive large language models (LLMs) exhibit a non-trivial capacity to "anticipate" long-range future tokens despite being trained predict only one token at time. Nevertheless, how to systematically profile, enhance and lever- age such capacity to improve LLM reasoning performance remains unclear. this paper, we propose Next Token-Bag Exploitation (Next-ToBE) to tackle this challenge. Next-ToBE quantifies LLM's anticipatory capacity by measuring how well tokens within predefined future window are pre-captured by the model's s current softmax probabilities. This capacity is strongly correlated with LLM gen- erative quality but often suppressed by the rigid one-hot objective in next-token prediction. To address this, we replace the one-hot target vector in next-token pre- diction with soft target distribution spanning additional future tokens. Specifi- cally, the immediate next token retains the highest importance, while more distant "look-ahead tokens" are also included to enrich supervision, with their impor- tance dynamically determined by temporal and semantic relevance patterns to in- ject newward-looking pressure. Besides, the fitting process emphasizes the model's 's intrinsic anticipatory tendency, thus preserving the confidence and fidelity of the pre-trained model to improve training stability. Overall, Next-ToBE not only ef- fectively activates LLM anticipatory capacity through fine-tuning, yielding no- table gains in reasoning performance with higher memory and computational ef- ficiency against the MTP baselines, but also shows great potential in pretraining setting by successfully cultivating this capacity from scratch. These highlight its value as an effective strategy to extend the prediction horizon of LLMs, enabling them to see further, and reason better.
INTRODUCTION
Large language models (LLMs) built on the auto-regressive next-token prediction (NTP) (Williams & Zipser 1989) have achieved remarkable success across diverse language modeling tasks (Yang al. 2025 Abdin al., 2024). Although NTP is restricted to generating only one token at a time, recent studies suggest that LLMs trained under this paradigm also exhibit anticipatory capacity, implicitly forecasting tokens beyond the immediate next one. For example, Pal et al. (2023) show that intermediate-laye can encode global response properties even before producing the first token; Wu et al. (2024) further show that this is attributed to gradients from future-token losses shaping earlier predictions.
hidden states can predict future tokens; Dong et al. (2025) report that LLMs
The anticipatory capabilities of LLMs mirror the human cognitive process of internally rehearsing before speaking (Barthel et al. (216)), i.e., "think first, generate later", which can be critical for tasks requiring multi-step planning and Tong-range logical consistency, such as mathematical reasoning (Ahn et al. 2024), code generation (Jiang et al. 2024), and human-computer interaction (Li al. 2023). However, emerging evidence suggests that the next-token prediction (NTP) paradigm may
"These authors contributed equally to this work.
1 "Correppondig author.

---

Published as a conference paper at ICLR 2026
inadvertently suppress this capacity. Deng et al. (2220) showed that the one-hot fitting objective in NTP inherently biases the model toward the marginal distribution of individual tokens, limiting its ability to capture Long-range semantic and temporal dependencies. Such emphasis step-level correctness could promote short-range pattern matching and undermine multi-step planning ability (Bachmann & Nagarajan| (024). Finally, penalizing plausible high-confidence alternatives (e.g., synonyms or paraphrases) deviating from the single ground-truth label reduces the model's semantic flexibility and undermines its ability to maintain coherent internal representations (Li & Lu 2021
To address these limitations of NTP, many work have focused on the new paradigm of "multi-token prediction" (MTP). MTP employs multiple parallel auxiliary output heads to predict several future tokens simultaneously (Qi et al.] [220;; Gloeckle et al. [222]] Cai et al.] (2024). Subsequent en- hancements use sequence modeling to tighten long-range dependency (Liu et al. (022a), improving sequence generation consistency by predicting non-adjacent tokens (Liu et al., 2025) or by masked- generation and knowledge-distee distiillaiionnn (Samragh et al. 2025). Meanwhile, efforts within NTP train- ing still continue: Gerontopoulos et al. (2025) incorporate register tokens into input sequences to share memory of future label tokens in advance; and Zuhri et al. (2025) propose an auxiliary loss to predict the rank of upcoming label tokens based their distance from the current step.
Although these advances offer substantial advantages, some concerns remain. First, higher com- putational and memory demands could be entailed, like additional parameters (prediction heads), register tokens or auxiliary loss calculation. Second, these approaches still rely on one-hot fitting objective (including MTP), thus inheriting some of its intrinsic limitations like reduced semantic flexibility or over-concentration of predicted probability mass. Third, few studies have systemat- ically profiled the anticipatory behavior in LLMs, making difficult to clarify its role in shaping model performance, or to effectively enhance it for meaningful gains in real-world applications.
In this paper, we introduce Probabilistic Next Token-Bag Exploitation (Next-ToBE), a simple method to profile, refine, and harness the anticipatory capacity of LLMs to enhance reasoning per- formance. The main idea is leverage the LLM's soft-max predictions at each step as an observable proxy for its internal anticipatory signal, and to promote consistency between this implicit planning and actual sequence of future tokens within a predefined window. Concretely, this is achieved by replacing the conventional one-hot objective of next-token prediction with a flexible token-bag dis- tribution constructed from the window of upcoming tokens. The construction of this distribution follows three principles: (1) The immediate next token is given the highest importance to preserve local coherence and prevent semantic drift. (2) The model's 's intrinsic anticipatory tendencies is re- spected preserve stability and fidelity of the pre-trained model in fine-tuning setting. (3) Distant future tokens are dynamically weighted by their joint semantic-temport relevance to mitigate noise from loosely related contextually unstable future tokens.
Overall, Next-ToBE provides rich supervision that activates and refines the internal anticipatory ca- pacity of LLMs. It is simple to implement, requiring only a modification of the target distribution within the standard next-token prediction framework and introducing no additional parameters. Ex- periments across the Qwen and Llama model families demonstrate that, on challenging reasoning tasks, base models fine-tuned with NeBE achieve consistent and notable performance improve- ments over strong MTP baselines, particularly in long-horizon and multi-step reasoning scenarios. Similar performance gains are also observed in the pretraining setting.
The contributions of this work are summarized as below:
1. We propose Next-ToBE, method profile and activate inherent planning capacity of LLM by replacing one-hot fitting in NTP with flexible target distribution over future tokens.
2. We propose a dynamic weighting scheme based on joint semantic-temporal relevance pat- terns to determine the importance of future tokens and capture multi-step dependencies.
3. We show that TeBt-ToBE delivers consistent and tangible gains in terms of language mod- eling and complex reasoning in both pretraining and mhe-tuning over strong MTP baselines.
2 RELATED WORK
In recent years, Next- Token Prediction (NTP) has achieved remarkable success in solving difficult problems Wies et al. 2023), showing powerful learability (Malach) 2023) and endowing L.MSs an
2

---

Published as a conference paper at ICLR 2026
ability to foresee content in the longer future (Wu et al. 2024 Dong et al. 2025 Cencerrado al.
2025 Mcanwhile, critics argue that teacher-forcing strategy overly relies on ground-truth prefixes
[Bachmann & Nagarajan 2024), leading exposure bias (Schmidt 2019) and error accumulation
[LeCun] 2023) that may hamper long-range reasoning (Bubeck et al. 2023 McCoy et al. 2023).
Recently, multiple-token prediction (MTP) architecture (Qi al. 2020; Gloeckle al.] 2024 Liu
iet al., 2024a; Chen et al. 2025bb Liu et al. 2025 Samragh et al. (025) and advanced inference
techniques (Cai al. [2024] Ankner et al. 2024/Chheng et al. 2024 have been proposed provide
richer supervisions during training and mitigate inference error accumulation. Researchers have also proposed transforming training data into non-causal format compel the models cap-
ture correlations between non-adjacent target tokens (Bavarian et al. 2022 Thankaraj et al. 2025 Gerontopoulos al. 2025). Due to space limit, detailed reviews are deferred to the Appendix B
Different from existing methods, our approach remains within the NTP framework but refines its training objective by replacing the one-hot target vector with a soft target distribution. This can be viewed as special form of "label smoothing" one-hot label for the immediate next token is softened by incorporating its temporal and semantic relations with subsequent tokens in the sequence. It requires no modification to LLM architecture and can be easily embedded to improve LLM anticipatory capacity and reasoning performance.
(Szeeennn (Szegedy al. 2016 Müller et al. 2209),, where the
METHODOLOGY
We begin by introducing a quantitative measure for token-level anticipatory capacity. A key ob- servation is that LLMs rarely produce strict one-hot vectors. Instead, the model ouiput typically assigns non-zero probabilities to multiple candidate tokens, resembling form of internal planning. To reflect this, we define the Future-tokens Hit Rate (FtHR), i.e., how well the model prediction at current step f, Pe Pa (V((r)))
E R |||x1
can pre-capture the upcoming k tokens, {x(+1, Tr42...,, Tt+k).
"ttkk
FIHR
Σ rets top, (P(M))))))
A(N
Here we consider top-m slots the LLM output vector, N is the length of the whole sequence, and V is the vocabulary set. Using this measure, highlight two key empirical observations:
LLMs possess intrinsic anticipatory behavior by pre-capturing future tokens. As illustrated in Figure la LLM's prediction at step t, P& (V((ae)) already contains future tokens, with FIHR metric ranging from 0.79 to 0.40 when considering windows of length from 2 to 10. These results were obtained using the Qwen-Math-1 1.5B model the NuminaMath dataset (Tang et al. 2024). In other words, at each inference step, the LLM does anticipate beyond the immediate next token.
0.8
Actual Prob. of generating TLLA
k=2
Future token Hit Rate
0.7
0.7
0.6
057
0.6
0.5
3.48
0.45
0.4
0.4
2
7 Future token bag (window) size k
8
10
10a
10'
10"
Rank of 2; in current prediction PaMMa
(a) Proportion of ground-truth tokens in [t, being captured by Po(Vlzg)), versus k.
(b) Prob. of generating x++* after k steps, rank of 2144 within the current prediction Ps (V 2g)).
Figure 1: The LLM's token-level pre-planning capacity (left), and its correlation with how faithfully an LLM can generate future token k-steps away in an auto-regressive manner (right).
2. LLM anticipatory capacity is linked to its ability to accurately generate future tokens. Figure T shows that, if ground-truth token Fe+h located & steps ahead receives higher probability
3

---

Published as a conference paper at ICLR 2026
in the current prediction Pe(V (in terms of higher rank), then I++4 is likely to be generated (in terms of the probability of auto-regressively generate T++ at step tk)). This suggests that enhancing the anticipatory capacity could practically improve token-level generative accuracies.
Since LLMs possess an intrinsic anticipatory capacity that is positively associated with performance, even enhanced
ask how this capacity be more fully activated gains. Unfortunately, the one-hot fitting objective forces the model to collapse its predicted proba- bility toward 'single correct answer', leaving little room for alternative possibilities and forward anticipation. To overcome this, we propose Next-ToBE, which replaces the one-hot objective with soft-target that effectively integrates broader future information.
achieve practical
3.1 PROBLEM FORMULATION
Next-Token Prediction(TTP): Given sequence {21, v and model Pa. the teacher- forcing strategy minimizes the following negative log-likelihood each time step i
N-1
CNTP
with Po(V (rgee the predicted probability distribution vocabulary V, conditioned Øgg.- Multi-Token Prediction (MTP): Instead of fitting only the next token, MTP make h parallel pre- dictions at time step I, each with an auxiliary head p(j) for j 1, ..., k responsible for predicting the ground-truth token steps away. MTP objective extends the standard NTP loss by minimizing the sum of the negative ligeliikl of these & future tokens,
9,
N- k
CMIP =
Σ
log pli) (a++s axge).
(3)
t=1
/=1
Next Token-Bag Exploitation (exx-TToB):: NeXI-ToBE replaces the one-hot target distribution in NTP with carefully designed soft target distribution over the future token bag, thereby effectively activating the anticipatory capacity LLMs. The loss function is written as
N-K-1
LNex--TBE
Σ
log Po(xe41lage))
WHet log Po aEg))
(4)
the immediate next token
look-ahead tokens
The first term is standard NTP objective, focusing the immediate next token (Z++1); while the second term encourages model to fit the subsequent k- 1 look-ahead tokens {#442,... Fetk). )- The coefficient balances the contributions of these terms, and the non-negative weights Waits further modulate the relative importance of the k - 1 look-ahead tokens.
Different from MTP that employs l prediction heads, Next-ToBE only employs single head and generates one token at each step in inference. In this sense, is in the middle hetween NTP and MTP: uses single head like NTP, but takes into account multiple future tokens each step like MTP. Interestingly, it is precisely through this one-token-at -aaeen------imee generative process that Next- ToBE continuously activates the latent anticipatory capacity of LLMs, thus making more accurate token-level predictions and achieving stronger reasoning performance.
3.2 WEIGHTING SCHEME IN TOKEN-BE BAG EXPLOITATION
We follow the three key principles in Section I for designing À and Wxet; in Eq. (4 to effectively capture long-range correlation while maintaining stable, effective learning process:
1. Importance of the immediate next token. The first term in the Next-ToBE loss, which is on fitting the immediate next token, should be assigned dominant importance. This is essential for preserving semantic coherence between the preceding context and the immediate next token. Moreover, the dense supervision of NTP objective is well known stabilize large-scale optimization
4

---

Published as a conference paper at ICLR 2026
Token-Baa Baken-Bag&a at step
(with & tokens)
Prefix Xet
Yyak
Mayi
Weat
weigftts
Inherent model preference
Token bag information
LIM
X
Soft prediction (frowe))
Temporal graph
Visting Prob.
P(u]ker)
Semantic groph
Rander
wisk
Po xt))
Vacibalayy
Trxc
Soft prediction (learnable)
hormaliv
Polmes ass))
Coest-TBBE
log Pa(z: 2 (ss)) +
log
Σ Pilandess))
Loss function
Immediate next taken
look-ahead tokens
Figure 2: Overview of the Next-ToBB architecture.
and accelerate convergence. To do this, we select a small weighting coefficient E (0, in Eq. to ensure that the first term dominates. To facilitate choosing A, normalize the second loss term-corme coreepodnnn to the k "look-ahead" which will discussed in section 3.3
The weights Wzes5 assigned future tokens Te+j in Eq. play crucial role in shaping the learning dynamics. We decompase the weights into two components, Wa+,, = Cg Berty where axety captures the intrinsic anticipatory preference of the base LLM with respect to the future tokens Warts" while Be,r, encodes their temporal-semanti relevance, as detailed below.
2. Inherent anticipatory preference of LLM. The output distribution of LLM already contains inherent anticipatory preferences that should be fully respected. This enables the model to retain prior knowledge and maintain predictive confidence, especially given the strong capabilities of base LLM acquired through large-scale pretraining. It also mitigates conflicts between the training signal and model's s existing prediction, possibly enhancing training stability and convergence rate.
To achieve this, we introduce set of weights, O,e,,, to track how the LLM's output distribution at the current step anticipates the k 1
tokens, (Ec-2:...;:14:
{ Poloett xxg)"" if Po(zets |xgge) > for 2, ...,....k
(5)
otherwise
Here, is a threshold to filter out low probabilities, and (.)7 (y = 1/10) is applied to calibrate remaining probabilities for proper loss re-weighting. Here, azz,z is frozen in training, i.c., it is obtained from the current "snapshot" of the model at step f, and does not contribute to the gradient.
Incorporating O'z,- into future token weights Wast, forces to respect LLM's intrinsic anticipa- tory signal. Specifically, if future token Ztty is pre-captured by LLM output distribution at the current step, it likely receives higher priority. This can be seen a form of self-distillation (Hahn & & Lu 2021), which helps preserve the knowledge and confidence of the base model.
Choi 2019
3. Semantic//emporal importance of future tokens. Not all of the k 1 look-ahead tokens equally important. Temporally, tokens farther from the current step t may be less relevant and more difficult predict, so their weights should decay with the temporal gap. Semantically, future token closely related to the immediate next token should have greater importance, as it can enhance local coherence and strengthen the supervision signal by jointly reinforcing the immediate next token. To capture these two preferences, we define the temporal scores and the semantic scores as
T(l j) exp(-)²/ (2h)) s(t + exp(xT+ where h is bandwidth controlling the temporal decay. These
j E (2.3,...kk) k): be combined
Bz, 7 () j)-
5

---

Published as a conference paper at ICLR 2026
However, Eq. 7 ignores the interdependence among the k k- look-ahead tokens. To simultaneously and dynamically account for the relations not only between the immediate next token (I++1) and the look-ahead tokens (f++2..... 2 but also among the look-ahead tokens themselves, adopt a sartloommwlll-bber two & x & matrices quantify pairwise temporal and semantic relations:
ranking scheme. Given the token bag {*++1, 12441. 2242,-........ Fetkk) we construct
W (i, exp(-li ((-li-ipP/(2h)))
W, (i,j) exp(xI,x*+t) i,j (1,2, k). (8)
where x; is the embedding of token z; from initial embedding layer of the LLM. Then, both W and W, are normalized into transition probability matrices, such that vandom-wall process (Pan
al. 2004 Tong et al. 2008) can be performed on top of them to re-rank the tokens (nodes).
To utilize both relational views, we combine the two matrices by either weighted sum or multiply, W w W (1 4) W or W WW
where oo [0, 1] controls the balance. Both mixing schemes above yield valid transition matrix which induces a graph of k nodes/tokens with Wij encoding their relations (Chen et al. 2025a). We then perform random walk on W-induced transition graph that starts from the immediate next token 2,+,, and iteratively transitions to its neighbors, each step with a jump-back probability p:
g(s+1) (1 p) W ( + pe(())
(10)
where (3() R denotes the ranking distribution of the is tokens iteration q, and ( a one-hot vector indicating the start node x This iterative process is convergent at
8 = (I - - w) (0)
(11)
where I is identity matrix. The convergent distribution 3 jointly accounts for the temporal and semantic relation among the look-ahead tokens. For convenience, we re-write 3 in Eq. [11 as
Bee+s = [}]-
(12)
Empirically, using matrix multiplication in Eq. ( to construct W corresponds to the "Alternate Similarity And Proximity Attention (ASAP-attention using matrix summation; Besides, weighting schemes based on random-wale scores 3 in Eq. (TT)
(Chen et al.) 2025a), which is better than
is notably better than simple point-wise score fusion in Eq. (7). See Appendix
E.3 for details.
We combine (multiply) LLM-preference based weighting 0f Eq. (5) and temporal- semantic weighting Bayss in Eq. (12). and normalize them to obtain the final weights
Woit3
(13)
Σi=2 Oz
The normalization ensures that the weights of the look-ahead tokens, {Wz+ez..... Waeee }, form valid probability distribution, which allows for interesting statistical interpretations. Overall, the weights Wx++5 integrate both the model's intrinsic capacity and the extrinsic signals present in the data. At step f, only those tokens that are simultancously pre-captured by model output Pa (a(((gg)) and actually appearing the near-future window [t, t + kj will be prioritized fitting targets.
3.3 NORMALIZED VERSION NEXT-TOBE TRAINING ORJECTIVE
Although LLMs can pre-capture the look-ahead tokens {æ++2.... } by giving them non-zero probabilities the current prediction Po (V( their actual values can be low (i.e., 10-"), leading heavily negative log-likelihood. This could dominate the loss and force the learning process to try very hard to raise these small probabilities, which may induce unstable training. To avoid this, we normalize the probabilities (o(2 (())) over the look-ahead tokens, as
n n-
Po y xrse)
NNexLToBB
log Po(ee115s))
W Waets log
(14)
Σi Po (2ee41))5))))
Empirically, the normalization is important the performance, as shown in ablation studies in Appendix E.I Eq. (14) can be seen as the mix of two cross-entropy losses: (1) the first is the cross-entropy between the N-dimensional LLM output distribution Po(V)=ge) and the one-hot dis- tribution encoding the next token I;-1; (2) the second the cross-entropy between the (k - 1)-
dimensional LLM prediction and weight-distributiin [W tr Wz.. over look-ahead tokens.

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
Following Mangrulkan et al. (2022). Lu et al. (0023) and Liu et al. (2024b), we adopt the Next-Tobe ToBE objective to fine-tune pre-trained LLMs on datasets spanning various reasoning tasks to evaluate the performance gains it can deliver, together with detailed ablation studies.
Datasets and Baselines. We used benchmarks from three domains. For mathematical reason- ing, following the prior work (Jiao et al,, 2024). randomly sampled 35,000 prompts from the Numina dataset (Tang et al.) 2024) for supervised fine-tuning, without any additional data engineering. For code generation and commonsense reasoning, used the CodeAlpaca-20k dataset (Wang et al. 2023b) and the Commonsense-155 dataset (Hu et al.) (023) as the training cor- pus, respectively. We have reported the results of the following competing methods: (1) Next-token prediction (NTP) objective for fine-tuning LLMs; (2) Medusa, the multi-token prediction (MTP) paradigm CCai al. 2024), and (3) Mutor (Gerontopoulos et al. 2025), which leverages shared tokens as registers to model future tokens within the NTP framework. Related works that lack code release or cannot be run smoothly were excluded from our side-by-side comparison.
Training details. Experiments were run on 4x Nyiiii RTX A6000. For mathematical reasoning, we used Qwen2.5-Math-1. 1,5/7B (Yang et al. 2024 and Llama-3. -nstruc (Dubey al. 2024) as base models; for code generation and commonsense reasoning, we used Qwen Owen-2.3-1.55 2.5-115777 ./7/B and 1-mna-3.--BB-instrucc All models were fine-tuned with LoRA adapters (Hu et al. 2022) (rank at 8),
and trained for 3 epochs with batch size of 128, except that for MTP (Cai et al. 2024), an additional one-epoch warm-up stage prior to the full training is used to train the auxiliary prediction heads. The maximum sequence length is 2048 for math and 512 for code generation and commonsense reasoning, for all competing methods. For Next-ToBE, the token bag size is 10, the threshold e in Eq. (5) is le 8, the bandwidth in Eq. (8) is h 2, the restart probability p in Eq. (11) is set to 0.3, and the loss coefficient in Eq. 14 is 0.05. More detailed settings are in Appendix D.
Evaluation Protocols. For mathematical reasoning, we have used Qwen-Math (Yang et al. 2024 evaluation codebase, in which models are prompted to reason step by step, and the final answer must be enclosed in "\ "Nboxed ". For code generation, evaluated with official unit-test har- nesses (Chen et al.] [202T] Austin et al., (021). reporting pass@1. For commonsense reasoning, we followed official evaluation codebase (Hu et al. 2023). Except for MBPP (Austin et al.) (2021) in the code task, which is evaluated under three-shot setting, all other benchmarks are evaluated in zero-shot. Greedy search decoding is used for all methods. Additional details are Appendix D.2
4.1 EXPERIMENTAL RESULTS
We empirically investigate the following research questions in this section:
Q1. Does Next-ToBE improve LLM in anticipating (pre-capturing) future tokens?
. Q2. Does Next-ToBE. improve LLM in accurately generating future tokens?
Q3. Does Nexi-ToBE improve LLM in complex reasoning lusks?
Q1 Diguue 3aproports Barpertt how the LLM pre-captures future tokens by plotting FUHR, (the Future-token Hit-Ratio), i.c., how the top-50 slots in the current-step prediction, Po ()) covers future tokens up to l steps, on Qwen-Math-1 1.5tth-1.BBBB model. As can be seen, after fine-tuning, the model exhibits a significant increase in the hit ratio across & E [2, 10), meaning that Text-ToBE effectively enhances the model's anticipatory (pre-planning) capability.
Q2-Firere 3b) shows the probability that an LLM correctly generates the next l: ground-truth tokens auto-regressively from step I, conditioned on prefix Jige- As shown, after fine-tuning, the token-level generative accuracy notably improves across various window sizes. Namely, higher anticipatory capacity indeed turns into higher generative quality, laying the basis for improved reasoning.
Tables and] report results complex reasoning tasks. Across 36 comparisons spanning 3 base model sizes and 12 categories in Mathematics, Code Generation, and Commonsense Reason- ing, Next-ToBE attains the highest accuracy in 35 cases, against strong competing methods such as MTP (Gloeckle et al. 2024) and MuToR (Gerontopoulos et al.) 2025). These results highlight effectiveness of Next-ToBE in enhancing complex reasoning across diverse tasks and model scales.
7

---

Published as a conference paper at ICLR 2026
Next-ToBe ToBE Finetaned
Nex- Finetuned
Next Finetured Base model
Token Rate
Base model
Generalivee socurary
Basa model
0.44
Densty
0.2
02
DA
Fulure token bag (windwe size
Generative longth (k steps)
Top token confidence (Prob.)
(a) Future-Tokens hit-ratio (antici- pation capacity) vs window size k.
(b) Token-bag generative accuracy versus token-bag size k.
(c) Distribution of top-token prob. after Next-ToBE line-tuning
Figure 3: After Next-ToBE fine-tuning, LLM anticipatory capacity (a) and token-level generative
accuracy (b) both improve; the next-token confidence (probability) slightly decreases (c).
Table I: Performance comparisons on five mathematical datasets across three different base LLMs.
Minerva Math
Olympiad Bench
Model
Method
MATH
AIME24
AMC23
Avg.
NTP
64.8
19.1
30.7
10.0
45.0
33.9
MTP
66.0
20.6
29.0
10.0
47.5
34.6
Qwen2.5-Maie
MuTToR
67.6
22.1
31.4
6.7
52.5
36.1
NoBI-ToBE
68.2
23.2
31,9
13.3
55.0
38.3
NIP
71.4
24.3
33.6
10.0
57.5
39.4
MTP
72.6
23.2
32.3
13.3
60.0
40.3
Qwen2.5- 1at22-5-Math-75
MUTOR
72.2
24.6
33.9
20.0
55.0
41.1
Next-ToBE
73.6
25.4
34.8
20.0
60.0
42.7
NTP
48.6
21.0
14.8
10.0
25.0
23.9
MTP
48.2
23,5
15.6
6.7
22.5
23.3
Llama3.1 1iiiu3.--B-iisttuuee
MUTTOR
49.6
23.2
15.3
6.7
27.5
24.5
Next-ToRI
50.6
23.9
16.3
10.0
32.5
26.7
For Mathematical reasoning (Table T]. Next-ToBE achieves an average accuracy of 38.3%, about 2.2%-4.49 higher than others on Owen2.5-Math-I1. 1.5B, with highest gains on AIME24 (+3.3%) and AMC23 (+2.5%). On Qwen2.5-Math-7B and Llama3.1-8P --8-iistuett Next-ToBE also attains the highest average scores. For Code generation (Table 2 Next-ToBE has an average accuracy of 55.1% on, being 3.9% and 2.2% higher than MTP and MuToR, respectively, on Qwen2.5-1.55 1.55B. When scaling to Qwen2.5-7B. its accuracy rises 75.2%, outperforming MuTTOR by 2.7% and NTP by 3.6%, stable gain model size grows. Even for an average 67.3% exceeding MuToR by 1.9% and NTP by 2.8%. For Commonsense reasoning (Table 2). Next-ToBE achieves average accuracy of 79.5% Qwen2.5-1.5B. and rises to 89.4% on Qwen2.5-7B, which 1.3% 3.3% higher than competing methods. On Llama3. -B-iisttrutt, Next-ToBE has averaged accuracy of 87.8%, about 1.2%-2.1% higher than other baselines. In addition, we extend our evaluation of Next-ToBE. to the larger Qwen3-14B model (Yang et al. ataal.2202 (025). As shown Appendix E.6 Next-ToBE consistently achieves the highest average accuracy across five mathematical benchmarks. Notably, it delivers absolute gains of 3.4% over MTP and 1.6% over MuToR, exceeding improvements observed on smaller Owen2.5-Math-77 models. This indicates that NeXt-ToBE exhibits favorable scalability as model size increases.
Next-ToBE delivers
Prefraining setting. Beyond fine-tuning, we also examined whether Next-ToBE remains effective in pretraining scenario, i.c., without any inherent anticipatory capacity when training from scratch. To this end, we trained a GPT-2 model (124M) (Radford al.) 2019) on WikiText- 103 (Merity) et al. 2017) using three different objectives: NTP. MTP, and Next-ToBE Detailed training strate- gies and evaluation results are provided Appendix E.7 As shown in Table Next-ToBE improves the Future-token Hit Ratio by 3.29% over NTP and 2.42% over MTP, indicating that anticipatory capacity can indeed be developed from zero. On the downstream HellaSwag QA benchmark, Next- ToBE further achieves absolute gains of +0.86% over NTP and +1.29% over MTP. These results demonstrate that, even in a pure pretraining scenario, Next-ToBE can successfully cultivate antici- patory behaviour from scratch and translate it into improved downstream reasoning performance.
8

---

Published as a conference paper at ICLR 2026
Table Performance comparison on Code Generation and Commonsense Reasoning tasks.
Code Generation
Commonsense Reasoning
Model
Method
HumanEval
MBPP
Avg.
PIQA
Hellsawage
ARC-C
ARC-E
SIQA
Avg.
NTP
50.0
55.2
52.6
74.4
74.7
77.7
Qwen2.5
MTP
48.9
53.4
51.2
78.8
74.3
718
87.6
68.4
76.2
(1.58)
MuToR
51.2
54.5
52.9
80.2
78.7
74.3
87.5
70.2
78.2
Nexi-ToBE
53.7
56.4
$5.1
80.7
79.3
77.1
89,9
70.4
79.5
NTP
74.4
68.9
71.6
88.7
90.4
88.3
95.6
77.7
88.1
Qwen2.5
MTP
76.2
69.3
72.8
87.4
77.6
87.8
(7B)
MuToR
75.6
69.3
72.5
88.9
91.3
88.2
95.8
88.4
NexI-ToBE
78.7
71.6
75.2
88.6
92.7
89.7
96.3
79.8
89,4
NTP
65.2
63.8
64.5
86.9
87.8
83.4
93.5
76.7
85.7
Llama3. 1 (8B-instruct)
MTP
67.1
62.3
64.7
88.2
90.1
84.8
93.4
76.7
86.6
MuToR
67.7
63.0
65.4
84.4
93.6
74.8
86.2
NeXI-ToBE
69.5
65.0
67.3
88.5
93.0
84.6
94.0
78.8
87.8
4.2 HYPERPARAMETER, SENSITIVITY
We conduct a series of ablation studies to cxamine the impact of window size and weighting-related hyperparameters to better understand the robustness and design choices of Next-ToBE.
First, we investigate how the token-bag bag size k affects per- formance (Appendix [.4). On code generation, Qwen2.5- 1.5B and Qwen2.5-7B achieve highest accuracy with k = 10, while Llama3. 1-8B-instruct peaks at This aligns with expectations: too small windows may fail to provide suf- ficient future context, whereas overly large windows may introduce noise and confuse the model.
Table 3: Ablation of key hyperparam-
eters on Qwen2.5-Math-1.. over mathematical benchmarks (averaged).
c(a)
Avg.
h(3)
Avg.
lc-4
36.6
37.4
We further conduct ablation studies on the hyperparameters related to the weighting scheme, including the threshold (for inherent anticipatory preference a) and the bandwidth h (for semantic/temporal importance 3), using Qwen2.5- Math-1.5B on five mathematical reasoning benchmarks.
le-6
37.1
2
38.3
lc-8
38.3
3
38.0
le-10
37.6
4
37.3
le-12
36.4
37.5
The average accuracy results under different settings summarized in Table 3, Overall, the ablation results demonstrate that the weighting-related hyper- parameters Next-ToBE exhibit stable behavior across broad range of values. Both the threshold and the bandwidth h show smooth performance curves with clear optima, rather than erratic fluc- tuations. This indicates that Next ToBE not overly sensitive precise hyperparameter settings.
4.3 TRADE OFF BETWEEN CONFIDENCE AND REASONING ABILITY
We examine the impact of the weight À in Eq. which balances the relative importance of fit- ting the immediate next token versus the look-ahead tokens. This trade-off plays critical role in reasoning performance, and also reshapes the confidence dynamics of LLM during generation:
(1) Confidence dynamics. In Figure 4a we evaluate the LLM's confidence by measuring its proba- bility of predicting the next token each generation step, averaged across the sequence. As shown, increasing X consistently reduces the model's next-token confidence. This happens because the Next-ToBE objective encourages the model to produce smoother predictions each step by fitting beyond the immediate next token. A similar trend is observed in Figure 3c where the distribution of the top-token confidence shifts toward a lower mean after Next-ToBE. fine-tuning-fe exam- plc, from 0.87 to 0.81 in mathematical tasks-compann compared to the base model. We further confirm that Next- ToBE does not introduce distribution drift: KL-divergence analysis over 1,024 step autoregres- sive generations shows that its token-by-toker predictive distribution remains closely aligned with that stronger reference model throughout long term generation. (Detailed Appendix E.8)
(2) Reasoning performance. How does next-token confidence relate LLM reasoning quality? As shown in Figure/4b] for code generation and math reasoning tasks with the Qwen1.51 series model, L.M performance improves with increasing À up point, after which it declines. This indicates

---

Published as a conference paper at ICLR 2026
044
Noxt Takan Contidenos
(6
NTV
Accuracy
Acourcay
Code
Code
Code
Math
Math
Math
022
1.09
5:0s ci abaide 01 02
04
6001
Edes aht
abaass
01
02
072
0.6
R24
Trade off parameter A
Trade-af oll parameter
Next Token Confidence
(a) Averaged next-token confidence
(b) Reasoning performance versus
(c) Reasoning performance vs (av- craged) next-token confidence.
(probability) vs x in Eq. 4..
À in Eq.
Figure 4: LLM confidence dynamics and reasoning performance versus trade-off parameter A.
that optimal trade-off exists between fitting the immediate next token and the more distant, look- ahead tokens. Figure 4 plots reasoning accuracy vs next-token confidence: for math, performance peaks around a confidence of 0.755, while in code generation, the optimal confidence is 0.786.
comprehensive study of LLM confidence dynamics and behavior is a broad research direction, which is related to over-confidence traces regularization (Szegedy et al. 2016), low-confidence traces filtering (Fu et al. 2025). and more others, which we leave for future investigation.
4.4 COMPUTATIONAL OVERHEAD
We evaluate runtime and peak memory usage for NTP, MTP, MuToR, and Next-ToBE on the 35K NuminaMath corpus using Qwen2.5. Math-1.5B under identical training settings.
Table 4: Runtime and memory consumption on mathematical training using Qwen2.5-Math-I..3
Model
Training Time
Peak Memory Consumption
As shown in Table Next-ToBE incurs only modest overhead relative to standard NTP: +26 minutes of training time (~20%) and
NTP
2h 04min
9,412 MB
MTP
2h 56min
16,559 MB
Mutor
2h 39min
39,486 MB
Next-ToBE
2h 30min
12,552 MB
+3.IGB of memory (~32%). Importantly, it remains far more efficient than multi-token prediction baselines, reducing peak memory by up to 68% and training time by up to 15% compared to representative MTP methods. Overall, Next-ToBE delivers substantial performance gains with significantly lower computational cost while retaining the simplicity of single-head head NTP architecture.
Besides, the random-wale walk weighting module introduces almost no computational burden. Comput- ing its stationary distribution only requires inverting small 10 10 matrix per step (from window size h = 10), taking roughly 10-4 seconds in PyTorch. Empirically, this component accounts for approximately about 6.2 minutes of training time (4. 1 % of 2.5 hour run on Qwen2.55Math-LL5B)) and its extra memory usage is negligible (34.6 MB out of 12.6 GB peak).
5 CONCLUSION
We presented Next-ToBE, a probabilistic training framework to profile, activate, and exploit the anticipatory capacity of LLMs. Next-ToBE introduces a principled trade-off between fitting the immediate next token and anticipating longer-range future tokens not via auxiliary prediction heads or architectural changes, but by directly leveraging the inherent probabilistic predictions of LLMs, which we found to be strongly correlated with generative quality. Across wide range of fine-tuning benchmarks, Next-ToBF consistently outperforms strong multiple-token prediction baselines, with substantial gains in reasoning performance and computational efficiency. Furthermore, anticipatory behavior also shown to emerge from scratch during Next-ToBE pretraining, highlighting its role as general and scalable training principle. Finally, we uncover compelling connection between the probability profiles induced by Next-ToBE and the confidence dynamics of LLMs, offering a new perspective for interpreting and controlling LLM reasoning behavior. We view Next-ToBE as a promising direction for rethinking pretraining across diverse sequential data, including code, time series, and multimodal modeling, where long-horizon anticipation essential.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This work was conducted in accordance with the ethical standards and code of conduct of ICLR and computer science research community. All experiments were carried out using publicly available datasets, and no personally identifiable or sensitive information was used. We have ensured that our methods, analyses, and results are reported transparently, without fabrication, falsification, or inappropriate manipulation. The authors adhered to principles of academic integrity, fairness, and respect all aspects of the research and manuscript preparation.
REPRODUCIBILITY
We have made every effort to ensure the reproducibility of our work. All code, scripts, and rel- evant resources used this study have been submitted alongside the paper. Detailed instructions regarding model configurations, training procedures, and evaluation protocols are provided in the supplementary materials to facilitate transparent replication of our results.
ACKNOWLEDGEMENTS
We are very grateful to the reviewers for their valuable comments and constructive feedback. This work is supported by the national natural science foundation of China (Grant No. 62276099).
REFERENCES
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, al. Phi-4 techni- cal report. arXiv preprint 2:Xix:2412.08900 2024.
Janice Ahn, Rishu Verma, Renze Lou, Di Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint 2:X1v:2402.00157, 2024.
Ibrahim M. Alabdulmohsin, Vinh Q. Tran, and Mostafa Dehghani. Fractal patterns may illuminate the success of next-token prediction. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10- 15, 2024, 2024.
Zachary Ankner, Rishab Parthasarathy, Aniruddha Nrusimha, Christopher Rinard, Jonathan Ragan- Kelley, and William Brandon. Hydra: Sequentially-deeende draft heads for medusa decoding. arXiv preprint 40X 2024.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2 2021.
Gregor Bachmann and Vaishnavh Nagarajan. The pitfalls of next-token prediction. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Con- feerence Track Proceedings, 2015.
Neil Band, Xuechen Li, Tengyu Ma, and Tatsunori Hashimoto. Linguistic calibration of long-form generations. In Forty-first International Conference on Machine Learning. 2024.
Mathias Barthel, Sebastian Sauppe, Stephen C Levinson, and Antje Meyer. The timing of utterance planning in task-oriented dialogue: Evidence from a novel list-completion paradigm. Frontiers in psychology, 7:1858, 2016.
11

---

Published as a conference paper at ICLR 2026
Mohammad Bavarian, Heewoo Jun, Nikolas Tezak, John Schulman, Christine McLeavey, Jerry Tworek, and Mark Chen. Efficient training of language models to fill in the middle. arXiv preprint wwxhv:2207.142555 2022.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Ka- mar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303. 12712, 2023.
Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. Medusa: Simple I1m inference acceleration framework with multiple decoding heads. In Proceedings of the 41st International Conference on Machine Learning, pp. 5209-5233 2024.
Iván Vicente Moreno Cencerrado, Arnau Padrés Masdemont, Anton Gonzalvez Hawthorne, David Demitri Africa, and Lorenzo Pacchiardi. No answer needed: Predicting Ilm answer ac- curacy from question-only linear probes. arXiv preprint 2509v 2025.
Jingyang Chen, Ping Li, Jiancheng Lv, Hongyuan Zha, Kai Zhang, and Jie Zhang. Learning tem- poral features with alternated similarity and proximity attention for time-series prediction. IEEE Transactions on Neural Networks and Leurning Systems, 2025a.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, al. Evaluating large language models trained on code. arXiv preprint (ttt::1007.03374/ 2021.
Michael K Chen, Xikun Zhang, Jiaxing Huang, and Dacheng Tao. Improving large language models with concept-aware fine-tuning. arXiv preprint drXiv:2506.07833, 2025b.
Yunfei Cheng, Aonan Zhang, Xuanyu Zhang, Chong Wang, and Yi Wang. Recurrent drafter for fast speculative decoding large language models. arXiv preprint arXiv:2403.090 209909999 2024.
Anna Dawid and Yann LeCun. Introduction to latent variable energy-based models: a path toward autonomous machine intelligence. Journal of Statistical Mechanics: Theory and Experiment, 2024(10):00411, 2024.
Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc' Aurelio Ranzato. Residual energy-based models for text generation. International Conference on Learning Represen- tations, 2020.
Zhichen Dong, Zhanhui Zhou, Zhixuan Liu, Chao Yang, and Chaochao Lu. Emergent response planning in 1lms. arXiv preprint arXiv:2502.06255, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Nouha Dziri, Ximing Lu, Melanie Selar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, et al. Faith and fate: Limits of trans- formers on compositionality. Advances in Neural Information Processing Systems, 36:70293- 70332, 2023.
Yichao Fu, Xuewei Wang. Yuandong Tian, and Jiawei Zhao. Deep think with confidence. arXiv preprint arXiv:2508. 15260, 2025.
Anastasios Gerontopoulos, Spyros Gidaris, and Nikos Komodakis. Multi-token prediction needs registers. arXiv preprint 2rXi::2505.10518. 2025.
Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozière, David Lopez-Paz, and Gabriel Synnaeve. Better & faster large language models via multi-token prediction. In Proceedings of the 41st International Conference on Machine Learning, pp. 15706-15734, 2024.
Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
12

---

Published as a conference paper at ICLR 2026
Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code gen- eration. arXiv preprint arXiv:2506.20633 2025.
Jiatao Gu, James Bradbury, Caiming Xiong, Victor O. K. Li, and Richard Socher. Non- autoregressive neural machine translation. In 6th International Conference on Learning Rep- resentations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Sangchul Hahn and Heeyoul Choi. Self-knowledge distillation in natural language processing. In Ruslan Mitkov and Galia Angelova (eds.), Proceedings of the International Conference on Re- cent Advances in Natural Language Processing (RANLP 2019), pp. 423-430, Varna, Bulgaria, September 2019. INCOMA Ltd.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2:3, 2022.
Edward S. Hu, Kwangjun Ahn, Qinghua Liu, Haoran Xu, Manan Tomar, Ada Langford, Dinesh Jayaraman, Alex Lamb, and John Langford. The belief state transformer. The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025.
Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Lee. Lim-adapters: An adapter family for parameter-efficiet fine-tuning of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 5254-52555 5276, 2023.
Chaeyun Jang, Moonseok Choi, Yegon Kim, Hyungi Lee, and Juho Lee. Verbalized confidence trig- gers selffverification: Emergent behavior without explicit reasoning supervision. arXiv preprint arXiv:2506.03723. 2025.
Juyong Jiang. Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint arXiv:2406.0051/ 2024.
Fangkai Jiao, Geyang Guo, Xingxing Zhang, Nancy F Chen, Shafiq Joty, and Furu Wei. Preference optimization for reasoning with pseudo feedback. arXiv preprint arXiv:2411.16345 2024.
Zhewei Kang, Xuandong Zhao, and Dawn Song. Scalable best-of-n selection for large language models via self-certainty. In 2nd AI for Math Workshop @ ICML 2025, 2025.
Yann LeCun. Do large language models need sensory grounding for meaning and understanding. In Workshop on Philosophy of Deep Learning, NYÚ Center for Mind, Brain, and Consciousness and the Columbia Center for Science and Society, 2023.
Boyi Li, Philipp Wu, Pieter Abbeel, and Jitendra Malik. Interactive task planning with language models. In 2nd Workshop on Language and Robot Learning: Language as Grounding. 2023.
Haoran Li and Wei Lu. Mixed cross entropy loss for neural machine translation. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning PMLR, 2021.
Research, pp.
Ziniu Li. Congliang Chen, Tian Xu, Zeyu Qin, Jiancong Xiao, Zhi-Quan Luo, and Ruoyu Sun. Preserving diversity supervised fine-tuning large language models. In The Thirteenth Inter- national Conference on Learning Representations, 2025.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024a.
13

---

Published as a conference paper at ICLR 2026
Shih- Yang Liu, Chien- Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu- Yu-Chiang Frank Wang, Kwang- Ting Cheng, and Min-Hung Chen. Dora: bright-decomposed low-rank adaptation. In Forty-first International Conference Machine Learning, 2024b.
Xiaohao Liu, Xiaobo Xia, Weixiang Zhao, Manyi Zhang, Xianzhi Yu, Xiu Su, Shuo Yang, See- Kiong Ng, and Tat-Seng Chua. L-mtp: Leap multi-token prediction beyond adjacent context for large language models. arXiv preprint arXiv:2505. 17505, 2025.
Junyi Lu, Lei Yu, Xiaojia Li, Li Yang, and Chun Zuo. Llama-reviewer: Advancing code review automation with large language models through parameter-efficien fine-tuning. 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE), pp. 647-658. IEEE, 2023.
Eran Malach. Auto-regressive next-token predictors are universal learners. arXiv:2309.0699 06979, 2023.
arXiv preprint
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. PEFT: State-of-the-art parameter-efficien fine-tuning methods.
thub.
com/ htmm///uggingfac//ettt 2022.
Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13:: 13338, 2023.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL httpst::: //openzeview net /ete/foo wwr.mooommmiiimmmoo ( 72udxe
William Merrill and Ashish Sabharwal. The expressive power of transformers with chain of thought. In The Twelfth International Conference on Learning Representations, 2024.
Rafael Müller, Simon Kornblith, and Geoffrey E. Hinton. When does label smoothing help? In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' A'Achhé-Bus Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Con- ference on Neural Information P Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 4696-4705, 2019.
Vaishnavh Nagarajan, Chen Henry Wu, Charles Ding, and Aditi Raghunathan. Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction. arXiv preprint arXiv:2504. 15266, 2025.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint arXiv:2502.09992, 2025.
Koyena Pal, Jiuding Sun, Andrew Yuan, Byron C Wallace, and David Bau. Future lens: Anticipating subsequent tokens from single hidden state. arXiv preprint arXiv:2311.04897, 2023.
Jia-Yu Pan, HyungJeong Yang, Pinar Duygulu, and Christos Faloutsos. Automatic multimedia cross-modal correlation discovery. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004.
Yudi Pawitan and Chris Holmes. Confidence in the reasoning of large language models. arXiv preprint ar arXiv:2412.15296 2024.
Benjamin Plaut, Khanh Nguyen, and Tu Trinh. Softmax probabilities (mostly) predict large language model correctness multiple-choice q&a. CoRR, 2024.
Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, and Ming Zhou Prophetnet: Predicting future n-gram for sequence-to te--ueee----eeeneeeeeee pre-training. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Lin- guistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Findings of ACL. pp. 2401-2410. Association for Computational Linguistics, 2020. 2020.
14

---

Published as a conference paper at ICLR 2026
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Mohammad Samragh, Arny Kundu, David Harrison, Kumari Nishu, Devang Naik, Minsik Cho, and Mehrdad Farajtabar. Your Ilm knows the future: Uncovering its multi-token prediction potential. arXiv preprini 1rXiv:5577.11831 2025.
Florian Schmidt. Generalization in generation: A closer look at exposure bias. In Alexandra Birch, Andrew Finch, Hiroaki Hayashi, Ioannis Konstas, Thang Luong, Graham Neubig, Yusuke Oda, and Katsuhito Sudoh (eds.), Proceedings of the 3rd Workshop on Neural Generation and Trans- lation. Association for Computational Linguistics, November 2019.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826, 2016.
Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. Mathscale: Scaling instruction tuning for mathematical reasoning. arXiv preprint arXiv:2403.00 0.8880 2024.
Abitha Thankaraj, Yiding Jiang. J Zico Kolter, and Yonatan Bisk. Looking beyond the next token. arXiv preprint arXiv:2504. 11336, 2025.
Hanghang Tong, Christos Faloutsos, and Jia- Yu Pan. Random walk with restart: fast solutions and applications. Knowledge and Information Systems, 14(3):327-345 24662677766 2008.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023a.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct: Aligning language models with self self-generated instructions. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 13484-13555 2023b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoe V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho "edds),.dvannee in Neural Information Processing Systems, 2022.
Noam Wies, Yoav Levine, and Amnon Shashua. Sub-task decomposition enables learning in se- quence to sequence tasks. In The Eleventh International Conference Learning Representattinss ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1(2):270-280, 1989.
Wilson Wu, John X Morris, and Lionel Levine. Do language models plan ahead for future tokens? arXiv preprint arXiv:2404.008:: 2024.
Yisheng Xiao, Lijun Wu, Junliang Guo, Juntao Li, Min Zhang, Tao Qin, and Tie-yan Liu. A survey on non-autoregressive generation for neural machine translation and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10): 1407-11427, 2023.
Minkai Xu, Tomas Geffner, Karsten Kreis, Weili Nie, Yilun Xu, Jure Leskovec, Stefano Ermon, and Arash Vahdat. Energy-based diffusion language models for text generation. In The Thirteenth International Conference Learning Representations, 2025.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-im relf-improvement. arXiv preprint arXiv:2409.12130 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint (00:2555500988.. 2025.
15

---

Published as a conference paper at ICLR 2026
Xinran Zhao, Hongming Zhang, Xiaoman Pan, Wenlin Yao, Dong Yu, Tongshuang Wu, and Jian- shu Chen. In Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
(FaR) improves confidence calibration of large language models.
Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards. arXiv preprint arXiv:2505. 19590, 2025.
Zayd MK Zuhri, Erland Hilman Fuadi, and Alham Fikri Aji. Predicting the order of upcoming tokens improves language modeling. arXiv preprint arXiv:2508. 79228, 2025.
16

---

Published as a conference paper at ICLR 2026
A USAGE OF LLM
In preparing this manuscript, the authors made only minimal use of a large language model (Chat- GPT). Its role was limited to refining grammar, finding typos and mis-spelled words improve readability. The study design, experimental procedures, data analysis, interpretation of results, and development of scientific ideas were carried out entirely by the authors without any reliance on the LLM. All substantive intellectual contributions remain solely the work of the authors, who carefully reviewed and verified the final text.
B MORE RELATED WORK
We synthesize the disparate arguments related to the next-token prediction paradigm from various research fields to clearly elucidate the nature and contributions of our method, ToB...
The auto-regressive next-token prediction (NTP) paradigm, which underpins modern language mod- els (Bahdanau et al., 2015 Merrill & Sabharwal 2024), remains a subject of ongoing debate. Critics argue that forchng training induces exposure bias-a discrepancy between training and test- ing conditions wherein the model is exposed to perfect, ground-truth contexts during training but rely on its own potentially flawed generations during inference (Schmidt 2219)] Bachmann & Nagarajan 2024). This bias may lead to the accumulation and propagation of errors, resulting in substantial de- viations from correct solutions (Dziri et al. [023 LeCun 2023), thereby compromising the model's
capacity for robust, long-range planning (Bubeck et al. 2023 McCoy al. 2023).
Conversely, proponents highlight NTP's powerful learnability (Malach 2023). which can transform intractable problems into step-by-step tractable tasks through methods like Chain-of-Thouueee
pervision (Wies et al., (023). This success in structured reasoning is further supported by deeper, implicit planning ability (Alabdulmohsin al. 2024), which has also been demonstrated in recent empirical studies. Probing the internal mechanisms, these studies show that LLMs acquire global information regarding the final response prior to generation (Wu et al., 2024; Dong al., [025] Cencerrado et al.) 2025). Collectively, these findings indicate that the NTP framework endows LLMs a robust and non-trivial planning capability. This perspective provides the central inspiration for our approach.
To enhance the implicit planning capabilities of NTP models, research has pursued several distinct strategies. A primary approach entails modifying the training data or model architecture to extend the model's predictive horizon. This includes data augmentation techniques that train the model to "fill in the middle" (Bavarian et al.) 2022). the incorporation of special tokens to explicitly predict future target tokens (Thankaraj et al. 2025 Gerontopoulos et al.] (2025), and architectural modifica- tions that enable bidirectional encoding and prediction (Hu et al. 2025). Although these methods
have demonstrated effectiveness, they may also introduce considerable complexity into the data
processing and training pipelines.
A second line of research seeks lighter-weight solutions by modifying the training objective itself. For instance, Zuhri et al. (2025) introduced auxiliary loss predict the rank of upcoming target tokens without altering the model's 's architecture. The method we propose advances this lightweight approach. Unlike the aforemennioned methods, it requires neither data augmentation nor architec- tural modifications. Instead, it directly exploits the model's inherent anticipatory signals by manip- ulating its self eeefipredittdd outputs to construct novel loss target.
Conceptually, our proposed soft "token-bag" target is grounded in a well-established technique: label smoothing (Szegedy et al., 2016). However, instead of smoothing with a uniform distribution over the vocabulary, our method provides model-informed target that regularizes overconfident predictions- a property shown to improve the generation of diverse and plausible results early machine translation tasks (Müller et al. 2019 Li & Lui 2021) and enhance the generalizability and diversity of reasoning results (Li et al. 2025) in the era of LLMs.
Additionally, the "inherent anticipatory signals" manifest as the model's predictive confidences, en- coded in the teven-level probability distributions that form a reasoning path (Wei et al. 2022; Wang
et al. 2023a| Pawitan & Holmes 2024). The quality of these confidence scores critical.
they
have been shown to strongly correlate with the final answer's correctness (Band et al. 2024 Plaut
et al. 2024 Zhao al. 2025 Jang al. 2025). The importance of these signals is highlighted by
17

---

Published as a conference paper at ICLR 2026
the effectiveness of various inference- time methods that leverage them, such modifying prompts to elicit better outputs (Zhao et al. (024). using decoding strategies like nucleus sampling to filter low-confidence tokens (Holtzman et al. 2020). aggregating multiple outputs via majority voting (Kang et al. 2025 Fu et al.) 2025). Instead of manipulating these signals post-hoe, our training- time method aims directly improve the quality and calibration of these anticipatory signals the source to foster more intrinsically robust and reliable reasoning.
Aside from previously introduced advanced NTP learning methods, Multi-Token Prediction (MTP) also enhances the implicit planning capabilities of LLMs (Qi et al., 2020 Gloeckle et al. 2024]
Liu et al., 2024a] Chen et al. 2025b; [iiu al. 2025 Samragh al., 2025). By using auxiliary
heads to predict multiple future tokens simultaneously. language models receive richer relationships between tokens during training. Furthermore, advanced inference techniques like speculative de- coding reduce generation steps (Cai et al., 2024) Ankner et al. 20244 Cheng et al. 2024), which can mitigate error accumulation and foster creativity (Nagarajan et al. 2025). Although MTP and our method both learn relationship between multiple tokens during training, they ultimately gener- ate single token independently during inference, which limits LLMs ability to capture the global joint distribution of entire sequence. This makes exposure bias and the associated problem of cumulative error remain unavoidable.
To overcome the limitations of token-level sequence modeling, researchers have explored paradigms that operate at the sentence level. Non-auto-regrresivve (NAR) models generate all tokens in parallel, directly capturing the full sentence distribution (Gu et al. 2018 Xiao et al.) (023). Diffusion models refine complete sequence through progressive denoising process (Gong et al 2023 Nie et al., 2025). Both NAR methods ensure consistency between training and testing behaviors, they can generate coherent long texts and even demonstrate potential in reasoning tasks (Gong el al.) 2025) Inversely, reliance on pre-specified output length limits their flexibility that makes auto-regressive
NTP models so versatile.
Energy-Based Models (EBMs) provide a unifying framework for text generation by introducing global energy function that evaluates entire sequences, thereby enforcing global coherence (Dawid) LeCun 2024). For auto-regressive models, Residual EBMs (Deng et al.] 2020) superimpose sentence-level energy scores-either by log-likelihood summation or by training additional model to compute them-on top of pre-trained models' token-level distributions, mitigating exposure bias and enhancing long-range coherence. Energy-based Diffusion Language Models (Xu et al., 2025) integrate energy functions into each denoising step, often guided by pre-trained auto-regressive models, to correct independence assumptions and improve global consistency. This line of work highlights EBMs as a promising bridge across generative paradigms, effectively combining the lo- cal fluency of token-by-toker generation with long sentence-level coherence. Consequently, this advancement shows potential in extending the boundaries of reasoning abilities in language models.
C ANTICIPATORY CAPACITY ANALYSIS
C.I FUTURE-TOKEN HIT RATES FOR DIFFERENT TOP-m VALUES AND FUTURE TOKEN-BAC WINDOW SIZES
6:0
6.M
Future toleen Hit Rate
token HIt
Future token HIt
05
45
0.4
0.
5
Future taken bag (window) 520 à
Puture taken bag (Widow) 520
Future taken bag (window) 520
(a) m 75
(b) m 100
(c) m 150
Figure 5: Average Future-tokens Hit Rate (FtHR) under different top-m settings.
18

---

Published as a conference paper at ICLR 2026
To further examine the anticipatory capacity of LLMs, report the Future-tokens Hit Rate (FtHR) across different values of (the number of top-ranked tokens considered). Recall that FtHR measures how often the ground-truth tokens the next steps fall within the top-m predictions of the model at the current step. For this analysis, randomly sample 100 mathematical reasoning problems from the NuminaMath dataset (Tang et al. 2024).
Figure results for three representative values of m (75, 100, and 150). Two consistent trends emerge across all settings: first, the hit rate decreases as the window size it grows, indicating that models more effective anticipating near-future tokens; second, larger m values yield higher hit rates, showing that anticipatory signals are spread over a broader set candidate tokens.
C.2 DETAILS OF ACTUAL PROBABILITY OF GENERATING WITH RANK IN CURRENT PREDICTION
We further investigate the connection between anticipatory capacity and token-level prediction ac- curacy. To obtain these results, we consider two complementary evaluation modes:
Teacher forcing(TF): At each step f. the model is conditioned on the ground truth prefix up to position
1. The probability of target token Tz+k is obtained as Po (==+**/<<+++++ We then pair this ground-truth probability with the rank of Te+k in the prediction distribution at step t, f.forming a sample for analysis. Since all probabilities are computed under the true prefix, this mode represents idealized setting.
auto-regressive mode: The model conditions on the ground-truth prefix up to step I, then auto- regressively generates k intermediate tokens before predicting Tz+4-- The probability of the target token is measured as Pø(I++* Igg, 2141:0+k-1), where 814144k-1 are the model's 's self- generated tokens. This setup reflects more realistic generation scenario, where prediction accuracy depends not only on the true history but also on the model's own generation quality.
of geneattig
Aotus of generaing
Actus Proh
04
22
10
10
10
10
10
101
30
10
10
12
10
21 cureet prediction 71
curvent prddction PMDS
nument prwdicion ROlF
(a) Qwen- Owen-Math-I Math-1.5B
(Teacher
(b) Llama3.1 1-8.----Bt---cc (Teacher
(c) Llama3.1 1-8B B- 1-8----isttuttt (auto- regressive mode)
Foreing)
Forcing)
Figure 6: Relations between token level probability and rank across different models and modes
After obtaining the rank of token Fi+& in the probability distribution at step and its actual gener- ating probability P partition the data into numerous logarithmically spaced bins (i.c. 300) based on rank. The mean probability within each bin then computed to reveal a coarse-grained trend. To further smooth the curve and highlight the overall pattern, we apply mouussian-weightee averaging to compress these bins into a smaller set of representative points (i.e. 15). Fig. [b] [b] presents the relationship between generating token probability and rank in the auto-regressive setting (Qwen- Math-1.5B). We next present results across different models and modes, as shown in Fig.
D TRAINING DETAILS
D.1 HYPERPARAMETER CONFIGURATIONS
We present the hyperparameter configurations used for Next-ToBe and baseline training methods across all the aforementioned tasks and three pre-trained language models. These detailed settings are provided to facilitate fair comparison and reproducibility. Unless otherwise stated, all experi- ments are trained for epochs with global batch size of 128 across all competing methods. The
19

---

Published as a conference paper at ICLR 2026
maximum sequence length is set 2048 for mathematical reasoning tasks and 512 for code gener- ation and commonsense reasoning tasks.
adopt the following hyperparameter settings: threshold 10 and power scaling factor y = 0.1 (Eq. )5; restart probability = 0.3 (Eq. (TT); and a multiplicative fusion of temporal and semantic relations to construct the RWR transition matrix; the bandwidth in Eq. (8) 2. addition, other key parameters such as the future token bag window size and the loss
For Next-ToBE,
coefficient À summarized in Table
Table 5: Best hyperparameter configurations for Next-ToBe on each reasoning task
Tisk
Mathematical Reasoning
Code Generation
Commonsense Reasoning
Qwen2.5
Qwen2.5
LJama3.1
Qwen2.5
Qwen2.5
Llama3.1
Qwen2.5
Qwen2.5
Llama3.1
Model
Math-1.5B
Muth-7B
8B-instrucl
1.5B
7B
8B-instruct
1.5B
7B
88-instruct
3e-5
5e-5
3e-5
5e-5
le-4
le-4
le-4
3e-4
3e-4
10
10
9
0.05
0.04
0.06
0.05
0.03
0.05
0.05
0.04
0.03
For NTP, in mathematical reasoning, set the learning rates to 3c-5, 5e-5 and 3e-5 for Qwen- Math-1.5B, Owen-Math-77B, and ---MAA-3.--SB-Inste learning rates for Qwen-1.5B, Qwen2.5-7B, and -.LMMA-..--B--nmstt 3e-4. For commonsense reasoning, the learning rates are set to le-4, 3e-4, and 3e-4.
respectively. For code generation, the
are set to 5e-5, le-4, and
For MTP, additionally introduce four auxiliary heads, each corresponding position, and attach them at the start with random initialization. To alleviate the instability caused by these randomly initialized heads, we freeze all other parameters and pre-train only the auxiliary heads for one epoch before commencing joint training. The learning rate are set to 1c-5, 5e-5 and 5e-5 for mathematical reasoning; 5e-5, le-4 and Ic-4 for code generation, and Ic-4, 3e-4 and 3e-4 for commonsense reasoning, corresponding to the Qwen- 1.5B series, Qwen-7B series, and LLaMA- 3.1-8B-Instruc models, respectively.
specific token
For Mutor, the learning rate are set to 3e-5, 5e-5 and 5e-5 for mathematical reasoning; le-5, le-4 and le-4 for code generation, and le-4, 3e-4 and 3e-4 for commonsense reasoning, corresponding to the Qwen-1.5B series, Qwen-7B series, and LLaMA-3.1 1--1--88--nsstee models, respectively.
D.2 PROMPT TEMPLATES
We provide all prompt templates used for training and evaluation.
Mathematical Reasoning Training and Evaluation Template
<| im_starr/ /oaett>syye >system
Please reason step by step, and put your final answer within \boxed(). 
>
(instruction)<lll in__nddd) <|im_start >assistant
20

---

Published as a conference paper at ICLR 2026
Zero-shot prompt template for Humaneval and commonsense reasoning evaluation
Below is instruction that describes task, paired with input that provices further context. Write response that appropriately completes the request.
#Hinstruction: (instruction)
##Response:
Prompt template for code generation and commonsense reasoning training
Below is an instruction that describes task, paired with an input that provices further context. Write response that appropriately completes the request.
##Inssruction: (instruction)
##VInput:
(input)
##Response:
Three-shot prompt template for MBPP evaluation
Below is instruction that describes task, paired with input that provices further context. Write response that appropriately completes the request.
##lInstruction
(instruction)
##IInput:
(testcase [0]
(testcase [1]]
(testcase [2]]
##Responsc:
E MORE EXPERIMENTS
E.
IMPACTS OF NORMALIZE TOKEN BAG PROBABILITY
Table 6 summarizes the effect of normalization on token bag distribution in Next-ToBE (see Eq. (44) during training on Qwen-Math-1.57 for mathematical reasoning. We observe that re- moving normalization causes significant drop in overall performance, even below that of the NTP baseline for Olympiad Bench and AIME24 tasks. This degradation arises because, without normal- ization, small-probablle a result, the model overemphasizes distant future tokens while underweighting immediate next token, ultimately destabilizing training and leading to collapse.
tokens in the token bag can produce disproportionately large losses. As
Table 6: Comparison of the Next- Next-ToBE objective with and without normalization across multiple mathematical reasoning benchmarks.
Method
MATH
MinervaMath
Olympiad Bench
AIME24
AMC23
Avg.
NTP
64.8
19.1
30.7
10.0
45.0
33,9
Next- ToBE w/o normalize
63.6
20.7
25.9
6.7
45.0
32.4
Next- ToBB
68.2
23.1
31.9
13.3
55.0
38.3
21

---

Published as a conference paper at ICLR 2026
E.2 DIFFERENT WEIGHTS OF TOKEN BAG INFORMATION
For better analyzing the joint temporal and semantic information, we visualize 30 curves of ranking distributions of with Qwen-Math-1.5B in mathematical tasks, considering multiplication fusion and weighted summation fusion.
0.14
Temproal semanic weight Ax.
Temproal semantic weight 8x.
0.09
0.12
0.08
0.10
0.08
0.06
0.06
0.05
8
10
10
Future token bag (window) size k
Future taken bag (window) size
(a) Multiplication Fusion
(b) Weighted Summation Fusion
Figure 7: Examples of the convergent random-walk distribution scores 3 plotted against the tem-
poral order of the tokens.
As shown in Fig. both the multiplication and weighted summation fusion methods exhibit over- all decreasing trend as the window size increases. The fluctuations observed, however, indicate that some more distant look-ahead tokens maintain significant semantic correlations with the immediate next token, resulting in higher weights despite their greater temporal distance from the current step. Further, multiplicative fusion (of the temporal proximity matrix W, and the semantic similarity matrix W.) leads to generally smoother weights than compared with weighted summation fusion, which we speculate one of the reasons why the former gives better results.
E.3 DISCUSSION ABOUT DIFFERENT APPROACUESS OF TEMPORAL/SEMANTIC IMPORTANCE FUTURE TOKENS
Table 7: Comparison between baseline training methods and Next-ToBE variants with different semantic-temporal
semantic temporal joint weighting strategies on five mathematical reasoning benchmarks.
Method
MATH
Minerva Math
Olympiad Bench
AIME24
AMC23
Avg.
Baselines
NTP
64.8
19.1
30.7
10.0
45.0
33.9
MTP
66.0
20.6
29.0
10.0
47.5
34.6
MuToR
67.6
22.1
31.4
6.7
52.5
36.1
Next-ToBe Variants
DM
68.0
22.1
31.1
13.3
52.5
37.4
WSF
69.2
22.8
31.7
10.0
55.0
37.8
MF
68.2
23.1
31.9
13.3
55.0
38.3
To investigate the impact of different strategies for integrating temporal and semantic information, design three variants of Next-ToBE and conduct experiments mathematical reasoning
(i) Direct Multiplication (DM) combines temporal and semantic weights by element-wiss product Eq. without considering interactions between future tokens.
(ii) Weighted Summation Fusion (WSF) combines temporal and semantic matrices with a weighted summation in Eq. (9) (the left equation). In this experiment, set 0.5.
(iii) Multiplication Fusion (MF) combines temporal and semantic information with a multiplication in Eq. O (the right equation). We keep this setting for all other experiments.
22

---

Published as a conference paper at ICLR 2026
As shown in Table all Next-ToBE variants outperform the NTP, MTP, and MuToR baselines; the simplest direct-mulipliicttion scheme (+DM) yields clear average gains (NTP: +3.5%, MTP: +2.8%, MuToR: +1.3%). Among the variants, WSF and MF bring additional improvements, with MF achieving the best overall performance. This suggests that explicitly modeling both semantic similarity and temporal proximity among look-ahead tokens is beneficial. The random-walk formu- lation was introduced not as a necessity but as a refinement that further enhances performance.
E,4 IMPACTS OF FUTURE TOKEN BAG WINDOW SIZES
75
70
Accuracy
Qwen2.55
Qwen2 5-78
55
4
8
10
12
14
Future token bag (window) size
Figure 8: Impacts of future token bag window size
The size of the future window plays critical role in determining the performance of Next-ToBE. Figure Tiguue/8jillstio Billustrates results from models trained the CodeAlpaca dataset, reporting average scores on the code generation benchmark with window sizes ranging from 4 to 14 across three models. We observe that Qwen2.5-1.35 and Qwen2.5-7B achieve their highest performance at window size of 10, while 1 1-B-Instruct reaches its peak at 8. Overall, accuracy tends to improve as the window size increases initially, but declines once the window becomes excessively large.
E.5 FUTURE-TOKENS HIT RATE DISTRIBUTION
In section have discussed whether Next-ToBE has improved LLM in anticipating future tokens, and found that after fine-tuning with NeBE-ToBE, the average Future token Hit Ratio on mathematical reasoning across Qwen-Math-1.53 exhibits significant improvement. For more intuitive understanding of this improvement, we present the Histogram of Future token Hit Ratio.
soree with Nech-TBBE
Frequency
0000
Frequency
5000
4000
3000
2000
0.4
05
0.5 1.0
0.4
0,5
0.8
1.0
Future token Hit Rate
Future token Hit Rate
(a) k
(b) 10
Figure 9: Histogram of Future-tokens Hit Rate (FtHR for different window sizes
As shown in Fig. the distribution of FtHR shifts rightward after Next-Tobe ToBB fine-tuning, compared to the base model. This indicates that fine-tuning lea relative high range, thereby demonstrating a clear improvement in anticipatory capability.
23

---

Published as a conference paper at ICLR 2026
E.6 BEHAVIOR ON LARGER MODEL
To assess whether the benefits of Next-ToBE persist as the model scale increases, we extend our study from 8B models larger 14B-parameter foundation model (Qwen3-14B). We fine-tuned the 14B model using The average accuracies on five mathematical reasoning benchmarks are reported in Table
and compared it against three baselines (NTP, MTP, and MuToR).
Table 8: Performance comparisons of different competing methods on five mathematical datasets using Qwen3-14B.
Method
Math
Minerva Math
Olympiad Bench
AIME24
AMC23
Avg.
NTP
64.2
29.4
24.0
3.3
40.0
32.2
MTP
64.6
30.1
24.3
3.3
35.0
31.5
MuToR 64.0
30.1
23.4
6.7
42.5
33.3
Next-ToBE 65.4
30.5
24.6
6.7
47.5
34.9
As shown in Table 8 Next-ToBE continues achieve the highest performance across all bench- marks on the 14B model, with absolute gains of 3.4% over MTP, 2.7% over NTP, and 1.6% over MuToR. These improvements comparable even larger than those observed on the smaller 8B model, demonstrating Next-ToBE exhibits strong scalability and remains effective on larger models.
E.7 PRE-TRAINING SETTING
Rather than restricting our study to the post-training scenario, we further explore the applicabil- ity of Next-ToBE in the pre-training stage. Below, describe detailed setup (with necessary adjustments for the pretraining scenario) and report results.
Model and Data. We pretrained GPT-2- style model (124M) from scratch on 100k samples ran- domly drawn from the training split of the WikiText-103 dataset, for 10 epochs.
Training Strategy. The trade-off parameter À in 14 was increased from 0 to 0.1 over 10 epochs following quadratic schedule; look-ahead token weights followed the simple scheme in Eq and the weight scheme over intrinsic anticipatory signals in Eq removed (as the model trained from scratch has no such capacity).
Evaluation metries. (1) perplexity (PPL) of the generated text; (2) Future-token Hit Ratio (FHR), which reflects anticipatory capacity; (3) Accuracy on the QA task. Metrics (1) and (2) are based on the Wikitext-10s test-split, metric (3) is computed on the Hellaswag test-set.
Table 9: Comparison of NTP, MTP, and Next-ToBE under pre-training settings for GPT-2 model.
Method
PPL 4
f FtHR 50
HellaSwag Acc
NTP
28.67
22.51
25.27
MTP
73.60
23.38
24.84
Next-ToBE
34.32
25.80
26.13
As shown in Table 9 ,the following observations can be made:
(1) Anticipatory capacity. Next-ToBE shows notably higher anticipatory capability against NTP and MTP (+2.4-3.3 FIHR, 10-15% higher). Given that the model is trained from scratch with no anticipatory capacity at all, these results provide compelling evidence that Next-ToBE not only enhances the anticipatory capacity by fine-tuning a pretrained L.M but effectively develop such capability from zero through pretraining, surpassing both NTP and MTP with the same training condition.
(2) Perplexity. Next-ToBE exhibits higher perplexity than NTP (34.3 vs. 28.7). This is entirely expected because Next-ToB: was designed exactly to trade the short-term, next-token-predictin certainty (as measured by PPL) for improved longer-range anticipattion/ressoinng This has been supported by gains in sequence-level generation accuracy (Fig. 3b) and performance on complex reasoning tasks (Table T& & 2 In comparison, MTP improves FtH by only 3% over NTP far
24

---

Published as a conference paper at ICLR 2026
below our 15% gain while incurring much larger PPL penalty (73.6, 2.15x ours). These results indicate that Next- achieves a more efficient balance between local token prediction and global sequence modeling.
(3) QA- On the HellaSwag commonsense QA benchmark, Next-ToBE improves accuracy by +0.86% over NTP and +1.29% over MTP, indicating that its enhanced anticipatory behavior indeed translates into better reasoning performance. Though our experiments were small in scale, the comparisons are strictly controlled, with all methods trained exactly the same data for the same number of epochs. So the higher QA-task accuracy of Next-ToBE provides clear evidence of its advantage.
E.8 DISTRIBUTIONAL DRIFT ANALYSIS
We investigate whether fine-tuning with Next-ToBE leads to distributional drift during long-horizon autoregressive generation. To assess this, we generate 1,024-token sequences using diverse Math- benchmark prompts (test set) and compare, at each decoding step t, the next-token distribution of our model (Qwen2.5-Math--7 fine-tuned with Next-ToBE) against a stronger reference model (0wen2.5-Math-PRM--7
using KI. divergence. Averaged over 100 sequences, the KI. divergence at final step (t = 1024) is 0.15 for (T[(ours and 0.17 for (T(frefouurs and the mean KI. across all 1024 steps is 0.19 and 0.26, respectively. These small and stable divergences indicate that Next- ToBE preserves the training-tinge time token distribution throughout free-running generation and does exhibit long-term drift.
25


Reviews:
[{"review_id": "PG44814cnx", "reviewer": "Reviewer_rAwF", "summary": "This work proposes a novel training objective that adds a more learning signal at every time step by training to model to predict future tokens as well as the next token using the same output projection from the model. Empirically authors show how existing models are already capturing this sort of information about the future in the next token distribution and call it anitcipatory knowledge. Then they hypothesize that improving anticipatory capability in the next token distribution will improve generative performance of the model.\n\nAuthors design the objective with careful balancing between next token and future tokens learning; where they introduce weights that are based on the look-ahead distance as well as other scaling params.\nExperiments show that their proposed method gives slightly better performance across the deck in most of benchmarks they consider compared to both next token and multi token prediction alternatives.", "questions": "How do you think anticipatory information can be further utilized ? e.g. during inference?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* well grounded motivation with empirical data point showing anticipatory knowledge already existing in the models.\n* experiments are clearly described and adequate baselines and related work methods were considered.", "weaknesses": "* the balancing weights and added renormalization make the whole objective a bit cumbersome. Not very clear what might be the optimal value of K i.e. either we want to get as much anticipatory as possible or how do we optimize it? \n\n* its known that MTP shows the substantial gains at scaling; so this method might as well show much more gains at scale. While this might be infeasible for authors due to compute constraints, absence of larger models and on more data is a minor weakness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Xx4THghJ39", "reviewer": "Reviewer_9ScA", "summary": "The paper proposes Next-ToBE, a training objective to improve the anticipatory capability in LLMs. Next-ToBE modifies the standard next-token prediction objective in LLM training by replacing the one-hot target vector with a soft distribution over additional future tokens. The model incorporates a weighting scheme that assigns a dominant weight for the immediate next token, preserves the model's own anticipatory preferences, and dynamically adjusts future token weights based on temporal and semantic importance. Experiments on diverse reasoning settings (math reasoning, code generation, and common-sense reasoning) using Qwen and Llama models show consistent accuracy improvement over NTP and MTP baselines. The approach requires no architectural change and reduces peak memory usage compared to MTP methods.", "questions": "See weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The method is well-motivated by the two key empirical observations. The defined future token hit rate metric and the corresponding analysis provide valuable insights into LLMs' intrinsic anticipatory behavior and how it's linked to generative accuracies.\n- The objective is straightforward, and the design choices are well-motivated and supported by theoretical justification and ablation studies.\n- The method is simple and requires no architectural or inference changes. It reduces peak memory usage compared to MTP methods.\n- The paper provides extensive experiments across diverse reasoning tasks and model families and demonstrates consistent improvements.", "weaknesses": "- The scope of baselines could be limited. The related work section acknowledges connections to label smoothing and other MTP-related baselines, but they are not evaluated as baselines, including ProphetNet, token order prediction, and label smoothing.\n- Direct quantitative and qualitative comparison with NTP is unclear. For example, the computational overhead for the weighting scheme and potential side effects (e.g., hallucinations) of Next-ToBE are not discussed. \n- Experiments are confined to relatively small models (8B), and how well the proposed method scales to larger models is unclear. Evaluation of larger models could strengthen the claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "stSfulmbN6", "reviewer": "Reviewer_ianY", "summary": "This paper provides a loss function that encourages an autoregressive LM’s prediction to include more information about future tokens, without changing the underlying Transformer architecture or compromising inference accuracy or speed. The idea is to use a **mixture of two cross-entropy losses**: (i) between the LM head and the correct **next** token (one-hot), and (ii) between the model’s **bag-renormalized** prediction over the next (k) tokens and a set of importance weights over those tokens. Next-ToBE **nearly unanimously** outperforms NTP, MTP, and MuToR across numerous math, code, and commonsense reasoning benchmarks. Since there is only one LM output distribution per position, inference speed stays comparable to NTP (unlike MTP). The appendix contains many additional informative experiments.", "questions": "1. How many tokens of CoT were used in the math evaluation protocols (a maximum is provided, but not mean/SD)? I would like to understand why autoregressive sampling does not appear to pull the Next-ToBE model off the token-by-token distribution. Conceptually, since the loss divides by the probability mass assigned to the future (k)-bag, the auxiliary term should mainly reshuffle probabilities **within** that set, but I would still expect the opposite effect from what Figure 3b shows.\n2. Does long-term Next-ToBE **autoregressive** sampling diverge from the training distribution—e.g., as measured by KL **to/from** a stronger language model?\n3. Does this technique still work if you train from scratch rather than fine-tuning a pretrained baseline? I ask in part because the alpha weights focus on tokens that already receive probability mass, which will start **at random** when training from scratch.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Clearly written, strong and extensively evaluated results, and an improvement on a central topic (raw language modeling itself).", "weaknesses": "The weighting scheme seems ad hoc and complicated, containing several hyperparameters (lambda, k, epsilon, gamma, h, rho) and the fusion of W_tau and W_s). However, the random-walk matrix is backed up by ablations, and upon reflection, the random walk here may not be so ad hoc.\nMissing some minor distributional diagnostics (see questions).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NUEm9RPUrE", "reviewer": "Reviewer_f6SD", "summary": "This paper proposes Next-ToBE, a new fine-tuning objective as an alternative to vanilla NTP. \n\nThe paper points out that regular LLMs only focus on predicting the very next word/token, which stops them from actually thinking ahead (but it does already exist to a certain extent).\n\nThe authors created their new training method, Next-ToBE that teaches the model to predict not just the next token, but also a few tokens down the line at the same time. Instead of simply predicting the next work, it predicts a probability of the next few words.\n\nThe key idea is that this helps the model develop better planning skills by making it look further ahead during training.\n\nThe authors demonstrate Next-ToBE's effectiveness by fine-tuning Qwen and Llama models on tasks in mathematical, code, and commonsense reasoning.", "questions": "- Why are all experiments/results shown on post-trained models? Why not evaluate this on a base model? Do the same properties hold there? I would consider raising my score if this can be answered.\n- How expensive is the random-walk weighting calculation?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well written, motivated, an executed\n- The empirical results (Table 1) look solid. In the ran experiments, Next-ToBE is clearly the best method.\n- The proposed Future-tokens Hit Rate (FtHR) is a nice metric\n- I appreciate the detail in training method, data, eval pipeline", "weaknesses": "- My biggest concern is that it's only done on models that are already post-trained. Pure NTP/MTP is typically done for pretraining, so I'm not sure why this is done on post-trained models. I think the results would be a lot more convincing if it was done on base models, on pretraining data, with full model tuning (not just lora).\n- The conclusion claims that it is \"simple to implement\", the method is quite complicated, especially compared to MTP. Particularly the part about the random walk on the graph. In its current form, it will likely not be adopted by many.\n- A lot of the method is heuristically driven. E.g. chosen values of lambda. How can others use this method? Will it always work out of the box on any model with the fixed hyperparams? I think more ablations of the hyperparams (alpha, beta, lambda) are necessary\n- It's unclear how much more efficient Next-ToBE is than other methods. There is overhead from computing the values for the lookahead tokens, but not sure how much.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)