Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning
Authors: Haonan Wang
Chao Du
Kenji Kawaguchi
Tianyu Pang
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Reasoning
Primary area: foundation or frontier models, including LLMs

Abstract: Majority voting has proven effective for close-ended question answering by aggregating parallel reasoning traces. However, it is not directly applicable to open-ended reasoning, such as code generation and web-based deep research, where a “majority” over complete solutions is ill-defined. We introduce THINKMERGE, a training-free, plug-and-play decoding strategy that runs K parallel reasoning traces and averages their next-token logits at synchronization points to produce a single coherent output. THINKMERGE integrates seamlessly with vLLM/SGLang and remains compatible with standard decoding techniques such as Top-p/Top-k. Empirically, it matches or surpasses majority voting on AIME and GPQA, while delivering consistent gains on open-ended coding tasks: on LiveCodeBench (hard), pass@1 improves by +8.28% for DeepCoder-14B-Preview and +7.58% for Qwen3-8B. Beyond code, we further show that THINKMERGE improves web-based deep-research agents (e.g., WebSailor-7B/32B) across GAIA, BrowseComp-en/zh, and XbenchDeepSearch. These results demonstrate that parallel test-time scaling can benefit open-ended reasoning without relying on voting over complete outputs.

Published as a conference paper at ICLR 2026
THINK IN PARALLEL, ANSWER AS ONE: LOGIT AVERAGING FOR OPEN-ENDED REASONING
Haonan Wang"I Chao Du 2 Kenji Kawaguchi Tianyu Pang
Naatioall University of Singapore haonan. wang@u. nus edu; kenji@comp. nus, edu. sg; {t (tianyupang, (uchao)@sea. com
SSea Al Lab, Singapore
ABSTRACT
Majority voting has proven effective for close-ended question answering by aggregating parallel reasoning traces. However, is not directly applicable to open-ended reasoning, such as code generation and web-based deep re- search, where a "majority" over complete solutions ill-defined. We introduce THINKMERGE, a training-free, plug-and-play decoding strategy that runs K parallel reasoning traces and averages their next-token logits at synchronization points to produce a single coherent output. THINKMERGE integrates seam- lessly with vLLM/SGLang and remains compatible with standard decoding techniques such Top-p/Top-k: Empirically, it matches or surpasses majority voting on AIME and GPQA, while delivering consistent gains on open-ended coding tasks: on LiveCodeBench (hard), pass I improves by +8.28% for DeepCoder- 14B-Preview and +7.58% for Qwen3-8B. Beyond code, we further show that THINKMERGE improves web-based deep-research agents (e.g.. WebSailor- across GAIA, BrowseComp-en/zh, and XbenehDeepSearch. These results demonstrate that parallel test-time scaling can benefit open-ended reasoning without relying voting over complete outputs.
INTRODUCTION
Recent advances in Large Language Models (LLMs) have been driven by test-time compute scaling. As evidenced by OpenAl's ol (OpenAI, 2024), DeepSeek-RI (Guo et al., 2025), etc., models gener- ate extended "think" segments that reflect intermediate hypotheses, derivations, and self-corrections prior to emitting the final answer (Chen et al., 2025b; Yang al., 2025c). Such sequential test-time scaling has established new paradigm: increasing the inference- computation (c.g.. longer reasoning traces) often leads to improved accuracy and mooblem-solving capability.
Yet simply lengthening the chain has diminishing returns and can even hurt, e.g., overthinking (Chen et al., 2024; Cuadron al., 2025), with studies showing that correct answers often appear in shorter traces (Zeng et al., 2025). A natural complement is parallel scaling: generating multiple reason- ing traces and combining their evidence, most effectively through majority voting close-ended tasks (Wang et al., 2022; Aggarwal et al., 2023; Brown et al., 2024; Knappe et al., 2024).
Many real-world workloads, however, are inherently open-ended. Coding assistants must output executable programs (Jimenez et al., 2024; Yang et al., 2025b), while autonomous deep research agents often need model context protocol (MCP) tool calling, multi-step plans and long-form expla- nations (OpenAI, 2021; Anthropic, 2025; Alibaba-NLP, 2025). In such settings, majority voting is undefined since there no single canonical answer, even though has been highly effective in math and QA (MAA, 2025; Cobbe et al., 2021; Rein et al., 2024). This gap motivates central question: Can the benefils of test-time parallel reasoning be extended open-ended tasks without relying on voting over complete outputs?
In this work, we address this question by introducing TEINK,EEGE, an inference- time framework that averages logits across parallel reasoning paths to construct single high-quality answer. Unlike
Work done during Haonan Wang's internships at Sea Al Lab.
Correspondence Tianyu Pang.

---

Published as a conference paper at ICLR 2026
Qwen3-1.7B Pass@N and Maj@N Performance
Qwen3-8B Pass@N and Maj@N Performance
85.6
Metrics
Rasei!
63.5
Passgil Majority@3 Pass@e
Kajority Pasa@e
80
81.6
78.7
61.4
Accuracy (%)
60-
Accuracy (9)
76.2
69.3
65.6 5
66.
61.7
65.9
44.3
60
10-
36.7
56.4
30-
26.7
46,0
20
22.8
41.6
AM24
AIME25
HMMT24
HMMT25
AIE221
AIME25
HMMT24
HMMM22
Figure I: Margins between Pass @1 and Pass @8 across (close-ended) tasks and models. Larger margins imply more room for majority voting to help.
majority voting, which selects among complete outputs, THINKMERGE enables the model to think in parallel but speak with one voice. Concretely, given a question, we run K diverse reasoning traces concurrently. At synchronization point (for example, after reasoning delimiter), we aggregate the next-token logits from all traces by averaging them, normalize the merged logits into a probability distribution, and sample the next token. The chosen token is then injected back into every trace, as if each had generated it, and the parallel reasoning continues step by step.
Through this iterative ensemble decoding, the model produces the guidance of multiple concurrent "thoughts". The approach is entirely training-free: it requires no fine-tuning or additional supervision, only multiple forward passes during inference. Moreover, it is plug-and-play and fully compatible with standard decoding strategies such as Top-p, Top-k, temperature, and repetition penalties (Shi al., 2024). Intuitively, THINKMERGE allows the model to explore a broader range of ideas in parallel and converge on more reliable answer.
single coherent solution that reflects
Empirically, we evaluate THINKMERGE on both closed-ended and open-ended reasoning tasks. On math and science benchmarks with well-defined answers, such as AIME (MAA, 2025) and GPQA (Rein et al., 2024), THINK matches or slightly surpasses the accuracy of majority vot- ing and its variants (Wang et al., 2022; Zeng et al., 2025). More critically, open-ended tasks where majority voting is not applicable, THINKMERGE yields consistent improvements over single-chain decoding. For example, on LiveCodeBench (Jain et al., 2024), Pass@1 I increases by +8.28 % for DeepCoder-14B-. Previiw and +7.58% for Qwen3-8B on the hard-level coding problems. Beyond coding, we further demonstrate that THINKMERGE also benefits web-based deep-research agents: running multiple WebSailor-7B/32B (Li et al., 2025a) trajectories in parallel and merging their logits in decoding answers improves pass @ by up +10.2% % points on ThenchDeeppSarce (Chen al., 2025a) and +4.8% points on GAIA (Mialon et al., 2023). THINKMERGE integrates seamlessly with inference frameworks such as vLLM (Kwon et al., 2023) and SGLang (Zheng et al., 2024), supports both online serving and offline batch decoding, and remains compatible with standard sampling con- trols (Top-p, Top-k, temperature, penalties). can thus be adopted as a simple drop-in augmentation to existing LLM deployments.
2 RELATED WORK
Majority Voting and Variants. Parallel scaling explores many candidate solutions and aggregates them (Brown et al., 2024; Zeng et al., 2025; Stroebl et al., 2024; Sun et al., 2024; Gui et al., 2024; Snell al., 2025; Liu al., 2025; Wu et al., 2025a; Jiang al., 2023; Li et al., 2025c; Chen et al.. 2023). Aggregation can happen at the solution level, either with seward-guide Best-of-N search (Sun et al., 2024) or guidance-free voting such as rule-based Majority Voting (Wang et al., 2022; Chen et al., 2023), with variants that adapt the sample count or filter candidates (Aggarwal et al., 2023; Xue et al., 2023; Huang et al., 2024; Knappe et al., 2024). While these methods deliver strong gains closed-ended tasks, they ill-defined for open-ended reasoning. where valid out- puts rarely repeat and "voting" is not meaningful. Instead, we ensemble logit-level (pre-softmax) during the answer generation phase across reasoning paths, reducing dependence on single con- sensus answer and turning extra test-time computation into performance gains on open- open-ended tasks.
Model-Based Aggregation. Beyond voting, several model-based aggregation methods have been proposed (Chen et al., 2023; Qi et al., 2025; Zhao et al., 2025; Jiang et al., 2023; Edge et al., 2024). These either (i) train separate scorer select among candidates, or (ii) prompt L.M compare
2

---

Published as a conference paper at ICLR 2026
CPQA PassgN Performance
AIME PassoN Performance
LiveCodeBench Pass@N Performance
100
PAXAINN (N4
(50
Ole 140
40
Oner Vaaggee
16
25 50
Figure 2: Trend of @ as the number of samples N increases. Gains are evident on AIME 2025,
GPQA Diamond, and the open-ended LIVECODEBENCH.
and summarize them. For example, LLM-Blender (Jiang et al., 2023) takes the top- candidate answers of a query, concatenates them with the input, and feeds the resulting sequence into a model that generates new answer intended to outperform all individual candidates. GraphRAG (Edge et al., 2024) adopts a similar high-level map-reduce pattern on the retrieval side: it first produces partial summaries and answers in parallel, then issues single LLM call to summarize these partial answers into a final response. Such approaches require an additional model to assign scalar scores or produce summaries, (in training) typically demand supervised tuning or domain adaptation to learn reliable judgments, and (in inference) incur at least one extra model forward over long, concatenated candidate outputs. In contrast, our method is training-fre free and performs logit-level aggregation during answer decoding, yielding a single output that integrates multiple parallel reasoning paths.
Probability- and PoE-Style Ensembles. A separate line of work ensembles next-token prob- abilities at each decoding step. Classical Product-of-Experty (PoE) (Hinton, 1999) combines expert predictions by multiplying their probability distributions. Recent LLM work adapts this idea prob- ability space. M-Ped (Guo et al., 2024) submits multiple prompt variants for the same input in batch mode and ensembles the next-token distribution by averaging the per-prompt probabilities. Other approaches ensemble different models: EVA (Xu et al., 2024) learns cross-model vocabulary map- pings to align output distributions into a shared space before averaging, while the agreement- based method of Wicks et al. (Wicks et al., 2025) constrains multiple models with different vocabularies to generate a shared surface string via an efficient search procedure. These probability-level token en- sembles primarily target settings with multiple prompts or multiple models, usually closed-form generation tasks such as machine translation or data-to-text. THINKMERGE is related in spirit but differs in how and where aggregation applied. We operate within a single reasoning model in the modern "think-then- anee-a"swer" paradigm, treat K chain-of Shain-off-thouhht traces as experts, and aggregate their pre-sofimas logits only during the answer phase, leaving the thinking phase fully diverse. This logit-space fusion can be viewed as PoE-style combination implemented as drop-in logit proces- sor in standard decoding framworks (e.g.. vL..M), and, more importantly, allows us to systematically convert extra test-time compute into accuracy and success-rate gains on open-ended reasoning and agentic deep-research benchmarks where solution-level voting is not directly applicable.
3 PRELIMINARY STUDY
To study the relation between ensemble gains and answer coverage the probability that among K sampled solutions at least one is correct (i.e., Pass K), we evaluate closed-ended math/science (4125 (MAA, 2025), 24'22 (Balunović 2024)) and open-ended coding (LiveCodeBench v5 (Jain et al., 2024)). For closed-ended tasks 14B (Yang et al., 2025a) and DeepSeek-RR 1-----R-iDilii--uiiee et al., 2025); for coding we test DeepCoder- 14B-Preview (Luo et al., 2025), Qwen3-8B, Swen3-Coder-30A33. --nttu-Intt Qwen3 TB-Thinking, Qwen3-Thine Think-30A.3 (Yang et al., 2025a).
al., 2025), GPQA (Rein al.,
we use
(Guo
Closed-ended tasks: Majority@ K and Pass K. For multiple-sampiinn at inference time, the empirical benefit of majority voting on closed-ended benchmarks is closely tied to the improved mar- gins between Pass K and Pass@1 1 and how quickly Pass@) K grows with K. Intuitively, when ad- ditional samples quickly increase the probability that the correct option appears (i.e., larger pass @ K gaps between K=1 and K>1), the vote distribution shifts toward the right answer, when pass @ K saturates, samples tend to reinforce the same wrong choice and voting yields little gain. As shown in Figure 1, there is clear margin between @ 8 and Pass@) @ for two reasoning models across four closed-ended datasets, indicating that parallel sampling meaningfully raises the chance of observing the correct option opttin--eence majority voting is expected to perform between these bounds.
Does parallel sampling help in open-ended settings? Unlike classification, open-ended problems do not admit direct vote over small, discrete label set. We therefore examine whether the existence
3

---

Published as a conference paper at ICLR 2026
LiveCodeBench Pass@N Easy Problems
LiveCodeBench Pass@N Medium Problems
LiveCodeBench PasagNN Hard Problams
Par
Passgg (%)
66
Devil Du 1008
Da
Guee 2041
Oue Care MAl
10
25
50
10
25
50
25
50
Figure 3:
LIVECODEBENCH stratified by difficulty. Hard questions exhibit larger Pass N gains N increases, indicating that parallel reasoning is potentially helpful on challenging instances.
signal captured by pass@N N (at least one good solution among N samples) still grows with N when evaluation is based on program execution or unit tests. Figure 2 plots Pass N as N increases. We observe a rapid rise on closed-ended AIME 2025 and GPQA Diamond, and a consistent increase on the open-ended LiVECODEBENCH, The positive slope on LIVECODEBENCH indicates that ensembling multiple reasoning trajectories can be potentially beneficial.
Where do the gains come from? To localize the effect, we stratify LIVECODEBENCH by difficulty. Figure shows that the increase in @uss@N more pronounced on the hard subset: difficult prob- lems benefit more from multiple, diverse reasoning attempts. This pattern mirrors closed-ended ob- servations- hard items accrue larger returns from additional samples--uggesting that ensembling over parallel thoughts might unlock solutions that single-pass decoding misses hard problems.
Takeaways and motivation. Across closed- and open-ended settings, Pass@N improves with N, and the gains concentrate harder instances. For closed-ended tasks, majority voting directly con- verts these gains into accuracy. For open-ended tasks, however, voting over free-form outputs is ill-posed. These findings motivate an open-ended ensembling mechanism that aggregates think- ing processes-precinee the goal of our approach that averages token-level logits across parallel reasoning paths.
METHOD
We propose training-free, plug-and-play decoding strategy that ensembles diverse chains of thought at the token level produce a single coherent answer for open-ended queries. The method proceeds in two stages: (i) generate K diverse reasoning traces up to a delimiter token, e.g. </think>; (ii) after the delimiter, decode one shared answer sequence by averaging the next- token logils across all K reasoning contexts at every autoregressive step.
Diverse Reasoning Generation. Given an input prompt or question Q, we prompt the LLM to pro- duce step-by-step reasoning ending with special delimiter marker (e.g. </think> to indicate the end of the thinking segment). To obtain diverse reasoning traces, we sample K independent chain-of-thee oothughht sequences R1, Ra,.... Rx from the model. Notably, official model cards for cent reasoning models recommend relatively high temperatures (e.g., 0.5-0.7 for DeepSeek and Qwen) (Guo et al., 2025; Yang al., 2025a). Because of the randomness introduced by the high temperature, the K reasoning paths are varied, exploring different plausible approaches or perspec- tives the problem. This step uses the model as-is, without any fine-tuning- are simply drawing multiple reasoning samples from the model's own distribution, which makes the procedure straight- forward integrate.
Ensembled Answer Decoding, Once desired number of reasoning chains reach delimiter markers (i.e. the end of the thought process), the model begins generating the answer portion jointly informed by all chains. At each autoregressive decoding step i of the answer, we query the model's 's next-token pre-softmax logits Mo (-) for each reasoning chain context. We then aggregate these logits by arith- metic mean. Formally, let μęı denote an empty answer prefix, for each chain & E (1,..., K). we define the logit vector over the vocabulary V as z Mo(Q, We.))) E R!V), we then ensemble on logits via arithmetic mean and only then apply softmax:
2,
1
(k)
Po(ys Q R..K, softmax(2:)lal)
K
for each possible token yi E V at that step. We then sample or select the next token y from this aggregated distribution P. Note, this ensemble step will not have impact on the up-following
4

---

Published as a conference paper at ICLR 2026
Question
Thinking Process
Answer
Pad Token
<think>
</think>
<think>
</think>
<think
</think
<think>
</think>
Answer
<think>
</think>
<think
</think>
Re-prefilling to Squeeze Bubble
Figure 4: Two-stage implementation. Stage I Map (Diverse reasoning). Sample K chain- of-thought traces up to a delimiter. Stage Reduce (Answer decoding). Left-pad the ques- rianonngnnnng contexts common length, re-prefill to "squeeze bubble", and then decode single answer sequence by averaging the pre-softmax logits across the K reasonings at every step.
Algorithm Ensemble-of offsmmble-of-Thoghh
Require: LLM Mo (outputs pre-softmax logits); query Q: number of traces K; reasoning temper-
ature Tihink; answer temperature Tans: decoding policy и (e.g., greedy top-k top-p repetition-
penalty); stopping rule STOP (cos/lengthhvalidatoor
1:
Parallel thinking: For & 1, K, sample a reasoning trace Rs ~ pol Tihink)) by running the model until reasoning end delimiter.
2: Initialize the shared answer prefix t- 2.
3: while not (y)p(y) do
4: for k: = 1 to K do
D fully parallelizable across k b next-token logits conditioned on (Q, Rig y)
5:
_(()) - (Q Rg,
6: end for
7:
lt K Σ ((k)
8:
(optional) Apply logit policy the averaged logits: E PROCESS(E::)
9:
Form the answer-step distribution P +- softmax( L/Tmm)
10: Select the next token Unext ((P)
b greedy: arg max; sampling: draw from P D append to the shared answer prefix
Y 4- y Vuext
12: Note: All K contexts implicitly share the updated y token at the next step via MQ(Q, RR. y).
13: end while
14: return
decoding strategies, such as Top-k, temperature, and penalty (Shi et al., 2024). This chosen token becomes the next word in the final answer and is also appended to each of the K contexts before proceeding to the next decoding step. By updating all contexts with the same generated answer token, ensure that subsequent probability predictions from each chain remain conditioned on a common partial answer. We repeat this token-level ensemble process autoregressively until an end-of-answer token is produced or another stopping criterion is met.
4.1
IMPLEMENTATION
Our method integrates cleanly with modern high-throughput inference stacks, including vLLM (Kwon et al., 2023) and SGLang (Zheng et al., 2024).
Two-stage pipeline. As illustrated Figure up to delimiter, and then (ii) left-pad all question contexts to the same length and re- prefill build an aligned KV cache. This "bubble squeezing" removes idle compute caused by unequal trace lengths and enables logit-level ensembling for the answer: at each autoregressive step we average the pre-softmax logits from the K contexts and decode single shared token. The design works in both online serving and offline batch settings for vLLM/SGlang with minimal changes. In practice, prefill modern optimized systems is fast; its overhead is negligible compared to decoding, so the two-stage variant remains efficient while being easy to instrument for ablations studies.
(i) batch-generate K diverse reasoning traces
One-step pipeline with Flex-Attention. Alternatively, we integrate ensembling directly into the decoding (Figure 5). We treat the K sequences
batch and rely flexable masks of Flex- Attention Torch Team, 2025) to silence padding tokens emitted by shorter traces when waiting the longest reasoning stream completes. After the delimiter, aggregate the pre-sofimax logits
5

---

Published as a conference paper at ICLR 2026
</think>
<think?
Batch Decoding
</think>
Prefix Question Attention
</hhnkk>
Flexible Attention Mask
Figure 5: One-step decoding with Flex-Attention Padding tokens produced by shorter reasoning traces when waiting for the longest trace are masked so they are not attended. After the delimiter (e.g., </think>), we average pre-softmax logits across all streams produce a single shared answer token at each step for all streams.
across the K contexts at every step and produce one shared answer token. We implement this variant in the HuggingFace Transformers generation pipeline (Wolf al., 2020), leveraging its stable Flex- Attention support (Hugging Face, 2024). At the time of our experiments, vLLM was in the process of integrating Flex-Attention into vLLM v1 (drisspg, 2025); we plan to open-source a vLLM implementation of the one-step variant once upstream support stabilizes.
For the controlled analyses and method variants in Section 4.2, we default to the two-stage vLLM pipeline, which maintains high throughput while providing convenient handling of reasoning traces.
4.2 DESIGN CHOICES AND VARIANTS
We study four orthogonal design reasoning trace processing startegies for when to start answer decoding and which reasoning traces to ensemble. Unless noted, we sample reasoning traces with temperature stop each at delimiter token, and then ensemble a subset of them to decode one shared answer by averaging pre-softmax logits.
(A) Merect-Merge. We decode K reasoning traces in parallel until their delimiter and immediately ensemble them to decode the answer. This is the default configuration used in most experiments. can be regarded as extra processing.
(B) K Early-Ready. To reduce tail latency from very long traces, we begin answer decoding as K traces have completed their reasoning segments, rather than waiting for all N(N > K) reasoning to be finished. Formally, let Reetty = {R Rg has emitted the delimiter). We start ensembling when |Ready 2 K. The answer is then decoded by averaging logits the currently available thinkings. This variant trades a small amount of diversity for lower latency and higher throughput, and is useful in online serving.
(C) Trimming (De-Repeat Suffix). Motivated by prior observations that models may emit repeated reflection fragments (e.g., "Wait", "Iimm", ", and "Alternatively" "') near the end of the reasoning phase and that overthinking can degrade performance (Wang et al., 2025), we remove degenerate repeated suffixes before re-prefill. Concretely, for each finished reasoning trace Rg. we detect the longest repeated suffix (e.g., via regex pattern matching with length thresholds) and trim it, producing Rı- trim RR). This preserves semantically useful steps while avoiding overweighting on the long and misleading words when decoding final answers.
(D) Shortest-K Merge Annti-Overtinnking). Prior work reports that excessive "overthinking" can correlate with worse final answers (Chen et al., 2024; Cuadron et al., 2025; Wu et al., 2025b; Sui et al., 2025). To bias toward concise, high-signal reasoning, we sort completed traces by their pre- delimiter length and select the K shortest from a reasoning pool with size N(N >> K) for logit ensembling: S argsort (flen( (Ien(R&)) R=)) 1:k: ensemble over (R h This variant leverages length- inductive bias to stay clear and on topic, and they help avoid late drift or repetition. Different from K K-Early-Ready, it waits for all N traces to finish so the shortest can selected globally, trading latency for anti-overthinking bias.
EXPERIMENTS
Experimental Setup. We evaluate THINKMERGE in two regimes. (i) For closed-ended reasoning, consider AIME 2025 (MAA, 2025) and GPQA Diamond (Rein et al., 2024), where each question

---

Published as a conference paper at ICLR 2026
Table 1: Performance of Majority Voting (MV) vs. THINKMERGE on AIME'25 across different strategies. Within each (model K strategy) group, the highest score within each category is bold; the highest within tier is underlined.
All-Reduce DirectMerge (A)
Early-Reade
Shortest-K Merge
Model
MV
Trimming (B)
MV
Ours (C)
MV
Ours (D)
K Qwen3-4B
63.3
66.7
66.0
66.7
66.7
70.7
65.3
Qwen3-14h 14B
70.7
72.0
72.7
72.0
73.3
75.3
74.7
RI-Distill Pesstll-Qwenn-2e Qwen-7B
40.7
41.3
42.0
40.7
41.3
50.7
48.0
K K=
Qwen3-4B
68.0
72.0
68.0
72.7
72.7
75.3
69.3
Qwen3-14hh 14B
73.3
72.0
73.3
76.0
73.3
78.0
76.0
RI-Distil-QQwen-BB
47.3
46.0
46.0
47.3
45.3
52.0
50.7
K= 8
Qwen3-4B
68.7
70.0
68.7
73.3
70.8
75.3
72.7
Qwen3-1444
74.0
78.0
73.3
77.4
78.0
80.0
78.7
D1-Distill-Qwen-777
46.7
45.3
48.0
46.7
45.3
52.7
52.7
Table 2:
Performance of Majority Voting (MV) and THINKMERGE on GPQA across different strategies. Within each (model x K x strategy) group, the highest score within each category is bold; the highest within tier is underlined.
All-Reduce DirectMerge (A)
Early-Reade
Shortest-K Merge
Model
MV
Trimming (B)
MV
Ours (C)
MV
Ours (D)
K K= 2 Qwen3-4B
49.2
50.3
50.0
49.4
50.3
52.0
52.6
Qwen3-14B
60.8
61.6
61.4
60.8
62.1
63.4
63.7
D.-Distil-Qwen--11
44.9
49.2
49.2
47.4
47.1
43.8
Qwen3-4B
51.2
52.2
52.2
51.4
52.4
53.7
51.8
Qwen3-14B
63.0
64.0
62.7
64.0
62.4
65.2
63.8
RI-Distill- Qwen-7B
50.3
48.9
49.0
50.3
48.9
50.2
47,4
K=8
Qwen3-4B 53.3
51.6
51.3
53.3
51.3
55.5
53.8
Qwen3-14P
63.9
64.1
63.0
64.1
65.9
63.7
Ri-Distill-Qwen--B
52.5
50.0
50.0
52.5
50.2
52.9
47.2
has unique ground-trutt answer and multiple samples can be aggregated via majority voting. (ii) For open-ended reasoning, we evaluate on LiveCodeBench v5 2222410-20222 20-20025.022 2222) (Jain et al., 2024), BrowseComp-en (Wei et al., 2025), BrowseComp-zth (Zhou al., 2025), GAIA (Mialon et al., 2023), and xBench-DeepSearch (Chen et al., 2025a), where majority voting is ill-defined and the quality of single coherent solution is what matters. For closed-ended tasks, we run five trials and report the mean accuracy in the main text, with meanistd in Appendix A.2; for LiveCodeBench, we report pass@1. 1.
For each question, generate K E {2, 4, 8) parallel reasoning traces and perform ensemble step at the answer phase by averaging pre-softmax logits across the K. For the Shortest- K Merge variant (anti-overthinkingg), K shortest by pre-delimiter length. Our processing strategies, (B) Early-Ready and (D) Shortest- Merge, also be paired with majority voting (MV). For fairness, we report MV under the same operation whenever it is applicable; thus, trimming cannot be applied to it. Methods that aggregate all completed traces- Majoriyy Voting, Direct-Merge, and De-Repeat Suffix Trimming-aud are grouped under the label All-Reduce in the tables. In contrast, Early-Ready and Shortest- operate on subset of traces during merging, so we place them in separate columns.
first produce pool of N 64 completed traces and ensemble the
For closed-endes tasks we evaluate Qwen3-4B, Qwen3-14B (Yang et al., 2025a), and DeepSeek-RI-Distill--- use DeepCoder-1443
(Guo et al., 2025). For the open-ended coding task we Awen3-Coderr:3AA3B-Innnttt
(Luo et al., 2025), Qwen3-8B, 4wen3-4B-Thinkiie (0725), and Qwen3-Think - (Yang et al., 2025a). For open-ended deep-research agent task, the WebSailor-3B, WebSailor-7B and WebSailor-32B (Li et al., 2025a). set the maximum sequence length to 32, 768, detailed sampling hyperparameters in Appendix A.

---

Published as a conference paper at ICLR 2026
Table 3: Effect of answer-phase temperature for THINKMERGE. Default vs. setting the answer- phase temperature to Tas== =033. The highest score within each category is bold; the tier is underlined.
Direct-Merge
Realy-Ready
Shortest-K Merge
Model
Default
Lams 0.3
Default
Tam 0.3 Default
Lans 0.3
==
Qwen3-4B
66.7
64.7
66.7
67.3
65.3
66.7
Qwen3-14B
72.0
71.3
73.3
73.3
74.7
74.0
RI-Distill-Qwae -wenn-BB
41.3
41.3
41.3
41.3
48.0
48.7
K =4
Qwen3-4B
72.0
70.0
72.7
72.0
69.3
70.7
Qwen3-14B
72.0
72.7
73.3
72.7
76.0
75.3
RI E1-Distill-ween-2
46.0
42.7
45,3
43.3
50.7
52.0
K
Qwen3-4B
70.0
68.7
70,8
69.3
72.7
70.7
Qwen3-14B
78.0
74.7
78.0
76.7
78.7
77.3
R1- RI-Distill-wen-7B
45.3
44.7
45.3
44.0
52.7
51.3
Table 4: LiveCodeBench Overall Pass (%). Row-wise best among merge settings is highlighted.
Model
Baseline
Direct-Merge
Shortest- K Merge
K=8
K=4
K=2
K=8
K=4
K=2
DeepCoder-I 4B-Preview
55.32
56.23
57.14
58.36
59.57
59.88
61.09
Qwen3-8B
57.14
53.19
56.53
59.57
58.31
56.53
58.05
Swen3-Coder-30A3
37.69
41.34
38.30
39,82
39.82
38.30
39.21
Qwen3-4B-Thhinking
63.53
60.79
62.01
62.61
62.31
64.13
63.83
Qwen3-Think-30A518
69.30
68.39
68.69
65.65
67.78
67.48
72.04
5.1 CLOSE-ENDED TASKS: COMPETITIVE WITH MAJORITY VOTING
On AIME and GPQA, THINKMERGE is competitive with majority voting (MV), often matching or slightly exceeding it when the merge is performed across all parallel thoughts ("All-Merge") For instance, on AIME, Qwen3-4B K=4 improves from MV 68. 68.0% to THINKMERGE 72.0% (+4.0%), and Qwen3-14p at K-8 improves from 74.0 to 78.0 (+4.0 %), shown in Table 1. On GPQA, THINKMERGE small K is reliably strong: at K-2, Qwen3-4B improves from 49.2% to 50.3% (+1.1 %). Qwen3-14B from 60.8% to 61.6% (+0.8 % ) and R I--Disiill-Qwen-- from 44.9% to 49.2% (+4.3 %) (Table 2). When applying Shortest- K Merge strategy. both THINKMERGEand AIME/GPQA (c.g., AIME
MV are boosted, but MV is generally stronger than THINKMERGE Qwen3-1444 at K=8: 80.0% vs. 78.7%; GPQA shows the same trend at K=4, 8). This indicates that when there is large reasoning pool, for math questions, shortest- K avoiding redundant self- reflection loops a strong inductive bias to select high-quality solutions, in which TEINKMERGE cannot help much.
Trimming repcated reflections. Our regex-based trimming variant (Ours+ (OursiTiimingg) shows mixed, model-dependent Sofectt--Smeeiines helpful (e.g.. AIME with R1-Distill-Wwen77B K=8: 48.0%), but often neutral or slightly negative. A sample-by-sampe reflection patterns vary widely across model-task combinations, making a single, robust pattern- matching rule difficult to design (and brittle rules risk removing useful content). Consequently, we don't use trimming in the subsequent open-ended experiments.
checking indicates that
Answer-phase temperature. Lowering the answer-phase temperature Tams offers no consistent gain. On AIME, many cells mildly drop at =033 especially for K-4, (e.g., All-Merge, Qwen3- 4B: 8 70.0% 70.0%-66. -6..7%) with modest increases on K-2 (e.g., Shortest Merge, Qwen3-4B: 65.3%-666 66) (Table 3). Our takeaway that: once the thinking phase already induces enough diversity, further "cooling" at the answer phase unnecessary.
5.2 OPEN-ENDED CODE: FEWER IS BETTER, AND "SHORTEST" BIAS LOSE EFFECTIVENESS
On LiveCodeBench, THINKMERGE outperforms single-pass baselines, with the most reliable gains small K. For DeepCoder-1455 P4e-rrevee.. overall pass@1 @1 improves from 55.32 *61.09 (+5.77 %) (best at Shortest-Fi K, K=2); for Qwen3-8B, it improves from 57.14-559.77 (+2.43 %) (best at All-Merge, K=2); see Table 4.
The "shortest" inductive bias is not universal. Prior math QA reports that "shorter chains are often better" attributing failures long, looping reflections. In code generation, Shortest- K Merge
8

---

Published as a conference paper at ICLR 2026
Table 5: DeepResearch agent benchmarks Pass @ (%). Row-wise best is highlighted.
Benchmark
Model
Baseline
THINKMERGE
N=2
N=4
N=
WebSailor-3Bl
32.22
33.49
15.04
5.34
GAIA
WebSailor-7h5
35.52
33.98
41.26
36,89
WebSailor-32B
46.64
48.55
51.46
50.49
WebSailor 3B
26.40
26,80
12.20
5,40
Xbench- Seanhh
WebSailor-7B
37.80
43.20
48.00
47.20
WebSaillor-2BB
50.40
50.20
55.20
57.60
WebSailor- 3B
4.70
6.30
3.50
2.50
BrowseComp-FN (200)
WchSaillo--BB
6.30
11.00
13.60
13.10
WchSailor-32PP
11.80
13.10
13.40
14.50
WebSailor-3B
8.67
11.76
4.15
2.77
BrowscComp-ZH
WchSailor-7h5
14.01
21.45
24.91
22.49
WcbSailor-32B 21.97
26.30
28.37
27.34
is not always benefit: shorter traces may omit necessary scaffolding (imports, helper functions) and
harm executability.
Helps most on Mediun/Hard Questions. The difficulty split shows that improvements concentrate on Mediun/Hard (Tables 13-14). On Hard, DeepCoder- 14B rises 20.69-228.9 (+8.28 %)) and Qwen3-8B 24.14-$31.72 (+7.58 %)). while Easy is largely saturated (Table 12). Due to space limits, full LiveCodeBench results across difficulty levels (Tables 12-14) are in Appendix A.3.
How many thoughts to merge? On closed-ended datasets, increasing K generally helps but shows diminishing returns beyond small K and depends on the base model: several cases K=4 already saturates, and K-=8 doubles the compute but offers little additional gain may even slightly regress. Consistently, majority voting also shows diminishing returns as N grows on those tasks; Figure in the appendix indicates saturation when N 8. For open-ended code, the saturation point is even earlier: K=2 is typically best, often outperforming K=4 and K=8. This is good news for prac- tical deployment. The strong performance is achievable with small ensembles, keeping affordable memory and computation costs for online serving.
Finally, test whether THINK MERGE also benefit agentic deep-research settings, where the model must interleave reasoning (e.g.. schhnk>.../thiinke and tool calls before pro- ducing answer. Concretely, evaluate three Tongyi-WebSailor agents Wetibs---eSai..r........ WebSailor-7BB and WebSailor-3BB-on four challenging web-based benchmarks: BrowseComp- en (Wei al., 2025), BrowseComp-zh (Zhou et al., 2025), GAIA (Mialon et al., 2023), and XbenehDeepSearcc (Chen et al., 2025a). CrowseComp-en/fe focus on hard-to-find, multi-hop factual queries in English and Chinese. Because BrowseComp-ert we randomly sample 200 questions as test subset, making it comparable in size to BrowseComp- zh (289 questions). GAIA requires robust tool use for multi-step real-world tasks; following prior work (Li et al., 2025b), evaluate the 103 text-only validation cases. XbenchDeepSearch tar- gets professional-style deep information retrieval. We use the WebSailor agent pipeline with the recommended decoding hyperparameters: temperature 0.6, top-p 0.95, and context length 32, 768.
large (1,266 questions total),
5.3 OPEN-ENDED DEEPRESEARCH AGENTS
Finally, we test whether THINKMERGE can also benefit agentic deep-research settings, where the model must interleave internal reasoning (e.g.. <think>..<<thinne with tool calls before pro- ducing an answer. We evaluate three Tongyi-WebSailor WebSailor-32B-o8 four challenging web-based benchmarks: BrowseComp-oo (Wei al., 2025), BrowseComp-zh (Zhou et al., 2025), GAIA (Mialon et al., 2023), and XhenchDeepSearch (Chen al., 2025a). BrowseComppen//2 focus on hard-to-find multi-hop factual queries English and Chinese. Because BrowseComp-es is large (1,266 questions in total), we randomly sample 200 questions as test subset, making comparable size to BrowseComp-zhh (289 questions). GAIA requires robust tool use multi-step real-world tasks; following prior work (Li et al., 2025b), we evaluate on the 103 text-only validation cases. XbenchDeepSearch targets professional-stlle deep information retrieval. We use the WebSailor agent pipeline (Alibaba-NLP. 2025) with the recom- mended decoding hyperparameters: temperature 0.6. top-p 0.95, and context length 32,768. For computational efficiency. Google Search API for the Serper API (Serper.dev) to reduce API fee costs; under this configuration, re-run all baselines and report the average five runs in Table 5.
Webbnns-eelSall WebSailor-7B, and
replace the evaluator model Owen2.5-72B with GPT-4.1 and swap the

---

Model Published as a conference paper at ICLR 2026
Table 6: Performance of Majority Voting, THINKMERGE, and Prob-Merge on AIME'25 across different numbers of samples K.
Majority Voting
THINKMERGE
Prob-Merge
K =
Qwen3-4B
63.3
66.7
62.0
Qwen3-144 1ren3-14B
70.7
72.0
68.0
RI-Distill- Bistill-Qwen-7BB
40.7
41.3
34.7
==
Qwen3-4B
68.0
72.0
65.4 4
Qwen3-14B
73.3
72.0
70.0
RI-Distill-Qwen-7B
47.3
46.0
32.7
==
Qwen3-4B
68.7
70.0
62.0
Qwen3-14B
74.0
78.0
69.4
RI R1-Distill-Qwen-7B
46.7
45.3
34.0
Scaling up the agent makes test-time ensembles effective. For the stronger 7B and 32B Web- Sailor agents, THINKMERGE consistently improves over the single-run baseline, often by a large margin. On XbenchDeepSSarch,, WebSailor-32B improves from 50.4 to 57.6 N (+7.2), while WebSailor-7B rises from 37.8 to 48.0 (+10.2). On GAIA, BBbSailor-32B reaches 51.46 at N=4, and WebSailor-7B improves from 35.52 41.26. The two BrowseComp benchmarks show a sim- ilar pattern. These results indicate that, once the underlying agent is sufficiently capable, running multiple research trajectories in parallel and then merging their answers is an effective way to trade test-time compute for higher performance.
In contrast, the 3B WehSailor agent only benefits from THINKMERGE at small N: across four benchmarks, 2 yields mild gains, but performance degrades noticeably at N=4, 8. This is con- sistent with "garbage in, garbage out" intuition: when most trajectories are low-quality or off-topic, ensembling more of them will not fix the errors and tively, small models tend to generate many such weak research trajectories, so averaging over too many of them "washes out" the good ones, suggesting that aggressive test-time compute scaling is only beneficial beyond a certain capability threshold.
dilute the few good traces. Qualita-
5.4 ABLATION: MERGE LOGIT vs. MERGE PROBABILITY FOR REASONING MODELS
Our THINKMERGE aggregates decoding the logit level: at each answer-time decoding step we take the arithmetic mean over the pre-softmax logits as described in Section 4. A natural al- ternative, more in line with prior work (Wicks et al., 2025; Xu et al., 2024; Guo al., 2024) token probability ensembling, is to first normalize each logit vector and then average probabilities:
Pt Σn softmax(2{**) We refer to this variant Prob-Merge. In both cases, aggregation is restricted to the answer phase; the <think> phase remains fully independent.
Table 6 compares Majority Voting (MV), THINK and Prob-Merge AIME'25 for three reasoning models and different numbers of samples K. Across all configurations, Prob-Merge is consistently weaker than THInKMERGE (logit-level merging) and often even underperforms MV, especially for the weaker RI RI-Distill-Qwen-7B model, where performance degrades sharply as K grows (e.g., 47.3 for MV vs. 32.7 for Prob-Merge at K-4). In contrast, logit-level DirectMerge either matches or improves upon majority voting in most settings. These trends suggest that, for rea- soning models, aggregating before normalization more robust than averaging already- normmlizd probabilities. Besides, from an implementation perspective, merging over probabilities also conflicts with the standard logit-proue agittprrceesor interface (e.g., top-k/top-p filtering) modern inference frame- works such as vLLM, which operate directly logits.
6 CONCLUSION
In this work, we introduce THINKMERGE, training-free parallel test-time scaling for open-ended reasoning. Given a prompt, we sample K diverse reasoning traces up delimiter, then decode single answer by averaging next-token logits across traces at every step; the chosen token is fed back to all contexts so the ensemble continues to guide subsequent tokens. THINKMERGE preserves compatibility with standard decoding controls and integrates naturally with modern inference stacks (e.g., vLLM, SGLang), making easy to deploy for both online serving and offline batch decod- ing. Empirically, on closed-ended math/science QA, the proposed method is competitive with, and sometimes exceeds, majority voting. On open-ended tasks, LiveCodeBench and deep research tasks ThinKMerGE improves overall p of several models without additional training.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This material is based upon work supported by the Air Force Office of Scientific Research under award number FA2386-24-1-4011 and this research is partially supported by the Singapore Ministry of Education Academic Research Fund Tier (Award No. TI 25 251RES2509).
REFERENCES
Pranjal Aggarwal, Aman Madaan, Yiming Yang, et al. Let's sample step by step: Adaptive- consistency for efficient reasoning and coding with llss. arXiv preprini arXiv:2305. 18600, 2023.
Alibaba-NLP. Deepresearch: Tongyi deep research, the leading open-source deep research agent. htw.aaabbaa..teeeeeeeeeeerae
https://githun www/Alibabaaaaaaaaooooaaaaaaaaaa NLP DeepResear ch, 2025. GitHub repository.
Anthropic. Claude code: Command line tool for agentic coding, 2025. URL https: //ocss claude. contte/comm/////e/ee http///ccccc//cc/// claude cwweeoooeoeeeeeeeeeo overview. Accessed: 2025-01-20.
Mislav Balunovié, Jasper Dekoninck, Ivo Petrov. Nikola Jovanović, and Martin Vechev. Math- arena: Evaluating Ilms on uncontaminated math competitions, February 2025. URL https:
/ //matharena ai/.
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.2 /787, 2024.
Kaiyuan Chen, Yixin Ren, Yang Liu, Xiaobo Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, al. xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations. arXiv preprint 15Xi:2550.11655 2025a.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like llms. arXiv preprint arXiv:2412.21117 2024.
Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311, 2023.
Zhipeng Chen, Yingqian Min, Beichen Zhang, Jie Chen, Jinhao Jiang, Daixuan Cheng. Wayne Xin Zhao, Zheng Liu, Xu Miao, Yang Lu, et al. An empirical study eliciting and improving rl-like reasoning models. arXiv preprint arXiv:2503. 045448 2025b.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, et The danger of overthinking: Examin- ing the reasoning-action dilemma in agentic tasks. arXiv preprint arXiv:2502.008235, 2025.
drisspg. Vllm FlexAttention work tracking. https: http://gghttuuuu com/ wwwoo//vinnn project/ vwwm/issues /19655, June 2025. GitHub issue #19765, vLLM Project.
Darren Edge Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph rag approach to query-focused summarization. arXiv preprint 1rXiv:2404.161333 2024.
Lin Gui, Cristina Garbacea, and Victor Veitch. BoNBon alignment for large language models and the sweetness of best-of-n sampling. In The Tiihth-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https:ttt...... /pwpeeeeevveeeeeeeeeeeeeeeeeeeee ew net. /f forum? i d= SKM1 rbX5.
Daya Guo, Dejian Yang. Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 22.2215119193 12948, 2025.

---

Published as a conference paper at ICLR 2026
Jiaxin Guo, Daimeng Wei, Yuanchang Luo, Shimin Tao, Hengchao Shang, Zongyao Li, Shaojun Li, Jinlong Yang, Zhanglin Wu, Zhiqiang Rao, et al. M-ped: Multi-prompt ensemble decoding for large language models. arXiv preprint arXiv:2412.18299 2024.
Geoffrey E. Hinton. Products of experts. In Proceedings of the 9th International Conference on Artificial Neural Networks (ICANN '99), volume 1999, pp. 1-6. IEE, 1999. ISBN 978-0-85296- 721-8. doi: 10. 10.1049/cp119991075
Siyuan Huang, Zhiyuan Ma, Jintao Du, Changhua Meng, Weiqiang Wang, and Zhouhan Lin. Mirror- consistency: Harnessing inconsistency in majority voting. arXiv preprint arXiv:2410.1085 2024.
Hugging Face.
Transformers flex attention implementation. https://githar
com/ http:///infcceettttsssorereeesse integratiions/fleeaatteennttttt py, 2024. Part of the Transformers library.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and lon Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07977 2024.
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. Lim-blender: Ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 141758 2023.
Carlos E Jimenez, John Yang, Alexander Wettig. Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimban. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024. URL https: haapp:s///pereeee /pperevvee..eeeeeeeeeeeee net/ forum? 1d=VTF ByNQM66.
Tim Knappe. Ryan Luo Li, Ayush Chauhan, Kaylee Chhua, Kevin Zhu, and Sean OrBrin Enhanc- ing language model reasoning via weighted reasoning in self-consistency. The 4th Workshop on Mathematical Reasoning and Al at NeurlPS 24, 2024. URL
forum? 200==220022210
Woosuk Kwon, Zhuohan Li. Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Prin- ciples, pp. 1116 2023.
Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baix- uan Li, Zhengwei Tao, Xinyu Wang. et al. Websailor: Navigating super-human reasoning for web agent. arXiv preprint #xXi:2507700292 2025a.
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yongkang Wu, Ji-Rong Wen, Yutao Zhu, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.2177, 2025b.
Zichong Li, Xinyu Feng, Yuheng Cai, Zixuan Zhang, Tianyi Liu, Chen Liang, Weizhu Chen, Haoyu Wang, and Tuo Zhao. Lims can generate better answer by aggregating their own responses. 2025c.
arXiv preprint ar wwxx:2503304000
Run-Ze Liu, Jing Gao. Jing Zhao, Kaiyan Zhang, Xiang Li, Biao Qi, Wen-Qiang Ouyang, and Bo- Wen Zhou. Can 1b llm surpass 405b Ilm? rethinking compute-optimal test-time scaling, 2025.
Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaoxiang Shi. Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Er- ran Li, Raluca Ada Popa, and Ion Stoica. Deepcoder: A fully open-source 14b coder at 03-mini level. ht:ps://pretty- o-ddido-b5.... www.7......i... www.ioonnnsitt..... -ateer--ttt---mi-----e -nini--avvll-cfff 122148800bbbeee33333333333322 2025. Notion Blog.
MAA. American invitational mathematics examination (aime). https: / /ww./r//aaaaaaa///////..//./////
maa - m www.iiattiiioni 1tt.tattttiiiaaooeeeaeeememm -compet onn 2025. Accessed: 2025-08-19.
12

---

Published as a conference paper at ICLR 2026
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. In The Tweifth International Conference on Learning Representations, 2023.
OpenAl. Openai codex, 2021. URL hwtpt::ttpppeeteee ccom/oodex.c..////oooo////oo Accessed: 2025-01-20.
OpenAl. Introducing openai ol preview, 2024. URL
Accessed: 2025-02-14.
PyTorch Team. FlexAttention Part II: FlexAttention for Inference. https://pytores orgg/ blog/ flexattentionn-Feee ttt.tteeeiioo-oo-oieeeeeeeeeeeee inffrenneeee.e.ee 2025. PyTorch Blog. Accessed: [insert date].
Jianing Qi, Xi Ye, Hao Tang, Zhigang Zhu, and Eunsol Choi. Learning to reason across parallel samples for IIm reasoning. arXiv preprint arXiv:2506.0900 2025.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-lovel google-prood q&a bench- mark. In First Conference on Language Modeling, 2024.
Serper.dev. Serper: The world's fastest and cheapest google search api. https: dev/. Accessed: 2025-11-18.
Chufan Shi, Haoran Yang. Deng Cai. Zhisong Zhang, Yifan Wang, Yujiu Yang, and Wai Lam. A thorough examination of decoding methods in the era of llms. arXiv preprint arXiv:2402.06925, 2024.
Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth Interna- fional Conference on Learning Representations, 2025. URL https: httpps://ppeereeeeee /ppnnevieeeeeeeeeeeeeeeeeeeeeeee .net// Faw.mmidd-FFA22---
Benedikt Stroebl, Sayash Kapoor, and Arvind Narayanan. Inference scaling laws: The limits of Ilm resampling with imperfect verifiers. arXiv preprint arXiv:2411.1750/ 2024.
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hamjic Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503. 164 (9,19,0 2025.
Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang. Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. Fast best-of-n decoding via speculative rejection. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. URL https:/ openreview.ned ftrr//ormmmmm
Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, and Tianyi Zhou. Wait, we don't 't need to" wait"! removing thinking tokens improves reasoning efficiency. arXiv preprint arXiv:2506. 08343, 2025.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. self-consistency improves chain of thought reasoning in language models, 2022.
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese Browsecomp: simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504. /2516, 2025.
Rachel Wicks, Kartik Ravisankar, Xinchen Yang, Philipp Koehn, and Matt Post. Token-level en- sembling of models with different vocabularies. arXiv preprint arXiv:2502.21265. 2025.
Thomas Wolf, Lysandre Debut, Victor Sanb, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- ger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: of natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in
13

---

Published as a conference paper at ICLR 2026
Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. As- sociation for Computational Linguistics. URL https::/ //ttp:::/www... aclweb. httpb/...gggaatoooggg.ooo 2020. 1emanls---ess...
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for Ilm problem-solving. In The Thirteenth International Conference on Learning Representations. 2025a.
Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Under- standing chain-of-thought length in Ilms. arXiv preprint arXiv:2502.07266 2025b.
Yangyifan Xu, Jinliang Lu, and Jiajun Zhang. Bridging the gap between different vocabularies for Ilm ensemble. arXiv preprint MxXi::2404.09492, 2024.
Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Xingzhang Ren, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, and Jiancheng Lv. Dynamic voting for efficient reasoning in large lan- guage models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 3085-3104, 2023.
Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujic Zheng. Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yugiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXiv preprint arXiv:2505.09388. 09388, 2025a.
John Yang, Carlos E. Jimenez, Alex L. Zhang, Kilian Lieret, Joyce Yang. Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R. Narasimhan, Diyi Yang. Sida Wang. and Ofir Press. SWE-bench multimodal: Do ai systems generalize to visual software domains? In The Thirteenth International Conference on Learning Representations, 2025b. URL https: // //wpnneeviw..... net www.://oom/ddddiiiiiiieo.oo
Shu Yang, Junchao Wu, Xin Chen, Yunze Xiao, Xinyi Yang, Derek FF Wong, and Di Wang. Un- derstanding aha moments: from external observations to internal mechanisms. arXiv preprint arXiv:2504.0299 2025c.
Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Yunhua Zhou, and Xipeng Qiu. Revisiting the test-time scaling of ol-like models: Do they truly possess test-time scaling capabilities? arXiv preprint 2502 12215, 2025.
Wenting Zhao, Pranjal Aggarwal, Swarnadeep Saha, Asli Celikyilmaz, Jason Weston, and Ilia Ku- likov. The majority is not always right: RI training for solution aggregation. arXiv preprint arXiv:2509.06870 2025.
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs. Advances in neural information processing systems, 37: 62557-62583, 2024.
Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al. Browsecomp-zhe zh: Benchmarking web browsing ability of large language models in chinese. arXiv preprint arXiv:2504. 19314, 2025.
14

---

Published as a conference paper at ICLR 2026
A MORE EXPRIMENT DETAILS
A. SAMPLING HYYPER-PARAMETE
For closed-ended benchmarks across all tested models, set temperature=0.6 and top-p=1.0 (i.e., no probabilistic truncation). For open-ended LiveCodeBench, we use model-specific settings: for -oween-Code-330--A33
we set temperature=0.7 and top-p=0.8 (because it non- reasoning model). For all other models, we set temperature=0..6 and top-p=0.95.
A.2
EXPERIMENTAL RESULTS WITH STANDARD DEVIATION
Table 7: Baseline (Majority Voting) on AIME 2025. Results are mean + std. For each n,
majority @ K is evaluated under three combination strategies.
Model
Direct- Direct-Voting Early-Ready Shortest-K Merge
K = 2
Qwen3-4B
0.633 + 0.047
0.667 + 0.047
0.707 + 0.032
Qwen3-14B
0.707 + 0.025
0.720 + 0.034
0.753 + 0.045
DeepSeek-RI-Distin -waill-
0.407 0.049
0.407 + 0.049
0.507 0.025
K =4
Qwen3-4B
0.680 0.034
0.727 + 0.033
0.753 + 0.017
Qwen3-14BB
0.733 + 0.021
0.760 + 0.025
0.780 + 0.034
DeepSeek-Ri-Disilll--eeee
0.473 + 0.044
0.473 + 0.044
0.520 + 0.016
==8
Qwen3-4B
0.687 + 0.054
0.733 + 0.047
0.753 + 0.017
Qwen3-144
0.740 + 0.025
0.774 0.013 0.467 + 0.021
0.800 = 0.021
-eesSeek-R---issil---eeee
0.467 = 0.021
0.527 0.033
Table THINKMERGE on AIME 2025, Results are mean + std under four combination strategies.
Model
Direct-Merge Suffix Trimming Early-Reay Shortest-K Merge
K 2
Qwen3-4B
0.667 0.059
0.660 + 0.068
0.667 = 0.027
0.653 + 0.017
Qwen3-14B
0.720 + 0.016
0.727 4 0.033
0.733 + 0.021
0.747 + 0.034
DeepSeek-RR RI Ristiili Distill-Qwen-T1 Qwen-7B 0.413 0.034
0.420 0.034
0.413 = 0.034
0.480 + 0.050
K =4
Qwen3-4B
0.720 0.016
0.680 + 0.031
0.727 + 0.033
0.693 + 0.025
Qwen3-14B
0.720 0.034
0.733 = 0.021
0.733 = 0.021
0.760 = 0.025
DeepSeet-R--Dstill--veee
0.460 0.033
0.460 0.039
0.453 + 0.027
0.507 + 0.033
=
Qwen3-4B
0.700 + 0.052
0.687 + 0.016
0.708 + 0.043
0.727 + 0.033
Qwen3-14B
0.780 0.045
0.733 4 0.021
0.780 + 0.034
0.787 1 0.016
DeepSeck RI- Distill
Distill-Qwen-P1111 Qwen-7B 0.453 0.045
0.480 + 0.054
0.453 = 0.034
0.527 + 0.025
15

---

Published as a conference paper at ICLR 2026
Table 9: Tempeeature study of THINKMERGE on AIME: parallel thinking uses the officially sug- gested temperature; the answer phase uses smaller T-3.3. Results are mean = std.
Model
Direct-Merge Early-Ready
Shortest-K Merge
K ===
Qwen3-4B
0.647 + 0.062
0.673 + 0.039
0.667 + 0.021 0.740 + 0.083
Qwen3-144
0,713 - 0.026
0.733 + 0.021
0.413 + 0.027
0.413 + 0.027
0.487 + 0.054
K=
Qwen3-4B
0.700 0.037
0.720 + 0.034
0.707 = 0.032
Qwen3-14B
0.727 + 0.025
0.727 + 0.033
0.753 + 0.034
DeepSeek-RI--issill---eees
0.427 + 0.039
0.433 + 0.042
0.520 + 0.016
K = 8
Qwen3-4B
0.687 + 0.034
0.693 + 0.039
0.707 + 0.025
Qwen3-14D3
0.747 0.017
0.767 + 0.021
0.773 + 0.025
DeepSeei-RI-DDstill--eeee
0.447 + 0.034
0.440 + 0.025
0.513 + 0.034
Table 10: GPQA - Baseline (Majority Voting). Results are mean + std. For each n, majority @n evaluated under three combination strategies.
Model
Direct- Direct-Voting Early-Ready
Shortest-K Merge
K ==2 2 Qwen3-4B
0.492 + 0.020
0.494 + 0.020
0.520 + 0.013
Qwen3-14B
0.608 + 0.015
0.608 + 0.015
0.634 + 0.021
DeepSeek-R1-Distill -waill-Qwen-7BB
0.449 0.025
0.474 + 0.009
0.449 0.011
K-4
Qwen3-4B
0.512 0.022
0.514 + 0.018
0.537 + 0.011
Dwen3-14B
0.640 + 0.012
0.640 + 0.012
0.652 = 0.007
DeepSeek-RI-Ditill--oe -ween--
0.503 + 0.015
0.503 + 0.013
0.502 + 0.010
K 8 8
Qwen3-4B
0.533 + 0.025
0.533 + 0.025
0.553 - 0.013
Qwen3-14B
0.639 = 0.011
0.641 + 0.011
0.659 + 0.008
DeepSeek-RI --i---R-DD-iiI---iee
0.525 0.021
0.525 + 0.020
0.529 + 0.019
Table 11: GPQA - Our method THINKMERGE, Results are mean + std under four combination
strategies.
Model
Direct-Merge Suffix Trimming Early-Ready Shortest-/K Merge
K 2
Qwen3-4B
0.503 0.016
0.500 + 0.018
0.503 + 0.016
0.526 + 0.006
Qwen3-14B
0.616 - 0.008
0.614 0.008
0.621 = 0.004
0.637 + 0.010
DeepSeck-R--Diaill--ee----
0.492 + 0.020
0.492 4 0.020
0.490 + 0.022
0.438 1 0.018
K = 4
Qwen3-4B
0.522 0.008
0.522 0.009
0.524 0.007
0.518 + 0.009
Qwen3-14B
0.630 1 0.012
0.627 4 0.014
0.624 0.010
0.638 1 0.013
DeepSock-R--iistil--eeeees Mistill-Qwen-711 0.489 0.010
0.490 + 0.009
0.489 # 0.010
K 88
Qwen3-4B
0.516 0.017
0.513
0.015
0.513 1 0.022
0.538 1 0.013
Qwen3- 14B
0.641 - 0.013
0.630 + 0.011
0.641 + 0.011
0.637 + 0.011
DeepSeek-RI- - R--ii--illillll--e---- 0.500 0.021
0.500 1 0.021
0.502 1 0.022
0.472 1 0.009
16

---

Published as a conference paper at ICLR 2026
A.3 LIVECODEBENCH EASY MEDIUM/ HARD LEVEL PERFORMANCE
Table 12: LiveCodeBench Easy Pass 1 (%). Row-wise best among merge settings is highlighted. The best score within each category is bold; the tier is underlined. Model
Baseline
Direct-Merge
Shortest-K K Merge
K=8
K=1
K=2
K=8
K=1
K=
DeepCoder-14ke 14B-eee-ieB-Preeeeee
98.77
98.77
96.30
97.53
97.53
98.77
97.53
Qwen3-8B
97.53
95.06
98.77
97.53
96.30
95.06
97.53
Qwen3-Coder -ode---ood--
90.12
93.83
90.12
87.65
90.12
88.89
93.83
Qwen3-4B-Thinking
98.77
98.77
98.77
97.53
97.53
97.53
97.53
Qwen3-Think-30A3B
98.77
98.77
98.77
97.53
98.77
98.77
98.77
Table 13: LiveCodeBench Medium Pass@) 1. Row-wise best among merge settings is highlighted. Model
Baseline
Direct-Merge
Shortest-At A Merge
K-8
K=1
K-2
K=8
K=4
K-2
DeepCoder-14P 14B----vi-eeePe
69.90
66.02
71.84
75.73
76.70
75.73
77.67
Qwen3-8B
71.84
63.11
66.99
68.93
69.75
67.96
66.99
36.89
42.72
34.95
41.75
41.75
38.83
37.86
Owwn3-4B-Thinking
76.70
69.90
75.73
76.70
72.82
76.70
75.73
Owen3-Think-30AA.
82.52
81.55
84.47
80.58
78.64
77.67
84.47
Table 14: LiveCodeBench Hard Pass (%). Row-wise best among merge settings is highlighted.
Model
Baseline
Direct-Merge
Shortest- K Merge
K=8
K=4
K=2
K=8
K=4
K=2
DeepCoder-14B 14B-11evie-rrr
20.69
25.52
24.83
24.14
26.21
26.90
28.97
Qwen3-8B
24.14
22.76
25.52
31.72
28.97
26.90
29.66
Qwen3-Codee #wen3-Coder-30AA 30d3
8.97
11.03
11.72
11.72
10.34
9.66
9.66
Qwen3-4B-Thinde Thinking
34.48
33.10
31.72
33.10
35.17
36.55
36.55
Qwen3-Think-30A.
43.45
42.07
40.69
37.24
42,76
42,76
48.28
17

---

Published as a conference paper at ICLR 2026
A.4 CLOSE-ENDED TASKS: MAJORITY VOTING SATURATES QUICKLY WITH N
Majority@&
Majority@N
sec: Auudi
100
Accurscy (ho
70
Accurscy (%)
70
60
50-
Owerdaas PPasiann
Owend 18 (Maj0ss
Ower1 48 (hossgn)
Owend-144 LeB (MU,000)
Qwen3-48 (MoggH]
DeepSeen-Kit R1-71 (Pasggg))
Qwann -4 (H41g30
GeecSieecl1- (Fasugho
(
Despoeck (Mage))
thasert
Dossbeox (haggh)
Figure 6: On Close-edned task AIME'25 and GPQA, majority voting saturates quickly with N.
B LLM USAGE
We used large language models (ChatGPT and Gemini) as writing and formatting assistants. In particular, helped refine grammar and phrasing, improve clarity, and suggest edits to figure/table captions and layout (e.g., column alignment, caption length, placement). The LLM did not con- tribute to research idcation, experimental design, implementation, data analysis, or technical content beyond surface-level edits. All outputs were reviewed and edited by the authors, who take full responsibility for the final text and visuals.
18


Reviews:
[{"review_id": "tvvyyrCmUe", "reviewer": "Reviewer_f3xd", "summary": "Authors present ThinkMerge which averages logits over multiple parallel reasoning traces to enable majority voting over open-ended QA. They outperform baselines on AIME, GPQA, and even LiveCodeBench.\n\nFirst, they demonstrate that Pass@N indeed goes up for all models on all tasks, both open- and close-ended. This serves as a basis for the assumption that majority voting should benefit performance. Further, they find that these gains do come from the model gaining “capability” on difficult questions as inference scale grows.\n\nTheir approach follows a map-reduce paradigm. First, they sample K CoTs and then left-pad them so that the end of the CoT (the `</think>` token) in each seq is aligned. To do this they autoregressively have each CoT effectively “vote” token-by-token by averaging the pre-softmax logits that come from each chain. \n\nThey considered a few other alternatives before landing on this approach. Across most tested models, their method outperforms simple majority voting when the chains are allowed to fully expand, rather than only using the shortest K, for AIME and GPQA. Additionally, on the open-ended livecodebench, their method consistently outperforms the baseline, and this shortest K early stopping approach still delivers gains. This is useful, as it means that potentially the method can be made more efficient for open-ended tasks.", "questions": "I think this is a pretty solid paper. What do you think about the weakness suggested?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Clear implementation of a straightforward approach to improve reasoning model performance.\n\nMethod delivers a clear improvement\n\nWell-described and easy to followN/A I think this is a solid paper.", "weaknesses": "No discussion of closed models here. I’m curious, how well does this close the gap between the open and closed models?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j0B8ZcZkWT", "reviewer": "Reviewer_CwCp", "summary": "This paper introduces ThinkMerge, an alternate to majority voting paradigm. Instead of doing full rollouts and then majority vote, the authors propose doing parallel rollouts only for the \"thinking\" tokens, and then decoding from there on by averaging the token logits at each auto-regressive step, and sampling the next token from the resulting distribution. The proposed method neatly fits into widely used inference engines such as vLLM and SGLang, and is compatible with other decoding choices such as top-p/top-k/penalty etc.", "questions": "Suggestions:\n\n- This work can be a good empirical contribution but would require some more work, especially around analysing which scenarios benefit from fusion approach and investigating it's limitations and failure modes. This is important for the ICLR communities oriented towards practicable solutions. \n- Since the emphasis is on open-domain reasoning, include more benchmarks such as Terminal Bench, other coding benchmarks etc.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Strengths: \n- The proposed method is quite simple to implement, doesn't require any training, and practically deployable on popular inference engines. \n- The technique addresses a limitation of majority voting in open domains, where the response is free form text (code/reports etc) where \"majority vote\" is ill-defined. \n- The four ablations seem well-motivated, and grounded in practical considerations.", "weaknesses": "Weaknesses:\n- My major concern is with the limited innovation/novelty. The proposed technique is a straight forward application of product of experts ensembling (See [1]). Within the context of application in LLMs, there is prior work in this area: \n[2] applies the exact same technique of token fusion in the context of having several prompt variants. They too generate logits by processing each prompt and then averaging the logits at each auto-regressive step. ThinkMerge can be thought of as a special case of this where each prompt is a reasoning trace generated before token fusion. A more general problem of fusion across LLMs with different vocabularies has also been explored by [3, 4]. So the novelty seems to be limited to applying this technique with parallel `<think>...</think>` generations. \n\n- The proposed fusion is naive, and can potentially fail in scenarios where the reasoning traces diverge too much. Suppose, that across parallel generations, the model explores entirely different strategies to solving the problem, the ensembling can collapse. This is not sufficiently explored in the paper. \n\n- Token probability ensembling literature needs to be covered more in Related section [2,3,4,5].\n\n- While the paper emphasizes open domain tasks, it's experimentally validated only with a single benchmark, LiveCodeBench. \n\n\n[1]: Product of Experts - Geoff Hinton\n[2]: M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models - Guo et.al, CORR 2024\n[3]: Bridging the Gap between Different Vocabularies for LLM Ensemble - NAACL 2024\n[4]: Token-level Ensembling of Models with Different Vocabularies - Rachel Wicks et.al (arxiv)\n[5]: Llm-blender: Ensembling large language models with pairwise ranking and generative fusion - Jiang et.al", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RlMHNnrt3Q", "reviewer": "Reviewer_bs9g", "summary": "The paper proposes THINKMERGE, a training-free way to make parallel chain-of-thought work for open-ended tasks where majority voting can’t, by averaging token-level logits from K parallel traces after the reasoning delimiter and decoding a single shared answer. On AIME/GPQA it matches or slightly improves over standard parallel CoT with voting, and on LiveCodeBench it gives clear pass@1 gains where voting fails. I’m slightly reserved about the reliance on logit access / synchronized delimiters and the fact that some gains on closed-ended tasks are modest, but the core idea is clean and the evaluation is broad enough to justify it. \n\nI would have preferred if the code was supplied to have a hand-on experiment with the approach, but I have not had time to reimplement it, however, theoritically it is intact.", "questions": "Please refer to weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Addresses a real gap: parallel CoT doesn’t work for open-ended outputs; this makes it usable there.\n- token-level logit averaging after a shared delimiter, easy to plug into vLLM/SGLang.\n- Evaluates on both closed-ended (AIME, GPQA) and open-ended (LiveCodeBench), so it’s not overfitted to math-only.\n- Keeps contexts synchronized while decoding, which is a nice engineering detail for practicality.", "weaknesses": "- This looks to be delimiter-dependent. It assumes clean think/answer separation; models that ramble or reflect will hurt alignment.\n- On AIME/GPQA it sometimes only matches or even loses to majority voting.\n- I’m skeptical about pure logit averaging, since it can dilute a minority-but-correct trace. A simple reweighting or confidence-based scheme might help; at minimum I’d like to see an ablation showing how often correct traces get downvoted by the ensemble\n- The method gives clear gains at small K (e.g. K=2), but increasing K further doesn’t reliably improve pass@1 and can even regress, likely because divergent traces interfere when logits are averaged. I think the approach is effective, but scaling k for more benefit is not working as expected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1ySTfPEQkH", "reviewer": "Reviewer_SGtm", "summary": "This paper proposes a new way to deal with open-ended questions with LLM, named ThinkMerge, in which authors used majority voting, map-reduce, etc.", "questions": "I have no questions, as the writing is very clear.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The paper is well structured and clearly written. I was able to easily understand the ideas the authors intended to convey.", "weaknesses": "I believe the primary issue lies in the lack of novelty. Although the paper claims to focus on open-ended questions, the datasets used, such as AIME and GPQA, are predominantly composed of closed-ended questions. This mismatch undermines the validity of the problem statement from the outset.\n\nSecondly, the proposed method, described as \"think merge,\" appears to be essentially a form of majority voting. The use of a map-and-reduce framework is also not new, as similar approaches can be found in many prior works, such as GraphRAG.\n\nIn terms of experiments, I don’t see any significant improvement on genuinely open-ended questions. Overall, in its current form, I don’t find the paper acceptable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)