Generalized Parallel Scaling with Interdependent Generations
Authors: Harry Dong
David Brandfonbrener
Eryk Helenowski
Yun He
Mrinal Kumar
Han Fang
Yuejie Chi
Karthik Abinav Sankararaman
Venue: ICLR 2026 Poster (2026)
Keywords: large language model
inference
scaling
reasoning
reinforcement learning
post-training
attention
tensors
Primary area: foundation or frontier models, including LLMs
TL;DR: To generalize and enhance parallel inference scaling for LLMs, we introduce Bridge, an architectural addition to LLMs that allows parallel generations for the same input to share information with each other throughout the decoding process.

Abstract: Parallel LLM inference scaling involves sampling a set of $N>1$ responses for a single input prompt. However, these $N$ parallel responses tend to be generated independently from each other, partitioning compute resources and leaving potentially useful information in one generation untapped by others. This is in contrast to response length scaling where past computation is used in all future steps. For higher quality responses and response sets, we propose Bridge to generate interdependent responses in parallel by rethinking batched LLM hidden states as holistic tensors rather than independent slices. With only a small amount (2.8\%-5.1\%) of new parameters, Bridge improves the relative mean accuracy gains from reinforcement learning with verifiable rewards by up to 39\% and boosts consistency of correct responses. Trained once, Bridge scales to any generation width, all with greater performance than independent generations, unlocking a more general mode of parallel scaling that effectively leverages information between sequences, compatible with any post-generation aggregation technique.

Published as a conference paper at ICLR 2026
GENERALIZED PARALLEL SCALING WITH INTERDEPENDENT GENERATIONS
Harry Dong 1.2 David Brandfonbremer' Eryk Helenowski'
Yun He Mrinal Kumar'
Han Fang' Yuejie Chi
Karthik Abinav Sankararaman'
1 Meta
Cammggie Mellon University
8 Yale University
harrydeandren cmu.
ABSTRACT
Parallel LLM inference scaling involves sampling a set of N > 1 responses for single input prompt. However, these N parallel responses tend to be generated independently from each other, partitioning compute resources and leaving poten- tially useful information one generation untapped by others. This is contrast to response length scaling where past computation is used in all future steps. For higher quality responses and response sets, we propose Br dge to generate inter- dependent responses in parallel by rethinking batched LLM hidden states holis- tic tensors rather than independent slices. With only small amount (2.8%-5.15 1)) of new parameters, Bri Bridge dge improves the relative mean accuracy gains from re- inforcement learning with verifiable rewards by up to 39% and boosts consistency of correct responses. Trained once, Bridge scales to any generation width, all with greater performance than independent generations, unlocking more general mode of parallel scaling that effectively leverages information between sequences, compatible with any post-generation aggregation technique.
INTRODUCTION
Scaling inference-time compute has given large language models (LLMs) substantial leaps in performance on difficult tasks. Many scal- ing methods concentrate resources to gener- ate a single high-quality response such as with chains-of-thought (CoTs) (Wei et al.] (2022) and decompositions of a problem into parallel substeps (Rodionov et al., 2025 Yang al. 2025b). However, there are also instances where a high-quality ser of responses for each input is needed, such as in the case of output synthesis, best-of-N selection, and synthetic data generation. Scaling this in parallel man- ner traditionally done by Monte Carlo sam- pling independent generations. Consequently, each generation is ignorant of the other rollouts, despite answering the same prompt. Indepen- dent generations for the same prompt leave po- tentially useful information derived from other responses unutilized, limiting the performance ceiling. In contrast, sequentially scaling CoTs ensures each sampled token can play role in the final output. Motivated by the potential of shared information across parallel generations stemming from the same prompt, we aim to leverage these interactions to enhance and generalize parallel inference scaling.
Prompt
Generation Hidden States
Hidden States
Responses
Bridge
find
First, to both sides...
followngg quadratic...
quaddratic has the form...
Features
F Feedforward
Let's recall the quadratic formula...
Tokens
Attention
Figure LLM hidden states are 3-D tensors, where attention and feedforward blocks explicitly transfer information between tokens and features, respectively. By instead treating parallel scaling generations as a single tensor rather than indepen- dent slices, our method, Bridge, operates along the batch axis, so that tokens from all sequences that share the same prompt share information throughout generation.
Work done during internship Meta

---

Published as a conference paper at ICLR 2026
There has been progress in integrating parallel dependence for inference, such breaking down reasoning steps into parallel paths (Rodionoy et al. 2025] Pan et al.j 2025a Jin et al., 2025 Yang et al. 2025b). There, parallel computation is funneled into single output, useful for generating one response but not a set of responses. Even so, they highlight the potential of mid-generation interactions between sequences. We seek extend parallel scaling with intendependence, which allows all N output sequences for one prompt to use all the compute and information available, not just single isolated partition. Thus, the challenge is finding a totally parallel method that uses N simultaneous threads to generate N responses with interdependence without extensive post- post-training.
Looking LLM hidden states operations reveals a clue to overcome these challenges (Figure ! For batch size B, sequence length S, and hidden dimension D. the hidden states per forward pass has 3-D shape x S x D. Attention and feedforward blocks blend information throughout each S x D slice with the batch axis kept independent. Even minor inter-sample interactions like Batch Normalizations (Toffe & Szegedy 2015) were substituted with Layer Normalizations (Ba et al.) 2016). While this is natural for heterogeneous batches where samples with wildly different inputs can be fed together without interference, parallel scaling, which draws many responses from a single input, exhibits uniquely homogeneous structure each output stems from the same input. Hence, there is the potential for useful information transfer during the generation process which we exploit.
We introduce Bri Bridge (Batch reasoning with interdependent generations), method that shares information across tokens that stem from the same prompt in a hatch for parallel scaling with in- terdependent generations. With minor architectural change LLMs, each token generated in batch can depend on tokens in other generation threads with the same prompt. In turn, our method improves reasoning performance evaluated both at the individual response level (accuracy) and re- sponse set level (coverage and G-Pass @ kiz (Liu et al. )025)). Furthermore, our method focuses on generation, so any
aggregation technique can be used. We push the following advancements for parallel scaling:
1. Parallelism with Dependence: Instead of generating in isolated silos, Bridge allows information to flow between sequences while maintaining complete generation parallelism. Thus, inference compute is pooled together for all tokens, rather than being partitioned. Bridge significantly increases the final performance after reinforcement learning with verifiable rewards (RLVR) on 12 benchmarks using multiple reasoning models.
2. Low Cost:
By adding only 2.8% to 5.1 % additional parameters, and warming up on a small supervised fine tuning (SFT) dataset (e.g. GSM8K (Cobbe et al. (02T)), Bri dge already significantly improves the effectiveness of RLVR.
3. Versatility:
Bri dge has no restriction on the width of parallelism and is robust to train- time and test-time width discrepancies. Trained once, all tested widths outperform indepen- dent generations in terms of accuracy, coverage, and consistency. Furthermore, Bridge does not rely on any heuristics or interventions at any point in the generation process.
Our extensive experiments multiple models and tasks show that Bridge effectively shares in- formation across multiple generations for the same input. For example, our method improves the relative benefit of RLVR compared to the next best method. With the same model, Bridge also increases the rate at which all responses to a single competition math problem are correct from 15.0% to 17.8%.
by 39% averaged over 7 math tasks,
Paper Organization. In Section 2. we cover relevant background on test-time scaling with an emphasis on parallel scaling. Then, we introduce Bridge in Section 3 detailing the algorithm, the training pipeline, and its implications. We demonstrate Br wridde's 's efficacy on a variety of reasoning datasets, evaluated both on sample-wise accuracy and on global response set quality in Section We go further and provide a thorough investigation of our method including varying the generation width, sequence length extrapolation, learned features, and output analysis Section 4.3
BACKGROUND & RELATED WORKS
We start with overview of test-time scaling methods for LLMs, emphasizing parallel methods. Although some methods combine parallel generations into one response, the problem of generating high-quality set of interdependent responses, which we aim to tackle, remains understudied.
2

---

Published as a conference paper at ICLR 2026
Test-time Scaling, In part due to the success of scaling LLM training (Kaplan et al. 2020 Hoff- mann et al.] 2022). there has been a growing interest in quantifying how far scaling LLM inference- time compute can push performance, especially on difficult reasoning tasks. The main axes of infer- ence scaling are generation length and the numher of generations. To scale generation length, I LLMs encouraged to produce long CoT's before arriving a final answer (Guo al.) 2025 Yang al.) 2025a Muennighoff et al. 2025) which is usually of higher quality than shorter CoTss In the case of extreme generation lengths, there appears to be diminishing or even negative returns, suggesting
limitation of current models to scale indefinitely along this axis (Gema et al. 2025). To scale along the number of generations axis, LLMs can output multiple responses for a single query, increasing the probability of a high quality response being generated (Brown et al.) 2024 Snell et al., 2024 Wu jet al., 2024. Manvi et al., 2024, Sun et al. 2024 Dong et al. 2025). However, independent gener- ations divide computational resources among themselves, oblivious to each other's progress, which leads to less significant performance gain with additional compute than length scaling (Mirtaheri et al. 2025). To promote parallel exploration during training, training with Pass @ k objective has shown promise which could be an interesting extension to our method (Chen et al. 2025b).
Post-generation Synthesis. Instead of selecting one response from pool of candidate responses, some works investigate ways to synthesize multiple responses together. One way to take an unweighted or weighted majority vote across responses (Wang et al.] 2022; Uesato et al., [022] Lightman et al. 2003 Li et al. 2023), but this geared mainly for discrete answers, and an effective synthesis of reasoning traces remains unclear. There are also approaches where multiple responses are concatenated and fed into an LLM to extract or combine information (Chen et al. 2023 Qi
et al., 2025 Zhao al. 2025). Our work's focus is the generation phase, many of these poxt-generation aggregation techniques can be seamlessly integrated.
Mid-generation Synthesis/running. There has also been some work in developing techniques to share information across outputs mid-generation. For instance, Hogwild! Inference (Rodionov al.) 2025) and Group Think (Hsu et al. 2025) share key-value caches across generation runs to collab- orate and decompose tasks into subtasks. Similarly, some methods concatenate outputs of parallel processes to aid in the main decoding thread (Pan et al. 2025a Jin et al., 2025 Yang et al. 2025b] Macfarlane et al., 2025). Training from scratch, ParScale (Chen al.] 2025a) fans outs an input into multiple paths within the model architecture then aggregates them to predict the next token. Whereas these previous methods funnel resources of N parallel processes to produce one output, method uses N parallel processes to simultancously generate N high quality outputs. This way, our design integrates inter-output dependency diddgenneaation while producing different responses simultaneously. flexibly suitable for post-generation synthesis, RLVR training, and synthetic data generation. Another line of work involves stopping unpromising outputs mid-generation to devote more resources other parallel generations (Fu et al., 2025] [uun al.) 2024). These works show excellent reductions in compute, and composing them with our method is of interest for future work.
High Order Tensors. Multiple works that have observed richer representations in tensors than their flattened matrix counterparts (Kolda & Bader, 2009 Papalexakis et al. (016). Converting this observation into tractable algorithms remains major area of research in theory (Tong et al.) 2022 Dong et al. 2023c] Luo & Zhang 2023 Qin et al., (025) and practice (Ho et al., 2019) Dong et al, 2023b). We take inspiration from these works to diffuse information throughout the entire LLM hidden states tensor instead of being constrained to matrix slices.
3 BRIDGE: CONNECTING GENERATION PATHS
Sharing information between samples diddgenneaation in the latent space gives rise to a couple tech- nical challenges. One is finding an effective and efficient way to achieve this. Attention and feedfor- ward blocks already pose serious static and dynamic memory bottlenecks, which we want to avoid accentuating while still improving accuracy. Second, we also need versatility to allow for any num- ber of parallel generations at test-time. Bridge overcomes these challenges with small attention- like blocks that fit into any LLM. We begin with a description of Bridge, its connections, and its implications in Section 3.T. followed by SFT warm up (Section 3.2 and RLVR (Section 3.3 details.
3

---

Published as a conference paper at ICLR 2026
Bridge
Batch
Norm
Prompt
Foodforward
Norm
Prompt g
Attention
Norm
Bridge
Attention Maeks
Architecture
Information Sharing
Figure 2: Our method design. (Left) A Bridge block and imput normalization layer are added after each feedforward block. (Right) A timestep's 's tokens stemming from the same input prompt attend to each other Bridge blocks, denoted by the arrows. Dotted arrows illustrate all the locations of information transfer to different sequences in Markovian fashion (token features only at the current timestep are shared to predict the next timestep's tokens). Attention is masked for tokens from different prompts and from completed generations. White squares are masked cells.
3.1 BRIDGE ARCHITECTURE
introduce Bridge, new transformer (Vaswani et al. 2017) block that introduces dependence between samples in a batch. At high level, Bridge performs attention between tokens, which share the same prompt and do not come from completed generations, a batch each timestep. We summarize method in Figure 2 and Algorithm
To start, we first describe self-attention layers. Define hidden states E BxSxD for batch size B, sequence length S, and hidden dimension D. Let [X]..,, and [X] be the b-th and s-th 2-D slices along the batch and sequence axes, respectively. Self-attention, parameterized by WA.0- WA. e p Dex and WAV. W E IR Dx Dvo, calculated independently for each sample
QA [ WA.
KA.5 = 1)b.,. WAKK.
VA, = X WAN;
[Attn(X))s
Softmax (MaskA (QA KT
X,b)) VA.
(1)
ERSKS
Bri dge blocks are similar, but attention between samples is calculated independently for each token index s. Letting WB,O, WB,K E Dx Dox and WB,v, WBO E Dx Dvo
QB.e = XX] WB.Q. KB., = [ WB.K. VB. = [x] R...W.Wa WB.V,
(eridge(x)).... Softmax (Maskg QB. KT B.,)) VB, WB.O-
ER I
There are 3 key differences between usual self-attention and Bridge beyond a transposition of X:
Instead of decoder mask, Bridge applies attention mask that omits attention kens from sequences stemming from different prompts and sequences that have completed generation. See Figure 2 for example.
No positional encoding is used to preserve sample position invariance.
Without attention previous tokens, Bridge's Markovian design does not maintain key-value cache.
We place Br i dge block after each feedforward block with residual stream and input normaliza- tion layer that mimics existing blocks, shown in Figure 2 Bridge is active during the prefill stage too, but since all hidden states for the same input are identical, Bri dge blocks act as linear layers.

---

Published as a conference paper at ICLR 2026
BFT Prompt
Promps
BFT Promot
Warmap Tran Sumple
BFT Promot
Warmup Training
Orignn Mocil
Figure 3: Warm up procedure. The original LLM generates candidate traces which are filtered by correctness and compiled into a dataset. SFT on this generated dataset only updates new parameters. The P-Match baseline substitutes Br Bridge blocks with MLPs matched in parameter count.
Connection to Efficient Attention for Tensors. Bridge unlocks the ability for an LLM to treat batch of LLM hidden states as 3-D (B x S x D) structure rather than stack of independent 2-D slices. In this way, the inputs are analogous to images, and the decoding process is like au- toregressively generating additional columns. With this interpretation, Br Bridge applying attention operations on different axes of an input is similar to axial attention (Ho et al. 2019) which was introduced first in computer vision accelerate encoder attention but has since seen wide success in various applications such as in medicine (Azad et al. 2024), materials science (Dong et al. 2023a). and algorithm discovery (Fawzi al. 2022).
Generation Interdependence. For B independent rollouts we sample the next token 0b,++1 from
p( [g, 1s,
for sample b, timestep input prompt q, and previously generated tokens 0b,1:-- With Bridge, the next token distribution becomes
p 1/g, {or 10=))
for each sample b. Conditioned on past tokens, Br bridge preserves independence between tokens at the same timestep. which allows next token sampling to still be performed in parallel:
(ob. 1 0b2. 1.= for + bz.
3.2 SFT WARM UP
While RLVR can be immediately applied with Bridge since these new blocks are initialized to have no contribution, we can also optionally warm them up with SFT for more sufficient training and better downstream performance. A desirable SFT dataset would include many reasoning traces to one prompt. To stay close to the original LLM's generation distribution, we create SFT datasets by first responding to prompts from an existing math dataset. Then, traces filtered for correctness. During training, these correct traces are fed together in the same batch to warm up Bridge blocks with SFT. All other parameters are frozen. Figure jillustrates the warm up procedure, and Table explores more in-depth on the benefits of warm up.
3.3 RLVR OBJECTIVE
We train LLMs with Bridge using GRPO (Shao et al. 2024). We use variant specified by Yu et al. (2025) which performs token-level normalization to reduce length bias. Letting the group size be G, the advantage of the i-th output o, to input q with reward r; is Å, r=-manllr for clipping threshold e, hyperparameter B, and policy no parameterized by 0, the objective
sul(ra....Fa) Then,
G
[(0)
1
min
clip
BBKKI. (Tol|Toss)
3KK..
}
|os|
im
sml
(3)
where
To 105,1:s-116
Now (oi,agg 10 ) (0,(
5

---

Published as a conference paper at ICLR 2026
Ten (o, {
1G j=1 Tom 1o
DKL (TT)
log
1.
To (orall (0 1)21))
no(0s,a1g foj.1:- )==1)
The key differences from the GRPO objective (and its variants) and our objective are not formulaic but rather inherently induced from the architecture of Bridge. Namely, the ratio and KL diver- gence terms now contain inter-sample dependence between relevant samples, breaking the original assumption of independent trajectories. By linking the advantages and logits in a group, the loss and gradients per output are intertwined with other outputs' that share the same prompt. In other words, gradients from all sequences, containing both positive and negative advantages, are back- propagated through each sequence because of Bri dge blocks. Further considerations with this setup are discussed in Appendix C Since Bri dri is just an architectural change, training is just limited to SFT and RLVR for reasoning problems. For instance, Br Bridge may also be applied for reinforcement learning from human feedback (RLHF) which is an interesting future direction.
EXPERIMENTS
We now showcase the benefit of Bridge across multiple models and math reasoning benchmarks. After describing our setup in Section 4.1, we first show that applying RLVR with Bridge blocks improves accuracy more than other methods. For instance,
with
Bridge blocks observes relative 39% further improvement with RLVR than the next best method (Section 4.2.1). Then, in Section 4.2.2, we demonstrate that Bridge also improves the output set quality across several metrics in terms of coverage and correctness consistency. Finally, in Sec- tion 4.3] we highlight some important characteristics of our method including the versatility of gen- eration width, length extrapolation, benefit of warm up, feature contributions, and output stability.
4.1 EXPERIMENTAL SETTINGS
Models and Baselines. We test Bridge on DeepSeek- PrepSeek-RI-Distilll RI--i--ii-----eeeeeee 1.5B. DeepSeek- R1- Distill- Shaill-Qwen-7B, and -eapseck-R--Disil-LLlaaane
which we abbreviate to DS- DS-Qwen- 1.5B, DS-
Qwen-7B, and DS-Llama-8B,, respectively (Dubey et al. 2024 Yang et al.] [024 Guo et al. 2025 We use 4 query and key-value attention heads for Bri dge. each with the same dimension as the original model's head dimension. This only adds 5.1%, 2.8%, and 3.4% extra parameters of the original DS-Qwen-1.5BB DS-Qwen-77e BB, and DS-Llama-8B models, respectively. Table 7 in Appendix B lists the exact parameter counts. Our parameter-matched baseline which we call "p- Match" adds 2-layer MLPs of the same size in the same positions as Bridge blocks which serves to show the limited effect of just adding parameters. Matched in parameter count, P-Match and Br i dge are also trained with the same warm up and RLVR pipeline. Both methods are initialized to have zero contribution.
top
Training. For the SFT warm up stage, we first use the original LLM to generate 8 response for each GSMSK (Cobbe et al. (02T) problem and then filter out incorrect responses and problems with one or fewer correct responses. We train only the additional parameters with Bri dgg and P-Match on this custom dataset for epochs and keeping the best checkpoint according to the perplexity on 500 validation problems (and their corresponding set of correct reasoning traces). This checkpoint is inserted in the model for RLVR where we train the full model on DeepScaleR-Preet Preview-Dataie (Luo et al. 2025) for 1000 gradient steps. DS-Qwen-1.5BB is trained with generation width while the others were trained with 4. The only reward is correctness of the generation. Our training hyperparameters are listed in Appendix A
Evaluation. We evaluate Bridge on 7 math benchmarks (MATH-500 (Hendrycks et al. 2021 Lightman et al. 2023). AIME24, AIME25 (AIME, (025). AMC23 (AMC) 2023). BRUMO25
(BRUMO 2025). CMIMC25 (CMIMC, 2025). and HMMT FEB25 (HMMT) 2025 and 5 chal-
lenging non- non-math benchmarks (XSum (Narayan et al. 2018), CNN/Daily Mail (Hermann et all 2015 See et al. 2017), GPQA (Rein et al. 2024), ZebraLogic (Lin et al.) 2025), and Countdown Pan et al. (025bb). Evaluating MATH-500 on every 100 training steps. the checkpoint with the highest validation accuracy is used to test on the remaining benchmarks. We evaluate across 4 re- sponses per MATH-500 sample and 32 responses per sample from the other benchmarks. Sampling temperature and top-p are set to 0.6 and 0.95, respectively. We set the generation width of Br ride

---

Published as a conference paper at ICLR 2026
Table I: Accuracy comparison across math benchmarks. In each section, the 4 rows from top to bottom are the performance of the original model, RLVR applied on the original model, P-Match (extra MLPs) with SFT warm up and RLVR, and Bridge with SFT warm up and RLVR. The rightmost columns show the average across all benchmarks and the average improvement over the original model. MATH-500, AMC23, BRUMO25, CMIMC25, and to MATH, AMC, BRU, CMI, and HMMT, respectively.
are abbreviated
Model
MATH
AIME24/25
AMC
BRU
CMI
HMMT
Avg
1 A
DS- DS-Qwen-1.5B
73.65
13.75 13.44
50.00
18.12
4.30
8.23
25.93
0.00
RLVR only
78.75
17.40 18.44
60.55
18.54
3.83
7.50
29.29
3.36
P-Match
78.65
18.12 19.17
60.62
20.94
5.08
8.54
30.16
4.23
Bridge
81.30
20.11 20.00
60.55
21.36
5.63
9.79
31.25
5.32
DS-Qwen-7B
82.15
23.44/21.88
66.02
23.75
5.63
11.98
33.55
0.00
RLVR only
88.15
29.06 23,85
74.30
28.33
7.97
12.60
37.75
4.20
P-Match
86.80
28.85 25.73
70.47
26.77
6.25
11.87
36.68
3.13
Bri Bridge
88.15
32.19 25.41
77.65
30.21
9.77
12.40
39.40
5.85
DS-Llama-88
73.40
15.42 13.12
57.97
15.62
2.73
8.23
26.64
0.00
RLVR only
76.70
18.12 18.12
63.44
15.83
5.47
10.52
29.74
3.10
P-Match
78.00
22.29
20.21
61.80
17.81
5.08
11.67
30.98
4.34
Bridge 80.15
24.76 18.18
66.36
19.91
6.02
11.93
32.47
5.83
to 8 for all tasks except MATH-500, which we set to 4 since we only evaluate on 4 responses per sample. We adapt our evaluations from the Lighteval framework (Habib et al. 2023)
4.2 REASONING PERFORMANCE
Here, we show the performance improvements of our method Br Bridge which leverages inter-sample information sharing for high quality generations. We evaluate performance both on per-output accu- racy (Section 4.2.1 and macroscopically, on the set of outputs generated per prompt (Section 4.2.2)
4.2.1 ACCURACY
Beginning with standard accuracy (Pass 1), compare the performance of the original model, original model with RLVR, P-Match with SFT and RLVR, and Bridge with SFT and RLVR on several math benchmarks. Results in Table show that in nearly all cases and average, Bridge obtains the highest accuracy compared to all other methods. In particular, the average performance improvements our method the original model is 26%, 39%, and 34% relatively of the next best method on DS-Qwen-l.5B, DS-Qwen-7B, and DS-Llama-8B) models, respectively. P-Match with parameter counts pegged Bridge improves accuracy from just pure RLVR most of the time but is much more inconsistent, such as in the case of DS-Qwen-7B. This indicates that the superior performance of Br rridge even though DS DS-Qwen-7B and DS-Llama-8B were trained with generation width 4, the evaluation results with width 8 are still stronger than the other independent sampling methods, showing the robustness of Bri dge. In addition, the improvement by Bri dge is greater for larger models, and scaling up to even larger ones remains of interest for future work. Although we train Br dge solely on math, observe degradation and sometimes improvement non-math tasks (Table
than that
solely attributed to additional parameters. Furthermore,
4.2.2 SET EVALUATIONS
Zooming out, we show Bridge also improves the consistency and coverage (i.e., the percentage of questions that have at least 1 correct response in the response set) across multiple generation attempts. To evaluate the set of responses to single input, we use the G-Pass@k, k, (Liu et al.) 2025) metrie, which paints more holistic picture of model potential (coverage) and consistency. correct output in à responses, G-Pass k, is the probability of 0 S fraction of & responses being correct. More formally, for responses and correct.
Whereas Pass@k k is the probability

---

Published as a conference paper at ICLR 2026
Table Evaluations on non-math tasks. Note that our training procedure only used math samples.
Rouge-1 (Lin 2004) scores are reported for summarization (XSum and CNN/DailyMail, Average accuracies are reported for GPQA, ZebraLogic,, and Countdown.
Model
XSum
CNN/DailyMail
GPQA
ZebraLogic
Countdown
DS- DS-Qwen-I.SB
15.72
22.11
33.14
30.90
28.77
RLVR only
14.81
22.79
32.45
30.90
28.15
P-Match
15.90
22.78
32.51
32.55
31.36
Bridge
17.17
24.07
33,90
33.15
34.84
DS- DS-Qwen-7B
18.03
24.19
43.94
40.00
49.55
RLVR only
17.24
23.52
43.56
41.25
49.93
P-Match
18.13
23.76
43.75
42.95
46.91
Bridge
18.16
24.55
45.77
42.60
52.70
DS-Llama-SB
18.23
23.84
35.80
41.25
14.04
RLVR only
2.25
1.65
39.46
43.25
29.23
P-Match
19.67
23.01
38.83
43.50
32.32
Br Bride
18.04
22.64
39.65
44.70
32.51
Grigall
Rak
15-
20
00.0001..00 D5-0Nen-TD 26-Ja11-00
DS- 15--0ee-1.98 C5 0w 25tiam-32
105-Owen-000 OS 0w0--7 DS
05-0m0135 25 Oeex3 OSllmmado
Figure 4: G-Pass@8.. averaged across AIME24, AIME25, AMC23, BRUMO25, CMIMC25, and SMMMTFFE225 Each chart measures the minimum number of correct answers (7 k) out of simultaneous responses, Bri sridge has the greatest coverage (7 k = 1) and answers correctly most consistently (r 1) the vast majority of cases. Higher is better.
responses,
(
Pass @k =
G-Pass = EE
As + 0, G-Pass@k;; is simply the coverage. On the other extreme, G-Pass @k the probability that all & responses are correct.
From Figure Br dge achieves higher G-Pass@8, values for nearly all values of r and models. This demonstrates that Bridge can achieve greater coverage without spreading out its responses to many incorrect answers. In other words, not only do Bridge blocks increase the probability of correct response in the response set more than the other methods, they also increase the frequency which they occur. Again, we note that Qwen-7B and DS-Llama-B0 were trained with generation width 4 yet they generalize well to evaluation width 8.
4.3 ABLATIONS AND ANALYSIS
Generation Width. The design of Bridge allows complete flexibility in the number of parallel generations, or generation width w, due to the removal of positional encoding. Here, we show its generalizability to other widths on DS DS-Qwen-77 which was trained on width of with RLVR. In Table 3 in all cases where w > 1, Bridge outperforms P-Match in terms of task-wise and global average accuracy. We also investigate the effect w on set quality in Figure 5 Again, generally see vast improvement upon the original model and P-Match with G-Pass @& settings. These results show not only the benefit of sharing information via Bridge but also the generalizability widths wider and thinner than its training width. At the extreme of w = 1, equivalent to independent generations, results average accuracy that falls between RLVR only and P-Match, indicating that Bri dge blocks do not harm independent reasoning.
for all
8

---

Published as a conference paper at ICLR 2026
Table 3: Accuracy across 32 samples of varying Bridge generation widths, w, with DS-Qwen- 7B which was trained at width with RLVR. A Bridge width of I is equivalent to independent generation. Tasks are abbreviated as described in T
Method
AIME24
AIME25
AMC
BRU
CMI
HMMT
Avg
?
DS-Qwen-7B
23.44
21.88
66.02
23.75
5.63
11.98
25.45
0.00
RLVR only
29.06
23.85
74.30
28.33
7.97
12.60
29.35
3.90
P-Match
28.85
25.73
70.47
26.77
6.25
11.87
28.32
2.87
Bridge (w
28.13
24.48
74.85
28.02
9.07
11.77
29.39
3.94
Bridge (w = 4)
31.57
25.63
76.93
28.65
10.16
13.13
31.01
5.56
Bridge (w 8)
32.19
25.41
77.65
30.21
9.77
12.40
31.28
5.82
Bridge (w = 16)
32.92
25.11
75.70
30.63
8.21
12.50
30.85
5.40
10
uridge (w
Bridge
G-Russgg Improveerrr
Bridge (w
Bridge (w 161
Correct out of
Figure 5: G-Pass@8, improvement upon the original DS-Qwen-7B model averaged across AIME24, AIME25, AMC23, BRUMO25, CMIMC25, and HOMMT_FEB22 with relation the eval- uation generation width w Bridge. The x-axis (T indicates the number of responses out of k that must be correct.
Generation Length. Bridge also shows strong generalizability along the length axis. We demonstrate its performance ing both individual and set performance. From Figure 6 our method scales smoothly and better than the other baselines in most cases. At the individual response level, method achieves the highest accuracy across all generation lengths. At the set level, Br Bridge blocks increase the number of that only had correct answers by 6.0% compared the next best 16K generation length, illustrating our method's 's consistency to generate correct answers.
extrapolate beyond its training length of 4096, again measur-
Cold Start vs. Warm up. Warming up Bridge blocks with SFT prior to RLVR outlined in Section 3.2 leads to improvements in performance, shown in Table The slight improvement implies that although it is prefered to warm up these new layers, is catastrophic if RLVR is applied directly from initialization.
Table 4: Accuracy comparison between cold start RLVR and RLVR with SFT warmed up Bridge blocks in DS-Qwen- 7B. Tasks are abbreviated as described in Table 16 responses were collected task except MATH-500 in which 4 were collected.
MATH
AIME24/25
AMC
BRU
CMI
HMMT
Avg
Cold Start
88.70
33.13 26.67
77.19
29.80
7.04
12.09
39,23
Warmed up
88.15
33.75 / 25.63
77.97
30.21
10.00
13.13
39.83
Feature Contribution. Having shown the improved performance brought by Bridge, we now briefly peer into the effect that it has on LLM hidden states. We measure this by finding the ratio between the output norm each block with the corresponding residual norm each token, with lower values suggesting relatively little effect the residual features (Figure (7). Surprisingly, we find Bridge blocks contribute little compared to its counterpart in P- P-Match, despite having a significant impact the performance.

---

Published as a conference paper at ICLR 2026
Accuracy
Coverage
2 out of Correct
4 out of Correct
70-
Orginn
24
206
16k
16C
18C
Sezuanee Langt:
Semutrec Langah
SAnpArIc Langan
Seqpuamoc Langs
Figure 6: From left to right, DS DS-Qwen-1.5B MATH-500 accuracy. coverage, G-Pass 4..,, and G- @ass@4, as generation length increases. We generate 4 responses per input.
P-Match
Bridge
Rudo (Outout: (Reidual)
Attantion Fecstorword Added MLF
Norm Ratin (Dulout (Rssttue))
Attenkion
101
Foodfooward
Bridge
100
166
L0-1
10
1.0
100
20.2
30
1.5
20
25
0
10 20
25
Layer
Layer
Figure 7: Ratio between feature norms of the block output and residual of every DS-Qwen-7B layer.
Output Stability. We additionally measure the effect of Br dge on the output tokens in Table 5 First, we find the average pair-wise BERTScores (Zhang et al. 2019) between MATH-500 responses, where higher scores indicate more similar output sequences. Our method has slightly higher BERTScore, meaning Bridge marginally increases output similarity but crucially does not collapse the distribution of outputs. Second, we measure the variance in the evaluation results for different responses to the same prompt. For this, we turn summarization tasks where the evaluation metric (Rouge) of single response is more fine-grained than the binary nature of math tasks. With the lowest variance, Bridge produces outputs with the most consistent quality.
DS-Qwen-7B BERTScores (F1) for MATH-500 and Rouge-1 variances of summarization tasks. Higher BERTSccres indicate greater similarity of outputs.
Table 5:
MATH-500 BERTScore CNN/DailyMail Variance
XSum Variance
DS-Qwen-7B RLVR only
89.93
16.14 14
21.21
90.06
27.96
33.15
P-Match Bridge
89.89
16.09
22.29
90.41
14.44
20.23
CONCLUSION
To generalize and enhance parallel inference scaling for LLMs, we introduce Bridge, novel and inexpensive architectural addition to LLMS that allows parallel generations for the same in- put to share information with each other throughout the decoding process. We demonstrate that our method improves both single sample accuracy and set-wise quality across multiple models and several reasoning tasks. We achieve this by rethinking hidden states in parallel scaling higher order tensors rather than disjoint slices. With this interpretation, this also plants the seeds for many exciting future directions such observing the the effect of Bridge blocks during pretraining or mid-training, post-training with a Pass@ k (Chen et al. (025b) or another global objective, and quantifying the benefit other modalities and tasks which can exhibit different levels of output homogeneity (Jain et al. (025). Such directions will push parallel scaling as much more effective axis of LLM inference scaling.
ACKNOWLEDGMENTS
We thank Bradley Brown for frequent discussions about this project.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
AIME,
AIME problems and solutions, 2025. URL utt.tttss//aatofocooeeeessootoo/
wiki/ /indaiindxx..e. 2hp/AIME Problems and Solutions
AMC.
American mathematics competition, 2023.
URL. httpp::/mmmt .argaaaa..oroaaa.oo.....oooooor org/
student- wwteentttroorammm/aaaaea amc/
Reza Azad, Ehsan Khodapanah Aghdam, Amelie Rauland. Yiwei Jia, Atlas Haddadi Avval, Afshin Bozorgpour, Sanaz Karimijafarbgloo,, Josepp Paul Cohen, Ehsan Adeli, and Dorit Merhof. Med- ical image segmentation review: The success of u-net. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv: 0.000006550 2016.
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint urXiv:2407.2 2024.
BRUMO. Brown university math olympiad 2025, 2025. URL http::tp///// /tttps://www... brumoorumooooooo www.urr/o/goo...oo......
Mouxiang Chen, Binyuan Hui, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Jianling Sun, Junyang Lin, and Zhongxin Liu. Parallel scaling law for language models. arXiv preprint arXiv:2505.10477 2025a.
Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.1331 2023.
Zhipeng Chen, Xiaobo Qin, Youbin Wu, Yue Ling, Qinghao Ye, Wayne Xin Zhao, and Guang Shi. Pass@ k training for adaptively balaneing exploration and exploitation of large reasoning models. arXiv preprint arXiv:2508. 20::::055.0.0.11. 2025b.
CMIMC. Carnegie mellon informatics and mathematics competition, 2025.
URL
https:
7/ 1/cmimc.nan mwwhh.a......mmmm cmu. edu/
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 1001:22110.14168 2021.
Harry Dong, Sean Donegan, Megna Shah, and Yuejie Chi. A lightweight transformer for faster and robust ebsd data collection. Scientific Reports, 13(1):11253, 2023a.
Harry Dong, Megna Shah, Sean Donegan, and Yuejie Chi. Deep unfolded tensor robust pea with self-supervised learning. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023b.
Harry Dong, Tian Tong, Cong Ma, and Yuejie Chi. Fast and provable tensor robust principal com- ponent analysis via scaled gradient descent. Information and Inference: A Journal of the IMA, 12 (3):1716-1758, 2023c.
Harry Dong, Bilge Acun, Beidi Chen, and Yuejie Chi. Scalable Ilm math reasoning acceleration with low-rank distillation. arXiv preprint arXiv:2505.07861, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint orXiv:2407.22 1783, 2024.
Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera-Paredes, Moham- madamin Barekatain, Alexander Novikov, Francisco R. Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, et al. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 4155, 2022.
11

---

Published as a conference paper at ICLR 2026
Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence, 2025. URL http:://axxx...ooo
Aryo Pradipta Gema, Alexander Hägele, Runjin Chen, Andy Arditi, Jacob Goldman-Wetzler, Kit Henry Sleight, Linda Petrini, Julian Michael, Beatrice Alex, et al. Inverse scaling in test-time compute. arXiv preprint arXiv:2507. 14417, 2025.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 129448888.1 2025.
Nathan Habib, Clémentine Fourrier, Hynek Kydliček, Thomas Wolf, and Lewis Tunstall. Lighte- val: A lightweight framework for Ilm evaluation, 2023. URL httts:://githtii com/
Srtgginnnfaceelihhteee
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021.
Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In NIPS, pp. 1693- 7701,2015
HMMT. Hmmt, 2025. URL/ttps:: www. hmmt org/
Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn, and Tim Salimans. Axial attention in multidi- mensional transformers. arXiv preprint 1912: 12180, 2019.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, al. Train- ing compute-optimal large language models. arXiv preprint arXiv:2203.15556 2022.
Chan-Jan Hsu, Davide Buffelli, Jamie McGowan, Feng-Time Ting Liao, Yi-Chang Chen, Sattar Vakili, and Da-shan Shiu. Group think: Multiple concurrent reasoning agents collaborating token level granularity. arXiv preprint arXiv:2505. 107, 2025.
Sergey loffe and Christian Szegedy. Batch normalization:
Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-456. pmlr, 2015.
Shomik Jain, Jack Lanchantin, Maximilian Nickel, Karen Ullrich, Ashia Wilson, and Jamelle Watson-Daniee Llm output homogenization is task dependent. arXiv preprint arXiv:2509.21267, 2025.
Tian Jin, Ellie Y Cheng, Zack Ankner, Nikunj Saunshi, Blake M Elias, Amir Yazdanbakhsh, Jonathan Ragan-Kelley, Suvinay Subramanian, and Michael Carbin. Learning to keep promise: Scaling language model decoding parallelism with learned asynchronous decoding. arXiv preprint arXiv:2502. 1/517, 2025.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 2020.
Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review, 51(3): 455-500, 2009.
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. Making language models better reasoners with step-aware verifier. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5315-
5333, 2023.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.22 20050, 2023.
12

---

Published as a conference paper at ICLR 2026
Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. Zebralogic: On the scaling limits of 1lms for logical reasoning. arXiv preprint arXiv:2502.01 100, 2025.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches oul, pp. 74-81, 2004.
Junnan Liu, Hongwei Liu, Linchen Xiao, Ziyi Wang, Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, and Kai Chen. Are your Ilms capable of stable reasoning?. 2025. URL https:
//aaxii.org/abs/22111 13147
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Tang, Manan Roongta, Colin Cai, Jeffiry Luo, Tianjun Zhang, Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing ol ol-preview with a 1.5b model by scaling rl, 2025. Notion Blog.
Yuetian Luo and Anru R Zhang. Low-rank tensor estimation via riemannian gauss-newton: Statis- tical optimality and second-order convergence. Journal of Machine Learning Research, 24(381): 1-48, 2023.
Matthew Macfarlane, Minseon Kim, Nebojsa Jojic, Weijia Xu, Lucas Caccia, Xingdi Yuan, Wanru Zhao, Zhengyan Shi, and Alessandro Sordoni. Instilling parallel reasoning into language models. In 2nd Al for Math Workshop @ ICML 2025, 2025. URL https:://peerrevieww Gorum?id=a3o4b333
Rohin Manvi, Anikait Singh, and Stefano Ermon. Adaptive inference-time compute: Lims can predict they can do better, even mid-generation. arXiv preprint arXiv:2410.02725, 2024.
Parsa Mirtaheri, Ezra Edelman, Samy Jelassi, Eran Malach, and Enric Boix-Adsera. Let me think! long ohain-of-thought can be worth exponentially many short ones. arXiv preprint arXiv:2505.21822. 2025.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, arXiv preprint arXiv:2501.19330 2025.
Shashi Narayan, Shay Cohen, and Mirella Lapata. Don't give me the details, just the sum- mary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint 1808 08745, 2018.
Jiay Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr. Learning adaptive parallel reasoning with language models. arXiv preprint arXiv:2504. 15466, 2025a.
Jiayi Pan, Junjie Zhang, Xingyao Wang. Lifan Yuan, Hao Peng, and Alane Suhr. Tinyzero. http://ggttubb.oom/laaaaa////// Pan.-aann//Tn.,.....m 2025b. Accessed: 2025-01-24.
Evangelos E Papalexakis. Christos Faloutsos, and Nicholas D Sidiropoulos. Tensors for data mining and data fusion: Models, applications, and scalable algorithms. ACM Transactions on Intelligent Systems and Technology (TIST), 8(2):1-44 2016.
Jianing Qi, Xi Ye, Hao Tang. Zhigang Zhu, and Eunsol Choi. Learning to reason across parallel samples for IIm reasoning, 2025. URL
09014
Zhen Qin, Michael B Wakin, and Zhihui Zhu. A scalable factorization approach for high-order structured tensor recovery. arXiv preprint arXiv:2506.1600 2025.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-level google-proof q&a bench- mark. In First Conference on Language Modeling, 2024.
Gleb Rodionov, Roman Garipov, Alina Shutova, George Yakushew, Vage Egiazarian, Anton Sinitsin, Denis Kuznedelev. and Dan Alistarh. Hogwild! inference: Parallel Ilm generation via concurrent attention. arXiv preprint arXiv:2504.06261 2025.
13

---

Published as a conference paper at ICLR 2026
Abigail See, Peter J. Liu, and Christopher D. Manning. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1073-1083, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10. 118553/1/P17--19999 URL https: 7/www. ac web. org/ htt.:o/gggaattoooo////1111100000
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.085 03314 2024.
Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang, Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. Fast best-of-n decoding via speculative rejection. arXiv preprint DrXiv:2410.20290 2024.
Tian Tong, Cong Ma, Ashley Prater-Bennette, Erin Tripp, and Yuejie Chi. Scaling and scalabil- ity: Provable nonconvex low-rank tensor estimation from incomplete measurements. Journal of Machine Learning Research, 23(163):1 2022.
Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song. Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint WxXv:2211.11275. 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- cry, and Denny Zhou. Self-connistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203. 111711 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24888 2022.
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. An empirical anal- ysis of compute-optimal inference for problem-solving with language models. arXiv preprint arXiv:2408..0077 2024.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint ar111:2412.15115. 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint #xXiv:2505.09388 2025a.
Xinyu Yang, Yuwei An, Hongyi Liu, Tianqi Chen, and Beidi Chen. Multiverse: Your lan- guage models secretly decide how to parallelize and merge generation, 2025b. https: /
09999
Qiying Yu, Zheng Zhang, Ruofci Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503.1 14476, 2025.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluat- ing text generation with bert. arXiv preprint arXiv: 1944009990 2019.
Wenting Zhao, Pranjal Aggarwal, Swarnadeep Saha, Asli Celikyilmaz, Jason Weston, and Ilia Kulikov. The majority not always right: Rl training for solution aggregation, 2025. URL
https: :tttss//azze //arxiv. org/ httg///b//200.0002
14

---

Published as a conference paper at ICLR 2026
A
TRAINING HYPERPARAMETERS
Table 6 lists the hyperparameters used for RLVR and SFT.
Table 6: Training hyperparameters.
Hyperparameter
RLVR
SFT & Warmup Data Generation
Mixed precision
BF16
BF16
Optimizer
Adam W
AdamW W
KL penalty
le-3
N/A
Learning rate
1e-6
2e-5
LR scheduler
constant
linear
LR warmup
10%
10%
Training steps
1000
varies
Batch size (rollouts samples)
448
32
Rollouts per sample
[2,8]
Total samples
7000
varies
Max length
4096
2048
Steps per rollout
8
N/A
Temperature
0.6
0.6
Heads
4
4
Generation width
or
[2,4]
B PARAMETER COUNT BREAKDOWN
Bri dge has a low memory cost, adding relatively very few parameters (2.8% to 5.1%) to LLMs. Table 7 shows the exact parameter counts for each model.
Table 7: Distribution of parameters (B) across embedding/head, attention (Att),, feedforward (FF),
and Br Bridge blocks.
Model
Embed & Head
Attn
FF
Bri dgde
Orig. Total
New Total
DS-Qwen-1.3 1.5BB
0.47
0.15
1.16
0.09
1.78
1.86
DS-Qwen-7B
1.09
0.82
5.70
0.21
7.62
7.82
DS-Llama-8F
1.05
1.34
5.64
0.27
8.03
8.30
C FURTHER GRPO CONSIDERATIONS
While our method inserts dependence between sequence and therefore their corresponding rewards, the sample permutation invariance of Bridge blocks means the unconditional rewards are still identically distributed. This implies
G EE=1'
E(Å,) = std(ri,...,,re = r nr'; stddrr....... ,..... = 0,
preserving unbiasedness of the advantage. To preserve outs for GRPO, one can generate multiple groups per prompt with Bridge and compute advantages between groups. Though this deserves exploration in future work, do not do this here it would be computationally expensive, and our single group setup is already empirically performative.
notion of independence between roll-
D BRIDGE PLACEMENT
Here, we examine the architectural placement Bridge blocks. Table X resulting MATH500 accuracy after applying RLVR on DS-Qwen-1.55 1.5BB with Bridge blocks added
we compare the
15

---

Published as a conference paper at ICLR 2026
after attention blocks after feedforward blocks, chosen architecture for the experiments. Since there is not significant difference, this implies flexibility placement, though we choose to stick with the one with the higher warmed up performance for experiments.
Tabbl 8: Effect MATH-500 accuracy when inserting Bridge blocks after attention blocks after feedforward blocks (chosen architecture) in DS-Qwen-1155B.
Placement
Cold Start
Warmed up
After Attention
80.20
80.20
After Feedforward
80.15
81.30
E SELF-CONSISTENCY
Since Bridge solely focuses on generation, we use any post-generation method to synthe- size or choose the final output. As an example, Table demonstrates Bridge coupled with self- consistency (Wang et al. 2022).
Table 9: Self-consistency accuracy across 32 samples. Tasks are abbreviated as described in Table |
Model
AIME24
AIME25
AMC
BRU
CMI
HMMT
Avg
DS-Qwen-1.5B
20.83
19.17
66.87
30.83
13.75
10.83
27.05
RLVR only
28.33
25.00
71.25
24.17
7.50
13.33
28.26
P-Match
25.00
26.67
72.50
28.33
8.75
15.00
29.38
Bridge
30.83
24.17
75.00
28.33
11.25
15.00
30.76
DS- DS-Qwen-7B
30.00
26.67
78.12
33.33
8.75
20.83
32,95
RLVR only
36.67
27.50
83.75
38.33
14.37
17.50
36.35
P-Match
37.50
29.17
81.87
36.67
13.13
15.00
35.56
Bridge
40.00
30.00
86.25
41.67
17.50
16.67
38.68
DS-Llama-8B
25.00
16.67
73.12
23.33
8.13
13.33
26.60
RLVR only
25.83
25.00
76.25
25.83
12.50
17.50
30.49
P-Match
25.83
25.00
70.63
25.83
14.37
19.17
30.14
Bridge 31.67
22.50
78.75
24.17
10.00
15.00
30.35
BRIDGE PSEUDOCODE
Algorithm sketches the pseudocode for Bridge blocks, following 2 Like normal self-attention, this can easily be extended to multiple heads.
Algorithm Bridge Block
Input: X
R BXXXD
Parameters: Wo. WK E DoR; Wv, W E DvD
Output: y
BBxXDD
Q, <- X Wo for
K.
X WK k for s I,..., S
V. X Wy for I...,,
Construct mask M, BxB for 5 L....
M, M.),,,d if generations by, by have the same prompt and are incomplete at token s.
M, by.b2
otherwise.
q.K. Dox + M,
D + Softmax
V. V.Wo for 1,
Return: y
16


Reviews:
[{"review_id": "l0AnjCxDM2", "reviewer": "Reviewer_vGTK", "summary": "In this paper, the authors propose a new technique, aptly called Bridge, for post-training language models to perform well at parallel generation. By allowing different sequence positions across a batch to effectively communicate with each other, each independent generation suddenly becomes interdependent. \n\nThe authors evaluate the technique on the canonical math datasets and produce reasonable baselines to compare Bridge against. They find that Bridge leads to improvements in both the pass@k and G-Pass@8_{tau}.", "questions": "Minor points:\n- How did you get the numbers 30%, 50% and 23% on line [323]? I can’t recreate them from the table using Avg col?\n- In line [365] I would have appreciated a definition of coverage.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Strengths:\n- The paper is well written. Coherent and understandable (albeit a few minor places I would have appreciated a little more context).\n- This approach seems reasonable for introducing interdependence across generations.\n- Reasoning in LLMs is an important area. AFAIK this is the first paper that takes a serious stab at introducing cross information sharing across parallel generations.", "weaknesses": "Weaknesses:\nI left the paper feeling like something was missing. The proposed architecture seems reasonable, but I have no idea for what it’s actually doing for the model? \n- Does the model begin to sample in semantically different directions given context from other models? I.e., does it increase diversity? \n- If one trace is suddenly going in the right direction, do other traces update their reasoning accordingly?\n\nSomething as simple as BERTScore (similar to here [1,2]) or some notion of entropy across traces in the different settings could provide some insights into what’s going on.\n\nA preliminary step in understanding what’s going on was done on line [457] but this seems both unsatisfactory and overly short. \n\n[1] https://arxiv.org/abs/2502.01697 \n[2] https://openreview.net/forum?id=gvsdQ72Peg&noteId=gvsdQ72Peg", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "US2T8K2meR", "reviewer": "Reviewer_T7CP", "summary": "The paper proposes an architectural modification to the transformer architecture to allow parallel generations to communicate with each other during generation. It shows notable improvements in accuracy on math benchmarks.", "questions": "Can you address the concerns around generalization to non-math domains, and appropriate comparison approaches?\n\nWhat is your hypothesis for why, as shown in Figure 7, norm ratios for Bridge blocks are so low?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper introduces a novel and clever technique that offers a nice performance improvement on math benchmarks at a modest cost. It offers an appropriate level of technical detail and analysis.", "weaknesses": "The paper only evaluates math benchmarks, so it's unclear how well the approach would work in other domains.\n\nIt's not clear that P-Match, rather than any of the alternatives described in the related works section, is the appropriate baseline.\n\nFigures lack CIs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C1DbwhFSBW", "reviewer": "Reviewer_YB53", "summary": "The paper proposes Bridge, a module for LLMs that applies attention between (past) tokens across parallel samples. The proposed method enables interdependence between traditionally independent parallel responses. The block is trained using RLVR and optional SFT warm up. The paper presented experiments showing that Bridge achieves better accuracy on math benchmarks over the base model.", "questions": "- Is there a latency cost due to the additional computation in Bridge?\n- How does Bridge perform compared against using existing methods enabling parallel computation (with and without interdependence)? Even though they eventually join together into a single response, a simple string post-processing step can split the response back into a set of responses by splitting on the special tokens these methods use for spawning parallel threads. How does Bridge compare to doing this?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 3, "strengths": "- The paper presents, to the best of my knowledge, a novel idea of enabling attention between tokens across samples in a batch, a dimension that typically maintains independence. \n- The evaluation applies Bridge and baselines to a variety of different math benchmarks", "weaknesses": "- The improvement in accuracy is rather small compared to baselines. Bridge does not perform much better than P-Match, which is the baseline given the equivalent amount of compute and in some cases also not much better than just RLVR only. As such, it seems most of the accuracy improvements are from additional compute/training over the original model rather than from the Bridge block design.\n- An important motivation to enable parallelism is to enable lower latency while increasing compute. The evaluation does not present the inference latency of Bridge and baselines\n- The evaluation lacks comparison to methods that enable parallelism during LLM decoding (such as those cited in the paper). Several of those methods (e.g. Multiverse, Pasta) already have mechanisms for encoding interdependency between different threads by enabling parallel threads to join together and respawn. The evaluation should evaluate against these methods to compare how Bridge's method of enabling interdependency performs.\n- The notation of Eq 1 and Eq 2 are difficult to parse", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "45wNTA8SSt", "reviewer": "Reviewer_hSjT", "summary": "Their method allows the LLM to attend to parallel generations from the same prompt. They do this with a new transformer block that attends to the previous token representation from the other generations. They initialize the layers to have no contribution, and fine tune the new architecture with SFT followed by RL. Their results show that this works better than adding the equivalent number of parameters to the transformer.", "questions": "none", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Novelty:\nExisting methods use multiple LLM generations, but theirs is the first I know of in which other generations influence the model during generation.\n\nClarity:\nThe paper was easy to understand, Figure 2 was clear and aided in understanding.\n\nResults:\nThe results are strong. I found figure 5 especially convincing, since it shows that increasing the generation width helps up to a point.\n\nSignificance:\nIt's common to sample multiple responses from LLMs, so this method is widely applicable.", "weaknesses": "Error bars\n\nNone of the plots have error bars and there’s only one run of each method. This makes it hard to tell if the differences are real in some cases where the method results are close to the baseline.\n\nReproducibility\n\nThe code is not provided, and there is no reproducibility statement. This is especially important for this paper since the main contribution is the new transformer layer so open sourcing the implementation would be very useful.\n\nBaselines\n\nThey don’t compare with any existing methods. Some existing methods take advantage of multiple generations to improve LLM accuracy, for example self consistency. The authors could compare against self consistency, or show the result of using self consistency on top of Bridge.\n\nMinor clarity issues:\n\nThe motivation in the introduction is high level and abstract. It says “Independent generations for the same prompt leave potentially useful information derived from other responses unutilized, limiting the performance ceiling.” but it doesn’t explain what this potentially useful information is. Add concrete examples to help the reader understand the motivation.\n\nThe paper doesn’t explain what RL with verifiable rewards is, leading to confusion which don’t already understand it. For example, the paper should explain if Bridge can also work with other types of RL for language models like RLHF, or if it’s only applicable to RLVR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)