Title: Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Generative Modeling; Model Architecture; Parallel Generation; LLM Reasoning
Abstract: Autoregressive Large Language Models (AR-LLMs) frequently exhibit implicit parallelism in sequential generation. Inspired by this, we introduce Multiverse, a new generative model enabling natively parallel generation. Multiverse internalizes a MapReduce paradigm, generating automatically through three stages: (i) a Map stage for adaptive task decomposition, (ii) a Process stage for parallel subtask execution, and (iii) a Reduce stage for lossless result synthesis. Next, we build a real-world Multiverse reasoning model with co-design of data, algorithm, and system, enabling rapid and seamless transfer from frontier AR-LLMs. For data creation, we develop Multiverse Curator, an automated LLM-assisted pipeline that transforms sequential reasoning chains into structured training data, avoiding costly human annotations. Algorithmically, we design Multiverse Attention to separate parallel reasoning steps while keeping compatibility with causal attention for efficient training. Systematically, we implement Multiverse Engine to support parallel inference. It features a dedicated interpreter that dynamically switches between sequential and parallel generation, triggered directly by the model. After a 3-hour fine-tuning with 1K examples, our Multiverse-32B stands as the only open-sourced non-AR model achieving performance on par with leading AR-LLMs of the same scale, evidenced by AIME24 & 25 scores of 54% and 46%, respectively. Moreover, our budget control experiments show that Multiverse-32B exhibits superior scaling, outperforming AR-LLMs by 1.87% on average using the same context length. Such scaling further leads to practical efficiency gain, achieving up to 2x speedup across varying batch sizes. We have open-sourced the entire Multiverse ecosystem, including data, model weights, serving system, supporting tools, as well as data curation prompts and detailed training and evaluation recipes.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a prompting, fine-tuning, and execution method that enables decomposing problems into parallel components. The system first decomposes the problem into subproblems, then generates them independently, and merges them into a solution for the problem. To fine-tune models, they propose the Multiverse 1K dataset, which contains explicitly annotated parallel branches.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n- Interesting and novel idea\n- Relatively simple\n\n# Weaknesses\n- Some details are not clear. See questions below.\n- For most of the examples, the amount of seedup in Table 2 is relatively low. This is surprising given the ~7 average branches reported in Tab 1. Perhaps the authors should consider reporting a parallelism metric where they compute the \"parallelness\" of the sequence, where they sum the length of the sequential steps, but they use the max of the branches as the length of a parallel block, and they divide this number by the original length.\n\nQUESTIONS:\n- Section 5.1 describes the process of creating the dataset, but this is in contrast with Sec 3.2, where the authors claim that the LLMs cannot decompose the problem into parallel branches. What is the key difference between Sec 3.2 and 5.1 that enables the LLMs to generate them? Isn't Sec. 3.2 Just a prompting issue then?\n- What are the baselines reported in Tab 2. exactly? Is Autoregressive 32B the same as Qwen2.5-32B-Instruct just fine-tuned on the task in the standard way? Is the difference between Multiverse-32B-zero and Multiverse-32B only the system prompt, or are they also fine-tuned differently? The text says \"variant prompted without the 'think in parallel' instruction\", but is this done only in test time or also during tuning? Also, if prompted without \"think in parallel\"  but fine-tuned with \"think in parallel\" in place and by generating parallel branches, does it still generate parallel branches?\n- What is the parallelism metric reported in Tab 2? The caption says \"ratio between the total number of generated tokens and the actual generation length\", but I don't understand what the difference is between the total number of generated tokens and the generation length. Is it something similar to what I described in the weaknesses?\n- How do the authors measure the implicit parallelism for Table 1 and Figure 3? Do they check the examples by hand? Is there some more advanced promoting technique that is used?\n- Fig 8: What is P?\n- Line 83: what are the varying batch sizes? \n- Line 132: For the hidden-state probe, the authors should probably use some internal layer of the model rather than the last.\n- Line 154: Why are there square brackets in $x_{[1:3,s]}$ and not in $x_{1,1:6}$?\n- Line 204: How does this rewrite work? Is it by prompting or by hand?\n\nLIMITATIONS:\nThe authors discuss the limitation of their work in a satisfactory way.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper converts existing single threaded dataset into multi-threaded ones. Furthermore, when fine-tuned on this dataset, the model learns to replicate the behavior on unseen problems, thus realizing the proposed benefits.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: \n\nThis is a burgeoning topic that see more and more attempts at addressing. This publication takes a very straightforward approach to get quite satisfactory results. This is the case where simple is good. The S1 dataset should be a boon for people to comprehend, extend, and improve upon this Multiverse.\n\nThough running the inference in a multi-threaded manner, the authors generated the multiverse training data in a single-threaded manner. Although it should be understood by experts in this field, most readers, especially junior ML practitioners, would not immediately comprehend how practical this is.\n \nWeakness: \n\nthe \"map reduce\" analogy is a weak one, I don't think it's that necessary. After all, most of the readers are not and will not be employed in cloud companies, and they don't and shouldn't be expected to - even if they hold a CS degree - instantly understand what map-reduce is. It's very simply a spawn-collect (scatter-gather, or even simpler, fork-join) pattern of programming.\n\nThe concept of \"plan - spawn - collect\" is fairly straightforward. There are certainly many other forms of parallelism.\n\nhere are still many unresolved issues in multi-threaded LLM as alluded to in this paper. As this is the first step to multi-threaded LLM, I probably should would refrain from calling this \"weakness\". I expect any void to be quickly filled.\n\nQUESTIONS:\nMultiverse attention. I can understand that the positional index being set to the same over the different <path> during training. There is a supposed benefit that the <path> follows directly after the prefix. Please explain why during inference different various paths can share the same positional index for the \"reduce\".\n\nThe argument given that \"current LLM cannot parallelize\" is rather weak. I do not object to that because whether or not the claim is valid, it does not take away the main contribution of the paper. I do recoomend to either improve the argument or to remove.\n\nI suggest that authors can cite works (don't know if published or not) which aim to allow multiple \"paths\" to also see each others progress, as opposed to just the common prefix, in the Related Works.\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper introduces Multiverse, a test-time scaling framework for LLM that enables parallel generation. Motivated by the observation that autoregressive models often generate reasoning trajectories with latent parallelizable branches, the authors propose a MapReduce framework that decomposes generation into three stages: Map, Process, and Reduce. The authors design three components: data curation, architecture design, and system implementation. Empirically. the authors demonstrate improvements on mathematical reasoning tasks.

STRENGTHS AND WEAKNESSES:
## Strengths
1. The paper demonstrates the inherent parallelizable features of autoregressive generation. The proposed method is novel; the MapReduce framework is elegant and practically useful.
2. The paper is well written with rich illustrations and clean empirical and structural analysis. 


## Weaknesses
1. The empirical improvement of the proposed method appears limited (Table 2, Figure 7). Error bars are not provided, so it's unclear whether the gains are statistically significant or within expected variance.
2. Ablations are missing: How important is each of the three components—data, Multiverse attention, scheduler? For example, would just fine-tuning Qwen2.5 on the structured outputs suffice?

QUESTIONS:
1. How important is each of the three components—data, Multiverse attention, scheduler? For example, would just fine-tuning Qwen2.5 on the structured outputs suffice?

2. In Line 126, how does the prompting test work? Can the authors clarify this?

In general, I lean toward acceptance despite the mentioned weaknesses and questions.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-03

comments: SUMMARY:
This paper proposes Multiverse, a natively parallel generative model based on the MapReduce paradigm. Autoregressive large language models (AR-LLMs) are constrained by sequential generation, while non-AR models like diffusion models struggle with complex tasks. Multiverse features three stages: Map for adaptive task decomposition, Process for parallel subtask execution, and Reduce for lossless result merging.

The model co-designs data, algorithm, and system: Multiverse 1K dataset transforms sequential reasoning into parallel structures via an LLM-assisted pipeline; Multiverse Attention modifies attention masks for parallel generation; Multiverse Engine enables dynamic scheduling between sequential and parallel modes.

STRENGTHS AND WEAKNESSES:
Unlike text diffusion models that often rely on rigid parallelization, Multiverse empowers language models to autonomously decide parallel generation strategies, leveraging GPU acceleration through dynamic task decomposition. This self-adaptive parallelism not only optimizes computational efficiency but also preserves logical coherence in complex reasoning, marking a significant advancement in bridging sequential AR-LLMs with parallel hardware capabilities.

QUESTIONS:
In the experimental section, are there more settings provided, such as the code? Additionally, the scores on AIME seem too low. Is it difficult to interpret the effectiveness of the method?
At a parallelism ratio of 1.3, the model achieves an 18.5% average speedup. the results seems not strong enough.

LIMITATIONS:
The experiments is not strong enough, with less than 2 times speed up. and the baseline is also not strong,
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-05