BeamAggR: Beam Aggregation Reasoning over Multi-source Knowledge for Multi-hop Question Answering (2024)

Abstract: Large language models (LLMs) have demonstrated strong reasoning capabilities. Nevertheless, they still suffer from factual errors when tackling knowledge-intensive tasks. Retrievalaugmented reasoning represents a promising approach. However, significant challenges still persist, including inaccurate and insufficient retrieval for complex questions, as well as difficulty in integrating multi-source knowledge. To address this, we propose Beam Aggregation Reasoning (BeamAggR), a reasoning framework for knowledge-intensive multi-hop QA. BeamAggR explores and prioritizes promising answers at each hop of question. Concretely, we parse the complex questions into trees, which include atom and composite questions, followed by bottom-up reasoning. For atomic questions, the LLM conducts reasoning on multi-source knowledge to get answer candidates. For composite questions, the LLM combines beam candidates, explores multiple reasoning paths through probabilistic aggregation, and prioritizes the most promising trajectory. Extensive experiments on four opendomain multi-hop reasoning datasets show that our method significantly outperforms SOTA methods by 8.5%. Furthermore, our analysis reveals that BeamAggR elicits better knowledge collaboration and answer aggregation.

heading: 1 Introduction
text: Large language models have showcased impressive performance across various NLP tasks (OpenAI, 2023; Touvron et al., 2023). Furthermore, chain-ofthought (CoT) prompting further enhances the reasoning capabilities of LLMs (Wei et al., 2022; Kojima et al., 2022; Chu et al., 2023). However, when the question surpasses the knowledge boundaries of LLMs, it leads to factual errors, also known as hallucination. Retrieval-augmented generation (RAG) assists LLMs by retrieving supporting knowledge
* Corresponding Authors: Ming Liu, Qianglong Chen
to alleviate factual errors, thereby drawing significant attention within the research community (Gao et al., 2023; Huang et al., 2023).
Early attempts adopt a retrieve-then-read framework for one-time retrieval (Lazaridou et al., 2022; Borgeaud et al., 2022; Izacard et al., 2023; Zhang et al., 2023b). They use the original complex question as the retrieval query and achieve satisfactory performance in single-hop questions (Joshi et al., 2017; Kwiatkowski et al., 2019). Nonetheless, when confronted with complex multi-hop questions, one-time retrieval suffers from the issues of inaccurate and irrelevant retrieval, which greatly impair their performance in multi-hop reasoning.
Recent work introduces iterative multi-round retrieval (Khattab et al., 2022; Press et al., 2023; Trivedi et al., 2023; Jiang et al., 2023b; Shao et al., 2023). They use the content generated by LLMs for retrieval and, in turn, use the newly retrieved content for reasoning. Through the iterative alternation between retrieval-augmented reasoning and reasoning-augmented retrieval, the retrieval is substantially improved. Meanwhile, a portion of research decomposes complex questions into simple sub-questions, employing sub-question retrieval to
1229
obtain more precise information (Zhou et al., 2023; Cao et al., 2023; Park et al., 2023; Su et al., 2023).
However, there are still significant issues with these methods. Iterative retrieval is tough to achieve precise retrieval aligned with the model’s reasoning. Sub-question retrieval grapples with the challenge of accurately aggregating answers, which causes cascading errors. Besides, when it comes to the open-domain setting, relying solely on knowledge from a single source proves inadequate for complex questions. Introducing multisource knowledge may encounter knowledge conflicts, rendering efficient collaboration challenging, thus impeding its applications.
To address the aforementioned challenges, our research focuses on the following question: How can models adaptively select and integrate knowledge from different sources during the reasoning, while reducing cascading errors in subquestions aggregation? Building upon this motivation, we propose Beam Aggregation Reasoning (BeamAggR) framework for knowledge-intensive multi-hop reasoning. Concretely, our method consists of three modules. (i) question decomposition: We begin by leveraging LLMs to decompose complex questions and convert them into question trees. The root node contains the original complex question, leaf nodes contain atomic sub-questions, and intermediate nodes contain composite questions that require compositional (or comparative) reasoning to obtain answers. Afterward, we traverse the question tree in post-order, employing bottom-up reasoning. (ii) complementary multi-source reasoning: For atomic questions, we conduct multisource reasoning, followed by fine-grained answer aggregation, thus fostering knowledge collaboration. The aggregated answers are then normalized into a probability distribution, serving as candidates for beam aggregation. (iii) beam aggregation: For composite questions, we enumerate the combinations of their dependent sub-questions and conduct reasoning. Finally, the reasoning results are probabilistically aggregated, and the most promising predictions are selected.
In summary, our method explores reasoning trajectories at each hop of questions and prioritizes paths with higher likelihoods, thereby bringing reasoning insight and reducing cascading errors. Besides, the complementary multi-source reasoning helps alleviate knowledge omission and conflict.
We evaluate our method on four open-domain multi-hop reasoning datasets: HotpotQA (Yang
et al., 2018), 2WikiMQA (Ho et al., 2020), MuSiQue (Trivedi et al., 2022), and Bamboogle (Press et al., 2023). The experiments are conducted using GPT-3.5-turbo (Ouyang et al., 2022) and Mistral (Jiang et al., 2023a). Experimental results show that our method significantly outperforms the baselines on four datasets, with an average improvement of 8.5% compared to the previous state-of-the-art method, demonstrating its superiority. Furthermore, thorough analysis reveals the superiority of our approach to knowledge collaboration and answer aggregation. Our contributions can be summarized as follows: • We introduce BeamAggR, a framework for open-
domain multi-hop reasoning, which outperforms the state-of-the-art methods.
• BeamAggR dynamically integrates multi-source knowledge in fine granularity during reasoning, fostering knowledge collaboration.
• BeamAggR broadens the scope of reasoning with beam combination and optimizes reasoning trajectories, mitigating cascading errors.

heading: 2 Related Work

heading: 2.1 Reasoning with Large Language Model
text: Wei et al. (2022) prompts LLMs to generate reasoning process before final answers, which is known as chain-of-thought (CoT) prompting. Since then, CoT prompting has been widely applied to enhance the reasoning capabilities of LLMs. Some work also designs instructions or clustering demonstrations for zero-shot reasoning (Kojima et al., 2022; Zhang et al., 2023c). Additionally, self-ensemble has been proven through extensive experiments to be an effective approach to improve performance. Wang et al. (2023) uses probabilistic sampling for multiple reasoning traces, while Qin et al. (2023) diversifies reasoning paths by using multiple languages CoTs. To address complex questions, Zhou et al. (2023); Dua et al. (2022) decompose them into sub-questions and solve them progressively, while Yao et al. (2023) models reasoning procedures as BFS or DFS search on reasoning trees.
In contrast, our method employs divide-andconquer strategy, breaking down complex questions into trees and addressing them through bottom-up aggregation reasoning.

heading: 2.2 Retrieval-augmented Reasoning
text: Knowledge-sensitive tasks may induce factual hallucinations in LLMs, thus necessitating external
retrieval for retrieval-augmented reasoning. Early work uses one-time retrieval, but they struggle to gather all the necessary knowledge to answer complex questions, resulting in knowledge omissions (Borgeaud et al., 2022; Lazaridou et al., 2022; Izacard et al., 2023). To address this, iterative retrieval is proposed. DSP (Khattab et al., 2022) engages in iterative interactions between retriever and reader through programmatically defined procedures. SelfAsk (Press et al., 2023) iteratively decomposes questions and solves them through the Google search. IRCoT (Trivedi et al., 2023) uses each reasoning step as a query for retrieval until obtaining the final answer. Similarly, ITER-RETGEN (Shao et al., 2023) conducts iterative retrievals by concatenating the output from the previous round with the original question. FLARE (Jiang et al., 2023b) introduces a lookahead mechanism, dynamically controlling the timing of retrieval based on reasoning confidence. Beam Retrieval (Zhang et al., 2023a) introduces an end-to-end framework designed to retrieve relevant paragraphs at each hop of questions through beam search. Meanwhile, some efforts achieve more precise retrieval by decomposing the problem into QDMR format (Wolfson et al., 2020). Cao et al. (2023) decomposes the problem into a tree, while Park et al. (2023) constructs a reasoning graph.
Compared to their approach, our method integrates diverse knowledge via complementary multisource reasoning. Besides, it explores reasoning trajectories at each hop of questions and prioritizes the most promising path, thereby eliciting better answer aggregation and reducing cascading errors.

heading: 3 Beam Aggregation Reasoning
text: Beam Aggregation Reasoning decomposes complex questions into trees and conducts bottom-up reasoning. Throughout the bottom-up reasoning, it intricately amalgamates diverse knowledge sources, thereby mitigating the dearth of knowledge. Furthermore, beam aggregation probabilistically consolidates answers and selects reasoning pathways, consequently diminishing cascading errors.
Firstly, we decompose complex questions, then perform bottom-up multi-source knowledge reasoning in a post-order traversal manner, and employ beam aggregation at intermediate nodes until reaching the root node to obtain the final answer. The procedure is depicted in Algorithm 1, with an overview of the methodology illustrated in Figure 2. We will introduce question decomposition in § 3.1, multi-source reasoning in § 3.2 and beam aggregation in § 3.3. Detailed definitions of notations in the algorithm and formulas can be found in Table 7. Task definition is given in Appendix A.1.
Algorithm 1 Beam Aggregation Reasoning Require: Complex multi-hop questions, Q Require: Multi-source knowledge retriever, R Require: Large language model, LLM Require: Candidate size in aggregation, k
1: Qdecomp “ LLMpQq 2: for N piq in PostOrderTraversepQdecompq do 3: if N piq is leaf-node then 4: â “ LLMpqpiq, Rq 5: apiq, ppiq “ Votepâqr1 : ks 6: else 7: initialize â, p̂ “ list, list 8: for c1, p1 in CartProdpsonspN piqqq do 9: q1 “ MaskFillpqi, c1q
10: at,pt “ VotepLLMpq1, c1, Rqq 11: â Ð Concatpâ, atq 12: p̂ Ð Concatpp̂, pt ¨ p1q 13: end for 14: apiq, ppiq “ Aggrpâ, p̂qr1 : ks 15: end if 16: end for 17: return aroot[1]

heading: 3.1 Question Decomposition
text: Multi-hop questions entail complex structures, such as bridge, comparison, composition and their integration. To address this, we parse complex questions into trees to express the reasoning structure. As shown in Figure 2 (a), the complex question Q is decomposed into a tree comprising four simpler sub-questions, Qp1q to Qp4q, with (compositional) dependencies among them. The root node represents the original complex question, the leaf nodes represent atomic sub-questions, and the intermediate nodes require compositional (comparative) reasoning to obtain answers. Following Cao et al. (2023); Su et al. (2023), we adopt #i as the placeholder for intermediate questions, enabling us to replace incomplete questions with solved subquestions as we traverse to that node. Specifically, the decomposed question is represented in QDMR format (Wolfson et al., 2020). Afterward, we tackle the complex questions in a bottom-up fashion, following a post-order traversal sequence. The tree is represented as a post-order traversal node sequence, Qdecomp “ tN p1q, N p2q . . . u, with each node containing a question, candidate answers, and the associated probabilities, N piq = tqpiq, apiq, ppiqu. It is noteworthy that the model may produce structural incorrect decomposition, which needs post-filtering
to ensure the validity of decomposition. We present the formal definitions in Table 7(b).

heading: 3.2 Complementary Multi-source Reasoning
text: To avoid the hallucination caused by lack of knowledge, we use four reasoning strategies combined with answer normalization to fuse information from diverse knowledge sources, as shown in Figure 2(b). The knowledge sources include implicit internal knowledge, explicit internal knowledge, Wikipedia knowledge, and web search knowledge.
as “ LLMpq,Ksq (1) where s P tclosebook, parametric, wiki, serpu is reasoning strategy, K is retrieved knowledge.
Internal Knowledge Reasoning For implicit knowledge reasoning, we prompt the model with chain-of-thought demonstrations to perform closedbook reasoning. There are also studies suggesting that the model’s parametric knowledge can serve as a retrieval source (Yu et al., 2023; Zhang et al., 2023b). We first prompt the model to generate parametric knowledge relevant to the question, and then employ it for explicit knowledge reasoning.
External Knowledge Reasoning We utilize Wikipedia and search engines as external retrieval for external knowledge reasoning. Regarding Wikipedia, we employ BM25 to conduct sparse retrieval over the full Wikipedia dumps. For search engines, we call the Google Search API and use the organic results as the retrieval content. After retrieval, we employ few-shot prompts to conduct reasoning on the retrieved documents.
Answer Normalization After completing four sets of independent knowledge reasoning, we merge them through voting to achieve knowledge fusion. Following that, we normalize the answerfrequency pairs to probability distributions for subsequent beam aggregation, as shown in Eq. (2).
pi “ exppfi{τqřk j“1 exppfj{τq
(2)
where f is frequency and τ is temperature.

heading: 3.3 Beam Aggregation
text: In beam aggregation, we conduct reasoning over combinations of candidate answers inherited from sub-questions to expand reasoning breadth and exploration space, as shown in Figure 2(c). We then select reasoning paths by maximizing the marginal probability distribution of predictions.
Beam Combination In intermediate nodes, we need to conduct reasoning based on the answers derived from previous sub-questions, and each subquestion is associated with a set of answers and probabilities, termed candidates (or beams). The intermediate question may depend on multiple subquestions, so we enumerate combinations among the candidates. Specifically, we compute the Cartesian product among the candidates, as shown in Eq. (3). To prevent combinatorial explosion, we restrict the exploration space with beam size. Afterward, for each combination, we substitute the placeholders in the question with candidate answers and conduct multi-source reasoning. We take a composition node with two sub-questions as an example.
txapxqi , apyqj y, ppxqi ppyqj | i, j “ 1, 2, . . . , ku (3)
where pxq,pyq denote sub-questions, k is candidate size, a is answer and p is associated probability.
As illustrated in Figure 2(c), Qp4q relies on Qp2q, Qp3q. We calculate the Cartesian product of candidates of Qp2q and Qp3q, and perform substitution to derive new questions. For example, Qp4q11 is obtained by substitute the #1 with ap2q1 and #2 with ap3q1 , “When did Austin become the capital of Texas?”, with P pQp4q11 q “ pp2q1 pp3q1 . Subsequently, we conduct multi-source reasoning on all substituted sub-questions.
Probabilistic Answer Aggregation We have explored numerous combinations of sub-questions in beam combination, yielding multiple sets of answers and probabilities. Next, we will aggregate the above answers according to the probabilities to determine the optimal reasoning path. This can be formalized as argmax for maximizing the marginal probabilities of predictions, as described below.
P pyq “ ÿ
qiPQ P py|x “ qiq ¨ P pqiq (4)
where Q is the original intermediate question, qi is a substituted question, P py|x “ qiq is the answers distribution of qi, and P pqiq is the weight of qi.
After probabilistic answer aggregation, we keep the top-k answers a and their probabilities p as candidates. The aggregated candidates will continue propagating bottom-up, participating in subsequent beam aggregations until reaching the root node. Upon reaching the root node, we regard the answer with the highest probability as the final answer.

heading: 4 Experimental Setup

heading: 4.1 Datasets
text: We evaluate BeamAggR on four open-domain multi-hop reasoning datasets. HotpotQA (Yang et al., 2018), 2WikiMQA (Ho et al., 2020), and Bamboogle (Press et al., 2023) consist of two-hop questions, and MuSiQue (Trivedi et al., 2022) contains questions with 2 to 4 hops. For HotpotQA, 2WikiMQA, and MuSiQue, we use the same development and test set provided by IRCoT (Trivedi et al., 2023). The test set and development set are both extracted from the original development set, consisting of 500 and 100 instances, respectively. For Bamboogle, we use all 125 instances as the test set, without a separate development set, and use the same hyperparameters with 2WikiMQA. The evaluation metric is token-level F1.
On all four datasets, we conduct experiments in the open-domain setting, employing the entire Wikipedia dumps and web search for retrieval.

heading: 4.2 Implementation Details
text: In the main experiment, we use GPT-3.5-turbo as the backbone. Since OpenAI has deprecated GPT3.5 (text-davinci series), we reproduce the baselines with GPT-3.5-turbo for a fair comparison1. Please refer to Appendix A.2 for more details.

heading: 4.3 Baselines
text: Standard Prompting (Brown et al., 2020) directly generates the final answer with few-shot demonstrations. We use a 20-shot demonstration.
Chain-of-Thought Prompting (Wei et al., 2022) generates reasoning steps before the final answer. We use a 20-shot demonstration.
One-time Retrieval involves using the original question as the query, concatenating sparse retrieval and Google search results into the prompt to guide the model to perform CoT reasoning.
Self-Ask (Press et al., 2023) adopts an iterative approach to decompose complex questions. It generates sub-questions iteratively based on existing reasoning, and then retrievals and answers them until reaching the final answer.
IRCoT (Trivedi et al., 2023) interleaves retrieval-augmented reasoning and reasoningaugmented retrieval until the retrieval information is sufficient to answer the question.
1https://platform.openai.com/docs/deprecations
FLARE (Jiang et al., 2023b) dynamically adjusts the retrieval timing based on the confidence of reasoning and conducts retrieval based on upcoming reasoning sentences.
ProbTree (Cao et al., 2023) parses the question into a tree, employing logprobs-based sub-question aggregation to obtain the final answer.

heading: 5 Experimental Results

heading: 5.1 Main Results
text: The experimental results on four multi-hop reasoning datasets are presented in Table 1. We observe that one-time retrieval can improve performance in HotpotQA, 2WikiMQA and Bamboogle. However, when facing the more challenging MuSiQue, onetime retrieval even impairs performance (24.7 Ñ 16.4), which indicates that inaccurate retrieval and knowledge conflicts may exacerbate hallucination.
The iterative retrieval approach has made notable progress compared to one-time retrieval by offering more comprehensive knowledge. Additionally, methods based on sub-question retrieval have more explicit queries, which leads to a further improvement in retrieval accuracy, serving as the best-performing baseline method.
As shown in Table 1, our method demonstrates superiority over all baselines. It outperforms the previous state-of-the-art, ProbTree, on all four datasets: HotpotQA (+2.5), MuSiQue (+4.0), 2WikiMQA (+3.7), and Bamboogle (+8.2). We attribute the improvement to three aspects: (i) Question decomposition leads to more accurate retrieval. Compared to iterative retrieval based on
generation content, sub-questions serve as clearer retrieval queries. (ii) Complementary reasoning facilitates collaboration between diverse knowledge sources. Our method dynamically selects and leverages knowledge from multiple sources with fine granularity, thereby mitigating knowledge conflicts and omissions. (iii) Probability-based beam aggregation mitigates cascading errors and optimizes the reasoning trajectories through consistency.
We observe significant improvements, particularly for Comp questions. Comparison questions involve interactions among sub-questions. With the aid of beam aggregation, the model can enumerate various combinations during reasoning, thereby encompassing a broader spectrum of possibilities (65.9 Ñ 74.2, 81.7 Ñ 89.9). Additionally, our method is proficient at addressing complex questions (3/4-hop), as shown in Figure 3. Our method exhibits a slower performance decline as reason-
ing depth increases (8.4% in 3-hop and 12.3% in 4-hop), indicating its ability to precisely aggregate answers and effectively alleviate cascading errors.

heading: 5.2 Results on Open-source Model
text: To prove the generalizability of our method to various models, we also conduct experiments on opensource models. We select Mistral-7B (Jiang et al., 2023a), the state-of-the-art LLM among similar scales. As shown in Table 2, Beam Aggregation significantly outperforms previous SOTA on all four datasets, demonstrating its model-agnostic nature and effectiveness. Furthermore, it is comparable to GPT-3.5-turbo on the Bamboogle dataset.

heading: 5.3 Ablation Study
text: Effect of Multi-source Reasoning We conduct two sets of ablations: removing a single knowledge source and removing a type of knowledge, as shown in Table 3(a). Removing any knowledge source results in a certain performance decrease, suggesting that each type of knowledge contributes to reasoning (a.1-a.4). Furthermore, the declining trends (4.1% internal knowledge and 9.7% external knowledge) indicate that external knowledge makes a greater contribution compared to internal knowledge. Using only internal knowledge for reasoning results in a severe performance drop (a.5). Nevertheless, our method still outperforms chain-of-thought reasoning, which reflects the effectiveness of aggregation reasoning. Completely removing internal knowledge leads to a substantial decline, as shown in (a.6). This suggests that internal knowledge can complement external retrieval, proving the effectiveness of our method in knowledge collaboration.
Effect of Beam Aggregation To validate the effectiveness of beam aggregation, we first employ
log-prob-based aggregation, which can only select a single knowledge source. As shown in Table 3(b), this leads to a notable decline, suggesting that coarse-grained polarized aggregation struggles to effectively integrate knowledge, thus underscoring the superiority of our beam aggregation. Probabilistic aggregation enables the distinction of the importance of answers. To investigate its effect, we conduct an ablation where the aggregated answers are treated with equal weights, as shown in (b.2). The performance drops by 5% and 4% on two datasets, suggesting that prioritizing reasoning trajectories can elicit better reasoning. Finally, we investigate the effect of candidate size. Larger candidate sizes result in broader reasoning breadth, but they also increase reasoning overhead. When it is set to 1, greedy aggregation strategy is employed. It is a cost-efficient variant of beam aggregation reasoning. As shown in (b.3), it causes a slight performance decrease, indicating that some erroneous reasoning may be corrected as the reasoning process progresses with the help of a broader scope of reasoning. We conduct a detailed analysis of reasoning performance and overheads in section 6.3.

heading: 6 Analysis

heading: 6.1 BeamAggR Facilitates Knowledge Collaboration
text: To investigate the impact of Beam Aggregation on knowledge collaboration, we conduct a prelimi-
nary experiment. We track the model’s utilization of knowledge from different sources during the reasoning process, as depicted in Figure 4. The polarity aggregation method based on log-probs excessively relies on reasoning on a single source of knowledge. In more than 2/3 of cases, it can only utilize knowledge from a single source. In contrast, our approach facilitates better knowledge collaboration in reasoning. In summary, our beam aggregation can effectively employ knowledge from multiple sources at a finer granularity.
Furthermore, we carry out a more detailed examination of the proportions of each type of knowledge in reasoning. We track the contribution of different knowledge sources to the final answer, as illustrated in Figure 5. Disparities in reasoning contributions across various datasets are observed. 2WikiMQA and MuSiQue tend to favor external reasoning, Bamboogle leans towards internal reasoning, and HotpotQA strikes a balance among different types of knowledge. It is noteworthy that, without manual adjustment of the weight of knowledge, Beam Aggregation can adaptively adjust its proportions during the reasoning process. This indicates that our method can effectively integrate multi-source knowledge.

heading: 6.2 Trends of Answer Aggregation in Bottom-up Reasoning
text: We conduct a detailed analysis of the trends in answer aggregation throughout the reasoning process. To assess the characteristics of the answer distribution during the bottom-up reasoning process, we define three metrics: diversity, consistency, and uncertainty. The definition and tendency are shown in Figure 6. It can be observed that through the reasoning process, the diversity of answers improves, indicating its exploration of a wider range of possibilities. In contrast, consistency and uncertainty continue to decrease as the reasoning depth increases. This suggests that our method is capable of filtering out reasoning from erroneous during the bottom-up aggregation, dynamically choosing appropriate knowledge sources and reasoning trajectories. Conversely, methods based on log-probs aggregation are affected by inaccurate aggregation and cascading errors, thus unable to achieve these, highlighting the superiority of our method.

heading: 6.3 Analysis of Reasoning Cost
text: Retrieval-augmented generation often involves frequent invocation of LLMs, resulting in significant computational overhead. We compare the performance and overhead of five RAG methods. As illustrated in Figure 7, previous methods exacerbate reasoning overhead while improving performance. In contrast, our method not only surpasses the previous SOTA in performance but also incurs lower overhead. Moreover, our method can further reduce reasoning overhead through greedy aggregation. In summary, Beam Aggregation is Pareto efficient in balancing performance and reasoning overhead. Detail statistics can be found in Table 5.

heading: 7 Conclusion
text: This paper introduces BeamAggR for knowledgeintensive multi-hop reasoning. BeamAggR utilizes a divide-and-conquer strategy, breaking down complex questions into a tree structure and conducting reasoning in a bottom-up fashion. At each step of reasoning, BeamAggR builds upon previous candidates to identify the most likely reasoning path. Furthermore, it incorporates multi-source reasoning to enhance knowledge collaboration. Overall, BeamAggR facilitates multi-hop reasoning through precise sub-question retrieval, efficient knowledge collaboration, accurate answer aggregation, and broad exploration of reasoning trajectories. Extensive experiments on four open-domain multi-hop reasoning datasets demonstrate its effectiveness.
Limitations
Our method involves beam combination across multiple candidates and self-consistency in each reasoning step, thereby increasing reasoning overhead. This overhead can be mitigated through greedy aggregation reasoning. The effectiveness of our framework is contingent upon the accurate decomposition of questions, which may pose significant challenges for large language models. Currently, we only use internal knowledge and unstructured external knowledge for reasoning. In future work, structured external knowledge, such as knowledge bases, can be integrated into reasoning to provide more comprehensive knowledge repositories (Li et al., 2024).

heading: Acknowledgements
text: The research in this article is supported by the National Key Research and Development
Project (2021YFF0901602), the National Science Foundation of China (U22B2059, 62276083), and Shenzhen Foundational Research Funding (JCYJ20200109113441941), Major Key Project of PCL (PCL2021A06). Ming Liu and Qianglong Chen are the corresponding authors.

heading: A Appendix
text: A.1 Task Defination
This paper focuses on open-domain multi-hop question-answering tasks. The task’s input is a multi-hop question Q that requires multi-step reasoning to solve, with each step of reasoning necessitating specific knowledge. Given Q as the query, the retriever R retrieves relative documents D “ tdiu|D|i“1. The retrieved documents D are then fed as context into the model for knowledge reasoning. y “ LMpQ,Dq. Our method employs LLMs with few-shot prompts and external retrieval to address the tasks.
A.2 Implementation Details of BeamAggR
Retrieval Setup We use the October 2017 Wikipedia dumps2 as the retrieval corpus, employing BM25 (Robertson and Zaragoza, 2009) implemented by Elasticseach as the sparse retriever. For the web search, we use the Google SERP provided by Serper3. For SERP results, we consider the top-3 organic results as well as the answer box (if available), resulting in 3 or 4 retrieval documents. We use wikipedia4 package to access raw Wikipedia content, and perform fuzzy matching to extract snippets in the document with fuzzywuzzy5 package. To ensure experimental fairness, we incorporate Google search results as additional knowledge for the baseline methods.
Hyperparameters Our main experiments are conducted using gpt-3.5-turbo-instruct provided by Azure OpenAI 12-01-preview version. The experiments on open-source models are conducted using Mistral-7B (Jiang et al., 2023a). Table 4 shows detailed hyperparameters.
2https://hotpotqa.github.io/wiki-readme.html 3https://serper.dev/ 4https://pypi.org/project/wikipedia/ 5https://pypi.org/project/fuzzywuzzy/
Evaluation We use token-level F1 as evaluation metric. Besides, if entity aliases are available, we calculate the maximum F1 score between the prediction and each ground truth alias.
Prompts For datasets except for Bamboogle, we adopt the question decomposition results provided by Cao et al. (2023). We convert their decomposition into our format and filter out those with invalid decomposition formats, regenerating them as necessary. For open-source models, limited by their capabilities, they conduct reasoning based on the question decomposition provided by GPT-3.5.
A.3 Details of Dataset
We provide statistics of the datasets, along with examples of each question type, as shown in Table 6.
HotpotQA consists of 2-hop Bridge and Comparison questions. Bridge: Inferring through the bridge entity to complete the 2-hop question. Comparison: Comparing two entities from the same category, including yes/no questions.
MuSiQue composites one-hop questions through bridge and composition into multi-hop complex questions, covering six categories of 2 to 4-hop questions. Bridge: Composite two single-hop questions into a 2-hop question through a bridge entity. Composition: A question is connected to two subquestions via two bridge entities, constituting a 3-hop question. Bridge and Composition: Combining 2-hop bridge questions and 3-hop composition questions in different order yields 4hop composition/bridge questions.
2WikiMQA includes four types of questions: bridge, inference, comparison, and bridge comparison. Bridge and comparison questions are similar to those in HotpotQA. Inference questions are similar to bridge questions, except they use inference relationships instead of bridge entities, such as grandfather of. Bridge-comparison: Comparing entities from two bridge sub-questions.
Bamboogle consists of 2-hop bridge questions, which are similar to 2hop (bridge) questions in MuSiQue. All questions in Bamboogle cannot be directly answered through search engines.
A.4 Details of Analysis
Distribution of knowledge integration In order to facilitate a fair comparison between log-probs and probabilistic aggregation in the preliminary study, we limit our statistical analysis to the proportion of internal and external knowledge contained within the top-1 answers in atomic questions. This ensures that the outcome of the statistics is not affected by the types of decomposition (iterative or sub-question retrieval) or different reasoning paths. The experimental results are averaged from 500 entries each from the HotpotQA and Musique datasets. We plot the discrete distributions (Histplot) and the kernel density estimate (KDEplot) in Figure 4. The y-axis is probabilistic density, for discrete distributions, the density is the percentage divided by the width of the bar. In this figure, the bar width w is set to 1{6 « 0.167. Contribution of each knowledge source Beginning with the final answer (the top-1 answer from the root node), we trace the reasoning path utilized from top to bottom, counting the knowledge sources that participated in the voting for subquestion answers along this path. Finally, we compute the percentage of knowledge sources within the entire path. To ascertain that this combination of knowledge aggregation led to the correct answer, we omit erroneous samples from our analysis.
Tendency of candidate answers distribution In the reasoning process, errors can be accumulated, leading to cascading errors. Through multiple reasoning paths and probabilistic aggregation, Beam Aggregation can gradually reduce the uncertainty. To illustrate this, we classify reasoning hop into three types: leaf, mid, and root. We statistically analyze the candidate answer distribution in each hop and use diversity, consistency, and uncertainty to
measure this distribution. Figure 6 shows a boxplot figure of these statistics. We also line the median value of each hop to show the trend more clearly. The analysis is conducted on the MuSiQue dataset, which has a clearer reasoning hop structure.
Token consumption We compare the efficiency between one-time retrieval (OneR), iterative retrieval (IRCoT, FLARE), and sub-question retrieval (ProbTree, Ours). We use a Pareto chart to visualize the token consumption of each method, as shown in Figure 7. The token consumption per instance and performance are averaged in four datasets. The upper left indicates better efficiency (less token consumption and higher performance). Detailed token consumption in each dataset is shown in Table 5. We will detail the method for calculating token consumption in the following section.
We measure the computational cost by evaluating the average token consumption per question. Specifically, the cost of each instance includes prompt tokens (demonstrations, question, retrieved documents) and completion tokens (reasoning traces, answer). For retrieval-augmented reasoning, the cost of a single API call is approximately 4000 tokens, whereas for non-retrieval reasoning, the cost is less than 1000 tokens.
A.5 Prompts We provide manually annotated demonstrations on the 2WikiMQA dataset for reference. Our prompts are derived from IRCoT6 (Trivedi et al., 2023) and ProbTree7 (Cao et al., 2023), to which we have made some modifications. The question decomposition is carried out using 24-shot demonstrations (Figure 10). For implicit internal knowledge, closed-book reasoning is conducted using 20-shot chain-of-thought demonstrations (Figure 11). For
6https://github.com/StonyBrookNLP/ircot 7https://github.com/THU-KEG/ProbTree
explicit internal knowledge, we first have the LLM generate parametric knowledge (Figure 12), followed by knowledge reasoning using 5-shot demonstrations (Figure 13). For external knowledge reasoning based on web search and Wikipedia, we use prompts of 5-shot and 3-shot (Figure 15, 14).
A.6 Formal Definition of Notations We describe the formal definition of the notations used in the algorithm pseudocode and formulas in this paper, as shown in Table 7.

heading: Future Work
text: In future work, structured external knowledge, such as knowledge bases, can be integrated into reasoning to provide more comprehensive knowledge repositories (Li et al. , 2024)