Making Long-Context Language Models Better Multi-Hop Reasoners (2024)

Abstract: Recent advancements in long-context modeling have enhanced language models (LMs) for complex tasks across multiple NLP applications. Despite this progress, we find that these models struggle with multi-hop reasoning and exhibit decreased performance in the presence of noisy contexts. In this paper, we introduce Reasoning with Attributions, a novel approach that prompts LMs to supply attributions for each assertion during their reasoning. We validate our approach through experiments on three multi-hop datasets, employing both proprietary and open-source models, and demonstrate its efficacy and resilience. Furthermore, we explore methods to augment reasoning capabilities via fine-tuning and offer an attribution-annotated dataset and a specialized training strategy. Our fine-tuned model achieves competitive performance on multi-hop reasoning benchmarks, closely paralleling proprietary LMs such as ChatGPT and Claude-instant1.

heading: 1 Introduction
text: The field of long-context modeling has garnered significant attention due to its importance in applications that demand extensive comprehension and generation capabilities (Lewis et al., 2020; Liu et al., 2023b). Techniques for long-context modeling (Chen et al., 2023a; Peng et al., 2023; Chen et al., 2023b) have been proposed with encouraging results on established benchmarks (An et al., 2023; Bai et al., 2023).
Nevertheless, we have identified a gap in the performance of these models when it comes to multi-hop reasoning tasks, where a model must navigate and synthesize information from disparate sources to answer complex questions. Evidence from key benchmarks such as LongBench (Bai
*Corresponding author. 1The dataset, model, and code are publicly available at
https://github.com/LaVi-Lab/LongContextReasoner.
et al., 2023), as well as our experimental results in Section 4.3, indicate that these long-context LMs underperform compared to leading multi-hop reasoning systems (Zhang et al., 2023). The reasons for this shortfall in multi-hop reasoning effectiveness are not yet fully understood.
We contend that the limitations in multi-hop reasoning observed in long-context LMs stem from two main issues: The inability to discern pertinent information within noisy contexts (Liu et al., 2023a) and the struggle to incorporate knowledge within the context effectively, particularly for smaller-scale models (Zheng et al., 2023a). To address these challenges, we introduce Reasoning with Attributions, a methodology that compels LMs to substantiate their reasoning by linking assertions to relevant context segments, such as citations (Gao et al., 2023) or direct quotations (Menick et al., 2022). This approach not only guides LMs to perform targeted information retrieval to identify the position of relevant contexts, thereby reducing noise, but also ensures their responses are wellgrounded in the source material. Our preliminary and comprehensive experimental findings, detailed in Sections 2.1 and 4.3, confirm the efficacy and resilience of this method across various multi-hop reasoning benchmarks.
Despite these advancements, smaller longcontext LMs exhibit continued difficulties in reasoning. We explore the potential for these models to improve through learning to reason and attribute simultaneously. Utilizing ChatGPT (Brown et al., 2020) to annotate the multi-hop reasoning dataset MuSiQue (Trivedi et al., 2022), we create a specialized dataset MuSiQue-Attribute for fine-tuning models in this dual capacity. We propose a potent learning strategy that leverages multi-task learning and data augmentation to fully exploit these annotations. Our experiments with five long-context LMs across three multi-hop reasoning datasets and two general instruction-following datasets reveal
2462
that our fine-tuned Vicuna-7B model (Zheng et al., 2023b) surpasses similar-scale baselines by a substantial margin, i.e., more than 20 points on average, and even outperforms ChatGPT and Claudeinstant on MuSiQue, albeit with a slight trade-off in other capabilities. This study illuminates a promising avenue to enhance the multi-hop reasoning capabilities of long-context LMs through a focus on attributions.
Our contributions can be summarized as follows: • We introduce Reasoning with Attributions, an
innovative reasoning paradigm that enhances both the performance and robustness of longcontext LMs in multi-hop reasoning tasks. • We provide MuSiQue-Attribute, an attributionannotated multi-hop reasoning dataset to support further research in this domain. • We develop a bespoke learning strategy that incorporates novel auxiliary tasks for multi-task learning and employs tailored data augmentation techniques. • Through rigorous testing on three multi-hop reasoning datasets and two general instructionfollowing benchmarks with both proprietary and open-source models, we demonstrate that our fine-tuned Vicuna-7B model achieves comparable multi-hop reasoning performance to ChatGPT with minimal impact on other capabilities. Additionally, our model shows resilience against varying degrees of contextual noise, underscoring the effectiveness of our methods.

heading: 2 Reasoning with Attributions

heading: 2.1 Pilot Study
text: The challenge of large language models becoming mired in irrelevant contexts, known as the “Lost in the Middle” phenomenon, has been documented across various NLP tasks, such as multi-document QA (Liu et al., 2023a) and mathematical reasoning (Shi et al., 2023). This issue is also apparent in multi-hop reasoning, which we illustrate later in Figure 2. Prior research has noted this problem but has not decoded the underlying mechanisms. For example, while Liu et al. (2023a) found that introducing the query before the context can aid in better information retrieval from the context, they did not achieve an improvement in QA performance using this query-aware approach. As we suggest in Section 1, the reasons might extend beyond mere retrieval challenges to include complications in ef-
fectively applying the retrieved knowledge. To tackle the issues outlined earlier, we introduce Reasoning with Attributions, a strategy that mandates language models to link the claims made during reasoning to specific sections of the provided context. This implicit requirement effectively decomposes a complex multi-hop question into two more manageable tasks: Pinpointing pertinent information within the context and constructing wellfounded claims based on that information.
We adapt the concept of Chain-of-Thought (CoT) (Wei et al., 2022) reasoning to create two distinct variants aligned with our attribution-based approach: Chain-of-Citation (CoC) and Chainof-Quote (CoQ). In CoC, models are prompted to reference citations corresponding to each step of the reasoning chain. CoQ goes further by requiring models to include direct quotations from the cited material for each reasoning step. An illustrative example highlighting the nuances between these methods is provided in Table 1.
The results of our preliminary study (Please refer to Section 4 for the setup), detailed in Ta-
ble 2, compare the efficacy of CoT, CoC, and CoQ when applied to two proprietary long-context LMs: ChatGPT (Brown et al., 2020) and Claudeinstant (Bai et al., 2022). Without further notice, ChatGPT always refers to gpt-3.5-turbo-1106 and claude-instant-1.2 for Claude-instant in this work. The findings suggest that both CoC and CoQ generally yield improvements over CoT, indicating that attribution-based reasoning enhances the precision and coherence of the models’ reasoning processes. CoQ appears to slightly underperform CoC, likely due to the increased complexity of producing exact quotations.
It is noteworthy that even in instances where CoT reduces the Answer Only (AO) performance, CoC is able to not only mitigate this decline but also surpass the AO baseline. This demonstrates the potential of CoC as a robust reasoning method. The success of our approach with various open-sourced models is further elaborated upon in Section 4.3. Based on these insights, we adopt CoC as our primary reasoning format in subsequent sections.

heading: 2.2 Dataset Curation
text: Our analysis, evidenced by the data in Tables 2 and 5, confirms that while reasoning with attributions holds promise, smaller open-source long-context language models significantly underperform compared to their proprietary counterparts in multi-hop reasoning tasks. To address this, we investigate whether training these models to perform attributions can boost their reasoning capabilities.
A hurdle in this process is the lack of attribution annotations within existing multi-hop reasoning
benchmarks. To bridge this gap, we have generated new annotations by prompting ChatGPT with 5-shot CoQ. This has been done to create CoT with attributions for 5,000 instances randomly selected from the answerable training set of the MuSiQue dataset (Trivedi et al., 2022). Although CoC generally outperforms CoQ, we chose CoQ for annotation because it provides more detailed information. This richness is beneficial not only for evaluating the quality of the annotations but also proves advantageous for the fine-tuning processes discussed in Section 3.
After generating the annotations, we implemented a filtering process to exclude annotations with any of the following errors (Please refer to Appendix E for implementation details):
• Incorrect Answer: The model’s predicted answer does not align with the reference answer, which typically indicates an erroneous CoT. • Non-Existent Attributions: Fabricated citations or quotes that do not correspond to the actual context are indicative of model hallucination. • Incorrect Citations: Citations do not match the manually identified supporting facts, suggesting flawed attributions. • Repeated Citations: Redundant citations contravene the multi-hop requirement of sourcing from multiple documents. • Extreme Quotes: Quotes that are either too terse (under five words) or excessively lengthy (spanning an entire document) lack utility.
Table 3 presents the substantial incidence rates
of each error type, which could negatively impact fine-tuning effectiveness. After filtering, we obtain a training dataset of 1,358 samples, referred to as MuSiQue-Attribute. The statistics of the MuSiQue-Attribute training set are outlined in Table 4. It is important to note that the hop distribution in MuSiQue-Attribute is skewed. This skewness arises both because generated CoT for questions with more hops is more prone to errors and because such questions represent a smaller fraction of the original MuSiQue training set. Additionally, we conducted a human evaluation of the MuSiQueAttribute quality in Appendix A.

heading: 3 Learning to Attribute in Reasoning
text: One intuitive approach to enhancing the multi-hop reasoning capabilities of LMs is to fine-tune them on our curated MuSiQue-Attribute, thereby teaching them to integrate attribution into their reasoning processes, specifically to generate CoC. Despite the simplicity of this method, our subsequent analysis in Section 4.4 demonstrates that this direct approach fails to produce robust results.
Multi-Task Learning. Beyond simply finetuning LMs on the MuSiQue-Attribute to learn to attribute in reasoning (denoted as LA), we propose three auxiliary tasks that serve as simplified analogs of LA. These tasks are designed to train LMs in conjunction with LA to enhance their proficiency in attribution-based reasoning:
• Answer Prediction (AP for short): This task focuses on direct answer prediction without the need for an explicit reasoning process. AP is intended to help LMs internalize the reasoning needed for straightforward questions where CoT is not required. • CoT Generation (CG for short): In the CG task, models are trained to generate a CoT before providing an answer. This is aimed at
developing LMs’ abilities to reason explicitly and methodically across multiple pieces of information for complex questions. • Quotes Identification (QI for short): This task trains models to pinpoint critical quotes for reasoning. QI is designed to fine-tune the ability of LMs to filter out irrelevant details and zero in on the pertinent segments of text, thereby sharpening the accuracy of reasoning. Figure 1 illustrates the distinctions between our primary LA task and the three auxiliary tasks.
Data Augmentation. A recognized limitation of direct fine-tuning on our MuSiQue-Attribute is the potential for models to develop biases, such as favoring certain locations of relevant documents (Liu et al., 2023a), sensitive to a fixed number of documents, or accommodating only a narrow range of noise levels. To counteract these biases, we have devised the following data augmentation strategies:
• Distractor Sampling: By randomly selecting a varying number of irrelevant documents, we modify the positioning of relevant documents and the total document count within the context. This approach also mimics the fluctuating noise levels encountered in real-world scenarios, training language models to cope with noisy contexts effectively. • Document Shuffling: Reordering the documents helps to remove any superficial positional cues that could lead to reasoning bias. For example, this ensures that models do not learn to associate the sequence of relevant documents with a fixed reasoning chain sequence. These data augmentation strategies are applied in sequence for each training instance.

heading: 4 Experiments

heading: 4.1 Datasets
text: Our method’s effectiveness in multi-hop reasoning is assessed on the following datasets: HotpotQA (Yang et al., 2018), 2WikiMultiHopQA (2Wiki for short) (Ho et al., 2020), and MuSiQue (Trivedi et al., 2022). For each question, we provide a context composed of shuffled relevant and irrelevant documents. These irrelevant documents are the official retrieved distractor documents. We adopt the development and test sets from Trivedi et al. (2023) for evaluation, which contains 100 and 500 examples respectively. The results we present are the mean values from three separate trials, each with a distinct random seed.
To understand the broader impact of enhancing multi-hop reasoning on LMs’ overall capabilities, we also conduct evaluations on general instructionfollowing benchmarks, namely MT-Bench (Zheng et al., 2023b) and AlpacaEval (Li et al., 2023c).

heading: 4.2 Models
text: The following long-context baselines are chosen in our experiments.
• ChatGPT (Brown et al., 2020). We choose gpt-3.5-turbo-1106, which supports a window size of 16K tokens. • Claude-instant (Bai et al., 2022). We choose claude-instant-1.2, which has a window size of 100K tokens. • LongChat (Li et al., 2023a). We use longchat-7b-16k, a 7B fine-tuned LLaMA model (Touvron et al., 2023a). It has a window size of 16K tokens. • LongLoRA (Chen et al., 2023b). We use LongAlpaca-7B-16k, which has a window size of 16K. • Vicuna (Zheng et al., 2023b). We use vicuna-7b-v1.5-16k, a 7B fine-tuned LLaMA-2 model (Touvron et al., 2023b). It supports a window size of 16K tokens.
We prompt all models with 5-shot to evaluate their multi-hop reasoning performance. These 5 demonstrations are randomly sampled from the 20 annotated training examples provided by Trivedi
et al. (2023). If the input length exceeds the window size, we drop the last demonstration until the input length fits. The prompts we used are presented in Appendix D.
Our model AttrLoRA is fine-tuned on vicuna-7b-v1.5-16k with LoRA (Hu et al., 2022), following hyper-parameters used in FastChat (Zheng et al., 2023b). For its training data, we perform augmentations to double the training data for all tasks in Section 3 except for the QI task. Note that we subsample same-sized instructiontuning data from the Alpaca dataset (Taori et al., 2023) and mix it with the reasoning data. These instruction-tuning data serve the purpose of minimizing the risk of hampering other abilities Vicuna already possesses before fine-tuning.

heading: 4.3 Main Results
text: Effectiveness and Robustness of Reasoning with Attributions. The results in Table 5 underscore the efficacy of our CoC prompting across three multihop reasoning datasets, benchmarked against five baselines. In 77% of the evaluated cases (disregarding instances of near-zero model performance) CoC outperforms CoT. Notably, Claude-instant exhibits strong results with AO, and its performance diminishes when CoT is used. However, CoC not only mitigates this decline but also attains results on par with AO, demonstrating the robustness of attribution-based reasoning. For a detailed analysis
of CoC’s robustness, particularly against varying degrees of contextual noise, see Appendix C.
Performance of AttrLoRA Against Proprietary Models. Table 5 presents a zero-shot performance comparison between our AttrLoRA and five-shot outcomes from various baselines. AttrLoRA surpasses baselines of comparable scale by an average margin of over 20 points. It exceeds the performance of two notable proprietary models on MuSiQue and delivers closely competitive results on the other two benchmarks.
In particular, AttrLoRA with AO achieves superior results on 2Wiki and surpasses CoT on HotpotQA. This can be attributed to the relative simplicity of these datasets, where explicit reasoning does not significantly enhance performance. For instance, CoT’s advantage is noticeably smaller on these datasets compared to MuSiQue for both the ChatGPT and Claude-instant. Additionally, according to Jiang and Bansal (2019), over half of the “bridge-type” questions in HotpotQA contain shortcuts, which can locate the answer by keyword matching, circumventing the need for the intended two-hop reasoning. Similarly, 2Wiki’s predictable nature, due to its question construction from a lim-
ited set of rules, simplifies the task for LMs. Another contributing factor is that AttrLoRA is trained on MuSiQue-Attribute, which does not encompass the full range of question types found in 2Wiki and HotpotQA, such as “comparison-type” questions. Resilience of AttrLoRA to Noisy Contexts. A key aspect of AttrLoRA is its robustness to contextual noise. To investigate this, Figure 2 illustrates AttrLoRA’s performance against varying degrees of synthesized noise. This synthesized noise is implemented by adding varied numbers of random irrelevant documents to the context. The data indicates that while the performance of baseline models markedly declines with increased noise, e.g., Vicuna drops by over 30 points on MuSiQue, AttrLoRA shows greater resilience, with a reduction of only about 10 points. Impact of Attribution Learning on General Abilities. Our investigation extends beyond multi-hop reasoning to examine how attribution learning affects AttrLoRA’s general instruction-following capabilities post-fine-tuning, as compared to the Vicuna baseline. The results in Table 6 from two
instruction-following benchmarks reveal that finetuning slightly compromises abilities beyond multihop reasoning in a 7B model due to capacity constraints. However, a closer analysis reveals that over 98% of the performance decrease is attributed to fine-tuning with Alpaca data (“+ Alpaca Data”), while multi-hop reasoning data incurs less than a 2% detriment. This is because the quality of Alpaca data is inferior to Vicuna’s, with the former being single-turn GPT-3 synthesized and the latter comprising multi-turn human-bot conversations.

heading: 4.4 Analysis
text: Attribution Quality of AttrLoRA. Drawing from the insights of Gao et al. (2023), we scrutinize the citation precision and recall for AttrLoRA, as presented in Figure 3. The model demonstrates high precision, indicating its proficiency in correctly attributing statements to pertinent documents. Nonetheless, the moderate recall highlights that AttrLoRA does not consistently identify all relevant documents, a potential consequence of the disconnected reasoning patterns observed in MuSiQueAttribute (detailed in Appendix A). Further exploration of the correlation between attribution quality and reasoning performance is in Appendix B. The Effectiveness of Multi-Task Learning. Our ablation study in Table 7 assesses our multi-task learning approach. Results indicate a marked enhancement in Vicuna’s reasoning capabilities upon fine-tuning with our dataset (“+ AP”). However, explicitly training Vicuna to generate CoT (“+ CG”) yields mixed outcomes: It benefits performance on
MuSiQue but adversely affects results on 2Wiki and HotpotQA. This discrepancy can be attributed to the relative ease of the latter datasets, where simpler questions and shortcuts reduce the effectiveness of complex reasoning strategies, as discussed in Section 4.3. Importantly, integrating the LA task (“+ LA”) mitigates the performance drops associated with CoT and notably boosts MuSiQue scores. This implies that attributions are instrumental in enabling the model to reason over complicated questions without compromising its ability to handle simpler queries. Finally, the addition of the QI task (“+ QI”) appears to further refine the model’s multihop reasoning proficiency, underscoring the value of our multi-task learning framework. The Effectiveness of Data Augmentation. We explore the impact of our data augmentation strategy, intentionally omitting the QI task, as it alone is insufficient for training models to conduct multi-hop reasoning. The data in Table 8 demonstrates that including augmented data generally enhances model performance across various datasets. However, augmenting CG and LA data does not yield improvements on 2Wiki. In this case, the model readily learns from a limited amount of annotated data due to the simplicity of the automatically generated questions within 2Wiki. Conversely, on MuSiQue and HotpotQA, which feature more complex and varied human-crafted questions, the model benefits from exposure to a larger dataset to accommodate the diversity of question formulations. The Effectiveness of Scaling Fine-Tuning Data. In Figure 4, we investigate how the expansion of fine-tuning data influences model performance. It is evident that incorporating additional data steadily enhances performance on MuSiQue and 2Wiki, while optimal results are attained with just 60% of our data for HotpotQA. This fact suggests that more
complex question answering, involving additional reasoning steps like MuSiQue and 2Wiki, demands a larger dataset. An intriguing discovery is that using a mere 20% of our data achieves approximately 85% of the peak performance. This highlights the efficiency of fine-tuning: Even a modest subset of multi-hop reasoning examples can significantly boost the model’s reasoning capabilities. Case Study. Table 9 presents a comparative case study where Vicuna and AttrLoRA are both prompted to generate CoC. Within the provided examples, AttrLoRA successfully produces coherent CoT and precisely attributes each claim. In contrast, Vicuna yields answers without engaging in an explicit reasoning process.

heading: 5 Related Work
text: Multi-Hop Reasoning. Multi-hop reasoning in open-domain question answering requires the synthesis and analysis of disparate facts across various documents to formulate a response. Key datasets in this field include HotpotQA (Yang et al., 2018), 2Wiki (Ho et al., 2020), and MuSiQue (Trivedi et al., 2022), which predominantly adopt a reading comprehension framework with pre-retrieved documents supplied by the creators. Traditional approaches often utilize a selector-reader model (Zhang et al., 2023; Zhu et al., 2021), where the selector is tasked with pinpointing relevant documents from the provided set, and the reader constructs an answer based on these selections.
Recent advances, however, pivot towards a paradigm that leverages long-context LMs (Khot et al., 2023; Trivedi et al., 2023). In this approach, the role of the selector is phased out, and instead, the entirety of the retrieved documents is processed by a long-context LM, which acts as the reader.
Our study aligns with this emergent research trend, particularly focusing on the use of attributions to enhance the performance of multi-hop reasoning within this long-context LM framework.
Context Utilization. The recent advent of longcontext LMs has shown promise (Li et al., 2023a; Zheng et al., 2023b; Chen et al., 2023b). However, these models often struggle with noisy contexts. Shi et al. (2023) demonstrate that superfluous sentences can significantly disrupt mathematical reasoning. Liu et al. (2023a) identify a relevant document position bias in multi-document QA. Wu et al. (2024) show that LMs could be easily distracted by retrieved irrelevant inputs.
To mitigate the impact of irrelevant context, Shi et al. (2023) prompt models to disregard such information and adopt self-consistency techniques (Wang et al., 2023). Creswell et al. (2023) suggest a two-stage approach that focuses on fact selection prior to reasoning. Echoing this approach, Yu et al. (2023) introduce Chain-of-Note which entails reviewing document relevance before providing an answer. Meanwhile, Yoran et al. (2023) examine automatic data generation for training more robust models. Our research contributes to this domain by investigating the use of attributions as a novel method for effective context utilization.
Language Models Attribution. Attribution in language models constitutes a nascent area of study, primarily aimed at identifying and mitigating hallucination (Li et al., 2023b). A line of research concentrates on post-retrieval answering: Models provide responses based on retrieved results with cited attributions (Nakano et al., 2021; Menick et al., 2022; Gao et al., 2023). Our research emerges from this foundation but diverges in its applica-
tion; We focus on multi-hop reasoning rather than hallucination reduction. Moreover, we delve into optimizing training methodologies to maximize the efficacy of scarce attribution annotations.

heading: 6 Conclusion
text: This study demonstrates that long-context LMs face challenges with multi-hop reasoning within noisy contexts. We introduce a reasoning paradigm that incorporates attributions, which significantly improves the reasoning capabilities of long-context LMs. Alongside, we contribute a new dataset annotated with attributions and study training strategies tailored for multi-hop reasoning. Our comprehensive experiments across five models and five benchmarks validate the superiority of our approach in enhancing multi-hop reasoning performance.
Limitations
The proposed reasoning with attributions method currently leverages citations and quotations, leaving room for future exploration of other attribution forms, such as URLs. This work concentrates on refining training strategies, yet the potential of custom model architectures for this task remains untapped. Additionally, our approach relies on contexts pre-supplied by dataset creators. Emerging research suggests that language models integrated with search engines can achieve enhanced outcomes. A promising avenue for further research lies in developing models that not only manage noisy contexts more effectively, as demonstrated in our work, but also actively engage with search tools to improve information retrieval.
Ethics Statement
Our dataset builds upon MuSiQue, adhering to its original copyright provisions; We distribute our supplementary annotations under the CC BY 4.0 license. While our model-generated annotations could potentially include misinformation or harmful content, our manual quality review in Appendix A did not encounter such instances. Throughout the human annotation phase, we did not gather any demographic data or information that could reveal the identity of the annotators. All annotators provided informed consent for the use of their annotations exclusively for research purposes.

heading: Acknowledgements
text: This work was supported by National Key R&D Program of China (Project No. 2022ZD0161200, 2022ZD0161201). This work is also supported by Hong Kong Research Grant Council - Early Career Scheme (Grant No. 24200223) and Hong Kong Innovation and Technology Commission Project No. ITS/228/22FP. This work was also partially funded by the Centre for Perceptual and Interactive Intelligence (CPII) Ltd under the Innovation and Technology Commission (ITC)’s InnoHK.

heading: A Human Assessment of MuSiQue-Attribute
text: We recruit a student possessing expertise in NLP and holding a Master’s degree in Computer Science to annotate 100 random samples from our MuSiQue-Attribute. The tool we used for annotations is shown in Figure 5. The annotation results reveal that 19% of the CoT annotations exhibited reasoning issues despite correct final answers, a problem referred to as reasoning unfaithfulness (Creswell and Shanahan, 2022). These erroneous CoTs fall into three categories: 5.26% with Disordered Steps where reasoning steps are improperly arranged, 78.95% with Missing Steps indicating omitted essential reasoning steps, also known as disconnected reasoning (Trivedi et al., 2020), and 10.53% with Incorrect Steps containing invalid reasoning steps. Our assessment also identified that 9% of questions in the dataset allowed for shortcuts (Jiang and Bansal, 2019), enabling models to find answers by keyword matching without reasoning.
The study further showed that within incorrect CoT annotations, 47.37% pertained to 2-hop questions, 36.84% to 3-hop, and 15.79% to 4-hop questions. Considering the distribution of hops in our dataset, the unfaithfulness issue affected 11.84% of 2-hop CoTs, 38.89% of 3-hop CoTs, and 50% of 4-hop CoTs, suggesting a decrease in reliability of LMs with the increase in reasoning steps required.

heading: B Attribution Quality and Reasoning Performance
text: Figure 6 presents a visualization of the relationship between citation precision or recall and multi-hop
reasoning performance. Our analysis identifies a notable positive correlation between citation precision and multi-hop reasoning capabilities. Table 10 provides statistical support for this observation, with correlation coefficients indicating a significant positive correlation. These results suggest the potential of using citation precision as a reference-free proxy for assessing multi-hop reasoning performance, which could be facilitated by Natural Language Inference (NLI) methods (Gao et al., 2023).

heading: C Robustness to Noisy Context
text: We assessed the resilience of CoT and CoC against varying degrees of contextual noise by evaluating the performance variability of different models across multi-hop reasoning benchmarks. Table 11 indicates that in 84% of the cases, CoC exhibits a smaller performance range compared to CoT. Notably, even in instances where CoC demonstrates a greater range, the disparity with CoT remains marginal. These findings suggest that CoC is generally more robust to noisy contexts than CoT.

heading: D Prompting Details
text: In this section, we highlight some details when prompting long-context LMs with various strategies, e.g., AO, CoT, CoC and CoQ.
• Each prompting strategy has its own instructions. • For CoT, CoC and CoQ, we add “Think stepby-step.” to the end of the question. • For few-shot prompting, we put demonstrations to different turns as the dialogue history. • Each demonstration is formatted according to Table 1.
The instruction for AO is (unique descriptions are underlined):
Write an accurate and concise answer for the given question using only the provided search results (some of which might be irrelevant). Do not say anything other than the answer itself.
The instruction for CoT is (differences to AO are underlined):
Write an accurate and concise answer for the given question using only the provided search results (some of which might be irrelevant). Start with an accu-
rate, engaging, and concise explanation based only on the provided documents. Must end with “The answer is:”. Use an unbiased and journalistic tone.
The instruction for CoC is (differences to CoT are underlined):
Write an accurate and concise answer for the given question using only the provided search results (some of which might be irrelevant) and cite them properly. Start with an accurate, engaging,
and concise explanation based only on the provided documents. Must end with “The answer is:”. Use an unbiased and journalistic tone. Always cite for any factual claim.
The instruction for CoQ is (differences to CoC are underlined):
Write an accurate and concise answer for the given question using only the provided search results (some of which might be irrelevant) and cite them properly. Start with an accurate, engaging, and concise explanation based only on the provided documents. Must end with “The answer is:”. Use an unbiased and journalistic tone. Always cite and extract word-for-word quotes for any factual claim.

heading: E Filtering Implementation
text: The implementation of each filtering strategy is detailed as follows:
• Incorrect Answer: Leveraging the evaluation script from QuAC2, we normalize both model predictions and official reference answers. We filter out data instances where the model prediction does not exactly match the answer.
• Non-Existent Attributions: Utilizing regular expressions, we extract all citations and verify if the predicted citations are present in the context. Each quote is checked for its exact presence in the cited document, and instances with fabricated citations or quotes are removed.
• Incorrect Citations: We assess the correctness of predicted citations using officially annotated supporting documents, ensuring the cited document is among the supporting documents. Any example with at least one incorrect citation is deleted.
• Repeated Citations: Regular expressions are used to extract all citations, and we check for duplicates. Examples with any duplicate citations are discarded.
2https://s3.amazonaws.com/my89public/quac/ scorer.py
• Extreme Quotes: We extract all quotes using regular expressions and tokenize them with NLTK3. Examples are kept only if all quotes contain more than five words but do not span entire cited documents.

heading: F Training and Inference Cost
text: For fine-tuning using LoRA (Hu et al., 2022), we employed ZeRO-3 (Rajbhandari et al., 2021) optimization and gradient checkpointing (Chen et al., 2016) techniques. The model was trained on 8 NVIDIA A100 80GB GPUs for no more than 14 hours. Approximately 4M parameters were tuned, constituting 6.22% of the total parameters. For inference, each test set was processed in about 30 minutes using a single NVIDIA A100 80GB GPU.
3https://www.nltk.org/

heading: Future Work
text: Limitations
The proposed reasoning with attributions method currently leverages citations and quotations, leaving room for future exploration of other attribution forms, such as URLs. This work concentrates on refining training strategies, yet the potential of custom model architectures for this task remains untapped. Additionally, our approach relies on contexts pre-supplied by dataset creators. Emerging research suggests that language models integrated with search engines can achieve enhanced outcomes. A promising avenue for further research lies in developing models that not only manage noisy contexts more effectively, as demonstrated in our work, but also actively engage with search tools to improve information retrieval