Venue: NAACL 2024

# Optimizing LLM Based Retrieval Augmented Generation Pipelines in the Financial Domain

Yiyun Zhao\({}^{1\lx@paragraphsign}\), Hanoz Bhathena\({}^{1\lx@paragraphsign}\), Prateek Singh\({}^{1\lx@paragraphsign}\), Saket Sharma\({}^{1\lx@paragraphsign}\), Bernardo Ramos\({}^{\lx@paragraphsign}\), Aviral Joshi\({}^{\sharp}\), Swaroop Gadiyaram\({}^{\lx@paragraphsign}\)

\({}^{\lx@paragraphsign}\)Machine Learning Center of Excellence, JPMorgan Chase & Co.

hanoz.bhathena@jpmchase.com

###### Abstract

Retrieval Augmented Generation (RAG) is a prominent approach in real-word applications for grounding large language model (LLM) generations in up-to-date and domain-specific knowledge. However, there is a lack of systematic investigations of the impact of each component (retrieval pipeline, prompts, generation models) on the generation quality of a RAG pipeline in real world scenarios. In this study, we benchmark 6 LLMs in 15 retrieval scenarios, exploring 9 prompts over 2 real world financial domain datasets. We thoroughly discuss the impact of each component in RAG pipeline on answer generation quality and formulate specific recommendations for the design of RAG systems.

+
Footnote †: Equal contribution.

## 1 Introduction

Recent years have seen tremendous improvement in the ability of large language models (LLM) such as GPT-4 OpenAI et al. (2023) and Llamada-2 Touvron et al. (2023) to address users' questions/queries in diverse domains (medical questions, math problems, code assistants etc). Despite LLMs acquiring immense parametric world knowledge during the pre-training, when adapting to real-world applications, their lack of customized domain-specific knowledge or knowledge of recent events Kandpal et al. (2023); Sun et al. (2023), frequently results in outdated responses or baseless responses not grounded in the user's domain of interest, also termed _hallucinations_ Bang et al. (2023); Rawte et al. (2023); Li et al. (2023). Hallucinations contribute to a lack of trust with users, and this unreliability is one of the biggest hindrances in the responsible deployment of LLM based systems for critical business applications in the financial domain.

Retrieval Augmented Generation (RAG) is the current go-to approach to connect LLMs to live/updated information sources. Existing works Lazaridou et al. (2022); Shuster et al. (2021); Ren et al. (2023) show RAG can reduce hallucinations and improve answer quality, without the need for highly expensive and sometimes brittle domain-specific fine-tuning.

Given a user query, a typical RAG system (Figure 1) employs a retriever system to fetch a list of documents likely relevant to the query from an information source (Retrieval). The documents are then fed into the context of the LLM, with users' query / conversation history, and specific instructions / prompts on how to generate a response "grounded" in retrieved information (Generation).

While there is growing number of proposals Jiang et al. (2023); Siriwardhana et al. (2023) to improve RAG systems (see the survey from Gao et al. (2023)), very few studies Chen et al. (2023b) systematically investigate the impact of each component (retriever, prompts, models) on answer generation quality and interactions among these various components. Our goal of this paper is to evaluate the efficacy and limits of RAG pipelines for Question Answering (Q&A) systems in the highly specialized financial domain.

In this study, we benchmark LLMs' answer generation quality and explore the following aspects: (i) Comparing different generative LLMs as answer generation models against each other and baseline (purely extractive) models; (ii) Examining how various LLMs handle differences in the quality of information retrieval; (iii) Exploring the impact of varying prompts on answer quality of RAG pipelines.

In line with our objectives, we curated two datasets from the banking sector featuring real user queries. These datasets were used to design test scenarios that mimic the retrieval of information at varying levels of quality. Additionally, we crafted prompts with distinct characteristics (e.g., level of detail in instructions, requirements for citations,

response format) and conducted evaluations on six LLMs. In addition to answer generation quality, we also evaluate LLMs' ability to adhere to instructions on aspects such as answer style, citation output format etc.

Our findings reveal that generative LLMs outperform baseline models in answer quality, even on metrics emphasizing extractiveness which could ideally have given extractive models an advantage. As expected, GPT-4 demonstrates superior performance over GPT-3.5-Turbo, which in turn outperforms the LLama-2 models. We observed that the generation quality is highly influenced by the quality of retrieval: LLMs tend to provide answers even when relevant source information is missing, a form of _pseudo-helpfulness_ varying from responding with content present in the context that is somewhat related but not addressing the user question, to hallucinations. The performance of even SOTA LLMs like GPT-4 declines with an increase in number of distractor documents retrieved or when relevant documents are not ranked higher in the retrieval list. Interestingly, we did not find a systematic impact of prompt characteristics on the quality of answer generation especially on OpenAI models: GPT-4 and GPT-3.5-turbo models are resilient to prompt variations, whereas the Llama-2 models exhibit more variability. Finally, GPT-4 and GPT-3.5-turbo exhibit compliance to instructions around structured formatting and language style over 90% of the time, whereas Llama-2 models struggle to follow instructions. To summarize, the main contributions of the current studies are as follows:

* We comprehensively assess various factors that contribute to the answer generation quality in LLM RAG systems, ranging from sensitivity to retrieval quality to the impact of different prompts, conducted across 6 LLMs on two datasets in the financial (banking) sector with real user queries, enabling the testing of RAG systems in realistic scenarios 1. Our evaluations on internal data absent in the LLMs' pre-training also represents a better estimate of real-world generalization of LLMs. Footnote 1: Unfortunately we cannot release this dataset due to confidentiality concerns.
* We conclude with specific recommendations for the design of RAG systems, grounded in the insights and findings derived from our empirical results.

## 2 Experiment Framework

This section introduces the design of the evaluation framework (see Figure 1). To summarize, we ran 1620 experiments to assess 6 LLMs in 15 retrieval conditions using 9 prompts over 2 datasets for 2 performance aspects.

### Evaluation Dataset Construction

In our experiments, we developed two RAG datasets from queries against two corpora: (1) **Banking webpages**: Public webpages with general information on banking products, and (2) **Banking policy guides**: internal guides for customer service executives detailing policies and protocols for customer assistance. For both the corpora, we had associated questions, which were either generated by the actual users and gathered from production system logs, or were generated by subject-matter-experts.

We chunked webpages/articles into about 100 word document chunks (also referred as documents) while preserving sentence boundaries. Chunks with majority content in a non-English language or those with fewer than 10 words were dropped. We paired each question to related documents via unifying subject matter experts' coarse

Figure 1: Evaluation Framework of a RAG system



annotation and two-staged models (see details Figure 6 in Appendix ). The questions paired with documents were then sent to the human reviewers to (i) assign a binary relevance label to every chunk and (ii) label an answer span within the chunk, served as reference answer.

### Simulated Scenarios of Different Retrieval Quality

The success of highly-customized RAG applications hinges on the quality of the retriever component. Understanding how LLMs are affected by retrieved documents is vital for developing effective enhancement strategies and further research in RAG systems.

We tackle this by designing different test sets. We sampled 800 user queries for each dataset and obtained retrieved document chunks using a retriever(OpenAI Embeddings2). We then manipulate the retrieved list to mimic retrievers of different qualities to address scenarios listed below.

Footnote 2: [https://platform.openai.com/docs/guides/embeddings](https://platform.openai.com/docs/guides/embeddings)

Q1. How does absence of retrieved "gold" document influence answer generation?We created two retrieval conditions: Retriever-Only (returning the retrieved set which may or may not contain gold chunk(s)) versus Retriever-W-GT (guaranteed to have gold document to the retrieved list)3.

Footnote 3: in case we need to add we place gold doc in the first position

Q2. How does the number of documents retrieved influence answer generation?We returned three conditions varying in the number of documents displayed to generation models: top_3, top_5, top_20.

Q3. How does order of retrieved documents influence answer generation?We further manipulated the display order of retrieved documents during LLMs' response generation. For Retriever-Only, we added a new condition where we simply shuffled the order of the documents to judge the sensitivity to order in general. For Retriver-W-GT, we added two conditions where we injected the gold document in the first or last position. In summary, we created 15 retrieval conditions. For retrieval_only, we designed 2 (retriever_only, reviewer_only_shuffled) x 3 (top 3, 5, 20); for retriever_w_gt, we designed 3 (retriever_w_gt, retriever_w_gt_first, retriever_w_gt_last) x 3 (top 3, 5, 20).

### Answer Generation Models

The quality of responses in Retrieval-Augmented Generation (RAG) systems is significantly influenced by the choice of Answer Generation Models. Here we compare the performance of several LLMs. We also report the performance of a RoBERTa Model, as our baseline.

Baseline ModelSince our dataset contains answer spans, for our baselines we use an encoder-only answer-span extraction model (Roberta finetuned on SQuAD2 Rajpurkar et al., 2016, 2018) and Natural-Questions Kwiatkowski et al. (2019) datasets).

Generative Large Language ModelsWe assess 6 frequently-used LLMs including gpt-3.5-turbo-0613, GPT-4-0613 OpenAI et al. (2023), and Llama-2-7B and Llama-2-13B (base and chat)Touvron et al. (2023). The details of experimental parameters for each model are fully specified in Table 1 in Appendix.

### Prompts

Previous research indicates that the performance of Large Language Models (LLMs) can be affected by the prompts used Chen et al. (2023); Zhu et al. (2023). In this work, we investigate the effect of prompting on generation in RAG pipelines. In particular, we created a set of prompts with variations in factors such as the verbosity of instructions, the need for direct quoting, explicit introduction of metrics within prompts, the requirement for citations, and specific response formatting, among other aspects. The full list of prompts experimented in the study can be found in Appendix (Figure 11, Figure 12, Figure 13).

### Evaluation Metrics and Aspects

To assess the performance of RAG system, we evaluate the answer quality, and instruction following ability of the LLMs.

Answer QualityDue to the extractive nature of our tasks (2.1) we followed Ren et al. (2023) using token F1 scores which show reasonable correlation with human subjects Adlakha et al. (2023). 4.

 

Instruction Following AbilityPractical RAG deployments typically require the LLM returns answer in a specific language style and may require citations in certain structured format that can be extracted from the model response. Thus, it is important to measure instruction-following performance.

* For structured output, we designed prompts that require pipe format (**answer_span>||<Document_ID>**) and JSON format ({ "text": **canswer_span>, "source_id": <Document_ID>}). We calculated the proportion of the output that correctly produced the expected formatting for each model.
* For language-style output, we designed a prompt that requires direct quoting from the source. Therefore, we calculated the the proportion of the sentences of responses that are directly taken from the source retrieved documents for each model.

## 3 Results

### Influence of Retrieval Quality on Answer Generation Quality

Figure 2 displays different models' performance on 5 retrieval conditions with 3 different number of retrieved documents for Banking Policy guides dataset (Complete results can be found in figure 9 and 10 in Appendix).

Does the presence of gold document matter?To address the question, we compare performance in Retriever-Only Conditionswith that in Retriever-W-GT-X Conditionsfor each model in Figure 2. Across models, we observe that performance generally improves when a source verified to contain answer is retrieved. In other words, even powerful LLMs such as GPT-4, are imperfect at rejecting to answer ("No Answer Found") in cases of retrieval failure6. However, GPT LLMs are better at gracefully handling retrieval failures compared to other models we consider. This behaviour represents a form of _pseudo-helpfulness_ in LLM based RAG systems, wherein LLMs try to be _helpful_ even when relevant information is missing in their context, overriding the typical expectation injected in RAG systems to only use information in the context that addresses the question. This phenomenon manifests as responses containing related content not addressing the user's question, and hallucinations.

Footnote 6: We consider “No answer found” or equivalent to be correct behavior in our metrics for cases where gold document is not retrieved

Figure 2: Answer generation quality (Token F1) of baseline model, GPT-4, gpt-3.5-turbo and Llama-2-13b for Banking policy dataset. Full results can be found in Appendix (Figure 9 and Figure 10).

 

Does the number of retrieved docs matter?To answer this question, we compare how models' performance varies across different top-k conditions (3,5,20). If the number of retrieved documents is irrelevant to answer generation quality for a model, we would expect the models' performance to not vary across different top-k scenarios. From Figure 2, we observe across models the influence of top-K varies between Retrieved-Only Conditions and Retrieval-W-GT-X Conditions.
* For [Retriever-W-GT-X Conditions], models' performance tends to decline when the number of retrieved documents increases: TOP-K-3 > TOP-K-5 > TOP-K-20. This decline is less pronounced when the gold document appears near the top of the retrieved list for GPT-4 and GPT-3.5-turbo. This indicates that models are in general sensitive to noisy documents in the retrieved list but GPT models are less distracted if the good document is placed in the top position.
* For [Retriever-Only Conditions], models' performance increases with the number of documents retrieved. This increase potentially is due to the improvement of gold document recall as LLMs are better at finding gold spans when they exist than rejecting to answer in absence of gold information.

Overall, our results indicate that we cannot reduce our retrieval optimization objective to maximize recall due to LLMs' sensitivity to retrieval noise.

### Does the order of retrieved documents matter?

To assess the influence of order, for each top-k, we compare model performance bar (that is bar with the same color) among retriever_w_gt_first v.s. retriever_w_gt t v.s. retriever_w_gt_last. From Figure 2 we observe across different LLMs (We ignore the baseline model whose pairwise implementation is insensitive to the display order by design), the performance varies noticeably across the different ordering conditions. Specifically, placing gold document in the first position leads to a better performance than in the last position. This discrepancy is more obvious when a higher number of documents are retrieved. For retriever_only conditions, where we compare against retriever_only_shuffled, the order does not show much influence presumably due to presence of retrieval failure cases that reduces the gap between naive retrieved ordering versus shuffled ordering. Overall, our results indicate that investing in a re-ranking system as part of retrieval optimization is still necessary with LLM based pipelines.

### Influence of Choice of Generation Models

This section compares different LLMs on answer generation quality. Figure 2 has shown that baseline models under-perform the other generative LLMs counterparts by a large margin. We compare the 6 LLMs using a head-to-head win rate across all experiments. Figure 3 demonstrates the win rate of a model when compared to another model across all the experiments, which shows the overall tendency: GPT-4 \(\triangleright\) GPT-3.5-turbo \(\triangleright\) Llama-2-13b > llama-2-13b-chat, llama-2-7b, llama-2-7b-chat.

Figure 4: Heatmap of Token F1 for 9 prompts X 6 Models on Banking Policy guides. Results for Banking webpages can be found in appendix (Figure 8).

Figure 3: Head-to-head pairwise win rate in Token F1 for Banking Policy guides. Results for Banking webpages can be found in appendix (Figure 7).

 



### Influence of Prompts

This section investigates model robustness to different prompts. Figure 4 captures the answer quality of different models (horizontal axis) with different prompts (vertical axis) for the Banking Policy dataset. We observe that GPT-4 is most robust to the prompts with max difference less than 4% followed by GPT-3.5-turbo about 6%. The two Llama-2 base models we tested show huge variations (around 15%) compared to the chat version. Though there is consistency in prompting trends across two datasets (Figure 8 in Appendix), we did not observe any salient effect of prompt features (such as verbosity, quoting or citation requirement etc) on answer generation quality.

### Instruction-following Accuracy

Structured OutputWe assess the proportion of models' output that followed correct output format (ll and JSON). From Figure 5, we observe that both GPT-4 and GPT-3.5-turbo follow instructions (> 94%) in both styles (lll or JSON). By contrast, for Llama-2 models, the chat models outperform base models but are far behind the OpenAI models in the "ll" citation instruction. All Llama-2 models tested barely produced a parseable JSON format with required fields.

Quoting as Answer StyleFigure 5 indicates that both GPT-4 and GPT-3.5-turbo quote more than 90% of the times. The Llama-2-base models also show high level of quoting (80%) compared to the chat versions (50%). However, qualitative analysis reveals that base models tend to copy line by line from the source text regardless of its relevance to the question, thereby, _cheating_ the metric. This can also be observed by comparing the completion to reference ratio (Figure 5): A high value of 2.9 and 3 indicates that Llama2 base models repeat up to 3 times of the expected completion.

## 4 Conclusion

We conducted thorough investigation of the influence of several components of a RAG pipeline on the overall generation quality. Based on our findings, we find retrieval optimization is an important part of the RAG pipeline design, even with high quality LLMs like GPT-4. Firstly, we recommend prioritizing retrieval recall, while tuning retrieval systems, as LLMs exhibit _pseudo-helpfulness_ when relevant gold document(s) are not retrieved. Additionally, improving precision of retrieval, either by using re-rankers or fine-tuned retrievers, will likely improve performance as they improve the gold document(s) rank in the retrieved list. We also find RAG systems to be sensitive to the presence of distractors in the context. We find a big delta between vendor LLMs (OpenAI) and smaller scale open-source alternatives in our experiments, with respect to sensitivity to prompting, overall quality, and instruction following on a domain specific use cases. Finally, for smaller-sized LLama-2 models we recommend simple instructions as they often fail to follow longer or more complicated instructions.

## Ethics Statement

All the work done and discussed in this paper meets and upholds the ACL Code of Ethics. User data wherever used was anonymized.

Figure 5: Accuracy of instruction following for structural formatting and language-style.





## References

* Adlakha et al. (2023) Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, and Siva Reddy. 2023. Evaluating correctness and faithfulness of instruction-following models for question answering. _arXiv preprint arXiv:2307.16877_.
* Bang et al. (2023) Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lorenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. _arXiv preprint arXiv:2302.04023_.
* Chen et al. (2023a) Banghao Chen, Zhaofeng Zhang, Nicolas Langrene, and Shengxin Zhu. 2023a. Unleashing the potential of prompt engineering in large language models: a comprehensive review.
* Chen et al. (2023b) Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2023b. Benchmarking large language models in retrieval-augmented generation.
* Gao et al. (2023) Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. _arXiv preprint arXiv:2312.10997_.
* Jiang et al. (2023) Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 7969-7992, Singapore. Association for Computational Linguistics.
* Kandpal et al. (2023) Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In _International Conference on Machine Learning_, pages 15696-15707. PMLR.
* Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. _Transactions of the Association for Computational Linguistics_, 7:453-466.
* Lazaridou et al. (2022) Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. 2022. Internet-augmented language models through few-shot prompting for open-domain question answering.
* Li et al. (2023) Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. HaluleEval: A large-scale hallucination evaluation benchmark for large language models. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 6449-6464, Singapore. Association for Computational Linguistics.
* OpenAI et al. (2015) OpenAI, ; Josh Achdiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Atlenschmidt, Sam Altman, Shyamal Ansdast, Red Avila, Igor Babuschkin, Suchir Balaji, Valeric Balcon, Paul Baltescu, Haiming Bao, Mo Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosic Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dolhan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Attty Eleti, Tyna Eleoundou, David Farhi, Liam Fedus, Niko Felix, Simon Posada Fishman, Justron Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gonitijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jan, Jaounce Jang, Angela Jiang, Roger Jiang, Hazohn Jin, Denmy Jin, Shiino Jonnotto, Billie John, Hewovoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kohdrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Luo, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinnney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Winnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Melby, Ashvin Nair, Reichiro Nakano, Rajeev Nayak, Avenol Neelakaran, Richard Ngo, Heyonwoob Noh, Long Ouyang, Cullen O'Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambtistata Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Pereleman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitcherp Song, Tolly Powell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Bae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ry

der, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Seslam, Kylan Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shocker, Pranav Shyam, Syzmon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natale Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine Thompson, Phil Tillet, Amin Tootocchanian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Frelipe Ceron Uribe, Andrea Vallone, Arun Vijayergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zelters, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. 2023. Gpt-4 technical report.
* Rajpurkar et al. (2018) Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't know: Unanswerable questions for squad. _arXiv preprint arXiv:1806.03822_.
* Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. _arXiv preprint arXiv:1606.05250_.
* an extensive definition, quantification, and prescriptive remediations. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 2541-2573, Singapore. Association for Computational Linguistics.
* Ren et al. (2023) Ruiyang Ren, Yuhao Wang, Yingqi Qu, Wayne Xin Zhao, Jing Liu, Hao Tian, Hua Wu, Ji-Rong Wen, and Haifeng Wang. 2023. Investigating the factual knowledge boundary of large language models with retrieval augmentation. _arXiv preprint arXiv:2307.11019_.
* Shuster et al. (2021) Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. In _Findings of the Association for Computational Linguistics: EMNLP 2021_, pages 3784-3803, Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Siriwardhana et al. (2023) Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Thairndu Kaluarchchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (rag) models for open domain question answering. _Transactions of the Association for Computational Linguistics_, 11:1-17.
* Sun et al. (2023) Kai Sun, Yifan Ethan Xu, Hanwen Zha, Yue Liu, and Xin Lunna Dong. 2023. Head-to-tail: How knowledgeable are large language models (llm)? aka will lms replace knowledge graphs? _arXiv preprint arXiv:2308.10168_.
* Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almdahiari, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Bleecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyn Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Innan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kolumann, Artem Koreneve, Punit Singh Koura, Marie-Anne Lachaviuc, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Tuning Mao, Xavier Martinet, Todor Mihalov, Pushkar Mishra, Igor Molbybo, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaogping Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models.
* Zhu et al. (2023) Kaiijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Neil Zhenqiang Gong, Yue Zhang, et al. 2023. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. _arXiv preprint arXiv:2306.04528_.

 

Appendix

### Limitations

Our research opens up various avenues for future investigation. Firstly, the RAG pipeline's inherent nature makes it challenging to comprehensively identify all relevant documents for a query. We aim to develop algorithms that can enhance automatic identification with greater precision. Secondly, our study presupposes that answers are derived from a single passage, which is not always true in practical scenarios. We plan to broaden this assumption to more realistic applications. Thirdly, our research is currently limited to six models from two families (OpenAI GPT and Llama); it would be intriguing to evaluate additional models like Claude, Mistral, etc., across a wider range of datasets. Finally, our study mimics the retrieval quality via manipulation of retrieved documents. It would be advantageous to examine various real-world retrieval systems to determine how improvements at the retrieval stage can translate into enhanced answer quality.

### Model Parameters

\begin{table}
\begin{tabular}{l c c c c} \hline
* & 3.9K & 200 & greedy \\ Llama-2-13b-chat &
- & 3.9K & 200 & greedy \\ Llama-2-7b &
- & 3.9K & 200 & greedy \\ Llama-2-7b-chat &
- & 3.9K & 200 & greedy \\ \hline \end{tabular}
\end{table}
Table 1: LLM Model Parameters. We used the set of parameters to balance the context length for retrieved document list and also completion length required to generate responses. Based on our estimates of reference answers, two OpenAI models can keep more than 99% for both inputs and outputs from being chunked and Llama-2 models keep 99% (both inputs and outputs) for the public webpage dataset from being chunked and 70% (inputs) and 90% (outputs) for the internal service dataset.
 



### Evaluation Dataset Generation

### Head-to-head comparison on Banking Webpage

Figure 6: Evaluation dataset creation pipeline: We paired each question to related documents via a two-staged pipeline. First subject matter experts (SMEs) annotated coarse grained document labels for certain queries. Given our requirement for more fine-grained query, chunk alignment pairs, a two stage model based approach was applied to get an initial mapping and then review it with human reviewers: (1) We took chunks from the SME provided higher level articles and use GPT4 as a pairwise evaluator of relevance. Each chunk in the SME annotated articles was paired with the query and GPT4 was instructed to return a binary label for relevance of the chunk against the query. (2) Since our SME annotations are not comprehensive, there could be articles in the corpus which they did not tag but could contain the answer. So we used a SOTA dense retriever to get top-10 chunks for a query from the entire document corpus. The union of the above selected chunks was provided to a team of human reviewers who (i) assigned a binary relevance label to every chunk and (ii) selected an answer span within the chunk which answered the question.Our datasets are used purely for evaluation purposes, not for any fine-tuning.

Figure 7: Head-to-head pairwise winrate in Answer generation quality (Token F1) for Banking webpage dataset.





### Prompt Variance heatmap

Figure 8: Head-to-head pairwise winrate in Answer generation quality (Token F1). Left figure shows the results for Banking Policy Guides (Left) and the Banking Public webpages (Right).





### F1 Performance for all datasets, baselines and conditions

Figure 9: F1 score for all models on the Banking Policy Guides dataset



## Appendix A Proofs

### Proof of Theorem 1.2

Proof of Theorem 1.3.: Let \(f_{i}=0\) and \(g_{i}=0\) be the set of all possible values in \(\mathbb{R}^{n}\), where \(\mathbb{R}^{n}\) is a subset of \(\mathbb{R}^{n}\). Then we have that \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for some \(n>0\). Since \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\), there exists an open neighborhood \(U\subset U\) such that \(\mathbb{R}^{n}\cap U\neq\emptyset\). Hence, by Lemma 4.5, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n<0\). Therefore, by Lemma 4.6, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n>0\). Thus, by Lemma 4.7, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n>0\). By Lemma 4.8, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n>0\). But then, by Lemma 4.9, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n>0\). So, by Lemma 4.8, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n>0\). Now, by Lemma 4.9, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n>0\). Hence, by Lemma 4.8, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n>0\). Hence, by Lemma 4.9, \(\mathbb{R}^{n}\subset\mathbb{R}^{n}\) for any \(n>0\). Hence, by Lemma 4.





### Prompt List

Figure 11: Prompts List 1

* [15] M. C. Bader, J. S. Hastings, and D. R. K. McKay, "A new approach to the "SOURCES" section b: a survey of _"Sources"_, vol. 30, no. 2, pp. 47-68, 1996.
* [16] A. G. Barnett, "The '\({}^{\prime}\)-\ 

* [25] M. C. Bader, A. D. Krizhevsky, and R. G. Hinton. The output of a deep neural network for image recognition. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 1-40, 2016.
* [26] J. L. Friedman, S. F. Friedman, and P. W. Friedman. "Generating adversarial networks with applications to machine learning." _Journal of Machine Learning Research_, 38:799-802, 2000.
* [27] T. H. Friedman, S. F. Friedman, and P. W. Friedman. "Learning from images using generative adversarial nets." in _Advances in Neural Information Processing Systems_, volume 1, pages 1-10, 2006.
* [28] Y. Bengio, I. Guyon, and E. Rabinovich. Generative adversarial nets. In _NIPS'03_, volume 1, pages 1398-1402, 2003.

[MISSING_PAGE_POST]