Towards Effective Long-Form QA with Evidence Augmentation (2023)

Abstract: In this study, we focus on the challenge of improving Long-form Question Answering (LFQA) by extracting and effectively utilizing knowledge from a large set of retrieved passages. We first demonstrate the importance of accurate evidence retrieval for LFQA, showing that optimal extracted knowledge from passages significantly benefits the generation. We also show that the choice of generative models impacts the system’s ability to leverage the evidence and produce answers that are grounded in the retrieved passages. We propose a Mixture of Experts (MoE) model as an alternative to the Fusion in Decoder (FiD) used in state-of-the-art LFQA systems and we compare these two models in our experiments.

## introduction long-form question answering (lfqa) is a generative qa task that produces informative and comprehensive answers, often requiring models to leverage external knowledge sources. retrieving supportive passages from large text corpora, e.g., wikipedia, is a prevalent approach to provide external knowledge for the generation model. however, the retrieved passages often suffer from noise and excessive length that poses challenges for the model. lfqa has largely been explored using eli5 @xcite , a community qa dataset, where the answers are provided by domain experts. finding supporting evidence for these questions is often challenging because the relevant information is usually fragmented across multiple documents. while several models @xcite @xcite have been proposed for lfqa, @xcite reveals that state-of-the-art models generate answers that are not grounded in the retrieved documents. our study examines the information contained in the retrieved documents question what are dentists actually doing when they scrape at your teeth with those metal picks? ## method given a question q, we retrieve a collection of passages @xmath0. . . p m from external corpora. we consider the optimal situation where only the most relevant information for the correct answers is extracted from the large set of retrieved passages p . we refer to this as the oracle. we use the oracle evidence in our models during training and inference. ## conclusion our study investigates the impact of including optimal evidence from external knowledge in lfqa. by employing three forms of evidence in oracle scenarios, we demonstrate that optimal evidence extracted from retrieved passages significantly improves the performance of lfqa systems. in addition, we propose an moe model for incorporating passages and extracted evidence. experimental results showed that our moe design uses the evidence more efficiently on the asqa dataset. we believe that these findings are encouraging for further exploration in the intelligent use of external evidence for lfqa to improve generation in non-oracle settings. these findings can be applied to improve all state-of-the-art approaches including llms.