Training Adaptive Computation for Open-Domain Question Answering with Computational Constraints (2021)

Abstract: Adaptive Computation (AC) has been shown to be effective in improving the efficiency of Open-Domain Question Answering (ODQA) systems. However, the current AC approaches require tuning of all model parameters, and training state-of-the-art ODQA models requires significant computational resources that may not be available for most researchers. We propose Adaptive Passage Encoder, an AC method that can be applied to an existing ODQA model and can be trained efficiently on a single GPU. It keeps the parameters of the base ODQA model fixed, but it overrides the default layer-by-layer computation of the encoder with an AC policy that is trained to optimise the computational efficiency of the model. Our experimental results show that our method improves upon a state-of-the-art model on two datasets, and is also more accurate than previous AC methods due to the stronger base ODQA model. All source code and datasets are available at https://github.com/uclnlp/APE .

## introduction open-domain question answering (odqa) requires finding relevant information for a given question and aggregating the information to produce an answer. the retriever-reader architecture, popularised by @xcite , has shown great success in this task. the retriever acquires a set of documents from external sources (e.g., wikipedia) and the reader extracts the answer spans from these documents @xcite @xcite @xcite . recently, @xcite ; @xcite ; @xcite showed that generative reader models that exploit an encoder-decoder architecture can significantly outperform previous extractive models, thanks to their better capability in aggregating and combining evidence from multiple passages. however, these generative models are much more computationally expensive than extractive models, and often need to be trained with a large number of passages, making it hard to train these models for most researchers @xcite . @xcite show that adaptive computation (ac) can significantly improve the efficiency of extractive odqa models at inference time. however, it requires fine-tuning all model parameters with a multitask learning objective, making it computationally challenging to apply this method to current state-of-the-art models. in this work, we explore an efficient approach to apply adaptive computation to large generative odqa models. we introduce the adaptive passage encoder (ape), a module that can be added to the encoder of an existing odqa model, which has the following features: 1) it efficiently reuses the encoder's hidden representations for calculating the ac priorities; 2) it does not require tuning of the base model and hence allows efficient training under limited resource; 3) it does not require confidence calibration. our experimental results on nat-uralquestions and triviaqa show that our method improves the performance of the state-of-the-art model fid @xcite , while also producing more accurate results (12.4% em) than the ac method proposed by @xcite . however, thanks to recent advances in sequenceto-sequence pretrained language models @xcite , generative odqa models @xcite @xcite ) achieve significant improvement upon extractive models, demonstrating stronger capability in combining evidence from multiple passages. we focus on generative models in this work. for ranking passages @xcite . recently, @xcite ; @xcite ; izacard and grave (2020a) achieved substantial increase in retrieval performance using dense representations. our work is based on the retrieval results from a dense retriever @xcite ), but we show that the proposed method can still improve the quality of the support passages despite the strong retrieval performance. nogueira and cho (2019); qiao et al. (2019) ; @xcite show that adding a separate cross-encoder re-ranker can improve the performance, but that comes with a significant increase of the computation at train or inference time. despite that our proposed adaptive passage encoder can be viewed as an encoder with an integrated re-ranker, the focus of our work is to improve the computational efficiency, namely, enhancing the performance without a substantial increase in computation. adaptive computation adaptive computation allows the model to condition the computation cost on the input. for example, @xcite al. (2020b); liu et al. (2020); xin et al. (2020) propose models that can dynamically decide to early exit at intermediate layers when the confidence at the layer exceeds a threshold. they show that adaptively early exiting can significantly reduce the computational cost for various sequence classification tasks. closest to our work, @xcite introduced adaptive computation for extractive odqa models. we extend adaptive computation to generative odqa models, and our approach can be incorporated in existing generative odqa models without finetuning the base model. ## method in this section, we will introduce the base model and how our proposed adaptive passage encoder works with it. ## conclusions in this work, we explore an adaptive computation method that can be efficiently applied to an existing generative odqa model. we find that, by replacing the encoder of generative odqa models with our proposed adaptive passage encoder, we can train an effective adaptive computation policy without tuning the base model. this allows applying adaptive computation to large state-of-the-art generative models, which was previously challenging computation-wise. our experimental results show that our method produces more accurate results than a state-of-the-art generative model on both naturalquestions and triviaqa, and it outperforms the previous ac method by a large margin. the analysis also shows that our approach achieves better passage quality that leads to improvements in odqa performance.