Calibration of Machine Reading Systems at Scale (2022)

Abstract: In typical machine learning systems, an estimate of the probability of the prediction is used to assess the system’s confidence in the prediction. This confidence measure is usually uncalibrated; i.e. the system’s confidence in the prediction does not match the true probability of the predicted output. In this paper, we present an investigation into calibrating open setting machine reading systemssuch as open-domain question answering and claim verification systems. We show that calibrating such complex systems which contain discrete retrieval and deep reading components is challenging and current calibration techniques fail to scale to these settings. We propose simple extensions to existing calibration approaches that allows us to adapt them to these settings. Our experimental results reveal that the approach works well, and can be useful to selectively predict answers when question answering systems are posed with unanswerable or out-of-the-training distribution questions.

## introduction with recent advances in machine reading, there has been a surge of interest in practical applications of the technology such as open-domain question answering @xcite and claim verification @xcite . due to various scale limitations in practical settings, these systems are seldom trained endto-end. such systems typically make use of a re-triever alongside a reader -the evidence is first retrieved from a large corpus and is then used by a machine reading model to provide an answer. as these systems are increasingly being deployed in the real world, it is important that they are not only accurate but also trustworthy. a way to make these systems trustworthy is to indicate when they are likely to be incorrect by providing a calibrated confidence measure in addition to the prediction. a naive solution for this is to use the system's output probability as the confidence. however, this confidence score is often uncalibrated @xcite ; i.e. it is not representative of the true correctness likelihood. 1 previous work @xcite @xcite has shown that large language models especially suffer from miscalibration. thus, several methods have been proposed to calibrate language models based on gradient-based calibration methods such as temperature scaling @xcite and featurebased forecasters @xcite . while gradient-based calibration is intuitive and easy to implement, feature-based forecasters require manual feature engineering. in this work, we contribute a simple method to calibrate practical retriever -reader machine reading pipelines. these systems typically include a hard retrieval step which makes gradientbased calibration infeasible. thus, we make use of the gumbel machinery @xcite ; specifically the gumbel top-k procedure of @xcite ; @xcite to obtain a differentiable sampling routine for the retrieval step. this sampler can then be combined with any gradient-based calibration technique such as platt's scaling. we conduct experiments on three different models -a generative and extractive open-domain question answering model and a claim verification model. we find that calibrating the retriever and the reader jointly is better than calibrating only the reader or the retriever . we also show that our approach can produce calibrated the systems follow the same architecture and are composed of a retriever and a reader. given the query, the retriever retrieves a set of k documents from the corpus along with scores for each of them. the reader then takes these as input and produces the output: a veracity label for claim verification and an answer span for the qa model. this can be seen as a probabilistic model with latent retrieval (d k shown in red). the goal of this paper is to calibrate the final output probabilities p(a|q). ## experimental details open domain question answering extractive we test the described calibration techniques on the open domain qa, using the pretrained models from @xcite . we perform our experiments on the natural questions (nq) dataset @xcite . we randomly split our validation set into two equal parts which we will call calib and valid. we use these splits for training and tuning our calibration models respectively. we use the test set of nq as our test set (test). during inference, we use the retriever to retrieve top 10 documents which are passed to the reader to extract the answer. ## results we now present the results of the various calibration techniques in ## analysis next, we attempt to verify the following claims: c1: the existing approach for calibrating only the reader doesn't result in a good calibration of the overall system. jointly calibrating the reader and the retriever model is better. c2: calibrated odqa systems do better selective prediction when they are allowed to not provide answers to some questions. c3: calibrated odqa systems are better at handing domain shifts in questions at test time. c4: calibrated odqa systems are better at handling unanswerable questions at test time. ## discussion and conclusion in this paper, we analyzed how various calibration techniques can be adopted to open-domain machine reading systems which are now being used in user-facing scenarios. we showed that in such systems that include a retriever, calibrating the system's confidence is not trivial and we proposed a technique that allows calibration of the system jointly. finally, we also provide an analysis on how the calibration techniques can help the model abstain from answering a question especially in settings where the model's prediction can be incorrect due to malformed or out-of-domain questions. while we do not find evidence to prove that one calibration method (e.g. a gradient-based method) is better that the other (e.g. a forecaster approach), it would be important to investigate these questions with more nuanced human studies.