Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023)

Abstract: Conversational Question Answering (ConvQA) models aim at answering a question with its relevant paragraph and previous question-answer pairs that occurred during conversation multiple times. To apply such models to a real-world scenario, some existing work uses predicted answers, instead of unavailable ground-truth answers, as the conversation history for inference. However, since these models usually predict wrong answers, using all the predictions without filtering significantly hampers the model performance. To address this problem, we propose to filter out inaccurate answers in the conversation history based on their estimated confidences and uncertainties from the ConvQA model, without making any architectural changes. Moreover, to make the confidence and uncertainty values more reliable, we propose to further calibrate them, thereby smoothing the model predictions. We validate our models, Answer Selection-based realistic Conversation Question Answering, on two standard ConvQA datasets, and the results show that our models significantly outperform relevant baselines. Code is available at: https://github.com/starsuzi/AS-ConvQA .

## introduction conversational question answering (convqa) is the task of answering a series of questions during conversation, taking into account a given relevant paragraph @xcite . contrary to traditional extractive question answering tasks @xcite that answer each question with the given paragraph just once, convqa aims at answering the current question using its previous questionanswer pairs taking into account the given paragraph multiple times. for example, as illustrated in figure 1 , the goal of convqa is to correctly answer the question q 3 based on the previous conversation * corresponding author : posing as a "juilliard-trained dermatologist" figure 1 : illustration of realistic convqa evaluation with three models: 1) using all predicted answers (all pred.); 2) not using predicted answers (no pred.); 3) only using probably correct answers while filtering out others (as-convqa, ours). the scores in the bar chart underneath represent the f1 scores measured by all test samples (see convqa has recently gained much attention as it follows the human's information seeking process through multi-turn interactions with others. however, it is also known to be quite challenging since it requires capturing all the information over the current question, previous conversation, and the given paragraph. to tackle this problem, a considerable amount of work focuses mainly on developing a model architecture for convqa @xcite @xcite @xcite . despite their successes, however, there remains a critical limitation in that they use the ground-truth answers (i.e., a 2 and a 1 ) in the conversation history during both training and evaluation steps. such an evaluation procedure is not applicable to the real-world scenario, since the ground-truth answers are not accessible when the user's query is posed. therefore, the supporting dialogue history for the current question should consist of the model's predictions ā1 and ā2 in the real-world application, instead of the nonexistent gold answers a 1 and a 2 . there is some recent work @xcite ) that considers such a realistic setting on evaluation. in particular, they propose to use the model's predicted answers (i.e., ā1 and ā2 ), instead of the ground-truth answers (i.e., a 1 and a 2 ), for its evaluation. however, in such a setting, the model faces inconsistency between training and evaluation since the model is evaluated with the predictions while trained with the ground-truth answers. to handle such a discrepancy, @xcite and @xcite suggest strategies that randomly decide whether to use predicted or gold answers for the input question during training. however, as figure 1 shows, using all predictions as the answer history is not effective: the performance difference is not so significant when compared to not using them at all. we see that this originates from a model's failure to answering previous questions. specifically, if a model incorrectly predicts an answer ā1 for the previous question q 1 , using the incorrectly predicted answer ā1 for the question q 2 not only affects the model's current prediction ā2 negatively, but also engenders further errors in the future prediction for q 3 . therefore, in this work, we propose a novel selection scheme for predicted answers from the conversation history, which filters out predictions that are likely to be incorrect, unlike the existing work that uses all the predicted answers including incorrect ones. the remaining step is then to identify possibly incorrect predictions. to this end, we propose to use the confidence and uncertainty of the model's prediction, which are measured by its likelihood and entropy, respectively. in particular, if the model predicts the previous answer with lower confidence (i.e., lower likelihood) or higher uncertainty (i.e., higher entropy) than a certain threshold, we regard the model's previous answer as probably incorrect, and remove it from the conversation history in answering the current question during evaluation. on the other hand, during training, we soften the sampling process so that, instead of using the hard threshold above, we sample a predicted answer based on its confidence or uncertainty (e.g., the lower the uncertainty, the higher the chance to include the predicted answer in the conversation history), in order to diversify the model's input. however, when dealing with confidence and uncertainty, we should be careful about a miscalibrated situation @xcite , which happens when uncertainty and confidence do not correspond to the error and accuracy of ground-truth correctness, respectively. in other words, if the model is not calibrated enough and the distribution for confidence and uncertainty is highly skewed over particular ranges, the highly uncertain or low confident yet valid predictions could be removed. therefore, to prevent such a performance degrading situation, we further calibrate models using a temperature scaling scheme @xcite before estimating the uncertainty or confidence. we refer to our method as answer selection-based realistic conversational question answering (as-convqa). we validate our method on two standard con-vqa datasets, quac @xcite and coqa @xcite , against diverse baselines on a realistic evaluation protocol. the experimental results show that our method significantly outperforms these baselines, and a detailed analysis supports the importance of uncertainty-and confidence-based answer selection schemes. our contributions in this work are threefold: however, the aforementioned work has a fundamental limitation on the evaluation protocol: they evaluate models based on ground-truth answers working as a conversation history, which are not available in a real-world setting. @xcite point out this problem of ground-truth history evaluation but use the ground-truth answers during evaluation as well, since they target at disambiguating pronouns in the question by comparing the predicted and ground-truth answers. alternatively, @xcite and @xcite use the model's predictions instead of the ground-truth answers during evaluation, and further train the model with predicted answers. however, they do not take into account the quality of predicted answers, where low-quality ones are not useful (see figure 1 ). thus, we propose to selectively use the predicted answers that are probably correct, based on their calibrated confidences and uncertainties. confidence and uncertainty as it is nearly impossible for models to always make accurate predictions, unreliable predictions become serious issues when deploying machine learning models to realworld settings. motivated to prevent such a risk, mechanisms of estimating the reliability of model's predictive probabilities based on confidence and uncertainty are recently proposed @xcite . we note that confidence is usually measured by the softmax outputs of models @xcite , and that uncertainty can be quantified by bayesian models, which can be approximated via monte carlo (mc) dropout @xcite . with much work on confidence and uncertainty estimations in computer vision tasks @xcite , related topics have been recently adopted for nlp tasks as well @xcite @xcite . while confidence and uncertainty estimation should also be considered in convqa, we believe that this venue is under-explored so far. in particular, since questions are asked sequentially, it is likely that untrustworthy predictions in the conversation history would negatively affect the performance. to tackle this, we propose to exclude low-confident or uncertain predictions when training and evaluating the convqa model. calibration confidence and uncertainty help interpret the validity of the model's prediction. however, it is not safe to rely on them when the model is not calibrated, where the correct likelihood does not match the predicted probability @xcite , or the model error does not match the predicted uncertainty @xcite . since deep neural networks are prone to miscalibration as the number of parameters has much increased, large pre-trained language models are also not free from this problem @xcite @xcite . one of the most prevalent approaches to calibrating the model is to rescale a logit vector before the softmax function for regularizing the probability, which is known as temperature scaling @xcite . while there exist lots of calibration schemes, including label smoothing @xcite and confidence penalty @xcite , in this work, we use temperature scaling as a calibrator, since it is simple yet effective while not changing the output class of the model prediction (i.e., only scaling logits). ## method we first introduce convqa. then, we describe our answer validating methods based on confidence and uncertainty values with their calibration schemes. ## experimental setups we explain datasets, metric, and models. please see appendix a for further implementation details. ## results and discussion in this section, we show overall performances of our proposed method along with detailed analyses. ## conclusion in this work, in order to tackle the challenge of inaccurately predicted answers in the conversation history, we proposed a novel answer selection scheme based on their confidence and uncertainty values. we further calibrated the output values of the model to match the model's predicted confidence and uncertainty to its correct likelihood and error, which makes our answer selection scheme more reliable. the experimental results and analyses demonstrate that as-convqa significantly improves the con-vqa model performance in a realistic evaluation setting without making any architectural changes.