Calibrated Interpretation: Confidence Estimation in Semantic Parsing (2023)

Abstract: Sequence generation models are increasingly being used to translate natural language into programs, i.e., to perform executable semantic parsing. The fact that semantic parsing aims to predict programs that can lead to executed actions in the real world motivates developing safe systems. This in turn makes measuring calibration—a central component to safety—particularly important. We investigate the calibration of popular generation models across four popular semantic parsing datasets, finding that it varies across models and datasets. We then analyze factors associated with calibration error and release new confidence-based challenge splits of two parsing datasets. To facilitate the inclusion of calibration in semantic parsing evaluations, we release a library for computing calibration metrics.1

## introduction when probabilistic models are used for decisionmaking or interaction, we not only want the model to be accurate but also to be calibrated, meaning that the probability the model assigns to a decision should roughly correspond to its likelihood of being correct. for example, if a model translates a user instruction into a program for a robot, the confidence that the program accurately captures the person's intent may inform whether the robot executes the program. because of its importance to safety and usability, a large body of work has addressed calibration, with most research considering single-timestep classification models (i.e., one output decision per input). however, probabilistic models are also commonly applied to multi-timestep tasks. in these tasks, the model generates a sequence of decisions, with each subsequent decision dependent on the input and the previous decisions. for example, text generation models typically predict sequences of words one at a time, with each new word conditioned on the previous outputs. although calibration has been measured in some text generation tasks like machine translation @xcite , determining the accuracy of generated text sequences can be challenging, making calibration-the relationship between accuracy and confidence-hard to estimate. as sequence generation models, especially large language models, play a role in a growing variety of tasks, measuring their calibration has become extremely important. the task of executable semantic parsing, where a model predicts an executable program from a natural language instruction, is often modeled as a sequence generation task. such parsing models are used in systems that interact with the real world, such as human-robot interfaces @xcite and digital assistants @xcite @xcite . since actions in these domains-especially physical domains-can have irreversible effects, the importance of ensuring model safety cannot be understated. thus, semantic parsing provides a clear motivation for having a well-calibrated sequence generation model: at low confidence, we may prefer for the system to defer action or request clarification, while when confidence is high, these actions may unnecessarily annoy a user and make the system unusable. this reasoning presupposes that the model's confidence is wellcorrelated with its probability of success. simultaneously, the constrained and executable nature of semantic parses makes accuracy easier to measure. in many text-based sequence generation tasks like machine translation, summarization, long-form question-answering (qa), and open-ended dialogue, evaluating the quality and correctness of a generation poses a variety of challenges, often due to the fact that there are many ways of stating roughly the same proposition in language. because calibration measures the relationship between accuracy and confidence, our ability to measure calibration will only be as good as our ability to measure accuracy. in executable semantic parsing, the model generates a program (rather than text) with a more restricted vocabulary and known syntactic rules. this generally limits the number of reasonable semantically equivalent outputs and makes quantifying accuracy easier. furthermore, the executable nature of the programs allows us to measure accuracy via denotation (i.e., the result of program execution) rather than form. these factors make it an ideal domain for benchmarking the calibration of sequence generation models. in section 4, we conduct what is to our knowledge the first large-scale investigation of calibration in sequence generation models as applied to semantic parsing tasks. we examine a variety of commonly used models and measure their calibration across four popular semantic parsing datasets drawn from two different domains: task-oriented dialogue (tod) and text-to-sql. we first document the model's calibration profiles, asking how well-calibrated modern semantic parsing systems are; this includes a large pre-trained model queried in a few-shot setting, as well as more traditional fine-tuned models. using qualitative and quantitative metrics, we find that most tod models are already fairly wellcalibrated. however, the same models are poorly calibrated on text-to-sql datasets. in section 5, we analyze various factors implicated in calibration, attempting to shed light on the differences documented in section 4. we first find that dataset size does not account for the difference in calibration between tod and sql models. using measures of input and program difficulty, we then explore the relationship between difficulty and calibration, finding that for sql programs, the difficulty of the input and output are associated with poor calibration. finally, we find that on tod datasets, each model's low-confidence examples are challenging for other models; this leads us to propose two confidence-based challenge datasets. because of calibration's importance to semantic parsing specifically-and sequence generation tasks generally-we introduce an open-source library for computing calibration metrics and plotting confidence, compatible with the transformers library @xcite . ## methods our first goal is to determine the typical calibration characteristics of models applied to semantic parsing tasks. for this, we choose a range of standard semantic parsing tasks, datasets and models. ## benchmarking calibration calibration can be measured at the token-level or the sequence-level. sequence-level calibration is most relevant to safety; in a system using an executable parsing model, the predicted program will be executed as a whole (i.e., as a sequence). however, absent a (potentially expensive) external model, sequence-level confidence scores will be composed of token-level scores, just as sequences are composed of tokens. moreover, token-level scores can reveal phenomena obscured by sequence-level scores. rather than only measuring the model's accuracy and confidence on a whole sequence, it may be more informative to examine on which types of tokens the model makes mistakes or has high calibration error. for example, token-level confidence scores can allow us to examine which specific functions the model is struggling on. to measure token-level accuracy against a reference program at time t, all predicted tokens at timesteps 1, . . . , t -1 must match the reference program's prefix; thus, we use teacher forcing, feeding the model the gold prefix up to the current timestep. that is, when predicting the confidence of token ŷt , we use tokens y 1 , . . . , y t-1 taken from the gold program p , not the predicted program p . note that this differs from the setting used for computing em accuracy in figure 1 , figure 2 , and figure 4 , where we compare the gold program @xmath0, . . . , y t to the predicted program @xmath1, . . . , ŷt . for sequence-level confidence, we use p . ## discussion and analysis in section 4, we found that for a fixed dataset, different models have varying calibration characteristics, even when the models have similar architectures and pre-training data. similarly, we saw that the same models often differ between datasets, and especially differ between domains (tod vs. text-to-sql). finally, we found that some models show conspicuous spikes in calibration error in some regions of the confidence space, and that these spikes hold across models. in light of these differences and quirks, we conduct additional analyses on factors associated with model calibration. ## conclusion broadly, our results show that calibration is a complex phenomenon with a multitude of influences. on a single dataset, different models vary in their calibration error; moreover, the same model often varies drastically between different datasets, indicating that calibration is a function of both the model and the dataset. taken together, these results point to the complexity of measuring calibration, and suggest that care should be taken when making claims about classes of models based on evidence from a limited number of models or datasets. given the utility of calibrated models and the importance of calibration to safety, we advocate for considering calibration in the standard evaluation suite for semantic parsing models, and release our metric and visualization suite as a standalone package to facilitate these comparisons.