  To build robust question answering systems, we need the ability to verify
whether answers to questions are truly correct, not just "good enough" in the
context of imperfect QA datasets. We explore the use of natural language
inference (NLI) as a way to achieve this goal, as NLI inherently requires the
premise (document context) to contain all necessary information to support the
hypothesis (proposed answer to the question). We leverage large pre-trained
models and recent prior datasets to construct powerful question converter and
decontextualization modules, which can reformulate QA instances as
premise-hypothesis pairs with very high reliability. Then, by combining
standard NLI datasets with NLI examples automatically derived from QA training
data, we can train NLI models to judge the correctness of QA models' proposed
answers. We show that our NLI approach can generally improve the confidence
estimation of a QA model across different domains, evaluated in a selective QA
setting. Careful manual analysis over the predictions of our NLI model shows
that it can further identify cases where the QA model produces the right answer
for the wrong reason, or where the answer cannot be verified as addressing all
aspects of the question.
