Enhancing Self-Consistency and Performance of Pre-Trained Language
Models through Natural Language Inference
Eric Mitchell, Joseph J. Noh, Siyan Li, William S. Armstrong,
Ananth Agarwal, Patrick Liu, Chelsea Finn, Christopher D. Manning
Stanford University
eric.mitchell@cs.stanford.edu

Abstract
While large pre-trained language models are
powerful, their predictions often lack logicalconsistency across test inputs. For example,
a state-of-the-art Macaw question-answering(QA) model answers Yes to Is a sparrow abird? and Does a bird have feet? but answers
No to Does a sparrow have feet?. To address
this failure mode, we propose a framework,
Consistency Correction through Relation Detection, or ConCoRD, for boosting the consistency and accuracy of pre-trained NLP models
using pre-trained natural language inference(NLI) models without fine-tuning or re-training.
Given a batch of test inputs, ConCoRD samples several candidate outputs for each input
and instantiates a factor graph that accounts
for both the model’s belief about the likelihood of each answer choice in isolation and
the NLI model’s beliefs about pair-wise answer
choice compatibility. We show that a weighted
MaxSAT solver can efficiently compute highquality answer choices under this factor graph,
improving over the raw model’s predictions.
Our experiments demonstrate that ConCoRD
consistently boosts accuracy and consistency ofoff-the-shelf closed-book QA and VQA models using off-the-shelf NLI models, notably increasing accuracy of LXMERT on ConVQA
by 5% absolute. See the project website1 forcode and data.

1

Introduction

Reliable and trustworthy AI systems should demonstrate internal self-consistency, in the sense that
their predictions across inputs should imply logically compatible beliefs about the world. However,even powerful large language models are known
to lack self-consistency (Ray et al., 2019; Elazar
et al., 2021; Kassner et al., 2021). For example, aquestion-answering (QA) model that answers thequestion Is a sparrow a bird? and Does a bird have1

https://ericmitchell.ai/emnlp-2022-concord/

Figure 1: ConCoRD first generates candidate outputs fromthe base pre-trained model, then estimates soft pairwise constraints between output choices, and finally finds the mostsatisfactory choices of answers accounting for both the basemodel and NLI model’s beliefs.

feet? with Yes is implicitly expressing the belief
that A sparrow is a bird and A bird has feet. If thesame model answers the question Does a sparrow
have feet? with No, the model expresses the logically incompatible belief A sparrow does not havefeet. In such cases, ascertaining the model’s “true”
belief is difficult, making interpreting and validating its behavior correspondingly challenging.
Prior work has improved model self-consistency
by training with specialized loss functions (Elazar
et al., 2021) or data augmentation (Ray et al., 2019),
or alternatively re-ranking model predictions basedon their mutual self-consistency using pre-written
logical constraints, such as “all mammals have fur”
(Kassner et al., 2021). However, the first classof methods requires expensive fine-tuning whichmight be impractical for many practitioners for verylarge pre-trained models, and re-ranking methods
require an explicit collection of the logical relations of interest, making scaling a challenge. Still,re-ranking-based approaches have the benefit of
not requiring fine-tuning, and we hypothesize that
their scalability limitations may be addressed byestimating logical relationships between model predictions on the fly. Specifically, we hypothesizethat existing pre-trained natural language inference(NLI) models can estimate logical relationships between an arbitrary pair of model predictions wellenough to provide an effective, scalable substitutefor explicit collection of such constraints. Leveraging these estimated constraints, we can construct a
factor graph representing a probability distribution
over model outputs that incorporates both the original model’s confidence scores and the NLI model’s
beliefs about logical relationships.
Our primary contribution is Consistency Correction through Relation Detection, or ConCoRD, aframework to improve the consistency and performance of a pre-trained base language model without fine-tuning by using more confident and better
attested model predictions to override less confident model beliefs. See Figure 1 for an overview.
To enable propagation of model beliefs, we estimate pair-wise logical relationships between model
predictions using a pre-trained NLI model. Using these pair-wise relationships, we define anundirected graphical model representing a distribution over responses accounting for both the basemodel’s beliefs and the NLI model’s estimates ofanswer compatibility. We efficiently find the approximate mode of this distribution among the basemodel’s top answer choices for each input as thesolution of a MaxSAT problem, which consistently
produces more accurate and self-consistent predictions than using the raw model predictions. In
Section 4.1 we find that ConCoRD produces an
8.1% absolute improvement in F1 of a pre-trained
Macaw model (Tafjord and Clark, 2021) on theBeliefBank QA dataset (Kassner et al., 2021). In
Section 4.2 we find a 5.0% absolute improvementin accuracy of a pre-trained LXMERT model (Tanand Bansal, 2019) on the ConVQA dataset (Rayet al., 2019), and in Section 4.3 we find that ConCoRD enables test-time model editing (Sinitsin
et al., 2020; Mitchell et al., 2022), updating model
predictions at test time when presented with new
information.

2

Related Work

Prior work for maintaining consistency in thequestion-answering space often involves additional

training to improve performance. Chen et al. (2021)
transform the Natural Questions (Kwiatkowski
et al., 2019) dataset question-answer pairs into
premise-hypothesis pairs, then uses an NLI modeltrained on this dataset as a decider for unanswerable questions. Alberti et al. (2019) generate questions from unlabeled texts, then filter them to ensure roundtrip consistency; pre-training on this
synthetic set improves performance on SQuAD
2.0 (Rajpurkar et al., 2018) and Natural Questions. Asai and Hajishirzi (2020) augment QApairs with their logically symmetric and transitive
counterparts through linguistic approaches to enhance cross-dataset QA performance. ConCoRD
differs significantly from these question-answeringspecific approaches because no fine-tuning of thebase model is needed and the methodology is not
specific to question-answering.
Similarly to ConCoRD, Kassner et al. (2021)
re-rank model predictions by solving an optimization problem defined by a combination of the basemodel confidence scores and pair-wise constraints
representing the logical compatibility of different
model predictions stored in a persistent memory,which they call BeliefBank. The key distinguishing property of ConCoRD is the fact that pair-wise
constraints between model predictions are dynamically estimated by a pre-trained NLI model, rather
than drawn from a fixed, pre-collected set of constraints. Dynamically estimating the constraints
has a variety of benefits, eliminating the need for
manually collecting the logical constraints of interest, automating the process of determining whether
a particular constraint applies to a particular pair ofpredictions, and likely inheriting improvements in
Natural language inference (NLI, MacCartney andManning (2008)) models over time.
NLI has long been used to maintain logical consistency in generated dialogue utterances (Welleck
et al., 2019; Dziri et al., 2019; Song et al., 2020), radiology report domain entities (Miura et al., 2021),
and summarization (Laban et al., 2022; Honovich
et al., 2022). Perhaps most similarly, Jung et al.
(2022) use NLI to estimate constraints between factual statements produced by GPT-3. These prior approaches support our intuition for using NLI models to improve logical consistency among batchesof answers. While the authors explore applications of this framework to multi-step reasoning
for True/False questions or statements, our work
focuses on applying this methodology to more gen-

Figure 2: An example factor graph for a simplified batch with two questions, q1 = What is the capital of Afghanistan? and q2 =What is the capital of Georgia?. Although Tbilisi is the most likely answer for both questions, the assignment of variables that isbest under the estimated contradiction constraint flips the answer to the first question to Kabul. The top-2 answer choices foreach question are sampled from the base model, and a soft contradiction constraint is detected between variables z1 (representing
the truth of the answer Tbilisi for q1 ) and z3 (representing the truth of the answer Tbilisi for q2 ).

eral settings, such as VQA, open-ended QA, andmodel editing.

3

Consistency Correction throughRelation Detection

ConCoRD contains three key components, the basemodel, a relation model (typically a pre-trained
NLI model), and an inference procedure that combines the predictions of the two models into a moreaccurate and self-consistent set of beliefs. Importantly, both the base model and relation model are
pre-trained, off-the-shelf models; ConCoRD does
not update any weights or require training data foreither model, using only a small validation set for
hyperparameter tuning. We next explain the function of each of these components when executing
ConCoRD.
3.1

Base Model

The core function of the base model in ConCoRD
is generating a set of candidate outputs for a given
input, which are ultimately re-ranked by the inference process (Sec. 3.3). Given a batch of N model
queries Q = {qi }, the first step of ConCoRD is togenerate a set of J candidate outputs for each queryÂi = {âi1 , ..., âiJ }, along with their corresponding likelihoods pθ (âij |qi ). Note that the candidate
outputs need not be an IID sample from the basemodel; for example, we might use beam searchwith a diversity bonus to produce a more diverse
set of candidates (Vijayakumar et al., 2018). Eachpair of query and candidate output forms a model
belief bij = (qi , âij ); the output of the base modelis the complete set of model beliefs B = {bij }and their corresponding normalized probabilities

2pij
θ . The base models in our experiments are pretrained question-answering models based on T5large (Raffel et al., 2020) and pre-trained visualquestion-answering models such as LXMERT (Tanand Bansal, 2019) and ViLT (Kim et al., 2021).

3.2

Relation Model

The relation model pϕ (·|x, x′ ) estimates themost likely logical relationship between an ordered pair of natural language utterances fromthe choices {none, fwd-entail, contradict,equivalence}.3 In addition to the model beliefs B, we define optional context statements
cijk = C(bij ), K relevant statements that may
be retrieved, generated, or manually written foreach model belief. The ability to incorporate context statements enables ConCoRD to modulate
model behavior independently for each input inthe test batch, rather than reasoning transductively
about pairs of test inputs. See Table 3 for examples of model beliefs and context statements. Inputs to the relation model are either pairs of twomodel beliefs (bij , bi′ j ′ ) or pairs of one model belief and one context statement (bij , cijk ). We define the most likely inter-belief relation as rij,i′ j ′ =argmaxr pϕ (r|bij , bi′ j ′ ), and similarly for beliefcontext relations rijk = argmaxr pϕ (r|bij , cijk ).
The output of the relation model is the set of mostlikely relations R = {rij,i′ j ′ } ∪ {rijk } and their′j′associated probabilities, which we denote as pij,iϕand pijk
ϕ . Our experiments use various pre-trained
NLI models based on RoBERTa (Liu et al., 2019)P
Normalized such that j pij
θ = 1.
Because relationships are estimated between orderedpairs of utterances, we can form an equivalence relation if
fwd-entail is predicted for both orderings of the utterances.
23and ALBERT (Lan et al., 2019) as the relation
model.
Question-answer to statement conversion.
While concatenating query qi and candidate
output âij to produce inputs to the relation model
is perhaps the simplest approach to estimating
soft constraints, we use a statement conversion
model to provide inputs to the relation model thatare closer to its training distribution. Instead ofdefining the belief bij = (qi , âij ) as concatenationof qi and âij , we define bij to be the statementfψ (qi , âij ), where fψ is the conversion model. We
fine-tune a small T5 model on a combination ofdata from (Demszky et al., 2018) and BeliefBank(Kassner et al., 2021) to produce a model that mapsa (question, answer) pair into a natural language
statement. Details about the fine-tuning procedureand data are provided in Appendix C.
3.3

which is equal to 1 for configurations where exactly
one of the nodes takes a value of 1, and 0 for allother configurations.
Binary factors ϕij,i′ j ′ (zij , zi′ j ′ ) and optionallyϕijk (zij , cijk ) encode compatibility between pairsof model beliefs (or model belief-context pairs):(
1
if rij,i′ j ′ (zij , zi′ j ′ )ϕij,i′ j ′ (zij , zi′ j ′ ) =ij,i′ j ′1 − pϕotherwise
where we define the relation function rij,i′ j ′ to evaluate to true if its arguments satisfy the underlying
relation, and false otherwise; ϕijk (zij , cijk ) is defined similarly to ϕij,i′ j ′ (zij , zi′ j ′ ) 4 . The inference
problem amounts to finding argmaxZ ϕ(Z), where!
ϕ (Z) =

Inference

ConCoRD’s inference procedure maps the set ofbeliefs B and pair-wise relations R into a choice ofthe most likely belief for each question. To define
the inference problem, we first define a binary decision variable zij representing the estimated truth
value of model belief bij . A value of 1 for node zij
in the maximum likelihood configuration meansthat âij is returned for query qi ; the problem includes a constraint that exactly one candidate answer is true for each query. The factor graph includes the set of variables Z = {zij }N,J
i,j=1,1 andvarious factors (functions mapping a subset of Z toa non-negative scalar) derived from the base modeland relation model’s beliefs and the hard constraintof returning only one answer per question. Factors
are defined such that more desirable configurationsof zij yield a larger product of the individual factors. First, unary factors ϕij (zij ) encode the basemodel’s beliefs about the likelihood of specific answers, and are defined as:( pij
if zij = 1
ϕij (zij ) = 1−pij(1)1otherwise
where pij = pθ (âij |qi ); in other words, the factor
takes the odds ratio if the corresponding statement
variable zij is assigned a truth value of 1; otherwise,
the factor takes value 1. In order to encode thehard constraint that exactly one output should be
returned for each query, we include a J-ary factor
ϕi (Zi ) for each group of nodes Zi = {zij }Jj=1 ,

Y
i

ϕi

Y
j

ϕij

Y
i′ j ′

ϕij,i′ j ′

!
Y

ϕijk

.

k

(2)
An approximate solution to this inference problem
can be efficiently found for most problems with a
MaxSAT solver such as RC2 (Ignatiev, 2019). We
omit arguments to the factors for conciseness. See
Figure 2 for a simple example of a factor graphwith a single inter-belief constraint and no beliefcontext constraints.
Entailment correction. Consider a belief b, a
set of its entailed statements S = {si }i , unary
factors ϕ(zb ) and {ϕ(zsi )}, and binary factors
P = {ϕ(zb , zsi )}i . Recall that an entailment relation rbsi (zb , zsi ) is satisfied (and the binary factor is maximized) if either zb = 0 or all zsi = 1.
Consequently, as the cardinality of {zsi |zsi = 0}
increases, the more likely it is that zb =Q 0 will maximize the product of all binary factors i ϕ(zb , zsi ).
This is true even if most entailed statements are true,i.e., |{zsi |zsi = 1}| ≫ |{zsi |zsi = 0}|. If most of
the statements entailed by a belief are true, assigning the belief to be false due to a small number of(potentially spuriously) false entailed statements
may be undesirable. To mitigate this outcome,we experiment with an additional type of factor
in which configurations satisfying entailments withboth zb = 1 and zsi = 1 are ‘rewarded’ more than4

We use this formulation only to accommodate settings
were multiple context statements are retrieved for each query;see Section 4.3. We do not have any ϕijk factors if we are
only using the model’s predictions within a batch of test inputs
as the premises for reasoning.

other configurations satisfying the entailment:
1− pb,si
ϕb,si (zb , zsi ) = 1
q ϕ
 1 − pb,si
ϕ

if zb , zsi = 1
if zb , zsi = 0
otherwise

Applying entailment correction consistently improves ConCoRD’s performance; see Appendix
Table 8 for a dataset-by-dataset breakdown.
3.4

Hyperparameters of ConCoRD

We introduce two key hyperparameters to ConCoRD. Because we do not know a priori the relative reliability of the base model and relation
model, we introduce the hyperparameter β ∈ [0, 1],
corresponding to a trade-off between the predictions of the base model and relation model. A
value of β = 1 corresponds to simply taking theraw predictions of the base model, while β =0 corresponds to optimizing purely for answers
that are self-consistent according to the relation
model, without considering the base model’s beliefs. The unary factors in the factor graph become
ϕβij (zij ) = (ϕij (zij ))β and ϕβij,i′ j ′ (zij , zi′ j ′ ) =1−β
ϕij,i′ j ′ (zij , zi′ j ′ )(and similarly for ϕβijk ). In
addition to β, we introduce a threshold λ for relation model confidence to filter out low-confidence
relation estimates. Thatis, we discard a relation′j′rij,i′ j ′ or rijk if pij,i<λ or pijkϕ
ϕ < λ, respectively. In practice, we find that the optimal β andλ vary across problems, perhaps due to the varying complexity of the model belief and context
statements (and therefore the reliability of the relation model’s predictions). Therefore, we use thehyperopt library (Bergstra et al., 2013) for automated hyperparameter optimization, using the Tree
Parzen Estimator (TPE) algorithm to tune β and λjointly. We use the optimal hyperparameters foundon the validation data for each problem to compute
test performance. Appendix H.1 details hyperparameter tuning for each experiment.

4

Experiments

Our experiments are broadly designed to answer
the high-level question: can ConCoRD leverage
the relational knowledge in pre-trained NLI models
to produce more accurate, self-consistent systembehavior, without additional data or fine-tuning?
Further, we investigate ConCoRD’s applicability toperforming test-time model editing (Sinitsin et al.,2020; Mitchell et al., 2022), or injection of new information, and ConCoRD’s sensitivity to the choiceof hyperparameters and types of relations detected.
4.1

Internal Consistency in Closed-Book
Question-Answering

Protocol. To evaluate the accuracy and consistencyof a set B of beliefs, Kassner et al. (2021) synthesize a gold standard for those beliefs and theinferred relations R. Following this prior work, we
assume the following is given:• A set of entities sm ∈ S• A set of unary predicates Pn ∈ P• A collection of “facts” (Pn (sm ))i , whose binary truth value is known• A directed graph of gold-standard constraints G(P, E), whose edges (Pn , Pn′ ) ∈E represent first-order logical formulae∀x (Pn (x) → Pn′ (x))
From these, we construct simple yes/no questions
using natural language templates. For example,
for fact Pn (sm ), if entity sm represents a lion andpredicate Pn represents an ability to drink liquids,
the template-generated gold question answer pair(qi , ai ) is Q: Is it true that a lion is able to drink
liquids?; A: Yes.
These questions are given as input to one of twosizes of a multi-angle question answering model(Tafjord and Clark, 2021), given a multiple choiceangle with choices Yes. and No. The questions andretrieved answers (qi , âi ) form a set of beliefs Bsm
for each entity. Since these are closed-book questions, no context statements are supplied; because
they are yes/no questions, only one candidate answer is obtained, i.e., J = 1. Question-answer
to statement conversion is applied to all questions
with a default answer of Yes. regardless of the answer âi , in order to provide the relation model withpositive natural language assertions from which toinfer sets of relations Rsm ; where the base modelanswers âi are No. we replace node zi in the factor
graph with its complement. Configurations Zsm
are found for each sm ∈ S which maximize Equation 2 given Bsm , Rsm and together form a global
solution Z.
Datasets. Kassner et al. (2021) provide a suitabledatabase with 12,636 facts (“silver facts”), eachindicating whether one of 601 predicates relates
to one of 85 entities, as well as 4,060 confidenceweighted first-order constraints manually gatheredfrom ConceptNet (Speer et al., 2017), forming aBase

ConCoRD

G.C.

Base

ConCoRD

Oracle

Model

F1

Con.

F1

Con.

F1

Con.

Model

Acc.

P.C.

Acc.

P.C.

Acc.

P.C.

Mac-LgMac-3B

0.831
0.855

0.835
0.871

0.914
0.931

0.920
0.947

0.862
0.905

0.934
0.936

LXM
ViLT

0.656
0.784

0.360
0.489

0.706
0.804

0.409
0.548

0.824
0.882

0.572
0.690

Table 1: F1 and consistency (1 - τ ) for two sizes of Macaw(Tafjord and Clark, 2021) QA models, comparing ConCoRD
to a naive QA baseline (Base) and ConCoRD with gold constraints (G.C.). ConCoRD significantly improves both F1 andconsistency for both models.

constraint graph G. Additionally, they provide
1,072 distinct “calibration facts”, each relating oneof 7 entities to one of 334 predicates.
We tune β and λ using a validation set of questions generated from the calibration facts, and evaluate test time performance with questions generated from silver facts.
Metrics. We measure accuracy using binary F1
between elements zi of the configuration Z maximizing ϕ(Z) (as in Equation 2), and the truth valueof facts (Pn (sm ))i . As in Kassner et al. (2021);
we use F1 for evaluation because gold answers arehighly biased towards true No. answers.
We compute consistency within batches ofquestions using the complement of of Li et al.
(2019)’s conditional constraint violation metric τ ,defined here as the proportion of relevant gold
constraints in G which are violated; a constraint∀x (Pn (x) → Pn′ (x)) is relevant iff, for some entity sm , there is some belief bi ∈ Bsm from fact
(Pn (sm ))i such that zi = 1, and there is some belief bj ∈ Bsm that corresponds to fact (Pn′ (sm ))j ;the constraint is violated when zj = 0.
Comparisons. ConCoRD is evaluated against a
naive baseline where only base model answers âiand probabilities are considered. A second baseline(G.C.) performs the inference described in Sec. 3.3,
replacing the inferred relations R with the gold
constraints from constraint graph G, rather thanthose estimated by the relation model.
Results. Results are shown in Table 1. ConCoRD
provides an absolute improvement of over 8% in
F1 and consistency for Macaw-Large and 7% for
Macaw-3B compared to the baseline. Notably, themargin of superiority of the Macaw-3B base model
is mostly preserved after applying ConCoRD, suggesting that ConCoRD may provide a significant
benefit even for very large models. A surprising
result is that ConCoRD shows marked improvements in F1 over the gold constraint baseline, suggesting that the detection and filtering of relations
ConCoRD provides may, in this setting, be an im-

Table 2: ConVQA accuracy (Acc.) and perfect consistency(P.C.) of LXMERT (Tan and Bansal, 2019) and ViLT (Kimet al., 2021) VQA models with and without ConCoRD. ConCoRD significantly improves accuracy and consistency of bothmodels. Oracle performance is top-2 performance, as ConCoRD attempts to select the best of the top 2 answer choicesof the base model.

provement over rigid adherence to the logical connections specified a priori in Kassner et al. (2021).
4.2

Internal Consistency in VQA

Protocol. The Visual Question Answering (VQA)
task involves a language model generating answers
to questions that are directly associated with images. VQA tests for robustness and generalizabilityof ConCoRD as it introduces an additional layer ofdifficulty; the task moves away from purely textbased tasks while expanding the answer space tothe vocabulary of the LM being used. The questions from the ConVQA dataset (Ray et al., 2019)and its associated images from the Visual Genome
dataset (Krishna et al., 2016) provide an apt setting
to assess ConCoRD, as the relatedness of questions for each image provide ample opportunity formodel self-inconsistency.
The ConVQA dataset consists of a set of images
each associated with a group of related questions
about the image, such as What color is the horse?
and Is the horse brown? for a picture of a brownhorse in a stable. We evaluate ConCoRD with two
VQA models, LXMERT (Tan and Bansal, 2019)and ViLT (Kim et al., 2021). For each group ofquestions Qn = {qni }i , we sample the top-2 candidate outputs {âni1 , âni2 } for each question, anduse a pre-trained NLI model to infer the most likely
pair-wise relations R between outputs from different questions. We use the RC2 MaxSAT Solver toestimate the configuration that maximizes Equation
2.
Metrics. We report accuracy as the proportion ofquestions answered correctly across all groups. We
infer consistency using a metric previously used in
the literature for the ConVQA dataset called "perfect consistency" (Ray et al., 2019). For all groupsof related questions, a group is perfectly consistent
if all its questions are answered correctly. Perfectconsistency then reports the proportion of question
groups that were perfectly consistent. While thisInput & Gold Answer

Generations

Added context

Q: What was the first capital cityof Australia? A: Melbourne

Canberra; Melbourne; Sydney; Inverell

Melbourne was the initial capital following the 1901
Federation of Australia.

Q: When does the implantationof the embryo occur?
A: around 9 days after ovulation

9 to 18 days; between 6
and 12 days; after the ovulation; on the 9th week

In humans, implantation of a fertilized ovum is most
likely to occur around 9 days after ovulation, however
this can range between 6 and 12 days.

Table 3: Success and failure in editing a model’s behavior with ConCoRD by adding new information to the context. The basemodel’s highest confidence answer is Underlined. Bold shows ConCoRD’s output after inference; with Teal, bold showing a
successful edit increasing F1 and Red, bold showing an edit that reduces F1.
F1
Model

Base

ConCoRD

OracleT5-Sm-NQ
T5-Lg-NQ
T5-3B-NQ

0.207
0.314
0.332

0.225
0.328
0.351

0.281
0.393
0.423

Table 4: Using ConCoRD to inject contextual information
into a model’s decisions at test time. Injecting gold Natural
Questions contexts consistently improves performance overthe base model without requiring fine-tuning.

is not a perfect measure of consistency as it excludes cases in which incorrect answers are consistent with each other, it still serves as a meaningfulproxy since the dataset was designed such that any
incorrect answer in a question group implies thepresence of inconsistency.
Datasets. We divide the ConVQA dataset into a"clean" (i.e. human verified and filtered) test set
and a non-test set (train + val + test as defined by
Ray et al. (2019)). From the non-test set, we sample 10,000 random images equivalent to 123,746
questions to be used as our validation set for tuning
our two hyperparameters. We use the clean test set– 725 images and 6,751 questions – to report our
final results.
Comparisons. ConCoRD is compared with a naive
baseline and a top-2 oracle upper bound. The naive
baseline is the answer with the highest VQA model
probability. Top-2 oracle upper bound selects thecorrect answer if present within the top-2 predictions of the VQA model. Top-2 is appropriate given
our use of the top-2 candidate outputs to generateinferences with NLI models.
Results. The final results for ConCoRD, baseline,
and oracle upper bound are shown in Table 2. ConCoRD increases the accuracy of LXMERT and
ViLT by 5% and 2% respectively, and the consistency of LXMERT and ViLT by 4.9% and 5.9% respectively. Examples in which ConCoRD correctly
and incorrectly selects a candidate output different
from the baseline output are shown in Figure 4 andFigure 5, respectively. In particular, the incorrect

scenarios demonstrate several failure modes that
may be in part responsible for the gap between
ConCoRD and the oracle upper bound, suggesting
further improvements of the components of ConCoRD will also continually improve ConCoRD.
4.3

Test-Time Information Injection

Protocol. We perform an additional experimentto evaluate ConCoRD’s ability to integrate external factual information into its inference process,rather than only using other predictions in the test
batch. Such an ability enables editing a model’s
behavior at test time, without re-training, as new
information becomes available. We use the Natural
Questions (NQ; Kwiatkowski et al. (2019)) dataset,rather than BeliefBank, to provide more challenging inputs to the relation model. Given a question
from NQ, a sentence from the ground truth context document containing information about theanswer is retrieved and provided as an additional
input to ConCoRD; we constrain the node representing this context variable in the factor graph tobe true. Constraints are predicted between eachanswer choice and the context statement. As in theother experimental settings, hyperparameters are
tuned on the validation set and applied on the test
set. See Appendix H for tuning procedures.
Metrics. Model performance is evaluated using
the SQuAD F1 score for overlapping tokens5 , following the same answer normalization protocols,
including lower-casing and removing punctuation.
Datasets. The NQ development set consists of
7830 open-book question-answer pairs, with both
long and short gold annotations in their context
passages. Since the NQ test set is not available,
we create a test and validation set from the NQ
validation questions as follows: we take the first
5000 questions to form our test set, and the rest to
be our val set, which we use for hyperparametertuning. Then each set is filtered such that only5
https://worksheets.codalab.org/bundles/
0xbcd57bee090b421c982906709c8c27e1

the answerable questions remain. “Answerable” isdefined as having a “short answer" span defined in
the annotations. This filtering process gives 2713
test entries and 1576 val entries.
Comparisons. ConCoRD is compared with a naive
baseline and an oracle upper bound. All of theseapproaches operate on the fixed set of QA modelanswers for a specific QA model (one of T5-SmNQ, T5-Lg-NQ, and T5-3B-NQ), specifically theset of top-4 answers for each question. The naive
baseline selects the answer with the highest QA
model probability, argmaxâij pθ (âij |qi ). The oracle upper bound approach selects the answer thathas the best score with the gold short answer span,argmaxâij F1 (âij , aij ).
Results. The results on the test set using the naive
baseline, ConCoRD, and oracle upper-bound arereported in Table 4. ConCoRD always outperforms the naive approach, demonstrating that theframework is useful even when each query input isprocessed independently (i.e., non-transductively).
However, despite providing a relative gain of as
high as 8.7% over the naive baseline, there is stilla gap between ConCoRD and the oracle. This gap
may be attributable to the complexity of the NQ
questions and context information compared withthe statements in prior experimental settings. Chenet al. (2021) demonstrate a significant gain in calibration performance from training on MultiNLI(Williams et al., 2018) to training on a combinationof MultiNLI and their NLI corpus adapted from
NQ, perhaps hinting that crucial knowledge present
in Natural Questions is not covered in MultiNLI,
partially explaining the gap between ConCoRD
and oracle F1 performance. Overall, these results
suggest that ConCoRD can reason between context
statements and model beliefs in addition to pairs ofmodel beliefs, improving performance even with
the increased complexity of the data.
Qualitative Analyses. Examples of “good” and“bad” edits (edits that improve and decrease theresulting F1-scores respectively) are presented inTable 3, with more in Appendix F. When the correct answer is not available in the candidate outputs,
ConCoRD is capable of pushing towards more partially correct answers and those that have more
overlap with the context.
4.4

Ablating Relation Types

Given that we consider two types of relations in
our experiments, contradiction and entailment, it

F1/Accuracy
Model

Task

Mac-LgBB
Mac-3B
BB
LXM
CVQA
ViLT
CVQA
T5-Sm-NQ
NQ
T5-Lg-NQ
NQ
T5-3B-NQ
NQ

ConCoRD Only cont. Only ent.
0.914
0.931
0.706
0.804
0.225
0.328
0.351

0.892
0.865
0.691
0.792
0.225
0.331
0.349

0.827
0.917
0.700
0.800
0.225
0.330
0.350

Table 5: Ablating the relation types considered in ConCoRD’sinference procedure. The Only cont. and Only ent. are theresults of applying ConCoRD with all entailment or contradiction relations removed, respectively. The ConCoRD column
is a reproduction of the results from Sections 4.1-4.3, for convenience. Value shown is F1 score for BeliefBank (BB) andNatural Questions (NQ) and accuracy for ConVQA (CVQA).
Note that hyperparameters β and λ are re-tuned on the respective validation set for each setting.

is natural to wonder the relative contribution ofthese to ConCoRD’s performance improvement;
Table 5 shows the results of this ablation. We re-run
ConCoRD with either entailment or contradiction
relations removed, re-tuning the hyperparameters
for both of the new settings (contradiction-only or
entailment-only). We find that the relative contribution of contradiction and entailment relations
varies significantly across models even within thesame task, but using both relation types always performs approximately as well or better than using
just one, suggesting that both types of detected relations from the NLI model carry useful information.
However, we observe in several cases, such as ViLT
and the T5 models, that the entailment and contradiction relations may encode somewhat redundant
information, as the performance when including
either type of constraint alone nearly matches thatof using both types.
4.5

Hyperparameter Sensitivity

We perform several experiments to clarify the relationship between the key hyperparameters, including the specific relation NLI model, β, and λ.
Impact of varying relation model. Table 6
shows a comparison of ConCoRD’s test performance for several NLI models for each setting; notably, the best-performing NLI model is not consistent across problems. While the Albert-XXL modelfrom Nie et al. (2020) is the strongest performing
model on NQ, the simpler RoBERTa-Large models
outperform it on BeliefBank and ConVQA.
Sensitivity to β and λ. Figure 3 shows the performance of ConCoRD on ConVQA with ViLT as

F1/Accuracy
NLI Model

Data

BB

ConVQANQ

Alb-XXL
RoB-Lg
RoB-Lg

ANLI
ANLI
MNLI

0.892
0.931
0.918

0.689
0.706
0.706

0.351
0.344
0.346

Table 6: Comparing ConCoRD’s performance for various NLI
models on BB (BeliefBank), ConVQA, and NQ. Performanceis measured as F1 score between predicted and gold text for
BB and NQ, exact match accuracy for ConVQA. We use
Macaw 3B for BB results, LXMERT for VQA results andT5-3B for NQ results. The best NLI model(s) in each column
are bolded; the best NLI model varies across problems.

Figure 3: Change in ConCoRD’s exact-match validation accuracy as λ (the NLI confidence threshold) and β (tradeoff
between base model and relation model beliefs) vary, holding
relation model RoBERTa-Large ANLI constant. By comparing the maximum value within each column or row, we
conclude that ConCoRD is relatively robust to the choice of
λ, which the choice of β is more important. Values are thoseencountered during tuning with base model ViLT on ConVQA
validation questions. Gray squares correspond to regions notevaluated during search, and asterisks (***) mark the region
where the maximum increase in accuracy occurs.

β (the tradeoff between base model and relation
model beliefs) and λ (the NLI confidence threshold) are varied, using the values explored duringhyperparameter optimization. Section H.2 of the
Appendix shows similar visualizations for different VQA experiments. If multiple hyperparameters within a grid element were explored, the best
performing configuration is shown. While the maximum value in each column is the same (0.04),
indicating that there exists a good value of β for
almost any λ, the converse is not true; for somevalues of β, no good value of λ exists. Thus, weconclude that the tradeoff parameter β is the more
important parameter to tune carefully.

5

Discussion & Conclusion

We have presented the ConCoRD framework for
enforcing self-consistency in pre-trained language
models using relations estimated by pre-trained
NLI models, showing that it improves over off-theshelf performance in a variety of settings without
requiring any fine-tuning. Our findings suggest thatexisting pre-trained NLI models can be a usefulbuilding block for boosting performance of NLP
systems by providing useful estimates of logicalrelationships between model predictions across various models and datasets for QA and visual QA.
ConCoRD also suggests several directions for
future work. Integrating ConCoRD with meth-ods that generate questions likely to elicit useful knowledge for answering the question at hand(Ray et al., 2019; Shwartz et al., 2020) may further
improve performance. In addition, integrating aframework such as ConCoRD with recent methods
for differentiation through black box combinatorial
solvers (Pogančić et al., 2020) may enable training of the entire base model, relation model, andinference pipeline end-to-end, potentially further
improving aggregate performance. Finally, ConCoRD’s general mechanism of re-ranking predictions by estimating the self-consistency of groups
of model predictions is applicable beyond natural language, and future work might investigate
its application to problems in vision or sequential
decision-making. We hope that ConCoRD may
serve as another promising example of integrating both neural and explicit symbolic inferencemachinery into a broader intelligent system that
outperforms any of its components individually.

6

Limitations

While our results suggest ConCoRD can effectively
leverage additional compute to boost model performance without fine-tuning, our work has some
limitations. Although ConCoRD is conceptuallyapplicable to generations from any language model,
our work focuses on question-answering settings toleverage existing self-consistency benchmarks. In
addition, ConCoRD increases the compute costs ofinference, although it does not require fine-tuning.
Further, our results suggest that the best NLI model
to use for ConCoRD may vary across domains, requiring some tuning. As NLI models improve, wemight hope that the final performance of ConCoRDlike systems should also inherit these gains, but Table 6 suggests that the factors that make a particular
NLI model well-suited to a particular problem are
not obvious, requiring further investigation.

Acknowledgements
The authors would like to thank the anonymous
reviewers for their helpful feedback during the review period, Gabe Mudel, Julie Wang, Cameron
Tew, Anthony Tzen, Kevin Yang, and Ian Ng forhelpful discussions and assisting with exploratory
experiments early on in the project, and Nora Kassner for providing helpful early guidance in configuring the BeliefBank experiments. CF and CM
are CIFAR Fellows. EM gratefully acknowledges
funding from the Stanford Knight-Hennessy Graduate Fellowship. JN is supported by Stanford University Medical Scientist Training Program grants
T32-GM007365 and T32-GM145402. SL acknowledges brownie bites from Target for providing a crucial fuel source for late night experiment-running.

References
Chris Alberti, Daniel Andor, Emily Pitler, Jacob Devlin,
and Michael Collins. 2019. Synthetic QA corpora
generation with roundtrip consistency. In Proceedings of the 57th Annual Meeting of the Association for
Computational Linguistics, pages 6168–6173, Florence, Italy. Association for Computational Linguistics.
Akari Asai and Hannaneh Hajishirzi. 2020. Logicguided data augmentation and regularization for consistent question answering. In Proceedings of the58th Annual Meeting of the Association for Computational Linguistics, pages 5642–5650, Online. Association for Computational Linguistics.
James Bergstra, Dan Yamins, David D Cox, et al. 2013.
Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms. In
Proceedings of the 12th Python in science conference, volume 13, page 20. Citeseer.
Jifan Chen, Eunsol Choi, and Greg Durrett. 2021. Can
NLI models verify QA systems’ predictions? In
Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3841–3854, Punta
Cana, Dominican Republic. Association for Computational Linguistics.
Dorottya Demszky, Kelvin Guu, and Percy Liang.
2018. Transforming question answering datasets
into natural language inference datasets. CoRR,
abs/1809.02922.
Nouha Dziri, Ehsan Kamalloo, Kory Mathewson, andOsmar Zaiane. 2019. Evaluating coherence in dialogue systems using entailment. In Proceedings ofthe 2019 Conference of the North American Chapter of the Association for Computational Linguistics:
Human Language Technologies, Volume 1 (Long andShort Papers), pages 3806–3812, Minneapolis, Minnesota. Association for Computational Linguistics.

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze,and Yoav Goldberg. 2021. Measuring and improving
consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031.
Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai
Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas
Scialom, Idan Szpektor, Avinatan Hassidim, andYossi Matias. 2022. TRUE: Re-evaluating factual
consistency evaluation. In Proceedings of the Second
DialDoc Workshop on Document-grounded Dialogue
and Conversational Question Answering, pages 161–
175, Dublin, Ireland. Association for Computational
Linguistics.
Alexey Ignatiev. 2019. Rc2: an efficient maxsat solver.
J. Satisf. Boolean Model. Comput., 11:53–64.
Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, andYejin Choi. 2022. Maieutic prompting: Logicallyconsistent reasoning with recursive explanations.
Nora Kassner, Oyvind Tafjord, Hinrich Schütze, andPeter Clark. 2021. BeliefBank: Adding memory to a
pre-trained language model for a systematic notionof belief. In Proceedings of the 2021 Conferenceon Empirical Methods in Natural Language Processing, pages 8849–8861, Online and Punta Cana, Dominican Republic. Association for Computational
Linguistics.
Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021.
ViLT: Vision-and-language transformer without convolution or region supervision. In ICML, pages 5583–
5594.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen,
Yannis Kalantidis, Li-Jia Li, David A Shamma,
Michael Bernstein, and Li Fei-Fei. 2016. Visual
genome: Connecting language and vision using
crowdsourced dense image annotations. In International Journal of Computer Vision.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti,
Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew
Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob
Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question answeringresearch. Transactions of the Association for Computational Linguistics, 7:452–466.
Philippe Laban, Tobias Schnabel, Paul N. Bennett, andMarti A. Hearst. 2022. SummaC: Re-visiting NLIbased models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics, 10:163–177.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman,
Kevin Gimpel, Piyush Sharma, and Radu Sori-cut. 2019.
ALBERT: A lite BERT for selfsupervised learning of language representations.
CoRR, abs/1909.11942.
Tao Li, Vivek Gupta, Maitrey Mehta, and Vivek Srikumar. 2019. A logic-driven framework for consistencyof neural models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference
on Natural Language Processing (EMNLP-IJCNLP),
pages 3924–3935, Hong Kong, China. Association
for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis,
Luke Zettlemoyer, and Veselin Stoyanov. 2019.
Roberta: A robustly optimized BERT pretraining
approach. CoRR, abs/1907.11692.
Hans-Andrea Loeliger. 2008. An introduction tofactor graphs.
https://people.binf.ku.dk/~thamelry/MLSB08/hal.pdf.

Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789,
Melbourne, Australia. Association for Computational
Linguistics.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, andPercy Liang. 2016. SQuAD: 100,000+ questions formachine comprehension of text. In Proceedings ofthe 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin,
Texas. Association for Computational Linguistics.
Arijit Ray, Karan Sikka, Ajay Divakaran, Stefan Lee,and Giedrius Burachas. 2019. Sunny and darkoutside?! improving answer consistency in VQA
through entailed question generation. In Proceedingsof the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International
Joint Conference on Natural Language Processing(EMNLP-IJCNLP), pages 5860–5865, Hong Kong,
China. Association for Computational Linguistics.

Bill MacCartney and Christopher D. Manning. 2008.
Modeling semantic containment and exclusion in natural language inference. In Proceedings of the 22ndInternational Conference on Computational Linguistics (Coling 2008), pages 521–528, Manchester, UK.
Coling 2008 Organizing Committee.

Vered Shwartz, Peter West, Ronan Le Bras, Chandra
Bhagavatula, and Yejin Choi. 2020. Unsupervisedcommonsense question answering with self-talk. In
Proceedings of the 2020 Conference on Empirical
Methods in Natural Language Processing (EMNLP),
pages 4615–4629, Online. Association for Computational Linguistics.

Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea
Finn, and Christopher D. Manning. 2022. Fast model
editing at scale. ICLR.

Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry Pyrkin,
Sergei Popov, and Artem Babenko. 2020. Editableneural networks. In ICLR.

Yasuhide Miura, Yuhao Zhang, Emily Tsai, Curtis Langlotz, and Dan Jurafsky. 2021. Improving factual
completeness and consistency of image-to-text radiology report generation. In Proceedings of the 2021
Conference of the North American Chapter of the
Association for Computational Linguistics: Human
Language Technologies, pages 5288–5304, Online.
Association for Computational Linguistics.

Haoyu Song, Wei-Nan Zhang, Jingwen Hu, and Ting
Liu. 2020. Generating persona consistent dialogues
by exploiting natural language inference. Proceedings of the AAAI Conference on Artificial Intelligence,
34(05):8878–8885.

Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal,
Jason Weston, and Douwe Kiela. 2020. Adversarial
NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meetingof the Association for Computational Linguistics. Association for Computational Linguistics.
Marin Vlastelica Pogančić, Anselm Paulus, Vit Musil,
Georg Martius, and Michal Rolinek. 2020. Differentiation of blackbox combinatorial solvers. In International Conference on Learning Representations.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi
Zhou, Wei Li, and Peter J. Liu. 2020. Exploring thelimits of transfer learning with a unified text-to-texttransformer. Journal of Machine Learning Research,
21(140):1–67.
Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018.
Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual

Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.
Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the Thirty-FirstAAAI Conference on Artificial Intelligence, AAAI’17,
page 4444–4451. AAAI Press.
Oyvind Tafjord and Peter Clark. 2021. Generalpurpose question-answering with macaw. CoRR,
abs/2109.02593.
Hao Tan and Mohit Bansal. 2019. LXMERT: Learning
cross-modality encoder representations from transformers. In Proceedings of the 2019 Conference on
Empirical Methods in Natural Language Processing
and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages
5100–5111, Hong Kong, China. Association for Computational Linguistics.
Ashwin Vijayakumar, Michael Cogswell, Ramprasaath
Selvaraju, Qing Sun, Stefan Lee, David Crandall,
and Dhruv Batra. 2018. Diverse beam search for
improved description of complex scenes. Proceedings of the AAAI Conference on Artificial Intelligence,32(1).

Sean Welleck, Jason Weston, Arthur Szlam, andKyunghyun Cho. 2019. Dialogue natural language
inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,
pages 3731–3741, Florence, Italy. Association for
Computational Linguistics.
Adina Williams, Nikita Nangia, and Samuel Bowman.
2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume
1 (Long Papers), pages 1112–1122, New Orleans,Louisiana. Association for Computational Linguistics.

A

Reproducing Macaw-Large Examples

The following configuration reproduces the MacawLarge behavior noted in the abstract and the introduction at https://huggingface.co/allenai/
macaw-large.
$answer$ ; $question$ = Is a sparrow abird? ; $mcoptions$ = (A) Yes. (B) No. ;$answer$ ; $question$ = Does a bird havefeet? ; $mcoptions$ = (A) Yes. (B) No. ;$answer$ ; $question$ = Does a sparrow
have feet? ; $mcoptions$ = (A) Yes. (B)
No. ;

B

Factor Graph Overview

A factor graph is a factorization of a function f
mapping a set of n variables Z = {zj }nj=1 to a nonnegative scalar. The factorization is represented
as a bipartite graph containing variable nodes andfactors; each zj is represented by one variable node,and each factor ϕi maps a subset of the variablenodes Zi to a non-negative scalar. The
Q value ofthe function is computed as f (Z) = i (Zi ). See
Loeliger (2008) for a more complete reference.

C

Question-Answer to Statement
Conversion Model Details

To convert question-answer pairs into declarative statements, we combine data from the Question to Declarative Sentence (QA2D) (Demszky
et al., 2018) and BeliefBank (Kassner et al.,2021) datasets to fine-tune a T5-base sequence-tosequence model. QA2D contains question-answer
pairs from five QA datasets; 95% of the pairs arefrom SQuAD (Rajpurkar et al., 2016). The goldstatements are from Amazon Mechanical Turk. The
BeliefBank questions are created from silver facts
using natural language templates as in Section 4.1,

and the yes/no answers are from the known binary
truth values of these facts. Our training dataset iscomposed of the full QA2D training dataset of 61kquestion-answer pairs and half of the BeliefBanksilver facts, for a total of 67k training examples.
Likewise, the validation dataset consists of the full
QA2D validation dataset of 10k pairs and half theBeliefBank silver facts, for a total of 16k validation
pairs.
The input to the QA statement conversion modelis the concatenation of the question-answer pairqi ∥ ai . Accuracy is evaluated by comparing theoutput sequence tokens to the gold sequence tokens.
Training occurs with a learning rate of 1e−4 for a
maximum of 50 steps, where each step consistsof 32k training examples, with early stopping if
validation loss does not decrease for 6 consecutive
steps. We ran the fine-tuning on NVIDIA GeForce
RTX 3090 GPU. Fine-tuning ended after 14 steps
with a final training accuracy of 0.764 and validation accuracy of 0.628. This took approximately
40 minutes. Table 7 demonstrates the model’s performance on a few validation examples.

D

Additional Modifications to ConCoRD

A timeout for solvers is imposed in order to prevent
the RC2 MaxSAT solver from running optimizationindefinitely. The average solve time per questionwas <4 ms for closed-book QA, <1 ms for VQA
and <20 ms for NQ (for NQ, the solve time is< 1/10th of the time needed for a forward passthrough the QA and NLI models). We found only
one batch of test questions for the closed-book QAtask and VQA task where the solver couldn’t find asolution efficiently, so we set a short timeout (30s
for CBQA, 10s for VQA, none required for NQ).
We also de-duplicate the list of inferred constraints before passing the statement and constraintgroups through the MaxSAT solver so that only thehighest-weighted constraints would remain among
their duplicates.

E

Entailment Correction Ablations

Table 8 shows the effects of entailment correction on ConCoRD test performance in closed-bookquestion answering and VQA experiments for different choices of base model, using the NLI relation model resulting in the best test set performance(RoBERTa-Large-MNLI).

Dataset

Input

OutputGold statement

SQuAD

Who established Yale’s residential college system? Edward S.
Harkness
How did Kuhn view the history of science? competing
paradigms or conceptual systems
Is it true that a poodle is a river?
No
Is a pigeon a living thing? Yes

Edward S. Harkness established
Yale’s residential college system.
Kuhn viewed the history of science as a competing paradigm
or conceptual system.

Edward S. Harkness established
Yale’s residential college system.
Kuhn viewed the history of science as competing paradigms
or conceptual systems.

A poodle is not a river.

A poodle is not a river.

A pigeon is a living thing.

A pigeon is a living thing.

SQuAD

BeliefBank
BeliefBank

Table 7: The QA statement conversion model outputs declarative statements from question-answer pairs. Out of the four
validation examples presented, three are correct. The Red, bolded portion of the output of the second example indicates how it
differs from the Teal, bolded corresponding portion of the gold statement.

Model

F1/Accuracy

Experiment

Model

Good Flips

Bad Flips

Naive w. E.C. w/o. E.C.

BeliefBank
VQA
NQ

Macaw-3B
LXMERT
T5-3B-NQ

723
576
168

277
238
69

Mac-Lg+Rob/ANLI 0.831
Mac-3B+Rob/ANLI 0.855
LXMERT+Rob/MNLI 0.656
LXMERT+Rob/ANLI 0.656
ViLT+Rob/MNLI 0.784
ViLT+Rob/ANLI 0.784

0.914
0.931
0.706
0.706
0.804
0.814

0.909
0.886
0.701
0.693
0.810
0.807

Table 8: Comparison of ConCoRD test performance vs. baseline with and without entailment correction (E.C.) acrossbase+relation models for closed-book question answering(Macaw) and VQA (LXMERT, ViLT) experiments (F1 forclosed-book QA, exact-match accuracy for VQA), showingthat the entailment correction improves performance for mostconfigurations.

F

Additional “Good” and “Bad” Edit
Pairs

More examples of good and bad edits in the Editing experiment are presented in Table 10. We also
include good (Figure 4)and bad flip (Figure 5) examples from the VQA dataset. For the bad flip examples in VQA, we include different failure modes
to demonstrate the types of potential ConCoRD
errors.

G

Good and Bad Flips

For each set of experiments on the test set, wereport the numbers of good and bad flips made by
ConCoRD in Table 9. It can be observed that the
number of good flips is consistently significantly
higher than that of bad flips.

H

Hyperparameter Search Details

H.1

Experiments

H.1.1

Closed-Book Question Answering

Hyperparameters (Section 3.4) are tuned jointly using hyperopt on the BeliefBank calibration dataset

Table 9: The numbers of good and bad flips in each of theexperiments performed. We define flips as choosing a different
candidate from the naive baseline for the multiple choice
experiments, and a binary truth value flip for BeliefBank.
“Good” flips are flips that improves performance, and “bad”
flips are those that are detrimental to performance.

(Section 4.1). The search space of β is uniform
between [0.05, 1.0], and for λ it is uniform between [0.5, 1.0]. hyperopt optimizes cumulative
F1 across all entity batches for 300 trials. To speedup tuning, we created caches of model beliefs Bsm
and relation sets Rsm for each calibration entity
sm . This was run on NVIDIA GeForce RTX 3090
GPU, and the largest NLI models took up to twohours to complete. Using these caches, hyperopt
tuning completes in less than an hour on CPU. The
best performance on the calibration facts for eachof the base Macaw models is reported in Table 11.
The results show that β is higher for the better basemodel Macaw-3B.

H.1.2

VQA

Hyperparameters are tuned jointly using hyperopt.
The search space for β is uniform over [0.05, 1], for
λ it is uniform over [ 31 , 1]. A total of 100 trials were
performed, updating parameters using TPE, on an
AWS g4dn.xlarge EC2 instance. Each search
took less than one hour. Table 12 shows the selected parameters and their exact-match accuracy
on validation questions.

ModelInput & Gold Answer

Generations

Added context

T5-Sm-NQ

Q: Who was the declaration of independence written for?
A: the Second Continental
Congress

Second Continental
Congress;the UnitedStates; the British Crown;
Great Britain

T5-Sm-NQ

Q: What is the scientificname for the calf muscle
A: gastrocnemius muscle

The serratus calf muscle;
gastrocnemius
muscle;The serratus calfi; The
serratus muscle

The United States Declaration of Independence is the statement adopted by
the Second Continental Congress meeting at the Pennsylvania State House (Independence Hall ) in Philadelphia on
July 4 , 1776 , which announced thatthe thirteen American colonies , then at
war with the Kingdom of Great Britain ,regarded themselves as thirteen independent sovereign states , no longer under
British rule .
Along with the soleus muscle , the gastrocnemius forms half of the calf muscle.

T5-3B-NQ

Q: Who is the actor thatplays Dr. Sean Murphy
A: Freddie Highmore

Freddie Highmore; Daryl“Chill” Mitchell; Dylan
Christopher
Minnette;
Javier Muoz

T5-3B-NQ

Q: Who is the founder of the
Ubuntu project
A: Mark Richard Shuttleworth

Linus Torvalds;
Mark
Shuttleworth; Richard St.
John Hopper; Richard St.
John Redmond

The series stars Freddie Highmore as
Shaun Murphy , a young surgical resident with autism and savant syndromeat San Jose St. Bonaventure Hospital .,Freddie Highmore as Shaun Murphy : A
surgical resident with autism and savant
syndrome .
Mark Richard Shuttleworth ( born 18
September 1973 ) is a South African entrepreneur who is the founder and CEOof Canonical Ltd. , the company behindthe development of the Linux - based
Ubuntu operating system .

Table 10: Editing a model’s behavior by adding new information to the context. The Underlined generation is the answer with thehighest QA model confidence. The Bolded generation is what ConCoRD selects after NLI inference. Teal, bolded generations
indicate that ConCoRD selects a generation with higher token overlap F1, while Red, bolded generations indicate that ConCoRD
selects a worse generation.

Figure 4: “Good” flip examples from the VQA experiments.
The green texts mark the correctly selected answers, while thered texts indicate incorrectly selected answers.

Figure 5: “Bad” flip examples from the VQA experiments.
The green texts mark the correctly selected answers, while
the red texts indicate the incorrectly selected answers. The
bolded texts are the correct answers, if generated within the
top-2 predictions. From top to bottom, the first image isan example of when the correct answer, "sheet," was notcontained in the predicted answers. The second image isan example of when the conversion of QA pair to statement
did not occur as intended and the NLI failed to generate theappropriate inferences that could be used to inform correctionof "background" to "buildings. The third image shows anexample of when an "incorrect" answer (sky) is effectively
the same as the "correct" answer (in sky)–only semanticallydifferent. The fourth image shows an example of when themodel strongly believed in an incorrect answer and changed
another correct answer.

Model

F1

β

λ

E.C.

Macaw-Large
Macaw-3B

0.919
0.94

0.753
0.804

0.855
0.873

True
True

Table 11: Validation performance on the BeliefBankcalibration facts. Both models achieve best validation
performance with the RoBERTa-Large ANLI model.
VQA

Acc.

β

λ

E.C.

LXMERT
ViLT

0.691
0.787

0.208
0.395

0.805
0.772

True
True

Table 12: Validation performance on VQA. Both models

achieve best validation performance with the RoBERTaLarge MNLI model.

H.1.3

Information Injection with Natural
Questions
For this round of experiments, we lower the bounds
for β and λ after some initial trials. The boundsof β are [0, 0.5] and the bounds of λ are [0, 0.6].
We run hyperopt for 200 trials (often taking approximately 2 to 3 hours on an NVIDIA GeForce
RTX 3090 GPU) for each of the three NLI models. Hyperopt optimizes for the highest tokenoverlapping F1 score in this experiment.
We report the best validation performance ofeach of the QA base models in Table 13.
Model

F1

β

λ

E.C.

T5-SmallT5-Large
T5-3B

0.227
0.331
0.353

0.112
0.081
0.072

0.540
0.413
0.477

True
False
True

(a) Base model LXMERT, with entailment correction.

(b) Base model LXMERT, without entailment correction.

Table 13: Validation performance on NQ. All models

achieve best validation performance with the ALBERTANLI model.

H.2

Visualizing Hyperparameter Search

Figure 6 shows increases in exact-match accuracy
as they vary with choices of λ, β, for additional
choices of base model for a VQA task, with andwithout entailment correction, complementing figure 3. Interestingly, choosing a different basemodel does noticeably effect the optimum valueof β; between figures 6b and 6c we see the nearoptimal region shift towards a value of β that gives
higher confidence in the base model where the basemodel produces “better” answers. However, theincrease in accuracy is similar, suggesting that withappropriate selection of β, ConCoRD can offer similar improvements over a range of choices of basemodel.

(c) Base model ViLT, without entailment correction.
Figure 6: As in figure 3, we show changes in exact-match
validation accuracy as a function of confidence threshold λand tradeoff parameter β, with several choices of base model,
with and without an entailment correction, holding relation
model RoBERTa-Large ANLI constant.