Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability (2024)

Abstract: While language models (LMs) can sometimes generate factually correct text and estimate truth values of individual claims, these generally do not reflect a globally coherent, manipulable model of the world. As a consequence, current LMs also generate incorrect or nonsensical content, and are difficult to edit and bring up to date. We present a method called Deductive Closure Training (DCT) that uses LMs themselves to identify implications of (and contradictions within) the text that they generate, yielding an efficient self-supervised procedure for improving LM factuality. Given a collection of seed documents, DCT prompts LMs to generate additional text implied by these documents, reason globally about the correctness of this generated text, and finally fine-tune on text inferred to be correct. Given seed documents from a trusted source, DCT provides a tool for supervised model updating; if seed documents are sampled from the LM itself, DCT enables fully unsupervised fine-tuning for improved coherence and accuracy. Across the CREAK, MQUAKE, and “Reversal Curse” datasets, supervised DCT improves LM fact verification and text generation accuracy by 3–26%; on CREAK, fully unsupervised DCT improves verification accuracy by 12%. These results show that LMs’ reasoning capabilities during inference can be leveraged during training to improve their reliability.

heading: 1 Introduction
text: There is increasing interest in using language models (LMs) as sources of information and tools for fact verification (Porter, 2023; Zhang and Gao, 2023). But today’s LMs cannot robustly perform either task: they are prone to generating factually incorrect information, contradict themselves, and are difficult to update with new information (Honovich et al., 2021a; Liska et al., 2022; Sun et al., 2023; Gilson et al., 2023).
∗Correspondence to akyurek@bu.edu.
Even if LMs are imperfect judges of factuality, however, they are quite reliable models of factual relations between pieces of text: they can identify logical and probabilistic relationships between statements (Williams et al., 2017), and generate text based on new information provided as input (Yehudai et al., 2024). For example, an LM that cannot answer How old was Charlie Chaplin when
9802
he died? may nonetheless answer correctly when prompted with Charlie Chaplin lived between 1889 and 1977, and recognize that this statement contradicts the claim Charlie Chaplin lived in the 21st century. How can we leverage LMs’ ability to reason about relations between claims to improve (and control) the text that LMs themselves generate?
Conceptually, standard supervised objectives cause LMs to assign high probability to statements in their training data, but not necessarily these statements’ logical consequences. Additional reasoning is required to determine the deductive closure of a training set (Armstrong, 1973)—the complete collection of inferences that can be made given the information initially available. An alternative procedure is needed to ensure that LMs assign high probability to a complete and consistent set of facts when they are trained and fine-tuned.
In this paper, we propose a new LM fine-tuning procedure we call Deductive Closure Training (DCT), which leverages inference-time reasoning as a source of training-time supervision. At high level, given seed text (which may be provided externally or LM-generated), DCT uses an LM to identify additional text implied by or contradicting this text, reasons globally about which portions of seed and generated text are most likely to be correct given this context, and finally fine-tunes on inferred-correct text. This approach builds on a large body of recent work (Mitchell et al., 2022b; Kassner et al., 2023; Hase et al., 2023) on inferencetime procedures for improving models’ factual correctness, showing that these techniques may be used at training time as well.
DCT may be applied in several different ways depending on the source of seed documents. If these are drawn from a trusted factual source, DCT may be used to perform supervised adaptation for factuality. If documents contain new information to be inserted into an LM, DCT provides tool for model updating (or “editing”; De Cao et al., 2021). Finally, if seed documents are generated by the model itself, DCT enables fully unsupervised finetuning of models for improved accuracy.
We demonstrate the effectiveness of DCT across three domains: fact verification (CREAK benchmark; Onoe et al., 2021), question answering with new information (on the MQUAKE benchmark; Zhong et al., 2023), and a synthetic test of edit propagation (on the “Reversal Curse” benchmark; Berglund et al., 2023). On these tasks, unsupervised and supervised applications of DCT improve
accuracy by up to 12% and 26%, respectively. These results show that, with little or no data, LMgenerated supervision can be leveraged to improve LMs’ coherence, accuracy and updatability.1

heading: 2 Related Work
text: DCT builds on several recent techniques for improving model accuracy via inference-time computation or training-time self-supervision.
Bootstrapping accuracy during inference A growing body of research adopts techniques that bootstrap language model performance at inference time. Tafjord et al. (2022); Bostrom et al. (2022); Weir and Van Durme (2022) and Jung et al. (2022) build self-guided semantic chains of reasoning to support inference. Suzgun et al. (2022) propose a set of procedures that bin model-generated candidate answers by semantic equivalence and later uses aggregated probabilities to select the highest ranked predictions, analogous to self-consistency (Wang et al., 2023) for textual outputs. Finally, recent work has shown promise in improving coherence by conditioning language models on relevant reference texts through retrieval augmentation (Mitchell et al., 2022a; Akyürek et al., 2023). Our approach builds on this line of work by using inference-time techniques to generate supervision.
Training for accuracy LMs greatly benefit from training or post-training techniques for improving accuracy, including instruction-tuning (Sanh et al., 2022), learning from feedback (Ouyang et al., 2022) and loss truncation (Kang and Hashimoto, 2020). Closest to our approach is the work of Hase et al. (2023) which leverages graph-structured representations of model “beliefs” to train a hypernetwork for model editing. DCT aligns with this thread in improving model training; it differs by requiring minimal or no external supervision.
Self-training Past work has also studied leveraging LMs themselves for performance improvements (Pan et al., 2023). Several studies use external tools (Schick et al., 2023), binary feedback (Pang et al., 2023; Liu et al., 2023) and natural language feedback (Bai et al., 2022) to improve capability or reduce harms. Others propose actuality and consistency metrics, which might be used for filtering bad answers in retrospect (Honovich et al., 2021b; Wang et al., 2020; Honovich et al.,
1Code is available at https://lingo-mit.github.io/ deductive-closure.
2022). Related to such approaches are methods that perform multiple inference attempts and aggregate them to get a more consistent answer (Wang et al., 2022; Yoran et al., 2023). Padmanabhan et al. (2023) fine-tune LMs on self-generated text without explicit implication generation or logical inference. Of immediate relevance to the current work, Li et al. (2023) and a concurrent study by Tian et al. (2023) use LM-generated factuality labels to rank or filter LM-generated data for fine-tuning; by contrast, DCT uses LMs to explicitly extrapolate from LM-generated or externally provided information, providing a single framework for both supervised model updating and unsupervised improvement.

heading: 3 Method

heading: 3.1 Preliminaries
text: Given a language model pLM that places a probability distribution over strings, our goal is to optimize pLM so that it is coherent (if pLM assigns high probability to statements P and Q, those statements must be logically compatible) and complete (if pLM assigns high probability to P , and P implies Q, then pLM must also assign high probability to Q). Together, these two properties imply that the LM is closed under logical deduction. Deductive closure is necessary condition for pLM to be truthful, and approximate deductive closure is generally agreed to be an important feature of human-like belief (Armstrong, 1973).
Deductive closure training begins with a set of seed documents si, which may comprise facts from a trusted source, new information provided by a user, or even text generated by pLM itself.2 At a high level, DCT works by using pLM to generate additional text implied by each seed document (i.e., true with high probability conditioned on s) or contradicting it. In Fig. 2, for example, the seed text (Country music originated in the United Kingdom)3 is used to generate statements (The UK is famous for country music), question-answer pairs (Q: Where did country music originate? A: England) and even multi-hop consequences (The steam train was invented in the UK; therefore, country music and the steam train were invented in the same country). Once they have been generated,
2While experiments in this paper focus on seed documents consisting of questions and declarative statements, this approach could be straightforwardly applied to larger pieces of text.
3Most editing benchmarks comprise counterfactual examples like this one.
DCT again uses pLM to reason about these documents as a set, identifying the subset of generated documents most likely to be true. Finally, DCT fine-tunes pLM on documents in this inferred-true set. In the following sections, we describe each of these steps in more detail.

heading: 3.2 Document Generation
text: The first step of DCT is to generate a set of related documents for each seed document (Fig. 2a) using
pLM. Formally, we first construct a set of textual prompts that instruct the LM to generate other documents entailed by and contradicted by the input, along with 1–5 examples. We denote these prompts primp and prcon respectively (see Appendix D for full prompt text). Then, we construct a collection of related documents Ri for each seed document si, i ∈ {1..n} as:
Ri = Ii ∪ Ci ∪ {si}, Ii = {rij ∼ pLM(· | primp, si)}, Ci = {rij ∼ pLM(· | prcon, si)}, (1)
where I and C denote generated implications and contradictions respectively. (Other procedures for generating related documents are also possible, e.g. by simply prompting pLM to generate similar text, as described in Section 5.1.) Note that the seed document si is included in Ri—this is crucial for detecting (and correcting) errors in the seed itself during unsupervised training.
This generation step may be followed by a double-checking step over Ri, in which we use the pLM to verify whether si entails / contradicts rij , and discard all rij for which pLM does not output yes with high probability (the prompt template is available in Appendix D). This step mirrors a variety of other recent methods in which models reevaluate their initial answers (Suzgun et al., 2022).

heading: 3.3 Consistency Evaluation
text: The previous step produces a collection of documents in the “deductive neighborhood” of each seed document. These documents may be mutually contradictory, and we wish to identify the subset most likely to be collectively true. To identify this subset, we leverage pLM’s ability to classify logical relations between documents, as well as the prior probability pLM assigns to each document. For example, if it is true that Emperor Meiji was the first emperor the Modern Japan, it cannot be the case that Emperor Meiji was the last Japanese emperor; if the former statement is very likely to be true, then the latter is likely to be false.
Formally, we first associate with the seed document si and every generated document rij a truth value tij ∈ {0, 1}. Given an assignment of documents to truth values denoted by Ti = {tij}, we compute the LM’s probability of Ti:
p(Ti | Ri) = ∏
j
pLM (tij | rij). (2)
We use prompting to estimate each pLM(tij | rij): we first condition pLM on a small set of document–label pairs where labels are one of {True, False}. Next, we use the normalized logits corresponding to the tokens true and false in the string pLM(rij is true) and pLM(rij is false), respectively. Refer to Appendix D for the prompt template. Next, we define a value assignment Ti = {tij} to be consistent if all implications and contradictions are respected.
c(Ti) = ∏
j:rij∈Ii 1[ti → tij ]
∏
j:rij∈Ci 1[ti → ¬tij ]
where ti denotes the truth value of the seed document, 1[a → b] is 1 iff b is true or a is false, and 1[a ̸→ b] is 1 iff b is false or a is false. We also provide an example for consistency computation across different truth value assignments in Table 6 in Appendix A. Finally, we select the most probable consistent assignment:
T ∗i = argmax T c(T | Ri) · p(T | Ri) . (3)
The procedure is depicted in Fig. 2b, with the highest-scoring truth value assignment shown in the blue-highlighted box.

heading: 3.4 Language Model Fine-Tuning
text: Finally, we fine-tune pLM only on the inferred-true4 documents, optimizing:
argmax θ
∑
i, j
tij log pLM(rij) . (4)
where θ parameterizes pLM. In practice, we do not train pLM to convergence, but instead for a fixed number of iterations.

heading: 3.5 Sources of Seed Data
text: Depending on how seed documents S are obtained, DCT-based fine-tuning may be used to improve models in several ways:
• Unsupervised fine-tuning for coherence: in this case, we sample the initial seed set from pLM itself, e.g. simply by prompting it to generate a set of documents on a topic of interest.
4For fact-verification tasks, it is possible to derive positive supervision from statements marked as false: if the consistency evaluation step infers that Meiji was the last Japanese emperor is incorrect, then we may generate a correct example of the form Verify the following statement: Meiji was the last Japanese emperor. False. We use this strategy for our experiments on fact verification.
• (Semi-)supervised alignment with a trusted source: in this case, the seed set comes from an external source of supervised data. If this data is known to be reliable, we fix each seed datum’s truth value ti = 1 during the evaluation step. This may be combined with the unsupervised procedure.
• Model updating, editing and continual learning: in this case, as with supervised updating, we treat descriptions of desired edits as seed documents, fix these truth values for these seeds to 1, and fine-tune both on these documents and all their implications only.
Note that in the latter two cases (where we fix the truth value of seed documents to 1), the evaluation step is greatly simplified, and simply discards all generated documents that are not logically consistent with the seed. In the case of unsupervised learning, this evaluation step can (and empirically does) cause LMs to re-label sampled seed documents as well as conditionally generated ones.
Generalizations of DCT We remark that the procedure described above is the basic implementation of a family of DCT-like approaches, within which many more sophisticated procedures are possible—for example: probabilistic DCT (computing marginal statement probabilities rather than hard truth assignments), contrastive DCT (replacing Eq. (4) with an objective that encourages true statements to be assigned higher probability than false ones), and multi-hop DCT (generating not just direct implications of documents, but a wider graph of related ones).

heading: 4 Formal Analysis of DCT
text: At first glance, it may seem surprising that this procedure (especially in its unsupervised form) can improve LM accuracy using only LM-generated text. In this section, we describe a set of assumptions under which DCT is guaranteed to improve accuracy on certain inputs. We focus this analysis on generation and evaluation of (question, answer) pairs, but it could be extended to the other tasks considered in this paper as well.
Informally, suppose:
1. Questions generated by the LM with high probability are likely to be correct. (Intuitively, high-probability questions will be ones that occurred frequently in the training set,
and are therefore more likely to be answered correctly; McCoy et al., 2023, though c.f. Lin et al., 2021.)
2. Given a question, prompting an LM with a related, correct question–answer pair increases the probability of a correct answer. (Intuitively, such prompts may steer models generally in the direction of truthfulness, as in Lin et al., 2021, and can provide concrete evidence useful for answering the new question.)
We wish to show that if these two conditions hold, DCT improves model performance.
For simplicity, we consider a minimal version of unsupervised DCT in which a single implication is generated from each seed statement, the check in Eq. (3) is not performed, and the LM is trained to convergence on data generated from an arbitrarily large number of seeds. Let q be some specific question of interest, let pLM(a∗ | q) denote the probability that pLM assigns the correct answer to q (before applying DCT), and let pDCT(a∗ | q) be the probability that the LM assigns after DCT. Let (q0, a0) denote a (question, answer) pair generated as a seed document, and a∗0 specifically the correct answer to q0. Finally, for convenience, define p(q0 | q) = pLM(q|q0) pLM(q0)∑
q′0 pLM(q|q′0) pLM(q′0)
(this is the
probability that the seed question was q0 given that the sampled question was q), and p(a0 | q, q0) via Bayes’ rule analogously.
Proposition 1. Suppose for some q that:
1. p(a∗0 | q, q0) ≥ p∗. (Conditioned on generating q during the document generation step of DCT, the probability that the generated answer to any seed question q0 contains a correct answer is (uniformly) at least p∗.)
2. Eq0|q pLM(a ∗ | q, q0, a∗0) ≥ pLM(a∗ | q) / p∗.
(In expectation, conditioning on a correct (q0, a0) pair increases the probability of generating a correct answer by at least 1/p∗.)
Then,
pDCT(a ∗ | q) > pLM(a∗ | q) . (5)
In other words, for any question q satisfying the two conditions above, unsupervised DCT increases the probability that pLM answers q correctly.
Proof is given in Appendix E.

heading: 5 Experiments
text: We evaluate Deductive Closure Training on a set of benchmark tasks measuring fact verification, question answering with new information, and a diagnostic model editing dataset. We use Llama-2-7B in all experiments. Additional qualitative results are provided in Appendix C.

heading: 5.1 Fact Verification
text: Task and training details We first evaluate whether DCT improves the models’ ability to classify factual claims. Our experiments use CREAK (Onoe et al., 2021), a dataset of claims about entities. We investigate four different learning settings: unsupervised, supervised, semi-supervised, and transductive, each using a different procedure for sampling seed documents. We report results on the CREAK development set. During DCT fine-tuning, we use a linear learning rate schedule until the training loss converges—this corresponds around 30 epochs for the majority of experiments unless otherwise indicated (see Appendix A for further details on experimental settings).
Evaluation and baselines Models are scored based on the fraction of claims they correctly label as true or false. For each condition, we compare to a state-of-the-art baseline. For unsupervised DCT, the baseline is an ordinary few-shot prompt. For supervised DCT, the baseline fine-tunes the LM on the provided true statements. For transductive DCT, we also compare to an inference-time baseline Graph-Inference similar to those described by Mitchell et al., 2022b and Kassner et al., 2023, which generates implications and contradictions for
each test example, performs reasoning as in Eq. (3), then directly outputs the inferred truth value for the example (with no fine-tuning). Unlike past work, we use the base LM to generate these graphs rather than a specialized pre-trained implication generation model. All results are presented in Table 1.
Results: Unsupervised DCT To generate seed documents, we query pLM 10 times, each time prompting the model to generate 10 diverse claims and sampling with a temperature of 0.9. We filter out the duplicate claims before continuing to sample implications and contradictions. The full method substantially outperforms a few-shot prompting baseline, and may outperform ablated versions of DCT that fine-tune only on seed statements assigned a high prior probability (labeled “seed only” in Table 2) or that do not perform the logical inference step described in Section 3.3 (labeled “− Consistency Eval”).
For these unsupervised experiments, we perform an additional evaluation specifically aimed at measuring logical coherence as well as factual accuracy. Here we use the contrast set in CREAK, which comprises 250 pairs of lexically similar examples with opposite truth values (e.g. Zendaya was raised in the US and Zendaya was raised in Scotland). In addition to accuracy, we compute the fraction of pairs that are labeled Both True (indicating incoherence) and Both Correct.
Here, DCT not only improves correctness but also reduces the number of incoherent predictions, decreasing the probability that pLM judges two contradictory statements to both be correct.
Results: Supervised & Semi-supervised DCT In the supervised case (Table 1), we utilize a small set of externally provided claims and associated ground-truth labels to initialize DCT seed nodes. We sample 20 claims from the CREAK training set and filter those labeled as true to use as our seed documents D. For semi-supervised learning, we pool together data generated following the unsupervised and supervised settings for fine-tuning.
All variants of DCT improve over an ordinary fine-tuning baseline; interestingly, examples generated supervisedly and self-supervisedly are complementary, such that semi-supervised learning improves over both results.
Results: Transductive DCT The previous evaluations assumed a strict train / test split. Here we study the behavior of DCT in a “transductive” setting (Gammerman et al., 1998) in which we have access to unlabeled claims from the evaluation set while updating the model. For each of the 1,371 claims in the validation set, we generate seed text by prompting the LM to generate a set of related claims, which are then used to generate additional implications and contradictions. In addition to the inference-time baseline described above, these experiments compare to an ablated version of DCT that trains only on the generated related claims.
As in other experiments, DCT outperforms the inference-time reasoning baseline as well as the related-text-only ablation.

heading: 5.2 Model Updating and Question Answering
text: Task and training details Language models often hallucinate wrong information and rapidly become out-of-date after initial training. As a consequence, there has been increased interest in specialized continual learning (or “model editing”) procedures for updating LMs with new information without full re-training. A key desideratum is
LMs should not simply assign high probability to the new fact, but all of its consequences: if we wish to update an LM encode the fact that the current U.K. prime minister is not Boris Johnson but Rishi Sunak, the LM should also produce text consistent with the fact that the current P.M.’s wife is not Carrie Johnson but Akshata Murthy. Past work has found that fine-tuning on edits, as well as many specialized editing procedures, fail to propagate such information.
Our experiments on this task use the counterfactual subset from MQUAKE (Zhong et al., 2023) dataset, which evaluates models on their ability to answer questions about new information not provided in their training sets. To apply DCT, we take as seed documents the text of the new information to be inserted into the model. During the generation phase, models are prompted to combine this information with other background knowledge related to the same topic (see Appendix D for prompting details), producing what we term Correlative Implications. Finally, because MQUAKE is a question answering dataset, we convert each generated statement into a question–answer pair using the LM, then fine-tune it on these pairs.
Evaluation and baselines We compare DCT to ordinary fine-tuning on new information and three state-of-the-art baseline approaches for model updating: a context distillation baseline by Padmanabhan et al. (2023), which fine-tunes LMs to behave out-of-context the same way they would with prompts containing the new information (see Appendix A for implementation details), a weight editing baseline by (Meng et al., 2023), and the retrieval baseline MeLLo (Zhong et al., 2023), which stores new text in an external memory. We evaluate the behavior of DCT and these baselines in settings where varying numbers of new pieces of information (between 10 and 1000) are provided, and report the model’s accuracy at question answering.
Results As shown in Table 3, DCT significantly outperforms fine-tuning, fine-tuning on continuations, weight editing, and MeLLo (the previous state-of-the-art on MQUAKE). Using correlative implications systematically improves over simple implications. Combining the two sets improves on average over using either in all settings. Our qualitative analysis in Appendix C reveals that correlative implications contain about 50% more new information than standard implications.

heading: 5.3 Sanity Checks for LM Consistency
text: Task and training details In addition to naturalistic question asking tasks like MQUAKE, there has been recent interest in developing precise tests of LMs’ ability to capture simple logical implications of new facts (e.g. assigning high probability to sentences of the form B is A after training on A is B). We investigate whether DCT can address these issues using the “Reversal Curse” benchmark (Berglund et al., 2023). We report results on two evaluations: first, a set of celebrity parent– child pairs with training examples like Jennifer Lawrence’s mother is Karen Lawrence and test examples Who is the child of Karen Lawrence?; second, a set of entity–description pairs with training examples like Olaf Scholz was the ninth Chancellor of Germany and cloze-style test examples The ninth Chancellor of Germany is .
Evaluation and baselines For these experiments, we compare to the fine-tuning baseline used in the original work of Berglund et al. (2023) as well as the fine-tuning on continuations approach by Padmanabhan et al. (2023). We use training examples as seed statements, and generate implications using the same prompt as CREAK experiments in 5.1. While we expect that a DCT-type approach specifically tailored for this benchmark could trivially re-generate all the test examples, our experiments in this section aim to evaluate whether a generalpurpose prompt can improve performance on a specific class of generalizations. Following Berglund et al. (2023), we report exact-match accuracy after removing punctuation and lower-casing. In this dataset, LMs are evaluated on a mix of questions
and cloze completion tasks featuring both training statements and their reversed forms.
Results Results are shown in Table 4. DCT improves accuracy on reversed statements without significantly hurting performance on original questions. Notably, however, DCT with this generalpurpose prompt does not completely solve this dataset, and we leave for future work the question of whether more extensive sampling or other procedures could further improve these results.

heading: 6 Conclusion
text: We have described Deductive Closure Training (DCT), a supervision procedure that optimizes models toward deductive closure—encouraging them to assign high probability to a logically co-
herent set of factual assertions.By doing so, DCT also improves the truthfulness and updatability of models, substantially increasing accuracy on a variety of fact verification and editing datasets in both supervised and unsupervised conditions. More generally, these results show that some factual errors in LMs stem not from limitations of their training data, but limitations of training algorithms. By using LMs themselves to reason about relationships between (and implications of) their predictions, they can be made more accurate with little or no additional supervision.
Limitations
While Deductive Closure Training (DCT) could in principle be applied to arbitrary graphs of relations between statements, here we have applied it only to a single layer of implications of seed data. All datasets used for evaluation involve English text, and it is possible that DCT behaves differently in different languages. Even within English, it is possible that exhibits systematic biases or differences in accuracy for certain types of factual content. While DCT can improve overall factuality, it may inadvertently perpetuate hallucinations within certain domains that could escape detection during our evaluations.
Ethical Considerations
While our experiments have focused on using DCT as a tool for bringing LMs into alignment with reliable sources, these techniques could also be used to optimize LMs toward generation of (logically consistent) false facts, increasing their effectiveness as tools for generation of misinformation.

heading: Acknowledgments
text: This work was supported partly by the National Science foundation under grant IIS-2238240, DARPA HR001118S0044 (the LwLL program), the Shared Computing Cluster administered by Boston University’s Research Computing Services, as well as a hardware donation from NVIDIA to MIT. Any opinions, findings, conclusions, or recommendations expressed here are those of the authors and do not necessarily reflect the view of the sponsor.

heading: A Experimental Details
text: We use the Llama-2-7B-hf checkpoint provided by HuggingFace Transformers library for all of our experiments. Code to reproduce the experiments will be made publicly available. While developing the codebase, the authors used GitHub Copilot via Visual Studio Code.
Generation We sample at temperature 0.6 and top-p 0.9 for all samples except for the set of seed documents for the unsupervised experiment in Table 1 where we used temperature 0.9 to obtain a diverse set of initial documents.
Training For fine-tuning we use the LoRA implemention via the PEFT library (Hu et al., 2022; Mangrulkar et al., 2022) and set rank to 8, alpha to 32 and dropout to 0.1. In the absence of a held-out development set, we set the learning rate to 0.0001 throughout, batch size to 4 and train for 30 epochs by default. We find that training loss typically converges after 30 epochs with the exception of the supervised experiments in Table 1 for which we train for 60 epochs. The transductive setting for CREAK results in substantially more training documents, hence we train only for 1 epoch. We use a linear learning rate scheduler with 100 warm up steps and AdamW optimizer. For fact verification training, we use weighted sampling as the class distribution is sometimes unbalanced.
Editing experiments We use the MQUAKE-CF subset from Zhong et al. (2023) and evaluate only on the multi-hop questions. Padmanabhan et al. (2023) proposes two techniques to introduce model updates based on fine-tuning: simple fine-tuning on continuations conditioned on the edit statement (which we call FT on Continuations) and context distillation on continuations. We find the former approach–fine-tuning the model on the continuations when the model is conditioned on the edit sequence–to perform better on MQUAKE than the latter. Hyperparameters used for MEMIT are available in Appendix A. For validation we use a set of held-out 50 edits.

heading: B Details for DCT
text: In Table 6, we consider a small graph consisting of one seed node (ri), one implication (ri1) and one contradiction (ri2). In the beginning, there are 8 candidate truth value assignment yet not all assignments are consistent within e.g. If ri if true,
then ri1 must be true and ri2 must be false. When computing the most probable assignment in Eq. (3), we only consider consistent assignments.

heading: C Qualitative Analysis
text: To better understand how DCT improves LM performance, we manually annotated about 350 generations from various experiments to assess whether (1) double-checking improves the precision of generated implications and contradictions; (2) whether DCT incorporates model internal knowledge when making new conclusions; and (3) whether generated text includes non-trivial new inferences.
Double-checking We evaluated whether the double-checking following DCT (Imp. + Cont.) improves precision. In the supervised setting for CREAK, we annotated 100 implications and contradictions generated using DCT (Imp. + Cont.). We found that 74 of these are valid. The doublechecking procedure removes about 2/3 of generations, resulting in 33. Among these, 27 are valid, raising the ratio of correct statements predicted by
the model from 76% to 82%.
Incorporating previous information The MQUAKE subset used in our experiments comprises difficult multi-hop questions. Hence, generations that incorporate existing information about the entities mentioned in the edit are especially useful. We compare the set of implications generated using the DCT (Imp.) and DCT (Corr. Imp.). Respectively, only 30% and 36% of generations involve strict logical implications; however, 78% and 69% were judged to be plausible given the edit. Furthermore, 24% and 33% of the generations incorporate new information supplied by the LM. For example, given an edit Chauncey Billups is associated with the sport of pesäpallo, the LM uses background knowledge Pesäpallo is popular in Finland to generate Chauncey Billups was born in Finland.
Novelty of inferences Lastly, we find that most implications made by the model on the “Reversal Curse” dataset are paraphrases or are trivial (Jennifer Lawrence’s mother is Karen Lawrence → Jennifer Lawrence has a mother) but some add world knowledge to the implication (Sadie Frost’s mother is Mary Davidson → Mary Davidson is the mother of a British actress, where the LM itself has supplied the knowledge about Sadie Frost). While generating implications, DCT often (but not always) generates test-set-like reversed implications on its own: the model reverses 22% of the statements of the form X’s parent is Y, 43% of statements of the form the person with property X is Y, but only 6% of statements of the form Person X has property Y. These findings suggest a strong bias toward generating text that starts with the person as opposed to the description. In general, most generated extensions are fluent, different from the source, and sometimes contain new information.

heading: D Prompt Templates
text: We use a set of fixed prompts to generate our graphs, calculate model-estimated probability for the correctness of a given statement, generating a set of seed documents and automatically converting statements into questions which are available in Tables 7 to 10.

heading: E Proof of Proposition 1
text: At optimality, pDCT(a∗ | q) (the probability that the updated LM assigns to the correct answer) will be the probability of a∗ given q marginally over all generated seed documents:
pDCT(a ∗ | q) =
∑
q0,a0
pLM(a ∗ | q, q0, a0) p(a0 | q0, q) p(q0 | q) .
We may decompose this according to whether the generated seed pair is itself correct:
= ∑
q0
p(q0 | q) [ pLM(a ∗ | q, q0, a∗0) p(a∗0 | q, q0)
+ ∑
a′0 ̸=a∗0
pLM(a ∗ | q, q0, a′0) p(a′0 | q0, q) p(q0 | q)
]
(where a∗0 denotes the correct answer to q0)
≥ ∑
q0
p(q0 | q) pLM(a∗ | q, q0, a∗0) p(a∗0 | q, q0) .
By assumption 1:
≥ ∑
q0
p(q0 | q)pLM(a∗ | q, q0, a∗0) p∗
= p∗Eq0|q pLM(a ∗ | q, q0, a∗0) .
By assumption 2:
≥ pLM(a∗ | q) .

heading: Future Work
text: Notably, however, DCT with this generalpurpose prompt does not completely solve this dataset, and we leave for future work the question of whether more extensive sampling or other procedures could further improve these results