From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question-Answering
Authors: Nathaniel Weir
Bhavana Dalvi Mishra
Orion Weller
Oyvind Tafjord
Sam Hornstein
Alexander Sabol
Peter Jansen
Benjamin Van Durme
Peter Clark
Venue: ICLR 2025 Poster (2025)
Keywords: microtheory
textual entailment
knowledge representation
natural language reasoning
text retrieval
automatic knowledge base construction
Primary area: interpretability and explainable AI
TL;DR: We introduce a question-driven algorithm to generate a list of LM-believed knowledge statements most relevant to answering questions in a dataset.

Abstract: Recent reasoning methods (e.g., chain-of-thought) help users understand how language models (LMs) answer a single question, but they do little to reveal the LM’s overall understanding, or “theory,” about the question’s topic, making it still hard to trust the model. Our goal is to materialize such theories - here called microtheories (a linguistic analog of logical microtheories) - as a set of sentences encapsulating an LM’s core knowledge about a topic. These statements systematically work together to entail answers to a set of questions to both engender trust and improve performance. Our approach is to first populate a knowledge store with (model-generated) sentences that entail answers to training questions, and then distill those down to a core microtheory which is concise, general, and non-redundant. We show that, when added to a general corpus (e.g., Wikipedia), microtheories can supply critical information not necessarily present in the corpus, improving both a model’s ability to ground its answers to verifiable knowledge (i.e., show how answers are systematically entailed by documents in the corpus, grounding up to +8% more answers), and the accuracy of those grounded answers (up to +8% absolute). We also show that, in a human evaluation in the medical domain, our distilled microtheories contain a significantly higher concentration of topically critical facts than the non-distilled knowledge store. Finally, we show we can quantify the coverage of a microtheory for a topic (characterized by a dataset) using a notion of p-relevance. Together, these suggest that microtheories are an efficient distillation of an LM’s topic-relevant knowledge, that they can usefully augment existing corpora, and can provide both performance gains and an interpretable, verifiable window into the model’s knowledge of a topic.

Published as a conference paper at ICLR 2025
FROM MODELS TO MICROTHEORIES: DISTILLING A MODEL'S TOPICAL KNOWLEDGE FOR GROUNDED QUESTION ANSWERING
Nathaniel Weir'; Bhavana Dalvi Mishra Orion Weller,, Oyvind Tafjord" Samuel Hornstein # Alexander Sabol Peter Jansen Benjamin Van Durme', Peter Clark
"Johns Hopkins University Allen Institute for AI
"Thomas Jefferson University 'University of Arizona
nweirê jhu. edu, petercaallena.. org
ABSTRACT
Recent reasoning methods (e.g., chain-of-thouue -f-houhh- help users understand how lan- guage models (LMs) answer a single question, but they do little to reveal the LM's overall understanding, "theory," about the question's 's topic, making it still hard to trust the model. Our goal is to materialize such theories here called microthe- ories (a linguistic analog of logical microtheories (Blair et al. 1992)) sentences encapsulating an LM's core knowledge about a topic. These statements systematically work together to entail answers a set of questions both engen- der trust and improve performance. Our approach is to first populate knowledge store with (model-generated) sentences that entail answers training questions. and then distill those down to core microtheory which is concise, general, and non-redundant. We show that, when added to a general corpus (e-g., Wikipedia), microtheories can supply critical information not necessarily present in the corpus, improving both a model's ability to ground its answers to verifiable knowledge (i.e., show how answers are systematically entailed by documents in the corpus, grounding up to +8% more answers), and the accuracy of those grounded answers (up +8% absolute). We also show that, in human evaluation in the medical domain, our distilled microtheories contain a significantly higher concentration topically critical facts than the non-distilled knowledge store. Finally, we show we can quantify the coverage of a microtheory for a topic (characterized by dataset) using a notion of p-relevance. Together, these suggest that microtheories are an efficient distillation of an LM's topic-relevant knowledge, that they can usefully augment existing corpora, and can provide both performance gains and an inter- pretable, verifiable window into the model's 's knowledge of topic
of
INTRODUCTION
What do language models (LMs) know about the topics they converse about? Despite their success, LMs are opaque users do not have good visibility into the LM's knowledge, creating challenges for engendering trust in the model. Recent work has tried to address this for answers to a single question by having the LM provide various explanations and chains of thought that support answer, e.g., (Danilevsky et al. 2020 Wei et al. 2022). However, such expositions only reveal snippets of the LM''s latent knowledge and do little convey the model's overall understanding, "theory", about the topic at hand, limiting a user's confidence in the model's behavior. Conversely, if we had method that could distill a model's overall theories into inspectable, verifiable form, this could help users see how answers follow from those theories.
Our goal is distill such theories which of logical microtheories (Blair et al., 1992)) as a set of sentences encapsulating an LM's core knowledge about topic. Our approach uses a question-driven methodology, where set of questions characterizes a topic (i.c., a topic is defined as a distribution over questions). We first prompt the
refer as microtheories (Mts) (a linguistic analog
Work done in part during internship at Allen Institute for AI Code for this work can be found at withttps:/// :// // wwwgithubb... com wwwp://i/////oeeeeoeeoo/ee/eeee r/ 27/mi crotheories/

---

Published as a conference paper at ICLR 2025
Test Hypotheses
hg: The cart will accelorate at speed of 125 més unil the student stops apolying the force
ha: The object drops slower than it would Earth
hg The planet with more muss experiences stronger gravitational pul
the slose distance than truck
Qs A sludent pushes a 12kg shopping cart with force of 16 nawlons (N)
Q3i An astronaut on the international Space Sistion drops oiject
Qei planots ortil star same
Q2: A car and a truck traveling the same speed approach stop sign
Newlon's 2nd law motion states that the equals mass Its accelcration
The gravity between two objects increases with the mass of the objects and decreases with the distance between the objects
Training Questions
Microtheory
Given set of topical training questions, we construct a microtheory, a set of statements articulating a model's core, reusable knowledge about that topic. These help prove (entail) answers to test questions.
Figure 1:
LM enumerate facts that entail answers to training questions. then identify the core, abstractable pieces of knowledge frequently reused for many questions in that set, then distill them down to core microtheory that concise, general, and non-redundant. The microtheory is intended to contain the core principles about the topic (e.g., Newton's laws of motion, for physics), extracted from the LM, that can help derive corpus-grounded answers to topical questions.
To answer questions with the help of microtheory, we use the well-known process of textual entail- ment (Dagan et al., 2005 Lai al. 2017). Textual entailment is a linguistic inference process that identifies when one (or more) statement(s) "reasonably implies" another (Magnini (2015), producing a (potentially multistep) entailment tree, analogous to a proof tree (Dalvi al. [021). While entail- ment lacks fully formal definition (modern entailment engines are instead trained on examples), it provides a well-defined mechanism for deciding if, and how, an answer systematically follows from set of statements. This allows us both use and evaluate microtheories for question-answeriig
We evaluate our microtheory approach in two domains using two existing datasets: K-12 science, using the ARC dataset (Clark et al.) 2018), and medicine, using the MedQA dataset (Jin et al.) 2021] We cluster each dataset into topics, construct microtheories on a training split of each topic, and test on the remainder. We find that when added to general corpus (here, Wikipedia), microtheories can supply critical, topical information not necessarily present in the corpus, improving both the model's ability ground its answers (i.e., show how those answers fully grounding up to +8% more answers), and the accuracy of those grounded answers (up to +8% absolute). We also show that, in a human evaluation the medical domain, our distilled microtheories contain a significantly higher concentration of topically critical facts than in the non- distilled knowledge store. Finally, we propose metric, p-relevance, quantify the completeness of a microtheory. By correlating p-relevance with training set size, we can predict how many training questions will be needed generate a Mt relevant to some p percent of test questions. In summary, our contributions are:
entailed by the corpus,
A method for distilling microtheories from a model, given QA dataset, that articulates the model's 's core knowledge about the dataset topic. This is the first method to try to distill model's overall understanding
topic in this way.
Evaluations that show microtheories can improve both model's ability ground its answers to verifiable facts in a corpus, and the accuracy of those answers.
A human evaluation in the medical domain, showing our method distills out topically critical facts from the general pool.
new metric, p-relevance to quantify the completeness of a microtheory for a dataset. Together, these suggest that microtheries are an efficient distillation of an LM's topic-relevant knowl- edge, that they can usefully augment existing corpora, and can provide both performance gains and an interpretable, verifiable window into the model's knowledge of a topic.
RELATED WORK
The notion of a microtheory small, coherent, topical representation of knowledge arose in formal logic to help organize large, logical KB into useful, topical subsets, each sharing the same underlying assumptions (context) (Lenat et al. 1990 Blair et al. 1992). A logical microtheory
2

---

the well-known Published as a conference paper at ICLR 2025
aimed to model a specific aspect of reality as self-contained, logical system, akin to a piece of "computational clockwork" (Clark et al. (004).. Our work here can be seen as a modern approach to this, similarly aiming to capture an understanding of topic but where the units of knowledge are in NI and reasoning is via textual entailment.
It has long been recognized that LMs contain substantial knowledge, and can even be viewed as new kind of "knowledge base" (Petroni et al., 2019, Bosselut et al. 2019). Some work has sought to materialize some of the model's 's factual knowledge either as knowledge graph triples (Alivanistos eet al. 2022 Hao et al.) 2022). or NI statements, e.g., BeliefBank (Kassner et al. (021). Other work has generated "belief graphs" showing how model-believed facts entail answers (Kassner et al.) 2023 Tafjord et al. 2022 Jung et al. 2022). However, while providing some visibility, these meth- ods only provide disconnected slices of the model's latent knowledge, at best related to a single question. Our work takes this further, aiming to extract a model's overall understanding of a topic.
One notable manual attempt topical, linguistic microtheory is WorldTree, corpus of several thousand statements (aiming to) cover all the knowledge required for a syllabus in elementary sci- ence (Xie et al., (020), constructed at significant expense. Our goal with microtheories can be seen as automating such construction process, and extracting such topical knowledge from language models directly (we later compare our Mts against WorldTree in $5.1 and $5.2).
More broadly, there has been substantial work in generating explanations with language models, to help users see the rationale behind model's answers (Wiegreffe & Marasovic 2021 Danilevsky
iet al.) 2020: Wei et al., 2022). b. However, much of this work is again question-centric, and sometimes lacks clear definition of whether an explanation adequately "supports" an answer. To more clearly define the relationship between supporting evidence and an answer, and for linguistic reasoning in general, there has been a resurgence of interest in textual entailment as a linguistic inference pro- cess, with numerous, high-quality multi-step entailment engines becoming available, e.g., SCSearch (Bostrom et al. 2022), NLProoffS (Yang et al. 2022), Entailer (Tafjord et al. 2022), IRGR (Ribeiro) et al.] 2022), Tree Wise (Weir et 2024), and MetGen (Hong et al.) 2002). In our work, we leverage this development to clearly define whether a microtheory supports an answer or not.
Finally, our work is distinct from work on library learning, which seeks to induce generalizations over data, e.g., (Muggleton) 1991 Wang et al. (002)). Rather, our goal here is to articulate general- izations that the LM appears to have already formed.
APPROACH
3.1 BACKGROUND: TEXTUAL ENTAILMENT
To make meaningful claims about microtheories, we need a clear definition of what it means for an answer to follow from microtheory (or a corpus in general). For this, notion of textual entailment in NLP: linguistic inference process that identifies whether one or more statements "reasonably imply" another (Dagan et al.) 2005). Modern entailment engines perform soft version of theorem proving by searching for entailment trees, analogous to proof trees, showing how a query hypothesis H follows from statements in a theory T, where both elements are stated in NL. Here, H is sentence (commonly a declarative answer to question Q). T = [t,..... ii.....,, f.]] is a corpus of text passages, e.g. sentences or documents, and an entailment tree is tree whose root H, whose leaves L = [/,,..., 44] are subset of texts in T, and whose individual steps have the form [p]....p;] FQ hw; j E {1...... : [T) - 1). where tQ denotes atomic textual entailment given the context of question Q (+= is typically learned from examples). Multiple alternative trees T; may be found supporting the same hypothesis H, analogous to finding multiple proofs for a hypothesis in formal logic. For large Ts. the search space can be very big. To handle this, entailment engines such as IRGR (Ribeiro et al. 2022) or TREEWISE (Weir et al., 2024) use iterative text retrieval to look up relevant statements and use them for grounding and decomposing hypotheses.
tree T; shows how H is entailed from a set of corpus texts L,, we say is grounded in the corpus. We also say the leaf statements L; provide argumentative basis for accepting H. If there are multiple trees with different leafsets, we say that L,, L2.... constitute alternative bases for supporting H.
Definitions:
3.2 CONSTRUCTING MICROTHEORIES
We now describe a method to automatically construct a microtheory, namely a list of model- generated NL assertions that are "most important" for entailing answers to questions about given topic, characterized by a set of training questions. Microtheories are intended to capture the core knowledge about the topic, and we evaluate different operationnalization of "most important"
3

---

Published as a conference paper at ICLR 2025
shortly. First, for given training set (Q1....Qu .al, extract from LM, via prompting, sets of facts that it would use to support the correct answer to each question ($3.2.)). Then, to reduce redun- dancies and promote generalizability, we apply series of optimizations to distill the core kernels of knowledge by identfying entailment relations ( (3.2.2). Finally, we propose (and later evaluate) three alternative ways of optimizing the microtheory contents given particular size budget ($3.2.3)
3.2.1 RAW FACT POOL EXTRACTION
We first prompt an LM to regurgitate facts on a question-by-questinn basis in order to populate an initial factpool F. We thhinnof-thought' reasoning (Wei et al.] 2022) catered towards extracting facts. Each example in the prompt contains question Qe. answer options to Qe the form of hypotheses h1.... hn, list of supporting statements for Qe (a question-specífic "theory"), and a list of entailment steps that compose subset of statements into complex inferences that answer the question by supporting the correct h. Prompting details for this GenFacts(() method can be found in $C As training exam- ples, we use EntailmentBank (Dalvi et al., 2021), a dataset of science questions paired with lists of supporting facts that combine into an entai ment tree for the correct answers. We define F as:
few-shot learning prompt that cues the LM to perform version of
V (eepp(GeneericcGeGaatttt////// [ta........................... (.]))
Because the LLM often generates context-specific sentences via GenFacts() that we wouldn't 't want to keep in our microtheory, e.g. about specific entities in the question (e.g. "the man is running"), "), we create a filter (KeeplfGGnnrric) in (1)) that prompts the LLM to classify each sentence as generic or context-specific and keeps only the generics.
3.2.2 SBERT AND ENTAILMENT-BASED CONDENSATION
As fact lists are extracted independently for each question. there is substantial redundancy within the unified pool. We use pair of techniques to remove statements and produce condensed pool C C F: Soft Deduplication. We identify paraphrases using Sentence Transformer (SBERT; Reimers & Gurevych] 2019) trained to maximize the embedding cosine similarity between same-neaning facts. We perform a linear pass of F and remove every fact f. for which 3f, E F.j i, (f)ERT(())))) SBERT(f,)) > for some threshold t.
Entailment Condensation. We perform a second pass through F find pairs fú, s.t. fi | fz. and remove f,. We use a fine-tuned neural cross-encoder to predict }. As it would be infeasible to check all 2 pairs of facts, we leverage SBERT to find the most promising candidates. We check for entailment between any two facts J1, Sj iff cos (SBERT((()))) (B)())())))))))) > u, 1
3.2.3 ENGINE USAGE-BASED OPTIMIZATION
After removing duplicates and entailments, we still see facts that serve the same functional roles in many questions but imply slightly different things, e.g. for physics:
(A) The force required to cause a given acceleration determined by the mass of an object
(B) The force acting on an object equals mass times acceleration
While this might be acceptable for a knowledge store with infinite storage, we instead seek more concise representation without redundancies. Towards this goal, we consider three alternative ap- proaches, illustrated in Figure 2] to identify some most effective statements to retain in the Mt. We will refer to the resulting sets as n-Mts Under this budget, we would not want to retain both (A) and (B) in the n-Mt if it meant discarding some (C) that is important to other questions.
Top-n Most Used Facts (Mtnsage). We seek the set of facts that best "covers" the argumentative bases for given dataset of hypotheses. One intuitive way to choose these facts is to feed them to an entailment engine and see which ones are used most frequently to explain hypotheses in the dataset. We refer this approach as the usage ntMt. Using questions Q Q1...Q. Qa and hypotheses hy ha:
Minsage
arg max
ΣΣ 8(f. LLAAEES(ENGINE(H,CCC..))
MCC.M HEM
A We use a - -aa---ppee-aaa-e2 for for L;, and sct f = .9 and 2 = .3. is hyperparameter; below, we observe that n represents trade-off hetween concision and performance.
4

---

Published as a conference paper at ICLR 2025
by Waste will au mounten.
Ay: It takee leee hear energy alfiuctee
by Glanes hot day
ke Wetter moles les in the ecample condensation
T22
L21
LAL1
LAS
LaL
LAB
Fi Viter loNiL
Is pressure uixudo
Exacoration lurning into
J Bolling and exaporation locil waporizztoo.
fe Cendemaatoon evasonation.
Distitted Microthaories
1: holling decicases Iowerr APl presopse
ft incheaniing and Noperticion boiling
AOOD
WOOM
Ce
4Mllec
FOOD
Figure 2: Comparison of distillation techniques for extracting n-Mts from a larger fact pool C based on their entailments (Is of different training hypotheses (hs)). The usage approach prioritizes facts based on the number of hypotheses for which they are used. This risks keeping facts that serve the same role in explaining the same hs (e.g., Is and fī are both kept, even though only one is needed to explain lt2, lıg, and h4). The quextion coverage (QC) approach maximizes the number of hts for which the r-Mt contains all of supporting argument's leaves (here l12, les. and Ita). This risks failing to cover some at all (e.g. h(). Partial coverage (PC) maximizes the total fraction of the argument covered for all questions (preferring the most covered argument for each h,
there is more than onc). In this example, coverage = .66+1+1+.5 3.16 (of a possible 4).
where ENGINE returns (possibly empty) set of entailment trees proving h2 from fact pool C. and LEAVES refers to the set of all facts used at least once in any of those trees. This objective does nor address the functional redundancy issue for fact pairs such as (A) and (B). If (A) and (B) serve the same role in entailing hypotheses, then both facts will be used for a similar number of questions; if one is in the top-n most used facts, the other will likely also be.
2. Maximum Question Coverage (Mtgc). We use an objective function minimizing the facts in a theory while maximizing the number of train hypotheses "covered." For each question Qj with bases Lj- = Lıj, Izj.... (returned by LEAFSETS), we try to fully cover at least one Ltj for as many questions as possible using S n facts. We use a linear program described S
n-Mioc
arg Hax
6(3La5 E LEANSSETS(ENNINN((ssC::))))) S
(3)
MCC.IMI=
3. Maximum Partial Coverage (Mtrc). Equation (3) poses question coverage as a binary variable:
for each Qi, either an entire basis is kept, or Q: is not covered. We consider variant that loosens this constraint, modeling the partial coverage of questions maximizes the per-question partial basis coverage (I.P described in $A.2). Given each question Q: associated with bases Lis, L try to best cover some Li--
| We use an objective function that
MMN Lagl
1-MMPc arg max MCC.M LU (LEAPSESSEENNNEE,,CCC
arg max
[azl
(4)
4 EXPERIMENTS CONSTRUCTING MICROTHEORIES
We apply our method to construct microtheories for two domains: ARC (Clark al. 2018). com- prising grade-school science questions, and MedQA (Jin et al. 202T). containing USMLE medical school exam questions. Our goal is to extract generalizable knowledge statements using training questions that can be readily applied to solve test questions. This requires train/test splits that are about the same topics. To do this, we create mini-splits of each dataset containing topically similar questions with 6-900 training questions using a hierarchical topic clustering algorithm we describe in $B For ARC, we make a split with 9 topic clusters including "Genetics and Evolution". "Force, Mass, Acceleration and Gravity", and "Plant Growth and Reproduction." For MedQA, make a split with 4 clusters including "The Effects of Smoking on Health" and "Kidney Function and Disorders." (See Figure for details).
4.1 MICROTHERY CONSTRUCTION DETAILS
To generate the initial fact pools 1 we use GPT-4 (gpt 4- 06113). For entailment reasoning, we use TIEEWISE Weir et al. 2024). a recent, off- the-shelf, search- based SOTA entailment engine
*For 3 questions, it might he better cover 70%, 70%, & 70% of basis for each than 100%, 100%, & 0%.
5

---

Published as a conference paper at ICLR 2025
ARC
ARC
MedQA
5000
Dataset Details
# Facts
1000
100
Questions
854/127/249
641/94/186
10
# Topic Clusters
9
4
Ext. Corpus
Wikipedia
Wiki+Textbooks
MedQA
Training Extraction Results
5000
[F]
20,561
16,722
# Facts
1000
CC
8,722
10,908
100
Qs with Proof
744 (89%)
562 (88%)
10
Min #Fs to Cover
800
890
Fact/Q Ratio
1.08
1.58
10
15 20 25 30
35 40
Questions per Fact
Figure 3: (Upper) Dataset details for the two domains. (Lower) Results microtheory extraction from training data. The last two rows result from the "min-fact" LP.
Histogram of training question per fact in C and 650-850 training questions. ARC facts are used more frequently than MedQA.
Figure 4:
that contains fine-tuned models to verify individual entailment steps (we use its tuned ChatGPT). To demonstrate our work is not closed-model dependent, we show MedQA results using Mixtral- 8x22B-Instruct-vO.] (Jiang et al.) 2024) in S We construct n-Mts for n=25 1000 using the usage, question coverage and partial coverage methods described in $3.2 Details are found in S
Figure 3 displays displays statistics about the number of questions and size of the resulting fact pools created
during extraction. It shows that the redundancy reduction techniques in $3.2.2remoon around 55% of ARC facts and 25% % of MedQA. Figure shows histograms for how frequently each fact in C was used in the 650-850 questions. In both datasets, a substantial fraction of facts (5000/8000) are never used in proof. A few dozen facts from the ARC pool are used in more than 10 questions; fewer from the MedQA pool are used this frequently, suggesting that MedQA questions target more diverse pieces of knowledge than ARC, though handful of MedQA facts are used in 233 questions. To investigate this difference further we implemented a "min-fact" linear program that found the smallest set of facts that fully covers a basis for every hypothesis the prover was able to ground. Figure shows that the 744 ARC training questions provable from C required minimum of 800 facts for "full coverage," a ratio of 1.08 facts per question, while MedQA required nearly twice that rate. This highlights that the total amount of knowledge statements required to answer MedQA questions is substantially higher, which is unsurprising given the domains grade school- versus medical school-level topics. We refer readers to $ for illustrations of the fact selection processes by Mlpc and Migc, examples of microtheory fact lists, and further qualitative analysis.
4.2 COVERAGE OF TRAINING HYPOTHESES
How well do the basge optimization methods cover argumentative bases for the training hy- potheses? Figure 5) shows the fraction of full and partial coverage by each Mt approach 5 First, as expected, coverage increases with the size of the Mt. Second, we see that our third optimization method, partial coverage (PC), performs best here, with Mipc (orange bars) consistently obtaining a higher coverage than either Mige or Mtnsage (green, blue). Finally, we observe full coverage is generally lower on MedQA than ARC, suggesting that the MedQA domain contains fewer reusable principles and more idiosyneratic facts than ARC. We later explore this conjecture further $5.3
EVALUATION
Our goal is that microtheories contain the core, reusable principles that underlie a topic. Note that microtheory (or any theory, for that matter) cannot contain all the knowledge required to answer all topical questions, given that an infinite long-tail of idiosyneratic facts is also required in addition to core principles. For example, to determine that a dropped pencil (say) will fall to the ground requires not just general knowledge of gravity, but also the idiosyneratic fact that a pencil is a denser-than-air object. To supply such facts, we assess our microtheories when combined with a large, general corpus (Wikipedia and textbooks, see I5. treating each source as flat retrieval index over knowledge statements. We evaluate along three dimensions:
1. Grounding (551): For the correct answers to the test questions, do microtheories improve the number of answers that can be fully grounded the (combined) corpus? If so, this would suggest
'ee define fractional coverage of a question Q, as in $3...3 the highest fraction of leaves amongst all argumentative bases Tay found by the engine for the question.

---

Published as a conference paper at ICLR 2025
ARC Training Coverage Rate (%)
ARC Grounding Rate (%6 of Test Hypotheses
100
Pull Coverago
100
T2
100
500
72 1000
84
75
Mige
Partial Coverage
Mipc
75
50
25
0
Cp Rd Us QCPC Rd Us QCTC Rd Us QCPC è WT
MedQA Training Coverage Rate (%)
100
MedQA Grounding Rate (96 of Test Hypothuses
100
71=
100
N 1000
80
80
75
50
50
25
25
0
10
25
50
100
200 500 1000
Fact Budget
Op Rd Us QC PC Rd Us QC PC Rd Us QCPC F
Figure 5: Rate at which different microtheory ap- proaches entail training hypotheses. Lighter bars the of fractional coverages for questions not fully covered by the Mt. Total coverage for ARC questions is generally higher than MedQA. The 1000-Mtpes effectively fully cover the train- ing hypotheses for which basis was found in both domains; since some hypotheses did not have ba- sis found, 100% coverage is impossible.
Figure 6: % of test hypotheses fully groundable after adding various microtheories (n-sized Random, Usage, QC, & PC Mts) to a base Corpus. We show the fraction of leaves grounded in the Mt (solid) and the base cor- pus (striped). We see optimized Mts outperform the random baseline (Rd), and trends show the tradeoff of conciseness vs. performance. F provides up +8% grounding over using just the corpus and exceeds the benefit of adding the handcrafted WorldTree corpus.
that microtheories are supplying important, topical information not necessarily present in the origi- nal corpus. We also measure what proportion of those groundings are supplied by the microtheory vs. the original corpus, and the impact of Mt size.
Question Answering ( (552): Using the entailment engine to answer the questions (i.c., selecting the entailed answer option, or the option with the highest-scoring entailment tree if more than one), do microtheories improve QA accuracy? If so, this suggests Mts supply important information to help downstream task. We evaluate this in our primary (science) domain, ARC.
3. Relevance ($5.3): To complement our empirical evaluations, we also perform a rigorous, (hu- man) expert evaluation of microtheory statements to assess, from a human perspective, whether they appear to be critical, core domain principles (rather than idiosyneractic facts).
5.1 GROUNDING
We first evaluate, for the correct answers to test questions, whether microtheories improve the num- ber of answers that can be fully grounded by the entailment engine (i.e., can be "proved" via entail- ment) when added to base, general corpus. For our primary domain (ARC), we use Wikipedia as our base corpus, and for MedQA, we use a combination of Wikipedia and the set of medical text- books released by Jin et al. (2021). These corpora
vastly larger than the size of the microtheories. Baselines. We compare the n-Mt methods against a baseline that samples n random facts from F. We also compare against using the entire pool F. For ARC, we also compare against the WorldTree resource, the version of which we use has 12,657 facts. We also consider using only the WoeldTree facts that its annotators labeled as "core" for at least one question in their dataset (69% of facts).
Test Grounding Results. Figure shoows the hypothesis grounding rates when various microthe- ories are added to the respective base corpora (Wikipedia, MedQA textbooks). A hypothesis is considered "grounded"' if the engine found at least one entailment tree for it rooted fully in the Mt, corpus, and/or context. Each bar is broken down into the portions of trees grounded to the Mt (solid) vs. the base corpus (striped) In both domains, comparing the first bar (Cp, base corpus only) and last bar (F, using the full fact pool microtheory). we see grounding rates are substantially
'If the prover finds multiple trees, we take the one with the highest fraction of microtheory leaves.

---

Published as a conference paper at ICLR 2025
higher, suggesting that microtheories improve the the model's ability to ground its answers to verifiable knowledge. This capability is important, as it allows models to show how answers systematically follow from inspectable sources.
For ARC, the engine grounds up to 8% more hypotheses (92% vs. 84%), and uses facts from F at rate of 75% (solid part of the right-hand bar, i.e., 3/4 of the required entailment knowledge comes from the Mt). In MedQA, the hypotheses are grounded at a similar rate, but 45% of the knowledge comes from the Mt instead of the corpus. For ARC, we see clear trend that microtheory size (budget) increases, the engine can ground more answers (overall har heights), and ground to more of the Mt knowledge (solid bars). For MedQA, the pattern is weaker: while larger Mts provide more grounding knowledge (solid bars), there is no clear trend in overall grounding rates, suggesting that in this domain, there may be fewer core, general principles vs. large number of idiosyncratic facts. We examine this conjecture further in $5.3
For ARC, compare against notable, manual attempt at microtheory called WorldTree, corpus of several thousand statements (aiming to) cover all the knowledge required for the ARC syllabus (ie al. 2020), constructed at significant expense (Figure 6, bar labeled WT). We see that the n=1000 Mts result in similar rates of overall grounding and grounding to the Mt yet were built automatically and are less than 10% the size of WorldTree. Thus, our approach can be seen as
automating this previously expensive process of constructing such resources.
5.2 QUESTION ANSWERING
QA Performance (%) by Theory and Budget
Figure 7 shows the results of running
85
n== 100
n=500
n= 1000
multiple-choice question answering on the
80
9 ARC topics. For each question, the an-
swer option that the entailment engine can ground in ("prove from") the corpus is se- lected, or the option with the highest scor- ing entailment tree is selected if multiple options are entailed. Using only Wikipedia as the knowledge source (Corpus, leftmost bar) achieves 69.2% QA accuracy. Adding 100 500 facts with any approach gener- ally does not improve on this and can de- crease it using 100 random facts When adding 1000-fact theories, all distillation methods add 4 points in QA (73%). Adding the entire fact pool F further improves per- formance by points (78%). These results suggest the following:
75
70
65
60
Cp Rel
Us
QC PC
Rul Us QC PC Rd Us QC PCWT F
Figure 7: Question Answering performance on ARC topical test questions using Wikipedia plus various Mts (Random, Usage, QC, PC) as knowledge sources. F adds nearly 10% over using only the base Corpus of Wikipedia despite be- ing a tiny fraction of its size. This exceeds the benefits of the handwritten WorldTree corpus. (n=1000)-Mts improve QA by 4%. This suggests that they contain important task information that is either not in Wikipedia or hard to find.
Adding Mts to Wikipedia shows a benefit to QA, despite their being orders of magnitude smaller.
Mt size correlates with QA accuracy. For the ARC subset considered, one needs to retain at least 500 facts to a noticeable improvement in QA accuracy.
The question-driven Mt extraction method can provide performance gains on par with a clean, hand-annotated corpus (WorldTree)
When extracting core facts from F, the choice of optimization method doesn't have a noticeable effect on QA, though each method outperforms random selection.
5.3 RELEVANCE
Finally, to complement our empirical investigation, we perform two subjective assessments of the contents of the microtheories. the first using (expert) human experts and the second using an LLM- process is
as-judge. This provides an orthogonal evaluation of whether our indeed extracting the more important, central facts about the topic. We are particularly interested in whether, and by how much, our microtheory condensation and optimization methods ($3.2.3) are selecting important facts from the general fact pool F.
This decrease results from the irrelevant facts distracting the entailment search, hurting the grounding rate.
8

---

Published as a conference paper at ICLR 2025
5.3.1 EXPERT RELEVANCE ANNOTATIONS
First, we collect human judgments of the relevance of Mt facts. We use the MedQA medi- cal domain due to easy access to domain experts. We asked experts how useful they would find the Mt facts when studying for and taking USMLE tests. We recruited two senior med- ical students to annotate the general relevance of facts from different MedQA microtheories.
We worked with the annotators to develop a 5- point rubric for scoring how essential fact is to the content that appears the USMLE. Annotation details and the full rubric are in y
Average MedQA Fact Relevance by Budget
5.0
Miramboo
Minsus
Mdpc
4.5
Average relevance ratings are shown in Figure 8 Both optimization approaches outperform a ran- dom baseline by at least half grade point; Mtrc is 0.6-1.2 higher, and never drops below a score of 4. (The annotation rubric score defines score of 4 as containing relevant exam content, though either too general or unspecific sufficiently answer an exam question.) In contrast, a ran- domly selected fact from the full pool frequently scores below 3.5, often containing context or background information less directly useful for answering exam questions. This suggests that the Mt extraction methods successfully distill out the most topically relevant facts from the general knowledge pool, successfully selecting the information most likely to support exam performance.
4.0
3.5
3.0
10
50
100 Budget
200
500
1000
Figure 8: Average expert-annotated task relevance score for microtheory facts. PC facts are more relevant than random and usage until the 1000-fact budget.
5.3.2 AUTOMATIC PER- TER-QUESTION RELEVANCE
As well as evaluating the Mt as a whole, we also explore whether Mts are relevant on a per-question basis, to see how often the Mt has something useful to contribute when answering new question. We ran a study to assess this, using an L.M as judge. Although necessarily approximate using an LLM the scorer, it provides further insights into our Mts. First, we say Mt is relevant if it contains at least one fact that is relevant for getting the question right, i.e., differentiating the correct answer from any incorrect options. To determine if fact is relevant, we use an existing scoring rubric developed by Jansen et al. 2021) for rating fact's relevance to question on a 0-5 scale, and here ask opt-40-20240 08 06 to make that judgment (see SK] for the rubric and the full prompt). We then measure the rate at which at least one 5-rated fact is in the Mt This provides a soft indicator that the Mt contains information that would be core to basis for the correct answer.
Figure 9 shows the change in per-question relevance rate as we increase n for MIpc. Our main observations are that Mtrc is substantially more relevant than Mtrandom (see Figure 23 for other methods) and that the ARC Mts contain 5/5 relevant facts at a much higher rate than MedQA. For example, the 100-Mipc obtains 72.3% relevance rate for ARC, while for MedQA, this rate drops to 30.1% %. Similarly, the 1000-Mtrc has around 90% relevance for ARC but only 70% for MedQA. Even looking at the entire fact pool F, is only 93% relevant for MedQA (right- hand bar in Figure ( while nearly 100% relevant for ARC. This suggests that it is easier to find highly reusable, general principles ARC's domain (science), e.g., basic laws of electricity, than in MedQA (medicine). Discussions with our medical experts reinforce this: while there are also general principles in medicine, e.g., "The body strives to maintain a stable internal environment," these are fewer and far between, and the USMLE questions instead often probe a student's long-tail knowledge of highly specific medical facts (e.g., symptoms of Hippel-Lindar disease). This sug- gests that some domains may be more amenable to distilling into microthoeries than others, and explains the more limited gains seen earlier with MedQA compared with ARC.
How COMPLETE ARE MICROTHEORIES?
How many training questions do we need to construct a "complete" microtheory? This is an impor- tant, practical concern for anyone wanting to extract microtheories for new domain. In one sense, microtheory is never complete, as there will always be topical questions that touch areas uncov- ered in training. However, we can make progress on this question in a different way. First, given a
Forr practical reasons, we only assess the top 270 facts retrieved from the Mt by the entailment engine (for Mts larger than 270). The engine uses SBERT retrieval encoder fine-tuned on science QA support facts.

---

Published as a conference paper at ICLR 2025
Per- Question Relevance (%0) by Burlget
# Questions Required for p- p-Relevance
100
Minsckon Stye
ARC
75
100
90
ARC
50
80
Social TQu
25
60
MedQA
40
2010
1000
20061
20
Med(qA
75
10
100
1000
10000
50
# Training Questions
25
Figure 10: Number of training questions (x-axis) needed to create JF that is relevant test questions p% the time (y-axis). While datasets like ARC and Social IQa (Sap et al. (2019) need 900-1600 questions for 90-relevance, but MedQA needs fär more (8900),
50
100 200
500
1000
16722
Fact Dudge
Figure 9: Per-question relevance rate for n-MTPps for varying ns. The rate is much higher in the ARC do- main than MedQA; at 1000 facts, the Mt is at all rele- vant to only 2/3 of MedQA questions but 90% of ARC.
distribution over topical questions, we say that a microtheory is relevant to a question if it contains at least one fact that is core the (correct) answer according to the method described in $5.3.2 We then say a microtheory is partially (p = e.g., 90%) complete if it is relevant to an arbitrary sampled question with probability p. Finally, ing from microtheory learning curves such as in Figure 10 For example, to obtain an Mt with a p-relevance of 90%, we will likely need 940 training questions for ARC. but 8953 training questions for MedQA [0] Conversely, given a training set of size n, we can predict the likely p-relevance of the Mt from the curve. Thus we would expect 100 questions to provide enough knowledge to be (p=60))relevaan for ARC but only 30-relevant for MedQA.
can predict this p-relevance by generating and extrapolat-
p-relevance gives practitioners a way of predicting whether Mts will be helpful for their new dataset: determining how much data they would need to collect for a new domain for the Mt to be fre- quently useful for the topic. If this number too high, then Mts might not be a viable solution for the domain this is likely the case for many datasets for which there is less of a shared body of knowledge that is reapplied to many questions, e.g. HotPotQA (Yang al., 2018) and Natural Questions (K (Kwiatkowski et al.) 2019), where each question asks about unique Wikipedia factoid. The number of training questions also has a large effect on the amount of compute used to construct microtheory, since calling the entailment engine to collect usage statistics and distill the most gen- cralizable facts requires many LLM calls per question. The smaller the training set necessary to obtain high p values, the less LLM calls; running the engine on 8953 MedQA questions to obtain 90-relevance would be making 1 100,000s of LLM calls. Future work might explore which kinds of QA datasets are realistically conducive Mts.
7 CONCLUSION
While LLMs have strong reasoning abilities, it is often unclear how their beliefs recurringly interact to answer questions on topic, significant barrier for understanding and deploying models. To address this, we have presented method for materializing a model's latent, topical knowledge into a microtheory, a linguistic analog of logical microtheories, articulating the model's core, reusable knowledge about the topic. contrast to explanation methods that show snippets of knowledge supporting individual answers, ours is the first method that attempts to distill model's overall un- derstanding of a topic. Our evaluations suggest that, when added to a general corpus, microtheories significantly improve model's ability to ground its answers, i.e., show how they systemat- ically entailed by corpus documents, as well as improving the accuracy of those grounded answers. In addition, human and L.LM-as-judge assessments of the microtheories suggest they contain high proportion of critical, topical facts, and that our construction process successfully identifies many of those important facts from a general pool. Together, these suggest that microtheries are an efficient distillation of model's topic-relevant knowledge, and can provide both performance gains and, for the first time, an interpretable window into a model's 's topical knowledge.
We fit (R² 1.00) modified Hill equation with an additional scaling parameter: y = Ve 4
10 Unlike in Figure 9 we use the original train/test sets here. We compute the per-question relevance of F. prior to any reductions, thus posing upper bound the relevance of any Mt distilled from the larger pool.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Dimitrios Alivanistos, Selene B'aez Santamar'ia, Michael Cochez, Jan-Christoph Kalo, Emile van Krieken, and Thiviyan Thanapalasingam. Prompting as probing: Using language models for knowledge base construction. In LM-KBC@ISWC, 2022.
Paul Blair, RV Guha, and Wanda Pratt. Microtheories: An ontological engineer's guide. Austin, TX:
MCC Technical Report. CYC-050-92, 1992.
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. COMET: Commonsense transformers for automatic knowledge graph construc- tion. In Anna Korhonen, David Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4762-4779, Florence, Italy, July 2019. Association for Computational Linguisties. doi: 10. 10.18663/v1/P11-144 1470. URL
Kaj Bostrom, Zayne Sprague, Swarat Chaudhuri, and Greg Durrett. Natural language deduction through search over statement compositions. ArXiv, abs/2201.0600 2022.
Peter Clark, John Thompson, and Bruce Porter. Knowledge Patterns, pp. 191-207.
Springer doi: 10.1007/
Berlin Heidelberg, Berlin, Heidelberg, 2004. ISBN 978-3-540-24450-0.
978-3-540-24750-1 10. URL fttt::/:/ooo..or//0..000///////3
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv: (003://800.5544 054400 2018.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entail- ment challenge. Machine Learning Challenges Workshop, 2005. URL https://api.
semant wcschhiarccrr..o.co www.aaarrrg/oooooStI....
Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pi- patanangkura, and Peter Clark. Explaining answers with entailment trees. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 202/ Conference on Empirical Methods Natural Language Processing, pp. 7358-7370. Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguis- URL htl.tt/ss///aaatttoooooooooooeee org/22211
tics. doi: 1/222//1/2021.mmmlpppp
emmn 1a-mapp-maate in. 585
Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. A survey of the state of explainable AI for natural language processing. In Kam-Fai Wong, Kevin Knight, and Hua Wu (eds.), Proceedings of the Ist Conference of the Asia-Pacife Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pp. 447-459, Suzhou, China, December 2020. Association for Computational Linguistics.
46
Shibo Hao, Bowen Tan, Kaiwen Tang, Hengzhe Zhang, Eric Xing, and Zhiting Hu. Bertnet: Harvesting knowledge graphs from pretrained language models. ArXiv, abs/2206.14268. 2022.
Ruixin Hong, Hongming Zhang, Xintong Yu, and Changshui Zhang. Metgen: A module-based entailment tree generation framework for answer explanation. ArXiv, abs/2205.02593, 2022.
Peter Jansen, Kelly J. Smith, Dan Moreno, and Huitzilin Ortiz. On the challenges of evaluat- ing compositional explanations multi-hop inference: Relevance, completeness, and expert ratings. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 202/ Conference on Empirical Methods in Natural Language Pro- cessing, pp. 7529-7542, Online and Punta Cana, Dominican Republic, November 2021. Asso- ciation for Computational Linguistics. doi: 10.18653/v1/20/1 http://..22/..ppp...... wwweeeennmamaeee main.596. URL https: wgy. oww.toollly....//22222 emn 596

---

Published as a conference paper at ICLR 2025
Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bam- ford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088 2024.
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What dis- case does this patient have? large-scale open domain question answering dataset from medical exams. Applied Sciences, 1114)):421, 2021.
Jachun Jung, Lianhui Qin, Sean Welleck. Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. Maieutic prompting: Logically consistent reasoning with recursive explanations. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1266-1273 Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v 1/2022. emnlp-main.88 82. URL https: htt///aaaaantoooooooooooooo wttgg//222022 emn main. 82
Nora Kassner, Oyvind Tafjord, Hinrich Schütze, and Peter Clark. BeliefBank:
Adding memory to a pre-trained language model for a systematic notion of belief. In Marie-Francine Moens, Xuanjing Huang. Lucia Specia, and Scott Wen-tau Yih (eds.). Proceedings of the 202/ Con- ference on Empirical Methods in Natural Language Processing, pp. 8849-8861, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguis- //acl antho 1 logy. org/2021,
tics. doi: 10. 10.18653/1/2021 201166331/222..mmmmmmmm./// @emmlp p- main. 697
Nora Kassner, Oyvind Tafjord, Ashish Sabharwal, Kyle Richardson, Hinrich Schuetze, and Pe- ter Clark. Language models with rationality. Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Language Pro- cessing, pp. 14190-14201, Singapore, December 2023. Association for Computational Linguis- tics. doi: 10.118653/11222233mmmmm www.:0233..mmmmmmmmm..mm.. @mmn1 ponnlp-mann... 877
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wai Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics. 7:452-466, 2019. doi: 20.1162/taclaa URL
http::://aaaanthhloggoooo org/ 001-1026666
Alice Lai, Yonatan Bisk, and Julia Hockenmaier. Natural language inference from multiple premises. In Greg Kondrak and Taro Watanabe (eds.), Proceedings of the Eighth Interna- tional Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 100- 109, Taipei, Taiwan, November 2017. Asian Federation of Natural Language Processing. URL
wrg...orgginn org//17-1011
Douglas B. Lenat, R. V. Guha, Karen Pittman, Dexter Pratt, and Mary Shepherd. Cye: toward programs with common sense. Commun, ACM, (3) 49, aug 1990. ISSN 0001-0782. doi: N0.1145/79173.79976 URL htttts: www.ooooooorr//oo 11..0.....111111 79176
Bernardo Magnini. Book reviews: Recognizing textual entailment: Models and applications by Ido Dagan, Dan Roth, mark sammons and fabio massimo zanzotto. Computational Linguistics, 41(1): 157-159, March 2015, doi: 10.1162/COLL..00211 URL htt.:t::://caaattooooo.y..
Stephen Muggleton. Inductive logic programming. New generation computing, 1991.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Kentaro Inui, Jing Jiang. Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joins Conference on Natural Language Pro- cessing (EMNLP-IJCNLP),, pp. 2463 2263,,243, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 20.18653/v1/D19. 1/50--1200.
URL httt:s::/aacaattho..
oogg BTg/D19-1255
12

---

Published as a conference paper at ICLR 2025
Nils Reimers and Iryna Gurevych. Centence-BERT: Sentence embeddings using Siamese BERT- networks. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural Language Processing (EMNLP-IJCNLA) pp. 3982- 3992, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v 1/1/8653////I1-1411111 1/119-1410,
Danilo Neves Ribeiro. Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henry Zhu, Xinchi Chen, Zhiheng Huang, Peng Xu, Andrew O. Arnold, and Dan Roth. Entailment tree explanations via iterative retrieval-generation reasoner. In NATCL-HLT, 2022.
Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulin Mike Gatford, et al. Okapi at tree-3. Nist Special Publication Sp, 109:109, 1995.
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Social IQa: Com- monsense reasoning about social interactions. In Kentaro Inui, Jing Jiang. Vincent Ng, and Xi- aojun Wan (eds.), Proceedings of the 2019 Conference guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4463 4473, Hong Kong. China, November 2019. Association for Com- putational Linguistics. doi: 10. (0.18653/v1/D19. 1454. URL https: haiE::://aclanhoology: .rgg
Empirical Methods in Natural Lan-
D19- 154
Oyvind Tafford, Bhavana Dalvi Mishra, and Peter Clark. Entailer: Answering questions with faithful and truthful chains of reasoning. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp- Abu Dhabi, United Arab Emirates, December 2022. Association for Computational org/
Linguisties. doi: 10. 101118655/110000 2022. www.p-main.n..... 134
134, URL
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi (Jim) Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. Trans. Mach. Learn. Res., 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. thain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837. 2022.
Nathaniel Weir, Kate Sanders, Orion Weller, Shreya Sharma, Dongwei Jiang, Zhengping Zhang, Bhavana Dalvi Mishra, Oyvind Tafjord, Peter Jansen, Peter Clark, et al. Enhancing systematic de- compositional natural language inference using informal logic. arXiv preprint arXiv:2402.14799, 2024.
Sarah Wiegreffe and Ana Marasovic. Teach me explain: A review of datasets for explainable natural language processing. In Thirty-fifth Conference on Neural Information Processing Sys-
tems Datasets and Benchmarks Track (Round 1), 2021. URL httppsss//penreviee..nn
Zhengnan Xie, Sebastian Thiem, Jaycie Martin, Elizabeth Wainwright, Steven Marmorstein, and Peter Jansen. WorldTree v2: A corpus of science-domain structured explanations and inference patterns supporting multi-hop inference. Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi. Hitoshi Isahara, Bente Mae- gaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 5456-5473, Mar- seille, France, May 2020. European Language Resources Association, ISBN 979-10-95546-3444 URL http:s:///acaaahoooooooo org/
wwtp////000000 2020 Irec- 1. 671
Kaiyu Yang, Jia Deng, and Danqi Chen. Generating natural language proofs with verifier-guided search. ArXiv, abs/2205.1 12443,, 2022.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question
13

---

Published as a conference paper at ICLR 2025
answering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Lun'ichi Tsujii (eds.), Proceed- ings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2369- 2380, Brussels, Belgium, October- October-November November 2018. Association for Computational Linguisties. doi: 10. 10.18653/1//D181225. URL ftI.https: //acl antho 1 logy. oggy.org/D18-1239
14

---

Published as a conference paper at ICLR 2025
A USAGE OPTIMIZATION DETAILS
A. n-MToc LINEAR PROGRAM
We the objective function to achieve the following goals, ordered by precedence:
Maximize the coverage of questions by least one proof tree.
Maximize the number of proof trees covered by the selected facts.
Minimize the number of facts included in the new theory.
We define the objective
C
d
min Σ
W24
ykj
(5)
Where z; is a binary variable indicating whether fact is included in the Mt, Ukj is binary variable indicating whether basis Laj for question j is covered by the selected facts, and 2j is a binary variable indicator whether question 9j has at least one basis fully covered. We set to be sufficiently high to dwarf the other terms, using them only for breaking ties. We impose the following constraints:
At most n. facts are kept.
Σ T S
(6)
Each Lyj is "covered" iff all its facts are kept.
Lxsl
Σ
xs 2 Vj
[Lall
Each question is "covered" if at least one of its bases is "covered"
VCJ+
A.2 n-MTPc ILP IMPLEMENTATION
We set the objective function to achieve the following goals, ordered by precedence:
Maximize the partial coverage of each question by the selected facts.
Minimize the number of facts included in the theory.
We define the objective
min Σ
x4
wp
Where x; is a binary variable indicating whether fact included in the Mt, variable indicating the maximal partial coverage of question by the selected facts, and ykj is continuous variable indicating the coverage of basis Laj for question j by the selected facts. zkj binary indicator of whether given Lag is the highest coverage of question j attained by the solution.
continuous
We impose the following constraints:
At most n facts are kept using Equation 6
Each Lęj is "covered" by the proportion of its facts that are kept.
VLkj.
ykj =
1
Σ sELis
(10)
Lkj
15

---

Published as a conference paper at ICLR 2025
Two constraints that enforce Pj equals the highest parital coverage for question j using the Big M method: The maximal partial coverage of each question is at least as large as the coverage of each of its bases.
vh5, Pj 2 Ukj - M( - -kk)
vk
(11)
and the maximal partial coverage of each question does not exceed the coverage of each of its bases.
vLj, Ps S Ukj
Mzkj
vk
(12)
Exactly one žkj is active per question.
Cd
VLJ.
2kj
=
(13)
1=1
B QUESTION/TOPIC CLUSTERING ALGORITHM
To create our topic-specific mini-splits of ARC and MedQA, we use a hicrarchical clustering al- gorithm that gathers questions targeting similar knowledge by iteratively prompting an LLM to "caption' questions with core targeted knowledge sentences and then clustering them using SBERT. Algorithm depicts our implementation of the question clustering algorithm used extract subsets of a broader QA dataset that target similar underlying core knowledge. For the embedding cluster- ing algorithm we use the Sentence' Transformer-bast based community detection algorithm that clusters sentences greedily according to some minimum cosine similarity threshold, which we set to 0.6.
Algorithm 1 Two-Step Clustering Algorithm for Question Grouping by Core Topic
procedure [WOSSTEFCLUSTERNNNG() a Hierarchically clustered questions with topic and hy- pertopic labels:
2: core facts
3: forall qi Q do
4: core_fact = =LMEEXTRAATCCORRFAACCY
5: core_facts += core_fact
6: end for
7: clusters
IIBERT.CLUSTER(COre.f.
8: topic_labels labels = 0]
9: forall Gj E clusters do
10: Jabic = LLM. GENERATETOPPCLAABLL
topicJabels += tabie_label
12:
end for
topic.clusters. clusters hapertopii.Jabels =Q
14:
forall te₁, E stopic_cluster do legperlopic..ater label
16:
W.M.GGNNR.AEEEYYERRPPPC...CC legpertopic.lubele += hapertopic.Jabel end for
17:
18:
19:
Assign topic labels and hypertopic labels to corresponding questions in Q
20:
return Q with hierarchical topie and hypertopic labels
C FACT EXTRACTION PROMPTING DETAILS
The LLM prompt generate relevant facts for given Q; is few-shot learning prompt using exemplars dynamically retrieved from EntailmentBank, Each exemplar's fact list comprises the gold WorldTree (Xie al. 2020) facts from the unstructured explanation graph, and its entailment tree is the gold entailment tree that uses a subset of the facts. Thus, while Q; might not be in the same ScienceQA domain as the exemplars, we cue the LM generate WorldTree-like facts that should serve similar roles as argumentative bases in entailment trees. We discard the model-generated entailment tree and take the generated fact list as candidates to be added to the fact pool.
Figure 11 shows an example dynamically constructed in-context learning prompt for fact extraction. Figure 12 shows an example output from Mixttra-&x222--nnstmee-- for MedQA question.
We use BM25 Rooherson al. 1995 for retrieval using the question the query.
16

---

Published as a conference paper at ICLR 2025
You are an excert theery ceneration ion ayacen that lists facts about prebian area that will help solve a reasening quastion. civen question, you genorate QUERY stanemence for each of the possibic ansvers to queat The quecies should be complete stalementa that exau.ly natch as possible.
choices as closely
Then, you generate a theery compeising simple FACTS about the world. These should be atacementa that you believe be locical and about the vorld that will help prove the quaries, a PACT either generip stat enens about world ("birds own fly*) or specific statament about the context that helps support the proof of the quezy (Muueacy is bizd").
For QURRY, you generate leass stalements.
Oree you generate the theory, you show how one of the QUERYS logicel ly Foll followe ows Fron the context. The se selected QUERY perseapunda to the answer choicw that lewl invw is cueree.. You can only clouse onw QUERY
To show your caa seasnnigg, you genecabe a PROOF of your thusen QUERY hygutheeis. Your FROOF is an "enta i lamat tres" that composes the PACTS of your theozy in order to show how the quazy is compesitionally entailed step
by
corclusion which should be used prenise Iature aleps. The last step curgosal lunalll provea hypellesia. then you zeuse the nav conciusion in later stap, you rafar to it by its id. Bach step 1n separated by a neuline
is composition
prenises into
H
QURSTION: ch of the fallowing act lons is nost likely part of a test to find the hardness of a mineral sample? (A] heating the sumple on has plate 18) scratching the sample with nall (C) hilting Lhe harmer (D) shining bright light
sample
cample
QUERIES:
(OURRY 41 lleatinng the sample hot plaate is nost likeyy part of best find the hardness of mirersl sample.
IQUERY BJ Scratcaing the cample with a nail is most likely part ot best to Hind the hareness ot mineral sample.
(oURnY CJ Hitting the sample with hamner is nows likely part of Lext to find the hardnwas of nineral
sample, FOURRY RI siining a bright light on the sample is most likely part of a test to find she hardness of a mireral samgle.
THRORY:
(FACT 1) mineral is kind ot material
IFACT 2) harcness is measure of nineral 's abilisy to resist soratching
(PACT an exangle of breaeing soneching down is scratching sonething
(FACT naterisl that
eatt can broken down by material that is hard IFACT 5) testing often requi res measuring
(PACT 6) naill i8 a kind of object
(FACT hardness
material object and incloses ordered values malleable / rigid hardness can be used scentity minerals
property
(FACT 3) nineral is kind ot solid / nnttural material
(FACT 10) naile are otten made of iron
IFACT 11) scraping obgest
snall particles break off of that object mincral can be scratched by a ringernail then that nineral soct (FACT 131 hardnese 16 kind at physical propesty
(FACT 12)
IFACT 14) hardness 18 an intensive property
(FACT 15)
is kind harcness
IFACT 16) experimentation zequires measuring
(FACT 171 an event is kind of action
(FACT 19) it material seratches casily then that matorial has 1oM hardnese
IFACT 19) minerel is kind object
(FACT 201
mineral nutricat is kind nutricat
(FACT 21)
abjects are made of naterials substances natter
IFACT 221
cock is made or minerals
(FACT 231
of (EACT 241 an object mads naterial taen that objest has the propertios that material
process is
of actions
IFACT 251 testing / identifyiigg the streak at mireral requires the powdered torm of that minerel.
PROOF Scratching the sample
nail
likely part
cind the hardness
nineral
sample.:
(STEF 1) natoricl that is soft can be broken dowr by a materisl that is hard an example of breakiac something down is scratching something -> intl: material that is sott can be scratched by a matorial. that is hard
(SIEF intl other mineral is sotter than that one ninerel
mincral is kind natorial int2: one nineral can scratch another mincral then that
(STBF 3) hardness is measura of ninezal 's abilisy to realst seratching & lnt2 +> typothasis: acratching the sampls wish nail
likely part of test find hardross nineral sample.
ANSHER: B
616 QUESTION: (quastion)
QUERIES: (cueries]
THEORY:
Figure 11: TheoryCoT in-context learning prompt for extracting WorkdTree-like facts from an LLM for a given question.
17

---

Published as a conference paper at ICLR 2025
QUESTION: A 47-jezz-old wonen with chronic opigaatric pain concs to the phycician because of a 1-monta histozy of interniltent lusse, foul-soelling stusis, the also had 6-2g (13-1b) weight losa. She has consuned 9-10 alocholic beveraçes daily for the sast years. Saven years ago, she traveled to Moxico on vacetion ahe has not been outside the large nel ropolliiienn area in which she resides since then, She appears malnourished, The atool is pale and loose: fucal fat content is elevated, An inmunoglobalin A anti-tiarue tranageetaniaas antibody assay is negative. Further evaluation la likely shew which of the following? (A) Tropheabiles on weodl nicrowopy, [8] Pancrestic ie cslel finasioss (C)
Villous atrophy of duodenal nuccss, [9] Positive Lactulose breath teat
QUERIES:
(QUERY Further avaluation is nost likely to show trophozoites on atcol nicroacopy.
[OURRY N. Forther evelustion is nost likely to show panoreatic alleification
(QURAY C1
Fuztlier evaluation is must likely to show villous alcuphy of duodenel nucose. (QUERY Further evaloation
likely show positive iastulose breath teat.
THRORY:
(FACT 13 Chronic spicastric pain can be sympton of various gastrointestimal disordars. Internittent,
Foul snell linn stools car sign of melabsorption,
(PACT Signifieant wiight loss can sign of iilra us nelsbaorpt ion.
(FACT 4) Long-tern, haavy alcshol consumption can lead to pancreatitis and malabsorpt.con
(PACT Travel to certain
expose irdividully to paras that cause gessrointes inal lasees. and Lissue camnglutannnnee and anuiy awaay rules out wellan
negaliie lamongglobalin deases
(FACT 7) Ha Walaboorpt ion can lead to elevated fecel far content.
(FACT Pale and lonse atouls can be sign of malahsorplion.
Pancrestic flcatiiooon can sigr chroric pancrestitis.
(PACT 10) Villous atrophy of dundenal mucosa can be sign of celiac disease or other rointet inal disorders.
(FACT 11)
positive lactulose breath test can indicate small intest inal bacterial overgrowch (91R0).
PROOF of Further evalustion is most Likely to show parcreatic calcitications
1STRP 13 cong-tern, heavy a cachell consunption fan can lead to pancreat is and nw sbsorptionn on 6 Siinifiiant weight lass cen be sign of rition
ne melabsorpi ion > Intls The patlen fa long tem, heavy alouho? consumption may have led to porcreatitie and malsbsorytion, csseing signiticent weigat loss,
loose, foul-ams lin steal a can be sign of nal shsorpt oon -> int The parient' intwenition it Leniy Soawe, Soul smelling scouls are libely due to na nalabsusgs ian oa caused by concreatiliv. 15TSF int2 Pancreatio siciissattions can de
(STRE 2) int & Interrittent,
chronis pancrestitis typothesis: zorther fon is likely to show parcreatio in celle Fieationss
Figure 12: Example ToT output for a MedQA question. The generated proof steps are thrown and the generic facts are added to the fact pool F. Context-specific facts are discarded.
D ENTAILMENT ENGINE IMPLEMENTATION DETAILS
To collect training proofs, we explore up to depth 2 for ARC hypotheses (these are easier to prove) and up to 40 expansions for MedQA.
D.1 RETAINING INFERENCES
When collecting the bases Laj, we want the entailment engine to best identify which facts most commonly reused across questions. Since the engine has the potential to miss certain combinations, we add a mechanism when finding bases for Q; After each call to ENGINE(h;, C; Q:)). we take all intermediate sub- hypotheses grounded by the engine facts in C in the process of proving h,, some S, and add them to the fact pool for all future questions j > i. Whenever the engine uses any subhypothesis s E S we replace with all the ways the engine grounded s using bases in C. E.g.. if the engine returns basis (f1,*] and was itself previously grounded by the sets [s). [f2, fa), we replace [fi, s] with [fi, f2, f's and [fi, f2, fa] 12
encourage reusing inferences from earlier training questions Q1 ... Qi-1
E QUALITATIVE EXAMPLES AND ANALYSIS
Figure 13 and Figure list the top-10 10 most frequent facts used for training questions, as well as a sample of 10 from the long tail of facts used for only one question. We also show which of these facts are retained in the partial coverage 100-MtpG- The top ARC facts are core pieces of knowledge very similar to those that appear in the hand-crafted WorldTree corpus; many of the facts in MedQA are about chronic obstructive pulmonary disease (COPD), reflecting that the Smoking topic is the largest in the training set and that COPD is common medical condition associated with smoking. Figure 15 depicts a subgraph of a much larger network of entailments connecting statements in C to 18 of the training hypotheses. It shows which of these statements were retained by the partial coverage Mt for their support of multiple related hypotheses. The subgraph illustrates 4 cliques; one for hypotheses about the gravitational force between opbjects, and 3 about the force, accelcration and mass of objects in motion. The core statements retained in the Mt are mainly versions of Newton's 's second law of physics, F = and Newton's 's Law of universal gravitation. While a singular fact
To limit combinatorial explosions, we produce maximum of 1000 bases per question via this "unfolding".
18

---

Published as a conference paper at ICLR 2025
QN
Mipc
any feature which is determined genes that be transmiited from parent to ollspting during reproduccioo is considered an inherited trait
23
Yes
the earth turns on its axis from west to cast once every 24 hours
22
Yes
sedinentary rocks are most often formed by the compaction and cemeniation of sediment, which is material trans- ported and deposited by wind, water, or ice
20
Yes
the tilt of earth's axis the reason for the changing amount of sunlight that each hemisphere receives, which in turn
20
Yes
is the driver for seasons
genes the instructions for physical traits that pass from parents offisring
20
Yes
Though Earth's revolution around the Sun does affect the duration of daylight and darkness, the basic cxistenee of the two phenomena is due to Earth's rotation on its axis
19
Yes
inheriting means receiving genetic information and traits from parent or parents
19
No
Physical traits of an organism, such as fur color, eye color, and size, can be genctically passed from parents to offspring
19
Yes
The rotation of the Earth regulates the daily eycle of day and night, not the sensonal cycle the rolation of the Earth on its axis is the movement of the Earth around a central point. once every 24 bours
19
Yes
18
No
Whales utilize ocean currents and temperature gradients to guide their migration routes
No
Mutations can be caused by various factors such as expusure to radiation, certain chemicals, or errors during DNA
No
replication
requires sunlight grow
No
lava beds igneous rock, formed from cooled lava wilting is a survival mechanism, limiting water loss by reducing the surface area exposed to the sun Mendel's experiments with pea planes were focused on herodity
No
No
No
Figure 13:
Top-10 10 most used and random 10 least used (at lcast once) facts in the ARC microtheory. The rightmost column depicts whether the fact is retained in the "partial coverage" microtheory (Mtrc) that reduces redundancy while maximizing training coverage.
Fact
Questions
Mfrc
The severe smoker with an estimated 40 pack-year history has high risk for chronic obstructive pul- monary diseuse (COPD) and lung cancer
45
Yes
The man's history of long term smoking puts him increased risk for candiovascular and lung dissuses, including pulmenary embolism
34
Yes
a heavy smoker with 25 pack-years history is high risk for lung diseases
32
No
Chronic obstructive pulmonary disease (COPD) is generally soon with a history of smoking and presents with symptoms such as shortness of breath. progreasive dyspuca, cough, and sputum production a 50 pack-year smoking history contributes to a multitude of bealth risks, including lung cancer findings such as blood and protein in the urine, as well as high scrum urea nitrogen and creatinine levels,
31
Yes
27
No
25
Yes
indicate kidncy damage
40 pack year smoking history, hypertension diabetes make one more likely have COPD man with pack-year smoking history has a very high risk developing health issues related to smoking
25
No
25
No
long exposure lung irritants, particularly labacco smoke, is the primary risk factor for developing chronic obstructive pulmonary disense
24
No
A patient's 's history of smoking cigarettes daily for 25 years can have a significant impact on their respira- tary function, leading problems such shortness of breath, especially after a surgical procedure
22
No
rapidly enlarging scalp lesion i child, especially blanching with pressure. is suggestive of vascular lesion such as a strawberry hemangioma
No
losing job due to issues such as tardiness is likely due to some negative circumstance impacting the individual's such alcohol abuse
No
Blondy diarthea can a symptom amochic dy dysentery caused by amoebic infection
No
fever common symptom in vasculitis because of the inflammation
No
Normal pressure hydrocephalus is characterized by cognitive changes, guil problems, and urinary incun-
No
tinence
Blood pressure 197/124 mm Hg considered dangerously high and needs be lowered immedistely beart, brain, skeletal musele and tissues, its levels increase when muscle or beart cells are injured
No
Creatinine phospbokinase an enzyme found
the liver breaks down bilirbbin so it can be removed from the body in the stool
No
Smoking and unprotected sex increase the risk of various infections, including urinary tract infections Regular kneeling put excessive pressure prepatellar bursa, potentially causing inflammation and pain
No
No
Figure 14: Top-10 10 most used and random 10 least used (at least once) facts in the MedQA microtheory. The rightmost column depicts whether the fact is retained in the "partial coverage" microthcory (Mtrc) that reduces redundancy while maximizing training coverage.
describing the law might theoretically serve the same purpose in proving all these hypotheses, We
19

---

Published as a conference paper at ICLR 2025
observe that the entailment engine prefers to separate facts representing different interpretations of the law (e.g. how to calculate net force, vs how to calculate the force required to move an object). This may also be a result of imposing depth-2 limit of the training proof search. It would perhaps require more "legwork" by the entailment engine to construct a chain of more than 2 entailments from single statement, e.g. just 'force equals mass times acceleration," to each hypothesis.
and dieu
Figure 15: Subgraph showing some of the statements from the 200-Mtrc ARC microtheory orrange and
their influence on proving training hypotheses ( green different question contexts (yellow) ). The partial coverage (PC) algorithm selects facts from that provide coverage for multiple questions. Facts from that are specific to only one or two questions maroon are not selected. Each dot (-) represents an entailment; inferences ( grey and green can be supported by alternative entailments the form of multiple entering edges. In this instance, the PC algorithm shows retain facts mostly related Newton's 's second law, which core to multiple subtopics grade school physics tests (motion, friction, gravitational attraction). Only the orange nodes are retained in the microthcory for test time inference; all other statements are discarded.
Figure 16 shows a similar subgraph for the MedQA microtheory, this time illustrating the facts retained by the full question coverage 100-Mtge- Hypotheses are fully thus grounded to Mt state- ments and the context passages of their respective questions. This graph also illustrates the effect of retaining inferences during the serial computation of the Laj leaf sets.
20

---

Published as a conference paper at ICLR 2025
Figure 16: Subgraph showing some of the statements from the 100-Mtee MedQA microtheory orange and
their influence on proving training hypotheses green ) in different question contexts ycllow related to clin- ical cases. Unlike the PC approach, the question coverage (QC) approach ensures full grounding of each train- ing hypothesis using facts from the microthery. As MedQA hypotheses often require many entailment hops, the inference retention mechanism described in SD.1 helps prove hypotheses using inferences ( light orange grounded from previous questions, some of which are omitted for space.
F RETRIEVAL IMPLEMENTATION DETAILS
The Wikipedia corpus used in $4 is structured as a flat index (i.e., a very long list) of paragraph chunks- specifically, the 2021-0I- 20 version of Wikipedia with 100 word chunks. For similar- ity, we split the textbooks into 100 word chunks as well. We retrieve paragraphs using two-phase retrieval;
BM25 (Robertson al. 1995) for first stage retrieval and rerank using Sentence- Transformer's 's ms- ms-marco- Maaroo-MimiLL---1122
(Reimers & Gurevych] 2019).
When we retrieve facts from both the Microtheory and Wikipedia (or Wikipedia++Textbookk) append the sets in that order. When we retrieve from Wikipedia+ Textbookss,s we interleave the results one-by-one give equal precedence the two sources. These design choices were chosen based on empirical performance.
G MIXTRAL RESULTS ON MEDQA
To demonstrate our work is not closed-model dependent, this section replicates the MedQA exper- iments from the paper using Mixtral-8x22B- - (Jiang et al., 2024) as the source of the microtheories instead of GPT-4. Figure 17 Figure 18 Figure 19, and Figure 20 each show Mixtral results analogous to an experimental result from the main paper using GPT4. We have commented on the similarities and differences with the original result in the caption of each figure.
H ADDITIONAL FACT USAGE METRICS
In addition to the grounding rate ($5.I), relevance rate ($5.22).. this section shows a pair of additional metrics for our considered microthe- ories: (1) the fraction of all facts in microtheory used by the entailment engine for least one
relevance rate (5...1). and automated
21

---

Published as a conference paper at ICLR 2025
Mixtral-8x22b Results
MedQA
Dataset Details
# Questions
641/94/186
McdQA
# Topic Clusters
5000
Ext. Corpus
Wiki+Textbooks
# Facts
1000
100
Training Extraction Results
10
[F]
11,082
[C]
8,183
5
10
15
20
# Qs with Proof
476 (74%)
Questiion per Fact
Min #Fs to Cover
918
Fact/Q Ratio
1.91
Figure 18: Histogram of training question per fact in C and 650 MedQA training questions us- ing Mixtral-8x22 multiple GPT-4 generated facts used for 25 or more training questions, -intrra--enerrtedd fact is used more than 22 times.
as the underlying LLM. While
Figure 17: Results of microtheory extraction from train- ing data using Mixtral-8x22b as the underlying LLM. The last two rows result from the "min-fase fact" LP. Compared with GPT-4, Mixtral generates 5000 fewer unique facts (1IK vs 16K) and finds proofs for 14% fewer questions.
Average MexQA Faet Relevance by Budget
MedQA Grounding Rate (56) of Test Hypotheses
5.0
100
1:= 100
R=500
1000
Microtheary Method
81
4.5
Minsag
Mipc
75
4.0
50
3.5
25
3.0
2.5
Cp Rd Us QC PC Rd Us QC PC Rd Us QC F
10
25
50
100
200
500
1000
Budget
Figure 19: % of test hypotheses fully groundable after adding various Mixtral- generated microtheories (n-sized Random, Usage, QC, & PC Mts) to a base Corpus. We show the fraction of leaves grounded in the Mt (solid) and the hase corpus (striped).
Average espert-annotaan task rel- evance score for Mixtral generated microtheory facts.
Figure 20:
test question, and (2) the average number of test questions for which each fact in the microtheory is used.
Figure 21 shows the fraction of microtheories used at least once at test time, Fndure 22 22swww the average number of questions for which any single Mt fact is used. In both cases, the Muusge.MMgc. and Mtpc have higher usage than the random sample baseline, while the full fact pool F scores very low, since most of the facts in the full pool are never used at test time. Expectedly, the Mussag approach scores highest on both metrics, given this is what it was optimized for; we sorted facts by their usage counts at training time (the same metric measured in Figuure 22) and took the top-n the Mt. We note that this does not directly correlate with Mt quality since, as we discussed in $3.2.3 sorting only by per-fact usage fails to remove functional redundancies. The test grounding rates in $5.1 should be considered more encompassing metric.
I EXTENDED RELEVANCE RATE RESULTS
Figure shows the rate at which the considered microtheory methods contain 5-rated relevant facts for the test sets in the topic mini-splits of ARC and MedQA.
J EXPERT RELEVANCE ANNOTATION DETAILS
We sampled 25 facts (fewer when necessary) from Mts with 70 budgets of size ranging from 10 to 1000. We evaluated the Mtasage Mtpc, and Mtrandom approaches. Both annotators rated each fact independently and then reconciled the (rare) disagreements. We computed agreement metrics across the full set of 314 annotated facts, finding Krippendorff of .81, Cohen's 's k of .809, and raw agreement rate of 863.
This numberia is not 7 x 3 x 25 because some Mts were smaller than 25 facts, the n- Mt-Mugg are all supersets of each other, and many other samples contained high overlap due to the extraction algorithm hehavior.
22

---

Published as a conference paper at ICLR 2025
ARC Questions per Fact
ARC Fraction Used Leard Ouce (96)
n2-000
1--0000
1.00
7:m 100
500
12=1000
0.75
0.50
0.25
0.00
Rd
QC
PC
Rd
Us
QC
PC
Rd
Us
QC
PC
F
Rd Us QC
PC
Rd
Us
QC
PC
Rd
Us
QC
PC
MedQA Praction Used At Least Once
(%)
MedQA Questious per Fact
1.00
1=100
N=0000
x==000
n==500
1=0000
0.75
0.50
0.5
0.25
0.0 Us
Rd
QC
PC
Rd
Us
QC
PC
Rd
Us
QC
PC
F
QC
PC
Rd
Us
QC
PC
Rd
Us
QC
PC
F
Figure 21: Fraction of a microtheory used in an entail- ment proof for at least one test question (out of 249 for ARC, 186 for MedQA). All distillation methods (n-sized Usage, QC. & Mis) are substan- tially more frequently than the random sample base- line, with the Usage having the highest fraction. The fraction used for the full raw fact pool J is very low.
Figure 22: Average number of test questions (out of 249 ARC, for MedQA) per fact microthe- ory. Distilled microtheories are used in more questions than the random baseline, with the average usage of Usage microthcory facts being (expectedly) highest.
ARC Relevance Rate by Theory
15.
Dolevtro Bate
113
T.L
0.5
Relanaal
MexIQA Relevancr Rate Theery
110
Mbye
Thaney
Figure 23: Rate (%) of microtheories containing least fact with 5/5 and 4/5 relevance determined by GPT-4o. Numbers above each bar are percentage
K
AUTOMATIC RELEVANCE ANNOTATION DETAILS
The prompt used for scoring the relevance of facts to given question is shown in Figure 26 The original rubric by Jansen et al. (02I contains 4 ordinal scores; we added a 5th differentiate between facts relevant to ruling out incorrect options (4) vs supporting correct options (5).
23

---

Published as a conference paper at ICLR 2025
Indicators
Exemplars
Statementifact that includes relevant associations" and IS specific, comprehensive, or pathognomonic enough provide question.
Deletion of 11-p-15 can cause
Beckwith- Wiedemann syndrame, which includes symptoms such as macroglossis, omphalocele, and hemihyyerrroppy
clear
Statementifact that provides information and/or associations about a topic domain that are fundamental to its understanding as whole
comprehensive
Tachyoardia (elevated heart rate) tachypnea (elevated respiratory rate) are signs of cardiac pulmonary distress.
Statementifact includes relevant associations specific, comprehensive, pathognomonic enough provide a clear answer to a test question
autosomal recesaive inheritance, both must be carriers of the mulated gene child inheni the discase
Statemmntifact that provides neither
Increased ciroulating estrogen can oczur men with liver disease, such as alcoholic cirmoois
comprehensive information and/or associations about topic domain that are relevant but not fundamental to its understanding
Osteaporosis is a condition characterized by low bone mass and deterioration bone lissue, leading to Increased bone fragiity and risk fractures.
Without knowing this statementifact, it would be difficult correctly answer a question on this topic
Definition that includes assumptions that are directly useful in answering question correctly
associations, context, andlor
Statement/fact that accurate provides context information about topic; does not provide information that would direclly result in getting a question right.
Symptoms drug-induced include and confusion
Without knowing
statement/fact, would NOT difficult to correctly answer question on this topic
Physician Program a program designed to help physicians with substance abuse or mental health issues
Definition that lacks relevant associations, context, and/or assumptions that are directly useful in answering question correctly
May include examples additional details that not directly facilitate correct answer
"Statement/fact that generic or common-sense; information widely known and/or lacks details relevance about given topic
spine part musculaskeletai system
Two dwo-ddimensoonal gel electrophoreeis not typically used to diagnose genetic disorders
specific context medical offering little value the diagnostic andior question-answering process
Sttatementffact that rarely helps
Decreseed activity apithellal Ne+ channels principal cells can be caused by increased activity luminal
principsl
Statement/fact that contains an element or component that factually untrue inaccurate
SStatementffact that useless, unclear, and/or completely irrelevant
The patient's BUN and creatinine levels slightly elevated.
Stattementfactt that never helps in the specific context of medical exams, offering value the diagnostic question-answering process
The cause of erythema nodosum often unknown, may related diseases.
systemic
Bloody diarthea be symptam Giardia
Statement/fact that is entiroly factually untrue or inaccurate
"Relevant association: A significant connection or correlation between two or more related factors (e.g.. diseases, symptoms, risk factors) that direclly impacts understanding, pathophysiology. diagnosis, treatment, andlor prevention of medical condition.
"Topic domains: subject, discipline, system medicine that encompasses many which many questions are derived from (e.g. cardiac system, renal system)
Figure 24: Rubric used collect expert relevance ratings for medical microtheory facts (1 2)
24

---

Published as a conference paper at ICLR 2025
Would medical student be in trouble if they didn't know this fact the morning of the exam? [yes =
Can you imagine up (or remember) question for which this statement would directly answer the question? (yes 5)
Would topic domain (e.g.. 'cardiovascular system") be hard to understand if you didn't
know this fact? [yes 5)
Could the information feasibly be important to understanding the correct answer or any of the key terms being tested by questions correct answers (e.g., definitions, etc.) but not the concept being tested itself? (yes 4)
Does the statement add correct and relevant supporting details that give context or depth to explanation, facilitating understanding of the concept? (yes Is the statement relevant to correct explanation of correct answer but generally not required for a complete and correct explanation? (yes = 3)
Is the statement semantic claim that correctly identifies a term that might be related to questions or correct answers but required produce correct answer or explain it? (yes
Is the statement partially incorrect or contains an element of untruth? (yes 2)
Is the statement "common knowledge"? (yes 2)
Does the statement have information that too general only marginally related given topic? (yes 2)
Is the statement completely irrelvant to correct and complete explanation of any feasible correct answer? (yes 1)
Is the statement completely incorrect? (yes= 1)
Figure 25: Rubric used to collect expert relevance ratings for medical microthcory facts (2 of 2)
25

---

Published as a conference paper at ICLR 2025
You sciantiric
scieatists deternine whether various explaining correct ansxer to quastion. Scientists give you question list tacts they baliave are selevant to the question. Your job is to deternine whachez each tast ia relevant to explaining the correct ansxer to the quastion using previded rubric for scoring:
rolevant
fact score cors/aritica: fect/ satt/kowlodes that explains way a correct answer is ecrrect wisential piece of infernal lun if explaining phanenenon in queation Loddles
it
following iadicatora:
fact has scoce cosw/critical: Caot/kapwindge that explaina way at incosrect ansaMs la Incoerect
it has following iadiestors:
fact
has tollowing iadiestors:
noderately inpostant pacessacy
correct explanation but not being explicitly cesced sunderlying sssurption racessary to undorstaadiag the temss used in the corract respense and explanation definex Lamts used in the promp. and/uz the cocrect innediate corsectly identify common sansa statenent that sopically rolevant not necossary diffaren:ia:ing cozrect Exom Incoerect rect
A fact walra detail explanation nissing this facl are awicher insursect nos incunglese Lrve zelavant atatement but not necassary for understanding shenomena in question
following ladiestors:
faut irselevant
has following indicatusa:
inporrect
least recessary and core to correct ly araver the imediate quest (le.. if womeone does not know the Fact. Lhey campo ansies the quest for, cucreetly).
nead
should only fact absclutely
the quest ions you yourself considering Down this stalement directly ansess the original queation? (yes 5)
Ta the information absclutely necessary to correct ansver the imeddiate questioor? tyee
Is the information lon absclutely necexsary explain another anaver la incorrect for the immediate
question? lyea
Ta the information on important underatending the correct ansuer or any the key zerne being tested by the immediate question or correct answer (e.gu definitions, ete.) but not the concegt being bested itselE? (yea 3)
ifies key answer a way thas relevant to the phenomenon being bested? (yea
statement
correctly
inmediate quest
Is the statenent relevant to a cocrect explanation ot the correct ansver but not required for complete and correct esplaracion? lyea selevent suppor Ling details but are not requized to make the explanation corpLete and correct? tyes - 2)
explanet ion Tx the statement a semanbic in clain that correct ly idincifies a term that is related so the innediaee guest ion explain it? (yea
the
give
depth
but not required La produce the currect the statenent completely irrelevant
cucreoi
correct and complete explanation the correct answer? tyee
1)
IS the statement incorrect? ryas
Figure 26: Prompt used to score the question relevance of retrieved microtheory statements. Instructions are based on the rubric from Jansen et al. 2021.
26


Reviews:
[{"review_id": "ygDBYcfjz5", "reviewer": "Reviewer_qyBe", "summary": "This paper presents a novel approach to enhancing question-answering systems by distilling a model's topical knowledge into what the authors term \"microtheories.\" The method involves generating a set of simple, logical facts about the world, which are used to construct entailment chains that support the answering of complex queries. The paper emphasizes the importance of creating relevant and context-specific facts, which can be either generic statements about the world or specific assertions that aid in proving a particular query. The authors introduce a system that generates query statements for each possible answer to a question, ensuring these queries align closely with the answer choices. The effectiveness of this approach is evaluated through expert-annotated task relevance scores, demonstrating that microtheory-based facts are more relevant than random or usage-based facts, particularly when constrained by a fact budget. This work aims to improve the reasoning capabilities of AI systems by providing a structured framework for knowledge representation and retrieval, ultimately leading to more accurate and grounded question answering.", "questions": "1. How is the *grounding* metric in section 5.1 calculated in detail?\n2. How do you detach the improvements brought by *microtheory* and that possibly brought by seemingly useful LLM-generated chain-of-thought?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper presents a novel concept of generating microtheories from models. By utilizing advanced models like GPT-4 and Mixtral-8x22B-Instruct-v0.1, the authors demonstrate a robust approach that is not dependent on a single closed model. This flexibility and innovation in methodology can potentially lead to more accurate and contextually relevant answers in reasoning tasks.\n\n2. Comprehensive evaluation. This thorough evaluation process ensures that the proposed method is not only theoretically sound but also practically effective on three domains. The focus on optimizing microtheory contents given a size budget highlights the paper's attention to practical constraints and efficiency, making it a valuable contribution to the field.", "weaknesses": "- Line 74: The *entailment* in NLP typically is typically referred to as textual entailment, which surely has a formal [definition](https://en.wikipedia.org/wiki/Entailment_(linguistics)). \n- Line 160-161, the method that you extract is using LLM prompting. The problem is, what's the inherent difference of your method compared to chain/tree/graph-of-thought, since you are both distilling knowledge from LLM as intermediate steps. I get it that you used the tricks like entailment consolidation to filter some meaningful microtheories, but there are many existing works focusing on filtering intermediate thoughts or supporting facts to facilitate CoT, which limits the novelty of your work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "guOZYD2URX", "reviewer": "Reviewer_4U6H", "summary": "The paper proposes a framework of distilling a model’s knowledge of a certain scientific topic into an inspectable, verifiable form of microtheories, and shows that adding microtheories in the original corpus improves entailment engine performance.", "questions": "1. I am confused about the \"training questions\", do you mean questions in the original training set of ARC and MedQA? Also, the word \"training\" appeared in this paper so many times in different contexts, \"training questions\" \"training hypothesis\" \"training set\" \"training proofs\"... but it seems like you are not training the QA module at all?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses an important problem of distilling a model’s knowledge of a certain topic into an inspectable, verifiable form\n2. The method proposed by the paper can extract facts that are deemed as critically relevant by human experts\n3. The distilled facts are helpful for entailment engines to perform QA tasks", "weaknesses": "1. The format of the input to the entailment engine is unclear. The engine takes in a mixture of original corpus such as Wikipedia and microtheories, but how these external information is structured is not clear.\n2. The claim \"Mt size correlates with QA accuracy\" in Section 5.2 seems to be an overstatement, as the authors only show three bins (100, 500, 1000).\n3. The fact that only adding more than 1000 facts can increase QA performance shows that current microtheory selection is unable to get the most relevant facts. In addition, 5000/8000 of facts for both datasets are never used in the proof. The authors should discuss 1) why it's happening and 2) the compute needed to retrieve a large enough fact corpus to improve QA performance, and the trade off between selection comprehensiveness and QA performance.\n4. Although the paper claims to generate microtheory from both GPT4 and Mixtral to show that their method not only works on closed models, there's no comparison of performance and discussion on how different models may affect microtheory generation. The paper also does not explain why they use GPT4 for ARC and Mixtral for MedQA.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "omptWGLGxI", "reviewer": "Reviewer_AcjP", "summary": "This paper presents a method for transforming a model's latent topical knowledge into a microtheories, encapsulating reusable knowledge about a topic. The authors also conducted evaluations demonstrating that, when incorporated into the general corpus, microtheories significantly enhance the model's grounding ability in its answers.", "questions": "1. In Figure 7, adding 10 random microtheories to the base corpus could make the performance drops a lot, could you please explain it?\n\n2. See weakness 2", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes an innovative approach to distill microtheories from a model, that aritculates the model's core knowledge about the dataset's topic.\n2. The author conducted comprehensive evaluations, including human evaluation to show that the microtheories can both improve the model's ability to ground its answer and improve accuracy.", "weaknesses": "1. Dependence on Training Questions and Answer Options: The method relies heavily on a pre-existing set of training questions and answer options to generate microtheories such as ARC. This dependency restricts its applicability to new corpora that lack such structured data, making the approach challenging to transfer across domains or to corpora where questions and answers aren't predefined.\n\n2. The paper could indeed improve by including more details on the filtering process in Section 3.2.2. Specifically, it doesn’t clarify how many microtheories were ultimately filtered out after applying the redundancy reduction techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)