SlowGuess/NAACL_2022_OCR huggingface dataset:
Peter West†‡* Chandra Bhagavatula‡ Jack Hessel‡ Jena D. Hwang‡  
Liwei Jiang†‡ Ronan Le Bras‡ Ximing Lu†‡ Sean Welleck†‡ Yejin Choi†‡*  
†Paul G. Allen School of Computer Science & Engineering, University of Washington  
‡Allen Institute for Artificial Intelligence
The common practice for training commonsense models has gone from human-to-corpus-to-machine: humans author commonsense knowledge graphs in order to train commonsense models. In this work, we investigate an alternative, from-machine-to-corpus-to-machine: general language models author these commonsense knowledge graphs to train commonsense models.
Our study leads to a new framework, **Symbolic Knowledge Distillation.** As with prior art in Knowledge Distillation (Hinton et al., 2015), our approach uses larger models to teach smaller models. A key difference is that we distill knowledge symbolically—as text—in addition to the resulting neural model. We distill only one aspect—the commonsense of a general language model teacher, allowing the student to be a different type of model, a commonsense model. Altogether, we show that careful prompt engineering and a separately trained critic model allow us to selectively distill high-quality causal commonsense from GPT-3, a general language model.
Empirical results demonstrate that, for the first time, a human-authored commonsense knowledge graph is surpassed by our automatically distilled variant in all three criteria: quantity, quality, and diversity. In addition, it results in a neural commonsense model that surpasses the teacher model's commonsense capabilities despite its 100x smaller size. We apply this to the ATOMIC resource, and will share our new symbolic knowledge graph and commonsense models<sup>1</sup>.
Prior works have suggested that pre-trained language models possess limited understanding of commonsense knowledge (Merrill et al., 2021; Talmor et al., 2021; Davis and Marcus, 2017) despite
<sup>1</sup>We will share this following the anonymity period. We have permission from OpenAI to release GPT-3 generations
Figure 1: Symbolic knowledge distillation extracts the commonsense from the large, general language model GPT-3, into 2 forms: a large commonsense knowledge graph  $\mathbf{ATOMIC^{10x}}$ , and a compact commonsense model  $\mathbf{COMET_{TIL}^{DIS}}$ . The quality of this knowledge can be controlled and improved by adding a critic model, making GPT-3 a stronger teacher.
otherwise stellar performance on leaderboards. As a result, symbolic commonsense knowledge graphs (Speer et al., 2017; Sap et al., 2019; Hwang et al., 2021) and corresponding neural representations (Bosselut et al., 2019; Hwang et al., 2021; Zhang et al., 2020b) have supplemented past models with commonsense capabilities. This has enabled diverse downstream applications, including interactive learning through a conversational interface (Arabshahi et al., 2021), persona- and affect-aware conversation models (Kearns et al., 2020), figurative language understanding (Chakrabarty et al., 2020, 2021), story telling (Ammanabrolu et al., 2021a) and fantasy games (Ammanabrolu et al., 2021b).
The common practice for commonsense knowledge graph construction sees humans spell out as many pieces of knowledge as possible. This pipeline goes from-human-to-corpus-to-machine, with commonsense models trained from human
Human Language Technologies, pages 4602 - 4625
authored knowledge graphs. Yet, high-quality, human-authored knowledge is expensive to scale, limiting coverage; this motivates an alternative: from-machine-to-corpus-to-machine. Prior efforts toward automatic commonsense knowledge graphs have resulted in considerably lower quality than human-written data (Hwang et al., 2021; Zhang et al., 2020b), which in turn leads to less reliable neural models (Hwang et al., 2021). Broad literature consistently shows machine-authored knowledge graphs underperform human-authored graphs (Etzioni et al., 2011; Mitchell et al., 2015; Bollacker et al., 2008).
In this work, we propose Symbolic knowledge distillation, a new conceptual framework towards high-quality automatic knowledge graphs for commonsense, leveraging state-of-the-art models and novel methodology. Most prior art for automatic knowledge graph construction extracts knowledge from raw text (Bhakthavatsalam et al., 2020; Zhang et al., 2020a; Zhou et al., 2020; Zhang et al., 2020b; Li et al., 2020). In contrast, our approach is motivated by knowledge distillation (Hinton et al., 2015) wherein a larger teacher model transfers knowledge to a compact student model ( $\S 2.1$ ). Our method differs from prior knowledge distillation in key ways: we distill a symbolic knowledge graph (i.e., generated text) in addition to a neural model, and we distill only a selective aspect of the teacher model. This selectively allows the student model to be of a different type (commonsense model), compared to the teacher (general language model), enriching the scope of distillation. An added benefit is that knowledge distilled as text is human readable: it can be understood and evaluated.
A general language model-GPT-3 in our case-is an imperfect commonsense teacher on its own, and the ability to evaluate distilled knowledge is useful in improving it. We empirically demonstrate that, by training a separate critic model to judge symbolic generation quality, a more precise teacher can be defined. Knowledge from this critical teacher is higher quality-even exceeding human-authored knowledge. Yet even before training a critic, our study makes the unexpected finding that the student model surpasses the commonsense of GPT-3, our knowledge source.
To test symbolic knowledge distillation against the human-to-corpus-to-machine paradigm, we compare with  $\mathrm{ATOMIC}_{20}^{20}$  (Hwang et al., 2021), which is a human-authored commonsense knowl
edge graph. We find that  $\mathbf{ATOMIC^{10x}}$ , our machine-generated corpus, exceeds the human generated corpus in scale, accuracy, and diversity with respect to 7 commonsense inference types that we focus on in this study. The resulting commonsense model,  $\mathbf{COMET_{TIL}^{DIS}}$ , not only surpasses the human-trained equivalent  $\mathbf{COMET}_{20}^{20}$ , but is also smaller, more efficient, and produces commonsense at a higher accuracy than its own teacher-GPT-3.
Symbolic knowledge distillation offers a promising new role for general language models, as commonsense knowledge sources, and humans, as small-scale evaluators to train critic models rather than authors of commonsense knowledge. Our work demonstrates that humans and LMs can be effective collaborators for curating commonsense knowledge graphs and training efficient and performant commonsense models.
2 Overview and Key Findings
Throughout our work, we describe the machine-to-corpus-to-machine methodology of symbolic knowledge distillation. We first go machine-to-corpus (§3), by decoding from GPT-3, then improve our knowledge with a specialized critic model (§4), and finally distill this knowledge into an efficient commonsense model (§5), going corpus-to-machine. Throughout this process, we evaluate against a human knowledge source, comparing our automatic knowledge graph  $\mathrm{ATOMIC^{10x}}$  and commonsense model  $\mathrm{COMET_{TIL}^{DIS}}$  to the human-authored  $\mathrm{ATOMIC}_{20}^{20}$  and resulting model  $\mathrm{COMET}_{20}^{20}$  (Hwang et al., 2021).
2.1 Symbolic Knowledge Distillation
Our proposed methodology parallels knowledge distillation (Hinton et al., 2015), a method for compressing a large or complicated teacher distribution  $P_{t}$  into a smaller/simpler student distribution  $P_{s}$ . Key to knowledge distillation is the notion of minimizing the cross-entropy between  $P_{t}$  and  $P_{s}$ :
$$
H \left(P _ {t}, P _ {s}\right) = - \sum_ {y \in Y} P _ {t} (y) \log P _ {s} (y) \tag {1}
$$
Knowledge is transferred to the student by encouraging it to match teacher predictions. Hinton et al. (2015) apply this to conditional classification: for
2In its simplest case, with temperature set to 1.0
X starts running	xEffect so, X	gets in shape	X sings a song	HinderedBy but not if	X can't remember the lyrics
X and Y engage in an argument	xWant so, X wants	to avoid Y	X is not well liked	xReact so, X feels	lonely
X learns to type fast	xNeed X needed	to have taken typing lessons	X takes care of a monkey	xAttr X is seen as	kind
X steals his grandfather's sword	xEffect so, X	is punished by his grandfather	X butts in	HinderedBy but not if	X is too shy to speak up
X takes up new employment	xIntent because X wants	to be self sufficient	X waits for the storm to break	xEffect so, X	is safe from the storm
Figure 2: Example automatically generated ATOMIC triples from our ATOMIC $^{10x}$  commonsense knowledge graph. Each example includes a generated event, relation (with natural language interpretation), and generated inference.
each training input,  $P_{t}$  and  $P_{s}$  are model predictions over label set  $Y$ . Typically  $Y$  is a tractable set, over which this sum can reasonably be calculated.
For distilling the knowledge of generative models, we can think of an unconditional language model (LM e.g. GPT-3) as  $P_{t}$ . This makes  $Y$  the set of all strings, over which LMs define probability. Unfortunately  $Y$  is an exponential set, intractable to sum over in Eq 1. Kim and Rush (2016) address this problem by simply taking the mode of  $P_{t}$  over  $Y$ , truncating most of the teacher distribution to the most likely sequence and discarding information.
Instead, we consider a sampling-based interpretation of the same objective:
$$
H \left(P _ {t}, P _ {s}\right) = \underset {y \sim P _ {t} (y)} {\mathbb {E}} [ - \log P _ {s} (y) ] \tag {2}
$$
which exactly equals the cross-entropy of Eq 1, at the limit under pure sampling from  $P_{t}$ .<sup>3</sup>
Yet distilling all knowledge from the teacher may not be desirable—our work is specifically focused on distilling commonsense knowledge from GPT-3. The ideal teacher  $P_{t}$  is a commonsense expert, but GPT-3 can approximate such a teacher, off-the-shelf, via prompting. This ability to select information is one explicit benefit of the sampling-based interpretation of Eq 2: while Eq 1 uses continuous logits over existing data, sampling gives discrete control over transferred information, by selecting which samples are elicited and used. For the general language model GPT-3, We encourage domain/quality with prompting, and sample truncation (Holtzman et al., 2020). We call this the loose teacher  $P_{t}^{L}$ -knowledge is generated and transferred from GPT-3, but without critical assessment of correctness ( $\S 3$ ).
In fact, sampling knowledge in Eq 2 offers even more control, as generations can be individually interpreted and judged. Given an indicator function  $A(x)$  for which knowledge  $x$  is correct, we can define a stronger teacher model. Using a Product of Experts (Hinton, 2002) between the loose teacher  $P_{t}^{L}$  and and the critic  $A(x)$ , we define a critical teacher:
$$
P _ {t} (x) \propto P _ {t} ^ {L} (x | p) \cdot A (x) \tag {3}
$$
In practice,  $A(x)$  is a textual classifier learned on human judgements, 1 for knowledge predicted to be correct and 0 otherwise. Thus, the critic gives control over the correctness and confidence of the knowledge that is transferred (§4).
2.2 Key Findings
Applying symbolic knowledge distillation in practice results in promising and surprising findings:
1. Learning symbolic knowledge from language models can be framed as a symbolic extension to knowledge distillation. In §2.1, we describe learning commonsense as a symbolic extension to knowledge distillation, with GPT-3 a knowledge source. We elaborate on this process with positive results in §3,4, and 5.
2. Symbolic knowledge distillation constructs a high quality knowledge graph at scale. Our method naturally yields a machine-generated commonsense knowledge graph, which can achieve impressive quality (§4), beyond that of human-authored data. An effective critic which filters incorrect generated knowledge is key.
3. A critical teacher results in a higher quality student. In §4, we show that making the teacher more critical results in higher quality knowledge, even as it reduces the scale of knowledge transferred. This demonstrates that quality matters, not
3 A useful consequence of this framing is that access to the full model distribution is not required. Our experiments (§3) use GPT-3, for which the distribution is not available, thus our method is applicable while knowledge distillation is not.
just quantity, as higher quality knowledge results in a higher quality commonsense model in §5 despite smaller scale data.
4. Critical teacher or not, a student can outperform the knowledge source. In §5, we show the unexpected result that all student models exceed the quality of GPT-3, the knowledge source.
5. Machines can win over humans for automatic knowledge graph construction. In §4 and §5, we show that machine generated knowledge and the resulting commonsense model can outperform their equivalents that use a human knowledge source. Our symbolic knowledge exceeds humans at scale, quality, and diversity. The resulting commonsense model achieves the most accurate commonsense KG completions.
3 Machine-to-Corpus Verbalization
Symbolic knowledge distillation begins by going machine-to-corpus, i.e. generating many commonsense facts, which results in a commonsense knowledge graph. §2.1 frames this as sampling to estimate the knowledge distillation objective-a student commonsense model learns from the generations of a teacher (GPT-3).
We start with a loose teacher, transferring knowledge by prompted generation with truncated sampling alone—this is in contrast to the critical teacher (§4) which explicitly judges and filters the generated samples. The loose teacher uses few-shot prompting as in Brown et al. (2020). We use a few-shot template:
where  $<\mathrm{EX}_i - \mathrm{INP}> / <\mathrm{EX}_i - \mathrm{OUT}>$  are human-authored, natural language ATOMIC entries, and  $<\mathrm{TASK-PROMPT}>$  is a description of the problem. Given such a prompt, GPT-3 generates the missing piece, output  $<\mathrm{EX}_N - \mathrm{OUT}>$  for input  $<\mathrm{EX}_N - \mathrm{INP}>$ , following the pattern of earlier examples (1 to N-1). We find important aspects for producing high-quality commonsense knowledge:
Examples should be numbered. e.g. <EX5-INP> might begin with "5") to indicate it is the 5th example.
- The format of  $\langle \mathrm{EX}_i - \mathrm{INP} \rangle$  and  $\langle \mathrm{EX}_i - \mathrm{OUT} \rangle$  should linguistically imply the relationship between them. See below for examples.
- <TASK-PROMPT> can be used to give extra specification to complicated problems.
3.1 Data: ATOMIC
We demonstrate symbolic knowledge distillation on the ATOMIC if-then resource (Sap et al., 2019). This follows an event-relation-inference (triple) format. The corpus links events (e.g.  $X$  attacks  $Y$ ) to relations, e.g. HinderedBy which describes what might hinder an event. For a relation/event, the goal is to generate a resulting inference, e.g.  $X$  attacks  $Y$  HinderedBy  $X$  is restrained.
Of the 23 relations from the most recent version-  $\mathrm{ATOMIC}_{20}^{20}$  we limit our investigation to 7 relations that correspond to causal commonsense knowledge: xAttr (how X is perceived after event), xReact (how X reacts in response to event), xEffect (what X does after event), xIntent (X's intent in event), xWant (what X wants after event), xNeed (what X needed for event to take place) and HinderedBy. We describe how verbalization is applied to ATOMIC data in 2 steps: generating underlying events (heads), then full examples (inference given event).
3.2 Event Generation
Events are context-free premises in ATOMIC involving PersonX (and sometimes a second PersonY) in various scenarios. These events form heads in knowledge graph triples. We generate events by filling in the elements of our template:
1. Event: X overcomes evil with good
2. Event: X does not learn from Y
···
10. Event: X looks at flowers
11.
The format is simple, as events are generated unconditionally. We use 100 high-quality events from the  $\mathrm{ATOMIC}_{20}^{20}$  corpus for our prompt, selected to avoid grammatical or logical errors, and minimize semantic overlap. We randomly sample 10 of these seed events for each generation batch, resulting in randomized prompts. We use nucleus sampling  $(p = 0.9)$  (Holtzman et al., 2020), and presence/frequency penalties of 0.5 from the GPT-3 interface. We generate 165K unique events using the 175B-parameter Davinci model<sup>4</sup> from Brown
the largest available version of GPT-3
et al. (2020) (human-authored  $\mathrm{ATOMIC}_{20}^{20}$  contains only 6.2K events).
3.3 Inference Generation
Generating ATOMIC inferences requires reasoning about events and relations together. We design verbalization templates for reach relation, with iterative design and small-scale verification by the authors<sup>5</sup> e.g. we prompt the xNeed relation as follows:
What needs to be true for this event to take place?
Event  $<  i>$  ： X goes jogging
Prerequisites: For this to
happen, X needed to wear running shoes
Event  $<  \mathrm{N}>$  : X looks at flowers
Prerequisites: For this to happen,
The language of this template implies the relation-specific task, both "Prerequisites:" and beginning with "for this to happen" suggest the xNeed relation. As well, we include an xNeed-specific <TASK-PROMPT>. We use 10 few-shot examples for each prompt.<sup>6</sup>
For each event/relation (165K X 7) we generate 10 inferences with the Curie GPT-3 model and earlier hyperparameters. Removing duplicate and degenerate (e.g. fewer than 3 characters) generations yields 6.46M ATOMIC-style data triples (examples in Figure 2). We call this  $\mathrm{ATOMIC^{10x}}$  as it contains an order of magnitude more triples than  $\mathrm{ATOMIC_{20}^{20}}$  for the 7 relations we study.
3.4 Evaluating a Generated Commonsense Knowledge Graph
Machine generation enables a large scale of unique generations at a much lower cost than human-authored knowledge (Table 1), but what kind of examples are produced by GPT-3, and how does it differ from knowledge produced by humans? In this section, we conduct an in-depth analysis to answer these questions.
Relation	\( \mathbf{A T O M I C_{20}^{20}} \)	\( \mathbf{A T O M I C^{10x}} \)
HinderedBy	77,616	1,028,092
xNeed	100,995	760,232
xWant	109,098	730,223
xIntent	54,839	965,921
xReact	62,424	1,033,123
xAttr	113,096	884,318
xEffect	90,868	1,054,391
Total Count	608,936	6,456,300
Est Total Cost	~$40,000	~$6,000
Est Cost Per Triple	~$0.06	~$0.001
Table 1: Number of unique triples with the given relation,  $\left| {\left( {\cdot ,\text{relation}, \cdot  }\right) }\right|$  . The estimated cost for ATOMIC  ${}^{{10}\mathrm{x}}$  comes at a fraction of a conservative estimation for ATOMIC  ${}_{20}^{20}$  crowdsourcing costs.
**Lexical Differences:** Diversity and Uniqueness  
Recent work finds that machine generations can be repetitive and lack diversity (Welleck et al., 2020; Holtzman et al., 2020); one way generated knowledge may differ from human-authored is less creative word choice, diversity, or more repetition.
To test this, we begin with lexical diversity (i.e. unique words used, Table 2). While there is variation by relation, the diversity of  $\mathrm{ATOMIC^{10x}}$  actually exceeds  $\mathrm{ATOMIC_{20}^{20}}$  here,  $5.2\mathrm{M}$  unique words to  $1.5\mathrm{M}$ . In addition, it contains significantly more strictly unique generated inferences (Table 2, unique tails).
BLEU Soft Uniqueness. Exact match (above) fails to capture the notion of similar text. Following the intuition of self-BLEU (Zhu et al., 2018), we define soft uniqueness to describe diversity of generations in a corpus. An inference  $x$  is softly-unique if:
$$
B L E U _ {2} (C, x) <   0. 5
$$
where  $C$  is the set of inferences for a given input (in our case, event + relation), and 0.5 is an empirical threshold. To find soft-uniqueness of a corpus, we iteratively remove examples until all are softly unique, i.e. low mutual lexical overlap; higher diversity means more such examples (thus a larger softly unique corpus is preferable). Softly-unique corpus sizes are given in Table 4 ("Size (div)").  $\mathrm{ATOMIC^{10x}}$  has a smaller fraction of softly-unique examples than  $\mathrm{ATOMIC_{20}^{20}}$ , yet it contains many more such examples.  $\mathrm{ATOMIC^{10x}}$  contains 4.38M such examples (full size 6.5M) vs.  $\mathrm{ATOMIC_{20}^{20}}$ , which has 560K (full size 600K).
See Appendix D for full prompts.
We also replace anonymous names ("X") with sampled generic names as this improved quality, See Appendix D. Once generation is complete, we substitute in generic markers ("X") for the final dataset.
for the largest, Davinci, 12M generations is computationally/monetarily intractable.
Length	Unique Tokens (K)	Unique Tails (K)
A20	A10x	A20	A10x	A20	A10x
xWant	4.69	5.16	322	784	69	152
xAttr	1.42	2.73	15	21	11	8
xEffect	3.92	4.66	216	864	55	185
xIntent	4.59	5.92	136	800	30	135
xNeed	4.51	5.97	289	1378	64	231
xReact	4.03	1.77	48	5	12	2
HinderedBy	7.93	7.49	522	1775	290	874
Events	5.20	5.32	109	881	6.2	165
Table 2: Average length, total unique tokens and total unique examples (in K, i.e. 1000s) by relation type and in events (bottom row) from ATOMIC $^{20}$  ( $A_{20}^{20}$ ) and ATOMIC $^{10x}$  ( $A^{10x}$ ).
Entropy	Cross Entropy	KL Divergence
H(D1) = 1.27	H(D1, D2) = 9.31	DKL(D1||D2) = 8.04
H(D2) = 7.80	H(D2, D1) = 41.48	DKL(D2||D1) = 33.68
Model-based Diversity Measurement. Lexical notions of diversity reward differences in surface form, which may not always reflect diversity of information, only format. Thus, we next study information-theoretic measures for diversity. Intuitively, diverse information should be less predictable, or higher entropy. With GPT-2 XL models finetuned on  $\mathrm{ATOMIC}_{20}^{20}$  and  $\mathrm{ATOMIC}^{10x}$  (\$5) we estimate entropy—roughly, how difficult it is for a model to capture the corpus information (Table 3). This is 4 times higher for  $\mathrm{ATOMIC}^{10x}$ , suggesting more content from a modeling perspective. We also estimate cross-entropy—how well a model trained on one corpus describes the other. From  $\mathrm{ATOMIC}^{10x}$  to  $\mathrm{ATOMIC}_{20}^{20}$ , this is 9.31, only 2 points higher than its entropy suggesting  $\mathrm{ATOMIC}_{20}^{20}$  is describable with information from  $\mathrm{ATOMIC}^{10x}$ . In reverse, this is 41.48 suggesting much of  $\mathrm{ATOMIC}^{10x}$  is not captured by  $\mathrm{ATOMIC}_{20}^{20}$ – $\mathrm{ATOMIC}^{10x}$  is surprising given only information from  $\mathrm{ATOMIC}_{20}^{20}$ .
Human Evaluation of Quality. Perhaps most importantly, we study the quality of knowledge in each corpus. We conduct human evaluation with Amazon Mechanical Turk. 3 annotators rate each triple resulting in "accepted", "rejected" or "no judgement". We evaluate 3000 examples from
Table 3: Entropy, cross-entropy, and divergence of  $\mathrm{ATOMIC}_{20}^{20}(D_1)$  and  $\mathrm{ATOMIC^{10x}}(D_2)$ .
Corpus	Accept	Reject	N/A	Size	Size (div)
\( \text{ATOMIC}_{20}^{20} \)	86.8	11.3	1.9	0.6M	0.56M
\( \text{ATOMIC}_{20}^{10x} \)	78.5	18.7	2.8	6.5M	4.38M
88.4	9.5	2.1	5.1M	3.68M
(criticlow)	91.5	6.8	1.7	4.4M	3.25M
95.3	3.8	1.0	3.0M	2.33M
(critichigh)	96.4	2.7	0.8	2.5M	2.00M
+ GPT-J	72.0	27.6	0.4	-	-
+ T5-11B LM	71.7	26.9	1.4	-	-
Table 4: Attributes of ATOMIC $^{10x}$  and ATOMIC $^{10x}$  (row 2) including the critic model ( $\S 4$ , rows 3 - 6) with various filtering cutoffs. Accept and Reject are by majority human vote unless any mark N/A. Size is in unique examples $^9$ . The highest precision corpus is ATOMIC $^{10x}$  with (critic $_{\text{high}}$ ), but multiple versions surpass ATOMIC $_{20}$ $^{20}$ . We also include alternate models (GPT-J and T5-11B) as the loose teacher.