Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (2022)

Abstract: The common practice for training commonsense models has gone from–human–to–corpus–to–machine: humans author commonsense knowledge graphs in order to train commonsense models. In this work, we investigate an alternative, from–machine–to–corpus–to–machine: general language models author these commonsense knowledge graphs to train commonsense models. Our study leads to a new framework, Symbolic Knowledge Distillation. As with prior art in Knowledge Distillation (Hinton et al. 2015), our approach uses larger models to teach smaller models. A key difference is that we distill knowledge symbolically–as text–in addition to the neural model. We distill only one aspect–the commonsense of a general language model teacher, allowing the student to be a different type, a commonsense model. Altogether, we show that careful prompt engineering and a separately trained critic model allow us to selectively distill high-quality causal commonsense from GPT-3, a general language model. Empirical results demonstrate that, for the first time, a human-authored commonsense knowledge graph is surpassed by our automatically distilled variant in all three criteria: quantity, quality, and diversity. In addition, it results in a neural commonsense model that surpasses the teacher model’s commonsense capabilities despite its 100x smaller size. We apply this to the ATOMIC resource, and will share our new symbolic knowledge graph and commonsense models.

## introduction prior works have suggested that pre-trained language models possess limited understanding of commonsense knowledge @xcite @xcite ) despite 1 we will share this following the anonymity period. we have permission from openai to release gpt-3 generations ## overview and key findings throughout our work, we describe the machineto-corpus-to-machine methodology of symbolic knowledge distillation. we first go machine-tocorpus ( §3), by decoding from gpt-3, then improve our knowledge with a specialized critic model ( §4), and finally distill this knowledge into an efficient commonsense model ( §5), going corpus-to-machine. throughout this process, we evaluate against a human knowledge source, comparing our automatic knowledge graph atomic 10x and commonsense model comet dis til to the humanauthored atomic 20 20 and resulting model comet 20 20 @xcite . ## machine-to-corpus verbalization symbolic knowledge distillation begins by going machine-to-corpus, i.e. generating many commonsense facts, which results in a commonsense knowledge graph. §2.1 frames this as sampling to estimate the knowledge distillation objective-a student commonsense model learns from the generations of a teacher (gpt-3). we start with a loose teacher, transferring knowledge by prompted generation with truncated sampling alone-this is in contrast to the critical teacher ( §4) which explicitly judges and filters the generated samples. the loose teacher uses few-shot prompting as in @xcite . we use a few-shot template: ## making the teacher more critical symbolic knowledge distillation requires a strong teacher model to maximize the quality of the generated knowledge graph and resulting student model ( §5). while the loose teacher (gpt-3 alone) results in a viable commonsense knowledge graph, evaluation shows this isn't a perfect commonsense teacher. thus, we multiply in a critic model, to filter lower-quality knowledge, correcting the teacher ( §2.1). with modest supervision (a small-scale human evaluation) we train a classifier to predict and discriminate unacceptable examples. we multiply this with the loose teacher §3, creating a critical teacher product of experts. in practice this means filtering atomic 10x to create new corpora that are higher quality, yet still larger scale than humanauthored atomic 20 20 . training a knowledge critic we gather a training set of correct vs. incorrect human judgments on a randomly-sampled set of 10k entries of atomic 10x , as in §3.4 but with one annotation per example. we take a (random) train/dev/test split of 8k/1k/1k. while this step requires human annotation, humans take on the role of high-level supervisors here-critiquing a small number of generations rather than authoring the entire knowledge graph as in previous work. indeed, the cost/complexity of this step is similar to a typical human evaluation, making it far cheaper/easier than eliciting humanauthored knowledge in past work. we train binary classifiers (critics) for human acceptability using roberta-large @xcite . we find pretraining on mnli results in the best model in terms of precision and recall, and we suggest this technique for future studies. we give more detail in appendix c, including baselines. our best model vastly improves the accuracy of atomic 10x ( size-accuracy trade-off using our critic to filter knowledge results in a natural trade-off between size and accuracy. we test several cutoffs for atomic 10x , i.e. confidence at which the critic rejects examples. we report humanmeasured accuracy (accept/reject column to understand what is filtered, we ablate the critic ( we find gpt-3 produces both independent awkwardly-phrased events/inferences (filtered by x-only models) and logical misalignments. the classifier, trained on validated knowledge triples, helps in both cases. the emap of our full model (identifies only awkward phrasings) achieves 87% ap, and our full model (which additionally identifies logical misalignments) improves to 94% ap. does filtering hurt diversity? one concern is that the critic may keep only similar "safe" examples, lacking novelty. we repeat our diversity analysis ( §3.4) for critical corpora ( ## corpus-to-machine: distillation the final step of symbolic knowledge distillation trains a compact model on the generated natural language knowledge graph. our base model is gpt2-xl trained on all of atomic 10x : we denote this model by comet dis til . we additionally train the model on critical versions of atomic 10x -crit low denotes training on the corpus achieving 91.5% accuracy, and crit high on the 96.4% accuracy corpus. models are trained for 1 epoch, with default parameters using the huggingface transformers library @xcite . ## related work commonsense knowledge graphs (ckg) ckgs provide knowledge for commonsense reasoning. some are manually constructed, e.g. 11 the slight difference in acceptability for gpt-3 from atomic @xcite . conceptnet @xcite contains taxonomy and physical commonsense, authored by humans or compiled from such sources. some ckgs are automatically constructed: transomcs @xcite extracts 18.48m tuples from syntactic parses and causalbank @xcite extracts 314m cause-effect pairs by pattern-matching. in contrast, we generate commonsense. extracting knowledge from lms past work uses models for automatic knowledge graph completion @xcite @xcite . yet, models are trained on existing resources; atomic 10x is generated without these. other works mine factual/commonsense knowledge directly from off-the-shelf lms @xcite @xcite , but not resulting in the quality at scale of atomic 10x . ## conclusions we introduce symbolic knowledge distillation, a machine-to-corpus-to-machine pipeline for commonsense that does not require human-authored knowledge-instead, using machine generation. knowledge is transferred from a large, general model to a compact commonsense model, through a commonsense corpus-yielding a commonsense knowledge graph and model. our resulting symbolic knowledge graph has greater scale, diversity, and quality than human authoring. symbolic knowledge distillation offers an alternative to humanauthored knowledge in commonsense research.