Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation (2022)

Abstract: We present Referee, a novel framework for sentence summarization that can be trained reference-free (i.e., requiring no gold summaries for supervision), while allowing direct control for compression ratio. Our work is the first to demonstrate that reference-free, controlled sentence summarization is feasible via the conceptual framework of Symbolic Knowledge Distillation (West et al., 2022), where latent knowledge in pre-trained language models is distilled via explicit examples sampled from the teacher models, further purified with three types of filters: length, fidelity, and Information Bottleneck. Moreover, we uniquely propose iterative distillation of knowledge, where student models from the previous iteration of distillation serve as teacher models in the next iteration. Starting off from a relatively modest set of GPT3-generated summaries, we demonstrate how iterative knowledge distillation can lead to considerably smaller, but better summarizers with sharper controllability. A useful by-product of this iterative distillation process is a high-quality dataset of sentence-summary pairs with varying degrees of compression ratios. Empirical results demonstrate that the final student models vastly outperform the much larger GPT3-Instruct model in terms of the controllability of compression ratios, without compromising the quality of resulting summarization.

## introduction we introduce referee, a new framework for sentence summarization that works by iteratively generating and distilling knowledge into successively better models. this allows referee to be [refer]ence fr[ee]-beginning by distilling from a large language model rather than with supervised data. yet, our method results in a more efficient, we then use generations from referee-distill to train a model in which these aspects are controllable: this is referee-control. our work follows the paradigm of symbolic knowledge distillation @xcite , which transfers implicit knowledge from a massive language model to a considerably smaller student model by explicitly generating knowledge in textual form. unlike traditional knowledge distillation @xcite where the teacher model and the student model are of the same type, symbolic knowledge distillation allows for the student model to be of a different type. our work differs from @xcite in three key aspects. first, our distillation is iterative: each student model becomes a teacher in successive rounds, refining and improving summarization at every step. second, referee controls for more than just overall quality, improving multiple model aspects in each round such as length, fidelity, and information bottleneck @xcite , then allowing explicit length control at generation time. third, our work is the first to show that referencefree, controlled sentence summarization can be formulated as symbolic knowledge distillation. referee works in two phases, illustrated in figure 1 . first, referee-distill uses a modest number of generated summaries from gpt-3 @xcite to produce high quality and compact summarizers @xcite . we follow an iterative approach; in each iteration we filter generations for desirable qualities, re-train a new and better summarizer, and finally generate new summaries for the next round. each round amplifies effects of the previous rounds, improving notions of summary quality like entailment or shorter length. second, referee-control uses these iteratively distilled summaries to train a model with explicit control: in our experiments, we use progressively shortened generations from each iteration to train a final summarizer with explicit length control. we find that referee demonstrates compelling empirical results compared to competitive baselines. referee-distill, even without explicit length control, is able to generate shorter summaries with more consistency and equal quality compared with the original teacher model (gpt-3, 16x larger in size) as well as a supervised model. moreover, referee-control, which has more direct length control baked in, demonstrates a sharp degree of control in length, and succeeds at generating high quality summaries at specified lengths with significantly higher accuracy than gpt-3. in sum, the promising empirical results of referee encourages further future investigation to extend the framework of symbolic knowledge distillation for reference-free, controlled text summarization. ## methods we first describe referee-distill (see §2.1), an iterative procedure to promote specific behaviors that may not be prevalent in the original data, while maintaining summary quality. we explore two different filters, detailed in §2.2. we then de-tail referee-control (see §2.3), a model that separates summaries into categorical variables and is iteratively trained to, summarize a given sentence within the desired category (e.g., a range of compression ratio). in this work we only consider categories that reflect different compression ratio, but the same approach could be applied to other types of control categories, such as style. ## primal-dual problem interpretation of summarization assuming summaries are fluent and factual, sentence summaries trade off between two variables: level of compression and level of information preservation. we are able to effectively fix the level of compression by introducing control codes, and then develop models to maximize information preservation. this is our primal problem. thanks to length-control codes, we can now also solve the dual problem: "what is the best shortest summary we could write?". written more precisely, given a fixed level of tolerance for losing information from the original sentence, what is the shortest summary we could write? furthermore, comparing similar-lengthed summaries also allows for fairer comparisons, since we are effectively measuring changes in only one variable. ## on gpt3's fidelity and length control we analyze gpt3-instruct curie's @xcite -3 ). the first three rows refer to three different datasets generated through three-shot prompting gpt3-instruct curie with summaries from different compression ratios (c.r.). following rows show results for the third and last iteration of our models, using each one of the two described filters (f 1 , f 2 , see §2.2). sentences correspond to a held-out set during training. besides using prompts that encourage shorter summaries, one can iteratively summarize through few-shot prompting. if f p (s) is the summary gpt-3 generates when prompted with p, then f p (f p (. . . f p (s))) = f n p (s) may also be a summary of s, possibly shorter. we find that successive application of the same prompt did not result in shorter summaries, i.e. these experiments motivate the need of more sophisticated approaches to length control and reliably summarizing without supervision. ## experiments dataset we create the corpora d and f by sampling contiguous sentence pairs from realnews @xcite news articles. we filter out sentences shorter than 50 characters. using gpt-3 as the teacher, we summarize sentences in d 0 and use the outputs with 60-80% compression ratio as our initial dataset c 0 , since it was the best one quantitatively and qualitatively. although this implies that we will not initially have enough short sum- training details we use off-the-shelf model wanli @xcite to create the filter f nli . we run 3 iterations of referee-distill with compression ratios for each training iteration are as follows: r 1 = 0.7, r 2 = 0.5, r 3 = 0.3. all generated data is decoded via beam search (with beam width 5). all experiments with f nsp are done with @xmath0.0025, empirically decided through preliminary exploration. we train referee-control with @xmath1. for more details, refer to appendix b. while having contextual information for f nsp is useful, many applications do not have contextual information available. therefore, we use the corpus c 0 ∪ . . . ∪ c t generated using f 1 as initial training data for referee-control with the goal of increasing its applicability. ## related work unsupervised summarization the vast majority of prior work in sentence summarization assumed access to large-scale text-summary paired datasets from which to train supervised models @xcite @xcite . nonetheless, these datasets are costly to create, and naturally-occurring summarization datasets (such as news highlights) are noisy and not easily found in other domains. therefore, recent work emphasized the need for developing unsupervised or self-supervised methods such as autoencoders @xcite , but in general they lead to less fluent summaries; more recent work has explored the information bottleneck principle @xcite instead of the reconstruction loss of autoencoders. our work contributes to this emerging line of research by demonstrating an entirely different method based on symbolic knowledge distillation. ## conclusions we presented referee, a framework for sentence summarization that can be trained without reference summaries, while allowing direct control for summary compression ratio. we uniquely proposed iterative symbolic knowledge distillation, where student models from the previous iteration of distillation serve as teacher models in the next. distilled models are significantly smaller than the original teacher, gpt-3, and empirical results demonstrated that the final student models vastly outperform the much larger gpt3-instruct model in terms of the controllability of compression ratios, without compromising the resulting summaries' quality. a useful by-product of this iterative distillation process is a high-quality sentence summarization dataset with varying degrees of compression, which we will release jointly with our models upon publication.