Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning (2022)

Abstract: Commonsense generation aims to generate a realistic sentence describing a daily scene under the given concepts, which is very challenging, since it requires models to have relational reasoning and compositional generalization capabilities. Previous work focuses on retrieving prototype sentences for the provided concepts to assist generation. They first use a sparse retriever to retrieve candidate sentences, then re-rank the candidates with a ranker. However, the candidates returned by their ranker may not be the most relevant sentences, since the ranker treats all candidates equally without considering their relevance to the reference sentences of the given concepts. Another problem is that re-ranking is very expensive, but only using retrievers will seriously degrade the performance of their generation models. To solve these problems, we propose the metric distillation rule to distill knowledge from the metric (e.g., BLEU) to the ranker. We further transfer the critical knowledge summarized by the distilled ranker to the retriever. In this way, the relevance scores of candidate sentences predicted by the ranker and retriever will be more consistent with their quality measured by the metric. Experimental results on the CommonGen benchmark verify the effectiveness of our proposed method: (1) Our generation model with the distilled ranker achieves a new state-of-the-art result. (2) Our generation model with the distilled retriever even surpasses the previous SOTA.

## introduction commonsense reasoning is the ability to make reasonable and logical assumptions about daily scenes, which is a long-standing challenge in natural language processing. recently, many discriminative tasks, such as commonsenseqa @xcite and swag @xcite , have been proposed to evaluate the commonsense reasoning abil- commongen proposes two main challenges to models, and it expects models to (1) reason over the commonsense relations among concepts to generate sentences in line with our commonsense; (2) possess the compositional generalization ability to generate realistic sentences with unseen concept compositions. experiment results @xcite show that large-scale pre-trained models (e.g., bart) alone is not competent for this task (see to enrich the source information, eki-bart @xcite ) first retrieves prototype sentences for the input concepts, and then feeds the concepts and retrieved sentences into the generation model. recent work, such as re-t5 @xcite , kfcnet @xcite , and kgr 4 @xcite , extends this retrieve-and-generate framework by introducing a binary classifier to re-rank the retrieved candidate sentences and filter out candidates irrelevant to the input concepts. one problem with these works is the discrepancy between training and re-ranking for their ranker. concretely, when training the ranker, they treat all retrieved candidate sentences as negatives, regardless of their relevance to the reference sentences of input concepts. however, during re-ranking, the ranker is asked to point out how these candidates differ in their relevance to references. another problem is that the re-ranking process of the cross-encoder ranker is very time-consuming, which is non-negligible, especially for online systems. in this paper, we also resort to the retrieve-andgenerate pipeline to solve commongen, yet further improve the retrieval module by alleviating the above problems. our motivation is to expect that the relevance scores of candidates computed by the ranker and retriever are in line with the gold quality scores between candidates and reference sentences measured with the evaluation metric. to achieve this, we first distill the gold rank knowledge of candidates measured by the metric to the ranker. next, we improve the retriever by transferring the metric knowledge from the distilled ranker to the retriever rather than directly distilling it from the metric (please refer to section 3.3 for more explanation). by doing so, the distilled ranker and retriever can select more relevant sentences than their counterparts without metric distillation. the contributions of this work are summarized as follows: (1) we propose to distill knowledge from the metric to ranker and retriever, termed dkmr 2 , for generative commonsense reasoning, which uses the metric-guided distillation to improve the ranker and a progressive distillation strategy to improve the retriever foot_0 . (2) we conduct extensive experiments on the commongen benchmark. our proposed model achieves a new state-of-the-art (sota) on both the v1.0 test set (43.37 vs. 39.15 on spice) and the official test set (v1.1) (34.589 vs. 33.911 on spice) of the leaderboard. (3) the performance of dkmr 2 with the distilled retriever is on par with dkmr 2 using the distilled ranker. as a result, the expensive retrieve-then-rank pipeline can be replaced with the distilled retriever at the expense of negligible performance. ## problem statement commongen is a constrained text generation task, with the goal of generating a coherent and plausible sentence s describing an everyday scenario us-ing an unordered concept set @xmath0, c 2 . . . , c m }. therefore, this task is typically formulated to maximize the conditional probability of s: ## methodology following the previous work @xcite , we resort to the retrieve-then-generate framework to solve commongen, which mainly consists of two modules, the retrieval module and the generation module. the retrieval module aims to retrieve relevant sentences to assist the generation module in generating desirable outputs. recent work @xcite @xcite extends this idea by introducing a ranker to the retrieval module. in this work, our retrieval module also resorts to the retrieve-then-rank pipeline, as illustrated in figure 1 . to be specific, the retrieval module mainly contains two models, the retriever and ranker, where the retriever is used to retrieve candidate sentences for the given concept set and the ranker further re-ranks the retrieved sentences. different from previous work, we improve the ranker by distilling knowledge from the gold quality scores between the candidate sentences and gold reference sentences computed by the evaluation metric. then, the distilled ranker will pass the distilled knowledge to the retriever, aiming to correct the retriever's inaccurate retrieval operations. in section 3.1, we first introduce the warm-up of the retriever. then, we will show how to distill knowledge from the metric, in turn, to the ranker and retriever in sections 3.2 and 3.3, respectively. finally, we will show how to generate sentences based on the retrieved sentences in section 3.4. ## conclusions this work presents dkmr 2 , a novel retrievalaugmented model for generative commonsense reasoning. unlike previous work, dkmr 2 enhances the retrieval module with the guidance of the evaluation metric. to be concrete, dkmr 2 first distills the order knowledge from the metric to the ranker and then teaches the key points summarized by the distilled ranker to the retriever. as a result, dkmr 2 achieves the state-of-the-art results on commongen. more importantly, dkmr 2 narrows the performance gap between the ranker and retriever, resulting in dkmr 2 with a distilled retriever being better than the previous baselines.