Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation (2022)

Abstract: Predicting the key explanation concept is essential for generating commonsense explanations. This paper introduces a method to predict the concept from pre-trained language models for commonsense explanation generation. Our experiment found that adopting a language model as the concept extractor and fine-tuning it with 20% training data can improve the quality and accuracy of the generated explanations over multiple evaluation metrics. Compared with conventional methods that search concepts over knowledge graphs, our method does not require the preparation and training models to search through knowledge graphs. To better understand the results from pre-trained language models, we also designed a metric to evaluate the retrieved concepts. Through analysis and experiments, we show the correlation between this metric and the performance of the generators, and we also show the importance of attaching concepts for generating high-quality sentences.

## introduction the natural language processing (nlp) field has gigantic advancements since the involvement of deep learning @xcite and large-size pre-training models @xcite @xcite . however, creating human-like ai systems requires models comprehending commonsense knowledge and reasoning, which is still a challenge for current nlp models @xcite . among all the research about making language models' responses fit common sense knowledge, we would like to focus on generating sentences to explain a statement with commonsense knowledge @xcite . an example of this task is shown in there are two ways for this task at present. one ## conclusion in this paper, we present a combining prompttuning and pre-trained language models concept extraction method for commonsense explanation generation task. we demonstrate that our method is more data-efficient for concept extractions than methods that select concepts from the knowledge graph. our method improves evaluation metric scores compared with baseline models. additionally, we analyze the method of selecting concepts from pre-trained language models and challenges to solve in future work.