CTRL sum: Towards Generic Controllable Text Summarization (2022)

Abstract: Current summarization systems yield generic summaries that are disconnected from users’ preferences and expectations. To address this limitation, we present CTRLsum, a generic framework to control generated summaries through a set of keywords. During training keywords are extracted automatically without requiring additional human annotations. At test time CTRLsum features a control function to map control signal to keywords; through engineering the control function, the same trained model is able to be applied to control summaries on various dimensions, while neither affecting the model training process nor the pretrained models. We additionally explore the combination of keywords and text prompts for more control tasks. Experiments demonstrate the effectiveness of CTRLsum on three domains of summarization datasets and five control tasks: (1) entity-centric and (2) length-controllable summarization, (3) contribution summarization on scientific papers, (4) invention purpose summarization on patent filings, and (5) question-guided summarization on news articles. Moreover, when used in a standard, unconstrained summarization setting, CTRLsum is comparable or better than strong pretrained systems.

## introduction neural summarization systems aim to compress a document into a short paragraph or sentence while preserving key information. there are two common categories of summarization systems: extractive summarization where models find and copy important portions of a documents @xcite @xcite , and abstractive summarization where models freely generates novel sentences @xcite @xcite . the focus of this work is on abstractive summarization. in a common setting, summarization methods take a source document as input and yield a generic summary which covers content selected arbitrarily by the model. however, to be useful, automatically generated summaries should cover content considered important by the readers. for example, figure 1 shows an nba basketball news article, and the reference summary aggregates the results of several matches. yet fans of certain players or teams might only be interested in the matches and statistics which include their entities of interest. motivated by this observation, in this work we focus on controllable summarization which allows the users to control the generated summaries. we propose ctrlsum, a framework to control summaries through a set of keywords. at training time, the model learns to predict summaries conditioned on both the source document and keywords, which are easily identified from training summaries. during inference, a control function is designed depending on the specific control aspect to map user preference to keywords to control the summary. while simple, keywords provide a clean separation of test-time user control and the training process -different dimensions of the generated summary could be controlled through engineering the test-time control function, while the training process and pretrained model remain unchanged. in contrast, typical controllable summarization methods @xcite @xcite @xcite predefine the specific control aspect (e.g. entity, length, topic, etc.) and rely on the corresponding control annotations during training. consequently, they require to train a separate model for each control aspect and cannot generalize to new control aspects at test time. figure 1 demonstrates such distinction. the training process in ctrlsum is agnostic to the specific control aspect, leading to possibly dwyane wade scored 21 of his 32 points in the first half and goran dragic added 20 as the miami heat handed lebron james another loss on his former home floor with a 106-92 victory over the cleveland cavaliers on monday …… [ignoring 60 tokens] james scored 16 of his 26 points in the fourth quarter for cleveland, which had its fourgame winning streak snapped. kyrie irving added 21. klay thompson scored 26 points, and stephen curry had 19 points and nine assists as the golden state warriors secured a playoff spot before beating the depleted los angeles lakers 108-105 …… [ignoring 400 tokens] ## the example control tasks in this section we define five controllable summarization tasks that we study in this paper and the respective control function g control . qualitative examples of each task are shown in ## experiments in this section we test ctrlsum quantitatively, and an extensive list of qualitative output examples in all the control tasks can be found in appendix h. ## related work previous work on controllable summarization often collects control codes as supervision @xcite . for example, @xcite utilizes an entity tagger tool to identify entities from the reference summary, and input those entities together with the document during training for entity control. they also study length control by training a separate model with length code as the additional input. other work mostly focus on a specific control task in limited settings @xcite @xcite ), yet ctrlsum aims for a more generic framework. keyword-guided summarization has been applied in different contexts or with distinct motivations. keywords are used as extra input to improve unconstrained summarization or reduce hallucinations @xcite @xcite @xcite . ctrlsum is also related to query-focused summarization @xcite @xcite , these methods often work with datasets that provide annotated queries, and did not focus on generic control tasks. ## conclusion in this paper we propose a generic framework, ctrlsum, to perform generic controllable summarization through a set of keywords. experiments on five different control aspects demonstrate the efficacy of our method.