GEMINI : Controlling The Sentence-Level Summary Style in Abstractive Text Summarization (2023)

Abstract: Human experts write summaries using different techniques, including extracting a sentence from the document and rewriting it, or fusing various information from the document to abstract it. These techniques are flexible and thus difficult to be imitated by any single method. To address this issue, we propose an adaptive model, GEMINI, that integrates a rewriter and a generator to mimic the sentence rewriting and abstracting techniques, respectively. GEMINI adaptively chooses to rewrite a specific document sentence or generate a summary sentence from scratch. Experiments demonstrate that our adaptive approach outperforms the pure abstractive and rewriting baselines on three benchmark datasets, achieving the best results on WikiHow. Interestingly, empirical results show that the human summary styles of summary sentences are consistently predictable given their context. We release our code and model at https://github.com/baoguangsheng/gemini .

## introduction text summarization aims to automatically generate a fluent and succinct summary for a given text document @xcite @xcite . two dominant methods have been used, namely extractive and abstractive summarization techniques. extractive methods @xcite @xcite @xcite identify salient text pieces from the input and assemble them into an output summary, leading to faithful but possibly redundant and incoherent summaries @xcite @xcite , where rewriting techniques could be used to further reduce redundancy and increase coherence @xcite . in contrast, abstractive methods @xcite ; by abstracting various sentences. ## related work abstractive summarizers achieve competitive results by fine-tuning on pre-trained seq2seq model @xcite . our work is in line, with the contribution of novel style control over the summary sentence, so that a model can better fit human summaries in the "sentence rewriting" and "long-range abstractive" styles. our generator is a standard seq2seq model with the same architecture as bart @xcite , and our rewriter is related to previous single sentence rewriting @xcite @xcite and contextualized rewriting @xcite . our rewriter uses the contextualized rewriting mechanism, which considers the document context of each rewritten sentence, so that important information from the context can be recalled and crosssentential coherence can be maintained. however, different from @xcite , which relies on an external extractor to select sentences, we integrate an internal extractor using the pointer mechanism @xcite , analogous to neusum @xcite to select sentences autoregressively. to our knowledge, we are the first to integrate a rewriter and a generator into a standalone abstractive summarizer. our model gemini can also be seen as a mixture of experts @xcite , which dynamically switches between a rewriter and a generator. a related work is the pointer-generator network @xcite for seq2seq modeling, which can also viewed as a mixture-of-expert model. another related work is the hydrasum, which has two experts for decoding. these models can be viewed as a soft mixture of experts, which learns latent experts and makes decisions by integrating their outputs. in contrast, our model can be viewed as a hard mixture of experts, which consults either the rewriter or the generator in making a decision. a salient difference between the two models is that our gemini makes decisions for each sentence, while previous work makes decisions at the token level. the goals of the models are very different.