Don’t Add, don’t Miss: Effective Content Preserving Generation from Pre-Selected Text Spans (2023)

Abstract: The recently introduced Controlled Text Reduction (CTR) task isolates the text generation step within typical summarization-style tasks. It does so by challenging models to generate coherent text conforming to pre-selected content within the input text (“highlights”). This framing enables increased modularity in summarization-like tasks, allowing to couple a single CTR model with various content-selection setups and modules. However, there are currently no reliable CTR models, while the performance of the existing baseline for the task is mediocre, falling short of practical utility. Here, we address this gap by introducing a high-quality, open-source CTR model that tackles two prior key limitations: inadequate enforcement of the content-preservation constraint, and suboptimal silver training data. Addressing these, we amplify the content-preservation constraint in both training, via RL, and inference, via a controlled decoding strategy. Further, we substantially improve the silver training data quality via GPT-4 distillation. Overall, pairing the distilled dataset with the highlight-adherence strategies yields marked gains over the current baseline, of up to 30 ROUGE-L points, providing a reliable CTR model for downstream use.

## introduction the abstractive text summarization task, aiming to generate accurate and coherent summaries from one or multiple documents, involves two principal sub-tasks: (a) identification of salient information in the input text(s) and (b) its consolidation into a coherent text. recently, @xcite proposed an explicit decomposition of these two subtasks, particularly concentrating on the latter as an isolated task termed controlled text reduction (ctr), as illustrated in figure 2 . this task takes as input a text with pre-selected marked spans ("highlights") and expects a reduced and coherent version of the text, covering precisely the content of these input spans. in addition to a baseline model, the authors also provided crowdsourced dev and test sets, along with automatically-generated silver training data, where each instance comprises of a document with highlighted content and the corresponding reduced text summary. the proposed adoption of ctr offers greater control over text generation, enabling modular summarization systems, where a single ctr model can be combined with various content selection strategies and user preferences. for example, the same ctr model could be used for generation from content selected for either generic or query-focused summarization (qfs; @xcite , or for long-form question-answering (lfqa; @xcite . it could even enable a human-in-the-loop scenario, where customized summaries can be created, based on users' preferences, as was recently demonstrated in @xcite . further, by excluding the subjective content selection requirement of the full summarization task, ctr offers a semantically welldefined and objective generation task, focused on coherent content consolidation. being recently introduced, there currently exists no high-quality ctr model, with the present baseline succeeding to cover only half of the highlighted details while pulling much non-highlighted content from the surrounding context. in this paper, we aim to design an efficient ctr model of sufficient quality for reliable integration in modular architectures. our proposed method, outlined in figure 1 , addresses two different shortcomings of the existing resources. first, we examine methods to intensify the highlights signal during training and inference, in order to yield better content preservation. second, we address the noise evident in the available silver training data, improving its quality using gpt-4. we begin our investigation by exploring strategies for enforcing the highlights signal. first, we explore the use of reinforcement learning (rl) during training to both accentuate the highlights signal and mitigate the impact of the data noise, adapting the recently-introduced quark algorithm @xcite , which aims to unlearn unwanted properties. since rl methods bias the model to conform to a reward function, in addition to the training data, we hypothesize it has the potential to reduce the impact of noisy data biases on the model's behavior. we then design a highlightaware decoding mechanism, following @xcite , which biases the model to better preserve the highlights content at inference time. finally, we address the inherent noise within the available silver training data, by employing gpt-4 (ope-nai, 2023) to generate cleaner training data for our model, essentially performing (symbolic) distillation from gpt-4. empirically, we demonstrate that each of the aforementioned strategies separately yields stateof-the-art results, surpassing the baseline model in terms of highlights content preservation. further, we show that gpt-4 is indeed effective in generating better silver training data, leading to further improvements. hence, our contribution in this paper is twofold: ## background this section provides the needed background regarding our task and the methods we employ. ## method this section presents the three core components of our contribution: highlights-driven reinforcement learning (rl) ( §3.1), highlight-attentive controlled decoding ( §3.2), and the steps taken for generating improved fine-tuning data with gpt-4 ( §3.3), corresponding to components (b), (c), and (a) in figure 1 , respectively. ## experimental setup base model throughout our experiments, our primary base model is the instruction-finetuned flan-t5 large model (flan-t5 large ; chung et al., 2022), further fine-tuned on the highlights-focused ctr dataset. the selection of this particular model is motivated by emergent research that indicates instruction-finetuned models manifest superior performance in tasks necessitating constrained generation @xcite @xcite . we will refer to this model as flan-t5 h , where h stands for "highlight-finetuned". in addition to this variant of flan-t5 h , we also show results of a large variant of the original pretrained ctr baseline model, which is the longformer encoder-decoder large model (led large ; @xcite , finetuned on the ctr dataset. we will refer to this model as led h . lastly, we also show the results of the few-shot gpt-4 (ope-nai, 2023), when guided with the same prompt used in our distillation process (see §3.3). ## discussion and future work our empirical assessments reveal the merits of each of the proposed methods. specifically, the utility of the distillation process emerges as complementary to both reinforcement learning (rl) training and controlled decoding. the latter methods still have room to play in both enforcing the content preservation constraints beyond the sheer training data, as well as in overcoming a certain level of noise that persists also in the gpt-4 generated data (as shown in §5.3). conversely, the combination of controlled decoding and highlight-centric rl approach did not yield improvements over the better-performing method (controlled decoding, in our experiments). given these strategies impact different stages of the generation process, namely training and inference, they may possess the potential for synergy. for example, the decoding strategy may be integrated into the rl training's sampling phase, possibly increasing the model's awareness of the decoding constraints during training. our performance analysis, detailed in §5.4, further supports this hypothesis by demonstrating that each technique amplifies different aspects of the highlights: one notably improves their coverage while the other augments adherence, suggesting the exploration of this potential synergy in future research. furthermore, the existing implementation of highlight-centric controlled decoding is computationally demanding, given its requirement to generate a complete completion for every candidate token at each generation step. @xcite motivated the need to generate entire summaries by the expectation of current faithfulness metrics for full summary inputs. yet, as our method does not employ faithfulness metrics, it would be insightful to explore how to leverage partial summaries, rather than complete ones, and thus to significantly reduce the computational overhead. finally, given the high-quality highlight adherence and coverage exhibited by our best models, investigating their incorporation into modular summarization pipelines emerges as a promising research direction. we suggest exploring this direction in future research. ## conclusion in this study, we addressed the lack of a highquality controlled text reduction (ctr) model by focusing on two pivotal aspects: the amplification of the highlight signals and the mitigation of noise within the training data. we started by proposing two distinct strategies aiming at augmenting the highlights signal. the first strategy emphasized this signal during training, where we combined an rl approach with a custom highlights-oriented reward. the second strategy was introduced during inference, where we employed a controlled decoding mechanism that prioritizes generation paths ensuring higher adherence to the highlights. furthermore, we addressed the intrinsic noise in the ctr dataset by generating new instances using gpt-4, significantly enhancing the dataset quality. empirical evidence shows the effectiveness of our proposed methodology. each of our highlightcentric strategies individually led to significant improvements over the baseline model in terms of highlight-matching capabilities. additionally, training on the gpt-4-generated data yielded further improvements, outperforming each of the non-distilled variants, trained on the original ctr dataset. in total, our highest-performing models achieved state-of-the-art results, outperforming the baseline by more than 30 rouge-l points, while also preserving comparable levels of coherency. future work would focus on improving the combination and efficiency of the different components, as well as on the incorporation of our best-performing models in modular summarization pipelines.