I nstruct CMP : Length Control in Sentence Compression through Instruction-based Large Language Models (2024)

Abstract: Extractive summarization can produce faithful summaries but often requires additional constraints such as a desired summary length. Traditional sentence compression models do not typically consider the constraints because of their restricted model abilities, which require model modifications for coping with them. To bridge this gap, we propose Instruction-based Compression (InstructCMP), an approach to the sentence compression task that can consider the length constraint through instructions by leveraging the zero-shot task-solving abilities of Large Language Models (LLMs). For this purpose, we created new evaluation datasets by transforming traditional sentence compression datasets into an instruction format. By using the datasets, we first reveal that the current LLMs still face challenges in accurately controlling the length for a compressed text. To address this issue, we propose an approach named length priming, that incorporates additional length information into the instructions without external resources. While the length priming effectively works in a zero-shot setting, a training dataset with the instructions would further improve the ability of length control. Thus, we additionally created a training dataset in an instruction format to fine-tune the model on it. Experimental results and analysis show that applying the length priming significantly improves performances of InstructCMP in both zero-shot and fine-tuning settings without the need of any model modifications.

## introduction sentence compression is a task of creating a concise summary from an original sentence while conveying its key information, by deleting words in the sentence. generally, sentence compression in extractive summarization provides more faithful summaries than abstractive summarization @xcite . ## problem statement the traditional approach to sentence compression is considered as a sequential labeling task @xcite @xcite @xcite . each source token in a sequence, represented as @xmath0, x 1 , ..., x n }, is processed using a sentence compression model to predict a corresponding label sequence, which is @xmath1, y 1 , ..., y n }, where y @xmath2, 0}. while the method is straightforward, it has limitations in incorporating additional constraints such as a desired length. addressing these requirements in the traditional approach typically involves modifications to the model, which is inefficient @xcite . to overcome these limitations, we utilize the recent powerful instruction-based llms for the sentence compression task @xcite . ## instruction-based compression in this section, we describe instructcmp. we consider "length priming" for a length constraint in it. we created new evaluation datasets by transforming traditional sentence compression datasets into an instruction format. to further improve performances of instructcmp, we also created a new training dataset in an instruction-based template. ## conclusion we proposed instructcmp to conduct sentence compression by incorporating length constraints without model modifications. for this new approach, we constructed new evaluation datasets by transforming traditional sentence compression datasets into an instruction format, while we also created new training datasets. additionally, we introduced "length priming" into the instructions and demonstrated its effectiveness in zero-shot and instruction-based fine-tuning settings on four benchmark datasets. we also conducted an indepth analysis, including the model size and type.