InstructCMP: Length Control in Sentence Compression through Instruction-based Large Language Models (2024)

Abstract: Extractive summarization can produce faithful summaries but often requires additional constraints such as a desired summary length. Traditional sentence compression models do not typically consider the constraints because of their restricted model abilities, which require model modifications for coping with them. To bridge this gap, we propose Instruction-based Compression (InstructCMP), an approach to the sentence compression task that can consider the length constraint through instructions by leveraging the zero-shot task-solving abilities of Large Language Models (LLMs). For this purpose, we created new evaluation datasets by transforming traditional sentence compression datasets into an instruction format. By using the datasets, we first reveal that the current LLMs still face challenges in accurately controlling the length for a compressed text. To address this issue, we propose an approach named “length priming,” that incorporates additional length information into the instructions without external resources. While the length priming effectively works in a zero-shot setting, a training dataset with the instructions would further improve the ability of length control. Thus, we additionally created a training dataset in an instruction format to fine-tune the model on it. Experimental results and analysis show that applying the length priming significantly improves performances of InstructCMP in both zero-shot and fine-tuning settings without the need of any model modifications.

heading: 1 Introduction
text: Sentence compression is a task of creating a concise summary from an original sentence while conveying its key information, by deleting words in the sentence. Generally, sentence compression in extractive summarization provides more faithful summaries than abstractive summarization (Cao et al., 2018).
∗ corresponding author

heading: Traditional Method

heading: Instruction-based Method
text: While traditional sentence compression methods used tree trimming, the approaches can be affected by parsing errors (Jing, 2000; Knight and Marcu, 2000; Berg-Kirkpatrick et al., 2011; Filippova and Altun, 2013). The introduction of LSTM-based Seq2Seq approaches aims to address this issue although their performance tends to degrade in handling longer sentences (Filippova et al., 2015). To solve this problem, Kamigaito and Okumura (2020) incorporated syntactic dependency trees into the Seq2Seq attention mechanism (Kamigaito et al., 2018) by jointly learning the dependency trees and sentence compression models. However, the stateof-the-art model required a considerable amount of ground-truth data for training (Filippova and Altun, 2013; Hasegawa et al., 2017).
Recently, unsupervised sentence compression has gained attention by exploiting BERT-based encoder models (Devlin et al., 2019). These models incorporated various scoring functions that target improving fluency and faithfulness in compres-
8980
sion without relying on ground-truth data (Niu et al., 2019; Zhou and Rush, 2019; Schumann et al., 2020; Ghalandari et al., 2022). However, these approaches are inefficient because they require extensive model modifications, such as including classifiers or modifying beam search for objective-specific fine-tuning.
In general, summarization requires additional constraints such as a summary length (Takase and Okazaki, 2019; Dou et al., 2021; Kwon et al., 2023a). The traditional task setting for sentence compression often did not consider this factor because of the restricted model abilities, which require model modifications to handle such constraints (Schumann et al., 2020; Ghalandari et al., 2022).
Recently, LLMs have gained considerable attention for their remarkable zero-shot task-solving abilities, especially under instruction-based settings (Ouyang et al., 2022; Wei et al., 2022a). Inspired by these latest advancements, we present Instruction-based Compression (InstructCMP), a novel approach to sentence compression that accommodates a length constraint through explicit instructions, without necessitating model modifications. To the best of our knowledge, this approach represents the first implementation of sentence compression in an instruction-based framework. For this purpose, we transformed traditional sentence compression datasets into an instructionbased format for evaluation.
However, recent LLMs do not consistently generate an output of the precise length, even when specific instructions to include such constraints are provided in a zero-shot manner (Zhou et al., 2023; Qin et al., 2023). Furthermore, as we validate it later, even when testing with the latest powerful models, such as ChatGPT (GPT-4) and ChatGPT (GPT4-1106-preview),1 accurately adhering to length constraints remains a substantial challenge.
To address this problem, we propose an instruction approach for better length control, which is named “length priming.” We incorporate additional length information (Misra et al., 2020) into the instruction. In addition to specifying the number of deleted words for the desired length, we include the length to be retained and the number of words in the source sentence in the instruction, without any external resources. To further improve length controllability, we additionally created a training
1https://chat.openai.com/

heading: Work Length Const. Mod.
text: dataset with the instructions to fine-tune the model using the dataset. Figure 1 shows the transformation process for an instruction format.
We conducted experiments on four benchmark datasets and performed an in-depth analysis to evaluate the effectiveness of LLMs in compressing sentences under the length constraint. The analysis considers the following factors: the model type and the number of parameters for the model size. Experimental results show that InstructCMP with length priming compresses sentences in a zero-shot setting while successfully keeping the desired length without model modifications. The performance can be further improved by fine-tuning it with the created instruction-based training dataset. The “length priming” method proves effective in both zero-shot and fine-tuning settings, as shown by significant improvements in the ROUGE metrics and adherence to the length constraint, even when using ChatGPT (GPT-4) and ChatGPT (GPT4-1106-preview). Our in-depth analysis also showed that InstructCMP can compress sentences while maintaining faithfulness. Our experiments show that instruction-based models like ChatGPT can effectively control the length when provided with more specific lengthrelated information.2

heading: 2 Problem Statement
text: The traditional approach to sentence compression is considered as a sequential labeling task (Filippova et al., 2015; Wang et al., 2017; Zhao et al., 2018; Kamigaito and Okumura, 2020; Schumann et al., 2020; Ghalandari et al., 2022). Each source token in a sequence, represented as x = {x0, x1, ..., xn}, is processed using a sentence compression model to predict a corresponding label sequence, which is
2Our code and datasets are available at: https://github. com/JuseonDo.
y = {y0, y1, ..., yn}, where yi ∈ {1, 0}. While the method is straightforward, it has limitations in incorporating additional constraints such as a desired length. Addressing these requirements in the traditional approach typically involves modifications to the model, which is inefficient (Schumann et al., 2020; Ghalandari et al., 2022).
To overcome these limitations, we utilize the recent powerful instruction-based LLMs for the sentence compression task (Touvron et al., 2023; Chung et al., 2022). Table 1 shows a comparison between previous work on traditional sentence compression and InstructCMP. Unlike the previous work, InstructCMP incorporates a length constraint directly into the instruction format, allowing models to process and learn the constraint as a part of their input. This allows an efficient and flexible solution for practical sentence compression, without extensive model modifications.

heading: 3 Instruction-based Compression
text: In this section, we describe InstructCMP. We consider “length priming” for a length constraint in it. We created new evaluation datasets by transforming traditional sentence compression datasets into an instruction format. To further improve performances of InstructCMP, we also created a new training dataset in an instruction-based template.

heading: 3.1 Instruction Template
text: Table 2 shows instructions that include a length constraint. The first instruction permits InstructCMP to compress text by deleting words without any constraints. However, in general, summarization requires a desired length for compressed texts (Makino et al., 2019; Dou et al., 2021; He et al., 2022; Kwon et al., 2023a). Length Priming. To apply the length constraint, we first construct an instruction that deletes words to meet a desired length (Constraint 2). It is easy to calculate the number of words to be deleted for any desired length.
However, LLMs do not consistently follow instructions, particularly when processing length constraints (Zhou et al., 2023; Qin et al., 2023). To address this issue, we propose the “length priming” method for the length constraint instruction for enhanced length comprehension. Constraint 3 considers the total length of the source text and the number of words that should be kept and deleted together. Considering such additional length infor-
mation can enable InstructCMP to recognize the length constraint more effectively. The number of words that should be kept is automatically calculated from the target desired length.
Constraint 3-1 applies the “length priming” only to the source text based on its length, whereas Constraint 3-2 applies it solely to the target text based on the number of words that should be kept and deleted together.

heading: 3.2 Dataset Creation
text: We consider four benchmark datasets. The Google dataset (Google) was automatically created by considering the syntactic dependency tree structure from news headlines (Filippova and Altun, 2013). The training, validation, and test datasets consist of 200,000, 1,000, and 1,000 pairs, respectively. For the test dataset used in the evaluation, the gold compression ratio is 0.45. The Broadcast (Broad) and BNC (BNC) datasets (Clarke and Lapata, 2008) comprise manually compressed sentences. Each of these datasets contains 1,370 and 1,629 evaluation pairs, respectively. The gold compression ratios of these datasets, which are 0.76 and 0.72 respectively, are longer than those of other evaluation datasets. DUC2004 (TASK1) (DUC) comprises 500 pairs with a gold compression ratio of 0.39. Unlike other evaluation datasets, this dataset includes abstract summaries as its ground truth.
We created new datasets by transforming traditional sentence compression datasets into an instruction format. For length constraint instructions, we inject lengths of ground-truth summaries.

heading: 3.3 Instruction-based Fine-tuning
text: To improve performances by leveraging LLM’s generalizability (Wang et al., 2022; Wei et al., 2022a; Chung et al., 2022), we also created a training dataset for instruction-based fine-tuning by sampling 5% of the training dataset from Google. Through this fine-tuning, we aim to enhance a model for better learning and improving abilities to handle length constraints in compressing sentences without any model modifications.

heading: 4 Experiments

heading: 4.1 Experimental Settings
text: Evaluation Metrics. F1 scores of ROUGE-1 (R1), -2 (R-2), and -L (R-L), the F1-score for kept tokens (F1), and the BERT score (BS) (Zhang* et al., 2020) were used to evaluate compression
quality. The ROUGE scores were calculated using the implementation provided by Google Research.3
To evaluate performances related to a length constraint, we calculated ∆CR, the difference between the model-generated compression ratio and the gold compression ratio. ∆CR evaluates how close the compression ratio of model-generated outputs is to the gold compressed summary (Kamigaito et al., 2018; Kamigaito and Okumura, 2020). Because InstructCMP can produce novel words, we counted the number of the novel words in the model-generated compressed summaries. Thus, novel represents the ratio of novel words that do not appear in the source text. Implementation Details. We employed the instruction-based open-source Llama2-13B-chat model (Touvron et al., 2023)4 as our backbone model. We tested various instructions on the validation dataset from Google and made selections based on their performance. To explore various parameter numbers, we experimented with 4-bit and 8-bit quantizations, as well as without quantization (Jacob et al., 2018) using PyTorch.5 We also evaluated the performance across various model sizes, including 7B and 70B, and compared various model types, specifically the encoder-decoder based models of FLAN-T5-XXL (11B) (Chung et al., 2022)6 and FLAN-UL2 (20B) (Tay et al., 2023).7
For instruction-based fine-tuning, we considered QLoRA, which can preserve the full 16-bit fine-tuning performance (Dettmers et al., 2023).
3https://github.com/google-research/ google-research/tree/master/rouge
4https://huggingface.co/meta-llama/ Llama-2-13b-chat-hf
5https://github.com/pytorch/pytorch 6https://huggingface.co/google/flan-t5-xxl 7https://huggingface.co/google/flan-ul2
QLoRA is an extended version of Low-Rank Adapters (LoRA) (Hu et al., 2022), an improved Parameter-Efficient Fine-Tuning (PEFT) (Mangrulkar et al., 2022) method for LLMs. This method combines low-rank and trainable matrices with the frozen weights in each layer of Transformer, building upon the foundational approach of LoRA. We incorporated low-rank matrices into the query and value weights using a LoRA attention dimension of 8. During training, we used 8-bit quantization for QLoRA, and during inference, we employed 4-bit quantization.

heading: 4.2 Main Results
text: Table 3 shows the performances of InstructCMP based on the Llama2-13B-chat model in a zeroshot setting, used directly without additional training, and in the QLoRA instruction-tuning setting, which involves fine-tuning of InstructCMP. Because prompting techniques for LLMs, such as few-shot (Min et al., 2022), directional stimulus (Li et al., 2023), and generated knowledge (Liu et al., 2022) methods, require external resources, we compared “length priming” to prompting techniques of chain-of-thought (Wei et al., 2022b) and treeof-thought in a single prompt (Yao et al., 2023; Hulbert, 2023) by adding them at the beginning of length constraint instructions (#2 in Table 2). Performance in Instruction-based Zero-shot.8 Even in a zero-shot setting, InstructCMP without a length constraint (#1 in Table 2) successfully compresses sentences, while it cannot necessarily meet the length. In applying length constraints with “length priming”, consistently improved performances are observed in both ROUGE and ∆CR. In addition, our “length priming” significantly outper-
8Experiments considering various instructions on the validation dataset from Google are detailed in Appendix A.
forms other prompting methods, chain-of-thought and tree-of-thought, in both length controllability and ROUGE metrics.
However, controlling the length of outputs for Google and DUC proved to be more challenging than Broad and BNC, specifically, in a zero-shot setting. We think this challenge arises from the nature of datasets, whose compression ratio is lower. Table 4 shows the results based on a target compression ratio of 0.2 and a target word count of 5 words, respectively. We observed that when the compression ratio is lower, the LLMs have difficulties maintaining both informativeness and length controllability. Performance in Instruction-based Fine-tuning.9 Following instruction-based QLoRA fine-tuning, the created training dataset further improves per-
9Experiments considering 0.5% and 1% randomly sampled training datasets from Google are detailed in Appendix B.
formances of InstructCMP. As shown in ∆CR for Broad and BNC, the model without the length constraint was trained to compress sentences more closely aligned with the gold compression ratio of Google. However, the performance degradation was observed on DUC when fine-tuning was applied using Google, due to the different natures of their abstractive and extractive ground-truth summaries.
Length Priming. The ablation results for “length priming” in instructions are presented in Table 6. We first compare performances of “length priming” in an unsupervised zero-shot setting. It significantly improved performances on all datasets in terms of ∆CR compared to w/o priming. Even in a supervised instruction-based fine-tuning, “length priming” largely improved performances in both ROUGE metrics and length controllability. The exception is on DUC because of its nature of the abstractive gold summary.

heading: Data Setting Output Gram. Faith. Info.
text: We also compare the effectiveness of “length priming,” using larger models, such as Llama2-70B-chat-hf, ChatGPT (GPT-4), and ChatGPT (GPT-4-1106-preview). Figure 2 shows the results. We confirm that “length priming” is essential for length constraints, even in the most recent and powerful LLMs.10

heading: 5 Analysis

heading: 5.1 Parameter Sizes
text: The left graph of Figure 3 shows the F1 score for kept tokens and the model-generated compression ratio (CR), compared to the gold compression ratio, based on zero-shot InstructCMP without a length
10When we additionally tested the chain-of-thought and tree-of-thought prompting methods on these larger models, their length controllability was similar to each other, which is similar to the results in Table 3.

heading: Data Method Instruction R-1 R-2 R-L F1 BS ∆ CR
text: constraint on the Llama2-chat model with 7B, 13B, and 70B parameters. On Google and DUC, the F1 scores increased with enlarging the model size, achieving compression closer to the gold compression ratio. However, on Broadcast and BNC, which have high gold compression ratios, InstructCMP with the 70B model compresses sentences more concisely, resulting in a compression ratio that significantly deviates from the gold compression ratio,
consequently decreasing F1 scores compared to the 13B model.
To further investigate this, we conducted human evaluations. We sampled 100 instances each from Google, Broad, and BNC. By using Amazon Mechanical Turk, we assigned in total 120 evaluators who obtained both US high school and US bachelor’s degrees for grading the results with scores from 1 to 5 (5 is the best) in terms of grammatical correctness (Gram), factual consistency (Faith), and a balance of redundancy and informativeness (Info). Table 5 shows the results. Because of the automatically constructed nature of Google, QLora and zero-shot settings can yield higher grammaticality scores than the gold summary. These results also indicate gold summaries of Broad and BNC are actually redundant (Ghalandari et al., 2022), and our instruction-based approach can generate faithful, informative, and grammatical summaries.
The right graph of Figure 3 shows the results of zero-shot InstrcutCMP without a length constraint on Llama2-13B-chat. Interestingly, there are no significant differences in performance among the 4-bit, 8-bit, and nonquantized versions.

heading: 5.2 Model Types
text: It is also of interest to draw comparisons with other instruction-based models, such as FLAN-T5XXL and FLAN-UL2, both of which employ the encoder-decoder architecture. However, they did not effectively compress sentences using instruction templates in Table 2. We think this is due

heading: Data Model Instruction R-1 R-2 R-L F1 ∆ CR
text: to the nature of their pre-training, which causes potential gaps between the pre-training steps and the instruction templates for extractive summarization settings (Kwon et al., 2023a). Thus, we used slightly modified instruction templates.11 Table 7 shows the results. Our “length priming” can improve length controllability by keeping ROUGE metrics compared to w/o priming.

heading: 5.3 Case Study
text: Table 8 shows the outputs of zero-shot InstructCMP based on the Llama-13B-chat model. The first example shows the controllability of the length constraint instruction. Even when instructed to delete zero words, InstructCMP follows the instruction correctly. The second example shows the flawless grammatical capabilities of LLMs (Mitrović et al., 2023). When deleting a single word can cause a grammatical error, InstructCMP can correct the error by paraphrasing, represented as novel in Table 3. The third example shows the output of InstructCMP in response to the length constraint. “Length priming” assists InstructCMP to compress a source text to meet a desired length, performing better than the length constraint without priming.

heading: 5.4 Comparison with the Baselines
text: We compare InstructCMP with traditional stateof-the-art (SOTA) baselines, specifically SCRL,12 which employs reinforcement learning optimized
11Experimental results using instruction templates in Table 2 and modified instruction templates are in Appendix C.
12https://github.com/complementizer/ rl-sentence-compression
in unsupervised settings, and SLAHAN,13 which recursively tracks parent and child words and leverages BERT embeddings optimized in supervised settings, trained on Google (Kamigaito and Okumura, 2020).
Following SCRL, we set a desired length of 11 for Google and DUC. In line with the previous work, we truncated model-generated outputs to 75 characters and used ROUGE recall scores for DUC (Schumann et al., 2020; Ghalandari et al., 2022). For Broadcast and BNC, the desired length was set to 75% of the length of the source sentence. Table 9 shows the results. Because zero-shot InstructCMP faces challenges in compressing sentences with length constraints when the gold compression ratio is low, we increased the model capability by using Llama2-70B-chat for Google and DUC instead of
13https://github.com/kamigaito/SLAHAN
Llama2-13B-chat. We observed comparable performances of InstructCMP to SCRL.
We also compare InstructCMP, based on Llama13B-chat, with SLAHAN. Following the previous work, we fine-tuned InstructCMP without a length constraint and achieved significant improvement, even after using 5% of the training dataset.

heading: 5.5 Increasing Training Dataset Size
text: We provide additional experimental results using larger datasets for QLoRA fine-tuning with 10% and 15% google training datasets. Table 10 shows the results. Three different benchmark results on Google, Broad, and BNC support that length priming is necessary, except for DUC due to its abstract summary nature, and indicate the generalization of the length priming instruction.

heading: 6 Related Work
text: Sentence Compression. Early studies on sentence compression in both supervised and unsupervised learning frameworks have used linguistic constraints, such as tree trimming methods (Jing, 2000; Knight and Marcu, 2000; Hori and Furui, 2004; Clarke and Lapata, 2006; Berg-Kirkpatrick et al., 2011; Filippova and Altun, 2013). To avoid potential parsing errors in the tree trimming, LSTMbased models have been introduced for deletionbased compression (Filippova et al., 2015) by jointly considering eye-tracking data (Klerke et al., 2016) and incorporating a score function of an ILPbased tree trimming method (Wang et al., 2017). Zhao et al. (2018) explored reinforcement learning for a syntax-based language model, that does not use explicit parsed trees. Kamigaito et al. (2018); Kamigaito and Okumura (2020) proposed Seq2Seq approaches that jointly learn sentence compression and dependency trees within their attention networks inspired by supervised head attention
(Kamigaito et al., 2017), an extensible approach to document-level summarization (Ishigaki et al., 2019) similar to the case of graph neural networks (Xu et al., 2020; Kwon et al., 2021). Alternatively, some recent work has utilized LLMs, such as BERT, for sentence compression to optimize fluency in unsupervised frameworks (Zhou and Rush, 2019; Niu et al., 2019; Schumann et al., 2020). Because a high-quality compressed sentence can infer from the original sentence, encoder-decoderbased autoencoder approaches have been also explored (Miao and Blunsom, 2016; Févry and Phang, 2018; Malireddy et al., 2020). For better optimization, reinforcement learning has been used (Wang et al., 2018; Ghalandari et al., 2022). Length Control. Despite the success of previous studies, practical summarization requires additional constraints such as a summary length for compressing sentences (Liu et al., 2018; Takase and Okazaki, 2019; Li et al., 2020; He et al., 2022). The approach for controlling the output to a desired length required modifying model parameters (Kikuchi et al., 2016), applying direct constraints (Takase and Okazaki, 2019; Makino et al., 2019; Kwon et al., 2023a), or splitting the training dataset into specific length ranges (He et al., 2022) due to the limited model abilities. Traditionally, sentence compression heavily relies on the model modifications for constraints such as lengths (Schumann et al., 2020; Ghalandari et al., 2022). Instruction-based LLMs. LLMs can perform various tasks in a zero-shot setting, using instructionformatted inputs (Brown et al., 2020; Radford et al., 2019). The emergence of instruction-based LLMs, such as ChatGPT and GEMINI,14 has demonstrated a significant improvement in performance, particularly in their zero-shot problem-solving abilities (Feng et al., 2023; Fang et al., 2023). Because performance varies greatly with various instructions, previous studies focused on finding better instructions (Zhu et al., 2023; Wang et al., 2023; Yao et al., 2023). Various prompting methods have been investigated, such as few-shot, directional stimulus, generated knowledge, chain-of-thought, and treeof thought (Min et al., 2022; Li et al., 2023; Liu et al., 2022; Wei et al., 2022b; Yao et al., 2023). These new types of LLMs mark the beginning of a new era in the field of natural language processing.
While the capabilities of these LLMs continue to grow with an increasing number of parameters,
14https://gemini.google.com/
challenges are introduced for these models in training and testing steps to provide robust and generalized outputs (Rae et al., 2022; Smith et al., 2022; Chowdhery et al., 2022; Chung et al., 2022; Brown et al., 2020; Tay et al., 2023). To address this issue, PEFT methods such as LoRA have been introduced. These methods combine low-rank and trainable matrices with frozen weights in each layer of Transformer and even consider quantization (Hu et al., 2022; Dettmers et al., 2023).
As a related approach to priming, label embedding (Xiong et al., 2021; Zhang et al., 2021) can also incorporate label-related information into the input to enhance generation, as mentioned by Kwon et al. (2023b). However, in contrast to priming, label embedding cannot precisely control the generation itself and requires additional training.
To conduct the sentence compression task with instructions, we focus on priming that incorporates additional constraint-specific information to enhance performance, particularly for the length constraint, rather than just paraphrasing instructions to direct the task.

heading: 7 Conclusion
text: We proposed InstructCMP to conduct sentence compression by incorporating length constraints without model modifications. For this new approach, we constructed new evaluation datasets by transforming traditional sentence compression datasets into an instruction format, while we also created new training datasets. Additionally, we introduced “length priming” into the instructions and demonstrated its effectiveness in zero-shot and instruction-based fine-tuning settings on four benchmark datasets. We also conducted an indepth analysis, including the model size and type.

heading: Limitations
text: Although our length priming successfully compresses sentences, it might be challenging to consider it in document summarization, which requires considering multiple sentences. Therefore, it remains a topic for future studies. In the future, we will consider sentence relationships for prompting to summarize documents. Furthermore, there can be cases where keyword constraints are required for controllable summarization to take into account the content of summaries, which also remains a potential area for future investigation.

heading: Acknowledgements
text: We would like to gratefully acknowledge the anonymous reviewers for their helpful comments and feedbacks.

heading: A Performance in Instruction Selection
text: To determine task-specific instructions, we manually composed several candidates and evaluated their performances on the validation dataset from
Google. Table 11 shows the results. Based on their performances, we selected the 5th instruction as our base instruction for the setting without a length constraint.

heading: B Performance in Varying Training Dataset Size
text: To investigate the impact of the training dataset size on performance, we also prepared 0.5% and 1% training datasets randomly sampled from the Google dataset. Table 12 shows the results of QLoRa fine-tuning for constraints in supervised settings. As observed, increasing the dataset size correlates with improved performance. Table 13 shows the results of an ablation study on “length priming”. Similarly, our “length priming” proves to be essential for performance improvements even in small datasets.

heading: C Performance of the FLAN Models and
text: Modified Instruction Templates
Table 14 shows the results for the FLAN models using instruction templates in Table 2. They did not effectively compress sentences, as denoted by ∆CR.
Table 15 shows the modified templates for the FLAN models.

heading: Data Size Instruction R-1 R-2 R-L F1 ∆ CR

heading: Data Size Instruction R-1 R-2 R-L F1 ∆ CR

heading: Data Model Instruction R-1 R-2 R-L F1 ∆ CR

heading: Future Work
text: Therefore, it remains a topic for future studies. In the future, we will consider sentence relationships for prompting to summarize documents. Furthermore, there can be cases where keyword constraints are required for controllable summarization to take into account the content of summaries, which also remains a potential area for future investigation