Ruler: A Model-Agnostic Method to Control Generated Length for Large Language Models (2024)

Abstract: The instruction-following ability of large language models enables humans to interact with AI agents in a natural way. However, when required to generate responses of a specific length, large language models often struggle to meet users’ needs due to their inherent difficulty in accurately perceiving numerical constraints. To explore the ability of large language models to control the length of generated responses, we propose the Target Length Generation Task (TLG) and design two metrics, Precise Match (PM) and Flexible Match (FM) to evaluate the model’s performance in adhering to specified response lengths. Furthermore, we introduce a novel, model-agnostic approach called Ruler, which employs Meta Length Tokens (MLTs) to enhance the instruction-following ability of large language models under length-constrained instructions. Specifically, Ruler equips LLMs with the ability to generate responses of a specified length based on length constraints within the instructions. Moreover, Ruler can automatically generate appropriate MLT when length constraints are not explicitly provided, demonstrating excellent versatility and generalization. Comprehensive experiments show the effectiveness of Ruler across different LLMs on Target Length Generation Task, e.g., at All Level 27.97 average gain on PM, 29.57 average gain on FM. In addition, we conduct extensive ablation experiments to further substantiate the efficacy and generalization of Ruler. Our code and data is available on the internet.

## introduction large language models (llms) have demonstrated remarkable capabilities across a variety of natural language tasks and are increasingly being utilized in various fields @xcite @xcite . a primary † min yang is the corresponding author. query: tell me how to make a cake in 30 words. word count of the @xmath0: to make a cake, preheat your oven to 350°f (175°c) and prepare an 8-inch round cake pan by greasing and flouring it. cream together 1 cup of granulated sugar and 1/2 cup of softened unsalted butter until light and fluffy, which takes about 3-4 minutes with an electric mixer. add 2 large eggs, one at a time, beat… failed figure 1 : existing llms lack the capability to follow instructions for generating texts of a specified length. the practical challenges highlight the complexity of achieving precise instruction following, particularly when users require control over the output's length. users frequently give llms various instructions, such as "tell me how to make a cake in 20 words", "use 50 words to write a post", "write a 300-word story for me" and so on. these instructions challenge the instruction following capability of llms. to explore how well llms handle such challenges, we focus on the scenario where users specify the target length of the responses. the question is posed, "can llms accurately generate with target length?" and introduce the target length generation task (tlg). we create a test dataset with various target lengths and introduce two evaluation metrics: precise match (pm) and flexible match (fm). our findings reveal that current llms generally perform poorly in this task, indicating considerable room for improvement. a discussion on the underlying causes is conducted, primarily attributing it to tokenization schemes and model training strategy. to address aforementioned issues, we introduce ruler, a model-agnostic approach designed to enhance the instruction-following capability of llms through meta length tokens (mlts). mlts are designed to control model's responses. by utilizing ruler, llms can generate responses that meet target lengths. we create a dataset with mlts d m lt for end-to-end training of llms. llms learn to generate mlt and the corresponding length response after training. during inference, if a target length is provided, ruler can transform it into a mlt and generate responses that meet the requirement. if no target length is specified, it first generates a mlt, then the response, ensuring its length aligns with the generated mlt. we apply ruler to various large language models and test them on tlg. each model demonstrates significant improvements. across all evaluated models, we observe a consistent improvement in both pm and fm scores at all levels. the pm and fm scores across all level showed an average improvement of 27.97 and 29.57.furthermore, to rigorously test the capabilities of ruler, we randomly sample the dataset provided by @xcite and assess ruler on multi mlt generation and self-generated mlt experiment to show the its effectiveness and generalizability. additionally, ruler is tested on six benchmarks to observe whether the models' overall performance is affected. our contributions can be summarized as follows: ## can llms accurately generate with target length? in this section, we examine the capability of llms to generate responses of a target length. initially, we introduce target length generation task (tlg). subsequently, we establish various target lengths and two evaluation metrics ( §3.1). we then detail the experimental setup and assess the ability of llms to generate responses at target lengths ( §3.2). finally, we present the outcomes of the experiments and analysis the underlying reasons( §3.3). ## ruler: meta length token controlled generation in this section, we first introduce ruler, encompassing the design of the meta length tokens (mlts), the data collection and the learning process associated with the models ( §4.1). subsequently, we detail the difference in the generation of ruler under two scenarios: tlg and non-tlg ( §4.2). ## conclusion this study initially investigate the instruction following abilities of llms and introduces target length generation task (tlg). additionally, we propose ruler, a novel and model-angnostic method that controls generated length for llms. ruler utilizes the mlt and end-to-end training to enhance model performance. experimental results demonstrate that substantial improvements in pm and fm scores across various models. moreover, two additional experiments are conducted to further validate the efficacy of the proposed method. finally, we assess overall performance across six different benchmarks to demonstrate its superiority.