P osition ID : LLM s can Control Lengths, Copy and Paste with Explicit Positional Awareness (2024)

Abstract: Large Language Models (LLMs) demonstrate impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. Despite these advancements, LLMs still encounter challenges with length control, frequently failing to adhere to specific length constraints due to their token-level operations and insufficient training on data with strict length limitations. We identify this issue as stemming from a lack of positional awareness and propose novel approaches—PositionID Prompting and PositionID Fine-Tuning—to address it. These methods enhance the model’s ability to continuously monitor and manage text length during generation. Additionally, we introduce PositionID CP Prompting to enable LLMs to perform copy and paste operations accurately. Furthermore, we develop two benchmarks for evaluating length control and copy-paste abilities. Our experiments demonstrate that our methods significantly improve the model’s adherence to length constraints and copy-paste accuracy without compromising response quality.

## introduction large language models (llms) have demonstrated remarkable capabilities in various domains, such as role-playing @xcite , creative writing @xcite , mathematical reasoning @xcite , and coding @xcite . these advanced llms often undergo multitask supervised instruction tuning (sft), endowing them with strong instruction-following abilities. additionally, an additional alignment training stage after sft further aligns llms with human values and needs. a notable outcome of this alignment training is that models tend to produce longer and more detailed responses, thereby enhancing the faithfulness, honesty, and helpfulness of their outputs @xcite @xcite . however, in many scenarios, longer responses are not necessarily better. users may often prefer outputs that follow specific length constraints or conditions. previous studies indicate that even the most advanced llms, such as gpt-4 @xcite , struggle to precisely follow length constraints @xcite . for example, when users request a model to generate a text strictly within 500 words, the model often produces outputs exceeding this requirement. we hypothesize that the llms' weak adherence to length control instructions can be attributed to two primary issues: intuitively, length control inherently requires an understanding of positional relationships within the text. for example, if a model is tasked with generating a summary that is exactly 50 words long, it needs to continuously monitor the number of words it has generated so far and how many words remain to meet the target length. this involves keeping track of its progress (i.e., the number of words being generated) within the text to ensure it stays within the specified limit. however, the model's focus on token-level operations and the lack of strict length-constrained training examples impair its ability to effectively and accurately monitor its progress within the text, leading to inaccuracies in following length control instructions. we refer to this lack of real-time awareness of the generated text's length by the models as the positional awareness issue. our work is not the first to focus on improving the model's positional awareness. for example, abacus embeddings @xcite enhances the model's mathematical computation capabilities by adding positional embeddings to each digit of a number. similarly, contextual position encoding @xcite employs a gate mechanism based on the query-key map to determine positional embeddings, making the positional embeddings context-aware and higher-level. this design improves the model's performance in tasks such as counting and selective copying. however, both of them have several limitations: (1) modifying the positional encoding is unfriendly to the offthe-shelf llms, as it may degrade their generalist capabilities, and it requires model re-training when altering the position encoding approach. (2) and they are not suitable for the closed-source llms which only provide invoking apis. furthermore, (3) these methods are only applicable in limited scenarios, such as some toy tasks or arithmetic tasks, and they demonstrate limited effectiveness when applied to more complex real-world instructions. to address these issues, we propose novel approaches to enhance the model's positional awareness: positionid prompting and positionid fine-tuning for length control tasks. in these approaches, we use positionid to denote the position of each unit in the text, where the unit can be defined as a word, a sentence, a paragraph, etc., according to specific requirements. for example, at the word-level, each word is assigned a unique positionid to indicate its position in the sequence, such as "the quick brown fox jumps over the lazy dog." can be converted into "the[1] quick[2] brown[3] fox[4] jumps @xcite lazy[8] dog @xcite ." when the position ids are assigned. similarly, at the sentence-level, "the quick brown fox jumps over the lazy dog. a swift auburn fox leaps across a sleeping canine." can be converted into "the quick brown fox jumps over the lazy dog. @xcite a swift auburn fox leaps across a sleeping canine.[2]" when the position ids are assigned. as shown in figure 1 , positionid prompting is a tuning-free technique that enables llms to count the number of units continuously during text generation. positionid fine-tuning involves training the model with data in a similar positionid prompting format, thereby enhancing the model's positional awareness. through different system prompts, this method allows flexible control to enable or disable the positionid prompting mode without compromising positional awareness. moreover, we validate another interesting feature brought by explicit positional awareness, namely, copy and paste (cp) abilities. to enable this, we propose positionid cp prompting, which involves a three-stage tool-use mechanism: (1) inserting position ids in the previous text upon generating a "<@xmath0, (2) continuing generating the tool call "<@xmath1, and (3) generating the tool call "<@xmath2.f., §3.2). to verify the length control (lenctrl) and copypaste (cp) abilities of llms, we constructed two datasets with diverse instructions: lenctrl-bench and cp-bench. lenctrl-bench includes a training set of 28,135 samples to enhance the positional awareness of open-source models, and a test set of 2,817 samples. cp-bench contains a test set of 182 samples carefully selected from the gpt-4 @xcite synthesized samples and the manually crafted samples. experiments on lenctrl-bench demonstrate that through positionid prompting and positionid fine-tuning, models can more accurately follow lengthcontrolled text generation instructions without compromising response quality. additionally, our work is the first to study the copy and paste tool-use abilities of current llms. experiments on cp-bench confirm the effectiveness of our proposed positionid cp prompting. it can not only accurately copy text spans, but also demonstrate superior response quality. ## datasets and benchmarks due to the lack of instruction datasets and benchmarks specifically designed to evaluate models' abilities to follow instructions with strict length constraints and to perform accurate copying and pasting, we construct two novel benchmarks: lenctrl-bench (length-controlled text generation benchmark) and cp-bench (copy-paste benchmark). both of these benchmarks are designed with diverse instructions, aiming to comprehensively evaluate the models' abilities to follow instructions while measuring their length control and copy-paste capabilities. ## method as shown in figure 1 , we have two pipelines: the positionid prompting and positionid fine-tuning are designed for length control ( §3.1); the posi-tionid cp prompting is used for precise copying and pasting ( §3.2). ## related works positional awareness of large language models (llms) significantly impacts their performance @xcite @xcite @xcite . techniques like abacus embeddings @xcite and contextual position encoding @xcite improve tasks such as mathematical computation and selective copying but require model retraining and are limited to open-source llms. they also do not address length control or real-world copying and pasting. previous research on length-controlled text generation @xcite @xcite includes methods like @xcite , which uses finetuning with length constraints but is only applicable to open-source models. our work introduces approaches suitable for both closed-source and opensource llms, focusing on length control and copying and pasting tasks. ## conclusion in conclusion, our work introduces positionid prompting and positionid fine-tuning to enhance large language models' positional awareness, enabling more accurate adherence to length control instructions without sacrificing response quality. additionally, we propose positionid cp prompting to address the challenge of copy-paste operations. by explicitly marking the position of text units, po-sitionid cp prompting allows models to accurately copy and paste specific spans of text. our experiments on lenctrl-bench confirm improvements in length control, while tests on cp-bench demonstrate the effectiveness of positionid cp prompting for copy and paste, highlighting its capability to maintain the integrity of the copied content and ensure precise placement during paste operations.