Length Representations in Large Language Models (2025)

Abstract: Large language models (LLMs) have shown remarkable capabilities across various tasks, that are learned from massive amounts of text-based data. Although LLMs can control output sequence length, particularly in instruction-based settings, the internal mechanisms behind this control have been unexplored yet. In this study, we provide empirical evidence on how output sequence length information is encoded within the internal representations in LLMs. In particular, our findings show that multi-head attention mechanisms are critical in determining output sequence length, which can be adjusted in a disentangled manner. By scaling specific hidden units within the model, we can control the output sequence length without losing the informativeness of the generated text, thereby indicating that length information is partially disentangled from semantic information. Moreover, some hidden units become increasingly active as prompts become more length-specific, thus reflecting the model’s internal awareness of this attribute. Our findings suggest that LLMs have learned robust and adaptable internal mechanisms for controlling output length without any external control.

## introduction large language models (llms) have gained considerable attention in recent years for their remarkable task-solving capabilities @xcite @xcite . llms are trained to predict the next token in a sequence. they can produce coherent and informative text, which demonstrates their implicit understanding of diverse linguistic structures @xcite @xcite . furthermore, they also learn when to stop generating text to ensure that the output adheres to appropriate length constraints @xcite . controlling output sequence length in llms is crucial for real-world applications, such as text summarization @xcite @xcite , machine translation @xcite @xcite , knowledge qa, and dialogue generation @xcite , that necessitate fitting content within specified length limits without losing informativeness. therefore, the number of studies attempting to improve length controllability has increased drastically @xcite @xcite . based on advancements in instruction-based llms, it is observed that injecting constraints into prompts can further effectively control output length without requiring model modifications @xcite . however, these prompt engineering methods mainly focus on external controls, and it has not been explored yet how llms internally encode and constrain output sequence length. understanding these internal mechanisms is critical for achieving precise length control, while enhancing the interpretability and robustness of llms in generation-based systems. herein, we aim to investigate how output sequence length information is encoded within the internal representations of general transformer architectures. specifically, we first investigate which components within llm transformer layers contribute to length control. our findings reveal that the outputs from multi-head attention mechanisms in the lower layers play a key role in determining and controlling output sequence length in a tunable and disentangled manner. for this work, we utilize a sentence summarization task, which often requires adherence to desired summary lengths, and employ models from the llama @xcite , @xcite , and qwen-2.5 @xcite families. we empirically demonstrate, based on human evaluations, that we can adjust output length during generation without losing the informativeness of texts by scaling specific hidden units within the outputs from the lower layers of multi-head attention mechanisms. for instance, multiplying certain hidden units with negative numbers results in longer text, while multiplying them by positive numbers generates more concise texts without losing informativeness. furthermore, certain hidden units related to length information show increasing activity as prompts become more specific regarding length constraints. these units appear to be directly involved in controlling output length, indicating that llms have learned to process length-related information as a distinct feature, partially disentangled from other semantic information. moreover, we find that the same highly activated hidden units are consistently involved in length control even after fine-tuning, regardless of length constraints in prompts @xcite . our code is available at https://github.com/mcat00/gilab_length . ## related work large language models. in recent years, llms have achieved considerable success due to their remarkable task-solving abilities, specifically in zero-shot settings @xcite . llms are broadly categorized into open and closed models. the open models, such as the llama or phi family, offer flexible access to modify their architectures, while the closed models, such as chatgpt, 1 have demonstrated remarkable reasoning abilities in various natural language processing tasks @xcite @xcite @xcite . recent studies have focused on finding better methods to prompt llms @xcite @xcite . ## finding length representations our goal is to understand whether and how length representations are encoded in llms when using various length-constraint prompts. for this, we extracted outputs from different components and layers of transformer architectures during text generation. we then applied regression to predict the generation time steps from these hidden states. summarization dataset. we used the google sentence summarization dataset foot_0 @xcite in an instruction-based format, following previous work foot_1 @xcite because recent studies on length control still faces challenges in managing it with llms @xcite . ## length representations in llms we explored which transformer layers and outputs contain length information, how length-specific prompts and quantization affect length representations, and the impact of fine-tuning on llms. ## disentangling length representations the previous section explored which components and layers contain length representations for output sequence length with varying prompts. while the second attention layer has a strong correlation with length representations, this does not indicate which hidden units are actually responsible for controlling the output sequence length. thus, the specific hidden units must be identified for a better understanding of llms' length control. ## discussion and conclusion we examined how llms encode output sequence length in their internal representations. our findings empirically demonstrated that the outputs from the second layer's attention mechanisms showed a strong correlation with the generation time step, thus indicating that length representations were captured early in the process. we also found that this pattern is consistent with different models with different sizes and continues to be robust even when 4-and 8-bit quantizations were applied. furthermore, we analyzed individual hidden unit from the second layer attention outputs and found that certain hidden units are highly activated and directly contribute to the process of representing length information. moreover, these units became more active when length-specific prompts such as priming were used. this finding indicates that llms adjust their internal representations based on the input prompts. furthermore, by scaling these lengthrelated hidden units, we effectively controlled the output sequence length without losing informative-ness, that indicates that length information is partially disentangled from semantic representations within llms. finally, our results revealed that fine-tuning further improves the llms' capabilities by reinforcing reliance on the top-k lengthrelated units. we also found the same activation of specific hidden units in the priming prompt are shared between zero-shot and fine-tuning settings, that indicates llms have constructed robust internal mechanisms for controlling output sequence length, and in-context learning performs similarly to implicit fine-tuning @xcite .