Brevity is the soul of sustainability: Characterizing LLM response lengths (2025)

Abstract: A significant portion of the energy consumed by Large Language Models (LLMs) arises from their inference processes; hence developing energy-efficient methods for inference is crucial. While several techniques exist for inference optimization, output compression remains relatively unexplored, with only a few preliminary efforts addressing this aspect. In this work, we first benchmark 12 decoder-only LLMs across 5 datasets, revealing that these models often produce responses that are substantially longer than necessary. We then conduct a comprehensive quality assessment of LLM responses, formally defining six information categories present in LLM responses. We show that LLMs often tend to include redundant or additional information besides the minimal answer. To address this issue of long responses by LLMs, we explore several simple and intuitive prompt-engineering strategies.Empirical evaluation shows that appropriate prompts targeting length reduction and controlling information content can achieve significant energy optimization between 25-60% by reducing the response length while preserving the quality of LLM responses.

## introduction large language models (llms) have seen rapid adoption in different applications, including virtual assistants for daily life, and in scientific research. however, the deployment of these models require massive computational resources, leading to significant energy consumption and carbon emissions. although a single inference operation may not consume much energy, such operations are performed enormously many times under deployment. for example, openai's chatgpt processes more than 1b queries per day, which consumes an estimated 0.5gwh of energy. in recent times, ninety percent of the total cloud computing demand for amazon web services was for model inference purposes @xcite . hence, llm inferences contribute to a significant energy impact @xcite and merit further investigation. our prior work established that the energy consumption during inference is closely correlated with the length of the response generated by llms @xcite . notably, output length has a greater impact on inference energy usage than input length because inputs may benefit from parallel processing and caching on gpus, whereas output generation is predominantly sequential. additionally, we demonstrated in @xcite that the energy expended during inference is largely independent of task complexity or type, depending largely on the output size. together, these findings highlight the potential for optimizing inference energy consumption by limiting the output length. a key consideration in this context is how popular llms behave -specifically, whether llms tend to generate more text than necessary for certain tasks. to investigate this, in this work, we benchmark 12 llms across 5 tasks/datasets spanning different use cases. we focus on factual questionanswering datasets primarily consisting of factual and procedural questions, deliberately excluding opinion-based questions, as determining their minimal response length can be challenging. our primary finding is that, for factual questions, llmgenerated responses are significantly longer than the target responses, regardless of whether the targets are human-written or machine-generated. it can be argued that longer answers are more useful to the users, since the additional information beyond the core information need -such as explanations, summaries, or conversational enhancements -can enhance user experience. but it is important to recognize that such enhancements come at the cost of increased energy consumption and environmental costs. a useful analogy is air travel via economy vs. business class, where the core need (timely travel) is distinct from optional enhancements that improve the experience but come at an additional cost. this is the first work to bring forward the trade-off between energy efficiency and user satisfaction, a pivotal contribution that unveils an entirely new perspective on llm responses. next, we categorize the llm-generated responses by identifying six distinct information categories through a comprehensive manual survey. this categorization reveals that llm responses often include, along with the minimal answers, explanations, examples, and reasoning information, as well as redundant or irrelevant text. in fact, our study reveals that, on average, the core answer constitutes only about 42% of the total responses. while the extra elements can enhance readability and potentially increase user satisfaction (for instance, users might appreciate supplementary context or a polite add-on), they also incur higher inference energy costs. consequently, we argue that llms should be aligned to generate minimal answers by default, incorporating only a limited amount of additional content and excluding extraneous or conversational tokens. finally, we explore several prompt-based strategies to generate more concise responses from llms. these include simple directives (such as "answer briefly"), providing in-context examples to guide the llm toward optimal response length, specifying a desired output length, and reducing non-essential content such as conversational and irrelevant information. we evaluate these prompt engineering strategies in terms of how they affect the length, energy, and information content of the llm responses. we also check how they affect the quality of llm responses in terms of rouge-l f1 scores with the target (gold standard) answers. empirical evaluation shows that some of our prompts achieve significant energy optimization by reducing the response lengths, while preserving (or sometimes even improving) the quality of llm responses. for instance, one of our best strategies is simply where we direct the llm to provide only the minimal answer; this strategy achieves up to 88% reduction in llm responses, which leads up to 58% reduction in energy consumption, while actually improving the response quality. ## some common strategies for shortening llm responses we now focus on some common strategies for shortening llm responses to achieve energy-efficient inference. we explore simple, intuitive white-box approaches, including response truncation and finetuning of llms, as well as black-box strategies based on prompt engineering. truncating responses refers to stopping the generation early, e,g., just after the minimal answer is obtained. however, we notice that in about 19% of the responses studied in section 3.4, the minimal answer is not at the beginning of the llm response. in most of these cases, the llms output conversational enhancers first to set up the context or for readability, followed by the minimal answer, thus limiting the scope of strategies for truncating responses. fine-tuning llms: next we explore whether simple fine-tuning can help in controlling the response length. toward that, we performed some basic fine-tuning experiments. we fine-tuned phi-3-mini (one of the llms that generate very long responses, as stated earlier) with a pool of human-written question-answers from the dolly and gooaq datasets, using lora with @xmath0, for 100 epochs. the models were trained on the target answer tokens with the hypothesis that it would enable the models to generate responses similar to humans on these particular datasets. we compare the average ratio of generated and target response lengths for the default model and the fine-tuned model. we observe that simple fine-tuning is leading to 1.24 times higher ratio for dolly and 1.26 times higher ratio for gooaq, in comparison with the default model. this is an unexpected result, where fine-tuning is (slightly) increasing the response length instead of shortening the length. this behavior could stem from extensive pretraining on large-scale data, making small-scale fine-tuning insufficient to alter its learned patterns. additionally, a model's intrinsic stopping mechanism may not be functioning as intended after the fine-tuning, causing it to overestimate the length for the responses. therefore, it is prudent to explore prompt engineering strategies leveraging llm's zero-shot capabilities, drawing on our observations in section 3. we explore this direction in the next section. ## prompt engineering strategies for controlling llm responses here we investigate various prompt engineering strategies aimed at generating concise llm responses, thereby reducing inference energy. ## concluding discussion in this work, we comprehensively explore output compression for energy-efficient inference. our specific contributions are as follows. (1) we are the first to systematically establish that large language models (llms) often generate very long responses which lead to high energy consumptions; we also strategy gemma-2-9b response query: 29 cm is equal to how many m? default here's how to convert centimeters (cm) to meters (m): **remember the conversion:** 1 @xmath1. **divide by 100:** to find out how many meters are in 29 cm, divide 29 by 100. 29 cm / 100 cm/@xmath2.29 meters therefore, 29 cm is equal to 0.29 meters minans 0.29 brief 0.29 m maddnored 0.29 **explanation:** there are 100 centimeters (cm) in one meter (m). to convert from centimeters to meters, divide by 100." query: <information from different @xmath3: what is a necromonger in riddick? default in the chronicles of riddick film franchise, necromongers are a fanatical warrior-religious order ... <long additional @xmath4. brief a necromonger is a member of a fanatical religious empire in the *riddick* franchise. they believe life is an enemy of the universe and seek to conquer and convert all others through violence. maddnored a necromonger is a member of a fanatical religious empire in the *chronicles of riddick* franchise. they believe life is an affront to the universe and seek to eradicate it through conversion or death. their society is highly structured with strict rules and rituals. query: how to connect iphone to lg tv to play music? default here's how to connect your iphone to an lg tv for screen mirroring, along with some important things to keep in mind: <long additional @xmath5. then, select ""screen mirroring"" in control center and choose your lg tv from the list. brief 1. **enable miracast on your lg tv.** (@xmath6. **turn on airplay mirroring on your iphone.** (control @xmath7. 3. **select your lg tv from the list of available devices.** let me know if you need detailed instructions! maddnored 1. **enable miracast on your lg tv.** (@xmath8. **turn on airplay mirroring on your iphone.** (control center). your iphone should now appear as a connection option on your lg tv. select it to mirror your screen. the notion of an ideal response length is subjective and varies significantly among individuals. as a result, a one-size-fits-all approach may not effectively meet the diverse needs of users. here we argue that the default behaviour of llms can be tuned so that they generate shorter responses, so that energy consumption is minimized. a user can ask for more details or longer responses if she desires. (3) use of standard datasets: our experiments have been performed on publicly available rich data for testing llms. however, the llms we used might have been pre-trained on parts of these standard datasets, causing them to behave in a certain way. this may lead to slight variations in results for real-world use cases, but the overall findings should be generalizable enough.