Demystify Verbosity Compensation Behavior of Large Language Models (2025)

Abstract: Recent work has revealed Large Language Models (LLMs) often exhibit undesirable behaviors, such as hallucination and toxicity, limiting their reliability and broader adoption. In this paper, we discover an understudied type of undesirable behavior of LLMs, which we term Verbosity Compensation (VC). VC is similar to the hesitation behavior of humans under uncertainty, compensating with excessive words such as repeating questions, introducing ambiguity, or providing excessive enumeration. We present the first work that analyzes Verbosity Compensation, explores its causes, and proposes a simple mitigating approach. Our experiments on five datasets of knowledge and reasoning-based QA tasks with 14 LLMs, reveal three conclusions. 1) A pervasive presence of VC across all models and all datasets. 2) The large performance gap between verbose and concise responses. We also demonstrate that this difference does not naturally diminish as LLM capability increases. 3) Higher uncertainty exhibited by VC responses across all five datasets, suggesting a strong connection between verbosity and model uncertainty. We propose a simple yet effective cascade algorithm that replaces the verbose responses with the other model-generated responses, alleviating the VC of the Mistral model from 63.81% to 16.16% on the Qasper dataset.

## introduction recent research has highlighted various undesirable behaviors of large language models, such as hallucination @xcite , toxicity @xcite , and ethical bias @xcite , which pose significant risks to users. among them, the verbose response issue where llms respond with excessive words has attracted more and more attention in the llm era because of unnecessary long output for solving problems @xcite and the unavoidable high cost of llmgenerated tokens. the existing work mainly focuses on the length of the response and its applications. researchers found that imposing a length constraint in the prompt can improve the performance of llms, under chain-of-thought @xcite and machine translation @xcite settings. @xcite found rlhf training favors the lengthy response. however, length is not enough to analyze verbosity as it provides a general overview but fails to capture key fine-grained features such as content structure. in this paper, we discover a type of undesirable verbosity behavior of llms. we term it verbosity compensation (vc). instead of focusing merely on the length, we analyze the frequency, types, and their relation to model performance. we also find vc is closely connected to the uncertainty of llms, demystifying the mechanism of the vc behavior, and improving the understanding of both vc and uncertainty. interestingly, vc is similar to the hesitation behavior of humans under uncertainty @xcite . figure 1 shows a motivating example. in the first response, llm generates a concise answer that is correct with low uncertainty. in the second and third responses, instead of generating an answer concisely, such as "16.5", llm repeats the question, and produces ambiguity, leading to a vc response with low performance and high uncertainty. vc is harmful and undesired for both users and servers. for the users, vc will lead to confusion and inefficiency @xcite . when an llm enumerates multiple answers, users are unclear about which one is correct. besides, vc leads to bias among users of different length preferences if verbose answers attain higher/lower scores. for the servers, the verbosity leads to unnecessary higher costs and higher latency because of useless tokens. to analyze the vc behavior systematically, we unify four long-context question-answering figure 1 : an illustration of comparison between concise and verbose responses. for each example, we ask the model to generate the response as concisely as possible. in the first response, llm generates a concise answer, while in the second and third responses, llm performs repeating, and ambiguity, leading to a verbose response with low performance and high uncertainty (detailed numbers in appendix b.2). next, we measure the uncertainty of model responses using perplexity and laplacian scores for open and closed-source models. we find that verbose responses exhibit higher uncertainty across all five datasets, suggesting a strong connection between verbosity and model uncertainty. finally, we leverage the connection between performance and vc to develop a routing algorithm that obtains significant improvements over the random selecting baseline and uncertainty-based routing. to mitigate vc in llms, we propose a simple yet effective cascade algorithm that replaces verbose responses with responses of larger llms. experiments demonstrate the efficacy of the proposed algorithm through tests on three model combinations: gemma to gemini, mistral to gpt-4, and llama to claude. the results show that our approach effectively alleviates the vc of the mistral model from 63.81% to 16.16% on the qasper dataset. the insights above can inspire the development of practical applications and effective mitigation strategies. future work can mitigate the uncertainty of the llms by alleviating vc behavior due to the proposed connections between them. ## verbosity compensation in this section, we first introduce the definition and quantification of vc, and then we propose the metrics for evaluating the correlation between verbosity compensation and performance, uncertainty, and alleviating it with llm routing. ## result and analaysis in this section, we analyze verbosity compensation and its connection with performance and uncertainty. then, we evaluate the cascade algorithm. ## conclusion in this paper, we define vc and propose a comprehensive benchmark to evaluate 14 llms, revealing they suffer significantly from five types of vc. we conduct a rigorous analysis and connect vc to 1) model performance and 2) model uncertainty, shedding light on future applications and mitigation. we propose a simple but effective cascade approach to mitigate verbosity compensation in llms, and our extensive experiments show it is highly effective.