GUARD : Glocal Uncertainty-Aware Robust Decoding for Effective and Efficient Open-Ended Text Generation (2025)

Abstract: Open-ended text generation faces a critical challenge: balancing coherence with diversity in LLM outputs. While contrastive search-based decoding strategies have emerged to address this trade-off, their practical utility is often limited by hyperparameter dependence and high computational costs. We introduce GUARD, a self-adaptive decoding method that effectively balances these competing objectives through a novel “Glocal” uncertainty-driven framework. GUARD combines global entropy estimates with local entropy deviations to integrate both long-term and short-term uncertainty signals. We demonstrate that our proposed global entropy formulation effectively mitigates abrupt variations in uncertainty, such as sudden overconfidence or high entropy spikes, and provides theoretical guarantees of unbiasedness and consistency. To reduce computational overhead, we incorporate a simple yet effective token-count-based penalty into GUARD. Experimental results demonstrate that GUARD achieves a good balance between text diversity and coherence, while exhibiting substantial improvements in generation speed. In a more nuanced comparison study across different dimensions of text quality, both human and LLM evaluators validated its remarkable performance. Our code is available at https://github.com/YecanLee/GUARD.

## introduction neural text generation models based on the transformer decoder @xcite have revolutionized natural language generation tasks such as story creation @xcite , context completion @xcite , or dialogue generation * equal contribution † corresponding author @xcite . decoding strategies play a crucial role in text generation with large language models (llms), affecting both the quality of the generated text and the computational efficiency during inference. the most commonly used strategies can be broadly categorized in deterministic @xcite and stochastic approaches @xcite @xcite . deterministic approaches are likelihood-based techniques that typically (over-)emphasize coherence at the cost of diversity and are prone to producing repetitive text, a phenomenon referred to as degeneration. stochastic approaches, on the other hand, aim to improve the diversity of texts, potentially sacrificing coherence and leading to semantic inconsistencies @xcite . this inherent coherence vs. diversity trade-off @xcite has motivated recent work to develop decoding strategies able to balance these two competing objectives. in particular, contrastive search (cs; @xcite introduced a weighted combination of model confidence (favoring coherence) and a degeneration penalty (promoting diversity) controlled by fixed, tunable hyperparameters throughout the generation. adaptive contrastive search (acs; garces arias et al., 2024) addressed this hyperparameter dependence by dynamically adjusting them based on the local uncertainty of the model. in doing so, this approach still has its limitations as it relies only on local uncertainty information, and it substantially increases the computational overhead at inference time. in this study, we aim to address the following research questions (rq): stochastic strategies based on sampling have emerged as a response to the limitations of deterministic approaches, particularly addressing text monotony and degeneration. temperature sampling @xcite introduces a hyperparameter τ that modulates the sharpness of the output distribution, enabling control over generation diversity. top-k @xcite and @xcite truncate the output distribution to avoid sampling from less reliable tail probabilities. typical sampling @xcite constrains the sampling distribution to tokens whose negative log-probabilities fall within a specific range of the model's conditional entropy, while min-p sampling @xcite introduces dynamic truncation based on the model's confidence. adaptive decoding (ad, @xcite employs an entropy-based confidence metric to optimize candidate selection. while these sampling methods effectively reduce degeneration, they risk inconsistency, compromising overall text quality @xcite . contrastive strategies typically aim at balancing coherence and diversity in text generation tasks. contrastive decoding @xcite leverages the differential between expert and amateur language models to select tokens that maximize their log-likelihood difference. cs evaluates top-k tokens using a combination of model confidence and degeneration penalty to enhance output diversity. acs builds upon cs by eliminating hyperparameters, addressing the unexplored impact of hyperparameter selection on generation quality @xcite , at the cost of additional computational overhead. however, its focus on immediate uncertainty overlooks temporal fluctuations that could benefit from more robustness. these limitations motivate guard, a novel strategy that enables an efficient, self-adaptive uncertainty-guided decoding. prefix: my eyes moved involuntary from the pages of the book that i was no longer reading, to the faces of the girls who were now looking over at me. my model output distribution eyes : 0.19 heart : 0.10 gaze : 0.05 …… instantaneous entropy global entropy loc  global  next token scoring heart : 0.31 eyes : 0.29 gaze : 0.15 …… token count cs glocal uncertainty-aware robust decoding (guard) weighting generation: heart beat faster as their gazes lingered upon me and, without even realising it myself, i was smiling back, my cheeks flushed and my eyes shining with happiness. reference: heart began to pound in my chest, and i could hear the blood rushing in my ears. i could feel the heat of the color burning on my cheeks. i was embarrassed, no, i was furious. top-normalization t k t  t k figure 1: guard leverages local and global shannon entropy deviations as proxies for model uncertainty, automatically adjusting cs parameters and incorporating a token count penalty for next-token selection. ## conclusion we introduce guard, a self-adaptive decoding strategy that adaptively balances text coherence and diversity by responding to fluctuations in longterm and short-term uncertainty. by integrating a token frequency penalty into cs, guard reduces repetitive outputs and computational overhead while maintaining context fidelity. theoretical analysis confirms that our estimation of global entropy preserves key statistical properties, such as unbiasedness and consistency. comprehensive experiments across multiple open-source models, datasets, metrics, human and llm evaluations demonstrate improvements in coherence, fluency, diversity, efficiency, informativeness, interestingness, and story development over established decoding methods. overall, guard offers a robust approach for improving open-ended text generation. future work will extend our method to additional text-generation tasks. we believe that investigating its integration with supervised fine-tuned models and exploring its performance in multilingual and low-resource settings might yield valuable insights.