ERGO : Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models (2025)

Abstract: Large Language Models (LLMs) suffer significant performance degradation in multi-turn conversations when information is presented incrementally. Given that multi-turn conversations characterize everyday interactions with LLMs, this degradation poses a severe challenge to real world usability. We hypothesize that abrupt increases in model uncertainty signal misalignment in multi-turn LLM interactions, and we exploit this insight to dynamically realign conversational context. We introduce ERGO (Entropy-guided Resetting for Generation Optimization), which continuously quantifies internal uncertainty via Shannon entropy over next token distributions and triggers adaptive prompt consolidation when a sharp spike in entropy is detected. By treating uncertainty as a first class signal rather than a nuisance to eliminate, ERGO embraces variability in language and modeling, representing and responding to uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO yields a 56.6% average performance gain over standard baselines, increases aptitude (peak performance capability) by 24.7%, and decreases unreliability (variability in performance) by 35.3%, demonstrating that uncertainty aware interventions can improve both accuracy and reliability in conversational AI.

## introduction large language models (llms) have become the primary interface for conversational ai systems, enabling users to interact through multi-turn exchanges. however, recent research has documented a critical limitation: llms often get 'lost' in conversation and experience substantial performance degradation in multi-turn conversations compared to single-turn interactions @xcite . this degradation manifests as ‡ lead author reduced accuracy, lower confidence, and a 112% increase in unreliability, posing significant challenges for real-world deployment @xcite . while prior work has measured this degradation, existing mitigation strategies remain limited. approaches based on task classification, retrieval, or context compression lack generality or require fine-tuning @xcite . we hypothesize that spikes in model uncertainty signal moments of conversational drift and by explicitly representing this uncertainty and monitoring its fluctuations, we can detect when an llm begins getting 'lost' in conversation. we introduce ergo (entropy-guided resetting for generation optimization), the first practical intervention framework that dynamically monitors internal uncertainty signals and resets context when needed. ergo computes shannon entropy over next-token probability distributions @xcite as an internal behavioral signal to detect spikes in uncertainty that indicate breakdown in comprehension. when such spikes occur, ergo triggers entropyguided prompt reconstruction, mitigating accumulated ambiguity and restoring coherence. unlike static prompt engineering, ergo's reconstruction is dynamically triggered by entropy thresholds and systematically preserves only those contextual elements that sustain both internal coherence and external task performance, discarding accumulated noise. a visual representation of this can be seen in figure 1 . empirical results demonstrate that targeted interventions based on uncertainty peaks not only recover task accuracy but also improve consistency across turns. in extensive simulations with incrementally revealed instructions, ergo improves average performance by 56.6% compared to standard multi-turn baselines, increases aptitude levels by 24.7% (best-case performance capability), and reduces the increased unreliability (variability in response consistency) observed in multi-turn settings by 35.3%. furthermore, ergo outperforms existing alternative strategies, and triggers resets with greater precision and timing compared to alternate baselines illustrating the potential of uncertaintyaware methods for robust conversational ai. to verify our findings and reproduce the results, please refer to the code repository found at the following link: https://github.com/haziq-exe/ergo 2 background and related works recent work has documented significant performance degradation in multi-turn llm conversations. @xcite demonstrated that model performance dropped by 39% on average in multiturn settings across six domains. @xcite formalized task-switch sensitivity using probability ratios, showing how conversation history compounds model confusion. while @xcite managed to mitigate average performance losses by 15-20%, their approaches faced substantial verbosity and practicality constraints (sec 5.4). agentbased frameworks @xcite explore systemlevel solutions but do not target fundamental model limitations during generation. ## conclusion our results show that ergo effectively mitigates multi-turn llm performance degradation by using shannon entropy to detect model confusion and trigger prompt restructuring. despite its simplicity, shannon entropy serves as a reliable signal for targeted context consolidation, minimizing unnecessary resets. ergo consistently outperformed existing methods, achieving 56.6% performance gains over standard baselines, improving aptitude by 24.7%, and reducing unreliability by 36.3%. correlation analysis confirmed that entropy-based resets reflect genuine model uncertainty rather than response length. as a practical, model-agnostic framework, ergo enhances conversational coherence in real-world deployments, with future work focused on advanced context consolidation strategies such as multi-stage summarization and adaptive techniques for long-form conversations.