ACON: Optimizing Context Compression for Long-horizon LLM Agents
Authors: name: Minki Kang, name: Wei-Ning Chen, name: Dongge Han, name: Huseyin A. Inan, name: Lukas Wutschitz, name: Yanzhi Chen, name: Robert Sim, name: Saravan Rajmohan
Organization: Microsoft

Large language models (LLMs) are increasingly deployed as agents in dynamic,
real-world environments, where success requires both reasoning and effective
tool use. A central challenge for agentic tasks is the growing context length,
as agents must accumulate long histories of actions and observations. This
expansion raises costs and reduces efficiency in long-horizon tasks, yet prior
work on context compression has mostly focused on single-step tasks or narrow
applications. We introduce Agent Context Optimization (ACON), a unified
framework that optimally compresses both environment observations and
interaction histories into concise yet informative condensations. ACON
leverages compression guideline optimization in natural language space: given
paired trajectories where full context succeeds but compressed context fails,
capable LLMs analyze the causes of failure, and the compression guideline is
updated accordingly. Furthermore, we propose distilling the optimized LLM
compressor into smaller models to reduce the overhead of the additional module.
Experiments on AppWorld, OfficeBench, and Multi-objective QA show that ACON
reduces memory usage by 26-54% (peak tokens) while largely preserving task
performance, preserves over 95% of accuracy when distilled into smaller
compressors, and enhances smaller LMs as long-horizon agents with up to 46%
performance improvement.