Title: KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
Venue: NeurIPS 2025
Accepted: yes
Keywords: LLM-based Multi-agent System; KV-cache reuse; Multi-context Agent Communication
Abstract: Multi-agent large language model (LLM) systems are increasingly adopted for complex language processing tasks that require communication and coordination among agents. However, these systems often suffer substantial overhead from repeated reprocessing of overlapping contexts across agents. In typical pipelines, once an agent receives a message from its predecessor, the full context-including prior turns-must be reprocessed from scratch, leading to inefficient processing. While key-value (KV) caching is an effective solution for avoiding redundant computation in single-agent settings where prefixes remain unchanged, it cannot be directly reused in multi-agent scenarios due to diverging prefixes introduced by agent-specific context extensions. We identify that the core challenge lies in the offset variance of KV-caches across agents. To address this, we propose **KVCOMM**, a training-free framework that enables efficient prefilling in multi-agent inference by reusing KV-caches and aligning cache offsets of overlapping contexts under diverse prefix contexts. KVCOMM estimates and adjusts KV-caches for shared content by referencing a pool of cached examples—termed *anchors*—that store observed cache deviations under varying prefixes. The anchor pool is maintained and updated online, allowing dynamic adaptation to distinct user requests and context structures. KVCOMM achieves over 70% reuse rate across diverse multi- agent workloads, including retrieval-augmented generation, math reasoning, and collaborative coding tasks, all without quality degradation. Particularly, when each fully-connected agent receives 1K input tokens with 512 prefix tokens and 512 output tokens under a five-agent setting, KVCOMM achieves up to 7.8× speedup compared to the standard prefill pipeline, reducing TTFT from ∼430ms to ∼55ms. Code is available at https://github.com/FastMAS/KVCOMM.


Peer reviews:
comments: SUMMARY:
This paper raises a key question: how can KV cache-based accelerated inference be effectively achieved in a multi-agent setting? It proposes a training-free calibration method, KVComm, which leverages KV cache offsets to enable kvcache reuse on placeholder texts with different prefixes.

STRENGTHS AND WEAKNESSES:
Strengths:
- This paper provides both theoretical and empirical evidence for a compelling observation: despite variations in prefixes, identical placeholder texts often yield similar KV caches, and the discrepancy between them can be effectively captured and exploited to accelerate inference in multi-agent settings.
- It is the first KV cache framework specifically designed for multi-agent scenarios, offering valuable infrastructure for advancing research in the multi-agent domain.
- The proposed method achieves effective acceleration without requiring any additional training, making it lightweight and practical.

Weaknesses:
- The writing is somewhat hard to follow, and Figure 3 is overly complicated, making it difficult to grasp the overall process.
- The paper introduces too many technical terms, making it unclear how the proposed method is concretely implemented based on the intuitive idea of “KV cache offset.”

QUESTIONS:
- What is the exact difference between the placeholder offset and the prefix offset? Could the author provide concrete examples to illustrate this? Why are both types of offsets necessary for multi-agent KV cache reuse?
- Is there any quantitative experimental result demonstrating the relationship between the number of anchors, GPU memory usage, and acceleration efficiency?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors identify the offset-variance problem that prevents naïve KV-cache sharing across agents, and propose KVCOMM—a training-free, anchor-based framework. The anchor pool will store representative anchors on the fly, interpolates and learns the offsets across layers to align future shared tokens, and reuses the saved anchors to accelerate the prefilling. KVCOMM can achieve up to 13.7 speedup with minimal performance degradations.\n\nSTRENGTHS AND WEAKNESSES:\n- **Strengths**\n  - Tackles the important problem of improving the efficiencies of multi-agent systems (MAS).\n  - Grounded with solid theory, providing formal error bounds and clear motivation.\n  - Reframes KV reuse as an *offset-alignment* problem and proposes an anchor-based interpolation mechanism.\n  - Includes extensive hyper-parameter ablation studies demonstrating the method’s robustness.\n\n- **Weaknesses**\n  - Experiments cover only a limited set of tasks and MAS setups; harder benchmarks such as Math500, AIME, and GAIA are not evaluated. \n  - Empirical comparison focuses mainly on a single baseline (cache blend). More possible baselines are helpful to get a comprehensive evaluation\n  - End-to-end (E2E) latency is not reported. It is unclear how much the speed-up of TTFT can be transferred to the E2E latencies and throughputs gains, which may be concerned more in the MAS settings.\n  - The anchor pools may introduce signifcant overhead, but this cost is not quantified. The KV cache could be very large for large models and complicated MAS trajectories. \n  - More fine-grained breakdowns are needed—for example, measuring the time to load retrieved elements into GPU memory when anchor pools are large and cannot be fully stored on-GPU.\n  - Sharing caches could raise safety or privacy concerns. Although this has already been discussed in the paper, no concrete measures is proposed to solve this concerns.\n\nQUESTIONS:\nIn the Table1, the performance of original settings on MMLU is quite low (47.1). Is it a typo?\n\nLIMITATIONS:\nMy main concerns are in the evaluation sections. It would be better if the author can provide more quantitative results. Please see my comments in the weakness section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses inefficient redundant KV-cache computation in multi-agent LLM systems caused by offset variance, where agents sharing overlapping contexts reprocess identical content due to divergent prefix lengths and dynamic placeholders, preventing static KV-cache reuse. The authors propose KVComm, a training-free framework enabling dynamic KV-cache reuse. KVComm employs an online anchor pool that tracks cache deviations under varying prefixes to estimate and align cache offsets for new contexts.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The authors effectively combine theoretical analysis (Propositions 1 and 2) with extensive experiments across diverse benchmarks (MMLU, GSM8K, HumanEval). They demonstrate significant results, including a 13.7× TTFT speedup and over 70% reuse rate under realistic multi-agent workloads. Ablation studies (Table 5) validate the necessity of each component. The work further includes comprehensive robustness evaluations, assessing sensitivity to hyperparameters (Table 6), request ordering (Table 4), and context length (Table 3).\n\nweakness:\n1.The paper assumes identical LLMs for all agents, but real-world multi-agent systems often leverage heterogeneous models (e.g., domain-specific LLMs). This severely limits practical applicability.\n2.Anchor matching relies on real-time embedding similarity calculations (Eq.5), yet latency overhead for long-context inputs is unmeasured. Given the softmax over anchors, this could negate prefilling speedup for large V. Please quantify this cost.\n3.While DroidSpeak solves cross-model KV-sharing, it's only cited without comparison. This omission undermines the claimed novelty.\n\nQUESTIONS:\n1.Proposition 1 assumes RoPE-based positional encoding. How does KVComm handle non-RoPE models? Experiments on such architectures would strengthen claims.\n\n2.The prefix segmentation requires rigid templates (Eq.1), making free-form agent interactions (e.g., unstructured debates) unsupported. The framework fails when placeholders aren't predefined—shown by omitting conversational benchmarks. \n\n3.Tests use ≤1K tokens (Table 3). How does reuse rate degrade for >4K contexts where cumulative positional deviations amplify?\n\nLIMITATIONS:\nIf the method only applies to strictly identical LLMs and rigidly fixed prefixes, and cannot accommodate free-form conversational interactions, the authors may wish to explicitly acknowledge these constraints in the limitations section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method for reusing (prefilling) cache in multi-agent systems called KVComm.  The method is similar to CacheBlend, but is more specific to the multi-agent setting and uses 'anchors' dictated by changes in the custom prefixes/prompts for the different agents to inform where the KV cache should be shared.  Results on several benchmarks can show some degradation at large prefill rates, but the speedup for the method is large and the benefits increase with the number of agents.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n* The problem itself is an important one - how do you save computation when sending many of the same tokens through agents of the same model?\n* The motivation for the method and general intuition is clearly presented\n* There are a lot of interesting results / graphs presented.  If I knew the subfield better, I'd love to dig into those at a detailed level\n\n## Weaknesses / Concerns\n* Your positioning of KVComm within existing methods wasn’t clear to me:\n  * You claim: “the first training-free, prompt-adaptive cache-sharing framework for efficient prefilling of multi-agent systems” <- why doesn't CacheBlend fit this?  Could you be more clear in your claim, since methods more general than multi-agent systems exist to do this? (The claim may be that it is more efficient specifically within multi-agent systems by leveraging X\"\n  * Is the method leveraging priors around prompts in a special way?\n  * What does “context-aware” mean?\n  * You mention and emphasize the assumption that other methods make that every agent runs the same model architecture.  It seems your method needs this assumption as well – is there a reason you emphasized it?  Could you make it clear that your method is similarly constrained (and taking advantage of this structure)\n  * Given the drastic difference between your method and CacheBlend on HumanEval, I think you should (1) very clearly explain what this difference is due to, and (2) make it part of your major differentiator/impact (what other problems is this likely to have a big impact on?  Where is CacheBlend being used that it doesn’t perform well?)\n  * Building up an anchor pool feels very similar to building up a database of past tokens themselves, i.e., speculative decoding methods – could you compare to them, at least in description / scenario?\n\n* I’m concerned that the results don’t line up with the CacheBlend results and know the authors said they didn’t optimize the prompts for the open source models.  I think it’s important to rerun these experiments to match CacheBlend as best you can so the numbers are comparable between papers.\n\n* Hardware/communications are usually a bottleneck in these situations.  It seems you don't address it in your paper - could you add clarification on:\n  * The fact that this method is constrained to single-GPU settings (as far as I understand it)? (and homogenous networks, which is a common assumption)\n  * Whether you needed to do anything special to store and retrieve the cache in an efficient way that was specific to this method?\n\n## Smaller suggestions\n* Cite “prior research” in “2 KV-Cache Sharing Scenario. Prior research has identified…”.\n* In the same section, point out that your method is ‘Selective Recomputation’ (It’s kind of weird to make that #2 out of 4 – put it at the beginning or the end?  Or explain why you’re working on that one in particular in this setting and make it into a separate paragraph, perhaps? \n* `Tab.` for table is hard to read.  Preference for `Table`\n\nQUESTIONS:\n1. Could you be more specific in your claims on novelty?\n2. Could you clarify the domain you're claiming your method should work well in? (specifically, do you intend for it to only be used in single-GPU, same-architecture situations?)\n3. As someone not very familiar with the field, I'd love more intuition behind the anchors of your method and the closest related thing in multi-agent or other areas in the literature (response editing, speculative decoding)\n4. It would be great to see experiments that line up with CacheBlend claims, and clear explanations for why that method is failing so badly on HumanEval / other times that it does.  Is the method dynamically recomputing the cache at a ~15% rate? Is this the vllm version?\n5. Please bold the 'best' results rather than your results in Table 1 (e.g., CacheBlend is better on MMLU with 2 agents, but it's not obvious from the current table formatting)\n\nLIMITATIONS:\nThe authors should limit the scope of their claims to single-GPU, same-architecture settings very clearly and acknowledge possible hardware communication bottleneck impact on performance in different settings, since this drives a lot of other KV cache works' considerations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}