KVComm: Enabling Efficient LLM Communication through Selective KV Sharing
Authors: Xiangyu Shi
Marco Chiesa
Gerald Q. Maguire Jr.
Dejan Kostic
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Multi-Agent Systems
Inter-LLM Communication
Multi-agent Debate
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose KVComm, a communication framework that enables efficient inter-LLM collaboration by selectively sharing key-value pairs, achieving near upper-bound performance with significantly reduced communication cost.

Abstract: Large Language Models (LLMs) are increasingly deployed in multi-agent systems, where effective inter-model communication is crucial. Existing communication protocols either rely on natural language, incurring high inference costs and information loss, or on hidden states, which suffer from information concentration bias and inefficiency. To address these limitations, we propose KVComm, a novel communication framework that enables efficient communication between LLMs through selective sharing of KV pairs. KVComm leverages the rich information encoded in the KV pairs while avoiding the pitfalls of hidden states. We introduce a KV layer-wise selection strategy based on attention importance scores with a Gaussian prior to identify the most informative KV pairs for communication. Extensive experiments across diverse tasks and model pairs demonstrate that KVComm achieves comparable performance to the upper-bound method, which directly merges inputs to one model without any communication, while transmitting as few as 30\% of layers' KV pairs. Our study highlights the potential of KV pairs as an effective medium for inter-LLM communication, paving the way for scalable and efficient multi-agent systems.

Published as a conference paper at ICLR 2026
KVCOMM: ENABLING EFFICIENT LLM COMMUNICA- TION THROUGH SELECTIVE KV SHARING
Xiangyu Shi, Marco Chiesa, Gerald Q. Maguire Jr., Dejan Kostié KTH Royal Institute of Technology (xi angyus, , mch ess, maguire, re, dmk) } @@kth.se se
ABSTRACT
Large Language Models (LLMs) are increasingly deployed in multi-agent sys- tems, where effective inter-model communication is crucial. Existing communi- cation protocols either rely on natural language, incurring high inference costs and information loss, tration bias and inefficiency. To address these limitations, we propose KVComm, a novel communication framework that enables efficient communication between LLMs through selective sharing of KV pairs. KVComm leverages the rich infor- mation encoded in the KV pairs while avoiding the pitfalls of hidden states. We introduce KV layer-wise selection strategy based on attention importance scores with Gaussian prior to identify the most informative KV pairs for communica- tion. Extensive experiments across diverse tasks and model pairs demonstrate that KVComm achieves comparable performance to the upper-bound method, which directly merges inputs to one model without any communication. while transmit- ting as few as 30% of layers' KV pairs. Our study highlights the potential of KV pairs effective medium for inter-LLLM communication, paving the way for scalable and efficient multi-agent system!"]
hidden states, which suffer from information concen-
INTRODUCTION
Large Language Models (LLMs) have catalyzed a paradigm shift from isolated model capabilities towards collaborative multi-agen systems (Guo et al. 2024 Tran et al. (025). CAMEL (Li et al., 2023), AutoGen (Wu et al. 2024), and ChatDev (Qian al., 2023) have demonstrated the potential of LLMs to collaborate effectively in multi-agent systems, achieving impressive results in various tasks. These systems leverage the strengths of individual LLMS and enable them to work together to solve complex problems that are beyond the capabilities of a single model (Yang et al., 2024a).
While multi-agent systems have shown great promise, they also introduce new challenges, particu- larly in the area of inter-agent communication. Effective communication between LLMs is crucial for the success of multi-agent systems. Explicit communication through natural language has been explored in several works, enabling the models to share information (Du et al. 2023), coordinate their actions (Sun et al. 2025), and make collective decisions (Yang et al. 2024b).
However, natural language communication leads to high inference costs due to the need for multi- ple decoding steps, and may not fully capture the rich information that needs to be shared between Large Language Models, as information is lost in the sampling process (Pham et al.) 2023] Ramesh & Li 2025) that occurs as each new token is produced. To address this limitation, recent works have explored alternative communication protocols that leverage the internal representations of LLMs. CIPHER (Pham et al., 2023) proposed to use the embedding space as the medium of communica- tion between LLMs. Namely, they pass the weighted average of the token embeddings from one LLM to another, facilitating more efficient information exchange. Rather than using the embedding space, AC (Ramesh & Li 2025) transmits the intermediate activations, specifically the last token's hidden state. They replace the last token's 's hidden state of the receiver's model (M,) with that of the sender's model (M,), allowing a more direct transfer of information. While these methods have shown promising results, they still face challenges in terms of communication efficiency and effec- tiveness. CIPHER (Pham et al. 2023) still requires multiple decoding steps, which can costly, Our code and data he found at http:://ghhhiiaoommm///////a///a

---

Published as a conference paper at ICLR 2026
Calibration
Inference M,
Output
M,
M,
Selected KV
Mg
Extract attention
Compute score
Layer
L
^
Model Input/Oulput
KV pairs of M,
KV pairs with score
Decoding activation
Attention within M,, M,
Prefill activation of MM,
Prefill activation of M,
Selected KV pairs
Attention models
Figure 1: KVComm framework for efficient LLM communication through selective KV sharing.
and AC (Ramesh & Li 2025) may lead to information loss as only limited activation information is transmitted.
We start with the question: What is the most effective way to communicate between LLMs? We argue that an ideal communication protocol should satisfy the following criteria: @ Effectiveness: should enable M, to effectively utilize the information from M,, @ Efficiency: It should minimize the computation needed by M, and the amount of data transmitted between models. @ Generality: It should be applicable to wide range of tasks and model architectures, ensuring its versatility in different scenarios. We choose to use activation information as the medium communication, as no decoding steps needed for M,, and M, can directly utilize the rich information encoded in the activations. We study different types of activation information (i.e., hidden states and KV pairs), and in Section 2.2 show that hidden states suffer from information concentration bias, where the last token's ' hidden state contains most of the information needed for the model's output. This makes it challenging to design an effective communication protocol using the last token's 's hidden state. Furthermore, we find that using all tokens' hidden states from a single layer of the sender M, does not guarantee effective communication. A dilemma arises: if the hidden states are taken from the early layers of M,, the computation benefit is limited since the computation cost is similar to concatenating the two inputs; if the hidden states are prepended the later layers of M,, the performance drops significantly.
Based on these observations, we propose KVComm, a novel communication protocol that enables efficient communication between LLMs through selective sharing of KV pairs. KV pairs are the most representative activation information in each layer, and sharing them does not interact with the hidden states of M, directly, while M, can decide how to utilize the information through the attention mechanism. To further improve the efficiency of communication. we propose a selection strategy to choose which (potentially concoiniiguus) layers' KV pairs to share. We formulate hypotheses that (H1) KV pairs from intermediate layers encode transferable semantic knowledge, more effective for
and (H2) KV pairs from layers exhibiting stronger attention distributions communication. These hypotheses are validated by our experiments in Sections 4.3 and 4.5) Based on these hypotheses, we define attention importance scores for each layer based on the average attention weights assigned to the context tokens. We also apply a Gaussian distribution centered certain layer as a prior on the attention importance scores. The intuition that the Gaussian distribution encourages selecting layers around a certain depth, which aligns with hypothesis H1. The general framework illustrated in Figure
We evaluate KVComm on diverse set of tasks with nine model pairs (see Section showing that it consistently outperforms existing communication protocols while significantly reducing the data transmitted between models. In summary, our work makes three key contributions:
We evaluate different types of activation information for communication between LLMs, and identify the limitations of using hidden states as the medium of communication. We show that the last token's hidden state suffers from information concentration bias, and point out dilemma that arises when using all tokens' hidden states.
We propose KVComm, novel communication protocol that enables efficient communication between LLMs through selective sharing of KV pairs. We design a selection strategy based on attention importance scores and Gaussian prior choose which layers' KV pairs share.
2

---

Published as a conference paper at ICLR 2026
This is the first approach that makes it possible to choose non-contiguous layers of KV. More- over, we show the feasibility of using a single context/question pair for guiding the selection for given pair of models, prior to deployment.
We conduct extensive experiments on diverse of tasks and model pairs, demonstrating that KVComm enables effective and efficient communication between LLMs, achieving com- parable performance the Skyline method, which is the upper-bound and directly merges the inputs without any communication, while reducing the computation costs by 2.5x to 6x. In particular, KVComm enables up to 3x reduction in communication relative approaches that transmit the entire set of KV pairs. Moreover, we demonstrate the performance benefits of non-contiguous selection of KV layers. Finally, we demonstrate the increase in performance that KVComm brings even over Skyline two datasets, further illustrating the need to com- municate in non-strictly textual manner.
2 PROBLEM AND MOTIVATION
2.1 PROBLEM FORMULATION
We formally define the problem of solving contextual task through the communication of two LLMs: M, and M, M2 takes as imput context C, and generates the required information Ic to be communicated. M, takes as input the query Q and the information Ic from M., and produces the final output. In this work, we limit the choices the two LLMs to (1) two instances of the same LLM, and (2) two models that fine-tuned versions of the same base LLM. The objective is to design a communication protocol that jointly optimizes the communication, computation efficiency, and the information fidelity between M, and M,.
2.2 WHY HIDDEN STATES FALL SHORT
When Decoder-Only LLMs infer, the input information flows through the model in the form of activation values, which refer to the intermediate results output by each decoder layer during the forward pass. We refer to the intermediate activation values that are passed between adjacent layers as hidden states. We also consider the KV pairs used in the attention mechanism within each layer as another type activation information. In this section, we investigate the effectiveness of using hidden states as the medium of communication by studying two questions: How important are hidden states of tokens at different positions in the sequence? (Section 2.2.I Are hidden states of all tokens effective for communication? (Section 2.2.2
2.2. TOKEN IMPORTANCE AT DIFFERENT POSITIONS
We begin with a simple experiment examining how token positions affect performance. Using 1amma-3.1881 on MMLU Social Science, we remove or retain the hidden state of only specific tokens at a given layer and measure the performance change. As shown Figure different to- kens vary in importance across layers, with the last token becoming most critical in later layers. This aligns with the intuition that the last token is often the most relevant to the current prediction. Thus, the last token's 's hidden state carries the most influential information for both model output and inter-LLM communication. Results on additional datasets and models are provided Appendix C
To ensure efficient communication with hidden states built on this observation, two conditions must hold: (1) M, must send at least the last token's 's hidden state, and (2) the communication protocol should preserve M,'s last token state much as possible. The protocol in Ramesh & Li] (2025) either replaces M, 's last token state with that of M, or averages the two, but both cause information loss M,'ss 's last token state, harming its performance.
2.2.2 UTILIZING ALL TOKENS
Another straightforward approach to ensure the last token's 's hidden state is preserved is to prepend all tokens' hidden states from M, to M,. The experiments presented in Figure 3 demonstrate that prepending all tokens' hidden states from M, to M, is effective if the hidden states are taken from the early layers of M,, and prepended to the early layers
HotpotQA with Llama-3.1
3

---

Published as a conference paper at ICLR 2026
Retain the taken
Remove the taken
Prepending Hidden
30-
Sender M, layer index (k)
0.6
25
0.8
20
Accuracy
0.5
None
0.5
15
0.4 3
Random
0.3
First
10-
0,3
5.
0.1
10
20
30
0.2
0.2
Receiver Mr layer index ()
0
5
10
15
20
25
0
5
10
15
20
25
Layer
Figure 3: Prepending helps only when using early-layer hidden states.
Figure 2: Compared to other token positions, the last token's hidden state is the most critical, especially in later layers.
of M,- Appendix siswwss experimental results on other datasets. We find that this method is caught in dilemma: (1) if the hidden states are taken from the early layers of M,, the computation benefit limited since is similar to concatenating the two inputs; (2) if the hidden states are prepended to the later layers of My, the performance drops significantly.
These findings suggest that while utilizing all tokens' hidden states can preserve the last token's information, it does not guarantee effective communication between LLMs.
EFFICIENT LLM COMMUNICATION THROUGH SELECTIVE KV SHARING
We propose a simple yet effective communication protocol that enables efficient communication between LLMSs by selectively sharing KV pairs. This approach addresses the limitations observed in previous methods by ensuring that the most critical information is preserved. Our design satisfies the three criteria outlined below: it enhances effectiveness by allowing M, to utilize essential context (@), improves efficiency by reducing unnecessary computation and transmission overhead (@), and ensures generality by being applicable across diverse tasks and architectures (3)
3.1 COMMUNICATION FRAMEWORK
For a given context C and query Q, M, processes the context C' and runs one forward pass (prefill
stage) generate the KV pairs {(k,, v-)] each layer 1, where / = 1, 2.2..... L and L is the total number of layers in M,. We apply selection strategy to choose a subset of KV pairs {(k), v/)), where i 1.2...., and M the number of selected layers. The selected KV pairs are then transmitted to Mr.
Mr processes the query Q and incorporates the received KV pairs during its forward passes (prefill and decoding stages). Specifically, at each layer of Mr. if corresponds to selected layer I the KV pairs from M3 are integrated into the attention mechanism. We simply concatenate the KV pairs from M, with those of M,: k, <- [k:kk/] and v +- vf] This integration allows M, to attend to both its own context and the information provided by M,, After processing the query Q with the integrated KV pairs, M, generates the final output.
3.2 KV SELECTION STRATEGIES
The communication protocol critically depends on the selection strategy for choosing which KV pairs to transmit from M, to M, Not all layers or attention heads contribute equally to encoding task-relevant knowledge. A fundamental question when designing selection strategies is: Which parts of the KV pairs encode the most relevant knowledge for communication?
Formally, given the set of candidate KV pairs {(k,, v') our goal is select a subset S S (1...., L.) such that the receiver's output retains maximal information from the sender, given
2The layer indices are 1-to-1 matched between M, and M, since only consider the case where the two models the same fine-tuned versions of the same hase I.I.M.
4

---

Published as a conference paper at ICLR 2026
constraint the number of selected layers |S| M, which is determined by the desired communi- cation efficiency. This can be formulated as the following optimization problem:
fa (Q,,(Q, ((K,, v')))))),
SC( SSL
where f()) is performance metric (e.g., accuracy, Fl score), and Mr [kk, v')hEs) denotes the output of the receiver model given the query Q and the selected KV pairs. Since direct compu- tation of this objective is intractable. we instead propose two hypotheses H1 and H2 that serve as priors for designing practical heuristies.
The first hypothesis H1 is that KV pairs from intermediate layers contain the most readily trans- ferable semantic knowledge. Prior analyses (Jawahar et al., 2019 Geva et al. 2020) suggest a hi- erarchy: early layers capture surface patterns, middle layers encode semantic abstractions, and late layers specialize task predictions. Thus, intermediate KV pairs should carry the richest generaliz- able information, making them most effective for communication. Experiment results in Section support this hypothesis.
Another hypothesis H2 is that KV pairs from layers exhibiting stronger attention distributions are more effective for communication. We quantify this notion of attention distribution using the at- tention importance score S',, defined in Equation (1) below. We deem layer l; to exhibit stronger attention distribution than 1 if St > stz Intuitively, a head consistently allocates high attention mass to the given tokens, its KV cache encodes salient contextual relations that are critical for the model's reasoning. Attention concentration thus serves as a proxy for the communication value of KV subset, suggesting that such heads should be prioritized for selection. This hypothesis is also validated by our experiments in Section 4.5
Our selection strategy based on these two hypotheses. We first define attention importance scores for each layer, which are calculated as the average attention weights that have been assigned to the context tokens by all heads in that layer during the prefill stage. We then take Gaussian distribution centered at certain layer as prior to select layers with high attention importance scores. The intuition is that the Gaussian prior encourages selecting layers around a certain depth, which aligns with hypothesis H1 that intermediate layers are more likely to contain transferable knowledge.
Mathematically, the attention importance score for each layer is computed as:
S
g.g.
HQQ
where Hi is the number of attention heads, [Q] is the number of tokens in the query, |C| is the number of context tokens, and ahe is the attention weight assigned by head at layer from token & to context token $1 is then normalized the range [0, 1] across all layers obtain the final attention
8 min, Sy
importance score S max, S. miny S!
We define a Gaussian prior centered at layer y with standard deviation a as pl = exp ---
The final selection score for each layer is computed as weighted combination of the attention importance score and the Gaussian prior:
s = + (1-
where a J0, 1] is a hyperparameter that balances the two components. We then select the top M layers with the highest selection scores St to form the subset S for communication.
For each model pair and dataset, the top M layers are selected based on the selection scores com- puted from a calibration set. The selected layers are then fixed and used for all samples in the test set. We found that a calibration set as small as a single sample is sufficient to obtain a robust selection that generalizes well to the entire test set, shown in the experiments in Appendix H
3.3 COMPLEXITY ANALYSTS
We analyze the computational complexity of our KVComm framework compared to baseline meth- ods. Compared to the NL.D (Du et al. 2023) method, our method does not require multiple decoding
5

---

Published as a conference paper at ICLR 2026
steps for Mag which significantly reduces the computation cost. When the number of tokens gen- erated during debate (Du et al. 2023) is large, the computation margin of our method over NLD is on the order of O(L(T, + T, + (d), where T, and T, are the number of tokens generated by M, and MM in the debate, respectively, and [Q] and d are the number of tokens in the query and the hidden dimension of the model, respectively. Compared method also reduces the computation cost, especially when M is small. The computation margin of our method over Skyline on the order of ((()) T,) M(Q T.))), where |C] is the number of tokens in the context.
the Skyline (Section (4.) method, our
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets We evaluate KVComm on a diverse set of contextual reasoning tasks. Gellowing/Ramess & Li (2025), we synthetically generate two datasets, Countries, which asks questions about coun- tries based on landmark information, and Tipsheets, which requires investment decisions from fi- nancial tips. Examples of these two datasets are shown in Table 3 in Appendix [.1 Moreover, we select six benchmarks, including HotpotQA (Yang et al. 2018), QASPER (Dasigi et al. 2021), MuSiQuest (Trivedi et al., (2022), two subsets of LongBench (Bai et al. 2024MMultiFilllQAac and 2WikiMQA), and TMATH (Qi et al. 2025). The last dataset is a mathematical problem-solving
dataset that contains hints context. We ROUGE-L Recall the evaluation metric for the last dataset, and F1 score for all other datasets. Statistics are summarized in Table ii Appendix [.
Models We conduct experiments nine different model pairs, shown in Table Appendix The model pairs include two instances of the same LLM and two models that are fine-tuned versions of the same base LLM. These models cover different families, including LLaMA (Dubey et al.) 2024). Qwen (wwn et al. 2024). and Falcon (Almazrouei et al. (023)
Compared Methods We compare KVComm with several representative approaches: Baseline
(no communication between M, and M,), Skyline (concatenating context C' and query Q as an up- per bound), Natural Language Debate (NLD) (Du et al., 2023), CIPHER (Pham et al.] [2023). and AC (Ramesh & Li 2025). Detailed descriptions for these methods are provided in Appendix B.4 Implementation details are provided in Appendix B.2 B.2
4.2 COMMUNICATION RESULTS
Table results on three model pairs fine-tuned from the same base LLM. The results on other model pairs are provided in Table g in Appendix F which show similar trends. We observe that KVComm consistently outperforms all baseline communication methods across datasets and model pairs. AC can outperform the Baseline method some datasets, but they still significantly worse than KVComm and Skyline, as hidden states of M, are corrupted during communication.
NLD and CIPHER can achieve performance close to that of KVComm or Skyline on Countries and Tipsheets datasets, which is because these datasets require only very small and highly salient amount of information to be transferred. For all other datasets, the sender has access to the entire context but not the question, and natural-language communication cannot reliably extract and trans- mit the task-relevant subset of information. As a result, NLD and CIPHER perform substantially below KVComm complex, long-context reasoning tasks. We conduct further experiments in Appendix to eliminate the influence of hyperparameters.
KVComm can achieve comparable performance to Skyline when selecting 70% of layers' KV pairs for communication, demonstrating the effectiveness of our selection strategy. Even when selecting only 30% of layers' KV pairs, KVComm can still outperform most baseline communication methods on many datasets, showing its potential for efficient communication with minimal overhead.
Note that KVComm can outperform Skyline M, may complement M, with stronger capabilities in certain aspects, and (2) selective KV sharing provides regularization effect, which helps Mr foeus the most relevant information and
datasets. We attribute this to two factors: (1)

---

Published as a conference paper at ICLR 2026
Table Communication results of different methods. Best results are bolded, second best underlined (excluding Baseline and Skyline). We report the results with M, for Baseline and Sky- line for fairness. KVComm (0.3/0.5/0.7)) denotes selecting 30%/50%770% of layers' KV pairs for communication, i.e., M 0. M [0.5L], M [0.7L].
Method
Countries
Tipsheets
HolpotQA
QASPER
MuSiQuest
MultiField -QA-en
2WikIM
TMATH
Ma: huihui-ai/Llama-3... --ntmm---------ttuutt------ Instruct-abliterattee
M,
suayptalha/DeepSeek. Miikk--R--Dsttiiiii
Baseline
0.05
0.32
0.23
0.05
0.02
0.11
0.27
0.34
Skyline
0.57
0.91
0.73
0.25
0.51
0.47
0.40
0.36
NLD
0.43
0.72
0.43
0.10
0.18
0.09
0.30
0.33
CIPHER
0.42
0.69
0.50
0.10
0.18
0.13
0.32
0.32
AC (mean)
0.03
0.45
0.25
0.05
0.02
0.13
0.23
0.35
AC (replace)
0.00
0.49
0.05
0.01
0.01
0.12
0.03
0.34
AC (sum)
0.02
0.46
0.23
0.05
0.01
0.34
VVCom (0.3)
0.46
0.45
0.46
0.09
0.28
0.15
0.28
0.35
KVComm (0.5)
0.57
0.81
0.57
0.27
0.32
0.51
0.36
0.35
K KYComm (0.7)
0.57
0.81
0.65
0.29
0.36
0.47
0.37
0.35
M.: Oriinn-hhen(Qwem2255-77.. hentaemmeeemm..-----ttuuure..e Unniruuoe--Unnnsoeeeed M,t Saapkkeeaas/essooe-Steeeeee
Baseline
0.01
0.36
0.13
0.05
0.03
0.08
0.09
0.35
Skyline
0.51
0.97
0.53
0.10
0.25
0.40
0.09
0.35
NLD
0.21
0.80
0.16
0.02
0.04
0.11
0.02
0.35
CIPHER
0.04
0.60
0.01
0.03
0.07
0.34
AC (mean)
0.00
0.00
0.03
0.00
0.00
0.08
0.01
0.01
AC (replace)
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
AC (sum)
0.00
0.00
0.02
0.00
0.00
0.07
0.04
0.03
KVComm (0.3)
0.04
0.26
0.02
0.01
0.01
0.09
0.08
0.31
VCCom (0.5)
0.19
0.88
0.28
0.07
0.12
0.26
0.10
0.33
KYComm (0.7)
0.41
0.89
0.41
0.21
0.25
0.29
0.15
0.34
M,
christofonu/llloon-llrrssss
M,, huihui-ai/Falcon3-77 hns.-nnttttaabbterattee
Baseline
0.08
0.36
0.21
0.06
0.04
0.09
0.23
0.31
Skyline
0.56
0.95
0.76
0.32
0.56
0.51
0.45
0.37
NLD
0.46
0.80
0.52
0.19
0.25
0.11
0.24
0.15
CIPHER
0.30
0.19
0.27
0.02
0.07
0.06
0.25
0.17
(mean)
0.01
0.46
0.25
0.06
0.04
0.09
0.23
0.31
AC (replace)
0.00
0.49
0.12
0.00
0.01
0.13
0.17
0.31
AC (sum)
0.01
0.46
0.25
0.06
0.03
0.10
0.24
0.31
KoComm (0.3)
0.46
0.69
0.59
0.19
0.40
0.35
0.29
0.32
KVComm (0.5)
0.40
0.92
0.63
0.25
0.44
0.45
0.34
0.35
KYComm (0.7)
0.19
0.96
0.55
0.26
0.42
0.51
0.31
0.36
avoid wasting its capacity on less important signals. This also explains why using fewer layers can sometimes yield better performance than using more.
Also note that the performance gain of KVComm is not substantial on TMATH. We attribute this to that pretraining gives LLMs solid capabilities in mathematical reasoning, which may not dra- matically benefit from additional context or hints. Moreover, AC performs relatively well on this dataset, which we consider is because the hints contain information about questions, so even if the last token's hidden states are corrupted, it can still generate some useful information.
4.3 BENEFIT SELECTIVE KV OVER ONE CONTIGUOUS CHUNK
DroidSpeak (Liu al. 2024b) chooses use one contiguous chunk of context for communication between L.MMs. Despite different problem settings, we evaluate KVComm by replacing the selec- tion strategy with two hyperparameters, which are two layer indices layer and layer,, then all layers between layer and layer, selected for communication. This is equivalent to using one contiguous chunk context for communication. We vary them to select different chunks layers.
Figure shows that using single contiguous chunk for communication yields good performance only in a small region the hyperparameter space, making it tricky to find the right hyperparam- eters. In contrast, the scatter and curve plots Figure demonstrate that KVComm consistently achieves the best or even outperforms the best contiguous chunk setting for the same number of layers. Line plots in Figure show that contiguous chunks most effective when taken from in-

---

Published as a conference paper at ICLR 2026
termediate layers, consistent with hypothesis H1 in Section 3.2 All results are HotpotQA with the 1aama-3.1-8B pair, with more in Appendix
4.4 ABLATION STUDY ON SELECTION STRATEGY
Table 2 compares KVComm with random selection. We find that KVComm consistently outper- forms random selection across different datasets and selection ratios. When the ratio is high (i.e., 0.7), the performance gap between our selection strategy and random selection becomes smaller, as more layers are selected and the impact of the selection strategy reduced. However, when the ratio is low (i.e., 0.3), our selection strategy significantly outperforms random selection, demonstrating its effectiveness in selecting the most informative layers for communication. Comparison results on other model pairs are provided in Table in Appendix G which show similar trends.
Table 2: Comparison with random selection. Best results for each selection ratio are bolded.
Method
Countries
Tipsheets
HotpotQA
QASPER
MuSiQuest
MultiField -QA-en
2WikiM -QA
TMATH
Mst
M, suayptalha/DeepSeeee RI- Distill- -Llma-3B
Random (0.3)
0.05
0.32
0.18
0.07
0.01
0.06
0.17
0.33
KYComm (0.3)
0.46
0.45
0.46
0.09
0.28
0.15
0.28
0.35
Random (0.5)
0.26
0.44
0.37
0.08
0.10
0.09
0.21
0.34
KVComm (0.5)
0.57
0.81
0.57
0.27
0.32
0.51
0.36
0.35
Random (0.7)
0.57
0.82
0.62
0.20
0.34
0.30
0.28
0.35
KYComm (0.7)
0.57
0.81
0.65
0.29
0.36
0.47
0.37
0.35
FL Score with One Contig, Chunk of Context
F1 Score w.r.. Layyrrss
Impact Chunk Pusition under Fixed #ayers
0.6
#layer= +layer-16
0.6
layerrem
20
Contig 1.0
F1 Score
0.4
0.4
0.9
10
0.2
0-0.7
0.2
0.06
0
0.0
10
20
30
10 layeream
15
20
layena
LLayers
Figure 4: Effective commu- nication with limited hyperpa- rameters.
Figure 5: KVComm achieves nearly the best or outper- forms contig. chunks.
Figure 6: Chunks intermedi- ate layers achieve the most ef- fective communication.
4.5 ATTENTION DISTRIBUTION ANALYSIS
validate hypothesis H2 Section 3.2 by selecting layers with different attention importance scores for communication. We select 9 layers with different levels of attention importance scores, and test the communication performance with Llama-3.2-3B model. The results are shown in Fig- ure 7 We find that selecting layers with higher scores achieve better performance, while select- ing layers with lower scores can diminish the performance. This validates hypothesis H2 that layers with higher attention importance scores more effective for communication.
4.6 SYSTEM EFFICIENCY
Mathematically, we have shown Section 3.3 that KVComm can reduce the computation cost compared to Skyline. We validate this through experiments on the Llama-3.2-3B model pair with Tipsheets and MultiFieldQA-en datasets. We report the relative FLOPs of KVComm and Skyline over AC in Figure 8 NLD and CIPHER are not included since they require multiple decoding steps for Mg. which makes the computation cost significantly higher than AC. We find that KVComm has significant computation advantage over Skyline, especially when selecting fewer layers for communication. This demonstrates the efficiency of our KVComm framework enabling effective communication with reduced computational overhead by 2.5x to 6x.
8

---

Published as a conference paper at ICLR 2026
In addition to FLOPs, also report the memory consumption among methods. KVComm similarly shows a substantial memory advantage over Skyline, as the reduced number of communicated layers not only lowers computation but also alleviates memory pressure. On Tipsheets, KVComm uses 23% to 73% less memory than Skyline. This further highlights the efficiency our framework in achieving lightweight inter-model communication.
Relative FLOPs and Memory Cost Over AC with Llama-3.2-38 (per Query)
Score Over Attention Level with Llama 3.2-30 3B
Tipsheets
MultiFicdOOA
HotporQ4
00001
200
#1 Scaree
0.4
Tipsheets
Countries
GRUOPS
2.0
0.0010
150
Wemoryy (GB)
1.0
0,005
0.2
0.
0,000
AC
Skylme
RyCanm (0.3)
KyComm (0.5)
AVCom (0.7)
AC
Styine
KVConu (8
KyCenm (0.5)
IVV Comm (0.7)
Attention Level (high + low)
Figure 7: Better communication per-
Figure KVComm requires less computation and mem- ory compared to Skyline.
formance with higher attention level.
5 RELATED WORK
LLM Inference Acceleration Lots of work has focused on accelerating LLM inference. Computation-level methods such as FlashAttention (Dao et al. 2022) and Memory- At- tention (Rabe & Staats 2021) reduce memory and speed up attention; system-level methods such vLLM (Kwon al.; 2023) and DeepSpeed-Inferennce (Aminabadi al.) 2022) improve overall throughput and latency; and model-level methods such as quantization (Lin et al. 2024) and prun- ing (M al., 2023) reduce model size and complexity. These works mainly focus working with only one model processing a single long input with the aim of minimizing computation cost. These approaches are orthogonal ours and can be combined with KVComm further improve efficiency.
Closest to our work are methods that reuse computation across decoding steps or requests. Gao et al. 2024) introduces a hierarchical KV caching system for all requests; Gim et al.) (2024) reuses prompt K caches across queries by decomposing inputs: Liu et al. (2024c) compresses KV caches into compact bitstreams; and Yao et al. (2025) combines multiple chunks' KV caches by selectively recomputing a few tokens. In contrast, our work targets communication across different LLMs, which is more challenging due parameter differences. Morcover, while prior methods reuse KV caches uniformly across layers, we enable selective sharing of KV caches from different layers, further improving efficiency. We do not compare with these works since they orthogonal to ours.
DroidSpeak (Liu et al.) 2024b) aims to accelerate inference for queries with shared prefixes. reuses the partial KV cache of these prefixes among different queries. Specifically, it empirically selects single contiguous chunk of layers and recomputes the rest with large calibration overhead, whereas our strategy flexibly selects non-contiguous layers with low overhead, without needing to recompute the remaining layers. Despite different problem settings, we compare their contiguous- chunk strategy with ours in Section 4.3 showing the advantages of our approach.
Ye et al. (2025) adjusts KV cache for shared content by referencing pool of cached examples- termed anchors that store observed cache deviations under varying prefixes. Our work goes beyond this related work by: 1) enabling a different type of communication, where the receiver does not have access to the context, 2) making it possible efficiently and selectively choose layers KV pairs that will be transmitted, and 3) being able to work effectively across different models that are fine-tuned from one model.
Inter-LLAn LLM Communication Communication between multiple LLMs has been explored in sev- eral recent works. Most works focus on using natural language as the medium of communication. For example, Du et al. (2023) proposed natural language debate framework where LLMs itera- tively critique each other's 's answers in natural language to improve the final answer. [iiagg et al. 2023) followed similar idea but introduced judge model to manage the debate process.

---

Published as a conference paper at ICLR 2026
CIPHER (Pham al. 2023) proposed using embedding space as the medium of communication. They pass the weighted average of the token embeddings from one LLM to another. Moreover, AC (Ramesh& & Li 2025) proposed to use the last token's hidden state as the medium of commu- nication. They replace the last token's hidden state of the receiver model with that of the sender model. Instead, propose tion than just using the last token's 's hidden state. We also propose more effective selection strategy for choosing which KV pairs to share, which can further improve efficiency.
the KV pairs the medium, which can preserve more informa-
KV Cache Optimization Several works have explored optimizing KV caches for a single LLM
by (1) compressing the KV caches reduce memory usage (Gie et al. 2023 Liu et al. 2024a) or (2)
managing the KV caches (offloading) to improve the inference speed (L.ee et al. 2024 Xiong et al.)
2024). As our work focuses on layer-wise selection KV caches for communication between two LLMs, these methods are orthogonal and can be combined with our method.
6 DISCUSSION
In this section, we discuss the limitations, clarify the scope of current design choices, and outline promising directions for future research. Additional discussions can be found in Appendix K
Heterogeneous Model Architectures Our current KVComm framework assumes that both LLMs share the same base architecture, i.e., identical models or fine-tuned versions of the same base LLM. This is because KV pair structures differ substantially across model families, making direct KV ex- change undefined. This architecture dependency is a practical limitation but not fundamental one. Future work could explore learning latent projections, adapters, or other transformation functions to enable KV exchange across heterogeneous architectures.
Multiple Sender/Receiver Extensions While we focus on single sender-receiver pair in this work, KVComm can be naturally extended to multiple senders and/or receivers. KVComm can integrate information from multiple senders by concatenating KV caches, and multiple receivers can independently select layers based on their own attention patterns. As shown in Appendix []] we mathematically extend our framework to multiple senders, and perform a preliminary experiment with two senders and one receiver, showing that multiple senders can improve performance due to diversified information sources. However, a systematic study of scaling behaviors in larger multi- agent networks remains future work.
Context-adaptive Online Calibration KVComm currently adopts a fixed layer-selection strat- egy after calibration for simplicity and computational efficiency, while context-adaptive selection is a promising extension. KVComm can naturally support online and dynamic selection. A demon- stration and analysis of this mechanism is provided in Appendix L
Layer Selection Priors Given our goal of keeping the method simple, efficient, and broadly repro- ducible, we opt for the Gaussian prior. Other alternatives, such as satropy-weighter or data-driven, are promising but introduce significantly higher complexity, e.g., larger calibration sets, training selector, or risking overfitting to a particular task distribution. Exploring more sophisticated priors is an interesting direction for future work.
7 CONCLUSION
In this work, we identified the potential of using KV pairs as an effective medium for communication between two L.Mss. We proposed a novel KVComm framework that enables efficient communica- tion by selectively sharing KV pairs between LLM models. We designed selection strategy based on attention importance scores and a Gaussian prior to select the most relevant layers. Extensive experiments on diverse datasets and model pairs demonstrated that KVComm can achieve compa- rable or even superior performance to the Skyline upper bound and other methods, while reducing communication costs by up to 3x. We highlight the generalization ability of our selection strategy, which can be effectively calibrated with only a single sample. Our work opens up new possibilities for efficient inter-L..M communication and paves the way for future research in this direction.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the Knut and Alice Wallenberg Foundation through Wallenberg Scholar Grant to Prof. Dejan Kostić. This work has been partially supported by Vinnova (the Swe- den's Innovation Agency). the Swedish Research Council (agreement No. 2021-04212), and KTH Digital Futures. We thank the anonymous reviewers for their insightful comments and constructive suggestions. We also thank Nicolae Filat for providing an early version of the code for CIPHER comparison and for helpful suggestions on the illustrations.
Computations were enabled by the Berzelius resource provided by the Knut and Alice Wallenberg Foundation at the National Supercomputer Centre. We also acknowledge the EuroHPC Joint Un- dertaking for awarding this project access to the EuroHPC supercomputer LEONARDO, hosted by CINECA (Italy) and the LEONARDO consortium through an EuroHPC Development Access call.
REPRODUCIBILIIY STATEMENT
We provide detailed descriptions of the datasets, model pairs, and implementation details in Ap- pendix B The code and synthetic datasets, Countries and Tipsheets, are uploaded GitHub to facilitate reproducibility upon the publication of this work.
REFERENCES
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Co- jocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, et al. The falcon series of open language models. arXiv preprint 1. 2023.
Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li, Du Li, Elton beepspeed-ifference::: Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, al. Deepspeed- inference:
enabling efficient inference of transformer models at unprecedented scale. In SC22: International Conference for High Performance Compuling, Networking, Storage and Analysis, pp. 1-15. IEEE, 2022.
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A bilingual, Pruceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 3119-3137, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 18653vV// 2024.acl- 172.
multitask benchmark for long context understanding.
http://aclla http::::////laaaooooogyoo 2002://4200022221111111002 aee4.acl-lon 172
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances in neural information processing systems, 1116344-16359) 2022.
Pradeep Dasigi. Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner. A dataset of informationn-seeking questions and answers anchored in research papers. arXiv preprint arXiv:2105.030/1, 2021.
Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving fac- tuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning. 2023.
Abhimanyu Dubey, Abhinav Jaubri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Bin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang, Djordje Jevdjic, Junbo Deng, Xingkun Yang, Zhou Yu, and Pengfei Zuo. (Cost-Efficient) large language model serving for multi-turn conver- sations with (CachedAttentiion). In 2024 USENIX Annual Technical Conference (USENIX ATC 24). pp. 111-126, 2024.
Suyu Ge. Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive kv cache compression for 1lms. arXiv preprint arXiw:2310.01801, 2023.
11

---

Published as a conference paper at ICLR 2026
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers key-value memories. arXiv preprint arXiv:2012.14913, 2020.
In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, and Lin Zhong. Prompt cache: Modular attention reuse for low-latency inference. Proceedings of Machine Learning and Systems, 6:325-338, 2024.
Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. SAMSum corpus: A human- annotated dialogue dataset for abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pp. 70-79, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 111/6653/1//11154099 URL wttph:://///. www. aclweb. oxc./antholooyy/1199-55
Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges, 2024. URL httttt:///aaiii/////e///// org/ abs/ 2202 01680
Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. What does bert learn about the structure of language? In ACL 2019-57th Annual Meeting of the Association for Compulational Linguisties, 2019.
Shuowei Jin, Xueshen Liu, Qingzhao Zhang, and Z Morley Mao. Compute or load kv cache? why not both? arXiv preprint arXiv:2410.03000 2024.
Woosuk Kwon, Zhuohan Li. Siyuan Zhuang. Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems princi- ples, pp. 61 1216666,222 2023.
Wonbeom Lee, Jungi Lee, Junghwan Seo, and Jaewoong Sim. {InfiniGen]: Efficient generative inference of large language models with dynamic {KV) cache management. In /8th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pp. 155-172, 2024.
Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Com- municative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991-5229 2023.
Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujuu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi- agent debate. arXiv preprint arXiv:2305.191 19118. 2023.
Lin, Jiaming Tang, Haotian Tang, Shang Yang. Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device I1m compression and acceleration. Proceedings of machine learning and systems, 6:87-100, 2024.
Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholamreza Haffari, and Bohan Zhuang. Minicache: Kv cache compression in depth dimension for large language models. Advances in Neural Infor- mation Processing Systems, 37: 37:119997-140031 2024a.
Yuhan Liu, Yuyang Huang, Jiayi Yao, Shaoting Feng, Zhuohan Gu, Kuntai Du, Hanchen Li, Yihua Cheng, Junchen Jiang, Shan Lu, al. Droidspeak: Kv cache sharing for cross-Ilm communication and multi-llm serving. arXiv preprint arXiv:2411.02822 2024b.
Yuhan Liu, Hanchen Li, Yihua Cheng. Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan et al. Cachegen: Kv cache compression and streaming for fast large language model serving. Proceedings of the ACM SIGCOMM 2024 Conference, pp. 38-56, 2024c.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. LIm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36:21702-21720 2023.
12

---

Published as a conference paper at ICLR 2026
Chau Pham, Boyi Liu, Yingxiang Yang, Zhengyu Chen, Tianyi Liu, Jianbo Yuan, Bryan A Plum- mer, Zhaoran Wang, and Hongxia Yang. Let models speak ciphers: Multiagent debate through 2023.
embeddings, arXiv preprint
Changyong Qi, Yu'an au'ang Wei, Haoxin Xu, Longwei Zheng, Peiji Chen, and Xiaoqing Gu. Tmath dataset for evaluating large language models in generating educational hints for math word problems. In Proceedings of the 31st International Conference on Computational Linguistics, pp. 5082-5093, 2025.
Chen Qian, Xin Cong, Cheng Yang. Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint arXiv:2307.07924, 6(3):1, 2023.
A Yang Qwen, Baosong Yang, B Zhang, B Hui, B Zheng, B Yu, Chengpeng Li, D Liu, F Huang, H Wei, et al. Qwen2. technical report. arXiv preprint, 2024.
Markus N Rabe and Charles Staats. Self-attention does not need o (n2) memory. arXiv preprint urXiv:2112.05660 2021.
Vignav Ramesh and Kenneth Li. Communicating activations between language model agents. arXiv preprint arXiv:2501.14022 2025.
Lijun Sun, Yijun Yang, Qiqi Duan, Yuhui Shi, Chao Lyu, Yu-Cheng Chang, Chin-Teng Lin, and Yang Shen. Multi-agent coordination across diverse applications: A survey, 2025.
Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc- Viet Pham, Barry O' O'Sullivan, and Hoang D. Nguyen. Multi-agent collaboration mechanisms: A survey of Ilms, 2025.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique Multihop questions via mnnge-hoppu question composition. Transactions of the Association for Computational Linguistics, 10:539-554 554, 2022.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- ger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. As- sociation for Computational Linguistics. URL https: /wwww/wwww.../../o///// acwweb. wttp://th//aaatgggogggoooooo 2020. emnlp demos.
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen 11m applications via multi- agent conversations. In First Conference on Language Modeling, 2024.
Yi Xiong, Hao Wu, Changxu Shao, Ziqing Wang, Rui Zhang, Yuhong Guo, Junping Zhao, Ke Zhang, and Zhenxuan Pan. Layerkv: Optimizing large language model serving with layer- wise kv cache management. arXiv preprint urXiv:2410.00428 2024.
Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Shaochen Zhong, Bing Yin, and Xia Hu. Harnessing the power of llms in practice: A survey on chatgpt and beyond. ACM Trans. Knowl. Discov. Data, 18(6), 2024a. ISSN 1556-4681. doi: 10.1145) 3649506.
Joshua C Yang. Damian Dalisan, Marcin Korecki, Carina I Hausladen, and Dirk Helbing. Llm voting: Human choices and ai collective decision making. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society volume pp. 1696-1708, 2024b.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio. William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018.
13

---

Published as a conference paper at ICLR 2026
Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, and Junchen Jiang. Cacheblend: Fast large language model serving for rag with cached knowledge fusion. In Proceedings of the Twentieth European Conference Computer Systems, pp. 94-109, 2025.
Hancheng Ye, Zhengqi Gao, Mingyuan Ma, Qinsi Wang, Yuzhe Fu, Ming- Yu Chung, Yueqian Lin, Zhijian Liu, Jianyi Zhang, Danyang Zhuo, et al. Kvcomm: Online cross-context kv-cache com- munication for efficient lim-based multi-agent systems. arXiv preprint arXi:2510012872, 2025.
14

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
Large language models, including ChatGPT, were employed provide assistance in improving the clarity, coherence, and fluency of the manuscript. These tools were used solely for language refinement, and all scientific content and interpretations remain the responsibility of the authors.
B EXPERIMENTAL SETUP
In this appendix, provide details about the experimental setup, including dataset details, implementation details, fine-tuned model pairs, and descriptions of compared methods.
B. DATASET
We provide sample prompts and expected answers for the Countries and Tipsheets datasets in Ta- ble 3 which are inspired by Ramesh & Li 2025). We also provide the statistics of all datasets used our experiments in Table 4 HotporQA, QASPER, MuSiQuest, and TMATH datasets are randomly sampled from their original datasets reduce the evaluation cost. Extended results on the full datasets are provided in Appendix
Table 3: Sample prompts and expected answers for Countries and Tipsheets datasets inspired by Ramesh & (205).
Dataset
Role
Content
C'
Uma is at the Mahaffie House. Which country Uma located in? United States
Countries
Q
Answer
C
Atlas LLC is under pressure amid softer trends; EPS -17%; won a sizable customer contract but faces lawsuit. Sable LLC shows clear momentum and improving execution; authorized a buyback but reported a cyber in- cident. Trace LLC looks balanced with a mixed near-term setup. You must invest in exactly one company from Atlas L.C., Sable L.C.C Trace LLC. Which do you choose? Sable LLC
Tipsheets
Q
Answer
Table Statisties of the datasets in our experiments.
Dataset
Size
Countries
200
Tipsheets
500
HotpotQA Yang et al. 2018
500
QASPER (Dasigi et al. 202T
500
TrsiQuestTTriveli et all 2022
500
MuliFieldQAAcn (Bai cl al 22024)
150
2WikiMQA (Bai al 2024
200
TMATH (Qi et al. 2025
300
B.2 IMPLEMENTATION DETAILS
We implement our KVComm framework based on the Hugging Face Transformers library (Wolf itt al. 2020), and models are loaded bíloat16 precision. We set the hyperparameters our selec- tion strategy
L/2, and 10, where L is the total number of layers in the model. For NLD and CIPHER methods, we set the number debate rounds to 2, and the maximum generation length to 256 in the debate process. For KVComm, is set to for Llama family models, and 0.8 for Qwen and Falcon family models. These values are obtained by validating on a left-out set. All experiments conducted on cluster of nodes, each equipped with Intel Intel@Xeon@Pee Xeon@Plainunn 8358 Processor
15

---

Published as a conference paper at ICLR 2026
@ 2.60 GHz and 4 NVIDIA A100 GPUs with 64 GB memory. We obtain the FLOPs with PyTorch Profile
B.3 MODEL PAIRS
We conduct experiments on nine different model pairs, shown Table The first four pairs consist of the same LLMs, while the last five pairs consist of models that are fine-tuned the same base L.LM.
Table 5: Model pairs in the evaluation. M, is the sender model, and M, is the receiver model.
Index
M,
Mr
Note
meta- Pectalllaaal-lama--2 1---BBB------B---- --8t-rrrruut
Tanz-llama/Llama->>
Same model
Beetallaam//Lamma33 3.2.aaam---a--mmmmmsss
In-sruuu
mela-llama/
Same model
3
Same model
#iiaacFalcoo3277-IInttrce
diuuacFallcon377B-nnnnre
Same model
5
1-8--Instttttttt
Llamm-33.1-8B
Fine-tuned on
puiiuu-aaiLLlmma-.22-1
Fine-tuned
whtp://wwen2557--mmmaeeee Uncensored
Fine-tuned
8
aaaiii/-///aooo.3711-.1- l---ntttcct-abblierreedd
Fine-tuned on 4
arcee-ail aaeeaaii/laama..... -.sma..--uppNNovva--I
deepseek Si/DeepSSek R--issi--------------
Fine-tuned
B.4 COMPARED METHOD DESCRIPTIONS
We compare our proposed KVComm framework with the following methods:
Baseline: M, processes the query Q without any communication from M,,
Skyline: M, directly processes the concatenation of the context C and query Q. This serves as an upper bound for performance.
Natural Language Debate (NLD) (Du et al. 2023): Each model generates an initial an- swer, and then they iteratively critique each other's answers in natural language for fixed number of rounds. Finally, one model produces the final answer based on the entire debate history. Compared to the original debate style setting, we use an information transserr style, which explicitly prompts M, that it has to summarize the context C' in its initial answer. We set the number of debate rounds to
CIPHER (Pham et al. 2023): Similar to NLD, but instead communicating in natural language, the models communicate by passing the weighted average of the token embed- dings from one L.MM to another. We use the same prompt as NLDD, and set the number of debate rounds to 2.
AC (Ramesh & Li 2025): Communicate with the last token's hidden state. Replace the last token's 's hidden state of M, with that of M,. We also test with mean and sum operations.
C TOKEN IMPORTANCE AT DIFFERENT POSITIONS
We add more details and experiments related to Section 2.2.1 in this appendix.
C.1 DETAILED EXPERIMENT PROCEDURE
Considering model We provide detailed description of the experiment procedure in Section 2.2.
M with L layers, given an input X with N tokens, we run partial forward pass until layer to obtain the hidden states (h/) /-1 Then, given specific token position k if we perform the Retain operation, we create a modified of hidden states 1N j= as follows:
h/
if f k
0.
otherwise
16

---

Published as a conference paper at ICLR 2026
If we perform the Remove operation, create the modified set of hidden states (h/) as follows:
0, ifi h otherwise
h
We then continue the forward pass from layer + 1 to layer using the modified hidden states (h) N as input, and obtain the final output the model. We evaluate the model's performance on the task with different token positions l and layer I.
C.2 MORE EXPERIMENTS ON TOKEN IMPORTANCE
We conduct the same experiment as in Section 2.2.1 on other datasets and models to investigate the effect of tokens at different positions in the sequence on the model's output. We report the results on MMLU Social Science, MMLU STEM, and MMLU Humanities using Llama-3. I-8B and Llama- We can see that the last token's 's hidden state plays the most critical role
3.2-3B models in Figure 9 the latter layers, which is consistent with the observation in Section 2.2.T
0.55
0.50
Last Note Raaddm
Acuracy
0.-
0.40
Nona
0.25
0.39
0.2
30
0.2
(a) Llama-3,2 3.2-3B on MMLU Social Science
(b) Llama Llama-3.1-8B 3.1-8B on MMLU STEM
Retbin the to
Remove the token
Retain the tocen
Remove the taken
3.52
2.45
0.45
3.62
None Random
3.35
Random
0.35
Filst
0.2
0.25
0.2
10
Layer
(c) Llama 3. 3.2-3B on MMLU STEM
(d) Llama-3.1 3.1-BB on MMLU Humanitics
Retain the token
Remove the token
0.60
3.60
0.55
0.59
Rundom
File
0.00
025
15
76
26
8
Layer
(e) Llama-3.2-3B MMLU Humanities
Figure 9: Effect of removing or retaining a token's 's hidden state across different positions on MMLU Social Science, MMLU STEM, and MMLU Humanities accuracy using Llama-3. 1.18BB and Llama- 3.2-3B models.
D UTILIZING ALL TOKENS
We add more details and experiments related Section 2.2.2 in this appendix.
D.1 DETAILED EXPERIMENT PROCEDURE
We provide a detailed description of the experiment procedure in Section 2.2.2 Considering two models M, and Mr, each with I, layers, given C and Q as input,
partial forward pass
17

---

Published as a conference paper at ICLR 2026
of M, until layer to obtain the hidden state Hi number tokens in C, and is the hidden dimension. Another partial forward pass M, is run until layer
R for all tokens in C, where |C| is the
obtain the hidden state Hj E TR xd for all tokens in Q, where [Q] the number of tokens in Q. We then modify the hidden states of M, at layer by prepending the hidden states from M, at layer & as follows:
H
H
We continue the forward pass from layer to layer L using the modified hidden states H7, as input, and obtain the final output of the model. We evaluate the model's performance on the task with different layers k and j.
D.2 MORE EXPERIMENTS ON UTILIZING ALL TOKENS
We conduct the same experiment in Section 2.2.2 Countries, Tipsheets, and HotpotQA datasets using 1aama-3.1-85 Llama-3.2-3B, and Qwen2.5-78 models. The results are shown in Figure T0)
We can the results consistent with the observation in Section 2.2.2
Llama-3.1-4
Llama-3 3.23B
Qwen2.5-78
Sender M, layer index (k)
0.6
25
20
0.4
15
0.3
10
0.2
0.1
0.0
10
20
10
20
10
20
Receiver M, layer index (i)
(a) Countries
Llama-3 3.1-8B
-lama-3.2-3B
Qwen2.5-78
Sender Ms layer index (k)
25
0.8
20
0.6
15
0.4
-0.2
0.0
10
10
10
20
Receiver M,, layer index (i)
(b) Tipsheets
Jamm-331-88
Llama-3.2-3
Qwen2.5-7B
Sender layer index (K)
25
0.6
20
F1 Score
15-
0.4
0..2
5
0.0
10
Receiver M, layer index (i)
(c) HotpotQA
Figure 10: Performance heatmap of prepending the hidden states from certain layers of M, to certain layers of M, on Countries, Tipshects, and HotpotQA.
18

---

Published as a conference paper at ICLR 2026
E EXPERIMENT RESULTS WITH EXTENDED DATASETS
To further validate the effectiveness of our KVComm framework, we conduct experiments on the full datasets of HotpotQA, QASPER, MuSiQuest, mainly HotpotQA-E, QASPER-E, and MuSiQuest-Fi Moreover, we include new human-created summarization dataset, SAMSum, which represents different task type. The statistics of these datasets are shown in Table 7 We report the results on these extended datasets Table 6 The results show similar trends as in Section 4.2 demonstrating
the robustness of our KVComm framework across different datasets and tasks.
Table 6: Communication results on extended communication tasks. The best results in each block bold, and the second best results are underlined.
Method
HotpotQA-E
QASPER-E
MuSiQuest-E
SAMSum
M,
M, Rhaypaalhhabeeepsskk-------iii--
Baseline
0.22
0.03
0.06
0.26
Skyline
0.77
0.52
0.25
0.33
NLD
0.45
0.10
0.18
0.28
CIPHER
0.51
0.10
0.20
0.28
(mean)
0,24
0.03
0.06
0.26
AC (replace)
0.06
0.00
0.01
0.26
AC (sum)
0.23
0.03
0.06
0.26
KVComm (0.3)
0.44
0.25
0.11
0.25
KYComm (0.5)
0.61
0.36
0.25
0.28
KYComm (0.7)
0.71
0.38
0.30
0,29
M, Prrinn/ieee0Ovvee222///lnnnnnnee
M, beepokkkatt//Bsppkettttttt-t
Baseline
0.15
0.04
0.06
0.25
Skyline
0.58
0.27
0.10
0.35
NLD
0.24
0.02
0.07
0.28
CIPHER
0.01
0.01
0.02
0.37
AC (mean)
0.03
0.00
0.00
0.01
(replace)
0.00
0.00
0.00
0.00
AC (sum)
0.03
0.00
0.00
0.04
KVComm (0.3)
0.02
0.00
0.01
0.18
KYComm (0.5)
0.21
0.14
0.08
0.30
KYComm (0.7)
0.40
0.34
0.21
0.35
M.: Orfionehristorrrfllooo---ttssss
M. mulhuiai//alcons-----issttt---l
Baseline
0.21
0.06
0.06
0.27
Skyline
0.78
0.60
0.33
0.36
NL.D
0.52
0.10
0.28
0.28
CIPHER
0.28
0.03
0.09
0.17
AC (mean)
0.24
0.06
0.07
0.26
AC (replace)
0.12
0.02
0.01
0.26
AC (sum)
0,23
0.05
0.06
0.26
KVComm (0.3)
0.59
0.15
0.40
0.28
KVComm (0.5)
0.59
0.22
0.46
0.31
KVComm (0.7)
0.59
0.26
0.36
0.32
Table 7: Statistics of extended datasets.
Dataset
Size
HotpotQA-E Yang et al. 2018)
7,405
OASPER-E (Dasig) et al. 202T
1,726
MuSiQuest-B Trivedi et al 2022
2,417
SAMSum (Gliwa et al. 2019)
819
MORE COMMUNICATION RESULTS
We provide more communication results different model pairs in Table 8 which show similar trends as in Section
19

---

Published as conference paper at ICLR 2026
Table More communication results of different methods. Best results are bolded, second best underlined (excluding Baseline and Skyline). We report M, for Baseline and Skyline for fairness. KVComm (0../0.5/0.7) denotes selecting 30%/50%/o0% of layers' KV pairs for communication,
i.c. M [ M [0.5L]. M [0.7L].
Method
Countries
Tipshects
HotpotQA
QASPER
MuSiQuest
MultiField -QA-em
2WIKIM -QA
TMATH
Mg: -aaa:ta.--a--3-----------
M,:
meta-llama/Llama
1a----..----------. Instruttt
Baseline
0.00
0.05
0.19
0.02
0.01
0.07
0.06
0.35
Skyline
0.62
0.92
0.74
0.35
0.54
0.56
0.52
0.36
NLD
0.58
0.87
0.52
0.13
0.25
0.17
0.10
0.36
CIPHER
0.57
0.84
0.57
0.13
0.25
0.36
AC (mean)
0.00
0.12
0.19
0.02
0.01
0.08
0.03
0.35
AC (replace)
0.00
0.36
0.15
0.02
0.01
0.07
0.05
0.35
AC (sum)
0.00
0.09
0.20
0.02
0.01
0.09
0.04
0.35
KCCom (0.3)
0.51
0.93
0.33
0.07
0.11
0.21
0.29
0.37
VCCom (0.5)
0.62
0.95
0.60
0.29
0.34
0.50
0.37
0.37
KVComm (0.7)
0.62
0.96
0.69
0.29
0.39
0.53
0.38
0.38
M.: meta- 1aaatm...-aam-m-33333.3333 Inn-truutt, Instruct; Mr: meta-Ilaman -.2-llam------mmmm.
Baseline
0.02
0.01
0.16
0.00
0.02
0.10
0.09
0.35
Skyline
0.56
0.87
0.72
0.23
0.45
0.45
0.37
0.38
NLD
0.51
0.71
0.49
0.09
0.18
0.11
0.07
0.34
CIPHER
0.45
0.73
0.46
0.08
0.17
0.09
0.07
0.33
(mean)
0.00
0.07
0.01
0.02
0.35
AC (replace)
0.01
0.37
0.13
0.01
0.02
0.06
0.03
0.34
AC (sum)
0.00
0.31
0.20
0.02
0.02
0.10
0.07
0.34
KCComm (0.3)
0.51
0.48
0.47
0.10
0.20
0.17
0.28
0.35
KVComm (0.5)
0.55
0.79
0.24
0.27
0.36
KVComm (0.7)
0.57
0.80
0.65
0.27
0.29
0.48
0.31
0.37
M,: Qwen//wwe2..5-77
M,: Qwen/Qwenz Owwen(Qwen..5777-Imnae
Qwwen/Qwen2.5577-IInneee
Baseline
0.00
0.32
0.19
0.05
0.03
0.06
0.17
0.32
Skyline
0.54
0.97
0.30
0.48
0.45
0.33
NLD
0.18
0.86
0.37
0.09
0.11
0.11
0.19
0.30
CIPHER
0.18
0.87
0.34
0.07
0.10
0.11
0.16
0.31
AC (mean)
0.00
0.37
0.15
0.01
0.02
0.10
0.20
0.33
(replace)
0.00
0.35
0.02
0.00
0.00
0.10
0.09
0.32
AC (sum)
0.00
0.41
0.14
0.02
0.02
0.08
0.17
0.32
KYComm (0.3)
0.04
0.31
0.06
0.02
0.01
0.19
0.19
0.32
KVComm (0.5)
0.57
0.92
0.49
0.18
0.20
0.40
0.25
0.32
KYComm (0.7)
0.56
0.98
0.72
0.29
0.48
0.45
0.35
0.33
M,: tiuuae/ tiiuaa/Falcon3-777 Innnstrrt M, tiiuae/Faleon3 B--nsrruuc
Baseline
0.06
0.33
0.19
0.01
0.01
0.09
0.21
0.31
Skyline
0.57
0.95
0.70
0.24
0.50
0.49
0.48
0.35
NLD
0.38
0.71
0.07
0.19
0.24
CIPHER
0.47
0.63
0.41
0.03
0.19
0.09
0.21
0.21
AC (mean)
0.03
0.51
0.22
0.04
0.04
0.09
0.22
0.32
AC (replace)
0.00
0.57
0.09
0.00
0.02
0.12
0.14
0.31
(sum)
0.04
0.51
0.22
0.04
0.03
0.09
0.22
KVComm (0.3)
0.06
0.67
0.41
0.12
0.22
0.41
0.23
0.32
KYComm (0.5)
0.16
0.94
0.52
0.22
0.33
0.47
0.33
0.32
KVComm (0.7)
0.23
0.96
0.54
0.22
0.32
0.47
0.29
0.32
M.: Luvraji/7/EvvooCodeLLaaaa --88--3.11-8--- -nnttrrrrut----ttttttt-- M..: Veam-ACCEToollACC----aaaaa
Baseline
0.00
0.07
0.04
0.00
0.01
0.08
0.01
0.34
Skyline
0.24
0.95
0.37
0.17
0.15
0.51
0.25
0.39
NLD
0.29
0.82
0.17
0.01
0.05
0.13
0.02
0.34
CIPHER
0.21
0.86
0.19
0.03
0.06
0.15
0.03
0.33
AC (mean)
0.00
0.31
0.03
0.00
0.01
0.11
0.01
0.34
AC (replace)
0.00
0.30
0.05
0.00
0.01
0.10
0.02
0.33
AC (sum)
0.00
0.27
0.04
0.00
0.01
0.09
0.01
0.34
Kommm (0.3)
0.12
0.95
0.12
0.05
0.04
0.26
0.19
0.36
KoComm (0.5)
0.55
0.98
0.15
0.14
0.43
0.38
KVComm (0.7)
0.53
0.97
0.51
0.22
0.25
0.49
0.33
0.38
M. arcce-ai/LLlamas> arce- S.w...---aper M.t deepseck-ai/DeepSeee Ri-eck--R-Dissill---a-aa -BB
Baseline
0.07
0.30
0.01
0.03
0.09
0.23
Skyline
0.55
0.80
0.52
0.17
0.40
0.41
0.16
0.29
Continued on next page
20

---

Published as a conference paper at ICLR 2026
Table 8 continued from previous page
Method
Countries
Tipsheets
HotporQA
QASPER
MnSiQuest
MultiField -QA-em
2WIKM -QA
TMATH
NLD
0.30
0.39
0.20
0.02
0.06
0.08
0.19
0.22
CIPHER
0.47
0.71
0.27
0.08
0.11
0.14
0.14
0.18
AC (mean)
0.00
0.31
0.08
0.02
0.02
0.09
0.14
0.25
AC (replace)
0.00
0.39
0.04
0.01
0.00
0.16
0.16
0.28
AC (sum)
0.02
0.34
0.07
0.02
0.02
0.08
0.16
0.24
KYComm (0.3)
0.09
0.52
0.10
0.01
0.03
0.09
0.08
0.28
K KVComm (0.5)
0.41
0.76
0.33
0.05
0.21
0.23
0.29
K KoComm (0.7)
0.53
0.76
0.47
0.12
0.28
0.31
0.14
0.29
G ABLATION STUDY ON SELECTION STRATEGY
We conduct ablation studies the selection strategy by comparing with random selection and selection based only attention importance scores. The results are shown in Table which show similar trends as Scction
Table 9: More comparison results with random selection. Best results for each selection ratio bolded.
Method
Countries
Tipshects
HotpotQ4
QASPER
MuSiQuest
MultiField -Q
2WikIM
TMATH
Ma: meta-llama/ Jawa-a..mmaa-- Instrrutt M.:
Random (0.3)
0.02
0.35
0.24
0.07
0.04
0.07
0.12
0.35
KYComm (0.3)
0.51
0.93
0.33
0.07
0.11
0.21
0.29
0.37
Random (0.5)
0.49
0.76
0.58
0.15
0.29
0.29
0.27
0.36
KYComm (0.5)
0.62
0.95
0.60
0.29
0.34
0.50
0.37
0.37
Random (0.7) 0.63
0.88
0.76
0.32
0.49
0.52
0.34
0.37
KYComm (0.7)
0.62
0.96
0.69
0,29
0.39
0,53
0.38
0.38
M.: Orion-Zhen(Qwenn2557BB-B htw.:eees------nsuuut-----eeeeee Iir--nsstruuc--nnnnooeeeddd M,: bespokclabw/BeepooeeSSttra
Random (0.3)
0.00
0.09
0.00
0.00
0.00
0.06
0.01
0.31
KVComm (0.3)
0.04
0.26
0.02
0.01
0.01
0.09
0.08
0.31
Random (0.5)
0.12
0.32
0.00
0.03
0.33
KYComm (0.5)
0.19
0.88
0.28
0.07
0.12
0.26
0.10
0.33
Random (0.7). 0.16
0.76
0.14
0.03
0.02
0.20
0.04
0.34
KYComm (0.7)
0.41
0.89
0.41
0.21
0.25
0.29
0.15
0.34
M.. chrisstfforr/alloom--ltrraatt M,: huihui-ai/Falcon33-75
haapi////f-atoon-----nntttttttie Instrucct-abliteraee
Random (0.3) 0.35
0.36
0.23
0.06
0.07
0.14
0.24
0.31
KVComm (0.3) 0.46
0.69
0.59
0.19
0.40
0.35
0.29
0.32
Random (0.5)
0.23
0.42
0.27
0.09
0.08
0.15
0.28
0.31
KYComm (0.5)
0.40
0.92
0.63
0.25
0.44
0.45
0.34
0.35
Random (0.7)
0.18
0.94
0.51
0.223
0.35
0.47
0.30
0.34
KYComm (0.7)
0.19
0.96
0.55
0.26
0.42
0.51
0.31
0.36
M.: meta-llamall 1la.mmmaaaaaa-33333--------- Instsrrt Mr meta-llama/ metta-laaa//llmma-3222 Jama-3,2-3BB 1amma33.-3B-Instruee
Random (0.3)
0.02
0.29
0.11
0.06
0.02
0.07
0.16
0.34
KYComm (0.3)
0.51
0.48
0.47
0.10
0.20
0.17
0.28
0.35
Random (0.5)
0.28
0.44
0.30
0.06
0.06
0.06
0.19
0.35
KVComm (0.5)
0.55
0.79
0.58
0.24
0.27
0.47
0.35
0.36
Random (0.7)
0.54
0.81
0.21
0.30
0.26
0.36
KVComm (0.7)
0.57
0.80
0.65
0.27
0.29
0.48
0.31
0.37
M.:
Instruct;; M.: Qwenn/wwen..577B--ne
Random (0.3)
0.00
0.34
0.05
0.00
0.00
0.08
0.10
0.30
KYComm (0.3)
0.04
0.31
0.06
0.02
0.01
0.19
0.19
0.32
Random (0.5)
0.00
0.32
0.10
0.02
0.02
0.10
0.16
0.32
KYComm (0.5)
0.57
0.92
0.49
0.18
0.20
0.40
0.25
0.32
Random (0.7)
0.41
0.71
0.28
0.04
0.04
0.21
0.17
0.32
KYComm (0.7)
0.56
0.98
0.72
0.29
0.48
0.45
0.35
0.33
M.: tiluae/Falcon3-11 B-lelconn----Instrree Instrrcct; M,t tliuae/Faleon3-11 -n-Inttuctt
Random (0.3)
0.01
0.35
0.18
0.04
0.03
0.12
0.21
0.30
KVComm (0.3)
0.06
0.67
0.41
0.12
0.22
0.41
0.23
0.32
Random (0.5)
0.04
0.41
0.03
0.05
0.24
0.31
KVComm (0.5)
0.16
0.94
0.52
0.22
0.33
0.47
0.33
0.32
Continued on next page
21

---

Published as a conference paper at ICLR 2026
Table 9 continued from previous page
Method
Countries
Tipsheets
HotporQA
QASPER
MuSiQuest
MultiField -Qaeen
2WIKM -QA
TMATH
Random (0.7) 0.19
0.95
0.51
0.20
0.29
0.42
0.26
0.32
KVComm (0.7)
0.23
0.96
0.54
0.22
0.32
0.47
0.29
0.32
M,:
1-8B-Instructt, M,; Ceam-ACEToolACE..
Random (0.3)
0.00
0.31
0.06
0.00
0.01
0.13
0.03
0.34
KVComm (0.3)
0.12
0.95
0.12
0.05
0.04
0.26
0.19
0.36
Random (0.5)
0.03
0.79
0.29
0.06
0.09
0.16
0.35
KVComm (0.5)
0.55
0.98
0.38
0.15
0.14
0.43
0.28
0.38
Random (0.7)
0.37
0.85
0.59
0.21
0.27
0.47
0.33
0.36
KVComm (0.7)
0.53
0.97
0.51
0.22
0.25
0.49
0.33
0,38
H CALIBRATION SET SIZE
0.8
0.7
Countries
HatpotQA
F1
Tipsheets
25
50
75
100
125
Calibration size
Figure 11: Effect of calibration set size. Calibration set size does not significantly affect the test performance.
We investigate how many samples are needed in the calibration set so that the selection strategy can generalize well to the test set. If smaller calibration set can achieve good performance the test set, would be more practical since would require less Tipsheets, and HotpotQA datasets using Llama-3.2-33 model. the results in Figure IT show, we can see that using only one sample in the calibration set can already achieve the same performance as using more samples (up 128 samples). This suggests that selection strategy can generalize well to the test set even sample in the calibration
obtain the selected layers. We conduct the experiment Countries,
with very small calibration set. In all other experiments in the paper,
IMPACT OF TRANSMITTED TOKEN LENGTH ON NLD
Transmitted token length important factor affecting the performance of natural language- based commu- nication methods like NLD, which refers to maximum number tokens generated by the sender model to communicate with receiver model. investigate the impact of transmitted token length on NL.D, we con- duct experiments on HotpotQA, MultiFicldQA-en, and 2WikiMQA datasets with different transmitted token lengths ranging from 64 to 1024 tokens. The results shown in Figure 12 We can see that the transmitted token length increases from 64 128, the performance of NLD improves. However, the transmitted token length continues increase beyond 128 tokens, the performance gains become marginal. This suggests that there is moderate transmitted token length (e.g., 128 tokens) is sufficient without incurring excessive com- munication overhead. In our main experiments, we set the transmitted token length to 256 tokens for NLD to ensure fair comparison with other methods.
MULTI-SOURCE KVCOMM
J.1 EXTENDING KVCOMM TO MULTIPLE SOURCES
KVComm can be naturally extended to multiple sources by integrating the KV pairs from different sender models. Mathematically, if have N, sender models Ma, Ma2......... M,y. and one receiver model M,, each sender Ms, processes the context C, and generates its own KV pairs {(k,, v(,)) at each layer I. The receiver Mr then receive the KV pairs from all senders and them compute the attention scores. The
22

---

Published as a conference paper at ICLR 2026
Result with Different Taken Lengths
0.25
Model Pair Index 6
0.20
Model Pair Index 7
0.15
Model Pair Index 8
0.10
2.00
400
600
800
1000
Transmitted
Length
Figure 12: Effect of transmitted token length on NLD. A moderate length is sufficient for NLD.
attention scores be computed follows:
H 121 N.
S HT b=19=1 cm1
where |Gi] is the number tokens in context Ct, and oh.q.i is the attention weight assigned by head layer from token @ to the context token of sender Ma.. The attention scores St are then integrated with the Gaussian prior to compute the selection scores.
Given the selection scores, a subset of KV pairs {(k v))) } can be selected from each sender model M., at each layer 1j. The selected KV pairs are concatenated to form the final KV pairs for the receiver model M,:
k' +- [K K2, kkk,, k.], v + (v' ;..
where corresponds to selected layer lj.
J.2 EXPERIMENT WITH TWO SENDERS AND ONE RECEIVER
We experiment with the scenario of two senders and one receiver to demonstrate the feasibility of extending KVComm to multiple sources. As shown in Table 10 we find that two senders can outperform one sender, for 17 out of 27 cases. We argue this is because of the diversification of information sources and agent thought. Owing to the usage of KV pairs, can naturally integrate multiple sources, while NLD and CIPHER cannot, suffering performance degradation.
K ADDITIONAL DISCUSSION
We have additional discussions the details and choices of our method.
all layers. For the receiver model, in each layer, we shift its positions by [C]. where |C] is the length context. For selected layers, we concatenate the KV pairs the sender at positions [0, (CI), and the KV pairs of the receiver follow at positions |[C], |C| + (Q)). For non-selected layers, positions (0, (Cl) are left empty (unattended), but the KV of the receiver still starts at position |C]. This approach ensures that all layers share consistent positional frame, so the attention mechanism
Positional Embedding Coherence KVComm is designed preserve positional coherence
the same offsets at every depth, avoiding positional drift across layers. We perform an ablation study to validate this design in Appendix M
Communication Cost Under the scenario where agents are connected with high-bandwidth links, the com- munication cost is relatively low compared to recomputation cost (Jin et al. 2024 Liu et al. (204c). KVComm more preferred when the information exchange volume is large (e.g., long contexts) and the communication bandwidth sufficient. scenarios with limited bandwidth, further compression of KV pairs or more aggres- sive layer selection may be necessary, which we leave for future work.
L CONTEXT-ADAPTIVE ONLINE CALIBRATION
A simple yet effective dynamic selection mechanism is recompute the selected layers every T queries, where T hyperparameter that he dynamically determined by server workload. We make fully mixed
23

---

Published as a conference paper at ICLR 2026
Table 10: Communication results for one sender and two senders scenarios. We bold the better result comparing one sender and two senders for each method. KVComm (0.3/0.5/0.7) denotes selecting 30%/50%/70% of layers' KV pairs for communication, i.e., M [0.3L]. M [0.5L],
M [0.7L]
Method
Sender
HotpotQA
MuSiQuest
2WikiMQA
M.. Saarre-aillllmmm--------epee
M yuvaji//Evvilooeelaaaa-....
M,: Temm-ACE7/too.AC22--aaa-----
Baseline
0.04
0.01
0.01
NA
Skyline
0.37
0.15
0.25
M
0.17
0.05
0.02
NLD
My and Ma
0.14
0.04
0.03
Mr
0.19
0.06
0.03
CIPHER
My and Ma
0.16
01.55
0.03
Mrs
D.12
0.04
0.19
KVComm (0.3)
My and Maz
0.16
0.06
0.23
M
0.38
0.[4
0.28
KVComm (0.5)
Ma and Maz
0.39
0.20
0.21
Mr
0.50
0.25
0.33
KVComm (0.7)
Me. and Mez
0.53
0.29
0.34
Maz Orion-zhen//Qwen2.5522 Instrutttttttt----e------- Instrructt-Unceesoreed
M, bespoketabs/Beeokk-SSttttsss
Baseline
0.13
0.03
0.09
NA
Skyline
0.53
0.25
0.09
My
-
0.16
0.04
0.02
NLD
Myy and M.2
0.18
0.06
0.02
M
0.03
0.03
0.03
CIPHER
M., and M.z
0.02
0.01
0.01
MM
0.02
0.01
0.08
KVComm (0.3)
My and Maz
0.02
0.05
MMa
0.28
0.12
0.10
KVComm (0.5)
May and Moz
0.32
0.18
0.08
M Ma, M.,
0.41
0.25
0.15
KVComm (0.7)
M., 0.50
0.24
0.24
Ma
Me, edrissofoor//hhcoon--llatasstt
M,: huthuraaiFaaccomm///Bnnntttttttt
Baseline
0.21
0.04
0.23
NA
Skyline
0.76
0.56
0.45
Mr Mr 0.52
0.25
0.24
NLD
M., and M.,
0.22
0.13
0.20
M.
0.27
0.07
0.25
CIPHER
My and M
0.15
0.04
0.14
Ma
0.39
0.40
0.29
KVComm (0.3)
My and M
0.51
0.30
0.27
0.6.
0.44
0.:7
KVComm (0.5)
Maj and Maz
0.49
0.43
0.35
Mr
0.55
0.42
0.31
KVComm (0.7) Ma and May
0.60
0.46
0.31
datasets, i.e., mixing all the samples from two datasets: Countries and Tipsheets; Countries and MultFFeelQAA- We then perform online calibration and evaluate with different calibration intervals T. As shown Fig- ure we find performance drops when increases, which is consistent with intuition.
Beyond periodic recomputation, more sophisticated adaptive mechanisms are also feasible, For example, the receiver model could leverage lightweight signals, such as token-level entropy and attention sparsity patterns, to trigger on-demand re-selection of informative layers. This is exciting direction for future work, and KVComm provides clean foundation for such extensions.
Additionally, illustrate different the selected layers are different datasets, we calculate the Kendall's Tau similarity of layer rankings for each pair of datasets across all models. As shown in Figure 14, some tasks share quite similar layer ranking for given model pair. e.g., model pair index 6 shares similar layer ranking for HotpotQA and MuSiQuest datasets. This phenomenon could guide the design of dynamic selection mechanisms future work.
24

---

Published as a conference paper at ICLR 2026
KVComm (0.3) Results on Mixed tasks with Online Calibration Countries MultifieldQA_een
Countries _ountries_TipShee TipSheets
0.6
Model Pair Index
Medel Pair Index
0.4
0.2
100
10a
10a
10a
10*
10a
10a
102
10a
10
Calibration Interval
Calibration Interval
Figure 13: Online calibration performance drops when the calibration interval increases.
Ranking Similarity Across Dataset
Model Pair Index 6
Model Pair Index
Model Pair Index 8
Tipsheets
1.0
Countries
HotpotGA-
0.9
MusiQuest-
MutiFieldOAeen
0.8
2WikMMDA
TMATH
Tichouts
Countries
HoroociAA
QASPER
MusiQuas
MutiFiiooa meoiioaaeee 290ii TIMATH
Ticsheeds
Caunriaa
HatporQA OASPER
MusiQuese
ManiFeao Fieeeeo ZWiMMOA THAATH
Tiasheets
Countries
NorcotAA QASPER
MusiQuss
Munor MonFeeoG en 21N0w0 TIMATH
Figure 14: Kendall's Tau similarity of layer rankings between different datasets.
M POSITIONAL EMBEDDING COHERENCE
Table 11: Comparison KVComm and KVComm-S, KVComm-S denotes shifting back the token positions of non-selected layers O) We bold the best results between KVComm and KVComm-S under the same settings.
Method
Countries
Tipsheets
HotpotQA
QASPER
MuSiQuest
MultiField -QA-en
2WikiM -QA
TMATH
M.: hulhui-aiLLlam--32--B
M,: suaypfalha/DeepSeek Risis--ili-isil--aaaaaa--
KVComm-S (0.3)
0.26
0.65
0.40
0.10
0.14
0.19
0.21
0.36
KYComm (0.3)
0.46
0.45
0.46
0.09
0.28
0.15
0.28
0.35
KVComm-S (0.5)
0.49
0.74
0.57
0.28
0.32
0.45
0.30
0.35
KVComm (0.5)
0.57
0.81
0.57
0.27
0.32
0.51
0.36
0.35
KYComm- (0.7)
0.52
0.76
0.65
0.30
0.39
0.46
0.32
0.35
KYComm (0.7)
0.57
0.81
0.65
0.29
0.36
0.47
0.37
0.35
M.:
M.:
KVComm-S (0.3)
0.00
0.20
0.02
0,02
0.01
0.09
0.05
0.34
KVComm (0.3)
0.04
0.26
0.02
0.01
0.01
0.09
0.08
0.31
KVComm-S (0.5)
0.04
0.90
0.33
0.13
0.18
0.35
0.16
0.35
KVComm (0.5)
0.88
0.28
0.07
0.12
0.26
0.10
0.33
K KVComm-S (0.7)
0.36
0.94
0.42
0.19
0.24
0.35
0.16
0.34
KYComm (0.7)
0.41
0.89
0.41
0.21
0.25
0.29
0.15
0.34
M,: christoforuflllcon--llrraaat M.::
KVComm-S (0.3)
0.47
0.71
0.54
0.10
0.36
0.19
0.26
0.32
KYComm (0.3)
0.46
0.69
0.59
0.19
0.40
0.35
0.29
0.32
KVComm-S (0.5)
0.36
0.97
0.67
0.27
0.46
0.34
0.36
0.34
KVComm (0.5)
0.40
0.92
0.63
0.25
0.44
0.45
0.34
0.35
KVComm-S (0.7)
0.21
0.95
0.59
0.26
0.52
0.46
0.37
0.36
KYComm (0.7)
0.19
0.96
0.55
0.26
0.42
0.51
0.31
0.36
We quantify the importance of positional embedding coherence hetween the sender and receiver models. We perform ablation experiment where, for non-selected layers, instead of shifting the receiver's takens
25

---

Published as a conference paper at ICLR 2026
position [C], place them back position creating positional inconsistency with selected layers. As shown Table [T] positional inconsistency does have detrimental effect performance, but overall, approach has merit.
N COMPLEXITY ANALYSIS DETAILS
compare computational complexity of our KVComm framework with the Skyline method and the NLD method. Recall that L is the total number of layers in the model, M the number of selected layers for communication. We use d to denote the hidden dimension of the model, and [Q] and |C] to denote the number of tokens in the query and context, respectively. Suppose M, would generate T, tokens in total, and the number of generated tokens is the same across different methods. For NLD, M. and M, would each generate T. and tokens for the initial answer. respectively.
Ignoring the embedding, output layers, and other minor components, the computational complexity of prefilling a sequence of length N with a single decoder layer is O(Na² + N²d), while the complexity of decoding a single token is Od (N))), where is the index of the generated token. Therefore, the total computational complexity M. to process the context C' is O(L(IC)d + [C)]d)).
total computational complexity of KVComm consists of three parts: (1) complexity M, to process
the context C, which is OLL[ + dd). (2) the complexity of Mr to process the query Q with the selected M KV pairs from Ma. which is (LL]Qid= + M([C] + [Q])QQd + (L M) Q)² and (3) the complexity of Mr to generate T, tokens with the selected M KV pairs from M,,, which is (Ld + M[[CI+ IQ| Tr)d (L (L-M) T.)d)). is:
Therefore, the total computational complexity of KVComm
T TKKVCom)) = (KVComm)= =(110(+
+0(11110+100 T TIQI) CM (IQ)+ ((Q)+
The computational complexity of Skyline method consists of two parts: (1) the complexity of prefillngg the concatenation of the context C and query Q. which is O(T(IC) + dQ + M[[C| + [Q)) and (2) the complexity of decoding tokens, which (O(TT(((+((()+----101000 ((C)+))+)++ Therefore, the total computational complexity the Skyline method is:
T(Skyline) = ((10)//))))=(((((()++++++)+)
+
The margin of KYComm over Skyline is:
((Skyline) T(KVComm) -T)
For NLD, the total computational complexity consists of three parts: (1) the complexity of M, to process the
context C' and generate T. tokens, which is (C//C//d+(C(()))) TT. ((C[++T.d) (2) the complexity of MM, process the query and generate T, tokens, which is O T.)d)), and (3) the complexity of M, to process the entire debate history and generate tokens, which OLL((T, + T, + [Q + (T, + T, (Q)) + (dd (T, T, + !QI + T, ).). Therefore, the total computational complexity of NLD is:
1Q] + Ltd ( + ([Q)
T(NLD) O of L. 21 + Tr o.)
10 +2 (T. WT+ (T.+T++IQ)) 121)²
+T. T. Q1) T. +TT1)+)
26

---

Published as a conference paper at ICLR 2026
The margin of KVComm over NLD is:
T(NLD) T(KVComm) = ((((2T++27, + + +Q1))d
+ +0
[ [T(TT² T (T.
+ +:10+++-10++70000 - -CM(1+1))) )a)
O USING ONE CHUNK OF LAYERS
We conduct the same experiment Section 4.3 on the HotpotQA dataset using other model pairs in Table 5
The results are shown in Figure We can see that the results consistent with the observation in Section
27

---

Published as a conference paper at ICLR 2026
F3 Score with One Contigg Chank of Conten.
F1 Scote MiL Slapers
Impact of Chunt Postion under fiael elayers
25-
0.6
layefer
20
15-
0.6
a=1.0
Comig.
0.4
0,5
am0.5
10
0.3
5-
0.2
gm0.6
0.2
elayers-11
0-...
elayers-15
10
bayers
#Layers
layefren
Llama-3.2-31 as both M, and M.
One Contig, Contact
Score #layer
elayers
0.5
25
#layers-10
20
0.6
0.6
Conti
dlayees115
tayorvon
0.4
0.4
0-1.3
0-1.0
0.3
0-0.8
0.2
0.2
0.1
0.0
10
20
10
bayer
Layyrs
layeffrem
(b) Qwen2.5-775 2---s-B-t- as both M, and Mr
Score with One Contig, Chark Contect Contest
Score
of Chunk Postion Fard
0.3
elayors-6
0.4
gm0.9
layefren
Scooe
15
0.3
0-0.7
10.
0.2
0m10.6
0.1
5-
0
0.0
10
20
0
10
20
10
15
layer,
#layers
layefnon
(c) Qwen2.5-78 B
as M, and Bespoke-Stratoss Stratos-ttttttt as Mr
F3 Score with One Contig, Chark Context
F1 Score w.t. #Layers
Impact of Chank Posiiion under rised #Layors
30
Contip
0.25
#byers=-1
0m1.0
#byers:1
t5yehee
20
0.4
d-08
Score
0-0.7
aw0.6
10
0.2
0-0.55
0.05
layeri
Llayers
yayer
(d) EvolCodeLlama-3 3.1-8-B..BBB....... -----ttttuctttttt M. and MaolACE-2-LLaamee 1.1--B Mr
Impect
#layen
0.7
0.6
0.6
bayofeun
Comtyy
0.5
g=005
0.4
0.0
5-
0.3
p-0.7
0.3
#byyer=-1
0.2
0.2
0.2
slayers-15
10
20
10
10
15
layena
layefhen
(e) Llama-3.2 3.2-3BB-Instruct-eeee
as M. and DeepSeck-RI RI-Distill-Llama 3B as M,
Figure 15: Experiment results of using one chunk of layers for communication on HotpotQA dataset using different model pairs.
28


Reviews:
[{"review_id": "w84riBeXjJ", "reviewer": "Reviewer_1arg", "summary": "In this paper, the author studies the interesting problem of inefficient communication in multi-agent LLM systems. The authors argue that existing methods, such as using natural language communication, suffer from high inference costs and information loss , while methods using hidden states of last token can be ineffective. To address this, the authors propose KVComm, a framework for communication via the selective sharing of KV pairs from a sender model to a receiver model. The framework is based on identifying the most informative layer's KV pairs to transmit. This strategy assumes that intermediate layers contain the most transferable semantic knowledge, and that layers with higher attention paid to the context are more valuable. Based on evaluation on 6 benchmarks, the paper highlights the efficacy of the proposed approach.", "questions": "Please clarify weakness 1.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written in general and the research problem has been well-articulated to the reader. It addresses an important and timely topic, communication among large language model (LLM) agents, which is relevant to the community given the growing interest in multi-agent systems.\n\n2. The proposed framework of using KV pairs as the medium for communication is intuitive and also looks technically sound. \n\n3. The paper shows dominant performance over existing communication protocols like AC, NLD, and CIPHER across a diverse set of tasks and model pairs demonstrating its efficacy. Further, the authors perform several ablations to support the claims.", "weaknesses": "1. The only concern is whether the proposed method generalizes to scenarios where the sender and receiver are LLMs with different architectures. In the experiments, the authors state that they \"limit the choices of the two LLMs to (1) two instances of the same LLM, and (2) two models that are fine-tuned versions of the same base LLM\". If its a limitation then the authors should clearly mention it.\n\n2. The KV cache sharing framework has some conceptual similarities with a prior work [1]. The authors are encouraged to clarify the distinctions of their approach relative to that work to better highlight the novelty.\n\n\n[1] DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6aesngmAnL", "reviewer": "Reviewer_1XmS", "summary": "The paper introduces **KVComm**, a novel communication protocol for multi-agent systems based on Large Language Models (LLMs). Instead of relying on natural language or hidden states, KVComm selectively shares key-value (KV) pairs across models using a strategy guided by attention importance scores and a Gaussian prior. The method significantly reduces communication and computation overhead while maintaining or surpassing task performance on diverse benchmarks.", "questions": "1. **How transferable is the KVComm strategy across model families?** The paper evaluates KVComm mostly on pairs of identical or fine-tuned models. How would this framework perform if the sender and receiver are *structurally different models*, e.g., GPT-style vs. Mistral-style? Would attention alignment or KV dimensions cause integration issues?\n\n2. **Can the selection of layers be made context-adaptive instead of fixed?**\n   The Gaussian prior is fixed during calibration. Could performance improve if KV layer selection is dynamically adjusted based on the current query or context (e.g., using entropy or other uncertainty measures)?\n\n3. **Why use a Gaussian prior for layer selection?**\n   While intuitive and empirically justified, the Gaussian prior assumes a unimodal distribution over informative layers. Have other priors or non-parametric approaches (e.g., entropy-weighted or data-driven learned selection) been considered?\n\n4. **Does the integration of KV pairs affect positional encoding or cause mismatch issues?**\n   Since KV pairs include positional biases (especially in rotary or absolute encodings), how does the concatenation affect the receiver’s decoding stability and accuracy?\n\n5. **What is the impact on latency and memory footprint during runtime?**\n   The paper discusses FLOPs and communication savings, but does the selective KV sharing induce memory overhead (e.g., extra caching or tensor reshaping), particularly for large batch sizes or streaming inference?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1: Using KV pairs rather than natural language or hidden states is a compelling, less explored direction that avoids known bottlenecks like information loss and concentration bias.\n\nS2: The use of attention importance with a Gaussian prior to guide layer selection is well-motivated and supported by empirical validation.\n\nS3: The authors benchmark across multiple datasets and LLM pairs, including ablations and comparisons with strong baselines like Skyline and AC.\n\nS4: The approach offers substantial computation and communication savings (2.5x–6x), making it relevant for scalable deployment scenarios.\n\nS5: The paper provides detailed setup information, dataset samples, and makes its code and synthetic datasets available.", "weaknesses": "To be honest, I carefully read this paper but I have not found some serious weakness. Below are just some suggestions that may lead to a more solid work.\n\n\n\nSuggest 1: The method is primarily validated on same or fine-tuned model pairs. Scalability to truly heterogeneous models remains unexplored.\n\nSuggest 2: The selection strategy is static post-calibration. Dynamic or context-aware layer selection could further enhance flexibility.\n\nSuggest 3: Several datasets are synthetically created or downsampled, which might not fully reflect real-world task complexity or communication needs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fd0yFH4JQh", "reviewer": "Reviewer_i79Z", "summary": "The paper proposes a framework for communication between LLMs in multi-agent systems. Instead of using natural language or hidden states to exchange information, it enables models to communicate efficiently by selectively sharing KV pairs from their attention layers. The authors introduce a layer-wise KV selection strategy, identifying the most informative layers for transmission. Experiments across eight model pairs and multiple reasoning datasets show that KVComm achieves performance comparable to an upper-bound “Skyline” setup while reducing computation and communication costs.", "questions": "See above", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The paper tackles an important but underexplored question: how to design effective and efficient communication protocols between LLMs in multi-agent systems. Both the *effectiveness* and *efficiency* of inter-agent communication are crucial for scaling such systems, making this direction timely and meaningful.\n* The authors conduct several preliminary experiments to justify their design choices, such as analyzing token importance and the limitations of hidden-state communication. These analyses help motivate the proposed selective KV sharing mechanism.\n* The paper compares against a wide range of baselines and evaluates on diverse tasks and model pairs.", "weaknesses": "* The proposed setup essentially reduces the “multi-agent” interaction to one model encoding the context and another generating the response. And since the paper uses KV-cache as medium, this feels closer to a *single-model sparse attention* or *cache reuse* scenario rather than true multi-agent communication. Moreover, if we really consider this problem under the \"multi-agent\" context, then transmitting activations or KV caches between agents incurs substantially higher communication cost (floating-point tensors) than natural language (token IDs). If minimizing \"transmission cost\" is a goal, as stated in Line 127, the paper needs stronger justification for why direct context forwarding or lightweight textual communication is not preferable.\n* Some explanations are vague or underspecified. For example, the term \"*strong* attention distribution\" seems not formal; Section 2.2.1’s experimental procedure (“remove or retain hidden states of specific tokens”) is poorly explained; and Section 2.2.2’s description of “prepending hidden states” is ambiguous, does it concatenate layer j's output from speaker model to the first position of layer k of receiver, and go through the remaining forward pass? Figures 3 also do not clearly define axes or mapping between sender and receiver layers.\n* The NLD baseline performs unexpectedly poorly across datasets. It is unclear whether this is due to an inappropriate setup or prompt mismatch. If the NLD agents still follow a debate-style dialogue rather than an information-transfer style, the comparison may not be meaningful, since the objective differs fundamentally.\n* A key challenge of activation- or KV-based communication is its poor transferability across models with different architectures or base weights. This greatly limits the generality of the proposed method. The current experiments only involve models fine-tuned from the same base, so the claim of broad applicability is not yet substantiated.\n* Several figures suffer from layout or readability problems, e.g., overlap in Figure 1 (“Extract attention”), overly tight spacing in Figures 2–3, and Figures 4–6.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vXrxZ0W6Zm", "reviewer": "Reviewer_XrXt", "summary": "The paper proposes KVComm for inter-LLM communication via selective KV-cache sharing. The specific techinique is guided by attention-based importance and Gaussian layer prior. Experiments show moderate gains over prior activation- or embedding-based protocols, yet KVComm still sometimes fall short of the full transmission baselines.", "questions": "Please refer to Weakness", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Selective KV-cache sharing with attention-based layer choice yields measurable gains over prior hidden-state or embedding protocols.  \n- Single-sample calibration generalizes across datasets, offering a lightweight deployment recipe.", "weaknesses": "1. I may misunderstand: the method appears restricted to two-agent pairs that share a parameter space (Table 5: same model or parent-offspring). If KVComm neither extends to >2 agents nor to heterogeneous backbones, its scope seems a bit narrow.\n\n2. The authors should clarify whether KVComm generalizes to fully-heterogeneous, multi-LLM scenarios (differing hidden dimensions & vocabularies); otherwise the contribution risks being incremental and may overlap prior work [1].\n\n3. Line 78 claims “using all tokens’ hidden states does not guarantee effective communication,” yet the same input-concatenation is later labeled “Skyline” and outperforms KVComm on Tipsheets/MuSiQuest (Table 1); this reads somehow contradictory.\n\n4. Excluding Skyline from “best result” bolding in Table 1 warrants justification; its inclusion would contextualize absolute gains.\n\n5. Fixing k % of layers manually is not adaptive; an automatic, data-driven or model-driven criterion to decide the subset size would strengthen practicality.\n\n6. suayptalha/DeepSeek-R1-Distill-Llama-3B seems to be an unofficial version; the official releases are DeepSeek-R1-Distill-Llama-8B and DeepSeek-R1-Distill-Llama-70B. Using an unofficial checkpoint complicates reproducibility a bit; why not adopt the official distillations?\n\n7. The authors may want to discuss the discrepancy and their unique contribution compared with [2], which seems to share similar motivation yet stronger adaptivity compared to this work.\n\n---\n\n[1] Dense Communication between Language Models\n\n[2] KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)