Communicating Activations Between Language Model Agents
Authors: Vignav Ramesh
Kenneth Li
Venue: ICML 2025 poster (2025)
Keywords: large language models
multiagent communication
embedding representation
multiagent debate
Primary area: deep_learning->large_language_models
TL;DR: We propose a simple technique whereby language models communicate via activations, which outperforms traditional natural language communication at a fraction of the compute required.

Abstract: Communication between multiple language model (LM) agents has been shown to scale up the reasoning ability of LMs. While natural language has been the dominant medium for inter-LM communication, it is not obvious this should be the standard: not only does natural language communication incur high inference costs that scale quickly with the number of both agents and messages, but also the decoding process abstracts away too much rich information that could be otherwise accessed from the internal activations. In this work, we propose a simple technique whereby LMs communicate via *activations*; concretely, we pause an LM $B$'s computation at an intermediate layer, combine its current activation with another LM $A$'s intermediate activation via some function $f$, then pass $f$'s output into the next layer of $B$ and continue the forward pass till decoding is complete. This approach scales up LMs on new tasks with *zero* additional parameters and data, and saves a *substantial amount of compute* over natural language communication. We test our method with various functional forms $f$ on two experimental setups—multi-player coordination games and reasoning benchmarks—and find that it achieves up to $27$% improvement over natural language communication across datasets with $<$$1/4$ the compute, illustrating the superiority and robustness of activations as an alternative "language" for communication between LMs.

Communicating Activations Between Language Model Agents
Vignav Ramesh
Kenneth Li'
Abstract
2023; Xi al., 2023; Wang al., 2024; Ahn al., 2022; Schick et al., 2023; Shen et al., 2023; Park et al., 2023;
Communication between multiple language model (L.M) agents has been shown to scale up the reasoning ability of LMs. While natural lan- guage has been the dominant medium for inter- LM communication, it is not obvious this should be the standard: not only does natural language communication incur high inference costs that scale quickly with the number of both agents and messages, but also the decoding process abstracts away too much rich information that could be oth- erwise accessed from the internal activations. In this work, we propose a simple technique whereby LMs communicate via activations; concretely, we pause LM B's computation at an intermediate layer, combine its current activation with another LM A's intermediate activation via some func- tion f. then pass f's output into the next layer of B and continue the forward pass till decod- ing is complete. This approach scales up LMs on new tasks with zero additional parameters and data, and saves a substantial amount of compute over natural language communication. We test our method with various functional forms on two experimental setups cetups-multi-planne multi-player coordina- tion games and reasoning benchmarks- find that it achieves up to 27.0% improvement over natural language communication across datasets with <1/4 the compute, illustrating the superior- ity and robustness of activations an alternative "language" for communication between LMs.
Nakano et al., 2022), communication between multiple co- operating agents has emerged as an intuitive approach to amplify the reasoning capabilities of L.LMs (Wu et al., 2023). Explicit communication in natural language between multi- ple LLMs has been shown to encourage divergent thinking (Liang al., 2023), improve factuality and reasoning (Du et al., 2023), enable integration of cross-domain knowledge (Sukhbaatar et al., 2024), and allow for modular compo- sition of abilities in complementary manner (Wu et al., 2023; Prasad et al., 2023).
A critical problem with natural language communication, however, is that it incurs extremely high inference costs that scale quickly with the number of agents as well as length and number of messages (Du et al., 2023; Yang et al., 2023; Wu et al., 2023). Restricting LLM communication to natural language also raises the question: as LLMs are increasingly capable of handling larger, more complex tasks (sometimes with 'super-human ability) (Wei et al., 2022; Burns et al., 2023), might they communicate more effec- tively in representations of higher dimension than natural language? While using natural language as communica- tive medium is appealing due to its interpretability, we claim that it may not be optimal for inter-LLM communication. Natural language generation uses only one token to repre- sent the model's s belief over the entire vocabulary, which risks losing information embedded within the model output logits (Pham et al., 2024); furthermore, model's belief over the entire vocabulary is itself not always better (for communicative purposes) than the model's (often richer) representation of the input in earlier layers. Indeed, Her- nandez et al. (2024) find that by around the halfway point of an LM's computation, it has developed "enriched entity representations" the input, where entities in the prompt are populated with additional facts about that entity encoded in the model's weights; but by the later layers these embed- dings are transformed into representation of the next word which leverages only parts of the previous, richer represen- tations, when that full embedding would be quite useful for communication.
1. Introduction
Language for the purpose of communication. As large language models (LLMs) have been increasingly used to power autonomous, goal-driven agents capable of reason- ing, tool usage, and adaptive decision-making (Yao et al.,
KKmmpnr Institute for AI, Harvard University, Cam- bridge, MA. USA. Correspondence to: Vignav Ramesh <vig- navramesh@ coatunccsh@olllc..hhe
Motivated by these concerns, this work outlines a simple technique whereby LLM agents communicate via activa- tions, thus enabling more efficient (i.e., higher-entropy) com-
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Communicating Activations Between Language Model Agents
munication fraction of the number of forward passes required at inference time. Concretely, we (1) pause a Trans- former LM B's computation at intermediate layer j in the residual stream; (2) combine its post-layer j activation with another LM A's post-layer k activation via some function f; and then (3) pass f"s output into the next layer j + 1 of B and continue its forward pass till decoding is complete. This aaproaahh scales up LLMs on new lasks by leveraging exist- ing, frozen LLMs along with zero task-specific parameters and data, applying to diverse domains and settings. Further- more, in requiring only partial forward pass through A and one forward pass through B, this method saves a sub- stantial amount of compute over traditional natural language communication, which we quantify in Section 3.2.
ful languages from scratch, even with centralized training, remains difficult (Lowe al., 2020; Chaabouni et al., 2019; Jaques et al., 2019).
With the emergence of large pre-trained language models, allowing communication between L.LMs in natural language has hence become a promising approach to enable coordina- tion among multiple LLM agents (Li et al., 2023). Recent works have demonstrated that such conversations enable integration of cross-domain knowledge (Sukhbaatar et al., 2024), modular composition of abilities in complementary manner (Wu et al., 2023), and improved task performance via splitting into subtasks (Prasad et al., 2023). Most notable is multiagent debate introduced by Du et al. (2023), where I.I .s provide initial responses and then make refinements by iteratively considering inputs from peers. While such methods have been shown improve performance vari- ous tasks over vanilla and majority-vote (Wang et al., 2023) style prompting, these experiments have only focused on large models (GPT-3.5/4 5/4, LLaMA2-70B and up), leaving the efficacy of debate on smaller, open-soure models underex- plored; our study addresses this gap by reimplementing Du et al. (2023) in experiments with smaller-scale (1 70B) models. More crucially, debate and similar natural language communication methods are extremely computationally ex- pensive, which this work addresses (Yang et al., 2023; Wu et al., 2023).
We validate our method by testing this approach with various funetional forms f on two experimental setups: two multi- player coordination games, where B is asked to complete task requiring information provided in a prompt to A; and seven reasoning benchmarks spanning multiple domains: Biographies (Du et al., 2023), GSM8k (Cobbe et al., 2021), MMLU High School Psychology, MMLU Formal Logic, MMLU College Biology, MMLU Professional Law, and MMLU Public Relations (Hendryeks et al., 2021). Our activation communication protocol exhibits up 27.0% improvement over natural language communication across these datasets, using <1/4 the compute. Critically, unlike prior work which test inter-LLM communication only on large-scale (> (0B) models (Du et al., 2023; Liang et al., 2023), we find that our approach generalizes across a wide array of L.LM suites and sizes, enabling even smaller L.LMs to unlock the benefits of communication.
Notably, Pham et al. (2024) propose CIPHER, which uses input (tokenizer) embeddings (as opposed to activations) to enable multi-agent communication; specifically, CIPHER passes the average tokenizer embedding (weighted by the LLM's next-token probabilities) between models. While (Pham et al., 2024) show this approach outperforms natural language debate, it (i) still faces substantial information loss relative to the model activations and (ii) does not save com- pute, as the number of these "average embeddings" passed between models is the same as the number of tokens passed between models in natural language communication.
In summary, our contributions are two-fold:
We propose novel inter-model communication proto- col for LLM agents that is purely inlvaationbbsedde
We perform comprehensive experiments to validate the improved performance of activation communication over traditional natural language communication. We also formally quantify our approach's compute savings over natural language communication, illustrating the superiority and robustness of activations as alterna- tive "language" for communication between LMs.
A related class of methods involves spending extra test-time compute reasoning in latent space (Geiping et al., 2025; Hao et al., 2024). Such latent reasoning approaches in- volving doing "chain-of of-thought in activation space," e.g. by grafting LM activations into other layers/later forward passes through the same model (c.g., a form of "recurrent AC" within single model); our approach can be viewed as doing exactly the same thing, but instead "outsourcing" the CoT to another model (and thus reaping benefits from greater diversity of thoughts/reasoning paths from distinct models).
2. Related Work
Multi-agent communication The field of multi-agent communication has a long-standing history. Notably, prior works emergent communication have showed that agents can autonomously evolve communication protocols when deployed in multi-agent environments that enable cooper- ative and competitive game-play (Sukhbaatar et al., 2016; Foerster et al., 2016; Lazaridou et al., 2017). However, re- cent experiments have demonstrated that learning meaning-
Activation engineering Activation engineering involves editing an LLM's intermediate layer representations during
a forward pass to create desired changes to output text (Li
2

---

Communicating Activations Between Language Model Agents
et al., 2024; Turner et al., 2023). Past work has explored extracting latent steering vectors from a frozen LLM to con- trol quality and content of completions (Subramani et al., 2022), as well as using "direction" vectors (computed as the difference in activations between two prompts) that en- able inference-time control over high-level properties of generations (Li et al., 2024; Turner et al., 2023). This work involves activation editing that similar to such prior works at high level, though for the purpose of communication between LLM agents.
Section 3.3.
3.1. Method
Consider two language models, A and B, and some setting in which B must perform task where it would benefit from knowledge given to A prompt/encoded in A's weights (example settings in Section 4.1/Section 4.2 respectively). We propose incorporating information from A's post-layer k: activation A into B's post-layer, j activation h (and vice versa, though for simplicity we henceforth only discuss the first direction) (Figure 1, left).
Model composition and grafting Composing expert models has been a recurring strategy to improve large mod- els, with different methods imposing different restrictions on the types of base LLMs that can be combined. Mixture of Experts (Shazeer et al., 2017) requires that all experts are trained simultancously using the same data; Branch- Train-Mix (Sukhbaatar et al., 2024) trains a single base LM multiple times on different datasets, then learns a router on outputs. Crucially, these methods do not work when nei- ther model can do the task at hand well (i.e., they solve the problem of choosing which of several outputs is best, not that of generating high-quality output by recombining the disparate abilities of the various base LMs).
More formally, suppose A and B (which have model di- mensions dA and da respectively) are given prompts AA and TB respectively, where A is of length L tokens and 2 B is of length tB tokens. We first run a partial forward pass of B until layer j (henceforth denoted (t)( to get hB E R Then we (I) run a partial forward pass of A until layer & to get (gx(x)) == AA E Rlaxds; (2) replace the activation of the last token (hB.))t E Rd +-
f(hA,,)), (hB,)tn)) for some function f Rda+ds + Rda; then (3) continue B's forward pass till decoding is complete, resulting in an output = Byx
Let a = (ha,k))a b = (hB,)tm, For sake of simplicity assume d = dg. We consider three non-learned functions f:
Model grafting, in contrast, seeks to merge different mod- els immediately prior to or at inference-time. Past works have explored this the parameter level (e.g., task vector averaging as in Ilharco al. (2023), which requires that the base models be well aligned), probability distribution / token level as in Shen et al. (2024) (which imposes few restrictions on the relationship between the base models, but by virtue of being liken-based can result in cascading errors during decoding), and activation level (e.g., CALM (Bansal et al., 2024) which learns an attention layer on top of two models' intermediate layer activations and thus ables broader integration of model abilities than token-level methods, but requires re-tuning of the attention mechanism for every model pair). In this work, we seek to unify CALM and other activation- level grafting techniques under a single framework, parameterized by the function f used to com- bine activations; crucially, we explore simple forms of f (c.g., sum, mean) that that-unlike Bansal et al. (2024) require zero additional task-specific parameters and data, and are far more compute-efficient.
f(a, b) a + b
(sum)
f(a.) b) (a + b)
(mean)
f(a.
(replace)
For cases where, due to differences in A and B's training, A and B's activation spaces are quite different, we propose learning task-agnostic (depends only on the models A and B) linear layer W E R&d x da that projects a onto B's s activation space. Note that this introduces zero additional task-specific parameters and data, as we propose learning this "mapping matrix" " W only once for each model pair (A, B) using general text, e.g. sequences from A and/or B's pretraining data mixes. We can then perform sum, mean, or replace with Wa. b instead of a, b. We propose training W to minimize MSE loss over dataset of N sentences
LISE (0 =
1 N Σ
N
20
Wy 2
3. Communicating Activations Between Language Models
'When da # dg, the sum, mean, and replace functions are defined follows. Let d min(da,dp) and the concatenation operator. Then f(a, b) = bimaxtdg -d0) o (sum), f(a, b)
We propose simple yet effective technique whereby lan- guage models communicate via activations. We detail our approach in Section 3.1; provide analytical models of the compute saved over natural language communication in Sec- tion 3.2; and discuss the intuition behind this approach in
-d,)))4- + (n,))(dndd,0)1110 bimassddg- -d.0) 1/4 (Bmaxídg - d.0) + (.0)) (mean), and f(a,b) blmax(dg -d0) amax(ds d,0) 11:A (replace).
3

---

Communicating Activations Between Language Model Agents
Each friend getz
16/4
A
hatel
Aggi1
W
har
has
f
hat
byj
legt
hgi
Janet bought cockion, each of her friends. Bow many cookies does each friend get?
give equal number
Figure I. Overview of activation communication. (Left) Our method involves (1) pausing a Transformer LM B 's computation at layer j in the residual stream; (2) combining its post-layer activation with another LM A 's post-layer h activation via some function f then (3) passing 's output into the layer j + of of B and continuing the forward pass till decoding is complete. (Right) Any be the sum, mean, and replacement functions, well
function I can be used to combine A and B's activations; explore letting as a task agnostic learned linear layer (details in Section 3.1).
where each (1 (f) pair denotes the final-token layer-26 activations A and at layers and j respectively given the same sentence as input.
a P-length input, 7 forward passes of given a P-length input, and the activation replacement procedure. This re- quires
33P2
3.2. Compute Analysis
(ADD))+T((E T(PVV L(SPDKH АP (2)
To understand the significance of activation communication, we must formally quantify the compute this procedure saves natural language communication. For simplicity sup- pose the following (similar calculations can be made for the cases where A and B have differing model architectures and/or are given different prompts):
+3HP² 4PPP))
FLOPs, where F(D) = O(D) for non-learned f and O(D2) when f is the mapping matrix.
In all practical cases, (2) is substantially lower than (1).
A and both have L layers (each with H attention heads, key size K, and feedforward size F'), dimension D, and vocab size V
3.3. Why should this work?
Recall that Pham et al. (2024) propose CIPHER- communicating the average tokenizer embedding (weighted by the LLM's next-token probabilities) between models. We build upon the intuition behind CIPHER, which goes follows: the token sampling process during decoding risks substantial information loss from the model's output logits, and communicating model's veighted-average tok- enizer embedding essentially entails communicating both that model's 's final answer and its belief in that answer (over the entire vocabulary).
A and B are both given prompt of P tokens
A can send B single M-token message
B must produce an output of I tokens, given its prompt and A's message
Traditional methods require M forward passes of A given a P-length input, plus forward passes of B given a (P+M) length input. Following Hoffmaan al. (2022), this requires
Communicating activations, then, can be thought of com- municating strict superset of (next-token prediction, belief over entire vocabulary). as activations of late-enough lay- ers essentially encode the model's entire knowledge about the provided context as well as its predicted completion and confidence in that completion (see Figures and 7 in Hewitt & Manning (2019) and Hernandez al. (2024),
M M(PVDD + L(SPDKH APPKH 3
4PDF)) M/VDD L(8(P M)DKH (I)
4(P KH 3H(P M)² 4(P M)DF))
FLOPs. In contrast, at inference time, our method requires only partial (up till the kith layer) forward pass of A given

---

Communicating Activations Between Language Model Agents
respectively, which show that linear probes tasked with pre- dicting certain output characteristics from Transformer's intermediate layer embeddings of its input work poorly for early layers, extremely well after around the halfway point of computation, but then prohe accuracy drops closer to the final layers). Indeed, these curves of probe accuracy by layer indicate that the final layers and LM head "throw away" information not useful for next-token prediction that very well could be useful for communicative purposes; this is precisely why our proposed activation communication technique is not an iterative approach (there is no notion of "rounds" like in debate and CIPHER, which require an addi- tional token budget to extract more and more information out of the LM), as one activation grafting step from A to B inherently communicates to B all of A's knowledge/beliefss about the prompt it was given. Moreover, the extra informa- tion over the model's 's next-token prediction and confidence that is encoded in its activations is what makes activation communication more performant than its natural language counterpart, as we will see in Section 4.
(X) setup, where the agents not allowed to communicate; "single-agent skyline," where single L.MM is given the concatenation of A and B's prompts; and traditional natural language communication, where A is asked to output a message that is then given to B along with z.. All decoding is done greedily.
Table 2 presents the results for both coordination games using different instances of the same model as the agents (A = B). Across the 3B and 8B model sizes, activation communication (AC) with f replace almost completely recovers the gap between the zero-communication (X) and the single-agent skyline (SKYLINE), outperforming natural language communication (NL) using far less compute. We hypothesize that replace more effective than mean and sum as the former guaranteed to output vector within B's activation space, while the latter two likely do not (e.g.. the norm of the vector outputted by sum will be around double that of a typical activation). Furthermore, most of the information B needs is likely contained in its represen- tations of previous tokens in the sequence, hence losing its final-token representation does not hurt.
4. Experiments
4.2. Reasoning Benchmarks
We test our method two distinct experimental setups: multi-player coordination games (Section 4. 1) and reasoning benchmarks (Section 4.2). Qualitative results are available in Appendix A.
Next, we test our methods on a variety of reasoning bench- marks, spanning several real-world tasks and domains.
Baselines We benchmark activation communication
4.1. Multi-player coordination games
against the following two baselines:
Drawing from existing literature on multi-agent communi- cation, we design two Lewis signaling games (Lewis, 2008; Lazaridou et al., 2016) to test the efficacy of activation com- munication (example prompts and answers in Table 1):
Single Model: A single LLM responds to the prompt in natural language.
Natural Language Debate (NLD) (Du et al., 2023); Each LLM provides an initial response to the given prompt. Then, for each of r - 1 subsequent rounds, each LLM is prompted to refine its previous response given the other agents' responses as input. Note that NLD is the most direct baseline for our approach, as it is -tate-of-the-art natural language communication protocol. We fix 2 in our experiments.
1. Countries, where A is given as input a string of the format "[PERSON] is at the [LANDMARK]" and B is asked "Which country is [PERSON] located in?"
2. Tip Sheets (inspired by Lewis et al. (2017)), where A is given a simulated "tip sheet" and B is asked to make an informed investment decision in accordance with the information in the tip sheet.
Note that we do not compare to Pham et al. (2024), as they communicate the imput (tokenizer) embeddings rather than activationssoutput embeddings between models, and hence require a shared tokenizer and embedding table between agents which is extremely restrictive and prevents applica- bility to our experimental setup.
We synthetically generate 100 (Countries) and 70 (Tip Sheets) different prompts and answers of the same format as the samples in Table 1, and report the proportion out of those samples that B responds with an exact string match to the ground truth answer. As baselines, we consider "silent"
To determine the values of k and j for activation commu- nication (AC), we compute the accuracy on Countries and Tip Sheets for every pair (k,j) E ".....,30]+ Based on these results (shown in Figure 2) as well as Table 2, we fix k = = 26 and f = replace for the following experi- ments.
NNote one important critique of multiagent debate: that in cases where multiple agents are uncertain about the answer, there is no reason why referencing other agents' answers would generate more factual reasoning. Both CIPHER and activation communication solve this problem, as some notion of model confidence is heing communicated along with its next-token prediction.
5

---

Communicating Activations Between Language Model Agents
Table I: Multi-player coordination games. Sample (prompt, answer) pairs for each game.
Game
Sample Prompts & Ground-Truth Answer
AA: "Alice the Acropolis of Athens."
Countries
BB "Which country is Alice located in?"
B's Expected Answer: "Greece"
AA: "Acme Inc. has taken a nosedive, as its quarterly earnings have dipped 8%. Meanwhile Doe LLC and Kiteflyer Labs have both reached record-high stock prices of 89, but Kiteflyer is involved in an IP lawsuit with its competitors." rg: "You must invest in company of {Acme Inc., Doe LLC, Kiteflyer Labs).
Tip Sheets
Which do you invest in?"
B's Expected Answer: "Doe LLC"
Table 2: Accuracies (%) on both coordination games using two identical LLaMA family models. Communication at layer
j = 26. 95% confidence intervals (1000 bootstrap iterations) reported in parentheses.
Model
Method
Accuracy (Countries)
Accuracy (Tip Sheets)
X
0.0 (0.0, 0.0)
38.6 (38.6, 39.4)
SKYLINE
84.0 (83.5, 84.1)
100.0 (100.0, 100.0)
NL
69.0 (68.7, 69.3)
74.3 (74.0, 74.6)
LLaMA-3.2-3B
AC (sum)
34.0 (33.9, 34.4)
50.0 (49.6, 50.3)
AC (mean)
36.0 (35.5, 36.1)
80.0 (79.8, 80.4)
AC (replace)
78.0 (77.7. 78.2)
90.0 (89.9, 90.3)
2.0 (L.9, 2.1)
54.3 (54.2, 54.5)
SKYLINE
86.0 (85.7, 86.1)
100.0 (100.0, 100.0)
NL
77.0 (76.6. 77.1)
85.7 (85.3, 85.8)
LLaMA-3.1-8B
AC (sum)
71.0 (70.9, 71.4)
85.7 (85.5, 86.0)
AC (mean)
70.0 (69.7, 70.3)
92.9 (92.7,93.1)
AC (replace)
83.0 (82.7, 83.1)
95.7 (95.6, 95.9)
Across all experiment configurations, we fix the decoding strategy to nucleus sampling with p = 0.9.
best performance by leveraging multiple models of distinct capabilities and sizes, relative to the added inference-time compute over a single forward pass through any single model?
Models We conduct most of our experiments using LLaMA- 3.2-3B and LLaMA-3.1 3.--BBBBBBBBBB as the two agents. Additionally, to test our approach's 's robustness and generalizability, we conduct experiments with models belonging to various other suites within the LLaMA family and of several different sizes.
Datasets We evaluate our technique on seven reasoning datasets that span various real-world tasks and domains: (i) Biographies (Du al., 2023), which asks the LLM to generate a factual biography of a famous computer scientist; (ii) GSM8k (Cobbe et al., 2021), variety of grade school math problems created by human problem writers; and (iii) datasets randomly drawn from MMLU (Hendrycks et al., 2021): High School Psychology (from the Social Sciences category). Formal Logic (from the Humanities category), College Biology (from the STEM category). Professional Law (from the Humanities Category), and Public Rela- tions (from the Social Sciences category). We evaluate randomly-sampled size-100 subset of each dataset.
Note that for these experiments, we restrict the setting to communication between different models (rather than multi- ple instances of the same model in Section 4. 4.1), since the same model would have identical activations for the same prompts, meaning no information would be communicated in the grafting process. We argue that the multiple-model setting is realistic (perhaps more than the setting of mul- tiple instances of the same model), as recent advances in LLM development have led to the release of models with specialized abilities (Singhal et al., 2023) and of different sizes (Dubey et al., 2024) that merit complementary usage. Our work thus answers the question: How can we get the
In experiments involving the mapping matrix W, we in- stantiate W E R-106xx3072 using Xavier initialization and
6

---

Communicating Activations Between Language Model Agents
Acc. (%)
Acc. (%6
60
20
15
k
k
(a) Countries
(b) Tip Sheets
Figure 2. 2D contour plots of accuracy over different values of and (the layers at which we access/edit activations for A/B respectively). & = 26 is roughly optimal (+) for both (a) Countries and (b) Tip Sheets.
train for 10 epochs on dataset of 3072 sentences' ran- domly drawn from the Colossal Clean Crawled Corpus (C4) (Dodge et al., 2021). We use batch size 32 and the Adam optimizer with learning rate 0.001.
outperforms both single-model baselines. fact, AC offers up 27.0% improvement NLD across six of the seven reasoning datasets. When applying W to A's acti- vation before performing the replacement function, we see even further gains of 2.6 50.0% over vanilla AC for four of the seven datasets. We hypothesize that the benefits from the learned linear layer are less consistent across datasets be- cause the subset of C4 data used to train W likely contains text more semantically similar some datasets than others, hence some datasets provide W with out-of-distribution inputs which reduces performance compared to vanilla AC.
Metrics We measure the accuracy of the final response for the single models and AC. For NLD, measure the accuracy of the majority-hell final-round answer across agents when the answer is automatically verifiable (numeric in GSM8k, multiple choice for the MMLU datasets) or the average final-round answer across agents otherwise (Biogra- phies).
While we fix A as the smaller model and B as the larger model in Table (so as to ensure decoding happens with the presumably more capable model), this need not be the case; swapping and yields results of 81.5 0.0 and 61.0 on Biographies and GSM8k respectively (without the linear layer). While these accuracies are lower than their non-swapped counterparts, notably they still are higher than both single-model baselines (and higher than NLD for Biographies); plus this is much more compute-efficient efficciefe as the smaller model is now the one requiring the full instead of partial forward pass.
For GSM8k and the MMLU datasets, we report the pro- portion of samples in the dataset for which the generated answer exactly matches the ground-truth answer. For Bi- ographies, following Du et al. (2023), we prompt an LLM judge (LLaMA-3.1 1--BB) to check whether each manually- decomposed fact a ground-trun truth biography is supported (1), partially supported (0.5), or unsupported (0) in the gen- erated biography, taking the mean of these scores over all facts the per-biography accuracy and the mean over all dataset samples the total accuracy.
Note that we find AC outperforms NLD on 48 of the 57 datasets in the full MMLU benchmark; complete MMLU results, as well as a suite of additional experiments, are shown in Appendix B.
Comprehensive evaluation with the LLaMA family Ta- ble 3 presents results on each of the seven reasoning bench- marks across various baselines and activation communica- tion. Notably, while NLD consistently outperforms LLaMA- 3. 3.2-3B, it does not always display a performance improve- ment over LLaMA-3.1 1.MMA-3.1-50 3.; but remarkably, AC consistently
Performance-compute tradeoff and generalization to dif- ferent model scales Thus far, we have been considering the absolute performance of AC with respect to NLD, for which our method attains state-of-the-- results; however the superiority of activations language for inter-LLLM
We use 3072 sentences as linear regression with d- dimensional input has sample complexity of (d) (Vapnik, 1999).

---

Communicating Activations Between Language Model Agents
Table Accuracies (%) on all seven reasoning benchmarks. NLD and all AC variants involve communication between LLaMA-3.24 3.2-3B (4) and ALaMA-3.11880 (B); the performance of these models individually are presented in the first two rows of the table. NLD typically improves performance over at least one of the single model baselines; AC-both with and without the task-agnostic linear layer- consisteeny beats both baselines and NLD as well.
Method
Biog.
GSM8k
HS Psych.
Logic
Col. Bio.
Prof. Law
Pub. Rel.
3.2-3B
79.4+0.0
58.0+4.99
30.0+1.0
16.0+0.8
11.0+0.7
0.0+0.0
26.0+0.1
3.1-8B
83.910.0
60.0.4..9
65.010.1
42.010.1
50.010.2
20.0.0..8
53.010.2
NLD
80.2+0.11
75.0+44.3
83.010.8
37.0+0.1
71.0=0.1
30.0+0.1
63.0+0.7
AC
84.610..0
64.0.1..8
85.010.8
47.010.11
78.010.9
30.010.1
74.010.1
AC (W)
86.8
66.014.8
70.0+0.1
35.0
79.0+0.9
45.0
63.0
communication is further illustrated by AC's larger ratio of performance improvement to added inference-time compute over individual LMs. Figure 3 displays the results of single models, AC, and NLD across model scales and suites within the LLaMA family on the Biographies dataset. Incoming arrows to AC and NLD nodes denote the base models be- tween which communication occurred. Not only does AC consistently outperform both single-model baselines unlike NLD, but also notice that the slope of each black line is far greater than the slope of each gray line, indicating that AC consistently achieves greater increases in accurucy per additional unit of inference-time compute (normalized by the compute of a single forward pass through LLaMA-3.2-15 on the given prompt) compared to NLD.
ACT (cxn))
Single
5.1-88
Avencary (9)
3.2
291
Compute (normalized FLOPs)
Communication across model f'amilies Table 4 displays results for AC between models from the Qwen-2.5, Gemma-2, 2, and LLaMA-3 familics. We see that AC beats NLD across the board, and beats both individual models for 4/5 of the model pairs on Biographies/GSMBk respectively- demonstrating the efficacy of AC irrespective of model archi- tecture, size, tokenizer, and training data. Moreover, these results are obtained without training W, meaning we do not need a separate projection layer between activation spaces to attain SOTA results, even for extremely distinct models! (We hypothesize this because we are only replacing B's last-token activation, hence B can learn from without an extreme alteration to its activation distribution (Moschella et al., 2023; Kornblith et al., 2019).)
Figure 3. Accuracy (%) vs. compute (# FLOPs normalized by single LLaMA-3. 2-1B 3-1-B forward pass) for various configurations of AC and NLD on the Biographies dataset. AC yields the greatest performance gains per additional unit of inference-time compute over each baseline
There are some limitations to this method. First, when not using the learned model-specific mapping discussed in Section 3.1, our method requires both models to have aligned embedding spaces, such that the activation of one model roughly retains its meaning in the other's activation space (note that unlike past works such Pham al. (2024) we do not require shared tokenizers or aligned vocabularies, only aligned embeddings). While less restrictive than past works (Pham et al., 2024), this assumption is somewhat limiting, but can be relaxed when we let f be the learned model-specific mapping; and in practice we find that even amongst different models in the LLaMA family, no such mapping is required for state-of-the results.
5. Conclusion
We present simple approach to enable effective and compu- tationally efficient communication between language mod- els by injecting information from the activations of one model into the activations of another during the forward pass. Salient features of this approach include: (i) Scales up LLMs on new tasks by leveraging existing, frozen LLMs along with zero additional task sask-specific parameters and data, (ii) Applies to diverse domains and settings, and (iii) Saves a substantial amount of compute.
Second, this method requires access to embeddings and will not work with black-box API access; however exploring API-only approaches is highly limiting, and recent releases of powerful open-source models (Dubey et al., 2024) merit the development of embedding- based techniques.
8

---

Communicating Activations Between Language Model Agents
Table 4: Individual model, AC, and NLD accuracies across three model families. Each cell displays two values: Biographies score / GSM8K score.
Model Pair (4, B)
B
NLD
AC
LLaMA-3.2-88 LLaMA-3.1-8B
79.4=0.0/ 08.0000/88.000000 58.0
83.9 60.0
80.2+0.1 / 75. 0.33+4..3
84.6=0.0/ 64.0#-188
Qwen-2.5-1.52. Qwen-2.5-3B
23.440.9/20.010.
15.5#1.1/550-1.1
63.211.1 66.00.1.
89.61110/
70.011.0
Gemma 2-2BB Gemma-2-98
23.011.1/45.011.
94.6100./ 80.0.0..
70.311.0/ 0/3L110/70.011.0.
88.110.7 90.00.0.7
Quen-2.5-1.5B, LLaMA-3.2-33
59.4 00.00
79.4+0.0/ 58.0+4499
75.4+1.0/ 75.00+1.
10.5=11.0/5.0t1..
LLaMA-3.2-3B, Gemma-2-22
79.4=0.0/ 58.00
15.01.1/45.0=1l
62.5#1.1
000.11/55.0:: 555.0#1.1
84.0#0.1
60.0#1.1
(wen-2.6-1.52) Gemma- 2B
59.4:0.9/ 20.0....9
83.0:111 45.00111
49.31 55.0.:1.1
73.0+1.1 55.0+111
Third, while concern might be the limited interpretabil- ity of communicating activations as opposed to natural language, we note the following. First, there is a funda- mental tradeoff between interpretability and information preservation (as activations, by virtue of being much higher- dimensional than the space of natural language, allow pro- portionally higher-entropy communication) (Pham et al., 2024), which merits discussion beyond the scope of this work. But second, we actually posit that our method sug- gests a new avenue towards interpreting LM activations: "translating" activations based on the beliefs they induce as messages in listening agents, similar to the method put forward in Andreas al. (2018). We recognize this as a promising avenue for future research.
P., Quiambao, J., Rao, K., Rettinghouse, J., Reyes, D., Sermanet, P., Sievers, N., Tan, C., Toshev, A., Vanhoucke, V., Xia, F., Xiao, T., Xu, P., Xu, S., Yan, M., and Zeng, A. Do as i can, not as i say: Grounding language in robotic affordances, 2022.
Andreas, J., Dragan, A., and Klein, D. Translating neuralese, 2018.
Bansal, R., Samanta, B., Dalmia, S., Gupta, N., Vashishth, S., Ganapathy, S., Bapna, A., Jain, P., and Talukdar, P. Llm augmented Ilms: Expanding capabilities through composition, 2024.
Burns, C., Izmailov, P., Kirchner, J. H., Baker, B., Gao, L., Aschenbrenner, L.,, Chen, Y., Ecoffet, A., Joglekar, M., Leike, J., Sutskever, I., and Wu, J. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision, 2023.
Additional directions of future work include using AC to allow large LMs to leverage small, tunable LMs as "knowl- edge bases" during decoding (Lee et al., 2024), as in col- laborative decoding (Shen et al., 2024) setups; and testing our approach on more complex coordination games (e.g., Lewis-style negotiation games (Lewis et al., 2017), Diplo- macy).
Chaabouni, R., Kharitonov, E., Lazaric, A., Dupoux, E., and Baroni, M. Word-order biases in deep-agent emer- gent communication. In Korhonen, A., Traum, D., and Mårquez, L.. (eds.), Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pp. 5166-51 1176,5175, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18663/11/119-15009 URL https:ttppsottc htattps://aclaantholleeeo
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Cobbe, K.,, Kosaraju, V., Bavarian, M., Chen, M.. Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems, 2021.
Acknowledgements
The authors are grateful to Jacob Andreas, Yoon Kim, and Sham Kakade for their valuable discussions and feedback.
Dodge, J. Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., and Gardner, M. Doc- umenting large webtext corpora: A case study on the colossal clean crawled corpus, 2021. URL https:
References
/ wwwxiv.org/abal. 2104. 08758.
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O.,
David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Ho, D., Hsu, J., Ibarz, J., Ichter, B., Irpan, A., Jang, E., Ruano, R. J., Jeffrey, K., Jesmonth, S., Joshi, N. J., Julian, R., Kalashnikov, D., Kuang, Y., Lee, K.-H., Levine, S., Lu, Y., Luu, L., Parada, C.,, Pastor,
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. Improving factuality and reasoning in language models through multiagent debate, 2023.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A.,

---

Communicating Activations Between Language Model Agents
Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravanku-
Ni, B., Hancock, B., Wasti, B., Spence, B., Stojkovic, B., Gamido, B., Montalvo, B., Parker, C.,, Burton, C., Mejia, C., Wang, C., Kim, C., Zhou, C., Hu, C., Chu, C.-H., Cai, C., Tindal, C., Feichtenhofer. C., Civin, D., Beaty, D., Kreymer, D., Li, D., Wyatt, D., Adkins, D., Xu, D., Testuggine, D., David, D., Parikh, D., Liskovich, D., Foss, D., Wang, D., Le, D., Holland, D., Dowling, E., Jamil, E., Montgomery, E., Presani, E., Hahn, E., Wood, E., Brinkman, E., Arcaute E., Dunbar, E., Smothers, E., Sun, F. Kreuk, F., Tian, F., Ozgenel, F., Caggioni, F, Guzmán, F. Kanayet, F., Seide, F., Florez, G. M., Schwarz, G., Badeer, G., Swee, G., Halpern, G., Thattai, G., Herman, G., Sizov, G., Guangyi, Zhang, Lakshminarayanan, G., Shojanazeri, H., Zou, H., Wang. H., Zha, H., Habeeb, H., Rudolph, H., Suk, H., Aspegren, H., Goldman, H., Damlaj, I., Molybog, I., Tufanov, I., Veliche, I.-E., Gat, I., Weissman, J., Geboski, J., Kohli, J., Asher, J., Gaya, J.-B., Marcus, J.. Tang. J., Chan, J., Zhen, J.. Reizenstein, J., Teboul, J., Zhong, J., Jin, J., Yang, J., Cummings, J.. Carvill. J., Shepard, J., McPhie, J., Torres, J., Ginsburg, J., Wang. J., Wu, K., U, K. H., Saxena, K., Prasad, K., Khandelwal, K., Zand, K., Matosich, K. Veeraragha- van, K., Michelena, K., Li, K., Huang, K., Chawla, K., Lakhotia, K., Huang, K., Chen, L., Garg. L., A. L., Silva, L., Bell, L., Zhang. L., Guo, L., Yu, L.., Moshkovich, L., Wehrstedt, L., Khabsa, M., Avalani, M.. Bhatt, M., Tsimpoukelli, M., Mankus, M., Hasson, M., Lennie, M., Reso, M., Groshev, M., Naumov, M., Lathi, M., Ke- neally, M., Seltzer, M. L., Valko, M., Restrepo, M., Patel, M., Vyatskov, M.. Samvelyan, M., Clark, M., Macey, M., Wang, M., Hermoso, M. J., Metanat, M., Rastegari, M., Bansal, M., Santhanam, N., Parks, N., White, N., Bawa, N.. Singhal, N., Egebo, N., Usumier, N.. Laptev, N. P., Dong, N., Zhang, N., Cheng, N., Chernoguz, O., Hart, O., Salpekar, O., Kalinli, O., Kent, P., Parekh, P., Saab, P., Balaji,, P., Rittner, P., Bontrager, P., Roux, P., Dollar, P., Zvyagina. P., Ratanchandani, P., Yuvraj. P., Liang, Q. Alao, R., Rodriguez, R., Ayub, R., Murthy, R., Nayani, R., Mitra, R., Li, R., Hogan, R., Battey, R., Wang, R., Maheswari, R., Howes, R., Rinotr, R., Bondu, S. J., Datta, S., Chugh, S., Hunt, S., Dhillon, S., Sidorov, S., Pan, S., Verma, S., Yamamoto, S., Ramaswamy, S., Lindsay, S., Lindsay, S., Feng, S., Lin, S., Zha, S. C., Shankar, S., Zhang. S., Zhang, S., Wang. S., Agarwal, S., Sajuyigbe, S., Chintala, S., Max, S., Chen, S.,, Kehoe, S. Satterfield, S., Govindaprasad, S., Gupta, S., Cho, S. Virk, S., Subramanian, S., Choudhury, S., Goldman, S., Remez, T., Glaser, T., Best, T., Kohler, T., Robinson, T., Li, T., Zhang, T., Matthews, T., Chou, T., Shaked, T., Vontimitta, V., Ajayi, V., Montanez, V., Mohan, V., Kumar, V. S., Mangla, V., Albiero, V., Ionescu, V., Poe- naru, V., Mihailescu, V. T., Ivanov, V., Li, W., Wang, W., Jiang, W., Bouaziz, W., Constable, W. Tang, X., Wang, X., Wu, X., Wang, X., Xia, X., Wu, X., Gao, X., Chen,
mar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Roziere, B., Biron, B.. Tang, B., Chern, B., Caucheteux, C., Nayak, C., Bi, C., Marra, C., McConnell, C., Keller, C., Touret, C., Wu, C., Wong, C., Ferrer, C. C., Nikolaidis, C., Al- lonsius, D., Song, D., Pintz, D., Livshits, D., Esiobu, D., Choudhary, D., Mahajan, D.. Garcia- Olano, D.,, Perino, D., Hupkes, D., Lakomkin, E., AlBadawy, E.,, Lobanova, E., Dinan, E., Smith, E. M., Radenovic, F., Zhang, F,, Syn- naeve, G., Lee, G., Anderson, G. I.., Nail, G., Mialon, G., Pang, G., Cucurell, G., Nguyen, H., Korevaar, H., Xu, H., Touvron, H., Zarov, L., Ibarra, I. A., Kloumann, L., Misra, I., Evtimov, I., Copet, J., Lee, J., Geffert, J., Vranes, J., Park, J., Mahadeokar, J., Shah, J., van der Linde, J., Billock, J., Hong, J., Lee, J., Fu, J., Chi, J.. Huang, J., Liu, J., Wang, J., Yu, J., Bitton, J.. Spisak, J., Park, J., Rocca, J., Johnstun, J., Saxe, J., Jia, J., Alwala, K. V., Upasani, K., Plawiak, K., Li, K., Heafield, K., Stone, K., El-Arini, K., Iyer, K., Malik, K., Chiu, K., Bhalla, K., Rantala- Yeary, L., van der Maaten, L., Chen, L., Tan, L.. Jenkins, L., Martin, L.., Madaan, L.., Malo, L.., Blecher, L.., Landzaat, L.., de Oliveira, L.., Muzzi, M., Pasupuleti, M., Singh, M., Paluri. M., Kardas, M., Oldham, M., Rita, M., Pavlova, M., Kambadur, M., Lewis, M., Si, M., Singh, M. K., Hassan, M., Goyal, N., Torabi, N., Bashlykov. N., Bogoychev, N., Chatterji, N., Duchenne, O., Çelebi, O., Alrassy, P., Zhang. P., Li, Vasic, P., Weng. P., Bhargava, P. Dubal, P., Krishnan, P., Koura, S., Xu, P., He, Q., Dong. Q. Srinivasan, R., Ganapathy, R., Calderer, R., Cabral, R. S., Stojnic, R., Raileanu, R., Girdhar, R., Patel, R., Sauvestre, R., Polidoro, R., Sumbaly, R., Taylor, R., Silva, R., Hou, R., Wang, R., Hosseini, S., Chennabas- appa, S., Singh, S., Bell, S., Kim, S. S., Edunov, S., Nie, S., Narang, S., Raparthy, S., Shen, S., Wan, S., Bhosale, S., Zhang, S., Vandenhende, S., Batra, S., Whitman, S., Sootla, S. Collot, S., Gururangan, S., Borodinsky, S., Her- man, T., Fowler, T., Sheasha, T., Georgiou, T., Scialom, T., Speckbacher, T., Mihaylov, T., Xiao, T., Karn, U., Goswami, V., Gupta, V., Ramanathan, V., Kerkez, V., Gonguct, V. Do, V., Vogeti, V., Petrovic, V., Chu, W., Xiong, W., Fu, W., Meers, W., Martinet, X., Wang. X., Tan, X. E., Xie, X., Jia, X., Wang, X., Goldsehlag, Y., Gaur, Y., Babaci, Y., Wen, Y., Song, Y., Zhang, Y., Li, Y. Mao, Y. Coudert, Z. D., Yan, Z., Chen, Z., Papakipos, Z., Singh, A., Grattafiori, A., Jain, A., Kelsey, A., Shajnfeld, A., Gangidi, A., Victoria, A., Goldstand, A., Menon, A., Sharma, A., Boesenberg, A., Vaughan, A., Baevski, A., Feinstein, A., Kallet, A., Sangani, A., Yunus, A., Lupu, A., Alvarado, A., Caples, A., Gu, A., Ho, A., Poulton, A., Ryan, A., Ramchandani, A., Franco, A., Saraf, A., Chowdhury, A., Gabriel, A., Bharambe, A., Eisenman, A., Yazdan, A., James, B., Maurer, B., Leonhardi, B., Huang,
B., Loyd, B., Paola, B. D., Paranjape, B., Liu, B., Wu, B.,
10

---

Communicating Activations Between Language Model Agents
Y., Hu, Y., Jia, Y., Qi, Y., Li, Y., Zhang, Y., Zhang, Y., Adi, Y., Nam, Y. Yu, Wang, Hao, Y., Qian, Y., He, Y. Rait, Z., Vitoto, Z., Rosnbrick, Z., Wen, Z., Yang, Z., and Zhao, Z. The Ilama 3 herd of models, 2024. URL https: / raxiv..... org/ abs/ 2407 21783.
Korblithh, S., Norouzi, M., Lee, H., and Hinton, G. Simi- larity of neural network representations revisited, 2019. htLhttp::/ http://aaxvv...oo /ate/////////// aaaaaaaa///10 00414.
Lazaridou, A., Peysakhovich, A., and Baroni, M. Multi- agent cooperation and the emergence of (natural) lan- guage. arXiv preprint arXiv: 2016.
Foerster, J. N., Assael, Y. M., de Freitas, N., and White- son, S. Learning communicate with deep multi-agent reinforcement learning, 2016.
Lazaridou, A., Peysakhovich, A., and Baroni, M. Multi- agent cooperation and the emergence of (natural) lan- guage, 2017.
Geiping. J., McLeish, S., Jain, N., Kirchenbauer, J., Singh, S., Bartoldson, B. R., Kailkhura, B., Bhatele, A., and Goldstein, Scaling up test-time compute with latent reasoning: A recurrent depth approach, 2025. URL https: / //tp/v/aavvva//aa//aa//eeeaaaava org/ abs 250.250022.0 05171.
Lee, J., Yang, E, Tran, T., Hu, Q. Barut, E., Chang, K.- W., and Su, C. Can small language models help large language models reason better?: Lm-guided chain-of- thought, 2024. URL 2404.0340 03414.
/ttp://///xxv..rr/////// /abs/
Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., and Tian, Y. Training large language models to reason in a continuous latent space, 2024. URL https:// arxiv. orgoorgggoo.... abs / 2412222222 06769.
Lewis, D. Convention: A philosophical study. John Wiley & Sons, 2008
Lewis, M., Yarats, D., Dauphin, Y. N., Parikh, D., and Batra, D. Deal or no deal? end-to-end learning for negotiation dialogues, 2017.
Hendryeks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding, 2021. URL https: //arxiv.org abs/ 2000 03300.
Li, G., Hammoud, H. A. A. K., Itani, H., Khizbullin, D., and Ghanem, B. Camel: Communicative agents for "mind" exploration of large language model society, 2023, URL https: /tt:://axxivi wtw/abs//////eoooo/e//////e/eaae /2303. 17760.
Hermandez, E., Sharma, A. S., Haklay, T., Meng, K., Watten- berg, M., Andreas, J., Belinkov, Y., and Bau, D. Linear- ity of relation decoding in transformer language models, 2024.
Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36, 2024.
Hewitt, and Manning, C. D. A structural probe for finding syntax in word representations. In Burstein, J., Doran, C., and Solorio, T. (eds.), Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume (Long and Short Papers). pp. 4129-4138, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.11665//1/N11-14199 URL https: http:///aaaaathhoocc org - 1419.
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y.. Tu, Z., and Shi, S. Encouraging divergent thinking in large language models through multi-agent debate, 2023.
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mor- datch, I. Multi-agent actor-critic for mixed cooperative- competitive environments, 2020.
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L. A., Welbi, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A. Osindero, S., Simonyan, K., Elsen, E., Rae, J. W., Vinyals, O., and Sifre, L. Training compute-optimal large language models, 2022.
Moschella, L., Maiorca, V., Fumero, M., Norelli, A., Lo- catello, F,, and Rodolà, E. Relative representations en- able zero-shot latent space communication, 2023. URL https://aran
org hbtp://bs//20... 15430.
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., Jiang, X., Cobbe, K., Eloundou, T., Krueger, G., Button, K., Knight, M., Chess, B., and Schulman, J. Webgpt: Prowser-assisted question-answeree with human feed- back, 2022.
Ilharco, G., Ribeiro, M. T., Wortsman, M., Gururangan, S., Schmidt, L.. Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic, 2023.
Jaques, N., Lazaridou, A., Hughes, E., Gulcchre, C., Ortega, A., Strouse, D., Leibo, J. Z., and de Freitas, N. Social influence as intrinsic motivation for multi-agent deep reinforcement learning, 2019.
Park, J. S., O' O'Brien, J. C.,, Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. Generative agents: Interactive simulacra of human behavior, 2023.

---

Communicating Activations Between Language Model Agents
Pham, C., Liu, B., Yang, Y., Chen, Z., Liu, T., Yuan, J., Plummer, B. A., Wang, Z-, and Yang, H. Let models speak ciphers: Multiagent debate through embeddings, 2024.
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z.,, Tang, J., Chen, X., Lin, Y., Zhao, W. X., Wei, Z., and Wen, J.-R. A survey on large language model based autonomous agents, 2024.
Prasad, A., Koller, A., Hartmann, M., Clark, P., Sabharwal, A., Bansal, M., and Khot, T. Adapt: As-needed decom- position and planning with language models, 2023.
Wang, X., Wei, J., Schuurmans, D., Le, Q. Chi, E., Narang, S., Chowdhery, A., and Zhou, D. Self-consistency im- proves chain of thought reasoning in language mod- els, 2023. URL https: /arxiv..vvvv.////a// org/ has//bb///// 2203. 11171.
Schick, T., Dwivedi- Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Zettlemoyer. L., Cancedda, N., and Scialom, T. Tool- former: Language models can teach themselves to use tools, 2023.
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Met- zler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., and Fedus, W. Emergent abilities of large language models, 2022.
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q. Hinton, G., and Dean, Outrageously large neural networks: The sparsely-gated offture-of-experts layer, 2017.
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, I.., Zhang, X., Zhang, S., Liu, J., Awadallah, A. H., White, R. W., Burger, D., and Wang, C. Autogen: Enabling next- gen Ilm applications via multi-agent conversation, 2023.
Shen, S. Z., Lang, H., Wang, B., Kim, Y., and Sontag, D. Learning to decode collaboratively with multiple language models, 2024.
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang. J., Jin, S., Zhou, E., Zheng, R., Fan, X., Wang, X., Xiong, L., Zhou, Y., Wang, W., Jiang, C.. Zou, Y., Liu, X., Yin, Z., Dou, S., Weng, R., Cheng, W., Zhang, Q., Qin, W., Zheng, Y., Qiu, X., Huang, X., and Gui, T. The rise and potential of large language model based agents: survey, 2023.
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., and Zhuang, Y. Hugginggpt: Solving tasks with chatgpt and its friends in hugging face, 2023.
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L.,, Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., Schaekermann, M., Wang, A., Amin, M., Lachgar, S., Mansfield, P., Prakash, S., Green, B., Dominowska, E., y Areas, B. A., Tomasev, N., Liu, Y., Wong, R., Sem- turs, C., Mahdavi, S. S., Barral, J., Webster, D., Cor- rado, G. Matias, Y., Azizi, S., Karthikesalingam, A., and Natarajan, V. Towards expert-level medical ques- tion answering with large language models, 2023. URL ht https:/ /ttp///axiii aaxiv.iii.. org/abs// 2305. 09617.
Yang, H., Yue, S., and He, Y. Auto-gpt for online decision making: Benchmarks and additional opinions, 2023.
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. React: Synergizing reasoning and acting in language models, 2023.
Subramani, N., Suresh, N., and Peters, M. E. Extracting latent steering vectors from pretrained language models, 2022.
Sukhbaatar, S., Szlam, A., and Fergus, R. Learning multia- gent communication with backpropagation, 2016.
Sukhbaatar, S., Golovneva, O., Sharma, V., Xu, H., Lin, X. V., Rozière, B., Kahn, J., Li, D., tau Yih, W., Weston, J., and Li, X. Branch-train-mix: Mixing expert 1lms into mixture-of-e ef-eeerrss 1lm, 2024.
Turner, A. M., Thiergart, L., Udell, D., Leech, G., Mini, U., and MacDiarmid, M. Activation addition: Steering language models without optimization, 2023.
Vapnik, V. N. An overview of statistical learning theory.
IEEE transactions on neural networks, (0(5):988-999) 1999.
12

---

Communicating Activations Between Language Model Agents
A. Qualitative Results
Give bullet point biography Joyce K. Reynolds, highlighting her contributiios and uchievements computer scientist.
Question:
Ground-Truth Biography:
Joyce Kathleen Reynolds was American computer scientist who contributed the development of protocols underlying the Internet.
She authored or co-authored many RFCs (Request for Comments) including Telnet, FTP, and POP protocols.
Reynolds worked with Jon Postel develop early functions of the Internet Assigned Numbers Authority and managed the
zone DNS, She received the 20066 Postel Award for her services to the Internet.
LLaMA-3. 2-38: Joyce K. Reynolds is renowned computer scientist and educator who has made significant contributions to the field of computer science and education.
LLaMA-3.1-8B: Joyce Reynolds was born in 1923 England. She studied mathematics at Girton College, Cambridge, graduating with a degree in 1945. Reynolds started National Physical Laboratory (NPL), where she worked on the development of the first commercial computer database, the Database Management System (DBMS).
computing 1945 the
She a pioneer in field artificial intelligence, with focus on human-computer interaction and user experience.
Reynolds has written extensively topics such usability, accessibility, human-centered design.
Reynolds was key member of the team that developed the first DBMS, which was a pioneering achievement in field of database management. Her work laid the foundation for modern database systems.
She is highly respected author, having written several books and articles on computer science and education. Reynolds has received numerous awards and honors for her contributions to the field.
AC: Joyoe K. Reynolds is renowned computer scientist and engineer, best known for her pioncering work in the development of the Internet's fundamental technologics.
was key member of team designed the Internet Protocol (IP) and Transmission Control Protocol (TCP), which form the basis of the modern Internet.
Reynolds' work on IP and TCP enabled the creation of a network that could efficiently route and manage data packets, know it.
paving the way the modern Internet
Reynolds has received numerous awards
honors for her groundbreaking work, including Internet Society's (ISOC)
highest award, the Jonathan B. Postel Service Award.
She continues to be an influential figure in the field of computer science and engineering, inspiring new generations of scientists and engineers with her pioneering achievements.
Figure Example of AC on Biographies dataset.
13

---

Communicating Activations Between Language Model Agents
Question: Christina planning birthday party and needs ,75 gift bags per invited guest, because 1/4 of "attendees don't 't show up. She invited 16 friends. Gift bugs are S2 each. How much will she spend?
LLaMA-3. 2-3B: Since 1/4 of attendces don't show we need to calculate 3/4 of the total invited guests find out how many will attend. 3/4 of 16 is (3/4) 16 12.
LLaMA-3. 1-8B: Christina invited 16 friends, but 1/4 of them won't show To find out how many will attend, we need to calculate 1/4 of 16 and subtract that from 16.
find 1/4 of 16, divide 16 by 16 4.
Now we know 4 guests won't come, we subtract from the total number of guests invited. 16 4 12 guests will attend.
Christina needs 0.75 gift bags per invited guest, so
guests, she needs 0.75 12 = 9 gift bags.
Christina needs.75 gift bags per invited guest. Since she invited 16 friends, but only 12 will attend, multiply .75 by 12. .75 12 =
Each gift bag costs S2, so the total cost will be 9 gift bags $2 per bag $18.
Gift bags are $2 each. Since Christina needs gift bags, we multiply 9 by 2. 9 2 $18.
AC: Since of attendees don't show up, 3/4 of will. find number guests that show we need to multiply the total number of invited guests by 3/4. So, 16 3/4 12 guests will show up.
2. Christina needs 0.75 gift bags per invited guest. Since she invited 16 friends, she needs 16 0.75 12
gift bags.
3. Each gift bag costs $2, Since she needs gift bags, total cost will 12 $24.
Figure 5, Example of AC on GSM8k dataset.
14

---

Communicating Activations Between Language Model Agents
According gate-control theory, which condition tends to close the gate? A) anxiely, B) depression, C) focusing on the pain, D) electrical stimulation
Question:
LLaMA-3. 3-3B:
The question is about the gate- control theory of pain, which suggests that certain stimuli can close gate to the central nervous system, thereby reducing the perception of pain. The options all conditions stimuli that can affect the perception of pain. The correct answer is the one that directly affects the gate, which is the focus
LLaMA-3.. 1-8B: Gate-control thcory suggests that the gate (the gate in the spinal cord that regulates pain) can closed by the activation of certain nerve fibers that carry nonpainful sensations. Anxicty, depression, and focusing the pain all increase pain perception, which would suggest that the gate open, not closed. Electrical stimulation, on other hand, can activate certain nerve fibers that carry nonpainful sensations, which close the gate and reduce pain perception. Therefore, the correct answer is D) electrical stimulation.
pain. When we focus on pain, it tends to close the gate, thereby reducing the perception of pain. The other options not directly related to the gate the same way. Anxiety and depression can affect pain perception, but they
directly close gate. Electrical stimulation can actually activate the gate, not close it. Therefore, the correct answer focusing on the pain.
AC: Step I: Understand the gate-control theory. gate-control theory of pain proposes that the nervous system has a "gate" that controls the flow of pain signals to the brain. The theory suggests that this gate be opened closed by various factors. Step 2: Identify the conditions that tend close the gate. According to the gate-control theory, conditions that tend to close the gate include factors that reduce the transmission of pain signals to the brain. These factors can include distraction, relaxation, and certain types stimulation. Step 3: Analyze the options. Anxiety (A) tends to increase transmission of pain signals to the brain, making it more likely open the gate. Depression (B) can have a complex relationship with pain perception, but not directly related closing the gate. Focusing on the pain (C) tends to increase the transmission of pain signals to the brain, making more likely to open the gate. Electrical stimulation (D) can be used to close the gate by overriding the pain signal with more intense stimulus. The final
<<D>>
Figure 6. Example of AC on MMLU High School Psychology dataset.
15

---

Communicating Activations Between Language Model Agents
Identify the conclusion of the following argument. From historical point view, genetics is young science. The principles that govern inheritance trails described less than 150 years ago. Furthermore, around the turn of the twentieth century, the laws of inheritancee were rediscovered. But even then, the importance of DNA was not really understood until the 1950s.
Question:
A) But even then, the importance DNA was really understood until the 1950s., B) The importance of DNA was not really understood until the 1950s., C) From historical point of view, genetics a young science., D) The principles that govern inheritance truits described less than 150 years ago.
LLaMA-3.24 3-233B The best answer The importance of DNA was not really understood until the 1950s.
LLaMA-3. 1-8B: The final answer <<A>>,
Explanation: The conclusion is the statement that
follows from the premises. In this case, the conclusion is the statement that follows from the two premises. The first premise states that genetics is a young science from a historical point of view, and the second premise states that the importance of DNA until the 1950s. The conclusion is the statement that summarizes the main point of the argument, which even then, importance DNA was not really understood until the 1950s. This is the correct answer because it the statement that follows from the premises and summarizes the main point of the argument.
really understood
AC: Step I: Identify the main claim of the argument. The main claim of the argument is that genetics young science from a historical point view. Step 2: Identify the supporting evidence for the main claim. The supporting evidence includes the fact that the principles that govern inheritance traits were described less than 150 years ago and that the laws of inheritance were rediscovered around the turn the twentieth century. Step 3: Identify the additional information provided in the argument. The additional information that the importance of DNA not really understood until the 1950s. Step 4: Determine the conclusion of the argument. The conclusion is the main claim of the argument, which is that genetics is young science from historical point of view. <<C>>
Figure Example AC on MMLU Formal Logic dataset.
16

---

Communicating Activations Between Language Model Agents
The members of a bird species capable of nesting in treetops or on the ledges of nearby cliffs make their homes only the treetops due the established presence more aggressive bird species the cliff ledges. With respect to habitat, the treetop-nesting birds are utilizing: A) their fundamental niche only, B) their realistic niche only, C) their ecological niche only, D) neither their fundamental niche nor their realistic niche.
Question:
LLaMA-3. 2-33B The treetop-nesting birds are not utilizing their fundamental niche because the cliff ledge not part of their natural environment. However, they are utilizing their realistic niche, which the portion of the fundamental niche that is actually available to the birds.
LLaMA-3. 1..1--..8888 1-8B:: The birds are using their ecological niche, which is the set of environmental conditions and resources that and occupy in given The birds using their fundamental niche, which is sct of all environmental conditions and resources that species can occupy theory. The birds are not using their realistic niche, which is the set of environmental conditions and resources that species with the constraint competition and predation. The birds are only using their ecological niche because they are limited by the presence a more aggressive bird species on the cliff ledges. Therefore, the correct answer is C) their ecological niche only.
species
and occupy given area, but
The troetop-nesting birds are not utilizing their ecological niche, which is the actual area used by the birds. Therefore, the correct answer is B) their realistic niche only.
AC: Step Define key terms: fundamental niche, realistic niche, ecological niche. The fundamental niche refers to the range of environmental conditions in which a species can survive and reproduce without any limitations. The realistic niche, the other hand, the range of conditions that species actually occupies due to biotic and abiotic interactions with other species. The ecological niche broader concept that encompasses both the fundamental and realistic niches. Step 2: Analyze the given scenario. the given scenario, the species capable of nesting in treetops or on cliff ledges makes their homes only in the treetops due to the presence of a more aggressive bird species on the cliff ledges. This that the treetop-nesting birds limited by the presence of another species their potential habitat. Step 3: Determine the type of niche being utilized. Since the peetop-nesting birds are limited by the presence of another species in their potential habitat, they are not utilizing their fundamental niche ecological niche, but are utilizing B) only their realistic niche, which the range of conditions that they actually occupy due to biotic and abiotic interactions with other species.
Figure 8. Example of AC on MMLU College Biology dataset.
17

---

Communicating Activations Between Language Model Agents
Question: A purchaser orally agreed with the seller to buy seller's 's house for $100,000 .000. The purchaser gave the seller written offer and post-dated check for 5% the price. Before the seller signed the offer form or deposited the check, he received an offer of his full asking price of $120, 000 from another party. The seller called the purchaser and called the deal off. The purchaser sued for specific performance on the basis that the oral understanding. along with the written offer and the check, constituted enforceable agreement. Will the purchaser likely prevail? A) Yes, because there was an oral agreement and the check for the down payment evidence of consideration being paid, No, because the offer not secepted-prinn to an acceptance the seller the right sell to anyone, C) Yes, because the buyer and seller had meeting of the minds evidenced by their oral understanding, the check and the written offer, D) No, because specifie performance does not apply residential real estate transaction.
LLaMA-3.3 1.2MA-3.22-BB. 2-3B: The most accurate answer would be: A). Here's the explanation: The purchaser and seller had an oral agreement to buy the house for $100,000, which a sufficient consideration support the enforecability of the agreement. The written offer and the post-dated check for 5% the price ($5,000) also constitute consideration, as they demonstrate the purchaser's commitment to the deal. The seller's receipt the written offer and the post-dated check prior to the offer from the other party indicates that the seller had already accepted the purchaser's offer and was willing to move forward with the deal. Therefore, the purchaser can argue that seller had already accepted the offer and was not free to accept the other offer.
LLaMA-3. 1-8B:
The purchaser and seller had valid contract. The check was consideration, and the oral agreement was enforecable. The seller had no right
off the deal after purchaser's offer was made. The seller could have sold anyone, but he was free call off the deal with the purchaser after the purchaser's offer was made. The purchaser had right enforce the contract. <<A>>
AC: The purchaser will not prevail because the offer was not accepted. The seller received another offer called deal before accepting purchaser's offer. The law requires that of the offer before contract can be formed. The seller's action in calling off the deal before accepting the purchaser's offer terminated the offer, and the purchaser's offer lapsed. <<B>>
acceptance
Figure 9, Example of AC on MMLU Professional Law datasct.
18

---

Communicating Activations Between Language Model Agents
Wait Weight Loss, diet pill brand, promises potential consumers if they one pill once day they will lose weight without having to change their diet or start exercising. However, many people who have used the product have complained that they haven't lost weight, and some have even gained weight. No- Wait Weight Loss has been unsuccessfil in the campaign: A) distribution, B) penetration, C) participation, D) identification
Question:
stage of the advertising
LLaMA-3.2 2- The best answer is B). Explanation: product failed to gain traction with consumers, which is referred the penetration stage of the advertising campaign. The penetration stage is the point time when product introduced to the market and has not yet reached significant level of adoption or usage. In this case, the product is not gaining any traction, and many consumers are not using it, which indicates that it has not yet reached the penetration stage of the advertising campaign.
-.. 1-8B: We need to consider the stages of the advertising campaign how they relate the product's performance. The stages are: 1) Identification: The initial stage... 7) Participation: The stage where consumers become engaged with the brand. We can infer that the product has not been able to convince potential consumers of its benefits, which the Persuasion stage. However, this is not correct answer. The product has been used by many people, and some have even complained about its performance, which suggests that the product has reached the Trial stage. However, this is also not the correct answer. The product has been unsuccessful in the stage of the advertising campaign, which suggests that product has not been able to gain market share and become a dominant player in the market, which the Penetration stage. Therefore, the correct answer is B) Penetration.
AC: The question is asking about the stage of the advertising campaign that No- Wait Weight Loss has been unsuccessful in. The stages of advertising campaign are: identification, awareness, knowledge, liking, persuasion, trial, and adoption. However, the options provided in the question are not the traditional stages the advertising campaign. The options are distribution, penetration, participation, and identification. However, if we map the options to the traditional stages of the advertising campaign, we that: distribution similar to the trial stage, penetration is similar to the adoption stage, participation is similar the awareness stage, and identification similar to the awareness stage as well. Since No- Wait Weight Loss has been unsuccessful in getting people to buy and use their product, it has been unsuccessful the trial stage the advertising campaign, which similar to the distribution stage. Therefore, the correct answer is <<A>>.
Figure 10. Example of AC on MMLU Public Relations dataset.
19

---

Communicating Activations Between Language Model Agents
Table
Reasoning benchmark performance when varying tokens modified during AC. All methods involve communi- cation between LLaMA-3.2 3.2-3B (A) and ILaMA-3.1-88) (B). The functional form is varied between last-token replacement, last-token summation, and summation for all tokens.
Method
Biog.
GSM8k
HS Psych.
Logic
Col. Bio.
Prof. Law
Pub. Rel.
AC (replace)
84.6+0.0
64.0+48
85.0=0.8
47.0+0.1
78.0=0.9
30.0+0.1
74.0=0.1
AC (sum)
79.7+0.0
66.0+4.7
65.0+4.8
42.0+
50.0+5.0
25.0+
37.0+4.8 15.0#3.6
AC (all tokens)
76.0=0.0
62.0=4.9
35.0+4.88
42.0=4.9
61.0+449
26.0+4.4
Table 6: Reasoning benchmark performance when sampling from A with CoT. All methods involve communication between -LaMA-3.2-38 (A) and LLaMA-3.1-8B (B).
Method
Biog.
GSM8k
HS Psych.
Logic
Col. Bio.
Prof. Law
Pub. Rel.
AC
84.6=0.0
64.0#
85.0+0.8
47.0+0.1
78.0+0.9
30.0+0.1
74.0=0.1 63.0+0.1
AC (W)
86.8+0.0
66.0
70.0
35.010.1
79. 49.9
45.0 45.010.1
AC (CoT)
82.1+0.0
66.0+4.0
80.0
26.0+
67.0+44.7
40.0
63.0+4.8
B. Additional Experiments
B.1. Modifying Activations of All Tokens
Recall that AC grafts the last-token layer-k activation of A into B's last-token layer-j activation. But is modifying just the last token activation enough to communicate information from A to B?
Note that after applying masked attention in each of the previous Transformer layers, the last token activation of attends to all tokens before it, hence incorporating information from the entire sequence. Indeed, this must be the case for activation communication recover the gap between the zero-comen communication and skyline setups on both coordination games, which (for Tip Sheets in particular) require information starting at the first few tokens of A's prompt to be communicated.
To verify this empirically, we experiment with summing the activations of all tokens in the sequence rather than just the last (we cannot replace all tokens as this would just replace B's layer-j activation with A's layer k-activation). Results are shown in Table 5.
just
Indeed, applying f to all tokens decreases performance relative to applying f to just the last token. Note that the fact performance generally decreases from replace to f sum, and further with all tokens, is expected. The high performance of AC with f = replace means that the edited last-token activation b retains some meaning in B's activation space; it is less likely for this to be the case when f = sum (at the very least b has norm roughly 2x that of B's original last-token activation), and when doing this for all tokens we'd expect performance to decrease even further as now all activation vectors, not just the last, are out-of-distribution with respect to B's activation space.
B.2. Incorporating Chain-of- Chain-of-Thought Prompting
How does AC perform in relation to NLD in cases where A might incur a long response (possibly with chain-of-thought for intermediate answer computation)? Le., does AC lose out on the benefits of CoT?
First, note that we still reap the benefits of CoT when we sample a completion from B after AC (where B gets all the information encoding A's "beliefs" about the prompt via AC, hence CoT on A's side is not needed). To verify this, we experiment with prompting A with CoT. generating full response, and then passing the layer-k last-token activation of the CoT response to B as part of AC. Results are shown in Table 6.
Indeed, we empirically find our above intuition (in orange) to hold, as there is no significant improvement over vanilla AC when generating from A using COT.
B.3. Learning W In-Distribution
Recall our reasoning about the AC (W) results from Section 4.2: "We hypothesize that the benefits from the learned linear layer are less consistent across datasets because the subset of C4 data used to train W likely contains text more
20

---

Communicating Activations Between Language Model Agents
Table 7: GSM8K performance when learning W in-distribution. All AC variants involve communication between LLaMA-3.2-3B (4) and LLaMA-3.1-8B (B).
AC
AC (W)
AC (Win dist)
64.0+4.8
66.0+4.8
78.0+4.1
Table 8: Reasoning benchmark performance of communication between identical models. Both NLD and AC involve communication between 2 instances of LLaMA-3.1 512-token completions are sampled with temperature 0.7 and debate run for 2 rounds.
Method
Biog.
GSM8k
HS Psych.
Logic
Col. Bio.
Prof. Law
Puh. Rel.
LLaMA-3.1-80
83.9=0.0
60.0+4.9
65. 05.0
42.0+0.1
50.0=0.2
20.0
53.0=0.2
NLD
80.810.0
70.0
85. 15.0
35.0.448
78.014.1
40.014.9
53.015.1
AC
83.7+0.0
60.0+4.9
85.0 33.0
40.0+4.9
74.0=4.4
40.0#449
79.0+4.1
semantically similar to some datasets than others, hence some datasets provide W with out-of-distribttion inputs which reduces performance compared to vanilla AC."
Indeed, we verify this hypothesis by training W on the GSM8k train set (to produce Win in disi) and then evaluating with this task-specific linear layer on the GSM8K test set. Results are shown Table 7.
Indeed, learning W in in-distribution significantly boosts performance, confirming our hypothesis. Unfortunately cannot
run this experiment for the other datasets, there is in-distribution training data available for MMLU (we use all public data for testing).
Hence, this suggests that AC (W) should unilaterally improve over vanilla AC we choose a training set with good coverage across many tasks and distributions, such that there are sentences semantically similar to prompts across the span of downstream task datasets.
B.4. Activation Space Similarity x AC Performance Gain
We conduct the following experiment: for each of the six pairs of models A, in the above experiment (see Table 4), we compute the increase Biographies performance with AC relative to the average individual performance of and B. We also compute the matrix analogue of the squared cosine similarity between the models' activation spaces,
I
where X is the matrix of A's activations 3072 sentences from C4 (the same dataset used to train W), Y is the corresponding matrix for B, and ||-|| denotes the Frobenius norm. This yields the plot in Figure 11.
There is clear positive correlation between the similarity of the activation distributions and the AC performance gain, as expected; the more aligned and B's activation spaces are, the more semantically meaningful and useful the embedding we graft from to becomes.
B.5. Communicating Activations Between Identical Models
Note that AC as described in Section 3.1 only supports communication between distinct models. We can extend AC to work for communication between identical models as follows: let and be instances of the same model. We can sample a completion from A with temperature and graft the last-token layer-k activation of the completion into B at layer j as part of the AC procedure. This still saves a substantial amount of compute over NLD between model instances, showing our technique can apply to this setting. Table 8 shows the results of this experiment.
Indeed, while communication between multiple model instances doesn't it always show improvement over the single model itself (a well-known result from (Du et al., 2023)), AC matches/outperformss NLD on five of the seven datasets.
The intuition behind debate between multiple identical model instances that sampling multiple completions (with temperature) from the same model yields diverse reasoning paths that can recombined into stronger final answer. The
21

---

Communicating Activations Between Language Model Agents
performance gain vs. "cosine similarity" (normalized squared Frobenius A,B's activation spaces
AC performance gain on Biographies (AC pert. avvii, B) pert.)
0.175
qwent qwen
0.150
0.125
gwen1.50 Se.n1.5b.lllaaa. 3b.llamm3b
0.100
Hameab,llamaBe
llama3ogemma22b
gwwn1:bb.gemeeeee Sb.gemma25
gemmas2.gemma9t
-0.025
0.35
0.40
0.45
0.50
0.55
0.60
0.65
|I (Í
Figure AC performance gain over average A/B individual performance Biographies, function matrix "cosine similarity" between A and B's activation spaces.
Table 9: Reasoning benchmark performance of AC and NLD with varying number of rounds. All methods involve communication between LLaMA-3.2-3B (A) and LLaMA-3.1-81 (B).
Method
Biog.
GSM8K
HS Psych.
Logic
Col. Bio.
Prof. Law
Pub. Rel.
NLD (1 round)
83.6+0.0
72.0+4.5
65.0
40.0+4.9
68.0+4.6
30.0+4.6
63.0+:..8
NLD (2 rounds)
80.2+0.1
75.0+4.3
83.0 83.0
37. 37.0+0.11
71.0+0.1
30.0+0.1
63.0+0.7
NLD (3 rounds)
80.1
79.0=4.1
70.0
45.0+5.0
63.0+4.88
40.0#:49
74.0+t4.4
NLD (4 rounds)
78.0#0.0
79.0=4.1
AC
84.6++0.0
64.0#
85.0+0.8
47.0 0..0
78.0+0.9
30.0+0.1
74.0+0.1
*Runs required too much compute
above experiment shows that the same intuition holds for AC-we are sampling multiple times from the same model, but passing responses between agents via AC rather than as NL messages.
B.6. Additional Rounds of Natural Language Debate
In Section 4.2 we fix NLD to agents and rounds, however we find in additional experiments that AC outperforms NLD even with additional rounds, highlighting the superiority and robustness of activations as an alternative "language" for inter-LM communication. Results are shown in Table 9; we see that for 5 of the reasoning benchmarks, AC beats NLD even with 3-4 rounds while using substantially less compute.
B.7. Full MMLU Benchmark Results
Table 10 below displays complete results of both AC and NLD on the full MMLU benchmark. Notably, AC matches/outperformms NLD on 48/57 datasets, with substantially less compute used, indicating its superiority and robustness as an alternative "language" for inter-LLM communication.
22

---

Communicating Activations Between Language Model Agents
Table 10: Comparison of NLD vs. AC the full MMLU benchmark (Hendrycks al., 2021).
Dataset
NLD
AC
Conceptual Physics
60.0 4.9
68.0 4.6
High School Chemistry
50.0 + 5.0
37.0 + 4.8
Security Studies
6.9
60.0
Jurisprudence
8.
3.6
Logical Fallacies
6..0
7.5
College Computer Science
4.00
4.00
International Law
5.00
59.0
Miscellaneous
3.0
95.0 2.2
Marketing
4.6
3.6
Elementary Mathematics
4.3
58.0
Machine Learning
4.9
4.9
High School Macrocconomics
5.0
4.3
High School US History
4.00
71.0 1 4.6
Human Aging
5.0
72.0 1 4.5
Astronomy
7.1
80.0 + 4.0
Computer Security
5..0
75.0 4.3
High School Statistics
5.00
42.0 4.9
Professional Medicine
79.0
6.80
Electrical Engineering
5.90
60.0 - 4.9
High School Computer Science
4.8
70.0 4.6
College Physics
5.00
28.0 4.5
Management
4.1
75.0 4.3
Moral Scenarios
4.9
4.9
World Religions
4.9
72.0 4.5
Virology
4.0
50.0 5.0
Philosophy
4.70
70.0 - 4.6
Abstract Algehra
5.0
28.0 + 4.5
High School Government and Politics
8.0
61.0 4.9
High School Biology
6..0
4.8
College Mathematics
6.80
24
Global Facts
3..0
37.0 4.8
High School World History
7.00
74.0 1.01: 4.4
High School European History
6.00
71.0 1 4.6
College Medicine
65.0
53.0
High School Geography
6.70
79.0
Anatomy
4.40
7..0
Human Sexuality
4.3
4.3
Medical Geneties
4.1
3.8
Professional Accounting
4..0
48.0 1 4.5
US Foreign Policy
89.0
90.0
Business Ethics
43.0
44.0 5.0
College Chemistry
41.0
47.0 5.0
High School Physics
4.0
47.0 - 5.0
Professional Psychology
54.0
55.0
Sociology
68.0
68.0 + 4.6
High School Microeconomics
95.0
95.0 2.2
High School Mathematics
55.0 5.0
Prchistory
7.30
60.0 4.9
Nutrition
4.50
70.0 4.6
Clinical Knowledge
4.30
65.0 4.8
Moral Disputes
4.80
60.0 -: 4.9
Econometrics
4..0
40.0 + 4.9
High School Psychology
83.0
85.0 0.8
Formal Logic
0.1
47.0 - 0.1
College Biology
0.1
7..0
Professional Law
3..0
30.0
Public Relations
63.0 0.7
74.0 0.1
Average
6.07
62.7 1 2.2
23


Reviews:
[{"review_id": "HdBOl84cPn", "reviewer": "Reviewer_VKxi", "summary": "The paper considers this fundamental question \"as LLMs are increasingly capable of handling larger, more complex tasks (sometimes with “super-human” ability), might they communicate more effectively in representations of higher dimension than natural language?\". It proposed a simple technique where LMs communicate via activations.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback, and address their comments below:\n\n1. > The paper considers rather simple datasets (2 multiplayer and 7 reasoning tasks) and two models (llama 3b and 8b).\n\nIn Appendix B.6, we display results on **the entire MMLU benchmark (57 datasets)**, spanning various domains and difficulty levels. We find that **AC matches/outperforms NLD on 48/57 datasets**, demonstrating our approach's value.\n\nWhile our initial experiments were only with LLaMA models, note: (1) These LMs are among *SOTA open-source models*, meriting our focus; and (2) we extensively vary both the LLaMA suite (LLaMA-2, 3, 3.1, 3.2) and parameter count (1-70B) as shown in Figure 3 of the paper. This is already a broad coverage of models/model sizes.\n\n**However, we share additional results using models from the Qwen-2.5 and Gemma-2 families below.** Each cell contains two results: Biographies score / GSM8k score.\n\n| Model Pair ($A,B$)                                           | $A$                              | $B$                              | NLD                              | AC                                        |\n| ------------------------------------------------ | -------------------------------- | -------------------------------- | -------------------------------- | ----------------------------------------- |\n| LLaMA-3.2-3B, LLaMA-3.1-8B  | $79.4\\pm0.0/58.0\\pm4.9$          | $83.9\\pm0.0/60.0\\pm4.9$ | $80.2\\pm0.1/\\mathbf{75.0\\pm4.3}$ | $\\mathbf{84.6\\pm0.0}/64.0\\pm4.8$                   |\n| Qwen-2.5-1.5B, Qwen-2.5-3B  | $59.4\\pm0.9/20.0\\pm0.9$          | $85.5\\pm1.1/35.0\\pm1.1$          | $63.2\\pm1.1/65.0\\pm1.1$          | $\\mathbf{89.6\\pm1.0}/\\mathbf{70.0\\pm1.0}$ |\n| Gemma-2-2B Gemma-2-9B      | $83.0\\pm1.1/45.0\\pm1.1$          | $\\mathbf{94.6\\pm0.9}/80.0\\pm0.9$ | $70.3\\pm1.0/70.0\\pm1.0$          | $88.1\\pm0.7/\\mathbf{90.0\\pm0.7}$          |\n| Qwen-2.5-1.5B, LLaMA-3.2-3B | $59.4\\pm0.9/20.0\\pm0.9$          | $79.4\\pm0.0/58.0\\pm4.9$          | $75.4\\pm1.0/75.0\\pm1.0$          | $\\mathbf{79.5\\pm1.0}/\\mathbf{75.0\\pm1.0}$          |\n| LLaMA-3.2-3B, Gemma-2-2B    | $79.4\\pm0.0/58.0\\pm4.9$ | $83.0\\pm1.1/45.0\\pm1.1$          | $62.5\\pm1.1/55.0\\pm1.1$          | $\\mathbf{84.0\\pm0.1}/\\mathbf{60.0\\pm1.1}$          |\n| Qwen-2.5-1.5B, Gemma-2-2B   | $59.4\\pm0.9/20.0\\pm0.9$ | $\\mathbf{83.0\\pm1.1}/45.0\\pm1.1$            | $49.3\\pm1.1/50.0\\pm1.1$          | $73.0\\pm1.1/\\mathbf{55.0\\pm1.1}$          |\n\nNote the following:\n- **AC beats NLD across the board**, and **beats both individual models for 4/5 of the 6 model pairs** on Biographies/GSM8k respectively----demonstrating the efficacy of AC irrespective of model architecture, size, tokenizer, and training data\n- These results are obtained *without* training $\\mathbf{W}$, meaning we do *not* need to train a projection layer between activation spaces to attain SOTA results, even for extremely distinct models! (We hypothesize this is because we are only replacing $B$'s *last*-token activation, hence $B$ can learn from $A$ without an extreme alteration to its activation distribution)\n\n2. > The experimental evaluation can be improved with better baselines than single model and NL debate.\n\nWe limit our focus to task-agnostic methods; existing model composition/grafting methods require substantial task-specific parameters/data & much more compute in the form of LM finetuning, layer or router training, etc., hence we do not compare to these.\n\nRegarding multiagent debate, the NLD setup that we evaluate against is the predominant method of NL communication. This is quite a strong/flexible NL approach, involving CoT and allowing varied numbers of agents/rounds. (In fact, as shown in Appendix B.5, we find **AC outperforms NLD even with 3-4 rounds of debate on 5 of the 7 reasoning datasets**.)\n\n3. > Why was k = j = 26 selected?\n\nSee Section 3.3, lines 201-222. This reasoning is precisely why we choose a layer around halfway through the LM; indeed, Hernandez24 find that by around the halfway point of an LM's computation, it has developed \"enriched entity representations\" of the input that would be quite useful for communication compared to the next-token representations of later layers.\n\nWe validate this empirically; Figure 2 shows 2D contour plots of accuracy over different $k,j$ values. For computational reasons we only do this hyperparameter sweep on the Countries and Tip Sheets datasets and simply cross-apply the optimal values here, $k=j=26$, to the reasoning benchmarks and find that the values seem to generalize well across datasets, which is quite interesting in its own right.\n\n5. > The explanation in line 25-255 is not convincing about why the experimental comparison with Pham24 was avoided. Could you please elaborate further?\n\n**Please see point #4 in the reply to Reviewer gYwD below.**\n\n**We hope that with the provided responses & additional strong results, you consider raising your score to support clear acceptance. We are happy to address any additional questions.**", "claims_and_evidence": "The primary claim in the paper is that LMs can communicate using activations. An LM B’s computation is paused at an intermediate layer, then its current activation is combined with another LM A’s intermediate activation via some function f, and then f’s output is passed into the next layer of B and the forward pass is continued till decoding is complete.  The experimental evaluation is done over 7 reasoning benchmarks and 2 multiplayer games. The activation communication protocol exhibits up to 27.0 improvement over natural language communication across these datasets using <1/4 the compute.", "methods_and_evaluation_criteria": "The presented method is rather simplistic but the experimental evaluation demonstrates its value.", "theoretical_claims": "There are no significant theoretical claims in the paper.", "experimental_designs_or_analyses": "The paper considers rather simple datasets (2 multiplayer and 7 reasoning tasks) and two models (llama 3b and 8b).", "relation_to_broader_scientific_literature": "The paper is an interesting contribution to multi-LLM inference.", "essential_references_not_discussed": "Yes, references are discussed. The paper demonstrates good familiarity with literature.", "other_strengths_and_weaknesses": "A rather simple idea of using activations is shown to have good empirical value on the considered experiments. \n\nThe weights learned for projecting activation from one model to another is task-independent. \n\nThe experimental evaluation can be improved with better baselines than single model and NL debate.", "other_comments_or_suggestions": "It would be useful to expand the empirical evaluation to more and diverse datasets.", "questions_for_authors": "Why was k = j = 26 selected as the layer? \n\nThe explanation in line 25-255 is not convincing about why the experimental comparison with Pham24 was avoided.  Could you please elaborate further?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5KtMmE2U2x", "reviewer": "Reviewer_gYwD", "summary": "The paper proposes an alternative method for communication between language models (LMs) that does not rely on natural language. Instead, the authors introduce a technique for LMs to communicate with activations. More specifically, intermediate activations from one model are injected into another's computation at an intermediate layer, allowing models to exchange information in a manner that the authors argue is more efficient than text. Through experiments, this method is shown to improve reasoning performance in multi-agent tasks, outperforming natural language-based communication across various datasets while using less compute as compared to natural language debate. The paper validates the approach through experiments on multi-player coordination games and reasoning benchmarks, demonstrating the method's robustness across different model sizes.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback, and address their comments below:\n\n1. > Lack of comparison with other similar approaches such as model merging\n\nTo adequately scope our paper, we chose to limit our focus to task-agnostic methods. Model composition/merging methods are extensively discussed in Sections 2 and 3.2, but they require a substantial amount of additional task-specific parameters and data, hence we do not compare to these. Furthermore these methods require much more compute in the form of LM finetuning, layer or router training, etc.; our approach is far more compute efficient.\n\n2. > The approach needs access to the models’ parameters which is not feasible for state-of-the-art close-source LMs.\n\nExploring API-only approaches is highly limiting (see AC's meta review of [1] at ICLR last year). Furthermore, recent releases of powerful open-source models [2] merit the development of embedding-based techniques.\n\n3. > Need for model-model pair training of W to map representations from one LM to another LM if architecture is different.\n\nNote that learning $\\mathbf{W}$...\n- needs to happen **exactly once** for each *model pair*\n- introduces zero additional task-specific parameters and data by virtue of requiring only general text, e.g. sequences from $A$ and/or $B$’s pretraining data mixes\n- is quite sample-efficient: as mentioned in the paper, we use $3072$ sentences to train $W\\in \\mathbb{R}^{4096\\times 3072}$, since linear regression with $d$-dimensional input has a sample complexity of $O(d)$ [3]\n\nFurthermore, we find empirically that *even when models have different architectures, tokenization schemes, or training distributions, we do* **not** *need to train a mapping matrix $\\mathbf{W}$ to attain SOTA results.*  **Please see point #1 in the reply to Reviewer VKxi above.**\n\n4. > The work by Phametal.,2024, though different, should also be a method to compare as a baseline to better understand the effect of using intermediate activations versus embeddings in multi-agent frameworks.\n\nPham24 propose communicating the *input (tokenizer) embeddings* between models, meaning the two models **must have the same tokenizer and embedding table** to even run their approach. This severely limits the applicability of their method; in particular, given that all our experiments use model pairs (e.g., LLaMA-3.2-3B and LLaMA-3.1-8B) with distinct tokenizers and/or embedding layers, we unfortunately cannot compare against Pham24. Also, note Pham24's approach still faces substantial information loss relative to the model activations and, more importantly, **does not save compute**, as the number of embeddings passed between models is *the same as the number of tokens passed between models in natural language communication*.\n\n5. > There are also highly similar approaches that also rely on using intermediate activations from a language model to inject into intermediate layers of the same language model to improve reasoning (e.g. https://arxiv.org/abs/2412.06769 and https://arxiv.org/abs/2502.05171). These approaches do not need another training stage to train a separate W to map activations from one LM to another. More discussion about these similar works would help the reader better understand the difference.\n\nThank you for raising this point; we have added discussion of these approaches to our paper. In summary, such latent reasoning approaches involve spending extra compute by doing \"CoT in activation space,\" e.g. by grafting LM activations into other layers/later forward passes through the same model; our approach can be viewed as doing exactly the same thing, but instead \"outsourcing\" the CoT to another model (and thus reaping benefits from greater diversity of thoughts/reasoning paths from distinct models).\n\nAlso, note that as shown above, we find that even when models have different architectures, tokenization schemes, or training distributions, we do *not* need to train a mapping matrix $\\mathbf{W}$ to attain SOTA results.\n\n**We hope that with the provided responses & additional strong results, you consider raising your score to support clear acceptance. Please let us know if there are any additional questions or concerns; we'd be happy to address them.**\n\n[1] Chau Pham, Boyi Liu, Yingxiang Yang, Zhengyu Chen, Tianyi Liu, Jianbo Yuan, Bryan A. Plummer, Zhaoran Wang, and Hongxia Yang. Let models speak ciphers: Multiagent debate through embeddings, 2024.\n\n[2] Abhimanyu Dubey et al. The llama 3 herd of models, 2024.\n\n[3] Vapnik, V. N. An overview of statistical learning theory, 1999.", "claims_and_evidence": "Yes, no issue.", "methods_and_evaluation_criteria": "Yes, no issue.", "theoretical_claims": "NA", "experimental_designs_or_analyses": "Yes, no issue.", "relation_to_broader_scientific_literature": "The proposed approach contributes to the field of multi-LM agent framework which is of interest to a broad ML community.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "Strengths:\nInnovation is timely with the popularity of multi-agent framework\nExperimental results show better computational efficiency compared to natural language baseline.\nWeaknesses:\nLack of comparison with other similar approaches such as model merging and single model with latent reasoning. These approaches, though not framed as multi-agent approaches, have close similarities in using activations from intermediate layers of an LM injecting into other intermediate layers of the same LM.\nThe approach needs access to the models’ parameters which is not feasible for state-of-the-art close-source LMs.\nNeed for model-model pair training of W to map representations from one LM to another LM if architecture is different.", "other_comments_or_suggestions": "The work by Phametal.,2024, though different, should also be a method to compare as a baseline to better understand the effect of using intermediate activations versus embeddings in multi-agent frameworks. \n\nThere are also highly similar approaches that also rely on using intermediate activations from a language model to inject into intermediate layers of the same language model to improve reasoning (e.g. https://arxiv.org/abs/2412.06769 and https://arxiv.org/abs/2502.05171). These approaches do not need another training stage to train a separate W to map activations from one LM to another. More discussion about these similar works would help the reader better understand the difference.", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "NA"}}, {"review_id": "TexVD8kDQr", "reviewer": "Reviewer_EeXt", "summary": "The paper proposes a novel method for inter-language model (LM) communication by directly exchanging model activations instead of using natural language.\nTested on two synthetic datasets (a coordination game and an investment decision task) and several reasoning benchmarks (GSM8k, MMLU subsets, Biographies), the method achieves up to 27% improvement over natural language communication while using less than one-fourth the compute. The paper compares activation communication with natural language debate, a common multi-agent method, showing superior efficiency and generalization across model sizes.\n\n## update after rebuttal\n\nI have considered the additional set of experiments and the preliminary analysis and I don't see any reasons why this paper shouldn't be accepted. I am increasing my score from 3 to 4", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback, and address their comments below:\n\n1. > All tested models come from the LLaMA family...\n\n**Please see point #1 in the reply to Reviewer VKxi above.** In summary, we test AC using models **across the LLaMA, Qwen, and Gemma families**, and find that **AC beats NLD across the board, and beats both individual models for 4/5 of the 6 model pairs** on Biographies/GSM8k respectively----demonstrating the efficacy of AC irrespective of model architecture, size, tokenizer, and training data.\n\n2. > The paper lacks an analysis of why the method work, e.g. inspecting the geometric properties...\n\nFirst, please see Section 3.3 for a theoretical grounding of this work.\n\nSecond, we conduct the following experiment: for each of the 6 pairs of models $A,B$ in the above experiment (see the table from response to Reviewer VKxi), we compute the increase in Biographies performance with AC relative to the average individual performance of $A$ and $B$. We also compute the matrix analog of squared cosine similarity between the models' activation spaces, $||Y^T X||_F^2/(||X||_F^2||Y||_F^2)$ where $X$ is the matrix of $A$'s activations on 3072 sentences from C4 (same dataset used to train $\\mathbf{W}$), $Y$ is the same for $B$, and $||\\cdot||_F$ is the Frobenius norm. This gives us the following plot (please click link if image not displayed):\n\nhttps://i.ibb.co/3y6XF7Z7/model-comparison-plot.png\n\nThere is a clear positive correlation between similarity of the activation distributions and AC performance gain, as expected; the more aligned $A/B$'s activation spaces, the more semantically meaningful/useful the embedding we graft from $A$ to $B$.\n\n3. > Though not fundamental to position the paper, broader comparisons...\n\nThanks for sharing this. We've included a discussion of this and similar papers (e.g., Kornblith19, Similarity of Neural Network Representations Revisited) in the paper.\n\n4. > A link that I didn't see in the paper...\n\nWe thank the reviewer for bringing this to our attention. The \"tuned lens\" and \"logit lens\" [1] are related to the theoretical intuition behind this approach that we share in Section 3.3; we have added a discussion of these papers to both Section 3.3 and related works.\n\n5. > In the appendix I found...\n\nThanks for pointing this out, we have ensured all experiments in the appendix are either moved to or discussed in the main body of the paper.\n\n6. > NLD debate is found to improve with increased number of agents...\n\nFirst, we want to distinguish between **\"agents\"** (distinct model *instances*) and **\"models\"** (distinct *LMs*).\n\nWith debate, more agents can help because we are sampling different outputs from each agent (model instance), and this can yield diverse reasoning paths that are recombined to produce stronger final outputs.\n\nThis isn't true with AC, as one activation grafting step from $A$ to $B$ inherently communicates to $B$ all of $A$'s knowledge/beliefs about the prompt it was given. **We argue this is actually a benefit of AC over NLD, as we don't require increasing token budgets to extract more and more information out of the LMs.**\n\nA similar argument can be made for the number of *rounds* in NLD. Indeed, as shown in Appendix B.5, for 5 of the 7 reasoning benchmarks, **AC beats NLD even with 3-4 rounds while using substantially less compute**, highlighting the superiority and robustness of activations as an alternative “language” for inter-LM communication.\n\nAC could theoretically scale to more than 2 *models*. While letting $f$ be the average function would work out of the box, we saw that direct replacement is much more effective. So, for instance, consider a setup (using the notation from Section 3.1 of our paper) where for any $i, j, k, m$ with $j < k$, we:\n\n1. run a partial forward pass $B_{\\leq j}(x_B)$ to get last-token activation $\\mathbf{b}_j$;\n2. run a partial forward pass $A_{\\leq i}(x_A)$ to get $\\mathbf{a}_i$;\n3. replace $\\mathbf{b}_j \\leftarrow f(\\mathbf{a}_i, \\mathbf{b}_j)$;\n4. continue $B$'s forward pass till layer $k$ to get last-token activation $\\mathbf{b}_k$;\n5. run a partial forward pass $C_{\\leq m}(x_C)$ to get $\\mathbf{c}_m$;\n6. replace $\\mathbf{c}_m \\leftarrow f(\\mathbf{b}_k, \\mathbf{c}_m)$;\n7. then continue $C$'s forward pass till decoding is complete.\n\nThis 3-model setup can extend to an arbitrary number of models. We leave this extension of our approach to future work.\n\n7. > The paper would be a clear accept with tests on a broader range of model families to confirm that activation communication is not specific to Llama's architecture or training paradigms, and additional insights on the effectiveness of the proposed approach.\n\n**We hope that with the provided responses & additional strong results, you consider raising your score to support clear acceptance. Please let us know if there are any additional questions or concerns; we'd be happy to address them.**\n\n[1] nostalgebraist. Interpreting gpt: the logit lens, 2020.", "claims_and_evidence": "The claims in the paper are partially supported by the experiments, which demonstrate that activation communication improves performance while reducing compute costs compared to natural language communication. The results across two synthetic datasets and multiple reasoning benchmarks provide some evidence for the claimed efficiency and effectiveness of the approach. The ablation studies on different activation combination functions further strengthen the argument that direct activation exchange can enhance inter-model communication. The application to Llama-only models make it for a partial support of the general claim that their methods can improve LLM performance", "methods_and_evaluation_criteria": "The proposed methods make sense for studying inter-model communication. The use of synthetic datasets is a notable strength. The countries and coordination game allow the authors to isolate the impact of communication without confounding factors. This controlled setup provides clearer insights into how activation-based communication influences performance.\n\nThe reasoning benchmarks used are fairly standard in the field, making the evaluation relevant and comparable to prior work.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experimental design is generally sound, with well-structured comparisons between activation communication and natural language-based methods. The use of both synthetic tasks and reasoning benchmarks helps validate the approach across different settings. However, a notable weakness is that all tested models come from the LLaMA family, meaning the study does not evaluate communication between models with different architectures, tokenization schemes, or training distributions. This limits conclusions about the method's general applicability. The comparison with single agent is also a (mild) concern that I have (see more below in the additional questions section).\n\nAdditionally, the paper lacks an analysis of why the method work, e.g. inspecting the geometric properties of the activations being communicated. As an example, understanding whether activation similarity between agents correlates with downstream task performance could provide insights into when and why activation-based communication is effective. Metrics like cosine similarity or rank correlation between activations before and after communication could help determine if performance gains are linked to latent space alignment between models. This type of analysis would strengthen the theoretical grounding of the method.\n\nThe paper would be a clear accept with tests on a broader range of model families to confirm that activation communication is not specific to Llama's architecture or training paradigms, and additional insights on the effectiveness of the proposed approach.", "relation_to_broader_scientific_literature": "The paper builds on prior work in multi-agent LLM communication by replacing costly natural language exchanges with direct activation transfer, reducing compute while improving performance. It extends activation engineering research by using intermediate activations as a communication channel rather than for single-model control. The method is related to model grafting but removes the need for learned routing or fine-tuning, making it more generalizable. Though not fundamental to position the paper, broader comparisons with cross-model embedding alignment techniques would strengthen the work’s positioning, e.g. Relative representations enable zero-shot latent space communication (Moschella et al.)", "essential_references_not_discussed": "Most essential references are properly discussed.\nA link that I didn't see in the paper is the one to papers like Eliciting Latent Predictions from Transformers with the Tuned Lens (Belrose et al), which, maybe with a different end goal, use a similar method. Discussing should be added in the related work section", "other_strengths_and_weaknesses": "The paper is clearly written and easy to follow, with well-structured explanations of the motivation, methodology, and experimental results. The key ideas are presented intuitively, making it accessible to both researchers familiar with multi-agent communication and those new to the topic. The figures and tables effectively support the narrative, particularly in illustrating the activation communication process and performance improvements.", "other_comments_or_suggestions": "In the appendix I found several interesting insights that should be either moved or at least discuss and referenced in the main body of the paper.\nFor instance, \n- the analysis on the interplay between CoT and AC: I would add it or at least discuss it in the main body of the paper. For me, it could be swapped with the compute cost analysis \n- the fact that AC over multiple instances of the same model, like NLD, doesn't always outperform the single model setup\n- the fact that AC is superior to NLD even with multiple rounds.\n\nFor instance, at the end of page 7, when mentioning the in-distribution training of matrix W, there's no reference to the experiment i in section B.3.\n\nThis additional information in the main body of the paper would make it even clearer and more informative", "questions_for_authors": "1. NLD debate is found to improve with increased number of agents (Figure 10 in the paper), do you think your method can compare with that (stronger) setup? I am thinking of instances of your method like f = average or a regularized W matrix learned to transfer across multiple models", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r2k6oqcBjv", "reviewer": "Reviewer_9iZt", "summary": "This paper studies the multi-agent communication problem in the LLM scenario. Specifically, it proposes using hidden representations instead of natural language. Experiments on several multi-agent collaboration datasets demonstrate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback, and address their comments below:\n\n1. > The theoretical depth of this paper can be further enhanced. For example, the result shows that the direct replacement strategy is the most efficient way. Basically, it means that we should discard all previous representations and directly use the projected ones. Do we have any assumptions behind this phenomenon? (e.g., what is the relationship between the two models such that this conclusion holds?)\n\nFirst, note that the direct replacement strategy only replaces $B$'s layer-$j$ activation of *the final token* with $A$'s last-token layer-$k$ activation. The embeddings at all other token positions remain the same. Hence, we are not actually discarding all previous representations: after applying masked attention in each of the previous Transformer layers, the last token activation of $A$ attends to all tokens before it, hence incorporating information from the entire sequence; and the previous token activations of $B$ are retained and this incorporate all of $B$'s \"thoughts\" regarding the full sequence.\n\nPlease refer to Appendix B.1 for additional discussion and comparison between the replacement, sum, and mean strategies for activation grafting.\n\n2. > What is the connection and difference of your model against the encoder-decoder model?\n\nThis is an interesting point; the direct replacement activation grafting approach could be seen as running an encoder-decoder Transformer, where the first $k$ layers of $A$ are the encoder, $B$ is the decoder, and we have a \"special cross-attention\" where----from layer $j$ onwards----all tokens in $B$ attend to only the last-token embedding outputted by the first $k$ layers of $A$. However $A$ and $B$ are language models used out-of-the-box, with no joint training, no requirement of shared tokenizer/aligned embedding space/similar training distributions/etc.\n\n3. > How to choose the hyperparameters of j,k\n\n**Please see point #3 in the reply to Reviewer VKxi above.**\n\n4. > What is the effect of the scale and distribution of the aligning data for training the transformation matrix?\n\nWe propose training $\\mathbf{W}$ to minimize MSE loss over a dataset of $N$ sentences\n$\\frac{1}{N}\\sum_{i=1}^N \\left\\|\\mathbf{z}^{(i)} - \\mathbf{W}\\mathbf{y}^{(i)}\\right\\|_2^2$ where each $(\\mathbf{y}^{(i)},\\mathbf{z}^{(i)})$ pair denotes the final-token layer-$26$ activations of $A$ and $B$ at layers $k$ and $j$ respectively given the same sentence as input. \n\nHence, the \"aligning data distributions\" are quite literally $A$ and $B$'s activation distributions; we do not scale or otherwise modify the activation vectors in any way before training.\n\n5. > Are you trying to train a unique matrix for each j,k pair? Since you are using the MSE loss, I wonder if such a training objective might work well given that the capacity of a matrix is not very big.\n\nYes we are, see point #4 above. However, note that for each model pair, we only need a single $(j,k)$ pair----indeed, we attain SOTA results with $k=j=26$ fixed.\n\nAlso, this training objective is quite standard in related literature. For instance, the \"model stitching\" paper [1] learns a 1x1 convolution between activations of two models at a specific layer, which is exactly equivalent to our method of training $\\mathbf{W}$; [2] also learns a linear projection between Transformer layers which is shown to be quite expressive.\n\nIndeed, we find our approach yields quite strong results; please see Appendix B.3 for additional discussion.\n\n6. > Have you tried directly aligning the representations from two models and adding one more layer on top of that?\n\nWe kindly ask for clarification on this question. If by \"directly aligning the representations from two models and adding one more layer on top of that\" you mean learning a linear layer that projects activations from $A$'s activation space to $B$'s, that is exactly what we do in the AC ($\\mathbf{W}$) method.\n\n**We hope that with the provided responses & additional strong results, you consider raising your score to support clear acceptance. Please let us know if there are any additional questions or concerns; we'd be happy to address them.**\n\n[1] Yamini Bansal, Preetum Nakkiran, Boaz Barak. Revisiting Model Stitching to Compare Neural Representations, 2021.\n\n[2] Alexander Yom Din et al. Jump to Conclusions: Short-Cutting Transformers With Linear Transformations, 2024.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "N/A no theoretical claims", "experimental_designs_or_analyses": "Yes. The experiments on the multi-agent database scenario make sense and can be used to demonstrate the effectiveness of the proposed method.", "relation_to_broader_scientific_literature": "The multi-agent communication problem has been a long-term topic in the AI community. Recently, due to the development of LLMs, people have started using natural language as a communication medium. As discussed in the paper, previous work already discussed that the natural language might not be the most efficient way and this paper follows the trend to use hidden representation but with a more careful design (e.g., train a transformation matrix)", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strength:\nThe paper is well-motivated and clearly written.\n\nLimitation:\n1. The theoretical depth of this paper can be further enhanced. For example, the result shows that the direct replacement strategy is the most efficient way. Basically, it means that we should discard all previous representations and directly use the projected ones. Do we have any assumptions behind this phenomenon? (e.g., what is the relationship between the two models such that this conclusion holds?)\n2. Many design choices need further clarification. For details, please see the question for authors section.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. What is the connection and difference of your model against the encoder-decoder model?\n2. How to choose the hyperparameters of j,k\n3. What is the effect of the scale and distribution of the aligning data for training the transformation matrix?\n4. Are you trying to train a unique matrix for each j,k pair? Since you are using the MSE loss, I wonder if such a training objective might work well given that the capacity of a matrix is not very big.\n5. Have you tried directly aligning the representations from two models and adding one more layer on top of that?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The core innovation of this work is to question whether natural language is the best communication channel for LM to LM information sharing. The implementation of this idea is to have activations from an intermediate stage of the first model be combined (via projection) with those of the second model which then continues computation. This softer channel is than evaluated on multiplay and reasoning settings.  The insight also begs the question as to what defines a “model”, is this two models or just a single one with a specific control flow. This therefore warrants evaluation via a single model.  These types of changes also beg questions about specific design choices (layer, projection learning, etc) which are not required when using NL.

Many of these concerns (benchmarks, models, and design) are resolved in the author response.
Decision: Accept (poster)