Cache-to-Cache: Direct Semantic Communication Between Large Language
  Models
Authors: name: Tianyu Fu, name: Zihan Min, name: Hanling Zhang, name: Jichao Yan, name: Guohao Dai, name: Wanli Ouyang, name: Yu Wang
Organization: Tsinghua-NICS-EFC

Multi-LLM systems harness the complementary strengths of diverse Large
Language Models, achieving performance and efficiency gains unattainable by a
single model. In existing designs, LLMs communicate through text, forcing
internal representations to be transformed into output token sequences. This
process both loses rich semantic information and incurs token-by-token
generation latency. Motivated by these limitations, we ask: Can LLMs
communicate beyond text? Oracle experiments show that enriching the KV-Cache
semantics can improve response quality without increasing cache size,
supporting KV-Cache as an effective medium for inter-model communication. Thus,
we propose Cache-to-Cache (C2C), a new paradigm for direct semantic
communication between LLMs. C2C uses a neural network to project and fuse the
source model's KV-cache with that of the target model to enable direct semantic
transfer. A learnable gating mechanism selects the target layers that benefit
from cache communication. Compared with text communication, C2C utilizes the
deep, specialized semantics from both models, while avoiding explicit
intermediate text generation. Experiments show that C2C achieves 8.5-10.5%
higher average accuracy than individual models. It further outperforms the text
communication paradigm by approximately 3.0-5.0%, while delivering an average
2.0x speedup in latency. Our code is available at
https://github.com/thu-nics/C2C.