Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism
Authors: Junfei Zhou
Penglin Dai
Quanmin Wei
Bingyi Liu
Xiao Wu
Jianping Wang
Venue: NeurIPS 2025 poster ()
Keywords: Collaborative Perception; Heterogeneous Collaboration; Diffusion model
Primary area: applications

Abstract: Multi-agent collaboration enhances the perception capabilities of individual agents through information sharing. However, in real-world applications, differences in sensors and models across heterogeneous agents inevitably lead to domain gaps during collaboration. Existing approaches based on adaptation and reconstruction fail to support *pragmatic heterogeneous collaboration* due to two key limitations: (1) Intrusive retraining of the encoder or core modules disrupts the established semantic consistency among agents; and (2) accommodating new agents incurs high computational costs, limiting scalability. To address these challenges, we present a novel **Gen**erative **Comm**unication mechanism (GenComm) that facilitates seamless perception across heterogeneous multi-agent systems through feature generation, without altering the original network, and employs lightweight numerical alignment of spatial information to efficiently integrate new agents at minimal cost. Specifically, a tailored Deformable Message Extractor is designed to extract spatial message for each collaborator, which is then transmitted in place of intermediate features. The Spatial-Aware Feature Generator, utilizing a conditional diffusion model,  generates features aligned with the ego agent's semantic space while preserving the spatial information of the collaborators. These generated features are further refined by a Channel Enhancer before fusion. Experiments conducted on the OPV2V-H, DAIR-V2X and V2X-Real datasets demonstrate that GenComm outperforms existing state-of-the-art methods, achieving an 81\% reduction in both computational cost and parameter count when incorporating new agents. Our code is available at https://github.com/jeffreychou777/GenComm.

Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism
Junfei Zhou
Penglin Dai Quanmin Wei!.
Bingyi Liu
Xiao Wu! Jiamping Wang
Sosthwest Jiaotong University EEgineering Research Center of Sustainable
Urban Intelligent Transportation, Ministry Education, China *Wuhan University of Technology *City University of Hong Kong {jeffreychou, wqm)-my swjtu. edu. cn penglindai@swe edu, cn byliu@whut edu. wuxiaohk@gmail.cmm com jianwang@ci edu. hk
Abstract
Multi-agent collaboration enhances the perception capabilities of individual agents through information sharing. However, in real-world applications, differences in sensors and models across heterogeneous agents inevitably lead to domain gaps during collaboration. Existing approaches based on adaptation and reconstruction fail to support pragmatic heterogeneous collaboration due to two key limitations: (1) Intrusive retraining of the encoder core modules disrupts the established semantic consistency among agents; and (2) accommodating new agents incurs high computational costs, limiting scalability. To address these challenges, we present novel Generative Communication mechanism (GenComm) that facilitates seamless perception across heterogencous multi-agent systems through feature gen- eration, without altering the original network, and employs lightweight numerical alignment of spatial information to efficiently integrate new agents at minimal cost, Specifically, a tailored Deformable Message Extractor is designed to extract spatial message for each collaborator, which is then transmitted in place of intermediate features. The Spatial Spatial-Aware Aware Feature Generator, utilizing conditional diffusion model, generates features aligned with the ego agent's s semantic space while pre- serving the spatial information of the collaborators. These generated features further refined by Channel Enhancer before fusion. Experiments conducted on the OPV2V-H, DAIR-V2X, and V2X-Real datasets demonstrate that GenComm outperforms existing state-of oftt--of-hhe-att methods, achieving an 81% % reduction in both computational cost and parameter count when incorporating new agents. Our code is available at http://hww..cooo////////////////
Introduction
In autonomous driving field, multi-agent collaborative perception has emerged promising paradigm for enhancing environmental understanding by enabling information sharing among agents, thereby effectively extending perception range and mitigating challenges such occlusions and long-range sensing limitations 2 3 Recently, numerous studies have been conducted to advance this field 4
g. Most of these works based the assumption of homogeneous collabora- tion, which limits their applicability in real-world scenarios, where collaboration typically involves heterogeneous agents with diverse sensor modalities and model architectures.
Existing approaches for heterogeneous multi-agent perception are predominantly non-generative and be broadly categorized into two types: iepptation-basele and reconstruction-bass meth- ods, illustrated in Figure [ (a) and (b). Adaptation- based methods include using a single adapter for one-stie stage transformation such MPDA employing two-stage adaptation with predefined
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

STAMP
Serfommm
Adapheniss
181%
Adoptor
sigg
Code Decoder
Briodder
Codle
Extractor
Resenatraction
Generator
Adop
Adoptor
.62%
Indicss
Spatia Message
Generation
Shared Codebook
Protocel
nuttber
(o) Adaptation
(b) Reconstruction
(C) Generation(ours)
(d) Scalability comparison
Comparison of heterogeneous collaboration strategies. (a) Adaptation-based based strategy: transform features via one or two-stage adaptation. (b) Reconstruction- haasdd strategy: reconstruct features ego agent using indices of a shared codebook. (c) Ours (Generation-basee features locally using collaborators' spatial messages, requiring only lightweight extractor tuning to support new agents without modifying the core module. (d) Our method shows superior scalability over (a) and (b).
Figure 1:
generate
protocol semantic space such as PnPDA I0] and STAMP[1] or adopting the BackAlign strategy such HEAL 12, which can be viewed as special form of adaptation that enforces the collabora- tor's semantic space to align with the ego agent's semantic space. Reconstruction- based methods CodeFilling [3), on the other hand, reconstruct features locally using indexing of shared codebook composed of encoded heterogeneous intermediate features. Despite their different mechanisms, both types of methods suffer from common limitations that fail support pragmatic heterogeneous collaborative perception. For instance. PnPDA [0] and STAMP[I] rely on predefined protocol semantic space, but the diversity of agents and vendors makes reaching consensus on a protocol space unrealistic. Similarly, methods like MPDA 99] and HEAL [ require retraining the fusion network or encoders, which disrupts the established semantic consistency among agents. When accommodating new agents, these methods and CodeFilling either require relatively high com- putational cost or introduce more parameters, thereby suffering from scalability constraints. These shared limitations present fundamental barriers to the real-world application of collaborative perception systems in heterogeneous environments, highlighting the core challenge of pragmatic heterogeneous collaborative perception: How can accommodate the emerging agents into the collaboration with minimal cost, while keeping the established semantic consistency among agents?
To address these challenges, we propose GenComm, a Generative Communication mechanism for heterogeneous collaborative perception that facilitates seamless perception across heterogeneous multi-agent systems through feature generation, without altering the original network, and employs lightweight numerical alignment of spatial message to efficiently integrate new agents at minimal cost, shown in Figure (c). The key idea behind GenComm is that each ego agent locally generates features for its collaborators using received spatial messages, ensuring that the generated features are aligned with the ego agent's 's semantic space while preserving the spatial information of its collaborators. To train the GenComm framework, initially conducted a homogeneous setting, the model learns three key components: the Deformable Message Extractor, responsible for capturing spatial messages; the Spatial-Aware Feature Generator, aimed at generating features based on the received spatial messages from collaborators; and the Channel Enhancer, designed to refine the generated features along the channel dimension before fusion. Although the spatial messages shared among agents may exhibit smaller domain gap compared to intermediate features, significant numerical discrepancies still arise due to inconsistent spatial confidence estimations across heterogeneous agents. Therefore, in the heterogeneous setting, each agent fine-tunes a lightweight message extractor specifically for its receiver to address the impact of numerical discrepancies across agents. Our method meets all the requirements of pragmatic heterogeneous collaborative perception simultancously and reduces transmission volume, thereby improving communication efficiency.
Accordingly, our contributions can be summarized follows:
We propose the first generation-basee communication mechanism called GenComm for heterogencous collaborative perception, enabling seamless perception among heterogeneous agents through feature generation without altering the original network, while leveraging lightweight numerical alignment of spatial message to accommodate new agents at minimal cost. Additionally, it brings the benefit of improved communication efficiency.

---

We design Deformable Message Extractor to extract key spatial information, which is used as conditions for a spatial-aware feature generator that generates features for collaborators aligned with ego's 8 semantic space and preserving the spatial information of collaborators, and Channel Enhancer is designed to refine the generated feature at channel dimension before fusion.
Extensive experiments on the OPV DAIR- V2X 14] and V2X-Ro Real [5 datasets demonstrate that GenComm outperforms state-of -ateeeof--haattt art baselines in both simulated and real-world heterogeneous settings. Moreover, the cost of accommodating new agent is reduced by over 81% % and 62% compared the leading adaptation- and reconstruction-base methods, respectively, highlighting its excellent scalability, shown in Figure (d).
2 Related Works
Collaborative perception. Collaborative perception breaks through the limitations of single-vehicle sensing by extending the ego agent's perception capability to occluded and long-range regions.
Among these, the most widely studied intermediate fusion approach [T6] I7 18 19 20 21 enables
individual agents to obtain more comprehensive features for downstream tasks by sharing and fusing intermediate features. Where2comm 4 and CodeFilling [3 have made efforts to balance communication overhead and performance, while methods such as CoAlign [5] and CoBevFlow [6] have been proposed to address feature misalignment caused by positional inaccuracies and temporal asynchrony.
[22] and V2X-R [3) incorporated radar data, exploiting the modality's 'ss robustness to adverse weather conditions to enhance the resilience of collaborative systems under challenging environments. While the above methods focus on enhancing collaborative perception under homogeneous settings, increasing attention has been given heterogeneous collaboration, where differences in sensing modalities and model architectures introduce new challenges.
Heterogeneous collaboration. Many existing works have made significant efforts to address the challenges of heterogeneous collaboration. Both BM2CP (24 and HM-Vi 25 leverage modality- specific characteristics to enhance heterogeneous feature fusion. MPDA
g adopts an adversarial
domain adaptation framework to transform features in 1 stage. PnPDA 10 and STAMP [11 maintains a shared semantic space, enabling semantic feature transformation 2 stage. HEAL 12 proposes Back strategy to align collaborator's semantic space to the ego agent's semantic space. CodeFilling (13) constructs a shared codebook by aggregating multi-domain features from all collaborators. Both feature transmission and reconstruction are performed through indexing into this shared codebook, which cleverly mitigates the domain gap among heterogeneous features by representing them a unified latent space. Although progress has been made, existing methods still fall short of simultaneously meet the requirements of pragmatic heterogeneous collaboration due to either intrusive design and limited scalability.
Feature generation. Diffusson models (26 are applied for Bird's Bird''sEyee- View (BEV) feature genera- tion, leveraging their powerful denoising capabilities to enhance spatial representations. DifBBE 28 further exploits the potential of diffusion models to generate more comprehensive BEV represen- tations, where condition-guider diffusion sampling produces richer BEV features for downstream tasks. V2X-R (3) adopts robust 4D radar features conditional inputs to denoise corrupted LiDAR features, resulting in cleaner representations. CoDiff (29) projects BEV features into a latent space and uses the projected representations conditional guidance for diffusion- based sampling.
3 Pragmatic Heterogeneous Collaborative Perception
In a pragmatic multi-agent collaborative perception system, each agent collaborates with heteroge- neous agents, while new agents continuously joining the collaboration. Assume there currently N agents, each equipped with a frozen pretrained perception network denoted as V The method for heterogeneous collaboration is represented by a learnable module T When a new agent joins the system, increasing the number of agents to N + 1, it is essential ensure that the integration incurs minimal parameter and computational overhead. This crucial due to the limited computational capacity of each agent and the need maintain scalability more agents objective is to optimize the parameters 0 of the heterogeneous collaboration module $ such that the overall perception error and computational cost ΣN+ Comp(4 ( of all N + agents are minimized. Subject to the constraint that each agent's 's perception network V remains fixed to preserve the
integrated. The
3

---

established semantic consistency among agents, the optimization problem is formulated as follows:
N+1
1+1
min
Σ dO,, Ue (2, oo ((Mj-nliea.))
Comp(11)
(I)
s.t. E'
Vi E (L,...,,
Here, O2 denotes the ground-truth output for agent i, -Mj. denotes the message extracted from agent j agent i, and the evaluation function d(-) measures the discrepancy between the ground-truth and the output predicted through collaboration, and G is the set of collaborators to agent i.
4 Methodology
In this section, we provide detailed description how the proposed GenComm is integrated into multi-agent perception system. We further elaborate on the underlying principles and the operational workflow of this mechanism. Designed to enhance system scalability, preserve the the established semantic consistency among agents, and enable efficient message sharing, our approach provides a more practical solution for building pragmatic heterogeneous multi-agent collaborative perception systems.
4.1 Framework overview
In a practical multi-agent collaborative perception scenario, we consider heterogeneous agents, among which one is designated the ego agent. For each agent i, we define G; as the set of collaborators to ego agent i. Each agent E {1,2,.... ..,.....N........ .V} receives an input observation X, which may consist of data from different sensing modalities such as images or LiDAR point clouds. To extract representations from these observations, each agent is equipped with an independent encoder fene which transforms the inputs into BEV features F.
Unlike previous methods that transmit full feature maps, our approach only requires the transmission of spatially-awar compressed representations. Specifically, after extracting features F; using dedicated encoder, each agent utilizes Deformable Message Extractor /mes_extract to capture spatial message, which then combined with the agent's meta information for transmission. Upon receiving messages from collaborators, the ego agent uses them as conditional inputs generate features that align to its own semantic space while preserving spatial information from the collaborators. A Channel Enhancer refines the generated features along the channel dimension before fusion. A fusion network is employed to integrate the ego features with the generated collaborator features, aiming to obtain more comprehensive global representations. The fused feature is finally passed to the decoder to obtain perception results. The following provides the mathematical representation:
F, (X,)
(2a)
Mj-+i = pd+ extract (F)), Vj E
(2b)
( hee. = fgeneratee (Fr
(2c)
2, = Shusion FFts (159)E):))
(2d)
Ô, Sácc(2)
(2e)
where F; E CG denotes the BEV feature of the ego agent i, with C. H,, and W; representing the channel, height, and width dimensions, respectively. Mj denotes the message extracted from agent j to agent i. The set { beg, represents the features generated by agent i for its collaborators.
2, is the fused feature of agent i, and Ó, denotes the final output obtained after decoding.
4.2 Model components
An overview of the system is presented in Figure 2 followed by detailed description of the three key components of GenComm. Additional detailed information about the model is provided in the Appendix A.3
Deformable message extractor. Extracting generalizable spatial information from features of differ- ent modalities processed by distinct encoders is a critical component of our approach. The extracted spatial information serves not only the transmitted message for inter-agent communication but

---

Encoder
Feature
Fusion
TniT.
8,
X;
Decoder
DME
Deformable Message
Spatial Generator
Output
Enceder
Offset
M.
Concatt
M.., Message
F
BEV feature
F
Genercted
Conv
Shifi
Resize
Elenent-wise muttiply
Fused fanture.
misse
CA
Gharnel Attention
U-Net G
Implemented age egent
Implemented collaboraters
Figure 2: Framework overview of GenComm. The Deformable Message Extractor extracts spatial messages, which serve conditions for the Spatial-Aware Feature Generator. Channel Enhancer further refines the generated features before fusion.
also spatial condition for the ego agent's 's feature generation. The quality of this spatial information directly determines the spatial fidelity of the generated features.
Intuitively, spatial information relevant to 3D object detection tasks can be represented by the confidence at individual pixels of BEV features. However, confidence a single pixel insufficient capture foreground-buckgroundd separation and contour information within BEV features. Therefore, we design a Deformable Message Extractor incorporating deformable convolution (30], which not only focuses on the pixel itself but also dynamically references surrounding pixels, thereby enhancing the model's capability to distinguish between foreground and background.
Formally, for an input feature map F, E RC. x an offset prediction network predicts a set of sampling offsets Apı, E 22 R.W, Given these offsets, spatial information is extracted using deformable convolution defined as:
M Resizer
Σ w(px) Fi (Po + P& + +pp..) C'''HH;WW, P.ER
Where R denotes the regular convolution sampling grid, le is the kernel size, and ur(-) represents the learnable attention weights. To guarantee compatibility across agents, learnable resizer is introduced to adapt the spatial information to the resolution the receiver. In this context, denotes the compressed channel dimension, while Mi-j corresponds to the extracted spatial message that serves as the basis for downstream communication and generation.
Spatial-awand feature generator: The ego agent upon receiving spatial messages (Mj-ii from collaborators, then leverages a conditional diffusion model [3 to generate feature representa- tions that align with its semantic space while preserving collaborator's spatial information.
Gaussian noise is progressively added to the initial feature Finit (where initialization from F,) over T time steps. Let Fe denote the feature at the denoising step f, then the diffusion process q can then be expressed as:
qF Funt) N (Fr
VarFuan (1 (1)
Specifically, given the initial feature Finits the noisy feature F, at timestep can be directly obtained via the closed-form of the forward diffusion process:
F. Va, Fmit + vI- az
1
(5)
This perturbed feature F, serves as the input to the denoising process, where a conditional U-Net co generates the feature representation at the previous timestep. At each generation step, the received message {Mi-nijje, is incorporated features:
conditioning input, guiding the U-Net eg to generate
(FLibea = en (MM LET,T-1,
After the final generation step T. the model generates features (A) that are aligned with the semantic space of ego agent while preserving the spatial information received from collaborators.
5

---

The generation process is directly supervised with the objective LOEN defined as the mean squared error between the generated features and their corresponding ground truth:
-
gen
2
jEG;
This design allows the ego agent to generates features aligned with the ego agent's semantic space while preserving the spatial information of the collaborators.
Channel enhancer. During the feature generation process, spatial information typically dominates, while semantic representations along the channel dimension are often overlooked. To ensure consis- tency between the ego features and the generated features in the channel dimension, we propose the Channel Enhancer. Specifically, we introduce the PConv operation [3 to enhance the representa- tional capacity of informative elements within the features. A gating mechanism [33] is incorporated to suppress redundant information along channel dimension, while channel attention is applied to emphasize critical features.
Firstly, we adopt PConv to reinforce the feature, then splits the reinforced feature into two parts along the channel dimension: a modifiable part Feom and a static part Fres. The modifiable part refined using a depthwise separable convolution and multiply by Fres. Following channel refinement, we apply a channel-wise attention mechanism suppress redundant channels and highlight informative ones. We compute soft attention score based on global context, the final refined feature is obtained. This module improves the channel-wis -wise expressiveness of the generated features, ensuring better alignment with ego-agent representations:
F = a (LN (Conv pw (ConVaw (Foom)) @ Fre)))
(8a)
= F' @
(8b)
Here, @ denotes element wise multiplication; GAP stands for global average pooling: FC denotes fully connected layer; LN represents a linear transformation; and the activation function.
4.3 Training strategy
The training process of GenComm consists of two stages. In the first stage, the model is trained in homogeneous multi-agent perception setting, where each agent constructs its own semantic space and learns three key components: a Deformable Message Extractor, Spatial-Aware Feature Generator and Channel Enhancer. After this first stage, the Deformable Message Extractor effectively captures spatial information, helping to avoid the domain gap issues commonly associated with intermediate features. However, it still suffers from numerical discrepancies, such as inconsistent confidence scores for the same pixels in the BEV map across heterogeneous agents. To mitigate this, in the second stage, each agent is initialized with a lightweight extractor tailored to the specific agent with which it will collaborate. Only the extractor is fine-tuned to align the spatial message, ensuring that the data received by the receiver is more consistent in numerical distribution with its own extracted spatial information. This alignment enhances the quality of the subsequently generated features.
4.4 Loss function
In Stage 1, we train our model in an end-to-end manner using combination of losses: a focal loss 34 for the classification loss Ceks a smooth LI loss 335 for the regression loss Creg, and mean squared error (MSE) loss for the generation loss Cgen- The classification and regression losses are standard 3D object detection losses, used to predict the confidence of each anchor and to regress the offset between the anchor and the ground truth object. Each loss is weighted by corresponding parameter a to balance its contribution, and the total loss is defined as Laugel an Lets + acy Crg + Ozg Ceg.gen- In Stage only the classification and regression losses are applied, resulting in Lauge2 = a Lets + a2 Crg.
5 Experimental Result
In this section, we evaluate the effectiveness of our approach using both simulated and real-world datasets, showing that our method delivers superior performance while integrating new agents at an ultra-low cost. Furthermore, we validate the contribution of each framework component through ablation studies.

---

Table 1: Performance on OPV2V-H and DAIR-V22 under various heterogeneous settings. AP is used to evaluate detection performance, while communication volume measures communication efficiency.
OPV2V-H
DAIR-V2X
Comm.
Fusion Network
Method
Volume
LF L
-
L
(loga)
AP50
AP70 f
AP50
AP70
AP30 1
AP50 1
MPDA
0.7668
0.5698
0.7369
0.5739
0.4246
0.3641
22.0
BuckAlign 12
0.7873
0.5841
0.6852
0.5240
0.4562
0.3727
22.0
CodeFilling 13
0.7218
0,5364
0.6661
0.5097
0.3848
0.3189
15.0
AttFuse 16
STAMP[I]
0.7594
0.5689
0.7258
0.5605
0.4468
0.3913
22.0
GenComm
0.8043
0.6332
0.7525
0.6005
0,4593
0.3786
16.0
MPDA
0.8495
0.6595
0.6873
0.5023
0.4717
0.3786
22.0
BackAlign T2
0.8553
0.6933
0.6907
0.5232
0,4902
0.3924
22.0
CodeFilling 13
0.8597
0.6886
0.5600
0.4156
0.4446
0.3557
15.0
V2X- 17
STAMP[I1
0.8442
0.6276
0.7508
0,5442
0.5421
0,4935
22.0
GenComm
0.8673
0.6991
0,7630
0.5759
0,5651
0,4665
16.0
5.1
Experiments setting
Datasets. We conduct experiments on three datasets: OPV2V-H DAIR-V2X 14 and V2X- [5. OPV2V-H is an extension of the large-scale le OPV2V I6 dataset, which was collected using the OpenCDA framework in the CARLA simulator OPV2V-H expands by incorporating additional sensor configurations, making it well-suited for heterogeneous collaborative perception research. DAIR-V22 -2X is the first realwwoldd dataset, the RSU in it equipped with high-resolution 300-channel LiDAR and a camera, while the vehicle has a 40-channel LiDAR and camera, enabling exploration of cross-agent collaboration under heterogeneous sensor setups. V2X-Real also real-world dataset but on larger scale, consisting of 2 vehicles and 2 RSUs (4 agents in total), which can be used to evaluate scalability in real-world scenarios.
Heterogeneous agents. Following prior work 12). four agents considered OPV2V-H: two equipped with LiDAR and two with cameras. For the LiDAR- -based agents, we adopt PointPillar and SECOND 39 the respective backbones, denoted as L and Lg where the superscript indicates the channel size of the LiDAR sensor. For the camera-based agents, EfficientNet and ResNet[40) are used as backbones, denoted CF and CR. respectively. For the V2X-Real dataset, we implement four agents with backbones of varying capacities (ranging from shallow to deep) and employ the PointPillar encoder for all agents, aiming investigate collaboration across different levels of feature extraction capability. The detailed network architectures and configurations are provided in the Appendix A.3 The substantial differences among these agents ensure the diversity and richness of our experimental setup.
Baselines. We incorporate several adaptation- based methods, including the one-stage method MPDA[9 and the two-stage method STAMP[1I) as well as BackAlign from HEAL [2] which enforces the alignment of collaborators' semantic spaces to the ego agent's 's semantic space. MPDA requires retraining both the fusion network and the task head, whereas BackAlign retrains the encoder to achieve feature alignment. STAMP introduces an additional protocol semantic space to facilitate stage adaptation. In addition, we include the reconstruction--iat reconstructs collaborator features using a shared codebook and the received indices.
method CodeFilling [3], which
Implementation details. For all baseline methods, first train base model for each agent in a homogeneous scenario, and then extend it to heterogeneous collaboration using their respective adaptation strategies. For our proposed GenComm, we train the entire system end-to-end in the homogeneous setting, and enable heterogencous collaboration by aligning lightweight Message Extractor for each newly introduced collaborator with respect the ego agent. All methods, including baselines and GenComm, are trained under the same settings for fair comparison on NVIDIA RTX 3090. Implementation procedures and specific configurations are detailed in the Appendix A.3
5.2 Quantitative results
Heterogeneous Collaboration. We evaluate our proposed method on three datasets encompassing both static and dynamic heterogeneous collaboration scenarios, covering two sensing modalities, four distinct encoder architectures, and two classical fusion method. In static collaboration scenarios,
7

---

Table 2: Performance comparison of different methods on two datasets as more agents are incorporated into collaboration. #P and #F denote the trained parameters and FLOPs, respectively.
OPV2V-H
1
+C
+ +C
Method
#P(M)
#F(G)
AP50
AP70
AP50 f
AP70
AP50
AP70 1
MPDA
0.7574
0.5497
0,6513
0.4786
0.6815
0.5123
5.75
51.33
BackAlign
0.6975
0.5288
0.7238
0.5398
0.7252
0.5408
31.18
211.38
Codefilling
0.6891
0.5234
0.637
0.4658
0.5981
0.4316
0.81
12.91
STAMP[1]
0.7609
0.5878
0.7819
0,5995
0.7829
0.6002
1.64
3.084
GenComm
0.7538
0.5951
0.7873
0.6174
0.7866
0.6184
0.31
0.615
V2X-Real
1228
+L
LH +
128
1.2 +
1 +LM +
+L
Method
#P(M)
#F(G))
AP30 f
AP50
AP30 t
AP50 +
AP30 f
AP50
MPIAA 9
0.6344
0.5725
0.6323
0.5751
0.6211
0.5672
5.75
51.93
BackAlign 12
0.6313
0.5822
0.6386
0.5878
0.6352
0.5896
31.18
211.38
CodeFilling 13
0.6273
0,5826
0.6284
0.5799
0,6081
0,5571
0.81
12.91
[TAMP[[]
0.6314
0,5881
0.6335
0,5893
0.6289
0,5882
1.64
3.084
GenComm
0.6848
0.6175
0.6961
0.6299
0.7144
0.6362
0.31
0.615
Por formercee AlYO
0.5
Poromeene APRC
AP7O
0.5
Porto
0.4
0.4
0.4
0.3-
killen
0.4
Boridlor
03
1007
S3MP
0.2
0.2
0.2
0.0
0.0
0.4
0.8
100
200
300
400
500
100
200
300
400 500
Pose Nolse Stdd deg)
Pose Noise (m degi
Time Dalyy (ms)
Time Delay (ms)
Figure 3: Comparision of performance with baselines on pose error and time delay setting.
as shown Table D our method outperforms the baseline methods in the majority of cases while reducing communication overhead by up to 64x. The performance improvement primarily stems from the tailored Deformable Message Extractor, Spatial-Aware Feature Generator and Channel Enhancer, which jointly enable precise spatial information extraction, high-quality feature generation, and rich semantic preservation-effcciivee narrowing the semantic gap across heterogeneous agents. In dynamic collaboration scenarios, illustrated in Table evaluate all methods the OPV2V-H and -22lReal datasets. In this setting, more agents progressively join the collaboration. Notably, GenComm, BackAlign, and STAMP maintain consistent performance gains as the number of collabo- rators increases, whereas MPDA and CodeFilling do not. Moreover, GenComm achieves the highest performance with the minimal computational and parameter cost. Importantly, only STAMP and our approach are non- ion-intrusive during heterogeneous collaboration.
Scalability analysis. We analyze the scalability our method Table 2 and Figure Thanks to the lightweight numerical alignment mechanism, our approach can continuously accommodate newly added collaborators with minimal computational overhead. Table we designate L as the ego agent and incrementally introduce three heterogencous agents for collbboation. Our method not only outperforms the baselines terms of accuracy, but also reduces parameter and computation costs by 80% compared to the latest state-of-the-an art methods STAMP. This demonstrates that our method maintains extremely low incremental cost as more agents join the collaboration, highlighting its strong scalability.
Robustness analysis. real-world applications, pose errors and time delays are as inevitable as heterogencous collaboration. To better assess the practicality of our method, we further evaluate its robustness under pose eтrors and time delays. Specifically, we introduce Gaussian noise and asynchrony input simulate pose errors and time delays. Experimental results demonstrate that our method exhibits superior robustness to both pose errors and time delay compared to state-of-tte of-the------rrt approaches shown in Figure

---

Table 3: Ablation Study on Message Extractor, Channel Enhance, and Extractor Align
Performance (AP50)
Inference Time (ms)
30
Perfomance
Communication
DME
CE
Align
AP50
AP70
0.2850
0.1922
24
0.7555
0.5961
0.7805
0.4911
0.7877
0.5480
64
0.7514
0.5709
Channel Size
0.8043
0.6332
Figure 4: Ablation study on the channel size the spatial message.
Ablation study. We conduct ablation studies to analyze the effectiveness of the proposed components, as shown in Table and Figure 4 which present the results of component-wise ablation and channels of message ablation, respectively. The results indicate that the Channel Enhancer plays a crucial role in refining the generated features, while the Deformable Message Extractor effectively captures accurate spatial information guide the generation process. Additionally, the lightweight numeric alignment helps mitigate spatial message discrepancies across agents. As illustrated in Figure 4, we conduct an ablation study on the channel size of the spatial message to investigate model performance, communication volume, and inference time. The red line denotes the model performance, the blue line represents the inference time of the diffusion model, and the red circles indicate the corresponding communication cost. Overall, our message design achieves balanced compromise across these three factors.
trade-off among
Visualization
The visualization results demonstrate that the semantic space of the generated features consistent with that of the ego agent, while fully preserving the spatial information of the collaborators. In terms of both feature quality and detection performance, our method surpasses CodeFilling.
(a) Collaborator 's feature
(b) Reconstructed by CodeFilling (c) Detection ouput of CodeFilling
(d) Semantic space of ego agent (c) Mi & Generated feature F (f) Detection ouput of GenComm
Figure 5: Visualization result of our generation method, Compared to CodeFilling, generated feature.F; is more consistent with the ego agent i's semantic space and preserving the spatial informa- tion from collaborator j. The prediction and GT shown red and green respectively.
Solution for Real- Application
In real-world deployments, we assume three vendors: A, B, C, and five heterogeneous agent types: A1, A2, B1, B2, B3, C, where each agent type refers to a specific combination of sensor and model architecture. Below we describe how GenComm can be practically applied in such scenarios:
Stage 1: Homogencous Pre-training. Each vendor trains their agents using GenComm in a homo- gencous collaboration setting. For example, vendor A trains agent A₁ in collaboration with other instances of A₁ using their private data belonging to vendor independently. In this stage, the fu-

---

Vendor A
Stagel
Stage2
Vendor B
Private data
Vendor C
Private model
Spatiglmessog
Vendor D
Private Platform
GenComm
Pair consensus between vendors
(1 A
)
Infra.
GenComm base training each vendor under homogeneous collaboration setting
WoaldWorld Application
Plag & Play Style
A-
Agent
Agent
Agent wants join the collab.?
B->A
AB
Collab.
Agent
C
HP
CB
Steps:
Reach consensus
B-C
I-C
w/ other vendors. Train specific
3
1
Legocy model DME
Agent 2
Infra
DMEs.
Specific DMEs Training under heterogeneous collaboration setting
We assume a mon-fully-connen fully-cconecttd communication graph in real-world applications, accordance with the consensus reached among vendors.
Figure 6: Application rationale of GenComm.
sion network, generation module, and Channel Enhancer are trained effectively under homogeneous scenarios.
Stage 2: Heterogeneous Collaboration. If two vendors reach collaboration consensus, they can enable heterogeneous collaboration by training specific Deformable Message Extractors (DMEs) between heterogeneous agents. For instance, if vendors A and B agree to collaborate, they train DMEs such as: DM F. DM F. -.. DM FB
These lightweight modules enable heterogeneous collaboration by deploying corresponding DMEs to each agent plug-and-plas non-intrusive style. When new agents join, vendors only need to deploy the corresponding DMEs without further modifications, thereby preserving the established semantic consistency among agents.
Conclusion
We propose GenComm, generative communication mechanism designed to support pragmatic heterogeneous collaborative perception. GenComm facilitates seamless perception across hetero- geneous multi-agen systems through feature generation, without altering the original network, and employs lightweight numerical alignment spatial information to efficiently integrate new agents at minimal cost. Comprehensive experimental results demonstrate that GenComm not only achieves strong performance, but also exhibits excellent scalability and communication efficiency. These advantages make GenComm a potential solution for large-scale deployment in real-world multi-agent systems, contributing to enhanced safety in autonomous driving and reduced risk of accidents.
Limitations. Although we assume a more realistic won-fflly-coommeee communication graph in our application rational, the approach still requires consensus among vendors, which may be hindered by factors such as commercial competition and the potential risks of malicious attacks.
10

---

Acknowledgment. This work was supported in part by the National Natural Science Foundation of China under Grant 62172342, Grant 62372387; Key R&D Program of Guangxi Zhuang Autonomous Region, China (Grant No. AB22080038, AB22080039); The Open Fund of the Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, China (Project No. KCX2024-KF0)).
References
[1] Yushan Han, Hui Zhang, Huifang Li, Yi Jin, Congyan Lang, and Yidong Li. Collabora- tive perception in autonomous driving: Methods, datasets, and challenges. IEEE Intelligent Trunsportation Systems Magazine, 15(6):31-151, 2023.
[2] Si Liu, Chen Gao, Yuan Chen, Xingyu Peng. Xianghao Kong, Kun Wang, Runsheng Xu, Wentao Jiang, Hao Xiang, Jiaqi Ma, al. Towards vehicle-to-everythin autonomous driving: A survey on collaborative perception. arXiv preprint arXiv:2308. 16714, 2023.
[3] Lei Yang, Kaicheng Yu, Tao Jun
Jun Li, Kun Yuan, Li Wang. Xinyu Zhang, and Peng Chen. Bevheight: A robust framework for vision-based roadside 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21611-21620, 2023.
[4] Yue Hu, Shaoheng Fang. Zixing Lei, Yiqi Zhong, and Siheng Chen. Where2comm: Communication-cfficee efficient collaborative perception via spatial confidence maps. Advances neural information processing systems, 35:4874 4886, 2022.
[5] Yifan Lu, Quanhao Li, Baoan Liu, Mehrdad Dianati, Chen Feng, Siheng Chen, and Yanfeng Wang. Robust collaborative 3d object detection in presence of pose errors. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 4812-4818, IEEE, 2023.
[6] Sizhe Wei, Yuxi Wei, Yue Hu, Yifan Lu, Yiqi Zhong, Siheng Chen, and Ya Zhang. Asynchrony- robust collaborative perception via bird's eye view flow. Advances in Neural Information Processing Systems, 36:28462-288 224777 2023.
[7] Rui Song, Chenwei Liang, Hu Cao, Zhiran Yan, Walter Zimmer, Markus Gross, Andreas Festag, and Alois Knoll. Collaborative semantic occupancy prediction with hybrid feature fusion in connected automated vehicles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pages 17996-18006, 2024.
[8] Quanmin Wei, Penglin Dai, Wei Li, Bingyi Liu, and Xiao Wu. Copeft: Fast adaptation framework for multi-agent collaborative perception with parameter-efficient fine-tuning, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 23351 -23359, 2025.
[9] Runsheng Xu, Jinlong Li, Xiaoyu Dong, Hongkai Yu, and Jiaqi Ma. Bridging the domain gap for multi-agent perception. 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 6035-6042. IEEE, 2023.
[10] Tianyou Luo, Quan Yuan, Guiyang Luo, Yuchen Xia, Yujia Yang, and Jinglin Li. Plug and play: A representation enhanced domain adapter for collaborative perception. In European Conference Computer Vision, pages 287-303. Springer, 2024.
[11] Xiangbo Gao, Runsheng Xu, Jiachen Li, Ziran Wang. Zhiwen Fan, and Zhengzhong Tu. Stamp: Scalable task and model-agnostic collaborative perception. arXiv preprint N.Ni:2501..8616 2025.
[12] Yifan Lu, Yue Hu, Yiqi Zhong, Dequan Wang. Yanfeng Wang, and Siheng Chen. An extensible framework for open heterogeneous collaborative perception. arXiv preprint 11X19 2024.
[13] Yue Hu, Juntong Peng, Sifei Liu, Junhao Ge, Si Liu, and Siheng Chen. Communication-efficiene collaborative perception via information filling with codebook. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15481 -15490, June 2024.
11

---

[14] Haibao Yu, Yizhen Luo, Mao Shu, Yiyi Huo, Zebang Yang, Yifeng Shi, Zhenglong Guo, Hanyu Li, Xing Hu, Jirui Yuan, al. D2x:-v2x: A large-scale dataset for cchicle-ifrratrrcttur cooperative 3d object detection. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 21361-21370, 2022.
[15] Hao Xiang, Zhaoliang Zheng, Xin Xia, Runsheng Xu, Letian Gao, Zewei Zhou, Xu Han, Xinkai Ji, Mingxi Li, Zonglin Meng, et al. V2x-real: a largs-scale dataset for cehicle-to-everythin cooperative perception. European Conference on Computer Vision, pages 450- Springer, 2024.
[16] Runsheng Xu, Hao Xiang, Xin Xia, Xu Han, Jinlong Li, and Jiaqi Ma. Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to-vehicle communication. In 2022 International Conference on Robotics and Automation (ICRA), pages 2583-2589 IEEE, 2022.
[17] Runsheng Xu, Hao Xiang, Zhengzhong Tu, Xin Xia, Ming-Hsuan Yang, and Jiaqi Ma. V2x-vit: Vehicle-to toverrthng cooperative perception with vision transformer. In European conference on computer vision, pages 107-124, Springer, 2022.
[18] Yue Hu, Yifan Lu, Runsheng Xu, Weidi Xie, Siheng Chen, and Yanfeng Wang. Collaboration helps camera overtake lidar in 3d detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2023.
[19] Shixin Hong, Yu I. Liu, Zhi Li, Shaohui Li, and You He. Multi-agent collaborative perception via motion-aware robust communication network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15301-15310, 2024.
[20] Yen-Cheng Liu, Junjiao Tian, Chih-Y Yao Ma, Nathan Glaser, Chia- Wen Kuo, and Zsolt Kira. Who2com: Collaborative perception via learnable handshake communication. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 6876-6883. IEEE, 2020.
[21] Binglu Wang, Lei Zhang, Zhaozhong Wang, Yongqiang Zhao, and Tianfei Zhou. Core: Cooper- ative reconstruction for multi-agent perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 87 8710-8720, 2023.
[22] Lei Yang, Xinyu Zhang, Chen Wang, Jun Li, Jiaqi Ma, Zhiying Song, Tong Zhao, Ziying Song, Li Wang, Mo Zhou, et al. V2x-radar: A multi-modal dataset with 4d radar for cooperative perception. arXiv preprint arXiv:2411.1096 2024.
[23] Xun Huang. Jinlong Wang, Qiming Xia, Siheng Chen, Bisheng Yang, Xin Li, Cheng Wang, and Chenglu Wen. V2x-r: Cooperative lidar-4d radar fusion for 3d object detection with denoising diffusion. arXiv preprint arXiv:241 0224111.04440 084402, 2024.
[24] Binyu Zhao, Wei Zhang, and Zhaonian Zou, Bm2cp: Efficient collaborative perception with lidar-camera modalities. arXiv preprint xxiv:2330..17002 2023.
[25] Hao Xiang, Runsheng Xu, and Jiaqi Ma. Hm-vit: Hetero-modal vehicle-to-vehicl cooperative perception with vision transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 284-295, 2023.
[26] Duy-Tho Le, Hengean Shi, Jianfei Cai, and Hamid Rezatofighi. Diffuser: Diffusion model for robust multi-sensor fusion in 3d object detection and bev segmentation. arXiv preprint arXiv:2404.0m 04629, 2024.
[27] Xin Ye, Burhaneddin Yaman, Sheng Cheng, Feng Tao, Abhirup Mallik, and Liu Ren. Bevd- iffuser: aod--and-pay diffusion model for bev denoising with ground-truth guidance. arXiv preprint arXiv:2502. 19694, 2025.
(28) Jiyy Zou, Kun Tian, Zheng Zhu, Yun Ye, and Xingang Wang. Diffbev: Conditional diffusion model for bird's eye view perception. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 7846-7854, 2024.
[29] Zhe Huang, Shuo Wang, Yongcai Wang, and Lei Wang. Codiff: Conditional diffusion model for collaborative 3d object detection. arXiv preprint arXiv:2502. 2025.
12

---

[30] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. In Proceedings of the IEEE international conference on computer vision, pages 764-773,
[31] Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Conditioning method for denoising diffusion probabilistic models. arXiv preprint a7Xiv:2108.02938, 2021.
[32] Jierun Chen, Shiu-hong Kao, Hao He, Weipeng Zhuo, Song Wen, Chul-Ho Lee, and S-H Gary Chan. Run, don't walk: chasing higher flops for faster neural networks. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pages 12021-12031, 2023.
[33] Shihao Zhou, Duosheng Chen, Jinshan Pan, Jinglei Shi, and Jufeng Yang. Adapt or perish: Adaptive sparse transformer with attentive feature refinmmen for image restoration. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2952-2963, 2024.
[34] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.
[35] Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference computer vision, pages 1440-1448, 2015.
[36] Runsheng Xu, Yi Guo, Xu Han, Xin Xia, Hao Xiang, and Jiaqi Ma. Opencda: an open coopera- tive driving automation framework integrated with co-simulation. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 1155-1162. IEEE, 2021.
[37] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. Conference on robot learning, pages 1-16. PMLR, 2017.
[38] Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12697-12705, 2019.
[39] Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 18(10):3337, 2018.
[40] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016
[41] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105-6114 14. PMLR, 2019.
[42] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rXiv:1412.69800 2014.
13

---

A Appendix
A.1 Advantages real-world applications
compare several heterogeneous collaboration methods with proposed generative approach in terms of their characteristics and capabilities. Specifically, "Multi-Mod." and "Multi- Enc." indicate whether each method demonstrates the efffectiveness of multi-modality and multi- encoder settings, respectively. "Non-Intru." refers to whether the pre-trained models or core modules remain non-intrusive. "Scal." " denotes scalable, indicating whether the method can seamlessly accommodate newly introduced agents at minimal cost. "Comm. Eff." reflects whether the approach communication-efficii These merits are crucial for the real-world applications of multi-agent systems. Among all existing methods, only GenComm simultaneously satisfies all these merits, which demonstrates that method is more practical than prior approaches and highlights the superiority of our proposed framework.
In Table A.I
Table A.1: Comparison of methods key properties.
Method
Publication
Multi-Mod.
Multi-Enc.
Non-Intru.
Scal.
Comm. Eff.
MPDA
9
ICRA 2023
HM-ViT 25
ICCV 2023
12
ICLR 2024
CodeFilligg
13
CVPR 2024
IO
ECCV 2024
STAMP
ICLR 2025
GenComm
A.2 Algorithmic Description of GenComm
Here, we present the algorithmic pipeline of the proposed method Algorithm which summarizes the key steps and provides clear overview of the overall mechanism.
Algorithm 1: GenComm: Overall Algorithmic Pipeline
Data: N: total number of collaborative agents; G5: set of collaborators for agent
Input: Xx. observation agent i
Output: Ó;: prediction of agent
for 1 N do
F. fene (Z)) &
// Feature extraction Message extraction
M = fine extrac (Fi) E RC, Vj E G.
end
The lightweight messages Miny are used in place of raw features F; for communication.
for
N do
Fin + Fi
// > Initialization
z IO,II)
F. = Vor, + /1 ai
Diffusion forward process
for 1 + TTT -1 1,...,00
(F/-)ees,
([F (M)-iljeallt
// Feature generation
end
{F, beg. (Fi (Feiaieg
{F, (J,be = onhancel (ee))))))
// > Channel enhancement
2, Shuioo (Fr, (F
// Feature fusion
O five (2:)
end
A.3
Implementation details
A.3.1
Component details
illustrates the detailed workflow of the Channel Enhancer module. Here, PConv denotes Partial Convolution (32, dwcony denotes Depth-wise Convolution, pwconv stands for Point-wise Convolution, and LN represents Linear Layer. The symbol @ indicates element-wiss multiplication.
Figure A.1
14

---

F
Channel Attention
PConv
dwconv
pwconv
LN
F
Fr
Figure A.I: Details of the channel enhancer
Table A.2: Configuration of the four heterogeneous agents
Symbol
Sensor
Encoder
Backbone
#Params (M)
OPV2V-H & DAIR-V2X
Agent
L
64-Channel Lidar
PointPillar (38]
Deep
6.58
Agent
CE
RGB Camera
EfficientNet 41
Deep
21.25
Agent
L
32-Channel Lidar
SECOND 39
Shallow
0.89
Agent
CR
RGB Camera
Resnet101
40
Deep
8.15
V2X-Real
Agent
L 228
128-
M88-Channel Lidar
PointPillar
38
Deep
8.07
Agent
L
128-Chan S3-Chhnnel Lidar
PointPillar 38
Medium
2.23
L (28
Agent
128- Ca-nnhenel Lidar
PointPillar 38
Shallow
1.06
M
Agent
128
128-Chan Channel Lidar
PointPillar 338
Identity"
0.76
Identity indicates that the backbone is implemented mn. Identity().
Channel Attention refers to applying an attention mechanism along the channel dimension, followed by weighting the features with the corresponding attention scores obtain the enhanced feature representation.
A.3.2 Heterogeneous agents details
For the OPV2V-H and DAIR-V V2X datasets, four heterogeneous agents designed by combining three types of sensor data, four encoder architectures, and two backbone networks, supporting heterogeneous collaboration, denoted L CF. CR and L shown in Table A.2 Specifically, the sensing modalities include a 64-channel LiDAR, a 32-channel LiDAR, and an RGB camera. Point cloud data processed using different encoders: PointPillars 38 and SECOND Image features are extracted using EfficientNet [4]] and ResNet-101 40). For the V2X-Real dataset, four agents are designed with varying feature extraction capacities, ranging from deep to shallow backbones and even without a backbone. All agents adopt the PointPillar encoder and are denoted as L L!
L M and L
In these agents, BEV Backbones of different scales are employed to extract spatial features. The shallow backbone consists of single block with 128 output channels, designed to retain fine-grained spatial information at minimal computational cost. The medium backbone includes two blocks with 256 output channels, providing balance between efficiency and representational capacity. The deep backbone adopts three blocks with 384 output channels, offering enhanced feature extraction capability.
A.3.3 Experimental setting details
Collaborative perception settings. In our collaborative perception setting, the maximum commu- nication range is set to [ 102.4 m, 102.4 m] along the z-axis and [-51.2m. m, 51.2 m] along the y-axis, covering total area of 204.8 m x 102.4 4 m. contrast, agents equipped with camera sensors have limited perception range of [-51.2 m, 51.2 m) along both axes (i.e., 102.4 m x
m. During training, the perception range for LiDAR -quupped agents
102.4 10).4m). During testing and
inference, the perception range is unified for agents to [- 102.4 m, 102.4 m) along the z-axis and [-51.2 51.2 m] along the yaaxis.
15

---

End 2 End
BackAlign
Input i
Input
Input
Input
STAMP
Encoder
Encoder
Encoder E
Encoder
Input
Input
Fusion Net
Fusion Net
Encoder
Encoder
Decoder
Decoder
Adapter
Adapter
MDPA
CodeFilling
Protacol Space
Input
Input
Input
Input
Encoder rd
Encoder j
Encoder i
Encoder
Reverter
Reverter
Adapter
Codebook
Fusion Net
Fusion Ne
Fusion Net
Fusion Net
Decoder
Decoder
Decoder
Decoder
Figure A.2: Illustration of the training strategies used in baseline methods when new heterogeneous agents join the collaboration.
The intermediate feature maps have spatial dimensions of [C, H, W = [128, 64, 128), corresponding to the number of channels, height, and width, respectively. For camera-based agents, due to the smaller perception area, the feature map size is reduced to [128, 64, 64].
GenComm settings We design the transmitted information to have a shape of [C", Hy, W,]. where C = 2 denotes the number of channels. The height H, and width Wy are determined dynamically based on the receiving agent's spatial configuration. The diffusion model is configured with a total time step T 3, and the denoising network eg implemented U-Net with 2 layers. Within the Channel Enhancer module, the channel dimensions of Fres and Fromy are both set to 64.
A.3.4 Training strategies between methods
Baseline training. We illustrate the training strategies the bascline method in detail in Figure A.2 The pipeline starts with training individual agents in homogencous setting, which serves as the foundation for subsequent heterogeneous collaboration. The baseline model is then trained on top of these pretrained models. End-to-end training requires retraining all modules of the heterogeneous agents. BackAlign [2] achieves heterogeneous collaboration by aligning the collaborators' semantic space with that of the ego agent, which necessitates retraining the collaborators encoders. MPDA 9 introduces a dedicated adapter for each collaborator and requires retraining both fusion network and decoder. CodeFilling [3] updates the codebook whenever new collaborator joins, ensuring the codebook includes representations for the new agent. STAMP[I] equips each agent with an adapter that maps its own semantic space shared protocol space, as well as a converter that maps the protocol space back to its native space. For all pre-trained model using AttFuse 16] as the fusion network, we train for 20 epochs with an initial learning rate of 0.002, using the Adam 42 optimizer. The learning rate is decayed by a factor of 0.1 1.1 at the 10th and 15th epochs. For pre- pre-trained model using V2X V2X-ViT(I7 as the fusion network, training is conducted for 30 epochs with the same initial learning rate, which is decayed by 0.1 at the 15th and 20th epochs. Based on the pretrained base models, Alignlign, MPDA, and CodeFilling are fine-tuned for additional epochs with an initial learning rate of 0.001, and the learning rate is decayed by factor of 0. at epoch 5. For STAMP, we follow its training schedule: the model fine-tuned for epochs with initial learning rate of 0.01, and the learning rate is decayed by 0.1 at epochs 1, 3, and 4.
16

---

Input
Input
Input &
Input
Input j
Input n
Encoder
Encoder
Encoder &
Encoder
Encoder P
Encoder reg
DME
DME
DME
DME
SAFG M
SAFG
CE
e
Keep original model
unchanged.
Fusion Net
Fusiion Net
2. Only the light weight extractor needs be aligned.
Decoder
Decoder X
Stagel: Base training under homogeneous setting
Stage2: Extractor align under heterogeneous setting
Figure A.3: Training strategy of GenComm
Table A.3: Robustness analysis on the real-world DAIR-VA V2X dataset with pose noise. Gaussian noise (0, of) is added to the and positions, and (00, of) to the yaw angle.
Noise level (of /o
0.0/0.0
0.1/0.1
0.2/0.2
0.3/0.3
0.4/0.4
Methods Metrics
AP30 t
MPDA 9
0.4296
0.4282
0.4214
0.3964
0.3522
BackAlign 12
0.4560
0.4528
0.4435
0.4187
0.3688
AttFuse 16
CodeFilling 13
0.3854
0.3840
0.3750
0.3526
0.3189
STAMP
0.4468
0.4449
0.4379
0.4162
0.377
GenComm
0.4608
0.4586
0.4476
0.4198
0.3726
MPDA
0.4717
0.4708
0.4550
0.4257
0.3825
BackAlign 12
0.4898
0.4874
0.4710
0.4367
0.3783
V2XVIT
CodeFilling 13
0.4443
0.4390
0,4266
0.3969
0.3521
STAMP
0.5421
0.5396
0.5327
0.5051
0.4586
GenComm
0.5624
0.5616
0.5438
0.5075
0.4398
Methods / Metrics
AP50 1
MPDA[9]
0.3685
0.3591
0.3170
0.2372
0.1624
BackAlign 12]
0.3725
0.3628
0,3181
0.2374
0.1652
AttFuse
CodeFilling 13
0.3185
0.3099
0.2697
0.2021
0.1495
STAMP
0.3913
0.3817
0.3383
0.2582
0.1899
GenComm
0.3801
0.3681
0.3195
0.2415
0.1653
MPDA 9
0.3779
0.3674
0.3178
0.2377
0.1702
BackAlign
0.3940
0.3819
0.3253
0.2359
0.1591
V2XVIT
CodeFilling 13
0.3559
0.3483
0.2920
0.2161
0,1504
STAMP
0.4935
0,4822
0,4266
0.3275
0.2317
GenComm
0.4649
0.4534
0.3875
0.2913
0.69
GenComm training. As illustrated in Figure A.3 we present the training pipeline of GenComm. In the first stage, the three key components of our method, namely the Deformable Message Extractor (DME), the Spatial-Aware Feature Generator (SAFG), and the Channel Enhancer (CE), are trained in homogeneous setting. After this stage, only the collaborator speccfi Extractor is trained to numerically align the transmitted messages with those of the receiver. This lightweight alignment strategy enables efficient adaptation to newly joined agents at minimal cost, thereby supporting pragmatic heterogeneous collaboration. The training hyperparameters for both stages are consistent with those used in the baseline setup.
17

---

Table A.4: Comparison of introduced latency across different methods.
MPDA
BackAlign
CodeFilling
STAMP
GenComm
Latency introduced (ms) 69.918
0"
13.390
17.934
20.7
Since BackAlign adopts an encoder alignment strategy without introducing extra modules, introduced latency is ms.
Table A.5: Performance of GenComm on OPV2V-H when dynamically adding agents.
64
Lp +
Ce
L + Cg + L
Lp + Cs + L + CR
Method
AP50
AP70
AP50 f
AP70
AP50
AP70 t
AP50 f
AP70
GenComm
0.7381
0.5848
0.7538
0,5951
0.7873
0.6174
0.7876
0.6184
A.4
Additional experimental results
A.4.1
Robustness analysis on DAIR-V2X
We further evaluate the robustness of the proposed method on the real-world DAIR- V2X dataset by simulating pose errors through the addition of Gaussian noise to agent poses. As shown in Table A.3] our method maintains strong robustness under noisy conditions, demonstrating good generalization capability to real-world scenarios.
A.4.2 Comparison of introduced latency among methods
In autonomous driving scenarios with tight real-time requirements, inference latency is critical factor that directly impacts system safety and responsiveness. We measured the introduced inference time for all baselines, averaging over 100 iterations after a 10-iteration GPU warm-up up to ensure reliability. GenComm's latency compared CodeFilling and STAMP and is smaller than the sensor data collection interval (100 ms in OPV2V-H). We consider this latency acceptable and unlikely to affect real-time performance.
A.4.3 Impact of dynamic agent participation
In real-world deployment, collaborative perception systems are often required function under dynamic collaboration settings, where participating agents may frequently join or leave the system. In Table report experiments
OPV2V-H and V2X-Real by dynamically introducing additional heterogeneous agents into the system, starting from single agent and scaling up to four collaborating agents. Table A.5 We take GenComm on OPV2V-H as an example, the AP70 score is 0.585 with only L When progressively adding Cg. L and CR. the performance increases to 0.595 (+0.010), 0.617 (+0.022), and 0.618 (+0.001), respectively. These results indicate that while adding more agents consistently improves performance, the marginal gains diminish as the collaboration scales up.
Conversely, if an agent leaves the collaboration, the performance drop is usually minor, especially when multiple agents remain. Additionally, the performance change depends on the entered or left agent's 's capability. For example, camera-based CF and CR contribute less than LiDAR-basee S
Table A.6: Performance under different degradation ratios.
LH
L
L 128
L 1.28
L
L
L
L
L128
Degradation Ratio
AP301
AP501
AP301
AP501
AP301
AP501
0.685
0.618
0.696
0.630
0.714
0.636
0.2
0.676
0.620
0.684
0.632
0.698
0.633
0.4
0.664
0.618
0,668
0.624
0.668
0.618
0.6
0.650
0.613
0.646
0.610
0.638
0.603
0.8
0.640
0.607
0.638
0.603
0.636
0.603
18

---

MPDA
STAMP
RackAlign
CodeFilling
GenComm
Figure A.4: Visualization of detection results four different scenarios. The comparisons illustrate that method delivers more accurate detections with reduced number of false positives compared to baseline approaches. L-P and C-E, denote the L and Cg respectively.
A.4.4 Impact of degraded information exchange
In practical communication scenarios, factors such as signal interference, bandwidth limitations, and device reliability inevitably lead partial message degradation during transmission. We further evaluate the robustness of GenComm under message degradation scenarios by randomly applying zero masks with different ratios on the V2X-Real dataset. The corresponding results are presented in Table A.6 However, the degradation keeps in an acceptable range, as the ego agent's 's perception helps maintain reasonable lower bound, with AP30 dropping by most 0.078 and AP50 by 0.033. When the missing ratio is S 0.4, collaboration still improves performance. When the ratio becomes larger, the messages lose most of their useful information and become closer noise, potentially harming performance.
A.5 Visualization
Figure A.4 presents additional visual comparisons of detection results across four different scenarios, demonstrating that method achieves more precise detections with fewer false positives.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the main claims made in the abstract and introduction accurately reflect the paper's 's contributions and scope. See Section T
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
20

---

Justification:
There is no any theretical result in our paper
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the messageion needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section 5.1 and Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [Yes]
Justification: the code can be found in supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com / ssiniiiiccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: see Section 5.1 and Appendix A.3
and
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We did not conduct multiple runs obtain error bars, 3D object detection experiments on large-scale datasets are computationally intensive and tome-consuming
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
22

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section 5.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Yes, the reserch is conducted in the paper conform.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section 7 .....oontrbbuting to enhanced safety in autonomous driving and reduced risk of accidents."
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See Section 5.1
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
Our code be found in supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
25

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
26


Reviews:
[{"review_id": "WC5cC1MB2s", "reviewer": "Reviewer_bQk5", "summary": "Summary：\n\nThe paper presents a novel approach for addressing the challenges in heterogeneous multi-agent collaborative perception, particularly the problem of effectively integrating new agents with minimal computational cost and without modifying existing models. The proposed method, called GenComm, utilizes a generative communication mechanism that allows for seamless collaboration among heterogeneous agents with different sensor modalities and model architectures.", "questions": "see the weakness", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength：\n\nThe introduction of the Generative Communication mechanism (GenComm) is a novel approach to solving the problem of pragmatic heterogeneous collaboration in multi-agent systems. This method avoids retraining the core components of agents and allows for scalable integration of new agents with minimal computational overhead. One of the key strengths of the paper is its focus on scalability. The paper clearly shows that GenComm allows for the addition of new agents with minimal impact on system performance and computational resources, which is crucial for large-scale, real-world applications.\n\nWeakness：\n\nThe symbols \\mathcal{M}_{j\\rightarrow i} and \\mathcal{X}_{i} first appear in equation (1) but are not explained; additionally, there appears to be a typographical error in equation (2e).\n\nFurthermore, the authors claim that GenComm's breakthrough lies in the fact that existing methods (such as feature alignment in BackAlign) rely on architecture adaptation, whereas GenComm avoids model modification through generative feature reconstruction.\n\n\nFurthermore, while the comparison results showcase performance metrics such as AP50 and AP70, there is no in-depth analysis of why GenComm outperforms these methods. The authors should explore in which specific areas their method demonstrates advantages (e.g., reducing communication costs, decreasing parameter counts, etc.) and discuss how these improvements relate to the limitations of existing work.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nWe truly appreciate your time and effort in reviewing our submission and sincerely thank **for recognizing our proposed method and its significance for real-world deployment and applications**. Your thoughtful feedback and valuable suggestions play an important role in helping us improve this work. Below, we provide detailed responses to the concerns and questions you raised.\n\n> **W1:** The symbols $\\mathcal{M}_{j \\rightarrow i}$ and $\\mathcal{X}_{i}$ first appear in equation (1) but are not explained; additionally, there appears to be a typographical error in equation (2e).\n> \n\nThank you for pointing out this issue, and we apologize for the confusion caused during reviewing the paper. **We will add explanations for all symbols before their first appearance, correct all typos, and carefully check all equations and notations to ensure clarity and accuracy.**\n\n\"The message of agent $i$  transmitted to agent $j$ is denote as $\\mathcal{M}_{i \\rightarrow j}$, the observation input of agent $i$ is denote as $ \\mathcal{X}_i $\"\n\nThis sentence will be added to Section 3 and we will revise the formulation (2e) in the revised version as: $\\hat{O_i}=f_{dec}^i(\\mathcal{Z_i})$\n\n> **W2:** Furthermore, the authors claim that GenComm's breakthrough lies in the fact that existing methods (such as feature alignment in BackAlign) rely on architecture adaptation, whereas GenComm avoids model modification through generative feature reconstruction.\n> \n\n**BackAlign is an intrusive method** that forces collaborators to align their feature space with that of the ego agent, which leads to modifications of the collaborators’ original model parameters. This is unrealistic in real-world applications. In contrast, **GenComm avoids** model modification by employing a generative communication mechanism to bridge the semantic gap between heterogeneous agents. This presents a key advantage of GenComm over BackAlign, especially when incorporating new agents into the system.\n\n> **W3:** Furthermore, while the comparison results showcase performance metrics such as AP50 and AP70, there is no in-depth analysis of why GenComm outperforms these methods. The authors should explore in which specific areas their method demonstrates advantages (e.g., reducing communication costs, decreasing parameter counts, etc.) and discuss how these improvements relate to the limitations of existing work.\n> \n\nThank you for your comment. We will provide a detailed analysis from both performance and advantage perspectives.\n\nFirstly, the improved performance over the baselines can be mainly attributed to the following reasons:\n\n**(i)** accurate spatial information extraction by the DME (Deformable Message Extractor).\n\n**(ii)** the high-quality generation process.\n\n**(iii)** the channel enhancement introduced by the CE(Channel Enhancer) module. \n\nTogether, these components we proposed help **preserve rich semantic information** and **effectively eliminate the semantic gap** introduced by heterogeneous agents better than existing methods, result in improving overall system performance.\n\nWe **will add this analysis into Section 5.2** of the Experiments section in the revised paper:\n\n“The performance gain mainly comes from accurate spatial information extraction (DME), high-quality feature generation, and channel enhancement (CE), which together preserve rich semantics and effectively reduce the semantic gap across heterogeneous agents.” \n\nNext, we provide a detailed discussion of the advantages of GenComm compared to existing methods.\n\nGenComm is a generative communication mechanism that generates high-quality feature maps by leveraging lightweight spatial information. **The benefits of this mechanism are three-fold:**\n\n**(i)** It avoids destructing the original network by using a generative approach;\n\n**(ii)** It reduces communication bandwidth by transmitting only compact spatial cues;\n\n**(iii)** It is scalable due to minimal cost to incorporating new agents.\n\n**We can see that GenComm achieves the best performance in terms of communication and computational efficiency,** as shown in Table 2 in our paper:\n\n|  | MPDA | BackAlign | CodeFilling | STAMP | **GenComm** |\n| --- | --- | --- | --- | --- | --- |\n| **#Params(M)$\\downarrow$** | 5.75 | 31.18 | 0.81 | 1.64 | **0.31** |\n| **#GFlops(G)$\\downarrow$** | 51.93 | 211.38 | 12.91 | 3.084 | **0.615** |\n\nFurthermore, we provide a detailed comparison of our method against existing approaches in terms of key features and advantages, as summarized in the table. The comparison attributes include:\n\n**“Multi-Mod.”**: whether the method has been evaluated with **multi** heterogeneous **mod**alities;\n\n**“Multi-Enc.”**: whether the method has been evaluated with **multi** heterogeneous **enc**oders;\n\n**“Non-Intrus.”**: whether it is **Non**-**intrus**ive, i.e.,  preserves the original model parameters without modifications;\n\n**“Scal.”**: whether the method is **scal**able;\n\n**“Comm. Eff.”**: whether it is **comm**unication-**eff**icient to save bandwidth usage.\n\n| Method | Publication | Multi-Mod. | Multi-Enc. | **Non-Intrus.** | Scal. | Comm. Eff. |\n| --- | --- | --- | --- | --- | --- | --- |\n| MPDA | ICRA 2023 |  | ✓ |  |  |  |\n| HM-ViT | ICCV 2023 | ✓ |  |  |  |  |\n| HEAL | ICLR 2024 | ✓ | ✓ |  |  |  |\n| CodeFilling | CVPR 2024 | ✓ |  |  |  | ✓ |\n| PnPDA | ECCV 2024 |  | ✓ | ✓ |  |  |\n| STAMP | ICLR 2025 | ✓ | ✓ | ✓ | ✓ |  |\n| **GenComm** | - | ✓ | ✓ | ✓ | ✓ | ✓ |\n\n**GenComm demonstrates clear advantages over existing methods in terms of both practicality and scalability, making it a potential solution for real-world multi-agent systems deployment.** These advantages are difficult to achieve simultaneously with existing methods, and we believe they are crucial for advancing the deployment of multi-agent systems in real-world applications.\n\n### **To summary:**\n\n**(i)** We will carefully check for typos and notation throughout the paper.\n\n**(ii)** We have added an analysis of the performance gain in GenComm.\n\n**(iii)** We provide a comprehensive comparison of GenComm with existing methods.\n\nWe sincerely appreciate the reviewer's constructive suggestions and believe that the additional analysis addresses the concerns and significantly improves the quality of our submission."}, {"review_id": "6MmUdkKgBv", "reviewer": "Reviewer_Ltr8", "summary": "This paper presents a novel approach to facilitate multi-agent communication that optimizes over communication capacity and computing power needed to adapt to new agents. It has clear real world applications in scenarios like multi-vehicle autonomous driving. Experiments show the method's effectiveness in reducing the communication capacity requirement.", "questions": "1. How is the multi-agent performance in terms of pairwise communication volume, etc.? Will it work among multiple settings on more than 2 agents, e.g., a clique-like communication graph or a centralized dispatcher or anything else that is meaningful in real world communication?\n2. The problem itself appears to be decentralized. Are there ways / metrics to evaluate the consistency among different agent's decoding results?\n\nI will consider increasing my score if the above concerns are addressed properly.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 4, "strengths_and_weaknesses": "Strengths:\nThe paper's main result is comprehensively evaluated and seems effective. The diffusion based multi agent perception has been proven useful in other settings, so it's not surprising to see it working in the autonomous driving environment, but it's interesting to see it actually also reduces communication volume.\n\nWeaknesses:\nThere's not much evidence in the experiments part that the multi-agent setting is working well with more than 2 agents.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns except when I was asking about evaluating consistency I was actually expecting some results of those metrics. That is my problem of course for not stating it clearly in the review so I won't take that into final reviews of the paper. I decide to increase my score to 5 in answer to the authors' response of new experiments in multi-agent and real world settings.", "rebuttal": "Dear Reviewer,\n\nThank you sincerely for taking the time to review our work and **for recognizing the effectiveness and efficiency of GenComm.** We greatly value your constructive feedback and insightful comments, which are essential to enhancing the quality of our paper. Below, we carefully address the concerns and questions you have raised.\n\n> **W1:** There's not much evidence in the experiments part that the multi-agent setting is working well with more than 2 agents.\n> \n\nIn response to the weakness, we provide further evidence showing that GenComm works effectively with more than two agents.\n\nIn Table 2 of the paper, we evaluate our method on the simulation dataset OPV2V-H using **four heterogeneous agents in collaboration** :\n\n1.Agent $\\mathbf{L}_P^{64}$ using LiDAR modality with 64 beams sensor and **Point Pillar** encoder；\n\n2.Agent $\\mathbf{C}_E$ using camera modality with **EfficientNet** encoder;\n\n3.Agent $\\mathbf{L}_S^{32}$ using LiDAR modality with 32 beams sensor and **SECOND** encoder;\n\n4.Agent $\\mathbf{C}_R$ using camera modality with **ResNet** encoder.\n\nAs shown in the results, the overall performance consistently improves with the addition of each new agent. This demonstrates that our method **scales well and works effectively** in scenarios involving more than two agents.\n\nWe will change the table header of Table 2 like below to clearly show that the total agent number is increasing:\n\n| Method/Metric | $\\mathbf{L}_P^{64}+\\mathbf{C}_E$ | $\\mathbf{L}_P^{64}+\\mathbf{C}_E+\\mathbf{L}_S^{32}$ | $\\mathbf{L}_P^{64}+\\mathbf{C}_E+\\mathbf{L}_S^{32}+\\mathbf{C}_R$ |\n| --- | --- | --- | --- |\n|  | AP50/AP70 | AP50/AP70 | AP50/AP70 |\n| **GenComm** | 0.754/0.595 | 0.787/0.617 | 0.787/0.618 |\n\nWhile DAIR-V2X is a real-world dataset, it does not fully demonstrate GenComm's effectiveness with more than two agents in real-world settings. To address this, we **additionally introduce another real-world dataset V2X-REAL[1],** which contains **four agents**. We conduct experiments on V2X-REAL to further demonstrate that our model outperforms existing methods in real-world settings with more than two agents. We evaluated four different agent types: $\\mathbf{L}_H^{128},\\ \\mathbf{L}_L^{64},\\ \\mathbf{L}_M^{128},\\ \\mathbf{L}_T^{64}$(Specifically, $\\mathbf{L}$ denotes LiDAR modality, 128 indicates the number of LiDAR beams, and the subscripts $H, L, M,$ and $T$ refer to different encoder scales.). The results are below.\n\n|  | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64}$ | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64} + \\mathbf{L}_M^{128}$ | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64} + \\mathbf{L}_M^{128} + \\mathbf{L}_T^{64}$ |\n| --- | --- | --- | --- |\n|  | **AP30/AP50$\\uparrow$** | **AP30/AP50$\\uparrow$** | **AP30/AP50$\\uparrow$** |\n| **BackAlign** | 0.631/0.582 | 0.638/0.587 | 0.635/0.589 |\n| **MPDA** | 0.634/0.572 | 0.632/0.575 | 0.621/0.567 |\n| **CodeFilling** | 0.627/0.582 | 0.628/0.58 | 0.608/0.557 |\n| **GenComm** | **0.685/0.618** | **0.696/0.63** | **0.714/0.636** |\n\n*Note：STAMP needs much longer training time, we will put the complement experiments in the final version if the paper is accepted.*\n\nWe see that GenComm outperforms existing methods on V2X-Real. **We believe these newly added experiments further demonstrate GenComm is working well with more than 2 agents on real-world dataset.**\n\n> **Q1:** How is the multi-agent performance in terms of pairwise communication volume, etc.? Will it work among multiple settings on more than 2 agents, e.g., a clique-like communication graph or a centralized dispatcher or anything else that is meaningful in real world communication?\n> \n\nGenComm still maintains low communication volume even in settings with more than two agents. We provide a detailed analysis below:\n\nFirstly, we report the **communication volume per single transmission** (indicates that the collaborator sends one message to the receiver during a single communication round.) for GenComm and baseline methods in Table 1 of the paper, the result as shown below:\n\n|  | MPDA | BackAlign | CodeFilling | STAMP | **GenComm** |\n| --- | --- | --- | --- | --- | --- |\n| Comm. Volume($log_2$ scale) $\\downarrow$ | 22.0 | 22.0 | 15.0 | 22.0 | **16.0** |\n\nIn our experimental setting, GenComm achieves a **64× reduction in Communication volume** per transmission compared to MPDA, BackAlign, and STAMP. As the number of agents increases, the number of communication times increases accordingly. Given a fixed per transmission volume, the total communication cost grows linearly with the number of transmission times. Therefore, the bandwidth-saving advantage of GenComm **naturally extends to multi-agent scenarios beyond two agents.**\n\nNext, we consider several practical communication structures to further analyze the communication volume under different system:\n\n- Clique-like communication graph: Consider a clique-like communication graph consisting of $N$ agents. In this setting, each agent simultaneously acts as both a receiver (receiving messages) and a collaborator (sending messages). Consequently, the total number of communications amounts to  $N×(N+1)$.\n- Centralized dispatcher: Assume a cloud server or a designated agent acts as a centralized dispatcher that receives, processes, and forwards messages to the other $N$ agents. In this scenario, the total number of communications is  $2\\times N$.\n\nBefore introducing GenComm’s communication topology, we first elaborate on its real-world application rationale (Presented as an image in Figure 4 of the supplementary material.):\n\nIn real-world deployments, we assume three vendors: $A,B,C$, and five heterogeneous agent types: $A_1,A_2,B_1,B_2,B_3,C$, **where each agent type refers to a specific combination of sensor and model architecture.** Below we describe how GenComm can be practically applied in such scenarios:\n\n**Stage 1: Homogeneous Pre-training.**\n\nEach vendor **independently** trains their agents using GenComm in a homogeneous collaboration setting. For example, vendor $A$ trains agent $A_1$ in collaboration with other instances of $A_1$ using their **private data** belong to vendor $A$ **independently**. In this stage, the fusion network, generation module, and Channel Enhancer are trained effectively under a homogeneous scenarios.\n\n**Stage 2: Heterogeneous Collaboration.**\n\nIf two vendors reach a collaboration consensus, they can enable heterogeneous collaboration by training specific Deformable Message Extractors (DMEs) between heterogeneous agents. For instance, if vendors A and B agree to collaborate, they train DMEs such as:$DME_{A_1 \\rightarrow B_1},\\ DME_{A_1 \\rightarrow B_2},\\ ..., \\ DME_{B_3 \\rightarrow A_2}$\n\nThese lightweight modules enable heterogeneous collaboration by deploying correspond DMEs to each agent. When new agents join, vendors only need to deploy the involved DMEs **without further changes.**\n\nFrom the above, GenComm’s real-world communication structure is as follows:\n\n- GenComm Real-world Application Rational: Agents of different types from various vendors can participate in collaboration once **pairwise consensus** is established between vendors. Instead of assuming a fully connected (clique-like) communication topology, GenComm adopts a **non-fully connected communication graph**, which is more realistic for real-world deployment. In this setting, the total communication volume depends on how many collaborators each agent connects with. Suppose there are $N$ agents $V_1, V_2, ..., V_N$, and agent $V_i$ communicates with $W_i$collaborators (where $W_i \\leq N - 1$). The total number of communication times is $\\sum_{i=1}^{N} W_i$. This design enables scalable and efficient collaboration with limited bandwidth requirements in real-world application.\n\n> **Q2:** The problem itself appears to be decentralized. Are there ways / metrics to evaluate the consistency among different agent's decoding results?\n> \n\nYes, we provide our insights about the metrics to evaluate the consistency among different agent's decoding results.\n\nFor a single agent, decoding results depend on the specific downstream task. For example, 3D object detection produces bounding boxes, semantic segmentation yields pixel-level labels, and occupancy prediction outputs binary occupancy values. Each agent can independently decode and evaluate results based on its own task and local ground truth.\n\nIn perception stage (what we focus in this paper),  for the same task, **using the same evaluation metrics(mAP for 3d object detection, mIoU for segmentation) can provide a reasonable indication of consistency among agents’ decoding results.** This is because, when evaluating within the same collaboration scene, the ground truth used by all agents is identical. For instance, in a 3D object detection task, all agents use the same ground-truth bounding boxes. Similarly, in an instance segmentation task, the segmentation maps are the same. Further, **the Intersection-over-Union (IoU) of bounding boxes or segmentation maps from different agents can also be used to directly evaluate consistency.**\n\nAdditionally, consistency among agents can also be evaluated at the prediction and decision-making stages. Specifically, we can assess **whether the collaborative decisions made after multi-agent fusion converge to the same or similar conclusions.**\n\n### **To summary:**\n\n**(i)** We provide evidence and add experiments to show that GenComm works well with more than two agents.\n\n**(ii)** We provide a detailed application rationale for GenComm and analyze the communication volume in several cases.\n\n**(iii)** We discuss how to evaluate result consistency between heterogeneous agents.\n\nThank you for the constructive feedback. We believe our newly added experiments and analysis responds well to the concerns and contributes to a better version of our paper.\n\nReferences\n\n[1] Xiang et al., V2X-REAL: A Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception. ECCV 2024"}, {"review_id": "ZVWb3KNOCV", "reviewer": "Reviewer_E5U6", "summary": "This paper proposes GenComm, a generation-based framework for collaborative perception in\nmulti-agent systems. Unlike adaptation- or reconstruction-based methods, GenComm enables\nagents to exchange compact spatial messages via a Deformable Message Extractor, which are\nthen used by a conditional diffusion model to synthesize aligned features at the ego agent. A\nChannel Enhancer further refines these features before fusion. This approach preserves\nbackbone integrity, maintains semantic consistency, and scales efficiently to new agents.\nExperiments on OPV2V-H and DAIR-V2X show that GenComm achieves state-of-the-art\nperformance with lower communication and computation costs", "questions": "1. How robust is the generative feature alignment process to inaccuracies in the spatial\nmessage (e.g., due to sensor noise or misalignment)?\n2. Can the proposed framework support dynamic collaboration where agents enter/leave\narbitrarily with partial observability (e.g., missing or stale messages)? How does\nperformance degrade?\n3. Could the authors provide empirical latency comparisons for the diffusion-based generation\nvs. adaptation-based baselines?\n4. The homogeneous pretraining stage seems to assume shared access to similar agents. How\nrealistic is this in open-world scenarios with proprietary vendors?", "limitations": "Yes.", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The proposed generation-based paradigm offers a novel and effective alternative to\ntraditional reconstruction approaches, enabling scalable heterogeneous collaboration.\n2. Extensive experiments on both synthetic and real-world datasets demonstrate superior\nperformance and significant cost savings.\n\nWeaknesses\n1. Real-world evaluation is limited: DAIR-V2X involves only two agents, which restricts claims\nabout large-scale applicability.\n2. The diffusion component, while effective, may introduce latency concerns in real-time\nsettings; inference efficiency is not quantified.\n3. The framework assumes a homogeneous pretraining stage, which may not be feasible in\ndecentralized or privacy-restricted environments.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nWe truly appreciate your thoughtful review and **thank for recognizing our method is scalable, achieves superior performance, and offers significant cost savings.** Your comments and valuable feedback have been instrumental in helping us improve our work. In the following, we respond to your concerns and clarify the relevant points.\n\n> **W1:** Real-world evaluation is limited: DAIR-V2X involves only two agents, which restricts claims about large-scale applicability.\n> \n\nThank you for pointing this out. We agree that conducting experiments solely on the real-world DAIR-V2X dataset, which contains only two agents, is insufficient to support our claim of large-scale applicability. Therefore, **we extended our experiments to another real-world dataset, V2X-Real [1]**, which includes **four agents.** We evaluated four different agent types: $\\mathbf{L}_H^{128},\\ \\mathbf{L}_L^{64},\\ \\mathbf{L}_M^{128},\\ \\mathbf{L}_T^{64}$(Specifically, $\\mathbf{L}$ denotes LiDAR modality, 128 indicates the number of LiDAR beams, and the subscripts $H, L, M,$ and $T$ refer to different encoder scales.). The results are below.\n\n|  | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64}$ | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64} + \\mathbf{L}_M^{128}$ | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64} + \\mathbf{L}_M^{128} + \\mathbf{L}_T^{64}$ |\n| --- | --- | --- | --- |\n|  | **AP30/AP50** | **AP30/AP50** | **AP30/AP50** |\n| BackAlign | 0.631/0.582 | 0.638/0.587 | 0.635/0.589 |\n| MPDA | 0.634/0.572 | 0.632/0.575 | 0.621/0.567 |\n| CodeFilling | 0.627/0.582 | 0.628/0.58 | 0.608/0.557 |\n| **GenComm** | **0.685/0.618** | **0.696/0.63** | **0.714/0.636** |\n\n*Note：STAMP needs much longer training time, we will put the complement experiments in the final version if the paper is accepted.*\n\nWe see that GenComm outperforms baselines on V2X-Real. **We believe these newly added experiments further demonstrate GenComm’s scalability and potential for real-world applicability.** We will revise the conclusion make it more reasonable: \n\n“These advantages make GenComm a scalable and **potential solution** for real-world multi-agent systems deployment.”\n\n> **W2:** The diffusion component, while effective, may introduce latency concerns in real-time settings; inference efficiency is not quantified.\n> \n\n**We evaluated the inference latency of the GenComm module under different numbers of spatial message channels in ablation study.** From Figure 4 in paper, we see that the inference time of GenComm is 20.7 ms in our experiment setting, which is significantly smaller than the sensor sampling interval of 100 ms. Therefore, **we consider this latency acceptable and unlikely to hinder real-time performance.**\n\nAt the same time, we understand the reviewer’s concern that reporting only GenComm’s latency is not convincing. **To address this, we have additionally measured the extra inference time introduced by all baselines.**  Please see details in response to **Q3.**\n\n> **W3:** The framework assumes a homogeneous pre-training stage, which may not be feasible in decentralized or privacy-restricted environments.\n> \n\nTo clarify the concern, we need clearly explain the GenComm’s Real-World Application Rational first (Figure 4 in supplementary):\n\nIn real-world deployments, we assume three vendors: $A,B,C$, and five heterogeneous agent types: $A_1,A_2,B_1,B_2,B_3,C$, **where each agent type refers to a specific combination of sensor and model architecture.** Below we describe how GenComm can be practically applied in such scenarios:\n\n**Stage 1: Homogeneous Pre-training.**\n\nEach vendor **independently** trains their agents using GenComm in a homogeneous collaboration setting. For example, vendor $A$ trains agent $A_1$ in collaboration with other instances of $A_1$ using their **private data** belong to vendor $A$ **independently**. In this stage, the fusion network, generation module, and Channel Enhancer are trained effectively under a homogeneous scenarios.\n\n**Stage 2: Heterogeneous Collaboration.**\n\nIf two vendors reach a collaboration censensus, they can enable heterogeneous collaboration by training specific Deformable Message Extractors (DMEs) between heterogeneous agents. For instance, if vendors A and B agree to collaborate, they train DMEs such as:$DME_{A_1 \\rightarrow B_1},\\ DME_{A_1 \\rightarrow B_2},\\ ..., \\ DME_{B_3 \\rightarrow A_2}$\n\nThese lightweight modules enable heterogeneous collaboration by deploying correspond DMEs to each agent. When new agents join, vendors only need to deploy the involved DMEs **without further changes.**\n\nNow we clarify that each vendor can perform stage 1 **independently with their own agent types and private data. Therefore, this stage involves no cross-vendor data or model sharing and raises no privacy concerns.**\n\nIn response to **Q4,** we give more clarification about privacy concern about real-world application.\n\n> **Q1:** How robust is the generative feature alignment process to inaccuracies in the spatial message (e.g., due to sensor noise or misalignment)?\n> \n\nWe evaluated **robustness** on OPV2V-H and DAIR-V2X (Figure 3 in paper, Table 3 in supplementary material), testing **pose noise (0–0.8 std)** and **latency (0–500 ms)** that cause **spatial misalignment**. **GenComm consistently outperforms baselines**, showing robustness to imperfect, noisy inputs in realistic settings.\n\n> **Q2:** Can the proposed framework support dynamic collaboration where agents enter/leave arbitrarily with partial observability (e.g., missing or stale messages)? How does performance degrade?\n> \n\nIn Table 2 of the main paper and response to **W1**, we present experiments on OPV2V-H and V2X-Real by **dynamically** introducing additional heterogeneous agents into the system one by one, starting from a single agent and scaling up to four collaborating agents. Taking GenComm on OPV2V-H as an example, the AP70 is 0.585 with only $\\mathbf{L}_P^{64}$. Adding $\\mathbf{C}_E, \\mathbf{L}_S^{32}$, and $\\mathbf{C}_R$ increases it to 0.595 (**+0.010**), 0.617 (**+0.022**), and 0.618 (**+0.001**), showing improvements but only marginal gains as more agents join.\n\n| **Method** | $\\mathbf{L}_P^{64}$ | $\\mathbf{L}_P^{64}+\\mathbf{C}_E$ | $\\mathbf{L}_P^{64}+\\mathbf{C}_E+\\mathbf{L}_S^{32}$ | $\\mathbf{L}_P^{64}+\\mathbf{C}_E+\\mathbf{L}_S^{32}+\\mathbf{C}_R$ |\n| --- | --- | --- | --- | --- |\n|  | **AP50/AP70** | **AP50/AP70** | **AP50/AP70** | **AP50/AP70** |\n| **GenComm** | 0.738/0.585 | 0.754/0.595 | 0.787/0.617 | 0.787/0.618 |\n\n**Conversely**, If an agent leaves the collaboration, the performance drop is usually minor, especially when multiple agents remain. Additionally, the performance change depends on the entered or left agent’s capability. For example, camera-based $\\mathbf{C}_E$ and $\\mathbf{C}_R$ contribute less than LiDAR-based $\\mathbf{L}_S^{32}$.\n\nWe also conducted experiments simulating missing message scenarios by applying random zero mask of different ratios. The results are shown below:\n\n| **Missing_ratio** | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64}$ | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64} + \\mathbf{L}_M^{128}$ | $\\mathbf{L}_H^{128}+\\mathbf{L}_L^{64} + \\mathbf{L}_M^{128} + \\mathbf{L}_T^{64}$ |\n| --- | --- | --- | --- |\n|  | AP30/AP50$\\uparrow$ | AP30/AP50$\\uparrow$ | AP30/AP50$\\uparrow$ |\n| **0** | **0.685/0.618** | **0.696/0.63** | **0.714/0.636** |\n| **0.2** | 0.676/0.62 | 0.684/0.632 | 0.698/0.633 |\n| **0.4** | 0.664/0.618 | 0.668/0.624 | 0.668/0.618 |\n| **0.6** | 0.650/0.613 | 0.646/0.610 | 0.638/0.603 |\n| **0.8** | 0.640/0.607 | 0.638/0.603 | 0.636/0.603 |\n\nOverall, missing information causes a performance drop. **However, the degradation keep in an acceptable range, as the ego agent’s perception helps maintain a reasonable lower bound**, with AP30 dropping by at most 0.078 and AP50 by 0.033.\n\nWhen the missing ratio is ≤ 0.4, collaboration still improves performance. When the ratio becomes larger, the messages lose most of their useful information and closer to noise, potentially harming performance.\n\n> **Q3:** Could the authors provide empirical latency comparisons for the diffusion-based generation vs. adaptation-based baselines?\n> \n\nWe measured the additional inference time for all baselines, averaging over 100 iterations after a 10-iteration GPU warm-up to ensure reliability.\n\n|  | MPDA | BackAlign | CodeFilling | STAMP | **GenComm** |\n| --- | --- | --- | --- | --- | --- |\n| **Latency introduced(ms)$\\downarrow$** | 69.918 | 0 | 13.390 | 17.934 | 20.7 |\n\nGenComm’s latency is compared to CodeFiling and STAMP and is smaller than the sensor collection interval (100ms). **We consider this latency acceptable and unlikely to affect real-time performance.**\n\n> **Q4:** The homogeneous pretraining stage seems to assume shared access to similar agents. How realistic is this in open-world scenarios with proprietary vendors?\n> \n\nIn response to **W3**, we have provided detailes on GenComm’s real-world applicability and clarified that the homogeneous pre-training stage can be independently conducted by each vendor without raising privacy concerns.\n\n**We further discuss Stage 2 in open-world scenarios with proprietary vendors.**  Collaboration occurs only when both vendors reach on a consensus in **privacy policies.** In contrast to protocol-based adaptation approaches such as PnPDA and STAMP, which require all heterogeneous agents to adopt a unified protocol feature space, or methods like Codebook and MPDA that assume joint training across all heterogeneous agents, or BackAlign which forces other vendors to align with the ego agent’s feature space. **GenComm offers a more realistic and scalable alternative that is both practical and privacy-preserving.**\n\nAt the end, we are grateful for the reviewer’s valuable suggestions and believe that our additional experiments and analysis has effectively resolved the raised concerns and improved the submission.\n\nReference\n\n[1] Xiang et al., V2X-REAL: A Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception. ECCV 2024"}, {"review_id": "na0hzeX4sV", "reviewer": "Reviewer_EBDZ", "summary": "This paper proposes GenComm, a generative communication mechanism for heterogeneous collaborative perception. Unlike prior adaptation- or reconstruction-based approaches, GenComm uses a diffusion-based feature generation framework guided by spatial messages extracted via deformable convolutions. It aims to support collaboration across agents with different sensors and models without modifying existing networks. The method introduces lightweight extractor alignment for handling domain discrepancies and demonstrates significant computational and communication cost reductions on OPV2V-H and DAIR-V2X datasets.", "questions": "Does the proposed method support, or potentially support, heterogeneous tasks? For example, if one agent is designed for object detection and another for end-to-end driving, would the proposed method support collaboration between these two agents? The authors are not required to experimentally demonstrate this.", "limitations": "The proposed method assumes a “point-to-point” setting where all agents have a dedicated DME module designed for the ego agent, which may not be ideal in real-world scenarios. The authors need to either address this concern directly or mention it in the limitations section.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n\t1.\tThe first generation-based method for heterogeneous collaborative perception.\n\n\t2.\tThe lightweight alignment mechanism enables the integration of new agents with minimal computational overhead (81% reduction).\n\n\t3.\tGenComm does not require retraining encoders or core modules, preserving each agent’s semantic space integrity.\n\n\t4.\tComprehensive experiments on both simulated (OPV2V-H) and real-world (DAIR-V2X) datasets with consistent improvements.\n\nWeaknesses:\n\n\t1.\tSome important terms are not clearly explained in the main manuscript. For example, the regression loss is neither clearly defined nor easily inferred from the context.\n\n\t2.\tThe advantage of the proposed method over previous protocol-based methods such as PnPDA and STAMP is not clearly demonstrated. While the authors claim that protocol-based methods require consensus on a protocol space, the reviewer believes that the proposed method also requires consensus—though on the feature space of the reference (first) agent. The authors need to clarify this concern.\n\n\t3.\tThe proposed method assumes a “point-to-point” setting where all agents have a dedicated DME module designed for the ego agent, which may not be ideal in real-world scenarios. The authors need to either address this concern directly or mention it in the limitations section.", "was_revised": true, "final_justification": "The author has addressed most of my questions and concerns, including\n\nW1: Terminology now clearly defined (classification/regression losses)\n\nQ1: Thoughtful theoretical analysis of cross-task support provided\n\nW3: Critical clarification that DMEs are per agent-type, not per-agent (addressing scalability)\n\nThe following weakness is partially resolved:\n\nW2: Two-stage deployment strategy and pairwise vs. global consensus distinction are helpful, but fundamental similarity to protocol-based methods remains - both require consensus, just at different granularities\n\nThe author believes that this work overall has a solid contribution to collaborative perception. I will suggest to accept the paper.", "rebuttal": "Dear Reviewer,\n\nFirst and foremost, I would like to extend my sincere gratitude for your time and effort in reviewing the submission and **thank for recognizing our work is efficient, scalable and effective.** Your insightful comments and constructive suggestions are invaluable to refining this work. In the following, we will address your concerns regarding the weaknesses and questions.\n\n> **W1:** Some important terms are not clearly explained in the main manuscript. For example, the regression loss is neither clearly defined nor easily inferred from the context.\n> \n\nThank you for pointing this, we apologize for the confusion we made when reviewing the paper.\n\nThe classification and regression losses in our loss function refer to standard components in 3D object detection. Specifically, the classification loss is used to predict the category of each anchor, and the regression loss predicts the spatial offset from the anchor to the ground truth. \n\n **We carefully checked any term in paper to eliminate misunderstanding and we will explicitly add the following sentence to the paper and:**\n\n\"The classification and regression losses are standard 3D object detection losses, used to predict the category of each anchor and to regress the offset between the anchor and the ground truth object.\"\n\n> **W2:** The advantage of the proposed method over previous protocol-based methods such as PnPDA and STAMP is not clearly demonstrated. While the authors claim that protocol-based methods require consensus on a protocol space, the reviewer believes that the proposed method also requires consensus—though on the feature space of the reference (first) agent. The authors need to clarify this concern.\n> \n\nBefore answering the advantage of GenComm over PnPDA and STAMP, We introduce GenComm’s real-world application rationale in detail:\n\nIn real-world deployments, we assume three vendors: $A,B,C$, and five heterogeneous agent types: $A_1,A_2,B_1,B_2,B_3,C$, where each agent refers to a specific combination of sensor and model architecture. Below we describe how GenComm can be practically applied in such scenarios:\n\n**Stage 1: Homogeneous Pre-training.**\n\nEach vendor **independently** trains their agents using GenComm in a homogeneous collaboration setting. For example, vendor $A$ trains agent $A_1$ in collaboration with other instances of $A_1$ using their **private data** belong to vendor $A$ **independently**. In this stage, the fusion network, generation module, and Channel Enhancer are trained effectively under a homogeneous scenarios.\n\n**Stage 2: Heterogeneous Collaboration.**\n\nIf two vendors reach a collaboration censensus, they can enable heterogeneous collaboration by training specific Deformable Message Extractors (DMEs) between heterogeneous agents. For instance, if vendors A and B agree to collaborate, they train DMEs such as:$DME_{A_1 \\rightarrow B_1},\\ DME_{A_1 \\rightarrow B_2},\\ ..., \\ DME_{B_3 \\rightarrow A_2}$\n\nThese **lightweight** modules are then deployed to the corresponding agents, enabling seamless heterogeneous collaboration without modifying their original backbones. When new agents join the collaboration, vendors simply need to deploy the corresponding new DMEs to the involved agents—**no further modification is required.**\n\nThis two-stage process ensures that GenComm is practical, modular, and scalable in real-world multi-agent systems.\n\nFurthermore, the Figure 3 and Figure 4 in supplementary material clearly show the application rational and how the baselines are trained.\n\nRegarding the comparison with protocol-based adaptation methods such as PnPDA and STAMP, we clarify that those approaches require all heterogeneous agents to reach consensus on a shared protocol feature space, which is **challenging in open-world scenarios involving proprietary vendors.** In contrast, GenComm only requires pairwise consensus between vendors to build a specific DMEs for heterogeneous agents, rather than relying on the feature space of the reference (first) agent, as BackAlign does. This significantly relaxes the constraint—reaching consensus between any two vendors is more feasible than achieving global consensus among all vendors, enabling bidirectional communication and better scalability in open environments.\n\n\n> **W3 & Limitations:** The proposed method assumes a “point-to-point” setting where all agents have a dedicated DME module designed for the ego agent, which may not be ideal in real-world scenarios. The authors need to either address this concern directly or mention it in the limitations section.\n> \n\nWe have elaborate the details about application rational of GenComm in response to **W2.**\n\nBased on the above, we can infer that this is practical from an implementation perspective, especially since **agents of the same type, meaning those with a specific combination of sensor  and encoder architecture, share same configurations**. It is important to distinguish between **the number of heterogeneous agent types, which is usually limited and tied to vendors**, and **the total number of agents on the road, which can be very large**. The number of DMEs that one agent type needs to maintain depends only on **how many other heterogeneous types it has reached consensus with** and **does not depend on the total number of agents**.\n\nMoreover, each DME is lightweight, containing about **0.078 million parameters**, which does not introduce significant storage or computational overhead.\n\nHowever, we agree there is a limitation: communication between all heterogeneous types still needs vendors to agree on building DMEs, **which may be affected by competition and intellectual property issues**. We will clearly mention this limitation in the revised paper:\n\n\"In real-world applications, although the homogeneous pre-training stage can be independently implemented by each vendor without any privacy leakage concerns, the second stage of building specific DMEs for agent types from other vendors requires pairwise consensus between different vendors.\"\n\n> **Q1:** Does the proposed method support, or potentially support, heterogeneous tasks? For example, if one agent is designed for object detection and another for end-to-end driving, would the proposed method support collaboration between these two agents? The authors are not required to experimentally demonstrate this.\n> \n\nThank you for this question, this is an **interesting and thoughtful question that provides valuable insights for both our current work and future research.**\n\nThe answer is **Yes**, the proposed method potentially support heterogeneous tasks **theoretically.** The analysis are below:\n\nWhile we do not directly demonstrate cross-task collaboration in this work, the proposed method is inherently compatible with heterogeneous agents performing different tasks. Since our design focuses on lightweight spatial feature sharing, **each agent can independently decode the received message for its own task-specific purpose** (e.g., object detection, end-to-end planning). This flexibility makes our method suitable for real-world multi-agent systems with specialized roles.\n\nFor instance, assume a agent $\\mathbf{D}$ is trained by a single task 3d object detection, a agent $\\mathbf{M}$ is trained by multi-task 3d object detection, segmentation, occupancy prediction. a agent $\\mathbf{U}$ is trained by a unifiled end-to-end paradigm like UniAD[1].\n\nThe same point of these three agents is the intermediate feature they used are all bev feature map, but contains difference semantic information due to difference suppervise signal. These agents also can share their spatial information to other agents, **the key point is that how to effectively leaverage the recieved message to fusion with their own feature and** **enhance the feature representation.** One possible solution is, add a specific message decoder on receiver, in the GenComm’s application rational is in satge 2, traininig a specific $DME_{i \\rightarrow j}$  on agent  $i$ (as the sender), and **a specific message decoder $MD_{i \\rightarrow j}$ on agent $j$ (receiver).** If the agent can use the received information to improve its own features, then the collaboration is effective.\n\n**This interesting topic can be explored in future work to investigate how generative communication mechanisms can enable effective collaboration in task-heterogeneous scenarios.**\n\n### **To summary:**\n\n(i) we have explain the terms\n\n(ii) we give detaled application rational of GenComm to further demonstrate the advantages of GenComm over PnPDA and STAMP.\n\n(iii) We discuss the collaboration between agents with heterogeneous tasks.\n\nWe truly appreciate the reviewer’s insightful feedback, and we are confident that the provided analysis helps clarify the concerns and enhances the overall quality of our work.\n\nReferences\n\n[1] Hu et al., Planning-oriented Autonomous Driving. CVPR 2023"}]

Author rebuttal:

Meta-review: This paper proposes a new multi-agent communication method, GenComm, a generative communication mechanism for heterogeneous collaborative perception. In particular, GenComm uses a diffusion-based feature generation framework guided by spatial messages extracted via deformable convolutions. It does not need to alter the original network and employs lightweight numerical alignment of spatial information to efficiently integrate new agents at minimal cost. Evaluations on simulated and real-world driving datasets validate the performance of the method.

The rebuttal provided new experiments on V2X-Real with 4 agents, a latency comparison, and provided additional clarification/discussions on results, application rationale, and the communication paradigm in real-world scenarios. All but one reviewers recommended accept or borderline accept. The only reviewer who has a negative rating did not engage in discussion. Considering the rebuttal and the discussions, the AC agrees with the majority of the reviewers and would recommend accept.
Decision: Accept (poster)