ParCon: Noise-Robust Collaborative Perception via Multi-module Parallel Connection

## Abstract

Abstract In this paper, we investigate improving the perception performance of autonomous vehicles through communication with other vehicles and road infrastructures. To this end, we introduce a novel collaborative perception architecture, called ParCon , which connects multiple modules in parallel, as opposed to the sequential connections used in most other collaborative perception methods. Through extensive experiments, we demonstrate that ParCon inherits the advantages of parallel connection. Specifically, ParCon is robust to noise, as the parallel architecture allows each module to manage noise independently and complement the limitations of other modules.
As a result, ParCon achieves state-of-the-art accuracy, particularly in noisy environments, such as real-world datasets, increasing detection accuracy by 6.91%. Additionally, ParCon is computationally efficient, reducing floating-point operations (FLOPs) by 11.46%.

## 1 Introduction

One of the fundamental components of autonomous vehicles (AVs) is the ability to perceive various driving environments. With the advance of deep learning, perception systems of AVs have demonstrated effectiveness in various studies, including object detection (Liu et al. []; Hu et al. []; Kumar et al. []) and segmentation (Zhang et al. []; Xu et al. []; Wu et al. []). However, the perception of a single vehicle alone still has limitations caused by occlusion and limited sensor range.

To overcome these, multi-agent collaborative perception has been pivotal in enhancing perception across various environments.
In particular, V2X (Vehicle-to-Everything) communications have enabled collaborative perception among heterogeneous agents such as vehicles and road infrastructure. Recognized methodologies, such as CoBEVT (Xu et al. []), V2X-ViT (Xu et al. []), and Where2comm (Hu et al. []), have significantly improved the performance of 3D object detection.

One of the major challenges in collaborative perception is that the ego agent must communicate with surrounding agents, inevitably introducing noise during communication. However, most V2X collaborative perception models employ a sequential architecture to connect agent-wise and spatial-wise modules (see Figure 1 (a)), which tends to amplify the impact of noise as it propagates through the entire process.
To solve this problem, parallel architectures have been adopted in various fields (Kim et al. []; Kang et al. []). When applied to V2X systems, parallel architectures offer significant benefits, including improved robustness to communication noise and enhanced performance. To leverage these advantages, we propose ParCon, a novel V2X collaborative perception model for 3D object detection that features parallel connections as shown in Figure 1 (b).

Figure: Figure 1: About this paper. (a) and (b) present two architectures. (c) and (d) ParCon holds state-of-the-art performance with a reduced number of model parameters and under various noises (details in Section 4.2)
Refer to caption: extracted/5924129/Fig/Fig1.png

In this paper, we introduce our novel model, ParCon, and present extensive simulation results on various collaborative perception datasets. We demonstrate that ParCon outperforms other state-of-the-art (SOTA) approaches in detection accuracy and exhibits substantially increased robustness to communication noise (see Figure 1 (c), (d)). In particular, this robustness enables the model to achieve significantly improved detection accuracy in noisy environments, including real-world datasets. Furthermore, ParCon uses fewer parameters and is more computationally efficient than other SOTA approaches (see Figure 1 (c)).

We summarize our contributions as follows.

- •
We propose a novel parallel connection-based collaborative perception model, called ParCon, for 3D object detection. Our model achieves state-of-the-art accuracy, computation efficiency, and robustness to noise.
- •
We demonstrate the advantages of parallel connection. Namely, we highlight the robustness of our model to communication noise by analyzing it at different levels of communication latency, heading errors, and localization errors.
- •
We utilize a channel compression layer (CCL) to fuse various types of information, including agent-wise and spatial-wise, global and local information, and make the overall fusion module lightweight, thereby significantly enhancing model efficiency.

## 2 Related Work

Collaborative Perception. Intermediate fusion, which fuses compressed features rather than raw sensor data, is the primary technique enabling collaborative perception to balance performance with communication demands. Previous studies obtain and share information using knowledge distillation (Mehr et al. []), multi-module (Xu et al. []; Xu et al. []; Yang et al. []), and spatial confidence-aware communication (Hu et al. []). Also, to reduce the impact of noise, many studies have attempted to develop robust models, such as V2Vnet(Wang et al. []), V2X-ViT(Xu et al. []), FeaCo(Gu et al. []), and CoAlign(Lu et al. []).
To the best of our knowledge, these models fuse the information in series, and the potential of using a parallel connection has not been explored. In this paper, we design a new collaborative perception model that connects the information in parallel to enhance model efficiency and noise robustness.

Computer vision. The introduction of convolutional layers has revolutionized the field of computer vision;
for example, AlexNet (Krizhevsky et al. []), VggNet (Simonyan & Zisserman []), ResNet (He et al. []), etc. Convolutional layers effectively capture local features while requiring fewer parameters than fully connected layers. However, they struggle to capture relationships between distant features.
Vision Transformer (ViT) (Dosovitskiy et al. []) leverages self-attention mechanisms to effectively capture global relationships, but at the cost of high computational complexity and challenges in maintaining translation equivariance. Several approaches, such as Swin Transformer (Liu et al. []) and Neighborhood Attention Transformer (Hassani et al. []), have been introduced to solve these challenges. Similar to these approaches, our approach fully exploits the strengths of both convolution and transformer architectures to capture local features and global context, respectively.

Figure: Figure 2: Overview of our proposed collaborative perception system. Our model consists of five steps: metadata sharing, feature extraction, feature sharing, fusion module, and detection head. The details of each component are discussed in Section 3.
Refer to caption: extracted/5924129/Fig/overall_architecture.png

## 3 Method

The overall architecture of ParCon is shown in Figure 2. In this section, we introduce the five main components: 1) metadata sharing, 2) feature extraction, 3) feature sharing, 4) fusion module, and 5) detection head.

### 3.1 Metadata sharing

The ego agent is the center agent that performs the object detection tasks, and the aux agents are auxiliary agents communicating to the ego agent.
Let the number of connected agents, including the ego agent, be $L$, and the ego vehicle always has an index of 1. At metadata sharing, each $l$-th aux agent for $l\in\{2,...,L\}$ sends their agent type $t_{l}\in\{\mathrm{I},\mathrm{V}\}$, timestamp, and pose to the ego agent.
Here, the agent type $\mathrm{I}$ refers to an infrastructure and $\mathrm{V}$ to a vehicle.

### 3.2 Feature Extraction

Most V2X collaborative perception models are based on intermediate fusion, which shares features extracted from raw point-cloud data. In ParCon, the feature extraction mainly consists of 1) PointPillars and 2) Sparse Resnet Backbone.

PointPillars. PointPillars (Lang et al. [])
splits point clouds into vertical columns. This enables PointPillars to use less memory and become faster than other voxel-based approaches, such as (Zhou & Tuzel []). Also, the results of PointPillars are 2D pseudo-images. Thus, it is proper to apply a 2D convolution layer. To achieve efficient extraction, we employ PointPillars to convert point clouds of all agents to 2D pseudo-images.

Sparse Resnet Backbone. Sparsity is an important property of point clouds. The works (Graham et al. []; Yan et al. []) suggest sub-manifold sparse convolution and sparse convolution, both of which are more effective and efficient for extracting features from sparse data. They extract feature $\mathbf{F}_{l}\in\mathbb{R}^{H\times W\times C}$ from the $l$-th agent’s 2D pseudo-image where $H$ is the height, $W$ the width, and $C$ the channels. All agents use the same backbone parameters. We deploy the Sparse Resnet (SpRes) backbone (Shi et al. []; Yin et al. []; Zhu et al. []), which reduces memory usage and works more effectively than conventional dense convolution.

### 3.3 Feature Sharing

The ego agent receives the aux agents’ features through communication. The features are first compressed to satisfy communication bandwidth and latency. Then, the ego agent receives the compressed features, decompresses them, transforms them into the ego agent coordinate, crops unnecessary information, and compensates for the time delay.

Compression and Decompression. For feature compression, all agents use the same encoder and decoder parameters. After an aux agent compresses its feature and sends it to the ego agent, the ego agent restores the feature using the decoder. We design an encoder and a decoder as a $3\times 3$ convolution layer to compress the channel size depending on the compression ratio $N$. The compressed feature $\mathbf{F}^{\prime}_{l}\in\mathbb{R}^{H\times W\times(C/N)}$ and decompressed feature $\mathbf{\bar{F}}_{l}\in\mathbb{R}^{H\times W\times C}$ are determined by

$$ $\displaystyle\mathbf{F}^{\prime}_{l}=f_{\mathrm{encoder}}(\mathbf{F}_{l}), \quad l=1,...,L,$ $\displaystyle\mathbf{\bar{F}}_{l}=f_{\mathrm{decoder}}(\mathbf{F}^{\prime}_{l} ),\quad l=1,...,L.$ $$

Transformation. The ego agent transforms the decoded features into the ego agent coordinate. Also, features outside the detection range are cropped to delete unnecessary information. After this process, we use a spatial-temporal correction module (STCM) used in Xu et al. []
to compensate for the time delay caused by communication latency. We denote by $\Gamma(\cdot)$ the sequential process of transform, crop, and STCM.
By concatenating $\Gamma(\mathbf{\bar{F}_{l}})$ along the agent dimension, the multi-agent feature $\mathbf{M^{0}}\in\mathbb{R}^{L\times H\times W\times C}$ is generated. That is,

$$ $\displaystyle\mathbf{M^{0}}=\underset{l\in[1,L]}{\parallel}\Gamma(\mathbf{\bar {F}}_{l}),$ $$

where $\parallel_{l}(\cdot)$ denote the concatenation along the agent dimension.

### 3.4 Fusion Module

Figure: Figure 3: Sub-modules in ParCon. (a) Agent-wise Attention (A-Att) sub-module with HRPE. (b) Spatial-wise Attention (S-Att) sub-module. (c) Spatial-wise Convolution (S-Conv) sub-module. These modules are discussed in Section 3.4.
Refer to caption: extracted/5924129/Fig/sub_modules.png

The overall architecture of the fusion module is described in Figure 3 (b). Our fusion module in ParCon mainly consists of three sub-modules: agent-wise attention (A-Att), spatial-wise attention (S-Att), and spatial-wise convolution layer (S-Conv). Each sub-module generates inter-agent complementary information, spatial global context, and spatially detailed information, respectively. We refer to the output of each sub-module as the intent of each sub-module. To prevent the model from being too heavy due to two transformer sub-modules (A-Att, S-Att), we use a channel-wise compression layer (CCL) to generate a compressed feature which is compressed the input feature channel-wise. This is one of the key components that makes our model efficient while maintaining state-of-the-art performance. After intents are generated from sub-modules, the output feature concatenates all intents and the compressed feature.

Channel Compression Layer (CCL).
To reduce the input feature size in a simple and information-maintained manner, we design CCL as one fully connected (FC) layer that compresses the feature channel-wise.
The input to the fusion module is the multi-agent feature $\mathbf{M}^{0}$, and the input at the $d$-th depth is $\mathbf{M}^{d}\in\mathbb{R}^{L\times H\times W\times C},d=1,...,D-1$ where $D$ is the maximum depth of the fusion module. Then, the compressed feature $\mathbf{M}^{\prime}$, output of CCL, at the $d$-th depth for each $d=0,...,D-1$ is given by

$$ $\displaystyle\mathbf{M}^{\prime}=f_{\mathrm{CCL}}(\mathbf{M}^{d})\in\mathbb{R} ^{L\times H\times W\times C/4}.$ $$

Although CCL is a simple FC layer, it plays two important roles in ParCon. First, by compressing the feature channel size from $C$ to $C/4$, CCL significantly reduces the number of parameters and GFLOPs in ParCon. Second, because CCL mixes various intents to generate $\mathbf{M^{\prime}}$, it contributes to improved performance by compensating for the deficiencies of individual modules and enhancing overall robustness.

Heterogeneous Relative Pose Encoding (HRPE). Several factors, such as 1) agent type, 2) relative angle, and 3) distance, affect the point-cloud distribution and, therefore, change the feature distribution. HRPE offers these three pieces of information to consider the inter-agent relationship.

According to the $l$-th agent type $t_{l}\in\{\mathrm{I},\mathrm{V}\}$, we define the period $w_{j}$ as the inverse of
the constant hyperparameter $\tau\in\mathbb{R}$ to some powers of integer $j\in[0,C/16-1]$. In particular,
$\omega_{j}=1/(\tau^{2j+1})$ if the agent type of $l$ is infrastructure, and $\omega_{j}=1/(\tau^{2j})$ if the agent type of $l$ is vehicle.
Using the approach in Piergiovanni et al. [], we make fixed encoding values $\mathbf{p}\in\mathbb{R}^{C/4}$ of the relative angle $\theta$ and distance $d$ between the ego and aux agents as

$$ $\displaystyle\mathbf{p}[4j:4(j+1)]=[\sin(d*\omega_{j}),\cos(d*\omega_{j}),\sin (\theta*\omega_{j}),\cos(\theta*\omega_{j})],$ $$

where $*$ is the multiplication.
The fixed encoding value $\mathbf{p}$ is added to the $l$-th compressed input feature $\mathbf{M}^{\prime}_{l}\in\mathbb{R}^{H\times W\times C/4}$, and then, the encoded feature $\mathbf{M}^{\mathrm{pos}}\in\mathbb{R}^{L\times H\times W\times C/4}$ concatenates the added feature along the agent dimension. That is, $\mathbf{M}^{\mathrm{pos}}=\underset{l\in[1,L]}{\parallel}(\mathbf{M}^{\prime}_
{l}+\mathbf{p}).$

Agent-wise Attention Module (A-Att). As shown in Figure 3 (a), we use the vanilla attention module available in the code of Xu et al. [], which we denote by $\mathrm{CAV_ATT}^{d}$ at the $d$-th depth. This module does not distinguish agent type and utilizes a multi-head self-attention transformer. The output ${}\mathbf{M}^{\mathbf{AA}}$ of A-Att at the $d$-th depth is given by

$$ $\displaystyle\mathbf{M}^{\mathbf{AA}}=\begin{cases}\mathrm{CAV_ATT}^{1}( \mathbf{M}^{\mathrm{pos}})+\mathbf{M}^{\prime},&\quad d=1\\ \mathrm{CAV_ATT}^{d}(\mathbf{M}^{\prime})+\mathbf{M}^{\prime},&\quad d\neq 1. \end{cases}$ $$

We use HRPE in the first depth of the fusion module to better interpret the other agent features.

Spatial-wise Attention Module (S-Att). We use Dilated Neighborhood Attention Transformer (DiNAT) (Hassani & Shi []) to interpret a global context. DiNAT maintains translation equivariance and reduces computation burden by resembling the convolution operation. Also, it efficiently widens the receptive fields by using the dilation rate. As shown in Figure 3 (b), we use one block of the original DiNAT module and make it lighter by reducing the depth, kernel size, and dilation value of the original DiNAT to enhance the overall efficiency of the model. The output $\mathbf{M}^{\mathbf{SA}}$ of S-Att is determined by

$$ $\displaystyle\mathbf{M}^{\mathbf{SA}}$ $\displaystyle=\mathrm{DiNAT}(\mathbf{M}^{\prime})+\mathbf{M}^{\prime}.$ $$

Spatial-wise Convolution Module (S-Conv).
We use convolution layers to capture the spatial local information. As shown in Figure 3 (c), S-Conv uses three blocks of convolution layers with batch normalization and activate function ReLU. Also, we apply residual connections only to the first and second blocks while omitting them in the third block. The output $\mathbf{M}^{\mathbf{SC}}$ of S-Conv is determined by

$$ $\displaystyle\mathbf{M}^{\mathbf{SC}}=\mathrm{S_Conv}(\mathbf{M}^{\prime})$ $$

Parallel Connection.
ParCon concatenates the intents of the three sub-modules and the compressed original feature $\mathbf{M^{\prime}}$. Then, the output of $d$-th depth fusion module $\mathbf{M}^{d}$ is generated by applying a multilayer perceptron (MLP) with the residual connection. That is,

$$ $\displaystyle\mathbf{M}^{\mathbf{O}}=\mathbf{M}^{\mathbf{AA}}\parallel\mathbf{ M}^{\mathbf{SA}}\parallel\mathbf{M}^{\mathbf{SC}}\parallel\mathbf{M}^{\prime},$ $\displaystyle\mathbf{M}^{d}=\mathrm{MLP}(\mathbf{M}^{\mathbf{O}})+\mathbf{M}^{ \mathbf{O}},\quad d=1,...,D.$ $$

### 3.5 Head

After passing the last depth, we get the final output $\mathbf{M}^{D}$. We extract the ego agent output $\mathbf{M}^{D}_{1}$ and apply two $1\times 1$ convolution layers, $f^{\mathrm{cls}}_{\mathrm{head}}(\cdot)$ and $f^{\mathrm{reg}}_{\mathrm{head}}(\cdot)$, for the detection box classification and regression, respectively. Specifically, $f^{\mathrm{cls}}_{\mathrm{head}}(\cdot)$ makes the classification output tensor $\mathbf{\hat{Y}}_{\mathrm{cls}}\in\mathbb{R}^{H\times W\times 2}$ that identifies whether an object is a vehicle or not, and $f^{\mathrm{reg}}_{\mathrm{head}}(\cdot)$ makes the regression output tensor $\mathbf{\hat{Y}}_{\mathrm{reg}}\in\mathbb{R}^{H\times W\times 14}$ that contains center of boxes $(x,y,z)$, size of boxes $(w,l,h)$ and heading of vehicle $\phi$. We use the total loss $\mathcal{L}_{\mathrm{total}}$ = $\mathcal{L}_{\mathrm{cls}}+\mathcal{L}_{\mathrm{reg}}$ where $\mathcal{L}_{\mathrm{cls}}$ is calculated using $\mathbf{\hat{Y}}_{\mathrm{cls}}$ and focal loss (Lin et al. []) and $\mathcal{L}_{\mathrm{reg}}$ is calculated using $\mathbf{\hat{Y}}_{\mathrm{reg}}$ and the smooth L1 loss (Berrada et al. []).

## 4 Experiments

### 4.1 Experimental Settings

Comparison Models and Datasets.
We compare our proposed architecture with state-of-the-art (SOTA) models, CoBEVT Xu et al. [], V2X-ViT Xu et al. [], Where2comm Hu et al. [], CoAlign Lu et al. [], on the datasets of V2XSet (Xu et al. []), OPV2V (Xu et al. []), and DAIR-V2X (Yu et al. []). V2XSet is a simulated dataset, including V2X scenarios, using the CARLA simulator (Dosovitskiy et al. []) and OpenCDA (Xu et al. []). OPV2V is a simulated dataset, only including V2V scenarios. Also, DAIR-V2X is a real-world dataset, including V2X scenarios, collected from one actual vehicle and one road infrastructure. We set the LiDAR detection range as $x\in[-140.8,140.8]$ and $y\in[-38.4,38.4]$ on V2XSet and OPV2V, and $x\in[-102.4,102.4]$ and $y\in[-38.4,38.4]$ on DAIR-V2X. A detailed description of the datasets and models is in Appendix A.1 and A.2.

Evaluation Metrics and Noise Settings. We calculate the Average Precision (AP) at the Intersection-over-Union (IoU) thresholds of 0.5 and 0.7 to evaluate 3D detection accuracy and use the number of parameters (# Params) and GFLOPs to compare model efficiency.
To consider communication noise while sharing the feature of connected agents, we add communication noise to training and inference data, which includes latency, heading noises, and localization noises.
The latency is based on the total time delay used in Xu et al. [] and is set to follow uniform distribution $U(0,t_{\mathrm{lag.}})$ with the maximum latency $t_{\mathrm{lag.}}$. The heading and localization noises follow normal distributions with zero mean and standard deviations $\sigma_{\mathrm{hdg.}}$ and $\sigma_{\mathrm{loc.}}$, that is, $\mathcal{N}(0,\sigma_{\mathrm{hdg.}}^{2})$ and $\mathcal{N}(0,\sigma_{\mathrm{loc.}}^{2})$, respectively.
We use three different noise settings: perfect, mild noise, and harsh noise. In the perfect setting, we let $t_{\mathrm{lag.}}=0,\sigma_{\mathrm{hdg.}}=0,\sigma_{\mathrm{loc.}}=0$, and in the mild noise setting, we let $t_{\mathrm{lag.}}=200,\sigma_{\mathrm{hdg.}}=0.2,\sigma_{\mathrm{loc.}}=0.2$. In the harsh noise setting, we use different values of $t_{\mathrm{lag.}}\in[0,500],\sigma_{\mathrm{hdg.}}=[0,1.0],\sigma_{\mathrm{loc
.}}=[0,0.5]$. Also, we train three types of models based on the noise settings: perfect, noise, and fine-tuned. The perfect model is trained in the perfect setting, the noise model in the simple noise setting (see Appendix A.3 for details), and the fine-tuned model fine-tunes the perfect model with the mild noise setting. We validate the perfect model, noise model, and fine-tuned model with the perfect setting, mild noise setting, and harsh noise setting, respectively.

Training Parameters. For a fair comparison, we train both our models and SOTA models using AdamW (Loshchilov & Hutter []), with a learning rate of 3e-4 and a weight decay of 0.01. We use the same learning scheduler, Cosine Annealing Warm-Up Restarts (Loshchilov & Hutter []), applying a warm-up learning rate of 2e-4. Regarding V2XSet and OPV2V, we train the models for up to 40 epochs and 10 warm-up epochs. We also train the models for up to 20 epochs and 5 warm-up epochs for DAIR-V2X. The models are trained and validated on an RTX 4090.

Model Parameters. After the feature extraction, we compress the channel size of the feature from $C=256$ to $C/N=8$ for the feature sharing. We set the channel size of multi-agent feature $\mathbf{M^{0}}$ as 256. Regarding the fusion module, HRPE in A-att utilizes the relative angle $\theta$ and distance $d$.
For V2XSet and OPV2V, we divide $\theta$ and $d$ into 20 ⁢ ° 20° 20 ⁢ ° and 25 m, and into 10 ⁢ ° 10° 10 ⁢ ° and 15 m for DAIR-V2X. In S-Att, the lightweight DiNAT (Hassani & Shi []) features two depths. For DiNAT’s hyperparameters, we use a $7\times 7$ kernel with dilation rates of 4 and 2 applied to each depth, respectively. The S-Conv comprises three blocks, each consisting of a convolution layer with a $3\times 3$ kernel.

**Table 1: Comparison of detection accuracy on V2XSet, OPV2V, and DAIR-V2X.**
| Model | Perfect | Mild Noise |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| V2XSet | OPV2V | DAIR-V2X | V2XSet | OPV2V | DAIR-V2X |  |
| AP@0.5/0.7 | AP@0.5/0.7 | AP@0.5/0.7 | AP@0.5/0.7 | AP@0.5/0.7 | AP@0.5/0.7 |  |
| No Fusion | 0.754 / 0.602 | 0.659 / 0.545 | 0.562 / 0.452 | 0.754 / 0.602 | 0.659 / 0.545 | 0.562 / 0.452 |
| Late Fusion | 0.885 / 0.764 | 0.852 / 0.768 | 0.573 / 0.397 | 0.844 / 0.552 | 0.726 / 0.539 | 0.549 / 0.364 |
| CoBEVT | 0.880 / 0.825 | 0.852 / 0.787 | 0.570 / 0.452 | 0.787 / 0.600 | 0.698 / 0.522 | 0.550 / 0.407 |
| V2X-ViT | 0.866 / 0.773 | 0.843 / 0.763 | 0.603 / 0.455 | 0.811 / 0.648 | 0.755 / 0.615 | 0.572 / 0.409 |
| Where2comm | 0.904 / 0.835 | 0.858 / 0.789 | 0.594 / 0.429 | 0.852 / 0.668 | 0.760 / 0.576 | 0.562 / 0.384 |
| CoAlign | 0.888 / 0.799 | 0.819 / 0.741 | 0.578 / 0.433 | 0.816 / 0.605 | 0.726 / 0.539 | 0.557 / 0.397 |
| ParCon (Ours) | 0.891 / 0.839 | 0.888 / 0.830 | 0.613 / 0.482 | 0.850 / 0.707 | 0.790 / 0.649 | 0.596 / 0.438 |

**Table 2: Comparison of efficiency.**
| Table 2: Comparison of efficiency.<br>Model<br># Params<br>GFLOPs<br>CoBEVT<br>11.14M<br>213<br>V2X-ViT<br>14.17M<br>287<br>Where2comm<br>8.69M<br>177<br>CoAlign<br>12.94M<br>96<br>ParCon(Ours)<br>5.57M<br>85<br>Table 3: Performance comparison between ParCon and ParCon-S at AP@0.7. ParCon-S is the sequential architecture model using the same sub-modules of ParCon.<br>Model<br>V2XSet<br>OPV2V<br>DAIR-V2X<br>ParCon<br>0.707<br>0.649<br>0.438<br>ParCon-S<br>0.690<br>0.642<br>0.430 |
| --- |

Figure: Figure 4: Robustness in various noise ranges on V2XSet, OPV2V, and DAIR-V2X.
Refer to caption: extracted/5924129/Fig/Harsh_Noise_comp.png

Figure: Figure 5: Robustness comparison between ParCon and ParCon-S in various noise ranges on V2XSet. ParCon-S is the sequential architecture model using the same sub-modules of ParCon.
Refer to caption: extracted/5924129/Fig/Par_Seq_HN.png

### 4.2 Quantitative Evaluation

Detection Accuracy.
Table 1 compares the proposed ParCon with the other SOTA methods on the simulation datasets, V2XSet and OPV2V, and the real-world dataset, DAIR-V2X. We also consider single-agent detection (No Fusion) and Late Fusion. We observe the following. i) In the simulated datasets, in the mild noise setting, ParCon achieves the SOTA accuracy increased by 5.83% compared with Where2comm on V2XSet and by 5.52% compared with V2X-ViT on OPV2V at AP@0.7. ii) Even in the real-world dataset, in the mild noise setting, ParCon achieves the SOTA accuracy increased by 6.91% compared with V2X-ViT at AP@0.7. In summary, our proposed collaborative perception model, ParCon, outperforms the SOTA models on all the datasets.
iii) Our model achieves the highest performance at AP@0.7, except in the mild noise setting of the DAIR-V2X dataset. In this specific case, the No Fusion approach shows the highest performance. This is because the DAIR-V2X dataset includes only one vehicle and one infrastructure, providing limited opportunity for improvement through collaboration. Thus, all the fusion methods, including ours, do not outperform the No Fusion approach in this scenario.

Model Efficiency.
Table 3 compares model efficiency in terms of # Params and GFLOPs. The # Params in ParCon is reduced by 56.95% compared with CoAlign and by 60.69% compared with V2X-ViT. The GFLOPs of ParCon is reduced by 11.46% compared with CoAlign and 70.38% compared with V2X-ViT. The efficiency stems from our CCL, which compress the channel size from 256 to 64.

Noise Robustness.
In Figure 4, we compare the noise robustness on the V2XSet, OPV2V, and DAIR-V2X datasets. ParCon always outperforms the SOTA models across all the datasets, even in the real-world dataset DAIR-V2X. At the same time, ParCon shows strong noise robustness (the lowest accuracy drop) among the SOTA models on the simulated dataset.
From zero noise to the maximum noise value, the accuracy (AP@0.7) of ParCon decreases by 4.90%/3.08%/10.01% on V2XSet and by 3.78%/4.32%/13.41% on OPV2V under latency/heading noise/localization noise, respectively. In contrast, V2X-ViT experiences larger drops of 9.78%/3.38%/13.62% on V2XSet and 7.07%/3.95%/15.18% on OPV2V. Similarly, Where2comm shows even greater declines, with decreases of 8.42%/5.37%/18.42% on V2XSet and 5.48%/8.12%/24.55% on OPV2V. You can see detailed values in Appendix A.5.

Importance of Parallel Connection.
To identify the effectiveness of parallel connection, we design the sequential architecture model, ParCon-S, which uses the same sub-modules of ParCon and connects them in series. Similar to ParCon, ParCon-S compresses the multi-agent feature with CCL and uses a fully connected layer to restore the channel size of the output feature to match the channel size of the original feature. Table 3 and Figure 5 show the comparison in terms of detection accuracy and noise robustness between ParCon and ParCon-S, respectively. As shown in Table 3, the parallel model always outperforms the sequential model at AP@0.7. The parallel model enhances the performance by 2.38%, 1.17%, and 1.82% compared to the sequential model on the V2XSet, OPV2V, and DAIR-V2X datasets, respectively. Furthermore, as shown in Figure 5, the parallel model outperforms the sequential model across various noise levels. In the harsh noise setting,
the accuracy of ParCon drops by 4.91%/3.08%/10.08% under latency/heading noise/localization noise, respectively, while ParCon-S experiences larger drops of 5.46%/3.51%/11.01% (See Appendix A.5 for detail values).
This comparison demonstrates that the parallel connection consistently outperforms the sequential connection across all datasets and noise settings.

### 4.3 Qualitative Evaluation

**Table 4: Summation of the absolute value of CCL weights in the perfect setting.**
| Figure 6: Visualization of the weights of CCL.<br>Table 4: Summation of the absolute value of CCL weights in the perfect setting.<br>Section<br>Depth 1<br>Depth 2<br>Depth 3<br>Sec.1<br>190.153<br>104.393<br>59.076<br>Sec.2<br>190.497<br>117.089<br>71.186<br>Sec.3<br>191.221<br>142.863<br>121.540<br>Sec.4<br>189.761<br>138.970<br>81.123<br>Table 5: Summation of the absolute value of CCL weights under the noise setting.<br>Section<br>Depth 1<br>Depth 2<br>Depth 3<br>Sec.1<br>228.254<br>103.984<br>48.986<br>Sec.2<br>226.726<br>117.295<br>70.545<br>Sec.3<br>224.471<br>172.468<br>145.967<br>Sec.4<br>221.155<br>158.333<br>84.261 |
| --- |

Investigation of CCL Weights. At CCL, the concatenated intents ($\mathbf{M}^{\mathbf{AA}}\parallel\mathbf{M}^{\mathbf{SA}}\parallel\mathbf{M}^{
\mathbf{SC}}\parallel\mathbf{M^{\prime}}$) are fused each other channel-wise. As we visualize the weights of CCL in Figure 6, it is noteworthy that after the second depth, the weights appear divided into four sections corresponding to the concatenated intents. Also, we observe that i) as shown in Table 5 and Table 5, the summation of the absolute weights in the third section of CCL, corresponding to the intent of S-Conv, shows the highest score, and ii) the summation of the absolute weights in the third section of CCL in the fine-tuned model is higher than in the perfect model.
These observations suggest that S-Conv, which generates spatial local information, plays a crucial role in noise rejection, and that CCL leverages this by focusing more on the intent of S-Conv. In essence, CCL aggregates the intents of specific effective modules to enhance overall performance and compensate for the limitations of other modules.

Visualization of Detection Results. Figure 7 shows the detection visualization of CoBEVT, V2V-ViT, and Where2comm in V2V and V2X scenarios of V2XSet. Our model accurately predicts bounding boxes that are well-aligned with the ground truth and successfully detects objects under occlusion. In contrast, other models display misalignments and fail to detect objects when occlusion occurs.

### 4.4 Ablation Studies

**Table 6: Effect of compression ratio at AP@0.7.**
| Table 6: Effect of compression ratio at AP@0.7.<br>Rate<br>Perfect<br>Mild Noise<br># Params<br>GFLOPs<br>$\times$1<br>0.839<br>0.670<br>19.69M<br>451<br>$\times$2<br>0.838<br>0.687<br>8.74M<br>161<br>$\times$4 (Ours)<br>0.850<br>0.707<br>5.57M<br>85<br>$\times$8<br>0.851<br>0.689<br>5.00M<br>64<br>Table 7: Effect of HRPE in the mild noise setting on V2XSet.<br>HRPE<br>AP@0.5<br>AP@0.7<br>✓<br>0.850<br>0.707<br>0.844<br>0.694 |
| --- |

**Table 8: Effect of sub-modules in the mild noise setting.**
| Table 8: Effect of sub-modules in the mild noise setting.<br>A-Att<br>S-Att<br>S-Conv<br>Original<br>AP@0.5<br>AP@0.7<br>✓<br>✓<br>✓<br>✓<br>0.850<br>0.707<br>✓<br>✓<br>✓<br>0.774<br>0.645<br>✓<br>✓<br>✓<br>0.848<br>0.702<br>✓<br>✓<br>✓<br>0.813<br>0.663<br>✓<br>✓<br>✓<br>0.842<br>0.683<br>Table 9: Ablation study of S-Att. Whether to apply the S-Att or vanilla model (ViT) in the mild noise setting on V2XSet.<br>S-Att<br>AP@0.5<br>AP@0.7<br># Params<br>GFLOPs<br>✓<br>0.850<br>0.707<br>5.57M<br>85<br>0.809<br>0.660<br>11.79M<br>75 |
| --- |

Figure: Figure 7: Visualization of detected boxes in the V2XSet Dataset. The green boxes represent the ground truth, while the red ones represent prediction. Our model ParCon shows more precise detection.
Refer to caption: extracted/5924129/Fig/Visualization_detection.png

CCL compression ratio.
We train ParCon with various compression rates in CCL to validate whether the reduction in feature size causes information loss. Table 7 shows that ParCon with the rate of $\times$4 yields the best performance at AP@0.7.
Compared with Parcon with the rate of $\times$1, Parcon with the rate of $\times$4 yields the accuracy enhanced by 5.50% in the mild noise setting, and the number of parameters and GFLOPs decreased by 71.68% and 81.15%, respectively. Based on this result, we select the rate of $\times$4 as the final setting.

Effect of HRPE. We use HRPE in A-Att to help the module understand the features of the other agents. Table 7 shows that HRPE enhances the performance of the ParCon in the mild noise setting. We notice that HRPE does not contribute much in the perfect setting. By appropriately setting the two hyperparameters for dividing $\theta$ and $d$, HRPE makes our model robust to noise.

Effect of sub-modules.
We present the ablation study results in Table 9. The inter-agent complementary information, which is related to A-Att, affects the performance of ParCon significantly. The performance AP@0.7 decreases by 8.76% by eliminating A-Att. The next contributing information is due to S-Conv; if S-Conv is absent, the performance AP@0.7 decreases by 6.12%.

Effect of S-Att. In Table 9, we compare our S-Att module with ViT (Dosovitskiy et al. []), which is vanilla model of S-Att. Although using S-Att reduces the number of parameters by 52.76%, it enhances the performance of AP@0.7 by 7.10% in the mild noise setting.

## 5 Conclusion

This paper has proposed a new collaborative perception model, ParCon, featuring the parallel connection of sub-modules. With the parallel architecture, ParCon effectively handles communication noises, such as latency, heading noise, and localization noise. Also, ParCon is efficient in terms of the number of parameters and GFLOPs as it uses CCL and S-Att. CCL reduces the feature size to make our fusion module highly lightweight with enhanced accuracy under the noise. We adopt DiNAT (Hassani & Shi []) to capture the spatial global context in S-Att, which enhances the performance while simultaneously reducing the number of parameters compared to the vanilla model, ViT (Dosovitskiy et al. []). Our work provides an important guideline for designing new collaborative perception architectures by demonstrating the benefits of parallel architecture, CCL, and S-Att, as well as their effectiveness and efficiency in handling communication noise.

Limitation and future work.
To focus on the effectiveness of the parallel architecture, we use the same sub-modules as in other SOTA models, with the exception of S-Att. In future work, we plan to design advanced sub-modules that can enhance synergy with the parallel architecture.

## Appendix A Appendix

### A.1 Dataset.

We use the V2XSet Xu et al. [], OPV2V Xu et al. [], and DAIR-V2X Yu et al. [] datasets to train and validate models in both V2V and V2X scenarios.

V2XSet. V2XSet is a simulated dataset supporting V2X perception, co-simulated using CARLA Dosovitskiy et al. [] and OpenCDA Xu et al. []. It comprises 73 scenes with a minimum of 2 to 5 connected agents and includes 11K 3D annotated LiDAR point cloud frames. The training, validation, and testing sets consist of 6.7K, 2K, and 2.8K frames, respectively.

OPV2V. OPV2V designed for multi-agent V2V perception. Each frame typically comprises approximately 3 CAVs, with a minimum of 2 and a maximum of 7. It includes 10.9K LiDAR point cloud frames with 3D annotations. The training, validation, and testing splits include 6.8K, 2K, and 2.2K frames, respectively.

DAIR-V2X. DAIR-V2X is a real-world dataset for collaborative perception. The dataset includes 9K frames from a vehicle and a road infrastructure, which is equipped with both a LiDAR and a 1920x1080 camera. The LiDAR of infrastructure is 300-channel, and the vehicle’s is 40-channel.

### A.2 Comparison Models.

We select CoBEVT Xu et al. [], V2X-ViT Xu et al. [], Where2comm Hu et al. [], CoAlign Lu et al. [] to compare ParCon.

CoBEVT.
CoBEVT presents a framework for multi-agent multi-camera perception that collaboratively produces BEV map predictions using both camera and LiDAR.

V2X-ViT.
V2X-viT attempts to utilize heterogeneous agents, such as vehicles and road infrastructure, and solve the problem of heterogeneity between them using a transformer with a graph structure.

Where2comm.
Where2comm introduces a spatial confidence map, enabling agents to share only spatially sparse yet perceptually important information.

CoAlign.
CoAlign introduces a hybrid framework combining intermediate and late fusion, and leverages an agent-object pose graph to align and optimize relative poses between agents, making the system more robust to noise introduced by pose estimation errors.

### A.3 Noise Settings and Trained Models.

We define four different types of noise as shown in Table 11, and three different types of trained model as shown in Table 11.
The latency is based on the total time delay used in Xu et al. [] and is set to follow uniform distribution $U(0,t_{\mathrm{lag.}})$ with the maximum latency $t_{\mathrm{lag.}}$. The heading and localization noises follow normal distributions with zero mean and standard deviations $\sigma_{\mathrm{hdg.}}$ and $\sigma_{\mathrm{loc.}}$, that is, $\mathcal{N}(0,\sigma_{\mathrm{hdg.}}^{2})$ and $\mathcal{N}(0,\sigma_{\mathrm{loc.}}^{2})$, respectively.
We use three different noise settings: Perfect, Mild Noise, and Harsh Noise. In the perfect setting, we let $t_{\mathrm{lag.}}=0,\sigma_{\mathrm{hdg.}}=0,\sigma_{\mathrm{loc.}}=0$, and in the mild noise setting, we let $t_{\mathrm{lag.}}=200,\sigma_{\mathrm{hdg.}}=0.2,\sigma_{\mathrm{loc.}}=0.2$. In the harsh noise setting, we use different values of $t_{\mathrm{lag.}}\in[0,500],\sigma_{\mathrm{hdg.}}=[0,1.0],\sigma_{\mathrm{loc
.}}=[0,0.5]$. Also, we train the three types of models based on the noise setting used in the training: Perfect, Noisy, and Fine-tuned. The perfect model is trained with the perfect setting, the noisy model with the simple noise setting, and the fine-tuned model is fine-tuning the perfect model with the mild noise setting.

| Table 10: Various types of noise setting.<br>Noise Setting<br>Latency<br>Hdg. Noise<br>Loc. Noise<br>Perfect<br>0<br>0<br>0<br>Simple Noise<br>100<br>$\mathcal{N}(0,0.2^{2})$<br>$\mathcal{N}(0,0.2^{2})$<br>Mild Noise<br>$U(0,200)$<br>$\mathcal{N}(0,0.2^{2})$<br>$\mathcal{N}(0,0.2^{2})$<br>Harsh Noise<br>$U(0,t_{\mathrm{lag.}})$<br>$\mathcal{N}(0,\sigma_{\mathrm{hdg.}}^{2})$<br>$\mathcal{N}(0,\sigma_{\mathrm{loc.}}^{2})$<br>$t_{\mathrm{lag.}}\in[0,500]$<br>$\sigma_{\mathrm{hdg.}}\in[0.0,1.0]$<br>$\sigma_{\mathrm{loc.}}\in[0.0,0.5]$<br>Table 11: Various types of trained model.<br>Training Type<br>Method<br>Perfect<br>Training model in the perfect setting.<br>Noise<br>Training model in the simple noise setting.<br>Fine-tuned<br>First, training model in the perfect setting.<br>Then, fine-tuning model in the mild noise setting. |
| --- |

### A.4 Attention Weights Visualization.

In Figure 8, we visualize the attention weights in A-Att, where the brighter colors indicate the areas that the ego agent focuses on. Some observations to notice are as follows: i) Our models show a clear difference between the regions that are paid attention to or not, compared to the V2X-ViT’s attention weights (Figure 8-(b)). ii) Our models have different attention tendencies. For ParCon in Figure 8-(d), the ego agent relies on its own confidence regions where its point clouds are dense and not occluded. Also, it relies on the aux agent’s confidence regions to incorporate wide-range information. In the sequential model in Figure 8-(c), the ego agent focuses on the areas where objects densely exist.
The tendencies are more explicitly identified in the infrastructure data, which has a wider detection range but is sparse and can be noisy at far distances. In ParCon, the ego agent takes the infra data relatively less, while the ego agent in the sequential model utilizes the infrastructure data heavily because the infra data has lower confidence than the data from nearby aux agents.

Figure: Figure 8: Aggregated LiDAR points and attention map. The white circle is the ego agent, and the blue and green circles are aux agents. Small grey circles are vehicles in the detection range.
Refer to caption: extracted/5924129/Fig/Scenario_3_w_colormap_v2.png

### A.5 Noise Robustness.

As shown in Table 12, 13, 14, and 18, we show the detailed values used in Section 4.2, which is ParCon outperforms the SOTA models across all noise ranges.

To support our claims of noise robustness, we analyze an additional comparison, and then divide the noise set into two subsets: 1) normal noise and 2) severe noise. The normal noise subset is based on the normal range defined in the work Xu et al. []. Hence, we define a normal noise subset as $t_{\mathrm{lag.}}\in[100\,\mathrm{ms},200\,\mathrm{ms}]$, $\sigma_{\mathrm{loc.}}\in[0.1\,\mathrm{m},0.2\,\mathrm{m}]$, and $\sigma_{\mathrm{hdg.}}\in[$$,$$]$, and the severe noise subset as $t_{\mathrm{lag.}}\in[300\,\mathrm{ms},500\,\mathrm{ms}]$, $\sigma_{\mathrm{loc.}}\in[0.3\,\mathrm{m},0.5\,\mathrm{m}]$, and $\sigma_{\mathrm{hdg.}}\in[$$,$$]$. In Table 15, 16, 17, and 19, we evaluate performance by averaging each component in the subset.

Normal Noise. Under normal noise conditions, ParCon outperforms SOTA models in terms of latency, heading, and localization noise. Moreover, ParCon demonstrates the lowest sensitivity to noise, decreasing by 0.03%, 0.18%, and 1.81& on V2XSet, 1.23%, 0.77%, and 2.28% on OPV2V under latency/heading noise/localization noise, and 0.92%, 3.49% on DAIR-V2X under heading/localization noise, repectively. Our models exhibit performance enhancements regarding localization and heading noise, while most SOTA models show reduced performance.

Severe Noise. Our models always outperform the SOTA models in severe noise conditions. Also, our models show the lowest sensitivity to severe noise. ParCon has low drop rate by 4.01%/2.13%/7.75% on V2XSet and by 3.38%/3.32%/10.29% on OPV2V under latency/heading noise/localization noise, respectively. Although the drop rate of our method is not always the lowest in some dataset, ParCon always keeps SOTA performance.

**Table 12: Robustness to various ranges of noises with detailed values on V2XSet.**
| Model | Maximum Latency ($t_{\mathrm{lag.}}$) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| 0 ms | 100 ms | 200 ms | 300 ms | 400 ms | 500 ms |  |
| CoBEVT | 0.764 | 0.759 | 0.738 | 0.712 | 0.696 | 0.684 |
| V2X-ViT | 0.747 | 0.738 | 0.721 | 0.701 | 0.683 | 0.673 |
| Where2comm | 0.741 | 0.741 | 0.722 | 0.697 | 0.688 | 0.679 |
| CoAlign | 0.718 | 0.711 | 0.695 | 0.677 | 0.670 | 0.665 |
| ParCon (Ours) | 0.760 | 0.765 | 0.754 | 0.737 | 0.729 | 0.723 |
| Model | Heading Noise std. ($\sigma_{\mathrm{loc.}}$) |  |  |  |  |  |
| 0.0 ⁢ ° 0.0° 0.0 ⁢ ° | 0.2 ⁢ ° 0.2° 0.2 ⁢ ° | 0.4 ⁢ ° 0.4° 0.4 ⁢ ° | 0.6 ⁢ ° 0.6° 0.6 ⁢ ° | 0.8 ⁢ ° 0.8° 0.8 ⁢ ° | 1.0 ⁢ ° 1.0° 1.0 ⁢ ° |  |
| CoBEVT | 0.764 | 0.766 | 0.760 | 0.750 | 0.735 | 0.721 |
| V2X-ViT | 0.747 | 0.748 | 0.745 | 0.739 | 0.731 | 0.721 |
| Where2comm | 0.741 | 0.740 | 0.731 | 0.720 | 0.710 | 0.701 |
| CoAlign | 0.718 | 0.717 | 0.712 | 0.705 | 0.698 | 0.691 |
| ParCon (Ours) | 0.760 | 0.760 | 0.757 | 0.751 | 0.744 | 0.737 |
| Model | Localization Error std. ($\sigma_{\mathrm{hdg.}}$) |  |  |  |  |  |
| 0.0 m | 0.1 m | 0.2 m | 0.3 m | 0.4 m | 0.5 m |  |
| CoBEVT | 0.764 | 0.752 | 0.711 | 0.634 | 0.545 | 0.465 |
| V2X-ViT | 0.747 | 0.737 | 0.717 | 0.693 | 0.668 | 0.645 |
| Where2comm | 0.741 | 0.730 | 0.700 | 0.661 | 0.628 | 0.605 |
| CoAlign | 0.718 | 0.703 | 0.671 | 0.632 | 0.599 | 0.573 |
| ParCon (Ours) | 0.760 | 0.754 | 0.738 | 0.719 | 0.700 | 0.683 |

**Table 13: Robustness to various ranges of noises with detailed values on OPV2V.**
| Model | Maximum Latency ($t_{\mathrm{lag.}}$) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| 0 ms | 100 ms | 200 ms | 300 ms | 400 ms | 500 ms |  |
| CoBEVT | 0.659 | 0.656 | 0.646 | 0.630 | 0.612 | 0.593 |
| V2X-ViT | 0.685 | 0.682 | 0.677 | 0.670 | 0.664 | 0.658 |
| Where2comm | 0.675 | 0.673 | 0.661 | 0.647 | 0.632 | 0.621 |
| CoAlign | 0.638 | 0.634 | 0.624 | 0.614 | 0.603 | 0.593 |
| ParCon (Ours) | 0.713 | 0.711 | 0.705 | 0.697 | 0.690 | 0.683 |
|  | 0.0 ⁢ ° 0.0° 0.0 ⁢ ° | 0.2 ⁢ ° 0.2° 0.2 ⁢ ° | 0.4 ⁢ ° 0.4° 0.4 ⁢ ° | 0.6 ⁢ ° 0.6° 0.6 ⁢ ° | 0.8 ⁢ ° 0.8° 0.8 ⁢ ° | 1.0 ⁢ ° 1.0° 1.0 ⁢ ° |
| CoBEVT | 0.659 | 0.656 | 0.646 | 0.630 | 0.612 | 0.593 |
| V2X-ViT | 0.685 | 0.682 | 0.677 | 0.670 | 0.664 | 0.658 |
| Where2comm | 0.675 | 0.673 | 0.661 | 0.647 | 0.632 | 0.621 |
| CoAlign | 0.638 | 0.634 | 0.624 | 0.614 | 0.603 | 0.593 |
| ParCon (Ours) | 0.713 | 0.711 | 0.705 | 0.697 | 0.690 | 0.683 |
| Model | Localization Error std. ($\sigma_{\mathrm{hdg.}}$) |  |  |  |  |  |
| 0.0 m | 0.1 m | 0.2 m | 0.3 m | 0.4 m | 0.5 m |  |
| CoBEVT | 0.659 | 0.639 | 0.575 | 0.490 | 0.416 | 0.351 |
| V2X-ViT | 0.685 | 0.679 | 0.575 | 0.490 | 0.416 | 0.351 |
| Where2comm | 0.675 | 0.661 | 0.615 | 0.569 | 0.536 | 0.510 |
| CoAlign | 0.638 | 0.621 | 0.569 | 0.517 | 0.480 | 0.456 |
| ParCon (Ours) | 0.713 | 0.706 | 0.688 | 0.663 | 0.639 | 0.618 |

**Table 14: Robustness to various ranges of noises with detailed values on DAIR-V2X.**
| Model | Heading Noise std. ($\sigma_{\mathrm{loc.}}$) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| 0.0 ⁢ ° 0.0° 0.0 ⁢ ° | 0.2 ⁢ ° 0.2° 0.2 ⁢ ° | 0.4 ⁢ ° 0.4° 0.4 ⁢ ° | 0.6 ⁢ ° 0.6° 0.6 ⁢ ° | 0.8 ⁢ ° 0.8° 0.8 ⁢ ° | 1.0 ⁢ ° 1.0° 1.0 ⁢ ° |  |
| CoBEVT | 0.448 | 0.446 | 0.440 | 0.433 | 0.427 | 0.421 |
| V2X-ViT | 0.449 | 0.447 | 0.441 | 0.434 | 0.428 | 0.423 |
| Where2comm | 0.404 | 0.404 | 0.400 | 0.396 | 0.391 | 0.387 |
| CoAlign | 0.408 | 0.407 | 0.403 | 0.399 | 0.394 | 0.391 |
| ParCon (Ours) | 0.466 | 0.464 | 0.458 | 0.452 | 0.445 | 0.440 |
| Model | Localization Error std. ($\sigma_{\mathrm{hdg.}}$) |  |  |  |  |  |
| 0.0 m | 0.1 m | 0.2 m | 0.3 m | 0.4 m | 0.5 m |  |
| CoBEVT | 0.448 | 0.439 | 0.424 | 0.408 | 0.397 | 0.388 |
| V2X-ViT | 0.449 | 0.439 | 0.426 | 0.411 | 0.401 | 0.394 |
| Where2comm | 0.404 | 0.398 | 0.387 | 0.376 | 0.368 | 0.363 |
| CoAlign | 0.408 | 0.402 | 0.393 | 0.384 | 0.376 | 0.371 |
| ParCon (Ours) | 0.466 | 0.458 | 0.441 | 0.426 | 0.414 | 0.406 |

**Table 15: Robustness to a subset of noises with detailed values on V2XSet.**
| Model | Maximum Latency ($t_{\mathrm{lag.}}$) |  |  |
| --- | --- | --- | --- |
| 0 ms | [100 ms, 200 ms] | [300 ms, 500 ms] |  |
| CoBEVT | 0.764 | 0.748 (2.09% $\downarrow$) | 0.697 (8.77% $\downarrow$) |
| V2X-ViT | 0.747 | 0.730 (2.24% $\downarrow$) | 0.686 (8.16% $\downarrow$) |
| Where2comm | 0.741 | 0.731 (1.35% $\downarrow$) | 0.688 (7.21% $\downarrow$) |
| CoAlign | 0.718 | 0.703 (2.09% $\downarrow$) | 0.671 (6.54% $\downarrow$) |
| ParCon (Ours) | 0.760 | 0.760 (0.03% $\downarrow$) | 0.729 (4.01% $\downarrow$) |
| Model | Heading Noise std. ($\sigma_{\mathrm{hdg.}}$) |  |  |
| 0.0 ⁢ ° 0.0° 0.0 ⁢ ° | [0.2 ⁢ ° 0.2° 0.2 ⁢ °, 0.4 ⁢ ° 0.4° 0.4 ⁢ °] | [0.6 ⁢ ° 0.6° 0.6 ⁢ °, 1.0 ⁢ ° 1.0° 1.0 ⁢ °] |  |
| CoBEVT | 0.764 | 0.763 (0.17% $\downarrow$) | 0.735 (3.79% $\downarrow$) |
| V2X-ViT | 0.747 | 0.746 (0.05% $\downarrow$) | 0.730 (2.17% $\downarrow$) |
| Where2comm | 0.741 | 0.735 (0.79% $\downarrow$) | 0.710 (4.16% $\downarrow$) |
| CoAlign | 0.718 | 0.714 (0.47% $\downarrow$) | 0.698 (2.75% $\downarrow$) |
| ParCon (Ours) | 0.760 | 0.759 (0.18% $\downarrow$) | 0.744 (2.13% $\downarrow$) |
| Model | Localization Noise std. ($\sigma_{\mathrm{loc.}}$) |  |  |
| 0.0 m | [0.1 ms, 0.2 ms] | [0.3 ms, 0.5 ms] |  |
| CoBEVT | 0.764 | 0.732 (4.26% $\downarrow$) | 0.548 (28.28% $\downarrow$) |
| V2X-ViT | 0.747 | 0.727 (2.58% $\downarrow$) | 0.669 (10.43% $\downarrow$) |
| Where2comm | 0.741 | 0.715 (3.61% $\downarrow$) | 0.631 (14.82% $\downarrow$) |
| CoAlign | 0.718 | 0.687 (4.26% $\downarrow$) | 0.601 (16.20% $\downarrow$) |
| ParCon (Ours) | 0.760 | 0.746 (1.81% $\downarrow$) | 0.701 (7.75% $\downarrow$) |

**Table 16: Robustness to a subset of noises with detailed values on OPV2V.**
| Model | Maximum Latency ($t_{\mathrm{lag.}}$) |  |  |
| --- | --- | --- | --- |
| 0 ms | [100 ms, 200 ms] | [300 ms, 500 ms] |  |
| CoBEVT | 0.659 | 0.644 (2.33% $\downarrow$) | 0.610 (7.43% $\downarrow$) |
| V2X-ViT | 0.685 | 0.662 (3.37% $\downarrow$) | 0.636 (7.13% $\downarrow$) |
| Where2comm | 0.675 | 0.661 (2.15% $\downarrow$) | 0.642 (4.97% $\downarrow$) |
| CoAlign | 0.638 | 0.626 (1.99% $\downarrow$) | 0.611 (4.26% $\downarrow$) |
| ParCon (Ours) | 0.713 | 0.705 (1.23% $\downarrow$) | 0.689 (3.38% $\downarrow$) |
| Model | Heading Noise std. ($\sigma_{\mathrm{hdg.}}$) |  |  |
| 0.0 ⁢ ° 0.0° 0.0 ⁢ ° | [0.2 ⁢ ° 0.2° 0.2 ⁢ °, 0.4 ⁢ ° 0.4° 0.4 ⁢ °] | [0.6 ⁢ ° 0.6° 0.6 ⁢ °, 1.0 ⁢ ° 1.0° 1.0 ⁢ °] |  |
| CoBEVT | 0.659 | 0.651 (1.24% $\downarrow$) | 0.612 (7.17% $\downarrow$) |
| V2X-ViT | 0.685 | 0.679 (0.85% $\downarrow$) | 0.664 (3.06% $\downarrow$) |
| Where2comm | 0.675 | 0.667 (1.27% $\downarrow$) | 0.633 (6.28% $\downarrow$) |
| CoAlign | 0.638 | 0.629 (1.45% $\downarrow$) | 0.603 (5.52% $\downarrow$) |
| ParCon (Ours) | 0.713 | 0.708 (0.77% $\downarrow$) | 0.690 (3.32% $\downarrow$) |
| Model | Localization Noise std. ($\sigma_{\mathrm{loc.}}$) |  |  |
| 0.0 m | [0.1 ms, 0.2 ms] | [0.3 ms, 0.5 ms] |  |
| CoBEVT | 0.659 | 0.607 (7.89% $\downarrow$) | 0.419 (36.40% $\downarrow$) |
| V2X-ViT | 0.685 | 0.670 (2.24% $\downarrow$) | 0.608 (11.30% $\downarrow$) |
| Where2comm | 0.675 | 0.638 (5.53% $\downarrow$) | 0.538 (20.32% $\downarrow$) |
| CoAlign | 0.638 | 0.595 (6.74% $\downarrow$) | 0.484 (24.12% $\downarrow$) |
| ParCon (Ours) | 0.713 | 0.697 (2.28% $\downarrow$) | 0.640 (10.29% $\downarrow$) |

**Table 17: Robustness to a subset of noises with detailed values on DAIR-V2X.**
| Model | Heading Noise std. ($\sigma_{\mathrm{hdg.}}$) |  |  |
| --- | --- | --- | --- |
| 0.0 ⁢ ° 0.0° 0.0 ⁢ ° | [0.2 ⁢ ° 0.2° 0.2 ⁢ °, 0.4 ⁢ ° 0.4° 0.4 ⁢ °] | [0.6 ⁢ ° 0.6° 0.6 ⁢ °, 1.0 ⁢ ° 1.0° 1.0 ⁢ °] |  |
| CoBEVT | 0.448 | 0.443 (1.01% $\downarrow$) | 0.427 (4.56% $\downarrow$) |
| V2X-ViT | 0.449 | 0.444 (1.04% $\downarrow$) | 0.429 (4.49% $\downarrow$) |
| Where2comm | 0.404 | 0.402 (0.50% $\downarrow$) | 0.391 (3.12% $\downarrow$) |
| CoAlign | 0.408 | 0.405 (0.78% $\downarrow$) | 0.394 (3.37% $\downarrow$) |
| ParCon (Ours) | 0.466 | 0.461 (0.92% $\downarrow$) | 0.446 (4.27% $\downarrow$) |
| Model | Localization Noise std. ($\sigma_{\mathrm{loc.}}$) |  |  |
| 0.0 m | [0.1 ms, 0.2 ms] | [0.3 ms, 0.5 ms] |  |
| CoBEVT | 0.448 | 0.431 (3.68% $\downarrow$) | 0.398 (11.13% $\downarrow$) |
| V2X-ViT | 0.449 | 0.432 (3.62% $\downarrow$) | 0.402 (10.37% $\downarrow$) |
| Where2comm | 0.404 | 0.392 (2.89% $\downarrow$) | 0.369 (8.63% $\downarrow$) |
| CoAlign | 0.408 | 0.398 (2.59% $\downarrow$) | 0.377 (7.34% $\downarrow$) |
| ParCon (Ours) | 0.466 | 0.449 (3.49% $\downarrow$) | 0.415 (10.75% $\downarrow$) |

**Table 18: Comparison robustness to various ranges of noises between ParCon-S and ParCon on V2XSet.**
| Model | Maximum Latency ($t_{\mathrm{lag.}}$) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| 0 ms | 100 ms | 200 ms | 300 ms | 400 ms | 500 ms |  |
| ParCon-S | 0.752 | 0.757 | 0.746 | 0.728 | 0.718 | 0.711 |
| ParCon (Ours) | 0.760 | 0.765 | 0.754 | 0.737 | 0.729 | 0.723 |
| Model | Heading Noise std. ($\sigma_{\mathrm{loc.}}$) |  |  |  |  |  |
| 0.0 ⁢ ° 0.0° 0.0 ⁢ ° | 0.2 ⁢ ° 0.2° 0.2 ⁢ ° | 0.4 ⁢ ° 0.4° 0.4 ⁢ ° | 0.6 ⁢ ° 0.6° 0.6 ⁢ ° | 0.8 ⁢ ° 0.8° 0.8 ⁢ ° | 1.0 ⁢ ° 1.0° 1.0 ⁢ ° |  |
| ParCon-S | 0.752 | 0.752 | 0.748 | 0.741 | 0.734 | 0.726 |
| ParCon (Ours) | 0.760 | 0.760 | 0.757 | 0.751 | 0.744 | 0.737 |
| Model | Localization Error std. ($\sigma_{\mathrm{hdg.}}$) |  |  |  |  |  |
| 0.0 m | 0.1 m | 0.2 m | 0.3 m | 0.4 m | 0.5 m |  |
| ParCon-S | 0.752 | 0.744 | 0.729 | 0.709 | 0.700 | 0.683 |
| ParCon (Ours) | 0.760 | 0.754 | 0.738 | 0.719 | 0.700 | 0.683 |

**Table 19: Comparison robustness to a subset of noises between ParCon-S and ParCon with detailed values on V2XSet.**
| Model | Maximum Latency ($t_{\mathrm{lag.}}$) |  |  |
| --- | --- | --- | --- |
| 0 ms | [100 ms, 200 ms] | [300 ms, 500 ms] |  |
| ParCon-S | 0.752 | 0.752 (0.05% $\downarrow$) | 0.719 (4.38% $\downarrow$) |
| ParCon (Ours) | 0.760 | 0.760 (0.03% $\downarrow$) | 0.729 (4.01% $\downarrow$) |
| Model | Heading Noise std. ($\sigma_{\mathrm{hdg.}}$) |  |  |
| 0.0 ⁢ ° 0.0° 0.0 ⁢ ° | [0.2 ⁢ ° 0.2° 0.2 ⁢ °, 0.4 ⁢ ° 0.4° 0.4 ⁢ °] | [0.6 ⁢ ° 0.6° 0.6 ⁢ °, 1.0 ⁢ ° 1.0° 1.0 ⁢ °] |  |
| ParCon-S | 0.752 | 0.750 (0.28% $\downarrow$) | 0.734 (2.45% $\downarrow$) |
| ParCon (Ours) | 0.760 | 0.759 (0.18% $\downarrow$) | 0.744 (2.13% $\downarrow$) |
| Model | Localization Noise std. ($\sigma_{\mathrm{loc.}}$) |  |  |
| 0.0 m | [0.1 ms, 0.2 ms] | [0.3 ms, 0.5 ms] |  |
| ParCon-S | 0.752 | 0.737 (2.05% $\downarrow$) | 0.689 (8.43% $\downarrow$) |
| ParCon (Ours) | 0.760 | 0.746 (1.81% $\downarrow$) | 0.701 (7.75% $\downarrow$) |

## Peer Reviews

confidence: 4
contribution: 1
initial_rating: 3
presentation: 1
questions: The term "module" is used extensively but inconsistently. Can the authors provide a precise definition of what constitutes a "module" in their architecture? Does it refer to distinct functional units, neural network blocks, or something else? How are the modules in the proposed parallel connection fundamentally different from the components in sequential architectures?
soundness: 2
strengths: The paper introduces a parallel connection architecture for collaborative perception, addressing the challenge of noise robustness in V2X systems. It demonstrates improved detection accuracy and computational efficiency compared to state-of-the-art methods through the use of a Channel Compression Layer (CCL) and spatial-wise attention. The evaluation spans multiple datasets and provides empirical evidence of robustness under various noise conditions, making a practical contribution to autonomous driving research.
summary: The paper introduces ParCon, a collaborative perception model for autonomous vehicles that employs a parallel connection architecture instead of the sequential connections used in existing methods. The main goal of ParCon is to enhance perception accuracy and robustness in noisy communication environments, such as those encountered in V2X (Vehicle-to-Everything) systems. The authors present ablation studies to validate the effectiveness of individual components, such as CCL and parallel connections, and provide qualitative visualizations to demonstrate the advantages of their model in noisy and occluded environments. They conclude with a discussion of the limitations and propose future directions for enhancing the architecture with advanced sub-modules.
weaknesses: 1. Writing: The use of the term "module" is vague and inconsistent throughout the paper. Does it refer to a computational block, a neural network component, or a specific feature processing unit? This ambiguity undermines the reader's understanding of the architecture.
Phrases like "manages noise independently" and "compensates limitations" lack specificity. How the proposed modules achieve these outcomes is poorly explained, leaving the mechanism unclear. The writing is repetitive and verbose, making it difficult to parse the main contributions and technical details. For example, the description of the Channel Compression Layer (CCL) is overly long, yet still leaves the reader unclear about its exact role. Key claims, such as computational efficiency and noise robustness, are repeated multiple times without sufficient empirical backing or theoretical analysis. 

2. Novelty: The primary claim of novelty, a "parallel connection" architecture, is incremental and not supported by any theoretical insights or innovative algorithmic developments. This feature feels more like an engineering optimization rather than a fundamental research contribution. Many components of the architecture (e.g., PointPillars, Sparse ResNet, attention mechanisms) are off-the-shelf methods, and the way they are combined lacks originality. The authors fail to show how these combinations lead to insights or advancements that are relevant to ICLR.

3. Relevance: The paper’s focus on a specific application (collaborative perception for autonomous vehicles) lacks sufficient theoretical contributions or broader applicability to the machine learning community. The proposed method is narrowly tailored to V2X scenarios, which limits its relevance for a broader audience. The paper does not introduce new machine learning techniques or paradigms but rather applies existing concepts in a slightly modified architecture.

4. Related works: The paper could benefit from a more comprehensive literature review, as some highly relevant works about collaborative perception are missing [1-6]. Including a broader range of recent studies would provide a stronger context for the contributions and better situate the proposed framework within the current state of research.

[1] Li, Y., Ren, S., Wu, P., Chen, S., Feng, C. and Zhang, W., 2021. Learning distilled collaboration graph for multi-agent perception. Advances in Neural Information Processing Systems, 34, pp.29541-29552.

[2] Li, Y., Ma, D., An, Z., Wang, Z., Zhong, Y., Chen, S. and Feng, C., 2022. V2X-Sim: Multi-agent collaborative perception dataset and benchmark for autonomous driving. IEEE Robotics and Automation Letters, 7(4), pp.10914-10921.

[3] Huang, S., Zhang, J., Li, Y. and Feng, C., 2024. Actformer: Scalable collaborative perception via active queries. ICRA 2024.

[4] Yang, D., Yang, K., Wang, Y., Liu, J., Xu, Z., Yin, R., Zhai, P. and Zhang, L., 2024. How2comm: Communication-efficient and collaboration-pragmatic multi-agent perception. Advances in Neural Information Processing Systems, 36.

[5] Su, S., Li, Y., He, S., Han, S., Feng, C., Ding, C. and Miao, F., 2023, May. Uncertainty quantification of collaborative detection for self-driving. In 2023 IEEE International Conference on Robotics and Automation (ICRA) (pp. 5588-5594). IEEE.

[6] Su, S., Han, S., Li, Y., Zhang, Z., Feng, C., Ding, C. and Miao, F., 2024. Collaborative multi-object tracking with conformal uncertainty propagation. IEEE Robotics and Automation Letters.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 2
questions: - Is this the first parallel collaborative perception model for V2X systems? If so, emphasizing this novelty in the paper could strengthen its contribution.
- How does ParCon manage relative pose errors? Communication errors often propagate differently across agents; addressing this could enhance robustness.
- Could you provide inference runtime metrics rather than only FLOPS to better understand the model’s real-world applicability?
- ParCon’s performance on the DAIR-V2X dataset under moderate noise conditions is inconsistent with other metrics, and the improvement via introducing s-att seems trival. Could the author explain more about the experimental results?
soundness: 3
strengths: - ParCon proposes the adoption of a parallel architecture in V2X systems, allowing each module to process information independently and mitigate the propagation of noise.
- The framework is specifically designed for parallel fusion in V2X systems, incorporating attention mechanisms to fuse diverse types of information effectively, leading to improved detection accuracy.
- The model performs well across a variety of noise conditions, showcasing resilience and robustness compared to sequential models, particularly under latency, heading, and localization noise, with reduced FLOPs demonstrate the model efficiency.
summary: The paper introduces ParCon, a noise-robust collaborative perception model designed to improve 3D object detection in V2X (Vehicle-to-Everything) systems. ParCon introduce a concept for parallel  ollaborative perception, and propose to employ a parallel architecture that independently processes data streams from multiple agents, enhancing noise resilience and computational efficiency. With attention modules tailored to fuse different information types, such as agent-wise and spatial-wise data, ParCon achieves better overall accuracy under varying noise conditions and reduces FLOPs.
weaknesses: - The paper appears to have been written hastily, with several typos, such as in the equations on lines 171 and 211. Many equations could be embedded within paragraphs for improved readability.
- Some descriptions, like the difference between sub-manifold sparse convolution and sparse convolution, lack clarity, making it challenging to understand the distinctions and implications.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. In Figure 6, what do the lines in Depth 3 represent?  
2. Why does the No Fusion value (0.754/0.602) in Table 1 differ from the Without A-Att value (0.774/0.645) in Table 8?
soundness: 3
strengths: 1. The writing is clear and easy to follow.  
2. The motivation and proposed method are clearly stated.  
3. This work introduces parallel module connections for collaborative perception.  
4. The experiments are thorough, encompassing three datasets that include both real-world and simulated data, as well as addressing two practical issues: communication latency and pose errors, along with detailed ablation studies on the modules.  
5. It is encouraging about the effectiveness, efficiency and robustness of the proposed method.
summary: This paper presents a new collaborative perception architecture that replaces the previous sequential module connections with parallel ones. Experiments demonstrate the robustness of this parallel architecture, and the proposed connections are also computationally efficient.
weaknesses: 1. Lack of novelty and deeper analysis regarding the design of parallel connections. The different modules utilize various operators for information extraction, resembling an ensemble. Could you conduct ablation studies comparing their parallel architecture to sequential and ensemble approaches? Should the parallel designs be more diverse to complement each other better for improved performance? What types of operators are necessary for feature extraction in the context of collaborative perception? Could you analyze which specific types of features or information each module extracts, and how these complement each other for collaborative perception tasks?
2. There is confusion regarding the effectiveness of S-Att. The ablation results in Table 8 indicate that the gain from S-Att is minimal, while Table 9 suggests its effectiveness is significant.  Could you explain this apparent discrepancy and provide additional analysis to clarify the true impact of S-Att? For example, could you conduct more detailed ablation studies or provide visualizations of the features extracted by S-Att compared to other modules?
3. There is insufficient analysis of the experimental results. The findings primarily demonstrate facts but lack an exploration of the underlying reasons for the observed outcomes. For instance, why do the designed modules and their parallel connections contribute to robustness? Could you visualize attention maps or feature activations to show how information flows through the parallel modules under different noise conditions?

---

confidence: 5
contribution: 1
initial_rating: 3
presentation: 2
questions: Clarification on Table 8: The meaning of "Original" is unclear. If it refers to a sequential setup, the performance gain might merely result from the larger model size. This distinction is essential for whether the performance gain is from the parallel design or purely from the extra model size.
soundness: 1
strengths: * The author benchmarked the performance on multiple datasets including simulation dataset (e.g., OPV2V, V2XSet) and real-world dataset (e.g., DAIR-V2X)
summary: This paper presents a method for addressing the noise in cooperative perception via the proposed multi-module parallel connection. The proposed method employs a parallel architecture instead of sequential processing.
weaknesses: * Lack of novelty: The paper presents a method closely related to V2X-ViT with two main modifications: a parallel architecture for multi-agent and spatial attention instead of a sequential setup, and a replacement of window attention with a CNN-based S-Conv and Dilated Neighborhood Attention Transformer for S-Att. However, the novelty and justification for these modifications appear limited, and key design choices lack sufficient exploration.
* Motivation is insufficient: The motivation for using a parallel rather than sequential architecture for better robustness against noise for perception module is insufficient, with no clear rationale for improved noise handling or scalability.
* The tested heading angle range is narrow, making it unrealistic for real-world applications. Please consider increasing the experimented value range for more realistic setups.