SiMO: Single-Modality-Operable Multimodal Collaborative Perception
Authors: Jiageng Wen
Shengjie Zhao
Bing Li
Jiafeng Huang
Kenan Ye
Hao Deng
Venue: ICLR 2026 Poster (2026)
Keywords: collaborative perception
multimodal
modal failure
modality competition
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: This paper presents a multimodal collaborative perception method capable of functioning effectively even when modal failure occurs or only a single modality is available during the collaboration process.

Abstract: Collaborative perception integrates multi-agent perspectives to enhance the sensing range and overcome occlusion issues. While existing multimodal approaches leverage complementary sensors to improve performance, they are highly prone to failure—especially when a key sensor like LiDAR is unavailable. The root cause is that feature fusion leads to semantic mismatches between single-modality features and the downstream modules. This paper addresses this challenge for the first time in the field of collaborative perception, introducing **Si**ngle-**M**odality-**O**perable Multimodal Collaborative Perception (**SiMO**). By adopting the proposed **L**ength-**A**daptive **M**ulti-**M**od**a**l Fusion (**LAMMA**), SiMO can adaptively handle remaining modal features during modal failures while maintaining consistency of the semantic space. Additionally, leveraging the innovative "Pretrain-Align-Fuse-RD" training strategy, SiMO addresses the issue of modality competition—generally overlooked by existing methods—ensuring the independence of each individual modality branch. Experiments demonstrate that SiMO effectively aligns multimodal features while simultaneously preserving modality-specific features, enabling it to maintain optimal performance across all individual modalities. The implementation details can be found in [https://github.com/dempsey-wen/SiMO](https://github.com/dempsey-wen/SiMO).

Published as a conference paper at ICLR 2026
SIMO: SINGLE--MODALITY-PEEBBBBL MULTIMODAL COLLABORATIVE PERCEPTION
Jiageng Wen :en'; :henggje Zhao Bing Jiafeng Huang", Kenan Ye. Hao Deng
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University 2 School of Computer Science and Technology. Tongji University
School of Mechatronic Engineering and Automation, Shanghai University [ _ggeng_wen, shengjiezhac, lizi ) @etonjji.edu....
ABSTRACT
Collaborative perception integrates multi-agent perspectives enhance the sensing range and overcome occlusion issues. While existing multimodal approaches leverage complementary sensors improve performance, they are highly prone failure-especingg especially when a key sensor like LiDAR is unavailable. The root cause is that feature fusion leads to semantic mismatches between single-modality features and the downstream modules. This paper addresses this challenge for the first time in the field of collaborative perception, introducing Multimodal Collaborative Perception (SiMO). By adopting the proposed Length- Adaptive Multi-Modal Fusion (LAMMA), SiMO can adaptively handle remaining modal features during modal failures while maintaining consistency of the semantic space. Additionally, leveraging the innovative "Pretrain-AAlin--Fus-- Fuse--- training strategy, SiMO addresses the issue of modality competition gennnrllly overlooked by existing methods-ensuriee the independence of each individual modality branch. Experiments demonstrate that SiMO effectively aligns multimodal features while simultancously preserving modality-specifie features, enabling it to maintain optimal performance across all individual modalities. The implementation details can be found in /ttps://githbb com/ dempsey-wen/Stiee SMMO
INTRODUCTION
Environmental perception tasks are essential make correct decisions for autonomous driving and robotic investigation. Multi-agent collaborative perception (MACP) is considered an effective strategy to overcome the issues of single-agent perception, including limited perception range and object occlusion. By sharing raw sensory data or perception features (Wang et al.) (020a).. each agent acquires diverse perspectives from multiple sources, thereby complementing information in occluded and remote areas. Many methods have been proposed to achieve a better and more precise collaborative perception (Liu et al. 2020b; Vadivelu al., 202T
Liu et al. 2020a Hu al. 2022
Lei et al. 2022 Lu et al. 2023).
Recently, collaborative perception with modal heterogeneity is gaining increasing attention. Initially, given the superior performance of point clouds in precise localization, most of the MACP methods rely on LiDAR. Considering the challenges that a single-
method can easily fail under certain conditions, some research (Xu et al., 2022a Hu al. 2023) has begun exploring camera- based collaborative perception methods as a supplement. Then, Zhao et al. (2023); Xiang et al. (003)); Lu et al. (2024); Gao al. (2025) have further explored the application of multimodal collaborative perception to leverage the advantages of multimodal sensors. However, these methods only consider integrating multimodal information to improve accuracy, ignoring the drawback that modal missing or damage could lead to the failure of the entire system, like a series circuit with multiple components.
"The work was done in Engineering Research Center of Key Software Technologies for Smart City Perception and Planning.
Conrespnnding author.
'Coresponding author.

---

Published as a conference paper at ICLR 2026
Series Circuit Not warking with any
(b) Existing Methods: Fuse without Alignment
colors indicate feature spaces
Fusiion
BASE
Z
Parallel Circuit Working with any effectiveness
Change in the number of dimensions
Change in the semantics of dimensions
(c) Ours: Align before Fusion
ratworkin
woikin) 9
Align 71
mm
Robustness Comparison: Existing Methods vs. Ours
not finiing
Keep condistency among unfused & fused.
Keep corsistency among multi palli--geett
Figure I: a) Existing methods perform like series circuit and fail with any modal failure, while ours performs like parallel circuit working with any effective branch. b) The common feature fusion methods cause space shifts, rendering unfused features not fitting to the downstream task heads. c) Aligning features before fusion keeps the consistency among unfused, fused and multi-agent features.
Although modal failure has been explored in single-agent scenarios (Yan et al. 2023 Ge et al. 2023 Wang et al. (2024a), MACP confronts unique and significantly more complex challenge: Hetero- geneous Modal Failure. Unlike single-agent settings that primarily require intra-agent alignment between features and local task heads, MACP demands strict inter-agent alignment. In realistic collaborative scenarios where agents suffer from different sensor failures (e.g., ego vehicle utilizing LiDAR collaborating with a camera-only neighbor), their transmitted features must reside a strictly unified semantic space to enable effective interaction. Existing single-agent robust methods fail to guarantee this cross-agent semantic consistency, leading to the breakdown of multi-agent fusion modules. Furthermore, the existing multimodal methods generally ignore modality competition (Huang et al. (202)), which often severely hinders multimodal joint learning. It causes models to favor easily converging branches, leaving more challenging ones inadequately trained and unable to function independently without the dominant branch. Even the dual-tower models that aim to extract multimodal features separately, like BEVFusion (Liu al., 2023), have to rely on the only effective branch, despite the other branches could have functioned with the redundant information of multi-sensors. To our best knowledge, our work is the first to tackle the significantly more complex challenge of dynamic, heterogeneous modal failures in MACP.
We propose the Single-Modality Modality-Operable Multimodal Collaborative Perception (SiMO) method, which allows the system to work normally with any effective modality, similar to a parallel circuit working as long as one path is connected as shown in Figure [ Compared to existing multimodal fusion methods, SiMO emphasizes aligning multimodal features before fusion and then integrating them within the same unbiased feature space. This approach not only ensures compatibility between single-modal and multimodal features for downstream modules but also maintains consistency in multi-agent features even when heterogeneous modalities fail during collaboration. We also delicately design the Length-Adaptive Multi-Modal Fusion (LAMMA) to handle feature fusion with modal missing. This plug-and-play fusion module can adapt to the integration of features extracted from different modalities in heterogeneous scenarios, while structurally providing foundation for aligning the feature spaces before and after fusion. To tackle the imbalanced training of multimodal branches introduced the "Pretrain-Alie -eeerain-AAiin---- Align-Fuse-RD (PAFR) training
caused by modality competition, strategy, enabling the extraction of comprehensive, independent multimodal features and bolstering the efficacy of individual modalities (Figure c)). We summarize our contributions as follows:
For the first time in collaborative perception, we address multimodal perception failure caused by missing modalities with SiMO, particularly when only RGB images are available.
We identify the inconsistency between pre- and post-fusion features as the primary catalyst for system collapse during modality failure, and propose LAMMA to accommodate varying quantities of modal features and structurally preserve semantic consistency fusion.
We point out that modality competition is a generally ignored obstacle to achieving the effectiveness of independent modal branches, and propose the PAFR training strategy to
2

---

Published as a conference paper at ICLR 2026
solve this problem and avoid the uncertainty of balancing branch convergence existing methods do.
Extensive experiments show that SiMO enables each modal branch to function alone with redundant multi-modal features and keeps the SOTA level performance.
RELATED WORKS
2.1
MULTIMODAL COLLABORATIVE PERCEPTION
HM-ViT (Xiang et al. 2023 first explores multimodal collaborative perception with agents randomly
adopting one modality of LiDAR point cloud or RGB images, achieving collaboration with modality
heterogeneity. HEAL (Lu al. 2024) further addresses the challenges of both modal and model
heterogeneity in collaborative perception. Based on HEAL, STAMP (Gao et al. 2025) implements a
multimodal and multi-task collaborative perception framework. BM2CP (Zhao et al., 2023) fuses point cloud and RGB image features before multi aggntt collaboration, and outperforms LiDAR-basee methods in 3D detection. CoBEVFusion (Qiao & Zulkernine (023) combines the two modalities to perform 3D detection and BEV segmentation based on BEV Fusion (Liu et al. 2023). However, these existing methods cannot continue to function when LiDAR fails. Though modal failure's significant impact in multimodal perception is recognized in (Yu et al., 2023). the issue remains largely unaddressed. CMT Yaan et al., 2023) pioneered multimodal perception with single-modality operation but did not analyze the mechanisms for adapting to modal missing. MetaBEV (Ge et al.) 2023) was the first to specifically tackle this problem, attributing it to feature misalignment that occurs modal missing, consequence of the pixel-level positional dependencies imposed by CNN+Concat fusion. UniBEV (Wang et al. 2024a) advocated identically structured feature extractors for semantic consistency, achieving more effective multimodal feature alignment than MetaBEV. Existing methods address modal failure only in single-agent, but struggle in the more complex MACP scenarios.
2.2 BALANCED MULTIMODAL JOINT LEARNING
Multimodal learning is typically believed to improve performance by integrating features of various modalities, outperforming single-modal approaches. However, real-world applications, jointly trained multimodal classification networks often underperform compared to their single-modalit counterparts. Wang al. (2020b) initially investigate the phenomenon, identifying overfitting re- sulting from increased model capacity from multiple branches the primary cause, and introduce Gradient Blending as a solution. Huang et al. (2022) firstly conclude the reason as modality com- petition and elaborate on how it hampers joint training of multimodal networks, suggesting that random initialization contributes to disparities in modal advantages. To rectify imbalanced training, Peng et al. (2022) prove that adjusting update gradients and dropout ratios for different branches during joint learning can ensure that both modal branches converge at a similar pace. |Wei et al.) (2025) argue that prioritizing the weaker modality for balance can limit the dominant modality's performance. Instead, they propose to assess the trainability of different modal representations based on their separability and adjust the relearning intensity accordingly. Yang al. (2024) highlight the importance of element- wise gradient modulation, considering the varying sigmificance of each network parameter. Current methods struggle to balance convergence rates and determine optimal modulation parameters, necessitating more deterministic approaches to address modality competition, especially in complex multimodal collaborative perception.
3 METHODOLOGY
3.1 THE ARCHITECTURE OVERVIEW
3D object detection is prevalent challenge in MACP, so we frame the problem as multiple agents using point clouds and multi-perspective images to detect objects in 3D. To facilitate the fusion of features from various agents, it is necessary to transform these features based on their respective poses. Consequently, Bird Eye's View (BEV) feature with explicit spatial semantics has emerged as a prevalent feature type in collaborative perception. As shown in Figure 2(a), the model processes these imputs-poin clouds X, and RGB images Xc-by extracting their BEV features and aligning
3

---

Published as a conference paper at ICLR 2026
(a) The Overview of SiMO
X
LIDAR Encoder
Aligner
Multi- Agent Fusion
Multi-Modal Fusion Module LAMMA Φ
Task Head h()
Careaaa
Alignar
X
Encoder
Rc
mmm
(b) LAMMA
(c) Balanced Multimodal I Learning
Conviieni
Modal A
unbalanced learaing with modallty competition
Length- Adaptive
Proj
Proj Jinv
Gammon
A-specile:
CounNolli
Fusion
MLP
@P
Modal
Unified Space
A-
Common
K,
automatically downgrade
K,
V
A
0
Modal A
balanced learning
comprehensive modality- sprciific featerss
SA
Com mon
A-speelfic
8
CA
SA
sum
0
0
Modal
Unified Space
Comman
D-spectl
Commion
spoolic
spodno
QKV: Query, Value S4: Self-Atiention
CA: Cross-Attemion
Figure 2: a) The overview of SiMO. b) LAMMA adaptively downgrades to Self-Attention fusion to keep consistent feature processing when modal failure happens. c) SiMO conducts balanced multimodal learning to keep modality-specifie features for branch independence.
them to similar feature spaces to attain Z,, and Ze, followed by the creation multimodal features Zum through the multimodal fusion Φ. The BEV features Zmm, shared by various agents are then fused to form a collaborative feature, and finally input to cls, reg and dir task heads produce 3D
bounding boxes Yreg and corresponding classification results Y Yat-- This process is described as:
8mlfm (Xm)), L, Z Zum (Z,, Zc)
Zm
(la)
(1b)
Ý (Zmmiz) i = 1,2,3,...
(1c)
where Z., represents the aligned BEV features, with f. denoting the feature extractor and gm the feature alignment module for modality m. & represents the multimodal fusion, V the multi-agent fusion, and h the task heads. Besides, the subscript L indicates the LiDAR modality, C' the camera, mm the fused multi-modality, and for agent order.
We adopt supervised training for multimodal joint learning. OPV2V-H (Lu al. 2024) provides 3D bounding boxes for all vehicle objects within the perception range of multiple agents. Each bounding box label Y encompasses its central coordinates (I, y, 2), dimensions (height, width, length h, w, I), orientation angle (0), and target class (Y,.)) Thus, the comprehensive 3D object detection task is essentially hybrid of seven-value regression task and classification task. We apply Focal Loss (Lin et al. (017)) for classification and Smooth-LI Loss (Gishhcck 2015) for regression. The total loss is follows:
L Y) LF Yalss Yess) + LSmooth- L. (Y &reg: Yreg).
3.2 WHY THE MODEL FAILS WITH SINGLE MODALITY?
Multimodal collaboration often uses concatenation, convolutional neural network (CNN), graph neural networks (GNN) (Scarselli et al. 2008). attention mechanisms, or Transformer (Vaswani) 2017) to blend features from various modalities. These methods either combine different modalities' features by merging their dimensionality (Liu et al.) 2023 Qiao & Zulkernine 2023), or integrate features into a new feature space (Xiang et al., 2023 Zhao et al. (023). These fusions create a disparity between the pre- and post-fusion feature spaces, making downstream task heads designed for fused features unsuitable for unfused ones. Consequently, when a modality is absent and feature fusion is inapplicable, these methods lose their effectiveness (Figure (b))
4

---

Published as a conference paper at ICLR 2026
Taking these points into account, we first opt integrate features from various modalities into single feature space using joint learning. Within the framework of joint learning, upon processing by identical modules post-fusion, it can be inferred that the features of multiple branches reside within the same feature space. Subsequently, via addition to fuse these features, we ensure that the feature space before and after fusion remains consistent. This approach enables both fused and individual modal features to be smoothly fed to downstream task heads (Figure I ()).
3.3 LENGTH-ADAN ADAPTIVE MULTIMODAL FUSION (LAMMA)
Additive fusion hinges on the effective alignment of multimodal features. While joint multimodal learning aids this goal, as Wang et al. (2024a) highlights, consistent feature processing architecture crucial for semantic alignment. However, in MACP's typical heterogeneous scenarios, diverse modal feature extraction is common. Therefore, we aim achieve consistent multimodal feature processing within the fusion module itself, avoiding additional requirements on other components. Furthermore, modal failure in collaborative settings implies variable number of input features, and detecting and responding to this change in complex, unpredictable real-world environments is challenging. To address these issues, we propose Length-Adaptive Multimodal Fusion (LAMMA), which is structurally designed for consistent multimodal feature processing and adaptive fusion.
Extracted features originating from distinct modalities exhibit considerable semantic variations, and multimodal fusion without prior semantic unification could result in mutual interference and detract from the model's efficacy. Hence, we apply ConvNeXt (Liu et al. 2022) as gr. and go aligners to align multimodal features Zg and Ze's 's semantics in both channel and pixel-wise. Then, all input features are added with a moodality-agoostee positional embedding avoiding modality-special treatment, and projected to more compact feature spaces for based length-adapte fusion robustly handles missing modalities, which a common challenge for existing methods based on CNN, RNN or Transformer. We achieve this by concatenating queries from different modalities in parallel, allowing for simultancous self-attention and cross-attention. The crossaattention mechanism facilitates both the transfer of complementary information and the alignment of shared features across modalities. The final integrated multimodal feature representation is then derived by additively combining these fused self- and aotsnnattnnion results. Hence, fusion is implemented as:
adaptive fusion. The attention-
Q
Wº [Z; Za] E bx22xx
(3a)
Km
W WK2 Zm R
(3b)
Vn
W E RR R
(3c)
Zattim
= MHA Km, Vn) E bx22nxd
(3d)
Z Lfseed_ Sum(Split(Z2)
pbxaxd
(3e)
Zyn
=
I + Z fused_B E Rxxxxd
(3f)
where [:] means feature concatenation, and WQ. WK, Wv are the linear weights for query, key and value respectively, shared by all modal features. querys for and are concatenated as Q. while keys and vol ues kept separated number of tokens and feature dimension respectively. Here A, B are used to refer to different but not specific modalities, as we do not assign the order of modalities, and LAMMA should treat all modalities consistently. Zatt_m attained with multi-head attention (MHA), are split to two parts in the size of (b, n. d), of which one for self-attention fusion and another for cross-attention fusion, and then summed up in element-wise to give Zfnsed m as enhanced representations for each input modality. Finally, Z Zfused_A A and Zfnsed n fused with addition for Zmm to avoid shifting the feature space. In the case of modal failure, the absent feature (e.g.. Z,)) in the query is empty. resulting Q [0: QB] Consequently, the fusion can be considered as essentially downgrading to a self-attention module (SA) without the need to distinguish failure happening (Figure ((b)::
Km and V m with m A, B. n, d are batch size,
QAKT
QBK?
Z Zjssed_m m
= Softmax( m V + Softmax(
(4a)
Vd
vd
Z Lusedd_A
0, Z fased SA KB, VB).
(4b)
Given that the model remains unchanged regardless of the presence or absence of modalities, and the input is consistently processed with exactly the same parameters, the semantic interpretation of
5

---

Published as a conference paper at ICLR 2026
the fused features is supposed to remain consistent. So, no matter if modality drops not, the fused features share the same semantics, ensuring optimal adaptability to the downstream task heads.
3.4
OVERCOMING MODALITY COMPETITION
3.4.1 MODALITY COMPETITION
To achieve object detection with only single modality, it is crucial to respectively extract full features from different modalities and align them in a shared space through multimodal joint learning. However, we have observed (details in Table of of Section A.9 that multimodal model trained jointly performs even worse than the best single-modalig phenomenon, termed as 'modality competition' by HHuan et al. (2022), is believed to result in insufficient training of certain branches in naive joint learning. Prior studies (Wang et al. 2020b
(LiDAR) model. The counter-intuine innuuiiee
Huang et al. 2022, Peng al., 2022 Javaloy et al.) 2022] Wei al. 2025 Yang et al. 2024) attribute these to factors like varying optimization distances or model overfitting, with various mitigation strategies proposed. However, we believe the phenomenon stem from more fundamental disparity
in how modalities present task-relevant information.
Building on these observations and existing research, particularly the faster convergence of LiDAR models, we hypothesize that modalities inherently vary in their efficiency at providing 'task-relevant information density'. For instance, extracting 3D spatial data is far more direct from point clouds than inferring it from 2D images. We contend this fundamental difference in 'information carrying efficiency' is primary driver of modal competition. Modalities with higher efficiency (c.g., LiDAR for 3D geometry) enable faster learning in their respective branches, which can then dominate the joint optimization process and suppress the thorough training of branches associated with lower- efficiency modalities (e.g., camera for 3D from 2D). This perspective suggests that issues like 'varying optimization distances' (Huang et al. 2022) or 'model overfitting' (Wang et al. (2220)) (potentially the more efficient modality) can be seen manifestations of this underlying efficiency gap. Given these intrinsic differences in 'information carrying efficiency,' we argue that modal competition largely inevitable within traditional end-to-end joint training paradigms, where an imbalanced learning dynamic naturally arises from co-optimizigg modalities with disparate ease of information extraction.
Pair Hi
pretrolned
3.4.2 SOLUTION
fidar wilfuse
Tuse
Toak
Recognizing this, our research shifts from attempt- ing to balance competition within such frame- work. Instead, we aim to circumvent its negative effects through an isolated multi-branch training approach (shown in Figure B). This strategy in- volves independently pre- pre-training each modality- specific branch to develop comprehensive feature extraction capabilities before any subsequent fu- sion. Our goal is not to climinate inherent modal differences but architect a training paradigm that prevents these differences from causing detri- mental competition, especially in the early learn- ing stages, thereby maximizing each modality's potential contribution.
Painchillar
of Randomyy Maddllly Chop RSP
Touker
Multi-
all.
Lift Splaet Shot 02)
S
Moduie
fuse
Eouatoo
Oc)
pretranedd com attfuse
all
fuse
Hoj
06.0
Figure 3: The training process of SiMO. (1) Load pretrained feature extractors. (2) Train
each aligner with the extractor frozen. (3) Train
LAMMA with two-modal input, freezing align- ers and task heads (w/o RD). (4) Fine-tune LAMMA with RD to adapt modal failure.
(Step 1) Instead of training both branches together, we consider training each branch independently to guarantee sufficient training for both branches as a strategy with more certainty. So, feature extractors that have been pre-trained through single-modal learning will be directly adopted after convergence of both branches. Then, we maintain these pre-traine trained extractors in a frozen state while advancing the training of the feature alignment module, fusion module, and task-specific heads.
(Step 2) Given that the effective features extracted by the pre-trained feature extractors reside in different feature spaces, the subsequent step involves training alignment module capable of aligning these features into the spaces compatible to the fusion module. We initiate the training process with the model accepting single-modal inputs (e.g. LiDAR), continuing until the model

---

Published as a conference paper at ICLR 2026
achieves convergence. Subsequently, freeze the trained alignment module (e.g. gi), and proceed to train the other aligner (with only camera inputs) until the model converges once more. This ensures that the second aligner (gc) can also effectively align its features to LAMMA compatible space. More aligners for other modalities can be trained and frozen in the same way.
(Step 3) Finally, with all alignment modules frozen, train the rest modules with multimodal inputs to achieve final convergence. All parameters of the common modules, including multimodal fusion, multi-agent fusion and task heads, are shared by all inputs without modality notations, so the common modules will be modality-independent
Thuoughout the training of feature extractors and fusion modules, our approach is prioritize single branch each time to preclude the emergence of modality competition, then freeze the branches for the common modules training. After the convergence of common modules, the model is fine-tuned with 0.5 possibility to Randomly Drop one modal feature in LAMMA (RD) in order to fit the model to the case of single-mode modal failure (Step 4).
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Dataset. We conduct our experiments on the public MACP dataset OPV2V-H (Lu et al. 2024 heterogeneous collaborative perception dataset based on OPV2V (Xu et al. 2022c) with supple- mentary of more sensor types. With the autonomous driving simulator CARLA (Dosovitskiy et al.. 2017) to simulate the road environments of cities and suburbs, OPV2V collects 73 scenes with 11,464 frames in total. The whole dataset (not including culver_test test set) is split into train, validate and test set with 6374, 1980 and 2170 frames respectively. Each frame contains the perception data of 2-7 Connected Automated Vchicles (CAVs), including the 800 600 RGB images the front, back, right and left cameras and the point clouds of 32-channel LiDAR. 3D bounding box annotations within the perception range of CAVs are provided for collaborative 3D object detection. In Section A.7 we also validate SiMO more datasets.
Model Settings. We adopt Lift--Slat-Shon (LSS) (Philion & Fidler| 2020) for camera encoder and PointPillar (Lang et al. 2019) for LiDAR encoder to independently extract BEV features of multi- view images and point cloud, and LAMMA for multimodal fusion. The aligners mantics of two features are two independent 3- layer ConvNeXt blocks (Liu et al. (202).. As for multi-agent fusion, we choose Autusion (Xu ct al.) 2022c) because the absence of trainable parameters in AttFusion facilitates the training of downstream modules shared among multiple load the pre-trained branches. Specifically, sattt__cmmatt___e_t f fuse and model parameters of opu2v_cams poo2v_lida___ fuse which both incorporate AttFusion with LSS for camera feature extraction and PointPillar for LiDAR, and then freeze the extractor to continue to train LAMMA. We also
Table 1: Quantitative 3D detection performances of models (in %).
Method
Modalit
APd:30
APM:0
AP
BMECP Zioe et 2003
91.55
91.31
unify the se-
BEVFusio Li al.
2023
(RD)
L+C(
93.33
UniBLV Wang all 2004g
LRRD)
91.75
((ID))
AlFusion Xu et 20122
68.31
96.65
95.09
52.1
25.50
87.16
L-C twle RD)
95.51
85.26
85 10
SIMO [AuFusian)
L4C(
96, 10
94.06
84.81
(RD))
95.11
94.02
82.35
(RE)
66.53
49.69
21.59
HEAL et 2024 (Pymikr TuCOn
98.22
58.00
96.6
68.45
60:48
36/07
LaC RD)
94.89
L+C((D)
95.30
iPynunid Fusoon)
(RD)
97.07
(RD)
apply the Pyramid Fusion model (Lu et al., (0044) as the main body of collaborative perception model with LAMMA to fuse multimodal features, veri- fying the effectiveness of LAMMA as a plug-and-play module to fit different collaborative perception frameworks. More details are in Scction[A.I
Train Settings. Our models are trained with Adam (Kingma 2014) optimizer and batchsize on
NVIDIA Geforece RTX 3090 GPUs The perception range is 102.4* 102.4 centered around the ego CAV. Average precisions at Intersection- -overiiioooooo---oeeeee Union thresholds of 0.30 (AP@30), 0.50 (AP@50), 0.70 (AP@70) are adopted measure the model performance. Inheriting the pretrained feature extractors open-sourced by HEAL (Lu et al., 2024), we freeze the parameters of extractor to further train the aligner and task heads. LiDAR branch training starts with initial learning rate (Ir) 2e-4
7

---

Published as a conference paper at ICLR 2026
and weight decay le-4, and reduces by 0.1 epoch 3 until convergence. After that, camera branch is trained with the same setting except task heads frozen. Freezing extractors, aligners and task heads, fusion module LAMMA is trained with Ir of 2c-5.
4.2 QUANTITATIVE RESULTS
Performance Analysis. We take BM2CP (Zhao et al. 2023). the only open-sourced MACP method with multimodal fusion, as our baseline, and we transfer BEVFusion (Liu et al. (023), a popular SOTA multimodal method for single agent, to collaborative perception by adding Pyramid Fusion for multi-agent collaboration. To control variables and limit the scope of comparison to multimodal fusion, we replace the LiDAR BEV feature extractor of BEVFusion with PointPillar and in the same settings with SiMO. Similarly, to compare with the SOTA methods for solving modal failure in single-modal scenarios, we adopt the CNW (Channel-Normalizee UniBEV the multimodal fusion, mirroring the other setup used for SiMO, and train the model with the Modality Dropout method proposed in its original paper. Moreover, we consider comparisons with the single-modal models of AttFusion and HEAL, which is the current SOTA collaborative perception method. Table TableeTshoee shows that none of BM2CP, BEVFusion and UniBEV cannot operate when LiDAR failing, while our SiMOs can effectively work with either single modal or multi modals, especially with only camera images. SiMO-AF is based on the pretrained models of AttFusion, and SiMO-PF based on Pyramid Fusion. Before fine-tuning with RD, SiMO-AF and SiMO-PF keep the upper performance limits of AttFusion and Pyramid Fusion (SOTA) respectively, which predictable as we adopt the same feature extractors of pretrained AttFusion and Pyramid Fusion. SiMO is designed to extract the comprehensive features for both LiDAR and camera to enable each branch, causing redundancy of the common feature of LiDAR and camera. So the results suggest that LAMMA can effectively handle the redundancy of two-modal features.
Weights) fusion proposed by
After fine-tuning with RD. both SiMO-AF and SiMO-PF achieve the adaptability modal failure of either LiDAR cam- era. SiMOs with RD maintain the performance levels as be- fore RD fine-tuning, without suffering severe deterioration caused by modal failure. Moreover, SiMO-PF with camera modal exhibits a significant improvement (12.36/9.155) 1555.75 for AP@ 30/ @ 50/@ 70) over Pyramid Fusion's 's which suggests that Pyramid Fusion does not take fully advantage of the camera ex- tracted features, considering their extracted features are exactly the same. While LAMMA can not only fuse the multimodal features, but also further refine the features for tasks.
100
re jeee nalnd
an co.66
Pysundd fasser iD
10cral
5468
15020
2000
Poinss
Figure 4: Comparison of SiMO- PF, BEVFusion and Pyramid Fu- sion (L) in varying extents of Li- DAR failure.
Robustness against Modal failure. To further demonstrate the robustness of SiMO against LiDAR failure, we evaluate the performances of SiMO-PF and Pyramid Fusion with varying numbers of LiDAR points, simulating the different degresses of LiDAR failure. Figure 4 shows that as LiDAR fails, the performances of Pyramid Fusion with LiDAR and BEVFusion drop rapidly and eventually fail to detect, while only SiMO-PF's performance degrades in much slower speed with the complementation of camera modality and finally converges to the level of SiMO-PF with only camera modality.
Adaptability to Heterogeneous Modal failure. The above compar- isons happen in homogeneous modal failure settings, where either LiDAR or camera is effective for all agents, while for heterogeneous modal failure, different agents in the collaboration graph lose distinct sensors. SiMO aligns BEV features into a consistent space for effective fusion regardless of failing modalities, so it is suppose
Table 2: Performances in homogeneous and heterogenous modal failures. L+C: both modalities, L: only LiDAR, C: only cameras, C-ego: heterogeneous failure with camera ego (C-L-C-L-...) L-ego: heterogeneous failure with LiDAR ego (I--C-L-C-...
METHnD
HEAL (PYRAMID FUSHON)
SIMO-PF
MODE
L
C
C-500
L-EG0
L+C
L
C
C-800
L-EG0
30
0.98
0.68
0.85
0.96
0.98
0.97
0.81
0.89
0.97
A
0.98
0.60
0.62
0.96
0.98
0.97
0.70
0.85
0.97
7P
0.39
0.92
to adapt to heterogeneous failure
well. In experiments, we simulate this using alternating failure pattern, where agents
8

---

Published as a conference paper at ICLR 2026
sequentially lose different modalities (e.g., Modality A, then Modality B, then Modality A, and so on). shows that compared to homogeneous failure, heterogeneous failure with LiDAR ego (the ego agent is equipped with LiDAR) results in only minimal performance decline, and models with camera ego significantly improve performances over camera-only setups. Moreover, compared HEAL,, which is designed for heterogeneous failure (collaboration among agents with heterogeneous modality), SiMO-PF achieves higher detection accuracies with both LiDAR ego and camera ego in the same heterogeneous failure settings without additional fine-tuning.
4.3 ABLATION STUDY
Table 3: Ablation study shows the necessity of our learning strategy, RD and LAMMA to adapt to modal failure.
The ablation study conducted to prove that achieving SiMO's adapt- ability to modal failure requires ap- plying our learning strategy, RD fine- tuning, and LAMMA simultancously.
STRATEGY
RD LAMMA
COHRESSONDING
EXPERIMENTS
(L+C/LC
MODAL FAILURE
NAIVE TRAINING W2O RD
0.9440.01M
NAIVE TRAINING WITH
0.11/00
SIMO wno RD
0.95A0.2640
The comparisons between BEVFu- sion and SiMO-PF in Table [ have demonstrated that even with RD fine- tuning, the model fails to adapt to modal failure without LAMMA for multimodal feature fusion. The performance SiMO before and after RD highlights the importance of RD in enhancing adaptability to modal failure (shown in Table 3). To further examine the impact of our joint learning method, we also trained SiMO-PF naively. As a result, regardless of whether RD fine-tuning was applied, HIMO-PF was unable to adapt to modal failure. The performance of the naively trained SiMO-PF with RD fine mie-ttming further confirms that without addressing modal competition through our training strategy, RD would only hinder the model. More ablation studies can be founded in Section A.9
BEVPISION WITH RD
0.K.A. 7201
SIMO WITH R.D
9.950.9448.45
The Effectiveness Verification of LAMMA. To quantitatively validate that LAMMA successfully aligns features from differ- ent modalities into a common, semantically consistent space while preserving their modality-specifie characteristics, we conducted both Procrustes analysis and t-SNE visualization.
by Sample
L-SNE Component
Jhish degree
Procrustes analysis (Table 4. which measures the disparity between two feature sets after optimal linear transformation, showed a dramatic reduction disparity between LiDAR and camera features after processing by LAMMA (from 0.6747 to 0.0472). This indicates that the features' geometric structures became highly congruent. Furthermore, t-SNE visualizations (Figure 6 revealed that while the features naturally formed dis- tinet clusters according to their modality (preserving modality- specific information), their intra-cluster topological structures exhibited high degree of mirror symmetry after alignment.
misner
symmeerry
stmilanity
Feature Types
b-NE Component 1
Figure 6: The t-SNE visualization for features of LAMMA.
1): SIMO-PF
white: point cloud of al agents reci detected results and confidences green: ground truch bounding boxes
2): HEAL (Pyramid Fusion)
(a) Full LiDAR
(b) 15,000 Lidar points
(c) 5,000 Lidar points
(d) Camera
Figure 5: Visualization of SiMO-PF's and HEAL's (Pyramid Fusion) detection results, ground truth and collaborative point clouds. (a)(b)(c are detection visualizations with all, 15000, 5000 LiDAR points (by occluding local point clouds), and (d) is detection visualizations with only camera images. SiMO-PF detects most objects even with 5000 LiDAR points, while HEAL misses the most.

---

Published as a conference paper at ICLR 2026
This suggests that semantic relationships between samples consistently encoded across modalities. These findings empirically confirm that LAMMA achieves its design goal of mapping multimodal features into unified space where they can be fused through addition without semantic shift, ensuring compatibility for downstream tasks under any modal failure. The implementation details and more results of t-SNE analysis be founded in Section A.10
4.4 QUALITATIVE ANALYSIS
a) shows that with enough LiDAR points, SiMO-PF has the same detection quality as Pyramid Fusion. When LiDAR becomes sparse even fails, SiMO-PF (the upper row) can still achieve reasonable performance, shown in (c). also reveals the necessity of applying extra modality to prevent the dangers posed by LiDAR failure. The comparison in (d) indicates the lead of SiMO-PF (Camera), which consistent with the above analysis that Pyramid Fusion does not fully utilize the extracted camera features. More visualizations can be found in Section
Figure 5
Table 4: Procrustes analysis.
PROCRUSTES
BEVFUSION
BEFORE LAMMA
LAMMA
AFTER
CAM VS LIDAR
0.8645
0.6747
0,0472
CAM
0.7297
0,3886
0.0215
LIDAR vS FUSED
0,5747
0.2773
0.0064
5 LIMITATIONS
While SiMO effectively addresses modality failure in collaborative perception, it exhibits certain limitations. First, single-modality operation performance is inherently limited by feature extractor capabilities. For instance, in single-view camera setups (e.g.. DAIR-V2X) the lack of multi-view parallax limits depth estimation accuracy. SiMO preserves baseline performance but cannot overcome this physical information bottleneck. Second, PAFR strategy, while ensuring deterministic convergence and mitigating modality competition, necessitates a multi-stage training pipeline. This inevitably increases the total training duration compared end-to-end joint training methods. Finally, LAMMA's additive nature, designed to preserve feature independence, lacks the implicit smoothing inherent in convolution-bai based fusion. This potentially makes the system more sensitive high- intensity sensor noise unless specific denoising modules are integrated.
6 CONCLUSION
This paper proposes SiMO to enable a multimodal collaborative perception system to function with various modal failures. SiMO believes that merging multimodal features in unified feature space is key to solving the problem, and it specifically addresses the issue of modal competition to ensure that each modality branch retains its independent functionality. Through the plug-and-play LAMMA and PAFR training methods, SiMO effectively aligns multimodal features while preserving modality- specific features. This makes it easy to adapt SiMO to other multimodal approaches without requiring modifications to existing methods, giving SiMO broader versatility.
ACKNOWLEDGMENTS
This work supported in part by the New Generation Artificial InteligennceNNttionl Science and Technology Major Project 20252D0123703, in part by the National Natural Science Foundation of China under Grant 61936014 and 62076183, in part by the Chenguang Program of Shanghai Education Development Foundation and Shanghai Municipal Education Commission under Grant 21CGA24, in part by Shanghai Municipal Science and Technology Major Project No. 10ZIDZZZ0000 in part by the Shanghai Science and Technology Innovation Action Plan Project 2251 I 105300, in part by the National Natural Science Foundation of China under Grant U23A20382 and in part by Fundamental Research Funds for the Central Universities.
REPRODUCIBILITY STATEMENT
To ensure the transparency and reproducibility of this work, we have provided the complete codebase in the supplementary materials, which includes implementation details of our experiments, training
10

---

Published as a conference paper at ICLR 2026
configurations, and data processing scripts. In addition, we have detailed the model architecture, hyperparameter settings, and all key steps in the training process in the appendix. We believe these materials will provide readers with sufficient information to easily reproduce our main results and conduct further research based on them.
REFERENCES
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In Conference robot learning, pp. 1-16. PMLR, 2017.
Xiangbo Gao, Runsheng Xu, Jiachen Li, Ziran Wang, Zhiwen Fan, and Zhengzhong Tu. Stamp: Scalable task and model-agnostic collaborative perception. arXiv preprint arXiv:2501.1861. 2025.
Chongjian Ge, Junsong Chen, Enze Xie, Zhongdao Wang, Lanqing Hong, Huchuan Lu, Zhenguo Li, and Ping Luo. Metabev: Solving sensor failures for 3d detection and map segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8721-8731, 2023.
Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 1440-1448, 2015.
Yue Hu, Shaoheng Fang, Zixing Lei, Yiqi Zhong, and Siheng Chen. Where2comm: Communication- efficient collaborative perception via spatial confidence maps. Advances in neural information processing systems, 35:4874-4886, 2022.
Yue Hu, Yifan Lu, Runsheng Xu, Weidi Xie, Siheng Chen, and Yanfeng Wang. Collaboration helps camera overtake lidar in 3d detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1243-92550 2023.
Yu Huang, Junyang Lin, Chang Zhou, Hongxia Yang, and Longbo Huang. Modality competition: What makes joint training of multi modai network fail in deep (provnn((prvvally). In International conference on machine learning, pp. 9226-9259 PMLR, 2022.
Adrián Javaloy, Maryam Meghdadi, and Isabel Valera. Mitigating modality collapse in multimodal vaes via impartial optimization. In International Conference on Machine Learning, pp. 9938-9964.
PMLR, 2022.
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.698, 2014.
Alex H Lang. Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Point- pillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12697-12705, 2019.
Zixing Lei, Shunli Ren, Yue Hu, Wenjun Zhang, and Siheng Chen. Latency-aware collaborative perception. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXII, pp. 316-332. Springer, 2022.
Yiming Li, Dekun Ma, Ziyan An, Zixun Wang, Yiqi Zhong, Siheng Chen, and Chen Feng. V2x- sim: Multi-agent collaborative perception dataset and benchmark for autonomous driving. IEEE Robotics and Automation Letters, 7(4): 14-19-999 2022.
Tsung- Yi Lin, Priya Goyal, Ross Girshick. Kaiming He, and Piotr Dollár. Focal loss for dense object detection, In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988. 2017.
Yen-Cheng Liu, Junjiao Tian, Nathaniel Glaser, and Zsolt Kira. When2com: Multi-agent perception via communication graph grouping. Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 4106-4115, 2020a.
Yen-Cheng Liu, Junjiao Tian, Chih- Yao Ma, Nathan Glaser, Chia-Wen Kuo, and Zsolt Kira. Who2com: Collaborative perception via learnalee handshake communication. In 2020 IEEE International Conference Robotics and Automation (ICRA), pp. 6876-6883. IEFF, 2020b.

---

Published as a conference paper at ICLR 2026
Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han. Bevfusion: Multi-task sensor fusion with unified bird's s-eye view representation. In 2023 IEEE international conference robotics and automation (ICRA), pp. 2774-2781. IEEE, 2023.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.
A convnet for the 2020s. Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition (CVPR), 2022.
Yifan Lu, Quanhao Li, Baoan Liu, Mehrdad Dianati, Chen Feng, Siheng Chen, and Yanfeng Wang. Robust collaborative 3d object detection in presence of pose errors. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 4812 4818. IEEE, 2023.
Yifan Lu, Yue Hu, Yiqi Zhong, Dequan Wang, Siheng Chen, and Yanfeng Wang. An extensible frame- work for open heterogeneous collaborative perception. In The Twelfth International Conference on Learning Representaations, 2024.
Xiaokang Peng. Yake Wei, Andong Deng, Dong Wang, and Di Hu. Balanced multimodal learning via on-the-fly fly gradient modulation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 8238-8247, 2022.
Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV 16, pp. 194-210. Springer, 2020.
Donghao Qiao and Farhana Zulkernine. Cobevfusion: Cooperative perception with iddr-camera bird's-eye view fusion. arXiv preprint arXiv:23 10.06008, 2023.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions on neural networks, 20(1):61-80, 2008
Nicholas Vadivelu, Mengye Ren, James Tu, Jingkang Wang, and Raquel Urtasun. Learning to communicate and correct pose errors. In Conference on Robot Learning, pp. 1195-1210. PMLR, 2021.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Shiming Wang, Holger Caesar, Liangliang Nan, and Julian FP Kooij. Unibev: Multi-modal 3d object detection with uniform bev encoders for robustness against missing sensor modalities. In 2024 IEEE Intelligent Vehicles Symposium (IV), pp. 2776-2783, IEEE, 2024a.
Tsun-Hsuan Wang, Sivabalan Manivasagam, Ming Liang, Bin Yang, Wenyuan Zeng, and Raquel Urtasun. V2vnet: -ehicle-to-vehicle communication for joint perception and prediction. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pp. 605-621. Springer, 2020a.
Weiyao Wang. Du Tran, and Matt Feiszli. What makes training multi-modal classification networks hard? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
12695-12705, 2020b.
Zhe Wang, Siqi Fan, Xiaoliang Huo, Tongda Xu, Yan Wang, Jingjing Liu, Yilun Chen, and Ya-Qin fa-Qin Zhang. Emiff: Enhanced multi-scale image feature fusion for rehicle-infrastrueture cooperative 3d object detection. In 2024 IEEE International Conference Robotics and Automation (ICRA), 2024b.
Yake Wei, Siwei Li, Ruoxuan Feng, and Di Hu. Diagnosing and re-learning for balanced multimodal learning. In European Conference on Computer Vision, pp. 71-86. Springer, 2025.
Hao Xiang, Runsheng Xu, and Jiaqi Ma. Hm-vit: Hetero-modal vehicle-to-vehicle cooperative perception with vision transformer. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pp. 284-295, 2023.
Hao Xiang, Zhaoliang Zheng. Xin Xia, Runsheng Xu, Letian Gao, Zewei Zhou, Xu Han, Xinkai Ji, Mingxi Li, Zonglin Meng. et al. V2x-real: largs-scale dataset for toveert-i-eerrythiing cooperative perception. In European Conference on Computer Vision pp. 455-470. Springer, 2024.
12

---

Published as a conference paper at ICLR 2026
Runsheng Xu, Zhengzhong Tu, Hao Xiang, Wei Shao, Bolei Zhou, and Jiaqi Ma. Cobevt: Cooperative bird's eye view semantic segmentation with sparse transformers. arXiv preprint ar arXiv:2207.02200 2022a.
Runsheng Xu, Hao Xiang, Zhengzhong Tu, Xin Xia, Ming-Hsuan Hsuan Yang, and Jiaqi Ma. V2x-vit: Vehicle-to-everythhing cooperative perception with vision transformer. In European conference on computer vision, pp. 107-124. Springer, 2022b.
Runsheng Xu, Hao Xiang, Xin Xia, Xu Han, Jinlong Li, and Jiaqi Ma. Opv2v: An open bench- mark dataset and fusion pipeline for perception with vehicle-to-vehicle communication. In 2022 International Conference on Robotics and Automation (ICRA), pp. 2583-2589. IEEE, 2022c.
Junjie Yan, Yingfei Liu, Jianjian Sun, Fan Jia, Shuailin Li, Tiancai Wang, and Xiangyu Zhang. Cross modal transformer: Towards fast and robust 3d object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 18268-18278 2023.
Yang Yang, Hongpeng Pan, Qing- Duag Jiang, Yi Xu, and Jinghui Tang. Learning to rebalance multi-modal optimization by adaptively masking subnetworks. arXiv preprint arXiv:2404.088. 083477 2024.
Haibao Yu, Yizhen Luo. Mao Shu, Yiyi Huo, Zebang Yang. Yifeng Shi, Zhenglong Guo, Hanyu Li, Xing Hu, Jirui Yuan, et al. Dair-v2x: large-scale dataset for rehicle-infrastruetue cooperative 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21361-21370, 2022.
Kaicheng Yu. Tang Tao, Hongwei Xie, Zhiwei Lin, Tingting Liang, Bing Wang, Peng Chen, Dayang Hao. Yongtao Wang, and Xiaodan Liang. Benchmarking the robustness of lidar-camera fusion for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3188 3198, 2023.
Binyu Zhao, Wei Zhang, and Zhaonian Zou, Bm2cp: Efficient collaborative perception with lidar- camera modalities. In Proceedings of The 7th Conference Robot Learning, Proceedings of Machine Learning Research, pp. 1022-1035, 2023.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 DETAILED ARCHITECTURE SETTINGS
We present more architecture settings of SiMO-AF in Table 5 for readers to quickly obtain some detailed information about the implementation. The same details can be found in the in the code. As for SiMO-PF, we adopt the exact settings of HEAL (Lu et al. 2024) to reuse their open-soureed model. The detailed settings can be found in opencood/bypes
yaml file
Baaiaaas__ammm__fuaeeeee yaml.
Table 5: Detailed SiMO-AF settings.
Modules
Submodules
Settings
Voxel Feature Encoder (VFE)
With normalization, absolute coordinates, filters644 Channels=64
PointPillar Scatter
ResNet: layens=]3, 5, 8), strides= 2, 2]. filters-[64, 128, 256].
Pointillar
BEV backhone
strides=[1, upsample_strides=(1. 4], upsample_fllerrs(1255 filters=[128, 128, 128]
Shrinker header
Convolutional layer: kernel _iiees,, stride=1, channel shrunk from 384 to 256
Aligner
Convnext: blocks=3, channels=256
Encoder
EfficientNet: img. feattureteee2,, downsample=x, with depih supervision
ResNet: layers=]3, 8J, strides= (2, 2, 2). filters:[64, 128, 256].
BEV backbone
LSS
upsample_stride==t. strides=(11 4), upsample filters=[128. 128, 128]
Shrinker header
Convolutional layer: dernel_size=3, stride=1, channel shrunk from 384 to 256
Aligner
Convnext: blocks=3, channels=256
LAMMA
Feature /eature_stride=22 feat_ caa__dim=56,, dim=256
Attussoo
dia=_dim226
Detect Head
Channels=256, anchors=2
A.2
ALGORITHM DESCRIPTION OF LAMMA
Algorithm presents the forwarding pipeline of our Length-Adaptive Multimodal Fusion module.
A.3 TRAINING AND COMPUTATIONAL COST ANALYSIS
Our training strategy to mitigate modal competition inevitably extends training duration. The training duration required for each step is detailed in Table can be observed that the Align+Fusion+RDD phase adds more training time than training single branch independently. This is because, apart from the pre-trained feature extractors, all subsequent modules necessitate retraining and fine-tuning to preclude any modal bias.
Although this training strategy, based on a pre-trained branch, increases the overall training time, offers greater certainty compared to grid-searching for the optimal training parameters of gradient modulation and forward dropout-and and surprisingly, the total training time ends up being shorter. The longer training duration arises specifically when compared methods that fail to address the issue of modal competition. We believe that extending the training period in controlled manner to tackle the problem of modal competition represents the lowest-cost solution available.
Table 6: Training stages and epochs.
STAGE
Training EPOCHS
PRETRAIN LUDARRAATTUSS
20
PRETRAIN (CAM (AM_AATFUSE)
20
ALIGN (SIMO-AF (L))
15
ALIGN (SIMO-AF (C))
5
FUSION (SiMO-AF (L+C))
5
RD FINE-TUNE
5
14

---

Published as a conference paper at ICLR 2026
Moreover, considering that the encoders are frozen in subsequent training. requiring only the training of downstream modules, this does not sigmificantly increase the requisite VRAM. Conversely, due to the separate training of the encoder and fusion module, the VRAM requirements are, in fact, reduced, which enabled us to complete SiMO's training on an RTX 3090 GPU.
Model size and computational cost analysis is presented in Table 7 While LAMMA'S parameters considerably larger than BEVFusion's 's multimodal fusion module, they comprise only 6.02% of the overall model, less than a Shrink Conv Layer. Thus, we contend this does not unduly increase model size. Similarly, FLOPS comparison shows LAMMA's incorporation does not substantially elevate computational burden.
Table 7: Parameter count and FLOPS comparison.
MODULE
SIMO-PF
BEVFUSION
LIDAR ENCODER (ENCODER_M))
768
768
CAMERA ENCODER (ENCODER_M2)
14,635,796
14,635,796
LIDAR BACKBONE (BACKBONE_MM) MM)
126,176
226,176
CAMERA BACKRONE (BACKBONE_M2)
267,136
267,136
ALIGNER (ALIGNER M2)
109.440
109.440
MULTIMODAL FUSION (MM (MM_FUSION FUSION)
1,312,512 (LAMMA)
73,856 (CONCAT+CONY)
PYRAMID BACKBONE (PYRAMID BACKBBNE
3,757.635
3.757.635
SHRINK CONY (SHRINK_ CONV)
1,475,072
1.475.072
DETECTION IEEADS
5,140
5,140
TOTAL PARAMETERS
21,789,675 0.0602
20,551,019
PROPORTION FUSION MODULE PARAMETERS
0.0036
FLOPS
223.95G
219.31G
A.4 COMMUNICATION COST ANALYSIS
We adopted HEAL'S Pyramid Fusion sequently aligning our communication bandwidth and latency with HEAL's. For the convenience of the readers interested in this issue, we calculated SiMO's communication volume using the same methodology as Where2comm (refer to Appendix 7.5 in Hu et al. 2022)), with the following results:
multi-agent fusion method without modification, con-
H x W x C 32
64 64 64 32
log2
loga
20
(5)
Here, and W represent the height and width the shared BEV features, respectively, and C is the number of feature channels. Multiplying by 32 indicates that features stored in float32 format, and dividing by 8 yields the communication volume in Bytes. The final metric is the base-2 logarithm of this value. The shared BEV feature map in SiMO has dimension of H-64, W=64, with C=64 channels.
For comparison, Where2comm's communication volume ranges from 13.84 to 25.93 (according to Table 4 in Hu et al. (2022)), demonstrating that SiMO's communication is also relatively concise. We believe that by integrating existing methods for reducing communication volume, SiMO's communication overhead can be further minimized in future work.
A.5 MORE VISUALIZATIONS
we can observe that by incorporating the camera modality, SiMO-AF's detection results have fewer false positives. AttFusion tends to misidentify objects similar size to vehicles as targets, as shown in the enlarged areas of the detection results. However, by integrating image features. SiMO-AF can correctly ignore such interferences. Additionally, SiMO-AF is more likely to mistakenly judge that there still targets behind the occluded area of the detected objects, leading to false positives. This situation often occurs in peripheral areas without the collaboration other agents, indicating that collaborative perception improve this condition.
From Figure 7
15

---

Published as a conference paper at ICLR 2026
(a) Full LiDAR points
(b) 15,000 LiDAR points
(c) 10,000 LiDAR points
(1)
HEAL PPrraid Fusion)
Figure 7: Visualization of SiMO-AF's (the 1st row) and Attuusion's 's (the 2nd row) detection results, ground truth and collaborative point clouds. (a)(b((c) detection visualizations with all, 15000 and 10000 LiDAR points. Typical detection errors are enlarged for analysis.
BM2CP' feature maps:
(a)
Camera
(b) LiDAR
(c) Fused
Figure The visualizations of BM2CP's features. (a) and (b) are respectively the camera and LiDAR feature before fusion, and (c) is the fused feature combining two modals.
A.6 THE LIMITATIONS OF BM2CP
From Table be seen that the performance of BM2CP somewhat inferior to that of AttFusion and Pyramid Fusion, even after combining two modal features. Here, we attempt to explore the reasons for this, but since the issue outside the scope of this paper's research, we include the analysis in Appendix.
We infer that the insufficient performance of BM2CP is mainly due to two factors. Firstly, the original setup of BM2CP was not adequately optimized to select the best hyperparameters. In contrast, the AttFusion method used is the result of continuous optimization in subsequent work, including the encoder and decoder replacement but only left AttFuse module the same the orignal, hence its performance better than the original BM2CP. Additionally, we attempted to visualize the features of the two modalities before fusion and the fused features in BM2CP, shown in Figure 8] We observe that compared to point cloud features, image features have significant locality. Their effective features concentrated in the vicinity of the agent, the perception range expands, the gain
16

---

Published as a conference paper at ICLR 2026
Camera Fewnises
EMA Sestures
Fuodd Ferrure
033163
033451
6390
0031
Figure 9: The visualizations of LiDAR and camera features aligned with LAMMA and the fused feature.
Algorithm LAAMA Forwarding
1: Input: Modality features Zm E Re*" th for m E M,, where M is the set of modalities (LM) = M)
2: Parameter: Positional encoding p RS Learnable matrices Wp Rdxe Wq
Rdxd WK E Rdxd Wy Rddd W out E Rexd
3: Output: Fused feature Zmm E Rexn
4:
5: Zeoncat < []
6: Add Positional Encoding and Initial Projection
7:
for .M do
8: Zm t- Zmm + P
> Add c-dimensional positional encoding to input features
9: Z + W,
Project features from e to
10: Zoonaa +
Zeonen: proj
> Concatenate projected features along the sample dimension
(d + d x Mn)
11: end for
13: Cross-Modal Query Initialization
Q Wq Zoneatt
Project to query space Q RRdxM
15:
16: Multi-Head Attention Fusion Blocks
17:
for MM do
18:
K,, f W K proj
Project to key space K,, E Rdxx > Project to value space Vm E Rdxx
19:
V
m + W v prjj
20:
Z (m)
all
MultiHead (Q, Km, V.))
Apply Multi-Head Attention; output d x Mn
Z (m)
Z (m) att
21:
att
LayerNorm
Zeconcat
A> Add residual connection and LayerNorm
22:
Z fused
(m)
LayerNorm MLP(Z (m)
(m) iit
Apply MLP and LayerNorm
23:
Z fused Sum(Split
(m)
(m) Tused
> Split d x Mn into M tensors of d x and sum them
end for
25:
26: Aggregation and Final Projection
27: Zmm Wont out (EmeM (m))
Sum modality features and project back channels
28:
29:
return Zum
Return the fused feature
that image features bring to the fused features relatively decreases. This might be because BM2CP does not have separate BEV backbone for image modality to encode and decode features, achieving feature alignment between the two modalities, but instead directly uses an attention mechanism to extract useful information for point cloud features from unaligned image features.
17

---

Published as a conference paper at ICLR 2026
Therefore, we believe that using separate BEV backbone to encode modality features into BEV features is beneficial for the semantic alignment of the two modality features and full utilization of the complementary information of multi multi-modalities. Moreover, aligning the semantics of multimodal features before fusion can bring a more effective fusion of modality information. For comparison, we have added visualizations of SiMO's features before and after fusion in Figure 2 which demonstrate that LAMMA effectively aligns the geometric structures of LiDAR and Camera features.
A.7 VALIDATIONS ON MORE DATASETS
As an emerging field, collaborative perception still lacks sufficient number of publicty avail- able datasets. Among the existing datasets, only
Table 8: 3D detection results on V2XSet of mod- els trained with our learning strategy and RD (in %).
OPV2V (Xu al. 2022ch. V2XSet (Xu et al. 20226). V2X-Sim (i et al, 2022). DAIR- V2X YYu et al. 2022) and V2XReal (Xiang al. 2024) provide multimodal data, while open-sourced real- world datasets are limited to DAIR-V2X and V2VREAL. Here, we validate SiMO on V2XSet to extend it to the Shhicle-to-vverthhing (V2X) scenario, and further explore DAIR-V22 and V2XReal for real-world validation.
Method
Modality
AP
AP
65,65
BEVFusion
2013
(.BB))
75.96
74,53
53.22
(RD)
HEAL ea 2024 (Pyne Poston
94.96
80,
56.09
46.72
31.71
(AlPusion)
SIMO
(xC))
C. (RI))
((RD)
36.40
52.10
86.95
$4.85
35,20
39.81
72.09
70:05
19.39
(Pyramid Fusion)
SIMO
LRD(
L (RD))
C. (KD)
91.09
93.66
68.16
90.44
92.66
56.42
$1.09
33.89
$5.85
V2XSet. V2XSet (Xu et al. 2022b) is large-
scale collaborative perception dataset that mirrors
the data structure and simulation environment of OPV2V. While OPV2V focuses on collaborative vehicles, V2XSet expands the scope of collaboration to include road infrastructure. Similar to the vehicles, these infrastructures are equipped with four RGB cameras for frontal, rear, right, and left perspectives, as well as a LiDAR sensor.
Table 8 demonstrate that SiMO maintains its adaptability to modal failure despite the perspective divergence among collaborative agents in vehicle-infrastructure scenarios. Moreover, SiMO and BEVFusion behave similarly to the results on OPV2V. For instance, SiMO-PF perform better with camera only, even compared with HEAL in camera-only setting. and BEVFusion cannot adapt to modal failure with RD fine-tuning.
DAIR-V2X V2X. DAIR-V2X (Yu et al. 2022 is the first large-scale real-world dataset designed for research on vehicle and road infrastructure collaboration. includes the perception data of a vehicle and a road infrastructure, both equipped with frontal RGB camera and LiDAR sensor. Table 2 shows that after RD, SiMO-PF shows notable accuracy drops with Rnll and dual modalities, while camera-only performance remains comparable to Pyramid Fusion. Similar results to other datasets suggest that SiMO may have adapted to modal failures, though the low accuracy of the camera-only approach makes this claim less reliable.
We attribute the poor camera-only results to DAIR- V2X's pingle-perspective setup. The LSS-based camera branch used in this study designed for multi-view fusion, extracting valuable depth infor- mation from parallax. In the absence of parallax across multiple views, it becomes challenging achieve convergence of the camera branch during Step 1. However, this issue does not stem from an inherent flaw in the SiMO fusion framework. Instead, SiMO's modular design allows for the flexible integration of more advanced single-view 3D detectors in the future-a key direction for our ongoing research. Furthermore, to our best knowledge, existing camera-based methods for DAIR-V -2X (e.g. Im VoxelNet Yu et al., 2022), EMIFF (Wang et al., 2024b)) achieve maximum AP@ 50 of 15.61% %. This suggests that the inherent limitations of the camera data in this dataset are the root cause of the lower performance.
Table 9: 3D detection performances of SiMO on DAIR-V2X (in %).
Mathol
Modulity
APG-30
APO:50
AP
HEAL La 2024
L
77.06
09.35
42.74
(Pyrmmh Farton
C
8.08
2.15
0.22
L-C L-C (wio RD)
74.56
64.51
29.31
SIMO
(RD()D)
68.07
51.82
18.84
(Pyramid Fucion) Fucioo)
L(RD)
66.89
52.33
19.35
(RD))
0.26
V2XReal. V2XReal (Xiang et al. 2024) is the latest real- wolld dataset containing a large scale of both multi-view RGB images and LiDAR point cloud in the V2X scenario. Although the dataset includes
18

---

Published as a conference paper at ICLR 2026
both point cloud and image data along with annotations, the absence of examples for importing image data in their open- source code, coupled with the annotation errors in some examples, has prevented us from successfully validating method this dataset.
A.8 THE EVALUATION OF ROBUSTNESS UNDER NOISY CONDITIONS
We introduced Gaussian noise with mean of 0 and standard deviation to the LiDAR point cloud coordinates to evaluate the robustness of SiMO under noisy conditions. The results Table To reveals a fascinating and crucial design trade-off: in noise-free or low-noise (a=0.1) environments, SiMO outperforms BEVFusion. However, as noise intensity escalates, SiMO's performance degrades more markedly than BEVFusion's.
Table 10: AP performance with LiDAR noise.
METROD
AP 30/50/70
LIDAR NOISE
0 = 0.1
or = 0.5
REVFUSION
0.95/0.94/0.81 94/0.81 8
0.95/0.94/0 943300 81
94/0, 9.30
0.77/0.76/0.5 76/0.55 0.81/0.78/0.55
0.93/0 92/0992/
SIMO PF
0. 0.98/0. 98/0.98/0.95
0,98/0.97/0.92
0.92/0. 90/0.73
0.71/0.67/0.43
Our analysis is that BEVFusion's 's advantage stems from its convolutional fusion mechanism. Convo- lution, as a potent local spatial filter, inherently learns to smooth and suppress noise at the data point level, achieving an implicit denoising effect.
SiMO's s characteristics are rooted its core design philosophy. Our method, through 'attention + addition', is meticulously designed to ensure that modal features, once mapped to shared space, largely retain their independence and integrity. While this design is paramount for addressing catastrophic modal failures (structural absence), it also implies that SiMO will 'trust' and faithfully propagate noise from individual modalities, as it lacks the local filtering/smont seeering/mmooiingg mechanisms inherent in convolution.
Regarding SiMO's performance degradation under high noise levels, this occurs because SiMO "trusts' features from different modalities, overlooking potential discrepancies in their noise levels. This underscores the importance of clean features in multi-modal fusion. To integrate SiMO's robustness to macroscopic failures with specialized denoising modules to achieve more comprehensive perceptual resilience, we propose augmenting the encoder in each modal branch with convolutional layers, similar to those in BEVFusion, smooth and filter noise. Concurrently, incorporating a suitable amount of random noise during training would enable the encoder to learn noise filtering, thereby extracting denoised features. This approach would allow the model to adapt to high- noise environments without requiring modifications to LAMMA itself.
A.9 THE CONTRIBUTION OF FACH STEP IN OUR LEARNING STRATEGY
Our learning strategy to overcome modality compe- tition includes 3 steps (Step 4 is applied for adapting to modal failure), respectively corresponding to the training of feature extractors, aligners and common downstream modules (LAMMA, multi-agent module and task heads). We conduct ablation experiments further explore the contributions of each step.
Table 11: The ablation study of our training method (measured in %).
AP#30
A7050
APo70
25.30
Wio ALBONERS
SIMO-AF
L.C
87.07
83.11
35.70
Since we inherited the pretrained AttFusion models as our branches, we start our training from Step 2, in which we focus on the training of Aligners gm- Despite the well-trained feature extractors having contributed to good performance, the inherent semantic differences between modalities can confuse the multimodal fusion module and worsen the performance. Aligners gm designed to map features from different modalities into feature spaces compatible with LAMMA, facilitating multimodal fusion. As shown in Table [I] SiMO-AF without aligners (in naively training) cannot converge to the same level as the best branch that is inherited from AttFusion's 's LiDAR model, indicating the necessity of aligners.
NAIVE TRAINING
SIMO-AF
L+C
96.00
94.26
78.49
SIMO-AF
DUR TRAINING
19

---

Published as a conference paper at ICLR 2026
In Step 3, freeze the branches to train the common modules. In comparison, naive training is not involved in freezing any components. Table [1 shows that without freezing the branches to avoid modality competition, the performance still suffers from decline of 8.76 in AP@ @0, even with the pretrained branches and aligners. Only SiMO-AF with aligners can deliver a comparable performance to the best branch with our learning strategy.
A. T-SNE COMPARISONS
To ascertain whether multimodal features achieve alignment, we conducted a t-SNE analysis on the unimodal and fused features of BEVFusion and LAMMA. We preserved 100 sets of the following features: (1) Raw features from BEVFusion encoders and fused features; (2) Raw features from our encoders (prior to alignment) and fused features; (3) Aligned L features, Aligned C features, and fused features from LAMMA. Subsequently, to mitigate computational intensity, we initially reduced the dimensionality of these features to 100 dimensions using PCA, followed by t-SNE visualization, which yielded the results presented in Figure TO
By comparing the t-SNE analysis results of BEVFusion and LAMMA, we can observe that both multimodal features from BEVFusion and LAMMA naturally form distinct clusters, highlighting clear modality gap in their representations. However, after LAMMA alignment, the clusters of multimodal features and the fused feature cluster exhibit striking structural similarities simlaarties-smmthinnn that was not evident in the features of BEVFusion and before LAMMA alignment. We believe this phenomenon reflects effective modality alignment, a claim further supported by quantitative Procrustes analysis results.
Notably, this alignment approach contrasts sharply with traditional contrastive learning methods. In contrastive learning, the typical outcome is the merging of multimodal features into a single large cluster, with each modality adopting identical distribution pattern. We attribute this result to contrastive learning's ' emphasis on minimizing inter-modalee modallty differences while emphasizing shared commonalities across modalities. Unfortunately, this process often leads to the loss of modality- specific features within the unified feature space- a critical drawback when comes to enabling independent operation for individual modalities. For this reason, instead of relying contrastive loss for modality alignment, we propose a novel training method that not only ensures effective alignment of multimodal features but also preserves the unique modality-speciff features.
A.11 THE EFFECT ALIGNER TRAINING ORDER
As mentioned in Section 3.3 SiMO de-
Table 12: The effect of aligners' training order (%).
signed to treat two modalities equally with- out any preset bias. So in the Step 2 of Sec- tion 3.4.2 the training order of aligners should not influence the adaptablity of modal failure. To confirm that, we train SiMO with two dif- ferent orders-LiDAR first and camera first.
ORDER
SiMO-AF (LIDAR FIRST)
SIMO-AF (CAMERA FIRST)
MODE
LeC
L
L+C
L
C
96.10
95.11
66.53
95.12
63.70
AP@
94.98
94.02
49.69
94.28
93.81
47,30
70
82.35
22.59
81.07
20.07
(1) Multimodal Pireed Feurares (BEVFasion)
(2) Multimodal & Fised Feeatunes (Defum LAMMA)
(3) Muliimobl & Fesod Festures (Afix LAMMA)
Feature Types
Types
Color by Sample
UDAR
dispbuusd
shilbary
Camera
1-SNE Component
INSEE Component
Rand
Component
Pigggh degree
stailluity
Peature Types Camera UDAR Fused
L-SNE Component
1-SNE Component 1
L-NN Component
Figure 10: Only features after LAMMA (3) show high degree of mirror symmetry similarity in
t-SNE analysis. Features' distributions of BEVFusion (1) and before LAMMA (2) show no similarity.
20

---

Published as a conference paper at ICLR 2026
Table 12 shows that SiMO can achieve the adaptability of modal failure regardeless of the training order of aligners.
Besides, reversed the input order of two modalities in LAMMA, and the minimal performance changes (-0.30%/0.36%% 3.3%% for AP@ 30/50/70) further proves that LAMMA does treat two modal features consistently without any modality perference.
A.12 THE EFFECT THE PROJECTION IN LAMMA
Since queries are concatenated for parallel computation LAMMA, the memory usage surges attention fusion, which makes LAMMA memory bottleneck. The convo- lutional Proj and the transposed convolutional Proj iinv in LAMMA aim to compress the BEV feature to process, saving the computation of Transformer block, and then restore the fused tokens back to BEV features. As the size of feature processed by Transformer is limited by GPU memory, Proj and Proj_inv in fact determine the size of modal features Zn. Without the ability to flexibly adjust feature sizes using Proj and Proj_inv, a model with fixed modal feature size of 64x64 cannot reach its maximum performance potential shown in Table
Table 13: The effect of feature size on
detection precisions (in %).
FEATURD Sizc
A
AP@50
AP@70
128 64*64
96.51 95.48
95.26
85.10
93.78
78.50
Table 14: Performance comparison with varying agents in heterogeneous modalities.
SETTING
HOMOGINBOUS MOBALITY
HETIROGINDOCIS MODALITY (L-800)
HETEROGUNIOUS MODALITY (C-650)
MODALITY
L-C
C
AlL
(1)
CAV_O
(C)
10AY_1
TCAV_2
(L)
(C)
1EAV_3 _
(L)
+CAY
A1x.
CAV_9
(C)
(L)
1CAY_1
"CAY_2
(C)
1CA
HCAV_4
(C)
APP-30 50
0.98
0.97
8.81
0.91
0.94
0.98
0.99
0.99
0.89
0.88
0.93
0.95
0.59
AP
0.98
0.97
0.70
0.97
0.93
0.98
0.90
0.09
0.85
0.84
0.50
0.94
0.82
AP
0.95
0.94
0.45
0.91
0.83
0.95
0.96
0.96
0.70
0.70
0.75
0.88
0.70
A.13
THE EFFECT OF THE NUMBER OF CAVS
Following related works (iiang et al. 2023 Lu et al. 2024). we conducted comparative experiments on varying CAV counts in heterogeneous failure scenarios. As shown Table 14 SiMO's perfor- mance generally improves with performance nears its upper limit. Conversely, in C-ego settings, performance initially improves with more agents but declines five CAVs. We posit this is due the final agent being camera-only, whose features introduce greater uncertainty, interfering with multi-agent fusion. This suggests Pyramid Fusion could be enhanced by assessing agent uncertainty. However, as SiMO does not involve improvements to multi-agent fusion methods, we do not consider this issue limitation of SiMO itself.
agents, especially in L-ego configurations, where collaborative
A.14 THE USE OF LARGE LANGUAGE MODELS (LLMS)
The study utilized LLMs to refine and enhance the paper based the author's original manuscript, as well as to make partial code modifications and assist bug fixes.
21


Reviews:
[{"review_id": "MStOg0Oaui", "reviewer": "Reviewer_Ntee", "summary": "This paper proposes SiMO to address a critical yet underexplored challenge in multimodal collaborative perception: system failure under partial modality loss. SiMO introduces two key components: i) Length-Adaptive Multi-Modal Fusion: a fusion module that handles variable numbers of input modalities via attention mechanisms and maintains semantic consistency before and after fusion through additive combination. ii) Pretrain-Align-Fuse-RD training strategy: a staged training protocol that mitigates modality competition by pretraining and aligning modality-specific branches independently before joint fusion, followed by fine-tuning with random modality dropout (RD). Experiments show that SiMO achieves state-of-the-art performance in full multimodal settings while maintaining strong single-modality performance.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-\tThe paper tackles robustness to sensor failure in collaborative perception, which is a realistic and safety-critical issue that is rarely explored in existing multimodal collaborative perception literature.\n-\tThe method is plug-and-play and demonstrated on two backbone frameworks (AttFusion and Pyramid Fusion), suggesting broad applicability.\n-\tStrong empirical validation: the paper presents comprehensive experiments across homogeneous/heterogeneous modality failures.", "weaknesses": "-\tWhile RD improves robustness as a data augmentation strategy, the 0.5 dropout probability is not justified theoretically or empirically. A sensitivity analysis would strengthen this design choice.\n-\tWhile LAMMA fusion is designed to handle missing modalities, how does it differ from simpler alternatives—such as replacing the missing modality’s feature with a zero tensor in existing fusion schemes?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y5coK7Hq4u", "reviewer": "Reviewer_AEwg", "summary": "The paper proposes SiMO (Single-Modality-Operable Multimodal Collaborative Perception), a multimodal collaborative-perception framework that (1) aligns modality BEV features into a unified semantic space, (2) fuses them with a Length-Adaptive Multi-Modal Fusion (LAMMA) module that is intended to gracefully degrade to self-attention when a modality is missing, and (3) applies a Pretrain–Align–Fuse–RD (PAFR) training schedule to avoid modality competition (pretraining branches, training aligners, freeze branches, then fuse + random-drop fine-tuning). Experiments on OPV2V-H and V2XSet (plus DAIR-V2X discussion) show that SiMO maintains reasonable detection performance with LiDAR failure / camera-only operation while prior multimodal fusion baselines collapse.", "questions": "In Figure 8, BM2CP’s feature visualizations are shown — why not visualize the features of your own method?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is, to the best of my knowledge, the first work in collaborative perception that explicitly tackles multimodal perception failure caused by missing modalities, with a particular focus on ensuring operability when only RGB inputs are available.\n\nIt identifies the inconsistency between pre-fusion and post-fusion features as the main reason for performance collapse during modality failure, and introduces LAMMA, a length-adaptive fusion module designed to maintain semantic consistency under varying modality combinations.\n\nThrough extensive experiments on multiple V2X collaborative datasets, the paper shows that each single-modality branch can function independently while maintaining competitive multi-modal performance, indicating that SiMO achieves strong robustness and communication efficiency under degraded sensory conditions.", "weaknesses": "The real-world validation is limited, and the camera results are inconsistent. On DAIR-V2X, the camera-only performance remains poor (the authors attribute this to the limitations of single-view LSS). This weakens SiMO’s claim of addressing single-modality operability in practice. The paper should (a) include stronger single-view camera baselines, or (b) tone down the claims regarding the single-view camera setting.\n\nComparisons with stronger or more recent modality-failure methods adapted to multi-agent settings (e.g., MetaBEV/UniBEV, contrastive alignment approaches, or simply stronger camera-only backbones) are recommended.\n\nThe scalability to more than two modalities and a large number of agents is unclear. How does the method perform with three or more modalities? Since the complexity varies with the number of connected queries, this should be discussed and, if possible, demonstrated experimentally.\n\nPose errors and asynchronous agents are not studied, yet collaborative perception is known to be sensitive to pose noise and temporal misalignment. The robustness to pose calibration errors or time delays should be evaluated or at least discussed (as prior works such as HEAL have shown this sensitivity).\n\nThe evaluation conducted solely on simulated datasets is not sufficiently convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A4pNq7GLHm", "reviewer": "Reviewer_gwXL", "summary": "This work presents the single-modality-operable multimodal collaborative perception framework, which is the first attempt to address modality failure in the context of multi-agent collaborative perception. The proposed SiMO employs Length-Adaptive Multi-Modal Fusion to effectively leverage the remaining modal features during modality failures while preserving semantic consistency across modalities. To reduce modality bias in modality competition, SiMO further proposes a “Pretrain–Align–Fuse–RD” training paradigm. The paper provides thorough analyses and extensive experiments, validating the effectiveness of the proposed approach.", "questions": "1. In the training process, Step 2 is sequential, aligning LiDAR first and then camera. This actually favors camera performance. My question is : when more modalities need to be aligned, how would this ordering be determined?\n\n2. What is the exact input and what are the parameters used for the t-SNE visualization? Please clarify the implementation details.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This work is the first to address the problem of modality failure in the context of multi-agent collaborative perception, and it proposes an effective and practical solution through the SiMO framework.\n2. The paper is supported by extensive experiments, complemented by insightful visualizations and in-depth data analyses.\n3. The manuscript is clearly written and logically structured and the technical content is easy to follow.", "weaknesses": "1. The paper does not clearly explain the unique challenges of modality failure in the multi-agent collaborative perception (MACP) setting compared to the single-agent case. Similar modality failures could also appear in single-agent multi-modality scenarios. The authors should clearly explain why existing single-agent modality-robust methods are inadequate in the MACP context, and ideally include comparative experiments to substantiate this claim.\n\n2. The training procedure is complex. It requires four separate end-to-end training stages, resulting in a large training overhead. The authors should provide a detailed analysis of its training robustness and efficiency.\n\n3. The description of “heterogeneous model failure” in Table 2 is unclear, making it difficult to understand what modalities the ego vehicle and neighboring agents respectively possess in this setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1Lq4udjXAF", "reviewer": "Reviewer_nBTy", "summary": "This paper introduces SiMO (Single-Modality-Operable Multimodal Collaborative Perception), a novel framework designed to enhance the robustness and reliability of multi-agent collaborative perception (MACP) systems against sensor failures. The central problem SiMO addresses is that existing multimodal fusion methods in MACP are fragile, similar to a \"series circuit,” failing completely when a critical sensor like LiDAR is unavailable.SiMO's solution functions like a \"parallel circuit,\" allowing the system to operate with any single, effective modality. It achieves this through two main innovations: 1) Length-Adaptive Multi-Modal Fusion (LAMMA): This plug-and-play fusion module first aligns features from different modalities (LiDAR and Camera) and then integrates them using additive fusion to maintain a consistent feature space before and after fusion. 2) Pretrain-Align-Fuse-RD (PAFR) Training Strategy: This strategy addresses modality competition to ensure that each individual sensor branch is comprehensively and independently trained, thus strengthening the performance of the single-modality operations.", "questions": "no questions.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. High Robustness Against Modal Failures: SiMO explicitly address the complex challenge of dynamic, heterogeneous modal failures in Multi-Agent Collaborative Perception, which is important for real-world safety-critical deployment of V2V.\n2. The core idea of aligning and fusing features using addition to ensure semantic consistency between pre- and post-fusion feature spaces is straight-forward and effective.\n3. Proposed LENGTH-ADAPTIVE MULTIMODAL FUSION (LAMMA) can handle varying input modalities. With RD training, the performance of camera-only collaboration gets better, indicating an alleviation in modality competition.", "weaknesses": "1. This framework necessitates multiple stage training, which increases the training complexity.\n2. Late fusion metrics are not reported.\n3. The results on DAIR-V2X is not persuasive. While authors attribute the poor performance to the single-camera sensor and out-of-date camera-based detector, it would be encouraging to implement a more modern camera detector to validate the SiMO's effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)