Availability-aware Sensor Fusion via Unified Canonical Space
Authors: Dong-Hee Paek
Seung-Hyun Kong
Venue: NeurIPS 2025 poster ()
Keywords: Sensor Fusion
4D Radar
LiDAR
Camera
Sensor Degradation
Sensor Failure
Autonomous Driving
Primary area: applications

Abstract: Sensor fusion of camera, LiDAR, and 4-dimensional (4D) Radar has brought a significant performance improvement in autonomous driving. However, there still exist fundamental challenges: deeply coupled fusion methods assume continuous sensor availability, making them vulnerable to sensor degradation and failure, whereas sensor-wise cross-attention fusion methods struggle with computational cost and unified feature representation. This paper presents availability-aware sensor fusion (ASF), a novel method that employs unified canonical projection (UCP) to enable consistency in all sensor features for fusion and cross-attention across sensors along patches (CASAP) to enhance robustness of sensor fusion against sensor degradation and failure. As a result, the proposed ASF shows a superior object detection performance to the existing state-of-the-art fusion methods under various weather and sensor degradation (or failure)  conditions. Extensive experiments on the K-Radar dataset demonstrate that ASF achieves improvements of 9.7\% in $AP_{BEV}$ (87.2\%) and 20.1\% in $AP_{3D}$ (73.6\%) in object detection at IoU=0.5, while requiring a low computational cost. All codes are available at https://github.com/kaist-avelab/k-radar.

Availabiliy--wwre Sensor Fusion via Unified Canonical Space
Dong-Hee Paek
Seung-Hyun Kong"
CCS Graduate School of Mobility
KAIST {donghee, saek,, skong) kr.kr
Abstract
Sensor fusion camera, LiDAR, and 4-dimensional (4D) Radar has brought a significant performance improvement in autonomous driving. However, there still exist fundamental challenges: deeply coupled fusion methods assume continuous sensor availability, making them vulnerable to sensor degradation and failure, whereas sensor-wise cross-attention fusion methods struggle with computational cost and unified feature representation. This paper presents availability-aware sensor fusion (ASF), a novel method that employs unified canonical projection (UCP) enable consistency in all sensor features for fusion and cross-attention across sensors along patches (CASAP) to enhance robustness of sensor fusion against sensor degradation and failure. As result, the proposed ASF shows a superior object detection performance to the existing statc-of-the-ar fusion methods under various weather and sensor degradation (or failure) conditions. Extensive experiments on the K-Radar dataset demonstrate that ASF achieves improvements of 9.77% in APBEV (87.2%) and 20.1% AP3D (73.6%) in object detection IoU=0.5, while requiring low computational cost, All codes are available at bhtpp://gitaan wttp:/oooccccaeeeaeee hetp://kiitt-vvlllle
Introduction
Autonomous driving technology has advanced rapidly, with multiple companies adopting multi-sensor fusion approaches that combine two or more sensors, such as cameras, LiDAR, and 4-dimensional
(4D) Radar, to achieve more robust and reliable perception (Badue et al. 202T Wang et al. 2020).
Cameras uniquely provide color information but struggle with depth estimation; LiDAR delivers resouutoooltion 3-dimensional (3D) point cloud data but is less reliable in adverse weather conditions (Zheng et al., (223a);; and 4D Radar, despite its relatively low angular resolution, offers robustness in adverse weather and directly measures relative velocity (Paek et al., [022] Palffy et al., 2022,
Kong et al.] 2024 [Sun and Zhang 2021). This complementary nature initiated sensor fusion between camera, LiDAR, and 4D Radar to improve perception performance and reliability compared to single-sensor Yan et al., 2023 Liang al. 2024 Liu et al. 2023a Zheng et al. (023b).
Most multi-modal sensor fusion methods can be categorized into two methods. The first is deeply coupled fusion (DCF), which directly combines feature maps (FMs) extracted by sensor-specific encoder, as illustrated in Fig. (a). While is simple to implement and computationally efficient with excellent performance in various benchmarks (Chae et al. 2024 Liu et al.) 2023a Liang et al. 2024 Zhao al. 2024a] Caesar et al., 2020; Geiger et al. 2013). assumes all sensors are functioning properly and consistently. This makes DCF vulnerable to sensor degradation due to adverse weathers, -amages and sensor failure. Moreover, DCF requires retraining the entire neural network when the number of sensors changes, the size of the fused FM (i.e., the input to the detection
ccoreepponiin author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

View trusforn
Pacte
Unified
W/M
Cross ottenton scress setsees aloss
Camen IM (FV)
Camara PM (B6%)
compling
caation
Fug FM
Cunera FM (P
Camaaa
projectio
canotical
pradhcs w/ pustliatur auanadiaaiios
Reforenee quaiics (Rry))
Fined FM
Avnilubilirr
bucon [Sectice 2]
Camer F31 elba
LIDAR 4D Radar TM (MEV)
Rada Fdd soukkadd
Okj GT
Mentcal
(boaity
LEDAR FM
Eilsts 00 02
Desecias brac*
Obl
Scnser combitation
foecton 33
① LiDAR or 4D Raiai FM vaa Casesa 231 contened
(a) Deoplyy fasion (DCP; 3DD
3D-LRRF)
epestion)
4D Radar FM
Coasee Sttea Fregue)
Pos-mbodding
Patckes Oo R/VV
(1) Addresing smoor degratation
(2) Adktrescing smoor fultnes
comping
Festure
Transfeencer decodee hesl
Availesiiiz-avnn
Availiiile -owar fasios
C Camana FM (PS)
Careera FM (Depch mibed(en))
Qai Obj
cg. weallers
Infertase wish sulliple
Festure compligg
Pos ee--enbedring
Ohj
Availshiirs-ona fasien*
Availbhilis-one owaes
faxims*
GT
2) Infertice ouh with 4D Rasie scrsor (... Craasa 1161. fel)
LIDAR 493 Hoddar TM
LIDAR
Damagad seface"
41 Kader IM (Po-ezbeldel)
(BEV)
tuinel
danggel date.
"Idertical newook weights acness eccmppe cases.
Scrost-nise alluntion fusium (SCF; s.g.
(c) Avallabolliy-awarr fusian
Figure I: Comparison of sensor fusion methods: (a) DCF (e.g.. 3D-LRF (Chae et al. 2024 (b) SCF (e.g., CMT (Yan et al. 2023)), ) and (c) ASE. FV, BEV, Obj.. GT, Lets and Lreg stand for 'rrontview'' 'bird's 's eye-view "objects' 'ground truths', 'classification loss', and 'regression loss', respectively. 'Feature coupling' refers to methods that combine features from multiple sensors to create new features. Optional components in dashed lines; for example, (Vora et al., 2020) combines camera and LiDAR features without transforming the camera viewpoint, while Yan et al. 2023) fuses features through transformer decoder head (Carion al. 2020) without explicit feature coupling. ASF does not apply feature coupling to ensure independence between sensors.
head) changes. The second method is sensor-wise cross-attentiin fusion (SCF), which divides features extracted from each sensor into patches with positional-embeddiin (e.g., depth information for camera (Liu et al. 2022 Wang et al. 2022)) and selectively combines available patches using handle cases where some sensors are degraded. However, SCF does not have sensor scalability, since the method does not project sensor-specific features into a
cross-attention (Fig. (b)). allowing
standardized representation (Zheng et al., 2024 Liu al.) 2023b Wang et al.) 2016). In addition, SCF incurs computational complexity that scales with the number of patches, resulting in substantial computational overhead when processing multiple sensors with numerous patches (Fent et al. 2024
Yan et al. 2023 Bai et al. 2022).
One of the fundamental limitations of existing fusion methods stems from inconsistencies in feature representation across different sensors (Shashua 2024 Yeong et al. 2025). Cameras produce 2D RGB images, whereas LiDAR generates 3D point clouds, and 4D Radar produces low-resolution tensors with power values. Therefore, each sensor extracts features with different representations for the same object, making consistent fusion challenging (as shown Fig. (a)). To address this inconsistency. an ideal strategy could project features from different sensors into a unified canonical space for fusion. The concept of 'True Redundancy (Shashual 2024)* that ensures sensor independence while maintaining canonical feature representation for any sensor combination, suggests a promising direction for highly reliable and robust sensor fusion.
Therefore, we propose availability aware sensor fusion (ASF) method (Fig. [[ in which each sensor performs independently while being complementarily fused through projection to a unified canonical space. As a result, the proposed method addresses the limitations of both DCF and SCF simultaneously. The key innovation of ASF is in two sub-modules; First, unified canonical projection (UCP) projects features from each sensor into a unified space based on common criteria (i.e., canonical). Since UCP is optimized using the same reference query for all sensors, inconsistencies in sensor features are climinated. While Fig. (a) shows sensor features represented without clear patterns, Fig. (b) demonstrates how UCP aligns the features from each sensor the fused feature. Second, cross-attentiog across sensors along patches (CASAP) estimates the availability of sensors through patch-wa cross-attention on features projected into the unified canonical space, assigning higher weights to features from available sensors and lower weights to features from missing or degraded sensors. Unlike SCF that applies the cross-attention across all sensors (N,) and patches (N,) (i.e., O(N, ONN,N,N, for N, queries), ASF only applies the cross-altention across sensors along patches (i.e., N.).. Because of this, ASF eliminates complex positional-embeddee and improves computationally efficiency. Additionally, it applies normalization ensure that (camera, LiDAR.

---

Convention dlmeuur:: Tiood
Canona LaDAR
Raiar
Convention of weathers:
Sleet, O A Nomual t HeRry unos
CASAP
I-SN companiii!
I-S oSNEooapooei
LSSE comconical
(a) Outpat features of scusor specifr-ee eucoders
(b) Outpat features of UCP
(c) Output features CACAN
Figure 2: Visualization of feature representation with t-SNE (van der Maaten and Hinton 2008 at different stages of ASF for 'Sedan' class. Red, green, blue, and gray dots represent features from camera, LiDAR, 4D Radar, and fused features, respectively. Symbols in solid lines such as circle and triangle, square, and star indicate normal, sleet, and heavy snow conditions, respectively. (a) Initial output features from sensor-specific encoders show inconsistent distribution across sensors. (b) After unified canonical projection (UCP), features become better aligned to the fused feature. (c) After cross-attention across sensors along patches (CASAP), features from available sensors form cohesive clusters (in dashed symbols) based on weather conditions. Note that in adverse weather, camera features show larger deviation due to the degradation. Additional visualizations are in Appendix B.
and 4D Radar) features can be processed consistently by the detection head regardless of sensor combination. In contrast to Fig. (b), where each sensor's features remain separate, Fig. 2 (c) shows how sensor features cluster together after CASAP (except for camera become useless adverse weathers, as shown in Fig.3). This enables ASF to flexibly handle sensor degradation or failure and to achieve the reliability embodied in the "True Redundancy' concept (Shashual 2024) for autonomous driving implementation.
To integrate the availability awareness into the detection head, we propose sensor combination loss (SCL) that optimizes learning outcomes across all sensor combinations. SCL considers individual sensor unavailability during the training, enabling the system maintain high performance in the presence unexpected sensor failure or adverse weather conditions. The effectiveness our proposed ASF method has been validated on the K- K-Radar dataset (Pack et al. 2022), demonstrating improvements of 9.7% in APBEV (87.2%) and 20. % in AP3d (73.6%) for detection performance at loU=0.5 compared to ttate-of-the-ar (SOTA) methods (Chae et al. 2024 Huang al.) [0025). which includes the performance in extreme situations such as sensor degradation or failure (i.e., unavailable).
The main contributions this paper are as follows: (1) We propose ASF based on UCP and CASAP that achieves superior performance to SOTA methods and robust performance against sensor degradation and failure. (2) We propose SCL for the loss function to optimize the detection performance for all possible sensor combinations. (3) Through extensive experiments the K- RaRad dataset, we demonstrate that ASF achieves the high performance with low computational load.
The remainder of this paper is organized as follows: Section introduces existing methods, and Section describes the proposed ASF in detail. Section[ppresent experimental settings and results on the R-Radar dataset, analyzing performance in various scenarios including sensor degradation and failure. Section Dection33conee concludes the paper with summary and discusses the limitations. All codes and logs for ASF are available at https://titteee com htte//aist--ae settt-vvllabbkraaaaeea
2 Related Works
2.1 Deeply Coupled Fusion (DCF)
DCF constructs fused feature map (FM) by concatenating FMs from each sensor. Most studies
focus on fusing camera with LiDAR or 4D Radar (Liu et al. 2023a Liang et al. 2024 Zheng et al.
2023b Xiong et al. 2023). or combining LiDAR and 4D Radar (Chae et al.) 2024 Huang et al.
2025). Implementations range from directly fusing the front-view (FV) camera image with LiDAR
or 4D Radar without view transformation (Vora et al. 2020 to applying Iearnable BEV transforms
3

---

(Philion and Fidler 2020) and concatenating at the BEV stage (Liu et al. 2023a Liang et al. 2024 Zheng et al. 2023b). DCF method is straightforward to implement and computationally efficient compared to SCF, demonstrating strong performance across multiple benchmarks.
Recent DCF studies have improved performance by applying feature coupling, where the FMs of each sensor are enhanced with FMs of other sensors using multi-layer perception (MLP) or attention mechanisms (Vaswani et al. 2017). 3D-LRF (Chae et al. 20224 demonstrated superior performance to the conventional DCF's (Liang et al. 2024 Liu et al. 2023a) by applying attention between LiDAR and 4D Radar voxel features before concatenation. L4DR (Huang et al., (025) achieved SOTA performance on the K-Radar dataset by weighting each sensor's s FM using coupled BEV FMs with LiDAR and 4D Radar. However, DCF assumes all sensors are functioning normally, as they rely on the fused FM constructed by concatenating FMs from all sensors, making DCF vulnerable to sensor degradation and failure. This limitation arises because the training process does not expose the model to potential sensor degradation or failure scenarios that commonly occur during deployment.
2.2 Sensor-wise Cross-attention Fusion (SCF)
SCF divides sensor-specific FMs into patches and dynamically combines them through cross-attention in a transformer decoder head (Yan et al. 2023). inherently accommodating sensor availability. TransFusion (Bai et al.) 2022) is the first SCF that addresses sensor availability, but its sequential fusion method (e.g., performing LiDAR detection first and then fusing with camera data) makes inference impossible when LiDAR is unavailable. CMT (Yan al., (023) presents availability- aware sensor fusion of camera and LiDAR data using a transformer decoder head without applying feature coupling to individual sensors. However, without feature coupling for camera, CMT relies on positional embedding to incorporate depth information into camera FMs (Liu et al. 2022), which results in 3D patches (H W x D). This leads to computational complexity O(N, N.N (where Ng. N.. and Np are the number of queries, sensors, and patches, respectively), causing explosive growth in computational cost and memory usage. For instance, CMT requires 8 A100 GPUs with 80GB VRAM to train with batch size of 16.
Recently, DPFT (Fent et al. 2024 creates independent FMs and projects sensor-agnostic query points onto different FMs to verify sensor availability, achieving 56. 1 AP3 at loU=0.3 (using only 4D) Radar and camera). Unlike methods that utilize entire FMs, DPFT achieves reasonable computational efficiency by employing deformable attention (Xia et al. (022) that considers varying receptive fields using only small number of key points. However, similar to CMT, DPFT performs object detection using variable object queries, which does not establish a common representation across different sensors (as illustrated in Fig.
3 Proposed Methods
3.1 Sensor Fusion Framework
The overall sensor fusion framework consists of three stages: (I) sensor-specific encoders (i.e., backbones) that extract same-sized bird's eye-view (BEV) feature maps (FMs) from each sensor data (i.e., RGB image, LiDAR point cloud, and 4D Radar tensor), (2) the proposed availability-aware sensor fusion (ASF) network that described in following subsection 3.2 and (3) a detection head that detects objects from the fused FM.
Focusing on our ASF contribution, we utilize established methods for the specifc encoders and detection head. Specifically, we adopt BEVDepth (Li et al. (023), SECOND (Yan et al. 2018)) and RTNH (Paek et al.) 2022) backhones for camera, LIDAR, and 4D Radar, respectively, along with SSD detection head (Liu et al.] 2016). Further specifications regarding the overall sensor fusion framework, such as sensor-specific encoders and detection head, are provided in Appendix A.
3.2 Availability-wware Sensor Fusion (ASF)
As illustrated in Fig. (c), the proposed ASF consists of two key components: unified canonical projection (UCP) and cross-attention across sensors along patches (CASAP).
Unified Canonical Projection (UCP). One of the key challenge in multi-modal sensor fusion is the inherent inconsistency of features from different sensors (Shashual 2024 Yeong et al. 2025). as

---

visualized in Fig. (a). To tackle this, we divide BEV FMs into an equal number of patches for all sensors and train projection functions to transform features from each sensor into unified space based on the same criteria (i.e., reference query in CASAP). To formally define our methods, we first represent the same-sized FMs of each sensor as:
(I) where C₂ denotes the channel dimension for sensor 8, H and W represent the identical height and width of BEV FMs for all sensors, respectively, and Sc, St. and Sn denote camera, LiDAR, and 4D Radar sensors, respectively. Each FM is then divided into patches Fý, with height Pnr and width Pw:
FM E R.. 8 E (Sc. SL,
(FMM) (F (F,,, ERCxXPXPP i=1::p)
(2)
where Tp(.) is the operation that divides each FM into patches, N2 (H/PH) (W/Pw) is the number of patches, which is identical across all sensors since each FM has the same spatial size, and 'I' denotes 2I. ' Note that since the patches are already spatially aligned (i.e., F F and F correspond to the same position), our method eliminates the use of computationally expensive pesitional-mmbeddingg (Liu et al., 2022) required for SCF (Yan et al. 2023 Fent et al., 2024). Then, we apply a parallel operation along patches that projects each patch to have the same channel dimension Cu. This is the UCP operation (-) that transforms sensor-specifie patches into patches in a unified canonical space. The UCP- -CPppeccsseed patch P; for each sensor is expressed as:
P = FF N. (-) LN(rroj (-)), Proj(.) GeLU(ML
F uez (IN(() p. ",, i=1::pp
(4)
where LN and n, denote layer normalization (Ba al. 2016) for training stability and the number of sequential projection functions incorporating MLP for transformation and GeLU (Hendrycks) and Gimpel| 2016) for non-linearity, respectively. While our framework allows for repetition of the projection function to increase non-linearity, with or 2 repetitions being sufficient (we use n, = 2, which aligns features as demonstrated in Fig. (b)). Note that (f" is trained separately for each sensor based on reference query, which results in alignment of features from all sensors with respect to the fused feature as shown in Fig. (
Cross-attention Across Sensors Along Patches (CASAP). The patches F., projected into the unified canonical space by UCP serve as keys (K) and values (V) for a trainable reference query Qref E RN, xC, (where N, is the number of queries Q), and we perform cross-attention across sensors along patches as:
(5) where Q'eef is the output of the cross-attention applied across sensors for the i-th patch. Since Qref is trained primarily on features that are mostly available in the training data, it naturally develops high correlation (i.e., high attention scores) with patches from available sensors after the training. Consequently, during inference, Q/e f. is predominantly composed of available F,, The number of heads in cross-attention is a hyper-parameter whose impact is analyzed in subsection 4.4
Q'ef, (nr.=-CoosAttt(----. K&V a.J F., i=
Compared to ASF, cross-attention in SCF is performed across all patches with respect to object queries Qobi in the transformer decoder head. This can be mathematically expressed as:
dtty--Crose -CrossAttn (Q=)( {&VEF F Sc FSc. SS SS: PSR FSR pe....... Ne,N," pe. pe.N, Ie. pe.N,
(6)
where Fs represents patches with rrsitional-embedding and Rea (usually larger than (Liu et al. pe
2022 Yan et al. 2023)) is the number of stacked transformer decoders. Eq. 6 shows the cross- attention across all sensors and all patches (i.c., the K&V set contains N,.N, patches, resulting in
ONN,NN computational complexity for N₂ queries). In contrast, in Eq. 5 the cross-attention is applied across sensors and along patches, which requires only O(N, computational operations. This is significant computational costs reduction as N, < N Moreover, demonstrated in Big.Baand Tab. [ ASF achieves better performance with only a single cross- coss-attention layer than SCF utilizing stacked cross-altention layers (i.e., Aha 26). 6).
Sequentially, ASF applies post-feature normalization (PN) N that has a similar structure to (+) with LN, to ensure that features can be processed consistently by the detection head regardless of
5

---

Ground trabss
Prediction
ditesionss
Falsc alamss
Numna weathrr
Ground tuth
C-L HR
IJ 7.3.225551 1961
11.3 11,3333300 3/50. [55]
69
(h)
(c)
Nertal wesher
43.5/56.5
6
49
Netmal wesher
100.0 [%]
100.0 [%6]
100.0 [56]
Advitsi weater
Ground mruth
C+L-R
11.2933 [%5]
12.4/92.854.00
(A)
(1
Adveme werter
L.R
CUR
C11
D
D
38 16.366.1 [%6]
34,6965.4
46.7/33 [%6]
Advers werher
A
100.0 [7]]
100.0 [S6]
100.0 [S]]
(p)
69)
(
Figure 3: Qualitative results ASF various sensor combinations. show results normal and adverse weather conditions in (a-i) and (j-r), respectively, where employed sensors noted in the top-left corner (C: Camera, L: LiDAR, R: 4D Radar, C*: damaged camera). Each subplot visualizes front-view camera image, LiDAR point cloud, 4D Radar tensor, and sensor attention map (SAM) showing attention score distribution from cross-attention in CASAP. In the SAMs, red, green, and blue represent attention scores for Camera, LiDAR, and 4D Radar, respectively. For example, a predominantly blue SAM indicates that 4D Radar receives the highest attention scores, meaning that 4D Radar is used primarily for detection in the scene. The bottom-left comerr of each subplot shows the proportion of attention scores in colored percentages (C/L/R[%]), Note that predictions are visualized all sensor data, when sensor is employed for detection (e.g., predictions from L+R are also visualized on the camera image).
the sensor combination. Therefore, PN enables camera, LiDAR, and 4D Radar features for the same object to be consistent. The set patches P,, with PN formulated as:
=N -NL,NNNNLLOOON... ER i=1:N Pa = F.,
(7)
((ProN((Proj( Proj() P
where n, denotes the number of sequential projections, with 1 or 2 being sufficient to increase non- linearity. Unlike Fig. (b) where features from different sensors occupy distinct regions, Fig. (c) illustrates how PN causes sensor features to converge into unified clusters.
Finally, to transform P., back the original BEV size (H x W), apply reshape operation T.(.). the fused FM FMusse
C, Since the size of P, is C2 x C (, (H/H))(() can be obtained as:
FM = =Tx(P, ER

---

Table 1: Performance comparison of 3D object detection K-Radar (Pack al. 2022) benchmark v1.0. C, L, and R represent Camera, LiDAR, and 4D Radar, respectively. The bold and underlined values indicate the best and the second-best, respectively. Note that the ASF models for sensors L+R and C+L+R share the same neural network weights trained with C+L+R, which means that ASF for L+R represents scenario where camera becomes unavailable. Nor., Ove., Sle., I...., and H.s.
denote 'Normal,, 'Overcast', 'Sleet', 'Light snow", and 'Heayy snow' respectively.
Methods
Sensors
IoU
Metric
Total
Nor.
Ove.
Fog
Rain
Sle.
L.s.
H.s.
BEV
41.1
41.0
44.6
45.4
32.9
50.6
81.5
56.3
0.3
RTNH Pack et al.
37.4
37.6
42.0
41.2
29.2
49.1
63.9
43.1
R
BEV
36.0
35.8
41.9
44.8
30.2
34.5
63.9
55.1
0.5
3D
14.1
19.7
20.5
15.9
13.0
13.5
21.0
6.36
BEV
76.5
76.5
88.2
86.3
77.3
55.3
81.1
59.5
0.3
RTNH
72.7
76.5
84.8
64.5
53.4
80.3
52.9
L
Pack et al.
BEV
66.3
65.4
87.4
83.8
73.7
48.8
78.5
48.1
0.5
3D
37.8
39.8
46.3
59.8
28.2
31.4
50.7
24.6
BEV
84.0
83.7
89.2
95.4
78.3
60.7
88.9
74.9
0.3
3D-LRF
74.8
81.2
87.2
86.1
73.8
49.5
87.9
67.2
L+R
Chac ct al.)
BEV
73.6
72.3
88.4
86.6
76.6
47.5
79.6
64.1
0.5
3D
45.2
45,3
55.8
51.8
38.3
23.4
60.2
36.9
BEV
79.5
86.0
89.6
89.9
81.1
62.3
89.1
61.3
0.3
L4DR Huang ct al.
78.0
77.7
80.0
88.6
79.2
60.1
78.9
51.9
L+R
BEV
77.5
76.8
88.6
89.7
78.2
59.3
80.9
53.8
0.5
3D
53.5
53.0
64.1
73.2
53.8
46.2
$2.4
37.0
BEV
88.6
88.1
90.3
99.0
89.1
80.4
89.4
78.7
0.3
87.3
86.6
89.8
90.7
88.6
80.0
88.8
77.5
L+R
BEV
87.0
86.2
90.2
90.8
88.8
78.2
88.6
71.0
0.5
ASF (Proposed)
3D
72.9
64.6
86.6
79.6
73.4
70.0
77.6
66.7
BEV
88.6
88.2
90.2
98.9
89.0
80.4
89.2
78.4
0.3
87.4
87.0
90.1
90.7
88.2
80.0
88.6
77.4
C+L+R
BEV
87.2
86.7
90.1
90.8
88.7
78.3
88.3
70.9
0.5
3D
73.6
71.8
87.0
79.4
73.0
67.5
78.0
66.4
where Cg is the quotient of Ca/(PH Pw) design C.= xP X Since the resulting channel dimension Cg may be insufficient for containing feature representation due to reduced channel dimensions after reshaping, in the implementation, we increase the number of patches by Consequently, the channel dimension of
a factor of up (i.e., Ny=ND i x Nhn-NN==,, FM increases from Cg to up Cg. and impact of this modification evaluated in subsection 4.4
3.3 Sensor Combination Loss (SCL)
Leveraging the consistent size of F (which serves as the input to the detection head) regardiess of sensor combinations, we propose an SCL that enables simultaneous optimization across multiple sensor configurations. The pro- posed SCL formalized as:
Table 2: Comparison of VRAM and FPS evaluated on the K-Radar benchmark v1.0. The bold and underlined values indicate the best and the second- best, respectively. The unit of VRAM and FPS are GB and Hz, respectively.
LSCL= (Cdl,s Lreg.s), #ESC
(10)
Methods
Sensors
VRAM
FPS
3D-LRF (Chae et
L+R
1.2
5.04
DPFT (Fent et al.
C+R
4.0
11.5
where SC represents the set of 7 possible sensor combinations (Sc-only, Sy-only, Sn-only, SL+ Sa. Sc+Sn. Sc+ and Sc++ where Sc. Sz, and S denote camera, LiDAR, and 4D Radar, respectively. Lete. and Creg.s are the classification and regression losses for each sensor combination. SCL explicitly prepares for the potential sensor unavailability by optimizing across all sensor combinations in the training, enabling the model to recognize that available sensors perform better than others (e.g.. 4D Radar outperform camera adverse weather). As demonstrated in Tab. SCL enhances the performance of the proposed ASF method when compared to ASF without SCL.
ASF
L+R
1.5
20.5
(Proposed)
C+L+R
1.6
13.5
7

---

Experiments
4.1 Experimental Setup
Dataset and Metries. K-Radar (Pack al. 2022) is large-scale autonomous driving dataset with broad range of conditions including time (day, night), weather (normal, rain, fog, snow, sleet), road types (urban, highway, mountain), and sensors (4D Radar, LiDAR, camera, GPS), Notably, K-Radar is the only dataset with data captured in adverse weather conditions.
For comparison with SOTA methods, we utilize two K-Radar benchmark variants. Benchmark v1.0 (Paek et al., 2022, Chae et al.) 2024, Huang et al. 2025) focuses on the 'Sedan' class within a driving corridor region of [0m, 72m] [-6.4m, 6.4m| 1m2m, 6m] (XxYxZ). For ablation studies and qualitative analysis, we use benchmark v2.0, which covers wider area [0m, 72m] x [-16m, 16m] [-2m, 7.6m] and includes both 'Sedan' and 'Bus Truck' classes. We evaluate using A Ap and A ABBEV at IoU thresholds of 0.3 and 0.5, while also reporting VRAM usage and FPS based on the same hardware setup.
Implementation Details. We implement the ASF single RTX3090 GPU with 24GB VRAM. ASF is trained for 11 epochs using AdamW (Loshchilov and Hutter 2017) optimizer with a learning rate 0.001 and batch size 2. The voxel size for the fused FM is set to 0.4m, con- sistent with (Pack al. 2022
Table 3: Performance comparison of ASF under vari-
ous sensor combinations on K-Radar (Pack al. 2022)
benchmark v2.0. We indicate the employed sensors (C:
Camera, L: LiDAR, 4D Radar) and report AP3p at IoU = 0.3 for 'Sedan' and 'Bus or Truck' classes. 'Nor., 'Ove.' 'Sle." and 'H.s.' refer to 'Normal', 'Overcast', 'Sleet', and 'Heavy snow', respectively. denotes the sensor unavailability, as shown in Fig. The bold and underlined values indicate the best and the second-best respectively. All ten ASF models share the same neural network weights trained for R+L+C. Performance un- der additional weather conditions (Fog, Rain, and Light Snow) and with other evaluation metric AP'BE is pro- vided in Appendix D.
4.2 Comparison
of ASF to SOTA Methods
Following the benchmark v1.0 of the K- Radar (Pack et al. 2022), we compare the proposed ASF with SOTA methods includ- ing 3D-LRF Chae et al. (024) and L4DR RTNH Paek et al.] 2022) for single-sensor per- formance. In addition to the detection per-
Class
Sensors
Total
Nor.
Ove.
Sle.
H.s.
(Huang al. (025), , and
R
47.3
40.7
58,8
45,9
56.5
L
73.0
73.0
86.1
64.9
54.5
C
14.8
14.9
7.71
formance comparison with DCF methods (Chae et al. 2024 Huang al. 2025), evaluate computational efficieccy against DPFT (Fent al. (024) which the only open-sourced SCF method available for K-Radar.
C*
3.71
3.72
3.17
L+R
77.3
77.7
87.3
74.4
65.4
Sedan
C+R
52.7
49.1
62.4
46.0
57.2
C+L
76.4
78.3
86.5
64.2
C+L+R
79.3
78.8
87.6
74.2
65.8
C'+L+R
77.6
78.2
87.7
74.4
65.4
C+L*+H +R
58.9
58.8
66.6
52.3
58.2
Detection Performance. As shown in Tab. D ASF significantly outperforms SOTA methods across various weather conditions. Compared previous SOTA L4DR (Huang et al.) 2025), ASF achieves substantial improvements of 9.7% in APBEV (87.2% vs. 77.5%) and 20.1% in APaD (73.6% vs. 53.5%) at IoU=0.5. These improvements are particularly re-
R
34.2
22.9
40.9
21.1
$1.2
I.
54.9
53.7
74.8
69.1
37.8
C
9,59
9.02
17.2
C
3.65
3.66
0.00
Bus
L+R
59.9
71.6
68.2
68.9
C+R
36.2
24.4
41.4
23.4
56.5
Truck
C+L
53.0
49.1
60.1
72.1
39.6
C+L+R
60.4
52.7
77.4
69.2
68.9
C'+L+R
60.1
52.1
72.0
70.9
69.1
C+L*+R
40.0
38.2
28.9
54.8
markable in challenging conditions like sleet (67.5% vs. 46.2% AP3D) and heavy snow (66.4% vs. 37.0% APsp).
Notably, both ASF configurations (L+R and C+L+R) use identical neural network weights yet maintain comparable performance, demonstrating the system's ability to gracefully handle sensor degradation. Even with only LiDAR and Radar, ASF achieves 87.0% A APEV and 72.9% APap 1oU=0.5, nearly matching the full sensor suite's performance.

---

Computational Efficiency. ASF achieves exceptional computational efficiency for real-time au- tonomous driving upplications. As demonstrated in Tab. 2 ASF with LiDAR and 4D Radar processes at 20.5 Hz, approximately 4x faster than 3D-LRF (5.04 Hz) using identical sensors. Even with all three sensors, ASF maintains 13.5 Hz. exceeding the 10 Hz threshold typically required for autonomous driving systems (Zhao et al. 2024b). This efficiency results from our CASAP, which applies cross-attention across sensors along patches rather than across sensors and patches. Fur- thermore, ASF maintains compact memory footprint (1.5-1.6 GB), comparable to 3D-LRF (1.2 GB) and substantially lower than DPFT (4.0 GB).
4.3 Addressing Sensor Degradation and Failure
A key advantage of ASF is the robust performance under sensor degradation failure. As shown in Tab. aab.3Jand and Fig. normal conditions (Fig.3 ASF effectively utilizes all available sensors with attention weights distributed according to each sensor's reliability. However, ASF's true value emerges in challenging scenarios, In adverse weather (Fig. (-r)), camera and LiDAR measurements are significantly degraded or disappear completely. In these critical situations, ASF automatically redistributes attention toward the more reliable 4D Radar, as evidenced by the predominant blue coloration in the sensor attention maps (SAMs) and corresponding attention percentages. Even with damaged sensors (denoted by in Tab. and Fig. 3), ASF maintains near-optimal performance. For example, with a damaged camera (C*), C*+L+R shows 77.6% AP3D. which is only 1.7% lower than with fully sensors (79.3%). This robustness stems from the unified canonical projection (which creates a common feature space) and the cross-attention mechanism (which estimates sensor reliability).
ASF dynamically adapts to different sensor combinations without retraining. Under
The qualitative results in Fig. demonstrate that in adverse weather, when LiDAR measurements disappear and camera visibility severely degrades, reliable object detection is only possible with active 4D Radar and ASF is fully using 4D Radar. Note that all results shown from the same ASF model with identical weights, illustrating how ASF dynamically adjusts attention to maintain detection performance across varying sensor availabilities.
4.4 Ablation Studies
Tab. preeents ablation studies of key ASF components, analyzing five factors: patch size (P), channel dimension (Cw), patches multiplier (np), number of attention heads (nx), and sensor combination loss (SCL). Our findings reveal that smaller patch sizes (P-2) improve performance through finer feature extraction, while balancing reduced channel dimension (C,=256) with increased patches multiplier (n, maintains or enhances re- sults; furthermore, increasing attention heads
Table 4: Ablation study of ASF. ASF performance for different components and parameters: P (patch size),
C. (channel dimension in unified canonical space), np (number of patches multiplier), n. (number of heads in CASAP) and SCL, using APaD IoU=0.3 for both 'Sedan' and 'Bus or Truck' on the K- benchmark v2.0.
Exp.
P
C.
71p
SCL
Sedan
Bus
(a)
5
512
76.1
45.7
(n4, = 16) benefits 'Bus or Truck' detec-
(b)
5
512
76.4
47.4
(c)
2
512
4
77.2
49.7
tion, and incorporating SCL consistently im-
(d)
2
256
8
8
77.6
57.9
proves performance across configurations by enhancing robustness varying sensor avail-
(e)
2
256
8
v
79.3
58.2
(f)
2
256
77.5
60.2
ability. The optimal configuration combines
(g)
2
256
8
V
79.3
60.4
P=2, 25=256, np np= =16 with SCL.
5 Conclusion
This paper introduces availability- aware sensor fusion (ASF), which addresses sensor availability challenges in autonomous driving by transforming features into unified canonical space through UCP and CASAP. Our approach maintains computational efficiency (O(N, while providing robust fusion for sensor degradation or failure. The proposed sensor combination loss further enhances robustness by optimizing across all possible sensor combinations. Experiments on the K- K-Radar datasct demonstrate significant improvements over SOTA methods (9.7% in APBEY and

---

20.1% in AP3D at loU=0.5), with consistent performance across various weather conditions and sensor combinations.
Limitations. Despite ASF's strong performance, the camera network's 8 capabilities remain a limi- tation. As shown in Fig. (g) and (p). camera-based object detection is less precise, particularly in adverse weather. In Fig. (c), while LiDAR and 4D Radar features are well integrated, camera features remain more separated in feature space, Enhancing the camera backbone could further boost system performance, especially in favorable weather conditions, where visual information is valuable.
Acknowledgment
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021RIA2C30083700
References
Claudine Badue, Rânik Guidolini, Raphael Vivacqua Carneiro, Pedro Azevedo, Vinicius Cardoso, Avelino Forechi, Luan Jesus, Rodrigo Berriel, Thiago M Paixao, Filipe Mutz, al. Self-driving cars: A survey. Expert systems with applications, 165: 13816, 2021.
Zhangjing Wang, Yu Wu, and Qingqing Niu. Multi-sensor fusion in automated driving: A survey, IEEE Access, 8:2847-2868, 2020. doi: 10. L.../ACCCSS.20199999000
Ziqiang Zheng, Yujie Cheng, Zhichao Xin, Zhibin Yu, and Bing Zheng. Robust perception under adverse conditions for autonomous driving based on data augmentation. IEEE Transactions on Intelligent Transportation Systems, 24(12):13916 ((11)))3911--39999 2023a. doi: 10.1109/TITS.2033. 3297318.
Dong-Hee Hee Pack, Seung- Hyun Kong, and Kevin Tirta Wijaya. K-radar: 4d radar object detection for autonomous driving in various weather conditions. Advances Neural Information Processing Systems, 35:3819-385 18939993829, 2022.
Andras Palffy, Ewoud Pool, Srimannarayana Baratam, Julian F. P. Kooij, and Dariu M. Gavrila. Multi-class road user detection with 3+1d radar the view-of-delft dataset. IEEE Robotics and doi: 10. 200p..AA.022231477777.
Automation Letters, 7(2):4961 -4688, 2022.
Seung-Hyun Kong, Dong-Hee Pack, and Sangyeong Lee. Rtnh+: Enhanced 4d radar object detection network using two-level preprocessing and vertical encoding. IEEE Transactions on Intelligent Vehicles, pages 1-14, 2024. doi: 10. 00.1099/TV.222444226666
Shunqiao Sun and Yimin D. Zhang. 4d automotive radar sensing for autonomous vehicles: A sparsity- approach. IEEE Journal of Selected Topics in Signal Processing, 15(4))879-891, 2021. doi: 10. 109//SSTPP.20211377996
Junjie Yan, Yingfei Liu, Jianjian Sun, Fan Jia, Shuailin Li, Tiancai Wang, and Xiangyu Zhang. Cross modal transformer: Towards fast and robust 3d object detection. In Proceedings of the IEEF/CVF International Conference Computer Vision, pages 18268-18278 2023.
Tingting Liang, Hongwei Xie, Kaicheng Yu, Zhongyu Xia, Zhiwei Lin, Yongtao Wang, Tao Tang, Bing Wang, and Zhi Tang. Bevfusion: simple and robust lidar-camera fusion framework. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS 22, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN 97817 13871088.
Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han. Bevfusion: Multi-task multi-sensor fusion with unified bird's-eye s-eye view representation. In 2023 IEEE international conference on robotics and automation (ICRA), pages 2774-2781. IEEE, 2023a.
Lianqing Zheng, Sen Li, Bin Tan, Long Yang, Sihan Chen, Libo Huang, Jie Bai, Xichan Zhu, and Zhixiong Ma. Refusion: Fusing 4-d radar and camera with bird's-eye view features for 3-d object detection. IEEE Transactions on Instrumentation and Measurement, 72:1-14, 2023b. doi: 10.1009/IM.20223333005
10

---

Yujeong Chae, Hyeonseong Kim, and Kuk-Jin Yoon. Towards robust 3d object detection with lidar and 4d radar fusion in various weather conditions. In 2024 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 15162-15172, 2024. doi: 10. 10.1/////73 2024.01436.
Yun Zhao, Zhan Gong, Peiru Zheng, Hong Zhu, and Shaohua Wu. Simplebev: Improved lidar-camera fusion architecture for 3d object detection. arXiv preprint arXiv:2411.05222 2024a.
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pages 1621-11631, 2020.
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11):1211-1237, 2013.
Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection. In European Conference on Computer Vision, pages 53 531-548. Springer, 2022.
Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon. Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In Conference on Robot Learning, pages 180-191. PMLR, 2022.
Xu Zheng, Yuanhuiyi Lyu, and Lin Wang. Learning modality-agnostic representation for semantic segmentation from any modalities. In European Conference on Computer Vision, pages 146-165. Springer, 2024.
Zhenghao Liu, Chenyan Xiong, Yuanhuiyi Lv, Zhiyuan Liu, and Ge Yu. Universal vision- lasiun-lagguage dense retrieval: Learning a unified representation space for multi-modal retrieval. In International Conference on Learning Representations (ICLR), 2023b.
Liwei Wang, Yin Li, and Svetlana Lazebnik. Learning deep structure- dings. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5005-5013, 2016.
structure-preserving preserving image-text embed-
Felix Fent, Andras Palffy, and Holger Caesar. DPFT: Dual perspective fusion transformer for camera-radar-based object detection. IEEE Transactions on Intelligent Vehicles (T-IV), 2024.
Xuyang Bai, Zeyu Hu, Xinge Zhu, Qingqiu Huang, Yilun Chen, Hongbo Fu, and Chiew-Lan Tai. Transfusion: Robust lidar-camera fusion for 3d object detection with transformers. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 1090-1099, 2022.
Amnon Shashua. True Redundancy, 2024. URL https::////. http:///wwwoooilleee.ee////tttae
cruudnedundancy//
De Jong Yeong, Krishna Panduru, and Joseph Walsh. Exploring the unseen: A survey of multi-sensor fusion and the role of explainable ai (xai) in autonomous vehicles. Sensors, 25(3):856, 2025. doi: 10.3390/s25030856.
Sourabh Vora, Alex H Lang, Bassam Helou, and Oscar Beijbom. Pointpainting: Sequential fusion for 3d object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4604-4612, 2020.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213-229. Springer, 2020.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 0-2579-2605 2008
Xun Huang, Ziyu Xu, Hai Wu, Jinlong Wang, Qiming Xia, Yan Xia, Jonathan Li, Kyle Gao, Chenglu Wen, and Cheng Wang. LAdr: Lidar- fusion for weather-robust 3d object detection. In Proceedings of the 39th AAAI Conference Artificial Intelligence (AAAI), 2025.
11

---

Weiyi Xiong, Jianan Liu, Tao Huang, Qing-Lome Han, Yuxuan Xia, and Bing Zhu. Lxl: Lidar excluded lean 3d object detection with 4d imaging radar and camera fusion. IEEE Transactions on Intelligent Vehicles, 2023.
Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In Computer Mision-ECCy 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part XIV 16, pages 194-2144211 210. Springer, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Zhuofan Xia, Xuran Pan, Shiji Song, Li Erran Li, and Gao Huang. Vision transformer with deformable attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4794 4803, 2022.
Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang. Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detection. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 1477-1485, 2023.
Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. In Sensors, volume 18, page 3337. MDPI, 2018.
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part 14, pages 21-37. Springer, 2016.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. 2016.
arXiv preprint
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus).
arXiv preprint
1oXx:16606.0815.. 2016.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 1rXiv:1711.05001, 2017.
Jingyuan Zhao, Wenyi Zhao, Bo Deng, Zhenghong Wang, Feng Zhang, Wenxiang Zheng, Wanke Cao, Jinrui Nan, Yubo Lian, and Andrew F. Burke. Autonomous driving system: comprehensive survey. Expert Systems with Applications, //bb..rg/10.0011//..w..22.20 2023.122366
2024b. ISSN 0957-4174.
doi: https:
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the paper's key contributions (ASF with UCP and CASAP, sensor combination loss).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper explicitly addresses limitations in conclusion, noting that camera network capabilities remain limitation, particularly in adverse weather conditions, and acknowledging that camera features are less well-integrated than LiDAR and 4D Radar features in the unified feature space.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
13

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The paper provides complete mathematical formulations with clearly stated assumptions for all components of the ASF (UCP, CASAP). Specifically, the computational
complexity analysis (O(N, vs. O ((,,),,NN,))) is mathematically justified in subsection 3.2] with all relevant variables defined and assumptions explicitly stated.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
The paper provides comprehensive details for reproducing the experimental re- sults, including dataset specifications (K-Radar benchmarks v1.0 and v2.0), implementation details (GPU, epochs, optimizer, learning rate, batch size, voxel size), network architecture specifies, and evaluation metrics. Additionally, the authors explicitly state that the code will be made publicly available after the review process.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
14

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The K-Radar dataset referenced in the paper is a widely used dataset and benchmark in autonomous driving research, making accessible to researchers the field. Additionally, the authors explicitly state that the code will be made publicly available following the completion of the review process, ensuring full reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/ wtipss.cc/pbblic///.o
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
The paper provides comprehensive experimental details subsection specifying dataset variants (benchmark v1.0 and v2.0), evaluation metries, implementation hardware (RTX3090 GPU), training parameters (11 epochs, Adam optimizer, learning rate 0.001, batch size 2), and architectural specifics.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
15

---

Answer: [Yes]
Justification: The paper provides suitable information about statistical significance through comprehensive performance breakdowns across different weather conditions and sensor combinations in Tab. and demonstrating the consistency of results. Additionally, the authors have included experiments in the Appendix that specifically analyze the impact of random seed variations, confirming the statistical reliability of the reported performance improvements.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative етror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper specifies the computing resources used, including hardware (single RTX3090 GPU with 24GB VRAM) and provides computational efficiency metrics (VRAM usage and FPS) in Tab. 2 Training details including epochs, batch size, and optimizer settings are clearly stated, giving sufficient information estimate resource requirements for reproduction.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of / http://wwwpp....pppaa/l///hccaaa
Answer: [Yes]
Justification: The research adheres the NeurIPS Code of Ethics by focusing on technical advancements in sensor fusion without involving human subjects sensitive data.
16

---

Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper discusses broader impacts in introduction, highlighting the positive societal implications of improved autonomous driving safety in adverse weather conditions. It addresses how the proposed ASF enhances reliability when sensors are degraded or compromised, which directly impacts safety during challenging environmental conditions.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The paper does not pose high risks for misuse as it focuses on a technical approach for sensor fusion in autonomous driving systems. It does not release pretrained language models, image generators, or scraped datasets that would require special safeguards against potential misuse.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
17

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper properly cites the K-Radar dataset when referencing it for experi- ments. While the license (CC BY-NC-ND) is not explicitly mentioned in the paper itself, this information is publicly available on the official K-Radar release page, and the authors' usage complies with the terms of this license.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pwwpssspppppseaeeeeeeepeepp thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The paper introduces a new ASF method with comprehensive documentation of the architecture, implementation details, and experimental results. The authors state that complete code will be made publicly available after the review process with appropriate documentation ensure reproducibility of all experiments and findings.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
18

---

The paper focuses sensor fusion technology for vehicle detection without conducting research with human subjects. While the K-Radar dataset contains pedestrians with faces blurred for privacy protection (as noted in the dataset documentation), this privacy measure was implemented by the dataset creators rather than being part of the research methodology presented in this paper.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
This paper does not involve any research with human subjects. The study focuses exclusively on technical methods for sensor fusion autonomous driving systems using the K-Radar dataset, where any human data (such pedestrian images) was previously collected and anonymized through face blurring by the dataset creators.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs were only used for editing purposes such as grammar checking, spelling corrections, and word choice refinements in the manuscript. This editorial assistance does not impact the core methodology, scientific rigor, or originality of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy /https://neurips.. http///arrrpppecooooeeee///ee/// for what should or should not be described.
19


Reviews:
[{"review_id": "wF1ZBXUDd8", "reviewer": "Reviewer_DZ4x", "summary": "This paper proposes a sensor fusion framework (Availability-aware Sensor Fusion) to address the degradation problem in autonomous driving sensors. ASF introduces two components: Unified Canonical Projection (UCP), which projects features from heterogeneous sensors into a unified representation space, and CASAP (Cross-Attention across Sensors Along Patches), which dynamically fuses sensor features based on availability. A Sensor Combination Loss (SCL) is adopted to improve robustness. Experiments on the K-Radar benchmark demonstrate significant improvements in both performance and efficiency over prior works.", "questions": "1.How does CASAP actually infer sensor reliability? Is there any theoretical justification or formal analysis behind its attention mechanism, or is it purely learned empirically?\n2.Can the method generalize to datasets without 4D Radar (e.g., nuScenes, Waymo)? If not, how tightly is the approach bound to K-Radar’s sensor setup?\n3.The influence of camera features remains unclear. Given that the limitation section identifies camera as a weak modality, how much does it actually contribute to detection performance in normal vs. adverse conditions?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1.ASF achieves state-of-the-art results under multiple weather and sensor failure conditions.\n2.Lower memory usage and higher inference FPS than existing methods. CASAP simplifies the standard SCF computation by reducing the cross-attention complexity.\n\nWeaknesses \n1.The camera-based object detection (particularly in adverse weather) remain unaddressed. Yet the performance on adverse weather is highlight in the paper.\n2.The experimental section primarily consists of straightforward performance tables under different sensor/weather configurations, but lacks in-depth diagnostic or interpretative analysis.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you sincerely for your thoughtful review. We address your concerns below:\n\n**Q1: How does CASAP actually infer sensor reliability?**\n\nThank you for this excellent question. CASAP empirically learns sensor reliability through cross-attention mechanisms trained on multi-sensor data. Features projected into the unified canonical space via UCP are optimized during training such that the reference query $Q_{ref}$​ (Eq. 5) achieves high correlation with features from reliable sensors. During inference, when a sensor is degraded or missing, its features exhibit low correlation with $Q_{ref}$​ and consequently receive lower attention weights. This is validated in Fig. 3, where 4D Radar assumes a dominant role when camera/LiDAR fail under adverse weather conditions. To facilitate understanding of this mechanism, we will release dynamic sensor attention map videos (inference_result_2.gif in the appendix zip file) on the project page.\n\n**Q2: Can the method generalize to datasets without 4D Radar?**\n\nThank you for raising this important point. ASF's architecture is inherently sensor-agnostic by design. We selected K-Radar as it uniquely provides multi-modal data under extreme adverse weather conditions where cameras and LiDAR experience complete failure due to heavy snow occlusion. Following your valuable suggestion, we are currently conducting experiments with ASF on nuScenes. Given the differences in sensor configurations and training settings between nuScenes and K-Radar, as well as ASF's sensitivity to hyperparameters (as shown in Table 4), the optimization process requires careful hyperparameter tuning across multiple experimental configurations. Upon completion, we will release the training code and model weights for both benchmarks on the project page.\n\n**Q3: Camera contribution analysis?**\n\nThank you for this insightful observation. While cameras are inherently vulnerable under adverse weather conditions due to their reliance on visible light, they nonetheless provide crucial contributions for long-range object detection. This is clearly demonstrated in inference_result_1.gif (in the appendix zip file), where red regions (indicating camera attention) dominate in distant object areas (upper-right portion of the video). This phenomenon aligns with findings from Wu et al.'s Virtual Sparse Convolution (CVPR 2023), which achieved 1st place on the KITTI 3D object detection benchmark, where their Figure 2 illustrates that virtual points (derived from camera features) are instrumental in achieving substantial AP improvements for distant objects. We are currently conducting quantitative analyses of distance-based sensor contributions, with results to be included in the revised Appendix in the camera-ready version.\n\n**W1: Camera performance in adverse weather**\n\nYour observation is indeed accurate. The K-Radar dataset comprises over 60% adverse weather frames, where heavy snow and sleet conditions result in complete occlusion of the sensor surfaces, leading to total loss of camera measurements (as demonstrated in inference_result_2.gif). These extreme conditions yield near-zero AP for camera-only detection, hence our notation of \"-\" (not available) in Table 3. This reflects the fundamental physical limitations of optical sensors rather than any methodological deficiency. ASF's primary contribution lies in gracefully handling such sensor failures through dynamic attention redistribution, maintaining robust performance despite individual sensor degradation. Comprehensive camera-only performance logs across all weather conditions will be made available on the project page, following the format provided in the supplementary materials (logs_10_seeds).\n\n**W2: Diagnostic analysis**\n\nWe appreciate this constructive feedback. In addition to the t-SNE visualizations (Figs. 1, 5) and attention maps (Fig. 3) currently presented, we are conducting comprehensive diagnostic analyses to provide deeper interpretative insights: (1) quantitative analysis of sensor contributions across different object distances, revealing how each modality's importance varies with range, and (2) statistical analysis of attention weight distributions under various weather conditions, quantifying the dynamic adaptation of sensor reliability. These additional analyses will be incorporated into the revised Appendix in the camera-ready version, offering a more thorough understanding of ASF's adaptive behavior under diverse operational conditions.\n\nThank you once again for your thorough review. We believe these addressed concerns and planned enhancements will substantially improve the paper's impact and reproducibility."}, {"review_id": "S1g19pXqGE", "reviewer": "Reviewer_mE3d", "summary": "The manuscript proposes a novel approach to sensor fusion designed to enhance robustness and reliability in autonomous vehicle perception by integrating data from multiple sensors (cameras, LiDAR, radar) based on their availability. \nTo this end, authors propose a cross-attention method and a unified canonical projection.\nThe unified canonical space allows a unified and consistent of features from different sensors, while the cross-attention across sensors along patches dynamically adjusts the important of features from the sensors based on their current availability and reliability. The objective is to prevent adverse effects on the fused data quality when one sensor is unavailable or sends low quality data. A sensor combination loss is used to optimize the detection performance across sensors. \nThe method is evaluated through experiments on the K-Radar dataset. Results show improvements on object detection performance metrics compared to state-of-the-art methods.", "questions": "How do you plan to address the limitations related to the camera features integration? Particularly in difficult weather conditions. \n\nThe method is evaluated on K-Radar. This is enough, but it would be interesting to see evaluations on other datasets or on real-world data. \n\nI would recommend to improve a bit the figures, which are very informative but slightly complex. Figure 3 would especially benefit from a simpler organization and more space for better readability.", "limitations": "yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "## Strengths \nThe proposed method is technically sound, novel, and very useful to the field, as it tackles a real issue in perception. Sensor fusion drastically improves performances and robustness, but can have negative impacts when some sensors become unavailable or degraded. This works provides a solution to this issue by dynamically changing the impact of the contribution of features from each sensor based on cross-attention. This is a clever use of cross-attention that solves a real-world problem. The contribution of the Unified Canonical Space for more consistent feature representation across sensors is valuable, and its usage with the proposed cross-attention method seems to succeed in reducing issues linked to sensor availability. \n The paper is well written, the equations describing the cross-attention method are clear, and the method is well evaluated on the K-Radar dataset. Visualizations of t-SNE plots and attention maps are an interesting addition. The results of experiments are also quite impressive, proposing significant improvement over the state of the art on a key task: object detection. The code is also already available on GitHub.\n\n## Weaknesses\nFigure 3, while very useful and containing interesting visualizations, could be made clearer for the reader. It would also be interesting to evaluate the method on different datasets, although I understand adequate datasets are not that numerous.", "was_revised": true, "final_justification": "This is a strong contribution, and I already had a favorable view of it. Authors made efforts to answer my minor remarks and provided clear answers. Other reviewers seem to have similar opinions. Therefore, I will not update my rating and leave it at 6.", "rebuttal": "We appreciate your thoughtful and positive review. We address your suggestions below:\n\n**Q1: How do you plan to address the limitations related to camera features integration? Particularly in difficult weather conditions.**\n\nThank you for this excellent question. Camera vulnerability in adverse weather stems from its fundamental reliance on visible light. Our strategy is to address this inherent limitation in two ways:\n\nFirst, ASF automatically assigns higher weights to other sensors (particularly 4D Radar) when camera performance degrades. As demonstrated in inference_result_2.gif (in the appendix zip file), when cameras fail due to heavy snow, ASF increases reliance on 4D Radar (blue) to maintain robust performance.\n\nSecond, we are exploring stereo-based approaches to enhance camera utilization. We have implemented Chen et al.'s DSGN++ (IEEE T-PAMI 2022) on K-Radar, achieving improved performance (26.5% and 26.3% AP3D for Sedan and Bus or Truck, respectively) compared to the monocular camera-based approach. We plan to integrate this as ASF's camera-specific encoder. However, it's worth noting that K-Radar's ZED 2i camera has a 12cm baseline compared to KITTI's 54cm, limiting improvement potential.\n\n**Q2/W2: The method is evaluated on K-Radar. This is enough, but it would be interesting to see evaluations on other datasets or on real-world data.**\n\nThank you for this important suggestion. While K-Radar is unique in providing extreme adverse weather conditions (complete camera/LiDAR failure), we agree that demonstrating generalizability is valuable. We are currently conducting ASF experiments on the nuScenes dataset. Upon completing hyperparameter optimization, we will release results and code on the project page.\n\n**Q3/W1: I would recommend to improve a bit the figures. Figure 3 would especially benefit from a simpler organization.**\n\nThank you for this constructive feedback. To address your concern about Figure 3's clarity, we will implement several improvements: (1) overlay GT bounding boxes on prediction results for easier comparison, (2) add clear separation lines between weather conditions, and (3) enhance captions to highlight key findings.\n\nAdditionally, recognizing that the current Figure 3 may be too dense with information, we will include 2 additional scenes in the Appendix using a 4×2 format. This larger format allows each subfigure to be displayed at a higher resolution, making details more visible and addressing your concern about clarity. We will also provide interactive visualizations (inference_results.gif in the appendix zip file) on the project page, enabling readers to dynamically observe sensor attention changes with better visual clarity.\n\nThank you again for your positive evaluation and constructive suggestions. We will incorporate these improvements to enhance our contribution."}, {"review_id": "le2S0d4Wjw", "reviewer": "Reviewer_HNwy", "summary": "This paper proposes a pipeline for 3D object detection by exploiting the fusion of multimodal sensor data.\nThe pipeline works with 3 steps:\nfirst, specific pre-trained feature extractors are used for each type of sensor (camera, LiDAR, radar).\nThis provides features from all sensors in the same BEV format.\nThen, the fusion is done in two stages, one to unify the features from different source in a common canonical space, then use cross-attention model that is trained on available data, making it give more attention to available sensors; This improves the performance by filtering noisy data from bad weather or broken sensors. The patch based optimization and the single common loss for all modalities allow faster computation.\nThis provides features that can be used for 3D object detection.\nThe final results show SOTA performance across different weathers and sensor failures against multiple baselines.", "questions": "How would stereo cameras impact the performance, as it would provide geometric information from the images?\nWould any other representation that BEV be possible and what impact would it have on performance?\nHow much of the performance variation can be attributed to the feature extractor of each modality?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Proposes a method to leverage multimodal sensors for 3D detection that is able to automatically filter noisy data through attention weight.\n- SOTA results in difficult situations (bad weather, broken sensors)\n- Paper is clear and well-written with in-depth experiences and analysis\n\nWeaknesses:\n- Camera information still provides less reliable performance as it does not provide geometric information like the LiDAR or radar.", "was_revised": true, "final_justification": "My questions were answered, no other concern.", "rebuttal": "We appreciate your thoughtful review. We address your questions and concerns below:\n\n**Q1/W1: How would stereo cameras impact the performance, as it would provide geometric information from the images?**\n\nThank you for this excellent question. Stereo cameras can indeed enhance performance by providing depth information. We have implemented Chen et al.'s DSGN++ (IEEE T-PAMI 2022) on K-Radar and conducted experiments. We achieved improved performance compared to monocular cameras: 26.5% AP3D for Sedan class and 26.3% AP3D for Bus or Truck class. This demonstrates the value of geometric information provided by stereo vision. These results validate that stereo vision substantially addresses the camera's geometric limitations you mentioned.\n\nHowever, K-Radar's ZED 2i camera has a short 12cm baseline compared to KITTI's 54cm, limiting depth estimation accuracy for distant objects. Nevertheless, we plan to integrate this stereo-based approach as ASF's camera-specific encoder. All experimental code and weights will be released on the project page upon completion.\n\n**Q2: Would any other representation than BEV be possible and what impact would it have on performance?**\n\nThank you for raising this important point. Several representations beyond BEV are indeed possible. (1) Voxel representation can directly represent 3D space preserving height information, but significantly increases memory and computational costs. (2) Polar coordinate representation suits 4D Radar but is less intuitive for downstream tasks (e.g., planning, segmentation). (3) Perspective view fusion benefits camera viewpoint but struggles with occlusion handling.\n\nWe chose BEV because it: (1) enables unified 2D representation for all sensor features, (2) directly supports autonomous driving path planning, and (3) provides balance between computational efficiency and performance.\n\n**Q3: How much of the performance variation can be attributed to the feature extractor of each modality?**\n\nThis is a very insightful question. While various feature extractors (termed sensor-specific encoders in our paper) have been developed for each modality, recent sensor fusion research typically fixes well-established extractors to isolate fusion module contributions, e.g., Lang et al.'s PointPillars (CVPR 2019) and Yan et al.'s SECOND (Sensors 2018) for LiDAR-specific encoders. As mentioned in Appendix A.1, we intentionally froze pre-trained encoders to clearly isolate ASF's contribution.\n\nAs you correctly note, better sensor-specific encoders would improve overall performance. As mentioned in Q1, we are experimenting with replacing the current monocular BEVDepth with the higher-performing stereo-based DSGN++ for ASF integration. Completed experimental code and weights will be uploaded to the project page to demonstrate how improved sensor-specific encoders impact performance.\n\nThank you again for your constructive feedback. We will carefully consider all points raised to improve our research, particularly regarding camera geometric information utilization."}, {"review_id": "TLOQBeKXKi", "reviewer": "Reviewer_Q33x", "summary": "This paper presents a method for a new angle on sensor fusion between photometric (e.g., image) and geometric (e.g., ToF including lidar/radar) modalities -- \"availability-aware\" sensor fusion, which they call ASF. \n\nThe method consists of two \"sub-modules\" which are (1) a unified canonical projection into a common space for all modalities, and the authors claim that this eliminates inconsistencies between modalities. Then, (2) the method addresses the problem of a missing sensor/modality by learning to estimate missing, or uncertain, modalities.\n\nQuantitative results on the K-Radar dataset are shown, and the proposed method is seen to outperform baseline methods of multi-sensor fusion.", "questions": "(Q1) Can the authors present results on the nuScenes dataset, which is canonical for multi-modal multi-sensor 3D object detection? This would greatly help to motivate the method, see W1 for more details. Alternatively, if the authors would like to provide a rebuttal for why the nuScenes dataset would be inappropriate for this setting, I would be open to this argument, but in this case I would still like to see at least one more relevant experiment against existing approaches.\n\n(Q2) Can the authors expand upon the claim that this method details with uncertainty in certain sensors? Have the authors done the necessary literature search on methods that quantify uncertainty in data, i.e., aleatoric uncertainty?\n\n(Q3) Again, these are minor points, but can the authors provide a plan to improve clarity and convincingness in the aforementioned areas of the paper (figure, readability)?\n\nI am inclined to keep my score of Accept given that my few concerns are addressed.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 4, "strengths_and_weaknesses": "(S1) First of all, the idea is well-motivated and the problem setting of dealing missing or damaged sensors is very interesting and relevant to real-world systems such as autonomous driving vehicles.\n\n(S2) Most parts of the method are described thoroughly with sufficient mathematical rigor, and importantly, are well-motivated. In other words, the authors provide a written justification of most of the design choices they made, e.g., removal of positional embedding and post-feature normalization, among others.\n\n(S3) The ablation and sensitivity choices are thorough and present quantitative backing for the design choices mentioned in S2.\n\n(W1) The authors claim that they only evaluate performance on the K-Radar dataset because it \"is the only dataset with data captured in adverse weather conditions.\" While I will not go so far as to call this assertion incorrect, it is important to note that \"adverse weather conditions\" is far from an objective term -- in my opinion, many datasets including the nuScenes dataset contain at least some amount of data captured in some degree of adverse conditions.\nImportantly, most autonomous driving works studying multi-modal (particularly, image-lidar) 3D object detection works are benchmarked on the canonical nuScenes dataset, such as UniTR (Wang et al., ICCV 2023) and SparseFusion (Xie et al., ICCV 2023). Can the authors present the results of their proposed sensor fusion for detection framework on this dataset, as has become the standard in this line of work? Besides nuScenes being the standard, it would be extremely valuable to see that the proposed method works well (enough) on more than one dataset. Currently, this is the main reason for the \"fair\" rating for Significance.\n\n(W2) In the introduction, the authors seem to equate a missing sensor and an uncertain sensor. While these concepts may seem tied together at face value, the concept of uncertainty is not brought up again in the rest of the paper. The authors should clarify whether their method actually addresses the problem of uncertainty, which in itself is studied in a large body of existing works.\n\n(W3) Minor weakness: Figure 3 would be more convincing if ground truth and prediction were overlaid, as it is currently hard to draw conclusions between many combinations who seem, at least to my eye, to show very similar performance. A form of error map or plot would be equally helpful. Relying on the captions containing quantitative results to tell the story means that it would have been more efficient to simply present another small table for the two examples shown.\n\n(W4) Very minor weakness: A few too many abbreviations that hinder the flow and readability of the paper. The authors could consider undoing some of the more unnecessary ones like autonomous driving (AD) and feature map (FM).", "was_revised": true, "final_justification": "While I maintain that the submission would have been more convincing with evaluation on nuScenes (or at least more than just one dataset), I believe the paper--as currently is--still warrants an acceptance.", "rebuttal": "We appreciate your thorough review and constructive feedback. We address your concerns below:\n\n**Q1/W1: Results on the nuScenes dataset**\n\nThe primary reason for selecting K-Radar is that this dataset contains extreme weather conditions such as heavy snow and sleet, where sensor surfaces become completely occluded (missing/failure), resulting in total loss of camera and LiDAR measurements. These extreme situations are essential for validating ASF's core contribution of maintaining robust performance through dynamic attention redistribution when individual sensors completely fail. This unique characteristic cannot be found in nuScenes or other datasets.\n\nThank you for this important suggestion. While K-Radar is unique in providing extreme adverse weather conditions (complete camera/LiDAR failure), we agree that demonstrating generalizability is valuable. We appreciate your suggestion and are currently experimenting with ASF on the nuScenes dataset. Since the nuScenes dataset differs significantly from K-Radar in sensor configuration and ASF requires careful hyperparameter tuning (as shown in Table 4), the adaptation process is ongoing. We will publicly release all experimental code and trained models on the project page once the optimization is complete.\n\n**Q2/W2: Clarification on uncertainty handling**\n\nThank you for pointing out this important distinction. We apologize for the confusion. To clarify, ASF addresses sensor availability rather than uncertainty quantification (e.g., aleatoric uncertainty). We will revise the words \"uncertain\" and \"certain\" used in line 64 of the introduction and line 217 of subsection 3.3 to \"degraded\" and \"available\". Thank you for clearly pointing out this terminology confusion.\n\n**Q3/W3: Figure 3 improvements**\n\nWe appreciate your constructive feedback. To improve Figure 3's clarity as you suggested, we plan three enhancements: (1) overlaying ground truth bounding boxes onto prediction results to facilitate direct comparison, (2) inserting clear visual separators between different weather conditions, and (3) revising captions to better emphasize key findings.\n\nFurthermore, acknowledging that Figure 3 currently contains dense information, we will add a minimum of 2 supplementary scenes in the Appendix formatted as 4×2 layouts. This expanded format enables higher-resolution display of individual subfigures, improving detail visibility and addressing your clarity concerns. We will also make interactive visualizations (inference_results.gif from the appendix zip file) available on the project page, allowing readers to observe sensor attention dynamics more clearly.\n\n**Q3/W4: Abbreviation reduction**\n\nThank you for your suggestion on readability. We will eliminate unnecessary abbreviations such as \"autonomous driving (AD)\" and \"feature map (FM)\", keeping only essential technical abbreviations like ASF, UCP, CASAP, and BEV. This will improve the flow and readability of the paper.\n\nThank you again for your constructive feedback that helps improve our work's clarity and impact. We will carefully incorporate all suggested revisions to make our paper better."}]

Author rebuttal:

Meta-review: The paper proposes a multimodal data (including cameras, LiDAR, radar signals) fusion method for autonomous vehicle perception. All four reviewers gave positive scores: two borderline accepts, one accept, and one strong accept. The major concerns raised by the reviewers include insufficient details and additional experimental results on the nuScenes benchmark. The authors addressed all the concerns during the discussion period. Based on all of these, the decision is to recommend the paper for acceptance.
Decision: Accept (poster)