Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
Authors: Xue Zhao
Qinying Gu
Xinbing Wang
Chenghu Zhou
Nanyang Ye
Venue: ICML 2025 poster (2025)
Keywords: Domain generalization
multi-camera 3D object detection
augmentation
Primary area: applications->computer_vision

Abstract: Improving the generalization of multi-camera 3D object detection is essential for safe autonomous driving in the real world. In this paper, we consider a realistic yet more challenging scenario, which aims to improve the generalization when only single source data available for training, as gathering diverse domains of data and collecting annotations is time-consuming and labor-intensive. To this end, we propose the Fourier Cross-View Learning (FCVL) framework including Fourier Hierarchical Augmentation (FHiAug), an augmentation strategy in the frequency domain to boost domain diversity, and Fourier Cross-View Semantic Consistency Loss to facilitate the model to learn more domain-invariant features from adjacent perspectives. Furthermore, we provide theoretical guarantees via augmentation graph theory. To the best of our knowledge, this is the first study to explore generalizable multi-camera 3D object detection with a single source. Extensive experiments on various testing domains have demonstrated that our approach achieves the best performance across various domain generalization methods.

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross- View Learning
Xue Zhao Qinying Gu
Xinbing Wang'
Chenghu Zhou'
Nanyang Ye
Abstract
and conditions vary widely. This performance drop occurs because camera data in practical applications often has dif- ferent distributions compared to the limited training data. As a result, enhancing the generalization of these models is critical for their safe deployment. Domain generalization (DG) aims to generalize model to an unseen target domain by learning from multiple source domains. However, gath- ering diverse sources of data for training is time-consuming and labor-intensive, especially in autonomous driving sce- narios, and cannot always guarantee improved performance. In this paper, we tackle a more practical yet challenging problem: boosting the generalization of 3D object detectors trained on single source domain. Focusing on single- domain generalization (SDG) not only addresses practical constraints but also provides more rigorous assessment of model adaptability.
Improving the generalization of multi-camert 3D object detection is essential for safe autonomous driving in the real world. In this paper, we con- sider a realistic yet more challenging scenario, which aims to improve the generalization when only single source data available for training, gathering diverse domains of data and col- lecting annotations is time-consuming and labor- intensive. To this end, we propose the Fourier Cross-Viee Learning (FCVL) framework includ- ing Fourier Hierarchical Augmentation (FHiAAu),, an augmentation strategy in the frequency domain to boost domain diversity, and Fourier Cross- View Semantic Consistency Loss to facilitate the model to learn more domain-invariant features from ad- jacent perspectives. Furthermore, we provide the- oretical guarantees via augmentation graph the- ory. To the best of our knowledge, this the first study to explore generalizable multi-camera 3D object detection with single source. Exten- sive experiments various testing domains have demonstrated that our approach achieves the best performance across various domain generaliza- tion methods.
In SDG for 2D image classification, previous works (Zhao et al., 2024; Qiao et al., 2020) focus on enhancing data di- versity through common 2D data augmentation techniques, such as geometric transformations, style transfer, or adver- sarial data generation. However, directly applying these approaches to BEV-based tasks introduces several chal- lenges. First, BEV representations are generated by pro- jecting multi-view 2D features using real-world physical constraints, which limits the use of strong geometric trans- formations, For example, large rotation of the input images will disrupt the spatial relationships, thereby affecting the spatial consistency of the BEV features. Second, style trans- fer techniques (Zhao et al., 2024) replace the original image statistics with those from the target style, but this often blurs the boundary between style and content (Lee et al., 2023), distorting important features and ultimately harming model accuracy and generalization. Third, adversarial generation methods (Goodfellow et al., 2020) suffer from unstable train- ing and mode collapse. While diffusion-based techniques (Ho et al., 2020) are more stable, they add significant com- putational and storage overhead, making them impractical for complex 3D detection models. Therefore, common 2D data augmentations cannot be effectively leveraged to create diverse training samples for BEV-based tasks. More im- portantly, for multi-camera 3D object detection, the natural availability of cross-view data offers a unique opportunity to learn Domain-invarian features, a potential that remains
1. Introduction
Multi-camera& 3D detection based on Bird's Eye View (BEV) representations has achieved rapid development, as BEV captures both spatial locations and semantic features with- out being heavily affected by occlusions. While existing models (Philion& Fidler, 2020; Huang et al., 2022: Li et al., 2023; 2022b) have achieved excellent performance on in- distribution datasets like nuScenes (Caesar et al., 2020), they struggle in real-world settings where the environment
Shanghai Jiao Tong University. Shanghai, China, 'Shanghai Artificial Intelligence Laboratory, Shanghai, China. Correspon- dence to: Nanyang Ye <ynylincoln @ @it...du..nn......
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross- Learning
CamCrash
Domeinl
Domain22
Domaina?
CrossVieew
Framel.ost
Single Sturee Demeeit
Tarper Domsins
en
Flg
Motiondiaa
LowLight
Data prints
Leon dimm contrention with rustt domatns
Bright
Eigo
(a)
(b)
(c)
Figure/ 1. (a) Detection results of different models: the proposed FCVL can improve the generalization of 3D detection on multiple target domains with single source training data. (b) Cross View Learning: make the most of the natural cross-view input improve the generalization. (c) Augmentation graph connectivity: augmentations of data from the same classes are assumed to be connected. FHiAug increases the augmentation graph connectivity between source and unseen domains.
underexplored in scenarios with limited training data.
feature levels, enhance domain diversity without requiring additional modules or specialized training strategies.
In response these limitations and challenges, we propose the Fourier Cross-Viee Learning (FCVL) framework includ- ing Fourier Hierarchical Augmentation (FHiAug), aug- mentation strategy in the frequency domain to boost domain diversity and Fourier Cross-Vies Semantic Consistency Loss to facilitate the model learn more domain-invariant features. Different from work (Zhao et al., 2024) expanding style statistics in the pixel domain, we utilize the Fourier transform introduce style variations while minimizing content distortion. This is motivated by the well-known property of the Fourier transformation: the phase compo- nent encodes high-level semantic information, while the amplitude component captures low-level image statistics (Xu et al., 2021). This separation allows us to independently manipulate style (low-level statisties) and content (high- level semantics) in the frequency domain. At the image level, we introduce Frequency Jitter, which perturbs both amplitude and phase components to create diverse samples that complement the single source domain. At the feature level, we propose Amplitude Transfer, a novel method for generating fine-gre -raiied style variations, ensuring domain di- versity in the latent space. To leverage the natural cross-vits -iew input, we design the Fourier Cross-Viee Semantic Consis- tency Loss to help the model acquire more robust features from adjacent perspectives. Furthermore, by leveraging aug- mentation graph theory (HaoChen et al., 2021; Wang et al., 2024), we provide a unique theoretical perspective on FCVL and establish its theoretical soundness.
We propose Fourier Cross- View Semantic Consistency Loss to facilitate the model learn more domain- invariant features from adjacent perspectives.
Leveraging augmentation graph theory, we provide a valid theoretical foundation for the effectiveness of FCVL (See Appendix A).
To the best of our knowledge, this is the first work to ad- dress generalizable multi-camera 3D object detection using single-source data. Extensive experiments across various test domains demonstrate that our approach achieves superior performance compared to existing domain generalization methods (See Fig. (a)).
2. Related Work
2.1. Multi-view 3D Object Detection
The recent advances in bird's eye view (BEV) representation exhibit great potential for MC3D-Det. The camera-based BEV models typically project 2D image features from six cameras explicit BEV feature maps in the 3D space and make predictions based on BEV features. In terms of the BEV projection, Lift-splat-shood (LSS) (Philion & Fidler, 2020), BEVDet (Huang et al., 2022) and BEVDepth (Li et al., 2023) distribute the 2D features into 3D space ac- cording the depth information while BEVFormer (Li et al., 2022b) adopts cross attention to query BEV features from 2D images. methods have achieved excellent performance on the in- distribution datasets, like nuScenes (Caesar et al., 2020). sift-sslat-shoot (LSS) proposes a view transform method that explicitly predicts depth distribution and projects im- age features onto bird's-eye view (BEV), which has been
In summary, our major contributions are follows:
method and transformer- based
Towards SDG for multi-camera 3D object detection, we present the Fourier Cross-Viens Learning frame- work to fully exploit natural cross-view input.
We propose FHiAug, a novel, efficient, plug-and and-play augmentation strategy that operates on both image and
2

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
proved practical for 3D object detection. Based on LSS, BEVDepth designs novel Depth Refinement Module to refine feature projection process (Li et al., 2023). BEV- Former further designs a transformer structure to automat- ically extract and fuse BEV features, leading to excellent performance on 3D detection (Li et al., 2022b).
Wasserstein Auto-Encoder (WAE) to relax the widely used worst-case constraint in meta-learning scheme. Adversar- ial generation can suffer from unstable training and mode collapse. It often requires a lot of experiments and fine- tuning to get a GAN to work well, making them impractical for complex 3D detection models.
2.2. Single Domain Generalization
3. Methodology
Domain Generalization (DG) aims to generalize a model trained on multiple source domains to a target domain which distributionally different (Muandet et al., 2013). Data aug- mentation has been a common strategy to regularize the training process avoid overfitting and improve generaliza- tion (Zhou al., 2023). CIRL (Lv et al., 2022) generates augmented images by causal intervention module with intervention upon non-causal factors. AGFA (Kim et al., 2023) trains the classifier and the amplitude generator ad- versarially to synthesize a worst-case domain for adaptation. Compared with these methods, our proposed method is sim- ple, stable, yet effective, without extra module design or special training strategy.
3.1. Overview of Fourier Cross-View Learning Framework
The overall structure of our framework is depicted in Fig.2 and the process is outlined in Algorithm 1 in Appendix B.
The FCVL framework is motivated by the cross-view con- sistency in multi-camera 3D object detection. For example, as shown in Fig. objects such as cars or pedestrians are often visible across multiple adjacent camera views. To capture this cross- view relationship, we implement Fourier Cross Semantic Consistency Loss, where fea- tures from nearby camera views are considered positive sam- ples, while those from distant views are treated as negative samples. However, its effectiveness limited the single- domain setting due to restricted feature diversity. Therefore, we propose Fourier Hierarchical Augmentation (FHiAug) to alleviate the bias in single-domain representations, FHi- Aug, on one hand, expands the domain diversity to force the model to learn from different feature distributions. On the other hand, it expands the quantity and diversity of the cross- view sample pairs, enabling the consistency loss to more effectively explore semantic alignments between adjacent perspectives.
This paper focuses on Single Domain Generalization (SDG) (Wang et al., 2023b) which is a more challenging yet realistic setting. Existing works enhance data diversity first using 2D data augmentation techniques, including three categories of data augmentation methods: photometric and geometric augmentations, style transfer, and data genera- tion. Chen et al. (2023) use photometric augmentations (e.g., Invert) and geometric augmentations (e.g., Rotation) to create diverse domains, then learn to analyze the causes of domain shift, and finally learn to reduce the domain shift for model adaptation. BEV representations are generated by projecting multi-view 2D features using real-world physical constraints, which limits the use of geometric transforma- tions. For example, large rotation of the input images will disrupt the spatial relationships, thereby affecting the spa- tial consistency of the BEV features'. Zhao et al. (2024) propose CPerb, a simple yet effective potssppeturrbation method with style transfer to enhance the diversity of the training data and introduce multi-route perturbation to learn Nomain-invariant features. Style transfer techniques (Nuriel al., 2021) replace the original image statisties with those from the target style, but this often blurs the boundary be- tween style and content (Lee et al., 2023), distorting impor- tant features and ultimately harming model generalization. Wang et al. (2023b) propose a style-complement module with generative adversarial network to enhance the gener- alization power of the model by synthesizing images from diverse distributions that are complementary to the source ones. Qiao et al. (2020) leverage adversarial training to create "fictitious" yet "challenging" populations and use "More experimental analysis can be found in the Appendix C.
In the following sections, we provide in-depth explana- tion of both the Fourier Hierarchical Augmentation and the Fourier Cross-View Semantic Consistency Loss.
3.2. Fourier Hierarchical Augmentation
Fourier Hierarchical Augmentation includes data augmen- tation at image level (Frequency Jitter) and domain per- turbation at feature level (Amplitude Transfer), which is a plug-and play method to boost domain diversity without extra module designing or special training strategies.
Frequency Jitter at image level For a single channel image x E R11xd2 the 2D Fourier transformation defined as follows,
di d ((x)
Σ x(m. exp
27i(
ma dj
dg
(1)
mm0 mm
where denotes Fourier Transform; d, and d, denote height and width of the image; u and v denote frequency coordinates; m and denote spatial coordinates.
3

---

Generalizable Multi-Camert 3D Object Detection from a Single Source via Fourier Cross- Learning
Fourier Cross-View Learning
Cross-
Adjacent perspectives
Domain diversity
Domain invariant
Frequein Prequency Jitter
Ammplttue Transfer
CosSsross- view
Muti-view
BEV features
Loss cross
Project
Styled features
FFT
FFT
Adjacent perspectives
Amplitude
Phase
Amplitude
Styled features
Phase P
Front view
Right view
Augmented
Uncertainty Estimation
(-,((,1))
x (0,1)
IFFT
cross-camera styles
iFFT
(r o*))
Pull closer
Push away
A o'AA-pl/opp'
Frequency Jitter
Amplitude Transfer
Los vtew
Image Level
Feature Level
Semantic Consistency
Figure Overview our FCVL. framework. FCVI. includes two major parts: FHiAug to boost domain diversity and Fourier Cross- View Semantic Consistency Loss to ensure domain-invarian BEV features. FHiAug consists of two stages. One is Frequency Jitter image level. The other is Amplitude Transfer at feature level. Notably, we achieve cross-camera augmentation via FHiAug, which means a set of surrounding views have different "styles". This forces the model to learn from diversified domains. Besides, cross-view features, we calculate Fourier Cross- View Semantic Consistency Loss to learn more domain-invariant BEV features from adjacent perspectives.
The amplitude components A and phase components P are then respectively expressed as:
mation to generate the augmented image at
F()) v) = A(z)(u, v) ee )+P())())))
(u,) u) = [F² + (1, ((x)(u, (u,
(4)
f [F
(2)
(lx)(u, u) = arctan
In the training phase, we set p; as the calling probability of Frequency Jitter and sample p ~ UO, 1). For image input 2r acquire the augmented Tang
where R(x) and (x) represent the real and imaginary part of F F(x), respectively.
Fang Frequency if Pt-
To generate diverse samples that complement the single source domain, we employ two strategies. First, we perturb the amplitude component using a hyperparameter, a, to cre- ate variations in low-level statistics. Second, we modify the intensity of the phase component with hyperparameter, 8, expose the model to previously less emphasized features (Chen et al., 2020).
This Fourier-based augmentation strategy, termed Fre- quency Jitter, manipulates both amplitude and phase ponents, as shown in Fig.7. The top row demonstrates adjustments to the amplitude, primarily affecting image brightness, which helps the model become robust to varying lighting conditions. The bottom row shows modifications to the phase component, creating samples with varying levels of semantic detail while preserving the overall structure. This controlled manipulation of semantic strength encour- ages the model learn more Nomain-invarian and robust features. Additional examples highlighting the effect of phase adjustments are provided in Fig. 10.
Á(z)(u, =) ((())(())))
(3)
o)) SP(x)(u,))))
where U(,, and the hyperparameter 7 control the strength of the augmentation amplitude; 3 ~ U(X, 1) and the hyperparameter À control the strength of the aug- mentation on phase.
Amplitude Transfer at feature level To implement do- main perturbation and create diverse virtual styles during training, we apply Amplitude Transfer based on the style
With new amplitude and phase component, we can form a new Fourier representation and use inverse Fourier transfor-

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
statisties of intermediate features. This approach aims to improve model robustness and generalization.
images, while the adjacent columns display styled variations. As observed, the augmented images exhibit different colors and textures, showcasing the effectiveness of the proposed method in generating diverse feature styles. The visualized results of FHiAug are shown in Fig.9. For the same image, via FHiAug, we can generate multiple styles.
Given an intermediate feature map X E RBxCXHXW, where B, C. H. and W denote batch size, number of chan- nels, height, and width, respectively, we first perform a Fourier transformation and extract its amplitude component A(X) E Bx We then compute the channel-wise mean (p) and standard deviation (r) for each instance's amplitude follows:
3.3. Fourier Cross-View Semantic Consistency Loss
For multi-camera 3D object detection, the input inherently includes cross-view data, which is beneficial for learning domain-invariian features. This has not yet been harnessed to improve generalization. As illustrated in Fig.3, consider car appearing in both the front and front-right views. Such cross-view targets are common in multi-camera inputs, pro- viding natural opportunities to observe the same object from different perspectives.
W
((A())) HW
A(X),
(6)
H
W
oA(A(X)) = HW
Σ
E
A(X) U(A(X)))?
Now, we acquire the style statistics (()) and (A(()) of the features. To achieve feature-level perturbation, differ- ent from (Xu et al., 2021) and (Zhou et al., 2021) to mix up different domains' style information directly, inspired by (Li al., 2022a) we make uncertainty estimation on ((A()) and or (AA()) with the variance as follows:
To exploit this, we propose the Fourier Cross-Viecs Seman- tic Consistency Loss to help the model learn more robust features from adjacent views. Unlike conventional consis- tency losses that operate in the pixel domain, we minimize the distance between the phuse distributions of the targets with the same semantics, as the phase component usually encodes high-level semantic information. Concretely, for adjacent views, we split the features into halves as shown in Fig.3. There is cross-vier -ves instance binding mechanism that the identical instance labels are assigned to cross-view instances of the same object. When taking a target object in one view the anchor, we search for objects with the same instance label in adjacent views as positive samples. For negative samples, we choose samples of different categories from other views. Next, we calculate triplet loss (Schroff et al., 2015) in the frequency domain to explore potential semantic similarity as follows:
Var(u(A(())) = 1
1
M (((E)) - E(M(A(X)))
B
Va(()(A(()))) = B
(A(X))
-(((A())))
b=1
(7) where B is the batch size and E denotes the mathematical expectations.
Next, we obtain new style statistics B and by random sampling from the Gaussian distributions:
((A(X)) = ((A())) &H V Var(m(((()))) cp 1),
pos
FHiAug (O
(A(()) = (A())) + ear (ar(((((())) Eor ~ (O 1).
'ang
(11)
nag ang
FHiAug (1. (E")),
(8)
Finally, we replace the original style statistics with the per- turbed values and perform an inverse Fourier transform to obtain the augmented feature map X:
a = P(u anchor) ),P (( ) = ((v aau
(12)
Cesoss = max (dist(a, p) - dist(a, n) + margin, 0), (13)
Á(x) = ((())) X A(X) (((())) + 3(A()))
where FHiAug is the proposed augmentation method; yanchor is the anchor example; y is the sample with the same cate- gory as anchor; o0 is the sample with different categories; P denotes calculating the phase components of different views after Fourier transformation; dist is the distance mea- surement; margin is a constant greater than zero
T (A(X))
(9) This allows us create diverse styled features in each train- ing iteration without explicitly defining content and style. During training, we set p/ as the probability of applying Amplitude Transfer and sample p ~ (0, 1). For a given feature input X, the augmented feature Xaug is generated as follows:
Overall, the training objective loss including detection loss Lae and consistency loss Leross can be written as:
Xang = Amplitmmp_eeeaeaeeeee.i CrmplitudeTTranfer((2.
(10)
Cuain
Lae + T Leross
(14)
We visualize the style variations of some pictures via Ampli- tude Transfer in 8.g.8. The left column shows the original
where y is the weighting parameter to balance different loss terms.
5

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
①
Split adjacent perspectives to halves
creasing from 0.3028 to 0.3567 (t 5.39%). FCVL achieves SOTA out-of-domain performance across different SDG or DG methods. On LLS-based framework, FCVL achieves SOTA performance on both BEVDepth and BEVDet as well. In terms of the average NDS of eight testing domains, our method achieves much better performance than BEVDepth (t 5.05%) and BEVDet (1 6.07%). Especially, FCVL im- proves the performance of BEVDepth by 10.87% for ad- verse weather conditions Snow and 10.19% for Low Light. Similarly, FCVL improves the performance of BEVDet by 8.07% for adverse weather conditions Snow and 12.50% for Low Light. FCVL has more stable generalization ability for adverse weather and light conditions on different 3D detec- tion frameworks. For worst cases Low Light and Snow, as shown in Fig.4(b). FCVL has shown significant improve- ment. Overall, as is shown in Fa), the proposed FCVL outperforms other methods with great margin on average of three frameworks (1 2.08%). Besides, for both transformer- based framework and LLS-based framework, FCVL has the superiority in stable maintenance of better generalization ability in eight testing domains, especially in Low Light, Motion Blur and Snow.
Cam front
Cam front right
Neg
Anchor
Pos
FHiAug
Push away
3
Pull same semantics closer
Figure 3. Illustration of Fourier Cross- View Semantic Consistency Loss.
4. Experiments
In this section, showcase the overall performance of our method. To comprehensively evaluate our method, we conduct extensive experiments across four distinct baseline models, four varied domain generalization methods, and three different datasets. To gauge the practical viability and performance our algorithm, we also evaluate it within real-world application scenarios.
To further evaluate our method, we extend our methods to the 3D detectors without explicit BEV features, such as Sparse4D (Lin et al., 2023) and multi-modal method, such as BEVFusion (Liu et al., 2023) as well. We list the experimen- tal results including different 3D detection schemes (explicit BEV or not, multi frames or not, etc.) in the Table 2. Our method can improve the out-of -ntt--i-iitriutiionn performance in all the settings, while maintaining the in-distribution perfor- mances as far possible.
4.1. Implementation details
We first choose three baselines BEVFormer, BEVDepth and BEVDet and experiment on nuScenes and nuScenes- C (Xie al., 2023). Besides, choose new SOTA Far3D (Jiang et al., 2024) as another baseline and experiment on dataset Argoverse 2 (Wilson et al., 2023). More details of datasets can be found in Appendix D.I. We use ResNet50 as the backbone and extend our method to different baselines re- spectively. All models are trained until full convergence for 24 epochs. All parameters in our framework are initialized from ImageNet. We apply an AdamW optimizer with the learning rate set to 0.0002 and set the batch size to 2 per GPU. All experiments are conducted with 4 GPUs. After training, we mainly report mean Average Precision (mAP) and nuScenes Detection Score (NDS) (Caesar et al., 2020).
More results on Argoverse 2 are shown in Table 3. We experiment on a new SOTA Far3D, which presents a sparse query-basedd method for multi-view 3D long-range detec- tion without explicit BEV features. To achieve training on one domain and testing on unseen domains, we sample data from sunny weather in Miami scenarios the training data and data from cloudy weather or other five cities as the ood test set. As is shown, our method improves the general- ization for long-range detection as well. In addition, we have visualized some detection results in Fig. 12. As can be seen, after being optimized by FCVL, Far3D exhibits great performance when the lighting conditions are poor.
4.2. Comparison with SOTA methods
We compare our method with some SOTA SDG and DG methods which involve rrequency-domaii data augmenta- tion (CPerb (Zhao et al., 2024), FACT (Xu et al., 2021)) and style transformation (DAC-SC (Lee et al., 2023)). PD- BEV (Lu al., 2025) working on BEVDepth, is proposed to ensure consistent and accurate detection and improve generalization via perspective debiasing.
4.3. Ablation Study on nuScenes
Effects of different components of FCVL Firstly, we analyze the effects of different components of Frequency Jitter at image level, as shown in Table 4. As is shown, jittering amplitude only(#1) jittering phase of the input only (#2) has impressive performance. When jittering both phase and amplitude (#3), Frequency Jitter improves the performance further. Next, we separately examine the ef-
The results on nuScenes and nuScenes-C are shown in Ta- ble 1. On transformer-basee framework, our FCVL can greatly improve the generalization of BEVFormer as shown in Table 1. The average NDS of eight testing domains is in-
6

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-Vicw Learning
Table Comparison with baseline methods on nuScenes and nuScenes- C. The table represents the results of NDS with ResNet50 as backbone. "Clean" denotes the normal validation set of nuScenes. "OoD Avg." is the average performance of eight testing domains, FCVL achieves SOTA out-of-distribution performance on three frameworks. PD-BEV +(Lu et al., 2025) has released the code for BEVDepth. Thus, we mainly compare our method with PDBEV on BEVDepth for fair comparison. The best and second-best results are highlighted in Red and Blue, respectively,
Cam Crash
Frame
Color Quant
Motion Blur
Low Light
Model
Clean
Bright
Fog
Snow
Ool Avg.
Lost
BEVFormer (Li et al., 2022b)
0,4362
0.3175
0.3246
0.3410
0.2549
0,4022
0.2461
0.3853
0,1510
0.3028
+CPerb (Zhao et al., 2024)
0.4356
0.3199
0.3292
0.3372
0.2548
0.4096
0.2420
0.3907
0.1661
0.3062
+DSU (I.i ct al., 2022a)
0,4359
0.3206
0,3322
0.3609
0.3425
0,4083
0.2458
0,3937
0.2601
0.3330
+DAC-SC (I.ce al., 2023)
0.4332
0.3085
0,2872
0.3703
0.3691
0.4161
0.3155
0.4093
0.3086
0.3481
+FACT (Xu et al., 2021)
0,4379
0.3181
0.3285
0.3436
0.2585
0,4100
0.2494
0.3916
0,1486
0.3060
+FCVL(Ours)
0.4375
0.3244
0.3374
0.3751
0.3748
0.4202
0.3078
0.4170
0.2969
0.3567
BEVDepth (Li et al., 2023)
0.4028
0.2654
0.2178
0.2801
0.2697
0.3072
0.1558
0.3080
0.0881
0.2365
PD-BEV t (Lu et al., 2025)
0.4094
0.2822
0.2316
0,3102
0.2842
0,3011
0.1411
0.3151
0,1091
0.2468
+CPerb (Zhao ct al., 2024)
0.4034
0.2698
0.2294
0.2847
0.2873
0.3180
0.1616
0.3164
0.1054
0.2466
+DSU (Li et al., 2022a)
0.4057
0.2722
0.2330
0.3065
0.3270
0,3462
0.2165
0.3249
0,1565
0.2729
+DAC-SC (Lee al., 2023)
0.4007
0.2714
0,2200
0.2846
0.2861
0.3284
0.1586
0.3172
0.1299
0,2495
+FACT (Xu et al., 2021)
0.4026
0,2670
0,2224
0.2872
0.2749
0.3276
0.1611
0.3141
0.0957
0.2438
(Our))
0.4050
0.2722
0.2346
0.3106
0.3318
0.3539
0.2577
0,3380
0.1968
0,2870
BEVDet (Huang et al., 2022)
0,3880
0.2508
0.1955
0.2409
0.2201
0.2591
0.1112
0,2633
0.0728
0.2017
+CPerb (Zhao et al., 2024)
0,3908
0,2590
0.2065
0.2479
0.2325
0,2643
0.1322
0,2752
0,0782
0.2120
+DSU (Li et al., 2022a)
0.3835
0,2582
0,2061
0.2814
0.3019
0.3128
0.1806
0.2961
0.1065
0.2430
+DAC-SC (Loe et al., 2023)
0,3884
0,2574
0,2046
0.2688
0.2644
0.2986
0,1450
0.2926
0.1028
0.2293
+FACT (Xu et al., 2021)
0,3907
0,2581
0.2054
0.2430
0.2277
0,2708
0.1230
0.2727
0,0773
0.2098
+FCVL(Ours)
0.3848
0,2579
0.2064
0.2928
0.3204
0.3244
0,2393
0,3156
0.1848
0.2677
Table 2. The table represents the effectiveness of our proposed method under different settings on nuScenes and nuScenes-C. "C" denotes camera input. "L" denotes lidar input. "Explicit BEV" means 3D detectors generate explicit BEV features. "Temporal" denotes whether utilizing multi frames. "Depth" denotes whether utilizing depth information. Bold fonts indicate the best results.
Table 3. Comparison with baseline methods on Argoverse 2 based
on Far3D (Jiang et al., 2024). The table represents the results of
MAP 1. "Clean" denotes the in-domain set.
Model
Clean
City
Cloudy
OoD Avg.
Far3D
0.1964
0.1140
0.0800
0.0970
+CPerb
0,2044
0,1172
0,0820
0.0996
Explicit BEV
+DSU
0.2072
0.1352
0,0904
0.1128
Model
Modality
Temporal
Depth
Clean
OnD Avg.
+DAC-SC
0.1904
0.1152
0.0830
0.0991
BEVFormer
0.4362
0.3028
+FACT
0,2052
0.1302
0.0898
0.1100
+FCVL(Ous)
0,4375
0.3567
(OVLL(Ours)
0.2170
0,1610
0,1082
0. 0.46 (1 2.18%)
BEVDepeh
0.4028
0,2365
+FCVL(Our))
0,4050
0,2870
BEVDepth
0.4828
0.4128
+ICVL(OUus)
/
v
0.4827
0.4291
last, we add our Lemes in the training (#6). This further enables the model to learn more domain-invariant features. Notably, the consistency loss is not only beneficial for the in-domain performance, but also boosts the out-of-domain performance.
BEVDet
0,3880
0.2017
+FCVL(Our)
0.3848
0,2677
Spanse4D3
0.5590
0.4431
+FCVL(Ous))
V
0.5592
0.4492
BEVFusion
L+C
0.7074
0.6865
+FCVL(Ous)
L+C
0,7123
0.6948
feet of the Amplitude Transfer at feature level (#4). It can be observed that enhancing at feature level alone can also lead to great improvement. Then, we combine Frequency Jitter and Amplitude Transfer (#5). Importantly, the com- bination of image and feature levels augmentations yields more significant performance gains. The image level aug- mentation expands the data distribution first, while feature level augmentation capitalizes on this augmented input to derive fine-grained style variations, further enhancing repre- sentation diversity. This hierarchical augmentation pipeline increases domain diversity collectively, thereby enhancing the model's 's ability to generalize to unseen scenarios. At
Effects of different inserted positions of Amplitude Transfer We evaluate the impact of different inserted po- sitions of Amplitude Transfer, as shown in Table 5. Inserted position of ResNet is numbered as follows: after first Conv O after Max Pooling layer 1, after first Resblock 2, after second Resblock 3, after third Resblock and after fourth Resblock 5, respectively. According to (Zhou et al., 2021). Resblock 1 to Resblock 3 contain domain-related informa- tion, which means domain-related information usually lies in shallow layers. Thus, in our method, Amplitude Trans- fer is inserted in Position 0-2. We make more experiments by increasing Position 3-5 gradually to find more suitable
7

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-Vics Learning
0.34
Ours
0.3038 (-0 (0.00%
Ours
FACT
0.2533 (.3.06%)
FACT
P_
P_
0.2755
DAC-SC
XDS
0022330 (-2.00%))
DSU
7.93%
+5.87%
0.31
CPerb
00:2530
(-1.90%)
CPerb
0.30
Baselines
0.2477 (-5.6 68%
BEVDet
Snear
0.29
0.22
0.1
0.0
0.1
NDS
NDS
(a)
(b)
(c)
Figure 4. (a) The average detection results of different methods including eight OoD domains under three baseline frameworks. As shown, the proposed FCVL outperforms other methods with great margin on average. (b) Worst cases analysis. Our method has shown significant improvement in the worst cases, Low Light and Snow. (c) Hyperparameters analysis of FHiAug. The strength of augmentation and À for Frequency Jitter; the probability for Frequency Jitter and py for Amplitde Transfer.
the out-of-domain performance gradually improves. After reaching a certain level of probability and intensity, fur- ther changes in the parameters will no longer cause drastic changes in ood performance, indicating that the model is stable against the
Table Ablation Study on different components of FCVL on BEVDepth (Li al., 2023). Amp. denotes Amplitude. Amp. Jitter means only jittering on amplitude component. Phase Jitter mcans only jittering on phase component. Jittering on both is the Frequency Jitter operated at image level. Amplitude Transfer is op- crating feature level. Laess denotes consistency loss. denotes performing corresponding operation. Bold fonts indicate the best results.
misspecifications as long as are within reasonable ranges. We set = 0.25, X = 0.5, p: 0.25 and py = 0.75 as the final setting.
Image Level
Feature Level
Amp. Jiner
Phase
Amp. Transter
Cara
Clean
OnD Avg.
The effect of Fourier Cross- View Semantic Consistency Loss alone are shown in Table 6.
00
0.4028
0.2365
#
0,4037
0.2735
0,2690
3
0,4037
0.2767
Table 6. The effect Fourier Cross-View Semantic Consistency
0,4022
0.2570
Loss alone. the weight of L
8
0,4004
0,2843
66
0,4050
0.2870
Quait
Bright
Light
Fog
Snow
Avg.
BEVDe
0,2508
0.1955
0.2409
0.230]
0.259
0.1112
0.2633
0,0728
0.2017
positions. As shown in Table 5, in terms of in- and out- of distribution performance, inserting Amplitude Transfer in Position 0-3 achieves both the best performance.
05
0,2467
0.1942
0.241
021322
0.2585
0.1328
0.2655
0.0641
0.2024
1.0
0.2501
0.1952
0.725
0.2822
0.2590
0.1467
0.2807
0.1140
0.2296
0.2808
0.1147
0.2778
4.5. Efficiency Analysis
Table 5. Effects of different inserted positions of Amplitude Trans- fer. Inserted position of ResNet is numbered as: after first Conv O, after Max Pooling layer after first Resblock 2, after second Res- block 3, after third Resblock 4 and after fourth Resblock S. "0-5" means inserting Amplitude Transfer from Position 0 Position S.
In this part, we make efficiency analysis delve into the proposed FCVL. We investigate how the method scales with increasing image resolution and computational complex- ity. The results are listed in Table 7. As it can seen, at larger image scales, FCVL can still significantly improve the model's generalization performance. Though there will a slight increase training time (+0. 11s per training step) during the training phase, the proposed FCVL only used during the training phase, it introduces no latency in inference phase. Without the need for more time-consuming or costly data collections, the proposed FCVL can improve the generalization performance almost for free, which is highly valuable for practical applications.
Model
Clean
OoD Avg.
BEVFormer
0.4362
0.3028
0-5
0.4404
0.3267
0-4
0.4393
0.3289
0-3
0.4421
0.3294
0-2
0.4404
0.3280
4.4. Hyperparameterss Analysis
In Frequency Jitter, there are three hyper-parameters. The hyperparameter 17 controls the strength of Amplitude aug- mentation; À controls the strength of Phase augmentation and p; is the provability of implementing Frequency Jitter. For Amplitude Transfer, we experiment on the probabil- ity py of implementing Amplitude Transfer. As shown in Fig.4(c), 4(c), initially, as the probability and intensity increase,
4.6. Evaluation in Real-world Autonomous Driving Scenarios
To further evaluate the performance of our algorithm in practical application scenarios, we test our method in the night with the model training on daylight samples only.
8

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
Table Efficiency analysis of FCVL. "Train" refers to the time takes for one training step when the batch size is 1. "Inference"
5. Conclusion
In conclusion, this paper addresses the challenge of Single Domain Generalization in multi-camera 3D object detection via Fourier Cross-Viees Learning framework. We propose a con-parametric Fourier Hierarchical Augmentation at both image and feature levels to enhance data diversity and Se- mantic Consistency Loss to facilitate model to learn more domain-invariant features from adjacent perspectives. Be- sides, via augmentation graph theory, we make valid theoret- ical guarantees. Extensive experiments have demonstrated that our approach achieves the best performance across vari- ous domain generalization methods.
refers to the time for inferring a single sample. "Memory" is the
consumed GPU memory during training with batch size 1.
Model
Resolation
Tranns)
Inferenecis)
Memory(MB)
OuD Avg.
BEVDs
256
5498
0,2017
+FCVL
256 x 701
0.361
0,073
7383
0 6.6 6.00%)
BEVDs:
1408
11698
0,2006
+FCVL
512 1408
0.605
0.143
20094
0.2394(1 3.89%)
More results can be found in Appendix D.3. This example in Fig.5 well demonstrates that our model can robustly deal with rapid environmental changes, such as variations in lighting conditions. As can be seen, in the distance where vehicles are dense and the lighting is very strong, the model can stably detect the targets. Although the model has only seen normal daylight samples, with our proposed FCVL., it also performs well under the extreme changes in light condition at night.
Limitations There are several hyperparameters to be tuned in FCVL. In the future work, we can explore additional techniques to avoid spending too much time on tuning hy- perparameters. Additionally, for Snow, we have already improved by 10,, but the performance is still much worse compared to the performance in other scenarios such as Low Light. Consequently, there is substantial potential for enhancement in adverse weather conditions.
Acknowledgements
Thanks to reviewers for your professional and insight- ful review comments. This work supported by National Natural Science Foundation of China (Grant No.62106139).
Figure 5. Visualized detection results of FCVL at night with light variations. The model is trained on only daylight samples with the proposed FCVL.
Impact Statement
4.7. Visualization Analysis with t-SNE
This paper presents work whose goal is to advance the field of Machine Learning and Autonomous Driving. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
We use t-SNE to visualize the BEV features from different domains of BEVDet and FCVL. In the Fig.6, source domain represented in red and other colors represent different target domains. We can find that the features of different domains extracted from BEVDet distant from each other and loosely distributed in the feature space. While, after optimization with FCVL, the distribution of four domains becomes more compact and connected, which is in line with augmentation graph theory. FCVL increases the augmenta- tion graph connectivity between source and unseen domains and improve the generalization a lot.
References
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O. nuscenes: A multimodal dataset for autonomous driving. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11618-11628, 2020. doi: 10.1099CVPR4260022000..11
60
Chen, G., Peng, P., Ma. L., Li, J., Du, I.,, and Tian, Y. Amplitude-phase recombination: Rethinking robust- ness of convolutional neural networks in frequency do- main. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 448-457, 2021. doi: 10.11009/CCV4992222200. 100051.
40
Source Domain
20
-20
Chen, J., Gao, Z., Wu, X., and Luo, J. Meta-causal learning for single domain generalization. In 2023 IEEF/CVF Conference on Computer Vision and Pat-
-40
-10
-00
-00
-10
-20
20
40
60
80
-60
20 BEVD&
-20
BEVDet
Figure I-SNE Visualization FCVI..
9

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Gross-View Learning
tern Recognition (CVPR), pp. 7683-7692, 2023, doi: 10.1 0T0./CCVRR577..223..0777777
Lee, S., Bae, J., and Kim, H. Y. Decompose, adjust, compose: Effective normalization by playing with fre- queney for domain generalization. In 2023 IEEE/CVE Conference on Computer Vision and Pattern Recogni- tion (CVPR), pp. 1 11776-11785, 2023.
Chen, P., Liu, S., Zhao, H., and Jia, J. Gridmask data augmentation. 2020.
doi: 10.1109/
CVPRS2729.2023.011
Finn, C., Abbeel, P., and Levine, S. Model-agnostic meta- learning for fast adaptation of deep networks. In Pro- ceedings of the 34th International Conference on Ma- chine Learning- Volume 70, ICML'17, pp. 1126-1135. JogR.org, 2017.
Li, X., Dai, Y., Ge, Y., Liu, J., Shan, Y., and Duan, L. Y. Un- certainty modeling for out-of-distribun generalization. arXiv e-prints, 2022a.
Li, Y., Ge, Z., Yu, G., Yang, J., Wang, Z., Shi, Y., Sun, J., and Li, Z. Bevdepth: Acquisition reliable depth for multi-view 3d object detection. In Proceedings of the AAAI conference on artificial intelligence, volume 37. pp. 1477-1485, 2023.
Goodfellow, I. Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley Farley, D.. Ozair, S., Courville, A., and Bengio, Y. Generative adversarial networks. Communications of the ACM, (3(11): 2020.
HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T. Prov- able guarantees for self-supervised deep learning with spectral contrastive loss. Advances in neural information processing systems, 34:5000-5011, 2021.
Li, Z.,, Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Qiao, Y., and Dai, J. Bevformer: Learning bird's "erd's-eee-vew repre- sentation from multi-camert images via spatiotemporal transformers. In European Conference on Computer Vi- sion, pp. 1-18, 2022b.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Lin, X., Pei, Z., Lin, T., Huang, L., and Su, Z. Sparse4d v3: Advancing end-to-end 3d detection and tracking. arXiv preprint arXiv:2311.1172) 2023.
Huang, J., Huang, G., Zhu, Z., Ye, Y., and Du, D. Bevdet: pengoppeffrrance multi-camert 3d object detection in bird-eye-view, 2022.
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, L., and Han, S. Bevfusion: Multi-task multi-sensor fusion with unified bird's s-eye view representation. In 2023 IEEE international conference robotics and aut- tomation (ICRA), pp. 2774 2781. IEEE, 2023.
Huang, X. and Belongie, S. Arbitrary style transfer in real- time with adaptive instance normalization. In 2017 IEEE International Conference on Computer Vision (ICCV), pp- pp. 1510-1519, 2017. doi: 10.1009//CV22017.111.
Lu, H., Zhang, Y., Wang, G., Lian, Q., Du, D., and Chen, Y.C. Towards generalizable multi-camera 3d object de- tection via perspective rendering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 5811-5819, 2025.
Jang. X., Li, S., Liu, Y., Wang. S., Jia, F., Wang. T., Han, L., and Zhang, X. Far3d: expanding the horizon for surround-view 3d object detection. In Proceedings of the Thirty-Eighth AAAI Conference Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence. AAAI Press, 2024. ISBN 978-1-57735-88 -87-7993-8889.....8 doi: 10. 10.1609aaai. 28833. URL https: / /tt.: 1609/ aaai. v3813...... 28033.
Lv, F., Liang. J., Li, S., Zang, B., Liu, C. H., Wang, Z., and Liu, D. Causality inspired representation learning for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8046-8056, 2022.
Muandet, K., Balduzzi, D., and Schölkopf, B. Domain gen- eralization via invariant feature representation. Computer Science, pp. 10-18, 2013.
Jiagg Y., Zhang. L., Miao, Z., Zhu, X., Gao, J., Hu, W. and Jiang, Y.-G. Polarformer: Multi-camera 3d object detection with polar transformer. In Proceedings of the AAAI conference on Artificial Intelligence, volume 37, pp. 1042-1050, 2023.
Nuriel, O., Benaim, S., and Wolf, L. Permuted adain: Re- ducing the bias towards global statisties in image classi- fication. In 2021 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 9477-9486. 2021. doi: 10. 11111//CVP24444..222000.00
Kim, M., Li, D., and Hospedales, T. Domain generalisation via domain adaptation: An adversarial fourier amplitude approach, 2023. URL https: / /arxiv...v. org/ abs/ 2302. 12047.
Philion, and Fidler, S. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to
10

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
3d. In Vedaldi, A., Bischof, H., Brox, T., and Frahm, J.- M. (eds.), Computer Vision ECCV 2020, pp. 194-210, Cham, 2020. Springer International Publishing. ISBN
Xu, Z., Liu, D., Yang. J., and Niethammer, M. Robust and generalizable visual representation learning via random convolutions. 2020.
978-3-030-5866-6.
Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez-Paz, D. mixup: Beyond empirical risk minimization. 2017.
Qiao, F., Zhao, L., and Peng, X. Learning to learn single domain generalization. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12553-12562 2020. doi: D0.110/CCPR44200020000 2020.. 01257.
Zhang, Y., Zhu, Z. H., Zheng, W. Huang, J., Huang, Beverse: Unified per-
G., Zhou, J., and Lu, ception and prediction in birds-eye-view for vision- centric autonomous driving, ArXiv, abs/2205.09743, 2022. URL https: /ppi. semanticscholar: org/ Cow.oooID:2408001110
Schroff, F., Kalenichenko, D., and Philbin, J. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 815-823, 2015.
Zhao, D., Qi, L.,, Shi, X., Shi, Y., and Geng, A novel cross-perturbation for single domain generalization. IEEE Transactions on Circuits and Systems for Video Technol- ogy, 2024.
Wang. H., Wu, X., Huang. Z., and Xing, E. P. High- frequency component helps explain the generalization of convolutional neural networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), pp. 8681-8691, 2020. doi: 10.1109/ CVPR422600.020.00877
Zhou, K., Yang, Y., Qiao, Y., and Xiang, T. Domain gener- alization with mixstyle, 2021.
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., and Loy, C. C. Do-
Wang, J.-Y., Du, R., Chang, D., Liang, K. and Ma, Z. Domain generalization via frequency-domain based fea- ture disentanglement and interaction. Proceedings of the 30th ACM International Conference on Multimedia, 2022. URL. https:/ /api. semant cscholar. org/ /as/Cooaa : :
main generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4339- 4415, 2023. doi: 10.1 1AL1099PALL22222319555999
Zhu, Z., Zhang, Y., Chen, H., Dong, Y., Zhao, S., Ding, W., Zhong, J., and Zheng, S. Understanding the robustness of 3d object detection with bird's v-ee- representations in autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 21600-21610, 2023.
Wang, S., Zhao, X., Xu, H.-M., Chen, Z., Yu, D., Chang, J., Yang. Z., and Zhao, F. Towards domain generalization for multi-view 3d object detection in bird-eye-views In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13333-13342, 2023a.
Wang. Y., Zhang, J., and Wang, Y. Do generated data always help contrastive learning?, 2024.
Wang, Z., Luo, Y., Qiu, R., Huang, Z., and Baktashmotlagh, M. Learning diversify for single domain generalization, 2023b.
Wilson, B., Qi, W., Agarwal, T., Lambert, J., Singh, J., Khandelwal, S., Pan, B., Kumar, R., Hartnett, A., Pontes, J. K., Ramanan, D., Carr, P., and Hays, J. Argoverse 2: Next generation datasets for self-driving perception and forecasting, 2023. URL httph:ttt///eett /ttp://///xvvooo abs/2301.00 00493.
Xie, S. Kong, L., Zhang, W., Ren, J., Pan, L., Chen, K., and Liu, 7. Robobev: Towards robust bird's eye view perception under corruptions, 2023.
Xu, Q. Zhang, R., Zhang, Y., Wang, Y., and Tian, Q. A fourier-based framework for domain generalization. In 2021 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 14378-14387, 2021. doi: 10. 11110//VR44447.222100000

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
A. Theoretical Analysis
To analyze the influence of data augmentation, we adopt the standard augmentation graph framework (HaoChen et al., 2021; Wang et al., 2024), where data augmentations induce interactions (as edges) between training samples (as nodes). Given natural data sample I E X, we use A(-IT) to denote the distribution of its augmentations. For any two augmented data r, x' X, define the adjacency matrix Wa as the marginal probability of and x' from a random natural data E Px:
W y'' = (
(15)
Le WWD - be the normalized graph Laplacian matrix, where D is diagonal degree matrix with the (z, x)-th diagonal element Dz , Wzz'
Based on the above augmentation graph framework, we construct the augmentation graph G(X, X, W) in the feature space for single source domain and augmented domains as shown in Ic). Then, we have the following theorem:
Theorem A.1. For the optimal encoder f", BEV projection module Paev. a learned classification head C" and regression head It on augmented data X, its linear probing error has the following generalization upper bound,
20 Ak+1
,Lf",PrEv C", C²,
4AMe Ne) (AYrrrer)
(16)
where denotes the labeling error caused by data augmentation; Ax+1 denotes the k + 1-th smallest eigenvalue of the Laplacian matrix L; (VVe: úe) denotes the average disagreement between de and the ground-truth labeling ye for classification; (yrr Fr) denotes the average disagreement between y, and the ground-truth labeling yr for regression.
Based on the generalization upper bound in Eq.16 (proof in AppendixA.1) 1), we can provide rigorous explanations to show that our method can increase graph connectivity A and reduce label error decrease the generalization loss.
First, as shown in Fig. (c)), few feature points are connected. There is large margin between the source and target domains. The proposed FHi plays a positive role in expanding graph connectivity XR+1, since it creates more diverse "middle" domains between single source data and unseen target domains. According to augmentation graph theory, with the increase of augmentation strength, the graph connectivity λ4+1 can be increased. Via increasing Ac+1 in 1q.16, 16, the generalization upper bound can be tighter and the generalization ability can improved.
can only get access the single source data, the connectivity of the graph is poor and only
However, common strong augmentation, such as strong geometric enhancement, also causes label error (larger in 1q.16) and increases the generalization loss. The proposed FHiAug augmenting in the frequency domain can effectively alleviate this issue. Next, we will provide theoretical analysis and show that FHiAug can ensure semantic consistency under strong augmentation strength to increase connectivity. As mentioned in Sec.3, input data X' can be decomposed into two components: phase Xp, and amplitude Xz, where X₂ contains semantic information about the label y. denoting the causal component, and X₂ contains more low-level information, denoting the non-causal components.
Assumption A.2. We assume the linear relationship between Xx and y.
y = XnO EE,
(17)
where e is the noise, Cov(X,, e) = 0, E[Xp] = O.
Theorem A.3. If input data X consists of all the phase components, X' = Xp. the optimal linear predictor & can be estimated without bias. Otherwise, the predictor é is biased.
For some style transferring methods in pixel domain, both phase and amplitude components are modified. In this situation, the predictor & is biased, which means that the predictor probably gives wrong prediction of label. While the proposed method FHiAug augments in the frequency domain and retains the phase congruency, avoiding label error effectively. At image level, Frequency Jitter only adjusts the intensity of phase component in the global. The distribution of semantic At feature level, we achieve style transfer with operating on amplitude component only. More proof for Theorem A.3 is in Appendix A.2.
information not changed.
12

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
A.1. Proof for TheoremA.1
Lemma A.4. Let (X, W) be the augmentation graph, be the number of underlying classes, There exists an extended labeling function i such that
Σ 2
W T((()), 20.
(18)
Lemma A.5. (Theorem B.3 (HaoChen et al., 2021)). Assume the set of augmented data X is finite. Let f be the optimal encoder. Then, for any labeling function ù X t- [r], there exists linear probe B such that
Ea- Pa-PP--A = (Il((T) - (+) &
42 9):
(19)
AR+1
where 1 denotes the + 1-th smallest eigenvalue of the Laplacian matrix The (y denotes the average disagreement between and the ground-truth labeling
According above lemmas, for detection task, the optimal encoder f*, BEV projection module B'EEY a learned classification head C and regression head R on augmented data X, its linear probing enror has the following generalization upper bound,
E(f,, PE O", C*)
<
+ 1/44Au Mc) + 4A(gr Vr)
A
(20)
2a
+ (yec Mc) + (yr,, Dr).
Ak+1
where a denotes the labeling error caused by data augmentation; A ( Fe) denotes the average disagreement between yee and the ground-truth labeling for classification; Ayr, 9-) denotes the average disagreement between y, and the ground-truth labeling yr for regression.
A.2. Proof for TheoremA.3
The optimal linear predictor for = Xyo + is
x² ( -xx
(21)
(22)
Φ + (EIx;,xx))) vl E(X, X,10)
(23)
Φ + (B(x77%)) (v Tth Xp)]
(24)
ø (B1x/
(25)
= Φ + (E|x, E(e] + Cov(Xp, e))
(26)
(27)
If input data X' is deteriorated due to data augmentation in pixel domain, the phase components, X, = Xx- is no longer a distribution with E[XY-] 0. Then, the predictor biased:
x- = Φ + (E[2 p- x
(E E(c)
(28)
B. Algorithm
The algorithm of the proposed method is illustrated 1.
13

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
Algorithm 1 The proposed algorithm (FCVL)
Input: Training data (z, y), detector network f with parameter 0,learning rate 8, probability p; to do Frequency Jitter, probability py to do Amplitude Transfer.
Output: The optimized network parameter 0*
1: while / < T do
2: # Fouurie-bbsed data augmentation at image level.
3: Sample Po ~ 1))
4: for (Ir, () do
5: Po S ps then
6: Perform Frequency Jitter according to Eq. 3.
7: Obtain augmented image & according to Eq. 4.
8: else +- x
9:
10: end if
11: end for
# Fourier-based domain perturbation feature level.
Sample pl (0,
14:
for intermediate features X do
15:
if P1 Ps then
16:
Perform Amplitude Transfer according to Eq.6 Eq.9.
17:
Obtain perturbed features X according to Eq.4.
18:
else x
end if
21: end for
22: #Cross-view Semantic Consistency Loss.
23: -- V Cramu (( y), 6);
24:
end while
25:
return Save the optimized network f(0").
C. More Analysis of Common Data Augmentation Techniques for Single Domain Generalization
In SDG for 2D image classification, previous works (Zhao et al., 2024; Qiao et al., 2020) aim to enhance data diversity using common 2D data augmentation techniques, such as geometric transformations, style transfer, or data generation. However, directly applying these approaches to BEV-based tasks introduces several challenges.
First, BEV representations are generated by projecting multi-view 2D features using real-world physical constraints, which limits the use of strong geometric transformations, such 270-degree rotations, as they would disrupt the spatial consistency of the BEV space. We add strong geometric enhancement experiment including significant rotation and translation on the image and the results are as follows in the Table 8. For one thing, it shows that strong geometric enhancement hurts in-domain performance, as large scale rotation or translation may destroy physical restraints in the real driving scenario. For another, geometric enhancement is not very effective in improving OoD performance.
Table 8. Strong geometric enhancement experiments including significant rotation and translation on the images.
Cam
Frame
Color
Motion
Low Light
Model
Clean
Bright
Fog
Snow
OoD Avg.
Crash
Lost
Quant
Blur
BEVDet
0.3880 0.3505
0.2508
0.1955
0.2409
0.2201
0.2591
0.1112
0.2633
0.0728
0.2017
strong geo
0.2338
0.1875
0.2249
0.2030
0.2371
0.
0.2511
0.0639
0.0900
Second, some style transfer techniques (Zhao et al., 2024; Nuriel et al., 2021) replace the original image statistics with those from the target style, but this often blurs the boundary between style and content (Lee et al., 2023), distorting important features and ultimately harming model generalization. These methods need remove the "style" in the pixel domain first.
14

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning
Some content cues will be removed inevitably.
Third, data generation methods including adversarial generation and catuusionnbssed techniques. Training Generative Adversarial Network (GAN) (Goodfellow et al., 2020) involves a competitive process between two neural networks: the generator and the discriminator. Adversarial generation can suffer from unstable training and mode collapse. It often requires a lot of experiments and fine-tuning to get a GAN to work well. While diffusion- more stable, they need significant computational and storage overhead, making them impractical for complex 3D detection models. Additionally, although we can spend much time generating a large number of samples, we would also require extra storage space. However, our method involves online augmentation and does not require any additional storage space.
techniques(Ho et al., 2020) are
Therefore, common 2D data augmentations cannot be effectively leveraged to create diverse training samples for BEV-based tasks.
Besides, we further clarify the differences between our method and other frequency-domait approaches. Compared with these methods(Xu et al., 2021: Lv et al., 2022; Kim et al., 2023), our method has major strengths in two aspects including accuracy and efficiency. Firstly, in the setting of single source data, our proposed method can enhance the generalization ability of the detectors by large margin. FACT (Xu al., 2021) mixes up two different domains' data in frequency, e.g. Cartoon and Photo from dataset PACS and achieves great OOD performance in the paper. But when training with only single domain, FACT can only mix the samples within the single domain and it indeed improves the in-domain clean set little, but the improvement of OOD sets is very slim in the single domain setting. Different from FACT, we first propose Frequency Jitter at image level to create diverse samples that are complementary to the single source domain. Then, at feature level, we introduce a novel method Amplitude Transfer to achieve style transfer without content distortion. Through uncertainty estimation, we can obtain uncertain feature statistics, which can gradually shift the features to more diverse domains through continuous training. Secondly, due to the high complexity of BEV-based 3D object detection models, our plug-and-play and non-paramerter data augmentation method can achieve better generalization results more efficiently. CIRL (Lv et al., 2022) generates augmented images by causal intervention module with intervention upon non-causal factors. AGFA (Kim et al., 2023) trains the classifier and the amplitude generator adversarially to synthesise a worst-case domain for adaptation. Compared with these methods, our proposed method is simple, stable, yet effective without extra module designing or special training strategies.
D. More Experimental Results
D.1. Datasets
To verify different methods' single domain generalization ability, we first utilize nuScenes (Caesar et al., 2020) as the single training source and nuScenes-C (Xie et al., 2023) as the testing sets. NuScenes-C is comprehensive dataset that encompasses eight distinct corruptions, including Bright, Dark, Fog, Snow, Motion Blur, Color Quant, Camera Crash, and Frame Lost. Each type of corruption has three different levels of corruption intensity (i.e., easy, moderate, and hard). These cight corruptions include diffcrent weather conditions, different light conditions, potential equipment damage situations. These scenarios are common oul-of-distribution problems in real-world application. We use eight distinct corruptions as our multi testing domains to evaluate the effectiveness of different DG methods.
In addition, we experiment on public dataset Argoverse 2 (Wilson et al., 2023). The Argoverse contains different driving scenarios across six major U.S. cities (Miami, Washington D.C. and so on), including various weather conditions such as sunny days and cloudy conditions. To adhere to the single domain to multi domain generalization paradigm, we take sunny-day data from Miami as the single-domain training set, while sunny-day data from other cities (with diverse urban road structures) as the first ood test set (City), and cloudy (dim lighting) data from other cities as the second ood test set (Cloudy).
D.2. Experiments with Random Seed
In early experiments, we find that the effect of random seeds on BEVDepth or BEVFormer is relatively small. We run our method FCVL three times on BEVDepth and the average NDS on clean testing set is 0.4004 + 0.0002; the average NDS of OoD sets is 0.2845 + 0.0003. The standard deviation for three trials is 0.0002 or 0.0003, which means the method is quite robust different seeds. Thus, in later experiments, we run our method with random seed.
15

---

Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross- Learning
D.3. Evaluation in Practical Application Scenarios
To further evaluate the performance of our algorithm in practical application scenarios, we collect large dataset consisting of sunny daytime and nighttime. We train the detection model with our method on 61716 samples of sunny daytime and test on daytime (6169) and night (8200) samples. More results can be found in Table 9. Notably, on the night testing set, FCVL can improve the mAP from 0.0420 to O. 1.1004(( 5.84%).
Table 9, Evaluation results (mAP f) in the real world autonomous driving scenarios.
Model
Daytime
Night
Baseline
0.2690
0.0420
+FCVL(Ours)
0.2687
0.1004(t 5.84%)
E. Visualization Analysis
E.1. Visualization of Proposed FHiAug
In this section, we visualize diverse styles generated via FHiAug. Visualization of Frequency Jitter at image level and visualization of style variations via Amplitude Transfer are shown in Fig.7, 8 and 9. To better illustrate the phase component, we provide more examples of changing phase components in 10g.
Figure 7. Frequency Jitter at image level. The top line is adjusting the amplitude. It mainly influence the image brightness. The bottom line is adjusting the phase. As it can been, main structures of different targets have been retained.
Figure& 8. Amplitude Transfer at feature level. The left cols original pictures. The other two cols are styled pictures.
16

---

Generalizable Multi-Camert 3D Object Detection from Single Source via Fourier Cross- Learning
Original Img
FHi Aug
Figure 9. Visualized results of FHiAAg. For the same image, via FlliAug, we can generate multiple samples, which can facilitate the model learn more domain invariant feature.
Original Img
Image reconstractedwith phase componentonly
Phase *0.5
Phase *0.75
Figure 10. The sample the top right is the image reconstructed with the phase only. As it can be the phase components mainly contain the semantic information. The images in the bottom show how the image changing when adjusting the strength phase components only. The image after the phase changing is similar dirty lenses and weather changes in real world, still preserving key BEV prediction information.
E.2. More Visualized Detection Results
Notably, FCVL greatly improves the performance for Snow. We visualize some detection results of these samples to compare the performance between baseline models and FCVL in Fig. Under the condition of Snow, baseline model misses detecting the small targets severely, while FCVL can greatly alleviate the problem of missing detection. Compared with CPerb, FCVL still shows more stable and more accurate localization and recognition ability.
17

---

Generalizable Multi-Camera 3D Object Detection from Single Source via Fourier Cross-Vicr Learning
Ground Truth
Baseline
CPerb
FCVI.
Figure 11. Visualized detection results of baseline and FCVL from Snow set.
Baseline
Baseline-FCVI.
Figure 12, Visualized detection results of baseline and FCVL on Argoverse 2.
18


Reviews:
[{"review_id": "669mWuz1RO", "reviewer": "Reviewer_4XDP", "summary": "Aiming to improve the generalization in only single source data available for training, this paper proposed Fourier Cross-View Learning (FCVL) framework. FCVL framework can leverage the Fourier transformation to separate high-level and low-level information within the image. Subsequently, it can make appropriate modifications to this information, so as to achieve the purpose of generalization. \nOverall, this framework outperforms other models compared in the paper, proving its generalization capability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive and constructive feedback!  We have addressed all the comments and incorporated additional experimental results to further validate our approach.\n\n__Q1: In this paper, use the nuScenes dataset as the training set and the NuScenes-C dataset as the testing sets. Why don't use NuScenes-C dataset as the training set to transfer to a simpler dataset? Are there any relevant experiments?__\n\nThe objective of this paper is to generalize models trained on a single domain (e.g., nuScenes) to multiple diverse application scenarios (e.g., NuScenes-C, which includes eight ood test scenarios). Such cross-domain generalization is more challenging and allows to rigorously validate the performance of the proposed algorithm under significant domain shifts.  \n\nBesides, \"use NuScenes-C dataset as the training set to transfer to a simpler dataset\" falls within the research paradigm of generalizing from multiple domains to unknown target domain. We conducted experiments under this paradigm. As shown in the table, our approach can still enhance generalization performance in this setting. \n\n| Model | NDS|\n| - | -|\n| NuScenes-C(train) | NuScenes(test)|\n| BEVDet| 0.1830 |\n| +FCVL| 0.1993(__+1.63\\%__) |\n\n__Q2:__ $d_1$ and $d_2$ denote height and width of the image. Thank you for bringing this detail to our attention and we have revised the manuscript accordingly.\n\n__Q3: The proposed Augmentation is similar to style transfer method with content restriction or diffusion-base method. Why not use the existing model for style transfer, instead of using fixed parameters to change the style of the image.__\n\nCompared with existing methods, FHiAug has advantages in three aspects: superior content integrity preservation, style diversity flexibility, and high efficiency. \n\n__Firstly, FHiAug demonstrates superior content integrity preservation.__ Style transfer techniques replace the original image statistics with those from the target style in pixel domain, which would blur the boundary between style and content, distorting important features.  In the training process, more data will be unintensionally simplified, leading to worse performances. Conversely,  FHiAug operating in the frequency domain decouples style manipulation from content preservation，effectively avoiding the interference between style and content in spatial domain. Both experimental comparisons against other style transfer methods (Table 1 in the paper)  and theoretical analysis validate the superiority of our approach.\n\n__Secondly, FHiAug has more flexibility in expanding style diversity.__ The hyperparameters of FHiAug are not entirely fixed. During each iteration, new style statistics  is randomly sampled from Gaussian distribution (please kindly refer to Equation 6–9 in paper), which __ensures the diversity of styles__ in each iterative process. However, diffusion-based techniques require conditional control, specific training data , or tailored architecture to achieve style generation for autonomous driving scenarios.\n\n__Thirdly, FHiAug exhibits better efficiency and extendability.__  Diffusion-based techniques demand substantial time for generation and  additional storage space to store the data.  If adopting an online generation approach, frequent calls to the generative model during training would significantly increase computational overhead, making them impractical for the training of complex 3D detection models. While, FHiAug is a plug-and-play online augmentation approach, which can be extended to other frameworks  with high flexibility. \n\n__To further validate the advantages of FHiAug__, we add more efficiency and performance analysis of diffusion-based method[1]. Under the same resolution, the generation method takes over ten times longer than FHiAug. Besides, we generate synthetic data following [1]. We incorporate synthetic data into the original dataset and train all the data on BEVDet. As can be seen from the generated images (https://drive.google.com/file/d/1M4gzNi_wVNPHvtWy_l0ddS5qnBqxpFLR/view?usp=sharing), some objects are distorted and the diversity of  synthetic data is quite limited. Compared with diffusion-based method,  FHiAug achieves superior generalization performance (__+4.19\\%__) much more efficiently.\n\n| Model | Resolution | time-consumed(s) | OOD Avg. | \n| -| -| -| - | \n| BEVDet |-|-| 0.2017| \n| FHiAug only| 256 $\\times$ 704 | 0.107 | __0.2579(+4.19\\%)__ | \n| MagicDrive[1] | 256 $\\times$ 704 | 1.5|0.2160|\n\n[1] Gao, Ruiyuan, et al. MagicDrive: Street View Generation with Diverse 3D Geometry Control. ICLR 2024.\n\n__Q4: The overall structure of the paper is confusing. The Relate Work should be introduced in front of the methodology, and the Relate work is not detailed enough.__\n\nWe have further modified the structure of the paper and put the related work section in front of the methodology. We also provide an additional related work section in the Appendix for detailed descriptions and broader coverage of all related work.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "In this paper, use the nuScenes dataset as the training set and the NuScenes-C dataset as the testing sets. Why don't use NuScenes-C dataset as the training set to transfer to a simpler dataset?\nAre there any relevant experiments?\n\nIn addition, Parameters d1 and d2 in Formula 1 are not introduced.", "theoretical_claims": "NA", "experimental_designs_or_analyses": "The proposed Fourier Hierarchical Augmentation is similar to the image style transfer method with content restriction or diffusion-base method.  Why not use the existing model for style transfer, instead of using fixed parameters to change the style of the image.", "relation_to_broader_scientific_literature": "The paper is related to multi-view 3D object detection. The proposed model leverages the Fourier transformation to separate high-level and low-level information within the image, which achieves better generalization ability.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n(1) Sufficient experiments have been made to prove the performance of the method.\n(2) Proposed baselines achieve better performance than comparisons and the experiments.", "other_comments_or_suggestions": "The overall structure of the paper is rather confusing, and the writing skills need to be improved. \nThe Relate Work should be introduced in front of the methodology, and the Relate work is not detailed enough.", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zBBWRCzg8K", "reviewer": "Reviewer_Ac9x", "summary": "The authors propose a novel generalization multi-camera 3D object detection framework using Fourier Cross-View Learning.\nVia the proposed Fourier Hierarchical Augemetatiion and Semantic Consistency Loss across views, this work consistently improves the generalization ability of the previous methods over multiple datasets.\nThe extensive experiments support the authors' claim, and the real-world demonstration shows the robustness of the proposed method for autonomous driving scenes.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are pleased that the reviewer found our paper __novel, interesting and effective__. Thanks very much for your acknowledgment, which is truly encouraging! We have addressed all the comments and further improved the manuscript. We are deeply grateful for your contributions to help elevate the quality of this submission.\n\n__W1:__ To enhance readability, we have reorganized the paper by moving the related work to Sec. 2. Besides, we have added a subsection in Sec. 2 to systematically review existing 2D augmentation techniques.\n\n__W2:The proposed domain generalization augmentation is only applied to perspective images/features. The motivation that the traditional 2D data augmentation won't work on BEV features is weakened by this.__\n\nWe apologize for any confusion. \n\nThe BEV representation is constructed by mapping 2D features from surrounding camera views into 3D space through physics-aware methods such as depth estimation. Although traditional 2D data augmentation are applied to images, when these augmented images are projected to BEV space, the artifacts introduced by augmentation will degrade the quality of BEV features.\n\nFirstly, strong geometric transformations (e.g. large-angle rotations, translations) can no longer be freely applied. Such transformations on 2D images would violate the spatial consistency between adjacent cameras', leading to distortion of the target's position or orientation in BEV space. This would degrade the perception system's reliability. This phenomenon exposes the limitation of common geometric augmentation in 3D perception. Geometric transformations must adhere to physical constraints derived from the multi-view geometry. We have conducted experiments to demonstrate this point. As shown in the table, natively applying strong geometric augmentations will not improve or even hurt the performance.\n\nSecondly, style transfer techniques replace the original image statistics with those from the target style, which causes  the interference between style and content and distorts content features. If the 2D features are impaired, the projected BEV feature will also be affected, ultimately hurting the 3D detection performance. While, our method decouples style manipulation from content preservation, effectively avoiding this limitation.\n\nCompared with these common 2D augmentations, the key advantage of our method is its ability to maximize sample diversity under physical constraints, while maintaining superior content integrity preservation.\n\n| Model | Clean | OOD Avg. |\n| - | -| - |\n| BEVDet  | 0.3880 | 0.2017 |\n| +strong geo | 0.3530 | 0.1749|\n\n__W3: Missing the explanation of the definition of City and Cloudy settings.__\n\nThank you for pointing this out and we have added more details in the manuscript. The Argoverse  contains different driving scenarios across six major U.S. cities (Miami, Washington D.C. and so on), including various weather conditions such as sunny days and cloudy conditions. To adhere to the single domain to multi domain generalization paradigm, we take sunny-day data from Miami as the single-domain training set, while sunny-day data from other cities (with diverse urban road structures) as the first ood test set (City), and cloudy (dim lighting) data from other cities as the second ood test set (Cloudy).\n\n__W4: The ablation studies (Tab. 4) are hard to understand. It should be revised.__\n\nThanks for your suggestion! More ablation studies and analysis are put in the body of the paper. The ablation studies include (1) effects of different components of FCVL and (2) effects of different inserted positions of Amplitude Transfer at feature level (this subsection has been moved from the Appendix D.4 to the main body of paper).\n\nWe have also revised the Table 4 to improve the readability. We have put the new table here ( https://drive.google.com/file/d/1BYMOE_trRM3vPfBfyxjt1dbW75p13Igv/view?usp=sharing ) . We hope this revised table will help readers understand the effect of each module in FCVL better. \n\n__Q:Since the method can improve the detector's generalization ability, how about the performance that trains the detector using the proposed framework on nuScenes/AV2 and test it on AV2/nuScenes?__\n\nThanks for your constructive suggestion, which further enhances our validation framework. As the number of surround-view cameras is different (NuScenes: 6 cameras, AV2: 7 cameras), we keep AV2's 6 cameras (ring_front_center, ring_front_left, ring_front_right, ring_side_left, ring_side_right, ring_rear_left) and align the Argoverse coordinate system to NuScenes. As the labeled categories between these two datasets are quite different, we mainly focus on two common categories (car and pedestrian) with mAP metrics. The results below demonstrate that our method achieves great improvements in cross-dataset generalization.\n\n| Model | mAP|\n| - | - |\n| nuScenes(train) | AV2(test) |\n| BEVDet | 0.1020|\n| +FCVL| 0.1246(__+2.26\\%__) |", "claims_and_evidence": "The claims of this work are Fourier Hierarchical Augmentation (FHiAug) and Fourier Consistency Loss. Among them, FHiAug can be further broken into Frequency Jittering (Amplitude and Phase) and Amplitude Transfer.\n\n- Frequency Jittering (Amplitude & Phase): It is supported by the ablation results (Tab. 4) and visualization (Fig. 7 & 9), which shows it can change the input image appearance and help the overall performance.\n- Amplitude Transfer: It is supported by the ablation results (Tab. 4) and visualization (Fig. 8 & 9), which shows it can further change the input image appearance through extracted image features and keep improving the overall performance.\n- Consistency loss: It is supported by ablation results (Tab. 4) showing its benefit.", "methods_and_evaluation_criteria": "In general, the method is very novel and interesting. FHiAug successfully generates more diverse training samples. With the cross-view consistency loss, the model manages to learn more generalizable features for multi-camera 3D object detection.\n\nFor the benchmarks, nuScenes-C is widely used and can successfully test the generalization ability of the methods. However, it seems the results for Argoverse 2 (AV2), i.e. Table 3, are missing the explanation of the definition of City and Cloudy settings.", "theoretical_claims": "All the proofs and theoretical claims look fine except for the second paragraph in the introduction, which the authors claim, \"however, directly applying these approaches to BEV-based tasks introduces several challenges. First, BEV representations are generated by projecting multi-view 2D features using real-world physical constraints, which limits the use of strong geometric transformations, such as 270-degree rotations, as they would disrupt the spatial consistency of the BEV space.\"\n\nIt seems to me a logical error here. The augmentation, including the proposed FHiAug, is all applied to the \"images,\" not the BEV features. Thus, using the claims here does not convince me why previous methods are not suitable for multi-camera 3D object detection.", "experimental_designs_or_analyses": "The experimental designs are well structured, and the experiments are extensive over five methods across two challenging datasets.\n\nThe analyses seem to be too short in the main paper. The efficiency analysis could be moved to the supplementary section, while the authors should focus more on the ablation studies. Tab. 4 also needs to be structured better so that the readers can easily understand which rows to compare and what the takeaway messages are.", "relation_to_broader_scientific_literature": "I believe this work has a broader scientific impact, given its novel idea of generating diverse training data through FFT (Fig. 2) and the real-world demo (Fig. 5)", "essential_references_not_discussed": "No. This paper doesn't have essential references not discussed.", "other_strengths_and_weaknesses": "Strengths:\n- The idea of generating diverse training samples via FFT is novel and interesting.\n- Cross-view consistency is intuitive and effective.\n\nWeakness:\n- Putting the related work in Sec. 4, which is not a common writing, and also putting the 2D data augmentation in the supplementary while it should be in the related work.\n- The proposed domain generalization augmentation is only applied to perspective images/features. The motivation that the traditional 2D data augmentation won't work on BEV features is weakened by this.\n- The explanation of generating City and Cloudy settings for AV2 is missed.\n- The ablation studies (Tab. 4) are hard to understand. It should be revised.", "other_comments_or_suggestions": "In general, I love the idea and proposed framework. Yet the unusual writing orders and some tables structure make the writing look not professional which should be improved.", "questions_for_authors": "- Since the method can improve the detector's generalization ability, how about the performance that trains the detector using the proposed framework on nuScenes/AV2 and test it on AV2/nuScenes? It will further strengthen the methods for domain generalization also across datasets (scenes and cameras).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yjNrKZjoQc", "reviewer": "Reviewer_xLGx", "summary": "The author proposes the Fourier Cross-View Learning (FCVL) framework including Fourier Hierarchical Augmentation (FHiAug),\nan augmentation strategy in the frequency domain to boost domain diversity, and Fourier Cross-View\nSemantic Consistency Loss to facilitate the model to learn more domain-invariant features from adjacent perspectives. According to the author, this is the first study to explore generalizable multi-camera 3D object detection with a single source.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your acknowledgment of our approach, which is truly encouraging! We have addressed all the comments and incorporated additional experimental results to further validate our approach. We sincerely appreciate your contributions to help elevate the quality of this submission. __All the tables are put here (https://drive.google.com/file/d/19yp9tYUu7XV-R4V69FW-Nzix8lZ8Kg5s/view?usp=sharing).__ Zoom in for better viewing.\n\n__W1: I find the FHiAug method quite novel.__   \n\nWe are deeply grateful for your acknowledgment of the novelty of FHiAug. \n\n__(1) establish the relevance of the proposed method to the multi-camera 3D detection task:__ \n\n__Relevance 1:__ The proposed FCVL framework leverages the cross-view consistency in multi-camera 3D detection input to enhance generalization. By introducing the cross-view consistency loss, the model is enforced to learn domain-invariant features that preserve semantic alignments across camera perspectives. However, its effectiveness is limited under the single-domain setting due to restricted feature diversity. Therefore, we propose FHiAug to alleviate the bias in single-domain representations. As shown in Figure 2 of the paper, FHiAug, on one hand, expands the domain diversity to force the model to learn from different feature distributions. On the other hand, it expands the quantity and diversity of the cross-view sample pairs, enabling the consistency loss to more effectively explore semantic alignments between adjacent perspectives. Ultimately, the FCVL framework achieves generalizable multi-camera 3D object detection with a single source.\n\n__Relevance 2:__ Compared to traditional 2D tasks, acquiring and annotating multi-camera 3D detection datasets is quite expensive. This method achieves remarkable improvements for 3D detection models at a relatively low cost without relying on large-scale annotated data. This solution offers a computationally efficient  approach to address real-world challenges in autonomous driving, underscoring its practical value.\n\n__(2) validate broader applicability:__  Besides, we conduct experiments on the 2D detection task. Similarly, following the paradigm of generalizing from a single domain to multiple domains, we train on the daytime-sunny set and test on the other four domains with different weather conditions at different times. As is shown (please refer to Table 1 in the link), our method  also effectively improves generalization for 2D detection tasks. \n\n__W2:__\n\n__(1)connection to the Fourier space:__ Given that the semantic information is contained in the phase components, our semantic consistency loss is computed by extracting semantic information from the phase information. We evaluate the semantic consistency of samples with  phase components. Please kindly refer to Equation 11–13 in our manuscript.\n\n__(2)relationship between this consistency loss and FHiAug:__ The proposed FCVL framework follows \"domain diversity first, then domain invariance\" paradigm. FHiAug expands the data distribution at both image and feature levels, while semantic consistency regularization enables the model to learn more domain-invariant representation. \n\n__(3)apply the loss to other augmentation methods:__ To validate this, we incorporate the consistency loss with DSU. The results are detailed in Table 2 in the link. This loss, when combined with other augmentation methods, can further enhance the generalization. With the same consistency loss constraints, our FCVL framework still demonstrates much more advantages. \n\n__In conclusion__, the proposed framework accommodates the specificity of multi-camera 3D detection while demonstrating \nextendability to other vision tasks. We leave it for future work to extend FCVL for general vision tasks. \n\n__W3 & W4:__ Thank you for the comments.  The proposed FCVL has achieved SOTA results with average performance improvements of  0.86%-2.47% on eight domains compared to other five methods across two distinct datasets and  four different frameworks . The __consistently significant improvements__ across __multiple experimental setups__ fully demonstrate the strong adaptability of our method. It can seamlessly adapt to diverse scenarios and consistently maintain superior and stable performance. Autonomous driving systems, as safety-critical systems, require consistent performance across diverse scenarios. We further list the results of large vehicles. Due to insufficient training data of large vehicles, detecting large vehicles is more challenging compared to common cars. We can observe consistently significant improvements over large vehicles. In particular, our approach achieves __+10.8\\%__ improvement for trailer, __+8\\%__ for truck, and __+7.1\\%__ for bus. Please  refer to Table 3 in the link.\n\nFurthermore, we list the results of mAP in Table 4 in the link. Our method maintains superior performance across multiple frameworks when leveraging FHiAug only and FCVL still achieves SOTA results with mAP metric.", "claims_and_evidence": "Yes, the author provides extensive experimental results to demonstrate the claims.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes, I have checked the correctness of proofs in the supplementary material.", "experimental_designs_or_analyses": "Yes. For issues about the experiments, please refer to the below part.", "relation_to_broader_scientific_literature": "The author proposes a new problem setting, which can be a good contribution to the literature. However, I keep doubt for the relationship between the author method and the problem setting.", "essential_references_not_discussed": "The author has discussed most of related works.", "other_strengths_and_weaknesses": "Strengths:\nThis paper introduces a new problem setting: generalizable multi-camera 3D object detection with a single source, which I believe is highly important. Moreover, the author's approach of applying augmentation in the frequency domain is quite novel. The experimental results also show performance improvements. Overall, I am inclined to accept this paper.\n\nWeakness:\n1. I find the FHiAug method quite novel. However, I believe it is a relatively general technique for RGB images. In contrast, the authors claim to be working on a new task: generalizable multi-camera 3D object detection. In my view, FHiAug has little to do with multi-camera or 3D detection specifically; rather, it is a more general method applicable to RGB images. Therefore, I do not see a clear connection between the proposed method and the novelty of the task itself. The authors should: (1) establish the relevance of their method to the multi-camera 3D detection task and (2) conduct experiments on tasks like 2D detection to demonstrate its broader applicability.\n2. The Fourier Cross-View Semantic Consistency Loss also does not seem to have a clear connection to the Fourier space; it appears to be a loss function applicable to various augmentation methods. I believe the authors should similarly (1) establish the relationship between this consistency loss and FHiAug and (2) apply the loss to other augmentation methods to validate its effectiveness.\n3. Although the experimental results show some improvements, the gains over the current state-of-the-art methods seem quite limited. For example, when using BEVFormer, the improvement is less than 1%. Given that the main contribution of the paper is FHiAug, I believe the authors should also provide results without the consistency loss, using only FHiAug, and compare them with existing methods to better demonstrate the effectiveness and improvements brought by the proposed approach.\n4. Although NDS is indeed a very important metric on nuScenes, I believe AP remains crucial for the detection task. I hope the authors can also provide AP metrics to specifically evaluate the model's capability in 3D bounding box prediction.", "other_comments_or_suggestions": "No, please refer to the above weakness part.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1R4RONJNIp", "reviewer": "Reviewer_dL4D", "summary": "This paper proposed a Fourier Cross-View Learning (FCVL) framework, which augments the data in the frequency domain and includes a contrastive-style semantic consistency loss to improve the model  generalization ability from a single source.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your  positive and constructive feedback! We have addressed all the comments and incorporated additional experimental results to further validate our approach.\n\n__Q1 and W1:__  In our approach, there is a cross-view instance binding mechanism that the identical instance labels are assigned to cross-view instances of the same object. This ensures their consistent assignment as positive sample pairs in contrastive learning. This effectively prevents the rear part of large vehicles from being misclassified as negative samples. Visualization analysis in the figure (https://drive.google.com/file/d/1X04hoOqohT-O3SmlxuWx843Ffm-LS_8N/view?usp=sharing) demonstrates that the front and rear components of large vehicles have consistent activation responses in feature maps. Besides, we further list the results (mAP) of large vehicles. We can observe consistently significant improvements over large vehicles. In particular, our approach has __10.8\\%__ improvement for trailer.\n\n| Method | truck | trailer| bus|\n| -| -| -| -|\n| BEVDet | 0.128 | 0.038| 0.222 |\n| +FCVL  | __0.208(+8\\%)__ | __0.146(+10.8\\%)__ | __0.293(+7.1\\%)__ |\n\n__Q2:__ Thank you for raising this question. As we mentioned above, in our approach, the identical instance labels are assigned to cross-view instances of the same object. When taking a target object in one view as the anchor, we search for objects with the same instance label in adjacent views as positive samples. This is because these objects inherently represent the same target observed from different angles, exhibiting strong correlations. For negative sample selection, we do not treat objects in the anchor's background as negatives. Instead, we choose samples of different categories from other views. This guarantees that negative samples are categorically distinct from the anchor, enhancing the model's ability to differentiate features across categories. Through this design, we effectively leverage cross-view consistency to improve model performance.\n\n__Q3:__ As highlighted in our discussion of limitations, there are some opportunities for improvement under extreme weather and low light conditions. For example, targets are occluded by fog or inherently difficult-to-detect small distant targets become even more challenging under extreme weather conditions. Some examples are put here. (https://drive.google.com/file/d/1d-hDTCbvTj3SOVffPHnlBlEnJIeShM-6/view?usp=sharing) These limitations can be further improved from two aspects: (1) weather-specific data augmentation combined with a multi-scale strategy and (2) multi-modal fusion integrating LiDAR with cameras against low light. In addition, under adverse weather conditions LiDAR performance may be degraded. To maximize sensor effectiveness across diverse scenarios, an adaptive cross-modal fusion scheme should be designed to achieve dynamic fusion of different modalities.\n\n__Q4:__ We would like to further clarify the advantages of our method over other frequency-domain approaches[1,2]. Compared with these methods, our method integrates __superior performance, high efficiency and extendability__. \n\nFirstly, in the setting of single source data, our proposed method can enhance the generalization ability of the detectors by a large margin. FACT[1] needs to mix up different domains' data in frequency to achieve great OOD performance. But when training with only single domain, FACT can only mix the samples within the single domain and it indeed improves the in-domain clean set's performance a bit, but the improvement  on the OOD sets is very slim. Different from FACT, we first propose Frequency Jitter at the image level to create diverse samples. Then, at the feature level, we introduce a novel method Amplitude Transfer to achieve fine-grained styles without content distortions. Via uncertainty estimation, Amplitude Transfer can obtain diverse feature statistics, which can gradually shift the features to more diverse domains through continuous training. \n\nSecondly, due to the high complexity of BEV-based 3D object detection models, our plug-and-play data augmentation method can achieve better generalization results more efficiently. AGFA[2] trains the classifier and the amplitude generator adversarially to synthesize the worst-case domain for adaptation. Compared with this method, our proposed method is more stable and effective without introducing sophisticated extra modules or special training recipes for stable performances. This also increases the extendability of our method to other frameworks. \n\nIn summary, the proposed method balances both performance and efficiency, and addresses real-world challenges in autonomous driving, underscoring its practical value.\n\n[1]. Xu, Qinwei, et al. A fourier-based framework for domain generalization. CVPR, 2021. \n\n[2] Kim, Minyoung, et al. Domain generalisation via domain adaptation: An adversarial fourier amplitude approach. 2023.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "The proof looks good, but I didn't check it carefully.", "experimental_designs_or_analyses": "Yes. Didn't find an issue.", "relation_to_broader_scientific_literature": "This method is applicaple for different kinds of detection network.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Pros:\n1. This paper augments the data in frequency domain by jitterring both amplitude and phase, which diversify the dataset. The phase term typically captures high-frequency features that may be more transferable.\n2. The designed contrastive loss, which utilizes the adjacent image regions, is also a good idea.\n3. The proposed method is adaptable to different approaches and achieved SOTA results with various baseline models.\n4. The t-SNE visualization and other visualization results validate that the learned features are domain-invariant and the images become more diverse.\n\nCons:\n1. There may be some false negative samples for the contrastive loss design since some large vehicles may distribute across several adjacent frames.", "other_comments_or_suggestions": "Please check my questions.", "questions_for_authors": "1. Regarding the semantic consistency loss, it appears to work well for small vehicles. But what happens when the vehicle is large? Will the rear part of a large vehicle be treated as a negative sample of the front part?\n\n2. Besides large vehicles, what if there are other vehicles in the background image, will they also be regarded as negative samples?\n\n3. Could you also show some examples where the proposed model still cannot detect correctly? How to further improve the model the future?\n\n4. The idea of augmenting data in the frequency domain is not new and have been tried by previous researchers. What are advantages of the proposed frequency domain augmentation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work proposes a strategy to boost domain diversity in the frequency domain, using Fourier Hierarchical Augmentation (FHiAug) and Fourier Consistency Loss for generalization of multi-camera 3D object detection trained on a single source domain to multiple domains. It received final scores of weak accept, accept, accept and weak accept from four reviewers. The reviewers appreciated the novelty of the proposed approach, its robust performance in improving cross-domain generalization under different settings and the comprehensive experiments conducted. The reviewers' concerns were addressed by the authors' responses. The AC concurs with the reviewers' consensus and recommends acceptance. The authors are encouraged to make the changes that they have promised in the rebuttal in the final camera ready version of the paper and its supplement.
Decision: Accept (poster)