MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection
Authors: Zhuoxiao Chen
Junjie Meng
Mahsa Baktashmotlagh
Yonggang Zhang
Zi Huang
Yadan Luo
Venue: ICLR 2025 Oral (2025)
Keywords: Test-Time Adaptation
3D Object Detection
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: LiDAR-based 3D object detection is crucial for various applications but often experiences performance degradation in real-world deployments due to domain shifts. While most studies focus on cross-dataset shifts, such as changes in environments and object geometries, practical corruptions from sensor variations and weather conditions remain underexplored. In this work, we propose a novel online test-time adaptation framework for 3D detectors that effectively tackles these shifts, including a challenging $\textit{cross-corruption}$ scenario where cross-dataset shifts and corruptions co-occur. By leveraging long-term knowledge from previous test batches, our approach mitigates catastrophic forgetting and adapts effectively to diverse shifts. Specifically, we propose a Model Synergy (MOS) strategy that dynamically selects historical checkpoints with diverse knowledge and assembles them to best accommodate the current test batch. This assembly is directed by our proposed Synergy Weights (SW), which perform a weighted averaging of the selected checkpoints, minimizing redundancy in the composite model. The SWs are computed by evaluating the similarity of predicted bounding boxes on the test data and the independence of features between checkpoint pairs in the model bank. To maintain an efficient and informative model bank, we discard checkpoints with the lowest average SW scores, replacing them with newly updated models. Our method was rigorously tested against existing test-time adaptation strategies across three datasets and eight types of corruptions, demonstrating superior adaptability to dynamic scenes and conditions. Notably, it achieved a 67.3% improvement in a challenging cross-corruption scenario, offering a more comprehensive benchmark for adaptation. Source code: https://github.com/zhuoxiao-chen/MOS.

Published as a conference paper at ICLR 2025
MOS: MODEL SYNERGY FOR TEST-TIME ADAPTATION ON HIDARBBASER 3D OBJECT DETECTION
Zhuoxiao Chen'
Junjie Mengi
Mahsa Baktashmotlagh"
Yonggang Zhangl
Zi Huang
Yadan Luoi
The University of Queensland
Hong Kong Baptist University
ABSTRACT
LiDAR-based 3D object detection is crucial for various applications but often experiences performance degradation in real-world deployments due to domain shifts. While most studies focus on cross-dataset shifts, such as changes in en- vironments and object geometries, practical corruptions from sensor variations and weather conditions remain underexplored. In this work, we propose a novel online test-time adaptation framework for 3D detectors that effectively tackles these shifts, including a challenging cross-corruption scenario where cross-dataset shifts and corruptions co-occur. By leveraging long-term knowledge from previous test batches, our approach mitigates catastrophic forgetting and adapts effectively diverse shifts. Specifically, we propose a Model Synergy (MOS) strategy that dynamically selects historical checkpoints with diverse knowledge and assembles them to best accommodate the current test batch. This assembly is directed by our proposed Synergy Weights (SW), which perform a weighted averaging of the selected checkpoints, minimizing redundancy the composite model. The SWs are computed by evaluating the similarity of predicted bounding boxes on the test data and the independence of features between checkpoint pairs in the model bank. To maintain efficient and informative model bank, we discard checkpoints with the lowest average SW scores, replacing them with newly updated models. Our method was rigorously tested against existing test-time adaptation strategies across three datasets and eight types of corruptions, demonstrating superior adaptability dynamic scenes and conditions. Notably, it achieved 67.3% improvement in challenging cross-corruption scenario, offering a more comprehensive benchmark for adaptation. Source code: http://ggihbb..om/muuoxxooccco//
INTRODUCTION
Recently, LiDAR-based 3D object detectors have exhibited remarkable performance alongside diverse array of applications such as robotic systems (Ahmed et al.
2018
Montes et al.
2020 Zhou
et al.
2022 Ye & Qian
2018) and self-driving cars (Deng al.
2021 Wang et al.
2020a Luo
et al.
2023b Qian et al. 2022 Wang et al. 2019 Chen et al. 2023b
|
Arnold et al. 2019: You et al.
2020
Meyer et al. 2019 Li al. 2019 McCrae & Zakhor 2020 Luo et al. 2023a Zhang et al.
2024).
However, when deployed in the real world, the detection system could fail on unseen test
data due to unexpectable varying conditions, which is commonly referred to as the domain shift.
To study this shift, researchers consider cross-dataset scenarios (e.g., from nuScenes (Caesar et al.) 2020) to KITTI (Geiger et al. 2012)) where object sizes and beam numbers are varying in the test domain (Ganin et al. 2016 Luo et al. 2021: Wei et al.] (2022). or adding perturbations to clean datasets (e.g., from KITTI to KITTI-C (Kong et al.) (023)) to mimic realistic corruptions caused by severe weather condition and sensor malfunctions (Hahner et al. 2022 Kong et al. [023] Dong ct al. 2023). However, in real-world scenarios, shifts generally do not arise from a unitary source, for example, considering detection system deployed in an unfamiliar northern city, it is likely to suffer severe weather conditions, such as heavy snow. This leads us to consider a new hybrid shift termed cross-corruption (illustrated in Fig. D. where cross-dataset gaps and corruption noises coexist across 3D scenes (e.g., Waymo (Sun et al. 2020) to KITTI-C).
"Correspondence Yadan Luo <y.luo@uq.edu..a

---

Published as a conference paper at ICLR 2025
Test-Time Steps
Waymo
KITTI
Waymo
1. Cross-Dataset
2. Corruption Shifts
3. Cross-corruption Shifts (hybrd object environmental shifts)
[object sizes, beam numbers]
Hogay, blur, cross-sensor...)
t=1
KITTI
KITTI-C
KITTI-C
Figure We investigate three distinct types of domain shifts that 3D detectors face during test time. Unlike previous work that focuses solely on cross-dataset shifts, our study comprchensively explores shifts arising from varying weather conditions, sensor corruptions, and the most challenging cross-corruption shifts, which encompass both object- and environment-rellsn relltte variations.
These simulated shifts prompt a critical research question: how can the 3D detector naturally adapt to a shifted target scene? Previous studies have identified unsupervised domain adapta- tion (UDA) as promising solution, by multi-round self self-training on the target data with the aid of pseudo-labeling techniques (Chen et al., 2023a Yang et al.) 2021 2022; Peng et al.) 2023 Li et al., 2023a), or forcing feature-level alignment between the source and target data for learning invariann representations {Luo et al.) 2021; Chen et al. 202I; Zhang et al.) 2021 Luo ct al. 2023c; Zeng et al. 2021). While effective. these UDA-based approaches require of- fline training process over multiple epochs with pre-collected target samples. Considering a de- tection system being deployed on resource-constrainee device in the wild, performing such time-consuming adaptation during testing is impractical. To this end, there is an urgent need for strategy that allows models to adapt to live data streams and provide instant predictions. Test-time Adaptation (TTA) emerges as a viable solution and has been explored in the general classification task, through (1) choosing small
of network parameters to update (e.g., matching
BN statistics (Wang et al. 2021 Niu et al.) (023)) ) or (2) employing the mean-teacher model to
provide consistent supervision (Wang al. 2022a 20236 Yuan et al. 2023 Tomar et 2023a
A recent work of MemCLR (VS et al. 2023). extends
the idea of TTA to image-based 2D object detection
Tent SAR
50
by extracting and refining the region of interest (Rol) features for each detected object through transformer- based memory module.
MemCLR CoTTA
40
30-
Ours
Despite TTA's advances in image classification and 2D detection, adapting 3D detection models test time remains unexplored. This motivates us first to conduct pilot study and investigate the effectiveness of these TTA techniques when applied to 3D detection tasks. As plotted in Fig. 2 methods that employ the mean-teacher model (e.g., MemCLR, and CoTTA) outperform the rest of the methods. This superiority stems from the teacher model's ability to accumulate long-temn knowledge, offering stable supervision signals and reducing catastrophic forgetting across batches
UDA
No Adapt.
Figure 2: The results (APap) of applying existing TTA methods to adapt SECOND (Yan al. 2018) from nuScenes to KITTI. Mean-teacher models are in green.
(Kirkpatrick al., 2017 Haaes al. 2020 Hu al. 2019: Lee al., 2017 Kemker et al. 2018). However, by treating all previous checkpoints equally through moving average (EMA) for each test sample, the teacher model may fail to recap and leverage the most critical knowledge held different checkpoints. For example, Fig. 3 shows very different characteristics of test point clouds 2++1 and It-2, while the teacher model still relies on a uniform set of knowledge (i.e., generated by EMA), without leveraging relevant insights from prior checkpoints tailored to each specific scene. Hence, no teacher model perfectly fits all target samples. We argue that an optimal "super" model should be dynamically assembled accommodate each test batch.
2

---

Published as a conference paper at ICLR 2025
In this paper, we present Model Synergy (MOS) strategy that adeptly selects and assembles the most suitable prior checkpoints into a unified super model, leveraging long-term information to guide the supervision for the current batch of test data. To this end, we introduce the concept of synergy weights (SW) to facilitate the model assembly through a weighted averaging of previously trained checkpoints. A model synergy bank is established to retain K previously adapted checkpoints. As illustrated in Fig. 3 to assemble a super model tailored to test batch Tt+1. checkpoints 1 and 3 are assigned lower SW due to redundant box predictions. To leverage the knowledge uniquely held by checkpoint 4 (e.g., concepts about cyclists and pedestrians), MOS assigns it with a higher SW.
To determine the optimal SW for the ensemble, we reformulate this as solving linear equation. intending to create a super model that is both diverse and exhibits an equitable level of similarity to all checkpoints, without showing bias towards any. The closed-form solution of this linear equation leads to calculating the inverse Gram matrix, which quantifies the similarity between checkpoint pairs. To this end, we devise two similarity functions tailored for 3D detection models from 1) output-leve, measuring prediction discrepancies through Hungarian matching cost of 3D boxes, and 2) feature-level, assessing feature independence via calculating the matrix rank concatenated feature maps. By computing these similarities for each new test batch, we determine the SW for the construction of the super model. This super model then generates pseudo labels, which are employed to train the model at the current timestamp for single iteration.
However, with more saved checkpoints during test-time adaptation, the memory cost significantly increases. To address this, introduce dynamic model update strategy that adds new checkpoints, while simultancously removing the least important ones. Empirical results evidence that our approach, which maintains only checkpoints in the model bank, outperforms the assembly of the 20 latest checkpoints and reduces memory consumption by 85%. The contributions of this paper are:
1. This an early attempt to explore the test-time adaptation for LiDAR- Rabedd 3D object detection (TTA-3OD). In addressing the challenges inherent to TTA-3OD, we propose a novel Model Synergy (MOS) approach to dynamically leverage and integrate collective knowledge from historical checkpoints.
2. Unlike mean-teacher based methods that aggregate all previous checkpoints, we identify and assemble the most suitable ones according to each test batch. To this end, we utilize the inverse of the generalized Gram matrix to determine the weights. We introduce similarity measurements for 3D detection models at both feature and output levels to calculate the generalized Gram matrix.
3. We conduct comprehensive experiments to address real-world shifts from (1) cross-dataset, (2) corrupted datasets, and (3) hybrid cross-corruption scenarios including eight types of simulated corruptions. Extensive experiments show that the proposed MOS even outperforms the UDA method, which requires multi-round training, when adapting from Waymo to KITTI. In a more challenging scenario of cross-corrupton, MOS surpasses the baseline and direct inference by 67.3% and 161.5% respectively.
RELATED WORK
Domain Adaption for 3D Object Detection aims transfer knowledge of 3D detectors, from
labeled source point clouds to an unlabeled target domain, by mitigating the domain shift across
different 3D scenes. The shift in 3D detection is attributed variations in many factors such as object
statistics (Wang et al. 2020b Tsai et al. (023b), weather conditions (Xu et al.
2021 Hahner et al.
2022). sensor differences Rist et al. 2019 Gu et al. [22TT] Wei et al. (2022), sensor failures (Kong
et al. 2023), and the synthesis-
gap (Saleh et | DeBottol et al.,
2021 Lehner et al.
2022). b. To bridge these gaps, approaches including adversarial matching of features (Zhang et al.
2021), the generation of enhanced 3D pseudo labels (Yang et al. 2021 2022 Chen et al. 2023a
Saltori et al. 2020 Wang et al. 2022b) You et al. 2022 Li et al. 2023b Wang al.
[023c] Peng
al.) 2023 Huang et al. 2024 Tsai et al. 2023a)), the mean-teacher model (Luo al. 202 Hegde
ict al., 2023) for stable adaptation, and contrastive learning ZZng et al., 2021 Lim et al.) (202)) for
tighter embeddings have been extensively investigated. However, implementing these cross-domain methods necessitates multiple epochs of retraining, rendering them impractical for scenarios where test data arrives in streaming manner.
Test-Time Adaptation (TTA) seeks to adapt the model to unseen data at inference time, by addressing domain shifts between training and test data (Wang al. 2024h Chen al. 2024). A very first.
3

---

Published as a conference paper at ICLR 2025
Target Data Batch
Model Bank F
EXPT
EXPT
CXPT
CKPT
x
fo
W(e+11)xf1
W(t+12))f22
W(t+1,3) +
W((+14)
fi+1
(0.05)
(0.35)
(0.10)
(0.50)
xt+2
W (ff
xf2
+
W(E+2,)
+
W ((eza))X44 fei
(0:25)
(0.05)
(0.30)
(0.40)
Figure 3: Illustration of the model synergy (MOS) that selects key checkpoints and assembles them into a super model l²=1 that tailors for each of test data Zt+1- "cyc." denotes the cyclist and "ped." denotes the pedestrian. MOS prioritizes checkpoints with unique insights that are absent other checkpoints with higher weights while reducing the weights of those with redundant knowledge.
work, Tent (Wang et al. 202I)) leverages entropy minimization to adjust batch norm parameters.
Subsequently, many works have adopted similar approaches of updating small of parameters, each with different strategies and focus (Niu et al. 2022] Mirza et al., 2022] Gong et al.] [022] Yuan
et al. 2023] Zhao al. 2023 Niloy al. 2024 Gong al. [2024] Wang al. [024a]] Niu al.
2023
Hong et al.
2023
Song et al. 2023 Nguyen et al. 20233] Niloy et al.) 2024). For instance,
EATA (Niu al. 2022 identifies reliable non-redundant samples optimize while DUA (Mirza
it al. 2022) introduces adaptive momentum in new normalization layer. NOTE (Gong et al., 2022)
updates batch norms at the instance level, whereas RoTTA (Yuan al. 2023) and DELTA (Zhao al.
2023) leverage global statistics for batch norm updates. Additionally lly, SOTTA (Gong et al. 2024)
and SAR (Niu al. 2023) enhance batch norm optimization through sharpness- aware minimization. Alternatively, some approaches optimize the entire network through the mean-teacher framework for stable supervision (Wang al. 2022a Tomar et al.) 2023b), generating reliable pseudo labels for self-training (Goyal et al. 2022 Zeng al. 2024). employing feature clustering (Chen al. 2022 Jung et al. 2023] Wang et al. 2023b), and utilizing augmentations to enhance model robustness (Zhang et al. 2022). Despite these TTA methods being developed for general image classification, MemCL. (VS et al. 2023) is the pioneering work applying TTA for image-based 2D) object detection, utilizing mean-teacher approach for aligning instance-level features. Nevertheless, the applicability of these image-based TTA methods to object detection from 3D point clouds remains unexplored.
3 PROPOSED APPROACH
Overall Framework. First, we establish the definition and notations for the Test-Time Adaptation for 3D Object Detection (TTA-3OD). When deploying a 3D detection model pretrained on source dataset, the test point clouds (x, ]== shifted or/and corrupted due to varying real- world conditions, where x4 is the t-th test point cloud the stream of test data. The ultimate goal of TTA-3OD is to adapt the 3D detection model sequence of target scenes in single pass during the inference time.
The core idea of our methodology is to identify the most suitable histori- cal models with distinct knowledge for each test batch Ft, and assemble them into super model ft facilitat- ing the model adaptation during the test time. To this end, we establish a model bank F = (fi...., fK of size K to preserve and synergize the previously trained models. The over- all workflow of our method unfolds in three phases, as shown in Fig. and Algorithm (in Appendix).
Warm-Up
Bank Update
20
fÆ+
Self-Train
Save
Remova
of Min ①
FK
fx
(2) Model Synergy
Average L
Ensembie
Allerntee
ØK+1
K+
W
f
fK+1
Infer
F
Find
Lxk
Self-Train
BK+L
fi
fx41
Figure 4: The overall workflow of our approach.
4

---

Published as a conference paper at ICLR 2025
Phase 1: Warm-up Initially, when the model bank F is not yet populated, train the model with the first K arriving target batches by pscudo labeling-basee -aaing self self-training. We save the model as a checkpoint each timestamp, i.f..... .fk }. into the F until reaching its capacity of K.
Phase 2: Model Synergy Once F is full, for every subsequent test batch 2,, checkpoints within F are assembled into a super model f to provide pseudo labels BA for supervision. For instance, as illustrated in Phase 2 of Fig. when inferring I we assemble the super model via F into fi++1 "(K+ + wCK + Then, pseudo labels B B*+1 are generated by f to train the current model fK+1 for a single iteration. When L batches are tested, we update the model bank.
Phase 3: Model Bank Update To maintain a compact model bank F, meanwhile, each model holds distinct knowledge, we dynamically update the bank fix its size at K. For every batches tested, we replace the checkpoint with the most redundant knowledge in F, with the newly trained fz. Phases 2 and 3 alternate until all target samples {x have been tested.
3.1 MODEL SYNERGY (PHASE 2)
As Phase 1 is straightforward, where the model bank is filled up with the first K checkpoints, elaborate on the details of Phase 2 in this subsection. An intuitive example is presented in Fig. B when inferring target point cloud, different historical models yield inconsistent predictions because these models learned characteristics from different batches at earlier timestamps. However, if these characteristies no longer appear in subsequent arriving test data, the model may suffer catastrophic forgetting, losing previously learned information. Our strategy to overcome this is to leverage and combine the long-term knowledge from prior checkpoints. But, not all checkpoints are equally important, as shown in Fig. 3 checkpoints 1 and 2 produce highly similar box predictions, whereas checkpoint 4 can detect objects that are missed by others. Treating these checkpoints equally leads to the accumulation of redundant information. Therefore, we propose synergy weights (SW) to emphasize unique insights and minimize knowledge redundancy, via weighted averaging of the historical checkpoints. Ideally, similar checkpoints (e.g., CKPT and 2 in Fig. 3) should be assigned with lower SW, whereas the checkpoint exhibits unique insights (e.g., CKPT 4 in Fig. 33 should be rewarded with higher SW. Through weighting each checkpoint, we linearly assemble them into a super model ft' that best fits the current test batch Ty. In the following section, we discuss our approach to determine these optimal SW for the super model ensemble.
3.1.1 SUPER MODEL ENSEMBLE BY OPTIMAL SYNERGY WEIGHTS
For each batch Tt, we assume there exist optimal synergy weights w E Eº for linearly assembling the historical checkpoints within the model bank F into super model ft as below:
Fw fe-
(1)
For simplicity, we use the same notation fo indicate the model parameters. The optimal synergy weights w can be deduced by solving the linear equation:
==FFFF)-1
(2)
which can be decomposed into two parts, (1) FF inverse of the Gram matrix, and (2) F fe: pairwise similarities between each model in F and the super model f Ideally, these similarities are expected to be uniformly distributed, ensuring f² remains unbiased towards any single model in F
for a natural fusion of diverse, long-term knowledge. Thus, we can simplify the Eq. 2 as:
w= (F F)
(3)
Now. to compute the optimal synergy weights w, delve into the first part of Eq. to compute the Gram matrix G = F F which captures the model parameter similarity for any arbitrary pair of models (fi, fz) in the bank F, as:
K
G G=F
RKKK fi.fs F.
(4)
í,j=1
where (ft denotes the inner product of the model parameters of fi and fz- Based the similarities captured in G, its inverse, G-1, essentially prioritizes directions of higher variance (diverse information) and penalizes directions of lower variance (duplicated information). Thus, the synergy weights calculated by knowledge duplicates. Previous studies (Dinu et al., 2023 Yuen 2024, Nejjar et al. 2003) have also demonstrated the effectiveness of using the - to determine importance weights.
G 11 are theoretically low for those checkpoints with
5

---

Published as a conference paper at ICLR 2025
3.1.2 INVERSE OF GENERALIZED GRAM MATRIX FOR 3D DETECTION MODEL
While the inverse of the Gram matrix is effective, evaluating similarity through the inner product of model parameters fi and f, is suboptimal, as models with distinct parameters may produce similar predictions on the same input test batch zy.- Hence, tailored to each input batch di., we compare intermediate features and final box predictions output by a pair of the models [2) To this end, we propose feature-level similarity function (:)((:;)) and an output-level box set similarity function (tex(3-)) to effectively quantify the discrepancy between any two 3D detection models (will be discussed in next section). By substituting the inner product with the proposed similarity functions, the original Gram matrix G evolves into generalized version G, specially designed for 3D detectors. 3) and Eq. (4). we can caleulate the G and the optimal synergy weights w as:
By combing Eq.
K
w = ( =FFFF) G- " G=
Sbbux (fi, x Steal Stea (fr
(5)
i.j=1
where each element represents the feature and prediction similarity between each pair of historical 3D detection models (z) in F. Next, the super model ft is linearly aggregated by all historical checkpoints F E RR weighted by w E I as follows:
fr Σ wifn w, E EWW
(6)
where the synergized super model fr is used to guide supervision of the current model fe. As illustrated in Fig. 4 for the test sample time f, assembled ft generates the stable pscudo labels B to train fi-1- + Je with a single iteration, as below:
B - ft (xt), fo train (z,,
where ..) indicates data augmentation applied the pscudo-labeled target point clouds. Recall Eq. (5). the key to finding the optimal synergy weights is to accurately measure the similarities of the features and outputs between a pair of 3D detection models. For each test batch 2t, utilize the intermediate feature map 2, and the final box prediction set B' output by modern 3D detectors 2021 Lang al. 2019 Qian et al., 2022 Mao et al. (023) for similarity measurement. We introduce a feature-level similarity function Siem (5.)) to assess feature independence between intermediate feature maps (z,, 2,), and an output-level similarity function Sour(::)) to gauge the discrepancy in box predictions (B', B'), where (2,, 2z) and (B', (B'B are generated by any pair of models (fis fz) within the model bank F.
(Yin et al.
3.3. FRATURE-LEVEL SIMILARITY SFEAT
To assess the similarity between feature maps (2, zz), utilize the rank of the feature matrix to determine linear independencies among the feature vectors in z, relative to those in Zj. Each single feature vector in the feature map, encodes the information of a small receptive field within the 3D scene. When inferring the input batch rı, if a feature vector in z; is linearly independent from the other in zj. it means z; and zj are either capturing information from distinct receptive fields, or focusing the same region but interpreting differently. To calculate such independencies, we concatenate z; @ 2, and calculate its r-n which identifies the maximum number of linearly independent feature vectors. A higher rank of z, @ z, signifies greater diversity of features, because fewer feature vectors can be linearly combined by others. To accelerate the computation, approximate the rank(.) by computing the nuclear norm of the feature matrix (Recht et al. 2010; Kang et al. 2015). With the input batch 2;, the feature-level similarity Steat between a pair of detection models fe,, fy), ) can be determined as follows:
Steat (s) = rank (z; @ 23))
2;, 23 E 22
(8)
D
where 2;, Zj are generated by fi and fj. respectively. H. W. D denote the height, width, dimension of the feature map, and is the maximum possible value of r.nk(.). Steat falls within the range from 0 to 1, and higher value of Steat indicates less feature independence between z, and zy, resulting in more similar feature maps yiclded by models fi and fy- Appendix A.I.I provides empirical evidence and in-depth analysis of the aak-based similarity metric.

---

Published as a conference paper at ICLR 2025
3.1.4 OUTPUT-LEVEL Box SIMILARITY SBox
To assess the similarity between box predictions (B', B²), we treat it as a set matching problem, since B' and B' represent sets of predicted bounding boxes that can vary size. To solve it, we apply the Hungarian matching algorithm (Carion et al. 2020; Stewart et al. (016), robust method for bipartite matching that ensures the best possible one-to-one correspondence between two sets of box predictions. We extend the smaller box set with @ to equalize the sizes of both sets to N. To determine an optimal bipartite matching between these two equal-sized sets, the Hungarian algorithm is used to find a permutation of N elements: p E P. which yields the lowest cost:
p = arg min pEP
Σ Lbox B B p(n)):
where B; is n-th box in the set B' predicted by the model fi. The box loss Lrox (+3-) is calculated by the ontersction-over--uiinn (IoU) plus the distance covering the central coordinates, dimensions, and orientation angles between pair of boxes: B, and its corresponding box indexed by p(n). The indicator function & [cc #0) means the cost calculated only when the category of n-th object is not
the padded The next step is to compute the total Hungarian loss (Carion et al. 2020) for all pairs
of matched boxes:
box (fr
1 (cl, #U) Lovx (B;; B (())))
(10)
1==1
where ğ is the optimal assignment computed in the Eq. ( We normalize the result to the same range as Steat using sigmoid function. A higher value of Sbox indicates that models fr and fa predict more overlapping boxes.
3.2 MODEL BANK UPDATE (PHASE 3)
In TTA-3OD, with the increasing number of test batches, simply inserting checkpoints at all times- tamps to the model bank F results in substantial memory cost and knowledge redundancy. As observed from our empirical results (Fig. [6. assembling the latest 20 checkpoints yields similar performance assembling only the latest 5 checkpoints. Hence, our goal to make the F retain only a minimal yet highly diverse set of checkpoints. To this end, we regularly update the model bank, by adding new models and removing redundant ones every time L online batches are inferred.
As illustrated in Fig. I upon adapting every L samples {æ1, Z1+1, I ). single checkpoint fi E F engages in the assembly L times, thereby I. synergy weights are calculated and stored into a list {w(.a): ((()),)) ,W(,,))) Given K checkpoints in F, synergy matrix is defined as L.K
W (wrL,))(= E _R_ After inferring L batches, we average across the L dimension to calculate the mean synergy weight for each checkpoint, denoted as:
[Wi,.. .... Kk} ) E IR
(11)
Finally, we remove the model with the lowest mean synergy weight in w and add the current model ft to the bank as follows:
F-F (ft), i= wwneeww
(ff)
(12)
where denotes the index of the minimum mean synergy weight in w. By updating the model bank this way, we maintain a fixed number of K checkpoints in F, each carrying unique insights.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets and -TA-30D Tasks. We perform extensive experiments across three widely used LiDAR-
based 3D object detection datasets: KITTI (Geiger et al., 2012). Waymo (Sun et al., 2020), and nuScenes (Caesar et al.) (020)). along with recently introduced dataset simulating rorl-world
corruptions, KITTI-C (Kong et al. 2023) for TTA-3OD challenges. We firstly follow (Yang et al.)
2021 2022 Chen et al. 2023a) to tackle cross-dataset test-time adaptation tasks (e.g. Waymo KITTI and nuScenes > KITTI-C), which include adaptation (i) across object shifts (e.g., scale and point density), and (ii) across environmental shifts (e.g., deployment locations and LiDAR beams). We also conduct experiments to tackle a wide array of real-world corruptions (e.g., KITTI KITTI-C) covering: Fog, Wet Conditions (Wet.), Snow, Motion blur (Moti.), Missing beams (Beam.),
7

---

Published as a conference paper at ICLR 2025
Table 1: Results of test-time adaptation for 3D object detection across different datasets. We report BEV / AD in moderate difficulty. Oracle means fully supervised training on the target dataset. We indicate the best adaptation result by bold.
Waymo -KTTTI
nuScenes KITTI
Method
Venue
TTA
APREy / AP
Closed Gap
APREy / AP
Closed Gap
No Adapt.
67.64 72.48
31.877 17.92
SN
CVPR'20 20
X
78.96 / 59.20
+72.33% 469.00%
40.03 /21.23
+37.55% / +5.66% +76.63% +66.21%%
ST3D
CVPR'21
82.19 61.83
492.97% 774.72%
54.94
Orucle
83.29 73.45
83.29 73,45
Tent
ICLR'21
65.09 73.
-16.29% 43,74%
46,907 18.83
-15.71% 11.71%//11.41 -1.64%%
CoTTA
CVPR'22
67.46/ 35.34
-1.15% / +17.10%
68.81 /47.61
+53.96% / +53.47%
SAR
ICLR'23
/5.81/30.39
-11.69% / 4.333%
61.34/ 35.74
+30.21%%
+32.09%
McmCLR
WACV'23
65.61 29.
-12.97% / 45..1%%
61.47 / 35.76
+330,62%
4-32. 13%
MOS
81.90 / 64.16
+91.12%
79.79%
71.13/51.1
+61.33% +55.78%
Table 2: Results of test-time adaptation for 3D object detection across different corruptions (KITTI KITTI-C) at heavy level. AP easy/moderate/hard difficulty of KITTI metric are reported.
No Adiptation
Tent
CoTTA
SAR
McmCLR
MOS
Fog
84.66/72.8/688.2 23
85.11/73.02/68.7 73
85 94/72.99//8.49
84,64/ 72.52/68.1 14
84.71/72.
85.22/74.02/99.11
Wet.
04.23378.82776.6 25
88.13/78.79/76.6 36
88. 6777666076.22
88,04/78 68/76.25
0p.32/81.91/77.77
Snow
F1.11/63922/99/
11.71/64.48//99.5
1L.3666.59/9945
72.29164.08/58 78
12.33/63.8//58.777 74
75.07767..87/6..11
Moti.
43.35/39, 23/38.23//.22
43.02/39. 15/38. 15
44,48/39. 68339968888866
42.89/39. 13/38.1133811111 12
12.86/38.44/77.1
01.70/41.63/40.23
Beam.
75.42/57.49/53.95
76.66//8801/55300
76.39/57.50/55 5505535 98
75.98/57.6778 75
75. 00/00/774///5...
78.74/60.4/555/
CrossT:
87.98/78. 4677.49
(7.62/77-92/744
86.78/76.0577 05/70.0//222
07.74477.55/7444
86 26.20/77.110 11/74.25
10.22279.10/755:47
Inc.
46.98/30.33/25.0 68
47,82/30.87/26.4 44
49 01.92/32.77/7..88 85
47.27/30.46/26.4
49, 00.99/32.08/27..4
59.1340.29/34.3
CrossS,
68.44/4,37/44000
69.89/46.75/41117
10.15477.1/44.0
1/46//4/////
11.17777728/40.99
71.87/99.28/43.88
Mean
70.77/58.56/5.74
71.11/58.62/54.66
71.49/58.71/54
70.89/58.31/54.77
71.13/8844//4461
74.16/61.82/57
Crosstalk (Cross.T), Incomplete echoes (Inc.), and Cross-sensor (Cross.S). Finally, we address the hybrid shift cross- cors-coorussions (e.g., Waymo - KITTI-C), where cross-dataset inconsistency and corruptions coexist in test scenes. The implementation details are included in Appendix(A.44
Baseline Methods. We compare the proposed method using voxel-based backbone (e.g., SECOND) with a diverse range of baseline methods: (i) No Adapt. refers the model pretrained from the source domain, directly infer the target data; (ii) SN (Wang et al. (020b) is weakly supervised domain adaptation method for 3D detection by rescaling source object sizes with target statisties for training; (iii) ST3D (Yang et al. 2222]) is the pioneering unsupervised domain adaption method for 3D detection with multi-epoch pseudo labeling- based self-training (iv) Tent (Wang et al. 2021 is an online TTA method which optimizes the model by minimizing entropy of its prediction; (v) CoTTA (Wang et al. 2022a) is an online TTA approach which employs mean- beaah-eechhrr to provide supervision with augmentations and restores neurons to preserve knowledge; (vi) SAR (Niu et al.) 2023) improves Tent by sharpness-aware and reliable entropy minimization for online TTA; (vili) MemCLR (VS et al. 2023) is the first work for online adaptation on image-based object detection, by utilizing mecan-teacher to align the instance-level features with memory module; (ix) Oracle means a fully supervised model trained on the larget domain.
4.2 MAIN RESULTS AND ANALYSIS
We present and analyze the main results of car detection using SECOND Yann et al., 2018) as the backbone 3D detector, when comparing the proposed MOS against baseline methods across three types of domain shifts. The results of multiple object classes are detailed in the Appendix A.5 and the examination of alternative backbone detectors (e.g.. PVRCNN (Shi et al. 2020) and DSVT (Wang et al. 2023a]), is presented in the Appendix A.7 Comparisons with linear and non-linear ensemble
methods are included in Appendix [..144
Cross-dataset Shifts. We conducted comprehensive experiments on two cross-dataset 3D adaptation tasks, as reported in Tab. [ when compared to the best TTA baseline CoTTA (Wang et al. (022a)) the proposed MOS consistently improves the performance in Waymo -> KITTI and nuScenes + KITTI tasks by large margin of 81.5 % and 7.4% in AP3D. respectively. This improvement significantly closes the performance gap 12% and 79.79% between No Adapt. and Oracle. Remarkably, the proposed MOS outperforms the UDA method ST3D (Yang et al. 2021), which requires multi- epoch training: MOS's 64. 16% vs. ST3D's 61. 3.. in AP3D. and even demonstrates comparable performance to the upper bound performance (Oracle): MOS's 81.90% vs. Oracle's 's 83.29% in AP when adapting from Waymo to KITTI. The results and analysis of additional cross- dataset transfer tasks (Waymo- + nuScenes) are provided in Appendix A.2
8

---

Published as a conference paper at ICLR 2025
Easy
Moderate
Hard
20 25
23:49
32.54
23.17
22
80
51.23
17.94
21.25
29:13
20.95
20.61
45.70
16.85
19.52
27:27
10:34
19.08
43.26
Wek.
44.86
33.36
32.70
27.74
30.94
30.14
30.06
26.
38.23
28.23
28.24
Snow
12.4
14:22
26.75
14.72
14:20
26
13,03
14.82
26.32
14.61
14.37
31:14
11.40
13 71
24.50
13:81
13:50
28.00 07.07
Mot..
8:26
9:12
13:74
8.85
8.54
26 21
8.30
9.82
13:03
9.56
9.36
24.65
8.16
9:87
13.75
9:50
9.34
24.50
Boam.
32.01
20.90
20.90
46
23.57
15:82
24.13
15:68
15.78
34.96
12.53
22.86
14.83
14.98
Creest.
25.07
28.74
24.54
24.74
42.44
21.71
23.77
28.65
23:18
23.39
40.98
20.03
27.63
21.64
21.77
Inc.
4.91
7.38
10.97
7.22
6.66
22.43
3.84
6.14
8.36 5.59
5.25
16.14
3.51
5.63
7.56
5.18
4.67
14. 16.96
Cramas
11:01
16
15.24
16.08
17:40
32
0:09
1167
10.67
10.55
11.03
21.63
7.40
9.36
10.08
19
No Adapt
COTTA
SAR
MamCLe
MOS
No Adapt
Tork
COTTA
SAR
MomCLR
MGS
No Adapt.
Rent
Co
SAR
MemcLL
MOS
Figure 5: Heatmap intuitively presenting the results of TTA-3OD across hybrid cross- cooss-coorrption shifts (Waymo - KITTI-C) in heavy difficulty. Darker//ligheer shades indicate lower/higher performance.
Corruption Shifts. We carry out comprehensive experiments (KITTI - KITTI-C) across eight different types of real-world corruptions to validate the effectiveness of the proposed MOS to mitigate these corrupiion shifts. As reported in Tab. 2 MOS surpasses all TTA baselines across every type of corruption, achieving AP3 gains of 3. 3.73%, 5.56%, and 4.78% at easy, moderate, and hard levels, respectively. Notably, MOS greatly addresses the most severe shift caused by incomplete echoes, improving 23.98% at the hard difficulty level. These promising results highlight MOS's s robustness to adapt 3D models to various corrupted scenarios, particularly those challenging ones.
corsspto Shifts. To address the hybrid shifts where cross-datases discrepan- cies and corruptions simultancously occur, we conduct experiments to adapt 3D detec- tors from Waymo to KITTI-C. We visualize the results utilizing the heatmap (in Fig. [5) is observed that the shades in the last col- umn (MOS), are significantly lighter than
Table 3: Ablative study of the proposed MOS on
Waymo - KITTI. The best result is highlighted in bold.
AP3o
APsEv
Ablation
Easy
Moderate
Hard
Easy
Moderate
Hard
we Ensemble
45.17
43.14
41.48
71.36
68.17
68.96
Mean Ensemble
48.56
45.89
44.37
72.72
70.34
70.15
MOS w/o Stea
67.02
59.98
58.76
91.85
81.90
79.77
MOS w/o Stor
72.99
61.40
60.09
88.52
80.15
79.06
MOS
74.09
64.16
62.33
91.85
81,90
81.78
those in all other columns (TTA baselines), indicating the superior performance of MOS over all other methods. It is worth noting that, the results without any adaptation (column 1) remarkably degrade, due to the challenging hybrid shifts, for example, only 3.51 % for incomplete echoes and 7.40% for cross-sensor approach greatly enhances adaptation performance for these two challenging corruptions by 97.99 % and 76.38 %, respectively, compared to the best baselines. Thus, prior TTA basclines fail to mitigate the significant domain shifts across 3D scenes, whereas the proposed MOS successfully handles various corss-conrn shifts, surpassing the highest baseline and direct inference by 67.3% % and 161.5% % respectively, across all corruptions.
hard level. Our
4.3 ABLATION AND SENSITIVITY STUDY
Ablation Study. To validate the effectiveness of each component in the proposed MOS, we conduct ablation experiments on Waymo - KITTI and report the AP across three difficulty levels. First, we remove the pivotal component in our method, model assembly, opting instead to simply use the model at the current timestamp for pseudo-label generation. Tab. shows that without assembly (row 1), the adaptation yields suboptimal results, achieving only 13. in moderate AP3D. Then we assemble the recent five checkpoints by averaging their parameters (row 2), and observe marginal improvement in moderate APıD to 45.89%. While our proposed assembly strategy (row 5), identifying and assembling the most useful 5 checkpoints through weighted averaging, significantly outperforms average aggregation by 39.8%. Next, we remove each of the proposed similarity functions used to calculate the synergy weights for the model ensemble. As shown in Tab. 3 removing either (:em(::)) or Sbox(e (:: )) leads to notable decrease in moderate AP by 6.9% and 4.5%. respectively, proving that both functions accurately gauge similarity between any pair of 3D detection models. Additional ablation studies can be found in Appendix A.I A.
Sensitivity to Hyperparameters. In this section, we examine how the adaptation performance of the proposed MOS is affected by the hyperparameters L and K. which determines the period of the model bank update(í.e. every L batches of test data) and the bank capacity. To study L, we conduct experiments with values ranging from 64 to 128. As illustrated in the Fig. (lef)). the APpEY (blue

---

Published as a conference paper at ICLR 2025
curve) remains stable, while the AP (green curve) increases as L grows from 64 to 96. This occurs because a larger L reduces the model update frequency. enabling the model bank to retain older checkpoints that hold distinct long-term term knowledge. Upon reaching 96, AD becomes stable with slight fluctuation of 1.67%.
To assess the informativeness and significance of the K checkpoints selected by the proposed MOS, we developed a comparative analysis between the standard MOS and a simplified variant: latest-firas MOS. This variant (crosses in Fig. 6 right) utilizes the most recent K
80
80
checkpoints and employs the same en-
SW-trst
semble strategy as MOS (i.e., Eq. (5)
* X
*
Wbst first
60
and Eq. (6). The standard MOS, that maintains K checkpoints with high synergy weights (SW) (Sec. 3.2).
APso
APrev
64
80
96
112
128
123
referred to as SW-first MOS (stars in Fig. 6 right). As shown Fig. 6 latest-first MOS presents obvious improvement in AP3 from 43.14% to 54.18% and in APBEV from 68.17% to 76.29% when assembling more latest checkpoints (K increases from to 5). However, expanding the ensemble to 20 models yields marginal increase to 56.16% in AD3D and 77.54% in APBEV. with a substantial memory cost. The SW-first MOS, which dynamically utilizes only 3 historical checkpoints based on averaged SW, out- performs latest-first MOS with 20 checkpoints by 18 18.42% in APaD and 5.84% in AP meanwhile, reducing memory usage by 85%. This efficiency highlights our method's s ability to selectively keep the most pivotal historical models in the model bank, optimizing both detection performance and memory expenditure.
Figure 6: Left: Sensitivity to the period of model bank update L. Right: Comparison between two MOS-based ensemble strategies (SW-first vs. latest-first) with increasing model bank size K. Both APBEV and AP3D are plotted when adapt- ing the 3D detector (Yan al. 2018 from Waymo to KITTI.
4.4 COMPLEXITY ANALYSIS
This section analyzes the complexity of the proposed MOS. Recall that K models in the bank F infer every test batch x, from the datasets {x,]f= the theoretical time complexity is asymptotically equivalent (I'). as K is constant. Concerning the space complexity, we assume that 3D detection network requires Nparam parameters for memory storage, and the space complexity is bounded by (. Theoretically. the proposed MOS exhibits a manageable linear complexity growth with respect to model size and dataset size.
80
We report the adap- tation speed (seconds per frame) and GPU mem- ory usage (MiB) of implemented methods in Fig. 7 It is observed that the proposed MOS significantly outperforms the top baseline, im- proving AP by 21.4% while only requiring additional 0.255 seconds per frame compared to the baseline average. In terms of memory usage, mean-teacher frameworks such as Mem- CLR (19,087 MiB) and CoTTA (15,099 MiB) consume more resources as they simultancously load dual networks in the GPU. The substantial consumption of McmCLR mainly stems from its extra transformer- based memory module. Compared to MemCLR, our MOS maintains a lower memory footprint at 17,411 MiB. Overall, MOS offers substantially superior adaptation performance with a manageable increase in memory and time consumption.
Time and Memory Usage
75
65
0.2
0.3
0.4
12000 14000 16000 18000 Memory Usage IMiB:
Seconds per frame
Tent
CoTTA
SAR
MemCLR R
Ours
Figure 7: Time and memory usage of method im- plementations when adapting the 3D detector YYan) et al. 2018) from Waymo to KITTI dataset.
CONCLUSION
This paper is an early attempt to explore the test-time adaptation for LiDAR-based 3D object detection and introduces an effective model synergy approach that leverages long-term knowledge from historical checkpoints to tackle this task. Comprehensive experiments confirm that MOS efficiently adapts both voxel- and point-based 3D detectors to various scenes in single pass, addressing real-world domain shifts sourced from cross- cross-dataset, corruptions, and cross-corruptions. The main limitation is its high computational demand and future efforts will focus on improving the time and space efficiency of the algorithm, through storing, and assembly of only a small portion of model parameters (e.g., selected layers modules) facilitate faster adaptation at test time.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work was supported by the Australian Research Council (DE240100105, DP240101814, DP230101196).
REFERENCES
Syeda Mariam Ahmed, Yan Zhi Tan, Chee-Meng Chew, Abdullah Al Mamun, and Fook Seng Wong. Edge and comnrr detection for unorganized 3d point clouds with application to robotic welding. In Proc. International Conference on Intelligent Robots and Systems (IROS), pp. 7350-7355, 2018.
Eduardo Arnold, Omar Y. Al-Jarrah, Mehrdad Dianati, Saber Fallah, David Oxtoby, and Alex Mouzakitis. A survey on 3d object detection methods for autonomous driving applications. IEEE Transactions on Intelligent Transportation Systems, 20(10):3782-3795, 2019.
Leo Breiman. Bagging predictors. Machine learning, 24:123-140, 1996.
Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11618-11628, 2020.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Proc. European Conference on Computer Vision (ECCV), volume 12346 of Lecture Notes in Computer Science, pp. 213-229, 2020.
Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proc. IEEE Conference on Computer Vision and Pottern Recognition (CVPR), pp. 295-305, 2022.
Zhuoxiao Chen, Yadan Luo, and Mahsa Baktashmotlagh. Conditional extreme value theory for open set video domain adaptation. In Proc. ACM International Conference Multimedia in Asia (MM Asia), pp. 20:1-20:8, 2021.
Zhuoxiao Chen, Yadan Luo, Zheng Wang, Mahsa Baktashmotlagh, and Zi Huang. Revisiting domain- adaptive 3d object detection by reliable, diverse and class-balanced pseudo-labeling. Proc. International Conference Computer Vision (ICCV), pp. 3691-3703, 2023a.
Zhuoxiao Chen, Yadan Luo, Zixin Wang, Zijian Wang. Xin Yu, and Zi Huang. Towards open world active learning for 3d object detection. CoRR, abs/2310. 1033911,,, 2023b.
Zhuoxiao Chen, Zixin Wang, Yadan Luo, Sen Wang, and Zi Huang. DPO: puttuueeturbation optimization for test-time adaptation 3d object detection. In Proc. ACM International Conference on Multimedia (MM), pp. 4138 4187. ACM, 2024.
Robert DeBortoli, Fuxin Li, Ashish Kapoor, and Geoffrey A. Hollinger. Adversarial training point clouds for sim-to-real 3d object detection. IEEE Robotics and Automation Letters, 6(4): 6662-6669, 2021.
Boyang Deng, Charles R. Qi, Mahyar Najibi, Thomas A. Funkhouser, Yin Zhou, and Dragomir Anguelov. Revisiting 3d object detection from egocentric perspective. In Proc. Annual Conference on Neural Information Processing (NeurIPS), pp. 26066-26079, 2021.
Marius-Constantin Dinu, Markus Holzleitner, Maximilian Beck, Hoan Due Nguyen, Andrea Huber, Hamid Eghbal-zadeh, Bernhard Alois Moser. Sergei V. Pereverzyev, Sepp Hochreiter, and Werner Zellinger. Addressing parameter choice issues in unsupervised domain adaptation by aggregation. In Proc. International Conference on Learning Representations (ICLR), 2023.
Yinpeng Dong, Caixin Kang, Jinlai Zhang, Zijian Zhu, Yikai Wang, Xiao Yang, Hang Su, Xingxing Wei. and Jun Zhu. Benchmarking robustness of 3d object detection common corruptions in autonomous driving. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1022-1032, 2023.
11

---

Published as a conference paper at ICLR 2025
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor S. Lempitsky. Domain-adverseen training of neural networks. Journal of Machine Learning Research, 17:59:1-5::35, 2016.
Yingbo Gao, Christian Herold, Zijian Yang, and Hermann Ney. Revisiting checkpoint averaging for neural machine translation. In Findings of the Association for Computational Linguistics, pp. 188-196, 2022.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the KITTI vision benchmark suite. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3354 3361, 2012.
Taesik Gong, Jongheon Jeong. Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. Note: Robust continual test-time adaptation against temporal correlation. In Proc. Annual Conference on Neural Information Processing (NeurIPS), pp. 27253-272666 2022.
Taesik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottananurak, and Sung-Ju Lee. Sotta: Robust test-time adaptation on noisy data streams. In Proc. Annual Conference Neural Information Processing (NeurIPS), 2024.
Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and Zico Kolter. Test time adaptation via conjugate pseudo-labels In Proc. Annual Conference on Neural Information Processing (NeurIPS), 2022.
Qiqi Gu, Qianyu Zhou, Minghao Xu, Zhengyang Feng. Guangliang Cheng, Xuequan Lu, Jianping Shi, and Lizhuang Ma. PIT: position-invariant transform for cross-fov domain adaptation. In Proc. International Conference on Computer Vision (ICCV), pp. 8741-8750, 2021.
Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin Dai, and Luc Van Gool. Lidar snowfall simulation for robust 3d object detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. (6343-16353, 2022.
Tyler L. Hayes, Kushal Kafle, Robik Shrestha, Manoj Acharya, and Christopher Kanan. REMIND your neural network to prevent catastrophic forgetting. In Proc. European Conference on Computer Vision (ECCV), volume 12353, pp. 466 483, 2020.
Deepti Hegde, Velat Kilic, Vishwanath Sindagi, Brinton Cooper, Mark Foster, and Vishal Patel. Source-free unsupervised domain adaptation for 3d object detection in adverse weather. In Proc. International Conference on Robotics and Automation (ICRA), 2023.
Junyuan Hong, Lingjuan Lyu, Jiayu Zhou, and Michael Spranger. Mecta: Memory-cconomic contin- ual test-time model adaptation. In Proc. International Conference Learning Representations (ICLR), 2023.
Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma, Dongyan Zhao, and Rui Yan. Overcoming catastrophic forgetting for continual learning via model adaptation. In Proc. International Conference on Learning Representations (ICI.R,, 2019.
Chengjie Huang, Vahdat Abdelzad, Sean Sedwards, and Krzysztof Czarnecki. Soap: Cross-sensor domain adaptation for 3d object detection using stationary object aggregation labelii.... In Proc. Winter Conference on Applications of Computer Vision (WACV), 2024.
Sanghun Jung, Jungsoo Lee, Nanhee Kim, Amirreza Shaban, Byron Boots, and Jaegul Choo. Cafa: Class-aware feature alignment for test-time adaptation. In Proc. International Conference on Computer Vision (ICCV), pp. 19060-19071, 2023.
Jean Kaddour. Stop wasting my time! saving days of imagenet and BERT training with latest weight averaging. CoRR, 1et./2299.1981... 2022.
Zhao Kang, Chong Peng, and Qiang Cheng. Robust pea via nonconvex rank approximation. In Proc. International Conference Data Mining (ICDM), pp. 211-220. IEEE, 2015.
Ronald Kemker, Mare McClure, Angelina Abitino, Tyler L. Hayes, and Christopher Kanan. Mea- suring catastrophic forgetting in neural networks. In Proc. Conference on Artificial Intelligence (AAAI), pp. 3390-3398, 2018.
12

---

Published as a conference paper at ICLR 2025
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinsker et al. Overcoming catastrophic forgetting in neural networks. In Proc. National Academy of Sciences (PNAS), pp. 3521-3526, 2017.
Lingdong Kong, Youquan Liu, Xin Li, Runnan Chen, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Robo3d: Towards robust and reliable 3d perception against corruptions. In Proc. International Conference on Computer Vision (ICCV), pp. 19937-19949, 2023.
Alex H. Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpil- lars: Fast encoders for object detection from point clouds. In Proc. IEEE Conference on Computer Vision and Putlern Recognition (CVPR), pp. 12697-12705, 2019.
Sang- Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung- Woo Ha, and Byoung-Tate Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In Proc. Annual Conference on Neural Information Processing (NeurIPS), pp. 4652-4662, 2017.
Alexander Lehner, Stefano Gasperini, Alvaro Paacos- Michael Schmidt, Mohammad- Ali Nikouei Mahani, Nassir Navab, Benjamin Busam, and Federico Tombari. 3d-vfield: Ad- versarial augmentation of point clouds for domain generalization in 3d object detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17274-17283, 2022.
Buyu Li, Wanli Ouyang, Lu Sheng, Xingyu Zeng, and Xiaogang Wang. GS3D: an efficient 3d object detection framework for autonomous driving. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1019-1028, 2019.
Ziyu Li, Jingming Guo, Tongtong Cao, Bingbing Liu, and Wankou Yang. GPA-3D: geometry-aware prototype alignment for unsupervised domain adaptive 3d object detection from point clouds. In Proc. International Conference on Computer Vision (ICCV), pp. 6371-6380, 2023a.
Ziyu Li, Yuncong Yao, Zhibin Quan, Lei Qi, Zhenhua Feng, and Wankou Yang. Adaptation via proxy: Building instance-aware proxy for unsupervised domain adaptive 3d object detection. IEEE Transactions Intelligent Vehicles, 2023b.
Jia Syuen Lim, Zhuoxiao Chen, Zhi Chen, Mahsa Baktashmotlagh, Xin Yu, Zi Huang, and Yadan Luo. DiPEx: Dispersing prompt expansion for class-agnostic object detection. In Proc. Annual Conference on Neural Information Processing (NeurIPS), 2024.
Yadan Luo, Zhuoxiao Chen, Zhen Fang, Zheng Zhang, Mahsa Baktashmotlagh, and Zi Huang. Kecor: Kernel coding rate maximization for active 3d object detection. In Proc. International Conference on Computer Vision (ICCV), pp. 18233 19233-18244, 2023a.
Yadan Luo, Zhuoxiao Chen, Zijian Wang, Xin Yu, Zi Huang, and Mahsa Baktashmotlagh. Exploring active 3d object detection from generalization perspective. Proc. International Conference on Learning Representations (ICLR), 2023b.
Yadan Luo, Zijian Wang, Zhuoxiao Chen, Zi Huang, and Mahsa Baktashmotlagh. Source-free progressive graph learning for open-set domain adaptation. JEEE Transactions on Pattern Analysis 2023c.
and Machine Intelligence, 12400
Zhipeng Luo, Zhongang Cai, Changqing Zhou, Gongjie Zhang, Haiyu Zhao, Shuai Yi, Shijian Lu, Hongsheng Li, Shanghang Zhang, and Ziwei Liu. Unsupervised domain adaptive 3d detection with multi-level consistency. In Proc. International Conference on Computer Vision (ICCV), pp- 8846-8855, 2021.
Jiageng Mao, Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. 3d object detection for autonomous driving: A comprehensive survey. International Journal of Computer Vision, pp. 1-55, 2023.
Michael Matena and Colin Raffel. Merging models with fisher-weighted averaging. In Proc. Annual Conference on Neural Information Processing (NeurIPS), 2022.
Scott McCrae and Avideh Zakhor. 3d object detection for autonomous driving using temporal lidar data. In Proc. International Conference on Image Processing (ICIP), pp. 2661-2665, 2020.
13

---

Published as a conference paper at ICLR 2025
Gregory P. Meyer, Ankit Laddha, Eric Kee, Carlos dallesp--Gozzale and Carl K. Wellington. Lasernet: An efficient probabilistic 3d object detector for autonomous driving. In Proc. IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 12677-12686, 2019.
Muhammad Jehanzeb Mirza, Jakub Micorek, Horst Possegger, and Horst Bischof. The norm must go on: Dynamic unsupervised domain adaptation by normalization. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14745-14755, IEEE, 2022.
Hector A. Montes, Justin Le Louedee, Grzegorz Cielniak, and Tom Duckett. Real-time detection of broccoli crops in 3d point clouds for autonomous robotic harvesting, In Proc. International Conference on Intelligent Robots and Systems (IROS), pp. 10483-10488, 2020.
Ismail Nejjar, Qin Wang, and Olga Fink. DARE-GRAM: Unsupervised domain adaptation regression by aligning inverse gram matrices. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11744-11754, 2023.
A. Tuan Nguyen, Thanh Nguyen- Nguyen-Tang, Ser-Nam Lim, and Philip H.S. Torr. Tipi: Test time adaptation with transformation invariance. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24162-24171, June 2023.
Fahim Faisal Niloy, Sk Miraj Ahmed, Dripta S. Raychaudhuri, Samet Oymak, and Amit K. Roy- Chowdhury. Effective restoration of source knowledge in continual test time adaptation. In Proc. Winter Conference on Applications of Computer Vision (WACV), pp. 2091-2100, 2024.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In Proc. International Conference on Machine Learning (ICML). pp. 16888-16905, 2022.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In Proc. International Conference Learning Representations (ICLR), 2023.
Xidong Peng, Xinge Zhu, and Yuexin Ma. CL3D: unsupervised domain adaptation for cross-lidar 3d detection. In Proc. Conference on Artificial Intelligence (AAAI), pp. 2047-2055, 2023.
Rui Qian, Xin Lai, and Xirong Li. 3d object detection for autonomous driving: A survey. Pattern Recognition, 130: 130: 108796, 2022.
Benjamin Recht, Maryam Fazel, and Pablo A Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM review, 52(3):471-501, 2010.
Christoph B Rist, Markus Enzweiler, and Dariu M Gavrila. Cross-sensor deep domain adaptation for lidar detection and segmentation. In Proc. Intelligent Vehicles Symposium, (IV), pp. 1535-1542, 2019.
Khaled Saleh, Ahmed Abobakr, Mohammed Hassan Attia, Julie Iskander, Darius Nahavandi, Mo- hammed Hossny, and Saeid Nahavandi. Domain adaptation for vehicle detection from bird's eye view lidar point cloud data. In Proc. International Conference on Computer Vision (ICCV), pp. 3235-3242, 2019.
Cristiano Saltori, Stéphane Lathuilière, Nicu Sebe, Elisa Ricci, and Fabio Galasso. Sf-uda 3d. Source- free unsupervised domain adaptation for lidar-based 3d object detection. In Proc. International Conference on 3D Vision (3DV), pp. 771 771-780, 2020.
Shaoshuai Shi, Chaoxu Guo, Jiang. Zhe Wang. Jianping Shi, Xiaogang Wang, and Hongsheng Li. PV-RCNN: poxnt-voxel feature set abstraction for 3d object detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10526-10535, 2020.
Junha Song, Jungsoo Lee, In So Kweon, and Sungha Choi. Ecotta: Memory- continual test-time adaptation via dist-distilledd regularization. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11920-11929 2023.
14

---

Published as a conference paper at ICLR 2025
Russell Stewart, Mykhaylo Andriluka, and Andrew Y Ng. End-to-end end people detection in crowded scenes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2325-2333, 2016.
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao. Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2443 2451, 2020.
OpenPCDet Development Team. Openpedet: An open-source toolbox for 3d object detection from point clouds. htttps://githube www. mml ab/
OpennCDet] hbmmmm///////eeetttt 2020.
Devavrat Tomar, Guillaume Vray, Behzad Bozorgtabar, and lcan-Philippe Thiran. Tesla: Test-time self-learning with automatic adversarial augmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.
2023a.
Devavrat Tomar, Guillaume Vray, Behzad Bozorgtahar, and Jean-Philippe Thiran. Tesla: Test-time self-learning with automatic adversarial augmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20341-20350, 2023b.
Darren Tsai, Julie Stephany Berrio, Mao Shan, Eduardo Nebot, and Stewart Worrall. Ms3d++: Ensemble of experts for multi-source unsupervised domain adaption in 3d object detection. CoRR, abs/2308. 059888, 2023a.
Darren Tsai, Julie Stephany Berrio, Mao Shan, Eduardo M. Nebot, and Stewart Worrall. Viewer- centred surface completion for unsupervised domain adaptation in 3d object detection. In Proc. International Conference Robotics and Automation (ICRA), pp. 9346-9353, 2023b.
Vibashan VS, Poojan Oza, and Vishal M. Patel. Towards online domain adaptive object detection. In Proc. Winter Conference on Applications of Computer Vision (WACV), pp. 478-488, 2023.
Dequan Wang. Evan Shelhamer, Shaoteng Liu, Bruno A. Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In Proc. International Conference on Learning Representations (ICLR), 2021.
Haiyang Wang, Chen Shi, Shaoshuai Shi, Meng Lei, Sen Wang, Di He, Bernt Schiele, and Liwei Wang. DSVT: dynamic sparse voxel transformer with rotated sets. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13520-13529, 2023a.
Jun Wang, Shiyi Lan, Mingfei Gao, and Larry S. Davis. Infofocus: 3d object detection for autonomous driving with dynamic information modeling. In Proc. European Conference on Computer Vision (ECCV), volume 12355, pp. 405-420. Springer, 2020a.
Qin Wang, Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7191-7201, 2022a.
Shuai Wang, Daoan Zhang, Zipei Yan, Jianguo Zhang, and Rui Li. Feature alignment and uniformity for test time adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20050-20060, 2023b.
Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hariharan, Mark E. Campbell, and Kilian Q. Weinberger. Pseudo- lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8445-8453, 2019.
Yan Wang, Xiangyu Chen, Yurong You, Li Erran Li, Bharath Hariharan, Mark E. Campbell, Kilian Q. Weinberger, and Wei-Lun Chao. Train in germany, test in the USA: making 3d object detectors generalize. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11710-11720, 2020b..
15

---

Published as a conference paper at ICLR 2025
Zixin Wang, Yadan Luo, Peng- Fei Zhang, Sen Wang, and Zi Huang. Discovering domain disentan- glement for generalized multi multi-source source domain adaptation. In Proc. International Conference on Multimedia and Expo (ICME), pp. 1-6, 2022b.
Zixin Wang, Yadan Luo, Zhi Chen, Sen Wang. and Zi Huang. Cal-sfda: Source-free domain domain-adaptive semantic segmentation with differentiable expected calibration enror. In Proc. ACM International Conference on Multimedia (MM), pp. 1167-1178. ACM, 2023c.
Zixin Wang, Dong Gong, Sen Wang, Zi Huang, and Yadan Luo. Tokens on demand: Token condensation training-fres test-time adaptation. CoRR, abs/2410.14722 14729, 2024a.
Zixin Wang, Yadan Luo, Liang Zheng, Zhuoxiao Chen, Sen Wang, and Zi Huang, In search of lost online test-time adaptation: A survey. International Journal of Computer Vision, pp. 1 1-34, 2024b.
Yi Wei, Zibu Wei, Yongming Rao, Jiaxin Li, Jie Zhou, and Jiwen Lu. Lidar distillation: Bridging the beam-induced domain gap for 3d object detection. In Proc. European Conference on Computer Vision (ECCV), volume 13699, pp. 179-195, 2022.
Mitchell Wortsman, Gabriel Ilhareo, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo Lopes, Ari S. Morcos, Hongseok Namkoong. Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proc. International Conference on Machine Learning (ICML), volume 162, pp. 23965-23999 2022.
Qiangeng Xu, Yin Zhou, Weiyue Wang, Charles R. Qi, and Dragomir Anguelov. SPG: unsupervised domain adaptation for 3d object detection via semantic point generation. In Proc. International Conference on Computer Vision (ICCV), pp. 15426-15436, 2021.
Yan Yan, Yuxing Mao, and Bo Li. SECOND: sparsely embedded convolutional detection. Sensors, 18(10):3337, 2018.
Jihan Yang, Shaoshuai Shi, Zhe Wang, Hongsheng Li, and Xiaojuan Qi. ST3D: self-training for unsupervised domain adaptation on 3d object detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10368-10378, 2021.
Jihan Yang, Shaoshuai Shi, Zhe Wang, Hongsheng Li, and Xiaojuan Qi. St3d++: denoised self- training for unsupervised domain adaptation on 3d object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
Cang Ye and Xiangfei Qian. 3-d object recognition of a robotic navigation aid for the visually impaired. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(2):441-450, 2018.
Tianwei Yin, Xingyi Zhou, and Philipp Krähenbühl. Center-based object detection and tracking. In Proc. IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 11784-11793, 2021.
Yurong You, Yan Wang, Wei-Lun Chao, Divyansh Garg, Geoff Pleiss, Bharath Hariharan, Mark E. Campbell, and Kilian Q. Weinberger. Pseudo-lidar+4: Accurate depth for 3d object detection in autonomous driving. In Proc. International Conference on Learning Representations (ICLR), 2020.
Yurong You, Carlos Andres Ruaz-Ruiz, Yan Wang, Wei-Lun Chao, Bharath Hariharan, Mark E. Campbell, and Kilian Q. Weinberger. Exploiting playbacks in unsupervised domain adaptation for 3d object detection in self-driving cars. In Proc. International Conference on Robotics and Automation (ICRA), pp. 5070-5077, 2022.
Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15922-15932, 2023.
Kevin Kam Fung Yuen. Inverse gram matrix methods for prioritization in analytic hierarchy process: Explainability of weighted least squares optimization method. CoRR, abs/2401.00 190, 2024.
16

---

Published as a conference paper at ICLR 2025
Longbin Zeng, Jiayi Han, Liang Du, and Weiyang Ding. Rethinking precision of pseudo label:
Test-time adaptation via complementary learning. Pattern Recognition Letters, 177:96-102, 102, 2024.
Yihan Zeng, Chunwei Wang. Yunbo Wang, Hang Xu, Chaoqiang Ye, Zhen Yang, and Chao Ma. Learning transferable features for point cloud detection via 3d contrastive co-training. Proc. Annual Conference on Neural Information Processing (NeurIPS), pp. 21493-21504, 2021.
Marvin Zhang, Sergey Levine, and Chelsea Finn. MEMO: test time robustness via adaptation and augmentation. Proc. Annual Conference on Neural Information Processing (NeurIPS), 2022.
Ruiyang Zhang, Hu Zhang, Hang Yu, and Zhedong Zheng. Approaching outside: Scaling unsu- pervised 3d object detection from 2d scene. In Proc. European Conference on Computer Vision (ECCV), volume 15069 of Lecture Notes in Computer Science, pp. 249-2666 Springer, 2024.
Weichen Zhang, Wen Li, and Dong Xu, SRDAN: scale-aware and range- aware domain adaptation network for cross-dataset 3d object detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6769-6779, 2021.
Bowen Zhao, Chen Chen, and Shu-Tao Xia. Delta: degradation-free fully test-time adaptation. In Proc. International Conference on Learning Representations (ICIR), 2023.
Zhengxue Zhou, Leihui Li, Alexander Fürsterling, Hjalte Joshua Durocher, Jesper Mouridsen, and Xuping Zhang. Learning based object detection and localization for mobile robot manipulator in SME production. Robotics and Computer-Integrated Manufacturing, 73: 102122,,, 2022.
17

---

Published as a conference paper at ICLR 2025
A APPENDIX
This Appendix provides additional details, experiments, and analysis of the proposed MOS, summa- rized as follows:
Algorithm. 1
Detailed algorithm of the proposed MOS framework.
Sec. A.1 Additional ablation and component study on more challenge transfer tasks:
Waymo + KITTI-C.
Study on similarity measurement methods (A.1.I)
Study on feature selection for similarity comparison A.1.2)
-
Study on MOS's ability to retain long-term knowledge 1.31.3)
Study direct use of trivial ensemble method (A.1.4)
Sec.
A.2:
Experimental results and analysis on additional transfer task: Waymo -> nuScenes.
Sec.
A.3
Discussion about related model ensemble methods.
Sec.
A.4
Implementation details on computing devices, hyperparameter selection, data
augmentations, pseudo-labeling, baseline losses and ensemble techniques in MOS.
Sec. A.5:
Experimental results and analysis across multiple classes to address both cross- dataset and cross--orruption shifts.
Sec.
A.6:
Visualizaion of 3D box predictions by checkpoints with different synergy weights.
Sec.
A.7
Experimental results with different backbone 3D detectors.
Sec.
A.8.
Qualitative study with visualized 3D box predictions from different methods.
A.1
ADDITIONAL ABLATTON/COMPONENT STUDY
To further validate the effectiveness of each component in the proposed MOS, we con- ducted ablation study
Table 4: Ablative study of the proposed MOS
Waymo - KITTI-C, under the corruption "incomplete echo". MOS cosine Stear refers the use of cosine sim- ilarity instead of rank-based similarity. MOS Rol Sceat denotes that Rol features from the second stage, rather than BEV maps, are used by MOS. The best result is highlighted in bold.
more chal- lenging transfer task: Waymo + KITTI- under the corruption "incomplete echo". As reported in Tab. 4 we observed signif- icant performance drop when either feature similarity box similarity 25,4% and 17.2% reductions removed: AP at hard difficulty, respectively. Compared the 5.7% and 3.6% drop in the Waymo > KITTI task, MOS proves to be especially effective more difficult cross-corruption transfer tasks.
APtp
APsev
Ablation
Easy
Moderate
Hard
Easy
Moderate
Hard
Siea
13.68
MOS w/o Fous
20.13
14.39
12.38
44.34
30.77
27.77
MOS cusine
17.74
12.51
11.04
42.20
29.01
26.18
MOS Rol Seur
19.13
13:73
11.84
44.14
30.72
27.71
MOS
22.43
16.14
14.96
44.18
32,05
29,58
A.1.1 RANK-BASED SIMILARITY VS. COSINE SIMILARITY
In this section, discuss the advantages of the rank-based similarity method and its superiority over the conventional cosine similarity. The key advantage is its ability to capture feature similarity only between a pair of point clouds but also within each single point cloud. In method, we concatenate 2; and 2 and compute the rank of the concatenated matrix. High similarity (low rank value) occurs two cases: (1) most features in z, are linearly dependent on features in Zj. indicating that z, and zj are similar, or (2) even if and z, are very different, the features within (or within 2,) are linearly dependent on each other, for example, point cloud contains cars only with very similar shapes and sizes. In this case, the concatenated feature will still produce a low rank value due contrast, to the absence of many diverse features although concatenated, indicating redundancy. cosine similarity is based purely on the comparison between pair of frames and ignores feature similarity within each point cloud. To support our claim, we provide experiments Tab. was observed that utilizing cosine similarity (row 3) results in significant performance drop of 29 % in AP Moderate compared to based similarity measurements (row 5). Furthermore, when comparing it MOS without feature similarity (row 1), there is performance gain. These findings
18

---

Published as a conference paper at ICLR 2025
underscore the importance of feature diversity within point clouds, which cosine similarity fails to capture.
A.1.2 SELECTION OF INTERMEDIATE FEATURES
In our method, we empirically utilize the BEV feature map of size 128 128 256 from the last layer of the encoder in both SECOND and PV-RCNN, as it is directly used to generate bounding boxes thus containing global fine-graine fine-grained information about the point clouds. We also conduct experiments extracted region of interest (Rol) features (i.e., 100 Rol features of dimension 128). As shown in Tab. 4 using Rol features (row 4) yields lower performance (3.1 % in AP3 Hard) than the BEV map (row 5). This occurs as Rol features focus only foreground objects, resulting in information loss in environments.
A.1.3 STUDY ON LONG-TERE KNOWLADGE
We further investigate the capability of MOS selecting and preserving checkpoints that contain long-term knowledge. Specifically, we perform experiments where the final stored checkpoints, after completing the test-time adaptation, are ensembled into the super model to infer the early data (ie., the first 32 point clouds). The results in Tab. confirm that even after running for long time and reaching the end of the target datasets, the final checkpoints maintain strong performance on the initial 32 point clouds, significantly outperforming the 32nd checkpoint by 239% in AP Furthermore, the high recall rate of 69.34 demonstrates that nearly all objects in the early batches are detected, highlighting the capacity of the saved checkpoints to retain diverse and long-term knowledge.
Table 5: Performance of the 32th checkpoint and final stored checkpoints of MOS on the Early Set (i.e., the first 32 point clouds), when adapting from Waymo to KITTI-C, under the corruption "incomplete echo". 0.5 denotes the recall metric calculated at an IoU of O.5.
MOS
APBEV
AP
Recall@005
MOS (32th Checkpoint) Early Set
15.03
4.14
64.04
MOS (Final Checkpoints) on Early Set
33.38
14.04
69.34
A.1.4 COMPARARION WITH TRIVIAL ENSEMBLE METHOSS
Least Squares. To investigate whether trivial en- semble methods can effectively solve for the op- timal synthetic model in the TTA-3OD task, we employed the simplest approach: least squares, to ensemble only single layer of 3D detection model. Specifically, we treat the frozen encoder as producing a fixed feature map z, and we ap- ply least squares (LS) to the final layer, which consists of only 256 parameters. As the results clearly demonstrate in Tab. 6 applying LS to a single layer with very few parameters (i.e., 256) of 3D detection model failed to address the TTA scenarios, yielding a notably lower-bound accuracy of 4.93 in AP
Table 6: Performance comparison between trivial ensemble methods and the proposed MOS in TTA- 3OD, when adapting from Waymo to KITTI-C, under the corruption "incomplete echo". AP at moderate difficulty level is reported.
APBEV
AP 3D
No Adapt
15.15
3.77
Least Squares (Last Layer)
18.28
4.93
Bagging (3 models) MOS
27.59
14.30
32.05
16.14
Bagging, We also explore a traditional non-linear ensemble method, bagging (Breiman 1996). This method involves training multiple models randomly sampled subsets of the training set, which makes it inapplicable for the TTA-3OD task. In our implementation, for each test batch, we randomly select one model from three to update the weights, followed by non-maximum suppression (NMS) with threshold of 0.7 to obtain the final predictions from the box outputs of all three models. Our empirical results show that bagging consumes 34,952 MiB of GPU memory due to the simultaneous loading of three 3D detectors, which is significantly higher than the proposed MOS method (17,411 MiB). Despite the heavy GPU usage, bagging outperformed by the proposed MOS method, with a 11.4% lower APaD. indicating that non-linear ensemble methods are computationally expensive and less effective, making them unsuitable for the TTA-3OD task.
19

---

Published as a conference paper at ICLR 2025
Table 8: [Complete Results] Cross-dataset results in moderate difficulty across multiple classes. We report average precision (AP) for bird's bird's-eye view (APBEv) ) / 3D (AP:d) of car. pedestrian, and eyclist with IoU threshold set 0.7, 0.5, and 0.5 respectively. We indicate the best adaptation result by bold, and the highlighted row represents the proposed method.
Task
Method
Car
Pedestrian
Cyclist
No Adapt.
27.48
46.29 43.13
4 443.84
SN
Wang et al.
2020b
78.96 59.20
53.72/ 50.44
44.61 / /4144
ST3D
Yang et al.
202
82.19 61.83
52.92 48.33
53.73 / 46.09
Oracle
83.29 / 73.45
46.64 41.33
62.92 60.32
Waymo KITTI
Tent (Wang al. 202
65.09 30.12
46.21/43.24
46.59/41.72
CoTTA
Wang et al.
2022a
67.46/ 35.34
49.02 44500
55.31 50.07
SAR Niu et al.
2023)
65,81 / 30.39
47.99/ 44.72
47.14/41.17
MemCLR R VS
2023
65.61 / 29.83
48.25 44.84
47.49 /4.67
MOS
81.90/ 64.16
50.25 46.51
54.12/
No Adapt.
51.84/ 17.92
34.57
17.61 11.17
SN Wang et al.
2020b
40.03 / 21.23
38.91/34.355
11.11/5.67
ST3D
Yang et al.
2021
75.94 54.13
44.00 42.60
29.58 21.21
Oracle
83.29 / 73.45
46.64/41.33
62.92 60.32
nuScenes + KITTI
Tent (Wang et al. 2021
46,90/ / 18.83
39.69 / 35.53
19.64 10.49
CoTTA
Wang et al. 2022a
68.81 / 47.61
40.90 36.64
20.68 13.81
SAR
Niu et al.
2023)
65,81 / 30.39
39.78 35.63
19.56 / 10.26
McmCLR (VS ct al.
2023
65.61 / 29.83
40.31 / 35.14
19.21 / 9.66
MOS
71.13 51.11
42.94 38.45
21.65 18.16
A.2 ADDITIONAL TRANSFER TASK
To further assess the effectiveness of the proposed MOS, we perform an additional challenging transfer task: from Waymo to NuScenes, with the APad results presented in Tab. 7 Notably, ST3D (ann et al. 202I), a multi-round UDA method, shows limited performance (i.e., 20.19), likely due to the significant domain gap (c.g., differences in beam numbers). Despite this challenge, the proposed MOS surpasses all TTA baselines, achieving state-of-the-art performance.
Table 7: Results (APıo at moderate level difficulty) of TTA-3OD on adapting Waymo nuScenes using SECOND.
Adapt.
ST3D
Tent
CoTTA
SAR
McmCLR
Ours
17.24
20.19
17.31
17.35
17.74
18.22
18.75
A.3 DISCUSSION ABOUT RELEVANT MODEL ENSEMBLE METHOODS
Gao et al. (2022) utilizes gradient information and parameter space advancements during checkpoint averaging. However, this method is infeasible for TTA-3OD due to noisy gradients from the lack of ground truth labels. Additionally, computation adaptation. Also, this work (Gao et al., (022)) requires optimization on development data (val set), which is unavailable during test-time adaptation. In contrast, our MOS measures output differences between checkpoint pairs, ensuring diverse knowledge without relying on gradients or development data.
gradient is expensive, hindering efficient
Kaddour 2022) examines EMA, varying the latest checkpoints (ckpt) and saving frequencies. We investigated the same ensemble strategy using the latest ckpt, shown in Fig. of the main paper. The "latest-first" strategy aggregates the most recent K ckpt. Our MOS strategy selecting the 3 most important ckpt, outperforms the ensemble of the latest twenty ckpt. This improvement arises because recent ckpt converge, leading similar parameters and loss of long-term unique knowledge, ultimately failing to generalize to the target domain. Our MOS effectively mitigates this issuc.
Matena & Raffel (2022) presents merging procedure using the Laplace approximation, approx- imating each model's 's posterior as a Gaussian distribution with the precision matrix as its Fisher information. This approach is infeasible for the TTA-3OD task. as it necessitates per-example data to estimate the Fisher matrix, which is unavailable in online TTA since the test set not provided in advance.
20

---

Published as a conference paper at ICLR 2025
Table 9: TTA-3OD results (easy/moderatc/hard AP3)) of pedestrian class under the cross-corruption
scenario (Waymo + KITTI-C) at heavy corruption level.
Adaptation
Wang p022
CoTTA (ang
SAR
Niu p025
McmCLR
VSS
20123
MOS
Fog
40488/26
1823.6
2.22226 68/33.0
29/36 69/34.0
94/23,
1126-02223777
32,38277 51/24,85
Wet.
19.1044
44841,24 74
49.13/44 58/01 1555
89.1 L4445 01/42.22
19.0984L5554.8 81
$1, 34/44446.. 85/45 16
Snow
47.22/22 26/99119 19
47,55442.79/23 79//99.99
66. 30/4 62/38.
47.42/22 85/39,54
6.68822 (6642.78299. A5
49, 2...464.82240 82/40
Moe.
27.18/25 02/23.20 29
27.47/25 25/23.43
27 27.28/25. 43/23.43
27:34/25 15/23.31
27, 44/25 19223 36
27.54/25 SL5/33 256
Brom
32.42/27 8W/5. 27
14, 50.00 55/28 18
32 22/22/7 41/25 13
34.83/00.7422854
54, 53/530 50778
54. 02770, 75/37 35
CrossI.
47 47.42433 08:90.37
47.86443 37/80.51
47. 47.76/43 29440 39
(5.133/33 65/40.71
47,87743. 58440.48
50, 70,75455 42/41.73
Inc.
89 28/54, 79/42.21
49.15444 (04...0002211 [1
89 36.66/55.. 39442. 77
49 29.22/644 7044224 24
49.01444 23.0144476442.11
$1. 91.66668834455. 3.31
CrossS.
22.46/18.4 40/76.08
37.70/22.52/033 82/23130
22.11/17,8511..00
27,99/23 2M/21.36
27/25/2377021.66
26, 74/22, (44/21.89
Mean
0820/340001447
19.3005.11/22448
38.18/440901522
39.35/35, 10/32.67
19,1725.11/32.0
40.41/362663324
Wortsman et al. (2022) supports our findings that averaging weights of models with different hyperpa- rameters improve accuracy and robustness. However, doesn't 't consider model weighting based on checkpoint importance and uniqueness, unlike our proposed MOS. Additionally, the "learned soup" and "greedy soup" methods in paper (Wortsman al., 2022) require a held-out validation set to compute mixing coefficients, making them unsuitable for test-time scenarios.
In summary, all the aforementioned papers focus on multi-epoch offline training, potentially requiring labels and validation sets, and neglect the estimation and selection of important ckpt for ensemble. In contrast, our ensemble method is uniquely designed for online Test-Time Adaptation (TTA) in 3D object detection.
A.4 IMPLEMENTATION DETAILS
Our code is developed on the OpenPCDet (Team 2020 point cloud detection framework, and operates on a single NVIDIA RTX A6000 GPU with 48 GB of memory. We choose batch size of 8, and set hyperparameters L = 112 across all experiments. We set the model bank size of K = 5 to balance performance and memory usage. For evaluation, we use the KITTI benchmark's official metrics, reporting average precision for car class in both 3D (i.e., AP:D) and bird's eye view (i.e., AP over 40 recall positions, with 0.7 IoU threshold. We set Sreat to a small positive value <= 0.01 once r-nk(-) reaches D, to ensure G is invertible. The detection model is pretrained using the training set of the source dataset, and subsequently adapted and tested on the validation set of KITTI.
Augmentations. We adopt data augmentation strategies from prior studies (Yang et al. 2022 2021 Luo et al. 202T Chen et al. 2023a) for methods requiring augmentations, such as MemCLR (VS) et 20.3). CoTTA [Wang et al. (202a)), and MOS. While CoTTA suggests employing multiple augmentations, our empirical results indicate that for TTA-3OD, utilizing only a single random world scaling enhances performance, whereas additional augmentations diminish it. Consequently, following the approach (Luo et al. [202]] we implement random world scaling for the mean-teacher baselines, applying strong augmentation (scaling between 0.9 and 1.1) and weak augmentation (scaling between 0.95 and 1.05) for all test-time domain adaptation tasks.
Pseudo-labeling. We directly apply the pseudo-labeling strategies from (Yang et al. 2021 CoTTA and MOS for self-training, using the default configurations.
2022
Baseline Losses. For Tent (Wang al. 2021 and SAR (Niu al. 2023), which calculate the entropy minimization loss, we sum the losses based on classification logits for all proposals from the first detection stage. For MemCLR (VS et al. 2023), we integrate its implementation into 3D detectors by /odding/writing pooled region of interest (Rol) features extracted from the second detection stage, and compute the memory contrastive loss. For all baseline methods, we use default hyperparameters from their implementation code.
Ensemble Details and Memory Optimization in MOS. the proposed MOS, we optimize memory usage by sequentially loading each checkpoint from the hard disk onto the GPU, extracting the output intermediate feature maps and box predictions needed for calculating the gram matrix and synergy weights. After saving the outputs for the currently loaded checkpoint, we remove it from the GPU before loading the next one. Once the synergy weights are calculated, in the model assembly process, we again sequentially load checkpoints, multiplying each model's parameters by the corresponding synergy weight and aggregate them into the supermodel. We report the optimized memory usage in Fig. 7
21

---

Published as a conference paper at ICLR 2025
Table 10: TTA-3OD results (casy/moderate/hard AD of cyclist class under the corssiconr
scenario (Waymo + KITTI-C) at heavy corruption level.
Adaptation
Wang 2012
CoTTA (ang
SAR
Niu et 2025
MemCLR
(3
20123
MOS
Fog
21,15/17 91/16 66
33.62/19
22 60/61/8 74/17.57
23,40/19
25.62/21.34200
Wet.
00.36699.00 61477.0 20
39.72 29.72/48. 57455
61. 11.36/9,,222 27147 .044
57,43/66 88443.96
57,76466 34M4.75
64.7./52.1144 60
Snow
18.87450.3 3787790.37/99.00 96
22,8//42225001 1I
22.1141,55/19,0
52.06/51 89/39 26
52,26M11 71:29 28
S7, 12,42 5343,
Moe.
34,62/99 25/27.33 33
36.7W299 04/27.31
10.37/51.1//99.3 34
37.53/29 78/28.
38. 15.19/29. 75:28. 19
4.3 63/32,57
Brom
32.48/22.44 42/21 26
16,08/25 03/2388
30.89/11 37/30.43
37 16/66/6666 21/24.74
56. 34/35 35:24.52 50
43,06/000 A3
CrossT.
59 29.56/08,33 75/46.20
58.72/49 26/06.51
62.14449. 13446 21
58,40 38.06/99 16/46.6]
58, 15.65448 57A16. 40
65, 1551156677999 30
Inc.
59 (9.62199000 03446.82
59.14447 S7/05. LI
59.89447.6229 .4
58.36/77 93445.51
58, (5.91188.88 28-15 688
69, 00.55449.57/47..2 57/47 23
CrossS.
18.38/11 40170.93
04.84415.04/44666
21.98/2.77122.26
26.19/15 2W114.5
25, 13.4611..067744449
30.34/18.66/17773
Mean
41 1188633,9001.79
5369674
1
43,8933,5022.17
43.9/34447/52.66
43.6874.3032.63
48 48,80037 99636996666 06
Algorithm 1 The Proposed Model Synergy for TTA-3OD
Input: fo: source pretrained model, {.t ) point clouds to test, F: model bank.
Output: fr: model adapted to the target point clouds.
Phase 1: Warm-up.
Start self-training with fo. and add each of trained f, into F until reaching capacity.
Phase 2: Model Synergy.
for each remaining batch 2; do
Calculate the inverse of generalized Gram matrix G-1, via Eq. Eq. Eq. Calculate synergy weights with G -- for each of model in via Eq. Assemble models within F weighted by w, into a super model fr, via Eq.
8 (9. Eq.
10
6
Generate prediction for I; pseudo label B' by ft. to train the current model fc, via Eq. Phase 3: Model Bank Update.
for each time of training/inferring L online batches do
Calculate the mean synergy weights w, via Eq. (II Remove the most redundant model in F and add Je into F, via Eq. 12)
end for end for
A.5 EXPERIMENTAL RESULTS ACROSS MULTIPLE CLASSES
Cross-Dataset Shifts. We conduct experiments to evaluate the proposed test-time adaptation method across various datasets for multiple object classes. As shown in Tab. 8 our proposed MOS consistently surpasses all test-time adaptation (TTA) baselines in APBEY for every class. Specifically, in the Waymo + KITTI task, MOS exceeds the leading baseline (CoTTA) by 3.3% for pedestrians and 2.6% for cyclists in AP3D- For the more challenging nuScenes + KITTI adaptation, which faces significant environmental shift (beam numbers: 32 vs. 64), MOS significantly outperforms COTTA by 4.9% for pedestrians and 31.5% for cyclists, highlighting CoTTA's limitations with more severe domain shifts. Notably, MOS also surpasses the unsupervised domain adaptive (UDA) 3D object detection method (ST3D) by 11.4% in the cyclist class for the task of Waymo - KITTI, demonstrating its effectiveness across all object classes and competitive performance against the UDA approach.
Cross- Cooss-Corruptio Shifts. Our experiments further explore adapting to 3D scenes under the chal- lenging cross corruptontttion shifts for pedestrian and cyclist classes. As shown in Tab. [ our MOS method surpasses all baselines across most corruptions, achieving the highest mean APıD: 40.41, 36.26 and 33.24 in easy, moderate and hard difficulty, respectively. However, for motion blur, beam missing, and cross sensor, the effectiveness of pseudo-labelling methods such as CoTTA and MOS is reduced. This is due the small size and rareness of pedestrian objects, which lead to less accurate pseudo labels. For cyclists, which are slightly larger objects than pedestrians, CoTTA improves AP3D in motion blur scenarios (CoTTA's 31.18 vs. MemCL.R's 29.75) but still struggles with harder corruptions including cross sensor and beam missing, as indicated in Tab. 1O] Conversely, MOS effectively addresses both cross sensor and beam missing challenges for cyclists, demonstrating supe- rior results: 14.9% and 18.6% higher than the best-performing baseline, respectively. Despite pseudo labelling's 's limitations with rare and small-sized objects under severe corruptions, MOS presents a comparable test-time adaptation ability for pedestrian objects and shows the leading performance for cyclists, demonstrating the effectiveness of leveraging diverse category knowledge from historical checkpoints.
22

---

In Fig. we visualize the predicted point clouds from different checkpoints in bank F of size Published as a conference paper at ICLR 2025
Table 11: Results TTA-3OD on adapting Waymo - KITTI using PV-RCNN (Shi et 2020) and
DSVT (Wang et al. 2023a). BEBE//APaD are reported. Note that, MemCLR requires 2nd stage Rol features, thus inapplicable to the one-stage DSVT.
Model
No Adapt
Tent
CoTTA
SAR
MemCLR
MOS
PV-RCNN
63.60/22.01
55.96/27.49
67.85/38.52
59.77/21.33
55.92/15.77
72.60/52.45
DSVT
65.06/27.14
63.94/31.07
66.63/34.51
66.12/37
+
77.38/55.41
A.6
VISUALIZATION OF SYNERGY WEIGHTS
K 3, each annotated with its synergy weight (SW) by Eq. 5 We note that CKPT 2 detects more pedestrians (row 2, column 2) that are overlooked by others. To capture this knowledge, the proposed MOS assigns CKPT 2 with a higher SW when inferring the right point cloud (column 2). However, for inferring the left point cloud (column 1), CKPT 2 fails to localize the car detected by CKPT 3, resulting in CKPT 3 receiving higher SW than CKPT 2. Regarding CKPT 1, since other checkpoints cover all its box predictions, is assigned the lowest SW minimize redundancy. Therefore, for each input point cloud, the proposed MOS employs separate strategy (i.e., weighted averaging by SW) combine checkpoints, ensuring the acquisition of non-redundant, long-term knowledge from historical checkpoints.
CKPT 1: 0.058
CKPT 1: 0.026
CKPT 2: 0.351
CKPT 2: 0.725
CKPT 3: 0.591
CKPT 3: 0.249
B
UR
Figure 8: Visualization of box predictions from different checkpoints in the model bank of size K 3, with corresponding synergy weight. These checkpoints are selected and stored the model bank with the proposed MOS when adapting the 3D detector (Yan et al. (018)) pretrained on Waymo (Sun et al. 2020). to the KITTI (Geiger al. (2012) dataset at test-time. Objects uniquely detected by only one checkpoint are marked in red, while those detected by two checkpoints are marked in yellow.
23

---

Published as a conference paper at ICLR 2025
SAR
COTTA
MOS
Figure 9: Visualization of box predictions from baseline methods and the proposed MOS, when adapting from Waymo (Sun et al. 2020) to KITTI (Geiger et al.] 2012) at test time. Predicted boxes are shown red, while ground truth boxes are shown in blue.
A.7 SENSITIVITY TO 3D BACKBONE DETECTOR
We evaluate the sensitivity of the proposed MOS when integrated with different two-stage, point- and voxel-based backbone detector: PVRCNN (Shi et al., 2020). and a recently proposed, one-stage, voxel transformer based 3D detector DSVT (Wang et 2023a.. We report the results of TTA baselines and the MOS for test-time adapting the PVRCNN and DSVT from Waymo to KITTI in Tab. [ It is observed that MOS consistently outperforms the leading baseline, achieving substantial improvements 36. 16% and 53.48% in AP l for PVRCNN and DSVT, respectively. This highlights that our model synergy framework is model-agnostic, maintaining its effectiveness across different 3D detectors.
A.8 QUALITATIVE STUDY
In this section, we present visualizations of the box predictions generated by three different test-time adaptation (TTA) methods: 1) the optimization-basel method SAR, the mean-teacher-based
24

---

Published as a conference paper at ICLR 2025
method CoTTA, and 3) the proposed MOS. The image at the botton is included for reference only. It evident that both SAR and CoTTA exhibit false negatives, misclassifying background regions as objects. Additionally, even the correctly predicted boxes from SAR and CoTTA poorly aligned with the ground truth. In contrast, the proposed MOS method produces no false negatives, and its predicted boxes show a high degree of overlap with the ground truth, resulting in higher Intersection over Union (IoU) score.
25


Reviews:
[{"review_id": "xddZp6ISOr", "reviewer": "Reviewer_rufS", "summary": "This paper introduces a novel approach for test-time adaptation in LiDAR-based 3D object detection: Model Synergy (MOS). The proposed method dynamically selects and assembles historical checkpoints to create a composite \"super model\" that adapts to domain shifts, including cross-dataset and corruption shifts. The assembly of checkpoints is controlled by Synergy Weights (SW) module, which minimizes redundancy by leveraging output-level and feature-level similarities. The method addresses practical scenarios like cross-corruption shifts, where both dataset and environmental discrepancies occur simultaneously. Experimental results across multiple datasets, including KITTI, Waymo, and KITTI-C, demonstrate that MOS outperforms SOTA test-time adaptation methods, achieving considerable improvements in both standard and complex scenarios.", "questions": "Most questions are mentioned in the weakness section above.\nWhether MOS can be generalized to other types of data, such as inputs combining LiDAR point clouds and camera images?\nHow checkpoints are selected and what \"unique insights\" would improve understanding of the model selection process?\nHow does the computational cost of calculating the synergy weights scale with the size of the model bank and batch size?\nAlso, I'm curious about if there's any evaluation of the impact of the proposed method on adaptation scenarios involving different types of LiDAR sensors or point cloud resolutions.\nI appreciate the response to these questions.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "The paper introduces a novel weighted ensemble of historical checkpoints approach, that helps mitigate catastrophic forgetting while adapting models to unseen domain shifts. This method achieves SOTA performance across multiple datasets and scenarios. \nThe proposed method also effectively reduces memory usage by dynamically updating the model bank, retaining only key models with high synergy weights, thus maintaining performance with low resource cost", "weaknesses": "The framework proposed by this paper is generally novel and effective, however, I still have some questions about the approach:\n1. The paper demonstrates strong results (outperforms the SOTA methods) for LiDAR-based 3D object detection but does not extend its findings to other modalities or sensors. I'm interested in whether MOS can generalize to other types of data, such as inputs combining LiDAR point clouds and camera images.\n2. While the paper contains empirical results for the chosen similarity metrics, more theoretical analysis for determining the synergy weights could be more thoroughly discussed.\n3. It would be better if the authors could provide more clarity on how checkpoints are selected and what \"unique insights\" would improve understanding of the model selection process, I may not quite follow what's the insights here.\n4. The rank-based feature-level and output-level similarity functions calculate the inverse Gram matrix, this method is novel but seems to be computational heavily. The paper provides some analysis of the MOS method in the last section, but it'll be great to have more discussion on how these similarity metrics scale with larger datasets or different types of models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CFRlwUenlY", "reviewer": "Reviewer_gqhp", "summary": "The paper explores the test-time adaptation for LiDAR-based 3D object detection and proposes a model synergy approach that leverages long-term knowledge from historical checkpoints to tackle this task. Experiments are shown to verify the approach by tested against existing test-time adaptation strategies across three datasets.", "questions": "To maintain feature diversity, the method assigns higher weights to checkpoints with unique insights. This may lead to more false positive results; how does the method address this issue?\n\nDoes improving accuracy at the cost of time and computational resources go against the original intent of test-time adaptation?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-The paper focuses on exploring test-time adaptation for 3D object detection and proposes a natural and effective model synergy strategy that dynamically assigns weights to historical checkpoints and aggregates knowledge to tackle task. \n-The paper analyzes and discusses the effectiveness of the synergy weights and makes task-specific modifications. \n-Experiments verifies the method's effectiveness under various conditions, outperforming comparison methods.\n-The paper is well-writing and easy to follow.", "weaknesses": "The theory on which the method is based has been considered in prior works. The paper makes minor modifications specific to the 3D object detection task.\nA few minor clarifications are required in approach introduction， I have elaborated in the following section", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZVCWYH8EVa", "reviewer": "Reviewer_REaw", "summary": "This work proposes MOS, a framework that dynamically weights a sequence of models to create a more robust model based on the richness of new knowledge. MOS is evaluated in a test-time adaptation setting for LiDAR-based 3D object detection tasks. Experiments conducted in cross-dataset, cross-environment and cross-dataset&environment settings demonstrate the effectiveness of MOS.", "questions": "- From Figure 7, the reviewer notes that both the time and memory costs of MOS are increased. Do the authors have any ideas for optimizing this additional cost?\n- The reviewer wonders about the average time cost of conducting one experiment.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The concept of weighting checkpoints based on newly acquired knowledge is intriguing.\n- The cross-dataset &environment setting can positively impact this field.\n- This work is well-supported by experiments and ablation studies.", "weaknesses": "- Additional visualization of detection results is needed, particularly a comparison between methods that do not use new-knowledge-based model aggregation and MOS.\n- From Table 9, the reviewer observes that the pedestrian detection results are unsatisfactory, and from Table 8, the reviewer notes that the cyclist detection results could be improved. This could benefit from adding 2D supervision, as rich contextual and color information in 2D images can facilitate the detection of small objects. A similar approach is suggested in recent work: Approaching Outside: Scaling Unsupervised 3D Object Detection from 2D Scene (ECCV '24). The authors are encouraged to include relevant references and discussions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)