Title: CodeMerge: Codebook-Guided Model Merging for Robust Test-Time Adaptation in Autonomous Driving
Venue: NeurIPS 2025
Accepted: yes
Keywords: test-time adaptation

autonomous driving
Abstract: Maintaining robust 3D perception under dynamic and unpredictable test-time conditions remains a critical challenge for autonomous driving systems. Existing test-time adaptation (TTA) methods often fail in high-variance tasks like 3D object detection due to unstable optimization and sharp minima. While recent model merging strategies based on linear mode connectivity (LMC) offer improved stability by interpolating between fine-tuned checkpoints, they are computationally expensive, requiring repeated checkpoint access and multiple forward passes. In this paper, we introduce CodeMerge, a lightweight and scalable model merging framework that bypasses these limitations by operating in a compact latent space. Instead of loading full models, CodeMerge represents each checkpoint with a low-dimensional fingerprint derived from the source model’s penultimate features and constructs a key-value codebook. We compute merging coefficients using ridge leverage scores on these fingerprints, enabling efficient model composition without compromising adaptation quality. Our method achieves strong performance across challenging benchmarks, improving end-to-end 3D detection 14.9\% NDS on nuScenes-C and LiDAR-based detection by over 7.6\% mAP on nuScenes-to-KITTI, while benefiting downstream tasks such as online mapping, motion prediction and planning even without training. The code is released at \url{https://github.com/UQHTy/CodeMerge}.

CodeMerge: Codebook-Guided Model Merging for Robust Test-Time Adaptation in Autonomous Driving

Abstract

Maintaining robust 3D perception under dynamic and unpredictable test-time conditions remains a critical challenge for autonomous driving systems. Existing test-time adaptation (TTA) methods often fail in high-variance tasks like 3D object detection due to unstable optimization and sharp minima. While recent model merging strategies based on linear mode connectivity (LMC) offer improved stability by interpolating between fine-tuned checkpoints, they are computationally expensive, requiring repeated checkpoint access and multiple forward passes. In this paper, we introduce CodeMerge, a lightweight and scalable model merging framework that bypasses these limitations by operating in a compact latent space. Instead of loading full models, CodeMerge represents each checkpoint with a low-dimensional fingerprint derived from the source model's penultimate features and constructs a key-value codebook. We compute merging coefficients using ridge leverage scores on these fingerprints, enabling efficient model composition without compromising adaptation quality. Our method achieves strong performance across challenging benchmarks, improving end-to-end 3D detection 14.9% NDS on nuScenes-C and LiDAR-based detection by over 7.6% mAP on nuScenes-to-KITTI, while benefiting downstream tasks such as online mapping, motion prediction and planning even without training. The code is released at https://github.com/UQHTy/CodeMerge .

1 Introduction

Real-world autonomous driving scenarios often encounter rapid and unpredictable environmental variations, such as sudden adverse weather conditions (e.g., fog, snow) or sensor malfunctions ( e.g., dropped frames, missing beams) arising from LiDAR and camera systems, as illustrated in Fig. 1. These abrupt disruptions momentarily render 3D perception modules partially or fully 'blind', propagating erroneous decision-making downstream and leading to severe safety hazards in the end-to-end autonomous driving (AD) pipeline. Consequently, a critical yet unresolved question emerges: Can perception models efficiently and robustly adapt onboard to address such unforeseen distributional shifts?

Test-time adaptation (TTA) offers a promising direction by enabling models to adapt online during inference. Prior TTA approaches typically handle shifts by aligning BatchNorm statistics [44, 34], enforcing consistency through data augmentations [46], or minimizing sharpness via adversarial perturbations [14, 34]. Nonetheless, when directly extending them to complex tasks such as 3D detection, these approaches often suffer from brittle optimization dynamics and fall into sharp local minima, which can lead to the loss of previously acquired generalization and the ability to cope with future task distributions [9].

LiDARCorruption

Bright

Fog

Motion

FrameLost

On-board

Detection

Dark

Snow

Quant

Crash

BeamMissing

Fog

WetGround

End-to-EndADSystem

LiDAR-basedDetector

Detect/Track

CrossAttention

Self-Attention

CrossAttention

Motion

Voxel Extractor

BEVLayer

Region Proposal

Classifier

Feature

Transfomer

k

Head

( :)

3DBox

Extractor

Mapping

Planning

米

LiDAR

》

Regressor

Transfomer

Head

FingerprintZ

Camera

EgoQuery

q

Recent studies improve long-term adaptation stability by leveraging model merging techniques [24] grounded in linear mode connectivity (LMC), which posits that models fine-tuned on different target samples but initialized from the same pretrained source model are 'linearly connected' in weight space. Thus, interpolating between such models has been shown to produce reliable pseudo-labels and alleviate model collapse issues in TTA [34]. Techniques such as Mean Teacher and its variants exponentially average the weights of past models, but often underutilize valuable diversity across past checkpoints. More recently, Model Synergy (MOS) [8] extends this idea by maintaining a buffer of TopK important checkpoints and dynamically merging them using synergy weights, computed via kernel similarities among each model's predictions of the current test sample. While effective, MOS incurs significant overhead from repeatedly checkpoint loading and performing K forward passes for calculating synergy weights, limiting its scalability in high-throughput driving applications.

In this work, we introduce a codebook-guided model merging (CodeMerge) approach for adapting 3D perception modules against various shifts at test time. The core idea is to represent each finetuned checkpoint Φ Θ ( t ) by a compact 'fingerprint' derived from the source model's penultimate activations. These fingerprints serve as keys in a model codebook, mapping to their corresponding checkpoint weights. Crucially, correlations in this low-dimensional fingerprint space reliably mirror those in the high-dimensional weight space (see Fig. 3), enabling informed merging decisions without loading full model parameters. CodeMerge employs ridge leverage scores to rank the informativeness of fingerprints, a technique theoretically linked to approximations of the inverse Hessian in the parameter space. This procedure needs memory that scales only with the fingerprint dimension and adds negligible latency, yet it lifts end-to-end 3D detection NDS by 14.9%, tracking AMOTA on the nuScenes-C corruption benchmark by 19.3%, and LiDAR-based detection 3D mAP by 7.6% on the challenging nuScenes-to-KITTI shift. These improvements seamlessly propagate to downstream motion prediction and planning modules without modification or additional training.

2 Preliminaries

We begin by formalizing the problem setting for test-time adaptation (TTA) in 3D object detection and reviewing model merging strategies that exploit linear mode connectivity in such context.

Task Formulation. Let Φ Θ (0) = ϕ Θ (0) ◦ h Θ (0) denote a pretrained 3D object detection model, comprising a feature extractor ϕ Θ (0) ( · ) : X ↦→ Z ∈ R d maps an input x ∈ X ( e.g., a point cloud or multi-view images) to a latent feature map z ∈ Z , and the head regresses 3D boxes h Θ (0) ( · ) : Z ↦→ Y ∈ R 7 . The goal of TTA is to sequentially adapt the model to a stream of unlabeled target-domain inputs D tar = { x t } T t =1 , which may exhibit significant distributional shifts or corruptions. The online adaptation must follow in a single forward-pass setting, incrementally evolving the model parameters Θ (0) → Θ (1) → . . . → Θ ( t ) to improve detection over time.

Linear Mode Connectivity (LMC). LMC [24, 12, 52] refers to the empirical property that two models Θ (1) and Θ (2) trained from a shared initialization (or sufficiently close regions in weight space), can be connected by a 'linear path' without significant loss degradation. Formally, for any

FingerprintZ

?

(0)

?

(0)

(0)

(t)

(t)

(t)

EMA

infer

MOS

CodeMerge

RLS merge

(ow-dimfingerpn

②

Model

2

?

CodeBook

Buffer

FingerprintZt-1

2

FingerprintZ2

λ ∈ [0 , 1] ,

This property facilitates efficient model merging through linear interpolation.

Implication for Model Merging in TTA. If LMC holds true between each pair of successive parameters (Θ ( t -1) , Θ ( t ) ) fine-tuned from Θ (0) , then their interpolated model should yield low loss. This underpins methods like Mean Teacher shown in Fig. 2, in which teacher models are recursively updated with an exponential moving average (EMA) with a decay factor β ∈ (0 , 1) :

Under LMC, this leads to approximately linear combinations of multi-task losses:

This shows that averaging can reduce variance from balancing multi-task losses. However, EMA's coefficients are solely based on time steps rather than model behavior, making it potentially suboptimal.

In contrast, MOS [8] (middle in Fig. 2) adaptively merges model parameters by solving a kernelweighted least squares problem over a buffer of K candidate checkpoints { Θ ( i ) } K i =1 . Given a test batch x t , the merged model is computed as:

where kernel matrix K ( t ) ∈ R K × K captures pairwise similarity between model outputs under the current batch. To evaluate ˜ w ( t ) i , MOS requires K forward passes over x t , making it more computationally intensive and thus hard to scale up the horizon K in TTA.

3 Our Approach

We introduce CodeMerge, a codebook-guided model merging scheme for efficient TTA in 3D object detection without triggering repeated inference across past models. To achieve this, we construct a model codebook (Sec. 3.1), where each checkpoint is represented by a compact fingerprint derived from intermediate features of a fixed source model. During inference, we compute curvature-aware ridge leverage scores (Sec. 3.2) in the fingerprint space. Finally, we perform a sign-consistent weighted merge of top-scoring candidate models (Sec. 3.3), promoting both stability and diversity.

3.1 Model CodeBook

At each step t , we maintain a model codebook for all past checkpoints along the adaptation trajectory, denoted as:

0.48

0.50

0.52

0.54

Pairwise Fingerprint Difference

0.85

0.90

0.95

1.00

1.05

Pairwise Parameter Difference

r = 0.902

= 0.867

SECOND: Waymo

KITTI

0.36

0.38

0.40

0.42

0.44

Pairwise Fingerprint Difference

0.97

0.98

0.99

1.00

Pairwise Parameter Difference

r = 0.862

= 0.676

SECOND: nuScenens

KITTI

0.47

0.48

0.49

0.50

0.51

0.52

Pairwise Fingerprint Difference

0.00

0.25

0.50

0.75

1.00

1.25

1.50

Pairwise Parameter Difference

r = 0.821

= 0.771

SparseDrive: Snow Corruption

0.48

0.50

0.52

0.54

Pairwise Fingerprint Difference

0.0

0.5

1.0

1.5

Pairwise Parameter Difference

r = 0.840

= 0.771

SparseDrive: Motion Corruption

Each entry is a key-value pair, where the key ˆ z i ∈ R d ′ is a low-dimensional fingerprint and the value Θ ( i ) is the corresponding checkpoint fine-tuned at time step i . To compute the key ˆ z i , we extract intermediate features from the i -th input batch x i using a pretrained feature extractor ϕ Θ (0) and randomly project them to a low-dimensional subspace for efficiency:

Here, RandProj( · ) : R d ↦→ R d ′ is implemented via a fixed Gaussian projection matrix where d ′ ≪ d ensures the keys are compact. As the test-time adaptation progresses, we update the codebook incrementally by appending new pairs, i.e., C ( t +1) ← ( ˆ z t , Θ ( t ) ) .

3.2 Curvature-Aware Merge Scores

To determine which checkpoints in the codebook should be merged at time step t , we first compute a merge score for each checkpoint Θ ( i ) ∈ C ( t ) using the ridge leverage score.

Definition 1 (Ridge Leverage Score (RLS)) . Let ˆ Z t -1 = [ˆ z 1 , . . . , ˆ z t -1 ] ∈ R ( t -1) × d ′ be the matrix of all stored keys (fingerprints), where ˆ z i be the fingerprint of the i -th candidate model Θ ( i ) . We define the ridge leverage scores of the fingerprint ˆ z i as

where λ is a regularization parameter. A high leverage score indicates ˆ z i is influential and lessly observed within the current feature space defined by past direction.

Theoretical Analysis. We now connect this leverage score to the inverse of curvature through the lens of LMC. We begin by revisiting the LMC assumption (Eq. (1)) through a second-order Taylor expansion around Θ (0) :

where δ i := Θ ( i ) -Θ (0) refers the model update direction and H is the Hessian at Θ (0) . In this view, the curvature along δ i is quantified by the quadratic term δ ⊤ i Hδ i . Its inverse δ ⊤ i H -1 δ i suggests δ i explores a novel region of the loss landscape, making it an indicator for selecting diverse checkpoints.

However, computing the full Hessian in high-dimensional parameter space is impractical, especially in TTA tasks. However, considering that 3D object detection models commonly use linear layers as final regression heads, we can effectively analyze curvature through the simpler and analytically tractable ridge regression setting. Specifically, assume a linear regression head parameterized by weights w ∈ R d and a fixed feature extractor ϕ ( · ) , yielding a ridge regression objective of the form:

where H w is Hessian matrix in parameter space. More precisely, this reveals the inverse of parameterspace curvature is linked to the proposed ridge leverage score under the low-rank surrogate ˆ Z ⊤ t -1 ˆ Z t -1 :

Empirical analysis (see Fig. 3) confirms that fingerprint vectors strongly correlate (Pearson correlation and Kendall Tau scores often exceeding 0.7) with parameter deltas, confirming that the geometry of fingerprint space reliably mirrors that of parameter space.

3.3 Model Merging

To perform stable model merging, we select topK high-scoring checkpoints based on ridge leverage scores, yet their associated parameter directions may exhibit destructive interference. To resolve such conflicts, we adopt a sign-consistent merging inspired by [54], which aligns model parameters based on majority sign consensus before merging. Let { Θ ( i ) } K i =1 denote the topK selected checkpoints and { s ( t ) i } K i =1 their corresponding leverage scores. For each parameter dimension j , we compute the majority sign sign maj ( j ) := mode( { sign(Θ ( i ) j ) } K i =1 ) , and zero out inconsistent components. The merged model is then given by:

where ⊙ denotes element-wise multiplication, and I [ · ] is a binary mask that retains only parameters aligned with the majority sign. This sign-consistent merge ensures coherent parameter updates and stabilizes adaptation under distribution shifts.

Optimization. Following the protocol in [8], we use the merged model to generate pseudo-labeled bounding boxes for self-training the LiDAR-based detector online. In realistic end-to-end AD systems (see Fig. 1), perception, mapping, and planning modules are often integrated into a monolithic architecture. For efficiency, we freeze all components except for the 3D box regression head. Experiments show that CodeMerge not only improves detection performance but also yields gains in downstream mapping and planning without requiring additional training or modifications (Table 2).

4 Experiments

4.1 Experimental Setup

Datasets and Tasks. We conduct comprehensive experiments across five benchmarks for end-to-end autonomous driving and outdoor 3D object detection: KITTI [13], KITTI-C [25], Waymo [40], nuScenes [4], and nuScenes-C [53]. For test-time adaptation in end-to-end autonomous driving, we pre-train models on the nuScenes driving benchmark and adapt them to eight real-world corruptions in nuScenes-C: Motion Blur (Motion), Color Quantization (Quant), Low Light (Dark), Brightness (Bright), Snow, Fog, Camera Crash (Crash), and Frame Lost. For LiDAR-based 3D object detection, we first tackle cross-dataset adaptation (Waymo → KITTI, nuScenes → KITTI) following [56, 57, 6], addressing both object-level shifts ( e.g. , scale and point density) and environmental differences ( e.g. , deployment location, beam configuration). We then evaluate adaptation to sensor failures and weather effects via KITTI → KITTI-C, covering Fog, Wet Conditions (Wet.), Snow, Motion Blur (Moti.), Missing Beams (Beam.), Crosstalk (Cross.T), Incomplete Echoes (Inc.), and Cross-Sensor (Cross.S). The detailed evaluation metric and implementation details can be found in Appendix A.1.

Baselines. We compare the proposed CodeMerge against a broad range of methods: (i) No Adapt. , the pretrained model evaluated directly on the target datasets; (ii) SN [48], a weakly supervised DA technique that rescales source objects using target size statistics; (iii) ST3D [56], the first UDA method for 3D detection, employing multi-epoch self-training with pseudo labels; (iv) Tent [44], an TTA approach that minimizes prediction entropy; (v) CoTTA [46], which combines mean-teacher supervision with stochastic augmentations for TTA ; (vi) SAR [34], enhancing Tent by sharpness-aware and reliability-aware entropy minimization; (vii) MemCLR [43], the first online TTA method that uses memory-augmented mean-teacher for 2D detection; (viii) Reg-TTA3D [58], which regularizes 3D box regression by enforcing noise-consistent pseudo labels during 3D TTA ; (ix) MOS [8], dynamically fusing a bank of topK checkpoints through kernel-based synergy for 3D TTA ; (x) DPO [9], flattening the test-time loss landscape via dual perturbations for 3D TTA ; (xi) Oracle , a fully supervised model trained with annotated target datasets.

4.2 Main Results and Analysis

TTA on End-to-End Autonomous Driving. We comprehensively evaluate our CodeMerge method on nuScenes-C [53] with the end-to-end SparseDrive model [41], covering five downstream tasks: 3D detection, multi-object tracking, online mapping, motion prediction, and trajectory planning under diverse corruptions. Table 1 shows CodeMerge consistently outperforms all baselines, including No Adapt, Tent, and the state-of-the-art MOS [8] in averaged results. In 3D detection, we boost mAP by 33.6% over no adaptation (0.1747 → 0.2334) and by 13.3% over MOS. CodeMerge also reduces mASE by 4.4% relative to MOS, and lowers mAVE by 19%. Under the Bright corruption, CodeMerge improves mAP by 11.9% over no adaptation, with consistent gains in other metrics. In multi-object tracking, CodeMerge improves AMOTA by 19.3%, reduces AMOTP by 13.8%, and raises recall by 16.5% when compared with the SOTA baseline, MOS. Notably, under the most safety-critical Lost scenario, the proposed method achieves the highest recall (0.1788) and lowest tracking error among all methods. Although only perception weights are adapted, downstream tasks benefit markedly. As reported in Table 2. CodeMerge increases online mapping mAP by 42.3% (0.2009 → 0.2859) over no adaptation, with +45.7% on lane boundaries and +39.5% on obstacles, especially +94.2% under Dark . For motion prediction, mADE and mFDE fall by 9.3% and 9.7% compared to no adaptation, respectively, while EPA (higher is better) rises by 13.8%. For planning, average lateral deviation falls 8.3% (0.7923 m → 0.7266 m) and collision risk drops 6.1% compared to no adaptation. These consistent gains achieved without touching non-perception modules, confirm

that the proposed lightweight, fingerprint-guided merging framework stabilizes the detector and unlocks robust performance across all autonomous driving tasks.

TTA on LiDAR-based Detection. We examine CodeMerge's performance in 3D object detection across two distinct types of domain shifts: Cross-dataset (Waymo → KITTI, nuScenes → KITTI) and Corruption-induced shifts (KITTI → KITTI-C). (1) Cross-dataset (Table 3). Compared with the nonadapted model, CodeMerge lifts APBEV by 25.1% and AP3D by 141% on Waymo → KITTI, closing 108.5%/84.5% of the domain gap and even surpassing the multi-epoch

ST3D and fully supervised Oracle in APBEV. On nuScenes → KITTI, it narrows the gap by 81.3%/73.15%, again outperforming the strongest TTA baselines (MOS, DPO) and exceeding

ST3D by +1.9% APBEV and +8.1% AP3D. (2) Corruption-induced (Table 4). Against KITTI → KITTI-C corruptions, CodeMerge raises mean AP3D by +7.7% over no adaptation and +2.6% over the best prior TTA baseline. Under Fog and Wet corruption, gains are pronounced: +9.9% (75.96 vs. 69.11) and +2.6% (79.82 vs. 77.79), respectively, indicating enhanced resilience to visibility and environment degradations. These results demonstrate that our latent-space, fingerprint-guided merging not only closes cross-domain gaps more effectively than existing TTA methods but also surpasses dedicated domain adaptation approaches, providing robust performance across diverse and challenging environments.

4.3 Ablation and Sensitivity Study

Impact of Checkpoint Selection Strategy. In Table 5, we compare five strategies for choosing K = 5 checkpoints under heavy Motion Blur: Random sampling, Recent (the latest five), KMeans++ clustering in feature space, Exponential Moving Average (EMA) model, and our Leverage-score ranking. Random yields a reduced detection mAP of 0.2740, weaker tracking (AMOTA = 0.2152), and planning (CR-Avg = 0.1120). Recent performs worst across all tasks (mAP 0.2480, AMOTA 0.1866, CR-Avg 0.1490), indicating catastrophic forgetting when only the newest checkpoints are merged. KMeans++ yields a marginal 0.17% lift in NDS over Random and reduces collision risk by 6.3%, reflecting its ability to capture diverse feature modes. However, KMeans++ is still outperformed by the proposed method (-3.8% mAP for detection), highlighting that pure feature clustering cannot match the important informativeness captured by leverage-score ranking. Futhermore, CodeMerge ( K = 5 , projection dimension d ′ = 1024 ) consistently outperforms EMA across all tasks: Detection mAP/NDS +0.0373/+0.0272; Tracking AMOTA +0.0372 and AMOTP -0.0810 ( ↓ better); Mapping mAP/AP ped +0.0355/+0.0300; Motion mADE/mFDE -0.0147/-0.0301; Planning L2-Avg/CR-Avg -0.0274/-0.0160. These gains indicate that our fingerprint-guided merging, which selects complementary checkpoints, offers stronger generalization than the uniform, time-local smoothing performed by EMA. Overall, the proposed Leverage-score selection consistently achieves the best results by explicitly identifying the most informative, complementary checkpoints carrying long-term knowledge.

の日四巴

CAM_BACK

四

Go Straight

Go Straight

SRONLLE

CAM_BACK

四

Go Straight

Go Straight

Impact on Number of Merged Checkpoints. Table 5 compares selecting K =3 , 5 , or 9 checkpoints (with d ′ =1024 ) for model merging under Motion Blur corruption. With only K =3 , detection mAP drops from 0.2851 to 0.2655, and tracking AMOTA falls from 0.2241 to 0.2077, indicating insufficient coverage of knowledge diversity. Increasing to K =9 recovers much of this gap (mAP 0.2818, AMOTA 0.2195) but yields only marginal gains, in mapping mAP (0.4167 vs. 0.4103 at K =5 ). The near-parity between K =5 and 9 suggests redundant information beyond five checkpoints. Balancing performance and memory fingerprint, we thus adopt K =5 in all experiments.

Impact on Dimension of Random Projection. We additionally examine the effect of varying the random projection dimension d ′ among {256, 512, 1024, 2048}. As Table 5 shows, at d = 256 , the performance is only slightly below that of d ′ = 1024 (mAP 0.2749 vs. 0.2851; NDS 0.4176 vs. 0.4264), demonstrating that very compact fingerprints still capture most of the critical variability. At d ′ = 2048 , results nearly match the d = 1024 but at twice the memory cost. Therefore, d ′ = 1024 offers the best trade-off between performance and fingerprint.

Efficiency Analysis. We further compare the GPU memory and total TTA runtime of CodeMerge against MOS on both the SECOND and SparseDrive detectors. Table 6: GPU memory (MiB) and total TTA As reported in Table 6, with SECOND, MOS

consumes 17.4 GiB and requires 1,813 seconds per adaptation run, whereas CodeMerge uses only 16.0 GiB ( -8.0% ) and completes in 1,054 seconds ( -41.8% ). The savings are even more pronounced on SparseDrive: MOS demands 39.4 GiB and 37,619 seconds, while CodeMerge needs just 29.8 GiB ( -21.0% ) and 27,359 seconds ( -37.48% ). These gains arise from our fingerprint-guided merging, which projects each checkpoint into a compact embedding and computes leverage weights on the fly (requiring only one extra forward pass), rather than loading and forwarding K full models as MOS does. This design drastically reduces memory footprint and latency, making CodeMerge well-suited for real-time autonomous driving applications.

Quantitative Analysis. We visualize predictions with CodeMerge (top row) against the non-adapted SparseDrive baseline (bottom row) in Fig. 4 to illustrate how on-the-fly merging enhances every stage of the end-to-end pipeline. In detection and tracking, CodeMerge produces tight, correctly aligned 3D boxes, but the baseline suffers a large number of missed or misplaced detections (highlighted in red dashed circles). In mapping, our method reconstructs dense, straight-lane boundaries and curb lines, validating its ability to preserve semantic consistency. In contrast, the baseline yields sparse, crooked lanes and missing curbs (highlighted in purple circles/arrows), degrading map fidelity. Finally, CodeMerge's planned trajectory remains centered in the lane and safely avoids dynamic objects, while the baseline's path drifts toward the curb (highlighted in an orange arrow) and even intersects an oncoming track, demonstrating unsafe behavior. In summary, these qualitative results confirm that

leveraging compact fingerprints and leverage-score-guided merging yields better detections, more robust tracking, and safer trajectories under severe real-world corruptions.

5 Conclusion

In this work, we address the challenge of online adaptation to domain shifts for both LiDAR-based and vision-centric end-to-end AD detection under extreme conditions. Our proposed CodeMerge framework effectively mitigates cross-dataset and corruption-induced distribution shifts, while reducing GPU memory consumption and inference latency by approximately 27% compared to state-of-the-art TTA methods. Notably, other downstream modules, such as mapping and planning, receive performance improvements without task-specific fine-tuning due to enhanced detection outputs. However, this study represents an early attempt to address robustness in end-to-end AD, and major experiments have been primarily conducted on the SparseDrive architecture. The primary bottleneck remains that popular architectures, such as UniAD and VAD, experience over tenfold performance degradation on nuScenes-C, hindering effective adaptation training. Future work will investigate strategies to further accelerate adaptation and enhance robustness under dynamic driving conditions.

References

Samuel K. Ainsworth, Jonathan Hayase, and Siddhartha S. Srinivasa. Git re-basin: Merging models modulo permutation symmetries. In Proc. International Conference on Learning Representations (ICLR) , 2023.

Devansh Arpit, Huan Wang, Yingbo Zhou, and Caiming Xiong. Ensemble of averages: Improving model selection and boosting performance in domain generalization. In Proc. Annual Conference on Neural Information Processing (NeurIPS) , 2022.

Y-Lan Boureau, Jean Ponce, and Yann LeCun. A theoretical analysis of feature pooling in visual recognition. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel .

Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 11618-11628, 2020.

Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 295-305, 2022.

Zhuoxiao Chen, Yadan Luo, Zheng Wang, Mahsa Baktashmotlagh, and Zi Huang. Revisiting domain-adaptive 3d object detection by reliable, diverse and class-balanced pseudo-labeling. In Proc. International Conference on Computer Vision (ICCV) , pages 3691-3703, 2023.

Zhuoxiao Chen, Yadan Luo, Zixin Wang, Zijian Wang, Xin Yu, and Zi Huang. Towards open world active learning for 3d object detection. CoRR , abs/2310.10391, 2023.

Zhuoxiao Chen, Junjie Meng, Mahsa Baktashmotlagh, Yonggang Zhang, Zi Huang, and Yadan Luo. MOS: model synergy for test-time adaptation on lidar-based 3d object detection. In Proc. International Conference on Learning Representations (ICLR) , 2025.

Zhuoxiao Chen, Zixin Wang, Yadan Luo, Sen Wang, and Zi Huang. DPO: dual-perturbation optimization for test-time adaptation in 3d object detection. In Proc. ACM International Conference on Multimedia (MM) , pages 4138-4147, 2024.

Nico Daheim, Thomas Möllenhoff, Edoardo M. Ponti, Iryna Gurevych, and Mohammad Emtiyaz Khan. Model merging by uncertainty-based gradient matching. In Proc. International Conference on Learning Representations (ICLR) , 2024.

Marius-Constantin Dinu, Markus Holzleitner, Maximilian Beck, Hoan Duc Nguyen, Andrea Huber, Hamid Eghbal-zadeh, Bernhard Alois Moser, Sergei V. Pereverzyev, Sepp Hochreiter, and Werner Zellinger. Addressing parameter choice issues in unsupervised domain adaptation by aggregation. In Proc. International Conference on Learning Representations (ICLR) , 2023.

Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, and Michael Carbin. Linear mode connectivity and the lottery ticket hypothesis. In Proc. International Conference on Machine Learning (ICML) , volume 119 of Proceedings of Machine Learning Research , pages 3259-3269, 2020.

Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the KITTI vision benchmark suite. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3354-3361, 2012.

Taesik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottananurak, and Sung-Ju Lee. Sotta: Robust test-time adaptation on noisy data streams. In Proc. Annual Conference on Neural Information Processing (NeurIPS) , 2024.

Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and J Zico Kolter. Test time adaptation via conjugate pseudo-labels. In Proc. Annual Conference on Neural Information Processing (NeurIPS) , 2022.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. In Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part III .

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR , abs/1512.03385, 2015.

Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In Proc. International Conference on Learning Representations (ICLR) , 2023.

Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. VAD: vectorized scene representation for efficient autonomous driving. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023 , pages 8306-8316. IEEE, 2023.

Jincen Jiang, Qianyu Zhou, Yuhang Li, Xinkui Zhao, Meili Wang, Lizhuang Ma, Jian Chang, Jian Jun Zhang, and Xuequan Lu. Pcotta: Continual test-time adaptation for multi-task point cloud understanding. In Proc. Annual Conference on Neural Information Processing (NeurIPS) , 2024.

Ian T. Jolliffe. Principal Component Analysis . Springer Series in Statistics. Springer, 1986.

Sanghun Jung, Jungsoo Lee, Nanhee Kim, Amirreza Shaban, Byron Boots, and Jaegul Choo. Cafa: Class-aware feature alignment for test-time adaptation. In Proc. International Conference on Computer Vision (ICCV) , pages 19060-19071, 2023.

Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El-Saddik, and Eric P. Xing. Efficient test-time adaptation of vision-language models. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 14162-14171, 2024.

Byungjai Kim, Chanho Ahn, Wissam J. Baddar, Kikyung Kim, Huijin Lee, Saehyun Ahn, Seungju Han, Sungjoo Suh, and Eunho Yang. Test-time ensemble via linear mode connectivity: A path to better adaptation. In Proc. International Conference on Learning Representations (ICLR) , 2025.

Lingdong Kong, Youquan Liu, Xin Li, Runnan Chen, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Robo3d: Towards robust and reliable 3d perception against corruptions. In Proc. International Conference on Computer Vision (ICCV) , pages 19937-19949, 2023.

Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribution shifts. International Journal of Computer Vision , 133(1):31-64, 2025.

Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, and Xinggang Wang. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025 . Computer Vision Foundation / IEEE, 2025.

Yadan Luo, Zhuoxiao Chen, Zhen Fang, Zheng Zhang, Mahsa Baktashmotlagh, and Zi Huang. Kecor: Kernel coding rate maximization for active 3d object detection. In Proc. International Conference on Computer Vision (ICCV) , pages 18233-18244, 2023.

Yadan Luo, Zhuoxiao Chen, Zijian Wang, Xin Yu, Zi Huang, and Mahsa Baktashmotlagh. Exploring active 3d object detection from a generalization perspective. In Proc. International Conference on Learning Representations (ICLR) , 2023.

Yadan Luo, Zijian Wang, Zhuoxiao Chen, Zi Huang, and Mahsa Baktashmotlagh. Source-free progressive graph learning for open-set domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(9):11240-11255, 2023.

Muhammad Jehanzeb Mirza, Jakub Micorek, Horst Possegger, and Horst Bischof. The norm must go on: Dynamic unsupervised domain adaptation by normalization. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 14745-14755. IEEE, 2022.

Jacob Morrison, Noah A. Smith, Hannaneh Hajishirzi, Pang Wei Koh, Jesse Dodge, and Pradeep Dasigi. Merge to learn: Efficiently adding skills to language models with model merging. In Findings of the Association for Computational Linguistics (Findings of EMNLP) , pages 15604-15621, 2024.

Shuaicheng Niu, Chunyan Miao, Guohao Chen, Pengcheng Wu, and Peilin Zhao. Test-time model adaptation with only forward passes. In Proc. International Conference on Machine Learning (ICML) , 2024.

Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In Proc. International Conference on Learning Representations (ICLR) , 2023.

Haoxuan Qu, Xiaofei Hui, Yujun Cai, and Jun Liu. LMC: large model collaboration with cross-assessment for training-free open-set object recognition. In Proc. Annual Conference on Neural Information Processing (NeurIPS) , 2023.

Alexandre Ramé, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, Patrick Gallinari, and Matthieu Cord. Diverse weight averaging for out-of-distribution generalization. In Proc. Annual Conference on Neural Information Processing (NeurIPS) , 2022.

Dominik Scherer, Andreas C. Müller, and Sven Behnke. Evaluation of pooling operations in convolutional architectures for object recognition. In Artificial Neural Networks - ICANN 2010 - 20th International Conference, Thessaloniki, Greece, September 15-18, 2010, Proceedings, Part III , 2010.

Hajin Shim, Changhun Kim, and Eunho Yang. Cloudfixer: Test-time adaptation for 3d point clouds via diffusion-guided geometric transformation. In Proc. European Conference on Computer Vision (ECCV) , 2024.

Ziying Song, Caiyan Jia, Lin Liu, Hongyu Pan, Yongchang Zhang, Junming Wang, Xingyu Zhang, Shaoqing Xu, Lei Yang, and Yadan Luo. Don't shake the wheel: Momentum-aware planning in end-to-end autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025 . Computer Vision Foundation / IEEE, 2025.

Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2443-2451, 2020.

Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, and Sifa Zheng. Sparsedrive: End-to-end autonomous driving via sparse scene representation. In Proc. International Conference on Robotics and Automation (ICRA) , 2025.

Yun-Yun Tsai, Fu-Chen Chen, Albert Y. C. Chen, Junfeng Yang, Che-Chun Su, Min Sun, and Cheng-Hao Kuo. GDA: generalized diffusion for robust test-time adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 23242-23251, 2024.

Vibashan VS, Poojan Oza, and Vishal M. Patel. Towards online domain adaptive object detection. In Proc. Winter Conference on Applications of Computer Vision (WACV) , pages 478-488, 2023.

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno A. Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In Proc. International Conference on Learning Representations (ICLR) , 2021.

Haiyang Wang, Chen Shi, Shaoshuai Shi, Meng Lei, Sen Wang, Di He, Bernt Schiele, and Liwei Wang. DSVT: dynamic sparse voxel transformer with rotated sets. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023 , pages 13520-13529. IEEE, 2023.

Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 71917201, 2022.

Shuai Wang, Daoan Zhang, Zipei Yan, Jianguo Zhang, and Rui Li. Feature alignment and uniformity for test time adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 20050-20060, 2023.

Yan Wang, Xiangyu Chen, Yurong You, Li Erran Li, Bharath Hariharan, Mark E. Campbell, Kilian Q. Weinberger, and Wei-Lun Chao. Train in germany, test in the USA: making 3d object detectors generalize. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 11710-11720, 2020.

Yanshuo Wang, Ali Cheraghian, Zeeshan Hayder, Jie Hong, Sameera Ramasinghe, Shafin Rahman, David Ahmedt-Aristizabal, Xuesong Li, Lars Petersson, and Mehrtash Harandi. Backpropagation-free network for 3d test-time adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2024.

Zixin Wang, Yadan Luo, Liang Zheng, Zhuoxiao Chen, Sen Wang, and Zi Huang. In search of lost online test-time adaptation: A survey. International Journal of Computer Vision , 133(3):1106-1139, 2025.

Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proc. International Conference on Machine Learning (ICML) , volume 162, pages 23965-23998, 2022.

Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. Robust fine-tuning of zero-shot models. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 7949-7961, 2022.

Shaoyuan Xie, Lingdong Kong, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Benchmarking and improving bird's eye view perception robustness in autonomous driving. IEEE Transactions on Pattern Analysis and Machine Intelligence , 47(5):3878-3894, 2025.

Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , 2023.

Yan Yan, Yuxing Mao, and Bo Li. SECOND: sparsely embedded convolutional detection. Sensors , 18(10):3337, 2018.

Jihan Yang, Shaoshuai Shi, Zhe Wang, Hongsheng Li, and Xiaojuan Qi. ST3D: self-training for unsupervised domain adaptation on 3d object detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10368-10378, 2021.

Jihan Yang, Shaoshuai Shi, Zhe Wang, Hongsheng Li, and Xiaojuan Qi. St3d++: denoised self-training for unsupervised domain adaptation on 3d object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Jiakang Yuan, Bo Zhang, Kaixiong Gong, Xiangyu Yue, Botian Shi, Yu Qiao, and Tao Chen. Reg-tta3d: Better regression makes better test-time adaptive 3d object detection. In Proc. European Conference on Computer Vision (ECCV) , volume 15101, pages 197-213, 2024.

Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1592215932, 2023.

Longbin Zeng, Jiayi Han, Liang Du, and Weiyang Ding. Rethinking precision of pseudo label: Test-time adaptation via complementary learning. Pattern Recognition Letters , 177:96-102, 2024.

Shuangfei Zhai, Hui Wu, Abhishek Kumar, Yu Cheng, Yongxi Lu, Zhongfei Zhang, and Rogério Schmidt Feris. S3pool: Pooling with stochastic spatial sampling. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , 2017.

Marvin Zhang, Sergey Levine, and Chelsea Finn. MEMO: test time robustness via adaptation and augmentation. In Proc. Annual Conference on Neural Information Processing (NeurIPS) , 2022.

Bowen Zhao, Chen Chen, and Shu-Tao Xia. Delta: degradation-free fully test-time adaptation. In Proc. International Conference on Learning Representations (ICLR) , 2023.

Tianpei Zou, Sanqing Qu, Zhijun Li, Alois Knoll, Lianghua He, Guang Chen, and Changjun Jiang. HGL: hierarchical geometry learning for test-time adaptation in 3d point cloud segmentation. In Proc. European Conference on Computer Vision (ECCV) , 2024.

A Technical Appendices and Supplementary Material

We include additional technical details in the following appendices:

Section A.1 (Implementation Details): Describes the full experimental setup, including training schedules and hyperparameter configurations.

Section A.2 (Evaluation Metrics): Provides definitions and explanations for all evaluation metrics used across detection, tracking, mapping, and planning tasks.

Section A.3 (Additional Visualizations): Presents qualitative results and visual comparisons illustrating the adaptation performance of the end-to-end AD system under various distribution shifts.

Section A.4 (Related Work): Summarizes the relevant literature in TTA and model merging.

Section A.5 (Runtime): Reports the computational cost and per-frame latency of each TTA step.

Section A.6 (Random Projection): Evaluates the determinism and robustness of our random projection strategy.

Section A.7 (Fingerprint Generation): Demonstrates that our fingerprint generation strategy significantly outperforms alternative methods.

Section A.8 (3D Applicability Analysis): Assesses CodeMerge across other detectors beyond SECOND, confirming its strong model-agnostic applicability.

Section A.9 (3D Robustness Analysis): Analyses the robustness of CodeMerge in heavy distribution shift occurs (e.g., Fog & LiDAR dropout).

Section A.10 (Backbone Adapted Analysis): Validates robustness performance when adaptation is applied solely to the backbone.

Section A.11 (End-to-end Generalizability Analysis): Verifies CodeMerge's extensibility and transferability across diverse end-to-end architectures.

A.1 Implementation Details

For the end-to-end autonomous driving task, we employ ResNet50 [17] as the backbone network to uniformly process image data from both nuScenes and nuScenes-C [53] datasets. All input images are resized to 256×704. We use a 900×256 instance query as input to the transformer layers. Our optimization strategy utilizes the AdamW optimizer, configured with a weight decay of 0.001 and an initial learning rate of 1 × 10 -7 . To balance computational efficiency and prediction accuracy, we apply a random projection module to reduce the dimensionality of query features extracted from the pretrained model, resulting in a compact 1024-dimensional feature vector, and manage predictions through a model bank with a limited capacity of five models. Through self-supervised training on detection and tracking heads, the model accurately predicts ten classes as well as the associated instance IDs. For the point cloud detection tasks, we adopt the SECOND [55] as our pretrained model. We configure the training with a batch size of 8, a learning rate of 0.01, and a weight decay of 0.01. Additionally, we utilize a 900×256 dimensional 3D feature vector as input to the leverage module, enabling efficient and effective model merging.

A.2 Evaluation Metrics in End-to-End AD

We follow standard evaluation protocols to assess each task module for end-to-end AD system. Detection Metrics. We use nuScenes metrics, including mean Average Precision (mAP) and five error-based scores: mean Average Translation Error (mATE), Scale Error (mASE), Orientation Error (mAOE), Velocity Error (mAVE), and Attribute Error (mAAE). Together, they evaluate spatial, geometric, and semantic aspects of 3D box predictions. The nuScenes Detection Score (NDS) aggregates these metrics into a single score for holistic performance evaluation.

Tracking Metrics. Tracking performance is measured using Average Multi-Object Tracking Accuracy (AMOTA), Precision (AMOTP), and Recall. These metrics capture association quality, localization precision, and coverage of tracked instances.

Online Mapping Metrics. We compute class-wise Average Precision (AP) for static map elements (e.g., lane dividers, crossings, road boundaries) and report mean AP across categories to reflect mapping accuracy and consistency.

Motion Prediction Metrics. We evaluate prediction with best-ofK trajectory metrics: minimum Average Displacement Error (minADE), minimum Final Displacement Error (minFDE), and Miss Rate (MR). We also report End-to-end Prediction Accuracy (EPA), which reflects cascading errors across detection, tracking, and forecasting stages.

Planning Metrics. We assess planning quality using two key indicators: collision rate, which measures the frequency of collisions during trajectory execution, and L2 distance to goal, which quantifies the Euclidean distance between the final position and the intended goal. Together, these metrics reflect the safety and goal-reaching accuracy of the planned motion.

A.3 More Visualizations

In Fig. 5, we present additional visualized predictions from both the non-adapted SparseDrive and the SparseDrive model adapted at test-time using the proposed CodeMerge, illustrating performance across a broader range of corruptions.

A.4 Related Work

Test-time adaptation (TTA) dynamically updates models during deployment to mitigate distribution shifts [26, 50]. Early approaches primarily focus on tuning BatchNorm layers via entropy minimization, adaptive moment estimation, global statistic alignment, or loss landscape smoothing [44, 31, 59, 63, 34, 14, 33]. Subsequent methods explore self-training with confidence-filtered pseudo-labels [15, 30, 60], feature-level consistency or contrastive regularization [5, 22, 47, 38, 64, 49], robustness through data augmentation [62, 42], and leveraging guidance from language models [23]. Extending TTA to more challenging perception tasks ( e.g. , image- or LiDAR-based object detection [28, 29, 7]), MemCLR aligns 2D detector features using a memoryaugmented teacher-student framework [43]; DPO stabilizes LiDAR-based detection via dual perturbation optimization [9]; Reg-TTA3D generates noise-consistent pseudo-labels to supervise lowconfidence 3D boxes using high-confidence ones [58]; and MOS enhances adaptation stability by dynamically merging the topK diverse checkpoints for supervision [8]. Despite their effectiveness, existing methods adapt only perception tasks, while adapting unified end-to-end autonomous driving systems at test time remains unexplored.

Model Merging studies how weight-space operations can effectively compose, refine, or repair vision models through checkpoint averaging, gradient matching, or arithmetic edits to task-specific weight vectors [32, 51, 10, 18, 1, 54]. Recent literature highlights the effectiveness of these merging techniques in enhancing generalization across tasks such as zero-shot learning [52], open-set learning [35], domain adaptation and generalization [36, 2, 11], and cross-domain tasks involving 3D LiDAR point clouds [20, 8]. In this work, we build upon the strengths of model merging techniques to enable efficient, on-the-fly adaptation within end-to-end autonomous driving pipelines.

A.5 Computational cost of each TTA step

The runtime reported in Table 6 is the total time aggregated over all adaptation steps across the entire test set. Below, we report GPU memory usage and per-frame latency alongside AP3D with SECOND on the nuScenes → KITTI transfer in Table 7.

CodeMerge

No Adaption

CodeMerge

No Adaption

CodeMerge

No Adaption

AM_BACK

Tun Right

Turn Right

Turn Right

Turn Right

Go Straight

Go Straight

Go Straight

Go Straight

Go Straight

Go Straight

Go Straight

Go Straight

Speed comparison. Compared with MOS [8], CodeMerge reduces per-frame latency by ~43% (0.49 → 0.28 s) while improving AP3D by +14.5% . Relative to TENT [44], it yields a +210.9% AP3D gain at essentially the same latency (0.26 vs. 0.28 s per frame). Compared with CoTTA [46], CodeMerge adds only +0.13 s per frame yet improves AP3D by +22.9% .

Memory comparison. CodeMerge uses less GPU memory than MOS (16,041 vs. 17,411 MiB) and slightly more than CoTTA (15,099 MiB), while achieving the highest AP3D among all methods. Compared with TENT, it requires +5,209 MiB (~5.1 GiB) but maintains comparable runtime and delivers markedly larger performance gains ( +210.9% ).

Overall, CodeMerge offers strong accuracy-efficiency trade-offs on SECOND [55] for the nuScenes → KITTI setting, it achieves the highest AP3D with competitive latency and memory, making it practical for real-time test-time adaptation (TTA) in dynamic deployment environments.

A.6 Random Projection Determinism and Robustness

We instantiate a fixed Gaussian random projection once before TTA and keep it frozen throughout adaptation. It is applied to the pretrained source feature extractor's intermediate activations ϕ Θ (0) ( x ) to produce low-dimensional fingerprints ˆ z (refer to Eq. (7)), ensuring all checkpoints are embedded into a shared, comparable space across time steps. Because both the fingerprinting features (from the frozen feature extractor) and the projection matrix are fixed, this mapping is training-free and lightweight .

As shown in Table 8, we verify robustness by resampling the projection matrix with three seeds (RP1RP3) and re-running the full pipeline. The results vary only slightly: For perception, NDS(0.4920 → 0.4946) and mAP(0.3630 → 0.3735); For tracking, AMOTA(0.3252 → 0.3347); For planning, L2Avg(0.6209 → 0.6255). These small ranges indicate that the inherent randomness of the projection has a minor impact on end-to-end AD performance.

Discussion This stability is expected because (i) the projection is a fixed Gaussian linear map instantiated once and kept frozen during TTA, applied to features from the fixed source encoder to produce low-dimensional fingerprints (Eq. (7)), ensuring all checkpoints are embedded in a shared space; and (ii) the fingerprint space faithfully mirrors parameter-space geometry (Fig. 3; Pearson r and Kendall τ typically > 0 . 7 ), so different random draws that preserve this linear embedding yield similar merge decisions. Overall, repeated experiments with different seeds exhibit only minor fluctuations, consistent with our design rationale for a stable, geometry-preserving random projection.

A.7 Alternative Fingerprint Generation Strategies

Lightweight pooling-based strategies. We first investigate a suite of computationally efficient feature-aggregation approaches, such as MaxPool [37], AdaptiveAvgPool [16], Lp-Pooling [3], and Fractional-Max Pooling [61]. While computationally lightweight, these approaches compress features into coarse statistics and underperform on the Waymo → KITTI transfer task. Our fixed

Gaussian Random Projection (RP) achieves AP 3 D = 63 . 23 , outperforming the pooling variants by +4.6%, +2.88%, +1.82%, and +1.44%, respectively in Table 9. This result suggests that simple pooling loses the discriminative structure required for behavior-aware merging.

Learning-based dimensionality reduction methods. We also consider learning-based techniques such as PCA [21], which requires fitting or updating decompositions on streaming features. However, these incur significant computational overhead at every TTA step, making them incompatible with our single-pass, real-time adaptation setting.

Pooling is high-throughput yet discards geometry and discriminative structure, while PCA-like methods impose computational costs incompatible with real-time operation. Our fixed random projection preserves model-space geometry, satisfies real-time efficiency requirements, and empirically delivers the strongest accuracy among all tested strategies.

A.8 Applicability beyond SECOND

To assess generality, we further evaluate CodeMerge on DSVT [45] under the Waymo → KITTI transfer in Table 10. CodeMerge attains 79.88 APBEV / 61.06 AP3D , outperforming the strongest baseline MOS [8] by +3.2% and +6.4% , respectively, and surpassing No Adapt by +125% in AP3D. Because CodeMerge builds on training-free random-projection fingerprints and codebook-guided merging rather than architecture-specific modules, these results indicate our TTA is model-agnostic and transfers effectively to detectors beyond SECOND.

A.9 Robustness under Fog & LiDAR Dropout

We conduct additional experiments adapting SECOND from KITTI → KITTI-C under heavy fog with increasing point dropout (0% → 25% → 50%). We report AP at the moderate level in Table 11. With stronger dropout, performance degrades gradually rather than catastrophically: AP 3D drops from 75.96 → 73.87 → 71.95 (-4.01%, ≈ 5.3% relative), and AP BEV from 88.16 → 85.73 → 85.04 (-3.12%, ≈ 3.5% relative). This smooth, monotonic decline suggests that our checkpoint merging remains reliable even under compounded shifts.

A.10 Backbone-only Adaptation

End-to-end AD framework. To verify the effect of merging and keep TTA lightweight, we freeze all modules except the 3D detection head. As shown in Table 12, we additionally enable backbone (ResNet) adaptation and observe negligible deltas relative to head-only TTA (e.g., NDS -0 . 0034 , mAP -0 . 0055 , AMOTA -0 . 0028 , map mAP -0 . 0097 , L2-Avg -0 . 0023 ), indicating that CodeMerge remains effective even when only the backbone is updated.

3D detection framework. Following prior works [8, 9], we also consider adapting the entire backbone (encoder, detection head, and all batch-norm layers). These studies have shown that, for test-time adaptation in 3D detection, full-backbone adaptation typically surpasses BN-only updates; CodeMerge fits seamlessly into this setting as well.

A.11 Generalizability Beyond SparseDrive

Early end-to-end models under corruption. Some previous end-to-end architectures (e.g., VAD [19]) collapse on corrupted scenes even before any TTA is applied. Under heavy snow , VAD attains only 0.0168 mAP and 0.0353 NDS for detection in Table 13, indicating that the base model's predictions are already unreliable and nearly collapsed. In this regime, online self-supervision becomes untrustworthy, so any TTA method has little signal to learn from. This reflects a robustness limitation of the base model, not of TTA or CodeMerge. Even so, attaching CodeMerge to V AD yields small but consistent gains across perception and planning, and does not degrade V AD's performance, underscoring that the bottleneck lies in model brittleness rather than our adaptation mechanism.

Recent architectures with non-collapsed baselines. When the backbone remains minimally competent under corruption, CodeMerge is highly effective and broadly applicable. On DiffusionDrive [27] and MomAD [39], CodeMerge delivers consistent improvements across detection, tracking, mapping, motion, and planning, and reduces collision rate under both Brightness and Snow corruptions, indicating scalability beyond SparseDrive.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the core contributions: a compact fingerprint-guided model merging framework, its theoretical grounding via leverage scores, and demonstrated improvements ( e.g. >100% gap closure on Waymo → KITTI and 7.7% AP boost under KITTI → KITTI-C)-all of which are borne out by the experiments.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have included detailed limitations in the conclusion section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The paper clearly states all assumptions underlying its theoretical results: linear mode connectivity of fine-tuned checkpoints, the Gaussian-Newton approximation of the Hessian, and regularized leverage-score derivation, including the spectrum bound and sampling error analyses that connect fingerprint correlations to optimal merging coefficients.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper provides exhaustive implementation details (frameworks, hardware, batch sizes, hyperparameters). All datasets, corruption settings, and model checkpoints are specified, and we commit to releasing our code and pretrained codebook, ensuring full experimental reproducibility.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.

In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We will publicly release our full implementation, including data preprocessing scripts, checkpoint codebook, and evaluation pipelines, alongside the pretrained SparseDrive weights and nuScenes/Waymo split configurations.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The paper details all training and testing settings, including dataset splits, optimizer type, learning rates, batch sizes, and checkpoint schedules, in both the main text and appendix, ensuring readers have sufficient information to reproduce every experimental result.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: While our results clearly demonstrate consistent gains, we did not report error bars, confidence intervals, or statistical tests across multiple runs.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We report in Section 4 that all experiments run on a single NVIDIA RTX A6000 GPU with 48 GiB of memory. We specify batch sizes for each task, and the frameworks used (SparseDrive / OpenPCDet), and provide per-checkpoint merging overhead. This level of detail is sufficient for reproducing both runtime and memory requirements.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our work uses only publicly available datasets (KITTI, nuScenes, Waymo) and standard benchmarks under their license terms. No personal or sensitive data is involved, and all experimental protocols conform to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no societal impact of the work performed

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We build on standard open-source assets-SparseDrive and OpenPCDet (Apache 2.0) and the KITTI, Waymo, and nuScenes datasets (with their respective academic licenses)-all of which are explicitly cited in Section 4.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA] .

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces CodeMerge, a novel and efficient framework for test-time adaptation (TTA) in the context of 3D perception for autonomous driving. The primary contribution is a method that avoids the high computational cost associated with existing model merging techniques. Instead of loading and performing inference with multiple full model checkpoints, CodeMerge creates a codebook where each checkpoint is represented by a compact, low-dimensional \"fingerprint.\" These fingerprints are derived from the penultimate features of a source model. The framework then uses ridge leverage scores calculated on these fingerprints to determine the importance of each past checkpoint for merging. The authors demonstrate the effectiveness of CodeMerge on both end-to-end autonomous driving systems and modular LiDAR-based object detectors, showing significant performance improvements on challenging corruption and cross-dataset benchmarks, as well as benefits for downstream tasks like mapping and planning.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n- **Significance**: The paper addresses a highly significant and practical problem in autonomous driving: robustness to real-world distribution shifts encountered during deployment. The evaluation of TTA within a full end-to-end pipeline (SparseDrive), demonstrating that improvements in the perception module positively impact downstream tasks. \n\n- **Originality**: The core idea of using low-dimensional fingerprints and ridge leverage scores to guide model merging is novel and insightful. The strong empirical correlation shown between fingerprint and parameter space differences provides good evidence for the validity of this latent-space approach.\n\n- **Quality and Clarity**: The paper is generally well-organized, and is supported by a comprehensive and rigorous set of experiments. \n\n# Weaknesses\n\n- As other popular architectures like UniAD and VAD \"experience over tenfold performance degradation on nuScenes-C, hindering effective adaptation training\" (Lines 272-273), this raises a significant question about the generalizability of CodeMerge. While the results on SparseDrive are strong, the paper lacks evidence that the approach would be effective on other end-to-end models, which currently limits the perceived scope of the contribution.\n\nQUESTIONS:\n1. The paper's success is demonstrated compellingly on the SparseDrive architecture. However, the conclusion mentions significant challenges in applying TTA to other architectures like UniAD. Could the authors elaborate on the underlying reasons for the performance degradation on these other models? Does this suggest a limitation in the TTA paradigm for certain architectures, or a property of CodeMerge itself? Answering this would greatly clarify the method's scope and potential for broader application. My score would improve if a convincing argument can be made for its generalizability.\n\n2. Section 3.1 states that fingerprints are computed using the initial pretrained feature extractor ϕΘ(0)​. For the end-to-end experiments, it is mentioned that only the 3D box regression head is updated (Line 141). Does this \"frozen feature extractor\" strategy also apply to the LiDAR-only detector experiments on SECOND, or are more parameters adapted and merged in that setting? Clarifying the scope of adapted parameters Θ(t) for all experimental settings would be helpful.\n\nAs I am not very familiar with this research field, if the answers are convincing enough, I would like to increase my scores.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors tackle test-time adaptation for 3D detection by compressing every intermediate fine-tuned checkpoint into a low-dimensional fingerprint. The compressed features are stored in a codebook and ranked by ridge-leverage scores. Then, the top-k weights are linearly merged. The experiments are conducted with both camera-based and LiDAR-based 3D detectors with various types of sensor degradations. Improved perception performance also benefits the downstream task, including mapping, motion prediction, and planning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n---\n- The paper is generally well-written and easy to follow.\n- The proposed idea of replacing K forward passes with a projection-space distance is clear.\n- Experiments cover both camera-based and LiDAR-based 3D detectors with various types of degradation.\n\nWeaknesses\n---\n- If I understand correctly, CodeMerge still requires a buffer of previously fine-tuned checkpoints, which means it needs gradient-based updates, and the paper assumes the model can back-propagate per every frame (or every N frames) to grow the buffer. This may be impractical in a real scenario. Regarding this, how many gradient steps are executed per frame to produce each checkpoint? Please correct me if I misunderstood.\n- Lack of robustness analysis. Once a heavy distribution shift occurs (e.g., fog + LiDAR dropout), the model may fail to accurately rank.\n- Similar to the above, does CodeMerge work if the backbone itself is adapted?\n- It would be beneficial to include more discussion on domain-adaptation methods (e.g, SN, ST3D). From my understanding, ST3D and SN are offline domain-adaptation methods, and CodeMerge is evaluated after every test frame. What are pros and cons about the two different directions?\n- Could EMA be another baseline for merging in Table 5?\n\nAdditional Comments\n---\n- Line 251: Quantitative Analysis --> Qualitative Analysis\n\nQUESTIONS:\nPlease see the above.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel model merge based TTA method for LiDAR 3D object detection and E2E AD models. In the proposed CodeMerge approach, previously extracted features and corresponding checkpoints are stored. At each time step, the features of these checkpoints are projected into low-dimensional features, fingerprints. RLS are then computed based on these fingerprints to guide the model merging process. Compared to prior model merge based TTA methods such as MOS, this approach demonstrates improved efficiency and performance. The effectiveness of the method is validated across multiple settings, including cross-dataset and clean-to-corruption adaptation.\n\nSTRENGTHS AND WEAKNESSES:\n[ Strengths ]\n1. The proposed fingerprint-based model merging method resolves the scalability limitations of prior model merging-based method, which is computationally expensive.\n2. The method is evaluated not only for LiDAR-based 3D object detection but also for end-to-end autonomous driving tasks, demonstrating its effectiveness as a TTA method across various application domains. This broadens its potential applicability.\n\n[ Weaknesses ] \n1. Regarding the efficiency analysis, does the reported runtime refer to a single adaptation step? If so, it would be helpful to specify the inference time for SECOND and computational burden introduced by each TTA step. Including runtime comparisons with other baselines such as TENT, CoTTA, and DUA would provide further evidence of the practicality of the proposed method.\n2. The paper employs random projection for generating fingerprints. Is there a rationale behind using random projections? Why are these projections redefined at each time step? Does the inherent randomness introduce performance variance? It would be helpful to clarify whether the method is robust across repeated experiments.\n\nQUESTIONS:\n1. Have alternative fingerprint generation strategies been considered?\n2. Can the proposed TTA method be applied to 3D object detection models other than SECOND?\n3. Since the fingerprint is generated using features extracted from a fixed encoder given the i-th input, can it reliably serve as a key of the model parameters themselves?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

comments: SUMMARY:
Robust 3D perception under unpredictable conditions is a key challenge for autonomous driving, but existing test-time adaptation methods struggle with instability in high-variance tasks like 3D object detection. While model merging approaches based on linear mode connectivity improve stability, they are computationally heavy due to repeated checkpoint access and multiple forward passes. This paper introduces CodeMerge, a lightweight framework that merges models in a compact latent space using low-dimensional fingerprints and a key-value codebook, enabling efficient model composition with minimal overhead. CodeMerge achieves strong results across benchmarks—improving 3D detection by 14.9% NDS on nuScenes-C and over 7.6% mAP on LiDAR-based detection.

STRENGTHS AND WEAKNESSES:
Strengths:

1. This paper is well written and easy to follow. The motivation is clear and reasonable.

2. The performances are good, improving 3D detection by 14.9% NDS on nuScenes-C and over 7.6% mAP on LiDAR-based detection.

3. The experimental results are extensive. Plenty ablation and sensitivity studies are provided which are convincing.

Weaknesses:

1. The authors claim that "Code and pretrained models are released in the supplementary material." In abstract but I can not find them.

QUESTIONS:
Please refer to the Weaknesses.

LIMITATIONS:
The authors provide one sentence in the Conlusion. But I don't think it is sufficient.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02