Unveiling the Hidden: Online Vectorized HD Map Construction with Clip-Level Token Interaction and Propagation
Authors: Nayeon Kim
Hongje Seong
Daehyun Ji
Sujin Jang
Venue: NeurIPS 2024 poster (2024)
Keywords: vectorized HD map
clip-level pipeline
clip-level token
interaction
propagation
Primary area: machine_vision
TL;DR: This paper introduces a novel clip-level pipeline to explicitly unveils the invisible map elements.

Abstract: Predicting and constructing road geometric information (e.g., lane lines, road markers) is a crucial task for safe autonomous driving, while such static map elements can be repeatedly occluded by various dynamic objects on the road. Recent studies have shown significantly improved vectorized high-definition (HD) map construction performance, but there has been insufficient investigation of temporal information across adjacent input frames (i.e., clips), which may lead to inconsistent and suboptimal prediction results. To tackle this, we introduce a novel paradigm of clip-level vectorized HD map construction, MapUnveiler, which explicitly unveils the occluded map elements within a clip input by relating dense image representations with efficient clip tokens. Additionally, MapUnveiler associates inter-clip information through clip token propagation, effectively utilizing long- term temporal map information. MapUnveiler runs efficiently with the proposed clip-level pipeline by avoiding redundant computation with temporal stride while building a global map relationship. Our extensive experiments demonstrate that MapUnveiler achieves state-of-the-art performance on both the nuScenes and Argoverse2 benchmark datasets. We also showcase that MapUnveiler significantly outperforms state-of-the-art approaches in a challenging setting, achieving +10.7% mAP improvement in heavily occluded driving road scenes. The project page can be found at https://mapunveiler.github.io.

Unveiling the Hidden:
Online Vectorized HD Map Construction with Clip-Level Token Interaction and Propagation
Nayeon Kim
Hongje Seong* Dachyun Ji
Sujin Jang'
Samsung Advanced Institute of Technology (SAIT)
(nayeon.XX)) hongje. seong, derek. ji, s.steve. jang) @samsung com
Abstract
Predicting and constructing road geometric information (e.g., lane lines, road markers) is crucial task for safe autonomous driving, while such static map elements can be repeatedly occluded by various dynamic objects on the road. Recent studies have shown significantly improved vectorized high-definition (HD) map construction performance, but there has been insufficient investigation of temporal information across adjacent input frames (i.e., clips), which may lead to inconsistent and suboptimal prediction results. To tackle this, we introduce a novel paradigm of clip-level vectorized HD map construction, MapUnveiler, which explicitly unveils the occluded map elements within clip input by relating dense image representations with efficient clip tokens. Additionally, MapUnveiler associates inter-clip information through clip token propagation, effectively utilizing long- term temporal map information. MapUnveiler runs efficiently with the proposed clip-level pipeline by avoiding redundant computation with temporal stride while building global map relationship. Our extensive experiments demonstrate that MapUnveiler achieves state-of-the-art performance on both the nuScenes and Argoverse2 benchmark datasets. We also showcase that MapUnveiler significantly outperforms state-of-the-an approaches in challenging setting, achieving + +10.75 7% mAP improvement heavily occluded driving road scenes. The project page can be found at https:/ //mapunveillrrrrrreeee/a github.io. io.
Introduction
Vectorized HD map construction (VHC) is task of predicting instance- wise vectorized representa- tions of map elements (e.g., pedestrian crossings, lane dividers, road boundaries). Such static map ele- ments crucial information for self-driving vehicles, including applications like lane keeping [1, 6], path planning [29, 24, 15], and trajectory prediction [30, 40, 11]. Prior approaches to constructing dense and high-quality HD maps typically rely SLAM-basee offline methods (e.g., [48, 37, 38]). Such an offline method generally involves series steps including feature extraction and selection (e.g., edge, plane), odometry estimation via feature matching, and mapping. However, these processes involve complicated and computationally burdensome tasks, limiting their use to offline applications.
cost-efficient and real-time alternative to existing expensive offline approaches. Current works on camera- camera-based VHC typically aim to extract unified 3D Bird's Eye View (BEV) features that cover the surrounding environment the ego-vehicle [20, 22, 23), relying on various Perspective View (PV) to BEV transformation methods [50, 32, 21. 5]. Subsequently, task-specific head follows to decode and
More recently, camera-based multi-view VHC has been actively investigated
"Equal contribution. t Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(t-2
BEV (t-1)
BEV (t)
(L-2)
BEV (t-1)
BEV (1)
BEV (--
BEV
(EV
GRU
GRU
GRU
Pred.
Pred.
Pred.
Pred.
Pred.
Pred.
Clip Tokens
Pred.
Pred.
Pred.
Visibility BEV
(a) framle-faame inference
streaming inference
(c) clip-level inference (ours)
Figure 1: (a) Existing approaches relying single-frame inference cannot capture the entire map information in the BEV features [26, 22, 23.. (b) Recent alternatives explore temporal information via streaming, but they cannot address the inherent nature of maps and propagate noise from previous timestamps. (c) We directly unveil hidden maps in BEV features by interacting with clip tokens that contain high-level map information. We visualize BEV features by ID PCA projection. The BEV features are extracted from (a) MapTRv2 [23], (b) StreamMapNet [46], and (c) our MapUnveiler.
predict map elements from the extracted BEV features. Despite significant progress, prior works still suffer from frequently occluded map elements caused by dynamic foreground objects such as vehicles and pedestrians, as described in Fig. I-(a). Moreover, the prediction performance degrades when applied to a larger perception range. To address such issues, prior works try to leverage temporal information extracted from stream of preceding feature frames [46, 41J. While such approaches have achieved improved online prediction performance, they still do not fully leverage the potential of temporal information across a longer range of frames. particular, prior works do not consider the cumulative impacts of occluded map elements, leading to noisy BEV feature generation, Fig. 1-(b) and Tab. 3. Moreover, such cumulative flaws can ultimately lead to degraded performance in longer perception ranges (see Tab. 1), which is crucial for the other hand, the key idea conventional SLAM-based offline methods is to stack and associate static features collected over longer window of frames (i.e., mapping features). This offline mapping strategy effectively addresses occlusion issues by leveraging diverse views of the map elements across multiple frames, while it requires manual human annotation and complex pipelines.
shown in
autonomous driving. On
Based on these insights, we introduce a novel clip-level construction framework, MapUnveiler, which incorporates the effective offline mapping strategy into sate--fftthe-ar online VHC approaches. In contrast to the direct global feature mapping used in offline methods, our MapUnveiler collects clip- level temporal map information and learns differentiable associations for efficient online inference. We generate compact clip tokens consisting of temporal map information within clip input and update BEV features with these tokens unveil hidden map elements that are visible in certain frames, as shown in Fig. 1-(c). Subsequently, we transfer the inter-clip tokens to the next clip's BEV features to facilitate establishment the long-tern intra-clip associations among map elements.
As result, MapUnveiler outperforms state-of-the-are methods on two widely recognized benchmarks: +1 +1.3% mAp on nuScenes [3] and +0.9% mAP on 2rgoverse2 [44]. We also observe a marginal increase in computational burden, as we exploit the benefits of the elip-level inference strategy, which contributions include:
allows an efficient inference of multi-frame inputs. In summary,
We propose MapUnveiler, an online VHC model that incorporates the offline mapping strategy and unveils the hidden maps by interacting multiple dense BEV features with compact tokens.
We introduce the clip-level pipeline to infer MapUnveiler online by mapping within a clip set of BEV features and propagating the map information subsequent timestamps, thereby building a global map efficiently.
Our method significantly improves a frame-level model on longer perception range settings (58.6% + 68.7%) and heavy occlusion splits (53.1% - 63.8%). and achieves state-of- the-art performance on two standard VHC benchmarks with marginal extra computations (15.6 FPS- + 12.7 FPS).

---

Ceneera Cemertorr
Map haees Uydevday loet Tetre
Map
Map Decoder
Map Queries
Write
Bacihone
to
Festures
Generater
Clip Raken
Vectorized Map
Clp set multiviie images
Memory
Memory
Unvellar
Fraie-levell MapNet
Queries
MapUnveiler
Figure 2: Our framework takes clip-level multi-view images and outputs clip-level vectorized HD maps. All components in the frame-level MapNet (i.e.. Backbone, PV to BEV, Map Decoder) are adopted from MapTRv2 [23]. The frame-level MapNet extracts map queries and BEV features independently at each frame. MapUnveiler generates compact clip tokens that contain clip-level temporal map information and directly interact with dense BEV features. With the updated BEV features, we construct high-quality clip-level vectorized maps. The generated map tokens and clip tokens then written to memory.
Related Work
Multi-View HD Map Construction. SLAM (Simultancous Localization and Mapping) [10] has been central technique for constructing accurate HD maps [48, 37, 38]. However, these meth- ods require memory-intensive, complex pipelines for global mapping of geometric features, and are therefore typically executed offline. Recently, deep learning approaches have emerged as an appealing alternative those expensive offline methods, enabling online HD map construction using cost-efficient multi-view camera sensors. The perspective-vier (PV) to bird's s-eee-vwwwww (BEV) transformation methods [50, 32, 21, 5] enable the generation of features from the surrounding environment of the ego-vehicles using camera sensors, even in the absence of precise spatial cues. BEVFormer [21] utilizes the deformable attention mechanism [52] to extract BEV features and predict rasterized semantic maps. However, it cannot generate instance-wise representation of map elements. To address this, HDMapNet [20] introduces heuristic method to group pixel-level se- mantic maps into vectorized representation. Similarly, VectorMapNet [26] proposes end-to-end learning approach to predicting vectorized map representations. Although such methods have demon- strated notable prediction performance in single- frame inference, they do not consider the temporal information from multi-frame inputs. More recently. StreamMapNet [46] and SQD-Map.Net [41] have proposed streaming feature paradigm [42], which aims leverage temporal information for improved temporal consistency across predictions. However, these methods propagate dense BEV features directly, incorporating map information from previous frames that may have been occluded and undetected, resulting in the accumulation of noise, as shown Fig. 1-(b). To address this issue, propose end-to-end clip token learning approach that combines offline mapping techniques with online strategies, aiming for high performance and computational efficiency.
Temporal Token Learning. With the rapid development of transformers [39], there has been significant interest in efficient token learning alongside dense features, egg., CNN representations. In particular, temporal token learning has emerged an attractive alternative memory-intensive spatio-temporal dense CNN representations [45, 31]. VisTR [43] extends DETR [4] into the 3D) domain to extract pptii-ttmpporal instance tokens that can be directly used for instance segmentation. IFC [17] proposes an efficient spatio-temporal token communication method, which replaces the heavy interactions within dense CNN features. VITA [13] learns efficient video tokens from frame- level instance tokens without dense CNN features. Cutie [7] updates CNN representations with tokens avoid opatio-temporal dense matching. TTM [36] introduces an efficient long-term memory mechanism by summarizing tokens into memory rather than stacking [2, 18, 34] recurrence [14, 8]. While the aforementioned approaches were designed to handle foreground instances, we discover the potential of token learning to construct background maps. By learning compact tokens and interacting with dense BEV features, we impose traditional mapping into online VHC model and enable online running.
3

---

Intra-clip Unveiler
Map Generator
Map Queries (Q"ar)
Map Takens (U,))))
parmors pay
Updater
BEV Features (Fder)
Updated BEV (12
Clip Taken Generator
Memory Read (1
Clip Takens (0
Map tyien posiiosa onteeding
DEV ludare unbecding
Figure 3: A detailed implementation of Intra-clip Unveiler. We use blue, red, and green arrows to indicate the flows of map tokens, BEV features, and clip tokens, respectively. In each attention and feed forward layer, standard layer normalization, dropout, and residual connections are followed.
3 Method
3.1 Overview
We present the overall architecture of MapUnveiler in Fig. 2. Given a set of synchronized multi-view images (i.e., clip inputs), model sequentially construct clip-level vectorized HD maps. We first extract frame-level BEV features and map queries, which are then used as inputs to MapUnveiler module. We employ memory tokens, which are written from the previous clip and facilitate the establishment of long-term temporal relationships. From the memory tokens and map queries, we generate clip tokens that embed temporal map element cues in compact feature space. This is the first step to understand clip-level map information. We then update BEV features with clip tokens, which is the core step of unveiling hidden maps. Using the updated (unveiled) BEV features, we extract map tokens and construct clip-level vectorized HD maps. After a clip-level inference, we generate new memory tokens using clip tokens, map tokens, and the current memory tokens. The new memory tokens are used for providing temporal cues for the subsequent clip-level inference. Since we opt for a clip-level pipeline, MapUnveiler efficiently infers with temporal stride S, performing clip-level inference only Nr /S times for sequence of Nr frames. In the following subsections, we detail each module in the proposed framework.
3.2 Frame-level MapNet
We adopt MapTRv2 [23] as our frame-level MapNet architecture extract clip of BEV features and map queries from synchronized multi-view images. We extract perspective view (PV) image features using a backbone network, then these PV image features are transformed into BEV features through a PV-to-BEY module. Following the setup of MapTRvv, we adopt ResNet50 [12] and Lift, splat, shoot (LSS) based BEV feature pooling [16] for our backbone and PV-to-BEV module, respectively. These BEV features are utilized for querying maps in the map decoder. With their BEV features, the map decoder outputs frame-level map queries which can be directly used for constructing vectorized HD) maps. Finally, the frame-level MapNet outputs BEV features and map queries, which are the results from the PV-to-BEV module and the map decoder, respectively. BEV features represent rasterized map features, whereas map queries embed vectorized map information; thus, we can directly construct vectorized HD maps using the map queries.
3.3 MapUnveiler
MapUnveiler is a novel framework designed unveil invisible map information that cannot be captured by frame-level BEV features alone. To avoid heavy computations, we adopted a clip-level inference scheme with temporal window (clip length) T and stride S. A detailed explanation of the inference scheme with the temporal window T and stride S is provided in Appendix (see Sec. A.1). Our MapUnveiler consists two main components: (1) Intra-clip Unveiler and (2) Inter-clip Unveiler. For each clip-level pipeline, our Intra-clip Unveiler generates vectorized maps for T frames. The Inter-clip Unveiler then writes memory tokens with the tokens generated in Intra-clip Unveiler to build global relationships.

---

3.3.1 Intra-clip Unveiler
Intra-clip Unveiler is composed of sequence of L layers. It initially takes clip set of frame-level map queries Quap, BEV features FBEV, and memory read Read (read at Inter-clip Unveiler, detailed in Sec. 3.3.2). In the first step, compact clip tokens are created by the clip token generator. The BEV updater then unveils hidden maps in the BEV features with the clip tokens. Finally, map generator outputs clip-level map tokens from the updated BEV features. The map tokens are directly used for constructing vectorized HD maps with perception heads. We illustrate the Intra-clip Unveiler in Fig. 3. In the followings, we describe the detailed implementation of cach module.
Clip Token Generator. Clip token generator yields clip tokens Uplip R.N.xC from frame-level map queries Qmap E I where No. NN, and N, denote the clip token size, number of predicted map element, and number of points per map element, respectively. To globally gather intra-clip map features, opt for naive cross-attention [39]. Through this step, we obtain compact clip-level map representations, enabling efficient intra-clip communication with small-scale features.
BEV Updater. The second step is the BEV Updater, which updates bev features FBEV RxxHxWxC with the clip tokens Upliip unveil the hidden map element information. In cross- attention, query is derived from the bev features FBEV, and the key and value are derived from the clip token Up The output of this step is robustly updated bev features U.BEV E T x enhanced via clip tokens for hidden areas relative to the original bev features. The main idea of BEV Updater to avoid heavy computation in spatio-temporal cross attention. To achieve this, we do not directly communicate intra-clip BEV features, but instead decouple the spatial BEV features and the temporal clip tokens. We then update the spatial BEV features with compact temporal clip tokens, effectively communicating apatiottmpoors information with reasonable computational costs. The updated bev features U/ BEE are used as value features in the next step.
Map Generator. The last step is the Map Generator, which generates map tokens U,map E R XNxNN,xxx using the updated BEV features U,BEV created in the previous step. The objec- tive of this step is generate a refined version of frame-level map queries. As illustrated in Fig. 3, the map generator uses deformable attention [52] and decoupled self-attention [22] mechanisms, following [22]. deformable attention, query is derived from the map queries Qumap, and the value is derived from the updated BEV features U,BEV Since the updated BEV features are -patio-temporally communicated, we directly extract map tokens. Each map token represents a vectorized map element through 2-layer Multi-Layer Perceptron (MLP). The map tokens Ummap are written to the memory of the Inter-clip Unveiler, and when the map tokens U.map of the L-th layer pass through the prediction head, vectorized maps are generated.
3.3.2 Inter-clip Unveiler
Inter-clip Unveiler propagates the tokens from previous clip input to the next one, thereby preserving the dense temporal information from the prior frames. As shown in Fig. 2, Inter-clip Unveiler writes map tokens Un and clip tokens U from the Intra-clip Unveiler to the memory. Here, adopt token turning machine (TTM) [36] to efficiently manage the long-term map information. In the followings, we describe the detailed implementation of read and write.
Read. We generate compact tokens that contain a global map information by reading from memory tokens and map queries. Following TTM [36], we read with the token summarizer [35] which efficiently selects informative tokens from inputs as follows:
Tread_ Read(U @mpp) SN, SN. (122-5:t-' QI@m)))))
where (Umeraory [[maa]] denotes the concatenation of two elements, and U.memory denotes mem- ory tokens for clip. We employ the Docation-based memory addressing used in [36] utilizing the positional embedding (detailed in Section 3.3.3). Note that the memory is not available in the first clip-level pipeline. Therefore, we initially write the memory token from learnable clip embeddings.
Write. We employ the write operation with the same token summarizer [35] that is used in [36]. The new memory U, Stt E IMMx is generated by summarizing the clip tokens UT map
5

---

tokens Uh and old memory U1-2sit's as follows:
memerry Sit = Write(U UUTT U(-25:t = SM ((U tha " (ss:ss)))))
(2)
where M denotes the memory token size. The newly generated memory through the write operation used in the read operation of the first layer of the Intra-clip Unveiler in the subsequential per-elip process. the tokens within the memory are not re- re-selected in the subsequent steps, it will be removed from the memory, and the selection mechanism will be determined through the learning. We employ the same memory addressing method used in the read operation. The write operation is applied in the last layer of the Intra-clip Unveiler, generating new memory that preserves the information of the clip tokens and map tokens.
3.3.3 Positional Embedding
While the standard transformer structure is permutation-invarian we require position information added with temporal information to predict map elements at the clip-level. For the BEV features (PB), we use a fixed 3D sinusoidal positional embedding, following [43]. For the map tokens (PMr), we use learnable positional embeddings used in frame-level MapNet [23] with newly defined learnable
temporal positional embeddings. For the clip tokens (Pc), we define new learnable positional
embeddings. Similarly, learnable positional embeddings are defined for the memory tokens that are used in read and write of the Inter-clip Unveiler.
3.3.4 Loss
Since our model is built on top of the frame-level MapNet (MapTRv2 [23]), we basically follow the loss functions used in MapTR [22] and MapTRv2 [23]. Specifically, we employ the overall loss functions
CMM MappUnvester = Aa C. + X² C.M + +x LM + A, CM CPVsee
(3)
Crrame_Mann MapNet Lone2one + Lone2many Edense:
(4)
LoneZone = X LE + CE + NE C
(5)
Ldenee = X₁ Edepth + X5. LBEVSeg + XN CpV
(6)
where LapUUneed nweiter and [Frame_MapNel indicate the loss functions for training MapUnveiler and frame-level mapnet, respectively. Latss Lypps Ldir. and LPVSeg denote classification loss [22], point- to-point loss [22], edge direction loss [22], and PV segmentation loss [23], respectively. Lone2one: Lane2mang. and f.dense are used for training frame-level MapNet and denote one-to-one loss [22], one- to-many loss [23], and auxiliary dense prediction loss [23], respectively. Ldepth and LBE denote depth prediction loss [23] and BEV segmentation loss [23], respectively. We set hyperparameters
to M = 2, A = 5, A = 0.005, A = 2, X = 2, X 5, X = 0.005, Xi" 3, X 1, and
A5" = 2. Note that we did not use 0me2many loss [23] to save GPU memory during the main training.
4 Experiments
4.1 Dataset and Metric
We construct experiments on two standard VHC benchmarks: nuScenes [3] and Argoverse2 [44] datasets. nuScenes offers synchronized six-view images with high-quality HD maps. It provides 1, 1,000 scenes and each scene consists of about 20 seconds. In this dataset, we follow the official scene split of 700, 150, and 150 for training, validation, and testing. respectively. Similarly Argoverse2 provides synchronized seven-view images with high-quality HD maps. It contains 1,000 scenarios and each scenario consists of about 15 seconds. Argoverse2 dataset provides the official scene split of 700, 150, and 150 for training, validation, and testing, respectively, which we follow.
For each dataset, we construct maps in two perception ranges: standard range of 60x and longer range of 100x50m. Additionally, we create challenging validation splits to demonstrate the efficacy of our model under heavy occlusions. Specifically, we collect the occluded frames if any dynamic objects exists within 2.5m around the ego vehicle. Thankfully, nuScenes provides 3D cuboid annotations for vehicles and pedestrians, allowing us to automatically create the new challenging split, and the result with this split is given in Sec. 4.4.

---

Table 1: Comparisons on nuScenes and Argoverse2 val sets. APp. AP,, AP, indicate the average precision for pedestrian crossing, divider, and boundary, respectively. FPS is measured using single
NVIDIA A100 GPU. are taken from the corresponding papers and are scaled based on the FPS of
MapTRv2 [23] for a fair comparison.
nuScenes
Argoverse2
Range
Method
Epoch
APp
APa
APs
mAP
FPS
Epoch
APp
APa APs
mAP
MapTRucCRR23) [22]
24
46.3
51.5
53.1
50.3
16,7*
6
54.7
58.1 56.7 56.5
MapVR/NeeddsS [47]
47.7
54.4
51.4
51.2
16.7*
54.6
60.0
58.0
57.5
GeMap| [49]
49.2
53.6
54.8
52.6
13.2*
PivotNetnccW2211 [9]
24
56.2
56.5
60.1
57.6
11.1*
30 m
BeMaaNettcwve:211 [33]
30
57.7
62.3
59.4
59.8
9.7*
MapTRy2jjucv24112 [23]
24
59.8
62.4
62.4
61.5
15.6
6
62.9
72.1 67.1
67.4
60 x
Stream/MapNet/wweeaaaa [46]
24
62.9
12.5*
6
62.0
59.5
63.0
61.5
SQD-MapNeetooopiin [41]
63.0
62.5
63.3
63.9
64.9
60.2
64.9
63.3
MGMapicVve:24 [25]
61.8
65.0
67.5
64.8
12.3*
[27]
24
63.4
68.0
67.7
66.4
14.2*
6
64.3
72.3
68.1 68.2
HVMM:: [51]
30
62.6
68.4
69.1
66.7
9.7*
6
69.0
69.5
70.3
69.6
MapUnveiler (ours)
24
67.6
67.6
68.8
68.0
12.7
6
68.9
73.7
68.9
70.5
100 X 50 m
MapTRRicLR:231 [22]
24
45.5
47.1
43.9
45.5
16.7*
6
47.5
MapTRy2 [23]
24
58.1
61.0
56.6
58.6
15.6
6
66.2
61.4 54.1
60.6
StreamMappettivccvaa [46]
24
62.9
63.1
55.8
60.6
12.5*
6
57.7
SQD-MapNellppepinn [41]
24
67.0
65.5
59.5
64.0
6
66.9
54.9
56.1
59.3
MapUnveiler (ours)
68.0
70.0
68.2
68.7
12.7
69.7 67.1
59.3
65.4
For fair comparisons with state-of-the-ar VHC methods, we follow the standard metric [20, 26] of average precision (AP) under several Chamfer thresholds {0.5m, 1.0m, 1.5m). We report the average AP in three Chamfer thresholds for each semantic map categories of pedestrian crossing, divider, and boundary. We average AP at three Chamfer thresholds and report the results for each semantic map category: pedestrian crossing, divider, and boundary. To validate the scalability in map categories, we further examine our model by learning semantic centerline, and the results are given in Appendix (see Sec. A.2).
4.2 Implementation Details
Our model is trained with 8 NVIDIA A GPUs using batch size of 16. We train MapUnveiler using AdamW [28] with a learning rate of 6 x 10-4 and weight decay of 0.01. We follow the standard setup [22, 23] that training models for the total epochs of 24 with image resolution of 800 450 and the total epochs of with an image resolution of 614 x 614 on nuScenes and Argoverse2 datasets, respectively. The number of the embedding dimension C, memory tokens M, and clip tokens N map elements N₁, points per map element Np are set to 256. 96, 50, 50, and 20 respectively. We use temporal window size T and stride S of 3 and respectively. The spatial size the BEV feature is 100 x 200. We pre-train the frame-level MapNet on the standard setting and then main train our model facilitate the exploitation of the frame-level map information. For fair comparisons, MapUnveiler is pre-trained for 12 and 3 epochs, and then main trained for an additional 12 and 3 epochs, resulting a total 24 and epochs on nuScenes and Argoverse2 datasets, respectively. During main training, we perform ceip-level inference three times and compute losses after each clip-level inference to effectively handle the GPU memory overhead.
4.3 Comparisons
We compare our MapUnveiler against state-of-the-art VHC methods trained with standard settings, which use ResNet50 [12] as a backbone, multi-camera modality, and train for 24 epochs and 6 epochs on nuScenes [3] and Argoverse2 [44] respectively. As shown in Tab. 1, MapUnveiler achieves -tate-of- the-art performance on all validation sets. Specifically, we surpass the state-of of-tthe-rt temporal model (SQD-MapNet [41]) by mAP of 4.1 % and 4.7% on two range settings of nuScenes validation set and 7.2% and 6.1% % on Argoverse2. We also outperform a heavy VHC model (HiMap [51]) by 1.3% and
0.9% on two benchmark sets. Notably, we boost frame-level model (MapTRv2 [23]) by 6.5% and
10.1% % on nuScenes and 3. and 4.8% on Argoverse2. The superiority of our approach is particularly
evident in the long-range setting: mAP on 60x 300 +000 1100050mm settings are 61.5% 11.5%%55666 58.6% (-2.9%)
7

---

Range 60 30 m
Range 100 50 m
Imput traages
Mapi la2 Stoweriighee
Durs
Input Images
GT
MagTRR2 SyramWlopier
Ours
Figure 4: Qualitative comparisons on tWO range variants of nuScenes val set: 60 30m and 10m. We compare our MapUnveiler with MapTRy2 [23] and StreamMapNet [46]. We marked significant improvements from Map and StreamMapNet using green boxes.
Table 2: Experimental results of the long training schedules.
nuScenes
Argoverse2 APa
Range
Epoch
APp
APa
AP,
mAP
Epoch
APp
AP,
mAP
24
67.6
67.6
68.8
68.0
68.9
73.7
68.9
70.5
60 x 30 m
48
69.5
69.4
70.5
69.8
12
69.0
74.9
69.1
71.0
110
71.0
69.1
71.8
70.6
30
72.5
74.2
71.9
72.9
68.0
70.0
68.2
68.7
6
69.7
67.1
59.3
65.4
100 x 50
48
68.4
71.2
68.3
69.3
12
70.4
66.8
59.3
65.5
110
71.2
71.7
72.2
71.7
30
71.7
67.9
62.6
67.4
(+0.7%) for MapTTRv2 [23] and ours. Although MapUnveiler incorporates temporal modules, we achieve reasonable inference speed (12.7 FPS) compared to frame-level MapNet (MapTRv2 [23], 15.6 FPS), surpassing both performance and speed compared to the state- of-the-art (HiMap [51], 9.7 FPS).
and
Qualitative Comparison. Fig. shows qualitative comparisons on two perception range settings on nuScenes benchmark. We compare our MapUnveiler with the state-of-the-an frame-level model (MapTRvv2 [23]) and temporal model (StreamMappeet [46]). As shown in the figure, MapTRv2 often mis-detects complex pedestrian crossings and cannot precisely predict dividers occluded by vehicles on the road. StreamMapNet struggles to accurately predict the boundaries of intersections, despite leveraging temporal information. In contrast, MapUnveiler consistently delivers accurate results for all map categories most cases. We provide more qualitative results on various scenes in Appendix and full-frame results through the supplementary video.
4.4 Analysis Experiments
In this study, present extensive experimental results on nuScenes benchmark and provide analysis.
Long Training Schedules. Tab. 2 demonstrates the performance improvement achieved through longer training schedules. We obtained (24, 48, 1 110) and (6. 12, 30) epoch models by pre-training for (12, 24, 24) and (3, 6) epochs, and then main training for (12, 24, 86) and (3, 6, 24) epochs on nuScenes and Argoverse2 datasets, respectively. As given in the table, the performance consistently improves with longer training, although the gains are relatively marginal. We conjecture that our MapUnveiler converges rapidly because we train the temporal modules from a pre- pre-trained frame-level MapNet. Since our model shows no significant gain from the 1 110 epoch model on nuScenes, we opt for 48 epochs for the remaining analysis experiments.
Robustness to Occlusion. Tab. presents results on our challenging validation splits collected based on the nearest dynamic objects (detailed in Sec. 4.1). For comparisons, we evaluate MapTRRv2 and StreamMapNer using the code and pre-trained weight provided in each official repository. As shown in the table, MapUnveiler surpasses MRp2 and Stream MapNet in all evaluated metrics. In partieular, existing approaches degrade performance significantly compared to the results on the standard
8

---

Table 3: Experimental results under heavy occlusions.
Table 8: Input variants write.
Method
AP,
APa
APb
mAP
Write
APp APa AP, mAP
Map [23]
24.1
62.2
55.2
47.1
None
65.6 67.6 68.0 67.1
StreamMapNet [46]
32.1
59.4
67.9
53.1
Unlip
68.7 68.9 69.6 69.1
MapUnveiler (ours)
38.6
80.6
72.2
63.8
Uskip + Ug
69.5 69.4 70.5 69.8
UL + -U LULL
UBEV
68.5 68.8 69.5 68.9
Table 4: Ablation study on the proposed modules.
Method
APp
APa
APb
mAP
Table 9: Temporal window size T and stride S.
T
S
APp p
APd
APb
mAP
MapTRv2 [23]
58.8
61.8
62.8
61.2
Intra-clip Unveiler
65.6
67.6
68.0
67.1
1
63.5
66.0
66.1
65.2
Inter-clip Unveiler
69.5
69.4
70.5
69.8
3
69.0
69.4
69.4
69.3
3
2
69.5
69.4
70.5
69.8
Table 5: Speed analysis of the proposed modules.
3
3
68.6
68.7
69.6
68.9
5
3
70.4
69.5
70.5
70.1
Method
FPS
GPU (MB) Params (MB)
MRp [23]
15.6
830.4
76.4
Table 10:
Variant memory token sizes.
Intra-clip Unveiler
13.1
1552.5
144.0
M
APp
APa
APb
mAP
+Iteer-llpp Unveiler
12.7
1614.9
213.9
24
68.2
68.3
69.1
68.5
48
68.4
69.3
69.9
69.2
Table 6: Results with freezing frame-level MapNet.
96
69.5
69.4
70.5
69.8
Read
APp
APd
APb
mAP
192
69.0
69.4
70.1
69.5
384
70.0
69.3
69.9
69.7
Freeze
64.9
66.5
68.7
66.7
End-to-End
69.5
69.4
70.5
69.8
Table 11:
Variant clip token sizes
N₂
APp
APa
AP,
mAP
Table 7: Input variants in read.
25
67.8
69.6
70.1
69.2
Read
AP,
APa
APh
mAP
50
69.5
69.4
70.5
69.8
Memory
68.3
68.5
70.0
68.9
100
67.9
68.1
69.6
68.5
Memory + Quap
69.5
69.4
70.5
69.8
200
69.3
68.8
70.2
69.4
split, i.e., $1.5%-+47.11 (-14.4%) in MapTRv2 and 62.9%->53.l (-9.8%) in StreamMapNet, MapUnveiler also shows a performance degradation 19.8%%-3..89 (-6.0%), but it demonstrates a smaller performance gap compared to previous studies.
Effectiveness of The Proposed Modules. Tabs. and show ablation study and speed analysis of our two proposed modules: Intra-clip Unveiler and Inter-elip Unveiler. We started with a frame- level MapNet, MapTRv2 [23], obtaining the result by reproducing it the result, Intra-clip Unveiler significantly boosts the performance from 61.2% to 67. 37.1% (+5.9% by mapping within clip, and it rarely requires extra computation (15.6 FPS -> 13.1 FPS). Inter-clip Unveiler further improves performance from 67.1% % to 69.8% (+2.7%) by building global relationship of the mapping. However, our approach requires approximately and three times more parameters compared to the frame-level MapTRv2 model during inference. This could be considered potential limitation of method, but the amounts not large.
system. As shown in
times more peak GPU memory
Frozen Frame-level MapNet. Tab. presents the results of MapUnveiler where frame-level MapNet is frozen and only Intra-clip Unveiler and Inter-clip Unveiler are trained. Despite the frozen model consuming the same amount of computation during inference, it effectively reduces GPU memory consumption and converges quickly during training. As given in the table, we obtained slightly lower performance compared the end-to-end training setting. Interestingly, however, the frozen model also achieves the state-of-the-an performance on nuScenes benchmark (current SOTA performance is mAP of 66.7%, the concurrent work of HIMap [51]). This indicates that our approach effectively unveils the hidden maps in the frame-level representations and does not rely solely on strong BEV representation learning.
Ablation Study on Read and Write. As given Tab. 7, reading only from memory is not enough, and feeding map tokens facilitates the understanding of the current map information (68.9% +69.8%)

---

We also ablate the input in write, and the result is given in Tab. 8. We significantly improve the per- formance by writing clip tokens to the memory (67.1%-69.. 1)). This suggests that the propagation of memory tokens facilitates constructing the global map. By additionally writing map tokens, we further boost the performance by explicitly providing the current map information (69. 19.>6% (...)... Unfortunately, however, we cannot obtain performance gain by writing dense BEV feature addition- ally. We conjecture that memorizing different types of features, i.e., vectorized representation of tokens and rasterized representation of dense features, may disturb training and tend to converge to sub-optimal state.
Temporal Window Size T' and Stride S. We train MapUnveiler with various temporal window size T and stride S. As shown in Tab. 9, increasing the temporal window leads performance improvement by interacting with more frames in a clip. However, training with a T = 5 setting requires > 40GB of GPU memory, limiting the GPU models. If we train MapUnveiler with T = 3, it consumes < 32GB of GPU memory. making it possible to train with various GPU models, and it achieves comparable performance. Additionally, selecting either too short (S (S = 3) a temporal stride yields sub-optimal results. Therefore, we opt for T = 3 and S = 2 as the default setting.
or long
Memory Token Size M. Tab. 10 presents the experimental results with various memory token sizes. Selecting memory token size of 96 or above does not significantly change the performance, whereas choosing a size smaller than 96 results in considerable performance degradation. This indicates that MapUnveiler does not require large memory token size to store the 50 clip tokens and 50x 20 map tokens, demonstrating that it performs memory-effciiently.
Clip Token Size No. Tab. 11 gives the results with various clip token sizes. MapUnveiler achieves state-of-the-art performance of 69.2% using only 25 clip tokens. We further boost the performance by using a larger clip token size of 50. However, employing clip token sizes beyond 50 disrupts learning and leads performance degradation.
More Analysis in Appendix. We provide ad- ditional experimental results including center- line, 3D map construction, geo-disjoint split, various backbones, and limitation analysis in Appendix.
4.5 Limitation
MapUnveiler takes temporally consecutive frames input. Therefore, if an intermediate frame is not correctly inputted due to commu- nication eπrors in real-world scenarios, the un- veiling within the clip may not be performed properly. We expect that the performance will recover quickly from the subsequent unveiling pipeline. Additionally, if the model cannot see clear region across all frames as shown in Fig. 5, MapUnveiler may fail to recognize the occluded map information.
Rlder
Indust
Figure 5: A limitation under heavy occlusions. Ma- pUnveiler initially roughly predicts the boundary where we highlighted in green. However, the in- visible regions are continuous, and MapUnveiler eventually predicts the region as having no bound- ary.
5 Conclusion
In this paper, we present a new VHC paradigm that constructs clip-level maps to efficiently incorporate conventional offline mapping strategies. In contrast to the recent temporal VHC models that do not consider the cumulative impacts of occluded map elements and directly propagate the noisy BEV features, we unveil the hidden map and noise in BEV features by interacting with compact clip tokens. To establish global mapping efficiently, we propagate the clip tokens instead of dense BEV features. With these two advanced modules, we propose MapUnveiler, which significantly outperforms previous works challenging scenarios such as long-range perception and heavy occlusions. Since we introduce a novel insight into online VHC approaches to incorporate mapping strategy efficiently, we hope that our research motivates follow-up studies to delve deeper into establishing global mapping online and leads practical VHC solutions.
10

---

References
[1] Angelos Amditis, Matthaios Bimpas, George Thomaidis, Manolis Tsogas, Mariana Netto, Said Mammar, Achim Beutner, Nikolaus Möhler, Tom Wirthgen, Stephan Zipser, et al. A situation-adaptvve lane-keeping support system: Overview of the safelane approach. IEEE Transactions on Intelligent Transportation
Systems, 11(3):617-633 6297, 2010.
[2] Mikhail S Burtsev, Yuri Kuratov, Anton Peganov, and Grigory V Sapunov. Memory transformer. arXiv
preprint 2rv::20066 1527, 2020. 3
[3] Holger Cacsar, Varun Bankiti, Alex H Lang. Sourabh Vora, Veniee Erin Liong, Qiang Xu, Anush Krishnan, Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving.
In CVPR, pages 11621-16331 2020. 2, 6,7
[4] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, pages 213 229. Springer, 2020. 3 [5] Shaoyu Chen, Tianheng Cheng, Xinggang Wang, Wemming Meng, Qian Zhang, and Wenyu Liu. Efficient and robust 2d-to-bev representation learning via geometry-guided kemnel transformer. arXiv preprint
arXiv:2206.044 04584, 2022. 1,3
[6] Zhilu Chen and Xinming Huang. End-to-end learning for lane keeping of self-driving cars. In IV, pages 1856-1866 IEEE, 2017.
[7] Ho Kei Cheng, Scoung Wug Oh, Brian Price, Joon- Young Lee, and Alexander Schwing. Putting the object back into video object segmentation. arXiv preprint arXiv:2310. /2982, 2023.
[8] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. In NeurlPS Workshops, 2014. 3
[9] Wenjic Ding, Limeng Qiao, Xi Qiu, and Chi Zhang. Pivotnet: Vectorized pivot learning for end-to-end hd
map construction. In ICCV, pages 3672-3682, 2023. 7
[10] Hugh Durrant-Whyg Whyte and Tim Bailey. Simultaneous localization and mapping: part i. IEEE robotics audomation magazine, 13(2)99-110, 2006. 20066 3
[11] Xunjiang Gu, Guanyu Song, Igor Gilitschenski, Marco Pavone, and Boris Ivanovic. Producing and leveraging online map uncertainty in trajectory prediction. arXiv preprint arXiv:2403. /6439, 2024. [12] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.
In CVPR, pages 770-778. 2016. 4, 7. 14
[13] Miran Heo, Sukjun Hwang. Seoung Wug Oh, Joon- Young Lee, and Seon Joo Kim. Vita: Video instance segmentation via object token association. In NeurIPS, volume 35, pages 23109-231200 2022. 3
[14] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9()): ((8):1735-1788) 1780, 1997. 3
[15] Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du Tianwei Lin. Wenhai Wang. et al. Planning- oriented autonomous driving. In CVPR, pages 17853-17862, 2023. I [16] Junjie Huang and Guan Huang. Bevpoolv2: A cutting-edge implementation of bevdet toward deployment.
arXiv preprint arXiv:2211.1111 20221.2222022...... 4
2022.
[17] Sukjun Hwang, Miran Hco, Scoung Wug Oh, and Scon Joo Kim. Video instance segmentation using frame communication transformers. In NeurlPS, volume 34, pages 33352-13363 2021. 3
[18] Hung Le. Truyen Tran, and Svetha Venkatesh. Learning to remember more with less memorization. In ICLR, 2019. 3
[19] Youngwan Lee and Jongyoul Park. Centermask: Real-time anchor-free instance segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13906-13915, 2020.
[20] Qi Li. Yue Wang. Yilun Wang, and Hang Zhao. Hdmapnet: An online hd map construction and evaluation framework. In ICRA, pages
IEEE, 2022. 3, 7
[21] Zhiqi Li, Wenhai Wang. Hongyang Li. Enze Xie. Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bev- former: Learning bird' v-eye-view representation from multi-camera images via spatiotemporal transformers. In ECCV, pages 1-18. Springer. 2022. 1.3
[22] Bencheng Liao, Shaoyu Chen, Xinggang Wang, Tianheng Cheng, Qian Zhang, Wenyu Liu, and Chang Huang. Maptr: Structured modeling and learning for online vectorized hd map construction. In ICLR,
2023. 1, 2. 5, 6, 7. 13
[23] Bencheng Liao, Shaoyu Chen, Yunchi Zhang. Bo Jiang, Qian Zhang, Wenyu Liu, Chang Huang, and Xing- gang Wang. Maptrv2: An end-to-end framework for online vectorized hd map construction. International
Journal of Computer Vision, pages 1-23, 2024. 1, 2, 3, 4, 6, 7, 8, 9, 13, 14
[24] Chang Liu, Seungho Lee, Scott Varnhagen, and H Eric Tseng. Path planning for autonomous vehicles
using model predictive control. In IV. pages 174-179. IEEE, 2017.
[25] Xiaolu Liu, Song Wang, Wentong Li, Ruizi Yang, Junho Chen, and Jianke Zhu. Mgmap: Mask-guided learning for online vectorized hd map construction. In CVPR, pages 14812-1482. 2024. 7
[26] Yicheng Liu, Tianyuan Yuan, Yue Wang, Yilun Wang, and Hang Zhao. Vectormapnet: End-to-end vectorized hd map learning. In ICML, pages 22352 22369. PMLR, 2023. 2, 3, 7, 13

---

[27]
Zihao Liu. Xiaoyu Zhang, Guangwei Liu, Ji Zhao, and Ningyi Xu. Leveraging enhanced queries of point sets for vectorized map construction. In ECCV, 2024. 7
[28]
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
[29]
Jean-Arcady Meyer and David Filliat. Map-based navigation in mobile robots:: Ii, review of map-lcarning
and path-planning strategies. Cognitive Systems Rexearch, 4(4):283-317. 2003. 1
[30] Sriram Narayanan, Ramin Moslemi, Francesco Pittaluga, Buyu Liu, and Manmohan Chandraker. Divide-
and-conquer for lane-aware diverse trajectory prediction. In CVPR, pages 15799-15808, 2021.
[31] Scoung Wug Oh, loon-Young Iee, Ning Xu, and Scon Jon Kim. Video object segmentation using
space-time memory networks. In ICCV, pages 9226-9235, 2019. 3
[32] Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly
unprojecting to 3d. In ECCV, pages 194-210. Springer, 2020. 1, 3,3, 4
[33] Limeng Qiao, Wenjie Ding, Xi Qiu, and Chi Zhang. End-to-end vectorized hd-map construction with piecewise bezier curve. In CVPR, pages 13218-13228, 2023. 7
[34] Jack W Rac, Anna Potapenko, Siddhant M Jayakumar, and Timothy P PLillicrap. Compressive transformers for long-range sequence modelling. In ICI.R, 2019.
[35] Michael Ryoo, AJ Piergiovanni, Anurag Arnab, Mostafa Dehghani, and Anelia Angelova. Tokenlearner:
Adaptive space-time tokenization for videos. In NeurlPS, volume 34, pages 12 12786-12797, 2021. 5
[36] Michael S Ryoo, Keerthana Gopalakrishnan. Kumara Kahatapitiya, Ted Xiao, Kanishka Rao, Austin Stone, Yao Lu, Julian Iharz, and Anurag Arnab. Token turing machines. In CVPR, pages 19070-19081, 2023. 3,
[37] Tixiao Shan and Brendan Englot. Lego- lego-loam: Lightweight and cooumd-op lidar odometry and mapping on variable terrain. In JROS, pages 4758-4765 IEFF, 2018. 1,3
[38] Tixiao Shan, Brendan Englot, Drew Meyers, Wei Wang. Carlo Ratti, and Daniela Rus. Lio-sam: Tightly- coupled lidar inertial odometry via smoothing and mapping. In IROS, pages 5135-5142. IEEE, 2020. 1,
[39] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NewrlPS, volume 30, 2017. 3,5
[40] Jingke Wang, Tengju Ye, Ziqing Gu, and Junbo Chen. Lip: Lane- based trajectory prediction for autonomous driving. In CVPR, pages 17134-17142, 2022. 1
[41] Shuo Wang, Fan Jia, Yingfei Liu, Yucheng Zhao, Zehui Chen, Tiancai Wang. Chi Zhang. Xiangyu Zhang, and Feng Zhao Stream query denoising for vectorized hd map construction. arXiv preprint
arXiv:2401.099 200002200.9912,.. 2024. 7,7,
[42] Shihao Wang. Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. In ICCV, pages 3621-3631, 2023. 3
[43] Yuging Wang. Zhaoliang Xu, Xinlong Wang. Chunhua Shen, Baoshan Cheng, Hao Shen, and Huaxia Xia. End-to-end video instance segmentation with transformers. In CVPR, pages 8741-8750, 2021. 3, 6 [44] Benjamin Wilson, William Qi. Tanmay Agarwal, John Lambert, Jagject Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays. Argoverse Next generation datasets for self-driving perception and forecasting. In NeurIPS Datasets and Benchmarks, 2021. 2, 6,
[45] Linjie Yang, Yuchen Fan, and Ning Xu. Video instance segmentation. In ICCV, pages 5188-5197, 2019.
[46] Tianyuan Yuan, Yicheng Liu, Yue Wang. Yilun Wang, and Hang Zhao. Streammapnet: Streaming mapping network for vectorized online hd map construction. In WACV, pages 7356-7365, 2024. 2, 3, 7, 8, 9, 13, 14
[47] Gongjic Zhang, Jiahao Lin, Shuang Wu, Zhipeng Luo, Yang Xue, Shijian Lu, Zuoguan Wang, et al. Online map vectorization for autonomous driving: A rasterization perspective. In NeurIPS, volume 36, 2023. 7
[48] Ji Zhang and Sanjiv Singh. Loam: Lidar odometry and mapping real-time. In Robotics: Science and systems, volume 2. pages 1-9. Berkeley, CA, 2014. 1,
[49] Zhixin Zhang, Yiyuan Zhang, Xiaohan Ding, Fusheng Jin, and Xiangyu Yue. Online vectorized hd map construction using geometry. In ECCV, 2024.
[50] Brady Zhou and Philipp Krähenbühl. Cross-view transformers for real-time map-view semantic segmenta-
tion. In CVPR, pages 37760-13769 2022. 1,3
[51] Yi Zhou, Hui Zhang, Jiaqian Yu, Yifan Yang. Sangil Jung, Scung- In Park, and ByungIn Yoo. Himap: Hybrid representation learning for end-to-end vectorized hd map construction. In CVPR, pages 15396-15406, 2024. 7, 8, 9, 13
[52] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-ene and object detection. In ICLR, 2021. 3,5 5
12

---

A Appendix
A.1 Clip-level Inference Scheme
Our MapUnveiler runs clip-level. Specifically, the k-th clip of C₃ includes
Cs = [fttt=ss+1
MapUnveiler infers from 0. fa represents frame at time t, and [Jt]t=k541 denote sequence of consecutive frames from kSS to &S + T. where S and T indicate the clip stride and the temporal window, respectively. Therefore, when T 3 and S 2, obtain clip sets such as Co [2, /3), C1 = {Sa, /4. Js) and C {/2x+1, J2443).
A.2 Predicting with Centerline
In Tab. 12, present quantitative comparison centerline predictions. MapUnveiler outperforms MapTRv2 [23] by significant margins on both nuScenes (54.0%~63.0055 +9.0%) and Argoverse (62.6%-468.0 +5,4%) benchmarks. This demonstrates the robustness of our approach various map categories.
Table 12:
Centerline predictions. APp APa. APp. APc indicate the average precision for pedestrian crossing, divider, boundary, and centerline, respectively.
nuScenes
Argoverse2
Method
Epoch
APp p
APa AP, AP. mAP
Epoch
APp APa AP, APc
mAP
MapTRv2uucv21 [23]
24
50.1 53.9 58.8 53.1 54.0
6
55.2 67.2 64.8 63.2
62.6 66.6 68.0
MapUnveiler (ours)
24
59.0
65.7 68.0
59.4
63.0
6
62.2
73.2 70.0
A.3 Extension to 3D Vectorized Map Construction
Tab. 13, presents a comparison on 3D map construction. In this study, we conduct experiment on Argoverse2 dataset. MapUnveiler achieves state-of-the- performance Tab. 13, demonstrating the efficacy of our approach even when extended to 3D VHC.
Table 13: Comparisons on Argoverse2 val set with 3D map construction.
Method
Epoch
APp
APa
APb
mAP
2ap [23]
6
60.7
68.9
64.5
64.7
HIMapicVPR [51]
6
66.7
68.3
70.3
68.4
MapUnveiler
6
66.0
72.6
67.6
68.7
A.4 A More Comparison Geo-disjoint Split
Tab. 14 presents experimental results on a recent gco-disjoint dataset split proposed in StreamMap- Net [46]. In this dataset, the performance of MapTRv2 [23] has been significantly drop from 61.5% to 36.6%. We also successfully achieve state-of-the-art performance on geo-disjoint dataset split.
Table 14: Experimental results on geo-disjoint dataset split proposed in StreamMapNet [46]. The results are obtained on nuScenes with 60x 30m range.
Method
Epoch
AP, p
AP,
AP,
mAP
VectorMapNeuucMM225 [26]
120
15.8
17.0
21.2
18.0
MapTR (CCR [22]
24
6.4
20.7
35.5
20.9
Stream [46]
24
29.6
30.1
41.9
33.9
UucvRv2 [23]
24
37.2
26.5
46.1
36.6
MapUnveiler (ours)
24
43.2
26.5
48.7
39.4
13

---

A.5 Various Backbones
We present experimental results with various backbones: ResNet--8 [12] and V2-99 [19]. As shown in Tab. 15, our method is not limited to MapTRv2 [23] with ResNet50, but can be extended to ResNet18 and V2-99 backbones.
Table 15: Experimental results with various backbones.
nuScenes
Argoverse2 AP, APa AP, mAP
Range Method
Backbone
Epoch
APp APa APb mAP Epoch
MapTRv2 [23]
R18
24
53.3 58.5 58.5 56.8
6
58.8 68.5 64
63.8
30 m
MapTRv2nucv241 [23]
R50
24
59.8
62.4 62.4 61.5
62.9 72.1 67.1
67.4
MapTRv2[ucv244] [23]
V2-99
24
63.6
67.1 69.2 66.6
64.5 72.2 70.1
68.9
MapUnveiler (ours)
R18
24
62.4
65.2 65.7 64.4
63.8 70.1 67.1
67.0
60
MapUnveiler (ours)
R50
24
67.6
67.6 68.8 68.0
68.9 73.7 68.9 70.5
MapUnveiler (ours)
V2-99
24
69.8
72.0 74.7 72.1
69.6 75.1 72.8 72.5
MapTRv2nuCv241 [23]
R18
24
52.7
57.3 51.5 53.8
60.3 57.6 49.6 55,8
100 50 m
MacVRv2uuCV241 [23]
R50
24
58.1
61.0 56.6 58.6
66.2 61.4 54.1 60.6
CipTRR2iucvan [23]
V2-99
24
62.6
67.8 65.2 65.2
68.5 62.1 58.4 63.0
MapUnveiler (ours)
R18
24
64.1
67.3 65.3 65.6
65.4 63.2 54.7 61.1
MapUnveiler (ours)
R50
24
68.0
70,0 68.2 68.7
69.7 67.1 59.3 65.4
MapUnveiler (ours)
V2-99
24
70.5
74.6 73.6 72.9
71.1 61.4 66.2
A.6 Randomly Dropped Intermediate Frames
In the limitations section (Sec. 4.5), we discussed that our MapUnveiler relies on temporally con- secutive frames, and the performance can degrade when intermediate frames are missing. However, we expected that the performance would recover quickly from the subsequent unveiling pipeline. To validate this, we evaluated two models with randomly dropped intermediate frames. Frames were dropped by converting multi-camera images into black images. The experiment conducted with drop rates of 20%, 10%, and 5%, and the results are given in Tab. 16. MapUnveiler is affected by dropped frames, but the performance degradation is reasonable compared to MapTRv2 [23].
Table 16: Experimental results with randomly dropped intermediate frames. The results are obtained on nuScenes with x0330m range
Method
Drop rate
AP,
p
AP
AP,
mAP
0%
58.8
61.8
62.8
61.2
5%
56.6
56.6
58.7
57.7
MapTRy2/ucv244 [23]
10%
53.4
54.8
55.3
54.5
20%
47.2
48.5
38.9
48.2
0%
69.5
69.4
70.5
69.8
5%
58.0
60.6
60.0
66.9
MapUnveiler (Ours)
10%
63.3
64.7
65.0
64.3
20%
66.2
66.9
67.6
59.6
A.7
More Qualitative Results
We present additional qualitative results in Figs. 6, 7, 8, and For all results, we provide multi-camera input and the results of MapTRv2 [23], StreamMapNet [46], and our MapUnveiler.
A.8 Broader Impacts
While our framework notably enhances the performance of online vectorized HD map construction relying on streaming multi-view camera sensors, does not assure flawless prediction of all map elements. Therefore, it is crucial to have backup plan ready for safety-critiaal real-world applications.
14

---

Range 60 30 m
Time
Input Images
GT
MapTRv2 StreamMapNet
Ours
Figure 6: Qualitative comparisons nuScenes val with 60 30m perception range setting.
15

---

Range 60 x 30 m
Time
Input Images
GT MapTRv2 StreamMapNet Ours
Figure 7: Qualitative comparisons on nuScenes val with x0 30m perception range setting.
16

---

Range 100 X 50 m
Time
Input Images
GT
MapTRv2 StreamMapNet
Ours
Figure 8: Qualitative comparisons on nuScenes with 100 x 50m perception range setting.
17

---

Range 100 X 50 m
Time
Input Images
GT MapTRv2 StreamMapNet
Ours
Figure 9: Qualitative comparisons on nuScenes val with 100 50m perception range setting.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The contributions our work are summarized the end of the introduction, aligning with the content of the abstract.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In Sec. 4.5 (Limitation) we addressed and outlined the constraints of the suggested method.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well sell-specffication asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken low lighting. Or speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess..
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknow ledgww in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
19

---

Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: In Sec. 4.2, provide implementation details in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
20

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We plan to release source codes upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //niss.. wttps.cccpublliiiee wwtppcccc//eaeeecccccccccccceeae htt.gguddee//eeeemiiiiieeeee /Cc////odimmsssseeeeee cassionnolccyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: In Sec. 4, we provide detailed information on the dataset and experimental setup.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
21

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma emror bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: In Sec. 4.2, we describe the computing resources used for training the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: / /wwurrrurpp...e/ www.blic//thicsuiieeee.
Answer: [Yes]
Justification: We abide by the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Appendix (Sec. A.8), we describe the broader impacts our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided relcasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Proper citation has been provided for the data, models, and code used in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "nyiAWoHIB9", "reviewer": "Reviewer_JH5U", "summary": "This paper proposes a clip-based vectorized HD map construction paradigm for the processing of long temporal sequence, in which occluded map elements are unveiled explicitly by efficient clip tokens. Through clip token propagation, MapUnveiler achieves effective utilization of long-term temporal map information by associating inter-clip information, in which clip tokens are propagated rather than dense BEV features. Experiments demonstrate that MapUnveiler boosts the performance on public benchmark datasets, also for more challenging setting like long-range perception and heavily occluded driving scenes.", "questions": "What is the experiment result for geometric-based dataset split as mentioned in [1] and [2]? Besides, what is the additional computing costs considering the injection of temporal clip token?\n[1] Yuan, Tianyuan, et al. \"Streammapnet: Streaming mapping network for vectorized online hd map construction.\" Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024.\n[2] Lilja, Adam, et al. \"Localization is all you evaluate: Data leakage in online mapping datasets and how to fix it.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "limitations": "Yes. The authors mentioned the weakness of their approach on the corrupted input.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written and easy-to-follow. Figures clearly conveys the intended message.\n2. “Unveiling the hidden” and clip token propagation are reasonable and effective strategy for static Map element detection, which is practical and alleviates the problem to some extent.\n3. The proposed method demonstrates strong performance on benchmark dataset, comprehensive experiments and ablation studies justify the model design.", "weaknesses": "1. As mentioned at line 227, this work is built on pretrained frame-level MapTRV2 and fine-tuned, thus the comparison can be unfair. Results without pretraining are required to verify your effectiveness.\n2. At line 53 and BEV Updater in line 151, for occluded features, how to select the tokens that are visible in certain frames? Seems tokens within the temporal window are fully utilized for BEV update by cross attention, how to determine whether these tokens contain unblocked information? More explanations are required.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for providing the insightful and constructive feedback. We appreciate your acknowledgment that the paper is easy to follow and that the proposed approach is effective. Below are our responses to each comment, and we will include all the results and comments in the revised version.\n___\n**W1. Results without pre-training**\n\nThank you for valuable feedback. We fully understand that our training pipeline may present an unfair advantage since previous works trained their models on vectorized HD map datasets for only 24 epochs, whereas we pre-train MapTRv2 for 24 epochs and fine-tune MapUnveiler for 24 epochs, amounting to a total of 48 epochs.\n\nTo address this concern, we conducted an experiment where we pre-train MapTRv2 for 12epochs/3epochs and then fine-tune MapUnveil for 12epochs/3epochs on either the nuScenes or Argoverse2 training set. Even though our proposed modules will be trained for only 12epochs/3epochs, this will undoubtedly provide fair comparisons, as we are training a total of 24epochs/6epochs. The results are given in **G1** of the global response.\n\nAs shown in the table, our method still demonstrates state-of-the-art performance of mAP on both nuScenes and Argoverse2 validation sets under a more fair setting.\n\nAdditionally, we tried to skip the pre-training stage and train MapUnveilr from scratch, but it failed to converge, achieving an mAP of 18.2% in the nuScenes (60x30m) validation set. This indicates that our method necessitates meaningful frame-level map features to learn map unveiling. We would like to kindly note that this pre-training strategy is not own approach but is commonly adopted for training networks with temporal information. For instance, StreamMapNet [45] and SQD-MapNet [40] trained their initial 4 epochs with single-frame inputs. MapTracker [A] employed a three-stage training process (BEV Encoder → Vector Decoder → All Parameters) to facilitate initial convergence.\n\n[A] Chen, Jiacheng, et al. \"Maptracker: Tracking with strided memory fusion for consistent vector hd mapping.\" arXiv preprint arXiv:2403.15951 (2024).\n___\n**W2. How to select the tokens that are visible in certain frames**\n\nThe model *automatically* selects the visible and valuable BEV regions through the learning of cross attention. As depicted in Figure 1-(c) in the main paper, each clip token within the temporal window is fully utilized for BEV update. We could limit the clip tokens to select manually choosen visible BEV regions, but we thought that this might not always provide an accurate solution for constructing a clear HD map (*e.g.,* the model might try to select an easy-to-see region even if there are no lanes). Hence, we opted to learn the selection mechanism in an end-to-end manner, which can minimize the losses of the constructed map. This approach is straightforward yet effective, as illustrated in Figure 1: compared to (a) MapTRv2 and (b) StreamMapNet, our BEV feature most clearly represents map elements.\n___\n**Q1. Experiment result for geometric-based dataset split**\n\nThank you for your insightful suggestion. We conducted additional experiments on the geometric-based dataset splits you suggested [1]. The results have been moved to the global response due to the character limit. Please find the result in **G2** in the global response.\n___\n**Q2. Additional computing costs**\n\nWe provided additional computational costs in Table 3 by injecting temporal clip tokens in two aspects: Intra-clip Unveiler and Inter-clip Unveiler. To provide rich information, we further measured GPU memory consumption during inference time and model parameters, and append Table 3. We present new Table 3 in the global response. Please find the result in **G3** in the global response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hLDba7LQ3O", "reviewer": "Reviewer_xabw", "summary": "This work presents a method called MapUnveiler, which aims to improve the construction of vectorized HD maps for autonomous driving. MapUnveiler uses a novel clip-level pipeline to unveil occluded map elements by relating dense image representations with efficient clip tokens and propagating inter-clip information. This approach leverages temporal information across adjacent input frames, addressing the limitations of single-frame and streaming inference methods. The model achieves state-of-the-art performance on the nuScenes and Argoverse2 benchmark datasets, demonstrating promising improvements in challenging scenarios with longer perception ranges and heavy occlusions.", "questions": "1. What do the map queries stand for? Can they be transferred directly to vectorized HD maps?\n2. Is the map decoder adopted from MapTRv2?\n3. Are map tokens generated from the intra-clip unveiler the refined version of map queries?", "limitations": "The limitation of dependency on temporally consecutive frames is discussed.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The introduction of a clip-level pipeline for vectorized HD map construction effectively addresses occlusion issues and leverages temporal information across multiple frames.\n2. The method utilizes clip tokens to propagate map information efficiently, reducing redundant computations and enhancing prediction consistency.\n3. Extensive experiments demonstrate that MapUnveiler achieves state-of-the-art performance on nuScenes and Argoverse2 benchmarks, particularly in challenging scenarios.", "weaknesses": "1. The community has noticed a severe data leakage issue with utilizing nuScenes and Argoverse2 datasets for online mapping evaluation {1, 2}, as these datasets are not intentionally built for online mapping. It might also be necessary to validate the proposed method on geo-disjoint training and validation sets.\n2. It would be good to see the analysis of added model compacity due to the introduction of the proposed intra-clip unveiler and inter-clip unveiler.\n3. It seems the proposed intra-clip unveiler and inter-clip unveiler are adaptable to any single-frame inference online mapping methods. It would be good to validate the effectiveness of the proposed modules on other baseline methods.\n4. The authors are encouraged to investigate the consistency of estimated HD maps across frames of the proposed method compared to existing methods with \"inconsistent and suboptimal prediction results\" (mentioned in Line 7).\n{1} Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps.\n{2} Localization Is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix It.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for providing thorough feedback and interesting suggestions. We are grateful for your acknowledgment that the introduction of a clip-level pipeline for vectorized HD map construction is effective and the proposed clip tokens propagate map information efficiently. Below are our responses to each comment, and we will include all the results and comments in the revised version.\n___\n**W1. Validation on geo-disjoint dataset**\n\nThank you for your insightful feedback. To address this concern, we conducted additional experiments on a recent geo-disjoint dataset split. Due to the character limit, we attached the full table in the global response. Please find the results in **G2** in the global response.\n___\n**W2. Analysis of added model compacity**\n\nThat would be a good analysis to provide rich information about our model for readers. Additional to the analysis of accuracy and computational costs of the proposed intra-clip Unveiler and inter-clip Unveiler (Table 3 in the main paper), we add GPU memory consumption during inference time and the model parameters. We present the table in the global response. Please find the results in **G3** in the global response.\n___\n**W3. Validate the proposed modules on other baseline methods**\n\nThat's a valuable suggestion. We believe that any single-frame inference online mapping methods based on DETR (which outputs both rasterized BEV and vectorized query features) can be adapted for our MapUnveiler. Therefore, it would be exciting to additionally claim our contribution in modularization. Consequently, we attempted these experiments with DETR-based models for which the code is available online (MapVR [46], MGMap [24], MapTRv1 [21]). Unfortunately, we are unable to present the results at this time due to a lack of resources and time to set up and conduct experiments within the short rebuttal period. Nevertheless, we would like to address the concern regarding this query, so we experimented with various backbone networks (ResNet-18 and V2-99 {3}). We present the result table in **G4** in the global response.\n\nAs shown in the table, our method is not limited to MapTRv2 with ResNet50, but can be extended to ResNet18 and V2-99 {3} backbones. This suggests that our method can also work for other various frame-level features as well.\n\nWe plan to implement our method on other single-frame inference online mapping methods and include the results in the final version.\n\n{3} An energy and GPU-computation efficient backbone network for real-time object detection, CVPRW. 2019.\n___\n**W4. Consistency of estimated HD Map across frames**\n\nThanks for your thorough feedback. Measuring the consistency of the model would be very helpful and further elaborate the contribution of our model. To quantitatively measure the consistency of the model, we would need to indicate the track ID of each estimated map element. However, this is challenging in our model because we simplify and propose a straightforward pipeline that does not require any complicated spatial warping process across time, which is required for streaming inference (StreamMapNet [45]), nor track ID annotation. Thus, we attempt to show the consistent results qualitatively in Figures 4, 5, 6, 7, and 8.\n\nWe recently found that MapTracker {4} presented a consistency-aware mAP (C-mAP) metric and measured it for baseline methods (*e.g.,* MapTRv2) by predicting track IDs through their tracking algorithm. It seems that we can also measure the C-mAP by applying the tracking algorithm proposed in {4}. To measure it, we have to re-train the model with the refined GT data proposed in {4}, so we are unfortunately unable to show the C-mAP result currently due to a lack of resources and time.\n\nWe plan to implement C-mAP and add the results in the final version.\n\n{4} MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mapping, arXiv:2403.15951\n___\n**Q1. Map queries**\n\nThat's correct. Map queries are trained with the same objective function as the map tokens, so it can be transferred directly to vectorized HD maps. Thus, MapUnveiler can be degraded to MapTRv2 if we transfer vectorized HD maps using map queries instead of map tokens. An interesting experiment can be conducted based on this idea; we can use MapUnveiler for both frame-level and clip-level scenarios (as sometimes we cannot utilize temporally consecutive frames due to unexpected communication/sensor errors in real-world scenarios). We experimented by replacing map tokens with map queries for every two frames, resulting in a slight performance degradation from 69.8% to 68.0% in nuScenes (60x30m).\n\nTo clarify this, we will add following explanation in Line #130 (we highlighted the new descriptions in *italics*):\n> ... the  map decoder, respectively. *BEV features represent rasterized map features, whereas map queries embed vectorized map information; thus, we can directly construct vectorized HD maps using the map queries.*\n___\n**Q2. Map decoder**\n\nThat's correct. The map decoder is adopted from MapTRv2. To clarify this, we will revise the caption in Figure 2:\n> ... vectorized HD maps. *All components in the frame-level MapNet (i.e., Backbone, PV to BEV, Map Decoder) are adopted from MapTRv2 [22].* The frame-level MapNet extracts ...\n___\n**Q3. Map tokens**\n\nThat's correct. The map tokens generated from the intra-clip unveiler are the refined version of map queries. To clarify this, we will revise the paper in Line #158:\n> ... in the previous step. *The objective of this step is to generate a refined version of frame-level map queries.* As illustrated in ...", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yGJBNRHryO", "reviewer": "Reviewer_gpqV", "summary": "The authors propose a new approach for constructing vectorized high-definition maps that exploits temporal information across adjacent input frames.  The model, which they call MapUnveiler, operates at the clip-level and consists of an intra-clip unveiler which generates vectorized maps for T frames and an inter-clip unveiler which uses a memory module to aggregate information between clips. The authors present results on two standard benchmarks, vectorized HD map construction benchmarks (nuScenes and Argoverse2) and demonstrate the model’s superior quantitive performance to several previously proposed approaches. They also show several qualitative examples of how MapUnveiler can better handle occlusions in the input images.", "questions": "1. Have the authors tried quantized models to reduce GPU memory? It could be interesting to see if the gains from larger window sizes outweigh the losses from quantization.\n2. The model still seems to struggle with some occlusions (a 6% drop from the standard split). Why do the authors think that is? Are these just very difficult cases or issues with the model?\n3. The one limitation that was discussed seems like it can be tested. How does randomly dropped intermediate frames affect model performance?", "limitations": "Only one limitation is included. I encourage the authors to think through other potential limitations.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- The paper is well-written and contextualized well within prior work.\n- The methodology is novel and well-motivated.\n- The results are strong on the two tested datasets, both quantitatively and qualitatively.\n- Many different analyses and ablations were included to justify the design decisions used within MapUnveiler and show its strengths.", "weaknesses": "1. The methods is dense and a bit hard to read. The architecture figures help but are also a bit difficult to parse through. It would be helpful to try to weave more intuition into the text.\n2. Claiming \"-9.8%\" is significant but \"-6.0%\" is comparable in the robustness to occlusion section seems a bit arbitrary (and potentially overstating MapUnveiler's performance, as a 6% drop is still considerable). I suggest the authors rephrase this sentence (and address similar claims in the paper).\n\nThere are several typos throughout the paper. I have enumerated some here, but encourage the authors to do a detailed proofread:\n- 127: With there\n- 129: mapnet -> MapNet\n- 161: bev -> BEV\n- 167 parenthesis \n- 192 backwards parenthesis \n- 294: In addition, if we choose too short", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are particularly encouraged that the reviewer finds our method novel and well-motivated. And we highly appreciate your constructive comments and suggestions. Below are our responses to each of your queries, and we will include them in the revised paper.\n___\n**W1. Weave more intuition into the text in 3. Method**\n\nWe will include our motivation in the method explanation to make it easier to follow and understand our method. For this, we will revise the paper as below (we highlighted the new and revised descriptions in *italics*):\n\n*Clip Token Generator (Line #149-150)*\n> ... , respectively. *To globally gather intra-clip map features, we opt for a naive cross-attention [38]. Through this step, we obtain compact clip-level map representations, enabling efficient intra-clip communication with small-scale features.*\n\n*BEV Updater (Line #155)*\n> ...  original bev features. *The main idea of BEV Updater is to avoid heavy computation in spatio-temporal cross attention. To achieve this, we do not directly communicate intra-clip BEV features, but instead decouple the spatial BEV features and temporal clip tokens. We then update the spatial BEV features with compact temporal clip tokens, effectively communicating spatio-temporal information with reasonable computational costs.* The updated BEV ...\n\n*Map Generator. (Line #161)*\n> ...  the updated BEV features $U_l^{BEV}$. *Since the updated BEV features are spatio-temporally communicated, we directly extract map tokens $U_l^{map}$. Each map token represents a vectorized map element through a 2-layer Multi-Layer Perceptron (MLP).* The map tokens ...\n___\n**W2. Rephrase some sentences**\n\nWe are sorry, we understand that it may be an overstatement. We will rephrase Line 261 as follows:\n> ~, while MapUnveiler shows comparable performance~ *MapUnveiler also shows a proformance degradation of 69.8%→63.8% (-6.0%), but it demonstrates a smaller performance gap compared to previous studies.*\n\nFollowing your advice, we also tone down following sentences (Line 242):\n> *Although MapUnveiler incorporates temporal modules, we achieve a* ~fast~ *resonable inference speed (12.7 FPS) compared to frame-level MapNet (MapTRv2 [22], 15.6 FPS)*, surpassing both performance and speed compared to the heavy state-of-the-art approach (HiMap [49], 9.7 FPS).\n___\n**W. Typos**\n\nThank you for your diligent efforts to enhance the quality of our paper. We will rectify the mentioned typos and strive to correct any other potential typos. We will thoroughly proofread the paper from the beginning.\n___\n**Q1. Model quantization**\n\nThat's an intriguing idea, and we hope to see the comparison. Unfortunately, we tried to implement the quantized models, but it was quite challenging to implement and evaluate the accuracy, memory usage, and speed of the quantized model. The primary reason behind this is that we developed our MapUnveiler using several custom addition and multiplication operators, which the FakeQuantize function does not support. Implementing it with TensorRT necessitates reconstructing the entire pipeline of our code, hence requiring additional time.\n\nBased on current knowledge, we can consider extreme quantization (4-bit) and a general solution used in TensorRT (16-bit and 8-bit). The performance drop due to 4-bit model quantization may be marginal in CNN architecture if we use advanced quantization algorithms such as [A]. However, transformer-based architectures currently lose more than 20% of the accuracy [B] with 4-bit quantization. As an alternative, if we adaptively combine FP16 and INT8 [C], we can accelerate the model speed x1.87 times with a marginal performance drop (about -1.19%). With this, we can boost MapUnveiler with a large window size (T=5) through quantization. However, the performance gain from a wide window size is marginal (mAP of 70.1% for T=5, and mAP of 69.8% for T=3) as shown in Table 7 of the main paper. We conjecture that our model effectively leverages long time dependencies through the inter-clip Unveiler, thus it may not be necessary to use a large window size. But it would be interesting to see the gains of the main model from quantization.\n\nWe plan to implement the quantized model and include the results in the final version.\n\n[A] QDrop, ICLR 2022\n\n[B] RepQ-ViT, ICCV 2023\n\n[C] https://github.com/DerryHub/BEVFormer_tensorrt\n___\n**Q2. The reason for strugglgling with some occlusions**\n\nWe found that MapUnveiler may be unable to unveil some regions if the regions are occluded for every frame. We visualize an example in **Figure 9 of the rebuttal PDF file** of the global response. MapUnveiler initially roughly predicts the boundary where we highlighted in green. However, the invisible regions are continuous, and MapUnveiler eventually predicts the region as having no boundary. As such, if the model cannot see a clear region across all frames, MapUnveiler may fail to recognize the occluded map information.\n___\n**Q3. Randomly dropped intermediate frames**\n\nA very constructive suggestion. We evaluated three models with randomly dropped intermediate frames. Frames were dropped by converting multi-camera images into black images. The experiment was conducted with drop rates of 20%, 10%, and 5%, and the results are given in **Table 13 of the rebuttal PDF file** of the globalresponse. MapUnveiler is affected by dropped frames, but the performance degradation is reasonable compared to MapTRv2.\n___\n**L. Other potential limitations**\n\nOne can be a potential limitation that was discussed in **Q2**; MapUnveiler is likely to fail in unveiling fully occluded roads across all frames.\n\nAdditionally, MapUnveiler requires marginal additional computational costs (15.6 FPS → 12.7 FPS) but requires approximately two times more GPU memory (830.4 MB → 1614.9 MB) and three times more parameters (76.4 MB → 213.9 MB) compared to the frame-level MapTRv2 model. We provide a comparison table in **G3** of the global response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "thlV3lCaCJ", "reviewer": "Reviewer_Hway", "summary": "This paper aims to improve vectorized HD map construction for autonomous driving. Inspired by the global feature association in traditional offline HD mapping, the proposed MapUnveiler processes input frames in a clip-based manner and hopes to resolve occlusions using information from previous frames. Built up MapTRv2, MapUnveiler introduces clip tokens together with the Inter-clip and Intra-clip Unveiler modules to update the map queries with temporal information. Experiments on nuScenes and Argoverse2 datasets demonstrate the superior performance of the proposed method, especially on highly-occluded scenes.", "questions": "The paper studies an important problem (temporal information association) in online HD map construction and proposes a reasonable method. However, the poor clarity and the potentially incomplete/unfair comparison results raise serious concerns about the paper's quality and reproducibility. My current rating is reject, and I will consider changing the score if the main weaknesses are properly addressed.", "limitations": "The limitations and broader impacts are adequately discussed in the paper.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "1. The idea of incorporating and aggregating clip-level information for online vectorized HD mapping is reasonable and is more akin to how humans drive. The proposed method has more thoughtful designs than early works such as StreamMapNet to better handle occlusions and incorporate long-range information. \n\n2. The proposed MapUnveiler obtains state-of-the-art results in various experimental settings. The improvements over previous methods are especially prominent in the large 100mx50m setting and the highly-occluded scenes collected by the authors. \n\n3. Extensive ablation studies enumerate the choices of almost all hyper-parameters or model components, which helps better understand and break down each element's contributions.", "weaknesses": "1. The clarity of the method description is poor, making it very hard to thoroughly understand the proposed architecture. Details are discussed below:\n    \n    - The method explanation is not self-contained:  i) The Inter-clip Unveiler section refers to the TTM and directly skips all details. There is no information at all about how is the compact memory token generated from the denser map queries;  ii) The \"loss\" section refers to MapTRv2 and again skips all details. The authors should not assume the general audience to be aware of the concrete details of TTM and MapTRv2. The core formulation of these components should be elaborated with texts or equations, while full details can go to the appendix.\n    - The definitions of the temporal window T and the stride S are unclear. Based on the text descriptions and the common definition of stride, my understanding of \"T=3 and S =2\" is that \"each clip has 3 frames, and every two consecutive frames have a temporal gap of 1.\" However, the symbols in L177-178 seem to suggest other meanings of T and S.\n    - The description of the inference mechanism is also vague. Is the MapUnveiler executed per frame or per clip? Figure 2 seems to suggest the per-clip inference where the predictions of T frames are obtained together. If this is the case, does it hurt the actual response frequency? \n    \n     In short, Section 3 of the paper lacks significant details, and I cannot properly understand MapUnveiler's exact formulation. Given that the authors answer \"No\" to Question 5 of the Checklist, I have to raise concerns about the paper's reproducibility.\n\n2. There is no detail on how the pre-training and fine-tuning are conducted. Do you initialize the MapNet by training MapTRv2? If this is the case, how are the training epochs split for the MapNet pre-training and the end-to-end MapUnveiler fine-tuning?  If the 24/6 epochs for nuScenes/Argo2 are only for the fine-tuning stage, then the comparisons in the main table are unfair, as other methods in the table have not fully converged. \n\n3. The main comparison results are incomplete. Most previous papers provide the nuScenes results of both short and long training schedules, but the main table only presents short-schedule results. Considering the last question about the pre-training and fine-tuning, the authors should complement the table with long-schedule results to show that MapUnveiler can obtain consistent performance boosts when all the methods are fully converged. This concern is backed up by the fact that MapUnveiler's improvement is much smaller on Argo2 compared to nuScenes -- based on my empirical experience, previous methods like MapTRv2 and its followups converge faster on Argo2, and training for 6 epochs is close to convergence. This probably suggests that the large performance gaps on nuScenes come from unfair training settings. \n\n4. Your interpretation of StreamMapNet and SQD-MapNet's Argo2 training epochs is wrong. These two methods employ a different frame sampling strategy at training time compared to MapTRv2, but their effective number of training samples is the same as MapTRv2. Therefore, the claim about the \"longer training schedules\" in the main table's caption is misleading.\n\n5. The venues in the main table are not accurate. HIMap[49] and MGMap[24] are accepted by CVPR2024, and the information was already available at the time of NeurIPS submission. Furthermore, a recent HD map construction method, MapTracker[A], also studies temporal modeling and should be very relevant, but it is missing in the discussion and related works.   \n\n    [A] MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mapping, arXiv:2403.15951", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your thorough and insightful feedback. We believe we can enhance the paper's quality and present more concrete results based on your comments. Below are point-by-point responses for your comments, and these will be included in the revised paper.\n___\n**W1-1. Explanation of TTM and MapTRv2**\n\nThanks for your valuable comment. Initially, we tried to avoid overclaiming in terms of model architecture, so we opted to skip the explanation of TTM and MapTRv2. However, we fully understand that this is not good for general readers. Following your advice, we will improve the presentation as follows.\n\ni) Inter-clip Unveiler\n\nIn Line #172, we will introduce an equation as\n> $U^{read}=Read(U_{t-2S:t-S}^{memory}, Q^{map})=S_{N_c}([U_{t-2S:t-S}^{memory}|| Q^{map}])$\n\nHere, the notation $[U_{t-2S:t-S}^{memory}|| Q^{map}]$ denotes the concatenation of two elements.\n\nIn Line #178, we will introduce an equation as\n> $U_{t-S:t}^{memory}=Write(U_{L}^{clip}, U_{L}^{map}, U_{t-2S:t-S}^{memory})=S_M([U_{L}^{clip}|| U_{L}^{map} ||U_{t-2S:t-S}^{memory}])$\n\nIf the tokens within the memory are not re-selected in the subsequent steps, it will be removed from the memory, and the selection mechanism will be determined through the learning.\n\nii) Loss\n\nTo provide detailed information about the losses, we will add following equations:\n> $\\mathcal L_{one2one} = \\lambda_c^F\\mathcal L_{cls}^F + \\lambda_p^F\\mathcal L_{p2p}^F + \\lambda_d^F\\mathcal L_{dir}^F$\n>\n> $\\mathcal L_{dense} = \\alpha_d\\mathcal L_{depth} + \\alpha_b\\mathcal L_{BEVSeg} + \\alpha_p\\mathcal L_{PVSeg}$\n>\n> $ \\mathcal L_{Frame-level MapNet} = \\beta_o\\mathcal L_{one2one}  +  \\beta_d\\mathcal L_{dense}$\n>\n> $\\mathcal L_{MapUnveiler} = \\lambda_c^M\\mathcal L_{cls}^M + \\lambda_p^M\\mathcal L_{p2p}^M + \\lambda_d^M\\mathcal L_{dir}^M$\n\nwhere $\\mathcal L_{one2one}$ is used for frame-level MapNet, and we set with $\\lambda_c^F=2, \\lambda_p^F=5, \\lambda_d^F=0.005$. \n$\\mathcal L_{dense}$ is an auxiliary loss using semantic and geometric information, and we set $\\alpha_d=3, \\alpha_b=1, \\alpha_p=2$.  $\\mathcal L_{MapUnveiler}$ is used for MapUnveiler, and we set $\\lambda_c^M=2, \\lambda_p^M=5, \\lambda_d^M=0.005$.\n\nDue to the character limit in the rebuttal, we conceptually show the revised explanation. We will further detail TTM, MapTRv2, and each loss in the Appendix.\n___\n**W1-2. The definitions of the temporal window $T$ and the stride $S$**\n\nTo provide a clear explanation, we will introduce the following equation.\n>$C_k =  \\lbrace f_t \\rbrace_{t=kS+1}^{kS+T}$\n\nThe notation $C_k$ represents the k-th clip, starting from k=0. Each frame is denoted by $f_t$, represents the t-th frame among the consecutive frames. Therefore, when T=3 and S=2, we obtain clip sets such as $C_0=\\lbrace f_1, f_2, f_3 \\rbrace$, $C_1=\\lbrace f_3, f_4, f_5 \\rbrace$, ..., and $C_k=\\lbrace f_{2k+1}, f_{2k+2}, f_{2k+3}\\rbrace$.\n\nThus, the temporal stride S refers to the clip stride, not the frame stride.\n\nAdditionally, it would be interesting to additionally set frame stride, as you mentioned. If we set the frame stride=2 and the clip stride (S)=1 (to see all frames), MapUnveiler achieves an mAP of 68.8% in nuScenes. This is lower than the model with frame stride=1 and T=1, (69.3%). We conjecture that temporally nearest frames provide the most rich information for unveiling maps because they have large spatially overlap regions, and temporally distant frames can effectively be exploited through the inter-clip Unveiler.\n___\n**W1-3. Inference mechanism**\n\nMapUnveiler excutes per-clip. Thus, it can cause response delay if the input frame rate is slower than 12.7 (which is the inference speed of MapUnveiler) because the model will wait until it collects T frames. To avoid this problem, we can set the clip stride (S) to 1 (but it will lead to a slight performance degradation, as presented in Table 7). Alternatively, we can fill in the intermediate frames' results with frame-level MapNet: during the response delay, we can directly construct an online map using map queries generated from the frame-level MapNet. In this case, however, the performance will drop from 69.8% to 68.0%.\n\n**W1-4. Open access code**\n\nWe are planning to release source codes upon acceptance.\n___\n**W2. Detail on how the pre-training and fine-tuning**\n\nWe end-to-end fine-tune MapUnveiler for 24/6 epochs from pre-trained MapTRv2, which was trained for 24/6 epochs. To address this concern, we present the freeze fine-tune setting in Table 4, but we understand it cannot fully address this concern. So, we additionally present experimental results where we pre-train MapTRv2 for 12/3 epochs and then fine-tune MapUnveiler for 12/3 epochs, totaling 24/6 epochs. The results are given in **G1** of the global response.\n___\n**W3. Long training schedules**\n\nThanks for your thorough feedback. We trained 110/30 epochs, and MapUnveiler achieves mAP of 70.6% and 72.9% on nuScenes and Argoverse2, respectively. As you commented, the performance improvement of our model is marginal. We think that we can converge more quickly because we start training from a pre-trained frame-level MapNet. Even though our model lags behind HIMap (73.7% and 72.7%) in the long training schedule setting, we would like to kindly emphasize the following three points: (1) HIMap is heavy (9.7 FPS) and runs slower than ours (12.7 FPS); (2) HIMap requires more training time to fully converge, which may cause overfitting; (3) HIMap was published arXiv two months ago when we submitted our paper, so this work stands as a contemporaneous work.\n___\n**W4. Caption in Table 1**\n\nThanks for the correction. We will remove † in Table 1.\n___\n**W5. HIMap[49], MGMap[24], and MapTracker[A]**\n\nThanks for the thorough check. We will revise them as CVPR2024 and will discuss with MapTracker. We would like to kindly inform the reason for notating as preprints: the official paper list of CVPR2024 was opened on the same day as the deadline for NeurIPS2024 submissions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their constructive and thorough comments. We are particularly excited that all reviewers acknowledged the idea of a clip-level pipeline as reasonable, novel, or effective for online vectorized HD mapping. We believe this rebuttal further enhance the paper through the valuable comments provided by the reviewers. We provided detailed point-by-point responses for all queries in each reviewer's rebuttal. Here, we provide global responses from **G1** to **G4**.
___
**G1. A more fair comparison**

We conducted an experiment where we pre-train MapTRv2 for 12epochs/3epochs and then fine-tune MapUnveil for 12epochs/3epochs on either the nuScenes or Argoverse2 training set. Even though our proposed modules will be trained for only 12epochs/3epochs, this will undoubtedly provide fair comparisons, as we are training a total of 24epochs/6epochs. The results are given below.
| Method (60x30m) | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP (nuScenes) | FPS | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP (Argoverse2) |
|---|---|---|---|---|---|---|---|---|---|
| MapTRv2 [22] | 59.8 | 62.4 | 62.4 | 61.5 | 15.6 | 62.9 | 72.1 | 67.1 | 67.4 |
| SQD-MapNet[40] | 63.0 | 62.5 | 63.3 | 63.9 | - | 64.9 | 60.2 | 64.9 | 63.3 |
| MGMap [24] | 61.8 | 65.0 | 67.5 | 64.8 | 12.3 | - | - | - | - |
| MapQR [26] | 63.4 | 68.0 | 67.7 | 66.4 | 14.2 | 64.3 | 72.3 | 68.1 | 68.2 |
| HIMap [49] | 62.6 | *68.4* | *69.1* | 66.7 | 9.7 | **69.0** | 69.5 | **70.3** | 69.6 |
| ***Map-Unveiler\* (ours)*** | *67.6* | 67.6 | 68.8 | *68.0* | 12.7 | 68.9 | *73.7* | 68.9 | *70.5* |
| Map-Unveiler (ours) | **69.5** | **69.4** | **70.5** | **69.8** | 12.7 | **69.0** | **74.9** | *69.1* | **71.0** |

| Method (100x50m) | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP (nuScenes) | FPS | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP (Argoverse2) |
|---|---|---|---|---|---|---|---|---|---|
| MapTRv2 [22] | 58.1 | 61.0 | 56.6 | 58.6 | 15.6 | 66.2 | 61.4 | 54.1 | 60.6 |
| StreamMapNet [45] | 62.9 | 63.1 | 55.8 | 60.6 | 12.5 | - | - | - | 57.7 |
| SQD-MapNet[40] | 67.0 | 65.5 | 59.5 | 64.0 | - | 66.9 | 54.9 | 56.1 | 59.3 |
| ***Map-Unveiler\* (ours)*** | *68.0* | *70.0* | *68.2* | *68.7* | 12.7 | *69.7* | *67.1* | *59.3* | *65.4* |
| Map-Unveiler (ours) | **68.4** | **71.2** | **68.3** | **69.3** | 12.7 | **70.4** | **66.8** | **59.3** | **65.5** |

In the table, we mark with an ***asterisk (\*)*** when we use a new fair training schedule (pre-train 12epochs/3epochs and fine-tune 12epochs/3epochs), and the other methods are directly copied from Table 1 in the main paper. Our method still demonstrates state-of-the-art performance of mAP under this setting.

___
**G2. Experimental results on geo-disjoint dataset splits**

We conduct experiments on a recent geo-disjoint dataset split: StreamMapNet where the performance of MapTRv2 has been significantly drop from 61.5% to 36.6%. The results are given in **Table 12 of the rebuttal PDF file**. For a fair comparison, our method uses short training schedule as discussed in **G1**. We also succesfully acheieve state-of-the-art performance on geo-disjoint dataset split. We will further evaluate on various geo-disjoint dataset splits in the final version.
___
**G3. Additional computing costs and analysis of added model compacity**

To analyze the model's capacity increased by the proposed modules, we measure GPU memory consumption during inference time and the model parameters in Table 3. The updated Table 3 is provided below.

| Method | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP | FPS | GPU (MB) | Parameters (MB) |
|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| MapTRv2 [22] | 58.8 | 61.8 | 62.8 | 61.2 | 15.6 | 830.4 | 76.4 |
| + Intra-clip Unveiler | 65.6 | 67.6 | 68.0 | 67.1 | 13.1 | 1552.5 | 144.0 |
| + Inter-clip Unveiler | 69.5 | 69.4 | 70.5 | 69.8 | 12.7 | 1614.9 | 213.9 |

As shown in the table, our methods require marginal additional computational costs (15.6 FPS → 13.1 FPS if we utilize clip tokens only within intra-clip tokens and 15.6 FPS → 12.7 FPS if we fully employ intra-clip and inter-clip tokens). However, our approach requires approximately two times more GPU memory and three times more parameters compared to the frame-level MapTRv2 model. This could be considered a potential limitation of our method, but the amounts are not large (2GB graphic cards can be purchased for under $25, and we commonly have more than 1GB of RAM and hard disks).
___
**G4. Experiments with various backbones**

Here we additionally present experimental results with various backbones: ResNet-18 and V2-99.

| Method (60x30m) | Backbone | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP (nuScenes) | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP (Argoverse2) |
|---|---|---|---|---|---|---|---|---|---|
| MapTRv2 [22] | R18 | 53.3 | 58.5 | 58.5 | 56.8 | 58.8 | 68.5 | 64 | 63.8 |
| MapTRv2 [22] | V2-99 | 63.6 | 67.1 | 69.2 | 66.6 | 64.5 | 72.2 | 70.1 | 68.9 |
| Map-Unveiler (ours) | **R18** | 65.5 | 68.4 | 68.2 | 67.4 | 66.5 | 71.1 | 67.5 | 68.4 |
| Map-Unveiler (ours) | R50 | 69.5 | 69.4 | 70.5 | 69.8 | 69.0 | 74.9 | 69.1 | 71.0 |
| Map-Unveiler (ours) | **V2-99** | **72.1** | **72.9** | **74.9** | **73.3** | **71.4** | **75.1** | **73.0** | **73.2** |

| Method (100x50m) | Backbone | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP (nuScenes) | AP$_{p}$ | AP$_{d}$ | AP$_{b}$ | mAP (Argoverse2) |
|---|---|---|---|---|---|---|---|---|---|
| MapTRv2 [22] | R18 | 52.7 | 57.3 | 51.5 | 53.8 | 60.3 | 57.6 | 49.6 | 55.8 |
| MapTRv2 [22] | V2-99 | 62.6 | 67.8 | 65.2 | 65.2 | 68.5 | 62.1 | 58.4 | 63.0 |
| Map-Unveiler (ours) | **R18** | 67.4 | 69.8 | 68.8 | 68.6 | 68.5 | 64.3 | 57.7 | 63.5 |
| Map-Unveiler (ours) | **V2-99** | **71.9** | **75.0** | **75.6** | **74.2** | **73.3** | **69.1** | **63.9** | **68.8** |

As shown in the table, our method is not limited to MapTRv2 with ResNet50, but can be extended to ResNet18 and V2-99 backbones. This suggests that our method can also work for other various frame-level features as well.

Meta-review: This paper initially received mixed reviews. After the rebuttal period, all reviewers recommend acceptance, praising the strong performance and extensive evaluations. There are some lingering concerns about presentation clarity, but the authors have been given clear guidance and they seem easy to address for the camera-ready submission.
Decision: Accept (poster)