IC-Mapper: Instance-Centric Spatio-Temporal Modeling for Online Vectorized Map Construction

Authors: submission  id
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 11:45:34

Abstract: Online vector map construction based on visual data can bypass the processes of data collection, post-processing, and manual annotation required by traditional map construction, which significantly enhances map-building efficiency. However, existing work treats the online mapping task as a local range perception task, overlooking the spatial scalability required for map construction. We propose \emph{IC-Mapper}, an instance-centric online mapping framework, which comprises two primary components: 1) \textbf{Instance-centric temporal association module:} For the detection queries of adjacent frames, we measure them in both feature and geometric dimensions to obtain the matching correspondence between instances across frames. 2) \textbf{Instance-centric spatial fusion module:} We perform point sampling on the historical global map from a spatial dimension and integrate it with the detection results of instances corresponding to the current frame to achieve real-time expansion and update of the map. Based on the nuScenes dataset, we evaluate our approach on detection, tracking, and global mapping metrics. Experimental results demonstrate the superiority of IC-Mapper against other state-of-the-art methods.

Main paper: 
 related work 2.1 visual multiple object tracking: 
 Traditional multi-object tracking algorithms perform the object associations based on detection results. Some early formulations [2, 3, 27] rely on Kalman filters [31] or optical flow algorithms [1] to estimate and match moving targets' positions from one frame to the next. Another kind of approach [29, 37] involves training additional target re-identification networks to assign the same ID to targets with similar features. In recent years, with the development of the Transformer [28] , query-centered end-to-end detection and tracking methods [35, 36] have been proposed, eliminating the complex post-processing and matching computations found in traditional methods. In the field of HD mapping, tracking modules are typically independent. However, experimental evidence suggests that performing tracking tasks end-to-end can compromise detection performance. Hence, recent vectorized HD map construction works, like MOTRv3 [33] and DQTrack [15] , take significant efforts to address this issue, while there is little work on using deep learning-based end-to-end tracking algorithms in mapping tasks.Offline map construction requires pre-collection of sensor data for the corresponding environment, followed by offline execution of three steps: map feature extraction, vectorized feature modeling, and map updating. Map feature extraction aims to extract map-related target features from image data, utilizing traditional geometric detection algorithms [22, 23, 32] , as well as deep learningbased semantic segmentation [13, 38] or object detection algorithms [11, 30] . The goal of vectorized feature 3D modeling is to transform features extracted from images into 3D space, traditionally achieved through IPM projection or using synchronized point cloud data. In recent years, deep learning-based 3D reconstruction algorithms [19] have also been used for feature modeling. Map updating aims to fuse features extracted from multiple frames. Some methods achieve this using traditional rule-based algorithms [9, 26] , while recent approaches also fuse multiple frame outputs using deep learning-based methods [39] . Overall, offline methods offer high-accuracy map reconstruction but are cumbersome and prone to error accumulation from various modules.Mapping. Vision-based online vector map construction is gaining attention for its real-time updates in dynamic environments and mapping efficiency. Traditional vision SLAM (Simultaneous Localization and Mapping) systems [6, 20, 21] use generic feature points for front-end representation, making it challenging to directly create vectorized maps. Qiao and others [24] construct a tracking and optimization module based on lane line representation for online lane map updates by using 3D lane line results obtained from a pre-trained detector [8] . In recent years, deep learning-based end-to-end map construction tasks have also garnered widespread interest. HDMapNet [14] , based on surround-view images, achieves end-to-end map instance segmentation tasks, followed by post-processing to complete vectorized instances extraction. VectorMapNet [18] and MapTR [17] accomplish end-to-end network output of vectorized map representations by designing decoders based on DETR [7] . StreamMapNet [34] further integrates temporal multi-frame information to enhance the accuracy of detection. However, existing deep learning-based online end-to-end map construction methods only focus on detecting map instances within a local perception range, unable to perform real-time updates and construction of global maps. 
method 3.1 overview: 
 This section provides an overview of the IC-Mapper. obtained at time step 𝑡, and 𝑇 indicates the total count of these time steps. The output comprises a series of vectorized map instancesS 1:𝑇 𝐺 = {(𝑐 𝑖 , 𝑃 𝑖 , 𝑖𝑑 𝑖 )} 𝑁 1:𝑇 𝐺 𝑖=1, corresponding to all the regions covered by the vehicle. Here, 𝑁 1:𝑇 𝐺 signifies the total count of map instances within the scanned area, where 𝑐 𝑖 identifies the category of the 𝑖-th instance, and 𝑃 𝑖 ∈ R 𝑁 𝑝 ×2 denotes the associated point set where 𝑁 𝑝 is the number of points. 𝑖𝑑 𝑖 provides a unique identifier for each instance. The entire architecture, as shown in Fig. 2 , builds upon the detector designed concerning StreamMapNet [34] and incorporates instance-based temporal association and spatial fusion modules to achieve end-to-end map detection, tracking, and updating.The detection module, building on StreamMapNet [34] , generates BEV (Bird's Eye View) feature maps concerning BEVFormer [16] . A DETR-based [7] decoder is designed to refine the points of map instances iteratively through a multi-layer Transformer. Furthermore, for temporal modeling, a GRU-based [10] approach is used for fusing BEV features, along with a query feature fusion method grounded in the TopK mechanism. To transition the instance's query features from one frame to the next, a Multi-Layer Perception (MLP) based query update module, 𝑈 𝑀𝐿𝑃 , is employed, which is supervised by the L1 distance between decoded coordinate points. The corresponding loss function is denoted as L 𝑡𝑟𝑎𝑛𝑠 . Given the detected map instances D 𝑡 = {(𝑐 𝑖 , 𝑃 𝑖 )} 𝑁 𝑡 𝑞 𝑖=1 and the ground truth setG 𝑡 = {( ĉ𝑖 , P𝑖 )} 𝑁 𝑡 𝑔𝑡 𝑖=1, where 𝑁 𝑞 and 𝑁 𝑔𝑡 are the number of detection queries and ground truth, we firstly employ bipartite matching to get the optimal instance-level and point-level assignment ( π and γ𝑖 ) as in [17] . Then the detection loss is defined as:EQUATIONEQUATIONwhere π𝑖 denotes the index of the detected instance corresponding to the 𝑖-th ground truth (GT) instance under optimal matching. Conversely, γ𝑖 ( 𝑗) indicates the index of the 𝑗-th point in the π𝑖 -th detected instance that matches the coordinates of point P𝑖 under optimal matching.In the temporal association module, as shown in Fig. 3 , for the current frame 𝑡, we maintain a tracking memory bufferA 1:𝑡 -1 = {(𝑐 𝑖 , 𝑃 𝑖 , 𝑄 𝑖 , 𝑖𝑑 𝑖 )} 𝑁 1:𝑡 -1 A 𝑖=1 to store 𝑁 1:𝑡 -1 A instances tracked in previous frames. The set of detected instances in the current frame 𝑡 is R 𝑡 𝑑𝑒𝑡 = {(𝑐 𝑖 , 𝑃 𝑖 , 𝑄 𝑖 )} 𝑁 𝑡 𝑑𝑒𝑡 𝑖=1, where 𝑄 𝑖 represents the query feature of the 𝑖-th instance. The temporal association module learns the matching relationships between instances in A geometric and feature dimensions. Specifically, we use Q 𝐷 ∈ R 𝑁 ×𝐷 as the query feature embeddings for the detected instances in the current frame. And Q 𝑇 ∈ R 𝑀 ×𝐷 represents the query feature of tracked instances from Tracking Memory Buffer, which have been updated by 𝑈 𝑀𝐿𝑃 . Here, 𝑁 and 𝑀 represent the number of detected and tracked instances, respectively, while 𝐷 denotes the feature dimension. Similarly, P 𝐷 ∈ R 𝑁 ×𝑁 𝑝 ×2 and P 𝑇 ∈ R 𝑀 ×𝑁 𝑝 ×2 are used to represent the point sets corresponding to the detected and tracked instances in the current frame's ego-vehicle coordinate system, respectively. In the geometric dimension, we encode the L2 metric matrix between P 𝐷 and P 𝑇 using a Feed-Forward Network (FFN); in the feature dimension, we encode the outer product of Q 𝐷 and Q 𝑇 using a Multi-Layer Perceptron (MLP). The encoding results from both dimensions are summed and then passed through another layer of MLP to obtain the fused metric relationship encoding result H ∈ R 𝑁 ×𝑀 ×1 .EQUATIONwhere each position in H is used to measure the similarity between the corresponding detected and tracked instances.In this section, we focus on more details about model optimization and inference on map instance tracking with the help of above designed learnable association. Optimization Objectives. Based on the ground truth (GT) ID labels, we construct a real binary metric matrix G ∈ R 𝑁 ×𝑀 to represent the matching relationships between detected and tracked instances. Each element of G is set to 1 if the corresponding detected instance (row index) and tracked instance (column index) match according to the GT, and 0 otherwise. Then, the cross-entropy loss between H and G is calculated to serve as the loss for the temporal association module:L asso = - 𝑁 ∑︁ 𝑖=1 𝑀 ∑︁ 𝑗=1 G 𝑖 𝑗 log(H 𝑖 𝑗 ) + (1 -G 𝑖 𝑗 ) log(1 -H 𝑖 𝑗 ) . (4)Here, G 𝑖 𝑗 represents the ground truth matching between the 𝑖-th detected instance and the 𝑗-th tracked instance. This approach effectively trains the temporal association module to improve its accuracy in matching instances over time. Inference Scheme. During the inference process, upon obtaining the similarity score matrix H ∈ R 𝑁 ×𝑀 ×1 , which delineates the affinity between current detection instances and historical tracking instances, the ID assignment to current detection instances can be formalized through the following steps:1. Threshold Application: A predefined threshold, 𝜃 , is applied to the scores in H , such that only those detection-tracking instance pairs with scores exceeding 𝜃 are considered for further processing.EQUATION)2. Optimal Matching: The most compatible detection-tracking pairs are selected based on the maximization of the total similarity score across all matches. This step ensures a one-to-one correspondence between detections and tracks, where each detection 𝑖 is matched with at most one track 𝑗 and vice versa, formalized as:EQUATIONwhere M denotes the set of matched detection-tracking instance pairs. 3. ID Allocation: Each detection instance 𝑖 in the current frame inherits the ID of its matched tracking instance 𝑗 from M. Detection instances without a match are assigned new IDs, indicating the emergence of new objects.EQUATION)4. Tracking Buffer Update: The tracking memory buffer is updated by R𝑡 𝑑𝑒𝑡 to reflect the current frame's detection instances and their assigned IDs, incorporating the additions of new objects and removal of unmatched tracks.To construct a global map that contains spatially continuous map instances, we further employ a spatial fusion module that fuses map instances from the current frame into the maintained global map. In this module, we firstly sample a fixed number of points from the intersection of the historical global map S 1:𝑡 -1𝐺 = {(𝑐 𝑖 , 𝑃 𝑖 , 𝑖𝑑 𝑖 )} 𝑁 1:𝑡 -1 𝐺 𝑖=1with the current perception range. These sampled points are then fused one-to-one with instances in R𝑡 𝑑𝑒𝑡 through cross-attention operations. Furthermore, we introduce a class-wise merging strategy to smoothly insert the upcoming detection result into the final global map S 1:𝑡 𝐺 . For the sake of brevity, we remove the id symbol 𝑖𝑑 from map instances S G = {(𝑐 𝑖 , 𝑃 𝑖 , 𝑖𝑑 𝑖 } 𝑁 𝑡 𝑖=1 since the following fusion and merge are performed based on tracking results. An illustration of this process is shown in Fig. 4 . 𝑖=1 for each detected map instance of the current frame, we employ 3 transformer layers where cross attention is calculated between the detected point sets {P 𝑖 } 𝑁 𝑡 𝑖=1 and the sampled historical points {P sample 𝑖 } 𝑁 𝑡 𝑖=1 . For the sake of clarity and brevity, we reformat these point sets as tensors omitting the instance index, denoted as 𝑷 𝑘 ∈ R 2 where 𝑘 represents the 𝑘-th layer, and we omit the self-attention and feed-forward network components. Point tensors are firstly encoded by several MLP layers and then standard multi-head cross attention (denoted as CrossAttn(𝑸, 𝑲, 𝑽 ), where 𝑸, 𝑲 and 𝑽 represents query, key, and value ) is applied:EQUATIONEQUATIONEQUATION)EQUATIONEQUATION)To be noticed, following [34] , a shared regression layer Reg is employed for all layers to predict the final point coordinates. The predicted points from the current layer serve as the query points input to the next layer. The output of the last layer is the final refined detection result which is used to infer an additional fusion loss. Here we simply employ a regression loss:EQUATION)where P𝑖 is the ground truth point set corresponding to the predicted instance. Hence, the final loss is the sum of the detection loss L 𝑑𝑒𝑡 , transition loss L 𝑡𝑟𝑎𝑛𝑠 , the tracking loss L 𝑡𝑟𝑎𝑐𝑘 and the fusion loss L 𝑓 𝑢𝑠𝑖𝑜𝑛 :EQUATION)This loss serves as the final end-to-end training target, where 𝛼, 𝛽, and 𝜆 are the weight factors. The collection of detection instances processed through the spatial fusion module is denoted as R𝑡 𝑓 𝑢𝑠𝑒𝑑 . 𝑖=1 . Hence we introduce an online merging algorithm based on Bezier curve fitting [4] applied for instances of class boundary or divider. It determines the point sets order during merging and fits a smoother result. For the pedestrian crossing class, we simply apply a union operation due to their quadrilateral closed shape. Here, we focus on the merging algorithm between a pair of map instance matches of class 𝑐 and therefore omit the index of it: (P 𝐺 , P 𝑡 , 𝑐). Here P 𝐺 denotes the global instance point sets and P 𝑡 denotes the detected instance point sets. This merging process is listed in Algorithm 2 and illustrated by Fig. 5 . 
experiments 4.1 experimental setup: 
 4.1.1 Dataset. The nuScenes [5] dataset is one of the most commonly used datasets in the autonomous driving domain. It collects sensor data from approximately 1000 scenes and annotates key samples at a frequency of 2Hz. Each sample provides imagery from six cameras along with the vehicle's global coordinates. Additionally, the dataset includes high-precision map files, allowing for the extraction of map feature data within the current local range for online mapping algorithm development. However, the original partitioning of the nuScenes dataset results in a significant overlap between the training and validation sets, which does not accurately reflect the model's true capability in map feature detection. Therefore, we follow the approach of StreamMapNet [34] to re-partition the nuScenes dataset and perform task evaluations based on this new partitioning.4.1.2 Metrics. Similar to previous related works, we focus on evaluating three types of vector map features: lane dividers, pedestrian crossings, and road boundaries. Given our mapping framework follows a detection-tracking-fusion and update process, we design specific metrics for local detection, instance tracking, and global map construction tasks to validate the advantages of the proposed method in various aspects.Detection Metrics. We conduct evaluations based on two scales of detection range: 100×50 (50m ahead and behind, 25m to each side) and 60×30 (30m ahead and behind, 15m to each side). Average Precision (AP) is used to assess detection capability, with 1.0m, 1.5m, 2.0m as the AP calculation thresholds for large-scale perception tasks, and 0.5m, 1.0m, 1.5m for small-scale tasks.Tracking Metrics. Since the original nuScenes dataset does not provide ID information for map instances, to fit our proposed instance tracking task, we introduce a post-processing tracking algorithm based on category and distance matching to provide ID labels for training and evaluation. Metrics from the multi-object tracking domain, such as MOTA, MOTP, and IDs, are introduced to evaluate our tracking performance for vector map instances.Map Construction Metrics. A simple merging strategy is employed on annotated data to construct global maps for the evaluation. We then introduce the Chamfer Distance (CD) from HDMap-Net [14] to assess the average error in global map construction.As we are the first to evaluate metrics across detection, tracking, and mapping tasks simultaneously, there are not many open-source and similar works available for comparison. This section will briefly introduce the baseline algorithms implemented for the comparison across various metrics. Detection Baselines. Three detection baselines are selected for the comparison because of their state-of-the-art performance. VectorMapNet [18] and MapTR [17] are two single-frame-based endto-end vector map detection networks, whereas StreamMapNet [34] utilizes multi-frame surround-view images as its input.Tracking Baselines. Due to the lack of prior work on visual tracking of vector map features, we reproduce two typical tracking algorithms for comparison. The first, referred to as Post-Track, which involves post-processing matching ns utilizes categories and positional distances from detection results. The second is MOTR-Track, similar to the MOTR [35] , which is an end-to-end detection and tracking approach where a subset of the detection queries in the current frame are initialized using tracking queries. This method explicitly assigns ground truth instances based on the ID value supervision.Mapping Baselines. Three algorithms have been selected as baselines for the mapping task. The first builds upon a detector trained using StreamMapNet, performing offline tracking and merging features based on the merging strategy in VMA-merge [9] . The second algorithm, inspired by HDMapNet [14] , initially clusters point sets detected across an entire image sequence to distinguish instances and subsequently fits the point sets of the same instances. PolyMerge [26] proposes to perform online map updates based on a rule-based algorithm. We reproduce the algorithm and find that our IC-Mapper produces better results under the Chamfer Distance (CD) metric.Our training approach is conducted in two stages. In the first stage, we jointly train the detection and tracking modules for 24 epochs on 8 GPUs with a batch size of 32, using the AdamW optimizer and set the learning rate to 𝑙𝑟 = 5 × 10 -4 . In the second stage, we freeze the parameters of the detection and tracking network modules and train the fusion module at a learning rate of 0.1 × 𝑙𝑟 . Regarding the data selection strategy, in the first stage, we follow the same approach as StreamMapNet, randomly splitting each video sequence into two parts. In the second stage, we sequentially load and train on the entire sequence. We abandoned the single-frame pre-training originally used in StreamMapNet because we found that incorporating the temporal association module allows for greater benefits when training directly on multi-frame data from scratch.. We design temporal association and spatial fusion modules for map instance tracking and global map construction tasks. However, we have observed that the IC-Mapper also achieves significant accuracy improvements in detection tasks compared to previous algorithms. As shown in Table 1 , our method surpasses existing map instance detection algorithms across various detection range configurations. Compared to StreamMapNet [34] , our method shows an average mAP improvement of 1.1 and 1.6 in the detection ranges of 60x30 and 100x50, respectively.. Table 3 demonstrates the performance differences in map feature tracking tasks between our algorithm and other algorithms. Traditional tracking algorithms maintain high levels of MOTA, while another end-to-end detection and tracking algorithm (MOTR) achieves fewer ID Switches.The strengths of our method are reflected in its solid performance across all tracking metrics. Additionally, for multitask learning, the interference between different tasks is an important measure, and our algorithm enhances the accuracy of the original detection tasks while maintaining high tracking performance, achieving a 12.3 higher mAP in detection compared to MOTR.. Table 2 compares the performance of different map construction methods, and the visualization results on select sequences are shown in Fig. 6 . Traditional tracking methods and VMA-based [9] instances substitution strategy struggle to effectively merge multi-frame detection results in the spatial dimension, often leading to localized omissions. Clustering and PolyMerge [26] , two mapping algorithms that do not rely on instance tracking, find it challenging to accurately capture the temporal correspondence of map instances, resulting in erroneous connections. Our end-to-end detection-tracking-fusion algorithm models both time and space dimensions using deep learning techniques, achieving superior mapping results. Additionally, our use of Bézier curve-based fitting also contributes to the mapping task metrics, with a detailed analysis available in Fig. 7 . 4 , we conduct ablation experiments to assess the impact of the temporal association module and spatial fusion module on detection accuracy. The experimental results indicate that the temporal association and spatial fusion modules designed for our tracking and mapping modules also significantly enhance the detection metrics. This improvement not only boosts the performance of the current tasks but also offers substantial advantages for subsequent mapping tasks, laying a strong foundation for more accurate and comprehensive map construction. We conduct evaluations on the geometric and feature-dimensional association matrices of the temporal module. The results show that the model in both dimensions achieves good tracking performance. Additionally, the design based on query features further enhances the detection task metrics. In our approach, the fusion of these two types of metrics is employed, resulting in a strong performance in both detection and tracking indicators. Table 5 : The tracking performance concerning different temporal association models, where the "-geo", "-query", and "all" means the calculations based solely on the geometric dimensions, the query feature dimensions, and a fusion of both dimensions for the association, respectively.Method MOTA MOTP ID-switch mAP Ours-geo 0.57 3.12 1.83 34.65 Ours-query 0.57 3.23 1.91 35.17 Ours-all 0.54 3.17  
conclusion: 
 We propose an instance-centric end-to-end detection-tracking-fusion framework for vision-based online construction of vectorized maps. We designed a temporal association module to match instances across frames and introduced a spatial fusion module to merge previous maps with current detection results. Based on such temporal and spatial modeling, our method achieves leading levels in metrics across detection, tracking, and mapping. Furthermore, we also encourage the community to ponder and explore more vectorized mapping tasks in long temporal sequences and large spatial scales. Limitations: We have introduced an end-to-end framework for detection, tracking, and map construction for the first time. However, the modeling of relationships between different sub-modules requires further exploration to enhance overall performance and integration. In addition, more datasets (besides the field-established and highly-quality annotated nuScenes) are preferred for the enhanced establishment of the method performance.  
: 
 which provides information beyond the perception range for autonomous driving vehicles, and is crucial for tasks such as localization, decision-making and trajectory planning. However, traditional processes for generating and maintaining HD maps are exceedingly complex, involving the pre-collection of a large amount of point cloud and image data, followed by offline feature extraction, postprocessing optimization, manual annotation, and quality inspection. While this can achieve high-accuracy maps, such a complex process makes long-term maintenance and updates not only costly but also difficult to quickly adapt to new environments, which limits the development of autonomous driving technology in unknown settings. Therefore, there is a growing urgency for affordable vision-based online map reconstruction technique. Most works treat the task of online vectorized map reconstruction as a local range perception task Early efforts firstly detect static features such as lane lines in images, then use camera parameters and Inverse Perspective Mapping (IPM) to project detection results into the vehicle's coordinate system [25] . In recent years, advancements in perception technologies have led to studies utilizing existing algorithms based on Bird's-Eye-View (BEV) representations to directly generate BEV space detection results from image data in an end-to-end fashion. [12, 16] . HDMapNet [14] is the first attempt to use images from multiple cameras as the input on the nuScenes [5] dataset, employing a neural network to provide local BEV space map segmentation results on an end-to-end basis. Subsequently, it utilizes post-processing methods such as clustering and fitting to transform the segmentation maps into vectorized representations. VectorMapNet [18] and MapTR [17] , by designing detection decoder structures similar to DETR [7] , achieve the end-to-end output of vectorized map representation. Further, StreamMapNet [34] introduces temporal modeling by using multi-frame images as the input to enhance the accuracy of map instance detection. However, existing deep learning-based approaches restrict the task of detecting map instances within a fixed BEV range, failing to achieve real-time construction and updates of a global map.Addressing this gap in research, as shown in Fig. 1 , this paper introduces an end-to-end online vector map construction method that takes a temporally continuous sequence of images as the input and outputs global vectorized map construction results in an end-toend manner. Nevertheless, constructing spatially continuous maps from temporally continuous observational data using deep learning is a challenging task. The primary difficulties are manifested in two aspects: (1) Capturing the temporal association of map instances. Identifying the same map instances across frames is crucial for subsequent fusion and merging to construct a global map. However, in complex traffic environments, the same map instance often appears differently in different frames due to the ego vehicle's movement, obstructions, changes in lighting, and other factors. (2) Maintaining the spatial consistency of the map. Map instances such as lane dividers and road boundaries often span large spatial scales, whereas the observational data from each frame can only cover a local range with the conflict between different frames.To address the difficulties outlined above, we propose IC-Mapper, an end-to-end vector map detection, tracking, and fusion scheme for online map updating and construction. Building on an existing vector map detection network, we first introduce an instancecentric temporal association module that matches detected instances with tracked instances in both geometric and feature dimensions, obtaining correspondence between map instances in adjacent frames. Secondly, we introduce an instance-centric spatial fusion module that performs sampling and encoding on the map maintained from previous frames in the spatial dimension and then executes fusion operation with the current detection results. Ultimately, the fused results are updated to the map, achieving continuous map construction. Our main contributions are as follows:• We propose an end-to-end framework that for the first incorporates all online mapping tasks in one network including the detection, tracking and the global map update. • We introduce an instance-centric temporal association module and a spatial fusion module to enable online tracking and fusion of map instances. • We extensively evaluate the online vectorized mapping tasks. Experimental results further illustrate the state-ofthe-art performance of our method compared to other mapping approaches across diverse metrics. 
 
 Appendix:  


Decision: The proposal concerns an instance-centric detection-tracking-fusion framework for global map construction based on sequence temporal input images. According to the 3 reviewers, the article is well-written and clear, offering contributions, but with weaknesses mainly at the level of experimentation, which led them to a decision around borderline/weak accept or reject. It seems to me that the answers provided in the rebuttal clarify the reviewers' expectations (somewhat surprisingly, this is not what R2 and R3 say), and that the paper could therefore be accepted, probably as a poster, given that the contributions highlighted are not major, but remain relevant to the ACM MM community.

Reviews: review: paper_summary: paper_summary: This paper presents IC-Mapper, a new framework for global temporal consistent vector map detection. It can detect vector maps, and merge them into a global map. First, IC-Mapper detects vector maps of each frame following StreamMapNet, which can aggregate information of multiple frames for local map detection. Then, it proposes a Instance-CentricTemporalAssociation module for map instance association. Finally, associated instances are merged based on the proposed spatial fusion module. Experiments on the large NuScenes dataset are conducted, and the effectiveness of the model is proved.
strength_weakness: strength_weakness: This paper is clear to read and easy to follow. The methodology is clearly claimed and multiple figures are provided for readers. The general design is solid and the motivation is convincing.
limitations: limitations: This paper is generally fine for me, but there are still some concerns about the experiments.

1. Is there any filtering steps in the section 3.4 (spatial fusion)? I believe the vector map detection of each step is not temporal consistent, so what if the associated instances have very different shapes (Or directly incorrect association)? Directly merging them is not a very smart method for me. Have you considered adding the confidence score into the merge algorithm for better performance?

2. IC-Mapper is based on SpatialMapNet. In Figure 1, IC-Mapper obtains a slightly higher score than SpatialMapNet. Are there any modifications to the SptialMapNet network that can explain the improvement?

3. It seems that Post-track has a very similar performance with IC-Mapper (34.1, 35.0), why does Post-track & VMA-merge have that bad results in Table 2? Is it because of the proposed IC-Mapper spatial fusion module? What if you use Post-track in IC-Mapper? In Table 4, it seems that the spatial module does not make the main contribution to improvement (+0.09mAP). I am confused here. Could you please do more experiments about different associations and merging method combinations? E.g., IC-Mapper association + VMA-merge; Post-track + iC-Mapper merge.

4. Following question 3, how could IC-Mapper merge outperform VMA-merge that far? Considering that the IC-Mapper merge is quite straightforward.

5. Do all baselines have exactly the same efforts for optimization? In Figure 6, if Post-track has quite good association performance, how could VMA-merge have such bad merging results? Besides, I noticed IC-Mapper has many duplicated detections (detecting one line two times) and baseline methods do not have such an issue. Could you please do some discussion here?

6. What about the model efficiency? Can it run online?

7. Cite more recent papers will make your paper more concrete and convincing, even if you do not compare them in your paper. Like
[1] Liao, Bencheng, et al. "Maptrv2: An end-to-end framework for online vectorized hd map construction." arXiv preprint arXiv:2308.05736 (2023).
[2] Xu, Zhenhua, Kenneth KY Wong, and Hengshuang Zhao. "Insightmapper: A closer look at inner-instance information for vectorized high-definition mapping." arXiv preprint arXiv:2308.08543 (2023).
[3] Ding, Wenjie, et al. "Pivotnet: Vectorized pivot learning for end-to-end hd map construction." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.
[4] Xie, Ziyang, Ziqi Pang, and Yu-Xiong Wang. "Mv-map: Offboard hd-map generation with multi-view consistency." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

In conclusion, I mainly have some concerns about the experiments. The authors may want to provide more information about it. I will raise my score if some of the aforementioned concerns are solved.
paper_summary: paper_summary: The authors propose IC-Mapper, an instance-centric online mapping framework, which comprises two primary components: 1) Instance-centric temporal association module. The former is designed for measuring the detection queries of adjacent frames in both feature and geometric dimensions to obtain the matching correspondence between instances across frames. The latter is responsible for performing point sampling on the historical global map from a spatial dimension and integrating it with the detection results of instances corresponding to the current frame to achieve real-time expansion and update of the map.
strength_weakness: strength_weakness: 1. The authors have implemented an end-to-end detection-tracking-fusion process that enables the construction of global maps, incorporating temporal tracking and spatial fusion modules.
2. The authors are among the pioneers to incorporate all online mapping tasks in one network including detection, tracking, and the global map update.
limitations: limitations: 1. It is suggested that the authors discuss any instances of lower performance relative to existing methods as observed in Table 1 and Table 3, offering a thorough analysis and potential explanations for these results.
2. It would be beneficial for the authors to elucidate the factors contributing to the performance decline of AP_{div} following the integration of the temporal and spatial modules, as detailed in the ablation study of Table 4, accompanied by a comprehensive discussion and analysis.
3. Table 4 indicates a comparatively marginal enhancement in performance attributed to the spatial module when juxtaposed with the temporal module. An in-depth discussion and analysis from the authors regarding this disparity would be insightful.
4. I would raise my score if the authors could effectively address my concerns.
paper_summary: paper_summary: This paper proposes an instance-centric detection-tracking-fusion framework for global map construction via sequence temporal input images. Based on basic detection, temporal association module is utilized to obtain the matching correspondence for instance tracking, while instance spatial fusion module is proposed to achieve map update and expansion. On nuScenes dataset, experiments on detection, tracking and global map construction demonstrate the superior performance compared with previous sotas.
strength_weakness: strength_weakness: 1. This paper is well-written and easy-to-follow.
2. Detection-tracking-fusion strategy seems to be interesting and reasonable for global map construction.
3. This paper achieves online mapping updating and construction, which can be practical for industries like mapping and autonomous driving companies
limitations: limitations: 1. For comprehensive assessment, experiments on Argoverse dataset are required. Besides, what is the inference speed and memory usage for each part of your strategy? 
2. Tracking and fusion are helpful strategy for utilizing inter-frame information, but in table 1, compare with previous sotas, the improvement seems limited and not as expected. 
3. The topK query selection from previous frame and fusion with the current strategy is similar to the instance temporal fusion and has already been used in StreamMapNet [1]. 
4. Bezier curve fitting may lose efficacy, which is hard to fit for some sudden change road structures or sharp corners. How do you handle such conditions? More visualizations for such scenes are required to show the global result.

[1] Tianyuan Yuan, Yicheng Liu, Yue Wang, Yilun Wang, and Hang Zhao. 2024. Streammapnet: Streaming mapping network for vectorized online hd map construction. In WACV. 7356–7365