QE-BEV: Query Evolution for Bird's Eye View Object Detection in Varied Contexts

Authors: yingxin  lai
hongrui  kou
tong  wu
jiawei  yao
ruixi 2024 liu
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-07 20:37:09

Abstract: 3D object detection plays a pivotal role in autonomous driving and robotics, demanding precise interpretation of Bird’s Eye View (BEV) images. The dynamic nature of real-world environments necessitates the use of dynamic query mechanisms in 3D object detection to adaptively capture and process the complex spatio-temporal relationships present in these scenes. However, prior implementations of dynamic queries have often faced difficulties in effectively leveraging these relationships, particularly when it comes to integrating temporal information in a computationally efficient manner. Addressing this limitation, we introduce a framework utilizing dynamic query evolution strategy, harnesses K-means clustering and Top-K attention mechanisms for refined spatio-temporal data processing.  By dynamically segmenting the BEV space and prioritizing key features through Top-K attention, our model achieves a real-time, focused analysis of pertinent scene elements. Our extensive evaluation on the nuScenes and Waymo dataset showcases a marked improvement in detection accuracy, setting a new benchmark in the domain of query-based BEV object detection. Our dynamic query evolution strategy has the potential to push the boundaries of current BEV methods with enhanced adaptability and computational efficiency. Project page: https://github.com/Jiawei-Yao0812/QE-BEV

Main paper: 
 introduction: 
 3D object detection is a pivotal task in various applications [3, 4] like autonomous driving, robotics, and surveillance [9, 13, 22] . In the field of 3D object detection, BEV (Bird's Eye View) algorithms [7, 15, 36] have gained increasing prominence due to their ability to provide a top-down perspective, simplifying complex 3D scenes [35] into 2D representations. This perspective aids in reducing computational complexity and enhancing the clarity of object localization. However, traditional query-based BEV methods have mainly relied on static queries [20, 21, 32] , where the query weights are learned during the training phase and remain fixed during inference. This static nature restricts the model's capability to effectively leverage both spatial and temporal contexts and adapt to complex scenes. While recent studies like CMT [34] and UVTR [12] have introduced dynamic queries. The dynamic queries enable the model with the ability to adaptively capture complex spatial-temporal relationships. However, these methods typically employ a position-guided query mechanism, interacting simply with features or tokens through basic methods like adding position encoding or direct concatenation. This limits their ability to explore unknown areas in space, particularly in complex or dynamically changing environments. Figure 1 illustrates the limitations of static query-based methods, such as DETR3D [32] , where queries are learnable during training but remain fixed during inference. This inflexibility hampers the model's responsiveness to dynamic changes within the scene. Existing dynamic query methods, while addressing some of these issues, often rely on simple position-guided interactions that do not fully capture the complexities of spatial and temporal variations in real-world environments. In contrast, our method introduces a dynamic querying mechanism that allows for queries to adapt to the input data iteratively.In this vein, we introduce QE-BEV, a novel method that pioneers the use of advanced dynamic queries in query-based 3D object detection, pushing the boundaries of what can be achieved with BEV (Bird's Eye View) algorithms. QE-BEV fundamentally redefines query dynamics through iterative adaptations, leveraging feature clustering for adaptive scene representation and employing a Top-K Attention mechanism that intelligently adjusts to the most relevant top-k feature clusters. This approach enables each query to flexibly and efficiently aggregate information across both proximal and distal feature clusters, enhancing the system's ability to interpret complex, multi-dimensional scenes. In summary, our contributions are as follows:• We introduce an innovative dynamic querying mechanism in QE-BEV, which utilizes Top-K Attention to dynamically focus on the most relevant feature clusters, enhancing the model's accuracy and adaptability to complex scenarios. • We incorporate a Diversity Loss within the Top-K Attention framework to ensure that attention is not only paid to the most dominant features but also to less prominent ones, boosting the robustness of our model across varied environments. • Our QE-BEV includes a Lightweight Temporal Fusion Module (LTFM) that significantly reduces computational overhead by reusing dynamic queries and their associated feature clusters, thereby streamlining the incorporation of temporal context. This model has been rigorously evaluated on the nuScenes [1] and Waymo [25] datasets, demonstrating significant improvements over state-of-the-art methods in terms of both accuracy and efficiency. 
related work: 
 Query-based Object Detection in 2D and 3D. Query-based object detection has gained significant advancements thanks to the introduction of the Transformer architecture [27] . Primary works like DETR [2] adopted a static query-based approach where queries are used to represent potential objects but do not adapt during the detection process. Various works [5, 26, 38] have focused on accelerating the convergence or improving the efficiency of these static query-based methods. However, these models, even when extended to 3D space [20, 32] , inherently lack the ability to adapt queries to complex spatial and temporal relationships within the data. [32] and PETR series [20, 21] . These approaches, however, do not fully account for the spatial-temporal dynamics of real-world scenes. Recent works have introduced dynamic queries to address this limitation, such as MV2D [33] , SimMOD [37] , CMT [34] , and UVTR [12] , which allow for updates in response to input data. Our work extends these concepts by incorporating a dynamic paradigm where queries adapt iteratively, enhancing the model's ability to capture complex and changing spatial-temporal relationships in 3D object detection. Temporal Information in Object Detection. Incorporating temporal information has been explored in various works [15, 19, 22] . However, these methods often introduce significant computational complexity and are constrained by the static nature of their query or feature representations. Our Lightweight Temporal Fusion Module (LTFM) not only efficiently integrates temporal context but does so in a dynamic manner, further emphasizing the shift towards a dynamic paradigm in 3D object detection. 
method: 
 In this section, we introduce QE-BEV, a novel method designed for effective and efficient 3D object detection. Traditional static query-based methods lack the dynamism required to capture the diverse nature of 3D spaces. In contrast, QE-BEV harnesses dynamic queries that undergo iterative updates, and thereby achieves unparalleled adaptability in discerning diverse object attributes. The key components of QE-BEV are shown in Figure 2 . QE-BEV is composed of multiple integral components that synergize to facilitate robust and precise 3D object detection. The framework includes a backbone network responsible for initial feature extraction. With the extracted feature, a Dynamic Query Evolution Module (DQEM) comes into play. First, DQEM exploits K-means clustering to groups features around each query, which brings adaptive structure representation for complex 3D scenarios. Afterwards, a Top-K Attention module is employed by DQEM to iteratively refine queries with their associated feature clusters. Finally, a Lightweight Temporal Fusion Module (LTFM) is incorporated to efficiently capture temporal context for each query.Initialization of Queries (Pillars). In the context of 3D object detection, the initialization of queries plays a pivotal role in the subsequent detection performance. In the BEV space, these queries, often referred to as "pillars", serve as reference points or anchors that guide the detection process. The query set 𝑄 can be represented as:EQUATIONwhere (𝑥 𝑖 , 𝑦 𝑖 , 𝑧 𝑖 ) is the spatial coordinates of the 𝑖-th pillar, indicating its position in the BEV space. 𝑤 𝑖 , 𝑙 𝑖 , ℎ 𝑖 are width, length and height of the pillar, respectively, providing the shape attributes. 𝜃 𝑖 is the orientation angle of the pillar, offering insights into its alignment in the BEV space. 𝑣 𝑥 𝑖 and 𝑣 𝑦 𝑖 are velocity components of the pillar, capturing its motion dynamics.In traditional methods like SparseBEV [19] , these queries and their associate features are initialized based on pre-defined grid structures and remain static throughout the detection process. Such static nature are designed to capture general object patterns but is not adept at handling diverse scenarios with complex intricate object details.On the contrary, in QE-BEV, the associated feature are grouped into a clustered structure, which well adapts to the complex 3D scene, and each pillar iteratively adjusts its attributes (like position, dimensions, or orientation) based on the associated feature clusters. Such dynamism renders the pillars better adaptability to the object attributes in the 3D scenes, leading to a more accurate and robust detection.K-means Clustering. In QE-BEV, K-means clustering is first employed to divide the surrounding features 𝐹 of each query into 𝐾 clusters 𝐶 1 , . . . , 𝐶 𝐾 . The surrounding features refer to the features within a predefined spatial neighborhood of each query point, determined by a fixed radius 𝑟 . These features include information such as geometry, texture, and color, and are extracted from multi-view data at the same time step.The rationale behind employing K-means clustering lies in its ability to partition the feature space into clusters within which the feature variance is minimized. This enable each query to focus on groups of coherent features rather than unorganized points, which is a more adaptive and structured representation, thereby enhancing the model's ability to discern the objects in 3D scenes. After K-means clustering, each query 𝑞 will have an associated set of feature clusters 𝐶 𝑘 , formally:EQUATIONand the cluster center:EQUATION)These clusters encapsulate the local patterns around each query, and provide the model with a more adaptive structured representation of the dynamic 3D scenes, serving as the foundation for the subsequent Top-K Attention steps.Top-K Attention Aggregation. To allow each query to aggregate features in a dynamic way, we introduce a Top-K Attention mechanism. For each query 𝑞, we compute the attention weights over its associated feature clusters 𝐶 𝑘 obtained from K-means clustering.Compute Attention Scores. For each query feature 𝑞 and each cluster 𝐶 𝑘 , compute an attention score.𝐴 𝑘 = (𝑊 𝑞 𝑞) 𝑇 • 𝑊 𝑘 𝜇 𝑘 (4)Here, 𝑊 𝑞 represents the weight vector for the query and 𝑊 𝑘 represents the weight vector for the cluster. The dot product measures the relevance between the query and each cluster.This step allows the model to measure the importance of each feature cluster with respect to the query, enabling more informed aggregations.Select Top-K Clusters. Sort the attention scores 𝐴 𝑘 in descending order and select the top-K clusters.EQUATIONThis selective attention mechanism enables each query to focus on the most relevant clusters, which may even be farther away, thus enriching the aggregated feature.Weighted Feature Aggregation. Aggregate the selected clusters using their attention weights to form the aggregated feature 𝑞 ′ to update each query 𝑞.EQUATIONThe weighted sum allows for a rich combination of features, enabling each query to adaptively focus on different aspects of the surrounding features.The aggregated feature 𝑞 ′ serves as the foundation for 3D object prediction. By allowing each query to aggregate information even from distant clusters, the model's capacity to capture long-range dependencies is significantly enhanced. Such capacity is particularly crucial in 3D object detection, where objects might have parts that are spatially separated but are contextually related.Diversity Loss for Balanced Feature Aggregation. The proposed Top-K Attention mechanisms has the risk of focusing excessively on the most relevant features corresponding to each query. While this approach is effective in capturing dominant patterns, it often neglects the long-tail or less prominent features that could be critical for certain edge cases or specific scenarios. For example, in a 3D object detection task involving vehicles and pedestrians, focusing solely on the most relevant features might capture the overall shape of a vehicle but miss out on smaller but important details like side mirrors or indicators, which are essential for precise localization and classification.To address this limitation, we introduce a Diversity Loss 𝐿 div . This loss function aims to balance the attention mechanism by ensuring that not only the most relevant but also the less prominent features are considered. Unlike conventional entropy-based losses, which are agnostic to the task at hand, our Diversity Loss is meticulously crafted for 3D object detection, ensuring a balanced attention distribution across different feature clusters, formally:EQUATIONwhere the following function serves as a critical component for stabilizing the gradient flow during the back-propagation process, especially when dealing with clusters of varying relevance:EQUATIONThis Diversity Loss brings several advantages. Firstly, it promotes a balanced feature representation by encouraging the model to pay attention to a variety of features, not just the most prominent ones. This is particularly useful for capturing less obvious but potentially crucial features. Secondly, the approach enhances the model's robustness, allowing it to adapt better to different scenarios and noise levels. Lastly, it fosters a more comprehensive understanding of the data, thereby improving the model's generalization capabilities.Dynamic Adaptation of Queries. After initializing the queries as pillars and performing K-means clustering to obtain feature clusters 𝐶 𝑘 , the next crucial step is dynamically adapting these queries based on the Top-K Attention mechanism. This dynamic adaptation is the key difference from SparseBEV, where the queries are static. In QE-BEV, each query not only captures local information but also dynamically updates itself to aggregate relevant features from a large scope of feature clusters. Initial Feature Aggregation. For each query 𝑞, aggregate the initial set of features using a simple average or any other aggregation method.EQUATIONThis initial aggregation serves as a baseline, capturing the immediate vicinity of the query. It acts as an anchor, grounding the subsequent dynamic adaptations.Top-K Attention Update. Apply the previously described Top-K Attention mechanism to adaptively update each query 𝑞 using its associated feature clusters 𝐶 𝑘 .EQUATIONHere, 𝑞 ′ is the aggregated feature obtained from Top-K Attention, and 𝛽 is a hyper-parameter that controls the blending of initial and dynamically aggregated features. This step allows each query to adaptively refine its feature representation based on both local and long-range information, enhancing its ability to capture complex patterns and relationships.Iterative Update. Repeat the K-means clustering and Top-K Attention steps, using the newly updated queries 𝑞 as the new pillars for the next iteration. Such iterative update ensures the queries continuously adapting to the varying feature landscape, thereby increasing the model's robustness and adaptability.By iteratively updating queries through a combination of K-means clustering and Top-K Attention, QE-BEV ensures each query is both locally and globally informed, thereby capturing richer and more balanced feature representations. This dynamic adaptation is a significant advancement over SparseBEV, where pillars remain static and cannot adapt to capture long-range dependencies.In QE-BEV, the key advantage of our Lightweight Temporal Fusion Module (LTFM) lies in its computational efficiency. Unlike traditional temporal fusion methods that rely on resource-intensive recurrent or convolutional layers, LTFM leverages the already computed dynamic queries 𝑄 and their corresponding feature clusters 𝐶 𝑘 , thereby avoiding additional heavy computations.Temporal Query Initialization. The temporal queries 𝑞 are initialized using a weighted combination of current and previous dynamic queries , thus reusing existing computations.EQUATION)By reusing the dynamic queries, we eliminate the need for separate temporal query extraction, thereby reducing computational overhead.Dynamic Temporal Aggregation. The Top-K Attention mechanism is applied directly to 𝑞, reusing the previously computed feature clusters 𝐶 𝑘 for both current and previous time steps.EQUATIONThis obviates the need for separate temporal feature extraction, further reducing computational cost. Query Update. The temporal queries 𝑞 are updated using the aggregated temporal features 𝑞 ′ , similar to the dynamic query update in the previous sections.EQUATIONThe update operation is computationally light, as it only involves basic arithmetic operations, thus bringing the computational efficiency.LTFM provides an efficient way to incorporate temporal context without introducing a significant computational burden. By reusing existing computations to avoid additional complex operations, LTFM offers a lightweight yet effective solution for temporal fusion.The computational efficiency of QE-BEV is one of its key advantages. Below, we quantify this in terms of time complexity: The overall time complexity is approximately 𝑂 (𝑛𝐾𝐼𝑑 + 𝑛 log 𝑛 + 𝑛), where 𝑛 is the number of data points, 𝐾 is the number of cluster centers, 𝐼 is the number of iterations in K-means, 𝑑 is the dimensionality of each data point. This is relatively low compared to methods that require more complex temporal fusion techniques such as RNNs or CNNs. As shown in Table 1 , QE-BEV achieves considerable higher efficiency than two common baselines. 
experiment 4.1 implementation details: 
 We adopt ResNet [6] as the backbone, the temporal module in our model is designed to be lightweight and we use a total of 𝑇 = 8 frames by default, with an interval of approximately 0.5s between adjacent frames. For label assignment between ground-truth objects and predictions, we use the Hungarian algorithm [10] . The loss functions employed are focal loss [16] for classification and L1 loss for 3D bounding box regression, augmented by our custom Diversity Loss 𝐿 div with a weight factor of 𝜆 = 0.1. The initial learning rate is 2 × 10 -4 , and it is decayed using a cosine annealing policy. In line with recent advancements, we adjust the loss weight of 𝑥 and 𝑦 in the regression loss to 2.0, leaving the others at 1.0, to better capture spatial intricacies. We also incorporate Query Denoising to stabilize training and speed up convergence, as suggested by the recent work StreamPETR [28] . For our K-means clustering, 𝐾 is set to 6. The number of Top-K clusters for attention is set to 4. The hyperparameter 𝛽 used for blending in query update is set to 0.6, and 𝛼 for temporal fusion in the Lightweight Temporal Fusion Module (LTFM) is set to 0.4. Our experiments utilize the nuScenes dataset [1] , a rich source of multi-modal sensor information encompassing 1000 driving sequences, each lasting around 20s. Annotations are available at a rate of 2 Hz for key frames. Each frame in the dataset offers a comprehensive 360-degree field of view through six camera sensors. For the task of 3D object detection, the dataset incorporates approximately 1.4 million 3D bounding boxes across 10 categories of objects. We adopt a similar task setting as in previous works [19] for Birds-Eye View (BEV) segmentation. The official evaluation metrics of nuScenes are comprehensive; they not only include mean Average Precision (mAP), which is calculated based on the center distance in the ground plane instead of 3D IoU, but also feature five additional True Positive (TP) error metrics: ATE, ASE, AOE, AVE, and AAE, to measure the errors in translation, scale, orientation, velocity, and attributes respectively. To provide a unified score that captures multiple facets of detection performance, the nuScenes Detection Score (NDS) is used, defined as:EQUATIONIn addition, we verify the advantage of QE-BEV on the Waymo dataset [25] , with the same experiment settings exploited in [28] .Table 2 presents the performance of our QE-BEV on the nuScenes validation dataset, compared with other state-of-the-art methods, which outperforms all other methods by a considerable margin. With a ResNet50 backbone and an input size of 704 × 256, QE-BEV achieves a nuScenes Detection Score (NDS) of 56.1, which is higher than the 54.5 achieved by SparseBEV. More significantly, when perspective pre-training is applied, indicated by the † symbol, As shown in Table 2 , in more complex configurations, such as using a ResNet101 backbone and an input size of 1408 × 512, QE-BEV outshines its competitors with an NDS of 61.1, exceeding SparseBEV's 59.2, making it the current leading approach.QE-BEV consistently maintains high mAP scores, proving its robust object detection capabilities. In terms of True Positive metrics like mATE, mASE, QE-BEV holds its ground well compared to SparseBEV and other competing methods. Moreover, the model also performs well on fine-grained evaluation metrics such as Object Orientation Error (mAOE) and Attribute Error (mAAE). The application of perspective pre-training not only improves nearly all evaluation metrics but also showcases the model's adaptability and flexibility. Table 3 compares QE-BEV with the SOTA methods on the Waymo validation dataset. It also reveals the advantage of QE-BEV, with the highest scores of mAPL, mAPH and mAP.The advantages of QE-BEV primarily stem from two inherent aspects: Firstly, the design of QE-BEV allows it to better capture long-range dependencies. In 3D object detection, different parts of an object might be spatially distant but contextually related. For instance, the front and rear of a car might be far apart in the BEV space, yet they belong to the same object. SparseBEV, being a static querybased method, might struggle in such scenarios since its query points are fixed and cannot dynamically adapt to the changing scene. In contrast, QE-BEV, through its Dynamic Query Evolution Module, can update its query points in real-time, thereby better capturing these long-range dependencies. Secondly, QE-BEV is better equipped to handle the dynamism of real-world scenes. Objects in real-world scenarios might move, rotate, or change their shape. SparseBEV, with its static query points, might falter in such dynamically changing scenes. However, QE-BEV, through its dynamic queries and K-means clustering, can dynamically adjust its query points, thus better adapting to the evolving scene.Dynamic Query Evolution Module (DQEM). For all ablation studies, we use ResNet-50 as the backbone and adopt the same training and evaluation protocols. The baseline model employs the standard cross-attention mechanism. The Dynamic-K Block integrates Dynamic Queries, K-means Clustering, and Top-K Attention as a unified module. We compare this with the baseline model that uses standard cross-attention. Table 4 shows that the introduction of the Dynamic-K Block results in an 4.2% increase in NDS and a 4.3% increase in mAP compared to the baseline. The Dynamic-K Block's significant performance boost can be attributed to its ability to focus on key features dynamically. Traditional methods with static query points, like the baseline model, might not be able to adapt to the dynamic nature of real-world scenes. In contrast, the Dynamic-K Block, with its integration of Dynamic Queries, K-means Clustering, and Top-K Attention, allows the model to dynamically adjust its focus based on the scene's context. This adaptability ensures that the model can give precedence to critical features, especially in complex scenes where objects might be occluded or distant from each other. Incorporating the Lightweight Temporal Fusion Module (LTFM) to the baseline model results in a 3.1% increase in NDS and a 2.8% increase in mAP. These improvements indicate that LTFM effectively captures the temporal dependencies without introducing significant computational overhead, thus validating its utility in our QE-BEV framework. The LTFM provides the model with crucial context about these object movements. By fusing information across time, the model gains a more comprehensive understanding of the scene, allowing it to predict object trajectories and interactions more accurately. LTFM consistently outperformed other methods like LSTMbased fusion, Convolutional LSTM fusion, and simple averaging across time. This can be attributed to LTFM's lightweight design and its adeptness at capturing crucial temporal dependencies without significant computational overhead.We further explored the temporal resolution at which the LTFM operates in Table 6 . Different scenarios might benefit from different temporal granularities. When comparing the performance of LTFM at different time intervals, such as every frame, every 2 frames, and every 5 frames, we observed that fusing information at every 2 frames provided the optimal balance between computational efficiency and detection accuracy. Selection of 𝐾 in K-means and Top-K Attention. As illustrated in Figure 6a , increasing the number of clusters 𝐾 initially improves both NDS and mAP. The performance plateau observed after 𝐾 = 6 in K-means clustering suggests that there's an optimal number of clusters that capture the scene's essence. Having too many clusters might over-segment the data, leading to redundant or even conflicting information. Similarly, Figure 6b shows that utilizing Top-K Attention with 𝐾 = 6 yields the best performance, highlighting the importance of selective attention. Including Diversity Loss improves both NDS and mAP, as shown in the supplementary material, indicating its effectiveness in balancing the attention mechanism and capturing a variety of features.Comparative visualization of query results for object detection is shown in Figure 4 . Our dynamic querying scheme is more adaptive to occlusions and rapidly moving objects, and aligns well across different timestamps. As can be seen from Figure 5 , although some targets are difficult to distinguish or locate and are misidentified by SparseBEV, QE-BEV is able to correctly detect the targets. This result confirms the excellent performance of the QE-BEV detector.  
conclusion: 
 In this paper, we presented QE-BEV, a novel approach to 3D object detection that leverages dynamic queries in BEV space. Distinct from conventional static query-based techniques, QE-BEV iteratively adapts queries to capture complex spatial and temporal relationships within the data. This dynamic paradigm offers a more flexible and adaptive mechanism for 3D object detection, effectively constituting a new frontier in the field. Our method integrates various novel components, including K-means clustering for feature selection, Top-K Attention for adaptive feature aggregation, and a Lightweight Temporal Fusion Module for efficient temporal context integration. These components collectively enable our model to outperform state-of-the-art methods on various benchmarks, thus validating the efficacy of the dynamic query-based paradigm.As future work, we aim to explore the applicability of dynamic queries in other vision tasks and to further optimize the computational efficiency of our model. We also plan to investigate the potential of incorporating more advanced temporal models to capture long-term dependencies in videos or large-scale 3D scenes. 
 
 Appendix:  


Decision: This paper proposes a framework utilizing dynamic query evolution strategy in BEV object detection. The problem is that previous implementations of dynamics queries does not leverage temporal information in a computationally efficient manner. Thus, this paper adopt K-means clustering and Top-K attention mechanisms for refined spatio-temporal data processing. Experiments on the nuScenes and Waymo datasets demonstrate its effectiveness.

The preliminary ratings are (br, ba, ba, wa). Key concerns are:
1. Insufficient motivation without an adequate introduction to static and dynamic query-based methods and the reason of the design.
2. Some writing and format issues.
3. Missing some fair comparison to current methods, and the performance is not sound in supporting the claim.
4. Missing analysis in the limitation and the failure cases.

After author rebuttal, the final ratings are (na, na, ba, wa), na for not-available. Reviewer 3R1q and R48b haven't provided their final rating. Reviewer Uy4t and 2ugj maintain their scores after the rebuttal, as the concerns 3&4 are resolved. However, concerns 1&2 still stay. AC stepped in, read the paper, review comments, the rebuttal and the reviewer's final justification. In the view of AC, concern 1 are sufficiently justified in the author's rebuttal by more ablation experiments and comparison to current methods. Given the absence of reviewers' feedback, AC maintain this viewpoint as it is. Concern 2 is viable given enough time to revise the paper.

Based on the comments above, , AC decides to accept the paper since the paper shows the effectiveness of the novel modules under fair setting. The authors should improve the manuscripts according to the comments above.

Reviews: review: paper_summary: paper_summary: This paper introduces QE-BEV, a novel method for improving 3D object detection in bird's-eye view (BEV) images, especially useful in autonomous driving and robotics. QE-BEV utilizes dynamic query evolution and a lightweight temporal fusion module, employing K-means clustering and Top-K attention mechanisms to enhance detection accuracy and efficiency. The method shows significant improvements over existing techniques on the nuScenes and Waymo datasets.
strength_weakness: strength_weakness: 1. Dynamic Query Evolution: Introduces a strategy for iteratively updating queries during detection, allowing the model to adapt to complex spatio-temporal relationships, enhancing flexibility and adaptability.

2. Efficient Feature Aggregation: Utilizes K-means clustering and Top-K attention mechanisms to efficiently aggregate features, ensuring the model focuses on the most relevant feature clusters, improving detection accuracy.
limitations: limitations: 1. The log file in the supplementary materials contains the author's name, violating the double-blind requirement.
2024-03-24 20:29:36,354 - mmdet - INFO - Start running, host: lizhiqi@HOST-10-142-40-14, work_dir: /home/jiawei/QEBEV/work_dirs/bevformer_Res50

2. Insufficient motivation: The authors do not provide an adequate introduction to static and dynamic query-based methods.

3. Writing: The writing needs improvement. In Figure 1, what do the tokens represent? Is K-means applied to the image features?

4. Experimental section: The experimental results lag behind current methods. The authors lack a comparison of single-frame experimental results with other methods.
paper_summary: paper_summary: The paper proposes QE-BEV, a novel dynamic query mechanism for Bird’s Eye View (BEV) 3D object detection in varied contexts. This method utilizes K-means clustering and Top-K attention mechanisms for refined spatio-temporal data processing, enabling real-time, focused scene analysis. Extensive evaluations on the nuScenes and Waymo datasets demonstrate significant improvements in detection accuracy with QE-BEV.
strength_weakness: strength_weakness: 1. The introduction of the dynamic query evolution strategy effectively combines K-means clustering and Top-K attention mechanisms, enhancing the model's adaptability and computational efficiency.

2. Through dynamic querying and the Lightweight Temporal Fusion Module (LTFM), the model efficiently integrates spatio-temporal context, reducing computational overhead.
limitations: limitations: 1. Figure 3 is not referenced in the text and is difficult to understand. The authors need to explain it in the corresponding section of the paper.

2. The authors should explain why they chose Top-K attention over other attention mechanisms, such as deformable attention.

3. The authors need to conduct more ablation studies on hyperparameter selection to improve the completeness of the paper. For example, the hyperparameter Lambda for the Diversity Loss (Ldiv) and the choice of the number of queries should be demonstrated.

4. The comparison in Table 1 does not effectively demonstrate the validity of QE-BEV. Firstly, the authors do not mention the reasons behind the design of relevant hyperparameters, such as the number of queries, which significantly impact both efficiency and accuracy. Secondly, the authors only present the FPS for each model but do not provide corresponding performance metrics (NDS, mAP). It is suggested that the authors provide these performance metrics alongside FPS to offer a more comprehensive comparison.

5. The authors need to unify the naming of the model components. For instance, the term "Dynamic-K Block" mentioned in Section 4.1.1 should be clarified as DQEM if they are the same.
paper_summary: paper_summary: The paper "QE-BEV: Query Evolution for Bird’s Eye View Object Detection in Varied Contexts" introduces a dynamic query evolution framework for 3D object detection in BEV images, crucial for autonomous driving and robotics. The model uses K-means clustering, Top-K attention, and a Lightweight Temporal Fusion Module (LTFM) to improve adaptability and efficiency.
strength_weakness: strength_weakness: 1. Innovative dynamic query mechanism for complex scene understanding.

2. Improved computational efficiency through LTFM.

3. Performance gains over existing methods.
limitations: limitations: 1. Could you please provide more explanation on the performance degradation of clustering too much i.e. K increase.

2. The paper would be strengthened by a more thorough discussion of the limitations and potential failure cases of the methodology. It is important for readers to understand the circumstances under which the approach may not be effective or could encounter issues, and how these might be addressed or mitigated.
paper_summary: paper_summary: This paper presents a novel method, i.e. QE-BEV, for 3D object detection that leverages dynamic queries in BEV space. QE-BEV distincts previous methods mainly by associating spatial and temporal relationship with the queries. The main components within QE-BEV utilize K-means, Top-K attention and dynamic adaptation. The proposed method achieves competitive resulting on existing 3D object detection tasks.
strength_weakness: strength_weakness: (1)The proposed dynamic query mechanism enables to integrate temporal information from prior implementations.
(2)The proposed Dynamic Query Evolution Module (DQEM) exploits K-means clustering to extract adaptive structure representation and Top-K Attention for refining queries with their associated feature clusters.
(3)The proposed Lightweight Temporal Fusion Module (LTFM) enables to efficiently capture temporal context for each query with attention mechanism.
(4)The overall architecture of the method proposed in the paper is concise and easy to implement, while also achieving the state-of-the-art results on public datasets.
limitations: limitations: (1)It is suggested that the authors to add a comparison between method calculation consumption and model parameter quantity in the experimental section, as the method proposed in the article has the characteristic of lightweight.
(2)It is suggested that the authors add some visualization of intermediate results in the experimental section to better illustrate the role of the module.