Unleashing the Potential of Unlabeled Data: Bidirectional Collaborative Semi-Supervised Active Learning for 3D Object Detection

Authors: Zengran Wang, Yanan Zhang, Jiaxin Chen, Di Huang
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Autonomous Driving, 3D Object Detection, Active Learning, Semi-Supervised Learning

Abstract:
To address the annotation burden in LiDAR-based 3D object detection, active learning (AL) methods offer a promising solution. However, traditional active learning approaches solely rely on labeled data to train an initial model for data selection, overlooking the potential of leveraging unlabeled data. Recently, attempts to integrate semi-supervised learning (SSL) into AL with the goal of leveraging unlabeled data have faced challenges in effectively resolving the conflict between the two paradigms, resulting in less satisfactory performance.
To tackle this conflict, we propose a Bidirectional Collaborative Semi-Supervised Active Learning framework, dubbed as BC-SSAL. Specifically, from the perspective of SSL, we propose a Collaborative PseudoScene Pre-training (CPSP) method that effectively learns from unlabeled data without introducing adverse effects. From the perspective of AL, we design a Collaborative Active Learning (CAL) method tailored for outdoor LiDAR scenes, which complements the uncertainty and diversity methods by model cascading, alleviating the dilemma of sampling rare classes. Extensive experiments conducted on KITTI and Waymo demonstrate the effectiveness of our BC-SSAL. Especially, on the KITTI dataset, utilizing only 2\% labeled data, BC-SSAL can achieve comparable performance to the model trained on the full set.

Decision: reject
Reviewer scores: 5, 3, 5, 5

Reviews and discussion:
<thread>
creation date:
1730543716906

thread_type:
official_review

<thread_object>
creator: reviewer_dsaa

<content>
summary:
This paper proposes an active learning and semi-supervised learning collaborative label-efficient 3D object detection method. By mining useful information from unlabeled data to enhance the performance of 3D detectors. The proposed method is experimentally validated on two widely used datasets.

strengths:
1. The paper presents an ideal for 3D object detection that collaborates semi-supervised learning and active learning bidirectionally, which is novel and interesting.
2. The design of the active learning scheme is reasonable, and the experiments have successfully validated the performance improvement brought by this module to the entire method.

weaknesses:
1. The proposed method has limited innovation; the CPSP module is highly similar to the 'Reliable Background Mining Module' presented in the existing SS3D[1]. The Reliable Background Mining Module employs a pre-trained detector to process unlabeled scenes, extracting foreground instances to construct a bank, and then uses gt-sampling data augmentation to generate new training data. The CPSP module follows the same procedure without any difference.

2. The motivation behind this paper is unclear, as there is no data proving the conflict between SSL and AL. It is suggested to provide experimental results of a naive combination of SSL  and AL  for 3D object detection, demonstrating that directly combining the two methods is unreliable. Additionally, the method does not offer a special design based on the proposed concept of 'bidirectional collaboration'.

3. The figures in the paper are sketchy and not aesthetically pleasing. 

[1] Liu. et.al. SS3D: Sparsely-Supervised 3D Object Detection from Point Cloud. CVPR2022

questions:
see Weaknesses*.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730608342369

thread_type:
official_review

<thread_object>
creator: reviewer_ww9a

<content>
summary:
The paper introduces BC-SSAL (Bidirectional Collaborative Semi-Supervised Active Learning) to enhance 3D object detection in LiDAR-based systems. The framework addresses the challenge of annotation burden by effectively leveraging unlabeled data through a combination of semi-supervised learning (SSL) and active learning (AL). Overall, this paper is technically detailed and the experiments are quite comprehensive. However, most of the designs in the paper are combinations of existing work, lacking novel designs, and the improvements in the experiments are not significant.

strengths:
1. The paper is well-structured, with a clear abstract, introduction, methodology, experiments, and conclusion sections that logically flow from one to the next.
2. The paper provides extensive experimental results on the KITTI and WOD datasets and conducts a large number of ablation studies.
3. The paper provides a multitude of figures that clearly demonstrate the design details of each module, facilitating the reader's understanding or reproduction of the methods described in the paper.

weaknesses:
1. It is not clear why the method is named ‘Bidirectional xx’, I cannot see any module design that is ‘bidirectional’.
2. I cannot see any special contributions from this paper. Most of the components are simple combinations of existing works. Such as the so-called CPSP is indeed a GT sampling data augmentation used in most of regular 3D object detectors.
3. The improvement of this method does not seem to be very significant. On the KITTI dataset, the most convincing category is the car, as there are enough objects in this category to make the conclusions more reliable. However, this method is not even as good as 3DIoUMatch.

questions:
1. Why is the paper titled 'Bidirectional xx'? Where is the bidirectional operation reflected?
2. What are the core differences between CPSP and GT sampling?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730636861013

thread_type:
official_review

<thread_object>
creator: reviewer_blqe

<content>
summary:
The paper introduces a Bidirectional Collaborative Semi-Supervised Active Learning (BC-SSAL) framework for 3D object detection in LiDAR data, aiming to reduce the annotation burden. BC-SSAL integrates semi-supervised learning (SSL) with active learning (AL) to leverage unlabeled data effectively. Experiments on KITTI and Waymo datasets show BC-SSAL achieves state-of-the-art performance.

strengths:
This method combines active learning with semi-supervised learning, proposing a new scheme for efficient label learning. The idea is interesting. The proposed method achieves the best performance compared to multiple baseline methods on two datasets.

weaknesses:
1. The Pre-train method proposed in this paper has limited novelty. The main content highly overlaps with SS3D[1]. Specifically, SS3D combines a pre-trained detector to mine potential instances in unlabeled scenes to generate an 'Instance Bank' and 'Broken Scene', and then uses gt-sampling to produce training data.
2. The effectiveness of the active learning module is missing validation. One of the main contributions of this paper is active learning, so it is necessary to supplement a separate performance comparison with the state-of-the-art (SOTA) active learning methods [2][3].
3. The method is called "Bidirectional Collaborative Semi-Supervised Active Learning," but no clear design addresses this specific collaboration. It is recommended that the authors clarify the specific conflicts between the two strategies, provide experimental data to demonstrate the consequences of such conflicts, and explain how these conflicts are resolved.
4. The semi-supervised scheme only employs HSSDA, lacking validation across different semi-supervised approaches. It is suggested that the authors validate the proposed bidirectional collaboration scheme across multiple representative semi-supervised methods. 

[1] SS3D: Sparsely-Supervised 3D Object Detection from Point Cloud. CVPR 2022
[2] KECOR: Kernel Coding Rate Maximization for Active 3D Object Detection. ICCV 2023
[3] Exploringactive 3d object detection from a generalization perspective.

questions:
What are the differences between the design of the CPSP module and the related model design in SS3D? Please also respond to the comments in Weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730685124326

thread_type:
official_review

<thread_object>
creator: reviewer_vc1w

<content>
summary:
This paper aims to reduce the annotation burden in LiDAR-based 3D object detection, the authors propose a Bidirectional Collaborative Semi-Supervised Active Learning framework (BC-SSAL). This framework combines Collaborative PseudoScene Pre-training (CPSP) to effectively utilize unlabeled data and Collaborative Active Learning (CAL) to enhance sampling, particularly for rare classes, in outdoor LiDAR scenes.

strengths:
- The task setting is practical, combining active learning (AL) and semi-supervised learning (SSL) to enhance 3D detection performance while minimizing annotation requirements.
- Good exploration and analysis are conducted on various strategies for integrating SSL with existing AL frameworks.

weaknesses:
- The proposed Collaborative PseudoScene Pre-training (CPSP) module shares similarities with the approach in [A], in which high-quality, high-certainty bounding boxes are stored in a memory bank, and point clouds from these boxes are integrated into scenes. Could you clarify and compare the conceptual and empirical differences between CPSP and [A]?

- The Active Learning (AL) sampling strategy lacks novelty, as entropy is a widely used general AL method. Moreover, the box diversity metric shows minimal improvement, as demonstrated in Table 4.

- Some typos: In lines 177 and 179, there should be a space before "(TMU)" and "(USS)." 


[A] Chen, Z., Luo, Y., Wang, Z., Baktashmotlagh, M., & Huang, Z. (2023). Revisiting domain-adaptive 3D object detection by reliable, diverse and class-balanced pseudo-labeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 3714-3726).

questions:
The impact of varying thresholds for confident object extraction has not been studied. Given that the model was initially pretrained on a limited dataset, its predictions may contain noise, suggesting that thresholds should be carefully tuned and selected. Could you explain the process for determining the optimal thresholds and demonstrate the impact of different threshold values?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732246552650

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response Reminder

comment:
We have taken your initial feedback into careful consideration and have provided our responses accordingly. Would you mind confirming whether our responses have satisfactorily addressed your concerns? If you believe we have successfully addressed your concerns, we kindly request that you consider adjusting your initial score accordingly. We understand that you are busy, and we would be very grateful if you could keep our response in mind during discussions with the AC and other reviewers. Please feel free to share any additional comments you may have.

Once again, we sincerely thank you for dedicating your time and effort to reviewing our work.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732520199669

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Reviewer-Author Discussion Period Ends in TWO Days

comment:
Thanks again for reviewing our paper. We hope that our response adequately addressed your concerns. As the deadline approaches, please let us know if you have any further questions before the reviewer-author discussion period ends. We understand your busy schedule, and we would greatly appreciate it if you could consider our response during discussions with the AC and other reviewers.

We are more than happy to address any additional concerns or questions you may have.

Once again, we sincerely thank you for dedicating your time and effort to reviewing our work.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732785460454

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Emphasizing Once Again the Overall Contributions of Our Work

comment:
We have noticed that some reviewers, due to the similarity of one module in our paper to existing methods, **have overlooked the overall contributions** of our work. However, we would like to clarify the following points:

1. **The central contribution of our paper** is the **resolution of the conflict between SSL and AL** in the SSAL framework, a challenge that has been largely overlooked by prior methods. By resolving this conflict, we propose a **bidirectional collaboration** where SSL improves AL through calibrated pseudo-labels, and AL enhances SSL by addressing **class imbalance** and improving **uncertainty estimation**. This conflict resolution is fundamental to the success of SSAL, and it **differentiates our work** from previous SSAL frameworks.
2. **Our CPSP module is not a direct replication** of existing methods. It introduces **novel design choices**, such as the **box bank deletion mechanism** and the **use of false-positive backgrounds**, which improve **model calibration** and **uncertainty measures**—features not present in prior work like SS3D. These unique contributions are essential to making SSAL work effectively.
3. Our **CAL module** makes significant contributions by addressing **class imbalance**, improving **uncertainty estimation**, and eliminating redundant samples using **similarity metrics**. CAL ensures more **effective sample selection**, particularly for rare categories like cyclists, which are often neglected in other frameworks.

Overall, our work provides **a comprehensive and impactful solution** to the SSL-AL conflict in SSAL, with **important implications** for future research and application in the field. We respectfully ask the reviewers and AC to evaluate the **overall contributions** of our work, rather than focusing on the similarity of one module. Dismissing the **holistic value** of our approach would be an unfair evaluation, and we hope that the **entirety of our contributions** will be taken into account during the review process.

Thank you for your time and feedback.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733124060793

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response Reminder: Approaching Deadline

comment:
**Thank you for reviewing our paper.** As the deadline is approaching, if our response has addressed your concerns, we kindly ask you to consider adjusting your scores accordingly. If you have any further questions or concerns, please let us know, and we will be happy to provide additional clarifications. Your feedback is greatly appreciated, and we look forward to your response before the reviewer-author discussion period ends.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper investigates a unified framework integrating semi-supervised learning (SSL) and active learning (AL) to provide a cost-effective solution for 3D object detection. By employing a nested refinement loop, the AL component selects instances that are both uncertain and diverse, while the SSL component self-trains on high-confidence instances, thereby reducing annotation overhead. Additionally, class-balanced learning is incorporated, and the proposed approach is evaluated on two widely used benchmark datasets. 

The paper’s strengths lie in the extensive experimental comparisons with numerous existing AL methods, demonstrating notable performance improvements across different classes. The results show performance improvements over numerous existing AL solutions, although these gains are somewhat expected due to the additional use of unlabeled data, which is not a typical consideration in mainstream AL methods. 

The primary concern raised by all reviewers is that the paper’s core technical contribution remains unclear. While the integration of AL and SSL is relatively underexplored, the chosen methods for measuring uncertainty and diversity seem quite standard, and the overall architecture appears to be a high-level combination of existing approaches. To strengthen this work in the future, it would be more beneficial to target a more specific technical challenge in AL and SSL (such as, but definitely not limited to, how to adaptively control the annotation budget, how to set the threshold for SSL, or preventing forgetting during iterative training without excessive data resampling) and this can help audience to better understand the core innovation and facilitate future work in this fields. It is recommended authors carefully go through the reviewers' suggestions and reposition the focus of this work, which would help strengthen the quality and impact of this work.