Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a "dense-to-dense" paradigm that utilizes the dense guidance of teacher to supervise the students. Specifically, Dense Teacher Guidance (DTG)'s dense supervision is achieved by Inverse NMS Clustering (INC) and Rank Matching (RM), which regularizes the consistency on NMS between the teacher and student. INC leads the student to group the candidate boxes into clusters in NMS as the teacher does, so that the student obtains the same grouping scheme of NMS with the teacher. Rank Matching is further introduced to align the score rank over the clustered candidates between teacher and student.
strengths_and_weaknesses: Strength:
1. The motivation of "dense-to-dense" paradigm is strong and clear. The limitation of "sparse-to-dense" is well presented.
2. The proposed Inverse NMS Clustering (INC) and Rank Matching are novel. The rank of the samples within each cluster learned by the teacher serves as informative dense supervision, which enables the student to reserve the same candidates as the teacher during NMS.
3. Ablation studies show the effectiveness of each proposed components or strategies.
Weakness:
1. Figure1 and 2 are not illustrative. These two figures fail to give an intuitively visual demonstration on how DTG-SSOD works.
2. Table 1 is expected to show more results under different labeling ratios, e.g., 20%, 30%, 50%.
3. Figure 3 is not illustrative. I don't understand it.
questions: Could you better illustrate Figure 3? It is poorly presented.
limitations: Despite fair technical merit and solid experimental validation, the visual illustration in this paper is pretty poor.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper focuses on the pseudo-labeling strategy in semi-supervised object detection. Unlike previous approaches which generate sparse pseudo labels from teacher detector for each image and match these sparse pseudo labels to dense proposal and object detection predictions from student detector, this paper proposes a "dense-to-dense" paradigm. Inverse NMS Clustering and Rank Matching components are proposed to instantiate the dense-to-dense paradigm. Experiments on COCO show that the proposed approach obtains the state-of-the-art semi-supervised object detection results.
strengths_and_weaknesses: ### Strengths
- The proposed approach is interesting.
- Most parts of the paper are well written and easy to understand.
- Very promising results are obtained by the proposed approach and detailed ablation experiments are conducted.

### Weaknesses
- Some parts of the paper writing can be improved. The Figure 2 is not very clear about how inverse NMS and Rank Matching are performed.
- The proposed approach is mainly applied to Faster R-CNN based detector. It's unclear how it can be applied to recent end-to-end object detectors (e.g., DETR) which don't have dense RPN and proposal-wise predictions.
questions: How could the proposed approach be applied to other object detectors, especially the detectors without dense RPN/proposal-wise predictions and NMS (e.g., DETR)?
limitations: It's unclear how it can be applied to recent end-to-end object detectors (e.g., DETR) which don't have dense RPN and proposal-wise predictions.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a novel SSOD method that guides the student network to behave like the teacher network's NMS. The proposed method enforces a dense-to-dense training signal instead of the more common sparse-to-dense strategy in other SSOD methods. The proposed method is composed of two parts: (1) INC which leads the student to group candidate boxes into clusters in NMS as the teacher does, and (2) RM which imitates the rank distribution of the teacher over clustered candidates.

The author demonstrates the effectiveness of the proposed method compared to SoTA SSOD methods. They also conduct thorough ablations and analyses to help understand the contribution of each component and the advantage of the proposed method over the sparse-to-dense strategy.
strengths_and_weaknesses: ### Strength

1. Enforcing the student to mimic the teacher's NMS behavior is somewhat novel in SSOD.

1. This work is well motivated for why using dense-to-dense is a preferred training strategy.

1. The authors conduct thorough ablations (Sec.4.4) and analyses (Sec.4.5) for each component of the proposed method as well as why the proposed dense-to-dense method works better than the sparse-to-dense methods.

1. As shown in Tab.2(b), It is surprising that including the box regression loss is beneficial for the final performance without any tailored method or tricks.

### Weakness

1. It would be good to verify whether (or to what degree) the student network actually mimics the teacher's NMS behavior.

1. In Eq.4, is $p^t_i$ probability or logit? It seems weird to pass the probability through exponential and softmax. Also, is $p^t_i$ the output from an additional fc/mlp head or just the output from the existing objectiveness/classification head of the detector?

1. Is RM applied to the R-CNN part or the RPN part as well?

1. In Tab.2, what is the setting here? Eg. the amount of labeled data. Could the findings generalize to other settings (eg. different amounts of labeled data)? In Tab.2(a), it makes more sense to compare with sparse-to-dense rather than supervised baseline. In Tabl.2(e), it is better presented as a figure containing the val curve of both conditions.

1. It would be helpful to also include the results of the sparse-to-dense counterpart in Tab.1.

1. Are Fig.3 cherry-picked results or randomly selected results? It would be good to also include the failure modes and provide analyses and discussions.

1. In Fig.1(b), Inverse NMS Clustering, what does that number of star and triangle shapes mean? Also, why do the student samples not contain the reserved box?
questions: Please see the questions in the weakness part of the "Strengths And Weaknesses" section.
limitations: In lines 148-149, the authors claim, "In theory, our SSOD method is independent of the detection framework and can be applicable to both one-stage and two-stage detectors." However, as Unbiased Teacher v2 [1] pointed out, it is non-trivial to generalize a method developed on a two-stage detector to a one-stage one. 

[1] Liu, Yen-Cheng, Chih-Yao Ma, and Zsolt Kira. "Unbiased Teacher v2: Semi-Supervised Object Detection for Anchor-Free and Anchor-Based Detectors." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper introduces a new dense-to-dense” distillation to supervise the training of object detectors. The authors consider the advantages of the dense-to-dense paradigm and point out that the sparse-to-dense paradigm could accumulate noise in the hand-crafted NMS and label assignment process. They propose the Inverse NMS Clustering (INC) and Rank Matching (RM) to instantiate the dense supervision. Finally, the method achieve SOTA performance in both accuracy and efficiency.
strengths_and_weaknesses: - The central insight of the work is simple and intuitive i.e. using dense supervision can be more informative regarding the information loss of NMS and score filtering. And the authors propose some solutions(INC and RM) to these problems.
- The paper presents extensive and insightful ablations showing the importance of various choices (e.g. temperature T, choice of target boxes in INC etc.), and overall, all the choices made are empirically well-justified.

questions: 1. Which is better between your method and the feature distillation(distillation on the final layer)?
2. The supervision in Inverse NMS Clustering is a "sparse-to-dense" paradigm, which is supervised by a reserved box. The author claims that the weakness of the "sparse-to-dense" at the beginning.
limitations: 1. The questions above confuse me. Q1,Q2 . 
2. The method is not very interesting, but the paper presents extensive and insightful ablations study about their methods.
review_title: null

Meta-review: metareview: 
 This paper proposes a dense-to-dense semi-supervised object detection method, where the teacher's NMS is used to guide the clustering and ranking of bounding box candidates from the student. This is motivated from potential noise resulting from sparse-to-dense pseudo-label supervision in existing methods. Results are shown on standard semi-supervised object detection benchmarks, with improvements over the current state of art. 

The reviewers all thought that the paper had an interesting idea, strong results, and thorough experiments, ablations, and analysis. Some concerns included generalization to other architectures (e.g. DETR or single-stage CNN), comparison to feature distillation, and poor communication especially through the figures. The rebuttal provided answers to these, including new experiments showing generalization to a single-stage method, and all reviewers have recommended acceptance (and the reviewer with borderline accept mentioned it is a good paper). As a result, I recommend accepting this paper as it provides an interesting new contribution to the common mean teacher paradigm. I highly encourage the authors to add new elements that came out in the rebuttal, especially generalization to single-stage methods and failure cases. 
confidence: Certain
Decision: Accept