Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This work tackles the task of semi-supervised object detection via a teacher-student method. The authors introduced a training regime where a teacher and student network, who share the same initial weights pre-trained on labeled data, jointly learns on unsupervised data. They find that label imbalance in the object detection task can lead to inefficient pseudo-label training under the usual SSL training pipeline, and therefore proposes to train the teacher network via exponential moving average to avoid bias in pseudo-labels.


Pros:
1. Revisiting of pseudo-labeling bias issue in semi-supervised object detection is good.
This paper analyze both classification and regression loss with different ratios of labelled data, which shows that classification branch can easily suffer from overfitting that limits current state-of-the-art semi-supervised object detectors. It shows that the misalignment between classification and regression branch not only exists in fully-supervised learning object detection, but also in self-supervised object detection. 

2. Experiments are solid and convincing.
Table 1 & Table 2 show that unbiased teacher consistently improves state-of-the-art methods CSD and STAC by a large margin on both COCO and VOC dataset.

Cons:
1. Process of VOC12 dataset.
In Table 2, since VOC12 has a large overlap with VOC07, is the VOC07 part removed from VOC12 as unlabeled data?

2. Novelty
Nevertheless, some contributions seem a bit straight-forward and without significant novelty. The use of EMA is a direct adaptation from successful methods from classification, while Focal loss is already known to tackle class imbalance. The authors might need to provide more insights on the use of these methods (e.g. explaining in more detail how EMA alleviates such bias).

review_title: effective approach and good results

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper addressed an essential task for large-scale application of object detection -- semi-supervised learning. It introduced a simple but effective Unbiased Teacher to solve the traditionally problematic data imbalance issue. 

Pros

(1) Provides strong evidences and analysis on the class-imbalance problem inherited in pseudo-labeling methods;
(2) Proposed 
      (a) an interesting learning paradigm where the teacher is the temporal ensemble of student networks;
      (b) focal loss in place of cross entropy;
(3) The experiment and ablation suggested that the resulting teacher model is not prone to class-imbalance-induced overfitting and the improvement from SOTA is significant.

Cons

Curious to know why there is a drop in AP_50 comparing to STAC. 
review_title: Simple yet effective solution

final_score: rating: 7: Good paper, accept
review_content: Paper summary:
This paper focuses on the pseudo-labeling bias issue in semi-supervised object detection (SS-OD), and proposes an Unbiased Teacher framework to address this issue. More specifically, the unbiased teacher framework combines the Mean Teacher model for semi-supervised image classification (Tarvainen and Valpola 2017) and Focal Loss (Lin et al. 2017) for fully supervised object detection to address the bias issue. Experiments on COCO and PASCAL VOC show that the proposed method obtains the state-of-the-art semi-supervised object detection results.


Pros:

+ The paper is well written and easy to understand.

+ The motivations of this paper are clear and interesting.

+ A very simple but effective solution is proposed.

+ Very solid experiments are conducted.


Cons:

- The main weakness of this paper is that the proposed method is a simple combination of the previous methods including Mean Teacher and Focal Loss.
In addition, for the foreground/background imbalance issue, it is straightforward to generate pseudo gts to address this issue, which also has been studied by previous semi-supervised object detection work (Sohn et al. 2020). For the class imbalance issue in object detection, it is also straightforward to use Focal Loss to address this issue.

- One minor thing: It would be better to show results of different detectors / CNN backbones.


Review summary:
In summary, I think this is a good semi-supervised object detection paper because of its simple but effective solution and solid experiments. So I would like to give a weak accept to this paper.


Post-rebuttal comments: 
The authors have addressed my concerns in their rebuttals. All reviewers give positive comments to this paper. So I would like to give an accept to this paper.
review_title: Good paper with solid experiments

final_score: rating: 7: Good paper, accept
review_content: +This paper presents a good work on semi-supervised object detection (SSOD), which is a very challenging task. Although there are great progresses on semi-supervised classification, the SSOD is lying behind. This paper shows very good results over the supervised baselines, even when all annotations are used in COCO. 

+The proposed method is very simple. It seems the paper is easy to be reproduced.

+It is a good idea to use EMA of mean teacher for SSOD. In addition, the focal loss (FL) is shown to be very useful in SSOD. 

A few questions:

-The default FL is mainly for +/- class imbalance. Do you have modified for imbalance among all positive classes?

-It seems FL is more useful than EMA. However, FL is not well ablated. For example, it is the balance between +/- classes more important or among all positive classes? How about the other SSOD, e.g. STAC, using FL?

-It is understandable that EMA model is more reliable. But I don't see it can be beneficial for class imbalance issue explicitly. 

-The comparisons with the baseline supervised models seem to be not very fair. For example, in the last column of Table 1, the supervised model uses 1x schedule (90k iterations), but the proposed method uses 4x schedule. From the Detectron2 github, the 3x model should have AP of 40.2. When compared with 4x supervised baseline, the gain of this paper should be much smaller. The same issue could also be in the other COCO-standard experiments. More fair comparisons should be presented.

-Don't quite understand Fig. 4. Too few description. What are burn-in limit and GT. Why are they constant?

-For the paragraph of "Class-imbalance on pseudo-labels", if you don't show results, why you write it in Sec. 4.1, instead of Sec. 4.2?

-It is very weird to cite Law & Deng, 2018, when you refer to AP50 saturation. There are tons of paper out there showing that. 

-For the AP curves, do you evaluate on the full val2017 set? It seems the data points are very dense. Isn't it expensive?

-I don't think the smoother curve of EMA teacher is the weights of teacher model is detached from the student model. 

=====updates========

Most of my concerns have been addressed by the rebuttal and all the other reviews are positive. l will remain my original recommendation.
review_title: Good paper on semi-supervised object detection

Meta-review: This paper proposed a new semi-supervised object detection approach using Unbiased Teacher to jointly address the pseudo-labeling bias and overfitting issues. Significant improvements over SOTA were reported on COCO and VOC. Reviewers agree that the proposed method is simple and effective, and the experimental results are solid and convincing.  While the novelty of technical contributions for individual components may not be very significant, the idea is simple and well executed with strong results and good presentation. Overall, the paper is recommended for acceptance (poster). 
Decision: Accept (Poster)