Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a method to improve the performance of object detectors with supervised training. The main ideas consist of two parts: (1) The human annotation of bounding boxes is often noisy or incomplete. Adopting a teacher model to generate pseudo boxes and combining them with original annotation can improve the performance of object detectors. (2) Strong augmentation may generate data which degrades the training performance. Adopting a teacher model to select easy targets in the augmented images for training can improve the performance.
In this paper, the author adopts an exponential moving average model as the teacher model. The proposed MixTraining strategy improves the performance (1.6~2.3 mAP) of several object detectors in COCO benchmark.

main_review: Strengths:
This paper is well written and easy to follow. The idea of adopting teacher model to handle the out of distribution data generated by strong augmentation is simple and effective.
The ablation shows the effectiveness of adopting pseudo boxes in the supervised object detection training. 

Weakness:
The proposed method could only address the missing label problem or inaccurate bounding box localization. As for false positive annotation, the proposed method can not handle.
Missing references for data augmentation works which adopt different augmentation policies for different data samples during training.
As claimed in this paper, the author considers strong data augmentation may degrade the performance of object detectors and provide experiments in Table 6. However, as shown in [1] and [2], the searched data augmentation can improve the performance significantly. Adopting the proposed method on [1] or [2] will make the experiments stronger.

[1] Learning Data Augmentation Strategies for Object Detection.
[2] Scale-aware Automatic Augmentation for Object Detection.

Comments:
Could the authors adopt the proposed method for one-stage object detectors? Why do the authors only provide results of two-stage detectors?

limitations_and_societal_impact: See above
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The study offers MixTraining, an object detection training technique that may be used to improve the performance of existing detectors. The suggested method improves data augmentation by leveraging augmentations of various strengths while eliminating strong augmentations of particular training samples, which could be harmful to training. It addresses localization noise and missing labels in human annotations as well. The efficacy of MixTraining is validated on the COCO dataset using different deep learning models and backbones.
main_review: This study utilizes augmentations of varying strengths while avoiding strong augmentations of certain training samples commendably. Psedo-labeling has been around for a long time. Overall, the authors enhanced a few existing techniques in an effective manner.

The authors demonstrate how to effectively use an augmentation technique and the value of human-annotated bounding boxes to bootstrap an object detector by extending its capabilities. They show that a model/backbone trained using the suggested mechanism outperforms a model/backbone trained using a typical setting.

The manuscript is well-organized, well-written, and easy to follow. The motivation of the study is clear and well-defined. The authors evidently explain how to apply an augmentation strategy to improve an object detector's performance.

The proposed mechanism is useful. The study does a great job of improving the performance of a detector by employing a previously less explored mechanism. However, in my opinion, the contribution of the paper is somewhat incremental which effectively uses the MixTraining technique to improve the capabilities of existing detectors.

To sum up, the authors demonstrate how an augmentation method and the efficacy of human-annotated bounding boxes can be extended beyond a typical training paradigm for object detection. I think this is an interesting approach that could prove beneficial to the research community. The proposed method can also be utilized effectively in real-world applications to enhance the performance of an object detector. Overall, I think this is a good study and am inclined to accept the paper.
limitations_and_societal_impact: I do not think there is any potential negative societal impact of this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: Object detection is a fundamental task in computer vision. In this paper a new paradigm of training, MixTraining, is proposed which utilizes augmentation of different strengths. Either standard augmentation and standard processing is used for an image or strong augmentation is applied (mixed data augmentation) and changed ground truth is used (loss is computed only on simple boxes and mixed training targets is used). As soon as human-annotated bounding boxes often are noisy (missing or inaccurate localizations) the other key ingredient when strong augmentation is applied to an image is to use only confident bounding boxes predicted by the model itself (fixes errors in human annotations) in addition to the confident bounding boxes from the ground truth (making simpler task as soon as strong augmentation is used). This key ingredient is obtained via pseudo-labeling by EMA detector (pseudo boxes). The idea is that this pseudo boxes and bootstrapping from the model itself help to detect images where strong augmentation can be expected to be helpful. MixTraining is shown to consistently improve various detectors on the COCO dataset.
main_review: **Originality**

Authors propose a novel paradigm of training for object detection when both normal and strong augmentations are involved and pseudo boxes are used to define how particular box is hard or not to use as a target in strongly augmented sample.  Augmentation, EMA and pseudo labeling are very well known, however, the latest  is more commonly used in semi/self-supervised setting and not for the supervised one to guide how hard or simple to detect particular bounding box. Known techniques are used from a new point of view.

**Quality**

There is no any theoretical analysis and work is purely experimental. However, all experiments are very well defined and looks reasonable and technically strong:
- different backbones and detectors are considered
- careful training schedules are used as long as different models and paradigms have different convergence speed (of course due to stronger augmentation used in MixTraining for example).
- extensive ablations are performed to understand the mixed targets and mixed augmentations dynamics
- extra analysis on pseudo boxes is conducted

However some extra information on choice of crucial MixTraining hyper-parameters is not provided/discussed:
- When is strong augmentation branch involved into training: right at the beginning or after EMA model reaches some good performance? Did authors observe any problem on bootstrapping with pseudo boxes?
- Are pseudo boxes regenerated every update? Does model generate pseudo boxes and strong targets once per N updates for all data?
- What is the dynamics of the parameters from above two bullets?
- Is probability to apply normal and strong  augmentations 0.5? Or do for the update step we apply both normal and strong augmentation for an image and compute Loss_normal + Loss_strong? Do we apply only strong/normal augmentation to the whole batch or mix of normal and strong ones?

Other comments:
- Is prediction score threshold tuned somehow, why is its value chosen to be 0.9?
- Would be great to have experiment also with on-stage detector to have broader experimental proof that method works.
- How were the augmentations and their parameters found for normal and strong augmentation strategies?
- In fig. 5 seems the learning rate decay happens at some point when there is a gap in model performances. Is it true? If yes then the best learning strategy is perform decay when the validation mAP is at maximum. Did authors try this schedule to avoid overfitting in SiTraining earlier to decay learning rate?

**Clarity**

The paper is very well written and organized (no any typos found during reading). Explanations with proper references on original works are well described too. Comparison with existed methods is fully covered in Section 2. All details on the reproduction steps are provided and the new training paradigm is clear described in the text and by Fig. 1.

**Significance** 

MixTraining is a novel general framework, which demonstrates ability on bootstrapping and self-improving even with noisy labels like missed or inaccurate localizations. MixTraining does not introduce extra parameters/computations at inference time, while improve leading detectors significantly via the training strategy itself (Faster R-CNN 41.7 -> 44 mAP; Cascade R-CNN 50.9 -> 52.8 mAP). This also opens potential research for semi-supervised learning as soon as bootstrapping via pseudo-labeling by EMA detector is demonstrated to work.

**Update on decision:**
Changing final decision from 6 to 7 as authors addressed all concerns in the below comments.
limitations_and_societal_impact: Didn't discussed in the paper. It is referenced to Appendix in the Checklist, however, Appendix was not attached to the submission. Either include in the Appendix and post it or add in the main text.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper introduces a new training paradigm named MixTraining for object detection that can improve the performance of existing detectors for free. The main idea is that MixTraining enhances data augmentation by utilizing augmentations of different strengths. MixTraining is found to bring consistent improvements across various detectors on the COCO dataset. It improves faster R-CNN from 41.7 to 44.0 and Swin-small from 50.9 to 52.8. 
main_review: Strengths:
1) The motivation to perform instance-wise augmentation for different instance is clear and make sense;
2) The method is straightforward and easy to follow;
3) The effectiveness of the method is validated on a strong baseline, which is a 50 + mAp detector on COCO dataset.
4) No further computation cost is found in the proposed method, MixTraining can bring improvement for free.
5) Good experimental results and ablation studies.

Weaknesses:
1) The network has to recognize each instance with different augmentations, which may slow down the training time.
limitations_and_societal_impact: The authors adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

Meta-review: The paper presents a new training procedure for object detection that relies on a teacher model and data augmentation. The approach adds a loss that is computed on a strongly augmented image but only on boxes that are above a threshold for the teacher (exponential moving average) model on the raw image. It is quite simple and generic, and provides a significant boost in final performance. The experiments are conducted on COCO with strong standard models (Faster R-CNN and Cascade-R-CNN) models. The reviewers have made constructive observations and all agree that this is a good paper. Overall, the paper is a significant contribution and is a welcome addition to the program of NeurIPS.
Decision: Accept (Spotlight)