Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a normalized calibration method for long-tailed object detection and instance segmentation task. Experimental resutls show that separately treating the background class and normalizing the scores over classes for each proposal are keys to achieving superior performance. Experiments are conducted on the LVIS dataset with diverse evaluation sets. Results shows that consistent improvement can be achieved by the proposed scheme, and  extensive analysis and ablation studies are provided.

main_review: The paper apply simple post-calibration techniques from standard multi-way to object detection and instance segmentation. The main idea is to scale down the logit of each class with respect to its train size. 
Overall, the paper is well organized, and the proposed scheme is clearly presented. The idea is straightforward and easy to implement, as only Hyper-parameter is introduced for tune in the scheme. The extensive experiments offer insights into various modeling choices and mechanisms of the scheme.
It would be better to conduct experiments on a more general dataset, (e.g. Open Images) to demonstrate its effectiveness.

limitations_and_societal_impact: Authors doesn't address the limitation and potential negative societal impact.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: A simple and effective post-processing calibration method is proposed for long-tailed object detectors. 
main_review: Strength
- The paper presents an interesting idea. It is quite effective and requires no re-training of the object detectors, making it a useful and low-effort tool to use. Experiments also show superior performance than other methods which requires a change of training.
- The experiments revealed that the key to performance improvement is normalization especially for the frequent classes, which is an important result and helps us better understand how the method works. 
- The paper is well written and clear. The illustrations to explain why the proposed method works are very useful.

Weakness
- The proposed method is purely about post-processing of classification scores in object detectors. Thus the localization part is not important. Therefore it makes sense to start by studying the long-tailed image classification problem and then apply it to object detection/instance segmentation problem.  It'll be better if the authors can show the efficacy of this method on classification problems as well.
- It makes sense that the proposed method works well with multi-class classifiers. However, for multiple binary classifiers, as described in Section 3.4 and Eq. 7, there's no normalization applied. In this case, the ranking of the boxes per class won't be changed because Eq. 7 is a monotonic function. So how is the performance improvement achieved?
- Although the proposed method works well in the experiments, the overall novelty is not high. 

Minor comments
- In Eq. 7, should it be dividing by the factor instead of multiplying it?
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a simple yet effective approach “Normalized Calibration” to post-process the class-specific logits from modern object detector using soft-max prediction for all object and one background classes. Experiments on LVIS has been conducted, and comparison with quite a few baseline methods are presented to show its superior performance for frequent, common and rare classes.
main_review: The idea seems to be original, which is to scale down the exponentialized logits by the class size in the training set and L1-normalize scores. Without L1-normalization, the calibration has a similar form to tar-normalization in [20] (e.g, scale down the logits before taking exponentiation) (also see table 3). It seems L1-normalization plays a more important role than logit calibration.

The paper writing is clean and easy to follow. External comparisons and internal ablative studies are convincing. There are enough clarity in the technical exposition.

One limitation is NorCal seems to work less well for detectors predicting sigmoid scores that are not necessarily L1-normalized (e.g. RetinaNet with focal loss). I would like to see more results using detectors using sigmoid rather than softmax to compute class scores.

Currently, only one task and one dataset LVIS is used for evaluation. The proposed NorCal seems to be generic enough to be evaluated on long-tailed classification such as ImageNet-LT and iNaturalist benchmark. Results will be more convincing if results on a 2nd dataset or task are provided.
limitations_and_societal_impact: I do not have concerns on the negative societal impact of this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors study the problem of "calibration" of long-tail object detection models. To be specific, they take an existing pre-trained object detector and modify its logits according to the number of samples in each class to obtain better logits/scores for each class. They demonstrate on the long-tailed LVIS dataset with different models that this significantly and consistently improves the performance of methods (in terms of both precision and recall). 
main_review: Strengths:
- A simple yet robust and generic method.
- Extensive ablation analysis. 
- Strong results.
- Clear and easy to follow text.

Weaknesses:
- The paper provides a very simple application of the normalization methods in other studies for object detection. In this sense, technical/methodological contribution is limited. Though, I favor simple solutions and I am not going to reflect this in my recommendation.

- This is a paper for addressing an imbalance problem for object detection and therefore, the authors should cite the following review paper on imbalance problems in object detection:

Oksuz et al., "Imbalance Problems in Object Detection: A Review", IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2021.

- Figure 2: It is not clear how these scores are obtained. Which split of the dataset is used? Which detector is used? How did you perform normalization for the plot on the left? Are the scores on the right not normalized? Why not? What do you mean by "another rare" classes? .. 

Minor comments:
- Figure 2: It would be nicer if the font sizes & styles of the text in the figure match the font size and the style of the main text. Moreover, it is very clear that the spacing between the figure and the text has been played with. This is against the formatting instructions.
- Footnote 2 on page 4: The period (.) is missing.
- Line 133: N_c has not been introduced yet.
- Line 136: "confidential scores" => "confidence scores".
- Line 148: "posts" => "posits".
- Line 149: "background patches" => "background regions".
- Line 231: "methods that trained" => "methods that are trained".

**AFTER REBUTTAL**

I've looked at the comments of the other reviewers and noticed that we are generally very positive about the paper. Some of us requested more experiments and the authors provided the results for them in their rebuttal, which supported the claims of the authors. Therefore, I will keep my original score of acceptance for the paper.
limitations_and_societal_impact: Yes, they have.
review_title: null

Meta-review: The reviewers were generally positive about this paper, and most of the raised concerns were cleared after the responce. Thus I recommend accepting the paper.
Decision: Accept (Poster)