Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work is motivated by the different nature of the "other"/background class in many vision tasks such as object detection and semantic segmentation. It introduces C+1 loss which consists of 3 individual loss terms that focus on the basic discriminability, intra-class compactness, and background margin. Basic discriminability loss is the conventional classification loss for C+1 classes, intra-class compactness uses center loss, and background margin loss is a margin loss acted on the center representation of all C classes and the features of background samples.
main_review: Strengths
- The introduced C+1 loss is widely applicable to many kinds of tasks that require classification.
- The experiments are done extensively on multiple tasks to demonstrate the effectiveness of C+1 loss: object detection, semantic segmentation, human parsing.

Weaknesses
- C+1 loss is a simple and straightforward combination of existing loss functions already explored in previous papers `Wen et al. (2016)`. The background margin loss is almost identical to the conventional margin ranking loss. If you take things that are working really well and combine them in a simple way, it is likely that the combination will work better than any of the individual things.
- The naming of "basic discriminability" is so arbitrary and not supported by anything. Why does it have to be "basic"?
- When the paper mentions it uses center loss for L_compact, it does not cite the any papers about center loss.
- The proposed properties of C+1 loss are not backed by any theoretical foundations or insights.
- Except for DeepLabV3 on Pascal VOC, the performance gains are not so significant (most are less than 1%) in the evaluated tasks.

summary_of_the_review: This paper absolutely feels like a simple repackaging of old methods with a new name (C+1 loss) and there are almost no good enough contributions to make a case for acceptance.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: The paper on hands tackles the problem of multi-class classification in the presence of a general  background class. This is illustrated by a specific loss function covering this aspect and demonstrated for a well known, but very old benchmark,
main_review: The paper tackles an interesting and fair aspect in the field of multiclass classification. However, the neither the idea nor the kind of solution are new. In addition, the experimental results are not promising and show just slightly different results compared to arbitrarily selected baselines. From this points of view, there is neither a novel contribution nor provides the paper thrilling new insights to the tackled problem. In addition, the paper is not written and structured very well. For instance, the mathematical writing and the bibliography need to be seriously checked!
summary_of_the_review: To summarize, the paper covers an interesting aspect, however, the overall novelty of the paper is very slim and the quality needs to be seriously improved. Thus, there is the clear recommendation not to accept the paper for ICLR!
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper addresses the problem of semantic segmentation/classification, in situations where one has C+1 classes at hand, including  C deterministic classes and a background class. The authors identifies the weakness of the classical softmax classification layer in this context, and propose a novel loss, that can better account for the particularity of the non-deterministic background class. Experiments are performed on VOC-2012, Pascal context, LIP and COCO2017. Quantitative results, ablation study and comparison with related work are reported. 

main_review: * Significance. The problem addressed by the author is important (the background class is present in most of the vision benchmarks) but few work (to my knowledge) attempted to tackle it in a rigorous way. 

* Novelty. The solution proposed by the authors (decompose the loss function into three terms, to enforce some constraints of the last layer feature space) is original, easy to implement and has the potential to inspire others. The properties identified by the authors (which translate into these loss terms) are well discussed and intuitive. 
 It might however be surprising  to maintain the basic loss (cross entropy preceded by a softmax) as such (since it probably acts against the compactness and background margin properties enforced by the two other terms). 

* Results. The authors report convincing results on four different datasets and two different tasks. The experiments (ablation study) validate the claims of the introduction. While the quantitative analysis consistently show improvement wrt to baseline or the sota, I believe that more challenging experiments could have
been performed for binary classification tasks (eg, human detection/classification, road sign detection), for which the background is often very diverse. 

* Clarity. The paper is fluid, a pleasure to read.
summary_of_the_review: I enjoyed reading the paper because it tackles a well identified problem with clarity and concision.
The problem is well defined, the solution is simple and explained with clarity. 
I believe that it belongs to the category of papers that I read once, keep it in memory, and come back to it regularly. 

---------------------------------------------------
Updates: Thanks for the authors' response, which partially addresses my concerns.  I appreciate the fact that the authors specifically point out in their paper that the background class should be treated in a differentiated way wrt other classes. This observation, in itself, in my view, constitutes the originality of the work --even though the technical solution is surprisingly simple. I encourage the authors to re-submit to an other venue.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper aims at dealing with the C+1 classification problem, where C foreground classes and a background class, which could consist of other classes that do not overlap with the C foreground classes. The paper additionally includes center loss for foreground classes and a modified center loss for background class(es), on top of the standard cross-entropy loss. Results are reported on PASCAL VOC, PASCAL Context and LIP datasets.
main_review: Strengths:
+ The employed additional loss functions improved the original methods without these additional loss terms.

Weaknesses:
- Lack of novelty. The C+1 classification is a straightforward and widely adopted setting, which is the default for segmentation task, while the paper claims that "We define the C + 1 classification problem ..."; The method itself is basically adding two additional regularization terms using the center loss, which is employed from existing methods.
- With the claim being handling C+1 classification problem, methods from the OSR literature should be compared. 
- Though an improvement on overall classification performance is shown, however, no evidence is provided showing that the representation is more compact and the margin has been full filled.
- No evidence is shown to support the conclusion of equation (3). The radii of the small and large balls should be demonstrated (e.g., by feature visualization).
- The writing is unsatisfactory, with grammatical errors and typos even in the abstract. For example, CoI is never mentioned before the short form appears; "In spite of ... use" ->  "In spite of ... using". The main context contains more issues...
summary_of_the_review: Overall, the paper severely lacks novelty. The method simply adopts a loss function introduced in the existing method and a slightly modified version of it to improve the performance. The claims are not well supported by the experiments. The writing is unsatisfactory.
review_title: null

Meta-review: This paper presents work on classification with a background class.  The reviewers appreciated the important, standard problem the paper considers.  However, concerns were raised regarding presentation, empirical evaluation, clarity, novelty, and signficance of the work.  The reviewers considered the authors' response in their subsequent discussions but felt the concerns were not adequately addressed.  Based on this feedback the paper is not yet ready for publication in ICLR.
Decision: Reject