Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a novel, simple and intuitive Aggregation Separation Loss (ASLoss), which aggregates the representations of the same class samples as near as possible and separates the representations of different classes as far as possible. The authors conduct extensive experiments on diffirent scenarios i.e. data insufficiency, class imbalance, and unclear class evidence to demonstrate ASLoss, and the results show that representations in deep spaces extracted by ASLoss improve classification performance and reaches the state-of-the-art level.
strength_and_weaknesses: Quality/Clarity: the paper is well written and the techniques presented are easy to follow. Its motivation is to aggregate the representations of the same class samples as near as possible and separate the representations of different classes as far as possible. And the authors design the corresponding Aggregation Separation Loss (ASLoss) to learning these representations. The method is validated our method on two image classification tasks and show promising results.

Originality/significance: the idea is incremental, which combines multiple loss functions (most are known) together to clarify confusion and improve separation over different classes. Extensive experiments are conducted to show its advantage on confusion-caused common conditions: data insufficiency, class imbalance, and unclear evidence. 
clarity,_quality,_novelty_and_reproducibility: Overall, it is a good paper based on the experimental results.
summary_of_the_review: The idea is incremental, where ASLoss combines multiple loss functions (most are known) together to clarify confusion and improve separation over different classes.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposed a new training loss that they claim to be helpful in data insufficient, class imbalance and unclear class evidence scenarios. The core idea is the newly created loss called ASLoss: it tries to maximize the similarities of samples belonging to the same classes; while minimize the similarities of samples across the classes. Despite the simple idea, the authors are able to obtain good results in Acne Severity Grading dataset and CelebA dataset.
strength_and_weaknesses: The biggest strength of this paper is clarity, the idea and motivation is clearly stated and verified through experiments.

The biggest concerns are:
1. Novelty: the idea seems to be not new. For instance in CosFace(https://arxiv.org/pdf/1801.09414.pdf) the author proposed to maximize the margin of Cosine distance.
2. Experiments: the authors claim that it will benefit the models in extreme cases such as: data insufficient, class imbalance etc. But they only experimented two (unpopular) datasets: Acne Severity gradient and CelebA. Clearly, more experimental supports are needed. 
3. Unable to calibrate this method with other baselines. Due to unpopular datasets, it is almost impossible to directly compare this method with baselines. Although authors compare quite a few baselines in Table 3, those are not official numbers. For instance in CBLoss(Cui et al.), Long-Tailed CIFAR, iNaturalist, ImageNet are studied. But in this paper the author ignored all those datasets. Because of this, we are not able to make fair comparisons.
4. (Minor) The formatting of this paper can be improved, many figures/plots are in low res.
clarity,_quality,_novelty_and_reproducibility: Clarity is good, the paper is easy to follow and clear to read in most places.
Quality and novelty are not good enough: the loss function seems not new enough. Quality needs to be improved.
Reproducibility is a concern: no directly comparison with prior works, datasets are too limited.

summary_of_the_review: The main reasons I tend to reject are listed in the previous sections.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a new loss function, e.g., Aggregation Separation Loss (ASLoss), to clarify confusion to improve image classification performance. 
Specifically, the ASLoss aggregates the representations of the same class samples as near as possible and separates the representations of different classes as far
as possible. Experimental results on two datasets (in the case of data insufficiency, class imbalance, and unclear class evidence) are conducted to illustrate the effectiveness of the proposed method.
strength_and_weaknesses: ## Strength
1. Overall, this paper is written rather clearly.
2. The considered problem is relevant to the community.


## Weaknesses
1. The "confusion" issue, as the authors claimed,  lacks a formal definition and seems a little confusing.
2. This paper lacks theoretical results to support the claims.
3. For the proposed ASLoss, the novelty is limited since the ideas of aggregating intra-class samples and separating inter-class samples are widely used in image classification. Besides, some parts of the notations are confusing and wrong. Specifically, in Eq.(1), the notation $N_{c_i = c_j}$  is confusing since what $i$ and $j$ come from? The similar issue also occurs for the notation $N_{c_i \neq c_j}$ in Eq.(2).
4. For the experiments, the hyper-parameters of the proposed method are selected on the test datasets during training and should be searched by another validation set. Moreover, this paper lacks ablation studies about the different parts of the proposed loss function.
clarity,_quality,_novelty_and_reproducibility: The clarity of this paper is rather well. The quality and novelty of this paper are limited. Please see the section "Weaknesses" for details. For the empirical results, it seems rather well to reproduce them.

summary_of_the_review: Overall, I think this paper proposes a new loss function for image classification where the novelty is limited and its effectiveness lacks theoretical support and ablation studies. Thus, the quality is poor and I suggest rejection.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper presents a loss function called ASLoss as an adjunct for classification loss to address the confusion issue under data insufficiency and class imbalance cases. The ASLoss can be decomposed into inner aggregation, outer separation and boundary constraint, which contraints the representation to have some invariant characteristics. The effectiveness is demonstrated by experiments.

strength_and_weaknesses: Pros:
The authors implement the ASLoss with different model architectures and show the effectiveness of the method. 

Cons:
The idea is exactly the same as metric learning loss. The three components, inner aggregation, outer seperation and boundary constraints are widely used in well-known paper like SimCLR and other contrastive learning paper or triplet loss paper. I do not see any references related to metric learning and the experiments in the paper are not strong enough to support that this method is very effective in the imbalanced learning or few-shot learning scenarios. To insert the ASLoss into classification problem is similar to the idea of supervised contrastive learning. I do not see enough novelty in this paper. 

Some other baseline models should be considered in imbalanced learning such as LDAM.
clarity,_quality,_novelty_and_reproducibility: I found the paper lack of novelty. This is very similar to metric learning.
summary_of_the_review: The authors ignored the work in metric learning e.g., contrastive learning and triplet loss, and proposed a loss function ASLoss which is very similar to those paper. I found the paper lack of novelty and the experimental results are not strong enough to support that this loss function is critic to this setup.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a new loss function designed to aggregate samples of the same class together and separate this of different classes. The reviewers point out that the idea is incremental, and I have to agree. In essence, all loss functions are designed to separate samples of different classes and the authors have not provided a theoretical justification or sufficient experiments to show that this procedure achieves something that's spectacularly different than prior deep learning architectures. The authors point out that data that fits all the challenging situations simultaneously is difficult to find/acquire.  In this case, the benefits of the method should be illustrated on data that only partially fulfills the criteria. If the situations when the method is best suited are extremely infrequent, that also makes the method less useful.

justification_for_why_not_higher_score: Limited novelty, lack of theoretical results, unconvincing experiments
Decision: Reject