Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors analyze the class-imbalanced learning problem through the lens of the loss landscape. 
Specifically, they examine the spectral density of Hessian of class-wise loss, in which they observe that for the tail class loss landscape, the solution converges to a saddle point. 
They further theoretically and empirically demonstrate that Sharpness Aware Minimization (SAM), a recent technique that aims to converge to flat minima, can be effectively used to escape saddle points for minority classes.
Borrowing SAM results in an increase in accuracy in the minority classes.

strengths_and_weaknesses: Strength

- The idea to interpret the long-tailed distribution problem with respect to the loss of landscape is novel.
- The authors validate the effect of the SAM for model training theoretically and visually.

Weakness

The proposed solution for the long-tailed recognition problem is not novel enough. It would be better to propose a novel method by modifying the existing SAM method to be especially helpful for the tail classes. Simply borrowing the existing method, which is generally helpful, might not be enough for complete research. Also, empirically in Table 1,2,3, it is hard to see the SAM is especially helpful for the tail classes (especially helpful to alleviate the long-tailed distribution). 

The experiments also seem to be weak. For example, in Table1, the authors should also evaluate their method with other imbalance ratios. 

Also, they should compare with more recent state-of-the-art long-tailed recognition works [1,2,3,4]; the baselines shown in the paper are too weak. In order to claim the effectiveness of the proposed method, the authors should apply the proposed method to the state-of-the-art methods and see the improvements instead of only applying the method on simple baselines.

[1] Self Supervision to Distillation for Long-Tailed Visual Recognition, ICCV 2021
[2] Parametric Contrastive Learning, ICCV 2021
[3] Influence-Balanced Loss for Imbalanced Visual Classification, ICCV 2021
[4] Distilling Virtual Examples for Long-tailed Recognition, ICCV 2021

========== ------- Comments after the rebuttal ------========
I have read the other reviewer's comments and the author's rebuttal, which addresses most of my concerns. Therefore, I would like to raise my score.
questions:  
limitations: The authors addressed the limitations of the proposed method and the potential negative social impact of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In the real-world, many datasets are imbalanced, which can degrade the trained model’s performance. To tackle this problem, this paper analyzes the class-imbalance problem by examining the spectral density of Hessian of class-wise loss. From the observation that the tail class loss converges to a saddle point, this paper proposes to use sharpness aware minimization (SAM). This paper justifies the proposed method theoretically and empirically on multiple benchmark datasets.
strengths_and_weaknesses: Strengths
- This paper justifies the proposed method with theoretical justification and various analyses.
- The proposed method improves the baselines. 
- The paper is well written.

Weaknesses
- The comparison baselines and the prior literature on long-tailed learning is limited.

questions: Please refer to the weakness.
limitations: The limitations and potential negative societal impact are addressed,
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors find that for the tail class loss landscape, the solution converges to a saddle point and the network thus suffers from poor generalization on minority classes. This work uses Sharpness-Aware Minimization (SAM) to escape saddle points and enhance the generalization performance, which has been theoretically and empirically demonstrated. Experimental results show that combining SAM with SOTA techniques leads to significant gains in performance on minority classes.
strengths_and_weaknesses: Strength 
1 The idea of studying the effect of negative curvature for class-imbalanced learning using deep neural networks is novel and interesting, which may pave a new way for further research on imbalanced classification.
2  Experimental results empirically show that escaping saddle points with SAM leads to a notable in crease in overall accuracy primarily due to the major gain in the accuracy on the tail classes.

weakness
1 The Algorithm should be presented and described in detail.
2 The background of Sharpness-Aware Minimization (SAM) shoud be described in detail. 

questions: How the sharp maximal point ϵ and weight w are updated in this work?

What is the difference between  θ and w? 
limitations: 1 The Algorithm should be presented and described in detail, which is helpful for understanding the proposed method.
2 The background of Sharpness-Aware Minimization (SAM) shoud be described in detail.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this paper, the authors find that the key issue for imbalanced classification is that the training for minority classes can lead to convergence to saddle points of their loss landscape. This phenomenon cannot be avoided in the reweighting methods. Consequently, the network suffers from poor generalization on minority classes. To escape from the saddle points, they introduce Sharpness-Aware Minimization (SAM) with a high regularization factor $rho$ to enhance the generalization performance. They theoretically and empirically demonstrate that SAM with high $rho$  is able to escape saddle points faster than SGD and converge to better solutions.
strengths_and_weaknesses: Pros:
1. Analysis the problem of imbalanced classification from the persepective of optimaization peocedure is novel.
2. The conclusion that saddle points cannot be avoided in reweighting methods is interesting.
3. Sufficient theoretically and empirically analysis.
questions: 1. Can you explain why the negative eigenvalues appear on imbalanced datasets and disappear on balanced datasets? Do they appear because of the negative gradients from the majority classes? If so, please analyse the differences and compare with the equalization loss (Equalization loss for long-tailed object recognition, CVPR 2020).
2. Is SAM only applied on the loss of the minority classes? I think providing an overall algorithm is better.
limitations: Does the saddle points exist in other methods, such as data augmentation and sample-level reweighting methods?
review_title: null

Meta-review: metareview: The paper studies the problem of saddle point escape for class imbalanced datasets and mostly makes two contributions from my perspective:
1) Analysis of the spectral density of the Hessian for class-imbalanced datasets. This observation is novel as far as I know.
2) A short analysis of SAM demonstrating it escapes saddle points

While the first point seems novel and of interest, I have some limited reservations regarding the second contribution. Theoretically, the authors provide a theorem demonstrating that the CNC condition derived in Daneshmand et al. holds with a larger constant. It is however unclear whether this is the reason for the superior performance of SAM in unbalanced datasets. Saddle points are often not prevalent in the loss landscapes of modern neural networks. The paper does not directly show that better performance is linked to saddles. I would like to encourage the authors to more directly highlight the importance of the CNC condition used in the analysis.

Overall, the reviewers are still rather positive about the paper and despite its shortcomings, it has the potential to encourage more research in this field. I, therefore, recommend acceptance and invite the authors to add a discussion of the shortcomings that should be addressed in future work.

Finally, I note that there is some recent work analyzing the dynamics of gradient descent under class imbalance:
Characterizing the Effect of Class Imbalance on the Learning Dynamics, Francazi et al.
The findings do not seem to be directly related but it's probably worth checking.

confidence: Certain
Decision: Accept