Title: QuanDA: Quantile-Based Discriminant Analysis for High-Dimensional Imbalanced Classification
Venue: NeurIPS 2025
Accepted: yes
Keywords: High dimensional data

imbalanced classification

quantile regression
Abstract: Binary classification with imbalanced classes is a common and fundamental task, where standard machine learning methods often struggle to provide reliable predictive performance. Although numerous approaches have been proposed to address this issue, classification in low-sample-size and high-dimensional settings still remains particularly challenging. The abundance of noisy features in high-dimensional data limits the effectiveness of classical methods due to overfitting, and the minority class is even difficult to detect because of its severe underrepresentation with low sample size. To address this challenge, we introduce Quantile-based Discriminant Analysis (QuanDA), which builds upon a novel connection with quantile regression and naturally accounts for class imbalance through appropriately chosen quantile levels. We provide comprehensive theoretical analysis to validate QuanDA in ultra-high dimensional settings. Through extensive simulation studies and high-dimensional benchmark data analysis, we demonstrate that QuanDA overall outperforms existing classification methods for imbalanced data, including cost-sensitive large-margin classifiers, random forests, and SMOTE.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes QuanDA, a novel classification method to handles the challenges of low-sample-size and high-dimensional settings. The core innovation lies in establishing a theoretical connection with quantile regression, which naturally accounts for class imbalance through the choice of appropriate quantile levels. Theoretical analyses are provided to prove the effectiveness in ultra-high dimensions for binary classification. Extensive experiments on seven high-dimensional data demonstrate the superiority over previous methods like cost-sensitive SVM, random forest, SMOTE, particularly under high imbalance conditions.\n\nSTRENGTHS AND WEAKNESSES:\n[Strengths]\n1. Enough Novelty: The linkage between quantile regression and the weighted Bayes risk is insightful and well-articulated. The jittering step elegantly resolves the discontinuity issue in binary response variables, enabling stable quantile optimization. This theoretical contribution is significant for imbalanced learning.\n\n2. Excellent performance. This paper demonstrates excellent performance in seven high-dimensional data demonstrate compared with other robust counterparts. This clearly shows the effectiveness and advantages of the proposed method.\n\n3. Rich Experiments and Analysis. The authors provide extensive experiments and analysis for the proposed method. I appreciate this. The experimental analysis with various ablation studies allows a better understanding of each module and overall performance.\n\n4. Good writing and organization. This paper is well-written and organized. Each section has a clear motivation. It’s easy to follow the ideas.\n\n[Weaknesses]\n1. Lacking Efficiency Comparison: The authors should provide the efficiency comparison to show the balance between extended modules and computation costs.\n\n2. More Comprehensive Ablation: It’s better to show the generality of the proposed method with various values of n/p in the simulated data.\n\nQUESTIONS:\nI do have some questions, see those questions in the weakness part.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of binary classification under high-dimensional, low-sample-size settings with severe class imbalance. The author proposes a novel method, QuanDA, which leverages quantile regression to construct a classifier aligned with a weighted Bayes decision rule. By selecting an appropriate quantile level, the method naturally incorporates information about class imbalance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:  \nS1: Theoretical foundations are solid. The paper demonstrates the correspondence between Bayes classification rules and quantile functions, effectively reformulating the classification task as a quantile estimation problem.  \nS2: The proposed method outperforms conventional approaches on both simulated and multiple benchmark datasets.  \nS3: The authors convert discrete labels into continuous variables, enabling the use of quantile regression.  \n\nWeaknesses:   \nW1: The experimental comparisons are limited to classical methods, without including more recent approaches such as \"Robust High-Dimensional Classification From Few Positive Examples.\"  \nW2: While L1 regularization is applied to QuanDA, DSDA, and logistic regression, no feature selection is conducted for methods such as SMOTE and Random Forest, making the comparison less fair.  \nW3: In the simulation study, the signal is restricted to the first five features, with the rest being noise. The paper does not investigate performance under varying sparsity levels (e.g., 10 or 20 informative features).  \nW4: SMOTE is based on nearest neighbor synthesis, but under the high-dimensional and sparse setting, nearest neighbor methods become less meaningful. It would be more appropriate to compare with improved variants such as Borderline-SMOTE.   \nW5: Algorithm 1 indicates that QuanDA involves multiple nested loops, potentially leading to high training time. It is recommended to report and compare training time with other methods.\n\nQUESTIONS:\n1.How does your method compare with more recent approaches specifically designed for imbalanced classification under few-shot settings?  \n2.What would the performance be if feature selection were applied to Random Forest and SMOTE methods?  \n3.Given that QuanDA involves multiple nested loops, is the training process computationally intensive?  \n4.In your simulation studies, how does the performance of QuanDA compare to other methods under varying levels of sparsity?  \n5.Under your experimental setting, what is the performance of advanced sample synthesis techniques tailored for high-dimensional data, such as Borderline-SMOTE?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a quantile-based discriminant analysis (QuanDA) method designed for high-dimensional imbalanced classification. Specifically, QuanDA directly models the quantile-based decision boundary tailored to the minority class to address class imbalance, while employing the elastic net penalty to enforce sparsity in the regression weights to handle high dimensionality.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The overall presentation is clear, and the progression from Bayes risk to quantile regression is reasonable.\n- The paper is easy to follow and includes open-source code.\n- The authors provide theoretical analyses to support the capability of QuanDA in handling the high-dimensional setting.\nWeaknesses:\n- The method seems primarily limited to binary classification. Although the authors claim that it can be extended to multi-class classification via the one-vs-one strategy, the experimental evidence in Table S.12 is inadequate, as it relies solely on synthetic data and two simple approaches (not designed for imbalance).\n- The paper conducts experiments on 5 comparison methods, however, these methods are relatively old. More recent approaches, such as [1-4], should be considered.\n- The paper conducts experiments on 7 datasets, however, this is still insufficient for a comprehensive evaluation. More benchmarks should be included to thoroughly evaluate the proposed method, encompassing balanced and imbalanced, low- and high-dimensional, small- and large-scale datasets.\n- Although the paper includes many metrics such as AUC, F1, and G-mean, these are presented solely as statistical results. I suggest the authors to incorporate more visualizations, as they are crucial for enhancing reader comprehension and facilitating new insights.\n\n[1] He, Haibo, Yang Bai, Edwardo A. Garcia, and Shutao Li. \"ADASYN: Adaptive synthetic sampling approach for imbalanced learning.\" In 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), pp. 1322-1328. 2008.\n[2] Xu, Lei, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. \"Modeling tabular data using conditional gan.\" Advances in neural information processing systems 32 (2019).\n[3] Sun, Yi, Lijun Cai, Bo Liao, Wen Zhu, and Junlin Xu. \"A robust oversampling approach for class imbalance problem with small disjuncts.\" IEEE Transactions on Knowledge and Data Engineering 35, no. 6 (2022): 5550-5562.\n[4] Sun, Hao, Jianping Li, and Xiaoqian Zhu. \"A Novel Expandable Borderline Smote Oversampling Method for Class Imbalance Problem.\" IEEE Transactions on Knowledge and Data Engineering (2025).\n\nQUESTIONS:\nFor the rebuttal, the following could strengthen the paper:\n- Include a more comprehensive evaluation on multi-class classification tasks.\n- Expand Tables 2 and S.8 to incorporate additional real-world datasets and comparisons with more recent methods, e.g., [1-4].\n- Present ablation studies for QuanDA. A key question is whether combining imbalance approaches with dimensionality reduction techniques such as PCA can achieve improved results in the HDLSS setting.\nIf my questions and concerns are well addressed, I will consider raising my score.\n\nLIMITATIONS:\nThe authors have discussed the limitations in their paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nHi folks, this likely won't be a traditional review since I'm not a traditional reviewer. I'm a \"subject matter expert\" on another paper submission, and it looks like I have been asked to help review some other submissions. I'm an expert in data management, not machine learning, so please take my opinions and thoughts with a grain of salt. Despite not being an expert in this area, I did my best to read your paper closely and provide comments that I hope are useful, even if they are from a more general CS point of view rather than the ML experts you may be used to!\n\nSummary: The authors propose QuanDA, a discriminant analysis technique for HDLSS. QuanDA uses an intuitive connection betwewn quantiles and weighted Bayes risk: by adding a small amount of noise to class labels, the \"quantiles\" of previously-binary variables become well-defined, meaning that quantile regression can be used. Sparse quantile regression is selected to deal with the high number of dimensions.\n\nSTRENGTHS AND WEAKNESSES:\nI was pleasantly surprised at the readability of this paper, especially compared to the other works I reviewed. The introduction actually introduces the problem and covers related work in detail, clearly setting up the problem. I did not fully understand the intuition behind the proof in S3, but this might not be the author's fault (although I think the paper would benefit from a clear, bold-faced, two-line description of the result!).\n\nD1) I had never thought of quantiles this way! S2.1.2 is a very nice piece of intuition. I will restate it here so the authors can decide how much weight to give the rest of my review: if if the majority-class probabilty at some point X exceeds 1 - P(majority), then the quantile of the data (which is itself binary, for binary data) flips from 0 to 1. This is exactly the cost-sensitive Bayes threshold for imbalanced data from Lin et al. \n\nD2) It seems to me that the jittering is more than just about numeric stability. Without jittering, I do not really understand what the quantiles \"are\" -- the jiterring makes the quantiles well-defined, the fact that it makes quantile regression numerically stable is a side-effect of this property, I think.\n\nD3) In S3, I do not understand where C6 comes from or what the intuitive motivation for this condition is. It almost seems like the authors sorted the conditions from \"most intuitive\" to \"random thing we needed to make the proof work\" -- I'm not against any of these conditions, I would just like a better understanding of the intuition behind them.\n\nD4) I was surprised to see that the authors used \"5%\" as an experimental representation of \"high class skew.\" I was expecting something closer to 0.01% or lower. Not a problem, just a surprise. The authors might want to make their understanding of \"skew\" more concrete earlier in the work. \n\nD5) I expected (kernel) SVMs to be included in the experimental analysis, since that's probably the technique I would've reached for in a problem with so many dimensions (SVMs and their cost-sensitive variant were also mentioned by the authors). The underconstrained nature of the problem (dims >>> n) explains the poor performance of other approaches.\n\nD6) The \"conclusion and limitations\" section is more of a \"conclusion and future work\" section, as limitations are notably absent. It's probably worth a few sentences here about how the current QuanDA assumes linear seperability, etc. This is obvious to anyone who fully reads the paper, but explicitly flagging the discussion is helpful for poor grad students looking for works that match their problems!\n\nD7) I think the best part about this paper was drawing the connection between Bayes risk and quantiles of (jittered) binary data. This is an intuitive idea that seems obvious in retrospect, but I had never personally considered (the best ideas often are!). The paper's presentation made the idea clear and intuitive, and I can honestly say I learned something reading it.\n\nQUESTIONS:\nQ1) Could the authors do a better job of discussing the limitations (both practical and theoretical) behind their work?\n\nQ2) Could the authors provide some intuition for (1) the proof assumptions and (2) the resulting theorem in S3 of the paper?\n\nQ3) Could the authors explain their choice of competitors for the experimental analysis? The experimental analysis kind of feels like the authors thought \"well, it's a NeurIPS paper, it needs some experiments!\" and then threw something together. Making it a little more structured (e.g., going through a more robust set of competitors) and highlighting key insights (e.g., maybe a case study of a failing case or of a particularly good result) would take the experiments from \"technically there and complete\" to \"actually a useful part of the paper.\"\n\nLIMITATIONS:\nYes -- while limitations could be discussed more clearly, the content is all there.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-04"}