QuanDA: Quantile-Based Discriminant Analysis for High-Dimensional Imbalanced Classification
Authors: Qian Tang
Yuwen Gu
Boxiang Wang
Venue: NeurIPS 2025 poster ()
Keywords: High dimensional data
imbalanced classification
quantile regression
Primary area: general_machine_learning

Abstract: Binary classification with imbalanced classes is a common and fundamental task, where standard machine learning methods often struggle to provide reliable predictive performance. Although numerous approaches have been proposed to address this issue, classification in low-sample-size and high-dimensional settings still remains particularly challenging. The abundance of noisy features in high-dimensional data limits the effectiveness of classical methods due to overfitting, and the minority class is even difficult to detect because of its severe underrepresentation with low sample size. To address this challenge, we introduce Quantile-based Discriminant Analysis (QuanDA), which builds upon a novel connection with quantile regression and naturally accounts for class imbalance through appropriately chosen quantile levels. We provide comprehensive theoretical analysis to validate QuanDA in ultra-high dimensional settings. Through extensive simulation studies and high-dimensional benchmark data analysis, we demonstrate that QuanDA overall outperforms existing classification methods for imbalanced data, including cost-sensitive large-margin classifiers, random forests, and SMOTE.

QuanDA: Quantile-Based Diseriminant Analysis for High-Dimensional Imbalanced Classification
Qian Tang School of Statistics University of Minnesota Minneapolis, MN, 55455 tang1015@umn. edu
Yuwen Gu Department of Statistics University Connecticut Storrs, Connecticut, 06269 yuwen. geweenng@ucoon..o edu
Boxiang Wang"
Department of Statistics and Actuarial Science University of lowa Iowa City, IA, 52246
boxiang-wang www.aaanggwnnnuioi......mo edu
Abstract
Binary classification with imbalanced classes a common and fundamental task, where standard machine learning methods often struggle to provide reliable pre- dictive performance. Although numerous approaches have been proposed to ad- dress this issue, classification in low-sample-sin and high-dimensional settings still remains particularly challenging. The abundance of noisy features in high- dimensional data limits the effectiveness of classical methods due to overfitting, and the minority class even difficult detect because of severe underrepresen- tation with low sample size. To address this challenge, we introduce Quantile-based Discriminant Analysis (QuanDA), which builds upon novel connection with quan- tile regression and naturally accounts for class imbalance through appropriately chosen quantile levels. We provide comprehensive theoretical analysis to validate QuanDA in ultra-high dimensional settings. Through extensive simulation studies and high-dimensional benchmark data analysis, we demonstrate that QuanDA overall outperforms existing classification methods for imbalanced data, including cost-sensitive aarge-aaggin classifiers, random forests, and SMOTE.
Introduction
High-dimensioon binary classification is fundamental yet challenging machine learning task. particularly in problems where sample sizes are small and class distributions are heavily imbalanced. The situation commonly arises in many application fields, such as disease diagnosties (Krawezyk)
et al. 2016;) Bae and Yoon 2015): Azari et al., (2055). where data acquisition is costly due to the involvement of human and animal experiments in clinical studies (Evans and Iidstad 2001). As result, the number of features often far exceeds the number of data points, leading to the so- called aiigh-imeensioallllwwssss -ossppllssssl (HDLSS) problem (Hall et al. 2005| Aoshhma et al. 2018). Besides data scarcity, class imbalance further complicates the challenge: the positive class, such as disease occurrence, is typically much rarer than the negative one. Similar challenges are prevalent in many other applications such image detection (Kubat al. 1997), cybersecurity (Cieslak et al. 2006). fraud detection (Wei et al. 2013 Sanz et al.] (2144), text categorization (Wu et al. (014)). and
fault diagnostics (Wu et al. 2018 Zhu and Song 2010 Santos et al. 2018).
'The corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

When data are highly imbalanced, identifying the minority class becomes no easier than finding needle in a haystack. Standard machine learning methods often become ineffective, as they tend to bias heavily toward the majority class and struggle to capture decisive patterns in the minority class. This bias can lead to poor generalization and unreliable prediction, and the challenge is further exacerbated in the HDLSS setting, where the minority class is severely underrepresented with the limited sample size compared to the data dimension. To address this challenging issue, many methods have been proposed in the literature to adjust for the class imbalance. Generally speaking, those methods fall mainly into three categories: (1) data-level adjustment, (2) algorithm-level adjustment, and (3) a combination of the first two categories. Comprehensive reviews include, but are not limited to Weiss )004);; He and Garcia 2009); Fernández (2018); Feng et al. (021): Rezvani and Wang (2023).
Two straightforward methods falling within the category of data-level adjustment are oversampling and undersampling, where the imbalance is mitigated by randomly duplicating samples from the minority class or eliminating samples from the majority class (Paula et al.) (015). However, in the HDLSS setting, neither approach is suitable. Undersampling can yield excessively small data set, as too many majority class samples must be discarded to match the minority class size. On the other hand, oversampling often leads to model overfitting, as individual data points may be replicated too many times, thereby distorting the classification boundary (Devi et al. 2020). Rather than replicating the minority class, an alternative strategy is to generate synthetic data to decrease the risk of overfitting. One well-known example of this strategy is the so-called Synthetic Minority Over-sampling TEchnique (SMOTE, Chawla al. (202).. Many of its variants have also been developed in the literature, such as FSMOTE (Gosain and Sardana 2019). MSMOTE (Hu et al. 2009 SMOTE-ENN (Muntasir Nishat et al. 2022), and SMOTE- (Ramentol al. 2012). 2). among others. SMOTE can be further integrated with ensemble learning techniques, such as SMOTEBoost (Chawla et al.] (2003) and WSMOTE (Abedin al.) 2023). However, SMOTE has been shown to not perform well in the HDLSS setting due to a strong bias toward the minority class (Blagus and Lusa 2013).
For algorithm-leve adjustment, a widely adopted framework is cost-sensitive learning, which assigns higher cost for data points that misclassified in the minority class. This approach is commonly classifiers, such as the support vector machines (SVMs, Cortes and Vapnik,
employed in 1995 /aannk 1995). Cost-sensitive SVMs {Lin et al. 2002 ZZng and Zhang 2023) place a different weight on each data point in empirical hinge loss, and the resulting classifiers are known to be Fisher consistent in terms of cost-sensitive Bayes risk (Lin 2002 | 2004). In the literature, most studies on cost-sensitive large-margin classifiers focus on standard classifiers or kernel machines, for example, Zhang et al. (2016); Shiin et al. 2017): Fu et al. (2018), while their performance on the HDLSS setting may not be reliable due to the so-called data pilling issue (Marron et al. 2007 Wang and Zou 2018). Moreover. the introduction of varying weights also affects the efficiency of their optimization algorithms. In addition to surge-margin classifiers, ensemble learning. such as random forest and boosting, can also deal with imbalanced data through algorithm-Ievel adjustments (Chen et al. 2004] Khalilia et al. 201T: Galar et al.] 2011 [aagaa and Cengiz| (022). Adjustments for imbalancee classification at both the data and algorithm levels have been extended to the deep learning framework as well, a survey of which can be found in Johnson and Khoshgoftaar (2019). However, deep learning- based approaches typically require a sufficiently large sample size, which is not the case in the HDLSS setting. Consequently, deep learning is not applicable in this context.
Despite extensive research on imbalanced classification, a critical gap remains: few approaches perform well when the sample size is extremely small and the number of features is extremely large. In the HDLSS setting, overly complex models are prone to overfitting, whereas an underfit model may fail to handle class imbalance and high dimensionality simultancously. To this end, it is essential to develop a direct yet effective classification method that can handle these challenges simultancously without relying on too much data.
In this paper, we propose Quantile- based Discriminant Analysis (QuanDA) for imbalanced classi- fication in the HDLSS setting. This method is based on quantile regression, which is widely used in statistics and econometrics to estimate the conditional quantiles of a continuous response given a set of features, providing a comprehensive view of the underlying distribution. The theoretical properties of quantile regression have been extensively studied (Belloni and Chernozhukov 201T Wang et al. 2012: Zheng et al., (2015) in the context of high- sigh--imensional regression. Although it may seem counterintuitive to apply quantile regression directly to a classification problem, we show
2

---

0.90
0.80
AUC
0.70
0.60
QuanDA dsda
logistic + RF SMOTE
20
5000
10000
50000
p
Figure 1: AUC scores comparing QuanDA with direct sparse discriminant analysis (dsda), logistic regression, random forest (RF), and SMOTE on simulated data with Σ = ARs, 400 and class imbalance To = 0.9. See details on data generation Section 4 The x-axis plotted with evenly spaced positions for clarity, and we label it with the original values of to preserve interpretability.
that the population minimizer of the cost-sensitive Bayes risk can be obtained from a conditional quantile of the binary class label. Building upon this connection, we propose QuanDA which directly fits quantile regression to the label for imbalanced classification. This idea echoes the established connection between least-squares regression and linear discriminant analysis (Hastie et al. 2009) Mai et al.] 2012), which is studied mainly for balanced classification. A pivotal component that enables QuanDA to perform effectively in the HDLSS setting is the jittering step, which introduces random noise to discrete class labels. This perturbation helps stabilize the algorithm and enhances its robustness when the sample size is limited. While a conceptually related idea was explored by Papandreou and Yuille 1 underlying motivations and implementations differ substantially. In their work, stochastic perturbations were applied to continuous energy potentials to induce a discrete label random field derived from an energy-basee formulation. We also impose sparse penalty, such as the lasso (Tibshirani] 1996), to automatically discard irrelevant features.
To give quick demonstration of our proposed method QuanDA in the HDLSS setting, Figure 1 presents AUC scores that compare QuanDA with several popular classifiers. When the number of features p is small, all the methods achieve satisfactory AUC scores. However, as p increases,
QuanDA, dsda, and logistic regression, which are specifically designed for high-dimensional data, exhibit slightly reduced performance but remain relatively stable. In contrast, RF and SMOTE struggle with high-dimensional data, and their performance deteriorates significantly. Overall, QuanDA consistently delivers the highest AUC scores and robustness to increasing dimensionality.
There are several notable features of QuanDA. First, by fitting quantile regression, we can select an appropriate quantile level to account for class imbalance, which is intuitive and straightforward. Second, QuanDA is flexible enough to incorporate various sparse penalties, for example, the group lasso (Yuan and Lin 2006) fused lasso (Tibshirani et al., 2005). to handle grouped or spatially structured features. Third, by formulating the problem into a standard quantile regression, QuanDA can be directly fitted using off-the-sheef algorithms for sparse quantile regression, such hdqr (Tang ct al. 2024) and fhdqr (Gu et al. 2018), which eliminates the need to design a specialized solver. We demonstrate through extensive simulations and benchmark data analyzes that QuanDA is highly competitive with the representative classifiers for imbalanced classification in the HDLSS setting, including cost-sensitive large margin classifiers, random forests, and SMOTE.
The remainder of this paper is organized as follows. Section 2 begins with the connection between the quantile regression and imbalanced classification, which followed by an introduction to the QuanDDA algorithm. Section 3 provides the theoretical analysis. Section 4 presents simulation studies and real-data applications. All technical proofs provided in the appendix.
3

---

2 Methodology
2.1 Background and motivation
We consider binary classification problem with training data, {xi: Vo where each x; R2 and the binary class label yi E {0, 1]. While {-1,1} } encoding is common in large-margin classifiers for geometrie interpretation, opt for the {0, }} encoding for ease of handling class probabilities. The two coding approaches are essentially equivalent after an appropriate transformation.
Let π P(Y 1) and To 1 loss of generality, we assume म < Ty. which means that class forms the minority class, whereas class O represents the majority class. Given the training data, our goal build decision function that assigns a new test data point Xnew to class ((Xnew) = 1 or class 0 if o (Xee)) O.
P(Y 0) denote the marginal class probabilities. Without
2.1.1 Review of Bayes risk
The performance of a classifier is fundamentally dictated by the Bayes risk, the lowest achieveable classification error given the true underlying distribution. Let I(-) be the indicator function. The Bayes risk is defined as the expectation of the 0-1 1 loss on the population level (Lin 2002):
R(Ø) =Exy [I(Y 0, ((X) = 1) I(Y = 1, (,) 0)],
whose minimizer gives the Bayes rule, say the theoretically optimal classifier under the 0-1 loss,
X arg min R(Ø) I(n)(X) 1/2).
where 77() = P(Y = 1 is the conditional probability for class 1.
To illustrate why the Bayes risk is unsuitable for direct application for imbalanced classification, note that R()) can equivalently written as
Tof (X)
I(((X) =00] T(x(X) = Mug (X) + 1g
Tig Tig (X)
Ex
Aug
(X)
T19 (X)
where (X) and g (X) are the conditional density of X given = 1 and = 0, respectively. easily seen that the Bayes risk tends to be biased toward the majority class when mi < no.
To address class imbalance, Lin (002) introduce weighted Bayes risk:
wp
Wi
I(Y = 1, 1 (() = 0]]
R( = Exy
T(Y = 0, ((X) = =) wo +
WD + W1
With these weights, the corresponding Bayes rule is given by
xx(X) =I =I(((110 (x(x)
Wi
(I)
wl w1
and the optimal Bayes risk is
R(o*) = Ex
wo
(-)) nxX)
w0
+
W1
n (X)II
(xX)
Wo
WD + 11
We + w1
we + wi
wo + w1
In the above framework, the most common choice weights is wo ग. and wi To. Given the class imbalanced, say < a higher penalty is thereby imposed on misclassifying the minority data. The framework can further extended account for unequal costs misclassification between the two classes. In particular, the weights can be adjusted to wa Ti and = ToCo, where Cy and Cj represent the costs of false positive and false negative, respectively. In practice, e₁ is set to higher value than cp. Incorporating such costs into the weighting scheme leads to the mean- wathhi- cooninggrupeceeee group-error criterion (Qiao and Liu| 2009) Qiao et al.) 2010). Accordingly, the Bayes rule becomes
6' 6'(X)
(1(x))
Wo
T1
(3)
wi + W1
ToCa + m

---

2.1.2 From quantile to Bayes risk
We now establish an intuitive connection between the Bayes risk and quantile functions. With E (0, 1), the quantile function defined as
(r = inf(y: Fr)x(1) 2 T)
where x x denotes the conditional distribution function of Y given X. Since Y is either 0 or 1, (r(r is also binary. Moreover, it follows that Qr(r X) if and only if P(Y 1[X) > 1-7, that is,
Qy [(() 1-
Consequently, setting r wt (wpp ww) nu- Qy X exactly coincides with the Bayes rule given in equation ①T. Likewise, the Bayes rule in equation corresponds to using T = w/ /0r//wu + w1) = (TTCI) ( TTC)).
X)
To estimate the quantile function, it is well known that (T x) = arg min E (Y Q(X))XX x],
where p=(u) u(T I(u < 0)) is the check or pinball loss. Note that the minimum of the objective function is E [p-(Y - Q0 = x] p-(-I(n(x) 1 -1) - mxxx) + (T(T(((x) s - ())n(x))
( (1- (x)),
if if > 1-7, if (x) S -
[T)(x).
Therefore, setting w /wwo + w1), we see that the minimum of the population risk under the check loss equivalent to the optimal Bayes risk given in equation (2).
2.2 bhasd discriminant analysis
We now introduce QuanDA based on the connection between quantile regression and Bayes risk. Because quantile regression is designed for continuous responses, directly applying it to binary class labels may lead to numerical instability. To address this, we propose to apply quantile regression on jittered class labels, and we shall show intimate connections to the Bayes risk framework.
Specifically, we craft jittered response = Y +U for uniform random variable U on [0, 1) that is independent of Y. It can be shown that Z is "almost" continuous, and Qy X) where [.] is the ceiling function that gives the smallest integer no less than its input. Importantly, the jittering does not actually affect the classification decision. To see this, for a given we define (T), which gives rise to
T P(Y + < 2 X) P(Y (U < x|X) + P(Y P( +1 +1$ 2/x)
(4)
- ( S 2 X +
+ +1
When (X( < T, we know < 1, because otherwise the right-hand side of equation 44 equals
(X)) + 17 (X)(22 1) > a. Therefore, taking 2 S 1, we see
Qy+u
1 7 X)
Hence 4u < 1 is sufficient condition to give Qy (T =
Likewise, when (X) > 1 1-T, = > 1; otherwise, the right-hand side of equation 4 equals - 7)(x), which is less than a. Thus, with > have
Qr+u (T x = == a -1 (1-m(x)) > 1,
()X)
which shows that (Tl > 1 is sufficient to give Qy = 1.
Consequently, we propose QuanDA by fitting quantile regression on the jitted response Z. To apply QuanDA to the HDLSS setting, we consider linear quantile regression model Qz Qz(r = x) a (T) + x (T) and adopt a sparse assumption on 3 (T), where
(a" (T), (T) arg min EE Epp (Z X TT3).
(5)
a
5

---

Algorithm Quantile-based Discriminant Analysis
Input: Training data: (xi, M/ )
#1.
for r = 1, 1.2...: 10 do Generate U(r) Uniform(0, 1).
Compute 2 Y Ulr).
for t = T - 0.05, T - 0.04. 0.04, 0.05 do
For each \₁ compute (t, (t, \1)) from
min 0.3
-a- x, + Ar113111.
pr (z, - -
Perform five-fold cross-validation to determine the optimal (1(t)(r) based on the AUC scores. end for end for for t = T- 0.05, r 7 0 0.05 do
0.04,
Compute @(t) 10 Σ (t, ()) and ((t) 1 E 3
3r (t. (7())1).
Calculate the AUC scores based on (a((),((()) to select the best t*.
end for
and the classification rule of QuanDA ((X) ( (x) > or o(X) otherwise.
In the sample version, we fit QuanDA using the training data ((x;, v.)] For challenging imbal- anced HDLSS classification, traditional dimension-reduction techniques such as principal component analysis (PCA) are often inadequate. Because PCA is unsupervised and does not incorporate class labels during dimensionality reduction, may result in substantial information loss, particularly for the minority class. A more effective strategy, as suggested by Mai et al. (002)) is to employ supervised methods that impose sparse penalties to perform feature selection directly within the classification framework. Motivated by this idea, we introduce an & fi-penalized formulation to achieve sparsity the classifier:
1 Σ 1=1
arg min a
Pr (z. - a- x 3) + 1111
xx,
where ||3]|1 is the l norm of B. The quantile level T is determined by wh/ w1/(w0 + wi) according to the weighted Bayes risk. The class proportions π and To be estimated by the sample proportions D [1 = 1)/n and Σi= 1(y: = 0)/n, respectively. For a new test data point Xnew. if â + x 3 > 1 or ((Xaew) otherwise.
the classification rule d(x,ew)
In our implementation of QuanDA, we use the elastic net penalty (Zou and Hastie 2005). A A1118111 Aa |811 instead to stabilize the algorithm.
Remark the implementation phase, to optimally tune the parameter т, we first determine the proportion of the minority class, wo. in the training data set. Next, we define a sequence ranging from To - 0.05 to To + 0.05, with increments of 0.01 between consecutive points. The optimal 7 value is then selected using five-fold cross-validation. Details are summarized in Algorithm
Remark 2. Another line of research called quantitative classifiers (Hennig and Viroli 2016 Pritchard
and Liu 2020 Berrettini et al.) 2024) is conceptually related but fundamentally different from QuanDA. These methods are distance-based classifiers, which assign class labels by computing a
distance between each data point and each class, and extend the idea of median- classifiers (Hall)
jet al.) 2009) by using quantiles of these distances to make predictions. Although QuanDA shares term "quantile-based" classification, its quantile regression perspective differs from these distance- based frameworks. the appendix, we shall compare QuanDA with quantileDA, representative quantile-based classifier, and show that QuanDA consistently outperforms qua

---

3 Theoretical Studies
By construction, the distribution function of Z Y U is continuous, but not smooth. In fact, does not have continuous derivatives only at {0. 2). The standard theory for quantile estimator would become problematic when the quantile turns out to one. This can be resolved by assuming that the set of x for which Qz(rlx) 1 has measure zero. This is feasible when there exists at least one continuously distributed covariate and that the conditional quantiles @z(r|x) are measurable functions of that covariate. We make the following assumptions to show the estimation consistency of the quantile regression for the high-dimensional model.
(C1) Y is binary random variable supported on {0,1) and X is random vector in R". The mean function of y given X, n)(x) = TE X = x), is strictly between and for almost every realization X ==
(C2) The exists at least one continuously distributed covariate in X.
(C3) Make Z Y + U, where U ~ Unif(0, 1) is independent of Y and X. The following restriction on the quantile process of Z given X x holds:
Qz(r|x) a + x 3 (T) for E (0,
where a² a²(r) (T) E IR and (T) E IP. Furthermore, if (iss) (T) denotes the components of 3 (T) corresponding to the continuous covariates in X, then X (T) o 0.
(C4) For a.e. x, the density f=(x(-Ix) e - (T) 3 given X = satisfies: (1) fel (u(x) is continuously differeeniablee almost everywhere, and fape (u|xx < f and f/ (u/x) - f for
a.e. in the support of E,: (2) fa + u/x) 2 for all in a small neighborhood of zero
(C5) Let A = Eje f1, .:. (T() + 0} and = card( ().. The covariates X satisfy
[ X
A
km v) =
inf
+ 82 > 0,
(5. A)CC... (8, A)70 AUAIA m)
A E RP, A || S u|| A I + = for some v > 0, A(A, m) C A' is the support of the m largest in absolute value components of A A for integer m 2 0. When = O, we take A(A, m) 0.
where Cu. {ō,
(C6) The covariates X satisfy
3 88/2
B(5 +
X T A 2
3/2
g 8 f (6. ()EC). (6. A)=0
inf
0.
E3
X
A
Assumption (CI) is standard and ensures that classification is feasible. Assumptions (C2)-(C3) ensure that (z(r(x) has zero probability of taking integer values. Assumption (C4) is standard for quantile regression and works for more general U than a uniform. Indeed, it is also possible to take U to have a beta distribution on [0, 1). Assumption (C5) serves as the sparse identifiability condition or restricted eigenvalue condition, which is commonly imposed in the nigh-dimensional statistical literature (Candes and Tao 2007]] Bickel et al. 2009). The sparsity nonlinearity coefficient q in Assumption (C6) controls the quality of minorization of the quantile regression empirical loss by a quardratic function over sparse neighborhoods of the true parameter. It is often assumed in the nigh-dimensional quantile regression (Belloni and Chernozhukov 2011). p(X), where
Theorem 3.1. Under conditions (C1)-(C6), with probability at least
p(A) 2exp
(oxp(-1x))
nd? -_x²)
2pexp
nx?
exp
16MM
s(1 + log p)
2M
ko(3, 1)
the lasso estimator (ār,, (2) of the quantile regression satisfies
8
2Mo
+ log
||ā - a²112 <
16
(2/8 + 11 +>V
/ Km (3,
Ko (o 1)
Ko (3, 1)
7

---

and
8
18s
2
2Mo
+ log (2V++1)+2 No(3, 1
-3x-33112
+
16
[V Km (3,
m
m
Ko (3 1)
provided that the growth condition
2Mo
logp (2/8+1)+ V k3(3, S 51112
32
Ko V
holds, where Mo max S E[X]]
By Theorem 3.1 one can typically choose the parameter À Cylog for the quantile lasso estimator, where > v2Mo is some constant. For example, one can set 2 M Note that given the design X, Mo can readily be obtained. Therefore, in principle, the parameter À in the quantile lasso regression is tuning free. This in similar spirit to square-root lasso Belloni al. (201T). With such choice of X, we can see that p()) o(1) as n, + which leads to
(1
logg
13x - 1 =
V Kig
provided -1 V lo (nro) ((1 and ko logp) -1 o(), by taking m s. When Ko and Ks are both positive constants, the quantile lasso estimator achieves the near-optimal rate /s /slog p/n, which implies that can be of exponential order of n, i... logp O(n7) for some 0 < 7 < provided logp = o(n).
Numerical Studies
The goal is demonstrate that QuanDA consistently outperforms all competing methods, including logistic regression (implemented in the R package glmnet (Friedman et al. 2010b), the direct sparse discriminant analysis (dsda) (implemented in the R package dsda (Mai al. (0122), random forest (RF) (implemented in the R package randomForees orest (Liaw et al. 2002)) and SMOTE (implemented in the package smotef amily (Siriseriwan, 2019)). In particular,
show that the performance of
QuanDA remains robust even in highly ambelanced scenarios, where other classifiers collapse.
While many other methods have been developed for imbalanced classification, they not tailored to the HDLSS setting and tend to collapse as illustrated in Figure [ Hence, we focus our comparison only representative set of methods that are either commonly used or designed for HDLSS data.
4.1 Simulations
In the simulations, the dimension p is set to 10,000, with a sample size n of 400 and randomly generated class labels. The data set is highly unbalanced, and the majority class (TT) comprises 85%, 90%, or 95% of the total, leaving the minority class (π²z ) at 15%, 10%, or 5%. The simulation data generate following the methodology described by Wann al. (2006). The positive class follows normal distribution with mean vector μ4 and covariance matrix E. The mean vector μ is set to 0.7 for the first five features and 0 for the others. The covariance matrix Σ is defined as:
Σ = (
D5x
O5x(p-5)
0 (pp-))x
I ((o-0)x(p-5)
where 25 takes the form of an autoregressive structure (AR defined as (pli-5l), or a compound symmetric structure (CS,) expressed (p + (1 p)ll(i = j)), for p E {0.2, 0.5, 0.73. The negative class has the same distribution except for the mean vector f4_ -4
We randomly split the simulation data into training set of size 200 and test set of size 200. In QuanDA method, we tuned the parameter by five-fold cross-validation and used the optimal from the candidate list given in Algorithm] To address imbalanced classification, we perform weighted logistic regression and weighted random forest, where the weights are determined based on the class proportions. Specifically, the weight for the majority class is set to 1/To and for the minority class,

---

Table I: The AUC scores of QuanDA for different combinations of X and 22 based on simulated data with = AR5, n = 400, and p 10,000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses.
To
A
A2
AUC
X2
AUC
0.85
0.1
0.1
0.873 (0.041)
0.1
0.01
0.870 (0.042)
0.01
0.1
0.798 (0.048)
0.01
0.01
0.805 (0.048)
0.9
0.1
0.1
0.843 (0.087)
0.1
0.01
0.834 (0.098)
0.01
0.1
0.750 (0.065)
0.01
0.01
0.758 (0.070)
0.95
0.1
0.1
0,506 (0.043)
0.1
0.01
0.506 (0.043)
0.01
0.1
0.682 (0.118)
0.01
0.01
0.674 (0.111)
Table 2: The comparison of AUC scores using simulated data with n 400 and p 10000. These scores are averaged over 50 independent runs with standard errors given in parentheses. More comparison results based on F1, G-means, and PRAUC are shown in the supplemental file.
QuanDA
dsda
logistic
RF
SMOTE
0.85
AR
0.923 (0.032)
0.922 (0.032)
0.915 (0.036)
0.786 (0.057)
0.770 (0.068)
AR;
0.872 (0.042)
0.861 (0.047)
0.842 (0.069)
0.763 (0.055)
0.767 (0.051)
ARG
0.849 (0.045)
0.840 (0.048)
0.821 (0.069)
0.749 (0.056)
0.746 (0.063)
CS2
0.947 (0.028)
0.949 (0.025)
0.945 (0.029)
0.786 (0.061)
0.791 (0.056)
CS3
0.907 (0.037)
0.907 (0.036)
0.893 (0.043)
0.770 (0.051)
0.774 (0.050)
CS7
0.874 (0.042)
0.868 (0.046)
0.846 (0.070)
0.757 (0.056)
0.756 (0.057)
Ty = 0.9
AR₂
0.913 (0.041)
0.908 (0.046)
0.890 (0.075)
0.723 (0.075)
0.730 (0.087)
AR;
0.855 (0.059)
0.852 (0.061)
0.811 (0.122)
0.718 (0.075)
0.726 (0.069)
ART
0.830 (0.065)
0.821 (0.082)
0.778 (0.
0.711 (0.080)
0.715 (0.087)
CS2
0.938 (0.037)
0.938 (0.038)
0.923 (0.073)
0.743 (0.080)
0.746 (0.071)
CS3
0.894 (0.048)
0.890 (0.051)
0.869 (0.078)
0.730 (0.068)
0.728 (0.073)
CS7
0,856 (0.061)
0.854 (0.062)
0.810 (0.122)
0.719 (0.074)
0.730 (0.067)
TO
0.95
AR
0.827 (0.115)
0.813 (0.143)
0.714 (0. 175)
0.661 (0.102)
0.675 107)
AR,
0.770 (0.116)
0.738 (0.148)
0.660 (0.169)
0.658 (0.095)
0.648 (0.098)
ART
0.740 (0.118)
0.715 (0.150)
0.645 (0.155)
0.649 (0.092)
0.652 (0.1
CS2
0.850 (0.118)
0.850 (0.132)
0.750 (0.184)
0.657 (0.098)
0.662 (0.101)
CS;
0.803 (0.121)
0.791 (0.149)
0.695 (0.170)
0.659 (0.097)
0,646 (0.1
CS7
0.772 (0.109)
0.744 (0.146)
0.668 (0.160)
0.645 (0.098)
0.653 (0.101)
set to 1/TI. For both logistic regression and dsda, we also employ five-fold cross-validation to select the optimal parameter '₁ given that A2 0.01.
Tables 2 summarizes the simulation results, averaged from 50 independent runs. We observe that method consistently achieves the highest AUC scores in the all the simulation settings. Moreover, the advantages of our approach become increasingly evident as the imbalance ratio grows. The empirical performance confirms the effectiveness of QuanDA in addressing imbalanced classification.
We then conduct a sensitivity analysis of the hyperparameters in our algorithm. QuanDA involves two regularization parameters Yr and A2. We begin by evaluating the AUC scores of QuanDA under various combinations of λι and A2, using simulated data generated with an AR, covariance structure, n = 400, and p = 10,000, To assess performance under different levels class imbalance, we consider To E {0.85, 0.9, 0.95). Table I show that, for fixed value of Ay. changes in x have little impact on QuanDA's performance. In contrast, varying x while holding x2 fixed has a more noticeable effect. Based our numerical experiments, cross-validation typically provides reliable choice for tuning A₁
Additional simulation results are provided in the supplementary materials, including comparisons of QuanDA with other widely used methods based on PRAUC, Fl score, and G-mean; see Tables S.I.s.

---

Table 3: The comparison of AUC scores on benchmark HDLSS data. The method achieving the
highest AUC for each data set is italicized.
QuanDA
dsda
logistic
RF
SMOTE
breast (42,233)))
0.949 (0.066)
0.926 (0.116)
0.944 (0.088)
0.891 (0.101)
0.895 (0.096)
leuk
(72, (euk
7128)
0.993 (0.016)
0.987 (0.044)
0.991 (0.028)
0.997 (0.007)
0.996 (0.009)
LSVT ( (126,309)
0.909 (0.047)
0.901 (0.054)
0.898 (0.052)
0.884 (0.060)
0.884 (0.058)
ovarian
1.000 (0,000)
1.000 (0.002)
1.000 (0.002)
1.000 (0.001)
1.000 (0.001)
prostate 6033)
0.969 (0.029)
0.965 (0.025)
0.963 (0.027)
0.942 (0.047)
0.940 (0.049)
4.2 Benchmark Data Applications
In this section, we demonstrate the performance of QuanDA using seven benchmark high-dimensional data (Mai and Zou| 2013, Sorace and Zhan
20033
Graham et al., 2010 Alon et al. [999] Golub
al. 1999 Singh et al. 2002 I'sanas et al.
2013). All the benchmark data are available at the UCI
Machine Learning Repository (Kelly et al. ranging from 309 to 22,283, Each data set is partitioned into two parts: 70% is used for training and the remaining 30% for testing. The model fitting and parameter tuning are performed on the training set, after which the classification accuracy of the model is assessed on the test set.
2023). Those data sets have varying dimensionality,
Table reports the average AUC scores from 50 independent repetitions. It shows that our method QuanDA overall outperforms all the other four methods both metrics in those benchmark data examples.
5 Conclusion and Limitations
In this work, we have developed Quantile-based Discriminant Analysis (QuanDA), a method specifi- cally designed to address imbalanced classification problems in high-dimensional, low-sample-size (HDLSS) settings. Extensive numerical studies demonstrate that QuanDA overall outperforms widely used imbalanced classification solvers, including cost-sensitive large-margin classifiers, random forests, and SMOTE. This shows that QuanDA is competitive in the challenging HDLSS scenarios.
Although this work focuses on binary classification, QuanDA can be naturally extended to multi-class settings. One straightforward approach the one-vs-one method (Friedman 1996; Hastic and Tibshi rani 1998), which decomposes K-class problem into K(K 1)/2 pairwise binary classification tasks. A binary classifier applied to each pair, and predictions are aggregated using majority voting. Another potential direction involves adapting the multiclass sparse discriminant analysis (msda) framework proposed by Mai and Zou (2015). Some preliminary results are presented Section S22 in the supplementary material, while comprehensive study including rigorous theoretical analysis is left for future work.
The strong empirical performance of QuanDA results in part from its simple structure, which makes it particularly suited for the HDLSS setting. Extending QuanDA to kernel learning and deep learning frameworks would be some promising future work for handling unstructured data.
In addition, key contribution of this work is novel connection between quantile regression and imbalanced classification. This connection enables the direct application of extensive quantile regression variants imbalanced classification problems. For example, Qiao al. (2023) proposed transfer learning to leverage external information for fitting quantile regression. The same framework can be directly integrated into our Algorithm I to transfer information to improve classification
accuracy. Meta-learning for quantile regression (Fakoor el al. (2023) also shows promise for extending meta-learning techniques to imbalanced classification through quantile regression.
Acknowledgment
We would like to thank four anonymous reviewers and AC of NeurlPS 2025 for their constructive comments. Tang's research is supported by IRSA Faragher Distinguished Postdoctoral Fellowship. Wang's 's research is supported by National Institute Health grant R01GM163244-01.
10

---

References
M. Abedin, C. Guotai, Hajek, and T. Zhang. Combining weighted SMOTE with ensemble learning for the imballncnb prediction of small business credit risk. Complex & Intelligent Systems, -5)3355933 2023.
U. Alon, N. Barkai, A. Notterman, K. Gish, S. Ybarra, D. Mack, and A. J. Levine. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences, 96(12):6745-6750 1999.
M. Aoshima, D. Shen, H. Shen, K. Yata, Y.-H. Zhou, and J. S. Marron. A survey of high dimension low sample size asymptotics. Australian & New Zeuland Journal of Statistics, 60(1))4-19, 2018.
A. Azari, V. Janeja, and S. Levin. Imbalanced learning to predict long stay emergency department patients. In 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 807-814. IEEE, 2015.
S.-H. Bac and K.-. Yoon. Polyp detection via imbalanced learning and discriminative feature learning. IEEE Transactions on Medical Imaging, 34(1 1):2379-22333 22933, 2015.
A. Belloni and V. Chernozhukov. f fe-penali quantile regression in high-dimensional sparse models. The Annals of Statistics, 39-1)-2-- 2011.
A. Belloni, V. Chemozhukov. and L. Wang. Square-root lasso: pivotal recovery of sparse signals via conic programming. Biometrika, 98(4):791-806, 2011.
M. Berrettini, C. Hennig, and C. Viroli. The quantile-based classifier with variable-wise parameters. arXiv preprint arXiv:2404.1 /3589, 2024.
P. J. Bickel, Y. Ritov, and A. B. Tsybakov. Simultaneous analysis of lasso and Dantzig selector. The Annals of Statistics, 37(4):1705- 17322
2009. 2009.
R. Blagus and L. Lusa. SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics, 14:1-16, 2013.
P. Bühlmann and S. van de Geer. Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, 2011.
E. Candes and Tao. The Dantzig selector: Statistical estimation when p is much larger than n. The Annals of Statistics, 25(6))2313-2351, 2007.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. SMOTE: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321-357, 2002.
N. V. Chawla, A. Lazarevic, L. O. Hall, and W. Bowyer. SMOTEBoost: Improving prediction of the minority class in boosting. In Proceedings of the Principles of Knowledge Discovery in Databases, PKDD-2003, Cavtat-Dubrovnik Croatia, pages 107-119. Springer, 2003.
C. Chen, A. Liaw, and L. Breiman. Using random forest learn imbalanced data. Technical Report 666, University of California, Berkeley, 2004.
D. Cieslak, N. Chawla, and A. Striegel. Combating imbalance in network intrusion datasets. In 2006
IEEE International Conference on Granular Computing, pages 732-737, 2006.
C. Cortes and V. Vapnik. Support-vector networks. Machine Learning, 20:273-297 1995.
D. Devi, S. K. Biswas, and B. Purkayastha. A review solution class imbalance problem: Undersampling approaches. In 2020 International Conference on Computational Performance Evaluation (ComPE), pages 626 6.1. IEEE, 2020.
C. J. Evans and S. Ildstad. Small Clinical Trials: Issues and Challenges. National Academies Press (US), 2001.
R. Fakoor, Kim, Mueller, A. Smola, and R. Tibshirani. Flexible model aggregation for quantile regression. Journal of Machine Learning Research, 24(162):1-455 2023.

---

Y. Feng, M. Zhou, and X. Tong, Imbalanced classification: A paradigm-based review. Statistical Analysis and Data Mining: The ASA Data Science Journal, 14(5):383-406 2021.
A. Fernández. Learning From Imbalanced Data Sets. Springer, 2018.
J. H. Friedman. Another approach to polychotomous classification. Technical Report, Statistics Department, Stanford University, 1996.
H. Friedman, Hastic, and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1-22, 2010.
S. Fu, S. Zhang, and Y. Liu. Adaptively weighted Sarge-margin angle-based based classifiers. Journal of multivariate analysis, 166:282-299, 2018.
M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera. A review ensembles for the class imbalance problem: bagging.. boosting- and hybrid approaches. IEEE Transactions Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4)):63-484, 2011.
T. R. Golub, D. K. Slonim, Tamayo, C. Huard, M. Gaasenbeek, J. P. Mesirov, H. Coller, M. L.. Loh, J. R. Downing, M. A. Caligiuri, et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring, Science, 286(5439):531-537 1999.
A. Gosain and S. Sardana. Farthest SMOTE: a modified SMOTE approach. In Computational Intelligence in Data Mining: Proceedings of the International Conference CIDM 2017, pages 309 309-320.80 Springer, 2019.
K. Graham, A. de Las Morenas, A. Tripathi, C. King, M. Kavanah, J. Mendez, M. Stone, J. Slama, M. Miller, G. Antoine, et Gene expression in histologically normal epithelium from breast cancer patients and from cancer- free prophylactic mastectomy patients shares a similar profile. British Journal of Cancer, 108(8):124--129, 2010.
M. Grandini, E. Bagli, and G. Visani. Metrics for multi-class classification: An overview. arXiv preprint XXXx::000.0055666 2020.
Y. Gu, Fan, L. Kong. S. Ma, and H. Zou. ADMM for high-dimensional sparse penalized quantile regression. Technometries, 60(3):319-333 2018.
P. Hall, J. S. Marron, and A. Neeman. Geometric representation of high dimension, low sample size data. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(3):427-444 2005.
Hall, D. Titterington, and J.-H. Xue. Median-based classifiers for high-dimensional data. Journal of the American Statistical Association, 104(488):1597-1608 2009.
T. Hastic and R. Tibshirani. Classification by pairwise coupling. The Annals of Statistics, 26(2): 451-471, 1998.
T. Hastie, R. Tibshirani, J. H. Friedman, and J. H. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2009.
H. He and A. Garcia. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 2109):1233-1288 2009.
C. Hennig and C. Viroli. Quantile-based classifiers. Biometrika, 103(2):435-446 2016.
Hu, Y. Liang. Ma, and He. MSMOTE: Improving classification performance when training data is imbalanced. In 2009 Second International Workshop on Computer Science and Engineering, volume 2, pages 13-17. IEEE, 2009.
J. M. Johnson and T. M. Khoshgoftaar. Survey on deep learning with class imbalance. Journal of Big Data, 6(1):1-54, 2019.
M. Kelly, R. Longjohn, and K. Nottingham. The UCI machine learning repository. School of Information and Computer Sciences, University of California, Irvine, 2023.
12

---

M. Khalilia, S. Chakraborty, and M. Popescu. Predicting disease risks from highly imbalanced data using random forest. BMC Medical Informatics and Decision Making, 11:1-13, 2011.
K. Knight. Limiting distributions for L1 regression estimators under general conditions. The Annals of Statistics, (6(2):755-777) 1998.
Krawezyk, M. Galar, Jeleñ, and Herrera. Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy. Applied Soft Computing, 38:714-722 726777476,62 2016.
M. Kubat, S. Matwin, et al. Addressing the curse of imbalanced training sets: one-sided selection. In Proceedings of the 14th International Conference on Machine Learning, volume 97, pages 179-186, 1997.
M. Ledoux and M. Talagrand. Probability Banach Spaces, Isoperimetry and Processes. Springer- Verlag Berlin Heidelberg, 1991.
A. Liaw, M. Wiener, et al. Classification and regression by randomForest. R News, 2(3): 2002.
Y. Lin. Support vector machines and the Bayes rule in classification. Data Mining and Knowledge Discovery, 6:259-200 2-559275,220 2002.
Y. Lin. A note margin-based loss functions in classification. Statistics & Probability Letters, 68 (1):73-82, 2004.
Y. Lin, Y. Lee, and G. Wahba. Support vector machines for classification in nonstandard situations. Machine Learning, 46:191-202, 2002.
Q. Mai and H. Zou. Sparse semiparametric discriminant analysis. Journal of Multivariate Analysis, 135:175-188, 2015.
Q Mai, H. Zou, and M. Yuan. direct approach sparse discriminant analysis in ultra-high dimensions. Biometrika, 99(1):29-42, 2012.
J. S. Marron, M. J. Todd, and J. Ahn. weiigte discrimination. Journal of the American Statistical Association, 102(480); 2007.
M. Muntasir Nishat, F. Faisal, I. Jahan Ratul, A. Al-Monsur, A. M. Ar-Rafi, S. M. Nasrullah, M. T. Reza, and M. H. Khan. comprehensive investigation of the performances of different machine learning classifiers with SMOTE-ENN oversampling technique and hyperparameter optimization for imbalanced heart failure dataset. Scientific Programming, 2022(1):3649006, 2022.
G. Papandreou and A. L. Yuille. Perturb-and-MAR random fields: Using discrete optimization to learn and sample from energy models. In 2011 Internutional Conference on Computer Vision, pages 193-200. IEEE, 2011.
B. Paula, L. Torgo, and R. Ribeiro. A survey of predictive modelling under imbalanced distributions. arXiv preprini MXXv::555.0063: 2015.
D. A. Pritchard and Y. Liu. Composite quantile-based classifiers. Statistical Analysis and Data Mining: The ASA Data Science Journal, 13(4):337-353, 2020.
S. Qiao, Y. He, and W. Zhou. Transfer learning for high-dimensional quantile regression with statistical guarantee. Transactions on Machine Learning Research, 2023.
X. Qiao and Y. Liu. Adaptive weighted learning for unbalanced multicategory classification. Biomet- rics, 661))159-1668 2009.
X. Qiao, H. H. Zhang, Y. Liu, J. Todd, Jnd S. Marron. Weighted distance weighted discrimination and its asymptotic properties. Journal of the American Statistical Association, 405(489)4401-414, 2010.
E. Ramentol, Y. Caballero, R. Bello, and F. Herrera. SMOTE-RSB*; hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using smote and rough sets theory. Knowledge and Information Systems, 33:245-2655 2012.
13

---

S. Rezvani and X. Wang. A broad review on class imbalance learning techniques. Applied Soft Computing, 143:110415 2023.
Saglam and M. A. Cengiz. novel SMOTE-based resampling technique trough noise detection and the boosting procedure. Expert Systems with Applications, 200: 10 11003, 2022.
Santos, Maudes, and A. Bustillo. Identifying maximum imbalance datasets for fault diagnosis of gearboxes. Journal of Intelligent Manufacturing, 29:333-351 2018.
J. A. Sanz, D. Bernardo, F. Herrera, H. Bustince, and H. Hagras. A compact evolutionary interval- valued fuzzy rule-based classification system for the modeling and prediction of real-world financial applications with imbalanced data. IEEE Transactions Fuzzy Systems, 23(4):973 2014.
Shin, Y. Wu, H. H. Zhang, and Liu. Principal weighted support vector machines for sufficient dimension reduction in binary classification. Biometrika, 104(1):67-8) 2017.
D. Singh, P. G. Febbo, K. Ross, D. G. Jackson, J. Manola, C. Ladd, P. Tamayo, A. A. Renshaw, A. V. D' Amico, J. P. Richie, et al. Gene expression correlates of clinical prostate cancer behavior. Cancer Cell, 1(2):203 209, 2002.
W. Siriseriwan. Smotefamily: A collection of oversampling techniques for class imbalance problem based on SMOTE. R package version, 1(1):15, 2019.
J. M. Sorace and M. Zhan. A data review and re- re-assessment of ovarian cancer serum proteomic profiling. BMC Bioinformatics, 4:1-13, 2003.
Q. Tang, Y. Zhang, and B. Wang. Finite smoothing algorithm for high-dimensional support vector machines and quantile regression. Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 478654478884 PMLR, 2024.
R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Staristical Society Series B: Statistical Methodology, 58( (8(1):267-293) 1996.
R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and Knight. Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(1):91-108 2005.
A. Tsanas, M. A. Little, C. Fox, and L. O. Ramig. Objective automatic assessment of rehabilitative speech treatment in Parkinson's 's disease. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(1): (11):181-199 2013.
A. van der Vaart and J. Wellner. Weak Convergence and Empirical Processes. Springer, New York, 1996.
V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 1995.
B. Wang and H. Zou. Another look at wisthnee-weighted discrimination. Journal of the Royal Statistical Society Series Statistical Methodology, 80(1):177-198, 2018.
L. Wang, J. Zhu, and H. Zou. The doubly regularized support vector machine. Statistica Sinica, 16: 589-615 2006.
L. Wang, Y. Wu, and R. Li. Quantile regression for analyzing heterogeneity in ultra-high dimension. Journal of the American Statistical Association, 107(497):214-222 2012.
W. Wei, J. Li, Cao, Y. Ou, and J. Chen. Effective detection of sophisticated online banking fraud on extremely imbalanced data. World Wide Web, 16:449-475, 2013.
T. Weiss. Correlates of posttraumatic growth in married breast cancer survivors. Journal of Social and Clinical Psychology, 23(5):733-746, 2004.
Q. WU, Y. Ye, H. Zhang, M. Ng, and S.-S. Ho. ForesTexter: efficient random forest algorithm for imbalanced text categorization. Knowledge-Bond Based Systems, 67:105-116, 2014.
14

---

Z. Wu, W. Lin, and Y. Ji. An integrated ensemble learning model for imbalanced fault diagnostics and prognosties. IEEE Access, 6:8394-84022 2018.
M. Yuan and Y. Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(1):49-6) 2006.
L. Zeng and H. H. Zhang. Sparse learning and class probability estimation with weighted support vector machines. arXiv preprint arXiv:2312.106168 2023.
C. Zhang, Liu, J. Wang, and H. Zhu. Reinforced angle-based multicategory support vector machines. Journal of Computational and Graphical Statistics, 2563):806-825, 2016.
Q. Zheng, L. Peng, and He. Globally adaptive quantile regression with ultra high dimensional data. The Annals of Statistics, 43(5):2225, 2015.
Z.-B. Zhu and Z.-H. Song. Fault diagnosis based imbalance modified kernel fisher discriminant analysis. Chemical Engineering Research and Design, 88(8)9936-951, 2010.
H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, (7(2))301-)))) 1000000--,,-0 2005.
15

---

SUPPLEMENTARY MATERIAL
S1 Additional Numerical results
S1.1
Simulations
All numerical experiments in this work were carried out Intel(R) Xeon(R) Gold 6430 (3.40
GHz) processor.
For high-dimann figh-dimenssonal imbalanced classification problems. relying solely on the AUC score is insuffi- cient fully evaluate the effectiveness of method. To offer more comprehensive assessment, we compare our proposed approach with several baseline methods using additional performance metries, including the area under the precision-recall curve (PRAUC), the Fl score, and the geometric mean (G-mean). Tables [..] to S.3 show that QuanDA consistently outperforms all competing methods across all evaluation metrics. Notably, quantileDA excluded from AUC and PRAUC comparisons, as it only produces predicted class labels rather than probability scores. Consequently, AUC and PRAUC cannot computed for this method. Furthermore, random forest, SMOTE, and quantileDA yield Fl scores and G-means of zero in certain settings, as they predict all samples as belonging to majority class, failing to identify any instances the minority class.
Tables S.4 report the AUC, PRAUC, FI, and G-mean scores for QuanDA and several widely used methods for imbalanced classification, evaluated on simulated data with 400 and p 5000. QuanDA consistently achieves superior performance across all evaluation metrics compared to the competing methods.
S.repprrt
S1.2 Real-data analysis
We further evaluate QuanDA against other methods using real-world datasets, focusing on PRAUC, F1, and G-mean scores. As shown in Tables ?? and ??, QuanDA consistently demonstrates superior performance across all metrics, while quantileDA exhibits the least effective results among the compared methods.
16

---

Table S.1: The PRAUC scores for five imbalanced classification solvers were evaluated using simulated data with n 400 and p 10000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses.
QuanDA
dsda
logistic
RF
SMOTE
0.85
AR
0,756 (0.086)
0.750 (0.088)
0.733 (0.100)
0.431 (0.098)
0.413 (0.1 10)
AR;
0.626 (0.103)
0.600 (0.
0.572 (0.123)
0.390 (0.088)
0.380 (0.100)
AR7
0.573 (0.101)
0.552 (0.107)
0,525 (0.120)
0.378 (0.086)
0.361 (0.092)
CS2
0.829 (0.072)
0.827 (0.075)
0.819 (0.086)
0.439 (0.100)
0.451 (0.118)
CS,
0.713 (0.100)
0.707 (0.098)
0.683 (0.115)
0.412 (0.101)
0.428 (0.086)
CS7
0.629 (0.103)
0.613 (0.112)
0.580 (0.133)
0.389 (0.108)
0.382 (0.105)
MO = 0.9
AR2
0.657 (0.136)
0.638 (0.160)
0.604 (0.174)
0.272 (0.083)
0.286 (0.098)
AR,
0,588 (0.140)
0.501 (0.144)
0,456 (0.182)
0.269 (0.105)
0.274 101)
AR7
0.452 (0.140)
0.443 (0.149)
0.396 (0.172)
0.265 (0.096)
0.263 (0.088)
CS2
0.737 (0.129)
0.734 (0.149)
0,696 (0.178)
0.294 (0.103)
0.296 (0.093)
CS3
0.600 (0.145)
0.589 (0.163)
0.551 (0.174)
0.288 (0.115)
0.280 (0.097)
CS7
0.508 (0.142)
0.503 (0.154)
0,453 (0.188)
0.264 (0.101)
0.273 (0.086)
TO
0.95
AR2
0.353 (0.211)
0.368 (0.224)
0,262 (0.239)
0.129 (0.081)
0.144 (0. 104)
ARs
0,243 (0.151)
0.243 (0.176)
0.184 (0.181)
0.120 (0.071)
0.116 (0.066)
AR7
0.211 (0.147)
0.207 (0.152)
0.150 (0.144)
0.122 (0.071)
0.121 (0.079)
CS
0.406 (0.228)
0.452 (0.258)
0.321 (0.271)
0.121 (0.078)
0.141 (0.091)
CS.
0.302 (0.197)
0.336 (0.216)
0.219 (0.205)
0.140 (0.089)
0.118 (0.070)
CS7
0.248 (0.141)
0.243 (0.183)
0.184 (0.177)
0.132 (0.086)
0.124 (0.070)
Table S.2: The Fl scores for six imbalanced classification solvers were evaluated using simulated data with n 400 and 10000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses. Notably, random forest, SMOTE, and quantileDA yield F1 scores of zero in this setting.
QuanDA
dsda
logistic
0.85
AR2
0.663 (0.069)
0.648 (0.109)
0.635 (0.079)
AR;
0.561 (0.071)
0.500 (0.097)
0.531 (0.084)
AR,
0.528 (0.072)
0.440 (0.107)
0.494 (0.087)
CS2
0.724 (0.072)
0.730 (0.085)
0.711 (0.078)
CS3
0.634 (0.076)
0.608 (0.104)
0.600 (0.089)
CS7
0.563 (0.074)
0.513 (0.094)
0.528 (0.095)
To = 0.9
AR
0.586 (0.088)
0.512 (0.199)
0.539 (0.126)
AR3
0.482 (0.097)
0.376 (0.161)
0.447 (0.112)
ART
0.444 (0.085)
0.334 (0.143)
0.404 (0.104)
CS
0.655 (0.084)
0.599 (0.201)
0.614 (0.118)
CS.
0.538 (0.089)
0.472 (0.185)
0.503 (0.1
CS²
0.486 (0.091)
0.389 (0.)
0.448 (0.103)
TO
0.95
AR
0.399 (0.161)
0.193 (0.238)
0.309 (0.189)
ARs
0.314 (3.136)
0.126 (0.169)
0.225 (0.172)
ART
0.288 (3.134)
0.109 (0.157)
0.196 (0.172)
CS2
0.453 17.179)
0.290 (0.271)
0.370 (0.203)
CS;
0.358 (0.151)
0.175 (0.219)
0.265 (8.1
CS7
0.313 (0.141)
0.141 (0.178)
0.225 (0.177)
17

---

Table S.3: The G-mean scores for six imbalanced classification solvers were evaluated using simulated data with 400 and p 10000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses. Notably, random forest, SMOTE, and quantileDA yield G-mean scores of zero in this setting.
QuanDA
dsda
logistic
70 0.85
2R
0.844 (0.047)
0.747 (0.093)
0.808 (0.072)
ARs
0.788 (0.052)
0.629 (0.084)
0.747 (0.087)
AR,
0.761 (0.053)
0.576 (0.097)
0.721 (0,091)
CS2
0.872 (0.047)
0.813 (0.072)
0.854 (0.071)
CSs
0.828 (0.055)
0.716 (0.090)
0.787 (0.088)
CS7
0.783 (0.056)
0.640 (0.079)
0.740 (0.100)
No = 0.9
AR2
0.812 (0.069)
0,632 (0.205)
0.775 (0.123)
AR,
0.757 (0.070)
0.513 (0.179)
0.719 (0.133)
ART
0.730 (0.073)
0,473 (0.163)
0.685 (0.166)
CS2
0.848 (0.062)
0.700 (0.205)
0.811 (0.103)
CSs
0.789 (0.078)
0.598 (0.189)
0.750 (0.127)
CS7
0.758 (0.088)
0.522 (0.181)
0.723 (0.124)
= 0.95
AR.
0.690 (0.152)
0.262 (0.306)
0.572 (0.310)
ARs
0.602 188)
0.194 (0.244)
0.456 (0.327)
ART
0.592 15.)
0.165 (0.222)
0.399 (0.334)
CS2
0.705 (0.162)
0.381 (0.332)
0.596 (0.317)
CSs
0.657 (0.158)
0,246 (0.287)
0,502 (0.319)
CS7
0.623 (0.126)
0,213 (0.249)
0.446 (0.328)
Table S.4: The AUC scores for five imbalanced classification solvers were evaluated using simulated data with n 400 and = 5000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses.
QuanDA
dsda
logistic
RF
SMOTE
0.85
AR.
0.931 (0.025)
0.925 (0.028)
0.919 (0.029)
0.830 (0.051)
0.829 (0.051)
AR;
0.882 (0.034)
0.871 (0.036)
0,861 (0.041)
0,801 (0.049)
0.804 (0.051)
ART
0.857 (0.038)
0.845 (0.042)
0.831 (0.048)
0.781 (0.059)
0.782 (0.065)
CS2
0.956 (0,018)
0.953 (0.020)
0.949 (0.023)
0.845 (0.044)
0.838 (0.054)
CS3
0.916 (0.029)
0.909 (0.031)
0.901 (0.034)
0.818 (0.047)
0.816 (0.057)
CS7
0.882 (0.035)
0.874 (0.039)
0.863 (0.044)
0.806 (0.055)
0.804 (0.055)
To 0.9
AR2
0.923 (0.039)
0.908 (0.052)
0.891 (0.081)
0.793 (0.066)
0.793 (0.075)
AR3
0.874 (0.047)
0.849 (0.073)
0.827 (0.098)
0.769 (0.071)
0.767 (0.075)
ART
0,846 (0.060)
0.816 (0.092)
0.794 10.)
0.744 (0.085)
0.755 (0.080)
CS2
0. 0.948 (0.031)
0.940 (0.039)
0.930 (0.055)
0.804 (0.071)
0.790 (0.089)
CS.
0.907 (0.043)
0.888 (0.061)
0.868 (0.096)
0.777 (0.081)
0.786 (0.071)
CS7
0.873 (0.050)
0.853 (0.072)
0.830 (0.098)
0.755 (0.084)
0.763 (0.070)
TO
0.95
AR
0.874 (0.081)
0.833 (0. 125)
0.760 (0.159)
0.706 (0.120)
0.690 (0.115)
AR;
0.821 (0.090)
0.780 (0.
0.698 (0.147)
0.689 (0.103)
0.683 (0.112)
AR7
0.793
(0.094)
0.747 (0.133)
0.674 (0.142)
0.681 (0.105)
0.680 (0.107)
CS2
0.901 (0.079)
0.871 108)
0.792 (0.163)
0.713 (0.108)
0.687 10.104)
CS;
0.855 (0.088)
0.813 (3.)31)
0.745 (0. 14.)
0.689 (2.
0.693 (0.104)
CS7
0.821 (0.089)
0.778 (3.137)
0.693 (0.141)
0.675 (0.108)
0.676 (0.105)
18

---

Table S.5: The PRAUC scores for five imbalanced classification solvers were evaluated using simulated data with n 400 and p 5000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses.
QuanDA
dsda
logistic
RF
SMOTE
0.85
AR
0.777 (0.065)
0.763 (0.080)
0.748 (0.075)
0.528 (0.110)
0.524 (0.113)
AR;
0.650 (0.078)
0.627 (0.089)
0.609 (0.090)
0.456 (0.107)
0.468 (0.097)
AR7
0.596 (0.080)
0.574 (0.096)
0,550 (0.095)
0.418 (0.109)
0.423 (0.117)
CS2
0.846 (0.051)
0.842 (0.058)
0.832 (0.061)
0.558 (0.111)
0.544 (0.128)
CS,
0.738 (0.073)
0.723 (0.083)
0,701 (0.085)
0.503 (0.092)
0.489 (0.116)
CS7
0.654 (0.081)
0.635 (0.094)
(0.09))
0.478 (0.104)
0.470 (0.109)
Mo = 0.9
AR2
0.682 (0.126)
0.655 (0.141)
0.625 (0.152)
0.376 (0.116)
0.373 (0.113)
AR,
0,555 (0.119)
0.507 (0.140)
0,477 (0.156)
0.331 (0.116)
0.323 (0. 106)
AR7
0.491 (0.113)
0.449 (0.141)
0.416 (0.151)
0.303 (0.110)
0.291 (0.091)
CS2
0.764 (0.115)
0.751 (0.127)
0.726 0..26(0.141)
0.381 (0.126)
0.381 (0.131)
CS3
0.634 (0.131)
0.603 (0.150)
0.569 (0.163)
0.340 (0.114)
0.351 (0.121)
CS7
0.548 (0.124)
0.514 (0.146)
0,480 (0.159)
0.327 (0.109)
0.308 (0.097)
TO 0.95
AR
0.428 (0.195)
0.390 (0.208)
03.150022100
0.186 (0.138)
0.167 (0.
AR5
0.315 (0.169)
0.281 (0.160)
0,218 (0.173)
0.149 (0.093)
0.144 (0.127)
AR7
0.263 (0.154)
0.248 (0.159)
0.186 (0.160)
0.137 (0.093)
0.142 (0.087)
CS2
0.527 (0.191)
0.471 (0.217)
0.372 (0.227)
0.185 (0.125)
0.167 (0.107)
CS.
0.400 (0.177)
0.346 (0.197)
0.269 (0.188)
0.162 (0.114)
0.165 (0.108)
CS7
0.305 (0.173)
0.286 (0.177)
0.197 (0.163)
0.145 (0.104)
0.146 (0. 100)
Table S.6: The Fl scores for six imbalanced classification solvers were evaluated using simulated data with 400 and p 5000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses. Notably, random forest, SMOTE, and quantileDA yield F1 scores of zero in this setting.
QuanDA
dsda
logistic
0.85
AR2
0.670 (0.072)
0.648 (0.109)
0.653 (0.071)
AR;
0.562 (0.073)
0.521 (0.090)
0.544 (0.071)
ART
0.525 (0.069)
0.463 (0.094)
0.500 (0.074)
CS2
0.733 (0.065)
0,732 (0.089)
0.725 (0.059)
CS3
0.632 (0.076)
0.613 (0.091)
0.618 (0.070)
CS7
0.569 (0.076)
0.530 (0.091)
0.550 (0.072)
To 0.9
AR
0.584 (0.093)
0.547 (0.151)
0.577 (0.)08)
AR3
0.478 (0.099)
0.403 (0.)
0.453 (0.093)
ART
0.444 (0.101)
0.342 (0.150)
0.419 (0.108)
CS
0.655 (0.091)
0.626 (0.141)
0.641 (0.116)
CS.
0.548 (0.101)
0.496 (0.159)
0.527 (091
CS7
0.484 (0.101)
0.408 (0.176)
0.456 (0.101)
TO = 0.95
AR
0.413 (0. 144)
0.279 (0.224)
0.293 (0.180)
ARs
0.337 (0.127)
0.187 (0.177)
0.234 (0.143)
ART
0.292 (0.122)
0.146 (0.183)
0.200 (0.121)
CS2
0.471 (0.155)
0.325 (0.265)
0.354 (0.201)
CS;
0.393 (0. 142)
0,228 (0.212)
0.258 (0.155)
CS²
0.335 (0. 134)
0.181 (0.191)
0.222 (0.134)
19

---

Table S.7: The G-mean scores for six imbalanced classification solvers were evaluated using simulated data with 400 and p 5000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses. Notably, random forest, SMOTE, and quantileDA yield G-means of zero in this setting.
QuanDA
dsda
logistic
Tu 0.85
AR
0.843 (0.037)
0.744 (0.094)
0.820 (0.060)
AR;
0.782 (0.045)
0.644 (0.075)
0.754 (0.070)
ART
0.754 (0.048)
0.594 (0.078)
0.730 (0.085)
CS2
0.874 (0.033)
0.809 (0.078)
0.855 (0.049)
CS.
0.822 (0.039)
0.718 (0.081)
0.801 (0.058)
CS7
0.787 (0.046)
0.649 (0.076)
0.758 (0.066)
TO
0.9
AR
0.802 (0.072)
0.664 (0.155)
0.793 (0.093)
ARs
0.752 (0.077)
0.535 (0.175)
0.721 (0.103)
ART
0.731 (0.075)
(0.188)
0.707 (0.138)
CS2
0.845 (0.066)
0.732 (0.128)
0.821 (0.109)
CSs
0.787 (0.081)
0.621 (0.171)
0.755 (0.102)
CS7
0.744 (0.087)
0.535 (0.200)
0.727 (0.104)
0.95
AR
0.698 (0. 124)
0,381 (0.289)
0.568 (0.290)
AR;
0.652 (0.142)
0,280 (0.249)
0.505 (0.287)
ART
0.610 147)
0.216 (0.256)
0.462 (0.282)
CS2
0.731
(0.134)
0,418 (0.323)
0.581 (0.299)
CSs
0.681 (0.122)
0.320 (0.279)
0.527 (0.296)
CS7
0.652 (0.123)
0.266 (0.260)
0.488 (0.295)
20

---

S2 Extension to Imbalanced Multi-class Classification
Although QuanDA is primarily designed for binary classification tasks, it can be extended imbal- anced multi-class classification under HDLSS settings using strategies. In this section, we present a simple example illustrate the effectiveness of QuanDA in handling multi-class classification using the one-vs-one strategy. We leave the development and evaluation of more advanced extensions to future work.
Motivated by the simulation design described in Section 4 we consider a three-class imbalanced classification problem in which each class follows multivariate normal distribution with common covariance matrix Σ. The class-specific mean vectors differ only in the first five features, with values set to 0.7, and -0.7, respectively. Two imbalance scenarios are examined. In the first scenario, the majority class ( To accounts for 80% of the total sample, while each of the two minority classes represents 10%. In the second scenario, the majority class comprises 90% of the total sample, and each minority class accounts for 5%. The total sample size is set to n = 100 or n = 400, and the feature dimension is p = 10, 000. We compare the performance of QuanDA with several competing methods, including MSDA, weighted random forest, and SMOTE. The macro Fl score (Grandini al. 2020) employed as the evaluation metric to assess the multi-class classification performance of each method. Table S.8 shows that QuanDA achieves the highest macro Fl score among the compared methods. In contrast, weighted random forest and SMOTE fails identify the minority classes, predicting all samples as belonging to the majority class.
Table S.8: The macro Fl scores for three imbalanced classification solvers were evaluated using simulated data with p = 10000. These scores represent the average results obtained over 50 independent runs with standard errors given in parentheses.
400
QuanDA
msda
RF
SMOTE
8:1:1
AR2
0.515 (0.073)
0.369 (0.031)
0.297 (0.000)
0.297 (0.000)
AR5
0.490 (0.069)
0.334 (0.023)
0.297 (0.000)
0.297 (0.000)
AR;
0.479 (0.071)
0.318 (0.021)
0.297 (0.000)
0.297 (0.000)
CS2
0.524 (0.068)
0.401 (0.029)
0.297 (0.000)
0.297 (0.000)
CSs
0.506 (0.072)
0.346 (0.020)
0.297 (0.000)
0.297 (0.000)
CS²
0.488 (0.061)
0.322 (0.017)
0.297 (0.000)
0.297 (0.000)
9:0.5:0.5
AR2
0.460 (0.067)
0.319 (0.016)
0.317 (0.000)
0.317 (0.000)
ARa
0.438 (0.064)
0.323 (0.028)
0.317 (0.000)
0.317 (0.000)
AR7
0.425 (0.065)
0.324 (0.028)
0.317 (0.000)
0.317 (0.000)
CS2
0.478 (0.068)
0.325 (0.031)
0.317 (0.000)
0.317 (0.000)
CSs
0.460 (0.070)
0.325 (0.033)
0.317 (0.000)
0.317 (0.000)
CS²
0.435 (0.071)
0.322 (0.023)
0.317 (0.000)
0.317 (0.000)
S3 Proof of Theorem 3.1
For ease of notation, we fix T E (0, 1) and drop all subscript wherever no confusion arises. Let
Qn 72
(3).
For À > 0, define the lasso estimator of the quantile regression by
(xx, = arg min Qn (3) + Σ 18
(6)
a
j=1
Let
Vn Vn B) = Qu - (n ,²) - (a 3) - Qn(a",
For some r > 0, set
Gu,uy = ((6, E Cu,oi ni
Σ
+ x A)² rz²..
21

---

Also, define
e(u, v,T)
sup
+ 5 1 + A)).
(6. A).
Let Faix and Jex be the distribution and density functions of cr, respectively. We shall write them as F and f for simplicity of notation in the proofs. The following proofs are based on a given X (i.e., conditional on X), but can be easily modified for a stochastic X.
Lemma S3.1. Under conditions (C1)-C3), with probability at least
(-x2)
nd² 2
2ppexp (-mxX²
nd²
1 - exxp
2M
the lasso estimator (āxx, 3xx of the quantile regression satisfies
00 E C3.1 {(6, A): dER, AER" AA S Aall + 10]]
where 8a 0x2 a and Â
3x B
Proof of Lemma S3.1 Let
1
{=
[(; S aa)),
n i=1
and c = [p) where
&
T(e {a')]as;, ssssp.
Note that (c. TT E 0Qn (a", 3B",, where the subdifferential taken with respect to a and J. By convexity of (m, B) and optimality of Bx), we have
0 2 (a, B.) Qn(a", , X(1I (11 (1/3*11)
2 ((āx - -a') E 33) + A(113x11 112 - |18"
2 - |cl Tax -a"l - ||E |0 11/32 - 13
Σ
13A.5 1871 Σ 1 1811
xEA=
jEA
which implies that
(A 11E1
Σ 13x.j 1311 1 (A =0)
Σ
18x.5 - 531 +5l- aaI.
(7)
JEAA
jEA
Under event & = {(sl S 2/2, | & 2/2), it follows from that
A x 11 - 31 2 + 184)-
The lemma then follows from Hoeffding 's inequality
X 2
Pr (1611100 d 2
Pr(E) > 1 Pr
[5]
2
2 21 Pr(1 (1| >
T - IlE, S <a')l>>> ))
n
i=1
Σ Pr
Σ
n
E 1(E, S <aa)|>> > 2
1=1
1=1
2 21 21-2 (
n122
nd² 2p exp
-2MM0
2
where Mo ma E [x]]]
22

---

Lemma S3.2. For u, u.T.t > 0, under conditions (CI)-(C6), with probability at least 1
exp -nt? |nnt2/n3rr /32r2)) we have
MMo
+ logp
U,) -
(1 (1+=)=+(1=+0)+
(2(21, v)
when 2 3. It follows immediately that, if one takes
2Mo
+ log
1 1-41 kul v)
(1 +++
then with probability at least 1 - expl -Mo(1 + (s + log p)/m/)////////n
n00 (u, u)] we have
2Mo
1+ + logp
v, S 8 nolu, v)
+ u1++)/+++o (1 (1)
Proof of Lemma S3.2
First, note that the check loss pr(.) is Lipschitz continuous with Lipschitz T). Let 8 = a- a", A B f", and define
constant max(r,
U.(6, A) p+(2; a x; 3) pr -
x,, gr)
=pr(1ī - x A) (r(),
where r. 2; 0² x. B² Ei -a, 1 i n. It follows immediately that
e(u, v,r) sup (6, A)E
Σ [U, A) EU,( A)
By Lipschitz continuity of the check loss, it follows that
(6, A) - (p- - 6 A p+(r!)) S max(T, r)16 x A S | x
(8)
A
isisn n.
Applying Massart's concentration inequality Bühlmann and van de Geer (011)), we have
2t
Fr(e(u,urr) 2 Ble(u, v, r)] + exp
8hg (u, v,r)
where ba (u, v,r) sup(s, A)EG. Σ var(U, A)). First, derive upper bound 82 (u, v,r). Note that by g and the Kauchy-Schwarc inequality
sup (8, A)EG ==1 (u,,,))
E EU. A) B(U,(6, A)))
< sup
16+ < 4nr
(8, (G.
Next, we show an upper bound on Efe(u, v, r)). Applying the symmetrization procedure van der
Vaart and Wellner (1966) and the contraction principle Ledoux and Talagrand (199T), we have
E[e(u, 2F
sup
Σ 1=1
(B A)
( AEGu....
7
2
E
sup
Edle(rt - - x A) p-(33))
(10)
L(8, A)E9..
<
-E E
sup A)EG....r
Σ E. A)
L(A.
where ,,,...,
. En are i.i.d. Rademacher random variables that are independent of E en and Pr(f: +1) 0.5.
23

---

For (6, A) E Gu.or, by condition (C1) and Cauchy-Schward inequality, we have
2² 2 x)(2 + | 2 ko(u, Kolu, v) A ll:
(11)
which implies that |6| S /rou/ and |ل S rv E s/Kolu,,,) Let & = i En) For any E E, we have
Eexp(LX} E)
II
(e twss + E tt(s)
2
12
II exp Poly = =exp
Σ
sspjsp
2 1=1
Letting t > 0, by Jensen's 's inequality we have
exp [ | El(xo] = cxp
tE max |
xx55) 51) - BExxp t max
1x E
OSiSP
o5i5D
=
max exp(t)X3 5)) OSSSPP
S
max
exx]'E + + =
0S15P
Σ E (cxx & + -
S >
2 |
exp
n.Mo
s 2(1 + p) exp
max
[| = 2(1 + exp
2 0SJSP
2
which implies that
E X =o) S $ |log 2 + log(1 + p)] n.Mo,
t, > 0.
2
Taking t = 22 2 log(1 11 + (n)) we obtain
[0 T E (oo) < v Mn
2 + log(1 p)] S v 2M / nn(1 + log p)
(12)
as long as p 2 3. It then follows from 10
12
and the Hölder's 's inequality that
Ele(u, r)] [ Ellbo)
sup (8, A)E.u..
(15) + |A]|i
Vn(1 4 2
log
sup
(10 + +|||)) |1))
n
(4. AjEG
4 M
nn +lgg))
sup
u1)81 (1 4) Abl
n
(4. A.EEG.....
2Mo
+ log
+ u)+++ (1+
Ko(u, 2))
n
The lemma then follows from
Lemma S3.3. Under conditions (C1) for any (ō, A) E Cu.,, we have
[[Qn(a o + A) Qn(a", (8))
1
1/2
min
/2 9
(8
A
2
Proof of Lemma $3.3
By Knight's identity Knight 1998), we have for any two scalars r and s,
r s - |r| = - [T[ > 0) [(r < 0) + 2 / [T(r < t) I(r s d)
24

---

It follows that for any 7 E (0, 1),
pr( - - - = (r- - - + 0.5 s) s|-(r) -rl
(.5 = -0..5 - [[r > - - < o]] + /
[(r -(r
(13)
= = [0] --] + / [Tr < - -
0
Let = 2 a' - x - a- 1 S <is n. By
13 and the mean value theorem, we have for
some Usst between and 1,
n
B Qu(aa + + A) Qu(a" , ,3)]]
Σ R
( - 5- x, A) p- (r:)
i=1
p+xx A
E
-r| - a') [T(E
[I(E S
[TE + +t) I(E, -IEE dt
a
S+xTA
[F(a" +t) dt
(14)
a
S+xTA
12 (s(a²) + +2222/0 5 + wn.s] dt
f a
f(a")(6
+ T A) 2
Σ 16 x A 3
6n
i=1
2 2n
6 0 X
A
88 x
A
6n
1=1
1m1
For (8, A) E Cu,os note that if
1/2
4
66 x A}2
S
f1/24
(15)
1=1
then by condition (C6) we get
n
8
+
x
3
+ x xT),,
6n
4n
which, together with 14 implies that for all (6, A) E Gy, 1/24/241
[ [6+xxAA)
E[Qn(a 5,9 A) - Qu(a", (()]
> 4n
When (15) does not hold, one can similarly apply the technique in the proof of Lemma 4 of Belloni and Chernozhukov 201 to show that for any (6, A) E Cu.v.
E Qn(a" + + A)
Qn ,8*)
[
min
Σ 1=1
(8
x
A 2
112
2 (8 16+x A)2
4n
This completes the lemma.
Proof of Theorem 3.1 Let G [(6,1)) C(3, n 16 (6+x x x 2 2 r2), where
2Mo
1+ 1+ log (2/8 + 1) + X
r. = 8f-1
1
(3, 1)
Ko (3, 1)
25

---

Moreover, let 8 = - a and - Bx -3". If we can show that
min Qn(a" + 6, 3r + A) -Q,(a", + X(113* + All 1-11111111) > 0, (16) (6, ().))
then by convexity of Qn this implies that Ei= (8* + x A
r under the event
{(0^, A E (3, 1)}. To show (16). first note that by Lemma S3.2 with probability at least
1 - exp (1 + logp)/ 1)], we have for all (6, A) E g",
Qn(a" + 6,3 + - - Qn(a", + A0113 + A 1 - (11))
2 E Qn(a" + 0 + A - Qn(a", e(3, 1.T.) + A
Σ
JEA
DEA
2 E[Qn(a" + 5 A Qn(a" 3r)] + 1
Σ
|4
Σ(AI)
(17)
JEA
jEA
MMo
+ log (2V3 + 1)r..
1
Ko(3, 1)
On the one hand, by Lemma S3.3 for any (d, A) E G we have
((Q + 0 - A) - (a , 2 min (fr2/4, gr.)
On the other hand, by condition (C1) and (11 we can see that
Aللl S r. vrvvas (3, 1).
Thus, it follows from 17 and the growth condition that
Qn(a" + 0 + A) - Qn(a",B1 + A(11S* A 111 - 11
f
2.oo
+ logp (2V3 1) + A s//o 1) r. > 0
2 = 2
16
4
(o(3, 1)
for all (6, A) E G by our choice of r.. By Lemma S3.1 and convexity of Qn, this implies that with
probability least
Pr(E)
expl exp[- [ppp[11660 (16 + logp)/ 1)] 2 - 1111(A))) (A),
have (8) A )
C(3, 1) and
n2 Σ 18
+x7 A 5
This, by condition (CI), further implies that
? 2 Km (3, 1) 1111112 + | AUTT mm)
As result, we obtain that
8 <
Tx
v Nim (3. 1)
and that
A AALA(A
Am))al
(18)
VKm (3m
Note that the jth largest in absolute value component of A A is bounded by || A l1//j. Therefore, it follows that
P
IR
1
,m)))
j2
m
4= 11
j=m+
A 1|2 + 18²1 <
18s
A A1l
112
+
8
m
18s
A A A .m)
+
2
8
m
m
26

---

which implies that
|
12 - 1
188
A
.m)
2
181p2
m
m
188
2
+
+
(3, 1)
m
m
This completes the proof of Theorem 3.1
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes].
Justification: The claims presented in the abstract and introduction accurately represent the contributions and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes].
We discuss the limitations of our work in the last section (Section 5) of the paper.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
28

---

Answer: [Yes].
Justification: The assumptions needed for theoretical results are included in the Section 3, and complete proof of each result is given in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes].
Justification:
We have included details of the experimental setup.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
29

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes].
Justification: The implementation of QuanDA is available at hittps://anonymous 4oper Sixence/statuu wttps//ttttttttttStatt 57FE
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
public/guides/ Soaalees//CoeSSuuumsisiiiiiii
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/cccppllligguddeeeeeeeeeeee Cooee///ddeeebmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]..
Justification:
We have included the details of experimental setup.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes].
Justification: For the experiments on synthetic data, we report error bars and the experimental settings for the random data.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
30

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes].
Justification:
We have included the details the computational resources in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes].
Justification:
We have conformed to the code of ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA].
Justification:
Our work does not have a direct societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
31

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA].
Justification:
The models or the data used in the paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA].
Justification: Our work does not existing assets.881
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
32

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA].
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA].
Justification: Our work does not involve research with human subjects crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA].
Justification:
Our work does not involve research with human subjects crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
33

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA].
Justification: Our work does not involve LLMs any important, original, non-standard components.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
34


Reviews:
[{"review_id": "IiTtTPuF76", "reviewer": "Reviewer_ajh1", "summary": "Hi folks, this likely won't be a traditional review since I'm not a traditional reviewer. I'm a \"subject matter expert\" on another paper submission, and it looks like I have been asked to help review some other submissions. I'm an expert in data management, not machine learning, so please take my opinions and thoughts with a grain of salt. Despite not being an expert in this area, I did my best to read your paper closely and provide comments that I hope are useful, even if they are from a more general CS point of view rather than the ML experts you may be used to!\n\nSummary: The authors propose QuanDA, a discriminant analysis technique for HDLSS. QuanDA uses an intuitive connection betwewn quantiles and weighted Bayes risk: by adding a small amount of noise to class labels, the \"quantiles\" of previously-binary variables become well-defined, meaning that quantile regression can be used. Sparse quantile regression is selected to deal with the high number of dimensions.", "questions": "Q1) Could the authors do a better job of discussing the limitations (both practical and theoretical) behind their work?\n\nQ2) Could the authors provide some intuition for (1) the proof assumptions and (2) the resulting theorem in S3 of the paper?\n\nQ3) Could the authors explain their choice of competitors for the experimental analysis? The experimental analysis kind of feels like the authors thought \"well, it's a NeurIPS paper, it needs some experiments!\" and then threw something together. Making it a little more structured (e.g., going through a more robust set of competitors) and highlighting key insights (e.g., maybe a case study of a failing case or of a particularly good result) would take the experiments from \"technically there and complete\" to \"actually a useful part of the paper.\"", "limitations": "Yes -- while limitations could be discussed more clearly, the content is all there.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "I was pleasantly surprised at the readability of this paper, especially compared to the other works I reviewed. The introduction actually introduces the problem and covers related work in detail, clearly setting up the problem. I did not fully understand the intuition behind the proof in S3, but this might not be the author's fault (although I think the paper would benefit from a clear, bold-faced, two-line description of the result!).\n\nD1) I had never thought of quantiles this way! S2.1.2 is a very nice piece of intuition. I will restate it here so the authors can decide how much weight to give the rest of my review: if if the majority-class probabilty at some point X exceeds 1 - P(majority), then the quantile of the data (which is itself binary, for binary data) flips from 0 to 1. This is exactly the cost-sensitive Bayes threshold for imbalanced data from Lin et al. \n\nD2) It seems to me that the jittering is more than just about numeric stability. Without jittering, I do not really understand what the quantiles \"are\" -- the jiterring makes the quantiles well-defined, the fact that it makes quantile regression numerically stable is a side-effect of this property, I think.\n\nD3) In S3, I do not understand where C6 comes from or what the intuitive motivation for this condition is. It almost seems like the authors sorted the conditions from \"most intuitive\" to \"random thing we needed to make the proof work\" -- I'm not against any of these conditions, I would just like a better understanding of the intuition behind them.\n\nD4) I was surprised to see that the authors used \"5%\" as an experimental representation of \"high class skew.\" I was expecting something closer to 0.01% or lower. Not a problem, just a surprise. The authors might want to make their understanding of \"skew\" more concrete earlier in the work. \n\nD5) I expected (kernel) SVMs to be included in the experimental analysis, since that's probably the technique I would've reached for in a problem with so many dimensions (SVMs and their cost-sensitive variant were also mentioned by the authors). The underconstrained nature of the problem (dims >>> n) explains the poor performance of other approaches.\n\nD6) The \"conclusion and limitations\" section is more of a \"conclusion and future work\" section, as limitations are notably absent. It's probably worth a few sentences here about how the current QuanDA assumes linear seperability, etc. This is obvious to anyone who fully reads the paper, but explicitly flagging the discussion is helpful for poor grad students looking for works that match their problems!\n\nD7) I think the best part about this paper was drawing the connection between Bayes risk and quantiles of (jittered) binary data. This is an intuitive idea that seems obvious in retrospect, but I had never personally considered (the best ideas often are!). The paper's presentation made the idea clear and intuitive, and I can honestly say I learned something reading it.", "was_revised": false, "final_justification": "", "rebuttal": "It is especially rewarding and encouraging to receive your positive feedback and we are very grateful for your valuable suggestions. The key innovation of this work is indeed the connection between two fundamental tasks: quantile regression and imbalanced classification. Your interpretation of this connection via the perspective of the flipping quantile and the clearly defined quantile due to the jittering going beyond the numerical stability is particularly helpful. In the revised paper, we will incorporate those insights to highlight this connection more fundamentally. Following your suggestions, we will also include real-world examples in the Introduction section to clearly illustrate the degree of class imbalance addressed in this work.\n\n**Could the authors do a better job of discussing the limitations (both practical and theoretical) behind their work?**\n\nThe main limitation of QuanDA is that the model is too simple. It has only a simple structure and may not adequately capture complex relationships. Because it is indeed challenging to handle imbalanced classification, especially in the HDLSS setting, a simple and straightforward approach like QuanDA could be effective in preventing overfitting. On the other hand, however, when sample sizes increase or class imbalance becomes less severe, QuanDA may be outperformed by more sophisticated methods, such as random forests or even deep learning (which completely fails in the HDLSS setting). To illustrate this point, we conducted additional simulations with $n=1000$ and $p = 10,000$, and we observed that the improvement in performance of QuanDA is less evident compared to the results presented in Table 1 of the main article.\n\n|$n=1000$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | | |$\\pi_0=0.85$| | |\n|$AR_2$|**0.951**|0.950|0.947|0.876|0.871|\n|$AR_5$|**0.901**|0.898|0.895|0.845|0.841|\n|$AR_7$|**0.876**|0.872|0.868|0.827|0.823|\n|$CS_2$|**0.971**|**0.971**|0.969|0.891|0.882|\n|$CS_5$|**0.936**|0.935|0.931|0.869|0.857|\n|$CS_7$|**0.905**|0.902|0.899|0.839|0.837|\n| | | |$\\pi_0=0.9$| | |\n|$AR_2$|**0.947**|**0.947**|0.943|0.852|0.850|\n|$AR_5$|**0.897**|0.892|0.889|0.821|0.824|\n|$AR_7$|**0.872**|0.866|0.863|0.814|0.803|\n|$CS_2$|0.968|**0.969**|0.965|0.868|0.859|\n|$CS_5$|**0.932**|0.930|0.927|0.842|0.840|\n|$CS_7$|**0.900**|0.897|0.895|0.826|0.816|\n| | | |$\\pi_0=0.95$| | |\n|$AR_2$|**0.936**|0.933|0.924|0.790|0.770|\n|$AR_5$|**0.887**|0.877|0.865|0.759|0.764|\n|$AR_7$|**0.864**|0.849|0.837|0.762|0.743|\n|$CS_2$|**0.959**|**0.959**|0.952|0.788|0.786|\n|$CS_5$|**0.920**|0.917|0.904|0.775|0.768|\n|$CS_7$|**0.889**|0.876|0.868|0.765|0.756|\n\nIn theory, this paper only provides consistency for parameter estimation in QuanDA. The theory does not cover feature selection consistency. \n\n**Could the authors provide some intuition for (1) the proof assumptions and (2) the resulting theorem in S3 of the paper?**\n\nA key discovery from our paper is the connection between the quantiles of the binary response and the Bayes rule. However, since the standard quantile regression is not directly applicable to the discrete response, from both a theoretical and a numerical perspective, we propose to bypass that issue via data (response) augmentation. Indeed, we find that the jittered response $Z=Y+U$ yields the desired quantile function $Q\\_{Y}(\\tau|\\mathbf{X})=\\lceil{Q}\\_{Z}(\\tau|\\mathbf{X})-1\\rceil$ through simple truncation, while itself being much more \"continuous\", a property that the standard quantile regression requires. Therefore, in establishing Theorem 3.1, we have assumed conditions (C1-C4) to make $Z$ look as close as possible to an absolutely continuous response variable to which standard quantile regression theory directly applies. Of course, there are some \"kinks\" in the distribution of $Z$ due to the discrete part $Y$. We handle that with assumptions (C2-C3), which makes sure that the quantile function of $Z$, $Q\\_{Z}(\\tau|\\mathbf{X})$, has zero probability of taking on value one exactly, to avoid pathological cases. Assumptions (C5-C6) are standard assumptions in high-dimensional statistical theory to make sure the high-dimensional coefficient vector $\\boldsymbol{\\beta}(\\tau)$ is identifiable.  See Bickel et al, 2009 and Candes et al, 2007 for examples of the design matrix $\\mathbf{X}$ where (C5-C6) are met.\n\nTheorem 3.1 establishes the estimation consistency (in terms of mean squared error) of our proposed estimator using nonasymptotic arguments (Wainwright, 2019), which means that our result holds for all pairs $(n,p)$. With proper choice of the regularization parameter $\\\\lambda=C\\\\sqrt{\\\\log{p}/n}$, our theorem shows that our estimator $\\widehat{\\boldsymbol{\\beta}}\\_{\\lambda}$ converges to the true coefficient vector $\\boldsymbol{\\beta}^{*}$ at a near-optimal rate $\\\\sqrt{s\\log{p}/n}$. Thus, if we are back to the traditional asymptotic statistical framework, by letting both $n$ and $p$ go to infinite, the estimation error $\\\\Vert{}\\widehat{\\boldsymbol{\\beta}}\\_{\\lambda}-\\boldsymbol{\\beta}\\^{\\*}\\\\Vert\\_{2}$ goes to zero as long as $s\\log{p}=o(n)$. This even holds when we let $p=O(\\exp(n^{\\gamma}))$ for some $0<\\gamma<1$, in which case $p$ could be much larger than $n$. \n\n**Could the authors explain their choice of competitors for the experimental analysis? The experimental analysis kind of feels like the authors thought \"well, it's a NeurIPS paper, it needs some experiments!\" and then threw something together. Making it a little more structured (e.g., going through a more robust set of competitors) and highlighting key insights (e.g., maybe a case study of a failing case or of a particularly good result) would take the experiments from \"technically there and complete\" to \"actually a useful part of the paper.\"**\n\nThank you for your constructive suggestion! We completely agree that a more structured presentation would enhance the message conveyed by the experimental section. In the revised paper, we will illustrate why those baseline methods are selected. We selected SMOTE and weighted Random Forest as baseline methods because sampling-based approaches and weighted loss functions remain widely used in practice for imbalanced classification problems, as highlighted in the recent textbook Statistical Foundations of Data Science by Fan et al. (2020). In addition, methods such as DSDA and sparse logistic regression have demonstrated strong performance in high-dimensional classification settings. Following your suggestion, we will add a more comprehensive set of competitors, including newer approaches such as ADASYN, Borderline SMOTE. We will also expand simulation settings to include scenarios with larger sample size and multi-class classification, where the performance of QuanDA is expected to be less competitive.\n\n**Comparison to kernel SVM**\n\nWe have included kernel SVM in our simulation studies for additional comparison. To better accommodate class imbalance, we incorporated class-specific weights into the SVM objective: the weight for the majority class was set to $1/\\pi_0$, and the weight for the minority class was set to $1/\\pi_1$. The table below reports the AUC scores when $n = 400$ and $p = 10{,}000$. As shown, QuanDA outperforms the kernel SVM and all other methods. Although kernel SVM is often considered robust in high-dimensional settings, its performance can still deteriorate substantially in the presence of many noisy variables. Although methods such as KNIFE (Allen, 2013, JCGS) attempt to integrate feature selection into kernel learning, their computational cost is often prohibitive. Therefore, we continue to recommend QuanDA for HDLSS scenarios due to its simplicity while strong empirical performance.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|**KSVM**|\n|-|-|-|-|-|-|-|\n| | | |$\\pi_0=0.85$| | | |\n|$AR_2$|**0.929**|0.922|0.915|0.786|0.770|0.618|\n|$AR_5$|**0.880**|0.861|0.842|0.763|0.767|0.615|\n|$AR_7$|**0.857**|0.840|0.821|0.749|0.746|0.612|\n|$CS_2$|**0.953**|0.949|0.945|0.786|0.791|0.619|\n|$CS_5$|**0.914**|0.907|0.893|0.770|0.774|0.617|\n|$CS_7$|**0.882**|0.868|0.846|0.757|0.756|0.615|\n| | | |$\\pi_0=0.9$| | | |\n|$AR_2$|**0.869**|0.852|0.811|0.718|0.726|0.599|\n|$AR_5$|**0.839**|0.821|0.778|0.711|0.715|0.592|\n|$AR_7$|**0.947**|0.938|0.923|0.743|0.746|0.591|\n|$CS_2$|**0.884**|0.850|0.750|0.657|0.662|0.597|\n|$CS_5$|**0.906**|0.890|0.869|0.730|0.728|0.596|\n|$CS_7$|**0.872**|0.854|0.810|0.719|0.730|0.592|\n| | | |$\\pi_0=0.95$| | | |\n|$AR_2$|**0.856**|0.813|0.714|0.661|0.675|0.580|\n|$AR_5$|**0.806**|0.738|0.660|0.658|0.648|0.581|\n|$AR_7$|**0.778**|0.715|0.645|0.649|0.652|0.584|\n|$CS_2$|**0.884**|0.850|0.750|0.657|0.662|0.579|\n|$CS_5$|**0.839**|0.791|0.695|0.659|0.646|0.579|\n|$CS_7$|**0.805**|0.744|0.668|0.645|0.653|0.584|"}, {"review_id": "vq85GNDt5L", "reviewer": "Reviewer_v5Li", "summary": "The paper proposes a quantile-based discriminant analysis (QuanDA) method designed for high-dimensional imbalanced classification. Specifically, QuanDA directly models the quantile-based decision boundary tailored to the minority class to address class imbalance, while employing the elastic net penalty to enforce sparsity in the regression weights to handle high dimensionality.", "questions": "For the rebuttal, the following could strengthen the paper:\n- Include a more comprehensive evaluation on multi-class classification tasks.\n- Expand Tables 2 and S.8 to incorporate additional real-world datasets and comparisons with more recent methods, e.g., [1-4].\n- Present ablation studies for QuanDA. A key question is whether combining imbalance approaches with dimensionality reduction techniques such as PCA can achieve improved results in the HDLSS setting.\nIf my questions and concerns are well addressed, I will consider raising my score.", "limitations": "The authors have discussed the limitations in their paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The overall presentation is clear, and the progression from Bayes risk to quantile regression is reasonable.\n- The paper is easy to follow and includes open-source code.\n- The authors provide theoretical analyses to support the capability of QuanDA in handling the high-dimensional setting.\nWeaknesses:\n- The method seems primarily limited to binary classification. Although the authors claim that it can be extended to multi-class classification via the one-vs-one strategy, the experimental evidence in Table S.12 is inadequate, as it relies solely on synthetic data and two simple approaches (not designed for imbalance).\n- The paper conducts experiments on 5 comparison methods, however, these methods are relatively old. More recent approaches, such as [1-4], should be considered.\n- The paper conducts experiments on 7 datasets, however, this is still insufficient for a comprehensive evaluation. More benchmarks should be included to thoroughly evaluate the proposed method, encompassing balanced and imbalanced, low- and high-dimensional, small- and large-scale datasets.\n- Although the paper includes many metrics such as AUC, F1, and G-mean, these are presented solely as statistical results. I suggest the authors to incorporate more visualizations, as they are crucial for enhancing reader comprehension and facilitating new insights.\n\n[1] He, Haibo, Yang Bai, Edwardo A. Garcia, and Shutao Li. \"ADASYN: Adaptive synthetic sampling approach for imbalanced learning.\" In 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), pp. 1322-1328. 2008.\n[2] Xu, Lei, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. \"Modeling tabular data using conditional gan.\" Advances in neural information processing systems 32 (2019).\n[3] Sun, Yi, Lijun Cai, Bo Liao, Wen Zhu, and Junlin Xu. \"A robust oversampling approach for class imbalance problem with small disjuncts.\" IEEE Transactions on Knowledge and Data Engineering 35, no. 6 (2022): 5550-5562.\n[4] Sun, Hao, Jianping Li, and Xiaoqian Zhu. \"A Novel Expandable Borderline Smote Oversampling Method for Class Imbalance Problem.\" IEEE Transactions on Knowledge and Data Engineering (2025).", "was_revised": true, "final_justification": "As the authors have discussed, the problem of binary classification in the HDLSS setting has important applications and is indeed worthy of investigation. Although the empirical experiments on application tasks are somewhat limited, the proposed method is interesting, and the theoretical analysis is thorough. Therefore, I will raise my score to 4.", "rebuttal": "Thank you for your feedback and positive comments on presentation, reasonable idea, open-source code, and theory. \n\n**Include a more comprehensive evaluation on multi-class classification tasks.**\nOur method is primarily designed for binary imbalanced classification in the HDLSS setting. The connection between Bayes risk and quantiles emerges only in the binary case. However, imbalanced multi-class classification under HDLSS conditions presents even greater challenges, and practical tools that are specifically designed for this case remain very limited. As such, we adopt a one-vs-one strategy as a pragmatic extension of QuanDA for practitioners and present some preliminary results.\n\nFollowing your suggestion, we conducted experiments on real-world data. We compared QuanDA with MSDA, Random Forest, SMOTE, and XGBoost. The data GDS1615 (Mai and Zou, 2015) contain three classes with an imbalanced ratio of 8:1:1. The table below reports the macro F1 scores for all methods. QuanDA achieves the highest scores across both datasets, highlighting its effectiveness in multiclass imbalanced classification tasks.\n\n||**QuanDA**|**msda**|**RF**|**SMOTE**|**XGBoost**|\n|GDS1615$_{(n=74,\\ p=254)}$|0.901|0.350|0.588|0.658|0.503|\n\nAlthough QuanDA via one-vs-one provides a promising solution, it only serves as a practical workaround for practitioners rather than a principled extension. A more rigorous path would be to formally extend QuanDA to the multi-class case, perhaps by drawing parallels to the extension from DSDA to MSDA. The multi-class extension is nontrivial and is beyond the scope of the current work. The theoretical justification is also challenging. We thus opt not to emphasize the multiclass setting in the current paper and reserve it for future research. \n\n**Expand Tables 2 and S.8 to incorporate additional real-world datasets and comparisons with more recent methods, e.g., [1-4]. More benchmarks should be included to thoroughly evaluate the proposed method, encompassing balanced and imbalanced, low- and high-dimensional, small- and large-scale datasets.**\n\nFollowing your suggestion, in the revised paper, we have included ADASYN (He et al., 2008) and EB-SMOTE (Sun et al., 2025). Regarding the other methods you suggested, the second method (Xu et al., 2019) is not for imbalanced classification. For the third method (Sun et al., 2022), under the HDLSS setting, we observed that the light-cone structure corresponding to each minority class sample was empty, hence the method is inapplicable. In fact, this confirms our claim that in the challenging HDLSS imbalanced scenario, overly complex methods are prone to overfitting. In contrast, QuanDA is built on a straightforward connection between Bayes risk and quantiles, thus effectively preventing overfitting.\n\nThe table below reports both AUC and PRAUC scores across all real datasets. QuanDA consistently achieves the best performance.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|**EB-SMOTE**|**ADASYN**|\n|-|-|-|-|-|-|-|-|\n| | | | |AUC| | | |\n|malaria$_{(71, 22283)}$|0.994|0.984|0.989|0.993|0.993|1.000|0.994|\n|LSVT$_{(126, 309)}$|0.928|0.901|0.898|0.884|0.884|0.888|0.892|\n|ovarian$_{(253, 15154)}$|1.000|1.000|1.000|1.000|1.000|0.932|0.931|\n|leuk$_{(72, 7128)}$|0.999|0.987|0.991|0.997|0.996|0.982|0.997|\n|prostate$_{(102, 6033)}$|0.980|0.965|0.963|0.942|0.940|0.945|0.915|\n|colon$_{(62,2000)}$|0.917|0.870|0.858|0.907|0.900|0.911|0.908|\n|breast$_{(42,22283)}$|0.973|0.926|0.944|0.891|0.895|0.920|0.895|\n| | | | |PRAUC| | | |\n|malaria$_{(71, 22283)}$|0.986|0.949|0.969|0.982|0.983|1.000|0.983|\n|LSVT$_{(126, 309)}$|0.892|0.836|0.822|0.852|0.849|0.854|0.856|\n|ovarian$_{(253, 15154)}$|1.000|1.000|0.999|0.999|0.999|0.844|0.839|\n|leuk$_{(72, 7128)}$|0.998|0.984|0.989|0.995|0.994|0.974|0.994|\n|prostate$_{(102, 6033)}$|0.976|0.958|0.952|0.901|0.899|0.912|0.818|\n|colon$_{(62,2000)}$|0.835|0.777|0.747|0.802|0.769|0.792|0.797|\n|breast$_{(42,22283)}$|0.964|0.900|0.923|0.859|0.864|0.881|0.869|\n\nTo provide a more comprehensive evaluation of QuanDA, we further conducted simulation studies where various factors (e.g., class imbalance ratio, feature dimension) can be systematically controlled. For the experiments below, we report AUC scores in the rebuttal due to space constraints and other metrics will be presented in the revised paper. We point out that QuanDA also consistently achieves the highest PRAUC scores across all settings. The following table reports the AUC scores when $n = 400$ and $p = 10{,}000$ under various imbalance ratios. QuanDA consistently achieves the best performance.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|**EB-SMOTE**|**ADASYN**|\n|-|-|-|-|-|-|-|-|\n| | | | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.929|0.922|0.915|0.786|0.770|0.705|0.783|\n|$AR_5$|0.880|0.861|0.842|0.763|0.767|0.704|0.759|\n|$AR_7$|0.857|0.840|0.821|0.749|0.746|0.686|0.748|\n|$CS_2$|0.953|0.949|0.945|0.786|0.791|0.713|0.787|\n|$CS_5$|0.914|0.907|0.893|0.770|0.774|0.701|0.771|\n|$CS_7$|0.882|0.868|0.846|0.757|0.756|0.691|0.756|\n| | | | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.923|0.908|0.890|0.723|0.730|0.669|0.728|\n|$AR_5$|0.869|0.852|0.811|0.718|0.726|0.646|0.727|\n|$AR_7$|0.839|0.821|0.778|0.711|0.715|0.650|0.710|\n|$CS_2$|0.947|0.938|0.923|0.743|0.746|0.657|0.729|\n|$CS_5$|0.906|0.890|0.869|0.730|0.728|0.661|0.740|\n|$CS_7$|0.872|0.854|0.810|0.719|0.730|0.655|0.727|\n| | | | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.856|0.813|0.714|0.661|0.675|0.604|0.659|\n|$AR_5$|0.806|0.738|0.660|0.658|0.648|0.622|0.650|\n|$AR_7$|0.778|0.715|0.645|0.649|0.652|0.623|0.635|\n|$CS_2$|0.884|0.850|0.750|0.657|0.662|0.612|0.668|\n|$CS_5$|0.839|0.791|0.695|0.659|0.646|0.614|0.661|\n|$CS_7$|0.805|0.744|0.668|0.645|0.653|0.619|0.655|\n\nWe then increased the sample size to $n = 1000$, and QuanDA continued to achieve the highest AUC scores. This demonstrates QuanDA's robustness as the sample size grows.\n\n|$n=1000$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|**EB-SMOTE**|**ADASYN**|\n|-|-|-|-|-|-|-|-|\n| | | |$\\pi_0=0.85$| | | | |\n|$AR_2$|0.951|0.950|0.947|0.876|0.871|0.798|0.872|\n|$AR_5$|0.901|0.898|0.895|0.845|0.841|0.790|0.842|\n|$AR_7$|0.876|0.872|0.868|0.827|0.823|0.776|0.818|\n|$CS_2$|0.971|0.971|0.969|0.891|0.882|0.813|0.883|\n|$CS_5$|0.936|0.935|0.931|0.869|0.857|0.802|0.858|\n|$CS_7$|0.905|0.902|0.899|0.839|0.837|0.783|0.835|\n| | | |$\\pi_0=0.9$| | | | |\n|$AR_2$|0.947|0.947|0.943|0.852|0.850|0.768|0.846| \n|$AR_5$|0.897|0.892|0.889|0.821|0.824|0.758|0.819|\n|$AR_7$|0.872|0.866|0.863|0.814|0.803|0.756|0.802|\n|$CS_2$|0.968|0.969|0.965|0.868|0.859|0.776|0.859|\n|$CS_5$|0.932|0.930|0.927|0.842|0.840|0.752|0.833|\n|$CS_7$|0.900|0.897|0.895|0.826|0.816|0.742|0.814|\n| | | |$\\pi_0=0.95$| | | | |\n|$AR_2$|0.936|0.933|0.924|0.790|0.770|0.709|0.774|\n|$AR_5$|0.887|0.877|0.865|0.759|0.764|0.690|0.759|\n|$AR_7$|0.864|0.849|0.837|0.762|0.743|0.699|0.754|\n|$CS_2$|0.959|0.959|0.952|0.788|0.786|0.674|0.791|\n|$CS_5$|0.920|0.917|0.904|0.775|0.768|0.703|0.778|\n|$CS_7$|0.889|0.876|0.868|0.765|0.756|0.682|0.761|\n\nFurthermore, we considered a balanced ($\\pi_0 = 0.5$) simulation setting with $n = 400$ and $p = 10{,}000$. Even in this balanced scenario, QuanDA continues to achieve the best performance, while the advantage is not as evident as the balanced case.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|**EB-SMOTE**|**ADASYN**|\n|-|-|-|-|-|-|-|-|\n|$AR_2$|0.940|0.938|0.938|0.836|0.839|0.804|0.833|\n|$AR_5$|0.889|0.883|0.883|0.810|0.810|0.783|0.821|\n|$AR_7$|0.864|0.857|0.857|0.792|0.799|0.764|0.782|\n|$CS_2$|0.963|0.960|0.962|0.847|0.854|0.815|0.847|\n|$CS_5$|0.924|0.922|0.921|0.827|0.827|0.792|0.830|\n|$CS_7$|0.891|0.887|0.887|0.808|0.808|0.787|0.790|\n\nLast, we would like to emphasize that our work specifically targets imbalanced classification in the HDLSS setting. It is particularly challenging in practice, where many complex models tend to fail. The main contribution of this paper is that QuanDA offers a simple yet effective structure that captures essential information while avoiding overfitting. In settings with low dimensionality, sufficiently large sample sizes, and more balanced classes, practitioners indeed have a wide range of complex models to choose from. However, these models often underperform in HDLSS, where QuanDA plays its unique role.\n\n**Present ablation studies for QuanDA. A key question is whether combining imbalance approaches with dimensionality reduction techniques such as PCA can achieve improved results in the HDLSS setting.**\n\nOur findings suggest the answer is NO. We conducted simulation studies incorporating PCA as a preprocessing step before classification. As shown in the table below, the performance of all methods declined, although QuanDA remained the best among them.\n\nThe primary reason is that PCA is an unsupervised technique and does not utilize class labels when reducing dimensionality. For challenging imbalanced HDLSS classification, this can lead to significant information loss, particularly for the minority class. A more effective strategy, as employed by QuanDA, is to use supervised approaches to apply sparse penalties (e.g., lasso) to perform feature selection directly within the classification framework.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.589|0.531|0.523|0.535|0.545|\n|$AR_5$|0.598|0.539|0.533|0.538|0.539|\n|$AR_7$|0.595|0.546|0.526|0.528|0.533|\n|$CS_2$|0.590|0.536|0.520|0.539|0.538|\n|$CS_5$|0.593|0.531|0.532|0.535|0.545|\n|$CS_7$|0.596|0.544|0.533|0.538|0.541|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.596|0.538|0.525|0.549|0.546|\n|$AR_5$|0.608|0.534|0.532|0.548|0.546|\n|$AR_7$|0.606|0.538|0.518|0.552|0.556|\n|$CS_2$|0.597|0.529|0.524|0.550|0.542|\n|$CS_5$|0.605|0.531|0.527|0.545|0.548|\n|$CS_7$|0.605|0.538|0.536|0.535|0.557|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.622|0.550|0.535|0.568|0.557|\n|$AR_5$|0.617|0.565|0.532|0.559|0.544|\n|$AR_7$|0.622|0.544|0.550|0.555|0.555|\n|$CS_2$|0.625|0.549|0.554|0.558|0.561|\n|$CS_5$|0.620|0.553|0.544|0.557|0.558|\n|$CS_7$|0.621|0.556|0.540|0.566|0.556|"}, {"review_id": "cH4U4uQdlR", "reviewer": "Reviewer_TFuB", "summary": "This paper addresses the challenge of binary classification under high-dimensional, low-sample-size settings with severe class imbalance. The author proposes a novel method, QuanDA, which leverages quantile regression to construct a classifier aligned with a weighted Bayes decision rule. By selecting an appropriate quantile level, the method naturally incorporates information about class imbalance.", "questions": "1.How does your method compare with more recent approaches specifically designed for imbalanced classification under few-shot settings?  \n2.What would the performance be if feature selection were applied to Random Forest and SMOTE methods?  \n3.Given that QuanDA involves multiple nested loops, is the training process computationally intensive?  \n4.In your simulation studies, how does the performance of QuanDA compare to other methods under varying levels of sparsity?  \n5.Under your experimental setting, what is the performance of advanced sample synthesis techniques tailored for high-dimensional data, such as Borderline-SMOTE?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:  \nS1: Theoretical foundations are solid. The paper demonstrates the correspondence between Bayes classification rules and quantile functions, effectively reformulating the classification task as a quantile estimation problem.  \nS2: The proposed method outperforms conventional approaches on both simulated and multiple benchmark datasets.  \nS3: The authors convert discrete labels into continuous variables, enabling the use of quantile regression.  \n\nWeaknesses:   \nW1: The experimental comparisons are limited to classical methods, without including more recent approaches such as \"Robust High-Dimensional Classification From Few Positive Examples.\"  \nW2: While L1 regularization is applied to QuanDA, DSDA, and logistic regression, no feature selection is conducted for methods such as SMOTE and Random Forest, making the comparison less fair.  \nW3: In the simulation study, the signal is restricted to the first five features, with the rest being noise. The paper does not investigate performance under varying sparsity levels (e.g., 10 or 20 informative features).  \nW4: SMOTE is based on nearest neighbor synthesis, but under the high-dimensional and sparse setting, nearest neighbor methods become less meaningful. It would be more appropriate to compare with improved variants such as Borderline-SMOTE.   \nW5: Algorithm 1 indicates that QuanDA involves multiple nested loops, potentially leading to high training time. It is recommended to report and compare training time with other methods.", "was_revised": true, "final_justification": "I recommend Borderline reject. As stated in my comment to the authors' response, several concerns still remain.", "rebuttal": "**How does your method compare with more recent approaches specifically designed for imbalanced classification under few-shot settings? (W1/Q1)**\n\nWe compared our method, QuanDA, with DIRECT, a recent method proposed in \"Robust High-Dimensional Classification From Few Positive Examples.\" The results, summarized in the table below for the setting with $n = 400$ and $p = 10{,}000$, demonstrate that QuanDA consistently outperforms DIRECT across various settings. \n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|**DIRECT**|\n|-|-|-|-|-|-|-|\n| | | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.929|0.922|0.915|0.786|0.770|0.614|\n|$AR_5$|0.880|0.861|0.842|0.763|0.767|0.613|\n|$AR_7$|0.857|0.840|0.821|0.749|0.746|0.612|\n|$CS_2$|0.953|0.949|0.945|0.786|0.791|0.615|\n|$CS_5$|0.914|0.907|0.893|0.770|0.774|0.614|\n|$CS_7$|0.882|0.868|0.846|0.757|0.756|0.613|\n| | | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.923|0.908|0.890|0.723|0.730|0.594|\n|$AR_5$|0.869|0.852|0.811|0.718|0.726|0.593|\n|$AR_7$|0.839|0.821|0.778|0.711|0.715|0.592|\n|$CS_2$|0.947|0.938|0.923|0.743|0.746|0.595|\n|$CS_5$|0.906|0.890|0.869|0.730|0.728|0.594|\n|$CS_7$|0.872|0.854|0.810|0.719|0.730|0.593|\n| | | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.856|0.813|0.714|0.661|0.675|0.557|\n|$AR_5$|0.806|0.738|0.660|0.658|0.648|0.556|\n|$AR_7$|0.778|0.715|0.645|0.649|0.652|0.556|\n|$CS_2$|0.884|0.850|0.750|0.657|0.662|0.558|\n|$CS_5$|0.839|0.791|0.695|0.659|0.646|0.557|\n|$CS_7$|0.805|0.744|0.668|0.645|0.653|0.556|\n\n**What would the performance be if feature selection were applied to Random Forest and SMOTE methods? (W2/Q2)**\n\nQuanDA performs feature selection via a lasso penalty, making it well-suited for high-dimensional, low-sample-size (HDLSS) settings compared to methods like Random Forest and SMOTE, which do not incorporate feature selection. To further assess the effect of dimensionality reduction, we conducted simulation studies using PCA on all the methods prior to classification. We observed that even after PCA, QuanDA continues to outperform the competing methods. The primary reason is that PCA is an unsupervised technique and does not utilize class labels when reducing dimensionality. In the context of challenging imbalanced HDLSS classification, this can lead to significant information loss, particularly for the minority class. A more effective strategy, as employed by QuanDA and DSDA, is to use supervised approaches that apply sparsity-inducing penalties (e.g., lasso) to perform feature selection directly within the classification framework.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.589|0.531|0.523|0.535|0.545|\n|$AR_5$|0.598|0.539|0.533|0.538|0.539|\n|$AR_7$|0.595|0.546|0.526|0.528|0.533|\n|$CS_2$|0.590|0.536|0.520|0.539|0.538|\n|$CS_5$|0.593|0.531|0.532|0.535|0.545|\n|$CS_7$|0.596|0.544|0.533|0.538|0.541|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.596|0.538|0.525|0.549|0.546|\n|$AR_5$|0.608|0.534|0.532|0.548|0.546|\n|$AR_7$|0.606|0.538|0.518|0.552|0.556|\n|$CS_2$|0.597|0.529|0.524|0.550|0.542|\n|$CS_5$|0.605|0.531|0.527|0.545|0.548|\n|$CS_7$|0.605|0.538|0.536|0.535|0.557|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.622|0.550|0.535|0.568|0.557|\n|$AR_5$|0.617|0.565|0.532|0.559|0.544|\n|$AR_7$|0.622|0.544|0.550|0.555|0.555|\n|$CS_2$|0.625|0.549|0.554|0.558|0.561|\n|$CS_5$|0.620|0.553|0.544|0.557|0.558|\n|$CS_7$|0.621|0.556|0.540|0.566|0.556|\n\n**Given that QuanDA involves multiple nested loops, is the training process computationally intensive? (W5/Q3)**\n\nThe table below reports the total computation time, including both training and tuning, for five imbalanced classification methods evaluated on simulated data ($\\pi_0 = 0.95$, $n = 400$, $p = 10{,}000$). We observe that QuanDA is approximately 10 times slower than alternatives such as DSDA and logistic regression. The primary computational cost comes from the jittering step, which is repeated 10 times to mitigate randomness and stabilize the results. For each individual jittering step, QuanDA runs at a similar speed, or even slightly faster, than DSDA and logistic regression. Even with 10 jittering steps, the entire computation for QuanDA completes within about ten minutes. Given the challenging nature of imbalanced classification in the HDLSS setting and QuanDA’s strong empirical performance, we believe the additional computational effort is worthwhile.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n|$AR_2$|760.88|67.98|89.15|22.80|57.86|\n|$AR_5$|691.75|63.87|83.60|22.34|57.68|\n|$AR_7$|689.90|62.30|84.16|23.35|58.73|\n|$CS_2$|687.76|69.31|92.41|23.01|57.12|\n|$CS_5$|891.53|85.40|104.00|25.66|63.94|\n|$CS_7$|857.99|81.86|99.29|25.27|67.83|\n\n**In your simulation studies, how does the performance of QuanDA compare to other methods under varying levels of sparsity? (W3/Q4)**\n\nFollowing your great suggestion, we evaluated all methods under different levels of sparsity by varying the number of informative features while keeping $n = 400$ and $p = 10{,}000$. Specifically, we considered settings where the number of informative features is $p_0 = 10$ and $p_0 = 20$. The tables below report the AUC scores under these conditions. QuanDA consistently outperforms the competing methods across both sparsity levels.\n\n|$p_0=10$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.953|0.944|0.940|0.898|0.897|\n|$AR_5$|0.894|0.877|0.870|0.851|0.857|\n|$AR_7$|0.866|0.848|0.842|0.830|0.827|\n|$CS_2$|0.989|0.987|0.986|0.921|0.927|\n|$CS_5$|0.963|0.956|0.952|0.903|0.905|\n|$CS_7$|0.928|0.918|0.910|0.872|0.870|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.950|0.936|0.932|0.869|0.875|\n|$AR_5$|0.892|0.865|0.854|0.828|0.831|\n|$AR_7$|0.862|0.830|0.825|0.813|0.809|\n|$CS_2$|0.986|0.983|0.982|0.873|0.890|\n|$CS_5$|0.959|0.950|0.946|0.866|0.873|\n|$CS_7$|0.924|0.910|0.899|0.847|0.845|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.912|0.873|0.810|0.769|0.762|\n|$AR_5$|0.854|0.796|0.731|0.751|0.754|\n|$AR_7$|0.821|0.761|0.687|0.732|0.726|\n|$CS_2$|0.956|0.938|0.887|0.783|0.777|\n|$CS_5$|0.916|0.885|0.817|0.761|0.776|\n|$CS_7$|0.878|0.833|0.766|0.761|0.760|\n\n|$p_0=20$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.967|0.951|0.957|0.955|0.953|\n|$AR_5$|0.902|0.878|0.882|0.891|0.891|\n|$AR_7$|0.871|0.847|0.849|0.861|0.856|\n|$CS_2$|0.999|0.999|0.999|0.991|0.991|\n|$CS_5$|0.991|0.987|0.989|0.977|0.979|\n|$CS_7$|0.969|0.959|0.961|0.954|0.954|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.964|0.952|0.951|0.940|0.942|\n|$AR_5$|0.902|0.880|0.873|0.886|0.879|\n|$AR_7$|0.870|0.844|0.832|0.850|0.847|\n|$CS_2$|0.999|0.998|0.998|0.977|0.980|\n|$CS_5$|0.989|0.984|0.986|0.963|0.967|\n|$CS_7$|0.966|0.957|0.956|0.935|0.940|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.946|0.907|0.853|0.888|0.884|\n|$AR_5$|0.872|0.827|0.769|0.820|0.812|\n|$AR_7$|0.834|0.782|0.706|0.797|0.787|\n|$CS_2$|0.993|0.979|0.959|0.912|0.912|\n|$CS_5$|0.976|0.951|0.921|0.904|0.901|\n|$CS_7$|0.942|0.917|0.848|0.880|0.878|\n\n**Under your experimental setting, what is the performance of advanced sample synthesis techniques tailored for high-dimensional data, such as Borderline-SMOTE? (W4/Q5)**\n\nWe have incorporated Borderline-SMOTE into our comparison to assess its performance under the HDLSS setting. The table below reports AUC scores for all methods when $n = 400$ and $p = 10{,}000$ across various imbalance ratios. We observe that QuanDA consistently outperforms Borderline-SMOTE and other competing methods.\n\nWe believe the main reason QuanDA outperforms Borderline-SMOTE is its simplicity. In the challenging HDLSS imbalanced classification setting, QuanDA directly leverages the connection between Bayes risk and quantiles, which yields a simple model to perform effectively. In contrast, Borderline-SMOTE is a more complex approach that typically requires a much larger sample size to achieve its full potential, and thus may not perform well under HDLSS conditions.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|**Borderline-SMOTE**|\n|-|-|-|-|-|-|-|\n| | | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.929|0.922|0.915|0.786|0.770|0.749|\n|$AR_5$|0.880|0.861|0.842|0.763|0.767|0.734|\n|$AR_7$|0.857|0.840|0.821|0.749|0.746|0.732|\n|$CS_2$|0.953|0.949|0.945|0.786|0.791|0.762|\n|$CS_5$|0.914|0.907|0.893|0.770|0.774|0.751|\n|$CS_7$|0.882|0.868|0.846|0.757|0.756|0.738|\n| | | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.923|0.908|0.890|0.723|0.730|0.685|\n|$AR_5$|0.869|0.852|0.811|0.718|0.726|0.704|\n|$AR_7$|0.839|0.821|0.778|0.711|0.715|0.694|\n|$CS_2$|0.947|0.938|0.923|0.743|0.746|0.707|\n|$CS_5$|0.906|0.890|0.869|0.730|0.728|0.692|\n|$CS_7$|0.872|0.854|0.810|0.719|0.730|0.686|\n| | | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.856|0.813|0.714|0.661|0.675|0.672|\n|$AR_5$|0.806|0.738|0.660|0.658|0.648|0.699|\n|$AR_7$|0.778|0.715|0.645|0.649|0.652|0.635|\n|$CS_2$|0.884|0.850|0.750|0.657|0.662|0.691|\n|$CS_5$|0.839|0.791|0.695|0.659|0.646|0.673|\n|$CS_7$|0.805|0.744|0.668|0.645|0.653|0.696|"}, {"review_id": "xO1K7zZFrL", "reviewer": "Reviewer_X6ME", "summary": "This paper proposes QuanDA, a novel classification method to handles the challenges of low-sample-size and high-dimensional settings. The core innovation lies in establishing a theoretical connection with quantile regression, which naturally accounts for class imbalance through the choice of appropriate quantile levels. Theoretical analyses are provided to prove the effectiveness in ultra-high dimensions for binary classification. Extensive experiments on seven high-dimensional data demonstrate the superiority over previous methods like cost-sensitive SVM, random forest, SMOTE, particularly under high imbalance conditions.", "questions": "I do have some questions, see those questions in the weakness part.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "[Strengths]\n1. Enough Novelty: The linkage between quantile regression and the weighted Bayes risk is insightful and well-articulated. The jittering step elegantly resolves the discontinuity issue in binary response variables, enabling stable quantile optimization. This theoretical contribution is significant for imbalanced learning.\n\n2. Excellent performance. This paper demonstrates excellent performance in seven high-dimensional data demonstrate compared with other robust counterparts. This clearly shows the effectiveness and advantages of the proposed method.\n\n3. Rich Experiments and Analysis. The authors provide extensive experiments and analysis for the proposed method. I appreciate this. The experimental analysis with various ablation studies allows a better understanding of each module and overall performance.\n\n4. Good writing and organization. This paper is well-written and organized. Each section has a clear motivation. It’s easy to follow the ideas.\n\n[Weaknesses]\n1. Lacking Efficiency Comparison: The authors should provide the efficiency comparison to show the balance between extended modules and computation costs.\n\n2. More Comprehensive Ablation: It’s better to show the generality of the proposed method with various values of n/p in the simulated data.", "was_revised": true, "final_justification": "The answers provided by the authors have addressed my previous concerns. Therefore, I keep my initial rating.", "rebuttal": "Thank you for your positive comments about novelty, performance, experiments, and writing of our work. We also appreciate your thoughtful suggestions. \n\n**Lacking Efficiency Comparison: The authors should provide the efficiency comparison to show the balance between extended modules and computation costs.**\n\nThe table below reports the total computation time, including both training and tuning, for five imbalanced classification methods evaluated on simulated data ($\\pi_0 = 0.95$, $n = 400$, $p = 10{,}000$). We observe that QuanDA is indeed slower than alternatives such as DSDA and logistic regression, with a runtime approximately 10 times longer. However, the primary computational cost comes from the jittering step, which is repeated 10 times to mitigate randomness and stabilize the results. For each individual jittering step, QuanDA runs at a similar speed, or even slightly faster, than DSDA and logistic regression. Even with 10 jittering steps, the entire computation for QuanDA completes within about ten minutes. Given the challenging nature of imbalanced classification in the HDLSS setting and QuanDA’s strong empirical performance, we believe the additional computational effort is worthwhile.\n\n||**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n|$AR_2$|760.88|67.98|89.15|22.80|57.86|\n|$AR_5$|691.75|63.87|83.60|22.34|57.68|\n|$AR_7$|689.90|62.30|84.16|23.35|58.73|\n|$CS_2$|687.76|69.31|92.41|23.01|57.12|\n|$CS_5$|891.53|85.40|104.00|25.66|63.94|\n|$CS_7$|857.99|81.86|99.29|25.27|67.83|\n\n**More Comprehensive Ablation: It’s better to show the generality of the proposed method with various values of n/p in the simulated data.**\n\nMany thanks for the great suggestion. We conducted additional experiments to evaluate the performance of all methods under varying sample sizes. Specifically, we fixed the feature dimension at $p = 10{,}000$ and varied the sample size $n$ from 200 to 1000. We do not consider higher values of $n$ because QuanDA is designed specifically for HDLSS setting. Practitioners have a wider choice of complex models when sample size is sufficiently large. The AUC scores reported in the tables below show that QuanDA consistently achieves the highest performance across all settings.\n\n|$n=200$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.883|0.831|0.811|0.689|0.707|\n|$AR_5$|0.837|0.776|0.747|0.669|0.685|\n|$AR_7$|0.813|0.747|0.734|0.666|0.684|\n|$CS_2$|0.909|0.864|0.851|0.680|0.690|\n|$CS_5$|0.866|0.812|0.779|0.680|0.677|\n|$CS_7$|0.836|0.776|0.747|0.670|0.674|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.829|0.775|0.712|0.662|0.671|\n|$AR_5$|0.791|0.721|0.663|0.638|0.656|\n|$AR_7$|0.770|0.698|0.653|0.645|0.659|\n|$CS_2$|0.845|0.799|0.755|0.653|0.664|\n|$CS_5$|0.812|0.754|0.689|0.651|0.653|\n|$CS_7$|0.785|0.725|0.682|0.661|0.653|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.804|0.664|0.636|0.657|0.657|\n|$AR_5$|0.787|0.637|0.625|0.639|0.678|\n|$AR_7$|0.781|0.629|0.618|0.647|0.672|\n|$CS_2$|0.807|0.686|0.641|0.633|0.651|\n|$CS_5$|0.801|0.662|0.644|0.640|0.638|\n|$CS_7$|0.794|0.636|0.623|0.621|0.658|\n\n|$n=600$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.939|0.937|0.934|0.821|0.830|\n|$AR_5$|0.886|0.879|0.878|0.796|0.799|\n|$AR_7$|0.860|0.851|0.851|0.780|0.780|\n|$CS_2$|0.962|0.962|0.961|0.832|0.835|\n|$CS_5$|0.923|0.919|0.917|0.817|0.813|\n|$CS_7$|0.888|0.883|0.882|0.796|0.798|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.936|0.932|0.923|0.793|0.780|\n|$AR_5$|0.885|0.870|0.854|0.754|0.764|\n|$AR_7$|0.857|0.841|0.821|0.756|0.761|\n|$CS_2$|0.959|0.958|0.953|0.802|0.800|\n|$CS_5$|0.920|0.915|0.905|0.775|0.781|\n|$CS_7$|0.886|0.876|0.857|0.764|0.763|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.904|0.887|0.867|0.697|0.699|\n|$AR_5$|0.850|0.830|0.766|0.691|0.689|\n|$AR_7$|0.818|0.783|0.725|0.679|0.700|\n|$CS_2$|0.931|0.923|0.901|0.688|0.710|\n|$CS_5$|0.885|0.874|0.835|0.703|0.702|\n|$CS_7$|0.847|0.835|0.764|0.700|0.703|\n\n|$n=800$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.940|0.940|0.937|0.847|0.853|\n|$AR_5$|0.887|0.882|0.881|0.819|0.824|\n|$AR_7$|0.862|0.854|0.853|0.806|0.805|\n|$CS_2$|0.963|0.964|0.962|0.864|0.863|\n|$CS_5$|0.924|0.923|0.919|0.841|0.835|\n|$CS_7$|0.890|0.887|0.886|0.815|0.817|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.934|0.932|0.926|0.825|0.821|\n|$AR_5$|0.882|0.874|0.867|0.796|0.795|\n|$AR_7$|0.857|0.848|0.841|0.784|0.777|\n|$CS_2$|0.959|0.959|0.955|0.842|0.833|\n|$CS_5$|0.919|0.914|0.912|0.814|0.801|\n|$CS_7$|0.885|0.880|0.873|0.789|0.790|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.924|0.920|0.897|0.760|0.747|\n|$AR_5$|0.865|0.855|0.823|0.733|0.728|\n|$AR_7$|0.836|0.826|0.792|0.730|0.724|\n|$CS_2$|0.951|0.948|0.935|0.753|0.742|\n|$CS_5$|0.907|0.897|0.869|0.736|0.745|\n|$CS_7$|0.869|0.861|0.828|0.734|0.729|\n\n|$n=1000$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.951|0.950|0.947|0.876|0.871|\n|$AR_5$|0.901|0.898|0.895|0.845|0.841|\n|$AR_7$|0.876|0.872|0.868|0.827|0.823|\n|$CS_2$|0.971|0.971|0.969|0.891|0.882|\n|$CS_5$|0.936|0.935|0.931|0.869|0.857|\n|$CS_7$|0.905|0.902|0.899|0.839|0.837|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.947|0.947|0.943|0.852|0.850|\n|$AR_5$|0.897|0.892|0.889|0.821|0.824|\n|$AR_7$|0.872|0.866|0.863|0.814|0.803|\n|$CS_2$|0.968|0.969|0.965|0.868|0.859|\n|$CS_5$|0.932|0.930|0.927|0.842|0.840|\n|$CS_7$|0.900|0.897|0.895|0.826|0.816|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.936|0.933|0.924|0.790|0.770|\n|$AR_5$|0.887|0.877|0.865|0.759|0.764|\n|$AR_7$|0.864|0.849|0.837|0.762|0.743|\n|$CS_2$|0.959|0.959|0.952|0.788|0.786|\n|$CS_5$|0.920|0.917|0.904|0.775|0.768|\n|$CS_7$|0.889|0.876|0.868|0.765|0.756|\n\nWe then fixed the sample size at $n = 400$ and varied the feature dimension $p$ from 5,000 to 20,000. Results for $p = 5{,}000$ and $p = 10{,}000$ are presented in the main paper. The AUC scores reported in the tables below demonstrate that QuanDA consistently achieves the highest performance across all settings, highlighting its robustness to increasing dimensionality.\n\n|$p=15000$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.934|0.930|0.926|0.756|0.765|\n|$AR_5$|0.886|0.873|0.861|0.731|0.743|\n|$AR_7$|0.862|0.849|0.830|0.732|0.746|\n|$CS_2$|0.958|0.955|0.952|0.751|0.760|\n|$CS_5$|0.920|0.913|0.909|0.754|0.762|\n|$CS_7$|0.888|0.876|0.864|0.748|0.750|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.923|0.918|0.894|0.732|0.734|\n|$AR_5$|0.873|0.856|0.828|0.723|0.700|\n|$AR_7$|0.846|0.832|0.787|0.701|0.721|\n|$CS_2$|0.946|0.945|0.932|0.739|0.729|\n|$CS_5$|0.905|0.900|0.866|0.717|0.721|\n|$CS_7$|0.873|0.861|0.821|0.708|0.712|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.864|0.817|0.752|0.647|0.631|\n|$AR_5$|0.822|0.764|0.698|0.647|0.649|\n|$AR_7$|0.798|0.745|0.674|0.655|0.651|\n|$CS_2$|0.882|0.851|0.772|0.635|0.657|\n|$CS_5$|0.845|0.796|0.735|0.640|0.656|\n|$CS_7$|0.814|0.766|0.698|0.649|0.658|\n\n|$p=20000$|**QuanDA**|**dsda**|**logistic**|**RF**|**SMOTE**|\n|-|-|-|-|-|-|\n| | |$\\pi_0=0.85$| | | |\n|$AR_2$|0.935|0.928|0.923|0.736|0.746|\n|$AR_5$|0.885|0.874|0.868|0.725|0.725|\n|$AR_7$|0.862|0.853|0.834|0.712|0.711|\n|$CS_2$|0.958|0.955|0.952|0.731|0.747|\n|$CS_5$|0.919|0.913|0.907|0.719|0.728|\n|$CS_7$|0.887|0.879|0.875|0.729|0.729|\n| | |$\\pi_0=0.9$| | | |\n|$AR_2$|0.923|0.899|0.880|0.714|0.696|\n|$AR_5$|0.869|0.836|0.834|0.690|0.688|\n|$AR_7$|0.844|0.817|0.802|0.685|0.677|\n|$CS_2$|0.949|0.943|0.911|0.699|0.711|\n|$CS_5$|0.908|0.879|0.867|0.701|0.701|\n|$CS_7$|0.871|0.837|0.838|0.674|0.692|\n| | |$\\pi_0=0.95$| | | |\n|$AR_2$|0.853|0.791|0.727|0.632|0.634|\n|$AR_5$|0.803|0.710|0.665|0.623|0.631|\n|$AR_7$|0.777|0.690|0.651|0.627|0.624|\n|$CS_2$|0.871|0.818|0.768|0.623|0.630|\n|$CS_5$|0.825|0.742|0.707|0.617|0.613|"}]

Author rebuttal:

Meta-review: ### Summary
This paper studies binary classification under high-dimensional, small-sample-size setting with heavy class imbalance. The authors propose a method which they call QuanDA, which builds upon a connection with quantile regression to construct a classifier which acounts for class imbalance via appropriate selection of quantile levels.
### Strengths
- The idea of exploiting the relation between binary classification and quantile regression is interesting.
- This paper also provides theoretical analysis on high-probability error bound of the estimator, which is summarized in Theorem 3.1.
- The proposal experimentally outperformed some conventional approaches.
### Weaknesses
- The proposal seems to be limited to binary classification.
- The experiments were limited (although the authors have added some more experimental results during the discussion phase).
### Reasons for decision
The rating/confidence scores from the four reviewers were 5/3, 4/4, 3/3, and 4/3, with the average 4 above the threshold of acceptance. The negatively-evaluating reviewer was not quite satisfied with the author response. Also, upon my own reading of the paper, I noticed several points that need to be addressed. Despite these, the reviewers valued the idea of relating binary classification with quantile regression is itself novel and interesting, as well as the solid theoretical analysis accompanying the proposal. Because of these, I would recommend acceptance, expecting that the authors would address these concerns in the final version.
### Additional points
- Jittering: Besides Reviewer ajh1's comment on jittering, I also feel that the idea of jittering introduced in section 2.2 of this paper has some resemblance with the idea of "perturb-and-MAP" proposed in [R1]. I would thus appreciate it if the authors could add some discussion on how they are related and how the former differs from the latter.
- Sparsity: Although not explicitly stated by the authors, one of the key factors affecting performance of the proposed method would be sparsity: The experimental setting was the dimension $p=10,000$ and the sample size $n=400$, and PCA-based feature selection will not work at all in such an extreme setting. Since  the discriminant feature has the dimension 5 in their setting, exploitation of sparsity would greatly help. Sparsity is, however, only briefly argued in lines 172-173 in formulating the proposal. This point should be mentioned more clearly.
- Line 124: achiev(e)able
- Displayed equation just below line 130: It seems incorrect because it takes expectation of a formula that contains conditional densities $g^\pm(X)$.
- Line 131: condition(al) density
- Line 134: I did not understand what "$w_0=\pi_1$ and $w_1=\pi_0$" mean.
- Line 162: $Q_Z$ here would be better written as $Q_{Y+U}$ to improve coherence in the notation.
- Line 176: The second appearance of $y_i=1$ should be $y_i=0$.
- Line 178: $\lambda_1\|\beta\|$ → $\lambda_1\|\beta\|_1$
- Line 180: compo(n)ent
- Line 181: $tau$ → $\tau$
- Line 209: $\mathbf{X}$ → $\mathbf{x}$
- Lines 227, 558, 569, 592: (C1-C6) → (C1)-(C6)
- Displayed equation just below line 228: $\\\|\hat{\alpha}\_\lambda-\alpha^\*\\\|\_2$ → $|\hat{\alpha}_\lambda-\alpha^\*|$
- Line 234: can readily (be) obtained
- Line 252: The simulation data (we)re generated
- Lines 256-257: The definitions of AR and CS should be exchanged.
- Tables 1, 2: Although the average scores of QuanDA were the highest, most of them seem to be insignificant: if one draws a figure with error bars with the standard errors, then those error bars would overlap. It would be nice if statistical testing be performed in order to demonstrate how significant they are.
- References:
  - Line 341: What is "GrC"?
  - Line 380: The page numbers "507-513" are missing.
  - Lines 404, 407: Capitalization of the journal titles is incorrect.
  - Line 422: The correct arXiv identifier should be provided.
  - Line 448: I am not sure about the format of the bibliographical information of this document. smote → SMOTE
  - Line 469: The volume number 16 is missing.
- Equation (6), line 559: $\alpha$ should not be in boldface.
- Proofs are not easy to read. I would suggest adding some more explanations in the development of mathematical formulas: Indicating what condition, inequality, etc. was used in each development will facilitate understanding of the reader.
- Lemma S3.1: It appears that the conditions (C4), (C5), (C6) are not used in the proof, so that it would be better to remove them from the statement of the lemma. The definition of $M_0$ should be provided in the statement, because otherwise one has to go back to the main text to see it while reading the proof.
- Second line of displayed equation just below line 564: $\zeta$ is a scalar, so that one does not have to take its transpose.
- Lemma S3.2: Again, it appears that the conditions (C5), (C6) are not used in the proof, so that it would be better to remove them from the statement of the lemma.
- Lines 603, 606: $\mathcal{C}(3,1)$ → $\mathcal{C}_{3,1}$
- Second and third lines of displayed equation just below line 618: $\hat{\delta}^\lambda$ should not be in boldface; All the norms of $\hat{\delta}^\lambda$ should be replaced with its absolute value, as $\hat{\delta}^\lambda$ is not a vector but a scalar.

[R1] George Papandreou and Alan L. Yuille, "Perturb-and-MAP random fields: Using discrete optimization to learn and sample from energy models," In International Conference on Computer Vision, pages 193-200, 2011.
Decision: Accept (poster)