A Generalized Geometric Theoretical Framework of Centroid Discriminant Analysis for Linear Classification of Multi-dimensional Data
Authors: Yue Wu
Jialin Zhao
Carlo Vittorio Cannistraci
Venue: ICLR 2026 Poster (2026)
Keywords: Linear classiﬁcation; Efficient and scalable algorithm; Geometric discriminant analysis; Centroid discriminant analysis; Nonlinear classification via kernel method
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: With the advent of the neural network era, traditional machine learning methods have increasingly been overshadowed. Nevertheless, continuing research on the role of geometry for learning in data science is crucial to envision and understand new principles behind the design of efficient machine learning. Linear classifiers are favored in certain tasks due to their reduced susceptibility to overfitting and their ability to provide interpretable decision boundaries. However, achieving both scalability and high predictive performance in linear classification remains a persistent challenge. Here, we propose a theoretical framework named geometric discriminant analysis (GDA). GDA includes the family of linear classifiers that can be expressed as a function of a centroid discriminant basis (CDB0) - the connection line between two centroids - adjusted by geometric corrections under different constraints. We demonstrate that linear discriminant analysis (LDA) is a subcase of the GDA theoretical framework, and we show its convergence to CDB0 under certain conditions. Then, based on the GDA framework, we propose an efficient linear classifier named centroid discriminant analysis (CDA) which is defined as a special case of GDA under a 2D plane geometric constraint. CDA training is initialized starting from CDB0 and involves the iterative calculation of new adjusted centroid discriminant lines whose optimal rotations on the associated 2D planes are searched via Bayesian optimization. CDA has good scalability (quadratic time complexity) which is lower than LDA and support vector machine (SVM) (cubic complexity). Results on 27 real datasets across classification tasks of standard images, medical images and chemical properties, offer empirical evidence that CDA outperforms other linear methods such as LDA, SVM and logistic regression (LR) in terms of scalability, performance and stability. Furthermore, we show that linear CDA can be generalized to nonlinear CDA via kernel method, demonstrating improvements on the linear version with tests on three challenging datasets of images and chemical data. GDA's general validity as a new theoretical framework may inspire the design of new classifiers under the definition of different geometric constraints, paving the way towards a deeper understanding of the role of geometry in learning from data.

Published as a conference paper at ICLR 2026
A GENERALIZED GEOMETRIC THEORETICAL FRAME- WORK OF CENTROID DISCRIMINANT ANALYSIS FOR LINEAR CLASSIFICATION OF MULTI-DIMENSIONAL DATA
Yue Wu Jialin Z Carlo Vittorio Cannistraci 1,2,3"
I Center for Complex Network Intelligence (CCNT) f
2 Department of Biomedical Engineering, Tsinghua University Department of Computer Science & Technology, Tsinghua University
ABSTRACT
With the advent of the neural network era, traditional machine learning methods have increasingly been overshadowed. Nevertheless, continuing research on the role of geometry for learning in data science is crucial envision and understand new principles behind the design of efficient machine learning. Linear classifiers are favored in certain tasks due to their reduced susceptibility to overfitting and their ability to provide interpretable decision boundaries. However, achieving both scalability and high predictive performance linear classification remains a persistent challenge. Here, we propose theoretical framework named geometric discriminant analysis (GDA). GDA includes the family of linear classifiers that can be expressed a function of centroid discriminant basis (CDBO)- the connection line between two centroids adjusted by geometric corrections under different constraints. We demonstrate that linear discriminant analysis (LDA) is subcase of the GDA theoretical framework, and we show its convergence to CDBO under certain conditions. Then, based on the GDA framework, we propose an efficient linear classifier named centroid discriminant analysis (CDA) which is defined as a special case of GDA under a 2D plane geometric constraint. CDA training initialized starting from CDBO and involves the iterative calculation of new adjusted centroid discriminant lines whose optimal rotations on the associated 2D planes are searched via Bayesian optimization. CDA has good scalability (quadratic time complexity) which is lower than LDA and support vector machine (SVM) (cubic complexity). Results on 27 real datasets across classification tasks of standard images, medical images and chemical propertics, offer empirical evidence that CDA outperforms other linear methods such LDA, SVM and logistic regression (LR) in terms of scalability, performance and stability. Furthermore, we show that linear CDA be generalized nonlinear CDA via kernel method, demonstrating improvements on the linear version with tests on three challenging datasets of images and chemical data. GDA's general validity may inspire the design of new classifiers under the definition of different geometric constraints, paving the way towards deeper understanding of the role of geometry in learning from data.
new theoretical framework
INTRODUCTION
Linear classifiers are often favored nonlinear models, such neural networks, for certain tasks due to their comparable performance in nigh-dimensiioal data spaces, faster training speeds, reduced tendency overfit, and greater interpretability in decision-making (Varoquaux al. (2017); Yuan et al. (2012)). Notably, linear classifiers have demonstrated performance on par with convolutional
Corresponding author: kalokagathos.agon @ gmail..com 'Research Center in Tsinghua Laboratory of Brain and Intelligence (THBI), Department of Psychological and Cognitive Seiences, Tsinghua University

---

Published as a conference paper at ICLR 2026
neural networks (CNNs) in medical classification tasks, such as predicting Alzheimer's disease from structural or functional brain MRI images (Schulz et al. (2020) Varoquaux & Cheplygina (2022))
These linear classifiers can be categorized into several types based on the principles they use to define the decision boundary or classification discriminant, as described below, where N is the number of samples, M is the number of features and k denotes iteration term:
The minimum distance classifier (MDC) (Duda et al. (2001) which is prototype-based classifier that assigns points according to the perpendicular bisector boundary between the centroids of two groups. This classifier has O M) training time complexity.
Fisher's linear discriminant analysis (LDA,, specifically refer Fisher's LDA in this study) is a variance-based classifier which can be trained in cubic time complexity O M2 M While faster implementations like spectral regression discriminant analysis (SRDA) (Cai et al. )2088) claim lower training time complexity, their efficiency depends on specific conditions, such a sufficiently small iterative term and sparsity in the data. These constraints limit SRDA's applicability in real-world classification tasks.
Support vector machine (SVM) (Cortes & Vapnik (1995) with linear kernel is maximum- margin classifier, which has a training time complexity of ( Fast implementations include Liblinear (referred as fast SVM) and SVM-SGD, which use coordinate descent and stochastic gradients respectively, achieving quasi- quaariiiac time complexity MK.
Perceptron (Minsky & Papert (169)) is a misclassification trigggerd ruled-based classifier. Its training time complexity is O(NN.M)
Logistic regression (LR) (Panda et al. (2022) ) is a statistics- classifier. It can be trained using either maximum likelihood estimation (MLE) or iteratively reweighted least squares, with time complexity of O(N + M and (()M²) respectively, and with O M) using the same coordinate descent technique in Liblinear.
Among linear classifiers, MDC offers the lowest training time complexity but suffers from limited performance due to its overly simplified decision boundary. Widely used methods such as I.DA and SVM are often favored for their strong predictive capabilities. However, these methods can be computationally expensive, particularly for large-scale datasets. Hence, achieving both high scalability and strong predictive performance remains challenging tradeoff, highlighting the need for new approaches that balance these competing demands.
To address this challenge, this paper makes the following 3 key contributions:
A geometric theoretical framework for classifiers:
(See Appendix. A. for explanation of the term theoretical framework.) This study introduces geometric framework, geometric discriminant analysis (GDA), to unify certain linear classifiers under a common theoretical model. GDA leverages a special type of centroid discriminant basis (CDBO), a vector connecting the centroids of two classes, which serves as the foundation for constructing classifier decision boundaries. The GDA framework adjusts the CDBO through geometric corrections under various constraints, enabling the derivation of classifiers with desirable properties. Notably, we show that: MDC is a special case of GDA, where geometric corrections are not applied to CDBO; linear discriminant analysis (LDA) is special case of GDA, where the CDBO is corrected by maximizing the projection variance ratio.
A high-performance and scalable linear geometric classifier: Building upon the GDA framework, we propose centroid discriminant analysis (CDA), a novel geometric classifier that iteratively adjusts the CDB through performance-dependeer rotations on 2D planes. These rotations are optimized via Bayesian optimization, enhancing the decision boundary's 's adaptability efficiently. CDA achieves lower training time complexity (quadratic) and is more efficient than LDA and SVM. Experimental evaluations on 27 real-world datasets of standard images, medical images and chemical property data, reveal that CDA consistently outperforms LDA, SVM and LR in predictive performance, scalability, and stability.
Nonlinear geometric classification via kernel method: For complex data where linear models are not enough, CDA supports nonlinear classification via kernel method. We demonstrated with challenging image and chemical datasets that kernel CDA improved over linear CDA and outperformed kernel SVM and kernel LDA. While the core CDA algorithm is efficient, the kernelized version inherits the standard computational bottleneck
2

---

Published as a conference paper at ICLR 2026
of kernel matrix construction. Nonetheless, while kernel CDA offers greater expressiveness and improved capability, linear CDA remains highly valuable for real-world tasks due to its superior training efficiency, interpretabilityy and reduced risk of overfitting.
More importantly, we emphasize that CDA not only achieves robust predictive performance but also offers superior computational efficiency. Unlike traditional methods such as LDA and SVM, which typically exhibit cubic time complexity. CDA operates with quadratic complexity, resulting in significantly faster runtimes in practice. These advantages make CDA particularly attractive for real-world applications, where scalability, interpretability, and efficiency classifiers remain widely used across scientific domains for their transparency and speed, CDA represents valuable advancement for practitioners seeking reliable and computationally lightweight solutions. Lastly, the GDA theoretical framework, from which CDA is derived, may inspire new classifiers under the definition of different geometric constraints.
essential. As linear
GEOMETRIC DISCRIMINANT ANALYSIS (GDA)
In this study, we propose a generalized geometric theoretical framework for centroid-based linear classifiers. In geometry, the generalized definition of centroid is the weighted average of points. For binary classification problem, training linear classifier involves finding a discriminant (perpendicular to the decision boundary) and a bias. In GDA, we focus on the centroid discriminant basis (CDB) which is defined as the unit vector from the centroid of negative class to positive class. Specifically, we focus particular discriminant termed CDBO, which is constructed from centroids with uniform sample weights. GDA theoretical framework incorporates all the classifiers whose classification discriminant is CDB0 adjusted by geometric corrections on CDBO, which is described in details in the following using an instance with LDA. Morcover, in the GDA theoretical framework, the classifier discriminants are ccaling-invariant which is explained in Appendix, A.2 Thus, throughout, Y * denotes generic constant independent of the variable of interest. The bias is also excluded from the analysis since it can be naturally determined after the discriminant is fixed (see Appendix. A..).
Without loss of generality, assume -oo-dimensional space (see Appendix. E for any-dimensional proofs). We derive the GDA theoretical framework referred to simply as LDA) and includes it under a certain geometric constraint. In LDA, the linear discriminant (L.) derived as the maximization of between-class variance to within-class variance:
generalization of Fisher's LDA (hereafter
of (w v - wi' vo) (w" T VI w vn) S
o² W''N w E w (20 Σ
where vo and V1 are the means of negative and positive class, Eg and E, are their covariance matrices, w is the projection coefficient. The maximum is obtained when NEw 11 Mu (Fisher (1936)), or N EYYLLD = 111 Py. where WLD is the normalized LDA discriminant in the GDA theoretical framework with normalizing constant 7, N is the total number of samples. Σ is the sum of within-class covariance matrices of each class Eg + Σx:
00
o02 nyy
o
o12
o
Σ
f o02 ya
zz
Ey
2y yv.
o1 ya
o yr
yy
vy
And the inverse of the covariance matrix is:
Σ
adj(2) Σ
ay
o2
0
by
2
1
Σ
22 2 vy
2
o2 yz
Zx
vy
2 By
o
ye
2
where adj(E) the adjugate of Σ. Let Au denote the unit vector of Av, then 1 10 714
HO) = 7Ap = A/,, Apy) T and WLD = 7 (M1 Mo) = >
[1)
Mo). Further,
2 o
70 vy
zy
App AHlel
[
111 1
Auu AHy
di
WLD
2 02
0 2 02
yx
a²
=
a²
2 Nv
1
by
<(AArr Apy,] T + Ceorection Apts Appy)
3

---

Published as a conference paper at ICLR 2026
where correction is correction matrix that acts as the second-order term associated with the sum of covariance matrices. Since WCDBO is the unit vector constructed from centroids with uniform sample weights (i.e., arithmetic means), it be written as WCDDO matrix, o o and let Cay = / oy Cra/vw o / 1, the linear discriminant in Eq. can be compressed into the following general form (Fig. a-c, general case):
[A AMy1" As in the covariance
WGD WLD 7(
Am Apy
Cory
A Auy
(WCDBO Conrection WCDBO)
Car
Crw/wal
Since from Eq. WLD can be decomposed into the basis WCDBO and correction, we write WGD to indicate that WLD geometrical discriminant (GD), discriminant geometrically modified from WCDRO- This geometrical modification can be intuitively interpreted as rotations on WCDBO-
Starting from Eq. 5 we have the following special cases to consider, which represent different forms of geometrical modification applicd to WCDBO:
Special case 1. If assume two variables have the same variance (o wgy) then Cra/vv 0. special case 1):
Eq. becomes (Fig. 4
WLD 7
ppz Apy
Cry
Apz
7 (CCDB + conrection WCDBO)
(6)
Cay
Afly
From Eq. there are two special cases:
Special case 1.1. If two covariance matrices are same (Eo E,), then the following also holds:
Cry "gy 3[(=-p.)ly-a)
Tay where Tay is the Pearson correlation coefficient (PCC) between
dz0y
x and y of the samples in each class. Equation 6 becomes (Fig. 4 special case 1.1):
WLD =>(
Ap
Tzy
Apu
7 (WCDBO + conrection WCDBO) WCDB))
(7)
Ap
Auy
Special case 1.1.1. From Eq. then the equation becomes (Fig. special case 1.1.1):
7
if there is no correlation between x and y variables (e.g., Tay = 0),
WLD = ((A Any))² [ Any WCDBO
(8)
which is equivalent to MDC method except for the bias. The second equal mark is from the fact that [Azz A/yy] is already a unit vector, thus = 1.
Special case 1.2. From Eq. 6 if two classes are symmetric about one variable, i.e., o0 12 +
ay
0, then Cxy oyy
<zu 0. In this case, the obtained discriminant is the same in Special case 1.1.1
(Fig.
special case 1.2):
WLD 7[Ap A [Apes Apy] WCDBO
Fig. 4 shows how LDA applies geometric modifications to WCDBO as a shape adjustment for different data shapes. When covariance matrices are similar (Special case 1.1), the correction term acts only when variables x and y have certain extent of correlation and increases with this correlation (Fig. 4 third row). Interestingly, when further assumptions are made that two variables have no correlation (Special case 1.1.1). or when two classes are symmetric about one variable (Special case 1.2), we can from Eq. 1 that WLD approaches WCDBO- which is indeed what we observe in the last two rows of Fig. Videos showing these special cases using 2D simulated data can be found from this link
The demonstrations of all these cases for higher-dimension
space are in the Appendix.
From the above derivation, Eq. 9 show that the solution of LDA can be represented by WCDB0 superimposed with geometric correction on WCDBO. and the geometric correction term is obtained under the constraint of Eq. [ which solves the maximization of the projection variance ratio. Without loss of generality, the conclusion can be extended other linear classifiers with different constraints imposed on the geometric correction term, for instance MDC, where the correction terms are all zero.
httpps:/drr wwwveiivvveeeeeee vee googl weweee.eeeeeeeeoeeeeeeeeeee wttp://fldderr.r/ooo wtt.ddderrs1///01/e..
WpHhe A75px A AV4xX0
usp-sharing

---

Published as a conference paper at ICLR 2026
Here, we propose the generalized GDA theoretical framework in which not only the geometric correction term can impose any constraint based on different principles, but also any number of correction terms act together on WCDnO to create the classification discriminant, given by:
WGD = (WWCDBO + C1 WCDDB + C2CCDBO + .... ..... CCWWB)))))
(10)
in which LDA only involves the first order base term WCDDO and the second order correction C₁+ shows that LDA is derived by the general GD equation imposing one variance-bass geometric correction in Fig. D shown in Fig. Th using an instance of 2d artificial data with specific covariance.
while higher order terms C2. C3. become O. Fig. |
(WWcnos + CyMcnno CgWcnnn + Crmenn))
c(a')
G. - C(a*) C. C,
CDB0
LD
Wio Wan (M C( (eenn)
Figure 1: (a) A 2D instance showing that LD (blue dashed) can be constructed from CDBO (black solid) using covariance correction which is geometrical correction. (b) The evolution from CDBO to LDD. The first equation the general form of geometrical discriminants (GD) that can be constructed by CDBO with geometric corrections. If only correction exists and this correction is covariance- related matrix, LD can be derived from the general expression. y denotes a normalizing factor. LD: Linear Discriminant; CDBO: Centroid Diseriminant Basis O.
Since GDA is framework rather than specific algorithm, it does not itself provide global theoretical guarantees on performance or convergence. Instead, such guarantees must be established for each concrete classifier instantiated within the framework. Therefore, in Section D we provide a section on CDA convergence proof, where we provide a formal mathematical proposition showing that, under the defined fitness function, CDA is guaranteed converge. This convergence relies solely on the monotonicity and boundedness of the objective sequence and therefore holds independently of the specific DD optimizer employed (e.g., BO, Brent's method, etc).
CENTROID DISCRIMINANT ANALYSIS (CDA)
Based on the GDA theoretical framework, in this study we propose an efficient geometric centroid- based linear classifier named centroid discriminant analysis (CDA), then introduce the extension to nonlinear classification via kernel method in section As described by Eq. io the GDA theory, the model of a geometric classifier can be expressed by the basis CDBO imposed by geometric corrections on this basis. We first give the conclusion that the final discriminant of CDA after n rotations is in (m) the form CDA = 7( (WCDDO + CIWDDBO) = WGD. where C1 = II" Acds I is the geometric correction operator term, is the identity operator, and Aeda the operator of single CDA rotation). The complete derivation of CDA in GDA theoretical framework can be found in Appendix. F
From the geometric point of view, CDA built on the basis CDBO in the GDA theoretical framework, then subjected to a series of geometric constraints guiding the rotations of the discriminant CDB in high-dimensional spaces. CDA follows performance-dependder training that, starting from CDBO involves the iterative calculation of new adjusted centroid discriminant lines whose optimal rotations on the associated 2D planes are searched via Bayesian optimization. The following parts together with Figure. 5 describe the CDA workflow, mechanisms and principles in detail.
Performance-associatee CDB Classifier: In GDA theoretical framework, CDB is defined as the unit vector pointing from the geometrie centroid of the negative class positive class. The geometric centroid is defined in a general sense which considers the weights of the data. The space of CDB consists of unit vectors obtained with all possible weights.
Apart from the discriminant line, a bias further required to realize classification by offering a decision boundary for data projected onto the discriminant. We perform search for this bias by
5

---

Published as a conference paper at ICLR 2026
checking the middle points of every two consecutive sorted projections. Thus, given N samples, there are N - 1 candidates. We name the best candidate as optimal operating point (OOP) and select OOP according to a performance-scove score, defined as (F (Fscorel + F "ccree" + AC'score) (see Appendix. for definitions), where pos and neg refer to evaluating the metric for each class. The performance score is comprehensive metric that simultaneously considers sensitivity, recall, and specificity, providing fair evaluation that accounts for biased models trained on imbalanced data, owing the more conservative AC-score metric (Wu & Cannistraci (2025)) ). With this OOP search strategy, any vector in the space of CDB is associated with performance-scvee score. The OOP search can be performed efficiently in (NNlogN) time (see Alg. for pseudocode). Importantly, during each rotation within the 2D plane, CDA explicitly selects the dirrectinn that maximizes the performance score and progressively refines this choice through continuous rotations. Because the optimization target is transparent at every step, CDA offers an inherently explainable learning process.
CDA as Consecutive Geometric Rotations of CDB in 2D planes: Our idea to start the optimiza- tion path from CDBO, continuously rotate the classification discriminant on 2D planes which there is high probability of better performance. To construct such a 2D plane with another vector, key observation is that samples whose projections onto CDB are close to the decision boundary (i.e. OOP) should have more weights, because these samples are prone to overlapping with samples from the other class, causing misclassification. Thus, we compute another CDB using centroids with shifted sample weights toward OOP (see next part). On the plane formed by these two CDBs, the best rotation is found by Bayesian optimization. For clarity, we have the following definition: the first vector in each rotation is termed CDB1, the second vector in each rotation is termed CDB2, the CDB searched with the best performance is termed CDA, and at the end of each rotation CDA becomes CDB1 1 for the next rotation. CDA rotation refers to this rotation process. Figure 5 shows the diagram for the first CDA rotation, where CDB1 equals to CDBO calculated using uniform sample weights. The CDA training stops when meeting any of the two criteria: (1) Reach the maximum 50 iterations. (2) the coefficient of variation (CV) of the last 10 performance-score
less than a threshold, indicating that the training has converged (see Alg. 1 and 4 for pseudocode).
Sample Weights Update Strategy: Given CDBI and the associated OOP in each CDA rotation, the distance of all projections to OOP can be obtained by d; = lq, oopl, Vi E (., which is then reversed by d, = dd min - (d)). in alignment with the purpose of giving larger weights points close the decision boundary. Since only the relative information of sample weights is important, they are 1-2normalized and decay smoothly from previous sample weights by a @ / ||ox @ drl12, where @ indicates the Hadamard (element-wise) product. Specifically, in the first CDA rotation, the CDB1 is obtained with uniform sample weights, which corresponds to CDBO in the GDA theoretical framework.
Rotation Optimization: The CDA rotation aims to search for a unit vector CDB with the best performance-score in a 2d plane spanned by CDB1 and CDB2. There are several optimizers that can realize this single- laggee-arrmeter search of rotation angle. Based on our findings in Section L we make the following recommendations. For maximum speed, Fibonacci search is the preferred choice, and BO is a strong second option. Be aware that BO seems to prefer log-transform of the data. For large-scale datasets, Fibonacci search remains the most efficient and still reliable method. In the main article, we proceed with BO due to its high performance. BO is a statistical-base technique to estimate the global minimum of function with as fewer evaluations as possible. CDA leverages this high-efficiency characteristic of BO to achieve fast training. BO has O(Z²) time complexity when it searches a single parameter, where Z is the number of sampling-and-evaluu strategy that grows BO sampling times from 4 to the maximum 10 with CDA iterations (see Appendix C.I. Figure 5p shows an instance of BO working process. Alg. 5 shows the pseudocode for the CDA rotation the black box function to optimize by BO.
CDA employs a
Finalization: As a finalization step, on the best plane CDA refines the discriminant with a null-model statistical test using 100 random CDB lines drawn from the plane (see Appendix. C.2 and Alg. 6).
Multiclass Prediction: Error-correcting output codes (ECOC) (Allwein et al. (2001)) is a vote/penalizatioonbaase method to make multiclass predictions from trained binary classifiers. To predict the class of new sample, multiple scores can be obtained from the set of trained binary classifiers. These scores are interpreted as they either vote for or are against a particular class. Depending on the coding matrix and the loss function chosen, ECOC takes the class with the highest overall reward or lowest overall penalty as the predicted class. In this study, the coding matrix for
6

---

Published as a conference paper at ICLR 2026
one-versus- one training scheme is selected, as it creates more linear separability (Acevedo al. (2022 2.24). ). For the type of loss function, hinge-loss is chosen, as our internal tests suggested that this loss leads to the best classification performance for all tested lincar classifiers.
4 EXPERIMENTAL EVIDENCES ON LINEAR CLASSIFICATION OF REAL DATA
In this section, we compared the proposed CDA with other linear classifiers including LDA, SVM and LR. The 5 tested SVM variants include the original SVM, dual and primal fast SVM implemented by Liblinear. SVM-SGD, and one fast SVM with BO hyperparameter search. LR uses fast imple- mentation by Liblinear (fast SVM and fast LR refers to the dual version unless otherwise specified). Additionally, we include baseline method, CDBO (equivalent to MDC equipped with OOP bias), to quantify the improvements made by CDA over centroid-based classification. Their theoretical time complexity is shown in Fig. 2h. Experiment details are in Appendix [J] An additional comparison with Partial Least Squares Discriminant Analysis (PLSDA, Barker & Rayens] (2003) a widely used and competitive linear method especially in chemometries, genomics, and other high-dimensional applications with both binary and inherent-multicde versions can be accessed in Section
a
(muu prediction AUROO)
AUROO)
Muthod
complestt
Aucesa bahat tsin ti (multidass prediciion AeddicionAAROO (moeet
LDA Fibonacci
CD90
DCNA + Kinph)
Top asciment
20
Average manking
g -J o. q. a g
AUROC
SVM fiok.
OONN NingN)
0.5
LDA
OUNM" H,14)
10-
0.5
0(N7)
Fast BVN,
20M--5rn.
((MMM)
100
1K
10K
100K
1M
Fosd UR
Training data size
Number of samples
-Nimmerr of feotures
levations
Average ranking of runtins
head re (emoved)
Running time (hour)
Its
lover hoverheeed 9 0
prial
0.5
100
1K
10K
100K
1M
Average Selary-aii datia
#Samples single-cel data
Figure 2: (a) Theoretical training time complexity of linear classifiers. (b-e) Performance 27 real datasets. (b) Top-2 occurrences and (c) average ranking of classifiers according to multiclass AUROC. Error bars represent standard errors. (d) Average ranking of training time on large datasets with average class-pair data size N x M > 10 (c) Lines of linear regression between training times and dataset sizes averaged across binary-pair in each dataset, in the log-log scale to reveal the scalability. (f-g) Large-scale scale data test. (f) Test set AUROC and (g) training time on large-scue scagee single-cell data with varying sizes of training samples by DDA-Fibonacci and fast SVM. CDBO: Centroid Discriminant Basis O; CDA: Centroid Discriminant Analysis; LDA: Linear Discriminant Analysis; SVM: Support Vector Machine; LR: Logistic Regression; BO: Bayesian Optimization.
Classification Performance Real Data: We assessed linear classification performance across
27 datasets, including standard image classification (Coates et al. 1) Cohen et al. (2117): Hull (1994); : Krizhevsky (2009): ; LeCun et
1998): Netzer & Wang (2011 Nilsback Zisserman
2008): ; Stallkamp et al. (201 1I; Xiao et al. 2017)) medical image classification (Yang et al. (2023), ). and chemical property prediction tasks Wu et al. (2018)) These datasets represent broad range of real-world applications and varying data sizes, enabling evaluations for both training speed and predictive performance. Image data were flattened to ID vectors; chemical formulas were processed by simplified molecular input line entry system (SMILES) tokenized encoding (see Appendix K) Each dataset was split into 4:1 ratio for training and test sets. The final model for each method was an unweighted ensemble of the five cross-validated models on the training set. We used one- versus- one training scheme and use external framework to realize multiclass prediction, instead of the native multiclass approaches (See reasons in Section TT-
Fig. 2b-e show the test set multiclass prediction performance. In Fig. 2p, CDA achieved a top- 2 occurrence of AUROC on 17 out of 27 datasets. outperforming all the other linear classifiers, indicating its stability and competitiveness. Fig. 2 shows that CDA achieved the highest average ranking around 3.3, followed by fast SVM BO and SVM, however, their extremely low average
7

---

Published as a conference paper at ICLR 2026
ranking of training speed in Fig. indicates the impracticality for large-scale datasets (Panel (d) shows results on large datasets with average class-pair data size N x M > 10", because in practice we care more about the time consumption large datasets rather than small ones). See S for other metrics and Section [S complete performance tables. These results of CDA is with BO optimizer and log-transform to stabilize training. while the comparison for different optimizers are in Section L and for the efficient variant CDA-Fibonacci see Fig. N
In Fig. 2 we performed linear regressions between training times and data sizes averaged across binary pairs within each dataset with overhead deducted (See Fig. B for original data points, and Fig. _p-c for results with overhead kept). The regression results show that CDBO exhibits the best scalability due to its simplicity. SVM primal and SVM-SGD are fast for small data size, however, CDA outperforms them when data size gets large. CDD not only improves significantly over CDBO in performance but also retains similar scalability. This improvement is driven by three key components: generalized centroids with non-uniform weights, sample weight shift strategy, and rotations by Bayesian optimization. The weighted average number of iterations required by CDA per dataset was 29.33 a small constant that does not contribute to time complexity. In contrast, the iterations required by SVM variants increase significantly with more challenging datasets to achieve a reasonable classification performance. Considering the diversity of tested datasets, CDA demonstrates itself as a generic classifier with strong performance and scalability, making it applicable to large-scale classification tasks across various domains.
We further test CDA (0177), with the largest two classes (See Section. H for details). We applied CDA-Fibonacci as a faster version for large data (See Section D). The results show that with growing sample sizes, CDA outperforms fast SVM on both classification AUROC (Fig. 2 and Appendix Fig. 6). and training speed (Fig. Dg. These results indicate that linear CDA is even more efficient and scalable than the flagship SVM method in efficiency. Hence, CDA as fast approach has the potential to drive large-scale real-world applications in fields such as biomedicine and autonomous driving.
large-scale dataset, the 1.3-million-cell mouse brain dataset (10x Genomies
In addition, we made comparison between CDA with two prevalent neural network architectures MLP and ResNet (See Appendix. [U.T]. In addition, we found that feasible to incorporate neural networks in CDA, using their extracted features to train CDA, which improves linear CDA performance. Importantly, CDA can be used to initialze linear layers or MLP in the final layers of a neural network architecture, outperforming their randomly initialized versions (See Appendix U.2)
We analyzed the relationship between the actual stopping iteration of CDA and the average binary classification performance (ps-score) across different datasets and iteration limits. The results revealed two distinet regimes. In the right half of Fig. 3 for tasks converging before 50 iterations, observed a significant negative correlation (Pearson's 's R = 0044), indicating that datasets with lower performance required more iterations to converge. This highlights the importance of allowing least 50 iterations, as early stopping before this point may prevent convergence for more challenging tasks. In contrast, for tasks exceeding 50 iterations, the correlation was weak and positive (Pearson's 's R 0.184), suggesting that beyond this threshold, the number of iterations no longer plays a significant role in ensuring convergence. This indicates that once 50 iterations are reached, CDA can stabilize regardless of the underlying task performance. To further validate this observation, in the left half of Fig. 3 we compared the distributions of ps-scores obtained under maximum iteration limits of 50 and 150. The estimated probability densities were nearly identical, and the Wilcoxon signed-rank test confirmed conditions (p=0.398). This result supports the conclusion that extending the maximum number of iterations beyond 50 does not provide systematic benefits in terms of classification performance. We also provide a similar convergence analysis for fast version CDA- -ibonncci in Appendix (O
CDA's Convergence Property:
significant difference between the two
PRELIMINARY TEST ON NONLINEAR KERNEL CDA
When data exhibit strong nonlinearity, linear classifiers often underperform. To address this problem, models must exploit these nonlinear patterns to enhance expressiveness and improve classification performance. In fact, linear CDA can be naturally extended to nonlinear variant using kernel method while retaining efficiency (see Appendix C.4). The main computational bottleneck lies in the kernel matrix computation, which is common cost shared by all kernel-based classifiers.
8

---

Published as a conference paper at ICLR 2026
Standard
fanelaral
12 image dasa max
tentrnns Cheer prpuery cars nay tretton 20 Mediica ensoe asto 1150 mar dentiinnl Cheer propary cata (150 max berations
Wilcoson peo.350 390
0.9
Arenag bendy cleediccinn posizas
Ri-0.48
0.7
Pearsors! Rud.1144
0.6
150 rase Fontones
Diatibuttion
Actual stoppin fansioon C23A
Figure CDA-BO convergence. The right half shows relationship between classification performance and the actual stopping iteration of CDA across datasets under two maximum iteration limits (50 and 150). A negative correlation was observed for tasks converging before 50 iterations (Pearson's 's R -0.48), while weak positive correlation was found for tasks converging after 50 iterations (Pearson's ' R = 0.184). The gray dash lines show linear regression of points till 50 and points between 51 and 150. The black vertical line indicates the 50-iteration threshold. In the left half, probability density distributions of ps-scores under 50 and 150 maximum iterations show substantial overlap.
To compare the performance of nonlinear CDA and other nonlinear classifiers, performed tests 3 challenging datasets, image dataset SVHN, chemical property dataset Clin'Tox and medical image dataset fracturemnist3d (see Appendix K for data processing). We choose these datasets because they are difficult to classify with linear classifiers, and we are interested in to what extent nonlinear classifiers improve over linear ones. We used a subset of SVHN samples (24,000) due to time limitation to test with kernel method. We compared linear/Gaussinr CDA (nCDA), SVM/nS VM and L.A/nL For kernel methods, we performed hyperparameter search for the gaussian parameter a. The data were divided into train, validation and test set, where on the validation set the hyperparameter was tuned. The training specifics are in Appendix [.2] The multiclass test set results (on Clin'Tox binary results since is a binary task) in Table I show that kernel CDA outperforms on out of datasets, while the performance gap compared to the best one (nLDA) is not large. Importantly, kernel CDA improves on linear CDA substantially, suggesting the potential of applying kernel methods on linear CDA for complex data. Despite the fact that gaussian CDA improves substantially over linear CDA in tasks such the SVHN image classification, we emphasize that linear CDA is still very useful and important, with high efficiency and low resource requirement in computation.
Table 1: Test set classification performance.
Dataset
Method
AUROC
AUPR
Fscore
ACscore
CDA
0.61520.02
0.63e0.02
0.61920.02
0.423=0.05
SWHN
aCDA
0.77720.01
0.782x0.01
0.78a0.01
0.73116.02
subsct
SVM
0.555#0.01
0.568=0.007
0.001
0,273+0.05
(image)
BSVM
0.73620.02
0.77620.009
0.75640.008
0.65420.03
aLDA
0.78620.01
0.7920.01
0.789:0.01
0.743:0002
0.567
0.56
0.351
ClinTox
aCDA
0.625
0.627
0.627
0.46
(chemical)
SVM
0.294
BSVM
0.500
0,481
0.480
0.000
aLDA
0.605
0.612
0.611
0.409
CDA
0.518:0.01
0.526:10.02
0.486:10.02
0.279 00.0
Fracture 3D
nCDA
0.625
0.62.10.04
0.607:10.04
0.507
(medical
SVM
0.576=20.009
0.579=00008
0.577+10.08
0.505=
image)
#SVM
0.608=0.06
0.591=:0.07
0.586=00.07
0.44::022
aLDA
0.608
0.626:10.02
0.605::0.02
0.491
6 CONCLUSIONS AND DISCUSSIONS
Linear classifiers, while inherently simpler, favored in certain contexts due their reduced tendency to overfit and their interpretability in decision-nis decision-making However, achieving both high scalability and robust predictive performance simultaneously remains significant challenge.

---

Published as a conference paper at ICLR 2026
In this study, the introduction of the Geometric Discriminant Analysis (GDA) framework marks notable step forward in addressing this challenge. By leveraging geometric properties of centroids fundamental concept across disciplines GDA provides a unifying framework for certain linear classifiers. The core innovation lies in a special type of Centroid Discriminant Basis (CDBO), which serves the foundation for deriving discriminants. These discriminants, augmented with geometric corrections under varying constraints, extend the theoretical flexibility of GDA. Notably, Minimum Distance Classifier (MDC) and Linear Discriminant Analysis (LDA) are shown to be a subset of this broader framework, demonstrating how they converge to CDBO under specific conditions. This generalization not only validates GDA but also sets the stage for novel classifier designs.
A key practical contribution of this work is the Centroid Discriminant Analysis (CDA), a specialized implementation of GDA. CDA employs geometric rotations of the CDB within planes defined by centroid-vectors with shifted sample weights. These rotations, combined with Bayesian optimization techniques, enhance the method's s scalability, achieving quadratic time complexity O(N M + Nlog.V. Across diverse intaactts-includin
standard images, medical images, and chemical property clas- sifications-CDA demonstrated superior performance in scalability, predictive performance, and stability. We emphasize that CDA not merely robustly outperforms most of existing linear classifica- tion methods, but crucially, it achieves this with superior computational efficiency. Specifically, CDA exhibits quadratic time complexity in the worst-case scenario, compared to the cubic complexity typical of established methods such as LDA and SVM, with significantly shorter runtimes that high- light its practical advantage in real-world applications. These findings hold particular relevance for the broader machine learning community, as linear classifiers remain widely used across numerous scientific domains where interpretability, scalability, and computational efficiency are critical. In such contexts, practitioners often prefer models that are not only robust and fast but also transparent and easy to deploy in real-worle decision decision-making scenarios.
For data exhibiting complex nonlinear structures that linear classifiers cannot adequately capture, linear CDA can be extended to nonlinear form via kernel methods. This generalization significantly broadens the applicability of CDA by enabling it to handle nonlinear patterns in the data. Together, linear and kernel-based CDA form a complementary toolkit for classifications. A practical strategy is to begin with linear CDA to see whether the predictions are satisfying, which offers fast and interpretable results. Our preliminary results also suggest that extending CDA with nonlinear kernels is promising direction for future research, since the variety of available kernels broadens the opportunity to tailor the mapping of original data onto higher-dimeesional spaces according to the specific structure of the data- where it may become more easily separable. Another future work is to explore an inherent multiclass extension of CDA (see Section TT. especially given that other classifiers such as LDA and SVM have their inherent multiclass approach. In the cases where C' is large and not negligible, inherent multiclass approaches offer attractive lower computational complexity with respect to the number of classes (C versus C(C - 1)/2 discriminants in pairwise schemes).
One limitation of this study is that the current CDA does not have a natively multiclass approach, hence can be interesting to investigate how to formulate CDA in natively multiclass way. However, in Section T we showed that using inherently multiclass LDA, SVM, and PLS-DA the performance does not improve and even drops significantly for SVM, a reason for which in this study we decided to focus on binary CDA classification and leveraging external multiclass framework. In addition, in this study, the GDA framework is used to show that LDA is special centroid-basee method case, and then is adopted to design CDA. CDA itself is presented as single, highly effecttve derivative, but the GDA framework's utility in systematically generating other novel classifiers is not investigated enough. Since GDA in the current formulation is framework that can include only centroid based classifiers, methods such as SVM that are based on large margin classification are not included in the GDA framework explain and design classifiers based linear centroid-based separability. The mission to investigate the extent to which GDA framework may inspire the design of new classifiers will be a not trivial point to expand in future studies.
In conclusion, the GDA framework and its CDA implementation represent a paradigm shift in classifier design, combining the interpretability of geometric principles with state-of-the- computational efficiency. This work not only advances the field of classification but also lays groundwork for innovative approaches to supervised learning.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the Zhou Yahui Chair Professorship award of Tsinghua University (to CVC), the National High-Level Talent Program of the Ministry of Science and Technology of China (grant number 202417100M 100011,0,,,11 CVC), and National Natural Science Fund of China Research Fund for International Senior Scientists (RFIS-III, grant number W2531064, to CVC). The authors thank Yuchi Liu and Mo Yang for their administrative support.
REFERENCES
10x Genomics. 1.3 million brain cells from e18 mice. ftttpp://suuportt 10xeeoom cs com/ single-eegeeeeeeeeeeeeeeeeee-- cew--ceelgeeeeeeee 3 cessed: May 2025.
neurons 2017, Ac-
Aldo Acevedo, Claudio Duran, Ming-Ju Kuo, Sara Ciucci, Michael Schroeder, and Carlo Vittorio Cannistraci. Measuring group separability in geometrical space for evaluation of pattern recognition and dimension reduction algorithms. IEEE Access, 10:22441-22471, 2022. ISSN 2169-3536. doi: 10.1 100///ccsss.022223555533.
2022
6152789
Aldo Acevedo, Yue Wu, Fabio Lorenzo Traversa, and Carlo Vittorio Cannistraci. Geometric separa- bility of mesoscale patterns in embedding representation and visualization of multidimensional data and complex networks. PLOS Complex Systems, 1:1-28, 1 2024. doi: 10.137 1
0000012. URL https::/doi. org/ 10. 1371/ journal. pes....na.....................m. pcsy. pcc.00000122
Erin L. Allwein, Robert E. Schapire, and Yoram Singer. Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research, 2001. ISSN 15324435.
M. Barker and W. Rayens. Partial least squares for discrimination. Journal of Chemometrics, 17(3):
166- 173, 2003. doi: 10. 10.1002/cem.88.
Deng Cai, Xiaofei He, and Jiawei Han. Training linear discriminant analysis in linear time. In Proceedings International Conference Data Engineering, 2008. doi: 10.1109/ 4497429.
Adam Coates, Honglak Lee, and Andrew Y. Ng. An analysis of single-layer networks in unsupervised feature learning. In Journal of Machine Learning Research, volume 15, 2011.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: Extending mnist to handwritten letters. Proceedings of the International Joint Conference on Neural Networks, volume 2017-May, 2017.
doi: 10.1109/IICNN..211.79666
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20, 1995. ISSN 15730565. doi: 10. 10.10//:::02222241111
Richard O. Duda, Peter E Hart, and David G. Stork. Pattern classification, second edition. NY: Wiley-Interscience, 2001.
R. A. Fisher. The of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 1936. ISSN 0050-1420, doi: 10.1111/j.1469) 1881444999-911111111 190...996.0..11........
Jonathan J. Hull. A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 1994. ISSN 01628828. doi: 10.1109/34.291440
Alex Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis. Department of. Computer Science, University of Toronto, 2009. ISSN 1098-6596. doi: 10.1.1.222.9220.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86, 1998. ISSN 00189219, doi: 1..799.
Marvin L. Minsky and Seymour Papert. Perceptrons, Expanded Edition An Introduction to Computa- fional Geometry. 1969.
11

---

Published as a conference paper at ICLR 2026
Masaya Misaki, Youngmoon Kim, Peter A Bandettini, and Nikolaus Kriegeskorte. Comparison of multivariate classifiers and response normalizations for pattem-informaten information firri Neurolmage, 53 (1):103-118, 2010. doi: http:://ww..mmma..00....00000000
Yuval Netzer and Tao Wang. Reading digits in natural images with unsupervised feature learning. Nips, 2011.
Maria Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Proceedings 6th Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP 2008, 2008. doi: 10.1109/CVGIP.2000..44
Nihar Ranjan Panda, Jitendra Kumar Pati, Jatindra Nath Mohanty, and Ruchi Bhuyan. A review logistic regression in medical research, 2022. ISSN 22296816.
Marc Andre Schulz, B. T. Thomas Yeo, Joshua T. Vogelstein, Janaina Courao-Miranadaa Jakob N. Kather, Konrad Kording, Blake Richards, and Danilo Bzdok. Different scaling of linear models and deep learning in ukbiobank brain images versus machine-learning datasets. Nature Communi- cations, 11, 2020. ISSN 20411723. doi: 10.00///414677-20-- 18037-..
Xi Shen, Emily Finn, Dustin Scheinost, Monica D Rosenberg, Maria Chun, Xenophon Papademetris, and R. Todd Constable. Using connectome-based predictive modeling to pre- dict individual behavior from brain connectivity, Nature Protocols, 12:506-518, 2017. doi: 10.10388hpro..211..177
Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. The german traffic sign recog- nition benchmark: A multi-class classification competition. In Proceedings of the International Joint Conference on Neural Networks, 2011. doi: (100///CCNN.2111.633333
Gaël Varoquaux and Veronika Cheplygina. Machine learning for medical imaging: methodological failures and recommendations for the future, 2022. ISSN 23986352.
Gaël Varoquaux, Pradeep Reddy Raamana, Denis A. Engemann, Andrés Idrooo, Yannick Schwartz, and Bertrand Thirion, Assessing and tuning brain decoders: cross-validation, caveats, and guidelines. Neurolmage, 145, 2017. ISSN 10959572. doi: 10. 1..1016...eurrommagee 201.:/mmmaaee221...00000.
Yue Wu and Carlo Vittorio Cannistraci. Accuracy score for evaluation of classification imbalanced data. Preprints, 1 2025. doi: 10. 000200044/mmpintt000002222111000 10. 1
URL httptttps/ org/
2178.
Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenet: A benchmark for molecular machine learning. Chemical Science, 9, 2018. ISSN 20416539. doi: 10.10399c7sc002644a.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017. URL httLlhttts///aee /tap:/i////a/v///ae wwwprrg/abb///// 0777707747
Jianeheng Yang. Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2 large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10, 2023. ISSN 20524463. doi: 10. 00.1038/44159--222 8-11----111-
Guo Xun Yuan, Chia Hua Ho, and Chih Jen Lin. Recent advances of lcrge-scale linear classification. Proceedings of the IEEE, 100, 2012. ISSN 00189211 19, doi: 10. 10.1099/RRO..22022218880
12

---

Published as a conference paper at ICLR 2026
A SUPPLEMENTARY: GDA THEORETICAL FRAMEWORK
A.1 DIFFERENCE BETWEEN THEORY AND THEORETICAL FRAMEWORK
We emphasize that GDA does not constitute new theory, but rather a theoretical framework. A theory in data science and machine learning provides a foundational explanation often accompanied by rigorous mathematical derivations, such as performance bounds or convergence guarantees. In contrast, theoretical framework offers structured set of concepts and assumptions, typically formalized through equations, that guide the design of methods and the interpretation of results without necessarily proving theoretical performance guarantees. Within this clarified scope, GDA serves as the framework under which we define and motivate an efficient geometric, centroid-base based linear classification method: Centroid Discriminant Analysis (CDA), which is discussed in Section 3
A.2 SCALING INVARIANCE THE DISCRIMINANTS IN THE GDA FRAMEWORK
GDA is a geometrical projection -assed theoretical framework. In this framework, the discriminant (the high-dimensional vector orthogonal the classification boundary) of the involved classifiers (including CDA), is normalized to unit vector before projecting data. The final classification depends solely on these projections; scaling the discriminant length by any factor (including terms like / in the derivation of GDA by LDA) uniformly scales all projections without changing their relative positions. Consequently, the optimal threshold (which can be found by search) and classification results remain unchanged. This normalization is explicitly implemented in CDA (see, for example, Appendix C.3 Algorithm 1, line 4: "Normalize WCDBI ..."). "), ensuring that only the direction is refined during iterations.
This property distinguishes GDA and CDA from approaches like SVM, where the magnitude of the discriminant vector is tied to the margin width during training. However, even for SVM, once the model is trained, scaling u and together leaves predictions unchanged because the decision function y H(w" x + b) is homogeneous in w and b. (y: predicted class; (-): Heaviside function; æ: sample; b: bias). In CDA training, magnitude carries no interpretive meaning- only direction matters. These also reflect that GDA reinterprets certain existing classifiers from the geometric projection perspective, which is an innovation that inherently differs from their own interpretations.
A.3 EXPLANATION OF THE DISCRIMINANT BIAS IN THE GDA FRAMEWORK
In practice, GDA-based methods methods-incllding CDA-expAAee explicitly normalize the discriminant to unit length before projection (see Algorithm 1, line 4). This normalization ensures that the training process refines only the direction of the discriminant, while the bias can always be computed afterwards by placing threshold between the projected class distributions. For binary classification, the bias may be chosen using variety of monotonic criteria-ee the midpoint between class means, the empirical risk minimizer on the projected line, or direct search for the threshold optimizing AUROC, accuracy, or any other performance metric. Because projection preserves the ordering of samples, all of these methods yield a valid bias once the discriminant direction fixed.
This perspective also clarifies that the discriminant and the bias play fundamentally different roles: the discriminant determines the orientation of the separating hyperplane, while the bias determines its position. The former is geometrically meaningful in GDA; the latter can always be adjusted post hoc without altering the interpretation the discriminant.
13

---

Published as a conference paper at ICLR 2026
A.4
GDA DEMONSTRATION WITH 2D ARTIFICIAL DATA
b
Bay
-f-mm
+8.1
General Case
Ozz Byy
auu (4y
y(Wcos> Courrectin Wccus)
0xx Bjy
Gormentes
More
[dtz [Ayy)
Cxy Aly
2rfandar
Gry
Special Case
= y(Wcoas Courrection Wened
Cxy
Σo Σ
Messe 11.00
Conmentir 2 Panadrrr
Movee 12:00
[dg (8yyy)
Tay Ae,
Auc
Special Case
Wi.
= y(Wenns Conrenction Wenne)
Fxy 0
MICOOK +8.41
-1-2.00
Special Case 1.1.1
[dite] [Auy]
WLD
Wcoeo
0 1
10.00
Special Case 1.2
[duy]] [Ayy.
Wi.p
Wizoas
Figure 4: The GDA theoretical framework in 2 dimensions showing how the relation between LD and CDBO evolves under specific conditions. (a) The relations between LD and CDBO proceeding from the general case to different special cases under the conditions shown along each arrow. (b) The specific expressions of L.D terms of CDBÓ and corrections corresponding to each case column (a). (c) Binary classifications models of LD (blue dashed) and CDBO (black solid) on different 2D data corresponding to each case in column (a). The lines show the direction instead of the unit vector of the discriminants. LD: Linear Discriminant: CDBO: Centroid Discriminant Basis 0. denotes generic normalizing factor.
14

---

Published as a conference paper at ICLR 2026
B CDA SCHEMATIC DIAGRAM
CDA projection
Original data
0B31 clia cartridd for 2001
CEA 1-n
Bayesian optimization
5000
Figure 5: Diagram for the first CDA rotation. (a) CDBI is obtained from specific sample weights. CDB2 is obtained from centroids with shifted sample weights toward the decision boundary. Darker points represent larger sample weights. CDBI and CDB2 form 2d plane which BO is performed to estimate the best classification discriminant. (b) The process of BO to estimate the best discriminant. The rotation angle / from CDB1 is the only independent variable search in BO. The estimated optimal line CDA serves as the new CDB1 in the next CDA rotation. CDB: Centroid Discriminant Basis; CDA: Centroid Discriminant Analysis.
C THE CDA ALGORITHM
C.I BAYESIAN OPTIMIZATION
One factor related to the effectiveness CDA is the estimation with varying precision in BO. We set the number of BO sampling times min(3 rot, 10) as default parameter during the CDA rotation, where rot is the rot-th rotation. During the first few CDA rotations, the BO estimation has relatively less precision. This gives CDA enough randomness first enter large region in which there more discriminants with high performance, under the hypothesis that regions close to global optimum or suboptimum have more hegh-ppeformance solutions that are more likely be randomly selected. Starting from this large region, BO precision is gradually enhanced to refine the search in or close to this region in terms of Euclidean distance, and force CDA converge. The upper limit of 10 CDA rotations still creates a small level of imprecision. in order to make CDA escape from small-size local minimum, to acquire higher performance, and improve training efficiency.
C.2 REFINING CDA WITH STATISTICAL EXAMINATION ON 2D PLANE
In CDA, the plane associated with the best performance is selected, but is uncertain whether there are
discriminants on this plane, since the precision might not be high enough with 10 BO samplings and even lower for less samplings. To determine whether the BO precise enough, statistical examination using p-value with respect to null-model is performed. On this best plane, we generate 100 random CDBs, run BO in turn for 10, 20 and to the maximum 30 BO samplings, until the p-value of training performance-score of the BO-estimated CDB with regard to the null model O, otherwise using the best random CDB. In this way, we are able to give confidence level of the BO estimated discriminant in the best CDA rotation plane. Alg. shows the pseudocode for the refining finalization step.
C.3 LINEAR CDA PSEUDOCODE
This study deals with supervised binary classification problem on labeled data {X, y) = {I,, ye Je with N samples and M features. The samples consist of positive and negative class data Xi and X2 with sizes N, and N2. respectively, and the corresponding labels are tokenized as 1 and O, respectively. The sample features are flattened to 1D if they have higher dimensions, such as
15

---

Published as a conference paper at ICLR 2026
image data. The following section introduces CDA in pseudo-code. CDA rotations, BO sampling and p-value computing involve constant numbers, thus, they do not contribute to the time complexity.
Algorithm CDA Main Algorithm (CDA)
O(NM + NlogN)
Input: N x M data matrix X, N x labels
Initialize a [1;1;... : 1| //N
{O(N))
Compute WCDB1 = Σ (-1)ct1
Σ 0,E mi
{O(NM))
No
c=1
Normalize WCDB1 WCDB1/ /I WCDBIll2 {0(1))
{O(M))
Initialize ps" = for i 1 to 50 do a = updateSampleWeights// X, y, at WCDB1)
{O(NM + N)g
Compute WCDB2 Σ (-1) c+1 1
Σ
a;
{O(NM))
Ne
E.EX:
Normalize WCDB2 = WCDB2/ WCDB22l2
{O(M)}
Set NBO min(i + 3. 10)
{O(1) Number of BO sampling)
(WCDA, p8) CdaRotation(wucnn WCDB2, X, NBO)
{O(NM + NlogN))
if ps > ps* then (WČDBI: WČDB2 ps*) (WCDBI, WCDB2, ps)
{O(M))
end
if coefficient of variation of the last 10 ps 0.001 then break (Early stop convergence)
end if
Update WCDBI = WCDA
{O(M)}
end for
Compute (WCDA. oop) = refneOnBBesPlaneeuunn wCDDZ X, y)
{O(NM + NlogN))
Output: WCDA, oop
Algorithm Update Sample Weights (updateSammleeeiighs)
O(N + Nlog.V
Input: N x M data matrix X, N x labels y. N x sample weights a, 1 x M vector WCDBI {O(NlogN))
Initialize oop = searchOOP( X, w)
Compute q X w w
{O(NM))
Compute d; = 14s oopl, Vi E (1,2,.... ,N)
{O(N)}
Compute d, = d min max d)
{O(N))
Update a a @ / @ 1
{O(N), Hadamard product)
Output:
Algorithm 3 Search Optimal Operating Point (searchOOP)
(NVogg)
Input: N x 1 projected points N x 1 labels y (O(Nlog.V), sort according q} Initialize cm = evaluateMetrics(gu [0; 0
Sort (q. y)
Nx1
{O(N) initial confusion matrix)
for to N do
Update cm = updateCM(yr. cm)
(0(1) scan each label and adjust cm)
Compute ps; evaluateMetrrcs(cmm
(O(1))
end for
Compute idæ arg max; ps;
(O(N))
Compute ps max(ps)
(O(N)) (0(1))
Compute oop 9rede
Output: oop, ps
16

---

Published as a conference paper at ICLR 2026
Algorithm 4 Approximate Optimal Line by BO (CdaRotation)
O(NM + NNlggN
Input: M lines WCDBI, WCDB2- x M data matrix X, N labels y, total BO iteration
NBO
Compute wath = WCDB2 CDBI (PCDR2Wenm
{O(M)}
Normalize Worth woth/ / W/ 2
{O(M)
Estimate /
BayessanOOtimmization heaamiiizattofeeevlvveeeetttttoo WCDBI, Worth: X, y),, NBo). 0
| 2 I {O(NM + NlogN)}
Compute WCDA WCDBI cos 0 warth sin 0
{O(M))
Compute q X WwDA
{O(NM)
Get the mean of (oop, ps) = searchOOP qq y) with 5-fold cross-validation
{O(Nlog.V))
Output: WCDA, oop, oop, ps
Algorithm 5 Evaluate the Line with Rotation Angle (evaluateRotation)
O(NM + NlogN)
Input: Rotation angle 6, 1 x M lines WCDB1, Worth N x M data matrix X, N labels y Compute WCDA = WCDBI cos 0 + Want sin
{O(M)}
Compute q X wCDA
{O(N M)}
Get mean ps = searchOOP(q.. y) with 5-fold cross-validation scheme
{O(Nlog))
Output: ps
Algorithm 6 Refine on the Best Model (refineOnBestPlane))
O(NM + NNogN)
Input: x M WCDBI and its orthogonal vector Wort, N M data matrix X, N x labels y for i to 100 do
Randomly choose
[-3] 21 x
{O(1)}
Compute w, uCDBI cos Worth sin {O(N))
{O(M))
Compute X u
Get mean (oopr, ps:); searchOOP(g) q, y) with 5-fold cross-validation end for
{O(MogN))
Sort idæ sortIdx [ps, ps,)) Compute p- = (idz(1)-1) 11-1)
{0(1) sort returning position indices) {O(1)}
Nr
Initialize NB0 = 0 while p * and NBO 30 do Increment NBO = Nno + 10 Compute (WCDA, oop, ps)
CdaRotation WCDBI, WCDB2 X, y. NBo)
{O(NM
Nlog.N}
Sort idæ sortIdx [ps, Js.]) (1d=(1)-1) N,
{0(1) sort returning position indices) {0(1)}
Compute
end while
if' p 3 0 then
Compute i = arg min; psr.i
{0(1))
Set (WCDA, oop, ps) (w,, o0pr.p8.). {0(1))
end if
Output: WCDA, oop.ps ps
C.4
NONLINEAR KERNEL CDA
The CDA algorithm involves computing the CDBs using the equation provided in Appendix. Algorithm I:
C.3
WCDB
Σ
2
(-1)o+1
1
Σ
a;
(1I)
N.
c=1
Combining this with the normalization step for each CDB WCDB < WcDn/ WeDa the equation be rewritten
17

---

Published as a conference paper at ICLR 2026
WCDB = nn Yr k) x 3
(12)
where is the vector of sample weights: n [1/NNap(tt] for all (1. the weight sum division for each class, and map(i) maps sample index class index c; y: = (-1) are the tokenized labels (+1 -1); k is the factor to normalize each CDB unit vector.
For the rotated CDBs used in Bayesian optimization (Appendix B, Algorithms 4-5), they can be expressed as linear combinations of CDB1 I and orthogonalized CDB2, mixing their corresponding B vectors. Thus, the sample coefficient tracked throughout the training process for each CDB.
The data projection can be expressed as:
XWCDB X x) XX
(13)
With this expression, the data projection can incorporate kernel methods to realize nonlinear classifi- cation, shown by the following equation.
q q= x=)
(14)
In fact, kernel methods implicitly projects data into high-dimensional space, then applying linear classifiers in the transformed feature space.
18

---

Published as a conference paper at ICLR 2026
D CDA CONVERGENCE PROOF
In this section we provide mathematical proof with proposition showing that, under the defined fitness function, CDA is guaranteed to converge. This convergence relies solely on the monotonicity and boundedness of the objective sequence and therefore holds independently of the specific ID optimizer employed (e.g., BO, Brent's method).
(k) Proposition. Let w CDB1 E RM be the CDB1 vector at iteration In each CDA iteration, (k)
WDDB determines the CDB2 vector WCDB2 via the CDA algorithm with sample weights up- (k)
date strategy CCDB2 = 9(X, W.DBI). Each discriminant has an associated performance met- ric ps = [(WCDB) bounded above by
the current iteration, the optimal vector found (k) (k) + CDB2 where (k+1) CDBI in the 2D plane spanned by CDBI and CDB2 WCDBI
(Ai, Xb) arg A (A} (AIwCDDI + A2wCn12). which then used as the new CDBI at (8)
iteration à 1. Since "CDBI E {Ai (Awwe (K) (DWCDB2) special with Å1 and X2 it follows that (k=11) 2 PSCDBI Therefore, the sequence PdCDB1 = f is non-decreasing and bounded above, and hence converges finite limit ps".
19

---

Published as a conference paper at ICLR 2026
E DERIVING LDA IN THE GDA THEORETICAL FRAMEWORK FOR M-DIMENSIONS (M>2)
In the GDA theoretical framework, we have demonstrated that linear discriminant (LD) can be expressed by the basis CDBO with different geometric corrections CDBO under different conditions, as shown in Fig. 4 The conclusions are not confined only to the 2-dimensional case but also applicable to the m- mndiimensional case (m >> 2). Here derive the generalization of the LD coefficient to the m-dimensional case. The discriminant of LDA is given by WLD = - (mu (4) = - WWCBB. where y is constant, and it is assumed that the within-class covariance matrix Σ is invertible, and E Eo + Σ1 is the sum covariance matrices of individual classes. Denote the elements in E, Eo, and E by ozj. OOij, and L' for all j 21,2..... ... m). respectively.
Special Case Assume that pairwise features have the same covariance a, and all pairwise features 8 (In experiments, we relax the conditions have similar variance and similar covariance). Then, Σ = 311 + al, where I is the identity matrix, and 1 is the all- one matrix. According to the Woodbury matrix identity, we have Σ-1 1 B(ma+
have the same covariance
2 7( (m 1m14)443 1N), where y is the matrix with all ones except on the diagonal. Thus, WT.D (WCDBO (m-1) 1 WCDBO). which corresponds to the row for special case 1 in Fig.
Special Case 1.1. Based the assumptions in Special Case 1, we further assume that the two covariance matrices are similar, i.c., Eo ~ E1. so that d0. 01,i for all i,j. We further have Tij Vi j. where Fij is the Pearson correlation coefficient (PCC) /o0,5 /o0,5
200,
o0,₁
de
200 vaoas
between features and j. Based on the assumptions made in this special case, all pairwise PCCs are the same, i.e., rij r for all * j. Thus, Σ-1 >(I [m-2])-41 1x), which shows that the geometric correction part is related to feature correlations. Accordingly, WLD = (WEDBO (m-2)r+1 1, WCDBo), which corresponds to the row for special case 1.1 in Fig.
Special Case 1.1.1. Based on Special Case 1.1, assume further that all pairwise features have no correlations (i.e., r = 0), then according to the equation, Σ-1 = 7I. Thus, WT.D = 7WCDBO = WCDBO. where y since WCDBO is already unit vector, which corresponds to the row for special case 1.1.1 in Fig. [ By these derivations, we show how LDA converges to CDBO under specific conditions of the data.
F DERIVING CDA IN THE GDA THEORETICAL FRAMEWORK
In this section, we give the mathematical demonstration of how CDA is formulated in the GDA theoretical framework. Since the correction matrix for LDA in Section 2 is a special case of linear operator, we extend more general correction term deriving CDA in the GDA theoretical framework. The construction of the geometric discriminant WGD can be realized by continuously rotating CDB0 planes that satisfy several constraints, shown by:
(n) WCDA Aodal (w (n) CDDRI a (n) X, y)
A (w CDA
(n-1)
a (n) X, y)
Aedu (eedal C a (n-1) x,y). 1.,),, X,
II Acda
(WCDBI, a (1)
(1)
(15)
II Acda
(WCDBO)
[[[ Aeda ()
WCDBO
WCDBO + WCDDO WGD
20

---

Published as a conference paper at ICLR 2026
where ACDA is the CDA rotation operator, C₁ (II" Acda I), in which I is the identity operator. This equation matches with Eq. I0 and involves only one correction term, where y = because both the CDBO and CDA are unit vectors and C₁ is overall matrix of high-dimensional rotation. For a given dataset X, are fixed, it is simplified by: (WCDA, ax) = Acda Acda (WCDBI, a. X, y) = Aeda Reda(Wtnn (CDBB (x) R IR m x R" which defines the linear operator that maps the line CDBI to CDA in each CDA iteration and maps the sample weights a to the updated sample weights.
In Eq. [5 the second equality holds because WCDA obtained at the end of each rotation is used as WCDBI in the next rotation. The fifth equality shows that the final CDA line can be written in a form that only depends on variables of the first iteration. This because, in the first CDA rotation, the initial sample weights (1) are uniform, and CDB1 is the same CDBO of the GDA theoretical framework. Thus, from the sixth equality, X, y. (2) are omitted. The last three equalities show that
the final CDA discriminant is a subcase of the generalized GDA theoretical framework.
The operator Acda be decomposed two sequential operators:
Aoda = ABO WCDBI, A (WCDBI Aswul WWDDI a. x.x.))..y).x8
ABO WCDBI, a)
(16)
where Acdb2(WDDig) a): R X Rd R Ra is the linear operator that maps CDB1 to CDB2 and updates sample weights by Aswu WCDBI a) R x R > IR" which corresponds to Appendix C.3 Algorithm 2. Ano (WCDBI, a) Rn x RR I x Ie" is the linear operator that maps two vectors CDB1 and CDB2 to the BO-estimated discriminant on the plane spanned by CDB1 and CDB2, and updates samples weights.
Addb2 performs the mapping to CDB2 through the following equation:
. (-1)et1 -i=1 a; c)
UCCDB
Σ
(17)
i= an:(y) c)
WCDB2
M/CDB2/ @BCD222
(18)
The operator ABO does the mapping CDA through the following equation:
Worth WCDB2
WCDB) "Dp WCDB1
(19)
W Worth = Worth/ @orth ||2
(20)
G arg max (cos OWEDRI sin OWCDD2)
(21)
0
w = (cos 0 WCDBI + sin 6 WCDB2)
(22)
Where the first two equations find the unit orthogonal vector to CDBI by the Cram-Schmidh process. The orthogonalization process ensures efficient search in the space of CDBs during the rotation. The last two equations estimate the optimal rotation angle using Bayesian optimization and the corresponding discriminant.
ps = Aps (WCDR): Rm + IR is the operator that outputs the performance score given a classification discriminant, which projects all data onto the discriminant and performs the OOP search of the score.
21

---

Published as a conference paper at ICLR 2026
G CLASSIFICATION PERFORMANCE EVALUATION
The datasets used in this study span standard image classification, medical image classification, and chemical property classification, most of which involve multiclass data. To comprehensively assess classification performance across these diverse tasks, we employed four key metrics: AUROC, AUPR, F-score, and AC-score. Accuracy excluded due to its limitations in reflecting true performance, especially on imbalanced data. Though data augmentation can make data balanced, it introduces uncertainty due the augmentation strategy and the data quality.
Since we do not prioritize any specific class (as designing a dataset-specific metric is beyond this study's scope), we perform two evaluations average. This approach applies to AUPR and F-score, whereas AUROC and AC-score, being symmetric about class labels, require only a single evaluation.
assuming each class positive- and take the
For multiclass prediction, a C-dimensional confusion matrix is obtained by comparing predicted labels with true labels. This is converted to binary confusion matrices by each time taking one class as positive and all the others as negative, and the final evaluation is the average of evaluations on these individual confusion matrices.
To interpret this evaluation scheme, consider the MNIST dataset and the AUPR metric as an example. In binary classification, for the pair of digits "1" and "2", calculates the sensitivity-recall for detecting "1" and the sensitivity-recat for detecting "2", then takes the average. For other pairs, the calculation follows the same pattern. the multiclass scenario, considers the sensitivity-recal for detecting "1" and the sensitivity-recall for detecting "not 1", then takes the average. The same approach applies to other pairs.
INDIVIDUAL METRICS
(A) AUROC
Area Under the Receiver Operating Characteristic curve involves the true positive rate (TPR) and false positive rate (FPR) follows:
AUROC
XPRX J==0
(23)
This measure indicates how well the model distinguishes between classes. A high score means that the model can identify most positive cases with few false positives.
(B) AUPR
Area Under the Precision- Recall curve considers two complementary indices: precision (prec) and recall (rec), defined as:
==0 AUPR prec (x)dx
(24)
AUPR measures whether the classifier retrieves most positive cases.
(C) F-SCORE
F-score (using Fi-score in this study) is the harmonic mean of precision and recall which contribute equally:
F Fscore 2 prec rec
(25)
prec + rec
(D) AC-SCORE
An accuracy-related metric AC-score (Wu & Cannistraci (025 ) addresses the limitations of tra-
ditional accuracy for imbalanced data. imposes a stronger penalty for deviations from optimal performance on imbalanced data and is defined as:
2 TPR TNR
AC Scoore
(26)
TPR+TNR
22

---

Published as a conference paper at ICLR 2026
This metric assigns equal importance to both classes and penalizes imbalanced performance by ensuring that if one class is poorly classified, the overall score remains low. Thus, AC-score provides more conservative but reliable evaluation of classifier performance.
(E) PERFORMANCE-SCORE (PS)
The proposed CDA classifier employs performance-depenee learning, guided by performance score (ps):
(Feee acore + Feee
score + ACscare)
(27)
This formulation ensures a balanced consideration of both F-score and AC-score.
Though CDA generic classifier utilizes relatively balanced metric, can be customized to specific applications as needed.
H EXPERIMENT ON LINEAR CLASSIFICATION OF LARGE-SCALE DATA
There are real-world problems where the samples are at million-level in addition to plenty of features, posing significant challenge train classifiers in reasonable time. One of these problems is omics data classification such as single-cell sequencing data. We collected the 1.3-million-cell mouse brain dataset (IOx Genomies (2017) ) and took the largest two classes to create a binary classification task (757,526 samples and 27,998 features), with varying number of samples by random subsampling till all samples were included. We compared CDA and fast SVM (dual optimizer), which one of the most efficient and scalable linear SVM, with regard to performance and speed. We applied CDA with Fibonacci optimizer as the fast version for large data. Original data of sequencing counts were used, and the training and test sets were created using 4:1 split. The results show that with growing sample sizes, linear CDA outperforms fast SVM not only on classification performance AUROC (Figure 2 and Appendix Figure 6 but also the single-core training speed (Figure ( These results indicate that linear CDA is even more efficient and scalable than the flagship SVM method in efficiency. Hence, CDA fast approach has the potential drive large-scale real-world applications in fields such as biomedicine and autonomous driving.
ACscore
0.4
0.2
1000
16
10K
100K
1M
100
18
1.K
100K
1M
0.0
1K
10K
100K
1M
#Samples in singieeeoaae cell data
Figure 6: Test set performance with varying sizes of training samples in single-cell mouse brain data, evaluated by (a) AUPR, (b) Fscore and (c) ACscore. CDA: Centroid Discriminant Analysis; SVM: Support Vector Machine; AUPR: Area under precision-recall curve; ACscore: Accuracy score.
I SUPPLEMENTAL TRAINING SPEED RESULTS
23

---

Published as a conference paper at ICLR 2026
CDBI
LDA
Fast SVM dual
Fast primal
Fast SVM BO
SVM-SGD
SVM
10
Training time (s)
10
10
109
10
4096
8192 Input matrix width N
16384
Figure 7: The training time with constant overhead kept. The training time of linear classifiers with increasing input matrix sizes. The number of features were set the same as number of samples N. Shaded area represents standard error. Log-scale is used for both axes reveal the scalability of algorithms. CDBO: Centroid Discriminant Basis O; CDA: Centroid Discriminant Analysis; LDA: Linear Discriminant Analysis; SVM: Support Vector Machine; BO: Bayesian Optimization; SGD: Stochastic Gradient Descent; LR: Logistic Regression.
b
23A
Fan
C
Average binary-pair train time (s)
10
Average binary-pair train time (8)
aul
Average binary-pair train time (s)
10"
(overhead removed)
1
102
10a
(overhead kepl)
(overhead kept)
10%
10
10
2
10
1
10a
10
Average binary-pair data size
10 Average binary-pair data size
10
10
10a
Average binary-pair data size
Figure 8: Running time results of linear classifiers on 27 real datasets. (a) Linear regression fit of the running time against data sizes, together with original running time data shown by the circles. Overhead is removed. (b-c) Running time and speed results with overhead kept. (b) Linear regression fit of the running time against data sizes. (c) The same as (b) with original running time data shown by the circles. The log-log scale is applied then linear regression is performed in (a-c) to reflect scaling behavior. CDBO: Centroid Discriminant Basis O; CDA: Centroid Discriminant Analysis; LDA: Linear Discriminant Analysis; SVM: Support Vector Machine: BO: Bayesian Optimization; SGD: Stochastic Gradient Descent; LR: Logistic Regression.
24

---

Published as a conference paper at ICLR 2026
J IMPLEMENTATION DETAILS
All data samples were converted into l representations without feature extraction or standardization throughout the paper, as the focus of this study to compare the general performance of various classifiers on common tasks, rather than optimizing feature extraction for specific applications such as image classification.
J.1 LINEAR CLASSIFIERS
Linear classifiers compared in this study include LDA, SVM, and fast SVM. Data were input to classifiers in their original values without standardization.
For CDA, the threshold for the coefficient of variation (CV) of performance-score was set to 0.001. 5-fold cross-validation is applied on the OOP search.
For LDA, the MATLAB toolbox function fitedised was applied, with pseudol Minddlliearrr as the discriminant type, which solves the pseudo-inverse with SVD to adapt to poorly conditioned covariance matrices.
For SVM, the MATLAB toolbox function fitesvm was applied, with sequential minimal op- timization (SMO) the optimizer. The cost function used the L2-regularized (margin) L1-loss (misclassification) form. The misclassification cost coefficient was set to C = 1 by default. The maximum number of iterations linked the number of training samples in the binary pair as 100 X Nuain- The convergence criteria were set to the default values.
For fast SVM, the LIBLINEAR MATLAB mex function was applied. For the dual form SVM using dual coordinate descent (DCD), the cost function used the L2-regularized (margin) Li-loss (misclassification) form; For the primal form SVM, both regularization and loss are in L2 form. The misclassification cost coefficient was set to C = 1 by default. A bias term was added, and correspondingly, the data had an additional dimension with value 1. The convergence criteria were set to the default values. Training epochs were set to 300 in general tasks and 100 in large-scale single-cell test.
For SVM-SGD, the passes of all training data are 10; batchsize is 10; learning rate is 1/V1+ + max; |,,122
Hyperparameter tuning was performed for fast SVM to achieve the its best performance. CDA, as well as CDBO, do not require parameter tuning.
Speed tests and performance test were from different computational conditions. To obtain classifica- tion performance in reasonable time span, GPU was enabled for SVM, and multicore was enabled for CDA and LDA. To compare speed fairly, all linear classifiers used single-core computing mode.
J.2 NONLINEAR CLASSIFIERS
For the experiments the nonlinear beemel-based approaches, on SVHN subset and chemical Clintox, the data were first divided into train-validation and test set with 5:1 and 4:1 ratio respectively, then the validation set was further divided into train set and validation with a 4:1 ratio.
The gaussian kernel parameter sigma was tuned on the validation set by Bayesian optimiza- tion with 30 sampling. For the tuned gaussian parameter, on SVHN subset gaussian CDA has coc ff = 0.2 and gaussian SVM has coeff 0.134; on Clin'Tox gaussian CDA has 1.34 and gaussian SVM has foeff = 68.9, where in gaussian kernel o² coeff www.ieeeeuurreeeeeeeeeiieeeeee
coeff
For CDA, the threshold for the coefficient of variation (CV) of performance-score was set to 0.001. For SVM, libSVM was implemented with 1.2rregularized (margin) L1-loss (misclassification) cost function and with default misclassification cost coefficient 1.
25

---

Published as a conference paper at ICLR 2026
K DATASET DESCRIPTION
The datasets tested in this study encompass standard image classification, medical image classification, and chemical property prediction, described in details Table. 1.
Table 2: Dataset description
Dalasel
&Samples
ATvaltrres
#Classes
Balancedness
Modality/source
Clasilication lask
Standard images
MNIST
imhalanced
image
digits
USPS
9298
256
10
imhalanced
image
digits
EMNIST
balanced
image
CI
60000
3072
10
balanced
image
objects
SVHN
99289
3072
imbalanced
image
house numbers
Rower
3670
1200
5
imhalanced
image
flowers
GISRB
26635
1200
43
imbalanced
image
trattic signs
STLIO
13000
2352
10
balanced
image
objects
FMNIST
70000
784
balanced
image
fashion objocts
Medical images
demmamnist
10015
2352
7
imbalanced inthllacedd
dernatoscope chest Ray fundus camera breast ultrasound blood cell microscope abdoeninal abdominal CT abdocninal
dermal discases
poeumonianmist
5856
pneumunia
retinamnist
1600
2352
5
imbalanced
diabetic retinopathy
breastamnist
inttalaneed imbalanced inttalaceed imhalanced inttalanced imhalanced
diseases
bloodmnist
17092
2352
8
blood discases
organammist
organs
organemnist
215883
784
11
human oegans
urgansmnist
organs
orgammnis33d
1472
21952
11
abdominal CT
human oegans
modulemmistdd
inttalanced
nodule mulignancy
fricturemnistdd
1370
21952
3
imhalanced
chest CT shape from abdominal
fracture types adrenal gland mass aneurysm excitatory/nhhiitory
adrenalmnistäd
1584
21952
imhalanced imhalanced
vesselmnist3d
1908
21952
2
shape from hrain MRA electron microscope
synapsemnist3d
1759
21952
imbalanced
Chemical formula
bace
1513
198
2
imbalanced imitllanced imbalanced imbalanced
chemical formula chemical formula chomical formula chemical Tormula
BACE1 cnzyme blood-brain barrier permesbility
BBBP
2050
clintox
1484
339
2
clinical toxicity drug activity
HIV
Large-scale single-cell sequencing data
Mouse brain
1306127
27998
10
imbulanced
single-cell sequencing
cell type
Data processing: Image data were used in their original value flattened vectors; chemical for- mula were processed by simplified molecular input line entry system (SMILES) tokenized encoding. SMILES line notation describing the structure of chemical entities using short ASCII strings. Chemical formula
stacked and aligned from the left since they have different SMILES lengths. The strings were mapped to natural numbers using a predefined dictionary, and those missing string positions were filled with 0. On these converted data performed different classifiers.
26

---

Published as a conference paper at ICLR 2026
L OPTIMIZER COMPARISON FOR CDA ROTATION
In this section we provide an expanded justification and empirical assessment of the one-dimensional (ID) optimizer used within the CDA iterative update. Although Bayesian Optimization (BO) was originally adopted, a ID optimization problem does not inherently require BO; several classical black-box optimizers can also be applied with low implementation overhead. To rigorously assess the optimizer choice, we conducted a meticulous study comparing BO with five alternative ID optimizers: (1) random search, (2) grid search, (3) Fibonacci search, (4) Brent's method, and (5) ternary search. All methods were evaluated under identical settings.
Figure [ summarizes the comparative performance of CDA under six different 1D optimization strategies across 27 real datasets, reported using AUPR, Fscore, and ACscore. CDA with Fibonacci and BO achieves the most frequent top-ranked positions, further confirming the empirical robustness of the proposed recommendations.
Based on these findings, several practical recommendations emerge. For maximum computational efficiency, Fibonacci search is the preferred choice and provides the most favorable balance between speed and stability. BO remains a competitive secondary option, particularly when oog-transformed data are used, BO tends to exploit smoothness more effectively in this domain. For large-scale datasets, Fibonacci search consistently offers the most robust and efficient solution.
(mnuticsse predicion AUROO AUROC)
Average ranting leds predicton AuRiC AUROC)
(PP)))
(mulicacs grea pee
Top-2 oecumnnce
20
Top-2 oosumence
20
Averaze ranking 5
15
15
Top-2 cocumenc (muticaass (Csoor)
Imuiliciooo prediction (Qvore)
(muticass prodictonn Froorc Ficore)
predciion (oore) )
Top-2 eocuretcc
Avango rarking
in:
Average rankinng of nundme
Avarage blaayypaa toain Sin
10"
80
Grd seacy
Ploessi saarcc
Anarch
10
Average binary-pair data size
Figure 9: Performance comparison of CDA using six different one-dimensional optimizers: Bayesian Optimization (BO), random search, grid search, Fibonacci search, Brent's method, and ternary search. Panels (a-b) show AUROC, (c-d) AUPR, (e-f) F-score, and (g-h) AC-score, reporting both top-2 occurrences and average ranking of multiclass performance across 27 real datasets. Panels (i-j) show the average ranking of running speed and the corresponding runtime plot. Error bars represent standard errors. CDA: Centroid Discriminant Analysis.
Table 3 reports the results on the same experiments with (i) final CDA performance, (ii) average running time, (iii) average number of CDA iterations, and (iv) average runtime per iteration, as summarized in the table below. Notably, across optimizers, CDA consistently converges to a similar performance level, demonstrating that CDA algorithm is well-posed and stable, regardless of the choice of ID optimizer. For runtime per iteration (iv), random search is the fastest due minimal structural overhead, whereas BO the slowest because of the cost associated with
27

---

Published as a conference paper at ICLR 2026
maintaining its surrogate model. CDA employs early-stop strategy, monitoring the stability of training performance across iterations. Among the tested optimizers, Fibonacci search provides the most stable training, yielding the lowest average running time and the fewest CDA iterations.
To further validate this conclusion, CDA with Fibonacci search was applied to a large-scale mouse- brain single-cell dataset containing up million samples for multiclass cell-type classification (see Section[H] The resulting performance and running time demonstrate that the Fibonacci based CDA significantly improves computational efficiency while maintaining high predictive accuracy, whereas even optimized SVM implementations require substantially longer runtimes.
The CDA convergence analysis with Fibonacci optimizer is provided in Section O
Table 3: CDA performance and efficiency across six 1D optimizers for original and log-transfommed
data.
Random search
Grid search
Fibonacci search
Brent's method
Ternary search
BO
Original data
Avg. performance (AUROC)
0.758
0.755
0.751
0.757
0.753
0.760
Avg. runtime (s)
4.681
4.339
4.137
2,473
2.838
3,230
Avg. number of rotations
36.278
46.330
43.811
29.689
29.252
35.441
Time per iteration (s)
0.136
0.096
0.099
0.099
0.107
0.114
tng-transformed data
Avg. performance (AUROC)
0.764
0.745
0.761
0.755
0.754
0.758
Avg. runtime (s)
4.693
4.313
4,003
2.503
2.546
3.355
Avg. number of rotations
35.796
46.330
43.507
30.244
26.178
35.759
Time per iteration (s)
0.134
0.095
0.098
0.098
0.113
0.108
28

---

Published as a conference paper at ICLR 2026
M STABILITY OF BO IN CDA WITH INITIAL FEW SAMPLING
To assess the stability of the BO-driven rotation process when only a few initial samples are available, we compared two BO schemes that differ only in the number of initial samples (all other BO hyperparameters and total BO budget were kept the same). In the first BO sampling scheme, the number of sampling set min(10, 10, max(4, Iter)) that gradually increase the number of sampling from 4 to 10 according to the current CDA iteration. In the second BO sampling scheme, the number of sampling is set always 10. The goal is to test whether the early-stage sampling searcity destabilizes convergence or substantially degrades multiclass performance.
The results Table 3 show that, across a wide set of image, medical and chemical datasets, perfor- mance is essentially unchanged between the two schemes (average AUROC 0.758 vs 0.754). The absolute average difference is 0.004, which negligible for our tasks and indicates that BO-driven rotations stable even when the initial sampling budget is small. It is even beneficial to use fewer sampling at initial CDA iterations.
Table 4: Multiclass AUROC for CDA with different BO sampling schemes.
BO with few initial sampling
BO control
with high initial sampling
Average AUROC
0.758
0.754
29

---

Published as a conference paper at ICLR 2026
N
SUPPLEMENTAL PERFORMANCES ON REAL DATASETS OF LINEAR CDA
(muliicios prediction Fscore)
(multicloes) prediction AUPR)
25
Awerage (uutiicless predicu prediction AUPR) "Y b.
Oo a G 2
Average ranking (mulliciass prediction Fecore)
Top accurence
20
Top-2 occurrence
10
6
20
(multiciass prediction ACscare)
Average ranking (multicioss prodiction ACccore) oo No
Top-2 oosurrence
20
Figure 10: Multiclass prediction performance on 27 real datasets of CDA-BO. Panels (a-b) show AUPR, (c-d) Fscore, (e-f) ACscore, reporting both top-2 occurrences and average ranking of multiclass performance across 27 real datasets. Error bars represent standard errors. CDBO: Centroid Discriminant Basis O; CDA: Centroid Discriminant Analysis; LDA: Linear Discriminant Analysis; SVM: Support Vector Machine; LR: Logistic Regression; BO: Bayesian Optimization
b
(multichss prediction AUROC)
(mullicteso) preciction AUROG)
(multiciaos preditioonul AUPR)
(multiciess predetion AUPR)
23
Top-2 occurrence
Averaag canling
Top-2 occurrence
20
Avarage ranking
do " & a
15
15
10-
10
(muliicless prediction ACscore)
Average ranking (multiciass prediction ACscore) e o di in to 10 a
(multidass prediction Fecore)
Average ranking (multidaass prediction Fecoor) d
25
Top-2 occureence
20
Top occurrence
20
20
70.
Figure 11: Multiclass prediction performance on 27 real datasets CDA-Fibonacci. Panels (a-b) show AUROC, (c-d) AUPR, (c-f) F-score, and (g-h) AC-score, reporting both top-2 occurrences and average ranking of multiclass performance across 27 real datasets. Error bars represent standard errors. CDB0: Centroid Discriminant Basis CDA: Centroid Discriminant Analysis: LDA: Linear Discriminant Analysis, SVM: Support Vector Machine; LR: Logistic Regression; BO: Bayesian Optimization
30

---

Published as a conference paper at ICLR 2026
O CDA-FIBONACCA CONVERGENCE
We analyzed the relationship between the actual stopping iteration of CDA and the average binary classification performance (ps-score) across different datasets and iteration limits. The results revealed two distinct regimes. In the right half of Fig. 12 for tasks converging before 50 iterations, we observed a significant negative correlation (Pearson's R 0.099), indicating that datasets with lower performance required more iterations to converge. This highlights the importance of allowing at least 50 iterations, as early stopping before this point may prevent convergence for more challenging tasks. In contrast, for tasks exceeding 50 iterations, the correlation was weak and positive (Pearson's is R = 0.089), suggesting that beyond this threshold, the number of iterations no longer plays a significant role in ensuring convergence. This indicates that once 50 iterations are reached, CDA can stabilize regardless of the underlying task performance.
To further validate this observation, in the left half of Fig. 12 compared the distributions of ps-scores obtained under maximum iteration limits of 50 and 150. The estimated probability difference between the
densities were identical, and the Wilcoxon signed- bank test confirmed two conditions (p=1). This result supports the conclusion that extending the maximum number of iterations heyond 50 does not provide systematic benefits in terms of classification performance. Note that the correlation before 50 in CDA-Fibonacee is weaker than CDA-BO in Figure] indicating that the Fibonacci optimizer helps CDA converge faster.
image
image
Wilcosoon
Chemical property (50 iteraskon))
signed rank test p=1
Standard
iteraitonnl
Average binury dessificttoo pa-score
Medical (150 merattons!
Medical data (150 max steratioss
Chemical property data (150 max iterations)
Pearson's R=-0.099
+ Pearson's
0.7
Ro00.089
+
0.6
*
*
50 max iterations 150 max ieeraioons
100
150
Distribution
Figure 12: Convergence characterization of CDA-Fibonacci. In the right half, relationship between average binary classification ps-score and the actual stopping iteration of CDA across four dataset categories (standard images, 2D medical images, 3D medical images, and chemical property data) under two maximum iteration limits (50 and 150). A negative correlation was observed for tasks converging before 50 iterations (Pearson's R = -0.099). while a weak positive correlation was found for tasks converging after 50 iterations (Pearson's 's R 0.089). The gray dash lines show linear regression of points till 50 and points between 51 and 1.50. The black vertical line indicates the 50-iteration threshold. In the left half, probability density distributions of ps-scores under 50 and 150 maximum iterations show substantial overlap. The Wilcoxon signed-ran test confirmed no difference (p=l), demonstrating that increasing the maximum iteration limit beyond 50 does not improve classification performance.
31

---

Published as a conference paper at ICLR 2026
P LOG-TRANSFORMING THE DATA FOR CDA TO HANDLE OUTLIERS
We performed controlled simulations under three conditions: Gaussian data, elliptic data, and log- transformed elliptic data. Each simulation consisted of 10,000 samples with 10 features, and the covariance structure was specified using a Toeplitz matrix. The distance between group centers was gradually increased to generate different levels of class separability. We included SVM in the comparison, since its margin baaggn--bseed formulation is known to provide degree of robustness to outliers.
The results shown in Figure 13 indicate that for Gaussian data, CDA performs comparably to LDA and SVM, confirming that it does not lose predictive power under ideal Gaussian conditions. For elliptic distributions, CDA is affected by outliers, as expected. However, applying a log-tran transforrr to the features effectively mitigates the influence of extreme values. After log- transforming the elliptic data, the performance of CDA improves substantially, and this preprocessing step benefits CDA more than it benefits LDA or SVM.
AUROC
AUPR
Fsccre
ACaczre
Goursian
stributtio data
07
(DA
055
35
15
05
05
15
05
Elppi
Log to--traaffarmedd atpie diotibution dats
Group center distance
Figure 13: Controlled simulation results comparing CDA, LDA, and SVM under different data distributions. Panels (a-d) correspond to Gaussian data, (e-h) to elliptic data, and (i-l) to log- transformed elliptic data. Within each group, the metrics shown are AUROC (a, i), AUPR (b, f. j), F-score (c, k), and AC-score (d, h, l). The simulations consist of 10,000 samples with 10 features, and the covariance structure specified using Toeplitz matrix. Group center distances were gradually increased to evaluate performance under varying class separability. These results illustrate that CDA maintains comparable performance to LDA and SVM for Gaussian data, is affected by outliers under elliptic distributions, and benefits substantially from log-transforming features.
We further applied the log-transform to 27 real datasets, with results summarized in Table 5 For CDA, both the winning rate (from 0.519 to 0.741) and the average performance increased. For LDA, although the winning rate improved, the average performance decreased, while for SVM, the log- nog-transform led an overall decline performance. These results suggest that the log transform well suited for variance stabilization and outlier control, and that CDA uniquely benefits from this preprocessing step. yielding enhanced overall performance. Based on these findings, we recommend incorporating log- transforr preprocessing step when applying CDA to data with heavy-tailed or elliptic distributions.
32

---

Published as a conference paper at ICLR 2026
Table 5: Binary classification performance comparison under original and log trtarrnnffomed data.
CDA
LDA
SVM fast
Original
Log
Original
Log
Original
Log
Average performance
0.800
0.805
0.758
0.742
0.765
0.757
Winning rate
0.519
0.741
0.481
0.688
0.704
0.407
33

---

Published as a conference paper at ICLR 2026
Q PERFORMANCE OF CENTROID-BASED METHODS IN DATASETS WITH MULTIPLE SUBCLASSES
We analyzed a dataset in which each class naturally decomposes into several subclasses, for example, due to illumination changes or rotations. To illustrate this, we selected two classes from the GTSRB dataset and applied Isomap dimensionality reduction (k-nearest neighbors, k=3), marking the ground- truth subclasses in the same color. The resulting embeddings are shown in Figure for the two groups.
We first applied a -rg-transform to the data to assess the potential impact of outliers. As indicated by the results T uuing CDA with Bayesian Optimization, the log- nog-transform did not produce significant difference, ruling out outliers as the primary source of variability. Therefore, the challenges in modeling this dataset can be attributed to presence of multiple subclusters within each class.
We then compared CDA, CDBO, LDA, and SVM on the GTSRB dataset. The results in Table 7 show that the purely centroid-based baseline CDBO performs poorly (AUROC = 0.589, close to random), confirming that single centroid is insufficient for classes with multiple subclasses. In contrast, CDA-while still centroid-driven -ato--diivve-inncorrrrttsss adaptive discriminant corrections and achieves AUROC 0.878, outperforming LDA (AUROC 0.821). This demonstrates that, among centroid-base methods, CDA is substantially more robust to intra-class multimodality. While all SVM variants achieve AUROC > 0.90, CDA remains competitive, offering strong balance between robustness and computational efficiency in datasets with multiple subclusters per class.
b
400
500
200
400
200
200
400
800
2000
-100
200
1000
1000
1000
-400
-2000
800
1000
Figure 14: Isomap dimension reduction results of two classes in GTSRB dataset (Pictures of different traffic signs). The same subclasses are marked with the same color according to the ground truth, which shows there are obvious subclusters.
Table 6: Multiclass prediction performance (AUROC) of CDA with Bayesian Optimization on the GTSRB dataset under original and transooaforeee data.
Data
CDA BO
Original
0.878+0.01 0.887=0.01
Log-transformed
Table 7: Multiclass prediction performance (AUROC) on the GTSRB dataset.
Method CDB0
CDA
LDA
SVM
AUROC
0.589#0.01
0.87840.01
0.82140.02
0.983+0.003
34

---

Published as a conference paper at ICLR 2026
R MORE ABOUT LINEAR CDA CLASSIFICATION
While the core CDA algorithm is efficient, the kernelized version inherits the standard computational bottleneck of kernel matrix construction. This bottleneck limitation of all kernel-based linear classifiers. When any linear method (e.g.. LDA, SVM) is kernelized, the dominant cost comes from constructing and storing the N N kernel matrix, which scales quadratically with the sample size. Thus, the kernelized CDA inherits the standard kemel- matrix bottleneck shared universally by kernel methods, rather than introducing new one. Although, we provide preliminary results that CDD can be formulated in the nonlinear versions via the kernel trick, we stress that in this paper we focused on linear CDA classification, because linear methods remain widely used in several scientific domains- particularly neuroscience, biomedicine, and social sciences where explainability, stability, interpretability, and transparent decision boundaries are essential. In many neuroimaging studies, for example, linear classifiers are preferred [Shen al. :); Misaki et al. (2010) because they provide direct insight into which brain regions contribute behavioral or cognitive prediction, making them standard analytical tool despite the inherent nonlinearity of the data. To further assess the capability of CDA under nonlinear data distributions, conducted additional experiment
specific CDA, but is an inherent
real fMRI-behavior prediction task, where the goal is to classify human subjects into four behavioral score categories (fluid intelligence (gF); 4 groups are low gF, low-middle gF. middle-high gF and high gF). Although the underlying neurobiological patterns are highly nonlinear, this setting reflects realistic domain where linear models are still favored for interpretability and reproducibility [Shen et al. (207); Misaki et al. (2010). The results (Table show that CDA consistently outperforms all other linear classifiers, including LDA, logistic regression, and several SVM variants. This demonstrates that CDA can better capture discriminative structures even when the data distribution is nonlinear, while maintaining the transparency advantages of linear models.
Table 8: Test set multiclass prediction performance on fMRI-behavior dataset (N=1076, M-64620, 4 classes).
Method
AUROC
AUPR
Fscore
ACscore
CDA
0.577
0.574+0.02
0.571#0.02
0.495+0.04
LDA
0.538+0.02
0.558+0.03
0.536+0.03
0.384+0.07
SVM fast
0.534+0.005
0.53640.005
0.53240.007
0.440.04
SVM SGD
0.54420.00.
0.547+0.007
0.545+0.007
0.42420.04
SVM
0.538+0.006
0.539+0.00 007
0,538+0.000
0.419#0.03
LR fast
0.537+0.004
0.539+0.005
0.536+0.006
0.4+0.05
35

---

Published as a conference paper at ICLR 2026
S FULL CLASSIFICATION PERFORMANCE ON REAL DATASETS OF LINEAR CDA
In the below tables give full classification performance 27 real datasets based on 4 metrics with binary classification (Table. 9-12) and multiclass prediction (Table. 13-16). In multiclass prediction, the performances on datasets with only two classes were filled with the one binary classification.
S.1 BINARY CLASSIFICATION
Table 9: AUROC (Binary Classification Performance)
EDA
Fist primall
Fae SVM
SVM
Standard images
0.965-0.002
0:901-00022
0.58+020022
0.98640.002
0.986o00002
0.98688.02
0,986511002
0.9862010022
EMNIST
0.6960011
0.797411.01
0.74120011
0.007500111
0.76220.01
0.78720001
0.75780002
SVEEN
0.52840.003
0.067=0.005
0.57849.004
0.53720.00
Bower
0.78340022
0.759+0.00
0:571=0.01
0.705#0.33
8.754=0.13
0.722=0.03
0.71+000
GTSRB
0.76740:03
0.97260.001
0942200002
0,99510.00
0.99540,0003
0.905.0.00M4
0.99
0,99510.0004
0,90110, 0005
IO
0.7254002
0.781+0.00
0.66760.01
0.75840 02
0,75760022
6.791:0.02
0.766:0.02
0.761:0.02
0.71240
FMNIST
09372001
0,975g00000
0.913800006
0.976250006
0.976400006
0.578:00005
0.975.00006
0:976300006
0,976611.006
Medical imuges
Germmnii
06826001
0,753
0.8
0.67623002
0,70830:02
0.69R200112
0.60820.02
0.712
poeumooiammist
0.8320
033320
0,912220
0,94120
0.94320
0,941a0
0.942401
0.941
6.9
edimmmiist
0363el000
0.66221..4
0616e0002
0 0651=0008
0.626e00
0.632
0.615
6222000
B 0419=0003
becsstanist
0.6600
0,76320
0,703a00
0.726e00
0,757:0
0.705ell
8.7348ll
0L685
bloddmnist
0,8940 02
0,94750.01
0.98300
0 051+0001
O 05556001
0.9550001
0.94660.0 01
0.96760.01
B 05544000
regarsmmist
0:80740.009
0.948-0.008
095+0008
0 0288+001
0.05+00.08
0.95%=8U008
0.930+0.01
D 0057+0008
0.95440.008
organennist
0.8920001
0.92530.01
0908200011
0.39520001
0.13200011
0.928
0.911
0.91920.01
0,50250002
organsmnist
08312001
0.83650.01
0.36660.01
0.34250022
0.8722001
0.868
0.85320.02
0.880001
0.54000000
organamint5d
0.92ka001
0.55720.000
0.5658211077
L1666200077
0.36620U177
0.9622000777
0.966220007
sodaicmtiatdd
0.7155g
0,70120
0.732M)
0.65750
0.70220
0.725all
0.9122l
0,734
0.73300
factunemmis13d
0U671s0016
0.556-0.01
0:525+0.04
0.576+0.007
0.57+00.08
0.6+00004
0.592+0.004
0-61240007
D. 0.59+0001
adretalmnist3d
0.65320
0,75620
0,69250
0.69720
0.61920
0.64720
0.63760
0.66520
064120
veselmnistäd
0.605500
0A8540
0,68110
0,61:0
0.64820
0.62840
0.60450
0,630
054400
0.539u0
0.54450
0,508.z)
0.527c0
0.518all
0.525at
0,539
Chemical formula
0.621:00
0.6842)
0.677s0
0.097all
0.637
BBBP
0.71120
0.74350
0.6932)
0.707.0
0.646:2
0.69725
0.71200
clintos
0.65c0
0.543e0
0.517e0
0.515e0
0.508el
0.514e0
0.515e0
HIV
0.537m)
0.506ee)
0.505ell
0.506er)
36

---

Published as a conference paper at ICLR 2026
Table 10: AUPR (Binary Classification Performance)
Duuset
CDBU
CDA
LDA
Fist SVM dual
Fost SVM primul
Fist SVM BO
SVM SGD
SYM
Fast LR
Staddrrd images
MNIST
0.85720.004
0.98200.00
0981=000C
0.98550.002
0.98624UN0
8,986.0.00
0.9%cz20012
0.986.0
0,986:13.02
USPS
0.90620.005
0.99200000 001
0198520002
0,9920.002
0.99201002
0,9920 DOL
0.99120001
0.99200002
0,991:9.001
EMNIST
0.93840.003
0972=00.01
0964-00001
0:97=0.001
0.97-1.001
8.97336 091
0.97600022
0.97=0.001
0.97=0.000
CIFAR10
0.69750.01
0.707=0.01
0,341:0.01
0,762+1101
0.784x0001
0.807
0.774.0.11
0.787=5.01
0.757401.00
SVIN
0,52840.003
6.8229.006
0.559x00000
0,57760.0 005
0.604:00044
06682400006
0.566a0.005
0634600006
0.56760.009
flsser
0.70440.00
0,341+0.00
0:571=0.01
0.712+0:00
0:707=0:03
0,759+0.83
0.75e0.03
0.712+0.00
0.73640.00
GITSER
0,257+0.003
0973+00.01
0.934-0.000
0,05-000003
0.996=0..003
6.996=0.0003
00901000000
0,995+0.0003
0.90-000003
STL,00
0.72480.02
0,76250.02
0.60720.01
0.759
0.757=0022
0.791:0182
0.772s0.02
0,762:0.02
0,71360000
FMNIST
0,93760.01
099550.006
0.973=00006
0.76600006
0,97220005
6.978
0.975600006
0.97600.06
0.97760.006
Medical Images
dermammust
0.5360.01
0,
0661:0.02
0,729
0.746:0002
0,74
0.6660000
6,552
0.7170000
yocomeniamnis
0,81750
0,3110
0.92240
0,937
0.94420
0.96620
0.92760
0.945e0
0.94610
retiumnist
0.61481.03
8.649
061220.02
0.4128002
0.34290.22
0.637211.33
0.62200.13
0.632
0.631a01.0
becastmnist
0.65300
0.75920
0.6920
0,74550
0,76620
0.71Ke0
0.726a0
0.725a0
0.73ka0
Nondanie
0.839=0.00
0.94750.01
0.80=6..22
0:93660001
0.95e0000
0.957
0.45=00011
0.957
0.956a0.01
eggnttatinis
0.902500009
0.95
0:929a0001
0.839 1008
0.941=0.11
0.556000006
0:955a10088
segutemtnist
0.901
0.3120.009
0.98520011
0.99311.11
0911:001
6.M32 109
0.913
0.9186001
0.90261..2
0.839+0.01
0.892-6.01
0867-6.01
0.64+6.0
0877001
0.89460.01
0.86+0.02
0.68-0.01
0.845+0.01
"egammiech)
0.934+0.009
0.53-00.06
0.04-00.00
0996-660007
0-966-0007
0.967=8,007
01965000017
8.97788.077
0.917+0.00
todilennistdd
0.720
0,771:0
0,74560
0,68560
0,20980
0.74960
0,71120
0.73260
0,75230
fracurrmmintdd
0,66360.85
0,50650.02
0531-0.05
055740.000
0.58600008
060660.004
0,640005
0.61850.01
600000001
adronamnieedd
0.6500
0.77440
0,70540
0.70840
0,62760
0 66340
0/65740
O. 0.59400
0666500
vesselnnisthd
0.58240
0.61110
0.59420
0.62740
0.69230
0.668111
0.54629
sysppemmiscd
0.55750
0.54260
0,54220
0.53360
0.55300
0.55360
0.53840
0.57240
0650
Chemaccal formula
0.6200
0,70450
0,68540
0.64360
0.67930
0.69900
0,65220
0.6420
0.085e0
BRBP
0.701a00
0.74750
0.712s0
0.751:0
0.723e0
0.742a0
clinsos
IU602:00
0.57etl
0.548:1)
0.553a0
0.575e0
0.5344ll
0.53cll
HIV
0.56580
0.55WA1
0.364a0
0.58540
0.596a
37

---

Published as a conference paper at ICLR 2026
Table 11: F-score (Binary Classification Performance)
Dutset
CDBO
CDA
LDA
Fast SVM doal Fist SVM gramal
Fas SVM BO
SVM SCDD
SVMI
Fost LR
Stadard imsgee
MNIST
0.957200004
0.9k520.002
0:81200002
0.885211002
0.996e20.012
0.9%cezuuK
0.9%c58002
0.98620.002
0.9862110022
USPS
0.06640.00 DBS
0.9R9-0.001
002
0.5000.002
0.99-0.002
0.99-0
0.91:88001
0.9900002
0.991
EMNIST
0:92640.03
0.972=0.001
0364400001
097H0,001
0.9200001
0.973x80001
0,97800002
03740001
0:97H0,002
CIEARIO
0.6663001
0.797:0.01
0.74150.01
0,74750011
0.78450001
0.80756.01
0,751=0.02
0.78750.01
0.35720002
SVIEN
0.52540003 003
0.66449.005
0.555400003
0.3120.01
0.56840.00 005
0,57420
0.471:0.01
0,57240006
0,56360.009
Bower
0,701a002
0,738
0.5720.01
0,709g000
0.705500.03
0.754:6.15
0.715
0.70940005
0.73281003
(ITSRB
0.74500.05
0,972-0.001
0:911=00022
0,90540.0003
0.996=41.8013
0.996a0
0,994
0.925+0.0083
0,905+0.0003
STE.IO
0.722x022
0.751=6.02
0.66620.01
0.756e0
0.75ka00
0.791=6.12
0.76e0.12
0.761=0.02
0.712=0003
FMNIST
09375001
0,975:0.006
0.93340000 006
0.97640.006
0.97640.006
0.97840.055
0.975.0.006
0,97640
0,97610.006
Medical imsgges
Germannist
0621
0,736...02
067720.02
0 0622:0.0
0.726x0.02
0 0.7220.02
0.595=0.05
0.33100.2
0.680.044
youumnniamiis
0.812:0
0.11s0
0,921+0
0.937a0
0.944:0
0.945e0
0,92500
0.944e0
0.96000
retinmmnist
0.5962002
0,6390003
06120002
0611a9004
0.3310005
0.586
0.6550005
bocastmisst
0.65120
0,75920
0,68450
0.73950
0,765:0
0.71640
0.725a0
0,722
0.712a0
bondmmist
0:94720.01
022422002
0192=0001
0195500011
0.95750.11
0.943=0.01
0.957=1001
0.949500000
055
092258002
0.95950U00
0.955a11008
reginemnist
091100011
0.911=0.01
0
regensmmie
033360001
066660.01
0,334-8022
0.80304.01
0.85+0.02
0.544+0003
ocpatmnistbd
sodalemtiaad
0.743e1)
U:70Gatl
0.731aa0
Insturemaiatd
0.512=0015
0.31420005
0.577880007
0.57940.007
vssclmmis/3d
smppeenield
0,5340
0.53940
0,4520
0,693a0
0,501
0,69360
0.51180
0,522
0.4860
bace
0.619u0
0,70640
0.68410
0.56920
0:678.0
698840
0.63440
0.63720
0.6910
BDBP
0.899s0
0.74760
0.718410
0.751=0
0,73640
0.71640
0.733a0
climox
0.54g0
0.569200
0.54724)
0,517g0
0,515g0
0.5220
0.50621)
0.513
0.51520
III
0.331a0
0.56220
0,548a1)
0,51120
0,504g0
0,511a8
0.501at
0,501
0.51120
38

---

Published as a conference paper at ICLR 2026
Table 12: AC-score (Binary Classification Performance)
Dutsee
CDBO
CDA
LDA
Fast SVM doal
Fist SVM primal
Fas SVM BO
SVM SCID
SVMI
Fast LR
Stadard imagee
MNISTT
0.95720000
0.98520.002
0:81200012
0.985211002
0.586e20.01
0.986z6u002
0.9862
0.9862110022
USPS
0.96640.00 DBS
0.9R9-0.001
0.92+00002
0.5940.002
0:99-0.002
0.99+0
8:91:48001
019900002
0.991
EMNIST
0:92220003
0.972=0.001
0364500001
097H0.001
0.09900.011
0.973
037400001
0:97H0,002
CIEARIO
0.6920001
0,796s0.01
0.765001
0.725:0.02
0.78420011
0.80726.01
0,72s0.02
0.78740001
0.75ha0.02
SVIEN
0.52440000 002
8.61469.044
0.49220008
0.35260.03
0.4444002
0,11900.02
0.30220.01
0.438:0
0:461:0.02
Bower
0.69ka0022
0.73320102
0.56720.11
0.20120005
0.69k20003
6,74g0013
0,675=00.05
0.74t103
0.72561103
(ITSRB
0.753e0:00
0.97=0.00
0:341:0002
0.995501.0044
11995x11.8013
1.995=000004
0,99000006
0.995e11.0044
0,90440,.0044
STE.1O
0.72x0.01
0,779:0.0
0.66450.01
0.75et1.02
0.755a0002
0.79s0.112
0.743e0.02
0.76a8
0.711=0100
FMNIST
093660.1
0,97560.006
0.97340 006
0.975e0.006
0:97640.005
0.978:0.005
0,97550.006
0,97640 006
0,97610.006
Medical imsages
Germannist
065840002
0.7246.02
04
0.53540
60220.05
0,620005
0.323
60640.05
05
youumnniamiis
0.85700
0.92240
0,908+0
0,94
0.942:0
0.9460
0,94240
0,94
0
retinamnist
06220.03
0.63200055
0.3072005
0.1132007
044240.04
0.54120.05
0.465
0.51820006
becastmiist
0.64120
0,75120
0,69820
0.62200
0,732:0
0,65840
0.72240
0,06120
0,59g0
bondmmist
0288960
0.546e11.1
029020002
01949:8101
019540001
0.954.0.111
0.94420
0.56c20011
039920001
0.94620.000
01952401009
0.95%20000
0.33=
0.954
reginemmiss
0.927
regensmmie
0:8224001
0.6620.01
0.864=8.01
0.825-0.02
0
0
oepatanistkd
0,96440.077
Insturemaistd
0,2912004
0,514
0.56240001
0.602200
vssclmmis/3d
0.547=0
smppeenieed
0,89840
0,50640
0.061220
0.10
0,19,0
0,19940
0:25340
0,239
0.13720
bace
0.6220
0,704400
0.678440
0.483a0
0.673a0
0.69110
0.58340
0.621
0.8820
BDBP
0.89340
0.71840
0.60110
0.63a0
0.63260
0.501
0.62640
66660
climos
0.63420
0.36520
0,238
0.080921)
0.0859210
O. 0.086921
0,036820
0.089951)
0,009000
III
0.471:0
0.46520
0.15921
0.060724)
0.027324)
0.040720
0,0205a00
0.02733g)
0.040720
39

---

Published as a conference paper at ICLR 2026
S.2
MULTICLASS PREDICTION
Table 13: AUROC (Multiclass Prediction Performance)
Ducaset
CDBU
CDA
LDA
Past SVM dual
Fist SVM grimal
Fist SVM BO
SVM SGD
SVM
Past LX
Standard images
MNIST
01997a1111
1.985a00.005
01505111005
0.96728.014
113652000055
0,96758U144
USSS
0.14+0001
0.971-6.034
0.949-0.000 036
0:974+0.005
0.974-0:005
0.973-0 004
0.974-6.004
0.77+00.05
0,97668.004
EMNIST
0.7336001
0.89620.008
0,879=0.019
0.891
0.58810.008
0.898=0.008
0.88310.01
0.89220.00s
0.87860.01
CIFARIO
0.599e11.02
01671
0.627750.111
0.64:51.00
0.661:0.02
0.681=0.01
0.66560122
0.62770.05
SVHN
0.522500.006
1163820.011
0.531=00017
0.536sl1.01
0.551
0.534g))006
0.550a00.11
0.54220.01
061+0.03
0.554+0.02
0,629H0
0.669e0.03
0.612.0.05
0.65t+0.05
0.98210.003
0.648H11002
0.G4R=0002
0.66369002
0.584e0.12
FMNIST
083610.03
0.1766.02
0.92.0.02
0.92440.00
0.93300.02
0.92760.02
0.91840.02
0:92460.02
0,924.0.02
Medical images
demmammist
061440.00
0.688.0.03
0.58840.02
0.59540.00
0,
0.60
0.500
0.80640.05
0.563+0.02
pacumeniannist
0.8720
095329
091240
0.9120
0.93320
0,94110
0.4229
09412
0,9422
0.575et100
B622411.44
D. 1.50240.03
0.59G+0004
0.60Ke0.04
061-0.04
0:50360.04
5.5%+0.04
brasimmist
0.6620
0,76340
0,70340
0.72ka0
0,75740
0,706
0.73400
0.70910
0.68810
bloddmest
0.799611.4
0.89320.002
0.89120005
0.7770000
0.8912000
organamnist
0.11321103
1.8882002
0.8520004
0.89120.00
0,950.02
0.89220112
0.89220.05
orpinemniet
0.0090001
1156740.03
0.8346.05
0.8T 181T00004
0,89-0000 01
0.862=0.03
0.33++0.01
0.84740.05
0.796+0.05
0.69110.00
0.76110.03
0.23510.03
0.7.0.03
0.33340.00
0.75420.03
0.73210.11
0.71650.03
0,70550.05
01667a1103
0.925=0.02
0:915a00122
0.834g0016
redulemminn3
0,715e0
0.78ta0
0,702H0
0.235e0
0.543H0
0.622H11.4
0.54400.04
0.576=0.01
0.57640.000
veselomiedd
0.539a0
0.546a0
0.50810
0,52740
0.51840
0.539e0
0.51740
Chemical formula
bace
0.70540
0.69340
BBBP
0,711:0
0.743:0
0.69330
0.66720
0,707a0
0.7120
0.68660
0.69710
0.71240
clitts
0.550
0.575ed
0.54k+t)
0.51750
0.515e0
0.519H0
0.5084]
0.51ee1
0.515e1
HTV
0.6:0
0.61640
0.53740
0,51a0
0.506e0
0.51.0
0.50560
0.50610
0.51a0
40

---

Published as a conference paper at ICLR 2026
Table 14: AUPR (Multiclass Prediction Performance)
Donasct
CDB0
CDA
Fust SVM
Fast SVM prinal
Fiet SVM BO
SVM
Standard images
MNIST
0399211111
0,958
11560211104
0.96720004
0.905200004
0:906000044
0.96728001
USPS
091840.11
0.971-6.005
0:909-0.005
0:974+0.004
0.974-0.005
0.934-0004
0.974+0.004
0-973+0.005
0.976:4.
EMNIST
0.77620.01
0,89620.008
0.88=00009
0.89140.008
0.88820.008
0.898:0.008
0.85120:009
0.89250.005
0.678
CHEARIO
0.681.01
3.675.01
0.62760.11
016468).12
0.66210011
0.68:8001
0651=000
0.666600122
0.61550.12 U2
SWHN
0.528801.00
1666220.11
0.5332001166
0.546a0).005
0.572501000
0,589500077
1.505200:11
0,59750008
0.55220.01
Rower
10000
0.665+0.02
0. 0.554+0.02
0.611+0.02
0 00
0.671=8.02
0.44+00011
0.612+0.02
0.652+0.02
GTSRD
0619:0001
0.89120.01
0.80520.01
0.98 150000
0581:0.000
0.98220.002
0.93940:005
0,98120.00
0,97550.00 003
STL10
0.00all102
0.63341.022
0.596600.12
0.648610.2
0.6475411000
0.663
0.6540.12
0.651
0.585=0.01
PMNIST
0336:0.03
Medical
0:59240.00
0.64500.02
0.633.0.0
0,62900.03
066500.01
0.649a0.03
0,50600.22
pacumoniannist
0.98420
0.22-0004
brreemmiat
0,653a0
0.79940
0,6940
0,74340
0.766400
0.71844
0.22600
0.72540
0.73640
Sloddmiit
0.78510.44
0
081840.03
50600.05
0.89560
089440.03
0.7220.00
0.897
89550000
ocganammist
08220.03
(189260.0
0.59920-12
12003
0189260000
0,900520002
0.57110000
0.89840.022
0,89520.02
organemnist
0318440.03
I18774MM22
0.838-0-05
0 1+0.44
0A39H0000
D. 0.87H0
0.83640.05
0.847+0.05
0.793+0.05 DS
organemnist
0.69720.02
0.76420.03
0.74130.05
0000310:03
0.73610.00
0,76220.03
0.734g0.00
0.748
0.89820.04
organmaisctd
01867211022
11913
0.93:102
0.92
1192520020
0:506600.12
U.N2a0U15
sodulemmist3d
0,720
0,77120
0.45201
0.70M60
0.749H0
0.711a0
0.33241
0.75220
Facturemnia&d
0.91751.04
5256-0.02
D. 0.5EH0004
0.57800.004
0.583+0000
O. 0.582=0 008
0.579-6.007
0.57900.005
O 0.585=0.D09
adrenalmniscdd
0,6560
0:77650
0,70440
0.70RA0
0.62720
0,66360
0.6720
0.68910
0.66550
0,582:0
0.671a
0.692a0
0164Gell
01061
synggammiisld
0.53740
0533+0
0.558e0
0.533e0
Chemical
0.6240
0.704a0
0.68500
0,6410
0,565a0
0.58340
0.55840
0.612:0
0.578a0
0.58410
0.601s0
0.58510
0.50640
41

---

Published as a conference paper at ICLR 2026
Table 15: F-score (Multiclass Prediction Performance)
Donasct
CDB0
CDA
Fust SVM
Fast SVM prinal Fiet SVM BO
SVM
Standard images
MNIST
0.99221111
11560211004
0.96220004
0.905
0:906000044
USPS
091740.11
0.971-6.005
0:969=0.005
0:974+0.004
0.973-00005
0.973+0000 004
0.974-0.004
0-973+0.005
0.976.4
EMNIST
0.733a001
0,89620.008
0,87950.009
0.89120.008
0.88820.008
0.89820.006
0.88320.009
0.89250.005
0.678
CHEARIO
0.59560.01
3.770.01
0.62760:11
0.634210.02
0.66210.11
0.67950.01
0.666600222
0.600500.12
SVHN
0.509801.006
11449201.11
0.529500RR5
0.52680.000
0.542201/007
0,542001
01521
0.55a0011
0.33
Rower
0.599+0.00
0.60+6.02
0.55340.02
0,629+0.02
0 00
0.66550.02
0.28+0.01
0.61+0.02
0.85+0.02
GTSRD
0.5861001
0.88610.01
0,78220.01
0.9820.003
0.81:0.000
0.982:0.003
0.95740:005
0.98.0.003
0,974g0000
STL10
0.60111102
0.63441:22
0.59720..12
0.646a10.02
0.644Get)
0.662:8.12
0144:0002
0:651
0,57e0002
PMNIST
0835e0.00
0.924e0.02
Medical
demuumint
0.571x0:00
0639.0.02
0.61500.03
0.533+0.01
0.624e0.05
0,572+0.02
pacumoniannist
0.98220
0:945
0.54740.04
0.592+0:04
bereemmiat
0,65140
0.7940
0.68660
0.73940
0.76540
0.71644
0.25400
0.7240
0.71240
Sloddmiit
0.779.0.04
0.7760.03
0.1720.03
S8744000
0.89410.0
088440.03
0.
0.89740.02
89440.00
ocganammist
0.11241003
0.29920.0
0.59920012
0.4722044
0189260000
0,900320002
0.309
0.898400022
0.89420002
organemnist
IL774
0.8376605
0A3PH0000
0.867=0002
0.847+0.05
0.789H0.0P DS
organemnist
0.685.0.02
0.7620.3
0.74
0,6920001
0.73610.00
0,76:0.03
0.72910.01
0.74850.05
0.88440.04
organmaisctd
0236251.02
1500
05222102
0.92440.02
0:9220002
0:502600112
0.S0PES
sodulemmist3d
0.695500
0.769200
0.4320
0,70820
0.74720
0.70K80
0.31211
0.751201
Facturemnia&d
0.60111.84
BA86-60022
D. D.54760044
0.57500.002
0.581+6000
0 0,579+0 008
0.575+6.006
0.57740.008
0.577H0.01
adrenalmmisldd
0,6500
0,77120
0,70340
0.707a0
0.62540
0.65940
0.51200
0,61110
0.65610
0.5680
0.663
0.625ell
synggammishd
0.529a0
493+0
0.501
0.409+0
0
Chemical
0,61940
0.704ah
56960
0,678e0
0.63740
0,531a0
0.56240
0.54840
0.511a0
0.504.0
0.501a0
0.501a0
0.50410
0.5
42

---

Published as a conference paper at ICLR 2026
Table 16: AC-score (Multiclass Prediction Performance)
Donaset
CDBO
CDA
LDA
Fast SVM dual
Fist SVM peimal
Fast SVM BO
SVMIGD
SVM
Post L8
Standard Imsaee
MNIST
0.868+0.0
0.9
095640.006
0.964+6.005
0.964+0.00 D05
0.96660 ORS
0.964+0.005
096540.05
0.966-0.885
USPS
0.90+0011
0.95=00005
0.98400.077
0,974+0.005
0.973+0.005
0:973+0.005
0.97440.004
0975+0005
0.975=0.004
UMINST
0,708a0002
0.854/0001
0.86320.01
0.87840.01
0,87420.01
0.886690.01
0.36720.01
0.879:0.01
0.85550.00
CIFARI0
0.404ad005
0.562=0015
0-48461.03
0.401=00.05
0.542200.1
0,579:80.13
0.48740006
0.548e105
SVHN
0:225e0.05
0,47860.03
0.34100.04
0.223+0.06
0.342:0.05
0.21660.00 04
0.234:0.06
0.247e0.05
0.224+0.00 06
Bower
0.69220.07
0,597=0.5
0.41749.05
0.55540.05
0.54220.04
0.59120.05
0.51850.08
0.506.0005
0.576.0004
GTSRR
0.296e51.0
0.85+=002
0.762+01:0
0181=40.004
0.951=0.004
0:381=00014
0.5660.006
0.982x11004
D:97e0
042550005
0,523.0.05
0413s0.04
0.31240.05
0.51a0.05
0.521:0.05
0.56460.
0:09200222
091229002
9912001
0.91720002
0.917601122
Imuage
Jermammiat
04
0.352+0.07
0.337+6.09
0.346-0.88
0:239+0.07
necucscniamiist
0,83700
0.93240
0,908.00
0.9620
094240
0.9110
091240
0.9.00
0.9334
rctiamnist
0.377a0.1
0.506+1007
4.40110.1
0.40600
0455+1.1
042240.1
0.26c00
A0Sel
Incasimmitt
0,541:0
0.75140
0,898.0
0.68240
0.73240
0.65810
072240
0.661
0.59
bloddmiss
0,742a0005
0.80420004
0.78423104
0.86540.002
0.8850.03
0.87820005
0.050004
0.833
0.578
orpansmnist
0:771=0.05
0.872-0.03
0.86+1100 0a
O 081440.05
0.574+0.03
0.887
0.54-0.04
0.885=0.00 o8
0.577+0.05
organemnist
0,766x0.04
0.818:0.03
0.79710.04
0.610.05
0,5012000
0.836.0.05
FS810.05
0.815:0.0
0.70840.09
0,5840
0.69:0.05
0.633.0.04
0.59440.06
0,6091000 05
0.67740.05
0.24:0.07
0.669:000 06
0.540.1
086553004
0910200.22
0.97740012
0.916a5102
0.73620.1
nofolemmist3d
0.207-0
0,773el
0.688-0
adicnalmmist3
0.426511
0L52Ke1
veselmmies3d
0.547.0
0.61500
0.58e0
0,4+0
0-43440
0.6700
0.375e0
0.113+0
sysppeemisstd
0,598.0
0.50610
0.061220
0.18940
0.1910
0.18940
025320
0,239
0.13710
Chemical Sormulaa
hace
0.62=0
0.704et)
0.678=0
0483+0
0.6344
0,69440
0.583e1]
0.621
688+0
DBOP
0.893:00
0.715
0,603:0
0.55330
0.6310
0.632200
0.50110
0,626e0
0.6hh10
clintox
(L365at)
0,258e0
0.089950
0.0869611
(L086950)
01066kel
0.0899500
0.0869611
HTV
0,471:b0
0.46+10
0,159H0
0.047-00
0.0275+0
0.0407+0
0 02005+0
0,0273e0
0 0.407400
43

---

Published as a conference paper at ICLR 2026
T COMPARISON WITH NATIVE MULTICLASS APPROACHES
The training time complexity of CDA for a multiclass problem is on the order of C2 * O(NM N log N), where C' is the number of classes, N is the number of samples per class, and M is the feature dimension. However, Since C' is typically much smaller than NM, it is negligible, and the overall complexity effectively becomes C O(NM M + N log N), similar to the binary case.
We think that exploring an inherent multiclass extension of CDA is an important direction, especially given that other classifiers such as LDA and SVM have their inherent multiclass approach. In the cases where C' is large and not negligible, inherent multiclass approaches offer attractive lower computational complexity with respect the number of classes (C versus C(C 1) discriminants in pairwise schemes).
In this section, we evaluated inherent multiclass LDA, SVM, and PLS-DA. The results in Table 17 show that although these inherent multiclass methods indeed achieve faster training times, their predictive performance consistently lower than that obtained by their binary counterparts combined with external multiclass strategy. This makes sense since these methods natively designed for binary classification such as the margin in SVM, and their optimization objective becomes less effective when extended to inherent multiclass setting.
Since we have evidence that the inherent multiclass approaches for LDA and SVM performed worse than their one-versus-one scheme, decided to leave the investigation the multiclass CDA to future study.
Table 17: AUROC (Multiclass Prediction Performance)
Daturet
C1380
CDA
LDA
Itheret multiclass LDA
SYM
Innerent muuticlss SVM
PLSOA
Icheren: multiclass PLSDA
Standard images
MNIST
0.607:0:11
0.958-0.006
0.92600.009
0.566-0.055
0.95
0.566000066
0:91 3:0.1
0:97100.004
3.9510.008
0.93340.085
0.967400007
CIFARIO
0,999:002
0.61100.02
0.62720001
3.651a0.01
0.653.0002
0661150.02
0.50450001
0:647:0002
SVEIN
0.522a0.00
0.6858.01
0.531=0.007
0.541200077
0.55850001
0.5020.003
0.508a0.007
0.532=0,005
0A140.05
016660001
0.554-002
0.6/6-0
8555-4002
0 0150740000
06616-0022
GIESRR
0.580-001
0 0.37--0.01
0.2214002
0.98140006A
0.93440.000
0.5514001
01902-00.11
STL.00
0.00720112
0465598.2
0,2KelLU2
0.523
0.65325002
B.02500/22
0.5g0.002
0.6195/102
PMNIST
0.836803
0917:0.00
092:0.02
0,90200022
8924:0.82
2.919:0.02
0.904:0.02
0.900.02
Medical Images
demarmist
0614-003
pesussiimmis
Q912251
retizamnis
0,57
0,5all 0009
0,6610
0.70300
0.70920
0.46700
0.53300
bloddmiic:
1J89540.01
0.716641
0 0.88-0.00
0.855000
O
1.89740622
0.540
0.67440
01303-00 04
orpatemaiit
0,50921013
0.86988.0
053328003
0,5420.03
0.347=0.03
0.5a0)
0.527
0.812211.03
organsminnst
0.094891113
0.76129.03
0 372556662
0.246
0.5af)
0,73
0.684611 03
irparnieed
0.667:003
0915:0.002
0501-1003
19925.0.02
0.540
0 508+0.008
0 0.0000.005
nodalemmistd
0.75550
0,78120
0732
0.2220
0.724e0
0.5g8
0.50KE0
0.48850
6.62241144
031840.01
0.54611044
0,3240000
U.5 0.5 UPM
0.560
0,47820113 13
0 00000000011
0.457e0
rescinnischd
0.521a0
0.555e0
sympomeieed
Chemical formala
bacs
3.62150
0.70520
0.684a0
0.606.0
0.637a0
0.69980
0.497a0
0.500a0
BBBP
0.71150
0.34220
0 0.693a0
0.69110
0,69720
0,67860
0.50Ka0
0.54720
cintes
0,644
0.57540
(1 543-00
0.51440
0.54940
0.67140
HIY
0.528
0.61640
0.37
033380
0,50840
0.51100
0.5140
0.49240
44

---

Published as a conference paper at ICLR 2026
U RELATION BETWEEN CDA AND NEURAL NETWORK
U.1 COMPARISON BETWEEN CDA WITH NEURAL NETWORK
Given that neural networks are prevalent, especially deep learning architectures such as ResNet, we made a comparison between linear CDA and them. Specifically, we compared CDA with ResNet-18 and included a on 6 3D medical image datasets of MedMNIST. The results are shown in Appendix Table. [8] The parameters of MLP and ResNet-18 are: 100 training epochs; SGDM optimizer; initial learning rate 0.01 with decay of 50% every 30 epochs. For MLP, used ReLU activation function; the number of hidden-layer neurons was set to approximately v #samples x #features. We can see that even compared with the more complex architecture ResNet, CDA still outperforms the adrenalmnist3d dataset. In addition, CDA demonstrates a better performance than ML.,, outperforming on 4 out of datasets.
MLP as the baseline for the simplest neural network. We tested
Table 18: Test set multiclass AUROC on MedMNIST3D datasets.
Method
CDA
MLP
ResNet-18
organmnist3d
0.913
0.936
0.977
nodulemnist3d
0.781
0.766
0,850
fracturemnist3d
0.518
0.564
0.611
adrenalmnist3d
0.756
0.635
0.737
vesselmnist3d
0.685
0.674
0.769
synapsemnist3d
0.544
0.507
0.656
U.2 COMBINING CDA WITH NEURAL NETWORK
We conducted two experiments using sophisticated, learned features. First, used CDA to initialize the final linear layer a ResNet-18 by taking an ImageNet-pretranne ResNet-18 and performing transfer learning SVHN. Instead of randomly initializing the final linear layer, initialized it using class-wise averaged CDA weights. This led to clear improvements over the baseline, yielding higher test accuracy and faster training, measured by area-across-epochs (AAE). Second, we evaluated Gaussian kernel CDA as a nonlinear model and compared it to single-hidden- layer MLP (512 input features + 45 hidden units + 10 output neurons). Because kernel CDA requires kernel matrix computations that scale quadratically with the number of samples, we used the same reduced dataset (24k/99k) employed in our first manuscript. Kernel CDA outperformed the MLP baseline, and CDA-initializes MLP achieved similar test performance to the randomly initialized MLP while training much faster, as shown by both AAE and the training curve comparison in Figure 15
These experiments demonstrate that CDA is well-performing and flexible: can act as strong initializer in neural networks-whethe for simple linear layer MLP MLP-improving both convergence speed and final performance, and can be extended to kernelized nonlinear variants that outperform standard nonlinear baselines.
When the data is complex with strong nonlinear relations, it also feasible to combine linear CDA with prevalent deep learning architectures by leveraging deep features. We validated this combined approach on SVHN by extracting features from the penultimate layer of an ImageNet- used the pretrained ResNet
ResNet-18 8 and training linear CDA on these features. For prediction, weights together with the trained CDA weights. As shown in Appendix Table 19 this combination sigmificantly outperforms applying linear CDA directly on the raw data, illustrating the strong potential of CDA for handling complex real-world datasets with substantial nonlinear structure.
45

---

Published as a conference paper at ICLR 2026
Table 19: Test set multiclass prediction performance on the SVHN dataset (ResNet-18 18 pretrained feature extractor + classifier).
Method
AUROC
AUPR
Fscore
ACscore
Acc
AAE
CDA initialized linear layer
0.79
0.797
0.795
0.748
0.664
0.637
Randomly initialized linear layer
0.78
0.795
0.789
0.73
0.653
0.621
Gaussian CDA
0.815
0.816
0.815
0.783
0.683
CDA initialized MLP
0.796
0.799
0.798
0.757
0.674
0.824
Randomly initialized ML.P
0.795
0.798
0.798
0.755
0.666
0.797
Hyperparameters: 150 epochs; Batch size 128; Initial LR = 0.005 with 50% decay every 40 epochs; L2
regularization 10 For nonlinear method tests (the last 3), we take subset=24000/9928 as total data due to
time limit perform kemel method full dataset
90
CDA initiaizzed MLP
Rendomly intielized MLP
80
70
Acc (%6
50
40
30
0
50
100
150
Epoch
Figure 15: Training curve comparison between CDA initialized MLP and randomly initialized MLP.
46

---

Published as a conference paper at ICLR 2026
V COMPUTATION CONDITIONS
The simulations were performed on a Lenovo P620 workstation (a AMD Ryzen Threadripper PRO 3995WX, 64 CPU cores maximum 4.308GHz, 256GB memory) with single-thread computing. The large-scale tests on single-cell data were performed on a server (two AMD EPYC 7H12 CPUs, each with 64 CPU cores at maximum 2.6GHz, 4096 GB memory) with single-thread computing. MATLAB 2023b was used as the platform to run the main algorithms. The "tic" and "toc" functions Matlab were used to measure the computation time.
W LICENSING
The code packages libSVM and liblinear are with BSD 3-Clause license. Matlab is with a paid proprietary license.
X CODE AVAILABILITY
The CDA algorithm has been implemented in both MATLAB and Python, and the code is publicly available at htttps://gtthtei cowp://ioonelaaallcceeeeraeeaeae
discriminant 1
Y CLAIM OF THE LLM USAGE
We used LLM-based tools to improve the language and flow; the principles, core logic, and innovations are entirely the authors'.
47


Reviews:
[{"review_id": "HQbtu3MBnK", "reviewer": "Reviewer_K7aC", "summary": "This paper introduces a Geometric Discriminant Analysis (GDA) framework, a unified geometric perspective for linear classifiers based on a Centroid Discriminant Basis (CDB0)—the vector connecting class centroids—adjusted by geometric corrections. Within this framework, the authors propose Centroid Discriminant Analysis (CDA), a linear classifier that iteratively rotates the discriminant in 2D planes using Bayesian optimization to maximize a performance score. CDA achieves quadratic time complexity, outperforming traditional methods like LDA and SVM (which have cubic complexity) in scalability. \n\nExtensive experiments on 27 real-world datasets (images, medical, chemical) show CDA's superior performance, stability, and efficiency. A kernel-based nonlinear extension (nCDA) is also introduced and shown to improve upon linear CDA and kernel SVM on challenging datasets.", "questions": "- how does the training time of CDA scale for a multiclass problem ?\n\n- The paper uses flattened raw pixels and simple tokenization. Do you believe CDA's performance is currently limited by this pre-processing? Have you tested it on more sophisticated, learned features (e.g., from an autoencoder) to see if it can serve as a powerful final-layer classifier?\n\n- In high-dimensional spaces, the 2D planes for rotation are constructed heuristically. How stable is the Bayesian Optimization process  especially in the initial rotations where the number of samples is low?\n\n- Can you show that the method is not loosing too much performance compared to LDA if the data is (exactly) Gaussian?\n\n- what about the performance on elliptic distributions? Maybe it could be interesting to show how this compare with \"robust\" methods build for elliptic distributions.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- computational efficiency of CDA, which achieves quadratic time complexity, \nThis potentially makes the CDA significantly fastethan traditional cubic-complexity methods like LDA and SVM.\n This is supported by  empirical evidence across 27 diverse real-world datasets, where CDA consistently outperforms established linear classifiers in terms of predictive performance, stability, and training speed, including on a large-scale single-cell dataset\n\n- the method offers  interpretability due to its geometric foundation and performance-dependent learning process, \n\n- the method can be extended to a kernelized nonlinear version (nCDA) demonstrates flexibility and improved capability on complex data, broadening its applicability.", "weaknesses": "-  The GDA framework is presented as a conceptual model and unifying perspective but does not seem to have rigorous theoretical guarantees?\n\n- The training process relies on several heuristic components, including Bayesian optimization for rotations and a sample reweighting strategy, which are difficult to follow and are maybe not robust?\n\n-  while the core CDA algorithm is efficient, the kernelized version inherits the standard computational bottleneck of kernel matrix construction, and the overall approach remains fundamentally designed for binary classification, relying on external methods  for multiclass problems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VJhbHfbceT", "reviewer": "Reviewer_QeqZ", "summary": "The work proposes a theoretical framework named geometric discriminant analysis (GDA), which includes the family of linear classifiers that can be expressed as function of a centroid discriminant basis (CDB0). A linear classifier named centroid discriminant analysis (CDA) is defined as a special case of GDA under a 2D plane geometric constraint.", "questions": "Please see the Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tA geometric theoretical framework for classifiers is proposed.\n2.\tA scalable linear geometric classifier is designed.", "weaknesses": "1.\tThe authors claimed that the proposed GDA is a framework includes the family of linear classifiers, but in fact only on CDA is designed. The claimed way for this is not suitable.\n2.\tThere are many classifier works, but the authors only compared LDA and SVM, which cannot reflect the sota of the propose method.\n3.\tIn the experiments, only LDA, SVM and LR are used as compared methods, which has no persuasiveness. There are many L\n4.\tThe proposed method uses kernel, so why not add kernel LDA for comparison?\n5.\tAs we all known, linear classifier is hard to discuss nonlinear data. In fact, the distribution of the data is always nonlinear, so how about the generation of the proposed method for nonlinear data?\n6.\tMinor errors: Where in line 147; Eq. equation 6 in line 177 and similar errors in other places.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aSjX8JyBia", "reviewer": "Reviewer_7WhT", "summary": "This paper introduces a generalized geometric theoretical framework called Geometric Discriminant Analysis (GDA) for linear classification. GDA unifies certain linear classifiers by defining them as geometric adjustments to a foundational Centroid Discriminant Basis (CDB0), the line connecting the centroids of two classes. Building on this framework, the authors propose a novel, efficient, and scalable linear classifier named Centroid Discriminant Analysis (CDA). CDA starts with CDB0 and iteratively refines the discriminant by performing performance-guided rotations on 2D planes, optimized through Bayesian optimization, achieving a lower quadratic time complexity compared to the cubic complexity of methods like LDA and SVM. The key contributions are the GDA framework itself, the high-performance CDA classifier, and a nonlinear extension of CDA via the kernel method.", "questions": "* The GDA framework is presented as a unifying concept. However, a key claim is that the framework can \"inspire the design of new classifiers.\" Please elaborate on how to use the GDA framework to design a new classifier systematically. A concrete, hypothetical example would greatly strengthen the claim of GDA's generative utility.\n\n* The use of Bayesian Optimization (BO) for the 1D search of the optimal rotation angle on each 2D plane is a notable design choice. Could you provide a more detailed rationale for this? Given that the performance-score evaluation for a single vector is computationally inexpensive, simpler methods like a fine-grained grid search might seem sufficient and potentially faster in terms of overhead.\n\n* The preliminary results for the nonlinear kernel CDA are promising but are limited to two datasets. Could you please provide more experimental evidence about this claim?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper demonstrates strong originality on two main fronts. Firstly, the introduction of the GDA framework presents a novel conceptual lens for viewing centroid-based linear classifiers. Instead of treating methods like Linear Discriminant Analysis (LDA) and Minimum Distance Classifier (MDC) as disparate algorithms, the paper reframes them as specific instances within a unified system. This is achieved by defining a foundational \"Centroid Discriminant Basis\" (CDB0) and characterizing different classifiers by the unique geometric corrections they apply to this basis. This unification is a creative and insightful contribution that provides a new language for understanding and designing these models.\n\nThe mathematical derivation of the GDA framework is well-executed, particularly in how it deconstructs LDA into the CDB0 and a covariance-based correction term, with clear explanations for various special cases (e.g., equal covariance, no correlation). This provides a solid theoretical foundation for the proposed concepts.\n\nThe empirical evaluation is convincing. The authors test CDA against a comprehensive suite of strong baselines (including multiple fast SVM variants, LDA, and LR) across 27 diverse, real-world datasets spanning standard images, medical imaging, and chemical properties. The breadth of this evaluation strongly supports the claim that CDA is a robust and generally applicable classifier. Furthermore, the inclusion of a large-scale experiment on a 1.3-million-cell dataset directly addresses the critical aspect of scalability, providing powerful evidence that CDA's theoretical efficiency advantage (quadratic vs. cubic complexity) translates into significant real-world performance gains. The rigor of the experiments, utilizing multiple metrics and cross-validation, further enhances the quality and credibility of the results.", "weaknesses": "* The framework is primarily used in a descriptive capacity to show that LDA is a special case. The paper proposes that GDA \"may inspire the design of new classifiers,\" but it stops short of demonstrating how. CDA itself is presented as a single, highly effective derivative, but the framework's utility in systematically generating other novel classifiers is not shown.\n\n* Rationale for Bayesian Optimization (BO). The algorithm uses BO to perform a 1D search for the optimal rotation angle on each 2D plane. While BO is a powerful technique, its use for a simple 1D optimization problem is not justified. Given that the performance-score function is relatively inexpensive to evaluate for a single vector, simpler methods might have been sufficient and faster.\n\n* The section on nonlinear kernel CDA feels underdeveloped compared to the thoroughness of the linear experiments. While the results on two datasets are promising, they are not sufficient to make a strong claim about its general effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YDzFyefDtt", "reviewer": "Reviewer_PuwL", "summary": "The authors propose a novel theoretical framework named Geometric Discriminant Analysis (GDA), which unifies a family of linear classifiers expressible as functions of a Centroid Discriminant Basis (CDB0)—the line connecting two class centroids—adjusted by geometric corrections under varying constraints. The authors formally demonstrate that Linear Discriminant Analysis (LDA) is a special case of GDA.", "questions": "In equation 10, how to determine various correction terms since  LDA is based on the first-order and second-order information of data.\n\nWhat are the theoretical explanations for obtaining the bias?\n\nIf each class of data has multiple cluster centers, using a single center may not be appropriate.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "GDA provides a unified geometric foundation for linear classifiers, explicitly connecting centroid-based discriminants (CDB0) to broader geometric constraints.\n\n GDA achieves higher accuracy, stability, and efficiency than LDA, SVM, and logistic regression (LR) on 27 real-world datasets.\n\nKernelized CDa extends linear CDA to nonlinear classification via kernel methods.", "weaknesses": "CDA initializes training from CDB0 (the line between centroids), which may be suboptimal if centroids are poorly estimated.\n\nIt is crucial to explore the model's extension to multi-class scenarios since traditional LDA is based on multi-class problems.\n\nWhile kernelized CDA extends to nonlinear problems, its performance depends heavily on kernel choice and bandwidth.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)