Robust Clustering using Gaussian Mixtures in the Presence of Cellwise Outliers
Authors: Pushpendra Rajpurohit
Petre Stoica
Prabhu babu
Venue: Accepted by TMLR ()

Abstract: In this paper we propose a novel algorithm for robust estimation of Gaussian Mixture Model (GMM) parameters and clustering that explicitly accounts for cell outliers. To achieve this, the proposed algorithm minimizes a penalized negative log-likelihood function where the penalty term is derived via the false discovery rate principle. The penalized negative log-likelihood function is cyclically minimized over outlier positions and the GMM parameters. Furthermore, the minimization over the GMM parameters is done using the majorization minimization framework: specifically we minimize a tight upper bound on the negative log-likelihood function which decouples into simpler optimization subproblems that can be solved efficiently.
We present several numerical simulation studies comprising experiments aimed at evaluating the performance of the proposed method on synthetic as well as real world data and at systematically comparing it with state-of-the-art robust techniques in different scenarios. The simulation studies demonstrate that our approach effectively addresses the challenges inherent in parameter estimation of GMM and clustering in contaminated data environments.

Published in Transactions on Machine Learning Research (01/2026)
Robust Clustering using Gaussian Mixtures in the Presence of Cellwise Outliers
Pushpendra Rajpurohit
cr2288143@itt..co.
Centre for Applied Research in Electronics Indian Institute of Technology, Delkú 110016, India
Petre Stoica
psoitt uu.se
Division of Systems and Control, Department of Information Technology Uppsala University, Uppsala, Sweden 75237
Prabhu Babu
Prabbu. Babu@caree ac.
Centre for Applied Research in Electronics Indian Institute of Technology, Delhi 110016, India
Reviewed on OpenReview: https: opw.revieeereeeeeee.emeee net/ forum? id- VVHPEgjdWk
Abstract
In this paper propose novel algorithm for robust estimation of Gaussian Mixture Model (GMM) parameters and clustering that explicitly accounts for cell outliers. To achieve this, the proposed algorithm minimizes a penalized negative log-likelihood function where the penalty term is derived via the false discovery rate principle. The penalized negative log- likelihood function is eyclically minimized over outlier positions and the GMM parameters. Furthermore, the minimization over the GMM parameters is done using the majorization minimization framework: specifically we minimize tight upper bound the negative log-likelihood function which decouples into simpler optimization subproblems that can be solved efficiently. We present several numerical simulation studies comprising experiments aimed at evaluating the performance of the proposed method on synthetic as well as real world data and at systematically comparing with state-of-the-art robust techniques in dif- ferent scenarios. The simulation studies demonstrate that our approach effectively addresses the challenges inherent in parameter estimation of GMM and clustering in contaminated data environments.
1 Introduction and literature
In machine learning, pattern classification, and other statistical fields, fundamental challenge lies in es- timating the parameters of the underlying distribution from observed data samples corrupted by outliers (Thomaz et al., 2004 Minh & Murino 2017: Bois et al.) 2024). For example, health monitoring systems may suffer from faulty measurements (Sangra & Codina 2015). while socio-cconomic surveys might in- clude deliberately falsified responses (Menold & Kemper 2014). To address these issues, robust statistical methodologics have been developed accurately estimate the parameters of the outlier corrupted data by first detecting outliers, then either down-weighting or completely eliminating them and finally estimating the parameters from the remaining uncorrupted data. The outliers may be casewise outliers (Huber 1964 Stoica et al. 2024), or cellwise outliers (Alqallaf et al. 2009), the latter being more challenging to deal with.
The inherent complexity of real-world data often cannot be adequately captured by a single distribution. To take this complexity into account, a mixture model that is a linear combination of several basic distributions can be employed. When these component distributions are Gaussian, the resulting model is termed a Gaussian mixture model (GMM). The superposition of multiple component distributions facilitates the representation of intricate density functions, and GMMs widely utilized to reveal possible underlying

---

Published in Transactions on Machine Learning Research (01/2026)
clusters of the data samples (Bishop & Nasrabadi 2006 Reynolds et al. 2009 Yang et al. 2012
Sharma
et al. 2024 Cleett-Albertos & Dutta 2023).
In many applications of GMMs the data are corrupted by outliers that do not belong to any of the mixture distributions. To handle outlier corrupted data, one approach is to partially relax the normality assumption by considering heavy-tailed distributions for the components. This framework includes models such as the mixture of Student's 's distributions (Peel & McLachlan, (0000) and the mixtures of contaminated normal distributions (Punzo & McNicholas, 2016). To address arbitrarily distributed outliers within the model-based clustering, Garcia-Escttt Escudero et al. (2008) introduced robust trimmed clustering (TCLUST), an approach that extends the minimum covariance determinant. (MCD) method for estimating the distribution parameters using classification trimmed likelihoods. In particular, TCLUST incorporates a concentration step within its expectation maximization (EM) algorithm, analogous to the one used in the FAST-MCD procedure [Rousseouw van Driessen, 1999). where a fixed proportion of the observations that deviate most from the mcan are considered to be outliers and are omitted from the parameter estimation step. A similar trimming method has also been implemented within mixture model framework in Neykov et al. (2007) and García- Escudero et al. (2014). blueRecent probabilistic formulations have also incorporated adaptive weighting to enhance robustness. Wang et al.] (2017) introducod a Bayesian data reweighting framework that learns per-sample weights jointly with model parameters, thereby reducing the influence of outlier observations and improving resistance to model misspecification. Similarly, Ghahramani & Jordan (1993) proposed an EM-based approach for supervised learning with incomplete data, where missing or uncertain entries are treated as latent variables, providing a probabilistic foundation relevant to handling cell-wise contamination in mixture models.
The goal of the present study is to develop an outlier-robust GMM parameter estimation approach that can detect the cell outliers. To achieve this, we employ the multiple hypothesis testing procedure of false discovery rate (FDR) to detect the outliers and employ the majorization minimizaaiinnn majorization-miiiiizztoon (MM) approach to estimate the GMM paramcters from the uncorrupted data. Unlike the classical EM algorithm, which necessitales the introduction of latent. variables and the computation of conditional expectalions, our MM- based approach for estimating the GMM parameters circumvents these steps and it is straightforward to understand and implement. The proposed method alternatingly estimates the cell outlier positions and the GMM parameters by minimizing a penalized negative log likelihood objective until convergence criterion is met, where the penalty term is derived from the FDR principle. The FM and MM yield identical parameter updates (means, covariances, and mixture proportions) for the non-outlier case (Sahu & Babu 2020). In the presence of outlicrs, we prefer MM, as given the latest estimate of outlier positions, EM is typically run until convergence but on the other hand, a single iteration of MM sufficient to decrease the penalized negative log likelihood objective.
The remaining sections of the paper are organized follows. In Section II, we introduce the data model and formulate the minimization problem of penalized negative log- likeliidl objective function with respect to the cell outlier positions and the GMM parameters. In Section III, we present a brief overview of the MM framework and the FDR-based multiple hypothesis testing. Section IV presents the derivations of cell outlier detection in the GMM and the GMM parameter estimation via the MM approach. This section offers an in-depth, by-sep exposition of the mathematical formulations, emphasizing the iterative optimization process that underpins our method. In Section V, we conduct an extensive simulation study on synthotic as well as real world data to evaluate the performance of the proposed framework under variely of simulation conditions, and we benchmark its performance against state-of-theeart robust parameter estimation and clustering techniques. Finally, Section VI concludes the paper.
2 Problem Formulation
In this section, we discuss the GMM as well as the GMM in the presence of cell outliers, and formulate the minimization problem with respect to the cell outlier positions and the GMM parameters.
2

---

Published in Transactions on Machine Learning Research (01/2026)
Let yt) be a p-dimensional vector that is randomly drawn from mixture of K Gaussian distributions, The corresponding mixture distribution is given by:
K TR TR (y(t;; Ex),
(
(1)
where the parameter vector 0 = ( Me, E k= comprises the mixing proportions, means, and covariance matrices of the components and (y(t); Mas Ex) denotes the distribution of the 4th component, viz:
exp (-2 (v(t)) ((x))
mu) E (y(t) pua)
N (y()): Ex)= (2T Σ
(2)
In many practical scenarios, the vector y() may be partially corrupted by cell outliers. To address this situation, we introduce a binary vector b(t) associated with y(t): the element (t) of the binary vector b(t) is set to the jth element of y(t) uncorrupted and to 0 if the jth clement of y(t) is contaminated. Based on the binary vector b(t), we construct a binary selection matrix B, E RP'** where pt is the number of uncorrupted elements y().. The matrix B, defined so that each column contains exactly one entry equal 1 corresponding to the uncorrupted elements of y()), with all other entries set O, For example, if
yt) = (t), (t), us (t)
and the corresponding indicators are (t) 1, 62(t) 1, and (t 0, then the matrix B, is given by:
1 0
B,
1
0
0
This structure B, ensures that BJ (t) extracts the uncorrupted elements of y(t), and thus B Σg B, is the covariance matrix corresponding these clements. Since (B,} can be directly obtained from {b(t)} and vice versa, the two notations are equivalent. We employ both representations because B, facilitates the expression of the uncorrupted data and its associated covariance (i..., B, (t) and B, E,),, while b(t) is more convenient notation when estimating the positions of the cell outliers.
In the presence of cell outliers, if an observed sample y(t) is drawn from the GMM then the distribution corresponding to the uncorrupted elements of the data is given by:
K
p b(b())
Σ
T (y(t); ph: Eg. b(t)),
(3)
where:
1
B, (B, E. -1 "B. (y(() (B(( ma)).
N (y():Ma De, b())
exp - (y(1) -1x)
/ (2T)a)
(4)
and p: = by(t)
Assune now that have cell-corrupted dataset D= [y(1), .(N)} consisting of N independent and identically distributed samples drawn from the GMM in (3). The objective is to estimate both the model parameters and the binary variables (b(t)). To facilitate this estimation, we define the following function that will be frequently referenced in subsequent derivations:
Pek (0,, b(t)) A log (TT N (y()) ER, b(t)))
(5)
where 0g {TTK: p4k, }.}. Note that 5 can be written as:
Iu (0x. b(t) = log n. (y(t) - p4
B B B, B. (y(t) 44) log B, B,
Pt
log(2
(6)
2
3

---

Published in Transactions on Machine Learning Research (01/2026)
Also note that:
TE N (y((); Σx. b()) = (0.e(Ox.b())).
(7)
For the dataset D, the likelihood function corresponding to the uncorrupted data can be expressed as:
N
2
II p(y());,b(())
(8)
Therefore the negative log-likellnn likelihood given by:
N
K
(0; _ log L0; (b(()) = Σ log
Σ
can ((x.)
(9)
For later use, also define the negative log-likelihodd of the tt observation y(t) follows:
4 (0.)(t),b(t)) & -log Σ (nn(O,
(10)
Minimizing 9 will flag too many cells as outliers. To prevent this from happening we add a penalty term to which penalizes the number of outliers. The problem of estimating the GMM parameters 0 and the binary variables {b(t)} will thus be formulated as:
minimize (N.). (b(e)) subject to
(0; D, (b(()) + E EE T
(11)
-11-1
π m O.
ΣΦ > 0
4k,
where N; N - -' (t) denotes the number of corrupted cells corresponding to the 7th variable across all observations and Ne is penalty factor whose choice will be based on the FDR as discussed later.
3
Preliminaries
In this section, we provide concise descriptions of the MM procedure (Sun et al. 2016 and the FDR principle (Stoica & Babu 2022). which will be useful in the algorithm development.
3.1
Majorization Minimization
Consider the following minimization problem:
minimize f(0),
(12)
AEB
where 0 denotes the optimization variable and θ represents the feasibility set.
Let 0= E ② denote the current estimate of 0 at the 2th iterative step. A surrogate function g (0 0*) is said to majorize the original objective function f(0) at #= if (Sun et al. 2016 Hunter & Lange 2004):
f(0) < ( 0²) VO
(13)
and
f(0²) = g (0 10))
(14)
Instead of directly minimizing (()), the MM procedure minimizes the surrogate function gr 0*). The minimizer of this surrogate function then taken as the updated estimate 0=+1 i.e.,
0++1 arg minimize gr 0*).
(15)
920
4

---

Published in Transactions on Machine Learning Research (01/2026)
The update in 15 guarantecs non-increasing sequence of objective function values:
f(0 g5(0"*)
g (0
f(0").
(16)
Thus, starting from an initial point 00 θ the MM procedure generates a sequence (0*} that monotonically decreases the objective function. For further details on the construction of surrogate functions, readers are referred to Sun (016)).
3.2
False Discovery Rate for Multiple Hypothesis Testing
For outlier detection, we utilize the multiple hypothesis testing procedure of FDR (Bnjamiiii & Hochberg,
1995 Stoica & Babu 2022). Let [Hr denote the set of N mull hypotheses and (T(t) the corre-
sponding test statistics. The multiple hypothesis testing, which is usually preferred to individual testing for
large N, (Stoica al. 2024) is often performed by controlling the FDR (Benjamini & Hochborg 1995). Let
" denote the individual false alarm probability (also called the significance level):
7 prob(reject H, H, true).
(17)
The FDR is defined as the expectation of the ratio of the number of incorrectly rejected hypothesis (false discoveries) and that of all rejected hepothesis (discoveries):
IR R
FDR = E
(FDR =0 for R = 0.)
(18)
To control the FDR at level a, we proceed in the following way. Let {T(t)} be ordered such that T(1) 2
((2) ... 2 (NN) and let the significance levels be given as:
== I,,,,,,, Nt Nn'
(19)
where 7) is the harmonic number:
= 1
+
N ~ log N + 0.577.
(20)
Also let ne be the following quantile of the distribution of (T(t)):
prob(T(t) 2 mh. H₂) 7o-
(21)
Finally let:
i max [t [(t)= nd.
(22)
FDR rejects the 7 hypothhses (H,,... H2) and accepts the remaining hypotheses (H 1 Hw). the test statistics are known to be independent positively correlated, then the following larger significance levels can be used:
7t t- N.
(23)
4 Proposed Algorithm: RobGMM
In this section, we present an iterative procedure for solving the optimization problem in 11
11
For the reader's convenience, we restate

---

Published in Transactions on Machine Learning Research (01/2026)
minimize (N.). (b(1))
1(0:2, + + Σ //s
(24)
subject to 711=1, π - 0, Σ - Vk.
where are obtained by means of the FDR procedure, see objective function alternatingly with respect to the outlier positions and the GMM paramcters. First, we exploit the FDR-based penalty in (24) to arrive at an estimate of outlier positions given estimates of the GMM parameters. Then, for given cell outlier positions, we minimize (24) over the GMM parameters using the MM procedure. The MM-based derivation of the parameter updates avoids the introduction of latent variables and the computation of conditional expectations, is typically required in alternative methods such the EM algorithm.
21 and 29 below. We minimize the above
4.1 Outlier Detection
In this subsection we describe the procedure for estimating the set (b,(1)) Our approach sequentially updates cach binary variable, beginning with {br (t)) while keeping the remaining variables (8)())
and the GMM parameters 0 fixed, followed by an analogous update for {b2(t) and so on. To facilitate this, we first consider the negative log-likelihood of the obscrvation y(t) given in (10) as a function of (b)(t)), using the latest estimates of (ū,())szi and 0:
fa (b) = l (0,0),,
(25)
Since bt( E {0,1), 1], the function
25 can be expressed in an affine form as
I (l) A (1) + (1 -1- b.()) S?(1),
(26)
where (t) and f() denote the values of fi (t)) at (t) and bt(t) = 0, respectively. Employing the representation in (26), the negative log- lieliiel associated with the uncorrupted data can be written up to additive constant as
Σ (t)),t))))
(27)
where:
T,(t)= f(t)) t= 1, N.
Each Ti(t) can be viewed as test statistic associated with the null hypothesis:
(t): The 7th cell of tth sample belong to one of the mixture components and therefore is not outlier.
Thus multiple hypotheses testing using FDR can be used to test these hypotheses jointly. Since the data samples are assumed to be independent (over t), the test statistics associated with these hypotheses are also independent and therefore the significance levels are given by (23 Let the thresholds ve be defined as in
21::
prob (Ti(1) 2 The H.)()) 7
(28)
By the log-likelihood ratio theorem (Stoica & Babu 2024b) the variable T:(() follows a chi-square distribu- tions with one degree of freedom under the null hypothesis that bx(t) 1. Using this observation in 28 leads to explicit choice of the penalty factor 7) in [11
at
prob (I,(t) 2 ne [T((t) x (1) N
1= N.
(29)
where a denotes predetermined false discovery rate, which is typically set at 0.05. Next observe that mini- mixing the likelihood in (11) with respect to (b)( and N, is equivalent to solving the following minimization problem:
N
N
min No. (6.0)
((()) +
(30)

---

Published in Transactions on Machine Learning Research (01/2026)
By sorting the test statistics (T) in descending order, i..., T2(1) 2 2 (NN and minimizing 30h with respect to (b:(1))E1: the minimum of 130 with respect to N; is obtained by evaluating:
N Σ
T(())
mh
(31)
t=
for all possible Nj's. Choosing the minimum of 31 by N1, and consequently the corresponding estimates {b:(t)]E> The updates of the remaining variables {by(t) are derived similarly.
yields the optimal number of corrupted cells, denoted
4.2 GMM parameter estimation
Using the latest estimates of the cell outlier positions (b(t)), the negative log- likellood in 9
becomes:
K ue Oz ((t)
(0.D,1b(())) -- Σ log
(32)
We will employ an MM algorithm minimize over 0. The objective in 32
is log-sum-exp func-
tion (Boyd & Vandenberghe 2004), therefore using Lemma in Appendix upperbound at 0 (the latest estimate of 0):
leads to the following tight
(, (b())) < -
Wek (Ox.B()) + constant
1=1 k=
(33)
St
θ 0
+ constant,
where
OcE (0r.D(e)
Wik
Σ K j= or (())
(34)
We minimize the surrogate function ss
(0 (010) obtain the updated estimate θ:
i = arg minimize
0
Ep) subject to π 11=117200 0
(35)
Using
6
si ( [|)0)
can be written (to with an additive constant) as
N K
(1000)) - px) B, B E. B. (y(1) - (x)) + log B Σ₁ B,
0 0
Wekk
log T3
1=1 k=1
(36)
Observe that 81
0 (010) separable in { and {Me, Ex), therefore the objective in (35 can be minimized
separately with respect to { and {pu, Ex).
4.2.1
Estimation of }
From 35 the minimization problem with respect to { is given by:

---

Published in Transactions on Machine Learning Research (01/2026)
maximize [ma]
Σ Ek=1 Wis log T
(37)
subject to
π = 1,
Letting À be the multiplier for the equality constraint
Σ Th
1, the Lagrangian for 37 is:
[((Ta),A)
Σ
Web log Th
A
Σ TK -1).
(38)
The update of Ty. is obtained by setting the derivative of the Lagrangian with respect to Th to zero:
ac((TR),A)
E Wike
+ X = O.
(39)
mTx
TT
The constraint Σ Th = 1 and 39
implies that:
Wikk
(40)
By substituting X from
40
into
39
we get the update of {T as:
T
Vk.
(41)
Wit
From 334 we get EN Σx=1 Wik N, therefore:
1=
Wikk
Vk.
(42)
N
4.2.2
Estimation of {u²}
Next consider the optimization problem with respect to {1r} for fixed (B.), (#x), and {Ex):
minimize (
Wik (y(t) m4x) B. B. E. B, (y(t) mu).
(43)
The solution to
43 which is quadratic least-squares problem, is given by:
N
Wek B, B, S.
B
Σ Whe B B, B. BTy(1)) Vk.
(44)
4.2.3 Estimation of {Σ+}
With given { and (B.} the optimization problem with respect to Eg is:
N
K Wik (1og B. Σg B, + qe (t) (B. E. B, gu(())
minimize (E+20)
(45)
1=1
where qu (t) = y(t) p.k. We tackle 45 using an one-step MM approach. An upper bound for the term
log B.
Σg B, in 45 for a given Σg Σg can be obtained using Lemma in Appendix A
8

---

Published in Transactions on Machine Learning Research (01/2026)
Wik log Br B₁ S Tr ER) + constant;
(46)
where
C -
Wik w
(B
(47)
Since both E& and Eg are positive semi-definite matrices, we can derive the following tight upperbound for the second term in 45 at Σか using Lemma 3 in Appendix A
N
Σ
q (B) BBB.))_B. qa(1) S Tr (D+EEE
(48)
where
D
Wek B.B BB (())) B, -B, Σg.
(49)
By combining 46 and
48
we arrive at the following surrogate minimization problem:
K
min
[I
(50)
E
Letting Ex
D ExD the above problem can be equivalently written as:
K
min (E
Σ
Tr
D.DDD..... + Tr (Ex')) ))
(51)
The matrix inequality
(D)...)))* -E - E. (DD...)))* -Ea o
(52)
implies that
Tr (DD. Ex) + II (Ey²) 21r
(DDC..)))+)
(53)
The minimum of left. hand side of 53 is obtained when (see 52
Ex D CDD
(54)
and thus corresponding minimizer of 50 is:
S D (DAC.DE)
(55)
The pseudocode of the proposed Robust. GMM (RobGMM) algorithm is summarized in Algorithm
1

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm 1 RobGMM
Input: {y(t)), 0.05
Output: (b)( {. (px), and {Ex}
Initial values of {TA), {ua): and {Ea} are obtained using suitable clustering technique (such K- means)
2: (b.(t) =
3: repeat
4: Obtain (t) using 30 and 33
5: Obtain [ using 42
Obtain {ps] using
7: Obtain {Σx} using 55
8: until Convergence
4.3
Convergence and Computational Complexity
The cyclic minimization of the objective in [1] with respect to {TR, E&}, and {Br guarantees the monotonic decrease of the objective with respect to the iterative steps. Consequently the convergence of the objective function evaluated at the iterates of the RobGMM can be proved using techniques from Razaviyayn
et al. 2013 : Sun et al. (2016). The following convergence criterion for the RobGMM algorithm can be used:
T( (B. Jourrent [(Orrevious (Be prrevious)
<,
I (Oprevions, (B, prrevoou)
where [(Ceurent (Bourent) }) and [(Orrvvious [Bprevooss) denote, respectively, the values of the penalized negative log-likelihood in evaluated the current and previous estimates of and B, In the simulation studies have used 10
In the following, we analyze the per-iteration computational complexity of the proposed algorithm in terms of the number of mixture components K, data points N, and data dimension p. The dominant cost arises from evaluating the set of binary selection vectors {b((), which initially incurs O(K ((K) operations. The computation of the mixing coefficients { requires O(KNp² + K²N) operations, the estimation of the means } requires O(Kp² KNp2 KN), and the estimation of the covariance matrices {Σx} adds a further OKKN'" + KN) cost. Hence, the overall per-iteration complexity of the RobGMM algorithm is O(KNN* K2N
The quartic dependence
primarily originates from the outlier detection stage, where the likelihood terms involve repeated inversions of covariance submatrices of the form (B, E, which costs O(p"). However, during outlier detection, the component covariances } remain fixed and B, changes only incrementally (typically by a single column addition or removal). This structure enables efficient updates of
[ B,) using rank-one update and downdate formulas derived from the Shernan-Morrison Morrison-Woodbury
identity and the Schur complement. (block matrix inversion lemma) (Sherman & Morrison, 1950: Woodbury
1950 Harville 1997
Zhang 2005 Stoica et al. 2005
Let the current inverse be
H ((Ex)s.s)*
(56)
where S is the index set corresponding to the uncorrupted dimensions. (S' = SU {i)), the inverse of the enlarged submatrix can be updated as
When new index i is added
H H 3
-HBa a-1
(EE)s
(57)
-
where (En)s and a (Ex) B8 HB. Conversely, when an index is removed S (i)), and the previous inverse is partitioned
F
?
H'
(58)
w
10

---

Published in Transactions on Machine Learning Research (01/2026)
the updated inverse corresponding to the reduced index set is obtained via the rank-one downdate
(<E)s,,)) -1 F ry
(59)
involve catrix-vector multiplications and outer products, each costing Opp)). where Pt S p. Hence, the per-update cost upper-bounded by O(r²), reducing the complexity of the outlier detection step from (KKNN*) to O(KNp"). Incorporating this improvement, the overall per-iteration computational complexity of the proposed RobGMM algorithm becomes O(KNp" + K²N).
Both updates in 57
59
5 Numerical Study
In this section the performance of RobGMM for clustering as well as robust GMM parameter estimation is compared with that of two state-of-the-art robust methods, namely TCLUST Garcia-Escudero et al,, 2008) and Student's mixture model (with degree of freedom = 5) (Peel & McLachlan 2000). We have also included the vanila GMM (Dempster et al.) 1977] in this comparison. Furthermore the RobGMM algorithm is also applied to the real-world Top Gear data set (Alfons 2021) for clustering and outlier detection. Our experiments are run using the MATLAB software on Intel-i7 processor with 64 GB RAM.
5.1 Clustering Performance
We evaluate the clustering performance of RobGMM algorithm on data generated using the simulation settings in Appendix B.1 We generate the outliers by randomly selecting 10% or 20% elements from the data matrix and modifying their values such that they are uniformly distributed in the interval [-20, 20). Fig. [ shows the uncorrupted data, the data corrupted with 10% cell outliers, and the clustering results obtained using competing methods and the proposed RobGMM algorithm. It be seen that RobGMM algorithm has excellent robust clustering performance.
The clustering performance of the RobGMM algorithm is compared with TCLUST and student's t mix- ture using two performance metrics namely Accuracy (Metz) 1978) and the Enhanced Multivariale Pearson Correlation Coefficient (EMPC) (Stoica Babu, 2024a). Evaluating these performance metrics requires computing the confusion matrix. Let G be the (K 1) (K + 1) confusion matrix in which the first K groups are for the uncorrupted data clusters and the (K + 1)th group assigned to the outliers. The element G.g of G shows the number of data points from the 5th cluster that were classified as belonging to the jth cluster. Using G the Accuracy and EMPC are obtained as follows:
Ek+1 K+l GkA
Accuracy
(60)
EKT Ef= Gig
1 K+1
(6x + Bk) Gk#
EMPC-
1,
(61)
K K+1
6R.8k
where 5, = = K+1 Gky and 3g -
ΣK+1 K+I Gas- Note that to match the obtained clustering order to the true clustering order, we consider all possible orders of the K clusters. For all these possible clustering orders, we construct the corresponding confusion matrices and compute the Accuracy for cach of them. The clustering order that gives the maximum accuracy is identified as a match to the true clustering order.
We consider two cases: in case 1, the performance metrics are computed by considering 4 clusters and one outlier group, in this case the competing algorithms do not detect outlicrs. The average Accuracy and EMPC performance metrics (computed using 500 Monte-Carlo runs) for RobGMM and the competing methods in case are shown in Table [ and Table 2 for 10% and 20% outliers... In case 2, we compute the performance metrics after adding an outlier detection step to the competing algorithms. We first implement the competing methods on the corrupted data estimate the distribution parameters (means, covariance matrices, and mixture proportions) and with these estimates we then run the FDR based multiple hypothesis testing to
11

---

Published in Transactions on Machine Learning Research (01/2026)
20
20
10
-20
-20
-20
-20
-10
-20
-10
20
-20
-10
10
(a) Uncorrupted data with sam-
(b) Data corrupted with 10% outliers
(c) Clustering obtained by GMM al- gorithm
ples and clusters
(square marker denotes an outlier)
20
10
-10
-10
-10
10
20
-10
10
20
10
(d) Clustering obtained by Stadent's mixture model
(e) Clustering obtained by TCLUST (l) Clustering obtained by the
algoritlun
posed RobGMM algorithm
Figure Uncorrupted and cutlier-corrupedd data, and clustering results obtained by competing methods and the proposed RobGMM algorithm.
12

---

Published in Transactions on Machine Learning Research (01/2026)
detect the outliers for the competing methods. The obtained results for case 2 are shown in Table and Table 4 From the results, see that the RobGMM achieves better performance metrics in both the cases. The degradation in performance metries for GMM, Student's mixture model, and TCLUST for case 2 when compared to case due to the fact that covariance matrices obtained via these methods are poorly estimated which leads to large misclassification of many inliers as outliers.
Table 1: Performance metrics for RobGMM and other methods for 10% outliers
GMM
Student's s mixture
TCLUST
RobGMM
(mean + std. dev.)
(mean + std. dev.)
(mean + std. dev.)
(mean = std. dev.)
Accuracy
0.708 + 0.100
0.821 = 0.106
0,900 + 0.048
0.981 1 0.008
EMPC
0.378 + 0.195
0.579 0.204
0.714 + 0.099
0.949 + 0.022
Table 2: Performance metrics for RobGMM and other methods for 20% outliers
GMM
Student's mixture (mean std. dev.)
TCLUST
RobGMM
(mean = std. dev.)
(mean std. dev.)
(mean = std. dev.)
Accuracy
0,580 + 0.094
0.682 = 0.079
0.718 # 0.100
0.962 # 0.008
EMPC
0.217 1 0.215
0.426 0.160
0.467 1 0.211
0.947 1 0.015
Table 3: Performance metrics for RobGMM and other methods for 10% outliers
GMM
Student's mixture
TCLUST
RobGMM
[mean std. dev.)
(mean dev.)
(mean std. dev.)
(mean std. dev.)
Accuracy
0.694 0.110
0.809 0.10
0.819 + 0.106
0.980 + 0.006
EMPC
0.134 + 0.220
0.344 = 0.211
0.349 + 0.215
0.950 + 0.017
Table 4: Performance metrics for RobGMM and other methods for 20% outliers
GMM
Student's mixture std. dev.)
TCLUST
RobGMM
[mean std. dev.) 0.542
(mean std. dev.)
(mean std. dev.)
Accuracy
1 0.092
1 0.012
EMPC
-0.063 0.218
0.226 0.200
0.192 + 0.198
0.948 0.022
5.2 Parameter Estimation Performance
The performance of the RobGMM algorithm for the GMM parameter estimation is compared with that of the competing methods in different scenarios using the following metries: normalized root mean squared error (NRMSE) and the Luilback-Loiberr (KL) divergence. The NRMSE of the estimated mean and covariance matrix for the 4th cluster are given by:
1|.4 P
NRMSE (ft))
(62)
|
Σ5 Ealle
NRMSE ΣΦ
(63)
E
where || || denotes the Frobenius norm. The KL divergence of the 4th component is defined as:
KL (HE Ex) (p.: Exx) 2 log Σg log
Σx - + Tr 1 + (ūx pa) T Ex (px - ma)
(64)
The cellwise outliers are generated by randomly selecting cell indices from the data matrix and perturbing their values by random shifts drawn from a uniform distribution. To ensure that these artificially introduced
13

---

Published in Transactions on Machine Learning Research (01/2026)
10
divergence component.
10%
NROMSE (n))
RebGMIM PITALETT
NIMSSE (X2.
RONGANN PCLUST Stadrni's GMM
Stadent's GMM
TELAST
10-2
10-4
GMM
10
10"
2
10
4
Outlier pereentage
Outlier peronstage
Outlier peremntgee
(a) NRMSE (Mi) outlier percent-
(b) NRMSE (Ei) vs outlier percent-
(c) KL divergence of component outlier percentage
age
age
10
KL. divergeacee for companees
10%
NRMSS (o)
NItSSB (Xx)
RIRGSE TCLUST
RUSGAAAI TCLOST
TCLUST
10
Studenc's
GMM
GMM
10-
2
Outlier percentage
Outlier perceutage
Outlier percentage
(d) NRMSE (f22) outlier percent-
(e)
NRMSE (Ez) vs outlier percent-
(f) KT. divergence of component vs outlier percentage
age
age
Figure 2: NRMSE and KL divergence for 2 clusters with p 2.
outliers do not overlap with the genuine cluster regions, each perturbed value is checked to verify that its distance from all cluster means exceeds three standard deviations of the corresponding clusters. If this condition is not satisfied, the perturbation is re-sampled until the shifted values lie outside the cluster regions, that is, beyond the 99th percentile of any cluster component. It is important to note that this restriction applies only to the outlier generation procedure; the outlier delection in RobGMM operates independently based on the estimated model parameters. The simulations are performed for K E (2 clusters with different choices of the number of dimensions (p (2. 25)). The simulation settings for these experiments are given in Appendix B.2 The obtained results for these simulations are shown Figure 2 Figure 3 Figure and Figure From the plots of NRMSE and KL. divergence versus outlier percentage, we observe that, compared to the competing methods, RobGMM exhibits superior parameter estimation performance across practical range of data dimensions.
5.3 Top Gear Dataset Clustering
The Top Gear dataset comprises various car models and their numerical feature specifications. We selected p p= 11 fully observed features corresponding to N 245 car models. We used a logarithmic transformation of the highly skewed variables price, displacement, brake horsepower (BHP), torque, and top speed (Alfons 2021). We then applied the RobGMM algorithm to the dataset. In Fig. 6 show the penalized negative log- ligliieldhood objective RobGMM iterations, and expected the objective converges monotonically.
The car models were grouped into four clusters using the RobGMM algorithm: performance and luxury sports cars such as the Aston Martin V12 Zagalo, Ferrari 458, Audi R8, Asion Martin Vanquish, Aston Martin Vontage, Audi A5 Sportback, Audi A7 Sportback, Audi R8 V10, Bentley Continental GTC, BMW Series, Corvette C6, Porsche 911, Porsche Borster, Jaguar XFR, and Lamborgh- ini Aventador. These cars are distinguished by their emphasis on power, speed, and exclusivity. Cluster consists largely of mainstream, compact, and family-oriented vehicles including models like Ford Fiesla,
Cluster 1 compriscs high-
14

---

Published in Transactions on Machine Learning Research (01/2026)
10
10'
divergenee compoosent
NBMSE (A))
10
NRMSEE (x4)
101
BolGMMM TCLAST
100
ROOGMN TOLUST Studean's & 6334
Student'ss
107
Strdient's
GMM
10
10:3
10%
10
10
10-1
2
1n
Outlier peromstage
Outlier peroeulagge
Outlier percentage
(a) NRMSE (Ma) vs outlier percent-
(b) NRMSE (Et) vs outlier percent-
(c) KL divergence component
age
age
outlier percentage
10P
101
KL. divergence component
10
10"
10
NROSSE (16)
TCLUST
NTtMSE (35)
TCLLSST
TELAST
10-4
107
10-1
10.1
10
10
10
2
10
4
Outlier percentage
Outller perorntage
Outlier percentage
(d) NRMSE (Ι²2) outlier percent-
(e) NRMSE (Ea) vs outlier percent-
(f) KL divergence of component outlier percentage
age
age
Figure 3: NRMSE and KL divergence for 2 clusters with 5.
109
divergmece compoment
NRASE (U))
10
RERGEE TCLUST
NIMSSE (X2)
RobeMM PCEASST
RabGaE CLLOST
Stadeon's
Suncent'' GKIM
GMM
Student's
10-3
GMM
10
10:4
2
10
4
Outlie percentage
Outlier peronulage
Outlier peromnagee
(a) NRMSE (m1)) outlier percent-
(b) NRMSE (Ex) vs outlier percent-
(c) KT. divergence component outlier percentage
age
10P
KL. divergrac companmutt
NRMSE (()
10-1
TCLUST
ROSESE
ATtMSI (32)
TOLUST
Students
RRCGMD TCLUST
Stadree's
10-3
GMM
10a
Student's
G343
10-1
10-4
10
10
10"2
2
10
4
Outlier percentage
Outlier percentage
Outlier percentage
(d) NRMSE (іτ²) outlier percent-
(e)
NRMSE (Ex) outlier percent-
(f) KT divergence of component outlier percentage
age
age
Figure 4: NRMSE and KT. divergence for 2 clusters with 10.
15

---

Published in Transactions on Machine Learning Research (01/2026)
10
102
10
10
10"1
1071
RobOMIK TELEST
NRAIS (jn)
TOLUST
REBGADE
NRMSB (po)
Sindmit' GMM
Student's
Stadene's
10
GMDE
NRMSE
30
GMM
30
10
10
Outlier peronstage
Outlier peronutage
Outller percentage
(a) NRMSE (fa) vs outlier percent-
(b) NRMSE (P2) vs outlier percent-
(c) NRMSE (Px) vs outlier percent-
age
age
age
10P
STRMS (3))
10
TELUST
XTMSS (22)
101
HobeMMM TOLUST
NRMSE (84)
101
ReSCME TELLST
Studeal's
Seadene's
GMM
10
10
10
Outlier percentage
Outlier parorntage
Outlier percentage
(d) NRMSE
Ei)
outlier percent-
(e) NRMSE (Σa) vs outlier percent-
(f) NRMSE (Es) vs outlier percent-
age
age
NT. divengncee for carponent
divergrace for component
10F
10/
30"
101
KL divergencce
109
RoOGMM
10a
BelGMM TELLSTT
TCLLIST
TCLUST
Staddent's
Studdent
Sturhent's S
GMM
C3DC
GMM
10
10
Outlier percentage
Outlier perceutage
Outlier percentage
(g) KL divergence of component vs outlier percentage
(h) KL divergence of component 2 vs outlier percentage
(i) KL divergence of component vs outlier percentage
Figure 5: NRMSE and KL divergence for 3 clusters with p 25.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Penalized negative log- likelihoooo
11000
10000
9000
8000
7000
6000
5000
4000
3000
60
80
100
Iterations
Figure 6: Penalized negative log-likelihood objective vs iterations of RobGMM on the Top Gear dataset
Honda Civic, Alfa Romeo Giulietta, Aston Martin Cygnet, Volkswagen Tiguan, Peugeot 107, Renault Clio, Renault Megane, SEAT Toledo, Toyola iQ, Volkswagen Passal, Cheerolel Spark, Hyundai i20, Suzuki Swift, and Volkswagen Golf. These cars are practical, widely available, and suited for everyday use. Cluster 3 in- cludes large SUVs and executive luxury vehicles such Rolls-Royce Phantom, Bentley Mulsanne, Audi Q7, BMW X6, Land Rover Range Rover, Rolls-Royce Ghost, Rolls-Royce Phantom Coupe, Toyola Land Cruiser V8, Volkswagen Towareg, Benlley Flying Spur, Jeep Wrangler, Nissan Palhfinder, Ssangyong Rodius, and Lexus RX, indicating a focus on comfort, size, and premium features. Cluster is composed hybrid, elec- tric, and niche vehicles including Audi TT Coupe, Audi TT Roadster, BMW i3, Chevrolet Volt, Fiat Doblo, Lerus CT 200h, Peugeol 207 CC, Peugeol 3008, Peugeol 308 CC, Subaru BRZ, Suzuki Grand Vilara, Toyola GT 86, Toyota Prius, Vaurhall Ampera, and Citroen DS5, with focus eco-friendliness and innovation.
In Fig. we show some examples of the cell outliers detected by RobGMM, which highlight feature values that deviate significantly from the typical characteristics of cars within their respective clusters (ten representative cars randomly selected from each cluster are shown in the figure). For instance, in Cluster 1, the Bentley Continental GTC is notably heavy for a performance sports car. In Cluster 2, the Peugeot 107 is exceptionally lightweight for a mainstream everyday vehicle, while the Aston Martin Cygnet and Toyota iQ exhibit unusually short body lengths despite being categorized regular city cars. In Cluster 3, the Bentley Mulsanne, Rolls-Royce Ghost, Rolls-Royee Phantom, and Rolls- Royce Phantom Coupe possess body lengths significantly greater than those of other luxury sedans and SUVs. In Cluster 4, the BMW i3 records an exceptionally high MPG value.
In a final experiment, we inject cellwise contamination by randomly selecting 1% of cells (independently for each dimension) and perturbing only those entries, leaving all other cells of the data sample unchanged. We then apply all the methods to the contaminated data. For GMM, Student's t mixture model, and TCLUST, detect the cell-outlicr positions using the FDR-based multiple hypothesis testing procedure applied to the distribution parameters (means, covariance matrices, and weights) estimated by these methods. The true and detected cell-outlier pasitions are shown in Figure g The results demonstrate that the proposed method detects the cell-outlier positions more accurately than the competing methods.
6 Conclusion
In this paper, we proposed a novel algorithm for robust estimation of GMM parameters and clustering that explicitly accounts for cell outliers. We minimized a penalized negative log- likelloodood function to estimate the GMM parameters where the penalty term was derived from the FDR principle. The penalized negative log-likelln ligelikoihooo function was cyclically minimized over the outlier positions and the GMM parameters. The minimization over the GMM parameters was done via MM framework in which minimized tight
17

---

Published in Transactions on Machine Learning Research (01/2026)
Aston Martin V12 Zagato
Alfa Romeo Giulletta
Aston Martin Vanquish
Aston Martin Cygnet
Aston Martin Vantage
Ford Fiesta
Audi A5 Sportback
Volkswagen Tiguan
Audi Sportback
Paugeot 107
Audi R8 V10
Renault Clio
Bentley Continental GTC
Renault Megane
Serles
SEAT Toledo
Corvette
Toyota
Porsche 911
Volkswagen Passat
MnDlsplacementi
In(Price)
(n(B)P)
In(Torque)
Acceleration
In(TopSpeed)
MPG
Weight
Length
Width
Height
In(Olsplacement)
In/Price)
In(BHP)
In(Toqqe))
Accaleration
In(TopSpeed)
MPG
Weight
Length
Width
Height
(a) Cluster
(b) Cluster 2
Audi Q7
Audi TT Coupe
Bentley Mulsanne
Audi TT Roadster
BMW
Land Rover Range Rover
Chevrolet Volt
Lexus RX
Fiat Doblo
Rolls-Royce Ghost
Lexus CT 200h
Rolls-Royce Phantom
Peugeot 207 CC
Rolls-Royce Phantom Coupe
Subaru BRZ
Toyota Cruiser
Toyota Prius
Volkswagen Touareg
Vauxhall Ampera
InDDipplceemet)
In(Price)
In(BHP)
Ini (n(ooque))
MPG
Weight
Langth Witth
Height
In(Dispaaeemntt
In(Price)
In(BHP)
In(Torque)
Acceleration
Spaeo))peed)
MPG
Weight
Length
Wiath
Height
Cluster
(d) Cluster
Figure 7: Cell outliers detected by RobGMM method in the Top Gear datasct (Violet pixel shows cell outlier).
18

---

Published in Transactions on Machine Learning Research (01/2026)
Traturro
10
50
100
2011
Samples
(a) Positions of synthetically generated outliers
Fosturea
30
100
150
200
Samples
(b) Synthetic outlier positions detected by RobGMM
Festaires
102
150
200
Samples
(c) Synthetic outlier positions detected by GMM
Postures
100
200
Samples
(d) Synthetic outlier positions detected by Student's t mixture model
Fostu10s
Samples
(e) Synthetic outlier positions detected by TCLUST
Figure 8: Cell outliers detected by competing methods in the Top Gear dataset (Violet pixel shows cell outlier).
19

---

Published in Transactions on Machine Learning Research (01/2026)
upper bound on the negative log-likelihood function. This minimization operation decoupled into simpler optimization subproblems that were solved efficiently. The present study can be extended to non-Gaussian mixtures, such as the Student's t mixture model, by deriving the corresponding penalized negative log- likelihood function, formulating the appropriate test statistics and their distributions under the Student's assumption, and subsequently estimating the distribution parameters using the MM framework. We pre- sented an extensive simulation study, comprising various experiments aimed at evaluating the performance of the proposed method on synthetic well real-world data and at comparing it with theaat robust GMM parameter estimation and clustering techniques.
Acknowledgment
This research was financially supported by the project Blending probabilistic and nonlinear representations (contract number: 2025-04318), funded by the Swedish Research Council.
References
Andreas Alfons. robusthd: An package for robust regression with high-dimensional data. Journal of Open Source Software, 6(67):3786, 2021.
Fatemah Alqallaf, Stefan Van Aclst, Victor J. Yohai, and Ruben H. Zamar. Propagation of outliers in multivariate data. The Annals of Statistics, 3711):111-33,, 2009.
Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57(1):289-300, 1995.
Christopher M Bishop and Nasser M Nasrabadi. Pallern recognition and machine learning, volune 4. Springer, 2006.
Alexandre Bois, Brian Tervil, and Laurent Oudre. A persistent homology-based algorithm for unsupervised anomaly detection in time series. Transactions on Machine Learning Research, 2024.
Stephen Boyd and Lieven Vandenberghe. Conver optimization. Cambridge university press, 2004.
Juan Cuesta-Albertos and Subhajit Dutta. On perfect clustering for gaussian processes. Transactions Machine Learning Research, 2023.
Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical sociely: series B (melhodologicall) 39(1):1-22 22, 1977.
Luis A. Carcía-Fscuderg, Alfonso Gordaliza, Carlos Matrán, and Agustín Mayo-Iscar. general trimming approach to robust cluster analysis. The Annals of Statistics, 36(3):1324 2008.
Luis A. Garcfa-Escudero, Alfonso Gordaliza, and Agustin Mayo-Iscar. constraincd robust proposal for mixture modeling avoiding spurious solutions. Advances in Data Analysis and Classificntion, 8(1):27 43, 2014.
Zoubin Ghahramani and Michael Jordan. Supervised learning from incomplete data via an EM approach.
Advances in neural information processing systems, 6, 1993.
David A, Harville. Matriz Algebra from a Statistician's Perspective. Springer, 1997.
Peter J. Huber. Robust estimation of location parameter. The Annals of Mathematical Statistics, 35(1):
73 101, 1964.
David R Hunter and Kenneth Lange. A tutorial on mm algorithms. The American Statisticion, 58(1):30-3) 37, 2004.
20

---

Published in Transactions on Machine Learning Research (01/2026)
Natalja Menold and Christoph Kemper. How do real and falsified data differ? psychology of survey response as source of lalsification indicators in face-to-face surveys. International Journal of Public Opinion Research, 26(1):41 65, 2014.
Charles Metz. Basic principles of ROC analysis. In Seminars in nuclear medicine, volume 8, pp. 283 298, 1978.
Hå Quang Minh and Vittorio Murino. Covariances in computer vision and machine learning. Morgan & Claypool Publishers, 2017.
Neyko M. Neykov, Peter Filzmoser, R. Dimova, and Plamen N. Neytchev. Robust fitting of mixtures using the trimmed likelihood estimator. Computational Statistics & Data Analysis, 52(1):299 308, 2007.
David Peel and Geoffrey J. McLachlan. Robust mixture modelling using the t distribution. Stafistics and Computing, 10(4))339 344, 2000.
Antonio Punzo and Paul D. McNicholas. Parsimonious mixtures of multivariate contaminated normal dis- tributions. Biometrical Journal, 58()):1506-15333 2016.
Meisam Razaviyayn, Mingyi Hong, and Zhi-Quan Luo. A unified convergence analysis of block successive minimization methods for nonsmooth optimization. SIAM Journal on Optimization, 23(2):1126 1153, 2013.
Douglas A Reynolds et al. Gaussian mixture models. Encyclopedia of biometrics, 741 7619-66663)) 2009.
Peter J. Roussecuw and Katrien van Dricsson. A fast algorithm for the minimum covariance determinant estimator. Technometrics, (3):212-2222 1999.
Nitesh Sahu and Prabhu Babu. New derivation for gaussian mixture model parameter estimation: Mm based approach. arxiv preprint arXiu:201..00023 2020.
Rosa Abellana Sangra and Andreu Farran Codina. The identification, impact and management of missing values and outlier data in nutritional epidemiology. Nutrición Hospilalaria, 31 (1)::1189 195, 2015.
Abhishek Sharma, Catherine Zeng, Sanjana Narayanan, Sonali Parbhoo, Roy H. Perlis, and Finale Doshi- Velez. Task-relevant feature selection with prediction focused mixture models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.
Jack Sherman and Winifred J. Morrison. Adjustment of an inverse matrix corresponding to a change in one element of given matrix. Annals of Mathemalical Statistics, 21(1):124 127, 1950.
Petre Stoica and Prabhu Babu. False discovery rate (FDR) and familywise error rate (FER) rules for model selection in signal processing applications. IEEE Open Journal of Signal Processing, 1:403-416, 2022.
Petre Stoica and Prabhu Babu. Pearson Matthews correlation coefficients for binary and multinary classi- fication. Signal Processing, 222:109511, 2024a.
Petre Stoica and Prabhu Babu. Penalized likelihood approach to covariance matrix estimation from data with cell outliers. IEEE Transactions on Signal Processing, 72:5616 5627, 2024b. doi: 10.1109/TSP.2024. 3507819.
Petre Stoica, Randolph L. Moses, et al. Spectral analysis of signals, volume 452. Pearson Prentice Hall Upper Saddle River, NJ, 2005.
Petre Stoica, Prabhu Babu, and Piyush Varshney. Robust estimation of the covariance matrix from data with outliers. JEEE Open Journal of Signal Processing, 5:1061-1072, 2024. doi: 00010///OSSP.22243477333
Ying Sun, Prablu Babu, and Daniel P Palomar. intoriizatioommmmmmizataain algorithms in signal processing, communications, and machine learning. IEFF Transactions on Signal Processing, 55(3):794-8T0 2016.
21

---

Published in Transactions on Machine Learning Research (01/2026)
Carlos E Thomaz, Duncan Fyfe Gillies, and Raul Queiroz Feitosa. A new covariance estimate for bayesian classifiers in biometric recognition. JEEE Transactions circuils and systems for video technology, 14 (2):214 223, 2004.
Yixin Wang, Alp Kucukelbir, and David M Blei. Robust probabilistie modeling with bayesian data reweight- ing. In International conference on machine learning, pp. 3646 3655. PMLR, 2017.
M. A. Woodbury. Inverting modified matrices. Technical Report Memo. Rep. 42, Statistical Research Group, Princcton University, 1950.
Miin-Shen Yang, Chien- Yo Lai, and Chih- Ying Lin. A robust clustering algorithm for gaussian mixture models. Pattern Recognition, 35(11):3950-3000 2012.
Fuzhen Zhang. The Schar Complement and Its Applications. Springer, 2005.
A Supporting Lemmas and Their Proofs
Lemma The log-sum-erp function can be lower-bounded any x as follows:
(x) 2 log
2 (x) + Vh (x) (x (x x),
(65)
with equality achieved at x = X. Here
X [1
Tpp
and hh (X) denotes the gradient of h (x) at x. The gradient of h (x) is given by:
Vh(x)=
(66)
c
Proof. The log-sum-cxp function h (x) is convex R Boyd & Vandenberghe 2004 therefore tight
lower bound for & (x) at any X can be constructed by using the following first order Taylor expansion:
(x) 2 h
Vh (x)
(x (x
(67)
Lemma 2. Let En and En be positive semi-definie matrices. Then the following inequality holds for any matrix B,:
T E. Tr Tr (B,
log B Eg B,
B. Bx + constant,
(68)
with equality for Σか = Σx.
Proof. From the arithmetic mean mean- geometric mean inequality (Snn et. al. 2016) we have that:
1/v
- B, EnB.
Br En B, By
Tt
B
Ey
(69)
or, equivalently,
1
log
B, Σg B,
- log
B, Σ B,
$ log
1 Tr
(B,
(B.'E,B,)
-1 T 1 ]]
B, E₁ By
llg(x))
(70)
p
p
22

---

Published in Transactions on Machine Learning Research (01/2026)
Since the function log(xr) is concave, it can be upper bounded by its first-order Taylor expansion at any point Zig
1
log(2) & log( + (x zo) In particular, when zp this inequality simplifies Lo:
(71)
30
log(r) S x -
(72)
Combining 70 and
72 yields
S IT (B. T Ey
-T m B. ExBi)
log
B B
constant,
(73)
where constant log
B. B. p.
Lemma 3. Let Eg and En be posilive semi-definile matrices. Then the following inequality holds for any matria By:
T B, EnBi)
Bi EB
B.
T E E B E,
(74)
with equality for Σg =
Proof. Consider the matrix:
B: Σx EB B B.
B
B. B.
B, Eg Ei 1/2 1/2
Σ
1-1/22
B E.
B 1//
Because this matrix is positive semidefinite, its Schur complements also positive semidefinite. In partic- ular:
B E = B. B.
B.
Be
B, Er
(75)
which is equivalent to:
B. E.
Bi ERB,) B. E
B, T
E
(76)
0
B Simulation Settings
The detailed simulation settings used in the numerical experiments are provided below.
B.1
For Clustering Performance
The simulation settings for evaluating the clustering performance are as follows:
Samples (N) 400
Dimensions (p) 2 (chosen small for the sake of displaying the clustering results)
Clusters (K)
Proportions (T) = [0.25. 0.25, 0.25, 0.25]
7
[10]
[6]]
Means: f41 =
142 =
-7
AA3 =
44 =
-5]
19.6 5.9
3.6
-3.0
5.8
-4.1
1.6
2.2 4.9
Covariances: Σ
Σg
Σ,
5.9
6.0
-3.0
5.9
-4.1
6.0
2.2
23

---

Published in Transactions on Machine Learning Research (01/2026)
B.2 For Parameter Estimation Performance
The experiments are performed with different choices of number of dimensions (o E {2, 5, 10, 25)). The synthetic data is generated for K E {2,3} clusters. Each cluster is modeled as multivariate Gaussian distribution with distinct mean and covariance structurcs, constructed as follows.
For cach feature dimension d E {I...., P), the mean values across clusters assigned random permutation of fixed set of base mean values, ensuring that the cluster centers remain well separated in every dimension:
Cluster Mcans:
Perm (5, 25}, Perm (5, 20, 35), K
K 2,
M
Mk E R
Pdk E
Cluster Covariances: For each cluster k symmetric positive definite covariance matrix Σn E Rexp was generated as
Σか = U
where U denotes the cigenvoctor matrix obtained from the cigendccomposition of random sym- metric matrix
R A
[Akks (O,,
The diagonal eigenvalue matrix A = uniform distributions with cluster-specific spectral ranges:
App) contains independent entries drawn from
u(6, 15), 1,
ARi U(4, 10), k k= 2,
u(3, 8), k= (if K =).
This procedure yields full-rank, well-conditioned covariance matrices with diverse eigenvalue spectra, thereby producing clusters of distinct orientation and spread.
Sample Generation: For each cluster k, data samples were generated as
X MxlT.N + E/ 1/22z, Zμ E N Zb (0,1L,
where cach column Tkdt of Zμ represents independent p-dimensional standard normal vector, E/ = U and 1 is a vector of ones length N.N. The total sample count and mixture proportions wore set as
400, K = 2,
[0.5, 0.5).
K 2,
π [0.3, 0.33, 0.37],
600, K 3,
K
24


Reviews:
[{"review_id": "wIIXjXECfr", "reviewer": "Reviewer_LrAE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces an algorithm for performing clustering and parameter estimation over K-dimensional multivariate Gaussian mixture models. The key challenge faced in the manuscript is the presence of what is called *cellwise outliers* (deliberate or non-deliberate corrupted features of observations in the dataset), which the authors consider to be around 1%-10% of their experiments. The modeling choice in the paper is to introduce a sort of hot-encoding binary variables, which mask or unmask the presence of outliers per data-point. In practice, this allows the selection of the sub-covariance matrices and sub-vectors of the Gaussian multivariate parameters to be estimated precisely. The learning algorithm is based on what is described as majorization-maximization (MM), which the current reviewer is not familiar with. In the end, after every step where hypothesis-testing (via false discovery rate of FDR) is made to update the beliefs on the presence of outliers via the binary variables, a constrained-type of optimization is done in closed form for the GMM parameters (mixing weights, means, covariances). Experiments are first conducted for a synthetic two-dim dataset, second for another with different outlier % rates, and third for another real-world dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "To me, the main strengths of the paper are: \n\n1) Closed-form solution for the estimation of the GMM parameters, and the exact computation of the computational cost for the algorithm --- which, unfortunately, is $p^{4}$ for $p$ being the dimension of the Gaussian components.\n2) The choice of using an additional *masking* variable to indicate/estimate the presence of outliers. I do believe it one of the correct ways to model this problem, and it has been longly considered in the literature.\n3) The introduction of the hypothesis-testing (FDR) method is interesting, and I liked how it combines with the maximization algorithm. However, I was not particularly familiar with this sort of solution for the GMM learning problem. As reviewer, I am usually more experienced in the EM-based solutions which have dominated the GMM problem since the 90s.\n4) Despite it is limited to a relatively small experiment with synthetic data, the performance included in Table 1 is impressive and a good sign that some modelling choices are correctly taken. I would've like to see the method tested in cases with misspecified $K$ for instance.", "audience_yn": "No", "audience_explanation": "However, I still have some concerns and *detected* points of weaknesses that I would like to highlight here:\n\n1) GMMs have been for at least *two* decades a key milestone and *starting point* for so many probabilistic modelling problems and challenges. Since the 90s, the EM algorithm stands out as one of the standard solutions, at its easy adaptation to non-Gaussian likelihoods, missing data, and optimization-based learning of parameters have defined a strong SOTA baseline that one should not ignore. In this regard, I see the current proposed work limited due to the lack of comparison and connection with such line of works. The problem introduced is somehow *niched*  to some potential application that is not entirely clear for the reader. Later, one quickly sees the problems that the model could have in three directions: i) larger dimensionality $p$, ii) larger rate of outliers and iii) consideration of non-Gaussian distributed data.\n2) For my taste as reviewer, the paper is somehow limited on the empirical results, and the exploration of the method performance. The synthetic data presentation is a bit *naive* with the original parameters and hyperparameters shown in pp. 13 for later discovering them. Whenever the method is relatively simple or easy to compute/analyse, one in the ML/AI community should expect larger experiments and results so the performance is characterised at a max level.\n3) The experiments and their results shown in Fig. 2 are OK to me, but inevitably remind me to the ones from Figure 1 in Ghahramani & Jordan (1993) where the solution is tested to rates ranging from 1% up to 100%. The way the paper addressed the problem is in many ways similar to such classical method, with EM used to estimate/fill missing data in a robust manner. The lack of discussion and the similarity of the proposed algorithm (taking the definition written in the beginning of section 4.1 pp.5) concern me significantly.\n4) The computational cost is correctly characterised, but I am highly concerned about its feasibility in real-world studies or methods with such super-cubical complexity on the vector parameters.\n5) Last but no least, I miss some references to well-known ML/AI outlier detection methods, and latent-variables (that's what $b$ is somehow, but without prior or admitted randomness). The way it is modelled with such masks, remind me to this work from D. Blei's lab some years ago: https://arxiv.org/pdf/1606.03860. I am aware that the scope is quite different, but still useful for the faced problems in the manuscript.\n\n---\n#### References:\n- _Zoubin Ghahramani, Michael I. Jordan_ (NIPS 1993) *Supervised learning from incomplete data via an EM approach.*", "requested_changes": "I would like to see additional progress on the main points I raised as weaknesses in my review. From all of theses ones, I would prioritize the EM connection and revision, which is perhaps the usual point-of-reference for readers familiar with GMMs. However, I also want to be honest in the sense that I see the current manuscript significantly under the TMLR threshold, parrticularly on the interest/fitting side with the potential readers/researchers/practitioners which focus on TMLR published works. In the end, the paper finds a solution, which does not completely scales or adapts to non-Gaussian problems for a GMM problem, well-known in the community and analyzed since the 90s in multiple ways not exactly revised in the paper.", "broader_impact_concerns": "N/A", "additional_comments": "### Minor issues\n1. I would've moved the numbers and settings of the numerical studies in pp. 10 and pp.13 to the appendix, which is the standard practice for venues like TMLR.", "extra_scores": {}, "extra_text": {}}, {"review_id": "o4WyPcKCuf", "reviewer": "Reviewer_a5w7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel algorithm for Gaussian Mixture Model (GMM)-based clustering that is robust to outliers. This is accomplished by introducing additional variables that index whether a data point is an outlier; specifically, which output dimensions for each data point are deemed to be outliers. The proposed likelihood function is minimized using a majorization minimization framework, with the resulting algorithm, RobGMM, having a computational complexity that is quadratic in the number of clusters ($K)$ and quartic in the output dimension ($p$).\n\nRobGMM is benchmarked against TCLUST and the Student’s t-mixture model on several synthetic datasets, with RobGMM shown to outperform both existing methods. Additionally, experiments on the “Top Gear” dataset demonstrate the capability of RobGMM in detecting outliers while clustering the data into interpretable clusters.\n\nThe proposed algorithm addresses a well-known issue of outliers when performing clustering. Outliers often distort the parameters of the GMM, which renders the inferred model to be less useful for clustering on unseen data. The relative simplicity of RobGMM, and optimizing the objective function by means of majorization-minimization, makes the algorithm appealing for users. The demonstrated performance on the chosen datasets also lend some credence to the fact that RobGMM does a better job of inferring the GMM parameters in the presence of cell outliers.\n\nWhile the relative simplicity of the proposed approach is appreciated, I think the authors have incorrectly justified the choice of thresholds ($_\\eta _t$) by framing it as controlling the false discovery rate (FDR) through the Benjamini & Hochberg procedure. In a multiple-hypothesis testing framework, the hypotheses have to be set beforehand; here, the choice of thresholds is in effect, setting the hypotheses based on the significance levels – this is likely p-hacking. Hence, an alternate justification has to be provided for this design choice. \nIn addition to this, the experiments are a little lacking in several key aspects:\n\n1.\tHow were the specific Gaussian parameters chosen for the synthetic experiments?\n2.\tAdditional experiments on higher-dimension datasets are needed for an extensive validation of RobGMM. The various algorithms should also be compared by time complexity as RobGMM scales quartically with output dimension.\n3.\tThe chosen numerical metrics appear to account for the accuracy of outlier detection as another cluster. \n\n   a.\tDoes the Student’s t-mixture model also include an additional outlier cluster? If not, this unfairly penalizes that model.\n\n   b.\tThe accuracy of outlier detection should probably be separated from that of the clusters. Poor outlier detection with accurate GMM parameter estimation could be an acceptable trade-off, especially when using heavy-tailed distribution models.\n4.\tWhy was only RobGMM was run for the real dataset?", "claims_and_evidence_yn": "Yes", "claims_explanation": "The experimental evidence does support the claims made but the justification for the thresholds has to be corrected.", "audience_yn": "Yes", "audience_explanation": "The paper approaches a known problem of clustering -- the presence of outliers. A novel approach to GMM-based clustering that is robust to outliers while also detecting outliers is a valuable contribution.", "requested_changes": "**Critical changes:**\n\n1.\tPursuant to the comment in the previous section, an alternative justification has to be made for how the thresholds in the hypotheses were selected.\n2.\tMore extensive experimental validation of RobGMM is required to more fully demonstrate that the proposed approach is utile for robust GMM clustering across a variety of settings, and comparable, if not better, than existing methods.\n3.\tClustering of the Top Gear dataset using RobGMM should be compared to that by other robust clustering approaches.\n\n**Suggested changes:**\n\n1.\tThe notation in (11) is confusing. The summation over $t$ runs to $N_i$ which itself is a sum over $N$ terms. This may be incorrect too, since $eta_N$ may never be added as a penalty but $b_i(N)$ contributes to some $N_i$.\n2.\tTypo: First line of Section 4 should read “For the reader’s convenience”.\n3.\tMore methods should be used as benchmarks for a comprehensive evaluation of RobGMM’s performance. \n4.\tIt is not clear in Figure 1 whether the various algorithms detect outliers. Also, the clusters for all the algorithms seem to be very similar.\n5.\tThe first scenario described in Section 5.2 lacks details.\n\n  a.\tThe avoidance of why outlier detections should not occur in the 99-th percentile of any cluster should be better explained. \n\n  b.\tHow were the values shifted?", "broader_impact_concerns": "I do not have any broader impact concerns with this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "zv67q10UaD", "reviewer": "Reviewer_VrhV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary**\n\nThe authors propose a majorization-minimization (MM) algorithm, RobGMM, for learning Gaussian Mixture Models with outliers. The main novelty is that their method accounts for partially corrupted data (cell outliers). The outlier detection is carried out via multiple hypothesis testing (FDR), which in turn provides a penalty term for the likelihood in the MM algorithm.\nThey test their method in clustering problems with synthetic data with manual corruption and in a real world dataset. Their method consistently outperforms other robust learning methods in several metrics.\n\n**Strengths** \n\n- The paper is clearly written and well structured.\n\n- The motivation (clustering under cell outliers) is relevant and their math is solid.\n\n**Weaknesses**\n\n- RobGMM has a complexity per-iteration of $\\mathcal{O}(KNp^4+K^2N)$, where $K$ is the number of mixture components, $N$ the number of data points and $p$ the problem's dimension. A dependence of $p^4$ is considerably high, and probably makes the method unfeasible for high-dimensional data. This doesn't invalidate their method, but at least a discussion of its limitations or a proposal for future improvements would be pertinent.\n\n- As far as I understand, in the experiments they compare RobGMM to robust learning techniques that, in their terminology, consider casewise outliers, while their method detects cellwise outliers. This might be a case of comparing apples and oranges. Is there any other method that considers cellwise outliers? If yes, why not use it as a benchmark? If not, the novelty should be emphasized. I am not familiar with the literature, but I feel the paper lacks more discussion of cellwise outlier detection methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The mathematical derivations and experimental setup are rigorous.", "audience_yn": "Yes", "audience_explanation": "Clustering contaminated data is an important real-world problem, and the paper addresses a difficult version of the problem (partially corrupted data).", "requested_changes": "Overall, this is a good paper worth publishing. However, I believe addressing the **Weaknesses** I mentioned above would significantly strengthen the submission.", "broader_impact_concerns": "No", "additional_comments": "I have a couple of related questions regarding the experimental setup: \n\n- In the synthetic data experiment, the choice of dimension 2 is justified by visualization. However, since your algorithm detects cell-wise outliers, isn't dimension 2 too small for exploiting the ability of RobGMM for detecting outlier dimensions?\n\n- Furthermore, as far as I understand, you create outliers by sampling uniformly distributed data in [-20, 20]. If I understood well, this doesn't create cell-wise outliers, but case-wise outliers. I feel the distinction between both and the ability of your algorithm to detect the former is not well highlighted by this experiment. I acknowledge this is done with the *Top Gear* dataset, but in that case it is not clear how you perturb the \"1% randomly selected cells\". \n\n- This is just out of curiosity: In the synthetic experiment, how do you visualize the outliers (for example in figure 1f)? I mean, given $(x,y)$, RobGMM might discard $x$ but keep $y$, are the points in 1f only those with no outlier dimension? In general, how often does RobGMM discard a complete data point by discarding each dimension? I guess this is controlled by $\\eta_t$, but it seems an interesting metric to assess the efficiency of the algorithm.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)