Towards a Unified Framework of Clustering-based Anomaly Detection
Authors: Zeyu Fang
Ming Gu
Sheng Zhou
Jiawei Chen
Qiaoyu Tan
Haishuai Wang
Jiajun Bu
Venue: ICML 2025 poster (2025)
Keywords: Anomaly Detection
Clustering
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: A unified theoretical framework models the intrinsic connections among representation learning, clustering, and anomaly detection.

Abstract: Unsupervised Anomaly Detection (UAD) plays a crucial role in identifying abnormal patterns within data without labeled examples, holding significant practical implications across various domains. Although the individual contributions of representation learning and clustering to anomaly detection are well-established, their interdependencies remain under-explored due to the absence of a unified theoretical framework. Consequently, their collective potential to enhance anomaly detection performance remains largely untapped. To bridge this gap, in this paper, we propose a novel probabilistic mixture model for anomaly detection to establish a theoretical connection among representation learning, clustering, and anomaly detection. By maximizing a novel anomaly-aware data likelihood, representation learning and clustering can effectively reduce the adverse impact of anomalous data and collaboratively benefit anomaly detection. Meanwhile, a theoretically substantiated anomaly score is naturally derived from this framework. Lastly, drawing inspiration from gravitational analysis in physics, we have devised an improved anomaly score that more effectively harnesses the combined power of representation learning and clustering. Extensive experiments, involving 17 baseline methods across 30 diverse datasets, validate the effectiveness and generalization capability of the proposed method, surpassing state-of-the-art methods.

Towards a Unified Framework of Clustering- based Anomaly Detection
Zeyu Fang' Ming Gu Sheng Zhou'
Jiawei Chen?
Qiaoyu Tan
Haishuai Wang*
Jiajun Bu
Abstract
Representation Learning
Clustering
Unsupervised Anomaly Detection (UAD) plays a crucial role in identifying abnormal patterns within data without labeled examples, holding significant practical implications across various domains. Although the individual contributions of representation learning and clustering to anomaly detection are well estaabissd,, their interdepen- dencies remain under-explored due to the absence of a unified theoretical framework. Consequently, their collective potential enhance anomaly de- tection performance remains largely untapped. To bridge this gap. in this paper, we propose novel probabilistic mixture model for anomaly detec- tion to establish theoretical connection among representation learning, clustering, and anomaly detection. By maximizing a novel anomaly- data likelihood, representation learning and clus- tering can effectively reduce the adverse impact of anomalous data and collaboratively benefit anomaly detection. Mcanwhile, theoretically substantiated anomaly score is naturally derived from this framework. Lastly, drawing inspiration from gravitational analysis in physics, we have devised an improved anomaly score that more effectively harnesses the combined power of rep- resentation learning and clustering. Extensive ex- periments, involving 17 baseline methods across 30 diverse datasets, validate the effectiveness and generalization capability of the proposed method, surpassing state-of-the-as methods.
Anomaly Detection
Mutually Dependent and Enhanced
Figure 1: Interdependent relationships among representation
learning, clustering, and anomaly detection.
within data in the absence of labeled examples (Chandola et al., 2009). It has long received extensive attention in the past decades for its wide-ranging applications in numer- ous practical scenarios, including financial auditing (Baku- menko Elragal, 2022), healthcare monitoring (Salem et al., 2014) and e-commerce sector (Kou al., 2004). Due to the lack of explicit label guidance, the key to UAD is to uncover the dominant patterns that widely exist in the dataset so that samples do not conform to these patterns can be recognized as anomalies. To achieve this, early works (Chalapathy & Chawla, 2019) have heavily relied on powerful unsupervised representation learning methods to extract the normal patterns from high -igh-sinensional and complex data such images, text, and graphs. More recent works (Song et al., 2021; Aytekin et al., 2018) 8) have utilized clustering. widely observed natural pattern in real- world data, to provide critical global information for anomaly de- tection and achieved tremendous success.
While the individual contributions of representation learning and clustering to anomaly detection are well-established, their interrelationships remain largely unexplored. Intu- itively, discriminative representation learning can leverage accurate clustering results to differentiate samples from dis- tinct clusters in the embedding space (i.c., O). Similarly, it can utilize accurate anomaly detection to avoid preserving abnormal patterns (i.e., @). For accurate clustering, it can gain advantages from representation learning by operating in the discriminative embedding space (i.c., Meanwhile, it can potentially benefit from accurate anomaly detection by excluding anomalies when formulating clusters (i.e., @). Anomaly detection can greatly benefit from both discrimi- native representation learning and accurate clustering (i.e., & @). However, these benefits hinge on the successful identification of anomalies and the reduction of their detri-
1. Introduction
Unsupervised Anomaly Detection (UAD) refers to the task dedicated to identifying abnormal patterns or instances
'Zhejiang Key Laboratory of Accessible Perception and In- telligent Systems. Zhejiang University, Hangzhou, China "The State Key Laboratory of Blockchain and Data Security. Hangzhou, China "New York University Shanghai, Shanghai, China. Corre- spondence to: Sheng Zhou <zhousheng zju@ zju.edu.cn>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Towards a Unified Framework of Clustering-based Anomaly Detection
mental impact on the aforementioned tasks. As depicted in Figure 1, the integration of these three elements exhibits significant reciprocal nature. In summary, representation learning, clustering, and anomaly detection are interdepen- dent and intricately intertwined. Therefore, it is crucial for anomaly detection to fully leverage and mutually enhance the relationships among these three components.
clustering results from this framework for anomaly detec- tion. We conduct extensive experiments with 15 baselines on 30 datasets from different data domains to evaluate the effectivvnesss of the proposed method. The results verify the effectiveness and generalization capability in detecting anomalies in real-world applications.
To sum up, we underline our contributions as follows:
Despite the intuitive significance of the interactions among representation learning, clustering, and anomaly detection, existing methods have only made limited attempts to ex- ploit them and fall short of expectations. On one hand, some methods (Zong et al., 2018) have acknowledged the interplay among these three factors, but their focus remains primarily on the interactions between two factors at time, making only targeted improvements. For instance, some strategies include explicitly removing outlier samples during the clustering process (Chawla & Gionis, 2013) or designing robust representation learning methods (Cho et al., 2021) to mitigate the influence of anomalies. On the other hand, recent methods (Song et al., 2021) have begun to explore the simultaneous optimization of these three factors within single framework. However, these attempts are still in the stage of merely superimposing the objectives of the three factors without unified theoretical framework. This lack of a guiding framework prevents the adequate modeling of the interdependencies among these factors, thereby limiting their collective contribution to a unified anomaly detection objective. Consequently, we aim to address the following question: Is it possible to employ a unified theoretical frame- work to jointly model these three interdependent objectives, thereby leveraging their respective strengths to enhance anomaly detection?
We propose unified theoretical framework to jointly optimize representation learning, clustering, and anomaly detection, allowing their mutual enhancement and aid in anomaly detection.
Based on the proposed framework, we derive a theoret- ically grounded anomaly score and further introduce a more comprehensive score with the vector summation, which fully releases the power of the framework for effec- tive anomaly detection.
Extensive experiments have been conducted on 30 datasets to validate the superior unsupervised anomaly detection performance of our approach, which surpassed the offteoof-the-ar through comparative evaluations with 17 baseline methods.
2. Related Work
Various UAD methods have been proposed based on differ- ent assumptions, making them suitable for detecting vari- ous types of anomaly patterns, including subspace-based models (Kriegel et al., 2009), statistical models (Gold- stein & Dengel, 2012), linear models (Wold et al., 1987; Manevitz & Yousef, 2001), density-based models (Bre- unig et al., 2000; Peterson, 2009), ensemble-based mod- els (Pevný, 2016; Liu et al., 2008), probability- mod- els (Reynolds et al., 2009; Zong et al., 2018: Li et al., 2022; 2020), representation--asse models (Ruff et al., 2018; Xu et al., 2023; Goyal et al., 2020; Qiu et al., 2021), and cluster- based models (He et al., 2003; Chawla & Gionis, 2013). Considering the field of anomaly detection has progressed by integrating clustering information to enhance detection accuracy (Li et al., 2021: Zhou al., 2022), we primarily focus on and analyze anomaly patterns related to clustering, incorporating a global clustering perspective to assess the degree of anomaly. Notable methods in this context include CBLOF (He et al., 2003), which evaluates anomalies based on the size of the nearest cluster and the distance to the nearest large cluster. Similarly, DCFOD (Song et al., 2021) introduces innovation by applying the self seff-training architec- ture of the deep clustering (Xie et al., 2016) to outlier detec- tion. Meanwhile, DAGMM (Zong et al., 2018) combines deep autoencoders with Gaussian mixture models, utilizing sample energy as a metric to quantify the anomaly degree. In contrast, our approach introduces a unified theoretical
In this paper, we try to answer this question and propose a novel model named UniCAD for anomaly detection. The proposed UniCAD integrates representation learning, clus- tering, and anomaly detection into a unified framework, achieved through the theoretical guidance of maximizing the anomaly-aware data likelihood. Specifically, we explic- itly model the relationships between samples and multiple clusters in the representation space using the probabilistic mixture models for the likelihood estimation. Moreover, we creatively introduce learnable indicator function into the objective of maximum likelihood to explicitly attenu- ate the influence of anomalies on representation learning and clustering. Under this framework, we can theoretically derive an anomaly score that indicates the abnormality of samples, rather than heuristically designing it based on clus- tering results as existing works do. Furthermore, building upon this theoretically supported anomaly score and inspired by the theory of universal gravitation, we propose a more comprehensive anomaly metric that considers the complex relationships between samples and multiple clusters. This allows to better utilize the learned representations and
2

---

Towards a Unified Framework of Clustering-based Anomaly Detection
framework that integrates representation learning, cluster- ing, and anomaly detection, overcoming the limitations of heuristic designs and the overlooked anomaly influence in existing methods.
observed data X:
max log p( X θ Φ)
max
log
(1)
3. Methodology
k
Σ ((x,) log Σ G ==1
In this section, we first define the problem we studied and the notations used in this paper. Then elaborate the proposed method UniCAD. More specifically, we first introduce a novel learning objective that optimizes represen- tation learning, clustering, and anomaly detection within a unified theoretical framework by maximizing the data like- lihood. A novel anomaly score with theoretical support is also naturally derived from this framework, Then, inspired by the concept of universal gravitation, we further propose an enhanced anomaly scoring approach that leverages the intricate relationship between samples and clustering to de- tect anomalies effectively. Finally, we present an efficient iterative optimization strategy to optimize this model and provide a complexity analysis for the proposed model in Appendix D.4.
= max
where с; represents the latent cluster variable associated with x,, and C₁ = A denotes the probabilistic event that x; belongs to the k-th cluster. The 8(x;) is an indicator function that determines whether a sample x; is an anomaly of value 0 or a normal sample of value
3.1.1. JOINT REPRESENTATION LEARNING AND CLUSTERING WITH ( b)
Based on the aforementioned advantages of MMs, we esti- mate the likelihood p(x, O h) with mixture models defined as:
K
②) = O
p C: k)0,
k=
Σ p(c: = kb) .)/ppx/sc = k, O, pk, Ex)
K
wk -p(xpp = k, U, pe: Ex), Σ
Definition 1 (Unsupervised Anomaly Detection). Given dataset X RND comprising N instances with D- dimensional features, unsupervised anomaly detection aims to learn an anomaly score o; for each instance x, in an unsupervised manner so that the abnormal ones have higher scores than the normal ones.
=
(2)
where + {ulk: Mk: Ex}. The mixture model is parame- terized by the prototypes pa, covariance matrices Σx, and mixture weights wi from all clusters. Σ Wi and k = 1,2 K.
3.1. Maximizing Anomaly-aware Likelihood
In practice, the samples are usually attributed to high- dimensional features and it is challenging to detect anoma- lies from the raw feature space (Ruff et al., 2021). There- fore, modern anomaly detection methods (Ruff et al., 2018; Zong al., 2018) often map raw data samples X (x:) E Nx into low-dimensional representation space Z = {z.) E R with representation learning func- tion z, = fx(x;) and detect anomalies within this latent representation space.
Previous research has demonstrated the importance of discriminative representation and accurate clustering in anomaly detection (Song al., 2021). However, the pres- ence of anomalous samples can significantly disrupt the effectiveness of both representation learning and cluster- ing (Duan et al., 2009). While some existing studies have at- tempted to integrate these three separate learning objectives, the lack of a unified theoretical framework has hindered their mutual enhancement, leading to suboptimal results.
Following this widely adopted practice, we model the distri- bution of samples in the latent representation space with a multivariate Student's: s-t distribution giving its cluster e; k. The Student's. 's-t distribution is robust against outliers due to its heavy tails. Bayesian robustness theory leverages such distributions to dismiss outlier data, favoring reliable sources, making the Student's s-t process preferable over Gaussian processes for data with atypical information (An- drade, 2023). Thus the probability distribution of generating
To tackle this issue, in this paper, we propose a unified and coherent approach that considers representation learn- ing, clustering, and anomaly detection by maximizing the likelihood of the observed data. Specifically, we denote the parameters of representation learning as O the clus- tering parameter as Q and the dynamic indicator function for anomaly detection as 8(-). These parameters are opti- mized simultaneously by maximizing the likelihood of the
3

---

Towards a Unified Framework of Clustering-based Anomaly Detection
x; with latent representation 2; given its cluster c; = k can be expressed as:
3.2. Gravity-inspired Anomaly Scoring
In practical applications, it is proved that anomaly scores derived from generation probabilities often yield suboptimal performance (Han et al., 2022). This observation prompts reconsideration of how to fully leverage the complex rela- lionships among samples or even across multiple clusters for anomaly detection. In this section, we first provide a brief introduction to the concept of Newton's Law of Univer- sal Gravitation (Newton, 1833) and then demonstrate how the anomaly score is intriguingly similar this cross-field principle. Finally, we discuss the advantages of introducing the vector sum operation into the anomaly score inspired by the analogy.
p(x:/e = HKk pel. Ex)
I( I 1/2 T(E) (1+ D (pn(()))
-
(3)
where 2, fx+) denotes the representation obtained from the data mapped through the neural network pa- rameterized by O. I denotes the gamma function while V is the degree of freedom. ΣE is the scale parameter. Dar 1k) = (
" (z: (4a) represents the Mahalanobis distance (MeLachlan, 1999). In the unsu- validation set or learning it is unnecessary, V is set as 1 for all experi- ments (Xie et al., 2016; Van Der Maaten, 2009). The overall marginal likelihood of the observed data x; can be simpli- fied as:
3.2.1. ANALOG ANOMALY SCORING AND FORCE ANALYSIS
pervised setting, cross- woossvvllidttngg/
To begin with, Newton's Law of Universal Gravitation (New- ton, 1833) stands as a fundamental framework for describing the interactions among entities in the physical world. Ac- cording to this law, every object in the universe experiences an attractive force from another object. In classical mechan- ics, force analysis involves calculating the vector sum of all forces acting on an object, known as the resultant force, which is crucial in determining an object's acceleration or change in motion:
K
[Etel 1/2
0(x,10,0) D)=
Wha
π
(Mi 146)2
(4)
3.1.2. ANOMALY INDICATOR 0(x;) AND SCORE O,
As we have discussed, the indicator function 0(x;) not only benefits both representation and clustering but also directly serves as the output of anomaly detection. Ideally, with the percentage of outliers denoted as I, an optimal solution for o(x,) that maximizes the objective function ((O, 4) entails setting all 5(x,)= 0 for x; among the I percent of outliers with lowest generation possibility p(x,/0,0 4), and otherwise 6(x,) = 1 is set for the remaining normal samples. Therefore, the indicator function is determined as:
K F iks with Fix = Σ
G mmmm
F 5,ttolll =
Fuk
2
rik
where Fas represents the k-th force acting on the object i. This force is proportional to the product of their masses, (m, and my), and inversely proportional to the square of the distance r'v between them. G represents the gravitational constant, and Fig is the unit direction vector.
Similarly, if denoting: FiR = p(x;, e k|O, +) = Wk-
5 1.pExl 1/2 the score of Equation (6) bears analogies to the summation of the magnitudes of forces
0, if f(xx((0,) () is among the lowest, 1, otherwise.
8(x,)
(5)
D+D (x (2:3
As this method involves sorting the samples based on the generation probability as being anomalous, the values of p(x; (O, +) can serve as a form of anomaly score, a classic approach within the mixture model framework (Reynolds al., 2009; Zong et al., 2018). This suggests that the likeli- hood of sample being anomalous is inversely related to its generative probability since a lower generative probability indicates a higher chance of the sample being an outlier. Thus the anomaly score of sample x; can be defined as:
G m.m.
0
with Fu
(8)
K Σ
Fik
where G = n mh We | 1/2 m, 1, and FER = / DM (z,, na) Here, Fix is taken as the mea- sure of distance within the representation space, modified slightly by additional term for smoothness. The constant G serves role akin to the gravitational constant in this analogy, whereas my resembles the concept of mass for the cluster. The notation m; suggests a standardization where the mass of each data point is considered uniform and not differentiated.
1
9 0)
Σ
K
Wk 1+ (H 12.
- DE 1/2
(6)
4

---

Towards a Unified Framework of Clustering-based Anomaly Detection
Mi
41
Algorithm Model Training for UniCAD
P4
Ha
Require:
Data points X, cluster number K, outlier ratio I,
Fis Fos
Fir + Foz Puz + Pos
Per
Pea
v
tolerance X, iterations t
Ensure: Network parameters θ mixture parameters
Hs
{wk, Mkx }
P2
Pz
1: Initialize θ and { Wk. Σx}}
(a) Scalar Sum
(b) Vector Sum
2: for i to t do
if i then
Figure 2: Analysis of gravitational force.
X; + X
5: else
6: Re-order points in X such that 01 2 ... On
3.2.2. ANOMALY SCORING WITH VECTOR SUM
7:
L + 121...... .........2..... .N
8:
X; X L [Remove top-1 anomalies)
Comparing Equation (7) with Equation (8), what still differs
end
is that, unlike simple sum of the scalar value, the resul- tant force F, F.pocal employs the vector sum and incorporates both the magnitude and direction Fix of each force. This distinction is crucial because forces in different directions
10: Update θ via Equation (15)
11:
while [](O), () jold (0, +)] > A do
12:
Jold (o, 1) < J(e, b)
13:
Calculate via Equation (10)
can neutralize each other with a large angle between them or enhance each other's effects with a small angle. Inspired by this difference, consider modeling the relationship between samples and clusters as vector, and aggregat- ing them through vector summation. The vector-formed anomaly score of' is defined as:
14:
Update fwks Mk, En} via Equations (12), (13), (14)
15:
end while
16: Calculate oR via Equation (9)
17: end for
18: return θ (wk Mkx Σx}
of
(9)
3.3. Iterative Optimization
E k= Fic- Fill
Given the challenge posed by the interdependence of the parameters of the network @ and those of the mixture model [wk, p4: Σx} in joint optimization, we propose an iterative optimization procedure. The pseudocode for training the model is presented in Algorithm 1.
where F. represents the unit direction vector in the repre- sentation space from the sample z, to the cluster prototype Hg and represents the Le norm.
3.2.3. ADVANTAGES OF VECTOR SUM
3.3.1. UPDATE +
The application of the vector sum principle extends beyond physical mechanics and finds relevance in various domains. In relational embedding (Bordes et al., 2013), for example, relationships can be represented as vectors. Aggregating these vectors allows for capturing complexities like transi- tivity, symmetry, and antisymmetry.
To update the parameters of the mixture model + {wk, pk: } we use the Expectation Expectation-Maximizie Maximization (EM) algorithm to maximize equation (1) (Peel & McLachlan, 2000). The detailed derivation is included in Appendix B.
E-step. During the E-step of iteration (t +-++1 1), our goal is to compute the posterior probabilities of each data point be- longing to the k--th cluster within the mixture model. Given the observed sample and the current estimates of the pa- rameters (t)) and (ttt, the expected value of the likelihood function of latent variable ct. or the posterior possibilities, can be expressed as:
Similarly, in our context, the vector sum can help capture more complex relationships along clusters. In Figure 2, sample is attracted to two groups of cluster prototypes, {MI H22) and (4)) with equal mass and distances. While both groups exert equal forces, we argue that their influences differ: a sample near two clusters with a large difference is more likely to be an anomaly than one near similar clusters. For instance, user liking both money- saving tips and luxury items is more anomalous than one liking two similar luxury items. The vector sum shows that the total force from (pi,Pa) is smaller, leading to a higher anomaly score, thus demonstrating its effectiveness identifying subtle distinctions among clusters.
p C. k10,
F(O)
(1+1) T
Σ p(x;, = 1 Φ(1)
K
(10)
The scale factor(Peel & McL.cchaan, 2000) serving as an
5

---

Towards a Unified Framework of Clustering-based Anomaly Detection
intermediate result for subsequent updates in the M-step is:
4. Experiments
(0) (1) i 1+ (M
(++1)
2
(11)
4.1. Datasets & Baselines
We evaluated UniCAD on an extensive collection of datasets, comprising 30 tabular datasets that span 16 diverse fields. We specifically focused on naturally occurring anomaly patterns, rather than synthetically generated injected anomalies, as this aligns more closely with real-world sce- narios. The detailed descriptions are provided in Table 4 of Appendix D. Following the setup in ADBench (Han et al., 2022), we adopt an inductive setting to predict newly emerging data, a highly beneficial approach for practi- cal applications. The code for reproducing our experi- ments is publicly available at https: /ttttss:/ge http///ggtttbbbt/ com/ BabelTower /UniCAD.
M-step. In the M-step of iteration (t+1), given the gradients (J().)) = 0, (J((.4) 0, and ( = O, we derive dwn
دنكة int(l NN. The updating process for {wh (t+1) (f+1) ((++)) } is as follows: 11
the analytical solutions for the mixture model parameters
wis Mk, and Σx. Assume the anomalous ratio is E [0, I], the number of the normal samples
The mixture weights we are updated by averaging the posterior probabilities over all data points with the num- ber of samples, reflecting the relative presence of each component in the mixture:
To assess the effectiveness of UniCAD, we compared it with 17 advanced unsupervised anomaly detection methods, in- cluding: (1) traditional methods: SOD (Kriegel et al., 2009) and HBOS (Goldstein & Dengel, 2012); (2) linear methods: PCA (Wold et al., 1987) and OCSVM (Manevitz & Yousef, 2001); (3) density-based methods: LOF (Breunig al.,
(++1)
(1+1) 1kk
nn.
(12)
1=1
The prototypes μe are updated to be the weighted aver-
2000) and KNN (Peterson, 2009); (4) ensemble-based meth-
age of the data points, where weights are the posterior probabilities:
ods: LODA (Pevný, 2016) and IForest (Liu et al., 2008); (5) probability-assdd methods: DAGMM (Zong et al., 2018), ECOD (Li et al., 2022), and COPOD (Li et al., 2020);
((+1) P4
(1+1) (++1) Take
(t+1)
((+t))
(6) cluster-based methods: DBSCAN (Ester et al., 1996),
Uik zi)
ik
U,
CBLOF (He al., 2003), DCOD (Song et al., 2021) and KMeans- (Chawla & Gionis, 2013); and (7) representation- based methods: DeepSVDD (Ruff et al., 2018) and DIF (Xu et al., 2023). These baselines encompass the majority of the latest methods, providing a comprehensive overview of the state-of-the-art. For a detailed description, please refer to Appendix D.2.
(13)
The covariance matrices Σg are updated by considering the dispersion of the data around the newly computed prototypes:
Σ
(++)
Ei=
ik
11+1) ((+1)
ik
(2;
1k
(++1) ((=
14 1)+
ΣØ=1 Tig
K
((+1)
4.2. Experiment Settings
(14)
In the unsupervised setting, we employ the default hyperpa- rameters from the original papers for all comparison meth- ods. Similarly, the UniCAD also utilizes a fixed set of parameters ensure fair comparison. For all datasets, we employ two-layer MLP with hidden dimension of d = 128 and ReLU activation function as both encoder and decoder. We utilize the Adam optimizer (Kingma & Ba, 2014) with a learning rate of 1e-4 for 100 epochs. For the EM process, we set the maximum iteration number to 100 and tolerance of -3 for stopping training when the objectives converge. The number of components in the mix- ture model is set h: 10, and the proportion of the outlier is set = 1%. We evaluate the methods using Area Under the Receiver Operating Characteristic (AUC-ROC) and Area Under the Frecision-Recal Curve (AUC-PR) met- rics (Han et al., 2022), reporting the average ranking (Avg. Rank) across all datasets. All experiments are run times with different seeds, and the mean results are reported.
3.3.2. UPDATE +
We focus on momaly-aware representation learning and use stochastic gradient descent to optimize the network parameters θ by minimizing the following joint loss:
L= c=- ②) g()),
(15)
where J(O, ④)- log Φ). An additional con- straint term g()) is introduced to prevent shorteut solu-
tion (Geirhos et al., 2020). In practice, autoencoder architecture is implemented, utilizing reconstruction loss s(0)= ||- 2 as the constraint.
These updates are iteratively performed until convergence, resulting in optimized model parameters that best fit the given data according to the mixture model framework.
6

---

Towards a Unified Framework of Clustering-based Anomaly Detection
Table 1: AUCROC of 10 unsupervised algorithms on 30 tabular benchmark datasets. In each dataset, the algorithm with the highest AUCROC marked in red, the second highest blue, and the third highest in green.
OC
DA GMM
DB
UniCAD w/ of
UniCAD w/ o
Dataset
LOF
IForest
ECOD
CBLOF
DCOD
KMeans--
DIF
SVM
SCAN
annthyroid
57.23
70.20
82.01
56.53
78.66
50.08
62.28
55.01
64.99
66.76
75.27
72.72
backdoor
85.04
85.79
72.15
55.98
86.08
76.55
81.91
79.57
89.11
92.87
87.28
89.24
breastw
80.30
40.61
98.32
N/A
99.17
85,20
96.86
99.02
97.05
77.45
98.15
98.56
cumpaign
65.70
59.04
71.71
56.03
76.10
50.60
64.34
63.16
63.51
67.53
73.52
73.64
celeba
70.70
38.95
70.41
44.74
76.48
50.36
73.99
91.41
56.76
65.29
81.38
82.00
census
54,90
47.46
59.52
59.65
58.50
60.17
72.84
63.33
59.66
67.84
glass
35.36
69.20
77.13
76.09
65.83
54.55
78.30
78.07
77.30
84.57
79.52
82.17
Hepatitis
67.75
38.06
69.75
54,80
75.22
68.12
73.05
48.38
64.64
74.24
75.53
80.62
http
99.59
27.46
99.96
N/A
98.10
49.97
99.60
99.53
99.55
99.49
99.53
99.52
lonosphere
75.92
90.59
84.50
73.41
73.15
81.12
90.79
57.78
91.36
89.74
92.04
90.37
landsat
36.15
53.90
47 47.64
43.92
36,10
50.17
63.69
33.40
55.31
54.84
49.60
57.37
Lymphography
99.54
89.86
99.81
72.11
99.52
74.16
99.81
81.19
100,00
83.67
99.29
99.73
mnist
82.95
67.13
80.98
67.23
74.61
50.00
79.96
65.23
82.45
88.16
86.00
86.64
musk
80.58
41.18
99.99
76.85
95.40
50.00
100.00
42.19
72.16
98.22
99.92
100.00
pendigits
93.75
47.99
94.76
64.22
93.01
55.33
96.93
94.33
94.37
93.79
95.12
95.52
Pima
66.92
65.71
72.87
55.93
63.05
51.39
71.49
72.16
70.44
67.28
75.16
74.87
satellite
59.02
55.88
70.43
62.33
58.09
55.52
71.32
55.97
67.71
74.52
72.46
77.65
satimage-2
97.35
47.36
99.16
96.29
96.28
75.74
99.84
86.01
99.88
99.63
99.87
99.88
shuttle
97.40
57.11
99.56
97.92
99.13
50.40
93.07
97.20
69.97
97.00
99.15
98.75
skin
49.45
46.47
68.21
N/A
49.08
50.00
68.03
64.34
65.47
66.36
72.26
69.69
Stamps
83.86
51.26
91.21
88.89
87.87
52.08
69.89
93.41
79.78
87.95
91.37
94.18
thyroid
87.92
86.86
98.30
79.75
97.94
53.57
94.74
78.55
92.26
96.26
97.66
97.48
vertebral
37.99
49.29
36.66
53.20
40.66
49.74
41.01
38.13
38.14
47.20
33.11
47.37
vowels
61.59
93.12
73.94
60.58
62.24
57.50
92.12
51.56
93.45
81.02
88.38
92.09
Waveform
56.29
73.32
71.47
49.35
62.36
66.41
71.27
63.47
74.35
75.33
71.81
74.29
WBC
99.03
54.17
99.01
N/A
99.11
96.88
94.92
97.45
81.27
97.68
98.93
Wilt
31.28
50.65
41.94
37.29
36.30
49.96
34.50
44.71
34.91
39.46
48.95
52.56
wine
73.07
37.74
80.37
61.70
77.22
40.33
27.14
82.18
27.36
41.69
82.72
95.25
WPBC
45.35
41.41
46.63
47.80
46.65
52.22
45.32
49.67
45.01
44.69
48.02
49.90
Avg. Rank
7.8
8,9
5.1
8.7
6,4
9.3
5.7
7.4
6.0
5.8
3.7
2.6
4.3. Performance and Analysis
ing a unified framework for joint representation learning, clustering, and anomaly detection.
Performance Comparison. Table 1 presents comparison of UniCAD with 10 unsupervised baseline methods across 30 tabular datasets using the AUC-ROC metric. The experi- mental results, which encompass 17 baselines, are included in Tables 5 and 6 of Appendix D.3. Our proposed Uni- CAD achieves the top average ranking, exhibiting the best or near-best performance on a larger number of datasets and confirming advanced capabilities. is noteworthy that there no -ee-size-fits-all unsupervised anomaly detection method suitable for every type of dataset, demonstrated by the observation that other methods have also achieved some of the best results on certain datasets. However, our model showcased a remarkable ability to generalize across most datasets featuring natural anomalies, as evidenced by methods statistical average ranking. As for clustering- such KMeans--, DCOD, and CBLOF, they mostly rank in the top tier among all baseline methods, supporting the advantage of combining deep clustering with anomaly detec- tion. However, our method significantly outperformed these methods by mitigating their limitations and further provid-
Effectiveness of Vector Sum in Anomaly Scoring. As demonstrated in Table 1, we compare the anomaly score og derived directly from the generation possibility with its vector summation form of According to our statistical findings, we observe that vector scores of consistently out- perform scalar scores o,, This indicates that the introduction of the vector summation, analogous to the concept of re- sultant force, makes a substantial difference in anomaly detection scenarios involving multiple clusters. The perfor- mance gains of the vector sum scores strongly demonstrate the effectiveness differences in the distinctions among multiple clusters and underscore the utility of this factor in the context of anomaly detection based clustering.
the UniCAD in capturing the subtle
Runtime Comparison. We present analysis of the run- time performance of various methods, including our pro- posed approach, as detailed Table 2. Our experiments, conducted on the backdoor dataset, reveal that while non-

---

Towards a Unified Framework of Clustering-basee Anomaly Detection
99.8
093
96
035
026
AJC AUC- I.C.ROC(S) (%)
AUC: (R(6)
853
878
023
99.4
92
99.2
0
Z
4
6
0222
lterations
(a) Optimization Analysis
(b) AUC-ROC Surface
AUC-PR Surface
Figure 3: (a) demonstrates the performance variations during the optimization process the satimage-2 dataset. (b) & (c) Analysis of cluster count k, anomaly ratio I.
Table Runtime Comparison. The runtime is reported in seconds (s).
importance of deriving both the optimization objectives and anomaly scores from the likelihood generation probability through a theoretical framework, which allows for unified joint optimization anomaly detection and clustering the representation space. Furthermore, the indicator function 8(x,) also contributes results further confirm the effectiveness of our UniCAD in mitigating the negative influence of anomalies in the clus- tering process, as the existence of outliers may significantly degrade the performance of clustering. summary, all these ablation studies clearly demonstrate the effectiveness of our theoretical framework simultaneously considering representation learning, clustering, and anomaly detection.
Phase
FFoest
KMeans--
DAGMM
DCOD
UniCAD
0.256 0.018
103.697 0.059
795.004
4548.634
246.113
Infer
4.190
16.190
0.079
performance increase. These
Table Ablation Study on 30 Datasets (AUC-ROC)
Gauss. Dist.
w/o Objective
w/o Indicator
Full Model 4.2
Metric
Avg. Rank
6.2
6.6
5.0
4.5. Hyperparameters Analysis
deep learning methods exhibit lower runtime, they often simplify the problem space excessively, failing to capture the complex non-linear relationships present in the data. In contrast, method, when compared existing deep learning techniques, demonstrates significant reduction in computational time. This indicates that our approach not only manages to efficiently model complex patterns but also achieves an optimal balance between computational efficiency and modeling capability.
This section analyzes how hyperparameters affect our model's performance during the iterative training process. As shown in Figure 3a, tracked iteration counts from 0 to 10 for the satimage-2 dataset, keeping other parameters constant. The AUC-ROC and AUC-PR curves demonstrated stable performance with only minor fluctuations initially, highlighting the convergence of the iterative EM optimiza- tion. We also conducted a sensitivity analysis on key hyper- parameters for the donors dataset, focusing on the number of clusters A and the outlier set proportion l. The results, shown in Figure 3, reveal that the optimal is generally lower than the actual anomaly proportion. Furthermore, pattern was observed with the number of clusters k, where the model performance initially improved with an increase in followed by a subsequent decline. This suggests the existence of an optimal range for the number of clusters, which should be carefully selected based the specific application context.
4.4. Ablation Studies
In this section, we examine the contributions different components UniCAD. Tables reports the results. We make three major observations. Firstly, the anomaly de- tection performance experiences significant drop when replacing the Student's distribution with a Gaussian distri- bution for the Mixture Model, highlighting the robustness of the Student's is distribution in unsupervised anomaly de- tection. Secondly, omitting the likelihood maximization loss (w/o [e, 2)) also results in a considerable decrease in overall performance. This observation underscores the
8

---

Towards a Unified Framework of Clustering-based Anomaly Detection
5. Conclusion
the 2000 ACM SIGMOD international conference on
Management of data, pp. 93-104, 2000.
This paper presents UniCAD, a novel model for Unsuper- vised Anomaly Detection (UAD) that seamlessly integrates representation learning, clustering, and anomaly detection within unified theoretical framework. Specifically, Uni- CAD introduces an anomaly-aware data likelihood based the mixture model with the Student-t distribution to guide the joint optimization process, effectively mitigating the im- pact of anomalies on representation learning and clustering. This framework enables theoretically grounded anomaly score inspired by universal gravitation, which considers complex relationships between samples and multiple clus- ters. Extensive experiments 30 datasets across various domains demonstrate the effectiveness and generalization capability of UniCAD, surpassing 15 baseline methods and establishing as a state-of-the-art solution in unsupervised anomaly detection.
Chalapathy, R. and Chawla, S. Deep learning for anomaly detection: survey, arXiv preprint 1ttt::9901.340000 2019.
Chandola, V., Banerjee, A., and Kumar, V. Anomaly detec- tion: A survey. ACM computing surveys (CSUR), 41(3): 1-58, 2009.
Chawla, S. and Gionis, A. k-means-: unified approach to clustering and outlier detection. Proceedings of the 2013 SIAM international conference on data mining, pp- 189-197. SIAM, 2013.
Cho, H., Scol, J., and Lee, S.-g. Masked contrastive learning for anomaly detection. arXiv preprint arXiv:2105.08795 2021.
Duan, L., Xu, L., Liu, Y., and Lee, J. Cluster-based outlier detection. Annals of Operations Research, 168:151-168, 2009.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al. A density- based algorithm for discovering clusters large spatial databases with noise. In ked, volume 96, pp. 226-231, 1996.
Acknowledgements
Geirhos, R., Jacohsen, J.-H., Michaelis, C. Zemel, R., Bren- del, W., Bethge, M., and Wichmann, F. A. Shortcut learn- ing in deep neural networks. Nature Machine Intelligence, 2(1 1):665-673, 2020.
This work is supported by the National Natural Science Foundation of China (Grant No. 62476245), Zhejiang Provincial Natural Science Foundation of China (Grant No. LTGG23F030005).
Goldstein, M. and Dengel, A. Histogram- based outlier score (hbos): A fast unsupervised anomaly detection algorithm. KI-2012: poster and demo track, 1:59-63, 2012.
References
Andrade, J. A. A. On the robustness to outliers the student-t process. Scandinavian Journal of Statistics, 50 (2):725-749, 2023.
Goyal, S., Raghunathan, A., Jain, M., Simhadri, H. V., and Jain, P. Droce: Deep robust one-class classification. In International conference on machine learning, pp. 3711- 3721. PMLR, 2020.
Aytekin, C., Ni, X., Cricri, F., and Aksu, E. Clustering and unsupervised anomaly detection with 22 normalized deep auto-encoder representations. In 2018 International Joint Conference on Neural Networks (IJCNN, pp. 1-6. IEEE, 2018.
Han, S., Hu, X., Huang, H., Jiang, M., and Zhao, Y. Ad- bench: Anomaly detection benchmark. Advances in Neu- ral Information Processing Systems, 35:32142-32159, 2022.
Bakumenko, A. and Elragal, A. Detecting anomalies in fi- nancial data using machine learning algorithms. Systems, 10(5):130, 2022.
He, Z., Xu, X., and Deng, S. Discovering cluster-based local outliers. Pattern recognition letters, 2003.
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., and Yakhnenko, O. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26, 2013.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint 1xXiv:1412.6999 2014.
Kou, Y., Lu, C.-T., Sirwongwattana, S., and Huang, Y.-P. Survey of fraud detection techniques. In IEEE Interna- tional Conference on Networking, Sensing and Control, 2004, volume 2, pp. 749-754. IEFE, 2004.
Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J. Lof: identifying density-based based local outliers. In Proceedings

---

Towards a Unified Framework of Clustering-based Anomaly Detection
Kriegel, H.-P.. Kröger, P., Schubert, E., and Zimek, A. Outlier detection in axis-parallel subspaces of high di- mensional data. In Advances in Knowledge Discovery and Data Mining: 13th Pacific-Asia Conference, PAKDD 2009 Bangkok, Thailand, April 27-30, 2009 Proceedings 13. pp. 831-838. Springer, 2009.
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., and Müller, K.-R. A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE, 109(5):756-795, 2021.
Salem, O., Liu, Y., Mehaoua, A., and Boutaba, R. Online anomaly detection in wireless body area networks for re- liable healthcare monitoring. IEEE journal of biomedical and health informatics, 18(5):141111511 2014.
Li, J., Izakian, H., Pedrycz, W., and Jamal, I. Clustering- based anomaly detection multivariate time series data. Applied Soft Computing, 100:106919, 2021.
Li, Z., Zhao, Y., Botta, N., Ionescu, C., and Hu, X. Copod: copula- based outlier detection. In 2020 IEEE interna- tional conference data mining (ICDM), pp. 1118-1123. IEEE, 2020.
Song, H., Li, P., and Liu, H. Deep clustering based fair out- lier detection. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, PP- 1481-1489, 2021.
Van Der Maaten, L. Learning parametric embedding by preserving local structure. In Artificial intelligence and statistics, pp. 384-391. PMLR, 2009.
Li, Z., Zhao, Y., HU, X., Botta, N., Ionescu, C., and Chen, G. Ecod: Unsupervised outlier detection using empirical cumulative distribution functions. IEEE Transactions on Knowledge and Data Engineering, 2022.
Wold, S., Esbensen, K., and Geladi, P. Principal compo- nent analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37-52 1987.
Liu, T., Ting, K. M., and Zhou, Z.-H. Isolation forest. In 2008 eighth ieee international conference data mining, pp. 413-422, 422. IEEE, 2008
Xie, J., Girshick, R., and Farhadi, A. Unsupervised deep embedding for clustering analysis. In International con- ference on machine learning. pp. 478-487. PMLR, 2016.
Manevitz, L. M. and Yousef, M. One-class svms for doc- ument classification. Journal of machine Learning re- search, 2(Dec): 2001.
Xu, H., Pang, G., Wang, Y., and Wang, Y. Deep isola- tion forest for anomaly detection. IEEE Transactions on Knowledge and Data Engineering. 2023.
McLachlan, G. J. Mahalanobis distance. Resonance, 4(6): 20-26, 1999.
Zhou, S., Huang, X., Liu, N., Tan, Q. and Chung, F.- L. Unseen anomaly detection on networks via multi- hypersphere learning. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pp- 262-270. SIAM, 2022.
Newton, I. Philosophiae naturalis principia mathematica, volume 1. G. Brookman, 1833.
Peel, D. and McLachlan, J. Robust mixture modelling using the distribution. Statistics and computing, 10: 339-348, 2000.
Zong B. Song, Q., Min, M. R., Cheng, W., Lumezanu, C., Cho, D., and Chen, H. Deep autoencoding gaus- sian mixture model for unsupervised anomaly detection. In International conference on learning representations. 2018.
Peterson, L. E. K-nearest neighbor. Scholarpedia, 4(2): 1883, 2009.
Pevný, T. Loda: Lightweight on-line detector of anomalies. Machine Learning, 102:275-304, 2016.
Qiu, C., Pfrommer, T., Kloft, M., Mandt, S., and Rudolph, M. Neural transformation learning for deep anomaly detection beyond images. In International conference machine learning, pp. 8703-870 87 8714. PMLR, 2021.
Reynolds, D. A. et al. Gaussian mixture models. Encyclo- pedia of biometrics, 741(659-663), 2009.
Ruff, L., Vandermeulen, R., Goemnizz N., Deecke, L., Sid- diqui, S. A., Binder, A., Müller, E., and Kloft, M. Deep one-class classification. In International conference on machine learning, pp. 4393 4302-4402, PMLR, 2018
10

---

Towards a Unified Framework of Clustering-based Anomaly Detection
A. Iterative Training Algorithm
B. Derivation of EM Algorithm
This appendix provides the detailed derivation of the Expectation--Maxiiiaaiio (EM) algorithm for optimizing the parameters of mixture model based on Student's t-distribution. The focus on deriving analytical solutions for the maximization of the parameters + (14: wk) of the mixture components. The EM algorithm alternates between two steps:
In the E-step, we calculate the posterior probabilities Tik, representing the probability of data point i belonging to cluster k given the current parameters. The posterior probabilities for a Student's 's t-distribution mixture model are formulated as:
Wk p(z: paa. 23k)
7ik
K Σラ= Wy plzi 20g)
(16)
where (2: Ex) denotes the Student's t-distribution for data point with respect to cluster k, and K the number of mixture components.
The Student's t-distribution is depicted as hierarchical conditional probabillity, resembling Gaussian distribution with an accuracy scale factor u, where its latent variable follows gamma distribution. Adopting a degree of freedom 1, the value of U,A is given by:
1
2
(17)
V + DM (24. pa)
DM (2s: 14)
In the M-step, we update the parameters = model, the likelihood function for Student's s-t Distribution Mixture Model (SMM) is represented as:
{wk, PA. and Σ&} using the derivatives obtained the previous steps. In our
NN
K
L(w, μz 2 =
Wk 1+ (z. M4) Σg (z: -44)
(18)
i=1 k=1
where wa are the mixture weights, Eg the covariance matrices, μ4 the means, and z; the data points.
The derivative with respect wa must consider the constraint that the sum of the mixture weights equals i.e. En Wh Hence, we introduce Lagrange multiplier to address this constraint and construct the Lagrangian L':
1.
L' E.A) L(w, u. Σ) + +
1
Σ k=1
Wk
(19)
The derivative with respect to wi is:
OL' ØL Own Owh
X,
(20)
Substituting the definition of L(w, ,1, E), we obtain:
ØL
Σ
plzz paas Σx)
Tik
(21)
Owh
Σラ Wj p(z, p1j. E,))
Wk
To solve for wk. we first multiply both sides of the equation by We and apply the constraint condition:
Σ
We
Tskk W/
0,
(22)
Upon further organization, we find that the Lagrange multiplier À actually equals the total number of data points (since Σ, Tsk = NR. where N& is the expected total number of data points belonging to the kth component, and the sum of all N₁ equals the total number of data points N).

---

Towards a Unified Framework of Clustering-based Anomaly Detection
Group Anomalies
Nearest Cluster
Generative Probability
Resultant Force
Figure 4: Score comparison with other methods.
Finally, can solve for wj:
Σ Tsk
(23)
N
This result indicates that the weight wi of each mixture component equals the proportion of the posterior probabilities of the data points it contains relative to all data points.
To update ss and Ex, we consider the conditional expectation of the data log-likelihood function:
N
Q( Ex)
Σ Tik
(log((T)
log |axl + log Usk
(24)
ki (z: Ex m16))
Maximizing (p4x, Στ) with respect to μ4 leads to:
aQ
Ta Uu (2>) pb -zi1 kik)
(25)
0pm
Setting Spg
aQ_ = 0 results in the updated mean M ((+1);
#k (++1)
ik
1)+ (++1)
uik
Σ
Th
(++1) (2+1)
Ugk
(26)
Considering the derivative (uu,, Σx)
with respect to Ex
aQ
N
Tik [Es - (2, - ms) x (z, - 7²)
(27)
k
2
Setting Byaz
aQ 0 yields the updated covariance matrix Σ
(1+1))
En
(++1) (1+1)
(z, 14
(z4
14/ ((+1))T
Tik
iik
(28)
Σラ= Tij
K
((+1)
C. Anomaly Score with Vector Sum
C.1. Toy Example
In the appendix, as illustrated in Figure 4, we investigated a toy example. We discussed a specific pattern of anomalies termed group anomalies, where small number of anomalous samples cluster together. It crucial to note that we do not claim this
12

---

Towards a Unified Framework of Clustering-based Anomaly Detection
anomaly pattern is common in real-world data; our goal is merely to point specific anomaly pattern that is challenging for traditional cluster-based anomaly detection methods to detect. Specifically, we utilize three Gaussian distributions with high variance (each generating 300 data samples) and one with lower variance (generating 30 data samples). Because the samples from the smaller Gaussian follow different generative mechanism and represent a minority in the dataset, we consider them anomalies.
We set the cluster number for KMeans--- and GMM at four, indicating that the Gaussian distribution comprising anomalous samples was also recognized as cluster. KMeans-- employs cluster-based approach, using the distance to the nearest cluster center as the anomaly score, while GMM uses probability-be the mixture model as the anomaly score. However, both approaches are ineffective in this scenario. Rather than identifying the small cluster as anomalous, they tend to misidentify samples on the peripheries of larger clusters anomalies.
approach, considering the samples' likelihood in
By contrast, our scoring method views the entire small cluster as more likely anomalous, followed by outlier samples on the margins of the larger clusters. This visualization provides a perspective that distinguishes our method from previous efforts.
D. Experimental Supplementary
D.1. Benchmark Datasets Details
Due to space constraints in the main text, we utilized 30 public datasets from ADBench (Han al., 2022), covering all different types of data. The details of the 30 datasets are presented in Table 4.
D.2. Baselines Details
A comprehensive overview of the unsupervised anomaly detection methods is presented below.
1.2.1. TRADITIONAL MODELS
Subspace Outlier Detection (SOD) (Kriegel et al., 2009): Identifies outliers in varying subspaces of a high-dimensional feature space, targeting anomalies that emerge in lower-dimensional projections.
Histogram-base -based Outlier Detection (HBOS) (Goldstein & Dengel, 2012): Assumes feature independence and calculates outlyingness via histograms, offering scalability and efficiency.
D.2.2. LINEAR MODELS
Principal Component Analysis (PCA) (Wold et al., 1987): Utilizes singular value decomposition for dimensionality reduction, with anomalies indicated by reconstruction errors.
One-class SVM (OCSVM) (Manevitz & Yousef, 2001): Defines a decision boundary to separate normal samples from outliers, maximizing the margin from the data origin.
D.2.3. DENSITY-BASED MODELS
Local Outlier Factor (LOF) (Breunig et al., 2000): Measures local density deviation, marking samples as outliers they lie in less dense regions compared their neighbors.
K-Nearest Neighbors (KNN) (Peterson, 2009): Anomaly scores are assigned based on the distance to the k-th nearest neighbor, embodying simple yet effective approach.
D.2.4. ENSEMBLE-BASED MODELS
Lightweight On-line Detector of Anomalies (LODA) (Pevný, 2016): An ensemble method suitable for real-time processing and adaptable to concept drift through random projections and histograms.
Isolation Forest (IForest) (Liu et al., 2008): Isolates anomalies by randomly selecting features and split values, leveraging the ease of isolating anomalies to identify them efficiently.
13

---

Towards a Unified Framework of Clustering-based Anomaly Detection
Table 4: Statistics of tabular benchmark datasets.
Data
# Samples
# Features
Anomaly
% Anomaly
Category
annthyroid
7200
6
534
7.42
Healthcare
backdoor
95329
196
2329
2.44
Network
breastw
683
9
239
34.99
Healthcare
campaign
41188
62
4640
11.27
Finance
celeba
202599
39
4547
2.24
Image
census
299285
500
18568
6.20
Sociology
glass
214
7
9
4.21
Forensic
Hepaitis
80
19
13
16.25
Healthcare
http
567498
3
2211
0.39
Web
Tonosphere
351
33
126
35.90
Oryctognosy
landsat
6435
36
1333
20.71
Astronautics
Lymphography
148
18
6
4.05
Healthcare
magic. gagic.gamma
19020
10
6688
35.16
Physical
mnist
7603
100
700
9.21
Image
musk
3062
166
97
3.17
Chemistry
pendigits
6870
16
156
2.27
Image
Pima
768
8
268
34.90
Healthcare
satellite
6435
36
2036
31.64
Astronautics
satimage-2
5803
36
71
1.22
Astronautics
shuttle
49097
9
3511
7.15
Astronautics
skin
245057
3
50859
20.75
Image
Stamps
340
9
31
9.12
Document
thyroid
3772
6
93
2.47
Healthcare
vertebral
240
6
30
12.50
Biology
vowels
1456
12
50
3.43
Linguistics
Waveform
3443
21
100
2.90
Physics
WBC
223
48
Healthcare
Wilt
4819
5
257
5.33
Botany
wine
129
13
10
7.75
Chemistry
WPBC
198
33
47
23.74
Healthcare
14

---

Towards a Unified Framework of Clustering-based Anomaly Detection
D.2.5. PROBABILITY-BASSE MODELS
Deep Autoencoding Gaussian Mixture Model (DAGMM) (Zong et al., 2018): Combines deep autoencoder with GMM for anomaly scoring, utilizing both low-dimensional representation and reconstruction enror.
Outlier Detection (ECOD) (Li et al., 2022): Uses ECDFs estimate feature densities independently, targeting outliers in distribution tails.
Copula Based Outlier Detector (COPOD) (Li et al., 2020): hyperparameter-frre method leveraging empirical copula models for interpretable and efficient outlier detection.
D.2.6. CLUSTER-BANE MODELS
DBSCAN (Ester et al., 1996): A density-based clustering algorithm that identifies clusters based the density of data points, effectively separating high-density clusters from low-density noise, and is widely used for anomaly detection in spatial data.
Clustering Based Local Outlier Factor (CBLOF) (He et al., 2003): Calculates anomaly scores based on cluster distances, using global data distribution.
KMeans-- (Song et al., 2021): Extends k-means to include outlier detection in the clustering process, offering an integrated approach anomaly detection.
Deep Clustering-based Fair Outlier Detection (DCFOD) (Chawla & Gionis, 2013): Enhances outlier detection with a focus on fairness, combining deep clustering and adversarial training for representation learning.
D.2.7. REPRESENTATION--ASER MODELS
Deep Support Vector Data Description (DeepSVDD) (Ruff al., 2018): Minimizes the volume of hypersphere enclosing network data representations, isolating anomalies outside this sphere.
Deep Isolation Forest for Anomaly Detection (DIF) (Xu al., 2023): Utilizes deep learning to enhance traditional isolation forest techniques, offering improved anomaly detection in complex datasets with minimal parameter tuning.
Each method's unique mechanism and application context provide rich landscape of techniques for unsupervised anomaly detection, illustrating the field's diverse methodologies and the breadth of approaches to tackling anomaly detection challenges.
D.3. Supplementary Experimental Results
In the appendix, we detail the statistical analysis conducted to compare the performance of various anomaly detectors. We obtained this diagram by conducting a Friedman test (p-value: 4.657e-19), indicating significant differences among different detectors. We utilized average ranks and the Nemenyi test to generate the critical difference diagram, as shown in Figure 5. is noteworthy that the vector version cxhibits significantly superior performance compared to the scalar version across more methods. The detailed outcomes for the AUCROC and AUCPR metrics, spanning 30 datasets and against 17 baseline approaches, showcased in Table 5 and Table 6.
D.3.1. DEGREES OF FREEDOM IN T- T-DIISRIBUTION
In fixed degrees of freedom scenarios, specifically when set to 1, the benefits of utilizing the t-distribution become less pronounced. Drawing from existing literature (Xie et al., 2016; Van Der Maaten, 2009), the flexibility to learn the degrees of freedom or to perform cross-vas coos-valldation on the validation set is particularly pertinent in unsupervised contexts. For the sake of simplicity and to minimize computational demands, we opted to maintain the degrees of freedom at 1, which provided robust performance while reducing complexity.
15

---

Towards a Unified Framework of Clustering-based Anomaly Detection
Table 5: AUCROC of 17 unsupervised algorithms on 30 tabular benchmark datasets. In each dataset, the algorithm with the highest AUCROC is marked in red, the second highest in blue, and the third highest in green.
Dataset
SOD
ILBOS
PCA
SVM
OC
LOF
KNN
LODA
IFurest
GMM
DCOD
COPOD
DB SCAN
CBLOF
DCOD
KMeame-
Deep SVDD
DIF
UniCAD (Scalar)
UniCAD
(Vector)
unnhhroodd
77.38
60:15
66.24
57.25
70.20
71.09
41.02
52.01
56.53
73,66
76.31)
50.08
62.25
55.01
64.99
76.09
66.76
75.27
72:72
backdoor
68.77
71.56
80.16
85.04
85.79
50.58
66.38
72.15
55.98
56.05
80:97
76.55
81.91
79:57
89.11
73.83
92.87
87.2
89.24
becsstw
93.97
95.13
80.30
98.
99.17
99.68
85:20
96.86
99:02
97.05
63.36
77.45
15.15
98.56
campaiga
69.16
78.55
72.75
65:70
39.04
72.27
51.67
71.71
76.10
77.69
50.50
64.34
63.16
63.51
67.53
73.52
73.64
celeto.
48.44
70.18
79.38
70.70
38.95
29.63
60.17
70.61
40.74
76.45
75.60
30.36
73.99
91.41
56.76
45.17
65.29
81.35
82.00
census
62.12
64.89
68.74
$4.90
47.46
66.85
37.14
59.3 52
59.65
67,83
69 07
58.50
60.17
72.84
63:33
54.16
59.00
67.00 11.90
67.84
glaes
73.
77.23
66.29
35 36
69
$2.25
75.13
77.
76.00
65.83
72.43
54.55
78, 50
78.07
77.30
55.71
64.57
79.52
62:17
Hapuritis
67.81
79:85
75.05
45.
74.24
8062
lutp
78.04
09.53
99.72
99.59
27.46
3.57
12,48
59 96-55
NIA
98.10
59 29
49.97
99.60
92.53
99.55
99.49
99.53
99.52
lonosphere
56.37
02. 29
79.19
75.92
90.59
$8.26
78.42
$4.50
73.41
T3.15
79.34
81.32
90.79
57.78
91.36
$3.94
K9.74
92.11
90.37
landsar
59.54
55.14
35.76
36.15
$3.90
$7.95
38.17
47.64
43.92
36.10
41.55
50.17
63.69
33.40
$5.31
62.48
54,84
49.60
57.37
mmphogaahy
71.22
$9,.00
99.82
99.56
89.80
55,91
65.55
99,85 61
72.11
99.52
99.80 18
74.16
99.81
81.19
100.00
71.91
83.67
99.29
99.75
muist
60:42
62.95
50.00
86.00
96.64
musk
74.00
100,00
100.00
58
41
89
95
99.
76.85
40
94
50/00
100,00
42:19
72.16
66.03
98.22
90.92
100,00
pendigns
66.29
93.04
91.73
93.75
47.99
72.95
89.10
94.76
64.22
93.01
90.00 50.68
55.33
96.93
94.35
94.37
27.32
93.79
95.12
95.52
Pima
61.25
71.07
70.77
66.92
65.71
73.43
65.93
72,87
$5,90
63.05
69. 10
$1.39
71.49
72.16
70.44
49.80
67.
75.
24.87
etellite
63.96
74,80
59.62
59.02
88
65.15 18
98
70:43
60.31
58.09
63.30
58.52
71.52
55.97
67.71
$7.40
74.52
72.46
77.65
satimuge-2
$3.08
97.65
96-25
97.21
75.74
86.01
99.87
99.88
shutde
69.51
98.63
97.
57.31
69.64
60.5
97,92
99.13
999
50.40
93.01
97.20
69.97
51.81
97.00
99.15
98.75
skia
60:35
60:15
45.25
29.45
46.47
71.46
45.75
68.21
NA
49,05
47.55
50.00
68.11
64.34
65.47
45,09
56.35
72.26
69.69
Stamps
73.26
90.73
91.47
K3.85
51.26
68.51
87.18
91.21
$8.59
$7.87
93.40
$2.08
09.89
93.41
79.78
59.48
$7.95
91.37
94.18
thyrid
92.81
95.62
96.34
67.92
$6.36
95,93
74.30
98.
79:75
97.94
96.30
53.57
94.74
78.55
92:26
52,14
96.25
97.66
97.48
38.13
47.97
woecs
92.65
72.21
65.29
61.59
03.12
07.26
70.36
73.94
60.58
62.24
53.15
57.50
92.12
51.56
93.45
40.87
81.02
53.
92.09
Wavelooe
68.57
66.77
65.45
56.29
73.32
73.78
60.13
71.47
49.35
62.36
75.03
66.41
71.27
63:47
74.35
53.94
75.35
71.81
74.29
WBC
94,60
96.72
98.20
99:05
$4.17
90.56
96.91
99.01
N6A
99.11
99.11
87.43
96.88
94.92
97.45
62.46
51.27
97.66
95,93
Wix
53.25
32.89
20.39
31.28
50.65
48.42
26.42
41.94
37.29
36.30
33.40
49.96
34.50
64.71
34.91
39.06
43.95
52.56
46.11
91.36
73.07
37.74
60:37
61,70
77.22
40.35
27.14
82, 18
27.26
WPBC
51.28
51.24
46.03
41.41
46.59
49.31
46.63
47.90
46.65
49.34
52.22
45.32
89.67
45.01
44.01
44.09
48.01
49.90
Avg. Rank
11.00
8.26
8,98
11.39
13.59
10.00
13:24
7.09
13.34
9.19
8.29
14:31
8.07
10.90
8.71
15.48
8.38
5.41
3.59
6
8
12
14
8
12
Galaxy (4.5)
(12) COF
CDPOD (7.3)
Galaxy (3.6)
GMM (6.6)
(6.8)
(1.4) DeepSVDD
(12) LOF
COF
DAGMM
IForest (7.5
CBLOF [ ()
PCA (7.5)
(12) DeepSVDD
(11) DAGMM
LOF
CBLOF (7
(12) LODA
HB05 (7. 7
(10) 50D
HB05 (7.6) 6)
(10) OCSVM
COPOD (7.5
(10) LODA
KMeanss (7.0 8)
(9.8) SOD
KMeans- (8.
(9.9) OCSVM
KNN (8.2)
(8.4) ECOD
GMM (8.2)
(B.6) ECDD
PCA (8.7)
KNN (8.3)
(a) AUC-ROC
(b) AUC-PR
Figure 5: Critical difference diagrams for AUC-ROC and AUC-PR.
D.3.2. ABLATION STUDY ON HYPERPARAMETER SETTINGS
An ablation study was conducted to evaluate the impact of hyperparameters k and l. A grid search was performed over various values of these hyperparameters across 30 datasets, benchmarking against 17 baseline methods. The comprehensive results, showeasing average ranks based on AUC-ROC, are summarized in the following table:
The findings indicate that the method exhibits robustness across specific parameter ranges. To ensure fair comparisons, a consistent parameter set (k+ = 10,1 = 1%) was applied, demonstrating strong performance across the majority of datasets.
Additionally, guidelines for selecting hyperparameters were examined. While techniques such as the elbow method and silhouette coefficient were considered for determining the optimal number of clusters, they proved to be time-consuming and exhibited weak correlation with anomaly detection performance. An ensemble learning approach, which involved random searches of values and aggregation of anomaly scores, showed promise in enhancing performance and model robustness for certain datasets. Future research will further explore this area.
D.4. Complexity Analysis
The complexity of each iteration in UniCAD involves three parts: constructing the outlier set, updating the network parameters O and optimizing the mixture model using the EM algorithm. Constructing the outlier set requires a sorting
16

---

Towards a Unified Framework of Clustering-based Anomaly Detection
Table 6: AUCPR of 17 unsupervised algorithms on 30 tabular benchmark datasets. In each dataset, the algorithm with the highest AUCPR is marked in red, the second highest in blue, and the third highest in green.
Datneet
SOD
HBOS
PCA
OC SVM
LOF
KNN
LOHA
IForest
DA GMM
ECOD
COPOD
DB SCAN
CBLOF
DCOD
KMeann--
Deep SYDD
DIF
UniCAD (Stalar)
UniCAD
(Vertor)
aonthyrood
18,84
16.99
16.12
10.37
15.71
16.74
7,05
20.47
9.64
25.35
16.55
7.60
13.74
10.01
15.41
21.25
18.93
26.37
25,13
backdoor
57.07
0.96
31.29
8.79
26.14
44.37
12.84
4.75
5:47
10.72
7.69
21.04
7,03
6.77
15:47
55.70
41.46
37.57
36.36
becosew
97.71
82.70
97.04
N/A
48.60
50.65
compaign
19:14
18:01
27.90 00
79.25
14.90
27.18
14.11
92.36
14.54
36.65
58.55
11.43
20.58
19661
18.96
16.75
76
27.55
27:12
calicto
2.36
13.52
15.89
10.78
1.75
3.14
4.04
8.96
1.95
13.96
13,89
2.32
11.22
17:48
3.19
2.73
5.44
15.12
14.66
census
8.54
8.68
10,02
6.82
5.48
9.04
5.03
7.78
9,03
9.46
9.92
7.52
7.52
10.92
8.13
8.42
7,42
9.70
9.75
10.05
20.26
13.57
24.58
14.06
8.46
18.
13.29
15.33
Hepodiis
24.73
37.73
36.85
29.44
13.67
21.95
30,90
22,90
41.50
22.31
19.53
30.04
34.93
36,08
43.37
hip
8.32
46.79
56.83
48.86
3.82
0.70
0.67
90.83
NIA
16.61
19
0.37
47.53
64.03
45.09
13.39
41.72
43.53
43.52
Toncophere
65.8 88.88
77
73.92
74.54
$8.07
90:41
73.04
50.41
64.97
64.09
89
63.04
89.77 77
47.63
91 36
43.24
87-45
50. 55
67.61
linden
26.36
22.03
16.15
16.21
24.60
34
15.86
19.81
24.48
16.24
17.45
20,80
31,05
15.57
22.40
36.92
24. 35
20.84
23.27
Lyuphogryyyy
22.00
44.54
12.34
34.58
96.66
munt
19:15
12.51
39.93
33.20
20,90
35.53
86
27.71
23.75
17:45
21.35
9:21
30,00
23.59
37.12
20L18
46.55
41.39
41.94
musk
7.59
100.00
99.89
10.61
2.82
9.65
47.60
59.51
32.76
50.13
34.79
3.16
100.00
2.87
37.55
8.78
70.70
97.53
99.96
pendigits
4.46
29:27
23.65
23.52
3.78
6.50
18.71
26,05
4.67
50.65
21.23
2.94
32,87
22.21
32,67
1.53
23.75
24.86
21.68
Pima
48,23
56.60
54.03
50.00
47.18
55. 14
44.09
$5.83
41.5
50.45
55.19
36.65
$2,99
50.24
$3.50
35.02
46 34.34
$4.66
54.25
satellite
47.23
67:25
59.64
37.68
50.01
61,94
58.33
37.56
68.92
71.68
75.13
sarimge-22
26 11
78:04
$3.89
82.71
4,30
39.16
50.52
95.45
22.07
54.
76.55
12.08
97.09
8.12
97.13
2.58
72.90
97.33
97.31
shurle
20.27
96:40
92.35
85.29
13.76
20
45.75
97.63
93:20
90.45
96, 56
7.68
79.80
81.82
32.66
12:41
67.23
92.05
92.36
skin
24.61
23 70
17.40
19.03
18.25
26.72
18.44
26.08
MIA
18.37
17.99
20.89
78.34
26.29
25.58
19.06
25. 36
28.57
28.72
Stamps
20.28
35.24
41.19
23.53
34.60
43.73
34.68
42.50
50.94
ibyoodd
23.56
50.99
44.34
21.23
20.81
34.98
14.08
16.06
51.06
19.64
9:48
29,58
10.56
31.69
2.70
50.36
64..99
60.16
vericrral
11.79
9.23
10,89
10.94
14.24
10.37
9.68
10.46
15:24
11.84
3.89
13.11
11,43
11.58
10.54
10.62
14.31
9.75
12.96
vowels
38.68
13:41
5.92
8:24
34,42
63:41
13.82
15.12
12:22
10,56
4.14
13.27
35.14
3.38
49.10
4.58
14.97
26.52
52.42
Winefform
9.66
5.56
5.79
4.37
11.33
13.04
4.71
6.24
3.11
4.76
6,90
5.33
17.91
4.26
19.74
4.41
11:28
6.49
7.8
WRC
73.56
66.55
$0.25
13:32
63.14
Wilt
5:53
3.54
3.13
3,62
5.05
4.73
3,36
4.23
4.00
3.69
5.38
3.74
4.62
3.76
4.65
4.05
4,80
5.19
wine
7.55
43.08
30.87
21.56
7.77
8.43
48.82
25.96
17.51
23.54
45.71
8.11
5.95
24.44
6.27
18.78
8.38
21.40
49.59
WPBC
25.62
23.04
23,01
22.95
20.29
21.49
25.39
22.42
22.49
21.24
22.83
23,86
21.08
22.86
20.58
25.00
20.73
22.71
24.90
Avg. Rank
10.83
8.19
8.31
11.14
13.34
9.36
11.79
7.29
11.96
9.36
9.53
14,91
8.53
11.97
9.03
13.41
9.10
6.31
4.74
Table 7: Learning vs. Fixed Degrees of Freedom
Metric
Learn
Fix v 1
AUC-ROC Avg. Rank
4.4 5.05
3.34 4.47
AUC-PR Avg. Rank
Table 8: Results of Hyperparameter Grid Search
A/
10
20
30
40
0.01
3.34
4.31
4.69
4.71
0.05
4.44
4.23
4.65
4.88
0.10
4.27
4.46
4.48
4.88
operation, for which we use Numpy's built-in quantile calculation with time complexity of O(N log N). Considering the number of network parameters along with the computation of the loss function, the computational complexity for optimizing θ is approximately O(TN Dd + Kd)... The EM algorithm for the Student's t mixture model includes two main steps: the E-step, where the complexity for computing the probability (or responsibility) of each data point belonging to each component is approximately ONN Kd), and the M-step, where the full computational complexity of updating the parameters (mean, covariance matrix) of each component is O(N K Kd²). In practice, we use diagonal covariance matrices, which reduces the update complexity to roughly (NNdd) If the EM algorithm requires T round to converge, its time complexity is approximately O(TN Therefore, the time complexity for t-iterations is OtNN(log N + +Td( D + K))).
17


Reviews:
[{"review_id": "1pr7mk8Rco", "reviewer": "Reviewer_GPqn", "summary": "The paper proposes UniCAD, a novel model for Unsupervised Anomaly Detection (UAD) that unifies representation learning, clustering, and anomaly detection within a single theoretical framework. By leveraging a mixture model with the Student-t distribution, UniCAD introduces an anomaly-aware data likelihood that enhances the joint optimization process, reducing the influence of anomalies on both representation learning and clustering. Additionally, the model formulates a theoretically grounded anomaly score inspired by universal gravitation, which captures complex relationships between data points and multiple clusters. Extensive experiments across 30 datasets from various domains demonstrate UniCAD’s superior performance and generalization ability, surpassing 15 baseline methods and establishing it as a state-of-the-art approach in unsupervised anomaly detection.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Experimental settings are not fair \n> Default parameters for baseline methods were used, which raises concerns. All methods have to re-run and be re-optimized for the datasets, similar to what you do for your solution\n\nThank you for highlighting the concern regarding the fairness of our experimental settings. In the paper, we also used the default parameters for our method, in line with the experimental setup used for the baseline methods. Specifically, we followed the ADBench benchmark [Han et al., 2022] to ensure a fair and consistent comparison. We acknowledge that tuning parameters could potentially improve performance, but as this is an unsupervised task, it is difficult to rely on techniques such as cross-validation for hyperparameter optimization. Therefore, we opted to maintain the same default settings across all methods to ensure fairness in the comparison.\n\n> The solution seems novel but several of its components are based on existing ideas. Clarifying novelty vs. only citing earlier work can help strengthen the contribution\n\nWe appreciate your feedback on the novelty of UniCAD. While components like mixture models and the Student’s t-distribution draw from existing ideas, the core contribution of UniCAD lies in its unified theoretical framework that integrates representation learning, clustering, and anomaly detection via anomaly-aware maximum likelihood estimation, alongside a novel gravity-inspired anomaly scoring mechanism. Specifically: (1) the joint optimization of these tasks with theoretical grounding, (2) the gravity-inspired vector-sum-based anomaly scoring capturing complex sample-cluster relationships, and (3) an efficient iterative optimization strategy are unique to our approach. To clarify this, we will revise the paper to explicitly distinguish between foundational concepts and our original contributions, supported by comparative experiments and theoretical analysis to underscore UniCAD’s distinctiveness.\n\n> Substantial runtime overhead\n\nWe acknowledge the concern about runtime overhead. In the paper (Table 2), we provided a runtime comparison showing that UniCAD is competitive with deep learning baselines like DAGMM and DCOD, despite the iterative EM algorithm. The efficient iterative strategy and parameter updates reduce computational overhead, with a complexity of $ \\mathcal{O}(tN(\\log N + Td(D + K))) $, making it feasible for large-scale datasets.", "claims_and_evidence": "The experiments overall support the claims. There are issues with parameter settings though", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Skimmed over the math derivations to check correctness", "experimental_designs_or_analyses": "There are issues with the experimental settings", "relation_to_broader_scientific_literature": "Good coverage", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n\n- Unified framework for anomaly detection that integrations representation learning and clustering\n- New anomaly score measure based on physics-inspired laws\n- Multiple datasets and methods used for experimentally demonstrating the potential of the idea\n\nWeaknesses:\n\n- Experimental settings are not fair\n\nDefault parameters for baseline methods were used, which raises concerns. All methods have to re-run and be re-optimized for the datasets, similar to what you do for your solution\n\n- The solution seems novel but several of its components are based on existing ideas. \n\nClarifying novelty vs. only citing earlier work can help strengthen the contribution\n\n- Substantial runtime overhead", "other_comments_or_suggestions": "N/A", "questions_for_authors": "see comments above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MSjdhkHMS9", "reviewer": "Reviewer_2n8B", "summary": "The paper presents UniCAD, a novel unsupervised anomaly detection framework that integrates representation learning, clustering, and anomaly detection into a unified theoretical framework. By maximizing an anomaly-aware data likelihood based on a mixture model with the Student-t distribution, UniCAD effectively mitigates the impact of anomalies on representation learning and clustering. The framework derives a theoretically grounded anomaly score inspired by universal gravitation, which considers the complex relationships between samples and multiple clusters. Extensive experiments on 30 datasets across various domains demonstrate the effectiveness and generalization capability of UniCAD, outperforming 17 baseline methods and establishing it as a state-of-the-art solution for unsupervised anomaly detection.\n\n## update after rebuttal\n\nThe authors have responded to most of the concerns in a clear and constructive manner. In particular, their clarifications on the theoretical differences between UniCAD and traditional GMM-EM models are well explained. Their justification for the vector-based anomaly scoring method is reasonable, and they have acknowledged the limitations of the gravitational analogy, with a plan to revise it accordingly.\n﻿\nHowever, I remain concerned about the limited addition of recent baseline methods. Although the authors incorporated NeuTraLAD (2024) and DROCC (2020), there is still a lack of advanced baseline methods from recent years, which weakens the claim of state-of-the-art performance. I encourage the authors to further improve the comprehensiveness of their baseline selection in the final version.\n﻿\nOverall, while the core idea and contributions are interesting and mostly well-supported, the incomplete experimental comparison affects the overall strength of the work. I maintain my final rating as “4: Accept”.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The paper lacks comparisons with the latest baseline methods. Among the 17 baseline methods included, only 3 are from after 2020, and there are no methods from 2024.\n\nThank you for noting the need for updated baselines. Given the rapid advancements in unsupervised anomaly detection, we have added recent methods, including NeuTraLAD (2024) and DROCC (2020), to our revised experiments. These methods are evaluated on the same datasets and metrics as UniCAD to ensure a fair and comprehensive comparison. The results are available at the following anonymous link: https://anonymous.4open.science/r/ICML2025_9292-4E31\n\n> Although the paper proposes an efficient iterative optimization strategy, the computational complexity of UniCAD remains high, especially for large-scale datasets. The iterative nature of the EM algorithm and the processing of high-dimensional data may lead to long training times, limiting its applicability in real-time scenarios.\n\nWe appreciate this observation. The computational complexity of UniCAD, detailed in Section 3.4 as $ \\mathcal{O}(tN(\\log N + Td(D + K))) $, is manageable with reasonable settings for iteration count $ t $ and cluster number $ K $. Table 2 demonstrates that UniCAD’s training and inference times are competitive with deep learning methods like DAGMM and DCOD. To address concerns about large-scale and real-time scenarios, we will provide additional runtime results on larger datasets and explore optimization techniques (e.g., parallelization or approximation) to further reduce training time, enhancing its practical utility.\n\n> Two minor issues. In 4.3, 'with additional experiments on other data domains presented in Appendix ??' In 3. Methodology, 'Finally, we present an efficient iterative optimization strategy to optimize this model and provide a complexity analysis for the proposed model.' However, I fail to find complexity analysis in the text.\n\nThank you for pointing out these issues. For the appendix reference in Section 4.3, we will ensure the additional experiments are fully detailed in the appendix and correct the placeholder “??”. Regarding the complexity analysis, it is provided in Appendix D.4; we will revise Section 3 to explicitly reference this section for clarity.\n\n> Please explain the difference between the proposed method and the typical learning of Gaussian Mixture Models (GMMs) with EM?\n\nUniCAD distinguishes itself from traditional GMM-EM through several key innovations.  Unlike GMM-EM which operates directly on raw features, UniCAD incorporates deep representation learning via neural networks to project data into a lower-dimensional space.  It introduces an anomaly indicator function $ δ(x_i) $ to dynamically filter out outliers during optimization, overcoming GMM-EM's assumption of normal-only samples.  For scoring, UniCAD replaces conventional likelihood probabilities with a novel gravity-inspired vector-sum approach that better captures complex sample-cluster relationships.  Additionally, it adopts Student's t-distribution mixture models instead of Gaussian distributions for improved outlier robustness.\n\n> Why is Anomaly Scoring with Vector Sum necessarily beneficial? Although the authors provide an explanation in Section 3.2.3, they do not discuss scenarios such as the following: If a sample point belongs to a normal cluster that happens to be centrally located in the feature distribution, could it inadvertently receive a high anomaly score?\n\nThe vector-sum anomaly scoring is beneficial because it comprehensively assesses a sample’s relationship with all clusters via directional “gravitational” forces, unlike scalar methods that consider only magnitude. For a sample in a normal cluster’s center, its strong “force” from that cluster dominates, yielding a large resultant force and thus a low anomaly score. In contrast, anomalies, weakly influenced by all clusters or pulled between them, have a smaller resultant force, increasing their score. \n\n> It is recommended to carefully reconsider the use of gravitation as an introductory concept in Section 3.2. In Section 3.2.1, the analogy between the equation of universal gravitation and the anomaly scoring formula appears somewhat forced, as their similarity is essentially limited to both being expressed as fractions. In Section 3.2.2, the connection is further reduced to merely involving vector summation. While the gravitational analogy provides an intuitive perspective, its relevance to the proposed method is not sufficiently substantiated.\n\nWe appreciate this critique and will revise Section 3.2 to de-emphasize the gravitational analogy as a direct equivalence. Instead, we will frame it as an intuitive inspiration, focusing on how vector-sum scoring models sample-cluster interactions as “forces” to capture complex relationships. We will provide detailed explanations and visualizations to substantiate its relevance, ensuring the method’s merit stands independent of the analogy.", "claims_and_evidence": "Yes. The claims are supported by both theoretical derivation and experimental results.", "methods_and_evaluation_criteria": "Yes, the evaluation criteria are reasonable to me.", "theoretical_claims": "Yes, I have checked the theoretical claims. Most are correct and I suggest the authors make the Equ 5 more clear where ‘the l lowest’ should be more accurately described.", "experimental_designs_or_analyses": "Yes, the experiments are extensive where over 30 datasets are evaluated, and the results are objective.", "relation_to_broader_scientific_literature": "I think both clustering and anomaly detection are fundamental task in scientific discovery. Although the clustering and anomaly detection have been widely discussed together, including the representation in the unified frameworks looks interesting and reasonable. But I suggest the authors provide broader type of datasets (such as images) for scientific usage.", "essential_references_not_discussed": "I do not have such suggestions.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper proposes a unified framework, UniCAD, which integrates representation learning, clustering, and anomaly detection by maximizing an anomaly-aware data likelihood. This unified approach helps to better understand the interdependencies among these tasks and allows them to mutually enhance each other, leading to improved anomaly detection performance.\n\n2. The paper not only introduces an anomaly score based on generation probability but also further proposes a vector summation-based anomaly scoring method inspired by the theory of universal gravitation. This method effectively captures the complex relationships between samples and multiple clusters, enabling more accurate anomaly detection.\n\n3. The paper conducts extensive experiments on 30 datasets from diverse domains, demonstrating the effectiveness and generalization capability of UniCAD. The results show that UniCAD outperforms 17 baseline methods on most datasets, establishing it as a state-of-the-art solution for unsupervised anomaly detection.\n\nWeaknesses:\n1. The paper lacks comparisons with the latest baseline methods. Among the 17 baseline methods included, only 3 are from after 2020, and there are no methods from 2024.\n\n2. Although the paper proposes an efficient iterative optimization strategy, the computational complexity of UniCAD remains high, especially for large-scale datasets. The iterative nature of the EM algorithm and the processing of high-dimensional data may lead to long training times, limiting its applicability in real-time scenarios.\n\n3. Two minor issues. In 4.3, \"with additional experiments on other data domains presented in Appendix ??\" In 3. Methodology, \"Finally, we present an efficient iterative optimization strategy to optimize this model and provide a complexity analysis for the proposed model.\" However, I fail to find complexity analysis in the text.", "other_comments_or_suggestions": "I think the symbols should be correctly used. For example, the vectors are represented by both bold symbols and symbols with arrows. I think this may cause misunderstanding.", "questions_for_authors": "1. Please explain the difference between the proposed method and the typical learning of Gaussian Mixture Models (GMMs) with EM?\n\n2. Why is Anomaly Scoring with Vector Sum necessarily beneficial? Although the authors provide an explanation in Section 3.2.3, they do not discuss scenarios such as the following: If a sample point belongs to a normal cluster that happens to be centrally located in the feature distribution, could it inadvertently receive a high anomaly score?\n\n3. It is recommended to carefully reconsider the use of gravitation as an introductory concept in Section 3.2. In Section 3.2.1, the analogy between the equation of universal gravitation and the anomaly scoring formula appears somewhat forced, as their similarity is essentially limited to both being expressed as fractions. In Section 3.2.2, the connection is further reduced to merely involving vector summation. While the gravitational analogy provides an intuitive perspective, its relevance to the proposed method is not sufficiently substantiated.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1h6taCZ2j4", "reviewer": "Reviewer_GMxn", "summary": "The paper presents an unsupervised anomaly detection framework with two key components: (i) a framework for joint learning of representation, clustering, and mixture models; (ii) a gravity-inspired anomaly scoring function based on mixture model outputs. Experiments on 30 datasets show it outperforms 17 baselines.\n\n## update after rebuttal\nFrom the response, I acknowledge that the gravity-inspired scoring function does have a certain degree of independent contribution to improving model performance. However, regarding the two key issues I raised—the concrete real-world application scenarios and the applicable scope of the scoring function — the authors have only promised to address them in future revisions, without providing specific experiments or supporting data in the current manuscript. Therefore, based on the present version, I will maintain my original score of 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> However, the paper does not elaborate on the specific scenarios in real datasets that this method can address. Although this scoring method theoretically holds advantages, further clarification is needed regarding its applicable scope.\n\nWe thank the reviewer for highlighting the need for more detail on specific application scenarios.   We acknowledge that the original manuscript lacked sufficient elaboration on real-world use cases.   To address this, we will revise the paper by adding a new subsection that discusses the method’s applications in diverse anomaly detection tasks.   Specifically, we will include examples such as financial fraud detection (e.g., identifying irregular transaction patterns), cybersecurity intrusion detection (e.g., detecting anomalous network traffic), and medical diagnostics (e.g., flagging unusual patient records).   These examples will be supported by experimental results to demonstrate the method’s performance advantages in these domains.\n\n> On the experimental specifics of the gravity-inspired scoring function: The paper states this function can more comprehensively measure sample-to-cluster relationships but does not clearly define its scope of application. What types of anomalies does this method excel at detecting? Can the authors provide concrete examples of its improved performance on different anomalies?\n\nWe appreciate the reviewer’s request for clarification on the gravity-inspired scoring function’s scope.   This method excels at detecting anomalies in datasets with complex, multi-cluster structures, particularly where anomalies lie at cluster boundaries or exhibit ambiguous cluster affiliations. We will add synthetic and real-data experiments showcasing improved detection of these anomaly types compared to traditional methods.\n\n> On verifying the independent contribution of the gravity-inspired scoring function: The paper claims this function significantly boosts overall performance, yet lacks in-depth validation of its individual contribution. Could ablation studies be conducted to analyze its impact? For instance, comparing model performance with and without this function under identical experimental conditions would clarify its specific effects.\n\nThank you for your valuable feedback. We would like to clarify that in Table 1, we have already compared the two anomaly scoring methods and treated them as two separate variants of the model. We apologize for not clearly indicating this distinction in the ablation section. We will revise the paper to highlight this and improve the explanation, ensuring that the independent contribution of the gravity-inspired scoring function is more clearly addressed.", "claims_and_evidence": "In Section 3.2 of the paper, a novel gravity-inspired anomaly scoring method is introduced, which takes into account not only the distance from a sample to each cluster center but also the directional information. The authors assert that by synthesizing vectors, this method can more comprehensively measure the relationship between a sample and multiple cluster centers, thereby aiding in capturing complex relationships within clusters. However, the paper does not elaborate on the specific scenarios in real datasets that this method can address. Although this scoring method theoretically holds advantages, further clarification is needed regarding its applicable scope.", "methods_and_evaluation_criteria": "The paper's method jointly models representation learning, clustering, and anomaly detection, incorporating a gravity-inspired scoring function for parameter optimization. This approach theoretically enhances anomaly detection performance. The model was evaluated using AUC-ROC and AUC-PR metrics and tested on 30 datasets against 17 baselines, demonstrating strong generalization.", "theoretical_claims": "The paper's theoretical claims are sound. I reviewed the construction of the gravity-inspired anomaly scoring formula in section 3.2 and the detailed EM algorithm derivation in Appendix B, and found no issues.", "experimental_designs_or_analyses": "The experimental design is generally sound. The authors conducted extensive experiments across 30 diverse datasets and compared them with 17 baseline methods, effectively validating the framework's superiority. However, the independent evaluation of the gravity-inspired scoring function in section 3.2 is inadequate. Its specific impact on overall performance hasn't been separately verified, making it hard to gauge the method's actual contribution. It is recommended to supplement with relevant ablation experiments for a more precise assessment of its effectiveness.", "relation_to_broader_scientific_literature": "Previously, anomaly detection and clustering were separate research paradigms. This paper presents a novel joint modeling framework for unsupervised anomaly detection, integrating representation learning, clustering, and anomaly detection to significantly enhance current methods. Based on a mixture model, this framework optimizes parameters by maximizing anomaly-aware data likelihood, reducing the impact of anomalous data on model training and improving detection performance.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "The gravity-inspired anomaly scoring method in the paper is notably innovative, offering a fresh perspective and solution for anomaly detection. This interdisciplinary approach provides new ways to tackle anomaly detection in complex data distributions and high-dimensional data.", "other_comments_or_suggestions": "No more.", "questions_for_authors": "1. On the experimental specifics of the gravity - inspired scoring function: The paper states this function can more comprehensively measure sample - to - cluster relationships but does not clearly define its scope of application. What types of anomalies does this method excel at detecting? Can the authors provide concrete examples of its improved performance on different anomalies?\n2. On verifying the independent contribution of the gravity-inspired scoring function: The paper claims this function significantly boosts overall performance, yet lacks in-depth validation of its individual contribution. Could ablation studies be conducted to analyze its impact? For instance, comparing model performance with and without this function under identical experimental conditions would clarify its specific effects.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No"}}, {"review_id": "cMkgAuCWo6", "reviewer": "Reviewer_jpXF", "summary": "This paper introduces UniCAD, a novel method for anomaly detection based on clustering and representation learning. The method leverages an anomaly indicator function and a mixture of student’s t-distributions to learn an anomaly score based on the mixture distribution density. Inspired by Newton’s law of universal gravitation, the authors further refine the anomaly score by adding positional information about a sample representation. In experiments, they demonstrate the efficiency of their method by comparing their approach against various baselines on 30 different datasets.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> It is unclear why the student’s t-distribution should work fundamentally better than a Gaussian mixture model. \n\nThe choice of the Student’s t-distribution over a Gaussian mixture model (GMM) is motivated by its heavy-tailed nature, which enhances robustness to outliers—a critical factor in anomaly detection. Unlike GMMs, where Gaussian tails decay rapidly and outliers can skew parameter estimates, the Student’s t-distribution mitigates this by assigning lower influence to extreme values. Our ablation study’s empirical evidence (Table 3) shows that performance significantly decreases when switching to a GMM.\n\n> The explanation of why vector information fundamentally improves the anomaly score is vague and should be elaborated better. ...\n\nThank you for your feedback. Due to space limitations, we provided a detailed explanation of the benefits of our scoring design in Appendix C and illustrated its differences from traditional clustering scores in Figure 4. We will emphasize these advantages more clearly in the next version to better highlight the rationale behind our approach.\n\n> Since the authors test their methods on tabular data only, they should compare to more recent works on this problem. ...\n\nThank you for introducing these excellent research works. Following your suggestion, we have properly cited these articles in our revised manuscript. Additionally, we have incorporated NeuTraLAD and DROCC as comparison baselines and reported the results in the following anonymous link: https://anonymous.4open.science/r/ICML2025_9292-4E31. \n\n> The method seems very overengineered, and many parts of the pipeline lack a clear motivation or intuition other than working better on the test set in the experiments.\n\nThank you for your input. We hope to clarify your misunderstanding regarding the design of our method. Far from lacking clear motivation, every module in our pipeline is purposefully designed, rooted in a unified optimization objective and theoretical framework. Our approach’s strength is validated through rigorous, fair comparisons across 30 diverse datasets spanning multiple domains, where it consistently achieves superior average performance, highlighting its robustness and broad applicability.\n\n> Other Comments Or Suggestions\n\nThank you for your detailed feedback. We will improve the clarity of Figure 2’s caption, define \"MM\" in Section 3.1.1, fix the broken reference in Section 4.3, and add an impact statement in the revision.\n\n> Could the authors provide a better intuition or motivation for adding directional information to the anomaly score? ...\n\nDirectional information enhances anomaly scoring by modeling samples as influenced by multiple cluster “forces,” identifying anomalies with weak or conflicting resultant forces.   It may hurt in datasets with irregular or overlapping clusters, where directionality could misclassify points.   For normal samples within clusters, since the \"force\" exerted by the center of their own cluster will be much greater than that of other clusters, the direction information of other cluster centers will be avoided from influencing the assessment of the abnormality degree of the samples.\n\n> Is there a particular reason why the authors learn representations using an autoencoder? ...\n\nThank you for your comment.  Autoencoders are widely used as a foundational framework for anomaly detection due to their ability to learn low-dimensional representations via reconstruction loss.  We appreciate your suggestion and agree that exploring alternative representation learning objectives holds potential for future work.  However, the focus of our approach is on anomaly-aware maximum likelihood optimization, which better guides representation learning.  Our ablation study (Table 3) demonstrates that performance significantly drops when only the autoencoder objective is used.\n\n> How did authors tune hyperparameters?\n\nWe used fixed hyperparameters ($ K=10 $, $ l=1\\% $) across all datasets for a fair comparison with baselines, which used default settings. These were effective on average, but tuning could improve results, as shown in the sensitivity analysis in Section 4.4.\n\n> Is there a clear intuition as to why student’s t-distribution mixture models work better than Gaussian mixture models?\n\nYes, the t-distribution’s heavy tails (Equation 3) make it robust to outliers, unlike GMMs’ exponential tails. This enhances clustering and scoring in anomaly detection, as shown in Table 3 (AUC-ROC drops with GMM). \n\n> Under which conditions does it make sense to view anomaly detection from a clustering perspective? ...\n\nThank you for your question. A clustering perspective is advantageous for anomaly detection when the data has natural groupings or multiple modes, allowing anomalies to be identified as deviations from these structures. However, in cases where clusters are absent or irrelevant to anomalies, density-based or distance-based methods may be more effective.", "claims_and_evidence": "This paper contains many design choices that are not very clearly motivated.\n- It is unclear why the student’s t-distribution should work fundamentally better than a Gaussian mixture model. The authors should elaborate on this point.\n- How to learn a good representation is only briefly mentioned. The authors use an autoencoder-based approach. Why is this a good choice, and how does it compare to other self-supervised representation learning approaches?\n- Drawing a parallel to Newton’s law of universal gravitation seems far-fetched, and the authors seem to just randomly define terms to make their formula fit the definition of Newton’s gravity.\n- The explanation of why vector information fundamentally improves the anomaly score is vague and should be elaborated better. Figure 2 is hard to understand and is not convincing.", "methods_and_evaluation_criteria": "Approaching the anomaly detection problem from a clustering perspective makes sense, and the experimental setup is reasonable. However, experiments are limited to tabular datasets, and baselines do not include recent advances in the field.", "theoretical_claims": "The paper does not contain any theoretical claims.", "experimental_designs_or_analyses": "The experiments are based on ADBench and seem to be sound and valid.", "relation_to_broader_scientific_literature": "The paper explores anomaly detection from a clustering perspective. This is an interesting perspective, and there are not too many prior works that explore the problem in a similar fashion.", "essential_references_not_discussed": "Since the authors test their methods on tabular data only, they should compare to more recent works on this problem. For instance, see [1], [2], and [3], which include more recent anomaly detection methods for tabular data. \n\n[1] Qiu, Chen, et al. \"Self-Supervised Anomaly Detection with Neural Transformations.\" IEEE Transactions on Pattern Analysis and Machine Intelligence (2024).\n\n[2] Li, Aodong, et al. \"Anomaly detection of tabular data using llms.\" arXiv preprint arXiv:2406.16308 (2024).\n\n[3] Qiu, Chen, et al. \"Neural transformation learning for deep anomaly detection beyond images.\" International conference on machine learning. PMLR, 2021.", "other_strengths_and_weaknesses": "### Strengths\n- This paper tackles anomaly detection from a clustering perspective, an interesting and novel angle for the problem.\n- I like the ablations in the experiment that demonstrate the contributions of specific parts of their method.\n### Weaknesses\n- The method seems very overengineered, and many parts of the pipeline lack a clear motivation or intuition other than working better on the test set in the experiments.\n- Experiments are restricted to tabular data only but do not include recent advances in tabular anomaly detection.", "other_comments_or_suggestions": "- Figure 2 is unintuitive and would benefit from a more extensive description in the caption.\n- In section 3.1.1, the authors use the abbreviation MM, which is not defined.\n- There is a broken reference at the beginning of section 4.3.\n- The impact statement is missing.", "questions_for_authors": "- Could the authors provide a better intuition or motivation for adding directional information to the anomaly score? In which scenarios do we expect this to help? Are there examples where it could hurt to add this information? How does directional information influence normal samples, which are expected to lay inside a cluster?\n- Is there a particular reason why the authors learn representations using an autoencoder? Why would you expect this to work better than more recent self-supervised approaches, such as masking or contrastive approaches?\n- How did authors tune hyperparameters?\n- Is there a clear intuition as to why student’s t-distribution mixture models work better than Gaussian mixture models?\n- Under which conditions does it make sense to view anomaly detection from a clustering perspective? Are there scenarios where tackling anomaly detection without clustering would be better? An experiment on datasets where we know about the existence of multiple modes would benefit this paper, as we could compare the chosen hyperparameter $k$ with the actual number of clusters in the datasets and evaluate its influence. E.g., the authors could take a multi-class dataset and define a set number of classes as normal whereas the rest is considered anomalous.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper received one accept, two weak accepts, and one weak reject. All reviewers agree that the combined use of clustering and representation learning for anomaly detection is a valid and meaningful approach. Initial concerns were raised regarding the use of a t-distribution in place of a Gaussian, the adoption of an autoencoder architecture, comparisons with outdated baselines, and the overall fairness of the experimental evaluation. While the authors' rebuttal appears to have satisfactorily addressed most of these concerns, some reservations persist—particularly regarding the use of autoencoders and the fairness of the evaluation protocol. Nevertheless, given that the majority of reviewers lean toward acceptance, I recommend the acceptance of the paper.
Decision: Accept (poster)