DRL: Decomposed Representation Learning for Tabular Anomaly Detection
Authors: Hangting Ye
He Zhao
Wei Fan
Mingyuan Zhou
Dan dan Guo
Yi Chang
Venue: ICLR 2025 Poster (2025)
Keywords: Anomaly detection
Tabular data
Tabular representation learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose to re-map data into a tailor-designed constrained space, in order to capture the underlying shared patterns of normal samples and differ anomalous patterns for tabular anomaly detection.

Abstract: Anomaly detection, indicating to identify the anomalies that significantly deviate from the majority normal instances of data, has been an important role in machine learning and related applications. Despite the significant success achieved in anomaly detection on image and text data, the accurate Tabular Anomaly Detection (TAD) has still been hindered due to the lack of clear prior semantic information in the tabular data. Most state-of-the-art TAD studies are along the line of reconstruction, which first reconstruct training data and then use reconstruction errors to decide anomalies; however, reconstruction on training data can still hardly distinguish anomalies due to the data entanglement in their representations. To address this problem, in this paper, we propose a novel approach Decomposed Representation Learning (DRL), to re-map data into a tailor-designed constrained space, in order to capture the underlying shared patterns of normal samples and differ anomalous patterns for TAD.
Specifically, we enforce the representation of each normal sample in the latent space to be decomposed into a weighted linear combination of randomly generated orthogonal basis vectors, where these basis vectors are both data-free and training-free.
Furthermore, we enhance the discriminative capability between normal and anomalous patterns in the latent space by introducing a novel constraint that amplifies the discrepancy between these two categories, supported by theoretical analysis. 
Finally, extensive experiments on 40 tabular datasets and 16 competing tabular anomaly detection algorithms show that our method achieves state-of-the-art performance.

Published as a conference paper at ICLR 2025
DRL: DECOMPOSED REPRESENTATION LEARNING FOR TABULAR ANOMALY DETECTION
Hangting Ye', He Zhao", Wei Fan', Mingyuan Zhou', Dandan Guo",, Yi Chang' 5. School of Artificial Intelligence, Jilin University' CSIRO's Data61² University of Oxford The University of Texas at Austin' International Center of Future Science, Jilin University Engineering Research Center of Knowledge- Driven Hauaan-Machine Intelligence, MOE, China yeht aemammlll ilu. edu. cn, he zhao@ csiros au, wei. fan@wrh. ox ac uk, mingyuan. zhou@mccombs utexas edu, (guodandan, yichang) @ jlu. edu.
ABSTRACT
Anomaly detection, indicating to identify the anomalies that significantly deviate from the majority normal instances of data, has been important role in ma- chine learning and related applications. Despite the significant success achieved in anomaly detection on image and text data, the accurate Tabular Anomaly De- tection (TAD) has still been hindered due to the lack clear prior structure infor- mation in the tabular data. Most state-of of-thhe-rtt TAD studies are along the line of reconstruction, which first reconstruct training data and then use reconstruction er- to decide anomalies; however, reconstruction training data can still hardly distinguish anomalies due to the data entanglement. To address this problem, this paper, we propose a novel approach Decomposed Representation Learning (DRL), to re-map data into a tailor-designed constrained space, in order to capture the underlying shared patterns of normal samples and differ anomalous patterns for TAD. Specifically, we enforce the representation of each normal sample in the latent space to be decomposed into weighted linear combination of randomly generated orthogonal basis vectors, where these basis vectors are both data-free and training-free free. Furthermore, we enhance the discriminative capability between normal and anomalous patterns in the latent space by introducing novel con- straint that amplifies the discrepancy between these two categories, supported by theoretical analysis. Finally, extensive experiments 40 tabular datasets and 16 competing tabular anomaly detection algorithms show that our method achieves state-of-the-ane performance.
INTRODUCTION
Anomaly detection (AD), which aims to identify the anomalies that significantly deviate from the majority normal instances, is crucial machine learning task. Tabular data, usually represented as
vectors of hetcrogeneous features, is a vital data type AD (Han et al. 2022] Yin et al. 2024) with
numerous applications, including cyber-security (Ahmad al. 2021). nando et al., [202T] Ye et al. 2023a) and financial fraud detection (Al-Hashedi & Magalingam] (021)
disease diagnosis (Fer-
In most real-world scenarios, labeled anomalies usually need to be manually annotated by domain experts, which is both expensive and time-consuming, and marking all types of anomalies is usually impractical (Chandola al. 2009 Ye et al. 2023b Guo al. 2023). Therefore, tabular anomaly detection (TAD) is usually implemented in a ole-class classification setting, where model is trained identify anomalies the absence of labeled anomalous samples, relying solely on the normal in- stances available during training (Schölkopf et al., 1999. [Sohn et al., 2021 Yin et al. (024). This approach fundamentally differs from supervised learning tasks, which access to examples from all data classes.
Existing approaches in this literature leverage the assumption that anomalies deviate from character- istic patterns present in the normal training data. By capturing these normal patterns during training, one can identify anomalies at the inference time. When considering perceptual data (e.g., image
"Corresponding authors.

---

Published as a conference paper at ICLR 2025
and text), many methods (Golan & El- Yaniv 2018 Liznerski et al., 2020) have demonstrated sig- nificant success by leveraging the structure of the input data. For example, images can be rotated, and the ability to distinguish between different rotations varies between anomalies and normal sam- ples (Golan & Yaniy 2018). However, unlike image or text, tabular data features can have values with different types (numerical and categorical), ranges and distributions. Thus, there is prior information on the structure of tabular data (Borisov et al. [022] Shenkar & Wolf 2022] Chang al. 2023; Thimonier et al. 2024; Ye et al., 2024). One of the general ideas for TAD is to train deep learning models that reconstruct training data and use reconstruction error to decide whether a test datum normal or anomalous following the assumption that a model can accurately reconstruct
normal samples while failing to reconstruct anomalous ones. Methods (Yin et al. 2024 Thimonier et al. 2024) built on top of this idea have shown promising performance.
Although fruitful progress has been made in the last several years, capturing the comprehensive nor- mal patterns that are distinet from anomalous patterns for tabular data remains a challenging task, as real-world data may exhibit data entanglement between normal and anomalous samples. This paper develops DRL, a decomposed representation learning framework for TAD that learns the underlying normal patterns shared among normal samples by re-mapping observations into tailor-designed constrained latent space, where normal and anomalous patterns are more effectively distinguished. To learn the shared information that distinguishes normal samples from anomalies, normal sample's s representation in the latent space to be decomposed into a linear combination of randomly generated orthogonal basis vectors with sample-species vectors are fixed during training and are data-free. In addition, to enhance the discriminative power between normal and anomalous patterns within the latent space, we introduce a novel constraint that amplifies the discrepancy between these two patterns, supported by theoretical analysis.
enforce each
weights. Notably, these basis
The contributions of this paper include: 1) Inspired by the effectiveness and popularity of reconstruction based approaches in TAD, we propose novel method that performs representation decomposition constrained latent space, effectively addressing the potential data entanglement issue between normal and anomalous samples, resulting in significant improvements over existing methods. 2) We propose a decomposed representation learning framework to enforce that each normal representation in the latent space could be decomposed into linear combination of shared orthogonal basis vectors with sample-specific weights, thereby capturing the underlying unique nor- mal patterns effectively. Importantly, the choice of basis vectors is data-free and the basis vectors themselves are training-free free. 3) Additionally, we develop a simple and theoretically sound constraint to further amplify the discrepancy between normal and anomalous patterns within the latent space. 4) We conduct extensive experiments on 40 TAD datasets and 16 competing benchmark algorithms, and report comprehensive results along with analysis and visualizations. These experiments show that our method achieves statc-of-the-arr performance.
PRELIMINARIES
Problem Formulation. Following previous works (Yin al. 2024 Thimonier et al. 2024), we implement AD in a one-class classification setting, i.e., only normal samples are available during training. We denote the training set as Dirain } where x; E X RRP.A N the number of normal samples in the training set and D is the number of input features. The test set Drest includes both anomalous and normal samples. The goal of AD is to learn a detection model from Divain- During inference, the detection model takes sample x 'Dest as input and outputs the predicted anomaly score of x, where a higher score indicates higher confidence that x is an anomaly.
Reconstruction-besse Approaches. These methods (Yin et al. 2024 Thimonier al., 2024) have demonstrated promising efficacy tabular AD by training models to accurately reconstruct normal approaches D to minimize the reconstruction loss within D Dtrain- Typically, A(: Q) first maps the sample from observation space to latent space, and then maps it back to the observation space to obtain the reconstruction of the sample. The O can be optimized by minimizing the reconstruction loss on normal training samples:
samples while failing to reconstruct anomalous ones. Standard reconstruction-bast consider learning a mapping A(: θ) R - IR,
N Σ d(x,, x;) = -min
N Σ d(x, A(x;; 0)), i=
min e
(1)
N
N
2

---

Published as a conference paper at ICLR 2025
Cadmennaly
VOMMO
Figure 1: T-SNE visualization for original data space and deep TAD models' latent space on real-world datasets. (a-d) are on the Abalone dataset and (e-h) are on the Cardiotocography dataset. "blue" and "orange" points indicate normal and anomalous samples respectively. (a) and (e) show that the normal and anomalous samples are entangled
prediction for observations by MCM. In (c) and (g), which depict the representations learned by MCM, we observe a notable overlap between normal and anomalous representations in the latent space. In contrast, (d) and (h) demonstrate that our proposed DRL achieves more discriminative separation for normal and anomalous representations in the latent space.
observed data space. (b) and (f) show
where d(x;, x.) measures the reconstruction loss for x;, which is often set to be a distance measure- ment such as the Euclidean distance. The reconstruction loss is typically employed as the anomaly score to detect anomalies. Recently, MCM (Yin et al., 2024 introduces learnable masking strat- egy to the input and aims to reconstruct normal samples well with access to only unmasked entries of the input, where generating the effective masks is still challenging in this field. Motivated by the importance of incorporating both feature-feature and sample-sample dependencies in tabular learning (Kossen et al. 2021 Gorishniy al.) 2024), NPT-AD (Thimonier et al., (024) trains Non- Parametric Transformers (Kossen et al. (202)) to reconstruct masked features of normal samples by utilizing the whole training set. However, NPT-AD involves high computational cost in terms of memory and time, due to its reliance on the training set during inference. Both NPT and MCM rely on the reconstruction about the samples, which usually suffer from the potential data entanglement in observation space, as shown in Fig. and Fig. 7 of Appendix A.
3 METHOD
3.1 MOTIVATION
Despite the effectiveness of aforementioned works based on their specially designed masks and reconstruction strategies in the observed data space, the issue of potential data entanglement remains unaddressed, which may impede the ability for TAD. As illustrated in (a) and (c) of Fig. real-world data may exhibit observation entanglement between normal and anomalous samples. Our empirical observations indicate that this phenomenon also exists for many other real world tabular datasets (see Fig. of Appendix I). (b) and (f) of Fig. [ indicate that while MCM (Yin et al. 20224), one of the best-performing methods, demonstrates promising performance in detecting anomalies, some mistakes caused by data entanglement persist. Ignoring observation entanglement in TAD under the one-class classification setting can lead to diminished discriminative power between learned normal and anomalous patterns, the overlap between normal and anomalous representations within the latent space of deep models may obscure the distinction between them, which is illustrated in (c) and (g) of Fig. [ We attribute this challenge to the intrinsic heterogeneity of features in tabular data, which aligns with recent findings (Grinsztajn et al. (2022) indicating that neural networks struggle to learn irregular patterns, particularly when confronted with numerous uninformative features present in tabular data. That is to say, using the reconstruction loss about the observed samples to distinguish the anomalous samples from the normal samples might be insufficient.
3.2 PROPOSED METHOD
To address the patterns from the representation h = f(x; 0,) produced by a feature extractor f(:;0)) : RR + R and performing representation decomposition in the constrained latent space, where the normal and anomalous patterns are more discriminative, rather than from the raw observations directly. In the following, we will elaborate on the specific learning procedure to capture the unique patterns of nor- mal representations in Section 3.2.1 then, we will provide constraint to further enhance the dis- criminative power between normal and anomalous patterns within the latent space in Section 3.2.2 also give the overall implementation of the proposed method in Section 3.2.3] As shown in (d) and (h) of Fig. ! with our proposed method, the representations of normal and anomalous samples in the latent space discriminative. An overview of the proposed framework is depicted in Fig.
challenges, we focus on learning the underlying unique normal
3

---

Published as a conference paper at ICLR 2025
[a]
Latent Space
(d)
X-f
Alignment
Learner
1D
h: f(x)
Bamin vectora
Alignnent Zoee
Feature Extrastor
Decomposition
(b)
[e]
(x)
w
Weight Learner +
Push
Input Data
h.
12
Separation
Figure 2: The DRL framework. During training. in step (a), to capture the unique patterns of normal representa- tions in the latent space, the representation h = fx; 0y) for cach normal sample is enforced by dccomposition loss (Eq. 3) be decomposed into a linear combination of shared orthogonal basis vectors
{Bx] k= with
sample-specific weights w. where w is computed by weight learner o(x; 0g) in step (b); in step (c). a separa- tion constraint (Eq. is introduced to further enhance the discriminative power between normal and anomalous patterns within the latent space; in step (d), incorporate alignment loss (Eq. (5) preserve intrinsic feature correlation of x motivated by standard reconstruction- based method. During inference, d(h, h) derived from the decomposition loss (Eq. optimized during training in step (a) is used as the anomaly score.
3.2. DECOMPOSED REPRESENTATION LEARNING LATENT SPACE
Normal samples, which are drawn from the same distribution, are considered to represent the "nor- mal" state. Thus, it is reasonable to assume that these samples share common statistical information that distinguishes them from anomalies. Inspired by techniques from dictionary learning, topic mod- cling, and matrix factorization (Tosić & Frossard 2011: Vayansky & Kumar, 2020),, can learn the shared information by enforcing that each normal sample's representation is decomposed into a linear combination of shared basis vectors (analogous to "topics" in topic modeling) with sample- specific weights (analogous to "topic proportion" in topic modeling). Unlike traditional dictionary learning topic modeling mainly performed in the data space, our approach assumes that the repre- sentation of each normal sample in the latent space can be effectively modeled as a mixture of fixed basis vectors with specific mixture proportions.
To accurately capture the statistical characteristics of normal samples while distinguishing them from anomalies, it is crucial that the shared basis vectors are sufficiently diverse to encapsulate the global structure of the normal data. To this end, we eliminate the dependencies among basis vectors by leveraging a set of orthogonal vectors as basis vectors
{ | R where
Be E R denotes the k-th basis vector and K denotes the number of basis vectors with K E. These basis vectors are derived using the classical Gram- Schhidt process (Leon et al. 2013). which constructs an orthogonal sequence { from a linearly independent sequence [qx I by defining Br inductively follows:
k-1
B
qk Br 2 2,
B = qı
G = qu
13 2
where Br qi: denotes the dot product. In practice, we randomly sample [ from the standard
Gaussian distribution and then obtain K orthogonal vectors as basis vectors by Eq. 2 which are fixed during the whole training procedure. Additionally, experimental results demonstrate that ours robust and alternative orthogonalization methods can also effectively build basis vectors.
Our objective is to decompose each normal sample representation h Oy) into linear combination of orthogonal basis vectors B = { weighted by a sample-specific vector w w= (w*)) ( h= E ΣK, where w denotes the weight associated with the je-th basis vector, and Σ denotes the probability simplex K Considering that the weights should be sample-specific to encapsulate the unique representation information in the latent space for an observed sample x, we apply a weight learner o(x; fz)) RDD R to explicitly compute the corresponding weight vector w. Consequently, a normal sample representation h that can be successfully decomposed in ビ w* Bk, where h serves the reconstructed version of h
the latent space is expressed as h Σk_ in the latent space. To ensure that normal sample representations are accurately decomposed using the orthogonal basis vectors, we formulate the optimization problem minimize the discrepancy
4

---

Published as a conference paper at ICLR 2025
between h and h as follows:
N
N
K
min edecomposition
min 0,
Σ
d( h.)
8f B N d(f(x;0))
min
Σ
Σ wf
i=1
(3)
N
min 0g
(x;; 0)) 0y) oxx;; Oz)
N
where N is the number of training samples, :((;,) is the distance measurement and {w)) o(x;; (a)) By ensuring that each normal training sample's representation is mapped to a region where can be expressed as linear combination of fixed orthogonal basis vectors, with the as- sociate weights explicitly computed, this approach effectively models the shared statistical infor- mation of normal patterns within the latent space. This shared information encompasses two per- spectives: (i) by utilizzig all features as input, we capture the inherent interactions among fea- tures, thereby acquiring comprehensive feature-feature dependencies among normal training sam- ples, and (ii) each training sample's representation is decomposed into a set of shared basis vectors with sample-spedn specific weights, thereby optimizing the model to capture sample-sample dependencies among normal training samples.
kk=1
3.2.2 CONSTRAINT: REPRESENTATION SEPARATION
To further enhance the discriminative power between normal and anomalous patterns within the latent space, we introduce a constraint designed to amplify the discrepancy between the two patterns. We now motivate the proposed constraint by giving a closer look at the discrepancy between normal and anomalous weights w {w*) 1m I computed by weight learner ((x; 0z).
Proposition 1 Let W. E N and w. E A denote the computed weights of normal and anomalous samples, where N and A denote the weight sets of normal and anomalous samples, respectively. Given w > where ΣK represents the probability simplex in IR." the expected discrepancy between normal and anomalous weights, Ew.EN, [[[ Wa Wall). can be amplified by creasing the variance of [|Wzl12 among N.
Proposition Denoting pww.. = E.. [Wa] as the centroid of the normal samples' weights, Ew,, EN w. EA [[|Wa- Hw.,l 11 B.,,EN [||wn - pw, 113), can amplified by increasing the vari-
ance of w,lll2 among N.
The proofs of Proposition 1 and Proposition 2 are provided in Appendix A. Since only normal samples are available during training, we implement this constraint by enforcing separation among the weights of normal training samples, i.e., we push each normal sample's corresponding weight W,, away from any other one. By sufficiently separating the normal weights W., we can promote increased variance in their L2 norms (Var(]) [ar[[[w,,) ww,,l2)). thereby promoting greater discrepancy between normal weights w, and anomalous weights Wa- Given that the reconstructed representation h can be expressed as h = Σk w* BR, where basis vectors B= [BA | are fixed, w could be viewed as the computed coordinates h in the subspace consisting of basis vectors. Therefore, the separation between W,, and wa promotes separation among normal and anomalous h. This process is achieved by minimizing the separation loss as follows:
min Leeparation max 0g
Hi+jjdd[w.
0s N(N 1)
i,j=1
(4)
min
Σ i,
N(N 1)
where d(.,)) is the distance measurement. This constraint amplifies the discrepancy between normal and anomalous patterns in the latent space, which is verified in Fig. 5 and Appendix A.10 1.
3.2.3 OVERALL ALGORITHM AND IMPLEMENTATIONS.
Motivated by reconstruction-batt methods Yin et al. 2024 Thimonier et al. 2024). we also in- troduce a strategy that incorporates the intrinsic feature correlations of the sample observation
5

---

Published as a conference paper at ICLR 2025
into representation h in the latent space. This approach helps prevent the model from extracting task-irrelevant representations while preserving the critical features of the normal data. Specifically, minimize the following alignment loss:
min Colignment 0y.0a
min 05,0g
d(x;, x,)
min 03.0g
Σ
N
N
i=1
(5)
N
min 05.0g
Σ
d(x;
where 0g: : I + IDD is the alignment learner, x; - -gh;;; (x](xx00);;0)) is the reconstruction of x,, and d(.,-) the distance measurement.
Altogether, DRL aims to minimize the following objective function w.r.t. f,
0s. 0g during training:
min Lall = min Edecomposition + X1 min Eseparation + A2 min Latignment, 0y.0s.0z
(6)
0y.0g
0,
05
where A1 A2 indicates the nyper-parameteed for balancing the total loss functions. The decomposed representation learning is achieved via Ledccoppsition (Eq.() to capture the unique patterns of nor- mal sample representations the latent space, Additionally, a separation constraint via Lseparation (Eq. 4 is introduced to further enhance the discriminative power between normal and anomalous patterns within the latent space. In practice, the separation constraint implemented within each mini-batch to reduce the computational cost. We also introduce strategy via Lalignment (Eq. (5) that incorporates the intrinsic feature correlations of the sample observations into representations.
During inference, we derive the anomaly score = d(h, h) = d( (x; fy), o(x; O)) from the Edecomposition (Eq. optimized during training, which performs reconstruction for representation in the constrained Tatent space, rather than in the observed data space (typically adopting d(x, x) anomaly score discussed in Section 2). A higher anomaly score indicates higher confidence that is an anomaly. We provide the training and inference process of DRL in Algorithm and of Appendix A.. respectively.
Given the potential entanglement between normal and anomalous samples in the observed data space, our proposed method offers a notable advantage by providing greater flexibility in adjust- ing representations within the latent space compared to the observed space (Ye et al. 2024 dan Guo et al., (0022). This flexibility allows to enforce constraints on the learned representations, making them more discriminative between normal and anomalous patterns, and thereby capturing the distinct characteristics of normal representations more effectively. The representation decom- position process seen by the model only corresponds to the normal samples during training, and separation constraint applied to further enhance the distinction between normal and anomalous patterns. This approach significantly impedes the ability exploit representation decomposition for anomalies, ensuring a more robust discrimination between the two categories.
4 RELATED WORK
Classical Anomaly Detection. One of the main settings for anomaly detection is one-class classifi- cation, that is identifying anomalies after observing set of normal samples (Sohn et al.) [021] Yin et al. 2024). Over the decades, various classical methods have been developed, consistently show- ing strong performance, particularly with tabular data. Density estimation This approach typically involves estimating the distribution of normal data using either parametric or non-parametric tech- niques, allowing for anomaly detection based on sample likelihoods under the estimated distribution. Traditional models include KDE (Parzen 1962) and GMM (Roberts & Tarassenko) 1994). Recently, ECOD (Li et al. 2022) has employed empirical cumulative distribution for this purpose. Addition- ally, some methods utilize local density estimation to identify anomalies, such as LOF (Breunig jet al. 2000). Classifationn-assed methods This category focuses on learning a decision boundary using only normal data. For example, OCSVM (Schölkopf al. 1999) creates this boundary by maximizing the margin between the imput data and the origin. Reconstruction based methods These methods are among the most widely used as discussed in Section including classical methods such as PCA (Shyu et al.] 2003). For a comprehensive overview of anomaly detection, we recommend
the following surveys (Pang al. 2021 Ruff et al. 2021 Chandola et al. 2009).

---

Published as a conference paper at ICLR 2025
Deep Learning-basee Anomaly Detection. As classical methods struggle to capture complex pat- terns and relationships in high-dimensional spaces (Pang et al., 2021), recent studies have prompted shift toward deep learning methodologies for tabular data. For example, ICL (Shenkar & Wolf, 2022) learns mapping relationships that maximize the mutual information between the feature sub- set and its rest. RAPP (Kim al. 2019) computes anomaly scores by comparing observations and reconstructions in both the input and hidden space of well-trained AutoEncoder (Chen et al.) 2018). In contrast, our approach re-maps observations into a specially designed constrained space and decomposes the representations into basis vectors to capture normal patterns. Inspired by dif- fusion models (Ho et al.) 2020), DTE (Livernoche al., 2024) simplifies the diffusion process to reconstruct observations for tabular anomaly detection. Recent advancements, including MCM (Yin) et al. 2024) and NPT-AD (Thimonier et al.) 2024), are discussed in Section 2 Among these, the Widely used methods primarily belong to reconstruction-basee approaches (Kim et al. 2019: Liver-
noche et al. 2024 Yin et al. 2024| Thimonier et al. 2024). However, the issue of data entanglement
between normal and anomalous samples is ignored.
5 EXPERIMENT & ANALYSIS
5.1 EXPERIMENT SETUP
Datasets. We conduct experiments on an extensive benchmark of 40 tabular anomaly detection datasets selected from Outlier Detection DataSets (ODDS) (Rayanal 2016) and Anomaly Detection Benchmark (ADBench) (Han et al. 2022). following previous works (Yin et al. 2024 Thimonier al. 2024). These datasets span diverse domains, including healthcare, science, and social sci- ences, and vary in characteristics from low-dimensional, low-scale to high-dimensional, large-scale. Detailed information about the dataset properties is provided in Appendix A.
Evaluation Metrics. Per the literature (Zong et al. 2018 Bergman & Hoshen 2020 Yin et al.,
2024 Thimonier al.) 2024), we construct the training set by randomly subsampling 50% of the normal samples. The remaining 50% of the normal samples are then combined with the entire set of anomalies to form the test set. Following previous work (Han al. 2022 Yin al., 2024). we employ Area Under the Frecision-Recat Curve (AUC-PR) and Area Under the Receiver Operating Characteristic Curve (AUC-ROC) as our evaluation criteria.
Baseline Models. We conduct comparative analysis between DRL and 16 other prominent meth- ods in the field of TAD. Specifically, we include classic techniques such as OCSVM (Schölkopf et al. 1999). KNN (Ramaswamy et al. 2000), LOF (Breunig et al. (000), PCA (Shyu et al. (003), IForest (Liu et al. 2008), and ECOD (Li et al. 2022). which continue to be widely used. In addition, compare our method against recent deep learning-basee baaen approaches, including DAGMM (Zong
ct al. 2018), Deep SVDD (Ruff et al. 2018), AutoEncoder (Chen et al.] 2018). RAPP (Kim et al.
2019
GOAD (Bergman & Hoshen 2020), NeuTraLLD (Qiu et al.) (222)). ICL Shenkar & Wolf
2022
DTE (Livernoche et al.) 2024), MCM (Yin et al., (024), and NPT-AD (Thimonier et al.
2024
More details about the implementation of baseline models
be found in Appendix A.2
DRL Details. The DRL architecture remains consistent across all datasets. Specifically, the feature extractor f(;0,) : RDD E and weight learner o(:30)) RD - R.K are implemented simple two-layer fully connected MLP with Leaky ReLU activation function. The last layer of weight learner is with Softmax activation function. The alignment learner g(:;0,) IR + RD is a linear layer. For the distance measurement -). we use L.2 distance for Ldecomposition (Eq. 3 Cosine distance for Exeparation (Eq. 4 and Lalignment 5). The default number of basis vectors K is set to 5, and these basis vectors are not updated during training. A₁ A2 is set to 0.06 and 0.1 for separation loss and alignment loss respectively. The hidden dimension E is set to 128, the batch size is set to 512, and the number of epochs is set to 200. Adam optimizer is employed, bounded by an exponentially decaying learning rate controller with 0.05 as initialization. To reduce the effect of randomness, the reported performance is averaged over 10 independent runs
'The source code is available at htttpps///gichhh...c///aaanitint
7

---

Published as a conference paper at ICLR 2025
(a) ALC--R performance [+))
(b) AUC-ROC performance (+)
(c) AUC-PR ranking
(4) AUC-ROC ranking
128
20 04 de 15
02 63
29
423456777 9100ER2
Figure 3: Comparison of all models' performance and ranking across different datasets in terms of AUC-PR
and AUC-ROC. The red triangles represent the average value over all datasets.
(a) Paralue AUC-PR performance (b) Puyauee AUGROC performance
(c) Pvalue of AUC-PR ranking
(cl Pivalie of AUC-RDC ranking
LOF
1000
RESTALAD
EL
MON
oc
PCr
ore toothug
Eoto Pox chr ocove
nandoggs
cor F
Corpan aaae
oir-
osc. surrs
Pick
Del
Figure 4: The wilcoxon test is conducted between different models across datasels. The significant dif- ference of different settings are provided. Blue color indicates the corresponding p-value is lower than 0.05
(significant), while white color indicates the corresponding p-value is higher than 0.05 (not significant).
5.2 EMPIRICAL RESULTS AND ANALYSIS
Main Results. We evaluated the performance and model ranking of DRL against 16 mainstream baseline models across 40 tabular datasets using two evaluation metrics. The results, compris- = 2720 numerical results,
ing 2(performance & ranking) (16-))))))))))))))))))111+++ (1++1)16m)mls))) (0daassstsss are summarized in boxplots, with each boxplot representing a model's performance across various datasets, as illustrated in Fig. 3 Detailed results, average standard deviations, and analyses of the baseline methods are available in Appendix AA.4 Overall, DRL demonstrates state-of-the perfor- mance, achieving an absolute average AUC-PR gain of at least 9.8% compared to other methods. Notably, DRL consistently enhances performance across datasets, regardless of dimensionality or size, as shown in Fig- of Appendix(AA4 We conduct the Wilcoxon signed-rant test (with a 0.05) [Woolson] 2007) to measure the improvement significance, as illustrated in Fig- In all settings, improvement of DRL over baseline models is statistically significant at the 95% confidence level. Additionally, provide T-SNE visualizations of real-world data in Fig. of Appendix [ revealing that data entanglement is a common issue in many tabular datasets. Even cases with- out data entanglement in the observation space, DRL enhances performance by making normal and anomalous patterns more distinguishable.
Different Basis Vector Initialization Methods. As utilize set of orthogonal vectors basis vectors to capture normal patterns, we systematically investigate various orthogonalization methods for generating these vectors, detailed in Table Specifically, Random indicates that randomly sample linearly independent vectors from the standard Gaussian distribution without applying any orthogonalization. The Householder (Schreiber & Van Loan 1989) transformation reflects a given vector across hyperplane defined by a unit vector. successively generating orthogonal vectors. Givens (Shalit Chechik 2014) rotation applies rotation matrix to a pair of coordinates, effec- tively zeroing out one component while preserving orthogonality. SVD denotes Singular Value
Table I: Comparison of basis vector generation strategies averaged over all datasets. Orthogonalization of basis vectors crucial for capturing normal patterns and distinguishing them from anomalies, with different methods performance, while learnable basis
yielding similar robustness. The Gram-Schmidt process achieves vectors underperform due to optimization complexity.
Metrics
Random
Househokder
Givens
SVD
GB (0.5. 1)
GS (0. 1.5)
GS (0. ()
GS (0, 1) (ours)
AUC-PX
0.667 (0001)
0.729 (0,0213
0.736 (0.024)
0.712 (0.023)
0.730 (0.028)
0.722 (0.030)
0.697 (0.032)
0.334 (0.028)
AUC-ROC
0.831 (Q,021)
0,851 (0.011)
0.849 (D-015)
0.837 (0.014)
0.851 (0.014)
01866000155
0.336 (0.024)
0367 (0.13)
8

---

Published as a conference paper at ICLR 2025
Table 2: Analysis on the effects of different components of DRL in terms of AUC-PR and AUC-ROC averaged over 40 datasets. The full results are provided in Table IT Table 12 Appendix A..0
Trnining
Inference
Evaluation Metric ADC-PR AUC-ROC
Variants
Edecomposttio
Exeperation Entigument
Weight kummer
Edccomposition
Calignment
0.6344
0.7882
0.6844
0,7999
0.708]
0.8123
0,5222
0,7066
(Observation)
v (Observation)
0,5407
0,7240
F
(Least square)
0.6964
0.8205
(MCM's repessentation)
/MCCMs representation)
0.6013
0.7273
(MCM w/ DRL.
0,6750
0,7837
0.6928
0.8286
0.6100
0,7720
(ours)
0.7344
0.8574
Decomposition (Klema & Laub 1980), which decomposes a matrix into orthogonal matrices and a
diagonal matrix capturing the singular values, thus revealing the matrix's latent structure. GS refers to the Gram-Schmin process (Leon et al. 2013) utilized in our study. In this experiment, we sample linearly independent vectors from Gaussian distribution, with the values in parentheses represent- ing the mean and variance. GS* (0, 1) indicates that the basis vectors are learnable during training, while they remain fixed in other settings.
Effectiveness of Key Components of DRL. We further analyze the effectiveness of key compo- nents of DRL, as illustrated in Table by comparing DRL against three groups of variants. Variant A to E explore training DRL with different loss configurations: (1) variant A, that removes both separation constraint and alignment loss when training DRL; (2) variant B, that removes separation constraint when training DRL; (3) variant C, that removes alignment loss when training DRL; (4) variant D, that uses only the alignment loss when training DRI, with the alignment loss directly serving as the anomaly score during inference; (5) variant F, that first constructs a set of orthogonal basis vectors, matching the dimensionality of the input features. It then employs the weight learner, as detailed in Section 3.2.1 to compute sample-specific weights. These weights are used to form linear combination of the basis vectors, yielding the reconstructed observations. The decomposition loss is applied directly to the observations, and this loss serves as the anomaly score. The removal of any of the components during training degrades the performance of DRL. Notably, the separation constraint significantly enhances performance, which we attribute its ability to improve the dis- criminative power between normal and anomalous patterns within the latent space. Variant F to H examine training DRL with different architectures: (6) variant F, decomposes each normal sample representation into linear combination of orthogonal basis vectors with sample-specific weights, and computes these weights using least squares instead of a weight learner; (7) variant G, that applies the decomposition loss to the well-trained representations of MCM, assuming that the fixed normal representations can be decomposed into orthogonal basis vectors; (8) variant H. that replaces DRL's feature extractor and alignment learner with MCM's encoder and decoder, effectively training MCM within the DRL framework. While Variant achieves promising performance, it still underperforms compared to DRL. DRL benefits from learnable weight learner that captures the global distribution of weights across all normal training samples, enabling it to accurately represent complex normal patterns. Variant G highlights that well-trained reconstruction- based representations may not lie within the latent space defined by the basis vectors, complicating the modeling of normal informa- tion. Moreover, variant H shows that while substituting DRL's components with MCM's leads to significant improvements over MCM itself, it still falls short of DRL due to the observation-centric strategy, which can introduce data entanglement. Variant I and J test the well-trained DRL with dif- ferent anomaly score configurations during inference: (9) variant I, that uses both decomposition and alignment losses as anomaly scores during inference; (10) variant J, that only uses alignment loss as anomaly scores during inference. The results indicate that decomposition loss is more effective than alignment loss in capturing normal information during inference.
Different Distance Metries. Besides, we also provide additional experiments to validate the effec- tiveness of different distance metrics for decomposition loss, separation constraint and alignment loss in Table 3 By default, we use L2 distance for decomposition loss, cosine distance for separa- tion loss and alignment loss. For separation and aligmment loss, L1, L2, and cosine distance metrics all yield promising performance, with cosine distance demonstrating relatively superior and more stable performance. The separation constraint exhibited more variability depending on the distance metric used. We attribute this to the fact that when maximizing the I.1 and 1.2 distance metrics, the

---

Published as a conference paper at ICLR 2025
Table 3: Analysis on the effects of various distance metrics in terms of AUC-PR across different datasets. The average results over 40 datasets are also provided. Full results are in Table and Table 14 Appendix A.
Different componants
Rackdoor
Tnngsog
1ympho
Pendigis
Vowels
Whe
Average
Decompositio wi Cosine
0.9033
0.9022
114307
0.9417
Separation w/ LI distunee
0.8784
0.6391
0.9185
0.8391
0.882
0.44
0.9389
0.7007
Alignment w/ LI distance
0.8868
0.6433
0.9001
0.9962
0.9218
0,3696
0.9401
0.7063
Seperition distance
0.6444
0.8998
0,8735
0,4635
0.9655
Alignmest DRL (ears)
0,8886
0.5915
0.6649
0.9238
1.0000)
0,9360
0,450s
0,9742
0.7344
lack of inherent range limitations can lead to excessively large distance values, which hinder the model's optimization process.
Discussion. In Fig. S present the average discrepancy between normal and anomalous observations over all data pairs which is la- beled "Original", alongside the discrepancy of latent representa- tions under "DRL. w/o S and "DRI.", where "S" denotes the sep- aration constraint. The results in Fig. 5 indicate that our proposed method enhances the discriminative distinction between normal and anomalous patterns within the latent space. Additionally, we visual- ize the T-SNE of the learned representations from DRL on the An- nthyroid dataset Fig- [6] The findings reveal that the observation entanglement between normal and anomalous samples in existing reconstruction based methods can obscure their distinction within the latent space. In contrast, DRL effectively separates the repre- sentations into distinct regions, thereby enhancing the discriminative capability between normal and anomalous patterns.
3.5
3.0
2.5
Driginal
Figure 5: The ratio of the mean L.2 distance between mal and anomalous samples to the mean L2 distance among pairs of normal samples across all datasets.
Hornel
Horeel
Anrthyraia
Narnal
Representation on of MCM Figure 6: Visualization for original data space and deep TAD models' latent space real-world datasets.
Original data
Representation of NPT-AD Representation of DRL (ours)
Additional Experiments. The computational efficiency detailed in Appendix A.5 We incorpo- rate the sensitivity analysis w.r.t. the number of basis vectors, the loss weight \ X2. the number of training epochs and the number of batch size in Appendix A.6 We also provide the weight visu- alization of normal and anomalous representations in Appendix A.7] DRL's robustness to anomaly contamination in Appendix A.8 and the model performance on different types of anomalies in Ap- pendix A.9, Furthermore, additional empirical evidence and theoretical analysis are provided in Appendix and Appendix A.II respectively.
CONCLUSION
We study the problem of anomaly detection for tabular data, typically implemented in one-class classification setting where the training data comprises solely normal samples. Most state-of-the-att TAD works are along the line of reconstruction; however, reconstruction on training data can still hardly distinguish anomalies due to the data entanglement. To address this problem, in this paper, we propose a novel approach Decomposed Representation Learning (DRL), to re-map data into a tailor- designed constrained space, in order to capture the underlying shared patterns of normal samples and differ anomalous patterns for tabular anomaly detection. Our approach decomposes each normal sample's representation into weighted linear combination of randomly generated orthogonal basis vectors, where these basis vectors are both data-free and training-free free. We further enhance the discriminative ability between normal and anomalous patterns through simple and theoretically sound constraint. The empirical results demonstrated the effectiveness DRL for tabular anomaly detection. Our work shed some light on the development of better algorithms for similar tasks.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The authors would like thank the anonymous referees for their valuable comments. In this work, Hangting Ye, Dandan Guo and Yi Chang are supported by the National Natural Science Foundation of China (No. 623B2043, No. U2341229, No. 62306125) and the National Key R&D Program of China under Grant (No. 2023YFF0905400),
REFERENCES
Zeeshan Ahmad, Adnan Shahid Khan, Cheah Wai Shiang, Johari Abdullah, and Farhan Ahmad. Network intrusion detection system: systematic study of machine learning and deep learning approaches. Transactions on Emerging Telecommunications Technologies, 32(1):e41500 2021.
Khaled Gubran Al-Hashedi and Pritheega Magalingam. Financial fraud detection applying data mining techniques: A comprehensive review from 2009 to 2019. Computer Science Review, 40: 100402, 2021.
Liron Bergman and Yedid Hoshen. Classification based anomaly detection for general data. arXiv preprint arXiv:2005.0233 2020.
Rajendra Bhatia and Chandler Davis. A cauchy-schwarz inequality for operators with applications. Linear algebra and its applications, 223:119-129; 1995.
Rajendra Bhatia and Fuad Kittaneh. Notes on matrix ceathhmetii-gemmettit mean inequalities. Linear Algebra and Its Applications, 308(1-3):203-21 2000.
Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey. IEEE transactions on neural networks and learning systems, 2022.
Markus M Breunig, Hans-PPter Kriegel, Raymond T Ng. and Jörg Sander. Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference Management of data, pp. 93-104, 2000.
Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM com- puting surveys (CSUR), 41(3):1-58, 2009.
Chun-Har Chang, Jinsung Yoon, Sercan Ö Arik, Madeleine Udell, and Tomas Pfister. Data-efficient and interpretable tabular anomaly detection. In Proceedings of the 29th ACM SIGKDD Confer- ence on Knowledge Discovery and Data Mining, pp. 190-201, 2023.
Zhaomin Chen, Chai Kiat Yeo, Bu Sung Lee, and Chiew Tong Lau. Autoencoder-bue based network anomaly detection. In 2018 Wireless telecommunications symposium (WTS), pp. 1-5. IEEE, 2018.
Dan dan Guo, Long Tian, Minghe Zhang, Mingyuan Zhou, and Hongyuan Zha. Learning prototype- oriented set representations for meta-learning. In International Conference on Learning Repre- sentations, 2022. URL
n www.mrorruuuuiio LerruummdddWWHH22S11ppp
Tharindu Fernando, Harshala Gammulle, Simon Denman, Sridha Sridharan, and Clinton Fookes. Deep learning for medical anomaly detection-a survey. ACM Computing Surveys (CSUR), 54(7): 1-37, 2021.
Izhak Golan and Ran Eaniy. Deep anomaly detection using geometric transformations. Advances in neural information processing systems, 31, 2018
Yury Gorishniy, Ivan Rubachev, Nikolay Kartashev, Daniil Shlenskii, Akim Kotelnikov, and Artem Babenko. Tabr: Tabular deep learning meets nearest neighbors. In The Twelfth International Conference on Learning Representations, 2024. Rtt/hhttss://oeneettt
net forum?
id=rhg TT5Sx
Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data? Advances in neural information processing systems, 35: 507-520, 2022.
11

---

Published as a conference paper at ICLR 2025
Dandan Guo, Long Tian, Chuan Du, Pengfei Xie, Bo Chen, and Lei Zhang. Suspicious object de- tection for millimeter-wave images with multi-view fusion siamese network. IEEE Transactions on Image Processing, 32:4088-4100 2023. doi: 10. 10.1109/T1P200333277777
Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. Adbench: Anomaly
detection benchmark. Advances in Neural Information Processing Systems, 15:32142-32159, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 68510 2020.
Ki Hyun Kim, Sangwoo Shim, Yongsub Lim, Jongscob Jeon, Jeongwoo Choi, Byungchan Kim, and Andre S Yoon. Rapp: Novelty detection with reconstruction along projection pathway. In International Conference on Learning Representations, 2019.
Virginia Klema and Alan Laub. The singular value decomposition: Its computation and some appli- cations. IEEE Transactions automatic control, 25(2):164-176, 1980.
Jannik Kossen, Neil Band, Clare Lyle, Aidan N Gomez, Thomas Rainforth, and Yarin Gal. Self- attention between datapoints: Going beyond individual input-output pairs deep learning. Ad- vances in Neural Information Processing Systems, 34:28742-28756. 2021.
Steven J Leon, Åke Björck, and Walter Gander. Gram-schmidt orthogonalization: 100 years and more. Numerical Linear Algebra with Applications, 20(3):492-532, 2013.
Zheng Li, Yue Zhao, Xiyang Hu, Nicola Botta, Cezar lonescu, and George Chen. Ecod: Unsu- pervised outlier detection using empirical cumulative distribution functions. IEEE Transactions on Knowledge and Data Engineering, 35(12): 2022.
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. 2008 eighth ieee international conference on data mining, pp. 413-422. IEEE, 2008
Victor Livernoche, Vineet Jain, Yashar Hezaveh, and Siamak Ravanbakhsh. On diffusion modeling for anomaly detection. In The Twelfth International Conference on Learning Representations, 2024. U net. wtt.:eeeoooommme11111311///
Philipp Liznerski, Lukas Ruff, Robert A Vandermeulen, Billy Joe Franks, Marius Kloft, and Klaus- Robert Müller, Explainable deep one-class classification. arXiv preprint arXiv:2007.00/66) 2020.
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. Deep learning for anomaly detection: A review. ACM computing surveys (CSUR), 54(2):1-38, 2021.
Emanuel Parzen. On estimation of probability density function and mode. The annals of mathe- matical statistics, 3363):1065-1076 1962.
Chen Qiu, Timo Pfrommer, Marius Kloft, Stephan Mandt, and Maja Rudolph. Neural transformation learning for deep anomaly detection beyond images. In International conference machine learning, pp. 8703-8714, PMLR, 2021.
Sridhar Ramaswamy, Rajeev Rastogi, and Kyuseok Shim. Efficient algorithms for mining outliers from large data sets. In Proceedingy of the 2000 ACM SIGMOD international conference on Management of data, pp. 427-438, 2000.
Shebuti Rayana. ODDS library, 2016. URL
Stephen Roberts and Lionel Tarassenko. A probabilistic resource allocating network for novelty detection. Neural Computation, 6(2):270-288) 284, 1994.
Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexan- der Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference on machine learning, pp. 4393 4402. PMLR, 2018
Lukas Ruff, Jacob R Kauffmann, Robert A Vandermeulen, Grégoire Montavon, Wojeiech Samek, Marius Kloft, Thomas G Dietterich, and Klaus-Rooh Robert Müller. A unifying review of deep and shallow anomaly detection. Proceedings of the IEEF, 10955)77567755 2021.
12

---

Published as a conference paper at ICLR 2025
Bernhard Schölkopf, Robert C Williamson, Alex Smola, John Shawe-Taylon and John Platt. Support vector method for novelty detection. Advances in neural information processing systems, 12, 1999.
Robert Schreiber and Charles Van Loan. A storage-efficient wy representation for products of house- holder transformations, SIAM Journal on Scientific and Statistical Computing, 10(1):53-57, 1989.
Uri Shalit and Gal Chechik. Coordinate-descent for learning orthogonal matrices through givens rotations. In International Conference on Machine Learning, pp. 548-556. PMLR, 2014.
Tom Shenkar and Lior Wolf. Anomaly detection for tabular data with internal contrastive learning. In International conference on learning representations, 2022.
Mei-Ling Shyu, Shu-Ching Chen, Kanoksri Sarinnapakorn, and LiWu Chang. A novel anomaly detection scheme based on principal component classifier. In Proceedings of the IEFF foundations and new directions of data mining workshop, pp. 172-179. IEEE Press Piscataway, NJ, USA, 2003.
João Vitor Valle Silva, Martin Andreoni Lopez, and Diogo MF Mattos. Attackers are not stealthy: Statistical analysis of the well-known and infamous kdd network security dataset. In 2020 4th Conference on Cloud and Internet of Things (CloT), pp. 1-8. IEEE, 2020.
Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, and Tomas Pfister. Learning and evalu- ating representations for deep one-class classification. In International Conference on Learning Representations, 2021. URL https: http::///ppeereeeeeeee http://wttteeta////t fwrum?rrmiooooooo htt.:t//ooooon2----CCCCCCCOoo
Hugo Thimonier, Fabrice Popineau, Arpad Rimmel, and Bich-Liën DOAN. Beyond individual input for deep anomaly detection on tabular data. In Forty-first International Conference on Machine Learning. 2024. URL https: / //tp///eeeieeeeeeooe/eeeeeoo net Eorum? -eeiiidd=chhiii chwpphpbppPPP3P33
Ivana Tošić and Pascal Frossard. Dictionary learning. IEEE Signal Processing Magazine, 28(2):
27-38, 2011.
Ike Vayansky and Sathish AP Kumar. A review of topic modeling methods. Information Systems, 94:101582, 2020.
Robert F Woolson. Wilcoxon signed-rank test. Wiley encyclopedia of clinical trials, pp. 1-3, 2007.
Hongzuo Xu, Guansong Pang, Yijie Wang, and Yongjun Wang. Deep isolation forest for anomaly detection. IEEE Transactions on Knowledge and Data Engineering, 35(12):12591 2023. doi: 10. 10.11099TKDDE..2223377222
Hangting Ye, Zhining Liu, Wei Cao, Amir M Amiri, Jiang Bian, Yi Chang, Jon D Lurie, Jim We- instein, and Tie- Yan Liu. Web-based long-term spine treatment outcome forecasting. In Pro- ceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp- 3082-3092, 2023a.
Hangting Ye, Zhining Liu, Xinyi Shen, Wei Cao, Shun Zheng, Xiaofan Gui, Huishuai Zhang, Yi Chang, and Jiang Bian. Uadb: Unsupervised anomaly detection booster. In 2023 IEEE 39th International Conference Data Engineering (ICDE), pp. 2593-26006 IEEE, 2023b.
Hangting Ye. Wei Fan, Xiaozhuang Song, Shun Zheng, He Zhao, Dan dan Guo, and Yi Chang. PTaRL: Prototype-based tabular representation learning via space calibration. In The Twelfth International Conference on Learning Representations, 2024.
net//oorumidd-32200
Jiaxin Yin, Yuanyuan Qiao, Zitang Zhou, Xiangchao Wang, and Jie Yang. Mem: Masked cell modeling for anomaly detection in tabular data. In The Twelfth International Conference on Learning Representations, 2024.
Yue Zhao, Zain Nasrullah, and Zheng Li. Pyod: A python toolbox for scalable outlier detection. Journal of machine learning research, 20(96):1-7, 2019.
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Dacki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In Inter- national conference learning representations, 2018
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
In this section, we provide the details of datasets, the implementations of baseline methods, the full comparison results with baseline methods, the computational efficiency details, hyper-parameter sensitivity analysis, robustness to anomaly contamination, model performance on different types of anomalies and visualization. The source code for implementing DRL will be released after the paper is published.
A.1 DATASETS DETAILS
We experiment on an extensive benchmark of tabular datasets following previous work (Yin et al, 2024). which spans diverse domains, including healthcare, natural science, social sciences, etc. We omit the KDD dataset since it presents a certain number limitations (Silva et al. (2220). Instead, according (Thimonier al. 2024). include three real-world datasets from (Han et al. 2022) that display relatively similar characteristics to KDD in terms of dimensions: Fraud, Campaign, and Backdoor. In addition, we extend the benchmark of (Yin al. (2024) by including other com- monly used tabular real-world datasets from ADBench (Han et al. (022). The dataset properties
are summarized in Table 4 The Table 4 shows the number of samples, the number of features, and the number of anomalies of each dataset used. Additionally, we provide T-SNE visualizations of real-world data in Fig. 7
Figure 7: T-SNE visualization real world datasets.
14

---

Published as a conference paper at ICLR 2025
Table Dataset properties. We use 40 commonly used tabular anomaly detection datasets this paper.
Samples
Dims
Anomalies
Abalone
4177
7
2081
Amazon
10000
768
500
Annthyroid
7200
534
Arrhythmia
452
274
66
Backdoor
95329
196
2329
Breastw
683
239
Campaign
41188
62
4640
Cardio
1831
21
176
Cardiotocography
2114
21
466
Census
299285
500
18568
Comm.and.crima
1994
101
993
Cover
286048
10
2747
Fault
1941
27
673
Fraud
284807
29
492
Glass
214
9
Hepatitis
80
19
13
Imgseg
2310
18
990
Ionosphere
351
33
126
Lympho
148
18
6
Mammography
11183
260
Mnist
7603
100
700
Musk
3062
Optdigits
5216
64
150
Parkinson
195
147
Pendigits
6870
16
156
Pima
768
268
Satellite
6435
36
2036
Satimage-2
5803
36
71
Shuttle
49097
3511
SpamBase
4207
57
1679
Speech
3686
400
61
Thyroid
3772
93
Vertebral
240
30
Vowels
1456
12
50
WDBC
367
10
WPBC
198
33
47
Wbe
378
Wilt
4819
5
257
Wine
129
10
Yeast
1484
507
15

---

Published as a conference paper at ICLR 2025
A.2 BASELINE MODELS DETAILS
We conduct a comparative analysis between DRL and 16 other prominent methods in the field of TAD, including classical non-deep methods and recent deep learning based methods. Specifically,
OCSVM (Schölkopf et al., 1999). KNN (Ramaswamy et al., 2000). LOF (Breunig et al.] 2000), PCA (Shyu al., 2003). IForest (Luu al. (2008) and ECOD Li al. 2022) represent classic AD approaches that continue to maintain popularity. In addition, we compare our method to recent deep
learning based methods, namely DAGMM (Zong et al.
2018). Deep SVDD (Ruf' al. 2018). Au-
toEncoder (Chen et al.] (2018), RAPP (Kim et al. 2019), GOAD (Bergman & Hoshen 2020). Neu-
TraLAD (Qiu et al. (2021), ICL (Shenkar & Wolf 2022). DTE (Livernoche et al. 2024). MCM (Yin)
al. 2024) and NPT-AD (Thmmonier et al. 2024
We use the popular PyOD python package (Zhao
et al.
2019) to implement OCSVM, LOF, PCA,
Forestt ECOD, Deep SVDD and AutoEncoder. We
the DeepOD python library (u et al. 2023) to implement GOAD, NeuTraLAD and ICL. The implementation of the other methods is based on their official open-source code releases. Following latest works (Yin et al., 2024: Thimonier al., 2024), We implement all baseline models' hyper- parameters following their original papers. All the methods are implemented with identical dataset partitioning and preprocessing procedures, following previous works (Yin al. 2024
A.3 DRL WORKELOW
Algorithm DRL training workflow.
Input: Training dataset Derain. feature extractor f(:00)) IRD 7 gE weight learner o(:; 0&), alignment learner 9(::0,) RE + RD, orthogonal basis vectors B (Bx] S R x obtained by Eq. [ hyper-parameters λ1,
0g has not converged do 2: Sample a mini-batch from Diraini
1: while , 0z.
3: for x, in B do
4:
Obtain sample representation h, by f fxx;; 0y);
5: Calculate the corresponding weights w; in latent space by 0x);;
6:
Obtain the estimated observation X; by g(;;; 0,);
7:
end for
8:
Compute Ldecomposition
1 [B| EXED d(f(xx(())) f o( 0g B) with Eq.
9:
Compute Leeparation [B((BI-1, -1 1 # j} dl d(o(xx; 0τ). d(x;; 0x)) with Eq. Compute Lalignment = [3 Σx,EB d(x, (xx;; fy); 0g)) with Eq. 5
10:
11:
Update 0y, 0s, 0g by minimizing Call Edecompoeition X1 Ceeparaaioon A2 Latiggmment
with Eq. through gradient descent. end while
12:
Output: Trained feature extractor f(:;0)), trained weight learner 0s).
Algorithm 2 DRL inference workflow.
Input: Test dataset Drest, trained feature extractor f(; Oy), trained weight learner o(; orthog- onal basis vectors B = (Gx 1k C R
K obtained by Eq.
1: for x; in Drest do
2: Obtain sample representation h, by f(x;; 0y);
3: Calculate the corresponding weights w, in latent space by o(x;; 0s);
4: Compute 8; d(f(xx;; 0f), ((x;; 1)) anomaly score (higher score indicates higher
confidence that x; is an anomaly) derived from the Edecomposition (Eq. 3
5: end for
Output: Anomaly score {st) {st) |Peasl
16

---

Published as a conference paper at ICLR 2025
A.4 FULL COMPARISON RESULTS WITH BASELINE METHODS
We compare the performance and model ranking of DRL against 16 mainstream baseline mod- els across 40 tabular datasets using 2 evaluation metrics. We summarize this large-scale experiment results (2(performance ranking) x x16 6+ 40 ((meerrcs) 2720 numerical re- sults) in boxplots, where each boxplot corresponding to each model shows its results across different datasets, as shown in Fig. 3 The full results and average standard deviation are provided in table [ and table [ We can observe that DRL achieves the highest average performance, and the lowest average ranking overall datasets for both AUC-PR and AUC-ROC. Specifically, DRL obtains an absolute average AUC-PR gain of least 9.8% compared to other methods. Considering the over- all performance and model ranking across 40 datasets, DRL achieves state-of-the-an performance. Furthermore, DRL consistently improves performance compared to baseline models w.r.t both di- mensionality and data size, as detailed in Fig. Additionally, conduct Wilcoxon signed-rank test (with a 0.05) (Woolson 2007) to measure the improvement significance, as illustrated in Fig. 44] In all settings, the improvement of DRL over baseline models is statistically significant the 95% confidence level. This demonstrates the superior effectiveness of DRL. compared to baseline models across different datasets. As the representative reconstruction-bass. detection, DTE, MCM and NPT-AD belong to the best-peri best-performing baseline methods according to the results, which is aligned with the results in their original papers. However, as discussed Sec- tion 3.1 relying on reconstruction loss from observed samples to differentiate anomalous samples may be insufficient due inherent data entanglement. RAPP computes anomaly scores by compar- ing sample observations with their autoencoder reconstructions in both the input and hidden spaces, yet does not address the underlying issue of data entanglement between normal and anomalous samples. In contrast, DRL enhances anomaly detection in tabular data by learning the shared un- derlying normal patterns normal samples through representation decomposition in a constrained latent space, where normal and anomalous patterns more discriminative. We provide TSNE visualizations real-world data Fig. / and our empirical observations indicate that data entan- glement is prevalent issue in many real-world tabular datasets. As shown in table 5 and table 66 even in cases without data entanglement in the observation space, DRL still improves performance by making normal and anomalous patterns distinguishable.
methods for tabular anomaly
17

---

Published as a conference paper at ICLR 2025
Table 5: Comparison of AUC-PR (1) results between baseline methods and DRL on 40 datasets.
JOCSVM
KNN
LOF
PCA
BCOD
DAGKM
DDD
RAPP
GOAD
NevTralAA
ICL
L/TE
30CM
NET-AD
DRL (ours)
Abulose
0.8499
0.8596
0:813
0,8393 0.8481
0.6554
0.7899
0.2173
0.5295
0:8125
(18233
0.7755
0.7655
0.4295
0,7471
0.2890
0,885
Amooun
0:304
0.0901
0:1117
n 1.107
0.1091
0.104
0.1077
0.1005
0.117
0.1179
11 1886
111077
1111
0. 0.1085
018611
0.1336
0.831
0.3525
[44513
0,5657
0.640
0.4002
0.2199
0.3235
0.529
0.3348
0.2122
0,3057
0.4144
0.6258
0,3215
0.62
0.5761
0.5399
06008
0.5277
0.5336
0,5067
0.466]
04664
0/636
03302
04487
0.409
0,0237
0,6155
0.0912
0.000
0.345
0.027
Backswer
0.1894
0.100
0.183
0:7291
04435
0.3555
0.991
08718
0.244
0
0:912
09924
09866
09762
0911T
01013
0:9966
Campeign
04657
0.00
0,40
0.5013
0.435
Carose
Commanddinz
Cantieooogrphyy
0.122
06619
08551
06667
0.16
0.51
01519
01311
03537
0,5912
0.2094
0,099
0.1157
0.0066
0,594
0.0854
011771
03475
0.109
01515
0,000
01011
0.667
03751
01506
06779
0535
1155
11101
06111
01562
0.595
1194
0.793 (3554
0.533 0.685
01124 0.20
03612
0/401
05655
0.9164
0.7699
0.51
Cover
0.019
0.001
0.016 00062
111555
RESSS
01922
0110
1201
10011
(6631) 01948
00114
0.83
Fasli
06882
6.608
0.100
01015
0.5945
115178
09237
0882
(0807)
(8223 01922
07315
0.669
Foodd
0.389
0.4533
140055
033366
0,009
0102
0000
0300
055
1035
5550
0122 (551
0 03666
0,614
Glass
0.0995
0:111
5100
00912
0407
00263
0000
(1490
0:151 01900
0
0.167
Heputits
0.3201
0.5828
0.0082
6
0438
6200
02297
03577
0.081 0,222
0 02803
05627
02811
1000
0.000 0.0120
Jooospane
0.890)
01591
Lympho
03100
0.9401
09982
0.4491
(18611
0.30
06003
0626
1066
[006
03351
(32004
0005
Marmuapphy
04179
0.331
04466
0.4165
0.3134
0.318
0235
02044
0526
1109
0.5665
033555
0.366
0.5406
Must
0.166
0.761
0:476
0.6099
0,530
0.3015
02723
0279
07133
119115
0.536
(3753
03477
Slakk
00814
099117
0.5179
0092
00982
1
1
0032)
1
11899
Opalgis
0.0992
0,8599
[44863
0.0002
0.157
0.009
00555
61155
0.1419
02486
0001
0.1209
0. 0.1534
05555
0-205
0.955
Pidkmon
0.7959
0.514
18811
01722
(1220
D DMIT
8991
Perdipis
6:9622
0.7855
04145
12220
00229
0-944 0.255
0.266
6.708
0.507
05006
62265
2282
0.023 02280
02221
6744
Saiingg
0,9555
0591
019192
0.884
07/11
11365
Shunte
0:5884
0331
05000 019677
0.9171
1.3615
03316
10611
(0055 (7457)
09151
Spannaac
08116
0.7585
06764
0.8144 0.8902
8.1135
05521
07785
03350
036
0SSN
0.459
(053)
0038
03356
05415
0,5413
Sperch
0.077
0.097
00011 01077 0,0151 [1003
R0A11
0011
00153
116N
B0116
10031
01255
11008
Thyndd
0.8151
0.590
0788 08114 09054 11000
0713
09091
11911
110H
(1811
08139
0.6626
Veisbaal
0.1517
0.1130
[12003
0.138]
0,194 11001
01005
01429
01594
(145)
1122
0:301
(1980
02279
0,3554
Nooels
0.3166
03227
0.008
2
022H
(1221
00037
0946
WDBC
04543
06671
0.9833
0.7224
09:84
09165
11229
01861 0.909
09112
WPBC
0,300
0364
04257
0
0.5017
60705
0144
0.254]
0.017
0.443
Wise
03624
0.9917
01251
111555
0.34%
0.800
6.1416
0.855
0166
119778
01469
(1995
0.9115
0.756
Yeal
94800
0,4377
044865
0.4678
0,4554
0.4043
0.5411
0.455
0.463
0.459)
0,587
0.5455
0.5017
0.0944
0,4631
0.4855
0,5416
AUC
0,5532
0,3254
sasking
9.425
8,55
8.7675
9,3575
11.225
9,0075
8.5125
6.4875
1995
Win
2
3
21
Awarage ad
0
0.017
0
0.056
0.038
0.022
0,042
0:915
0:021
0,033
0,048
0.064
0.004
0.028
18

---

Published as a conference paper at ICLR 2025
Table 6: Comparison of AUC-ROC (1) results between bascline methods and DRL on 40 datasets.
[orsvs
TarpSViI
Ausilisondor
NerTralAn
NYTEAD|
0.7894
0.7154
0:3344
0.351
0.4967
0,6785
0,5463
0,7019
0:6242
0:3407
0.6992
0.6208
0.5821
0:9143
0.5006
Amaazo
0.5768
0.543
0.5379
0.5256
0.55%4
0.5157
0.567
0.611
0,509
Antoriiiia
0.7688
0.7953
0.7688
0.7584
0.3534
0.199
0,1233
0.784
0.676
0.58
0.79
05145
05114
0.7185
0.7742
Hacidos
0.8664
0.379
6.2344
6.7995
0/361
0.585
07143
0.851
09834
0.9399
60128
0.925
0.154
09965
0.961
00954
0.9985
Breast
0,993
0:7714
6.9577
0.938
D.779
0.965
0.700
0.9955
09117
0.7157
09458
05735
09275
03955
0.954
0.998
Campaie
0.702
0.7324
6,761 0.7707
007299
0.7555
0.5755
0.224
03011
0.7348
0433
04682
07241
0.7995
0.5902
0.3915
0.7744
Caddo
0.9644
0:8979
69562
60555
0922
0627
0:651
00013
0.567
08729
06812
0.734
05514
0.725
0.963
03355
0.562
0/923
0/419
67499
00224
02619
0.478
07117
09:74
16001
030
67188
BM
Coauu
0.7158
06554
0.71
63975
0.122
00227
07002
0.675
0.5115
04685
0,00
00715
0.714
Commaand
0.5003
0.8713
03759
01952
0.627
0.7042
07511
09173
010055
0.500
0.504
Fasll
0.5882
0.3705
0.4663
0.5587
(L500)
0.500
0.550
03071
0.019
0,5142
0.355
0.771
01554
01963
00621
08279
Pruul
0,9543
03225
0,9575
69445
0963
0.8015
03271
05335
0.0255
04888
03800
00100
01883
00055
08565
0.9514
Class
0.515
0:4441
01567
0.541
115771
06631
11965
06337
0,1741
04117
0335
05231
03355
0.000
Hepaliis
0.4055
0.4622
0,5386
08122
10225
06000
0000
0.521
0.5724
06554
02011
08822
05289
05522
R8122
Ingwe
0:2014
0.471
8.4407
166699
1533
02813
03776
08217
08121
01577
Jannopteee
0.5765
0.9177
03766
0.009
02041
9850
0.865
09653
0.003
02225
02725
0.954
Lyngtac
0.9177
01000
0286
0965
69727
0968
02566
0.977
Must
Mommaraphy
0:3266
0.9511
05151
09759
069201
06117
05534
0.9136
0,9712
Mink
0.5
0.917
00521
0.008
03649
00543
0.9952
Opellght
0.633
0-4862
8.966
0.5817
09133
0,5977
031T1
0.75
01521
01947
0.9317
0,9972
Pickitace
0,503
0.455
0.4206
06927
0.768
6355
0:2715
04488
0:226
016672
0.365
0.816
Pecdlgia
0,9635
0.990
09906
0.447
D9966
012255
0.774
0:3181
09659
05142
0.00
02013
03957
0,999
Ptraa
0:113
04723
0,611
6.115
06772
01981
00038
64133
0017
19:21
(1225
07022
00555
(UM]
Saellie
0.7291
02200
1000
0720
0.2244
07374
0280
0.966
0.333
Shunle
SpunBioc
0.5961
0.7882
0:3895
0.345
0.9583
0313
0,993
0.814
0358
0.688
0300
033305
0121
09963
0.7716
0,3842
05307
03994
03124
08031
02041
03134 05063
11911 11575
0.5551
0,5005
0,592
041299
Speech
0.3673
0.3561
0,3159
0,3558
013112
0.35%
03077
03633
0.3134
0,5005
0.489
02531
0.361 02453
0,5925
0,5821
Thymid
0.9855
0.0554
69916
11571
113811
04417
0,1573
0430
00511
0191
0595
0.8787
0.9911
Varsheal
0.2654
0.1171
0.217
0.1146
0-114
122005
0.3058
0.2362
0311)
02501
0515
05357
0.5347
068266
Soorlc
0.3166
06111
672887
02219
0:00
1181 08009
(USSI
0,7033
09259
0.95
09508
0.802
00419
0,9953
wYSc
will
0.7933
0.7195
0.7566
0,3607
013748
0.5011
0.34
01663
0.053
0.51g
038338
05533
062)
0.354
0,3505
0,9557
Wine
0.455
0.917
0.456
0/4667
06571
0,7433
0.3133
[U506]
0.5356
0.8622
0,5366
0.9353
6915
0.9944
0.5535
0,9222
Vesel
0.4083
0.4166
0.4511
0.4156
D/0095
0.1397
0.119
0.2645
04
0.5488
0,006
0:615
0.4789
0.1191
0.513
Average AUC--DCC
0.7181
0,772M
0.7667
0,2427
0/75%
01714
0,4167
0
0/7617
0.6477
0,7591
07737
0.7755
0.8185
0.8574
Average nanking
10.3
9.3
78
9.4
8.975
11,625
13:425
1065
8.9125
9.6525
12.5
5.115
9,015
8.1875
7,275
5,6125
2.45
Win
0
0
0
20
Avcragesal skl
0
0.012
0
0.047
0.034
0632
0.004
0.009
0.017
0.026
0.029
0.046
0.003
0.003
0.5
0.0
AUC
0.5
0.4
0.2
0.3
10002 100000
300303
1000
10203
150000
300000
32
1020
1000
Dims
Figure& 8: Model performance w.r.t. different sample size and dimensionality. We categorized all datasets into four intervals based on sample size: < 1000, 2 1000 & < 10000, > 10000 & S 100000, 2 100000 & 300000. For dimensionality, the intervals are defined as: < 25, 2 25 & S 50, > 50 & < 100, 2 100& & < 1000. We provide the average performance for each interval across different datasets.
19

---

Published as a conference paper at ICLR 2025
A.5 COMPUTATIONAL COST
We provide the runtime in seconds of DRL for the training and inference phase on a single GTX 3090 GPU, as shown in Table 7
Table 7: Runtime in seconds of DRL for the training and inference phase.
Ahalone
Amazon
Anmthyroid
Amhythmia
Backdoor
Campaign
Train Inference
2,4855
8.6840
6.5400 0.0248
0.3842
77.9249
0.3972
32.1423
1.8538
0.0095
0.0225
0.0027
0.1546
0.0660
0.0091
Cardiotocograhhy
Census
Comm Cummandd.crmm
Hepariiss
Train Inference
1.7553 0.0085
247.3347
0,9703
236.5932
1,0365
233.1834
0.3029
0,2812
0.5250
0.0106
0.3724
0.0109
0.3743
0.0022
0.0025
Imgsey
Tonosphere
Lympho
Marmmggrphyy
Optdigits
Train Inference
166713 0.0060
0.3324 0.0025
0.2868
8.6523
5,2673
2.5212
3.8596
0.2775
0,0022
0.0256
0.0145
0.0060
0.0127
0.0029
Pendigits
Pina
Satellite
Satimage-2
Shuttle
SpamBase
Speech
Thyroid
Train Inference
5.8137
0.7752
3.6661
4,9489
40.0011
2 2.6029
2.8119
3,5806
0.0103
0.0067
0.0172
0.0163
0.0929
0.0175
0.0128
0.0094
Vertebral
Vowels
WDBC
WPBC
Yeust
Train Inference
0.3003 0.0023
1.5365 0.0065
0,3250 0.0024
0.2854
0.3436
4.0054
0.2858
0.9532 0.0091
0.0024
0.0024
0,0080
0,0022
Table 8: Runtime seconds of DRL and other baseline methods for the training and inference phase averaged over all datasets.
GC3V34
L.C
PCA
BO00
DAUMM
KAPY
Neslod.
J.L.
K0.M
NPFAA
DKLL
Infvewie
1099
151533
0.016
83113
04044
1.609
0/666
71 77:89
4.796
11:095
1.8066
8001)
A.6 HYPER PARAMETER SENSITIVITY ANALYSIS
We incorporate the sensitivity analysis for the number of basis vectors, the loss weight Å1, A2, the number of training epochs, and the number of batch size in Fig.
A.7 REPRESENTATION WEIGHT OF WEIGHT LEARNER VISUALIZATION
The weight visualization in Fig. 10 shows that the separation constraint effectively diversifies the weights of normal representations, while the weights for anomalous representations differ signifi- cantly from those of normal representations.
A.8 ROBUSTNESS TO ANOMALY CONTAMINATION
Our method is implemented in a one-class classification setting where the training set consists en- tirely of normal data. However, real-world anomaly detection applications frequently encounter contaminated training sets, requiring models to be robust against small levels of dataset contami- nation. To analyse the robustness of DRL w.r.t anomaly contamination, in the case of anomaly contamination ratio of 0%, 1%, 2%, 3%, 4% and 5%, as shown in Fig. [ The results indicate that DRL exhibits greater stability and consistently outperforms other models, demonstrating its superior robustness to anomaly contamination.
conduct experiments
A.9 PERFORMANCE ON DIFFERENT TYPES OF ANOMALIES
While extensive public datasets are available for benchmarking, they often contain a mix of differ- ent types of anomalies Existing works (Han et al. 2022) have summarized four common types of anomalies and proposed methods for generating them. Following this framework, we conducted experiments generate synthetic anomalies based on the realistic Shuttle dataset and assess the
20

---

Published as a conference paper at ICLR 2025
0.80
AUC-PR
0.75
AUC:
0.70
AUC PR AUC-ROC
0.725
0.65
9
11
13
19 40 60 80100
0.06
0.2
0.4
0.6
0.9
(a) Number basis vectors
(b) Weight A for separation loss
0.850
0.84
0.79
0.800
0.69
0.775
AUC-ROC
0.750
0.59
AUC-PR
0.725
AUC-ROC
0.49
0.2
0.4
0.6
100
150
200
250
300
350
(c) Weight for alignment loss
Number of training epochs
0.850
0. 0.825
0. 0.800
AUC-PR AUC-ROC
0.775
0.750
0.700
128
1024
(c) Number of batch size
Figure Sensitivity analysis. The results are averaged over all datasets.
30
35
10
25
(a) Annthyroid
(b) Thyroid
Figure 10: Weight visualization of normal and anomalous representations. For each subfigure, the left 20 columns and right 20 columns correspond to the weights of normal and anomalous repre- sentations respectively. For each dataset, randomly select 20 normal samples and 20 anomalous samples for visualization.
1.00
0.3
3.0
0.66
4-
DIL
DA
0.50
-2- O41
3.0
MCV
16/4
84
030
4000
(a) Abalone
(b) Abalone
(c) Tonosphere
(d) Tonosphere
Figure 11: Model performance w.r.t. different contamination ratios (the percentage of anomalies in the training set).
21

---

Published as a conference paper at ICLR 2025
performance of DRL on these specific anomaly types. For details the generation process of the four types of anomalies, please refer to (Han et al., 2022). The four types of anomalies are as fol- lows: Local anomalies refer to the anomalies that are deviant from their local neighborhoods; Global anomalies are samples scattering throughout the entire space, while being far from the normal data distribution; Dependency anomalies represent samples that do not adhere to the correlation struc- ture among normal data; Clustered anomalies refer to groups of data with similar characteristics but significantly different from normal data. The results on these different anomaly types are provided in T The results show that DRL exhibits strong performance across all anomaly types.
Table 9: The AUC-PR results on different types of anomalies.
lforest
LOF
OCSVM
BCOD
NeuTralAD
JCL MCM
NPT-AD
DRL
Local
0.3494
0,5256
0.3847
0.4324
0.4247
0.3284
0,5103
0,5125
0.6742
Cluster
0.9888
0.9772
0.9909
0.9271
0.9995
Dependency
0.6333
0.325
0.5
0.625
0.1213
0,3333 0.9992
0.6429
0.673
0.6913
Global
0.9992
0.9453
0.9847
0.9996
0.999
A.
ADDITIONAL EXPERIMENT RESULTS
Table 10: The comparison between E [||w. Www, and E w,, - - separation loss over all datasets. Wn and w. denote the computed weights of normal and anomalous samples respectively. Pw.. = E [w,], represents the the center of normal weights.
12 under the influence of
abalone
amazoe annthyyoid arrhythmia backdoor bresstw campaign
cardio
Cardistacography
census
D,2550
0.1126
0.1717
0.5577
0.1916
0.1738
0.0057
0.3347
0.2055
0.0270
wio sepention
Hww
0,1411
0.1021
0.0744
0.3935
0.0935
0.0307
0.1836
0.1439
0.0274
Gap
0.1139
0.0105
0.0973
0.1642
0.0921
0.1431
0.0166
0.1511
0.9616
-0.0004
0.5647
0.3276
0.2891
0.2080
0.3035
0.2411
0.8362
w/ scporation
[ Mww. Gap
0.2316
0.1928
0.1859
0.4338
0.1678
0.0480
0.1964
0.2077
0.1739
0.1206
0.3331
0.1348
0.119
0.1827
0.1215
0.16
0.6114
0.1838
0,0672
0.7156
comm. crime cover
Fault
Fond
glass
Dleptiii
imgoeg
Sonosphere
lympho
mammography
[[ Jawey
0:3816
0.2053
0.1154
0.1011
0.1766
0.0853
0.0645
0.1679
0.3146
0.2532
w/o separation
0.0794
0.0337
0.1353
0.0235
0.1570
0.0737
0.0655
0.0676
0,0437
0.1768
Gap
0,3022
0.2416
0.0199
0.0778
0,0196
0.0116
0.1000
0,2709
0,0761
0,7966
0.3100
0.1681
0.1914
0.3291
0,1919
03310
0.1949
0.4917
0.4266
w/ sopuration
0.1136
0.668
0.14
01135
0.2307
0.1749
0.1422
0.0853
0.1512
20722
Gup
0.683
0.2472
0.0107
0.0779
0.1081
0.017
0.1738
0.1096
0,3405
0.2244
moist
musk
optóigiis
Parkinson pendigis
pima
saneliic
satimage22
shuttle
SpumBase
||ws
0.7646
0.2715
0.0612
0.4004
0.0537
0.4218
0.1488
000811
0.3905
1305
wio separttion 2.
0,6229
0.1382
0.0553
0.2336
0,0399
0,3810
0.1460
004493
0.1179
0.0542
Gap
0,1417
0.1333
0.0059
0.1668
0,0138
0.0408
0.028
0.438
0.2726
0.0764
||wa
0.7950
0.3669
0.0807
0,5022
0.1953
0.4723
0 0.1666
0.1601
0.801
O 0..845
soparation
[w
pww.
0.6390
0.1457
0.0558
0,2582
0.1527
0.4199
0.1672
00076
0.1448
Gap
0.156
0.2212
0.0144
0,252
0.0426
0.0524
0.0294
0.0225
0.3253
0.2785
sperch
thyroid
veriebral
vowels
WDBC
WPBC
w/c
Wilt
wine
[||wa
Mww, [15]
0.0422
0.2963
0.3867
0.0403
0,0769
0.0631
w/o separation 2
WW Pww
0.0333
0.0958
0.0504
0.1161
0,3755
0.0432
0.0300
0.0608
0.5375
0.0672
Gap
0.0089
0.2005
40,0188
-0.0125
0.311
-0,0229
0.2916
0.0039
0.2519
-0,0043
E J[w,
0,2306
0.4663
0.0991
0.1912
0.4423
0.2614
0.4883
0.2343
0.8883
0.2191
sopuration
[w
pww.
0.1303
0,1142
0.1472
0,089
0.0621
0.1037
0.1129
0.1091
Gup
0.1003
0.3521
0.0346
0.014
0.353
11993
0.3846
0.1114
0.3009
0.11
22

---

Published as a conference paper at ICLR 2025
abalone
amazon
annthyroid
arrhythmia
ba cacooor
breastw
cardio
Cardiotocographty
Comm.and.crimg
cover
fault
fraud
glass
Hepatitis
imgseg
ionosphere
lympho
aammegraphy
mnist
musk
optdigits
Parkinson
pendigits
pima
satellite
Salimage-2
shuttle
SpamBase
speech
thyroid
vertebral
vowels
WDBC
WPBC
wbc
Wilt
wine
yeast
Tay
hpp
hg
Figure 12: The T-SNE visualization of representation by feature extractor o and the reconstructed representation h by linear combination basis vectors wB over all datasets. h, and h, correspond to normal and anomalous representations respectively.
23

---

Published as a conference paper at ICLR 2025
Ieprommattinn of
Repeoritio dDN
Inntisaation
Tepnieteto ofEEL
Reomsortaioon of (O0
Eneresoraate OfCRL
Condee peceeccy
and
CPTY Pwtere
Septursian ai
of 20
Tall
arddgt
Nothon
Reptoratan
fasmanntaio CEERL WDOC
vattona
eeteory
Repriser DIBB DR.
Iepeorstan ADA
Figure 13: T-SNE visualization of normal and anomalous representations extracted by the feature extractor f. illustrating the impact of separation loss across all datasets. Each pair of subfigures cor- responds to the same dataset: for instance, the first and second subfigures in the first row correspond representations without and with separation loss, respectively.
24

---

Published as a conference paper at ICLR 2025
SeLace
STRCE
Nppoortttin
Repuariniio GOAD
Represartatoon MAP?
Inprussaaion 5010
Npprssmtation MCM
Ripresmiatio DfL
Eadion!
Ippoooitatinal
Pephsentaiin
Resresntiiion
Inpreseaaton CO1O
canpaigo
corli
Appesirtaiona
Reprosat Drprasatiiinn
Hipresentatte OE
comm
corm atda
cheer
Repustitaion
fxti
Dat
Fird
of WORR
Ay
Ingies
inggng
anoathere
Imnophere
oppoontatina of
Repmosntatin CORD
Nepressnation 5OND
Reprosntiiles MCM
brete
breto
of
Aspressttaionn haps
GOOD
Figure 14: T-SNE Visualization of different deep TAD models' latent representations across all datasets (the former 20 datasets). This figure serves as an extension of Fig. 1 with each group of four subfigures corresponding to the same dataset.
25

---

Published as a conference paper at ICLR 2025
meir
Tert
Ippnortttion of
Repmariaiion GOAD
Rezrwarras: Rgoreaartaionn BAP
INprinnTtion 5010
Represematie of
optsgts
9%
pendigtt
Repnsentaiin
salnopa22
vilmage
appesoration of
Reproticiod of
of WOOO
Pocrentttion CERAP
Hipreseenatee of
Tyyoed
Repasintaion
Apprisntaainn
Repreisntatee DED
verttr
vetebral
vouet
ongennittiinaa of
wac
WB
WR
of
Pepmssntatin GORD
Teansuntei
IepreseTatio DOAD
Represetaales MMCM
Regrrartttee DTL
GOND
Figure 15: T-SNE Visualization of different deep TAD models' latent representations across all datasets (the latter 20 datasets). This figure serves as an extension of Fig. | subfigures corresponding to the same dataset.
with each group of four
26

---

Published as a conference paper at ICLR 2025
campaign
Hopatitis
ionosphere
0.80
0.10
[iw, w..B1
0.40
Varil
0.10
0.12
E[]],; Me B]
El[ - Mull B3
0.10
Varil
0.05
Vantl
200
Epocls
sppors
Eoochs
lympho
mammography
SpamBase
0.50
0.40
0.30
0.30
lww, w..111
0.20
0.30
Vart
E[]]w, Me B)
Ef]l,, - Mall (
0.10
Varil ml
Varl]iwklls!
0.10
Epachs
thyroid
wbc
Wilt
0.50
0.40
0.30
[ws (allE]
E[[[w; Ma ())
0.15
Iww
0.20
Varil wil 2)
0.20
0.10
0.10
Ellww, - Mal ( Vanl]
0.10
0.00
100 Epochs
1
10C
200
Epochs
Epochs
[] W, Wa [12] and Var(|| l12) (l)2)) during training. The results could verify Proposition I in Section 3.2.2 and wa denote the computed weights of normal and anomalous samples respectively.
Figure 16: Progression of E
27

---

Published as a conference paper at ICLR 2025
Erwasdtr
Hapatiile
2.075
0.175
50232
0.1000
0,0250
20135
0.050
0.0.50
3.353
0.0000
1.022
0.00%
5,0002
0,0050
200
jonoushere
[ymppha
SparBess
0.100
0.1902
0.8200
4.1392
Mm
4.1300
0.0000
0.500
0,4200
5,0003
0.0400
1.023
0.0000
0,2200
0.002
0.0000
200
160
300 Epseh
Spode
Spochs
Cardittocggrshhy
2,0003
0/032
0.0600
4.0000
2,0000
0,0000
100
220
100
200
320
203
Anomaly
seperation
Anomaly
soparation
Anomally
segaration
Anomaly
separation
Figure 17: Visualization of test samples' anomaly score by DRL with and without separation loss as training progresses. We report the average anomaly score of normal and anomalous test samples respectively. The anomaly score is calculated by decomposition loss (Eq. 3).
28

---

Published as a conference paper at ICLR 2025
brcasoM
Hicpatiis
lonssp
Dacomsentonn sepersiinn Dacomssstion aecerator
Decempasiiio bus sena otter Dakompasiil bos axparaday
Cecumpos tin reperation Decompoo w/o suparatian
0.001
5A952
0.005
1A362
20332
0.0000
0.0005
0.0010
2,013
sio
5,0225
tympho
misk
vateilite
Dacomzasiton rezeratio
0.0000
Decompasionn loss
0,5200
Cecompot
3.023
0.0:15
0,0050
2,015
4A112
0,0250
2,0003
0.0023
200
100 Spachs
300 Epocts
Spoda
Spamflaee
satimage-2
speech
3,0003
Decomsnsitinn seperation Docoesontioon lis se
Decempastton tro sotto
0.0440
Decampos seperdton
2olom Bol los ed's aipaaooian
Decamppo too tos who iepusetiin
1.004
0.5020
0.0000
3,002
0.0000
0.0000
1.001
0.0000
0.0220
100
220
320
275
320
Figure 18: Effect of separation loss on decomposition loss (Eq. 3 convergence. We report the average decomposition loss of training samples. The dataset dimension is as follows, breastw: 9; Hepatitis: 19; ionosphere: 33; lympho: 18; musk: 166; satellite: 36; SpamBase: 57; satimage-2:
36; speech: 400.
29

---

Published as a conference paper at ICLR 2025
brcaste
Hicoattls
lonosp
2.002
bairing secarat
Taading om aiseparation
0.3200
Thainiing las wno axparaday
Training losm sesarstion
The ring loss IN siperation The ring bas w/o suparatian
0.002
0.3750
50902
0.0000
0.1500
5.002
2.002
0.0400
0,0750
1.002
0:0000
0,0000
5.000
ado
sdo
tympho
munic
vatwiliee
2,0903
0.3400
1.002
trakiing rucoratir
Training
0,2400
The
0.0000
5.000
0,2000
0.0000
0.3200
20991
0.0400
0.0800
5.000
0.0000
5.002
0.0000
200
160 Spachs
320
Spoch
Epects
Spamllane
satimage-2
speech
burning seperatio
Trennin tno separatio
spperdion
7,1007
Ycenig lis wis seserat
Treeaig las wdos sqparoiinn
0,3500
To rang fos wo jepuettt
2,0002
0.0250
0,3400
7,0802
0.0000
0.1200
5/602-
0,1100
1.003
0.0050
0,0900
0.0000
0,0000
2.003
100
320
310
275
320
Figure 19: Effect of separation loss on overall training loss (Eq. 6) convergence. We report the average overall training loss of training samples. The dataset dimension is as follows, breastw: 9; Hepatitis: 19; ionosphere: 33; lympho: 18; musk: 166; satellite: 36; SpamBase: 57; satimage-2:
36: speech: 400.
30

---

Published as a conference paper at ICLR 2025
Table 11: Full analysis on the effects of different components in DRL in terms of AUC-PR. This table serves as an extension of 2
DRL (ours)
Abalone
0.8764
0.8786
0.8804
0.7281
0.6711
0.8827
0,7974
0.8453
0.8573
0.7621
0.885
Amazon
0.1067
0.1079
0.1119
0.0139
0.1073
0.1125
0.1027
0.1160
0.1215
0.1096
0,1206
Annhyroid
0.4534
0.5666
0.6775
0.3213
0.2802
0,6702
0.4729
0.5326
0.5629
0.6260
0.6761
Arrhythmia
0.6261
0.6266
0.6267
0.474
0.4257
0.5967
0.5413
0.5952
0.627
0.5827
0.627
Backdoor
0.7504
0.8742
0.8845
0,5214
0.1093
0.841
0.5174
0.7487
0.8941
0.8787
0.8915
Breastw
0.9974
0.9975
0.9979
0,8959
0.9828
09980
0.9388
0.9939
0.9965
0.8966
0.9966
Campaign
0.4772
0,4995
0.5209
0.3968
0.1423
0.4875
0,4120
0,4742
0.4863
0.4264
0,5013
Cardio
0.8261
0.8671
0.8676
0.6247
0.8732
0,809
0.7663
0.8275
0.8322
0.7026
0.8325
Cordiotocorrahyy
0.7543
0.7543
0,755
0.6165
0.6128
0.7197
0.6678
0.7103
0,754
0.7147
0.754
Census
0.1667
0.2339
0.2731
0.1918
0.4062
0.1717
0.1927
0.2474
0.2642
0.1469
0.2649
Comm.and and.erime
0.917
0.9179
0.9183
0,7525
0.7524
0.89
0.8415
0.8957
0.917
08111
0,9164
Cover
0.8044
0,8264
0.8602
0.286
0.0107
0.8821
0.7110
0.7692
0.783
0.6998
0.783
Fault
0.6245
0.6372
0.6454
0.5143
0.5939
0.6425
0.5667
0.6227
0.6563
0.6059
0.6649
Fraud
0.0703
0,4723
0.6597
0.3105
0.2165
0.5905
0.3987
0.6608
0.7586
0.7773
0.6614
Glass
0.0959
0.1052
0.1242
0.1076
0.0972
0.0969
0.1068
0.1550
0.167
0.0792
0.167
Hepatitis
0.5056
0.5138
0.5533
0.4555
0.6289
0.6467
0.0159
0.6248
0.6186
0.6421
0.6627
Imgseg
0.9088
0.9153
0.9306
0.6685
0.6954
0.9161
0.6947
0.9079
0.9098
0.8318
0,9238
lonosphere
0.9804
0.9814
0.9627
0.9788
0.9039
0.9779
0.9225
0.9889
0.9909
0.1155
0.9895
Lympho
0.8179
0.8734
0.9762
0.8151
0.9151
0.9686
0.9970
0
Mammography
0.3237
0,5114
0.5135
0.1945
0.5104
0.8107
0.2743
0.3196
0.3344
0.6452
0.8406
Muist
0,882
0,8894
0.8894
0.6568
0.1092
0.7681
0.8167
0.8861
0.887
0.6484
0.887
Musk
0,9274
0.9533
0.8983
0.9637
0.9885
0.0327
Optdigits
0.6387
0.6405
0.6578
0.1194
0.0521
0.854
0.6032
0.6429
0.6459
0.7712
0,9356
Parkinson
0.92
0.9209
0.921
0.8221
0.9311
0.9199
0.7982
0.8347
0.921
0.9318
0.921
Pendigits
0.6647
0,2225
0,7704
0.7594
0.7102
0.7157
Pina
0.7448
0.7449
0.745
0.6105
0.6905
0.690M
0.6602
0.7049
0.7449
0.6053
0.7449
Satellite
0.8428
0.8521
0.8652
0.7484
0.7711
0,8489
0.7857
0.8061
0.8634
0.7856
0.8692
Satimage-2
0.2661
0,9621
0.9748
0,8812
0.9154
0,92293
0.9295
0.8725
0.9668
0,8340
0.9703
Shuttle
0,974
0.9796
0.9849
0.7824
0.9656
0.965
0.9113
0.9452
0.9623
0.8714
0,9819
SpamBase
0.8437
0.8486
0.8489
0.7133
0.8186
0.8157
0.7918
0.7865
0.8342
0.8401
0,8413
Specch
0.0482
0.055
0.0608
0.0321
0.0304
0.0466
0.0435
0.0455
0.0525
0.0245
0.0584
Thyaodd
0.7967
0.5174
0.4143
0.7357
0.8626
Vertebral
0.2216
0.2224
0.2242
0.1469
0.18
0.1726
0.2322
0.2460
0.2457
0.1689
0,2854
Vowels
0.3518
0,3993
0.4007
0,3411
0.1422
0.442
0.3429
0.3819
0.1964
0.3160
0.4506
WDBC
0.9
0.9909
0.9909
0.9894
0.9808
0.1167
WPBC
0.4126
0.4355
0.4696
0.338
0.4126
0.4068
0.4173
0.4439
0.3747
0.4574
0,5017
Whe
0.9405
0.9522
0.9527
0.8159
0.8531
0.8119
0.9256
0.9408
0.9742
0.111
0.9742
Wilt
0.4074
0.412
0.4226
0.195
0.3372
0.4207
0.4100
0.4295
0.4474
0.5401
0.4543
0.9909
0.9184
Yeast
0.4847
0.4864
0.4952
0.3882
0.5083
0.5073
0.3765
0.5064
0.5099
0.5240
0,5416
Average
0.6344
0.6844
0.7081
0.5222
0.5407
0.6964
0.6013
0.6750
0.69228
0.6100
0.7344
31

---

Published as a conference paper at ICLR 2025
Table 12: Full analysis the effects of different components in DRL in terms of AUC-ROC. This table serves as an extension of Table
B
D
E
11
DRL (ours)
Abulone
0.7575
0.7633
0.7638
0.6373
0.5582
0.7963
0.6050
0.5976
0.7833
0.7531
0.8086
Amazon
0.4891
0,5075
0.5171
0,4861
0.5497
0,5465
0.5466
0.5806
0.5227
0.5386
0,5699
Anathyroid
0.7381
0.8321
0.8927
0.7371
0.7211
0.8047
0.6219
0.6959
0.8805
0.7925
0,9234
Arthythmia
0.7339
0.734
0.7341
0.6754
0.5584
0.7342
0.8022
0.8093
0.7541
0.5867
0.7742
Backdoor
0.929
0.9291
0.9425
0.8507
0.718
0.8995
0.9185
0.9646
0.9787
0.9799
0,9788
Breastw
0.9574
0.9574
0.9579
0.8957
0.9818
0.998
0.9721
0.9665
0.9967
0.8413
0.9968
Campaign
0.7176
0.7637
0.7684
0.7114
0.7874
0.7336
0.6750
0.9199
0.7793
0.7247
0.7714
Cardio
0.9135
0.9263
0.9263
0.9712
0.9659
0.8734
0.9404
0.8676
0,9562
Cardiotocography
0.8
0.8001
0.8005
0.7529
0.6859
0.8167
0.6358
0.8225
0.82
0.8195
0,8400
Census
0.6084
0.6897
0.7691
0.5964
0.837
0.5856
0.6709
0.7615
0.7509
0.6210
0.7714
Comm.and.crime
0,8047
0.8055
0,8075
0.6332
0.5696
0.8028
0.8330
0.7512
0.8192
0.7436
0,8461
Cover
0.9479
0.9501
0.9502
0.8694
1528
0.9908
0.6772
0.6223
0.9604
0.9627
0,9804
Fault
0,5459
0.5505
0.5728
0-4811
0.5475
0.6273
0.5314
0.6017
0.6239
0.6079
0.6279
Fraud
0.8534
0,8545
0.8996
0.7345
0.9403
0.944
0.9031
0.9258
0.9102
0,9149
0.9314
Glass
0.5382
0.5425
0.5192
0.5858
0.7161
Hepatitis
0.6885
0.693
0.7881
0.6919
0.81
0.7089
0.5361
0.5353
0.7466
0.7276
0,8122
Imgseg
0,8516
0.8718
0.8722
0.7681
0.5608
0.8751
0.7869
0.8323
0,889
0.8636
0,9093
Tonosphere
0.9306
0.932
0.9353
0.8685
0.8838
0.9666
0.9506
0.9743
0.9274
0.8322
0,9840
Lympho
0.9483
0.9506
0.9577
0.893
0.993
0.9176
0.9470
0.9507
1,0000
Mammography
0.8124
0.8605
0.6967
0.8574
0.8999
0.8942
0.9034
0.7484
0.9455
Mnist
0.9295
0.9321
0.9321
0.8219
0.2673
0.875
0.6193
0.9161
0.9312
0.9164
0.9712
Musk
0.9546
0.9573
0.96
0.8999
0.8621
0.9818
0.6706
1.0000
Optdigits
0.9254
0,9302
0.9305
0,7737
0.4667
0.9487
0.7806
0.9948
0.9653
0.8000
0.9972
Parkinson
0.6247
0.6315
0.6318
0.5715
0.6976
0.6658
0.3728
0.4927
0.6532
0.7035
0.6732
Pendigits
0.9372
0.9392
0.9534
0.8031
0.9449
0.8833
0.9430
0.9694
0.9903
095777
0.9979
Pima
0.725
0.7251
0.7253
0.6204
0.6857
0.7243
0.6671
0.7506
0.7451
0.6231
0.7651
Satellite
0.7575
0.7653
0.7064
0.7569
Satimage 2
0.9038
0.9573
0.9573
0.8992
0.9789
0.9953
0.9629
0.9865
0.9955
0.9247
0.9968
Shuttle
0.9588
0.959
0.9594
0.8992
0.9919
0.9965
0.9951
0.9846
0.9993
0.9885
0.9992
SpamBase
0.7967
0.8035
0,8039
0.7543
0.8148
0. 8066
0.7621
0,8049
0.8118
0.8071
0.8289
Speech
0.5775
0.5804
0.5848
0.3215
0.4268
0.5867
0.4622
0.4830
0.5631
0.4541
0.5821
Thyroid
0.942
0.9425
0.9477
0.862
0.8319
0.9876
0.9522
0.9831
0,9832
0.9288
0.9911
Vertebral
0.4616
0.4708
0.4781
0.1381
0.4032
0.3637
0.2511
0.4726
0.5254
0.5540
0.6216
Vowels
0.8097
0.8131
0.7871
0.934
0.7652
0.7504
0.7562
0.8511
WDBC
0.96
0.96
0.96
0.9
0.9994
0.9994
0.9380
0.9466
0.8832
1.0000
WPBC
0.4956
0,4998
0.5023
0.4235
0.5451
0.5599
0.5022
0.5167
0.5064
0.5930
0.6198
Wbe
0.9504
0,9528
0.9531
0.8606
0.9713
0.9511
0.7803
0.9821
0.9763
0.8174
0.9963
Wilt
0.8794
0.8861
188866
0.6467
0.8752
0.9357
0.4442
0.3820
0. 0.8736
0.8827
0.9357
Wine
0.96
0.96
0.96
0.8983
0.9041
0.9597
0.8717
1.0000
Yeast
0.4109
0.4163
0.4206
0.3655
0.4712
0.5084
0.4451
0.3294
0.4706
0.5379
0.5120
Average
0.7882
0,7999
0.8123
0,7066
0.7240
0.8205
0.7273
0.7837
0.8286
0.7720
0.8574
32

---

Published as a conference paper at ICLR 2025
Table 13: Full analysis on the effects of different distance metrics in DRL in terms of AUC-PR. This table serves an extension of Table
Decompositio Casine distance
Separation distance
Alignment distance
Separation distance
Aligament distance
DRL (ours)
Abalone
0,8350
0.8792
0.8535
0.876
0.8631
0.885
0.09844
0.1094
0,1006
0.1028
0.1206
Annthyroid
0.6824
0.6551
0.6484
0.6494
0.6497
0.6761
Anhythmia
Backdoor
0.8808
0.8784
0. 8868
0.8786
0.8886
0.8915
Breastw
0.8851
0.9961
0.9957
0,9958
0.9943
0,9966
Campaign
0.4972
0.4344
0.4678
0.4609
0,4811
0.5013
Cardio
0.8253
0.8158
0.7921
0.8123
0.8209
D.8325
Cardiotocograply
0.7318
0.7458
Censas
0.2109
0.2358
0.2421
0.2204
0,2333
D.2649
Comm.and.crime
Cover
0.7663
0.7134
0.7793
0.7423
0,7613
0,783
Fault
0.6534
0.6391
0.6433
0.6444
0.6576
0,6649
Fraud
0.6097
0.6739
0.6551
0.6297
0.6444
0.6614
Glass
0.2792
0. 1.11
1.8787
0.1449
0.097
0.167
Hepatitis
0.6486
Imgseg
0.9036
0.9185
0. 9004
0.8998
0.9125
D. 0.9218
Tonosphere
0.9526
0.969
0.9895
Lympho
0.9868
0.8391
0.9762
0.8901
Mammography
0.8145
0.7645
0.8211
0,7972
0.8041
0.8406
Mnist
0.8579
0,8893
0.7912
0.8243
0.835
0.887
Musk
0.9705
0.8125
8484
0.9335
0.951
Opidigits
Parkinson
0.9175
0.9442
0. 9237
0.9476
0.9314
0.921
Pendigits
0.9029
0,8820
0.9218
0.909
0
Pima
0.6978
0.7398
0.7440
0.7486
0.7584
0.7449
Satellite
0.8426
0.8364
0.8212
0.8499
0.8099
0.8692
Satimage-2
0.9185
0.9174
969648
0.9345
0.9491
0.9703
Shuttle
0.9622
0.9318
0.9718
0.9674
0.9671
0.9819
SpamBase
0.8327
0.8629
0.8113
0.8852
0.8205
Speech
0.0545
0.0310
0.0342
0.0398
0.0408
0.0584
Thyroid
0.8561
0.8113
0.8210
0,8209
0.8299
0.8626
Vertebral
0,2312
0.2681
0,2542
0.2686
0,2781
0.2854
Vowels
0.4307
0.4400
0.3696
0.4635
0,4425
0.4506
WDBC
0.9954
0.9928
1.0000
0.93
0,9319
WPBC
0.4896
0.4911
0.4843
0,4984
0,4943
0.5017
0.94177
Wilt
0.4146
0.4384
0.4105
0,413
0,4238
0.4543
Wine
1.0000
1.0000
0.9714
Yeast
0.5308
0.5010
0.5049
0.5206
0.4817
0.5416
Average
0.7130
0.7007
0.7063
0.708
0.7134
0.7344
33

---

Published as a conference paper at ICLR 2025
Table 14: Full analysis on the effects of different distance metrics in DRL in terms of AUC-ROC. This table serves as an extension of Table
Decomposition Casine distance
Separation distance
Alignment distance
Separation distance
Aligament distance
DRL (ours)
Abalone
0.7934
0.7744
0.7610
0,7691
0.7748
0.8086
0.5525
0.5413
0.5649
0.5109
0.5258
0.5699
Annthyroid
0.88228
0.8342
0.8931
0.8672
0,9031
0.9239
Anhythmia
0.7667
0.7376
Backdoor
0.9607
0.9036
0. 0.9248
0.9339
0.9372
0.9788
Breastw
0.9838
0.9932
0.9958
0.9961
0,9944
0,9968
Campaign
0.7644
0.7423
0.7188
0.7229
0.736
0.7714
Cardio
0.9327
0.9110
0.9586
0.9139
0.9236
D. 0.9562
Cardiotocograply
0.8134
Censas
0.7659
0.69%7
0.7083
0.7169
0.7354
D.7714
Comm.and.crime
0.8346
Cover
0.9742
0.9417
0.9297
0.9394
0.952
D 0.9804
Fault
0.6495
0.5632
0.6057
0,5749
0.6028
0.6279
Fraud
0.9248
0.9243
0,9330
0.9362
0,9417
0.9314
Glass
0.6785
0.6729
0.6724
0.6887
0.6736
D. 0.6969
Hepatitis
Imgseg
0.9010
0.6974
0. 7939
0.8724
0.882
D. 9.99
Tonosphere
0.9794
9562
Lympho
0.9965
0.9507
0 9977
0.9005
Mammography
0.9473
0.8969
0.8556
0.8891
0.8753
0.9455
Mnist
0.9709
0.9686
0. 0.9417
0.9352
0,9464
0.9712
Musk
0.9921
0.9006
9.99908
0.9295
0.9619
Opidigits
0.9631
Parkinson
0.6688
0.7287
0.6738
0.7423
0.6984
0.6732
Pendigits
0.9712
0.9321
0.9136
0.9275
Pima
0.7607
0.7524
0.7505
0.7495
0,7773
0.7651
Satellite
0.8596
0.7916
0.7999
0,7952
0.8021
0.832
Satimage-2
0.9335
0.8948
9696
0.9191
0.988
0.9968
Shuttle
0.9779
0,9860
0. 9979
8803
0,9973
0.9992
SpamBase
0.8496
0.8717
0.8696
Speech
0.5123
0.4773
0.5173
0,5078
0.4901
0.5821
Thyroid
0.8935
0.9424
0.9856
0,9619
0.9879
0,9911
Vertebral
0.6276
0.55U5
0,5848
0.5546
0,5857
0.6216
Vowels
0.8009
0.7824
0.8255
0.7909
0.7999
0.8511
WDBC
0.9749
0.9732
1,0000
0.9542
0.9955
WPBC
0.6096
0.5325
0.5702
0,5438
0,5796
0.6198
0.9636
Wilt
0.5033
0.7463
0.8637
0.8795
0,8407
0.9357
Wine
1.0000
1.0000
0.9933
Yeast
0.5078
0.4622
0.4857
0.4602
0.4668
0.512
Average
0.8318
0.8012
0.8225
0.8109
0.8243
0.8574
34

---

Published as a conference paper at ICLR 2025
A.11 THEORETICAL ANALYSIS
Proposition 1 Let Wa E N and Wa E A denote the computed weights of normal and anomalous samples, where N and A denote the weight sets of normal and anomalous samples, respectively. Given w EEK, where ΣK represents the probability simplex TRKK, the expected discrepancy between normal and anomalous weights, SwW.NNWWEE. [[] Wa [2). can be amplified by in- creasing the variance of W among N.
Proof. The expected discrepancy between normal and anomalous weights is formulated as:
E [1 Wn. wall [1 =E [(w, w.) " (w,, Wa]]
E
w.
[
+ E [[wall]]
2E [w wo
=Var([| E² w,, [||]]
+ EE
[/waall
-2E [ w.]
1
Var([/ ) F.2
+ E
)²
2 [Iwallellwalll
VK
K
2
~var([w<xl12)
2E [ 1] Var(I
2.
K
K
where Var(-) denotes the variance, is probability simplex as discussed above and satisfies Ekm w = 1 and S w S 1, /|wl22 = (wokz2 Σ w
- is given by Arith-
VK Wa E [[l [|a|| walla] is given by Cauchy-Schwarz inequality (Bhatia & Davis, 1995). and || la
K
metic Geam-Gee Mean Inequality (Bhatia & Kittaneh, 2000 W.T.. S EER | Twll| S
arises from the condition about the simplex. The lower bound of E Ww, wa [ is given by Bar([)w..lle x 2. To increase this lower bound, can enhance
Let Wn N and Wa A denote the computed weights of normal and anoma- lous samples. where N and A denote the weight sets of normal and anomalous samples, respec- fively. Given w E >x where Σ represents the probability simplex in R denoting pww. Ew,EN EN [wn] as the centroid of the normal samples' weights, Ew,.EN w.EE [llwa pw., 12
Proposition 2
EW [llWn- Mww. [13].
can be amplified by increasing the variance of [[ wwll2 among N.
Proof. EwnEN,Wac. [[[Wa Hww 11.]
EWLEN [[|wn M.w 12] is formulated as:
E Wa Mw,,, 112 - E [l|wa Mw,, 1131
E W. pww, 112 || flw,, 11ā]
E [ (Wa (ww.)) ) ((Wa - (lw.)) (Wn - p.w.. (TWn - pww).
=I E [wT W. 2w T
a Mw.. + ww,, uw. (w] WwT.. wn, 2w lww. + pow, (ww..)
=E [wT Wa + w,,
T Wn 2w]] T Wn
2w T Mw.
2w pww.]
=E ||waa]] [12]
+ E [ww, x12]
2E
w
112] + 2E [w:]] Mw.
w Mw..
=E ||waal] ||2]
+ Var( Wn.l vr.l1z)
+ E
||2]
2E
[w,, [12]
+ 2E
[w"] M.....
2E (w Hww.
(8)
>Var( [| (12) + E [[all
+ E
[www
|2]
2E
[1
+ 2E
[wit H.w.. ]
2E (w'd Hww]]
1
~Var([Ww..ll)) 12) + TE
)2
+ E
2F
[||w.]ll2
wwwwlll [|]] [w.l||/[w,n,1
2TE w.allalluw...11
VK
K
1
1
2 Var([[ (12) + IE
+ E
2E
[1
1] [1-21[14 [1
VK
[K
=Var([[wall) [2))
K 4,
where Var(-) denotes the variance, |/w/|2 = Σx (w+
bkn wn VK VK is given by Arith-
metic Mean-Geometric Mean Inequality BBhatia & Kittaneh
2000). E w'' Mw,
E [[ Mww. <
E alllwalllll [1z] is given by Cauchy- Cauchy-Schwarz Inequality (hhaiia & Davis 1995). Uww Σ is
K
given by W, Σx and w/|22 & arises from the conditions
Ex_ w 1 and < w 1.
35

---

Published as a conference paper at ICLR 2025
Therefore the lower bound of Iw,, E.W. [ll.. Mww. [1 -.,,EN [|wn, Mww [1 Var(1 (n.ll)
is given by
KR 4. To increase this lower bound, enhance Var( (arl//w,ll2).
A.12 LIMITATION
The primary limitation of the current DRL. method is that it is designed specifically for tabular anomaly detection. Although our method can be used different data types for the anomaly detec- tion task by introducing representation decomposition, ture for the weight learner and alignment learner due to the difference between data types. In the future, plan to explore its application to other data types, where incorporating prior structural knowledge from these data types might be possible solution.
may need design the specific architec-
36


Reviews:
[{"review_id": "98lLcWKRG0", "reviewer": "Reviewer_hqNX", "summary": "This paper introduces Decomposed Representation Learning (DRL), a novel approach for addressing the entanglement issue in representation learning within reconstruction-based Tabular Anomaly Detection (TAD). DRL mitigates this problem by employing orthogonal bases in a training-free, data-free manner to remove latent space dependencies, implemented through the Gram-Schmidt process for Decomposition Loss. It also maximizes variance in the weights of normal latent bases, enhancing discrepancy with abnormal latents as supported by Proposition 1, and reconstructs the original input from latent representations to prevent task-irrelevant features. The paper validates DRL’s effectiveness with extensive experiments across 40 ADBench datasets and 15 baseline models, achieving state-of-the-art performance through detailed ablation studies and T-SNE visualizations.", "questions": "1. What would happen if the basis vectors were set as unit vectors, for example, [1,0,0,0],[0,1,0,0],[0,0,1,0],[0,0,0,1] ?\n2. The paper states that cosine distance improves performance as L2 distance leads to optimization issues. Could the authors explore how performance might change with L1 distance, or clarify why cosine distance was chosen specifically over both L2 and L1?\n3. Proposition 1 proposes that maximizing variance of weights attached to basis vectors in normal latents enhances separation from abnormal latents. Are there trade-offs in terms of model stability or convergence when applying this approach?\n4. How sensitive is DRL’s performance to hyperparameter choices, such as the number of orthogonal basis vectors or weights in loss terms? Are there specific recommendations or guidelines for tuning these parameters across different types of tabular datasets?\n5. What are the main limitations of DRL as identified by the authors?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The paper effectively addresses limitations in reconstruction-based methods by comparing DRL with recent state-of-the-art models such as NPT-AD and MCM, demonstrating its superior performance. This is a novel contribution to the field of tabular anomaly detection.\n2. Theoretical support for the separation loss, specifically Proposition 1, provides a solid basis for the model's structure and validates its design choices.\n3. Extensive experimentation across 40 ADBench datasets and 15 baseline models showcases the robustness and effectiveness of the DRL method, and ablation studies and T-SNE visualizations add valuable insights into its workings.", "weaknesses": "1. Lack of KNN Baseline Comparison: While the paper includes comparisons with several state-of-the-art methods, there is no performance comparison with KNN, a model often effective in tabular data tasks. Including KNN as a baseline would enhance the experimental section by providing insights into how DRL performs relative to a widely recognized tabular anomaly detection approach. I recommend the authors consider adding KNN results and explain how DRL’s design is particularly beneficial over KNN in anomaly detection.\n\n2. Loose Bound in Separation Loss Calculation: In calculating the separation loss lower bound, the authors remove all terms involving squared expectations, which might lead to a relatively loose bound. It would be helpful if the authors could explore alternative derivations for this bound that might be tighter or justify the sufficiency of the current approach. Additionally, it would be useful to discuss how this looseness could potentially impact performance or theoretical guarantees of the DRL method.\n\n3. Clarity in Section 2 and Lack of MCM Limitations: Some portions of the paper, particularly Section 2 (Preliminaries), could benefit from clearer explanations. For instance, a more detailed explanation of theta would aid readers in following the theoretical framework more easily. Additionally, while the authors discuss the disadvantages of NPT-AD, the limitations of MCM are not mentioned. Including a brief discussion on MCM's limitations would help balance the comparison and highlight DRL's unique advantages more clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vLfN6WYYYZ", "reviewer": "Reviewer_hbkx", "summary": "This paper proposes a new method for tabular anomaly detection under the one-class setting via learning decomposed representations of normal samples. The key idea is to decompose normal representations into a weighted linear combination of data-free and training-free orthogonal basis vectors. Furthermore, a separation loss, supported by theoretical analysis, is designed to enhance model’s discriminative ability between normal and anomaly samples. Extensive experiments conducted on 40 datasets demonstrate the effectiveness of the proposed method.", "questions": "see the weakness part.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1.\tThis paper is well-written and easy to read.\n2.\tThe proposed decomposed representation learning method sounds reasonable for tabular anomaly detection.\n3.\tThe authors provide a theoretical analysis to support the proposed separation loss.\n4.\tThe comparative experiments conducted on 40 tabular datasets are quite extensive.", "weaknesses": "1.\tI appreciate that this paper highlights the issue of normal and anomaly samples being entangled in raw space. However, the argument that normal and anomaly latent representations are entangled in reconstruction-based methods, leading to diminished discriminative power, seems somewhat unfounded. Since these models are biased towards reconstructing normal samples, it is expected that representations of the two classes would be entangled in latent space. Based on this, the motivation for learning decomposed normal representations is unclear to me. What are the specific advantages of learning decomposed representations for tabular anomaly detection?\n2.\tThe visualized t-SNE results demonstrate that the proposed method can extract non-entangled normal and anomaly representations. However, the authors may need to explain the rationale for leveraging orthogonal basis vectors to learn such representations. Could vectors with other relational properties achieve the same effect? What are the specific advantages of using orthogonal basis vectors for decomposed representation learning?\n3.\tI appreciate the authors’ efforts in conducting extensive ablation studies. However, what is the performance when using cosine distance for the decomposition loss? Additionally, how does the model perform if only alignment losses are used as anomaly scores during inference?\n4.\tThe sensitivity analysis of vector number is good. I wonder if the vector number is somehow related to the feature number of input data.\n5.\tI appreciate the theoretical support for the separation loss. However, as the separation loss is applied between samples, is it sensitive to the batch size? Additionally, if the training normal samples are highly similar to each other, does this loss face convergence challenges?\n6.\tCan the proposed method be applied to other data types (e.g., image or time series data)? In other words, why is the proposed method specifically useful for tabular data.\n7.\tThis is a minor point, but the paper lacks a discussion on limitations and future work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fs30aDfWkn", "reviewer": "Reviewer_Sgt1", "summary": "This paper introduces Decomposed Representation Learning (DRL), a new framework for anomaly detection in tabular data. This approach aims to overcome the limitations of traditional reconstruction-based methods, which often struggle with data entanglement, especially in tabular settings where feature heterogeneity can obscure the separation between normal and anomalous samples. DRL remaps data into a latent space, enforcing each normal sample's representation as a linear combination of orthogonal basis vectors that are both data-free and training-free. Furthermore, DRL introduces a constraint to amplify discrepancies between normal and anomalous patterns in the latent space.", "questions": "Please refer to weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. This paper is well-motivated, and the technical details are illustrated properly.\n2. The proposed method is supported by a theoretical analysis of the constraint to maximize the discrepancy between normal and anomalous samples.\n3. The authors conducted comprehensive experiments to demonstrate the effectiveness of the proposed DRL.", "weaknesses": "1. DRL relies on decomposing the representation of normal samples into linear combinations of orthogonal basis vectors, and this decomposition assumes that normal samples are well described by fixed orthogonal bases in the potential space. However, the true distribution of normal samples in complex, high-dimensional feature spaces may not be captured simply by a small number of basis vectors. The reviewer is concerned about whether the decomposition may not be sufficient to completely express the normal sample features if the distribution of high-dimensional data is complex or contains nonlinear feature associations, which in turn leads to inaccurate anomaly detection.\n\n2.  The decomposition and separation constraints of this method require the computation of unique weight vectors for each sample, which may incur significant computational costs on high-dimensional data and large-scale datasets. Also, the introduction of the weight learner increases the model complexity, especially when the number of orthogonal basis vectors is large. The reviewer is concerned about the efficiency of the proposed method. Although authors have shown the runtime in A.5, more comparison with other baselines in terms of runtime will improve the persuasiveness of this aspect.\n\n3. The reproducibility of this work is limited. The reviewer could not validate this work as the source code was not released.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "82b07kGRVG", "reviewer": "Reviewer_TJDb", "summary": "The authors present a Novel reconstruction-based outlier detection (R-OD) method for tabular data called DRL. DRL's novelty among other R-OD methods is that it focuses on representing the data $\\mathbf{x} \\in \\mathbb{R}^d$ as a linear combination of a randomly selected basis $\\mathcal{B}$ of $\\mathbb{R}^D$, with $d>D$. To this end, the network learns an encoder $f_{\\theta_f}$ from $\\mathbb{R}^d$ to $\\mathbb{R}^D$, a decoder $g_{\\theta_g}$ and a weight learning function $\\phi_{\\theta_\\phi}$ that is in charge of learning such weight representation. The authors also include a novel loss function to train DRL, focusing on 3 different aspects. (i) It learns an embedding function $f$ that agrees with a linear combination representation of the embedding $\\phi$  in the $L_{decomposition}$ loss, (ii) it focuses on separating the normal samples between themselves in the $L_\\text{separation}$ loss and (iii) it reconstructs the embedding by $f$ with $g$ in the $L_{aligment}$ loss. Furthermore, the authors include a theoretical result that motivates the use of $L_\\text{separation}$, by proving that an increase in the total $L_2$ distances of the representations of inlier data during training, leads to a greater expected distance between an inlier and an outlier (Proposition 1). \n\nAdditionally, the authors include an extensive list of real-world experiments in the main text, with 40 real-world datasets and 15 relevant competitors. In particular, the authors tested DRL's One-class classification (OCC) performance, provided an ablation study, and studied different types of distances and base selection strategies. The appendix includes further experiments with different types of synthetic outliers, sensibility to the parameters, robustness study and computational cost analysis among others.", "questions": "I kindly ask the authors to address the following questions & concerns in order to improve the manuscript:\n\n1. The performance of the method is not properly verified (see T.1,T.2,T.3). It will greatly improve the paper to address, particularly T.1 and T.2. For T.2, I suggest to prove that $\\|w_a - \\mu_{w_i} \\|$ has a lower bound greater than the increase in $\\|w_i\\|$, where $\\mu_{w_i}$ is the centroid of the inlier's representation in basis $\\mathcal{B}$.\n\n2. The authors should consider addressing concern E.1, to at least experimentally verify T.2. Is there any particular reason why not all datasets and competitors were considered (at the very least as an image in the appendix)?\n\n3. The authors should consider rewriting section 5.2. It is unclear to me the difference between the ablation study presented in the section and the *Comparison of DRL and Variants in the Observation Space*. For instance, why are they separate sections if they seem to try to study the same thing? Why is there no reference to the datasets used in *Comparison of DRL and Variants in the Observation Space*, and why only the average PRAUC and not the full table of results? Why did the authors consider only 7 datasets out of the total 40 in the ablation study?\n\n4. The author should strongly consider releasing the code for the method. Reproducibility is a crucial part of the scientific process, and without the code, this paper cannot be reproduced. Is there any strong reason as to why the code was not available at the time of the review?\n\nI will consider changing my rating upon successfully addressing these questions during the rebuttal. \n\n\n\n### Additional remarks that did not influence the score\nA couple of additional remarks that might improve the manuscript:\n\nR1. P-values are not a scalar metric. This means that a difference between .20 to .90 is the same as a difference between .0501 and .0502 (when selecting a critical value of 0.05) [Lavine]. Thus, authors should consider removing Figure 4 for something more meaningful (see [Campos et al] results section).\n\nR2. The Wilcoxon signed-rank test is a single-population comparison test. This means that it is strictly designed to work in single comparisons. The setting presented in the experiments is a multiple comparison setting, thus a multiple comparison test should be used in this regard (see, for example, the Conover-Iman test [Conover]). \n\n\n### References \n\n[Melia & Zhang] Meilă, M., & Zhang, H. (2023). Manifold learning: what, how, and why. https://arxiv.org/abs/2311.03757\n\n[Lavine] M. Lavine, “Introduction to Statistical Thought” \n\n[Campos et al] Campos, G.O., Zimek, A., Sander, J. et al. On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study. Data Min Knowl Disc 30, 891–927 (2016). https://doi.org/10.1007/s10618-015-0444-8\n\n[Conover] WJ Conover (1979), Multiple-Comparisons Procedures.\n\n\n\n### Update\n\nAfter the discussion with the authors, I have decided to increase my score from 3 to 6. See the comments below for more information.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper has a nice flow in the presentation.\n2. The idea of using a linear combination of a projected random basis to represent the data is an interesting idea with big explainability potential.\n3. The paper is well motivated, as the problem of OCC in tabular data is an important task in ML.\n4. There is a large list of different experiments.\n5. The authors use a statistical comparison test to provide a statistical significance analysis of the main experimental results.\n6. The authors include the pseudo-code of the method in the appendix.", "weaknesses": "There is not enough evidence to support the claims that the authors make in the paper, both theoretically and experimentally. \nParticularly:\n\n### Theory\nT.1. The authors claim that DRL \" (...) assumes that the representation of each normal sample in the latent space can be effectively modeled as a mixture of fixed basis vectors with specific mixture proportions\" (L160-161). This assumption is central to the method's idea, and it goes unsupported both in theory (no theoretical example of data behaving as such) and practice (no example proving that given such a theoretical example, one could extract exactly such representation). As an example, consider the manifold learning (ManL) literature. Assuming that data comes from a lower dimensional manifold $\\mathcal{M}$ is a big assumption, however, there are examples of such data being generated by synthetic means, and also examples of ManL methods properly learning the representations ---see figure 5 in [Meila & Zhang]. \n\nT.2. The authors further claim that Proposition 1 proves that the outliers are going to be far away from the inliers (i.e., separated). While technically true on average, Proposition 1 does not \"(...) amplify the discrepancy between the two (inlier and outlier) patterns\" L224. What proposition 1 shows is that, if one increases the variance of the learned weights $w_i$, the process of measuring the distance between an inlier's representation and an outlier's representation will be, on average, higher. This, however, does not imply that, necessarily, you will increase the distance of the outliers with the total set of inliers. For example, one could obtain a representation that places a large set of outliers in the centroid of the inliers, with the remaining inliers being sparse around the centroid.\n\nT.3. DRL's performance is not explained in the theoretical derivations. The authors focus on proving how the representation that they learn can \"separate\" outlier from inlier, but it is not clear how an increase in distance can affect the final scoring function (they use $L_\\text{decomposition}$ as a final score). Particularly, it is not clear to me how it can affect the encoder $f$.\n\n\n### Experiments\nWhile this paper contains a large collection of experiments, they do not focus on verifying the theoretical claims of the method.\n\nE.1. The authors include experiments to try to prove the claim mentioned in weakness number T.2 in Figure 1. However, out of the total list of datasets (40) and competitors (15), they only use 2 datasets and 1 competitor without any clear reason. This does not contain sufficient evidence that the outliers and the inliers can be separated by Proposition 1. \n\nE.2. Weakness T.1 addresses that the random basis reconstruction assumption has no example in theory. In practice, this assumption seems to not be properly explored. The authors include an ablation study in Table 2 where they compare different versions (including one employing only the basis reconstruction assumption) of DRL. However, they only included 7 out of the 40 datasets introduced earlier. \n\nAdditionally, the authors include in Figure 5 a comparison between different variants of DRL. Particularly, variant B \"applies the decomposition loss to observations, assuming that each normal sample can be decomposed into a set of orthogonal basis vectors (...)\" L472-473. This variant ranked third to last among 8 different versions of the method. Furthermore, it is not possible to compare this performance to the other detectors as the authors do not say which methods they use, and only report an average PR-AUC. \n\nE.3. At the time of this review, there is no code available for DRL, making it not reproducible.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)