Data-Efficient and Interpretable Tabular Anomaly Detection
Authors: Chun-Hao Chang
Jinsung Yoon
Sercan O Arik
Madeleine Udell
Tomas Pfister
Venue: Submitted to ICLR 2023 (2023)
Keywords: Anomaly Detection
Interpretability

Abstract: Anomaly detection (AD) plays an important role in numerous applications. In this paper, we focus on two understudied aspects of AD that are critical for integration into real-world applications. First, most AD methods cannot incorporate labeled data that are often available in practice in small quantities and can be crucial to achieve high accuracy. Second, most AD methods are not interpretable, a bottleneck that prevents stakeholders from understanding the reason behind the anomalies. In this paper, we propose a novel AD framework, DIAD, that adapts a white-box model class, Generalized Additive Models, to detect anomalies using a partial identification objective which naturally handles noisy or heterogeneous features. DIAD can incorporate a small amount of labeled data to further boost AD performances in semi-supervised settings. We demonstrate the superiority of DIAD compared to previous work in both unsupervised and semi-supervised settings on multiple datasets. We also present explainability capabilities of DIAD, on its rationale behind predicting certain samples as anomalies.

Under review as a conference paper at ICLR 2023
DATA DFFIIIEFI AND INTERPRETABLE TABULAR ANOMALY DETECTION
Anonymous authors Paper under double-blind review
ABSTRACT
Anomaly detection (AD) plays important role in numerous applications. In this paper, we focus on two understudied aspects of AD that are critical for inte- gration into real-world applications. First, most AD methods cannot incorporate labeled data that are often available in practice in small quantities and can be cru- cial to achieve high accuracy, Second, most AD methods are interpretable, a bottleneck that prevents stakeholders from understanding the reason behind the anomalies. In this paper, we propose a novel AD framework, DIAD, that adapts white-box model class, Generalized Additive Models, to detect anomalies using a partial identification objective which naturally handles noisy or heterogeneous features. DIAD can incorporate small amount of labeled data to further boost AD performances in semi-supervised settings. We demonstrate the superiority of DIAD compared to previous work in both unsupervised and semi-supervised set- tings on multiple datasets. We also present explainability capabilities of DIAD, its rationale behind predicting certain samples anomalies.
INTRODUCTION
Anomaly detection (AD) has numerous real-world applications, especially for tabular data, including detection of fraudulent transactions, intrusions related to cybersecurity, and adverse outcomes in healthcare. When the real-world tabular AD applications are considered, there are various challenges constituting a fundamental bottleneck for penetration of fully- aullmmeat machine learning solutions:
Noisy and irrelevant features: Tabular data often contain noisy or irrelevant features caused by measurement noise, outlier features and inconsistent units. Even change in a small subset of features may trigger anomaly identification.
Heterogeneous features: Unlike image text, tabular data features can have values with signif- icantly different types (numerical, boolean, categorical, and ordinal), ranges and distributions. Small labeled data: In many applications, often small portion of the labeled data is available. AD accuracy can be significantly boosted with the information from these labeled samples as they may contain crucial information
representative anomalies and help ignoring irrelevant ones. Interpretability: Without interpretable outputs, humans cannot understand the rationale behind anomaly predictions, that would enable more trust and actions improve the model performance. Verification of model accuracy is particularly challenging for high dimensional tabular data, as they are not easy to visualize for humans. An interpretable AD model should be able to identify important features used to predict anomalies. Conventional explainability methods like SHAP (Lundberg Lee. 2017) and LIME (Ribeiro al.] 2016) are proposed for supervised learning and not straightforward to generalize to unsupervised semi-supervised AD.
Conventional AD methods fail to address the above their performance often deteriorates with noisy features (Sec. they cannot incorporate labeled data, and cannot provide interpretabilityy
In this paper. we aim to address these challenges by proposing Data-efficient Interpretable AD framework, DIAD. DIAD's 's model architecture is inspired by Generalized Additive Models (GAMs) and G (see Sec. 3). that have been shown to obtain high accuracy and interpretability for tabular data (Caruana al., 2015: Chang al. 2021b Liu et al. 2021), and have been used in applications like finding outlier patterns and auditing fairness (Tan et al.) 2018). We propose to employ intuitive notions of Partial Identification (PID) as AD objective and learn them with a differentiable GA

---

Under review as a conference paper at ICLR 2023
Model (GA?M)
Test Data Prediction
Unlabeled
Labeled data
16
Score
21
data
2
AB
1.2
GA'M Interpretability
Training with
Fine- tuning AUC loss
Anommaly score
anomaly
Optional If iobeted dota extsty
F1 (e.g. cell size)
Overview of the proposed DIAD framework. During training, first unsupervised AD model fitted employing interpretable GA²M models and PID loss with unlabeled data. Then, the trained unsupervised model is fined-tuned with a small amount of labeled data using differentiable AUC loss. At inference, both the anomaly score and explanations are provided, based on the visual- izations of top contributing features. The example sample in the figure shown have an anomaly score, explained by the cell size feature having high value.
Figure 1:
(NodeGA M,, Chang al. a0) Our design is based on the principle that PID scales high- dimensional features and handles heterogeneous features well, while the differentiable GAM allows fine-tuning with labeled data and retain interpretability. addition, PID requires clear-cut thresholds like trees which are provided by NodeGA While combining PID with NodeGA M, we introduce multiple methodological innovations, such as estimating and normalizing sparsity metric as the anomaly scores, integrating regularization for an inductive bias appropriate for AD, and using deep representation learning via fine-tuning with a differentiable AUC loss. The latter is crucial to take advantage of small amount of labeled samples well and constitutes more 'data-efficient" method compared to other AD approaches e.g. DIAD improves from 87.1% to 89.4% AUC with 5 labeled anomalies compared to unsupervised AD. Overall, our innovations lead to strong empirical results DIAD outperforms other alternatives significantly, both in unsupervised and semi-supervised settings. DIAD's outperformance especially prominent large-scale datasets containing heterogeneous features with complex relationships between them. In addition to accuracy gains, DIAD also provides rationale on why an example is classified as anomalous using the GA GA graphs, and insights on the impact of labeled data on the decision boundary, a novel explainability capability that provides both local and global understanding on the AD tasks.
Table Comparison of AD approaches.
Unlabeled data Noisy features Heterogenous features Labeled data Interpretability
PIDForest DAGMM
X
x
GOAD
Deep SAD
SCAD
V
X
X
DevNet
X
V
X
DIAD (Ours)
RELATED WORK
Overview of AD methods: Table summarizes representative AD works and compares DIAD. AD methods for training with only normal data have been widely studied (Pang & Aggarwal 2021
Isolation Forest (IF) (Liu et al. 2008) grows decision trees randomly the shallower the tree depth for a sample is, the more anomalous it is predicted. However, it shows performance degradation when feature dimensionality increases. Robust Random Cut Forest (RRCF, (Guha et al.. 2016)) further improves IF by choosing features to split based on the range, but is sensitive to scale. PID- Forest (Gopalan et al.) 2019) zooms on the features with large variances, for more robustness to noisy irrelevant features. There are also AD methods based generative approaches, that learn to reconstruct input features, and use the error of reconstructions or density to identify anomalies. Bergmann et al. (2019) employs auto-encoders for image data. DAGMM (Zong et al. 2018) first
2

---

Under review as a conference paper at ICLR 2023
learns auto-encoder and then uses Gaussian Mixture Model to estimate the density in the low- dimensional latent space. Since these are based on reconstructing input features, they may not be directly adapted to high-dimensional tabular data with noisy and heterogeneous features. Re- cently, methods with pseudo-tasks have been proposed as well, A major one is to predict geometric transformations (Golan & Yaniv 2018, |Bergman & Hoshen] 20T9) and using prediction errors to detect anomalies. QQu et al. (2021) shows improvements with a set of diverse transformations. CutPaste (Li et al.) 2021) learns to classify images with replaced patches, combined with density estimation in the latent space. Lastly, several recent works focus on contrastive learning. [aac al. 2020) learns to distinguish synthetic images from the original. [Sohn al. (202I) first learns cerrbbutioonaugmenttee contrastive representation and then uses one-class classifier to identify anomalies. Self-Contrastive Anomaly Detection (SCAD) (Shenkar Wolf, 2022) aims to distin- guish in-window vs. out-of-window features by sliding window and utilizes the error to identify anomalies.
Explainable AD: A few AD works focus on explainability as overviewed Pang Aggarwal (2211a). Vinh et al. (2016); Liu et al. (2020) explains anomalies using off-the-shelf detectors that might come with limitations as they are not fully designed for the AD task. [Lizneski et al. (202T) proposes identifying anomalies with one-class classifier (OCC) with an architecture such that each output unit corresponds to a receptive field in the input image. Kauffmann et al. (2020) also uses an OCC network but instead utilizes a saliency method for visualizations. These approaches show the parts of images that lead to anomalies, however, their applicability is limited to image data, and they can not provide meaningful global explanations GAMs.
Semi-supervised AD: Several works have been proposed for semi-supervised AD. Das et al. (2017), similar to ours, proposes a two-stage approach that first learns an on unlabeled data, and then updates the leaf weights of IF using labeled data. This approach structure learned in the first stage, which we show to be crucial for high performance (Sec. 6.4). Deep SAD (Ruff et al. 2019) extends deep OCC DSVDD (Ruff et al., 2018) to semi-supervised setting. However, this approach is not interpretable and underperforms unsupervised OCC-SVM on tabular data in their paper while DIAD outperforms as regression problem and achieves better sample complexity with limited labeled data. Yoon et al. (2020b) trains embeddings in self-supervised way (Kenton & Toutanova 2019) with consistency loss (Sohn et al.; 2020) and achieves state-of-the-ar semi-supervised learning accuracy on tabular data.
not update the tree
DevNet (Pang et al. (019b) formulates AD
3 PRELIMINARIES: GA AND NODEGA'M
We first overview the NodeGA 2M model that we adopt in our framework, DIAD.
GAMs and GA Ms are designed to be interpretable with their functional forms only fo- cusing on the 1st or 2nd order feature interactions and foregoing any 3rd-order or higher interactions. Specifically, given an input x E IR, label y, a link function (e.g- g is log(p/1 p)) in binary clas- sification), the main effects for j(th) feature fj. and 2-way feature interactions fiz,, the GA M models expressed
G:
GA G g(v)= fo Σ, f +,,
for Enn
(1)
Unlike other high capacity models like DNNs that utilize all feature interactions, GA are re- stricted to only lower-order, 2-way, interactions, so the impact of fy or fag can be visualized inde- pendently as a 1-D line plot and 2-D heatmap, providing a convenient way to gain insights behind the rationale of the model. On many real-world datasets, they can yield competitive accuracy, while providing simple explanations for humans to understand the rationale behind the model's decisions.
NodeGA M:
NodeGA P (Chang et al. 2021a) is a differentiable extension of GA which uses the neural trees to learn feature functions f and fyy- Specifically, NodeGA M consists of L layers where each layer has m differentiable oblivious decision trees (ÓDT) whose outputs are combined with weighted superposition, yielding the model's 's final output. An ODT functions decision tree
3

---

Under review as a conference paper at ICLR 2023
with all nodes at the same depth sharing the same input features and thresholds, enabling parallel computation and better scaling. Specifically, an ODT of depth C' compares chosen C' input features to C' thresholds, and returns one of the 2 possible options. F chooses what features to split, thresholds bº, and leaf weights W E RR².. and its tree outputs h(x)) are given as:
(HO (x) 14) @ (T( Fz(())
[(FF(()) I(CC F (
h(z)= = -w
(2)
where II is an indicator (step) function, @ is the outer product and is the inner product. To make ODT differentiable and in GA form, Chang et al. (2021a) replaces the non-differentiable opera- tions 5e and II with differentiable relaxations via softmax and sigmoid-like functions. Each tree is allowed to interact with at most two features so there are no third- or higher-order interactions in the model. We provide more details in Appendix. B
PARTIAL IDENTIFICATION AND SPARSITY AS THE ANOMALY SCORE
AD objective given its benefits in minimizing the adversarial impact of noisy and heterogeneous features (e.g. mixture of multiple discrete and continuous types), particularly for tree-based models. By way of motivation, consider the data for all patients admitted to ICU - we might treat patients with blood pressure (BP) of 300 as anomalous, since very few people have more than 300 and the BP of 300 would be in such "sparse" feature space.
We consider the Partial Identification (PID) Gopalan et al. 2019)
To formalize this intuition, we first introduce the concept of 'volume". We consider the maximum and minimum value of each feature value and define the volume of a tree leaf as the product of the proportion of the splits within the minimum and maximum value. For example, assuming the maximum value of BP is 400 and minimum value is O, the tree split of 'BP 2 300' has volume 0.25. We define the sparsity 81 of tree leaf 1 as the ratio between the volume of the leaf Vi and the ratio of data in the leaf Di as 81 = Vi/D,. Correspondingly, we propose treating higher sparsity more anomalous- let's assume only less than 0.1% patients having values more than 300 and the volume of 'BP > 300 being 0.25, then the anomalous level of such patient is the sparsity 0.25/0. 22.00.1. To learn effectively splitting
regions with high vs. low sparsity i.e. high v.s. low anomalousness, PIDForest (Gopalan et al. 2019) employs random forest with each tree maximizing the variance of sparsity across tree leafs to splits the space into a high (anomalous) and a low (normal) sparsity regions. Note that the expected sparsity weighted by the number of data samples in each leaf by definition is 1. Given each tree leaf I, the ratio of data in the leaf is Dj, sparsity 81:
V1
B[s] = Σ [Dit]] = Σ, (D DI
Σ,M
(3)
Therefore, maximizing the variance becomes equivalent to maximizing the second moment, as the first moment is 1:
max Var[s] max Σ, Disi
max Σ, V2/2D.
(4)
5 DIAD FRAMEWORK
In DIAD framework, we propose optimizing the tree structures of NodeGAPM by gradients to max- imize the PID objective the variance of sparsity - meanwhile setting the leaf weights W in Eq. 2
as the sparsity of each leaf, so the final output is the sum of all sparsity values (anomalous levels)
across trees. We overview the DIAD in Alg. [
Details of DIAD framework are described below.
Estimating PID The PID objective is based on estimating the ratio of volume Vi and the ratio of data Dj for each leaf I. However, exact calculation of the volume is not trivial in an efficient way for an oblivious decision tree as it requires complex rules extractions. Instcad, we sample random points, uniformly in the input space, and count the number of the points that end up at each tree leaf as the empirical mean. More points in a leaf would indicate a larger volume. To avoid the zero count in the denominator, we employ Laplacian smoothing, adding constant ő to each count ! Similarly, we estimate D, by counting the data ratio in each batch. We add 5 to both Vi and D,-
It is empirically observed to be important to set large 6, around 50- 50-100, to encourage the models ignoring the tree leaves with fewer counts.
4

---

Under review as a conference paper at ICLR 2023
Normalizing sparsity The sparsity and thus the trees' outputs can have very large values up to 100s and can create challenges to gradient optimizations for the downstream layers of trees, and thus inferior performance in semi-supervised setting (Sec. 6.4). To address this, similar to batch normalization, we propose linearly scaling the estimated sparsity to be in [-1, 1] to normalize the tree outputs. We note that the linear sealing still preserves the ranking of the examples the final score is a sum operation across all sparsity. Specifically, for each leaf I, the sparsity st is:
81 D1,
28t/ (max 81 min 8t)
(5)
Temperature annealing We observe that the soft relaxation approach for tree splits in NodeGA EntMoid (which replace II in Eq. ( does not perform well with the PID objective. We attribute this to Entmoid (similar to Sigmoid) being too smooth, yielding the resulting value similar across splits. Thus, we propose to make the split gradually from soft to hard operation during optimization:
Entmoid (z: + as optimization goes by linearly decresing T P-0 0
66)
Updating leafs' weight When updating the leaf weights W in Eq. 2 in each step to be sparsity, to stabilize its noisy estima- tion due to mini-batch and random sam- pling, we apply damping to the updates. Specifically, given the step sparsity sf for each leaf I, and the update rate 7 (we
Algorithm 1 DIAD training
Input: Mini-batch X, tree model M, smoothing 8, wi is an entry of the leaf weights matrix W (Eq. for each tree and leaf I
X = MrarMMMxxTaasooo(X.. min=-1, max=1) X U[-1, (Data uniformly from [-1, 1]}
use 7 = 0.1):
w/ (1 7)mm (6-1) + 78f-
(7)
Ed M(X), EB _M(v)) {Count how many data in each leaf of tree for X, Xu. See Alg. 2
Regularization To encourage diverse trees, we introduce novel regularization approach: per-tree dropout noise on the
Eti = Et + 8, Ey
Et + 5 (Smooth the counts}
Vu En
ge
(Volume ratio}
estimated momentum. We further restrict each tree to only split on p% of features randomly to promote diverse trees (see
Du Ene EV
(Data ratio)
Appendix. for details).
Ml = {Second moments}
V
P/r
Lm = -E,e Mn (Maximize the second moments)
Incorporating labeled data At the sec-
Set Va/ Pu {Sparsity)
ond stage of fine-tuning using labeled
280 Se1 [max Eel minn 1) (Scale to [-1, 1] (Eq. 5)
data, optimize the differentiable AUC
Wel (1 (1-7)uu + ysu {Update tree weights Eq. Optimize LM by Adam optimizer
loss (Yan et al.] 2003 Das et al., (017) which has been shown effective in imbal- anced data setting. Note that we optimize
both the tree structures and leaf weights of the DIAD. Specifically, given a randomly-sampled mini- batch of labeled positive/negative samples Xp/XN, and the model M,, the objective is:
LrN 1/|Xr||XN Σ
Expexx,axxxx max(M(a..) _Mzx),,))
We show the benefit of this AUC loss compared to Binary Cross Entropy (BCE) in Sec. 6.4
Training data sampling Similar to Pang et al. (2019)), we upsample the positive samples such that they have similar count with the negative samples, each batch. We show the benefit of this over uniform sampling (see Sec. 6.4).
Theoretical result DIAD prioritizes splitting informative features rather than noise (Appendix
Given uniform noise Fn, and non con-unnform uniformfeatures v DIAD prioritizes cutting za over Fa because the variance of sparsity of va is larger than In as sample size goes infinity.
Proposition 1
6 EXPERIMENTS
We evaluate DIAD on various datasets, in both unsupervised and sumi-super settings. Detailed settings and additional results are provided in the Appendix. Code will be released upon acceptance.
5

---

Under review as a conference paper at ICLR 2023
6.1 UNSUPERVISED ANOMALY DETECTION
We compare methods on 20 tabular datasets, including 14 datasets from Gopalan et al. (2019) and 6 We run and average results with different random seeds.
larger datasets from Pang et al. 2001 9a)
Baselines We compare DIAD with SCAD (Shenkar & Wolf 2022), a pecently-poposee deep learning based AD method, and other competitive methods including PIDForest (Gopalan et al.] 2019). COPOD (Li et al. PCA. k-nearest neighbors (kNN), RRCF (Guha et al. 2016 LOF (Breunig et al.] 2000)) and OC- SVM (Schölkopf al., 2001). To summarize perfor- mance across multiple datasets, we consider the averaged AUC (the higher, the better), as well as the average rank (the lower, the better) to avoid a few datasets dominating the results.
DIAD
#IFercst
2020
COPOD
0.25
0.00
-025
RRCF-
-050
LGF
-0.5
-100
RDForrst
coPOD
RCA
SQAD
Lor
Sable2shows that DIAD's performance is better or par with others on most datasets. Compared to the PIDFor- est which has similar objectives, DIAD often underper- forms on smaller datasets such as on Musk and Thyroid, but outperforms on larger datasets such as on Backdoor, Celeba, Census and Donors. To analyze the similarity of performances, Fig. shows the Spearman correlation be- tween rankings. DIAD correlated with SCAD as they both perform better larger datasets, attributed better utilizing deep representation learning. PIDForest under- performs larger datasets, and its correlation with DIAD low despite having similar objectives.
Figure 2: The Spearman correlation of methods' performance rankings. DIAD is correlated with SCAD as they both perform better in larger datasets. PID- Forest underperforms on larger datasets, and its correlation with DIAD low, despite having similar objectives.
Table 2: Unsupervised AD performance (% of AUC) 18 datasets for DIAD and 9 baselines. Met- rics with standard error overlapped with the best number are bolded. Methods without randomness don't have standard error. We show the number of samples (N) and the number of features (P), ordered by N.
DIAD PIDForest
GIF
IF
COPOD
PCA
SCAD
KNN RRCF
LOF OC-SVM
N
Vowels
78.3
74.0 in 79.0
74.9
49.6
60.6
90.8
97.5
80.8 1ss
5,7
77.8
IK
12
Siesmic
72.2
73.0
53.3
70.7
72.7
68.2
65.3
74.0
69.7
44.7
60.1
3K
Musk 90.8
100.0
121
100.0 1.
94.6
100.0
93.3
57.3
Satimage
99.7
98.2
98.9
9
97.4
97.7
98.0
93.6
99.2
46.0
42.1
6K
36
Thyroid 26.1
88.2=0s
57.6
81.4 n9
77.6
67.3
75.9
75.1
74.0
26.3
54.7
7K
as
81.4=os
56.4 +s
78.6 66
78.0
79,2
79.3
63.4
=0.
43.7
67.0
NYC n7
57.2
49.0
56.4
$1.1
64.5
69.7 54.4
32.9
50.0
10K
Mammo. 03 84.8
$2.5
90.5
88.6
69.8
83.9
83.2
28.0
87.2
11K
CPU 91.9
93.2 Lai
78.1 1ine
91.6
93.9
85,8
87.5 19
72.4
78.6
44.0
79.4
18K
81
81,6
73,9
82.7
80.9
83.4
81.8
74.7
79.6
23K
Campaign
11
78.6
64.1
7..4
78.3
73.4
72.0
72.0
65.5 13
46.3
66.7
41K
62
smtp
86,8+
91.9 =01
76.7
90.5
91.2
82,3
82.2 89.5
88,9
9.5
84.1
95K
3
Backdoor
91.1
74.2
68.9
44.8
88.7
91.8
75.4
28.6
86.1
95K
Celeba
77.2
67.1
61.6
70
75.1
78.6
75.4
56.7
61.7
56.3
68.5
203K
Frand
95.7 1na
94.7
10.4
94.8
94.7
95.2
95.5
93.4
87.5
52.5
94.8
285K 29
Census
65.6
=3.4=11
58.8 235
61.9
67.4
66.1
58.4
64.6
45.0
53.4
299K 500
http
19.
99.2 _a
91.1 1.
1ap
99.6
567K
Donors 87.7 61.1
80.3
7
81.5
82.9
65.5 +
61.2
64.1
=en
40.2
70.2
619K 10
Average 82.5
80.7
71.2
81.2
81.0
80,5
80.3
70.6
76.8
40.2
71.0
Rank
3.6
4.4
6.3
4.0
4.2
4.2
4.7
6.6
6.7
9.8
6.8
Next, we show the robustness of AD methods with additional noisy features. We follow the experimental settings from Gooalan al. (2019) include 50 additional noisy features which are randomly sampled from 1, on Thyroid and Mammography datasets, and their noisy ver- sions. Table. shows that the performance of DIAD more robust with additional noisy fea-
We did not use all 30 datasets in ODDS used in SCAD (hhnnarr & Wolf 2022 because some small overlap with datasets from (Gopalan 2019

---

Under review as a conference paper at ICLR 2023
tures (76.1~771J) 85.0-+83.1 while others show significant performance degradation. On Thyroid (noise), SCAD decreases from 75.9 -49.5, KNN from 75.1->49.5, and COPOD from 77.6-+60.5
Table 3: Unsupervised AD performance (% of AUC) with additional 50 noisy features for DIAD and 9 baselines. We find both DIAD and OC-SVM deteriorate around 2-3% while other methods deteriorate 7-17% on average.
DIAD PIDForest
GIF
COPOD PCA
SCAD
KNN
RRCF
LOF OC-SVM
Thyruid
76.1
88.2 2
27.6
81,4
67.3
212
74.0=05
26.3
Thyroid (noise)
71.1
76.0
49.4
64,4
60.5
61.4
49.5
136
49.5
53.6
50.8
49.4
Mammography
85.0
84.8 ams
82,5
85.7 1os
90.5
88.6
69.8
117
83,9
83.2
28.0
87.2
Mammography (noise)
83.1 =44
82.0
=2.7=14
71.4
72.4
76.8
69.4
81.7
79.1 =
37.2
87.2
Average
3.5
7.5
9.1
15.6
17.6
8,9
13.4
13.9
12.2
-16.8
2.7
6.2 SEMI-SUPERVISED ANOMALY DETECTION
Next, we focus on the semi-supervised setting and show DIAD can take advantage of small amount of labeled data in superior way.
We divide the data into 64%-16%% -20% train- val-test splits and within the training set, we consider that only a small part of data is la- beled. We assume the existence of labels for small subset of the training set (5, 15, 30, 60 or 120 positives and the corresponding negatives to have the same anomaly ratio).
Table 4: Performance in semi-supervised AD set- ting. We show the average % of AUC across 15 datasets with varying number of anomalies.
No. Anomalies
15
30
60
120
DIAD
87.1 89.4 90.0 90.4
89.4 91.0
DIAD w/o PT
86.2 87.6 88.3 87.2 88.8
CST
85.3 86.5 87.1 86.6 88.8
The validation set is used for model selection and we report the averaged performances eval- uated on 10 disjoint data splits. We compare
DevNet
83.0 84.8 85.4 83.9 85.4
to 3 baselines: (1) DIAD w/o PT: optimized with labeled data without the first AD pre-training
stage, (2) CST: VIME with consistency loss (Yoon et al.] (220aa) which regularizes the model to
make similar predictions between unlabeled data under dropout noise injection, (3) DevNet (Pang
et al. 2019a): a state-of-the-art semi-supervised AD approach. Further details are provided in Ap-
pendix
1.2
(a) Vowels
(b) Satimage
(c) Thyroid
(d) NYC
1.90
3.0
AUC
3330
1300
2875
1550
Drutk
380
2850
0.5
122
No. Anomalies
No. Anomalics
No. Anomalies
No. Anomalics
(e) CPU
(f) Campaign
(g) Backdoor
(h) Donors
1.52
AVC
235
0.000
019
235
370
272
62
AM
25 No. Anomalies
45
41
164
6
45
201
131
25 No. Anomalies
45 120
No. Anomalies
No. Anomalies
Figure 3: Semi-supervised AD performance 8 tabular datasets (out of 15) with varying number of anomalies. Our method 'DIAD' (blue) outperforms other semi-supervised baselines. Summa- rized results can be found in Table. 4 Remaining plots with tabular datasets are provided in Appendix. G

---

Under review as a conference paper at ICLR 2023
Fig. 3 shows the AUC across of 15 datasets (the rest can be found in Appendix. G The proposed version of DIAD (blue) outperforms DIAD without the first stage pre-training (orange) consistently on 14 of 15 datasets (except Census). which demonstrates that learning the PID objective with unlabeled data improves the performance. Second, neither the VIME with consistency loss (green) or DevNet (red) always improves the performance compared to the supervised setting. Taole|4shows the average AUC of all methods in semi-supervised AD. Overall, DIAD outperforms all basclines and shows improvements over the unlabeled setting. In Appendix. E average ranks metric rather than AUC.
we show similar results in
6.3 QUALITATIVE ANALYSES ON DIAD EXPLANATIONS
Explaining anomalous data DIAD provides value to the users by providing insights on why sample is predicted as anomalous. We demonstrate this by focusing on Mammography dataset and showing the explanations obtained by DIAD for anomalous samples. The task to detect breast cancer from radiological scans, specifically the presence of clusters of microcalcifications that appear bright on mammogram. The 11 images are segmented and preprocessed using standard computer vision pipelines and 6 image-related features are extracted, including the area of the cell, constrast, and noise. Fig. 4 shows the data samples that are predicted to be the most anomalous, and the rationale behind DIAD on feature contributes more for the anomaly score. The unusually- high 'Contrast' (Fig. a)) is a major factor the way image differs from other samples. The unusually high noise (Fig. (b) and 'Large area' (Fig. (e)) are other ones. In addition, Fig. d) shows 2-way interactions and the insights by it on why the sample is anomalous. The sample has 'middle area' and 'middle gray level', which constitute a rare combination in the dataset. Overall, these visualizations shed light into which features are the most important ones for sample being considered anomalous, and how the value of the features affect the anomaly likelihood.
(d) Area x Gray Level (Sp=0.05)
(a) Contrast (Sp=0.38)
(b) Noise (Sp=0.21)
(c) Area (Sp=0.18)
Gruy Level
Sparsity
Sparsity
Sparsity
12
a.2-
Contrast
Noise
Area
Area
Figure 4: Explanations of the most anomalous samples on the Mammography dataset. We show the top 4 contributing features ordered by the sparsity (Sp) value (anomalous levels) of our model, with 3 features (a-c) and I two-way interaction (d). (a-c) x-axis is the feature value, and y-axis is the model's 's predicted sparsity (higher sparsity represents more likelihood of being anomalous). Model's predicted sparsity is shown
the blue line. The red backgrounds indicate the data density and the green line indicates the value of the most anomalous sample with Sp as its sparsity. The model finds anomalous as it has high Contrast, Noise and Area, different from values that majority of other samples have. (d) x-axis is the Area and y-axis is the Gray Level with color indicating the sparsity (bluc/red indicates anomalous/normal) The green dot is the value of the data that has 0.05 sparsity.
Qualitative analyses the impact of fine-tuning with labeled data Fig. visualizes how pre- dictions change before and after fine-tuning with labeled samples on Donors dataset. Donors dataset consists of 620k educational proposals for K12 level with 10 features. The anomalies are defined as the top 5% ranked proposals as outstanding. We show visualizations before and after fine-tuning. Figs. [5] a & b show that both 'Great Chat' and 'Great Messages Proportion' increase in magnitude after fine-tuning indicating that the sparsity (as signal of anomaly likelihood) of these is consistent with the labels. Conversely, Figs. 55 & d show the opposite trend after fine-tuning. The sparsity definition treats the values with less density as more anomalous in this case 'Fully Funded's =0 is treated as more anomalous. In fact, FFully Funded' is a well-known indicator of outstanding pro- higher
posals, so after fine-tuning, the model learns that Fuull Funded'= in fact contributes anomaly score. This underlines the importance of incorporating labeled data to improve AD accu- racy. Appendix. H shows another visualization in Thyroid dataset.
8

---

Under review as a conference paper at ICLR 2023
(b) Great Messages Proportion
(a) Great Chat
(c) Fully Funded
(d) Referred Count
Output
Output
Output
3002
Output
1000
-20623-
-3.02
0.30
3/2T1
34535
0.7
04 04
92
04
00
Figure 5: AD decision visualizations on the Donors dataset before (orange) and after (blue) fine- tuning with the labeled samples. Here the darker/lighter red in the background indicates high/low data density and thus less/more anomalous. In (a. b) we show the two features that after fine-tuning (blue) the magnitude increases which shows the labels agree with the notion of data sparsity learned before fine-tuning (orange). In (c, d) the label information disagrees with the notion of sparsity; thus, the magnitude changes or decreases after the fine-tuning.
6.4
ABLATION AND SENSITIVITY ANALYSIS
Table 6: Semi-superrised AD perfor- mance with 25% of the validation data.
Table 5: Ablation study for supersuee AD.
No. Anomalies
5
15
30
60
120
25% val data (4% of total data)
DIAD
89.4 90.0 90.4 89.4
91.0
No. Anomalies
5
15
30
60
120
Only AUC
88.9 89.4 90.0 89.1 90.7
Only BCE
88.8 89.3 89.4 88.3 89.2
DIAD
89.0 89.3 89.7 89.1 90.4
Unnormalized sparsity
84.1 85.6 85.7 84.2 85.6
DIAD w/o PT
85.4 87.1 86.9 86.4 87.9
No upsampling
88.6 89.1 89.4 88.5 90.1
CST
83.9 84.9 85.7 85.6 88.2
Only finetune leaf weights
84.8 85.7 86.6 85.7 88.3
DevNet
82.0 83.4 84.4 82.0 84.6
To analyze the major constituents of DIAD, we perform ablation analyses, presented in Table 5 We show that fine-tuning with AUC outperforms BCE. Sparsity normalization plays important role in tinintnnng, since sparsity could have values up to 10 which negatively affect fine-tuning. Upsampling the positive samples also contributesto performance improvements. Finally, to compare with Das et al.] (017) which updates the leaf weights of a trained IF (Liu et al., 2008) to incorpo- rate labeled data, we propose a variant that only fine-tunes the leaf weights using labeled data in the second stage without changing the tree structure learned in the first stage, which substantially decreases the performance. Using differentiable trees that update both leaf structures and weights is also shown to be important.
In practice we might not have large validation dataset, in Sec. 6.2 thus, it would be valuable to show strong performance with small validation dataset. In Table we reduce the validation dataset size to only 4% of the labeled data and find DIAD still consistently outperforms others. Additional results can be found in Appendix. D We also perform sensitivity analysis in Appendix. F thtt varies hyperparameters in the unsupervised AD benchmarks. Our method is quite stable with less than 2% differences across a variety of hyperparameters many different datasets.
DISCUSSIONS AND CONCLUSIONS
As unsupervised AD methods rely approximate objectives to discover anomalies such as recon- struction loss, predicting geometric transformations. or contrastive learning, the objectives inevitably would not align with labels on some datasets, as inferred from the performance ranking fluctuations across datasets. This motivates for incorporating labeled data to boost performance and interpretabil- ity to find out whether the model could be trusted and whether the approximate objective aligns with the human-definee anomalies. In this paper, we consider the general PNU setting, with both pos- itive (P) and negative (N) samples available and we propose a novel interpretable AD framework, DIAD. Our framework consists multiple novel contributions that are key for highly accurate and interpretable AD: we introduce a novel way to estimate and normalize sparsity, modify the archi- tecture by temperature annealing, propose novel regularization for improved generalization, and introduce semi-superviee
AD via supervised fine-tuning of the unsupervised learnt representations. These play crucial role in pushing the state-of-the tftte--f-theaatt art in both unsupervised and semi semi-supervised AD benchmarks. Furthermore, DIAD provides unique interpretability capabilities, crucial in high- stakes applications such in finance healthcare.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
We provide our hyperrpaaameters in Appendix !
We will provide code upon paper acceptance.
REFERENCES
Liron Bergman and Yedid Hoshen. Classification-basee anomaly detection for general data. In International Conference on Learning Representations, 2019.
Paul Bergmann, Sindy Löwe, Michael Fauser, David Sattlegger, and Carsten Steger. Improving un- supervised defect segmentation by applying structural similarity to autoencoders. In VISIGRAPP (5: VISAPP), 2019.
Markus M Breunig, Hans- Kriegel, Raymond T Ng. and Jörg Sander. Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 93-104, 2000.
Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Mare Sturm, and Noemie Elhadad. Intel- ligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1721-1730, 2015.
Chun-Has Hao Chang, Rich Caruana, and Anna Goldenberg. Node-gam: Neural generalized additive model for interpretable deep learning. In International Conference on Learning Representationss 2021a.
Chun-Hao Chang, Sarah Tan, Ben Lengerich, Anna Goldenberg, and Rich Caruana. How inter- pretable and trustworthy are gams? In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pp. 105, 202
Shubhomoy Das, Weng-Keen Wong, Alan Fern, Thomas G Dietterich, and Md Amran Siddiqui. In- corporating feedback into tree-based anomaly detection. arXiv preprint arXiv: 700.00941, 2017.
Izhak Golan and Ran El- Yaniv. Deep anomaly detection using geometric transformations, Advances in neural information processing systems, 31, 2018.
Parikshit Gopalan, Vatsal Sharan, and Udi Wieder. Pidforest: Anomaly detection via partial identi- fication. Advances in Neural Information Processing Systems, 32:15809-15819, 2019.
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, An- drew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint S.ti::170.006611 2017.
Sudipto Guha, Nina Mishra, Gourav Roy, and Okke Schrijvers. Robust random cut forest based anomaly detection on streams. In International conference on machine learning, pp. 2712-2721. PMLR, 2016.
Jacob Kauffmann, Klaus-Robert Müller, and Grégoire Montavon. Towards explaining anomalies: a deep taylor decomposition of one-class models. Pattern Recognition, 101:107198, 2020.
Jacob Devlin Ming-Wes Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT, pp. 4171- 4186, 2019.
Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister. Cutpaste: Self-supervised learning for anomaly detection and localization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9664-9674, 2021.
Zheng Li, Yue Zhao, Nicola Botta. Cezar lonescu, and Xiyang Hu. Copod: copula- outlier detection. In 2020 IEEE International Conference on Data Mining (ICDM), pp. 1118-1123. IEFE, 2020.
10

---

Under review as a conference paper at ICLR 2023
Brian Liu, Miaolan Xie, and Madeleine Udell. Controlburn: Feature selection by sparse forests. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp- 1045-1054, 2021.
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. 2008 eighth ieee international conference on data mining, pp. 413-422. IEEE, 2008.
Haoyu Liu, Fenglong Ma, Yaqing Wang. Shibo He, Jiming Chen, and Jing Gao. Lp-explain: Lo- cal pictorial explanation for outliers. In 2020 IEEE International Conference on Data Mining (ICDM), pp- 372-381, IEEE, 2020.
Philipp Liznerski, Lukas Ruff, Robert A. Vandermeulen, Billy Joe Franks, Marius Kloft, and Klaus Robert Muller. Explainable deep one-class classification. In International Confer- ence on Learning Representations, 2021. URL htttps::/opeereviiee ew. net wtt.weeww..w..eeeeeeeeeee forum?io
A5VV3UyI0Z
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predic- tions. In Guyon, U. Luxburg, S. Bengio, Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp- 4765-4772 Curran Associates, Inc.. 2017. URL http:/ /ttp:://ppprra.... nips.cc/pape.........ccccpppppp. 7062 a wnw-nniiiie se--unifeeddppppp approcch to interpreting model www.epppddiiiiioieeeo.e
Guansong Pang and Charu Aggarwal. Toward explainable deep anomaly detection. In Proceed- ings of the 27th ACM SIGKDD Conference on Knowledge Discovery amp; Data Mining, KDD '21, pp. 4056-4057, New York, NY, USA, 2021a. Association for Computing Machinery. ISBN 9781450383325. doi: 10.11457344548834477744
URL https:://ci.orgg/
3447548. 3470794
Guansong Pang and Charu Aggarwal. Toward Explainable Deep Anomaly Detection, pp.
4056-4057. Association for Computing Machinery, New York, NY, USA, 2021b. ISBN 9781450383325. UtLattss: http//oi.oogr./// 33573333311.1113375553.8.83 3470794
Guansong Pang, Chunhua Shen, and Anton van den Hengel. Deep anomaly detection with deviation networks. In Proceedings of the 25th ACM SIGKDD international conference knowledge discovery & data mining, pp. 353-362, 2019a.
Guansong Pang, Chunhua Shen, and Anton van den Hengel. Deep anomaly detection with deviation networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 353-362, 2019b.
Ben Peters, Vlad Niculae, and André FT Martins. Sparse sequence-to tosstueeceeeeeeeeeee models. In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1504-1519, 2019.
Chen Qiu, Timo Pfrommer, Marius Kloft, Stephan Mandt, and Maja Rudolph. Neural transformation learning for deep anomaly detection beyond images. arXiv preprint arXiv:2103.16440 2021.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference 2016.
on knowledge discovery and data mining, pp. 1
Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexan- der Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference on machine learning, pp. 4393-4403 PMLR, 2018
Lukas Ruff, Robert A Vandermeulen, Nico Gömniz,, Alexander Binder, Emmanuel Müller, Klaus- Robert Müller, and Marius Kloft. Deep semi-supervised anomaly detection. In International Conference on Learning Representations, 2019.
Bernhard Schölkopf, John C Platt, John Slawe-Taylor Alex J Smola, and Robert C. Williamson. Estimating the support a high-dimensional distribution. Neural computation, 13(7):1443- 2001.
11

---

Under review as a conference paper at ICLR 2023
Tom Shenkar and Lior Wolf. Anomaly detection for tabular data with internal contrastive learning. In International Conference on Learning Representations, 2022. URL httt:://ppeeeeee..e
net/Eorum?ics h
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.
Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, and Tomas Pfister. Learning and evalu- ating representations for deep one-class classification. International Conference on Learning Representations, 2021. URL
net http://wwwcccpeeeeeeee.ecooeetaa (trer//rrrmmmmm-CCCSSSSSe
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems, 33:11839-11 1852, 2020.
Sarah Tan, Rich Caruana, Giles Hooker, and Yin Lou. Distill-and-compar:: Auditing black-box models using transparent model distillation. In A/ES, 2018.
Nguyen Xuan Vinh, Jeffrey Chan, Simone Romano, James Bailey, Christopher Leckie, Kotagiri Ramamohanarao, and Jian Pei. Discovering outlying aspects large datasets. Data mining and knowledge discovery, 30(6):1520-1555)) 2016.
Lian Yan, Robert H Dodier, Michael Mozer, and Richard H Wolniewiez. statistic. In Proceedings of the
Optimizing classifier
performance via an approximation to the wilcoxon-mann-whing 20th international conference on machine learning (icml-03), pp. 848-855, 2003.
Jinsung Yoon, Yao Zhang, James Jordon, and Mihaela van der Schaar. Vime: Extending the suc-
cess of self-and semi-supervised learning to tabular domain. Advances in Neural Information Processing Systems, 33, 2020a.
Jinsung Yoon, Yao Zhang. James Jordon, and Mihaela van der Schaar. Vime: Extending the suc- of self-and semi-supervised learning tabular domain. Advances in Neural Information Processing Systems, 33, 2020b.
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Dacki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection, In Inter- national conference learning representations, 2018
12

---

Under review as a conference paper at ICLR 2023
A
PSEUDO CODE FOR SOFT DIFFERENTIABLE OBLIVIOUS TREES ALG.
Here, we show the pseudo code of differentiable trees.
Algorithm Soft decision tree training
Mini-batch X E BxD Temperature T2 (Zi, +0) Symbols: Tree Depth C, Entmoid Trainable Parameters: Feature selection logits G', 2² E RP, split Thresholds b E RC split slope S R..
Input:
G G G2 G .... ".....'e DXC (Alternating G¹, G so only 2 chosen features per tree]
G
G X -EEtMax(F//TTT dim=0) e RBxx (Weighted sum to soft-select features with temperature
Ti}
for c = 1 to do
He o (G 62) S
{Soft binary split of the feature value with temperature T2}
end for H
(1 (
BB2 {Soft counts between [0, 1] per leaf)
(1 - (1-
F = sum(e, dim=0) E 2 Return: E count
(Sum across batch to get total counts per leaf}
B DETAILS OF MAKING TREE OPERATIONS DIFFERENTIABLE
Both (xx) and would prevent differentiability. To address this, (xx) is replaced with weighted sum of features with temperature annealing that makes gradually sharper:
F*))) Σ sr,entma,, (G" T 0,
where G E R a trainable vector per layer e per tree, and entmax, (Peters et al. 2019 is the ent- max normalization, as the sparse version of softmax whose output sum equals to 1. As T 0, the output of entmax gradually becomes one-hot and F=(x) picks only one feature. Similarly, the step function I is replaced with entmoid, which is sparse sigmoid with outputs between 0 and 1. Dif- ferentiability all operations (entmax, entmoid, outer/inner products), render ODT differentiable to optimize parameters W, be and Ge (Chang et al. 2021a).
C PROOF OF PROPOSITION 1
Given uniform noise In and non-uniform features DIAD prioritizes cutting x. over xn because the variance of sparsity of za is larger than In as sample size goes to infinity.
Proposition 1
Here, we show that the variance of sparsity of uniform features would go to 0 under large sample sizes. Without loss of generality, we assume that the decision tree has single cut in I (0, 1) in a uniform feature E [0, 1]. and we denote the sparsity of the left segment as and the right as Vi $2. The sparsity $1 is defined as D. where the V1 = l is the volume, and the D; is the data ratio i.c. Dj = where is the counts of samples in segment between and 7, and is the total samples. Since In uniform feature, the counts e₁ become a Binomial distribution with n samples and probability 1:
C Bem(n, Bern(n, 1 l).
As 0o, Dj because E[D; Eleil / and V Var Verfel ((1-)) Therefore, as number of examples grow, the sparsity 8; = Vi
n
D. + = 1. Similarly, $2 + 1. For any uniform noise, since both sparsity $1, $2 converges to as n - 00 no matter where the cut is, the variance of sparsity converges to O. Thus, the objective of DIAD which maximizes the variance of sparsity would prefer splitting other non-uniform features since there is no gain variance sparsity when splitting the uniform noise.
13

---

10a (red, Under review as a conference paper at ICLR 2023
D SEMI-SUPERVISED AD RESULTS WITH SMALLER VALIDATION SET
When we have small set of labeled data, how should we split it between the train and validation datasets when optimizing semi-supervised methods? In Sec. 6.2 we use 94%-16%-209 for train- val-test splits, and 16% of validation set could be too large for some real-world settings. Does our method still outperform others under smaller validation set?
To answer this, we experiment with a much smaller validation set with only 50% and 25% original validation set (i.e. 8% and 4% of total datasets). In Table 7 show the average AD performance across 15 datasets with varying size of validation data. With decreasing validation size all methods decrease the performance slightly, method still consistently outperforms others.
Table 7: Summary of Semi-supervised AD performances with varying size of validation set (4%, 8% and 16% of total datasets). We show the average of AUC across 15 datasets with varying number of anomalies. Our method DIAD still outperforms others consistently.
25% val data (4% of total data)
50% val data (8% of total data)
No. Anomalies
5
15
30
60
120
5
15
30
60
120
DIAD w/o PT
85.4
87.1
86.9
86.4
87.9
85.7
86.9
88.0
86.9
87.5
DIAD
89.0
89.3
89.7
89.1
90.4
89.2
89.7
90.0
89.2
90.6
CST
83.9
84.9
85.7
85.6
88.2
84.2
85.7
85.8
86.2
87.9
DevNet
82.0
83.4
84.4
82.0
84.6
83.0
85.0
85.5
83.6
85.5
100% val data (16% of total data)
No. Anomalies
5
15
30
60
120
DIAD w/o PT
86.2
87.6
88.3
87.2
88.8
DIAD
89.4
90.0
90.4
89.4
91.0
CST
85.3
86.5
87.1
86.6
88.8
DevNet
83.0
84.8
85.4
83.9
85.4
E THE AVERAGE RANK PERFORMANCE OF SEMI-SUPERVISED AD RESULTS
The average AUC for semi-supervised AD results (Table 4 might not represent the entire picture, so we provide the average ranks as well Table Our method still consistently outperforms other methods.
Table 8: Average ranks of AUC across 15 datasets in the Semi-supervised AD result.
No. Anomalies
5
15
30
60
120
DIAD
1.3
1.3
1.3
1.3
1.2
DIAD w/o PT
2.3
2.6
2.6
2.5
2.9
CST
3.2
3.1
3.1
3.0
2.7
DevNet
3.1
3.0
3.1
3.2
3.2
F SENSITIVITY ANALYSIS
We perform sensitivity analyses from our default hyperparameter the unsupervised AD bench- marks. We exclude Census. NYC taxi, SMTP, and HTTP datasets since some hyperparameters can be run, resulting in total datasets each with different random seeds. In Fig. 6 besides show- ing the average of all datasets (blue), we also group datasets by their sample sizes into 3 groups: 10a (green, 5 datasets), and (4)
(1) > 100 (Orange, datasets), (2) 10. > 6 datasets). Overall, DIAD shows quite stable performance between 82-84 except when (c) No. trees= 50 and (h) smoothing < 10. We also find that hyperparameters: (a) batch size, (b) No. Layers, and (d) Tree depth that the large group (orange) has opposite trend than the small group (red). Large datasets yield better results with smaller batch sizes, larger layers, and shallower tree depths.
14

---

Under review as a conference paper at ICLR 2023
(a) Batch Size
(b) No. Layers
(c) No. Trees
Tree depth
Arg AUC (%)
Avg AUC (%)
Avg AUC (%)
Avg AUC (%)
68-
2
Ve 1901 196 2008 2901 som
10
15 10 25
20
35
40
200
205
300
465
25
25
20
44
(f)
Column Subsample
(g) Per-tree Dropout
(h) Smoothing
(i) LR warmup
Avg AUC (5)
Avg AUC (%)
Avg AUC (%) (%))
R 3
Avg AUC (%)
R 2 2
Mo
2
R
1500 1290
Figure 6: Sensitivity analysis. Y-axis shows the average AUC across 14 datasets, and X-axis shows the varying hyperparameters. The dashed line is the default hyperparameter. We show groups: (1) All datasets (Blue), (2) N > 10y (Orange), (3) 10a > N > 10 (green), and (4) N < 10 (red).
G SEMI-SUPERVISED AD FIGURES
We experiment with 15 datasets and measure the performanceunder a different number of anomalies. We split the dataset into 64-16-20 train-val-test splits and run times to report the mean and standard error. We show the performance in Fig, 7
H MORE VISUAL EXPLANATIONS
We show another example of DIAD explanations on the "Celeba" dataset. Celeba consists of 200K pictures of celebrities and annotated with 40 attributes including "Bald", "Hair", "Mastache", "At- tractive" etc. We train the DIAD on these 40 sets of attributes and treat the "Bald" attribute as outliers since it accounts for only 3% of all celebrities. Here we show the most anomalous example deemed by the DIAD in Fig. g The top 4 contributing factors are shown in (a-d), showing Gray Hair, Mustache, Receding Hairline, and Rosy Cheeks are very anomalous in the data. We also show the top 4 interactions in (e-h), indicating the combination of Rosy Cheeks with Mustache, Goatee, Necktic and Side Burns are even more anomalous decmed by DIAD.
We also show the least anomalous example deemed by DIAD in the Celeba dataset in Fig. The lack of "Receding Hairline", "Rosy Cheeks", "Pale Skin", and "Gray Hair" are pretty common and thus DIAD outputs negative normalized sparsity value.
We show another example of DIAD explanations on the "Backdoor" dataset. It consists of 95K sam- ples and 196 features that record the backdoor network attacks with the attacks anomalies against the 'normal' class, which is extracted from the UNSW-NB 15 data set. In Fig. TO] we show two most anomalous examples deemd by DIAD, The Fig. I0 I0a-cc a-c) shows the top 3 contributing factors for one example and the "protocol=HMPP solely determines its high abnormity since the rest of the two features have only little sparsity. A user can thus decide if he wants trust such explanation and finds out if such protocol is indeed anomalous. The Fig. [- shows the top 3 contributing factors for the other example and both the "protocol=ICMP" and sttat==ECO" contributes to its high sparsity (1.5, and 1.2 respectively). And other features are relatively quite normal.
We further show another case study of DIAD explanations on "Thyroid" datasets before and after fine-tuning to further demonstrate the discrepancy between unsupervised AD objective and labeled anomalies. Thyroid datasets contains 7200 samples with 6 features and 500 of labels are labeled as "hyperfunctioningg" In Fig. [] visualize 4 attributes: (a) T3, (b) T4U, (c) TBG, and (d) TT4. And the dark red in the backgrounds indicates the high data density by bucketizing the x-axis into 64 bins and counts the number of examples for each bin. First, in T3 feature, before fine-tuning (orange) the model predicts a higher anomaly for values above 0 since they have little density and have mostly white region. After fine-tuning the labeled data (blue), the model further strengthens
15

---

Under review as a conference paper at ICLR 2023
(a) Vowels
(b) Siesmic
(c) Satimage
(d) Thyroid
1.09
AUC
6575
0.70
0.99
3.83
1340
6650
0825
3965
8
ub
No. Anomalics
No. Anomalics
No. Anomalics
No. Anomalics
(c) Mammography
(f) AT
(g) NYC
(h) CPU
0.00
AUC
3.54
019
0.52
0.7-
130
0.8
Desfic
3.50
No. Anomalics
No. Anomalics
No. Anomalics
No. Anomalics
MT
(i) Campaign
(k) Backdoor
Celeba
ALC
0.20
040
1.00
0.3
0.32
0.70
35
No. Anomalies
No. Anomalies
No. Anomalies
No. Anomalies
(m) Fraud
(n) Census
Donors
AUC
0.75
0.992
01627
6881
77
08875
6.73
40
135
No. Anomalies
No. Anomalies
No. Anomalics
Figure 7: Semi-supervised AD performance tabular datasets (out of 15) with varying number of anomalies. Our method 'DIAD' (blue) outperforms other semi-supervised baselines. Table. summarizes the comparisons.
16

---

Under review as a conference paper at ICLR 2023
Dot
Gray Hair (Sp=1.6)
Mustache
Receding
Rosy Cheeks (Sp=1.1)
(a)
(b)
(c) Hairline (Sp=1.15)
(d)
(Sp=1.3)
Sparsity (Sp)
Snarsity (Sp)
Snarsity (Sp)
Sparsity (Sp)
00
Mustache
Goatee
Rosy Cheeks Necktie
Rosy Cheeks
(e)
X Rosy Cheeks (Sp=0.5)
(f)
Rosy Cheeks (Sp=0.3)
(h)
x Side Burns (Sp=0.2)
(Sp=0.3)
Sparsity (Sp)
Sparsity (Sp)
King Chockee
Spursity (Sp)
33
Sparsity
2.4
22
02
Figure 8: DIAD decision making visualizations of the most anomalous image the CelebA dataset. The top 4 contributing mains are shown in (a-d) where the green dots are the image's attributes and the blue line is the model's prediction. This celebrity has gray hair, Mustache, receding hairline, and rosy cheeks which make DIAD predict him very anomalous in the dataset. The top 4 2- way interactions are shown in (e-h) where the combination of the Rosy Cheeks with Mustache (e), Goatee (f), Necktie (g), and Side Burns (h) make him even more anomalous.
Receding
Rosy Cheeks (Sp=-0.08)
Pale Skin (Sp=-0.07)
Gray Hair (Sp=-0.07)
(a)
Hairline
(b)
(c)
(d)
(Sp=-0.1)
Sparsity (SD)
Sparsity (Sp)
32
Sparsity (SD)
Sparsity (Sp)
646
0.6
1.0
0.2
64
68
10
04
24
04
Figure 9: DIAD decision making visualizations of the least anomalous celebrity in the CelebA dataset, showing its least anomalous features. The lack of "receding hariline", "rosy cheeks", "pale skin", and "gray hair" make DIAD deem him as the most normal subject indicated by the negative sparsity.
its belief that values bigger than 0 are anomalous. However, in T4U, TBG and TT4 features, before fine-tuning (orange) the model indicates higher values are more anomalous because has larger volume and relatively small data density (white). But after fine-tuning (blue) the labels the model
17

---

Under review as a conference paper at ICLR 2023
Protocol = HMP (Sp=2.8)
No. connections of
CT source Dport (Sp=0.002)
(a)
(b)
(c)
the same dest (Sp=0.05)
Sparsity (Sp)
Sparsity (Sp)
Sparsity (Sp)
-02
0225
-0.6
-180
-1.0
0.4
Source bits per second (f)
Protocol ICMP
State ECO (Sp=1.2)
(d)
No. of HTTP flows
(Sp=1.5)
(Sp=0.01)
Sparsity (Sp)
Sparsity (Sp)
20-
Source bps
-0075
45-
6.0
0.0
-23
-05
110000
-1.0
-100
1,0077
0.00
0.25
0.50
0.75
1.00
No. of HTTP flows
Figure 10: DIAD decision making visualizations of 2 most anomalous examples in the Backdoor dataset. (a-c) shows the top 3 contributing factors of one example and the "protocol=HMP" is solely responsible for abnormity prediction. (d-f) shows another example that both "protocol=ICMP" and "State=ECO" are both contributing to large abnormity value.
moves an opposite direction that the smaller feature value is more anomalous. This shows that the used unsupervised anomaly objective, PID, is in conflict with the human-defined anomalies in these features.
T3
(b) T4U
(c) TBG
(d) TT4
Output
Outuur
Output
Output
00
32
05
00
C44
030
0.2
0.0
0.5
Figure 11: AD decision making visualizations on the Thyroid dataset before (orange) and after (blue) fine-tuning on the labeled samples. Here the darker/lighter red in the background indicates high/low data density and thus less/more anomalous. In (a) T3, the labeled information agrees with the anomaly specified in PID, after fine-tuning the magnitude increases. In (b, c, the label information disagrees with the anomalous levels specified in PID especially when the value are small: thus, the magnitude changes or decreases after the fine-tuning.
We show more experimental results to see how methods perform under noise injection in Table following the procedures described in Sec. /cc./oaan and Table In additional to Thyroid and Mammo- graph, we further compare with Siesmic, Campaign, and Fraud. We find that overall DIAD and OC-SVM only deterioriates around 1-2% while others can deterioriate up to 3-1 11% on average, showing DIAD's superiority of noise resistance.
I HYPERPARAMETERS
Here list the hyperparameters for both unsupervised and semi-supervised experiments.
18

---

Under review as a conference paper at ICLR 2023
Table 9: Unsupervised AD performance (% AUC) with additional 50 noisy features for DIAD and baselines. We find both DIAD and OC-SVM deteriorate on average 1-2% while other methods deteriorate 3-11% on average. We ignore the average of LOF since most of the ROC are below 50%.
DIAD
PIDForest
GIF
COPOD
PCA
SCAD
ANN
RRCF
LOF OC-SVM
Thyroid
76.1 25
88.2
67.6 81.4
77.6
67.3
75.9+12
75.1
74.0
26.3
54.7
Thyroid (noise)
71.1
26.0
42 14.4
49.5
49.5
53.6
50.8
Mammography
85.0
84,8
82.5
85.7
90.5
88.6
69.8
83.9
83.2=02
28.0
87.2
Mammography (noise)
83.1 1o1
82.0
72.7
71.4
72.4
76.8
6M.4
81.7
79.1
37.2
87.2
Siesmic
72.2 2.4
23
53.3 =4 70.7 2:00
72.7
68.2
65
74.0
69.7=11
44.7
58.9
Siesmic (noise)
71.8
+
56.9 =12 62.0
58.0
74,0
69.7=63
44.7
Campaign
71.0
78.6111
64.1
70.4
78.3
73.4
12.0
72.0
65.5
46.3
66.7
Campaign (noise)
70.6
63.7 =43 67.6
71.1
72.5
71.6
67.6
59.3
40.3
64.5
Fraud
95.7 02
94,7+
80.4 =68
94,8 +.
94.7
95.2
95 o2
93.4
87.5
52.5
94,8
Fraud (noise)
95.6
94
51.1 2:66
1.7
78.6
61.6244
21.3
Averuge
1.6
9.4
11.3
I.1
UNSUPERVISED AD
Since it's hard to tune hyperparameters in unsupervised setting, for fair comparisons, all baselines with default hyperparameters. Here we list the default hyperparameter for DIAD in Ta- ble 10
Here we explain each specific hyperparameter:
Batch size: the sample size of mini-batch.
LR: learning rate.
y: the hyperparameter used update the sparsity in each leaf (Eq.
Steps: the total number of training steps. We find 2000 works well across datasets.
LR warmup steps: we do the learning rate warmup (Goyal et al. 2017) that linearly in- creases the learning rate from 0 le-3 the first 1000 steps.
Smoothing ó: the smoothing count for our volume and data ratio estimation.
Per tree dropout: the dropout noise we use for the update of each tree.
Arch: we adopt the GAMAtt architecture form the NodeGAM (Chang al. 2021a
No. layer: the number of layers of trees.
No. trees: the number of trees per layer.
Additional tree dimension: the dimension of the tree's output. If more than 0, it appends an additional dimension in the output of each tree.
Tree depth: the depth of tree.
Dim Attention: since we use the GAMAtt architecture, this determines the size of the attention embedding. We find tuning more than 32 will lead insufficient memory in our GPU, and in general 8-16 works well.
Column subsample (p): this controls how many proportion of features a tree can operate on.
Temp annealing steps (K), Min Temp: these control how fast the temperature linearly de- cays from 1 to the minimum temperature (0. 1) in K steps. After K training steps, the entmax or entmoid become max or step functions the model.
I.2 SEMI-SUPERVISED AD
We adopt 2-stage training. In the 1st stage, we optimize the AD objective and select best model by the validation set performance under the random search. Then in the 2nd stage, search the following hyperparameters with No. anomalies=120 to choose the best hyperparamter, and later run through the rest of 5, 15, 30, and 60 anomalies report the performances.
Learning Rate: [5e-3, 2e-3, 1e-3, 5e-4]
19

---

Under review as a conference paper at ICLR 2023
Table 10: Default hyperparameter used in the unsupervised AD benchmarks.
LR warmup steps
Per tree
Batch Size
LR
7
Steps
Smoothing
Dropout
Arch
2048
0.001
0.1
2000
1000
50
0.75
GAMAtt
Temp annealing
No.
No.
Addi.
Tree depth
Column Subsample (p)
Dim Attention
Min Temp
layers
trees
tree dim
steps (K)
3
300
12
0.4
1000
0.1
Loss: ['AUC", 'BCE"].
Then, for each baseline we use the same architecture but tune the hyperparameters:
CST: the overall loss is calcualted as follows (Eq. 7, 8, 9 in Yoon et al. 2020a
L.final Tug 3T..
The supervised loss L. is:
L, = (e (BC
The consistency loss Lu is:
Lu Eg-Px wwppmmmmmmm....mmm (
where the gm (æ, m)) is to use dropout mask m to remove features and impute it with marginal feature distribution, and the masks are sampled K times. Since the accuracy is quite stable across different B. and when 2 20 (Fig. 10, Yoon et al. 2020a)), we select 3 1 and K 20, and search the dropout rate Pm from [0.05, 0.1, 0.2, 0.35, 0.5, 0.7] and the learning rate [2e-3, 1e-3).
DevNet: they first randomly sample 5000 Gaussian samples with mean and standard deviation and calculate the mean ug and standard deviation dg:
UR Σ Ti where T₃ ~ (0, 1),
OR standard deviation of (ri, T2... [5000
Then they calculate the loss (Eq. 6, 7 in (Pang al. 2019a)):
L (1 v)(dev(x)) + max dev(x)) where dev(x: 6(x) uR
GR
The & is the deep neural network and the a set to 5. In short, they try to increase the output of anomalies (y 1) be bigger than a and let the output of normal data (y 0)
to be close to O. We tune learning rates from [2e-3, Ic-3, 5e-4] for DevNet.
20


Reviews:
[{"review_id": "z_uwjyHUuK", "reviewer": "Reviewer_XPd4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents an approach for tabular anomaly detection which is based on Partial Identification (PID) and on Generalized Additive Models (GAM) and extensions. The method works also in semisupervised settings and compares well with alternatives, as shown in a quite extensive experimental evaluation.", "summary_of_the_review": "Paper which proposes a carefully tailored pipeline for anomaly detection of tabular data. Some doubts on its significance. Large experimental parts which should be completed with a rigorous statistical analysis.\n\nUPDATE AFTER THE REBUTTAL. \nI carefully read the responses, the clarifications, and the additional material, and I thank the authors for the significant efforts made in clarifying my doubts. Even if I consider that the paper has potential, I’ll maintain my score unchanged, since I still have few doubts, especially on the significance and on the analysis of the results (even if I acknowledge the efforts made by the authors in improving these aspects in the rebuttal)", "strength_and_weaknesses": "Positive points\n- The paper is well written and easy to read\n- The topic is definitely interesting\n- The structure of the manuscript is clear\n- Experiments are extensive\n\n\nNegative points\n- Significance of the proposed method. \nMy main concern is about significance of the proposed method. It seems that authors start from ideas presented in PID and PIDforest, for anomaly detection, and optimized such framework and created an effective pipeline by adding several carefully chosen ingredients. The resulting framework is definitely well performing, as shown in the experiments, but I’m wondering how large is its methodological contribution from the Anomaly Detection perspective (but of course this is my personal opinion). As for the theoretical contribution, I think it is not so relevant, being reduced to the three lines of page 5 (by the way, the proof of the proposition, found in Appendix, is not clear to me, especially the sentence which concludes the proof)\n\n- Conclusions from the experiments.\nExperiments are very extensive, based on 20 tabular datasets and involving many different competitors. One comment which applies to the whole analysis. The tables show bold values, which are meant to highlight the best results; however these do not derive from a rigorous statistical analysis, but simply by the rule “Metrics with standard error overlapped with the best number are bolded” (from the paper, caption table 2). Without a rigorous statistical evaluation conclusive observations can not be derived. I suggest authors to use the Friedman test followed by a post-hoc Nemenyi test, and to show results via critical diagram – see for example\n\nDemšar, Janez. \"Statistical comparisons of classifiers over multiple data sets.\" The Journal of Machine learning research 7 (2006): 1-30.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "n7Un4AKwt4L", "reviewer": "Reviewer_qGLb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new interpretable method for unsupervised or semi-supervised anomaly detection on tabular data, in the presence of noisy or unlabeled data. ", "summary_of_the_review": "In summary, the authors propose an innovative approach to AD on tabular data. The experimental evaluation shows good promise, but is not entirely convincing. ", "strength_and_weaknesses": "Strengths:\n- Innovative approach based on generative additive models.\n- The model is explainable.\n- There is a little technical contribution (Prop 1) creating some insight into the method's inner workings.\n\nWeaknesses:\n- Section 6.1. on unsupervised AD experiments has some problems in terms of the chosen datasets and baselines: [1] determined [2] as the overall best performing deep AD method on tabular data. However, the method [2] is missing in the comparison here. On the 14 datasets from Gopalan et al., the proposed approach shows inferior performance, except for SatImage, where a tiny improvement of 0.4% is achieved. Out of the 9 datasets from Pang et al., only 6 have been selected. To avoid dataset selection bias, all 9 should be analyzed. Besides NeuTraL, also DSVDD - a common baseline - is missing in the comparison. \n- In the noise experiments, only two datasets have been analyzed, which is insufficient to draw conclusions.\n- Section 6.2 on semi-supervised AD: These experiments look better. The approach achieves higher AUROC than the Devnet baseline. However, the original Devnet paper reported on AUPRC, which is why I cannot confirm that the present results are consistent with their analysis. The semi-supervised DSVDD baseline is missing.\n\n\n\n[1] Maxime Alvarez, Jean-Charles Verdier, D'Jeff K. Nkashama, Marc Frappier, Pierre-Martin Tardif, Froduald Kabanza: A Revealing Large-Scale Evaluation of Unsupervised Anomaly Detection Algorithms. https://arxiv.org/abs/2204.09825\n[2] Chen Qiu, Timo Pfrommer, Marius Kloft, Stephan Mandt, Maja Rudolph Proceedings of the 38th International Conference on Machine Learning, PMLR 139:8703-8714, 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "O-6hnlSJ5t3", "reviewer": "Reviewer_1y6T", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a new anomaly detection method, namely DIAD, that uses Partial Identification (PID) as an objective to perform anomaly detection optimization with a tree structure of an existing generalized additive model. It is also flexible to use an additional loss function, such as AUC optimization or BCE loss, to utilize auxiliary labeled anomaly data in the semi-supervised setting. Due to the use of GAM model structures, it also offers a two-way feature interaction based explanation of detected anomalies. The effectiveness of the method is evaluated on 18 tabular datasets.", "summary_of_the_review": "Considering both the pros and cons above, the paper is at the borderline and towards the reject side.", "strength_and_weaknesses": "The work aims to tackle both the interpretability and the capability of utilizing a few labeled anomaly examples in anomaly detection. It has the following strengths:\n- Both of the interpretability and the sample-efficient capability are important to different ML tasks, including anomaly detection. The paper addresses both of these two important aspects.\n- The presented method can work in both unsupervised and semi-supervised settings. Limited AD methods have such a property.\n- Experiments on 18 datasets show the effectiveness of the method against popular unsupervised/semi supervised methods\n\nSome issues that may require further investigation include:\n- The technical novelty of the method is weak. The methods like PID were used in anomaly detection, and the AUC loss is a widely used loss function and it does not bring much improvement over BCE. The full method seems to be a combination of these widely used components.\n- The method is weak in the unsupervised anomaly detection setting, which does not show clear improvement over the closely related competing methods like PIDForest and IF. There are also other more effective improved IF methods that tackle the studied issues in the paper, such as [a-c] and references therein. It would be more convincing to review these studies and possibly include them in the comparison.\n- In terms of the semi-supervised setting, the proposed method involves two stages of training, unsupervised training and fine-tuning. I wonder whether the competing methods like CST and DevNet are trained from scratch, or only fine-tuned in a similar way as the proposed method. Questions here include how much the contribution is it from the unsupervised training in the semi-supervised setting? Does such a training strategy also help improve other existing semi-supervised methods?\n- The work argues a two-way feature interaction-based anomaly explanation, but the presented result in Figure 4 is only the explanation of individual features. Figure 4(d) is a two-way interaction example, but the results indicate only individual features, e.g., the Area feature, are sufficient for the detection and interpretation.\n- The interpretation results are limited to qualitative ones, and do not involve comparison to other methods, and thus, they are not convincing enough.\n- There exist a large number of gradient backpropagation-based methods for anomaly explanation, especially on image data. This type of methods is applicable to tabular data, too. The authors are suggested to elaborate more based on more related work in the discussion in Related Work. \n\n**References**\n- [a] \"Sparse modeling-based sequential ensemble learning for effective outlier detection in high-dimensional numeric data.\" In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1. 2018.\n- [b] \"Extended isolation forest.\" IEEE Transactions on Knowledge and Data Engineering 33, no. 4 (2019): 1479-1489.\n- [c] \"Generalized isolation forest for anomaly detection.\" Pattern Recognition Letters 149 (2021): 109-119.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "0AO-62KUMG", "reviewer": "Reviewer_235S", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper develops an interpretable anomaly detection framework, DIAD,  by combining the NodeGAM structure with the PID objective. The NodeGAM provides an interpretable mechanism, and PID can be trained for anomaly detection. Overall, the framework makes sense to me. The paper includes detailed experimental evaluation by comparing with several state-of-the-art baselines on multiple datasets.", "summary_of_the_review": "I feel that the proposed approach has some merits, but I am also not very sure whether the combination of NodeGAM and PID is trivial or challenging. Meanwhile, the term \"interpretable anomaly detection\" in the title really draws my eye, but the evaluation in this part is weak in my view. I understand currently there is no golden metric to evaluate the interpretability, but maybe showing more cases could be more convincing, just like the papers in the CV area. \n\nI also have one more question, which may be outside the scope of this paper: If the purpose is to achieve sample-wise interpretation (explaining the anomalies), could we apply other post-hoc interpretation techniques to interpret PIDForest? ", "strength_and_weaknesses": "Strength\n1. The paper targets an important research problem, interpretable anomaly detection.\n2. The proposed framework leverages the existing techniques and makes sense.\n3. Detailed evaluation is conducted to show the performance of anomaly detection.\n\n\nWeakness:\n1. In the unsupervised setting, the proposed DIAD can achieve better or comparable performance on large datasets, but may not be as good as baselines when the datasets only have small amounts of samples. Do the authors have any insight into such observations? \n\n2. My major concern is in the interpretation part. How to achieve interpretable anomaly detection based on DIAD is not very clear to me. For example, given an anomaly, do we explain the result based on the ranking of features evaluated by the sparsity values or based on the feature-wise function? Meanwhile, as Mammo. only consists of 6 features, I am also curious about the interpretability of the proposed model on high-dimensional data. I believe for low-dimensional data, it is not very challenging for domain experts to exam the anomalies. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject