Unsupervised Anomaly Detection in The Presence of Missing Values
Authors: Feng Xiao
Jicong Fan
Venue: NeurIPS 2024 poster (2024)
Keywords: missing data
anomaly detection
deep learning
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Anomaly detection methods typically require fully observed data for model training and inference and cannot handle incomplete data, while the missing data problem is pervasive in science and engineering, leading to challenges in many important applications such as abnormal user detection in recommendation systems and novel or anomalous cell detection in bioinformatics, where the missing rates can be higher than 30\% or even 80\%. In this work, first, we construct and evaluate a straightforward strategy, ''impute-then-detect'', via combining state-of-the-art imputation methods with unsupervised anomaly detection methods, where the training data are composed of normal samples only. We observe that such two-stage methods frequently yield imputation bias from normal data, namely, the imputation methods are inclined to make incomplete samples ''normal", where the fundamental reason is that the imputation models learned only on normal data and cannot generalize well to abnormal data in the inference stage. To address this challenge, we propose an end-to-end method that integrates data imputation with anomaly detection into a unified optimization problem. The proposed model learns to generate well-designed pseudo-abnormal samples to mitigate the imputation bias and ensure the discrimination ability of both the imputation and detection processes. Furthermore, we provide theoretical guarantees for the effectiveness of the proposed method, proving that the proposed method can correctly detect anomalies with high probability. Experimental results on datasets with manually constructed missing values and inherent missing values demonstrate that our proposed method effectively mitigates the imputation bias and surpasses the baseline methods significantly. The source code of our method is available at https://github.com/jicongfan/ImAD-Anomaly-Detection-With-Missing-Data.

Missing--nn..... - Unsupervised Anomaly Detection in The Presence of Missing Values
Feng Xiao
Jicong Fan 1,2-
The Chinese University of Hong Kong, Shenzhen, China Shenzzen Research Institute of Big Data, Shenzhen, China xi xiaofeng... ds@gmail... com
fanjicongâcuhk. edu.
Abstract
Anomaly detection methods typically require fully observed data for model training and inference and cannot handle incomplete data, while the missing data problem is pervasive in science and engineering, leading to challenges in many important applications such as abnormal user detection in recommendation systems and novel or anomalous cell detection in bioinformatics, where the missing rates can be higher than 30% or even 80%. In this work, first, we construct and evaluate a straightforward strategy, "impute-then-deteee""
combining state-of-the-ar imputation methods with unsupervised anomaly detection methods, where the training data are composed of normal samples only. We observe that such two- stage methods frequently yield imputation bias from normal data, namely, the imputation methods are inclined to make incomplete samples "normal", where the fundamental reason is that the imputation models learned only on normal data and cannot generalize well to abnormal data the inference stage. address this challenge, we propose an end-to-end method that integrates data imputation with anomaly detection into unified optimization problem. The proposed model learns generate well-designed pscudo-abnorma samples mitigate the imputation bias and ensure the discrimination ability both the imputation and detection processes. Furthermore,
provide theoretical guarantees for the effectiveness the proposed method, proving that the proposed method can correctly detect anomalies with high probability. Experimental results datasets with manually constructed missing values and inherent missing values demonstrate that our pro- posed method effectively mitigates the imputation bias and surpasses the baseline methods significantly. The source code of our method is available at hhttps:/ // github. com jicngg/iioooaaaaaaoo an/ ImAD LmAA-AAnmmaly-Deeeeeeee Anomall-
1 Introduction
Anomaly detection (AD) [rrenng et al. 2000 Schölkopf et al. 2001 Liu et al. 2008 Pevny 2016
Zong al. 2018 Ruff et al. [018 Cai and Fan 2022 Fu al. 2024 Zhang al. 2024 Xiao
ct al. 2025 aiming at identifying anomalous or novel samples in data, is crucial machine learning problem. It finds extensive applications in many high-stakes fields such as biology, healthcare, finance, and cybersecurity. Data missing or incompleteness, a persistent and unavoidable issue in many real-world situations, often arises during the processes of data collection, transmission, and storage. Moreover, in fields like bioinformatics (e.g. single-cell RNA sequencing) [Zhang and Zhang 2018), psychology (e.g. questionnaire data) [Schlomer al. 2010), and recommendation systems (e.g. user-item interaction data) [Shani and Gunawardana 2011 Fan et al. 2024], the data missing rates are often higher than 30% 80%. Indeed, the missing data problems lead to
"Correspondngg author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

many challenges for anomaly detection, such as detecting anomalous cells or rare cell types based on incomplete single-cell RÑA sequencing data (Fa et al. 202T] and identifying abnormal users in recommendation systems YYng and Cail 2017]. Regrettably, most existing AD methods necessitate complete data both the training and test sets, rendering them ill-equipped to handle datasets with missing values. Consequently, addressing the AD challenge in the context of incomplete data becomes both necessary and inevitable.
A naive strategy is filling the missing values by statistical characteristies such as mean or median and then performing anomaly detec- tion. Taking two real-world datasets "Adult" and "KDD" as examples, we consider the mechanism missing completely at random and fill the miss- ing entries with the variable means and then perform a classical AD methods Isolation For- est (Liu et al. 2008) and two deep learning based AD methods (Deep SVDD |Ruff et al.) 2018) and NeutraL AD [Qiu et al. 2021]). The results are shown in Figure I The detection accuracies of the four methods degrade signifi- cantly with the missing rate increases. This verified the failure of the naive strategy and the difficulty of unsupervised anomaly detection with missing values. Besides the naive imputation, one may
Adali
KDD
lond
Diso SVEp
AURRY
AUROC
2.50
Cd
Deng Sert
Missag Rate(N)
Minsng Rate(t))
Figure I: Performance (AUROC) degradation of anomaly detection methods with increasing miss- ing rate on Adult and KDD datasets.
consider using more powerful imputation algorithms (Dempster et al. 1977 Pigott 2001 Candes
and Recht. 2012 Stekhooeen and Bühlmann 2012 Gondara and Wang 2018 Yoon et al. 2018 Fan
et al.) 2020 Muzellec et al. 2020] to recover the missing values and subsequently implementing AD
algorithms on imputed data. We refer to this strategy as "immuue-then-deteea".
is worth noting that, for unsupervised anomaly detection, where the training set is composed of only normal samples, such ceppuut-then-deteed methods would yield imputation bias for normal data, i.c., the imputation methods are inclined recover an abnormal sample with missing val- ues as "normal" as possible during the inference, which leads to lower recall higher false neg- ative rate. Figure clearly shows the negative impacts of the imputation bias, which is worth studying and addressing. The main challenge is that the training set and test set do not satisfy the condition of identical distribution and the impu- tation model trained only on incomplete normal data does not generalize well to incomplete ab- normal data. In Section 4.2] we quantitatively and comprehensively evaluate the "impute-then-deter" methods using srate-of-the-as imputation algorithms and AD algorithms.
Muk
KDD
date Imputed MissForest
camplete
Imputed Missfores:
66-
Impubed GAIN
Impuled GAN
1.0-
0.5
Rocall
8.4
0.12
Doop SVID
Deop SYDO
Figure 2: The degradation of recall rate of abnor- mal data on "impute- "impute-then-deteet" methods.
To tackle the aforementioned problem, in this paper, we propose an end-to-end method, called ImAD, for unsupervised anomaly detection on incomplete data. The main idea of ImAD is to integrate data imputation and anomaly detection into unified optimization objective and alleviate imputation bias by automatically learning to generate pseudo-abnormal samples. Note that the pseudo-abnormal samples are by-products of the training process and we do not use any extra data in all experiments. Our contributions are summarized as follows.
We study the imputation bias problem of the "impute- thenuteeetthe--eeett pipeline and quantitatively evaluate their detection performance.
We propose novel method ImAD for AD incomplete data. To the best of our knowledge, it is the first end-to-end unsupervised AD method in the presence of missing value.
We provide theoretical guarantees for ImAD, proving that it can correctly detect anomalies with high probability.
We compare ImAD with more than 9 baselines on 11 real datasets of various domains, covering datasets with manually constructed missing values and datasets with inherent missing values.
2

---

2 Related Work
Missing Data Imputation Data imputation fills missing data with plausible values and provides imputed data for downstream tasks such as classification, clustering, and visualization. As the missing data problem is prevalent in many fields, the study on missing data imputation is extensive, and many algorithms have been proposed in the past decades. |ayer et al. [2019] pointed out that there are approximately 150 implementations available to handle missing data. These methods can be roughly organized into three categories. The first category is based the iterative regression model. such as well-known methods Multiple Imputation by Chained Equations (MICE) [Royston and White 201T] and MissForest (Stekhoven and Bühlmann 2012] that trains random forests on observed data through an iterative imputation scheme. The second category is the matrix completion methods (Candes and
Recht 2012 Mazumder et al.) [2000) Fan and Chow 2018 Fan al., 2019 2020 The third category is based on deep learning especially deep generative models [Fan and Chow [017]] Yoon et al. 2018
Liet al. 2019 Muzellec et al.
2020 , For instance. Yoon et al. (2018) proposed generative adversarial
imputation network (GAIN) based on generative adversarial network (GAN) [Goodfellow et al.
2014] and (Tashiro et al., 202 proposed conditional score-based diffusion models for probabilistic
time-series imputation (CSDI) based diffusion model [Sohl- et al.] 2015]. Indeed, these deep imputation methods often achieve state-of-the-art performance in the tasks of missing data imputation, when the distributions of the training data and testing data are identical. However, their performance in recovering the missing values for unsupervised AD is rarely studied.
Anomaly detection on incomplete data The research on anomaly detection in the presence of missing values is very limited. To the best of the authors' knowledge, [Zemicheal and Dietterich 2019] is the first work evaluating the detection performance of anomaly detection methods combined with different data imputation techniques. Their experiments of anomaly detection on a few UCI datasets with missing values showed that implementations of unsupervised anomaly detection methods such as Isolation Forest [Liu et al.] 2008) on incomplete data should always include algorithms for handling missing values and the imputation contributes improving the detection performance of anomaly methods. |Fan et al.] (2022) studied the problem of statistical process monitoring with missing values and proposed fast incremental nonlinear matrix completion method for online and sequential imputation. [ardd et al. [2023) provided a study of existing unsupervised anomaly detection methods on GAN-imputed data.
It's worth noting that the strategies used in [eeicceal and Dietterich) 2019 Fan et al. 2022 Sarda ct al. 2023 are two-stage methods, where the imputation models trained the traming dataset that does not contain any abnormal data or only contains very few unlabeled outliers. As a result, the imputation model will not generalize well on abnormal data during the inference and will use the learned pattern of normal data to fill the missing values of abnormal data, which makes the abnormal data similar to normal data and hence lowers detection accuracy. In contrast, our method integrates data imputation and anomaly detection into a unified process, and alleviates the imputation bias via introducing pseudo-abnormal samples, and hence achieves superior detection accuracy.
3 Proposed Method
3.1 Problem Formulation and Our Motivation
Given n samples X1, X2, ... Xx, drawn from an unknown distribution Dx C R"' the goal of unsupervised AD is to learn a decision function
m - 10 by utilizing only these samples, such that f(x) = 0 if E Dx and f(x) = 1 if x & DDx- We consider the scenario that II E Rm contains missing values. Let M E (0, 1]nxm be a mask matrix
x=
xJ_ [ determined by some missing mechanism M such MCAR, MAR, or MNAR, where M,,, 1 means Xi, is observed and M. = ( means X is missing. Then the observed incomplete matrix is
X
xx,] M(X) X o M
(1)
where @ is the Hadamard product. Equation | implies that the missing values of X are temporarily filled with zeros. In many scenarios such as gene expression data analysis, recommendation systems, and questionnaire surveys, the data missing rate in X is often high. Training an anomaly detection model f on X and using it to detect anomalies in new incomplete data has practical significance
3

---

such as detecting anomalous cells or rare cell types in bioinformatics, identifying abnormal users in recommendation systems, and recognizing unusual subjects using questionnaires of psychology.
As mentioned before, conventional AD methods are vulnerable to missing values and a good impu- tation algorithm can raise the detection aceuracy of anomaly detection method to some extent. However, the strategy sinpute-then-detes and hence cannot provide satisfactory detection performance. Therefore, in this work, we aim to provide an end-to-end unsupervised anomaly detection method in the presence of missing values to mitigate the imputation bias and improve the detection accuracy. The most challenging problem is that the imputation model (denoted as Z) trained only on incomplete normal data cannot generalize well to incomplete abnormal data. To solve the challenge, we take the following strategy and consideration.
is inclined to make incomplete abnormal samples normal
We propose to learn a model that can generate some pseudo-abnormal samples, and then learn an imputation model from both the original normal data and the generated pseudo-abnormal samples. Thus, the learned imputation model can generalize well to incomplete abnormal data during inference and recover the missing values with high accuracy, which further improves the accuracy of anomaly detection. However, we encounter the following issues.
It is non-trivial to generate meaningful pseudo-abnormal samples that are similar enough to real ones. The reason is that the distribution (i.e., D2)) of training data is unknown and the data dimension is often high.
The incompleteness of X further increases the difficulty of generating pseudo-abnormal samples. On the other hand, the generated pseudo-abnormal samples should not be too far from the normal data, where a large gap will make the learned imputation model fail to impute the abnormal samples close to normal data and cause the abnormal samples be hard to detect.
The generating model, imputation model, and detection model should be coordinated with each other and as a whole ensure the reliability of the inference.
3.2 Learning Framework of ImAD
To address the aforementioned challenges, we propose to find a d-dimensional latent space Z where the normal data are lying and then generate pseudo aaborraabnormal samples around the normal samples in Z. The samples in Z will be mapped back by neural network to the original data space, yielding reliable pseudo-abnorma data.
E D2
E D2
We define D, as the latent distribution of the normal data in Z and define D3 as the latent distribution of pseudo-abnormal data in Z. Since the patterns of normality are limited and the patterns of abnor- mality are unlimited, we let D2 be a truncated Gaussian distribution (a hyperball denoted by B3 with radius T1) in Z and assume that the remaining region of Z is the abnormal region, denoted as Z \ B. It should be pointed out that there is no need to define Dz in the entire space 2 \ B. which will be explained in the discussion for Theorem (b) and further supported by Theorem 3.4 in Section 3.4 Instead, we only need to define D3 in a small region of Z B that encloses B, which will reduce the uncertainty of random sampling (or samples size equivalently) and make easier for mapping the samples back to the original data space. Thus, we define Dd as a hypershell surrounding B and let D; be a truncated Gaussian. The radii of the two hyperspheres forming the hypershell are rı1 and ra respectively, where T2 > r1. An illustration of Dr and D3 in 2-D space is shown in Figure 3 where D₂ and D3 are truncated Gaussian from N (O0, 0.52. 12) and (2, respectively. The theoretical analysis for sampling from D2 and D2 is in Appendix A We learn a reconstructor R Rd - R." to transform the samples drawn from D, to the original data distribution Dx, i.e.,
12
Figure 3: Visualization of Dr
and D2 2-D latent space 2.
Dx RD2)).
(2)
R is actually reconstruction model that recovers the original data from the latent space Z. With D3 and R, we can obtain a distribution D; of pseudo-abnormal data in the original data space as
Dx : R(D2))
(3)
The samples (denoted by x) drawn from D3 are reasonable pseudo-abnormal samples, which will be explained by the discussion for Theorem 3.2 a) in Section 3.4

---

Reconstructor
X 2MP()OM
Projector
P22 K2a
725
K2
Imputer
Kat R3g Rxa
= 1
X 1(x)
P(X)
Xyp
R(Z)
saning paint penstastturmal
Figure 4: ImAD framework. X and X denote the normal and bgeudo-abnonen data with missing values, respectively, while X and X are the corresponding imputed data.
Now we use a projector P Rm - Rdd transform Dx and Dx into Dz and D3 respectively, i.e.,
D₂ P(Dx), D2 P(Dx).
However, the training set X -M(X) is incomplete, and we need to learn an imputation model I to recover the missing values, i.e., X I(X). More generally, denote
Dx = I(Dx).
(5)
We hope that the imputation model also able to recover the missing values of the generated pseudo-abnormal samples if they have, though they are complete. We thus remove some values of the generated pseudo pseudo-abnormal samples x Dx using missing mechanism M and let D (Dx).
The missing values are then recovered by
: I((2)
(6)
This step mitigates the problem of imputation bias encountered by the "impute--hen--eteet"" methods.
Let E, Ep, and R denote some distance or discrepancy measure between distributions. We here show how to achieve the goals of (2). (3) ( (5). and (6) in unified optimization problem. First, for normal data, we solve
minimize (I(2,),, D& M) + Ep(P(Ds), D.) E Ex M) I,P,R
(7)
For the generated pseudo-abnormal data, solve
minimize & ( M) ((()))(((((())))))
(8)
I.P.R.
Let E. be finite-sample estimation of &.. Combining and obtain the objective of ImAD:
minimize I,P,R
{I X
MM
EP(((( X IZ: Z)
En(R(P(()) X M)
2000
((AD)
((RE
where x R(2) M, I(X), and [: denotes the row-wise concatenation of two matrices.
In (9). the samples in Z are drawn from D, and the samples in Z are drawn from Dz. The roles of the three parts of the objective function in (9) are analyzed as follows.
DDI denotes the data imputation loss. With this loss, the imputation model will be able to recover the missing values of normal data and abnormal data.
5

---

L(D)) denotes the anomaly detection loss. With this loss, the anomaly detection model will be discriminative and be able project normal data and abnormal data into different regions in Z. (RE) denotes the reconstruction loss. This loss is to ensure that D2 and D3 are meaningful.
We see that our method ImAD couples data imputation with anomaly detection to a unified optimiza- depicts the overall framework of ImAD, where the green and red arrows show the flow paths of normal data (starting from X) and pseudo-abnormal data (starting from Z) respectively. The reconstructors Figure share parameters.
tion objective. Figure 4
3.3 Specific Implementation of ImAD
We use three neural networks h So and Go with parameters eh 0, to model I, P and R respectively. For E, we consider two different cases, the samples are pair-wise, we directly use the square loss, which is simple and efficient. Thus, in LDI and C for CRE is masked by M. When the samples are not pair-wise, we take advantage of the Sinkhorn distance [Cuturi] 2013] derived from the optimal transport theory. The Sinkhorn distance between two distributions Der and Dy supported by their finite samples U V (V1, V2,"",," "2, Vn, D. is defined as
the square loss, and the square loss
{U1, U2
un.) ~ D, and
Sinkhom V) = min C) + Σ Ps (u)), 3
s.t. P1 = a11=
(10)
where P X is the transport and C R x,, X*, is the metric cost matrix. The two probability vectors a and satisfy - 1, = 1, and 2 0 is a trade-off between the Wasserstein distance and entropy regularization.
By applying hop Jo. No. square loss, and Sinkhorn distance to we obtain the following problem:
minimize (fa X),, Z) + all Salho (96 (Z) o M) 0.0.0
((AD)
(11)
3|[[ X
Ap [X; xI)) @ [M + A|I go go o M
((D)
((K)
Solving the problem 11 we get well trained imputer hg- and projector se- For a new sample Xnew containing missing values, we define an anomaly score s(.) by
s s(Xnew) = fl (hey- (Xnew))ll
(12)
which is the distance the origin in the latent space. If s (Xhnew) Fı, Knew is detected as abnormal. Otherwise, Xnew is treated as a normal sample.
3.4 Theoretical Guarantees for ImAD
WLOG, we assume fo. 9d. and hg all have L layers, where 0 (W(, W W o {W?, W ...... W a matrix as || llo and l12,1 respectively. We also make the following assumptions.
and vi (Wh, W W J. Denote the spectral norm and 2, 1norm of
Assumption 3.1. For fe. 9o. and hgg the following conditions hold: 1) |/W/lb ay, [| S ag. [W/ll S On, E [L]: 2) || Il2.1 S by, || S by. || S by VI E [Z]; 3) all activation functions in fo. Po. and he are p-Lipschitz continuous; 4) the maximum width of the layers in fo, Io, and h2 is d.
The following theorem can be used to obtain some deterministic guarantee for ImAD.
Theorem 3.2. Under Assumption 3.1 have:
(a) [gg (z) - go (z) S p² a z|z z holds for any z, 2;
(b) Ilf (h. (x) fo [ 2d a/ - X|| holds for any x and X
Theorem 3.2 a) indicates that in the latent space Z, if an abnormal sample ă ~ Da is close to normal sample z ~ Dz, the original data space, the corresponding abnormal sample still close to the normal sample x provided that ag is not too large. This means the generated pscudo-abnormal

---

b), let's consider an incomplete abnormal sample x and assume that its closest incomplete pseudo-abnormal sample generated by the å on the outer hypersphere (shown in Figure 3 is x*, where || x B. Then in the latent space, we have || 201 S a Let the radii of the inner and outer hyperspheres be ri and r2 respectively. Now we can conclude that if r2
samples are practical and useful. For Theorem b,2
> p 2 is outside the decision region given by the inner hypersphere and hence x is successfully detected as an abnormal sample.
Now we study the theoretical guarantees for our ImAD in the sense of expectation. Let 11 be the thresholds for the anomaly score defined by (12) to determine whether a sample is normal not. Let 72 be the radius of the outer hypersphere enclosing Dz. Let 8x be the average anomaly score of the (incomplete) normal training data, i.c., & - xx,) Let & := A. E:=1 Ir2 s(x,)
where x; are the (incomplete) pseudo-abnormal samples generated during the training stage. With these definitions and Assumption 3.1] the following (proved in Appendix presents the theoretical generalization ability of our ImAD.
Theorem 3.3. Suppose the squared anomaly score x of normal data is always upper-bounded by %, 1r - =xx) of the pseudo-abnormi data is always upper-bounce by F, and the absolute output of fe is always upper-bounded by ô. Suppose the samples in X and X. are independently drawn Dx
and & respectively. Define K. = a < (1 I az
2/3 + I bb ((an) 2/3)) 3/2 A r9 2. and
Å r - e
(a) For normal data from Dx, over the randomness of X.
P P [x()] > ril S 86
(13)
By-488RIn n )² /(992)) and R = U pdnnd2)))) where 5 = cxp (-2m(4
dlIn(dd).
(b) For abnormal data from Diy, over the randomness X,
P ri 1-5,
(14)
where 8 8= 2exp -2m (Ā 18418 2 /(999) and R =
XI Vdln
Theorem 3.3 means that normal sample, in expectation, is detected as anomalous with probability at almost d, where ó close to zero under some mild conditions such as L is not too large and n is
not too small. In other words, a false alarm happens with low probability. Theorem 3.3 (b) means that
an abnormal sample drawn from Dg. in expectation, can be successfully detected with probability at least 1 6, where 8 is close to zero under some mild conditions. Theorem (b) also indicates that a larger r2 is better. It is worth noting that here we only focus on Dz. which defined by Dz,, M. and gø. D can be regarded data. The anomalous samples drawn from space out of D. are much easier to detect, which is further supported by the following theorem (proved in Appendix
distribution of difficult anomalous data that are close to normal
Theorem 3,4. Let be a constant satisfying [e o he gz(z) -o fa o hy go('')l 2 cilz z'll for any z, z' and assume that [l.f o hy o (o) - -Ol e. Any samples drawn from the space out of D can be correctly detected if cr2 E ri.
4 Experiments
4.1 Datasets, Baselines, and Implementation Details
We compare ImAD with "impute-then-des" theeeeeeeeee" methods on 11 publicly available tabular datasets from various fields, including seven datasets with manually constructed missing values and four datasets with inherent missing values. In all experiments, only incomplete normal data are used in the training stage, but there are both incomplete normal and abnormal data during the inference. The statistics of all datasets are in Table [ and detailed description of all datasets is in Appendix Considering the "impute-then-detees
strategy, for data imputation, we use MissForest (Stekhoven and Bühlmann 2012 and GAIN (Yoon et al.) 2018). For anomaly detection, we use Isolation Forest [Liu et al.
7

---

2008] Deep SVDD [Rff et al. 2018), NeutraL AD [Qiu et al. 2021 and DPAD (Fu et al. [024]. The pairwise combination between the imputation and anomaly detection methods yields eight "impute-thenn-dtec"" baselines.
Table 1: Statistics of datasets, The "normal" and "abnormal" denote the number of normal and abnormal samples, respectively. "missing samples rate" means the proportion of samples with missing values and "missing entries rate" means the proportion of all missing values.
dataset
hilld
Teatures
instances
normal
abnormal
Adult
incure census
14
30,162
22,658
7,508
Botnet
cybersecurity
115
13,113
27,494
without
KDD
cybersecurity
121
494,021
396,743
97,278
inherent
Arrhythmia
medical diagnosis
274
452
320
132
missing
Speech
speech recognition
400
3,686
3,625
61
values
Segerstolpe
cell analysis
1,000
702
329
372
Usoskin
cell analysis
25,334
610
232
378
dataset
hield pallern recognition
Teatures
instances
missing samples rall
missing entries rale
with
Titanic
9
79.46%
10,79%
inherent
Mo
recommendation system
498
6,040
100%
82.41%
missing values
Bladder
cell analysis
23,341
2,500
100%
86.93%
Seq2-Heart
cell anulysis
23,341
4,365
100%
88.51%
We use MLPs to construct the three modules of ImAD, Adam Kingma and Ba 2015) as the optimizer and set coefficient of entropy regularization term in Sinkhorn distance to 0.1 all experiments. Other experimental dyper-parameterd are provided in Appendix [ Sensitivity analysis of hyperpparaaees provided Appendix A detailed description of distinct missing mechanisms, including MCAR, MAR, and MNAR, is provided in Appendix [ In this study, we let the missing rate mr be 0.2 or 0.5, which is consistent with the previous data imputation works (Yoon et al., 2018 Muzellec et al. 2020). We use the AUROC (Area Under the Receiver Operating Characteristic curve) and AUPRC (Area Under the Precision-Ree Reecal curve) evaluate the detection performance. ALL experiments were conducted on 20 Cores Intel(R) Xeon(R) Gold 6248 CPU with one NVIDIA Tesla V100 GPU, CUDA 12.0. We report the average results of five runs.
4.2 Experimental Results on Datasets with Manually Constructed Missing Values
Before presenting the numerical results, we show the effectiveness of the generated pseudo-abnormal samples learned for the Botnet dataset in Figure 5 where directly let the latent space 2 be 2-D for convenient visualization. We see that the pseudo-abnormal samples cover the region of real abnormal samples, which matches our motivation and expectation.
sample
noraal sample
missing rate 0.2
pscudo abnormal sample missing rule
Figure 5: Two-dimensional visualization on Botnet.
The results of anomaly detection with missing data under the setting of MCAR are shown in Table 2 and more results under MCAR are provided in Appendix K In Table "Mean-Filling denotes that the missing values are filled with feature means.
We have the following observations from the Table 2
The detection performance of "'impute-then-deted
methods does not decrease with the increas- ing of missing rate from to 0.5 in some cases (emphasized by underline), which indicates the adverse impact of imputation bias for the detection algorithm. The main reason is that

---

Table 2: Detection performance in terms of AUROC and AUPRC (%, mean and std) on datasets with manually constructed missing values under MCAR. mr denotes the missing rate. The best result in each case is marked in bold.
KDD
Adult
DI Methods
Methods
AUROCIS)
AUPRC(SS)
AUROC(S)
AUPRC((S)
mr- 0.2
mi 0.5
107 0.2
mi 0.5
101 m0.2
mr m 0.5
ror 0.2
wI m 0.5
I-Forest
74.71(1.92)
57.44(5.71)
$1.19 31)
64.3715.95)
59.63(0.50 (5)
56.01(0.84)
57.99 (.3))
59,25(0.72)
Maan-illing
Deep SVDD
Nectral AD
95.4K(.17)
96.63(1 .05)
87.39(1.72)
85.32())26)
95.09(1 36)
88.691 36)
62.53(5.35 84.91(1.11)
88. 1012.87)
(0)))
54.64(1.08)
57.37(2.55)
63.20/2.86) 26)
62.28(6.81)
57.43(1.16)
59.05(3.19)
DPAD
52.55(2.41)
53.73(2.45)
55.21(3033)
53.5112.46)
61.9300.17)
19.17(0.35)
66.7200 10)
62.42(0.13)
I-Forest
94.90(1.95)
93.24/2.55) 13.37(2.15)
93.21(1.92
60.0641 69)
60.73/0.69
57.12/2.16 16)
56.80(1.27)
MissForest
Neutral AD
Deep SVDD
93.58/2.46) (4)
1.72)
92. 24)
91)
03.87(1 .7)
85,77 95)
94.88 2.86
791
62.33(4.66)
58.79(1 L8)
55. 55.11(33.40 11)
61. 61.21(2.24)
60.07 60)
55. 35.31/2.9 91)
(2.22(3.611
55-45(1.72)
DPAD
70.65(4.61)
10.80(1.00 1.08)
76.39 (8)
62.0770.000
64.3900
663.82200223 23)
68.6800 31)
64.80(011)))
I-Forcet
82.78/3.88 80)
79.9-10.39)
90,33<1 58)
89.52(1.07)
68. 59.53/0.91)
57.05(1 (2)
56.87(1.09)
Deep SVDD
88.68/4.87)
88. (8.544
$8.36,3 42)
85.4515.67)
58.65 44)
65 2.(22.0))
57.61(4.24
59 59.55/2.34)
GAIN
Nevtral AD
90.48/3 24)
84, ()
84,61(1 30)
84.08(1.71)
55.01/ L8)
56.44/2113
53.00 80)
59.05(3.97)
DRND
20.34/8.20 20)
00.99)
22.29 10.59)
94.49(0.04)
0.188.
(2)
68. (8.21)
ImAD (Ours)
97.01(0.33)
90.78(1.35)
95,96.0 18)
91. (8,50()
(6.51(2.12)
73.42/2.08) (1.19(1663)
71.50(2.02)
Antytthmia
Speech
DI Methods
AD Methods
AUROC(S)
AUPRC(S))
AUROC(S)
AUPRC(S)
mz 0.2
tar
mi
sur 0.5
I-Forest
78.83(1.88)
76.29(1.62)
75.83(0.86)
76.05(2.16 16)
26.28(1.45)
34,54(2.03)
36. (0))0))447)
38,98/0.77 79)
Mesa-Filling
Deep SVDD
66.22(1.89)
62.60/4.67)
71.76(1.8) 81)
66.46(3)93)
53.9015.43)
32.80().57)
57.52(4.23) (2))
(3.75(2.67)
Neutral
79.02(1.91) 74.87(2.15)
81.02/1 97)
76.79(2.34)
49.54(3.911)
50.65(4.58)
50.79 50.79/2.26)
51.82(1.5) .7)
DPAD
78.33(1.47)
73.37(3.23)
78.97(1 46)
74.63(2.41)
40.22(2.77)
42. 42.22(4.16) 16)
46,266 46.26(2239) (9)
45. 45.58(2.01)
I-Forest
60.72(0.62)
81.54(0.95)
27.91(1 85)
(7.95(0.97)
28 (8)55)
29.09(1.14)
36.83/1 .06)
37.29(0.76)
MissForest
Deep SVDD
72.690.991
75.80X4.07)
70.94/0.75 75)
27 55)
60.37(0.87)
40: 14(4. 30)
58.9331
42.00/22 16)
Neutral, AD
44,30/2.11 47,35(4.81)
11)
50.87 53)
50.12(2.55)
56.51(4.87)
14.77().77)
55. 15.4444.30 36)
52, (..97)
DPAD
80.79(1.11)
82.64(1.29
80.35(1.76)
8.43))1.42)
44.8115.10 10)
43.32(3.71)
48.57(3.26)
47.63(3.88)
I-Forest
87.19(081)
76.29X1.5
76,40N1 86)
76.29(1.39)
29.33(0.59)
29. 29.23(2.133) 13)
(9.9220021)
40 (0.0440063)
GAIN
Neutral. AD
Deep SVDD
17,14(5.41)
37.96(5.09)
48,86/2.355 35)
42.57(2.55 56) 33.98/4.12)
59. 69.35(2.58) 58)
54.03(2.45)
56.8014.89) (2.35(1.96)
51.95(1.79)
16. 46.54((2100 10)
57.24(5.51)
54.76/4.55 58)
(4.38(0.96)
(7.54(L.75)
55.05(5.55)
DPAD
79.07(1.29)
10.11(4.13)
76.48(0.97)
79.67(4
(1.77)
26.46% 24.95(2.116)
49, 49.69%2.188
ImAD (Ours)
82.24(1.76)
81,76(1.19)
83.74(1.85)
(3.37(
61.94/2.77)
58.66(1.40)
600.43(333)
(8.13(1.48)
lower missing rate implies a simpler imputation task, leading to a more pronounced imputation bias from normal data, which makes the abnormal data more "normal", thereby increasing the difficulty of detection for such two-stage methods.
The "impute-then-deteet"" methods with "MissForest" (simple and shallow imputation algo- rithms) achieve better detection performance than those with "GAIN" (generative and deep imputation model) in most cases, suggesting that sophisticated imputation module may not contribute positively to subsequent anomaly detection because the identical distribution assump- tion does not hold here. The outstanding recovery ability leads to a pronounced imputation bias and further affects the detection task.
Compared with all baselines, ImAD achieves better detection performance in almost all cases. Besides, different from the "impute-there teieeetthee-eeee" methods, the performance of ImAD increases with the changes of missing rate from 0.5 to 0.2 all cases. This indicates that the imputation module of ImAD generalizes well on incomplete abnormal data and the generated pseudo- abnormal samples can alleviate the bias.
4.3 Experimental Results on Datasets with Inherent Missing Values
We report experimental results on the four datasets with inherent missing values in Table where the naive imputation methods "Zero-Filling" and 'Mean-Filling" are also considered. Observing Table 3 we notice that the naive imputation methods are insufficient for subsequent detection tasks when facing high missing rates and the imputation bias impacts the detection accuracy of "impute-then- detect" methods. Our ImAD outperforms all baselines in all cases. It indicates that our proposed method is practical and effective for real-world anomaly detection with missing data.
4.4 Impact of Different Missing Mechanisms
Given a real dataset, the missing mechanism is usually unknown and difficult to estimate. It is expected that when the missing mechanism M in generating (incomplete) pseudo-normal samples is closer to the missing mechanism M in the real data, the performance of ImAD should be better. In this section, analyze the impact of different M on the detection performance of ImAD. Note that
9

---

Table 3: Detection accuracy (AUROC and AUPRC (%, mean and std)) on four real-world datasets with inherent missing values. The best result in each case is marked in bold.
Methods
Methods
AUROOQ
Titanic
AUPRC(5)
AUROU(S5)
Movicl.ensIMM
AUPRC((5
(UR
Blokke
AUPRCI(5]
AUROUT))
See2-
AUPROC(5)
I-Ferest
77,44(0.29)
72,74(0.77)
35.53/0.43)
40.87(0.32)
32.22/0.72)
19.23/0.41
48.78/0.63)
46.25/0.54)
Zen-Filling
Neutral.
Deep SVDD
49.94(0.58)
51.03(1888)
47.10(1.11)
53.19(3211)
41.38(0,98)
39.2910.79)
44.05(0.72)
(3.4400.56)
63
68.40/2.57)
76.32(2.44)
71.12(0.90)
82.84(3335
78.
65. 65.19(0.95)
DPAD
79.50(0.91)
79.07(1.30)
44.10(3446)
46.12/2.29)
99.99/0.00)
99.99/0.00)
95 95.31(0.32)
93.38/0.69)
I-Forest
79.60(0.665
78.64(0.94)
44.06/3.12) 41.47(0.46) 36.30)0.75)
46.63(3 88)
54.69/2.33 32)
51.98/2.07)
Mean-Hiling
Neutral
Deep SVDD
66.15/2.00)
53.87(0.09)
$2.41
(3.38)2.77)
38.34(0.69)
48. (8(.06)
46 (0.93)
81 (1.9733344)
15.15
78.89(3.78)
99.38/2.64
89
75. 16116111 05)
71. 71.12(0.55)
86.5718.50
DPAD
67.02(0.66)
69.850 (0.85(
47.74(4.08)
48.52/2.89)
97 97.52(1.43)
97,88:1
77.93(10.20)
(6.37)
I-Forest
79.72(0.28)
78, 00.5000.33 331
36, (4(3 (44)
41 (5)
44 44.53(2.05 84)
46 84(1 94)
64. 64.588 98)
5K S6
MissForest
Neutral.
SVDD
60.4618.59)
54.6334.442
52. 12)
7.73)
$6,04(0.49)
18.14(1.18)
5.707(1.722)
(410.4)
95. S3(1 (155)
68.0112.50 66.41(4.66)
97.04/0.81 81)
91.50(1.15
94. 20.29(0 42)
92. 30(0455)
90.87(1.16)
DPAD
68.18(1.45)
(0.31
47.500(440)
48 (3.09))
(6.96(1130) (1)30)
97.229 (77)
7% 78.0355.20 20)
74.67(5.33)
I-Forest
79.46(0.79)
78 691699999
64.84(1.44) (44)
62.63 (99)
45 45.77(2.300 39)
47 47.62222.19) 19)
64 64.622(333) 31)
58 (2(2.22)
Deep SVDD
70.59(4.54)
66/43(1.17)
58.99(
56.68(2.11)
76.78:447)
Ncutral. AD
55.71(2.74)
51,2522255)
50,72(2. 88)
51,4712.30 38)
65 3013 98)
68/4 69)
91 41,45
90 7M0 36)
DPAD
78.12(0.97)
77.41 (04)
59 19.66(1.86)
58 (1.)))9)))
06.89(1.885
97.25(1 56)
74 14-99(2.77 76)
73. 16( 1.93)
ImAD (Ours)
$2,09(0.99)
81.3910.84)
66.32(1.36)
65.34(1.35)
100/0.00)
100:0.00)
96.62(0.11)
96.40/0.19)
for the synthetic incomplete data, we accurately know the missing mechanism. The experimental results are reported in Table On real incomplete data, our method is robust to the setting of missing mechanism M and has better overall performance when M is MCAR. Therefore, based on Occam's ' Razor principle and the empirical results, we recommend using MCAR as the missing mechanism for the generated aecudo-bnnormal samples when the real missing mechanism is unknown. On the other hand, shown in Table 4. synthetic incomplete data, detection performance degrades when M is different from M.
Table 4: Performance comparison of different missing mechanisms M.
Missing Mechanism Pscudo-Abnonee Samples
Missing Mechanism M
Dataset
MCAR
MAR
MNAR
of Normal Data
AUROC(%)
AUPRC(S}
AUROC(%)
AUPRC((S)
AUROC(S))
AUPRC(SS)
Titanic
Unknown
82.09
81.39
79.06
77.08
80.50
79.17
MovieLensIMM
Unknown
66.32
65,34
63.14
63.39
61.44
60.91
Bladder
Unknown
100.00
100.00
99.95
99.95
100.00
100.00
Seq2 Heart
Unknown
96.62
96.40
96.79
96.60
95.56
Adult
MCAR
71.19
71.50
64.11
66.44
67.28
66.72
Adult
65.66
67.23
74.61
70.74
71.14
Adult
MNAR
70.69
69.17
68.35
68.78
71.60
68.97
4.5
More Experimental Results
The appendices contain the following additional results: I. Performance gain from pseudo-abnormal samples (Appendix G); II. Influence of the constrained radii Tı, 12 (Appendix H): III. Sensitivity analysis of hyperparameters (Appendix[H [ IV. Impact different missing rates for training and test set(Appendix K V. Results of MAR and MNAR (Appendix K).
5
Conclusion
This paper proposed ImAD, the first end-to-end unsupervised anomaly detection method on incom- plete data. ImAD integrates data imputation with anomaly detection into a unified optimization objective and automatically generates pseudo-abnn abnormall samples to alleviate the imputation bias. We theoretically proved the effectiveness of ImAD and empirically evaluated ImAD on multiple real-world datasets. The results showed that ImAD mitigates imputation bias from normal data and provides an effective solution for unsupervised anomaly detection in the presence of missing values. One limitation of this work is that we haven't considered the applications on incomplete image data and incomplete time series.
10

---

Acknowledgments
This work supported by the General Program of Natural Science Foundation of Guangdong Province under Grant N0.22224/151001111 the National Natural Science Foundation of China under Grant 60.62276236, the Guangdong Provincial Key Laboratory of Mathematical Founda- tions for Artificial Intelligence [2023B212201000 Shenzhen Science and Technology Program ZDSYS2023062609 3020066 Shenzhen Stability Science Program 2023, and Hetao Shenzhen- Hong Kong Science and Technology Innovation Cooperation Zone Project (No. (No.HZQSW WS-SCCYBB- 2024016). The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References
Peter L Bartlett, Dylan Foster, and Matus J Telgarsky. Spectrally-norrmaiizedd margin bounds for neural networks. Advances in neural information processing systems, 30, 2017.
Barry Becker and Ronny Kohavi. http:://doi.or//1.22244.4 0.2443//
Adult. UCI Machine Learning Repository, 1996. DOI:
Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pages 93-104, 2000.
Jinyu Cai and Jicong Fan. Perturbation learning based anomaly detection. Advances in Neural Information Processing Systems, 35, 2022.
Emmanuel Candes and Benjamin Recht. Exact matrix completion via convex optimization. Commu- nications of the ACM, 55(6):111-11) 2012.
Zhenpeng Chen, Jie M Zhang, Max Hort, Mark Harman, and Federica Sarro. Fairness testing: A comprehensive survey and analysis of trends. ACM Transuctions on Software Engineering and Methodology, 2023.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1): 1-22, 1977.
Botao Fa. Ting Wei, Yuan Zhou, Luke Johnston, Xin Yuan, Yanran Ma, Yue Zhang, and Zhangsheng Yu. Gapelust a light-weight approach distinguishing rare cells from voluminous single cell expression profiles. Nature Communications, 12(1):4197, 2021.
Jicong Fan and Tommy Chow. Deep learning based matrix completion. Neurocomputing, 266: 540-549, 2017.
Jicong Fan and Tommy WS Chow. Non-linear matrix completion. Pattern Recognition, 77:378-394 2018.
Jicong Fan, Lijun Ding, Yudong Chen, and Madeleine Udell. Factor group-sparse regularization for efficient low-rank matrix recovery. Advances neural information processing Systems, 32, 2019.
Jicong Fan, Yugian Zhang, and Madeleine Udell. Polynomial matrix completion for missing data im- putation and transductive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3842-3849, 2020.
Jicong Fan, Tommy W. S. Chow, and S. Joe Qin. Kernel- statistical process monitoring and fault detection in the presence of missing data. IEEE Transactions on Industrial Informatics, 18 (7):4477-44) 4487, 2022. doi: 10.1 10.1109//TI..20 109/TT1.002131113377
Jicong Fan, Rui Chen, Zhao Zhang, and Chris H.Q. Ding. Neuron-enhanced autoencoder matrix com- pletion: Theory and practice. In The Twelfth International Conference on Learning Representations, 2024.

---

Dazhi Fu, Zhao Zhang, and Jicong Fan. Dense projection for anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages
2024.
Lovedeep Gondara and Ke Wang. Mida: Multiple imputation using denoising autoencoders. In Advances Knowledge Discovery and Data Mining: 22nd Pucific-Asia Conference, PAKDD 2018, Melbourne, VIC, Australia, June 3-6, 2018, Proceedings, Part III 22, pages 260-272. Springer, 2018.
Ian Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
Soyeon Caren Han, Taejun Lim, Siqu Long, Bernd Burgstaller, and Josiah Poon. Glocal-k: Global and local kernels for recommender systems. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 3063-3067, 2021.
Xiao Han, Lu Zhang, Yongkai Wu, and Shuhan Yuan. Achieving counterfactual fairness for anomaly detection. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 55-66. Springer, 2023.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations, 2015.
Steven Cheng-Xian Li, Bo Jiang, and Benjamin Marlin. Misgan: Learning from incomplete data with generative adversarial networks. arXiv preprint arXiv:1902 2019.
M. Lichman. Uci machine learning repository, 2013, URL http::/arcciieee...... ics wuc.iiis.......................i ecuuml
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. 2008 eighth ieee international conference data mining, pages 413-422. IEEE, 2008
Inke Mayer, Aude Sportisse, Julie Josse, Nicholas Tierney, and Nathalie Vialaneix. R-miss-tastic: a unified platform for missing values methods and workflows. arXiv preprint arXiv: 04822, 2019.
Rahul Mazumder, Trevor Hastie, and Robert Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. The Journal of Machine Learning Research, 11:2287-2322, 2010.
Yair Meidan, Michael Bohadana, Yael Mathov, Yisroel Mirsky, Dominik Breitenbacher, Asaf, and Asaf Shabtai. detection of botnetttetttttoeeeoeooooeeeeeeet botteetattackss_aaaaaaaa attaccs_n_naaaaa baiot. UCI Machine Learning Repository, 2018. DOI: https://oi.oggg1022233///551
Boris Muzellec, Julie Josse, Claire Boyer, and Marco Cuturi. Missing data imputation using optimal transport. In International Conference Machine Learning, pages 7130-7140. PMLR, 2020.
Tomáš Pevny. Loda: Lightweight on-line detector of anomalics. Machine Learning. 102:275-304, 2016.
Therese D Pigott. A review of methods for missing data. Educational research and evaluation, 7(4): 353-383, 2001.
Chen Qiu, Timo Pfrommer, Marius Kloft, Stephan Mandt, and Maja Rudolph. Neural transformation learning for deep anomaly detection beyond images. In International Conference on Machine Learning, pages 8703-8714, PMLR, 2021.
Shebuti Rayana. Odds library, 2016. URL httpp://///ddd http::///dddd....ssoos..ee sto.....oooo...oce.
Patrick Royston and lan R White. Multiple imputation by chained equations (mice): implementation in stata. Journal of statistical software, 45:1-20, 2011.
Donald B Rubin. Inference and missing data. Biometrika, 6313)558--592, 1976.
12

---

Lukas Ruff, Robert Vandermeulen, Nico Goermitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference on machine learning, pages 4393 PMLR, 2018.
Kisan Sarda, Amol Yerudkar, and Carmen Del Vecchio. Unsupervised anomaly detection for multivariate incomplete data using gan-based data imputation: A comparative study. In 2023 s Mediterranean Conference on Control and Automation (MED), pages 55-60. IEFE, 2023.
Nicholas Schaum, Jim Karkanias, Norma F Neff, Andrew P May, Stephen R Quake, Tony Wyss- Coray, Spyros Darmanis, Joshua Batson, Olga Botvinnik, Michelle B Chen, et al. Single-cell transcriptomies of mouse organs creates a tabula muris: The tabula muris consortium. Nature, 562(7727):367, 2018.
Gabriel L Schlomer, Sheri Bauman, and Noel A Card. Best practices for missing data management in counseling psychology. Journal of Counseling psychology, 57(1):1, 2010.
Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. Estimating the support of high-dimensional distribution. Neural computaation 13(7):143314771, 2001.
Ása Segerstolpe, Athanasia Palasantza, Pernilla Eliasson, Eva-Marie Andersson, Anne-Christine Andréasson, Xiaoyan Sun, Simone Picelli, Alan Sabirsh, Maryam Clausen, Magnus K Bjursell, et al. Single-cell transcriptome profiling of human pancreatic islets in health and type 2 diabetes. Cell metabolism, 24(4):593-60) 607, 2016.
Guy Shani and Asela Gunawardana. Evaluating recommendation systems. Recommender systems handbook, pages 257-297, 2011.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan. and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, PMLR, 2015.
pages
Daniel Stekhoven and Peter Bühlmann. Missforest-non- non-paramttric missing value imputation for mixed-type data. Bioinformatics, 28(1):112-118) 2012.
Yusuke Tashiro. Jiaming Song. Yang Song, and Stefano Ermon. Csdi: Conditional ccore-based diffu-
sion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804 24816, 16, 2021.
Dmitry Usoskin, Alessandro Furlan, Saiful Islam, Hind Abdo, Peter Lönnerberg, Daohua Lou, Jens Hjerling- Leffler, Jesper Haeggström, Olga Kharchenko, Peter V Kharchenko, et al. Unbiased clas- sification of sensory neuron types by large-scale single-cell ma sequencing. Nature neuroscience, 153)):4451133, 2015.
Feng Xiao, Jianfeng Zhou, Kunpeng Han, Haoyuan Hu, and Jicong Fan. Unsupervised anomaly detection using inverse generative adversarial networks. Information Sciences, 689: 2025. ISSN 0020-0255. doi: https://doi.org// 10. 101 2t..202222.33...
Zhihai Yang and Zhongmin Cai. Detecting abnormal profiles in collaborative filtering recommender systems. Journal of Intelligent Information Systems, 48:499-518, 2017.
Jinsung Yoon, James Jordon, and Mihaela Schaar. Gain: Missing data imputation using generative
adversarial nets. International conference on machine learning, pages 5689-5698. PMLR, 2018
Tadesse Zemicheal and Thomas G. Dietterich. Anomaly detection in the presence of missing values for weather data quality control. In Proceedings of the 2nd ACM SIGCAS Conference on Computing and Sustainable Societies, COMPASS 19, page 65-73, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781 17814503671411. doi: 10.1 10.145/331
URL httphhttt////olt orgo 10. 1145/3314344 3332490
I Lihua Zhang and Shihua Zhang. Comparison of computational methods for imputing single-cell ma-sequencing data, IFFF/ACM transactions on computational biology and bioinformatics, 17(2): 376-389, 2018.
13

---

Yunhe Zhang, Yan Sun, Jinyu Cai, and Jicong Fan. Deep orthogonal hypersphere compression for anomaly detection. In The Twelfth International Conference on Learning Representations, 2024.
URL htt.pppwwweeieeeeeeeeoorrrraaaaa
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International conference on learning representations, 2018.
14

---

A Analysis for Sampling in Latent Space
To project the normal data onto target distribution and generate pseudo-abnormal data, our proposed method involves sampling from two latent distributions 2., D3 ~ (0, o²1,). In this section, we provide a lower bound for the constrained sampling radius r, given a sampling probability p that provides probabilistic guarantee, namely, to obtain N points from truncated Gaussian, we need to sample N/p times from a Gaussian distribution. Subsequently, we perform sampling within the truncated Gaussian distribution with constrained radius r.
For target distribution Da, compactness is to ensure a clear and reliable decision boundary between normal and abnormal data. Therefore, we select truncated Gaussian from (O0, o La) as target distribution D3 and bound D2 in d-dimensional radius r hyperball centering at origin. For radius r, we have the following proposition.
we expect that it is compact and can be easily sampled, in which the
Proposition A.1. Let Fa denote the cumulative distribution function (CDF) of the chi-square (p),
distribution (d). For a given probability < < 1, when r 2 ov (p), the sampling probability in Da satisfies P(||2|2 < r2) 2 p where z = |2,, : zal and ; itd N(0,0²)
Proof. We have
dd ~ N(O,1) = Σi- ot
21
iid. ~
N(0,02) =
σ² ~ (d).
(15)
Let Y Σ
we get
a²
P(Y < FJ (p)) =
d
2i
P (p) =
(16)
P
2 Fr (v)
2
9) p.
P
@ov Fr 1Tt (p)
Therefore, r 2 o Fí (0) - ((|2|22 < 2p.
According to the analysis in Section 3.2, we select truncated Gaussian from ((0, as target distribution D2 and bound D2 between two d-dimensional hyperspheres with radii rı, 72 respectively, centering origin, where re > T1. For radius F,, r2, we have the following proposition.
Proposition A.2. Let Fa denote the cumulative distribution function (CDF) of the chi-square distribution x² (d). For given probability p <1, when rı S L (p1), r2 ō0 Fa (p2) and satisfies p P2 Pı, the sumpling probability in Dr satisfies P(r? < ||z||2 < r2) 2 p where (0) z [21, [2, 22, ... za] and , za ~
itd.
Proof. According the proof for Proposition A.I
we have
Fa (p) = P[[|2| < <T²) 2 p. r 2 8
(17)
Therefore,
S /v Fr (pt)
= 2(||2|2- < < S Pl. and
(18)
22 2 /v FJ (p2) P[|||| < < ž p2.
Therefore, get P (|z/|| < r3) P(||2||
<rt) P (1 < | < <
P2 P1
15

---

As shown in Figure we set radius of D2 equals to radius of Dz. Also, we maintain the settings ri in all experiments make the introduced pseudo-abnormal samples are far from the normal data.
B Proof for Theorem 3.2
Proof. Recall that Io was defined as
(o(z) = oo
(19)
Then for any z, Rd, we have
(z) - 2o [2)
=l 02 -
<pl (-. (-.--2
- ///p:///W)aaaaaa///./ooo\/o\---3
Spl (,
(20)
L
II
[ l1
||z - 21|
1zz
This finished the proof for part (a) of the theorem. The proof for part (b) is similar and omitted here for simplicity,
C Proof for Theorem 3.3
We define the following model class
F ( fo o hep R R)
(21)
where So and his satisfy the Assumption 3.1 and the sum of squares of the outputs of so hig corresponding the definition of the anomaly score, meaning s(x)² π (x)))) The following lemma (proved by Appendix provides the covering number bound of F.
Lemma C.1. Under Assumption 3.1 for any > 0, it holds that
X
In (n(
(20)) ?
In (c, Fx. 1.
<
22
2/3
2/3 3
where 6² a2F oZt and ς 1 T.
bn.
Suppose the loss function is y-Lipschitz, follows from Lemma C.I
that
IN (5, 2o 11 In
F 11-
(22)
X 2
With the covering number, we can bound the Rademacher complexity by the following Dudley entropy integral:
Lemma C.2 (Lemma A.5 of [Bartlett al. 2017], reformulated), Let F,, = to be real-valued function class laking values in [0, 7), and assume that O E Fz.. Then
4or
12 TVR v (e,.l
Rx(F))
inf a>d
Vn
16

---

Combining 22
and Lemma C.2
and letting R := +1 1 in(22)) p20(22) ( dr²2 we obtain
40y vn
12
nive
R
Ro(Fa)
inf a=0
+
de
ya
12R
n2
inf a>0
+
vni
4y
12
(23)
n
7
where we have chosen a v
The following lemma the classical generalization enror bound based on the Rademacher complexity, Lemma C.3. Given hypothesis function space F mapping
X to R and y O, define F. == {(x,)) + ((f(x),)) : f E F}. where 1,0. v) I 7. Then, with probability at least
1 å over sample X of size every f E F Fstiisiies (f) - (f) + 2Rx(Fx) + 37V (2/5)
Now using Lemma
C.3 and inequality
23
have
87 24R In
In (2/6)
L, S Lt,(()
+ 371
(24)
n
2n
For Theorem 3.3 a)) the loss function is ((.. 1/ yl 9. where y and g s(x) due
to the definition of the anomaly score. This also means that the Lipschitz constant μ of & is 1. We assume that the squared anomaly scores on the normal training data are upper bounded by y. Let
(f) = >; s(x,)² and (f) ED, [x(x)²] It follows from (24) that
(x,)² 8y + 24Rlnn
In (2/0)
ED (x)
+ +37%
(25)
n
2n
Let rı be the threshold determined by the training data to judge whether a sample is anomalous or not. We let
Σ sxx)
8y + 24R
+ 371
In (2/0)
r
(26)
71 1=1
n
2n
and solve for 6:
2n r 52
2442
8 = 2exp
(27)
9-2
where 52 ET sx..) Then we rewrite (25) as
Ep (xx) 1
(28)
which holds with probability least 1 - 8. In other words.
[x(x) > + < 8,
(29)
which implies
[s ~
(30)
because both s(x) and are nonnegative. We complete the proof for Theorem 3.3 a).
For Theorem b,),), R = 1 X 2 (2d ) (20) (t function
We consider the following loss
( 2 - xx)
(31)
The Lipshitz constant of this loss is 1. Suppose the loss is upper bounded by 5. Similar to the proof for Theorem 3.3 (a), have
n
85 ln + 35
In (2/8)
ED, [((x)] S 71
((xx) +
(32)
2n
17

---

Let
85 + 24R hn
In (2/6)
(x)) +
3/1
7
(33)
n
2n
1=1
and solve for d:
2n T
85/4 In
= exxp
(34)
94.2
where E= -.= ((2)). Then we rewrite 32 as
Ep, [12 s ( S.
(35)
which holds with probability at least 8. In other words,
P ED, [12 5 s(x) - 2 1-
(36)
Since Ir3 - -xx)) implies that s(x)2 2 r T, letting = rf. arrive
ED [s
(x)
| 6,
(37)
which also means
P
ED [x(
2 n
21 -
(38)
because both (x) and rı are nonnegative. Renaming as ö, we finish the proof. Note that in the theorem of the main paper, we have put outside the constant 4 in R². That's why the constant 24 becomes 48.
D Proof for Theorem 3.4
Proof. The anomalous samples (denoted by x) drawn from space out of Dxx are much easier to detect. The reason is that, in the latent space, these anomalous samples (denoted by ž') are sufficiently far from the normal region (Dz). According to the definition of Dx we have = f foh(() = (()
According to the definition of the anomaly score, need to measure I OJl. We have
x Oll
= If o g(ž) f g
+ g(0) 0ll
2 o ((ž) f h g(0) - ho - Oll
2c12 011 h
2cr2
where is some constant depending on the networks and we have assumed that [|ʃ g() S Now suppose that 72 sufficiently large such that cr2 - E > rı, then / -0] > rı, meaning that z' is outside the inner hypersphere and hence can be detected successfully. Nevertheless, determining an exact is still an open problem for neural networks.
E Proof for Lemma C.1
Proof. In F, can be regarded as additional layer of the neural network, where the activation function for each element of z is square, the weight matrix for the output is a vector consisting ones, and the activation function for the final output is linear. Thus, fe Iy has 2L + layers. For the square activation function, the Lipschitz constant is 20. For the final output layer, the spectral norm of the weights is Vd, which is equal to the l2.1 norm because is a vector.
Lemma E.1 (Theorem 3.3 of BBrrlettt al. 2017 Let fixed nonlinearities (ø1...., OL) and reference matrices (M1...., ML) be given, where os is pi-Lipschitz and o, = 0. Let spec- tral norm bounds (S1,.... SL), and matrix (2, 1) norm bounds (bz...., bb) be given. Let data matrix X paxd be given, where the rows correspond to data points. Let Hx denote
18

---

the family of matrices obtained by evaluating X with all choices of network FA Hx == { (x*) : A (AI...., AL), -1lAill S 8t, ||A - M, 112,1 S bzj. where each matrix has di-
mension most W along each axis. Then for any > 0,
| In (2W2)
L
b;
2/3
3
In (Hx,e, (Ix) <
11
$333
84
Then using Lemma
E. and Assumption 3.1
we can obtain
X
p In
lnd
(20)
In xx.
22
where k = az ah
2L and Ç = 11+ +
by
2/3
bn.
2/3
3
This finished the proof.
F
Time Complexity Analysis
The notations used in the complexity analysis are explained as follows:
Tt, m to denote the number of samples of the training phase and inference phase, respectively. Missforest is well-known data imputation algorithm based on random forest (O(t, 2 n log n.)
where i denotes the number of trees, v denotes the number of attributes.
T, g. Ta, Tae: Toc denote the iterations of corresponding methods.
L and d denote the number of layers of the neural network and the maximum width of the layers of the corresponding models, respectively.
t denotes the number of trees of I-Forest and is the maximum iterations of the Sinkhorn algorithm.
P. v. K denote the key parameters of the corresponding methods.
Table 5: The time complexity of training and inference.
DI Methods
AD Methods
Time Complexity (Tmining)
Time Complexity (Inference)
I-Forest
OfT' OfT'-pC: ver log - by Volog
Ofe(ts a n) + fa mm log al)
MissForest
SVDD
O(T plt) nlogn) (Tue + MndFL -n])
Oleltt loga) (muL m))
DJT -Pth nlggn)
Neutral. AD
OIT play e loggo) 4 T(nd + w.K)) KY)
O o loge) + (mdPT + K))
DPAD
O(T plt pll----n ve-nlogn)) + T[nd Lxn))
Olplls mm log n) (ndd [oddL+)on)
I-Forest
OC(T + Tayod b- (logs))
Oimal E molgoo Olned L+ (mdd m)) O (InP - LyndT, O [md]L mml)
GAIN O(T TginPL)
Deep SVDD
Orr, Taind L (Tor Too) + +))
NeutraL AD
O + Ta)nd TinP - K))
K))
DPAD
Olli + T.lnn Tmn2L +2))
ImAD
O(mad L+ m)
G
Gain of Detection Performance from Pseudo-Abnormal Samples
G.1 ImAD Benefits from Learning Pseudo-abnormal Samples
In this section, we explore the influences of introduced pseudo-abnormal samples for detection performance. On all datasets used in our experiments, remove the pseudo-abnormal samples in the training process and only use incomplete normal data to train ImAD. The experimental results shown in Ta and Table 7 where the detection performance of ImAD is improved in all the cases when introducing pseudo-abnonie generated useudo-abnorme samples are practical and effective for anomaly detection incomplete
samples into the training process. This indicates that the
data.
G.2 ImAD's Pseudo-abnormal Samples Can Improve "impute-then-detecr" Methods
Furthermore, we save the generated pseudo-abnormal data from the training process of ImAD on Titanic and Bladder, and then we add them into the training set for data imputation of "impute-then- detect" methods. The related results are provided in Table g We that the pseudo-abnormal
19

---

samples learned by our ImAD can improve the performance of reason that with the pseudo-abnormal data, the imputation algorithms. i... MissForest and GAIN, generalize better on the test data. These results further confirm the effectiveness of the generative module of our ImAD
methods. The
Table 6: Gain of detection performance of ImAD from pseudo-abnormal samples on datasets with manually constructed missing values.
AUROC(S)
AUPRC(%)
Datasets
Settings
mnr0..2
ml=0.55
mr=0.22
mr=
ImAD w/o pscudo-abnormal samples
65.30 (4.36)
68.15 (5.66)
67.49 (3.98)
69.50 (4.35)
Adult
ImAD
76.51 (2.12)
71.19 (1.63)
73.42 (2.08)
71.50 (2.02)
ImAD w/o pseudo-abnomin samples
96.00 (2.03)
91.50 (1.59)
94.70 (1.03)
92.09 (1.14)
KDD
ImAD
97.01 (0.33)
90.78 (1.35)
95.96 (0.18)
91.58 (0.32)
ImAD w/o pscudo-abnormal samples
99.78 (0.05)
99.38 (0.16)
99.78 (0.05)
99.40 (0.15)
Botnet
ImAD
99.71 (0.22)
99.53 (0.25)
99.68 (0.24)
99.58 (0.20)
ImAD w/o pscudo-abnormal samples
78.28 (4.03)
79.04 (0.73)
76,98 (3.55)
78.27 (1.88)
Arrhythmia
ImAD
82.24 (1.76)
81.76 (1.19)
83.74 (1855
83.37 (1.36)
ImAD w/o pseudo-abnormal samples
53.22 (3.62)
47.28 (4.27)
53.47 (4.66)
49.92(3.13)
Speech
ImAD
61.94 (2.77)
58.66 (1.40)
60.43 (3.33)
58.13 13 (1.48)
ImAD pseudo-abnormal samples
97.29 (0.93)
96.53 (1.15)
96.62 (0.90)
96.60 (1.16)
Segerstolpe
ImAD
(0.88)
(0.67)
98.98 (1.18)
96.85 (0.54)
ImAD w/o pseudo-abnormal samples
79.07 (2.13)
76.67 (1.27)
79.43 (0.94)
78.57 (2.48)
Usoskin
ImAD
84.95 (1.29)
79.23 (2.49)
85.48 (2.34)
80.06 (3.40)
Table 7: Gain of detection performance of ImAD from pseudo-abnormal samples on datasets with
inherent missing values.
Datasets
Settings
AUROC(%)
AUPRC(S6)
ImAD w/o pseudo-abnormal samples
76.14 (1.66)
75.46 (1.32)
Titanic
ImAD
82.09 (0.99)
81.39 (0.84)
ImAD w/o pscudo-abnornal samples
60.62(1.35)
60.29(1.44)
MovieLensIM
ImAD
66.32 (1.36)
65.34 (1.35)
ImAD w/o pscudo-abnormal samples
99.90 (0.21)
99.87 (0.29)
Bladder
ImAD
1.00 (0.00)
1.00 (0.00)
ImAD w/o pseudo-abnormal samples
95. (0.71)
94.27 (1.18)
Seq2-Hear Heart
ImAD
96.62 (0.11)
96.40 (0.19)
Table 8: Gain of detection performance provided by the pseudo-abnormal samples for "impute-then- detect" methods.
Settings
Titanic
Bladder
Imputation Method
AD method
AUROC(%)
AUPRC(%))
AUROC(%)
AUPRC(%)
MissForest
IForest
79.72
78.50
44.53
46.84
MissForest with pseudo-abnormal data
IForest
79.24
78.72
50.33
51.67
GAIN
IForest
79.46
78.69
45.77
47.62
GAIN with pseudo-abnornal data
IForest
79.91
79.47
50.29
52.83
H Influence of Constrained Radii 11,72 for Detection Performance
In this section, we explore the influences of constrained radii r1, 12 detection performance. We change the latent dimension d = {4. 8, 16, 32, 64, 128, 256, 512) and carry out related experiments. Detailed experimental settings and results are provided in Table and Figure and Figure respectively.
20

---

Adult
Adult
missing_rate= rate=002
AUROC(
80
m rate=0.5
AUPRC(%)
missing rate=
60
60
16
32
64
128
256
512
16
32
64
128
256
512
Latent Dimension
Latent Dimension
Botnet
Botnet
AUROC(%)
100
AUPRC(%)
100
90
90
missing_rad rate 0.2
missing_ raweegrraaaaaaeeeeeee 0.2
missing_ran rate 0.5
missing raaeenng_rrreeeee 0.5
80
16
32
64
128
256
512
16
32
64
128
256
512
Latent Dimension
Latent Dimension
KDD
KDD
100
AUROC(%)
AUPRC(%)
90
90
missing_rade rate=0.2
missing_raed rate=aater 0.2
missing rate=0.5
missing rate= 0.5
70
70
16
64
128
256
512
32
128
256
512
Latent Dimension
Latent Dimension
Figure 6: The detection performance Adult, Botnet and KDD with different latent dimension.
As showed in Table 9 we change the dimension d of latent space and then get r oV Fd (p) (See
Proposition A.1 and set target distribution Dz O..00.52 I,), D2 ~ \O0 Ia) and set p = 0.9.
Table 9: The constrained radii r,, 12 under with different latent dimensions.
Latent Dimension (d)
Radius
16
32
64
128
256
512
r1 = 0.5 F7 (0.9)
1.39
1.82
2.42
3.26
4.44
6.10
8.45
11.76
72 F1 (0.9)
2.78
3.65
4.85
6.52
8.88
12.20
16.90
23.52
Figure and Figure shows the fluctuation of detection performance with different latent dimension d. It can be observed that method is not very sensitive to changes in the radii 71 and r2, but its performance degrades with a reduction in the latent dimension. This is reasonable since the smaller latent dimension results in more information loss.
I Ablation Study and Sensitivity Analysis of Hyperparameters
For hyper-parameterr ax 3, 3 used in our experiments, we vary them in a large range to analyze the sensitivity of ImAD under MCAR. For imputation module indispensable part in the presence of missing values. The average results are shown in Figure g
B, it cannot be set to 0 because the
21

---

Arrhythmia
Arrhythmia
AUROC%%)
80
AUPRC(%)
70
70
Dussing_rate-dnn 0.2
missing_rateri rateeatteO. 0.2
60
missing_rates 0.5
missing_ Bessing_atteeOl 0.5
50
16
64
128
256
512
32
128
256
512
Latent Dimension
Latent Dimension
Speech
Speech
80
80
AUROC(66)
missing_rate raaee=0.2
AUPRC(%) AUPRC(%)
m
rate- 0.2
70-
missing, rate=0.5
missing rate=
60-
16
64
128
256
512
32
128
256
512
Latent Dimension
Latent Dimension
Segerstolpe
Segerstolpe
AUROC(%)
100
100
missing_ratee ratem 0.2
Maasiiggratee--s
missing_ratee rate=0.
missing_rate-m0 0.5
Latent Dimension
Latent Dimension
Usoskin
Usoskin
AUR000%)
80
AUPRC(%)
70
70
missing rate= 0.2
missing rate=0.2
missing_rates
missing_ratees 0.5
50
16
32
64
128
256
512
16
32
64
128
256
512
Latent Dimension
Latent Dimension
Figure 7: The detection performance Arrhythmia, Speech, Segerstolpe and Usoskin with different
latent dimension.
Missing Rate (0.2)
Missing Rate (0.2)
Missing Rate (0.2)
150
AUROC(%)
AUROC(5)
100
0.01
Missing Rate (0.5)
Missing Rate (0.55
Missing Rote (0.5)
AUROC(5)
AUROC1)
AUROC(5)
0
Figure 8: Sensitivity analysis hyperparameters a, 3, on Adult dataset.
22

---

J Detailed Experimental Implementations
J.1 Dataset Description
Adul Becker and Kohavi 1996] is from the 1994 Census Income database with 14 variables including both categorical and continuous variables. The samples of income S 50K are regarded as normal data, and the samples of income > 50K are regarded as abnormal data. Data
preparation follows the previous work (Han et al. 2023].
KDD Lichman 2013] is the KDDCUP99 10 percent dataset from the UCI repository and contains 121 variables including both categorical and continuous variables. The attack samples are regarded as normal data, and the non-attack samples are regarded as abnormal data.
Arrhythmia Rayana| 2016) is an ECG dataset. It was used to identify arrhythmic samples in five classes and contains 452 instances with 274 attributes.
Speech |Rayanal 2016 consists of 3686 segments of English speech spoken with different accents and is represented by 400-dimensional so-called i-vectors which are widely used state- of-the-art features for speaker and language recognition.
Segerstolp (Seersstolpe et al. 2016 is scRNA-seq dataset of human pancreas islets which includes six cell types: "alpha", "beta", " "delta", "ductal", "endothelial" and "gamma". In our experiments, "alpha" is regarded as normal data and "beta" is regarded as abnormal data. Usoskin [Usoskin et al. 2015) is dataset employed for the analysis of sensory neuron cells, specifically originating from the mouse lumbar dorsal root ganglion. The dataset encompasses four distinct cell types: non-peptidergic nociceptor cells (NP), peptidergic nociceptor cells (PEP), neurofilamen--contaiinng cells (NF), and tyrosine hydroxylase-containing cells (TH). In our experiment, TH is regarded as normal data and PEP is abnormal data.
Botner [Meidan al. 2018 is a public botnet datasets for the IoT. it was gathered from commercial IoT devices authentically infected by Mirai and BASHLITE. There are 7,062,606 instances in the original datasets. In our experiments, we use "Ecobee Thermostat" subset of the original data, in which "benign traffic" is regarded as normal data and "gafgyt attacks" is regarded as abnormal data. The "gafgyt_attacke "attatkks""" has five attack types and we randomly select 1,000 samples from each type as abnormal data of the test set.
Titanic [Chen et al. 2023] a classification dataset to detect the survival on the Titanic. In our experiments, we nine features including "gender, ticket, cabin, age, sibsp, parch, fare, embarked and pelass". The instances that did not survive are considered normal samples and those that survived are considered abnormal(or unusual) samples.
MovieLens IM [10 Han al. [2021] contains 1,000,209 anonymous rating approximately 3,900 movies made by 6,040 MovieLens users who joined MovieLens in 2000. Due the missing rate of the original dataset is near 95%, we remove some columns with quite high missing rate and obtain new dataset with 82% missing rate. Since the age of all samples is divided into seven groups, we chose the middle five groups (18 < age < 56) as normal samples and the remaining abnormal samples.
Bladder is a cell transcriptome data from the model organism Mus musculus, in which contains cell types (bladder cell, bladder urothelial cell, endothelial cell and leukocyte). We use the instance from "bladder cell" as normal samples and those from "leukocyte" as abnormal samples.
|T|
Seq2-Hearl 12 Schaum et al. 2018 is a single cell transcriptome data from the model organism Mus musculus, containing nearly 100,000 cells from 20 organs and tissues. There are 8 cell types in this data. We use the instances with "fibroblast" type as normal samples and those with "myofibroblast" type as abnormal samples.
2 hntps://arciive.i..s htwp:/cchiiiiiii.c..cdeeeeeeeaee www......dd///atsse///aauu.
hhtps://kdd. http://ww...........ddddddaddeed wtt..d/ddddaabbssssdddupyyy//
" "http:/oddds..
htt.r/owwo....///attttttiiaaaaee datasstt
'https://odds... ..s.ss..........s.s. .t......ss.s..t.........oo www.ssttooooooccioe wwwpoookddd/sppecc-heeeeeeee -dtaset//
"https://cblast. gao www.-aa.ooooo...o orgg///odooo
"htrps///iimaarssonaabbooo www.aa.rrggrr////
Im/
hhtpss//chllsttt
12 phttss//cblast.t wat..gaa..bo//doonllee..
23

---

J.2 Missing Mechanisms
In this work, we evaluate the detection performance of all the baselines under the three distinct missing mechanisms and follow the previous work [Muzellec et al. 2020) to set the missing value generation mechanism.
A detailed explanation of our implementation is provided as follows.
missing completely random if the missingness is independent of the data. In our implementation, each entry is masked according to the realization of a Bernoulli random variable with parameter p {0.2, 0.5).
MCAR:
missing at random if the missingness depends only on the observed values. In the MAR setting, for all experiments, fixed subset of variables that cannot have missing values is sampled. Then, the entries from the remaining variables are masked according to logistic model with random weights, which takes the non-missing variables as inputs. bias term is fitted using line search to attain the desired proportion of missing values.
MAR:
MNAR: missing not random if the missingness depends on both the observed values and the unobserved values. In the MNAR setting, first, we sample subset of variables whose values in the lower and upper p-th percentiles are masked according to a Bernoulli random variable, and the values in-between are left not missing.
J.3 Sampling in Target Distribution
In our experiments, we select two truncated Gaussian distribution (O, II) with different o as target distribution Dz, Da and set = 0.5, a 1.0 respectively. For target distribution Dz (0 0.5² Ia), according to the Proposition A.1 we set constrained radius 0.5 F. (p) where denotes the latent dimension and set = 0.9. Similarity, for target distribution D3 ~ NN Ia), we
set 11 = and r2 = Fd (p) and set 0.9.
J.4 All Baselines
For the data imputation method used in experiments, GAIN MissOT code and the hyperparameters are fine-tuned as suggested in the original paper. For MissForest, missingpy 15 which is library for missing data imputation in Python implement the
official
MissForest [Stekhoven and Bühlmann
2012 algorithm. For anomaly detection method, Deep
SVDD 16 Rubin 1976). Neutral. AD
Qiu et al. 2021 and DPAD (Fu et al. 2024 we official
code and the hyperparameters are fine-tuned as suggested in the original paper. For Isolation Forest, to implement the Isolation Forest [Liu al. 2008 algorithm.
we use scikit-learn
J.5
Hyper-parameter Settings
The hyperparameters used in our experiments are provided in Table 10
K
More Experimental Results
In this section, we conduct experiments on the Speech dataset with a missing rate mr e (0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8). The related results are visualized in Figure 9 where the detec- tion performance of with the increasing of missing rate from 0.1 to 0.8. Moreover, proposed method outperforms all baselines in almost all cases.
then-detenn"" methods does not degrade and some of them even improve
15 "hans://mppiigggrooccciiiisseprp
"http:://gthhub... "htpp://gthuu..co//ooccccc aomboschrreaacch//aaaaaaa.. 18

---

Table 10: Hyperparameters settings of the proposed method on all datasets.
Datasets
Missing rate
Latent dimension
Learning rate
B
mr=0.2
0.0002
5
20
Adult
mr=0.5
4
0.0002
1
10
mr=0.2
32
0,0001
1
1
Botnet
mr=0.5
32
0.0001
1
mr=0.2
32
0,0001
5
KDD
mr=0.5
32
0.0001
5
mr=0.2
128
0.000T
Arrhythmia
mr=0.5
128
0,0001
1
mr=0.2
128
0.0005
0.2
0.1
Speech
mr=0.5
128
0.0005
0.2
0.2
mr=0.2
128
0.0001
1
Segerstolpe
mr=0.5
128
0.0001
m==0.2
128
0.0001
0.2
0.2
0.2
Usoskin
mr=0.5
128
0.0001
0.2
0.2
0.2
Titanic
4
0.0001
0.1
0.1
0.01
MovieLens 1M
128
0.005
1
Bladder
128
0.0001
0.01
Seq2-Heart
128
0.0001
Koyd Bvod
AL200000
Poyd M05
Kerd
Kadl 20700
Maaf
DAN-A
40
DRB RE.0
DND
0.2
0.4
0.6
0.8
0.1
0.3
0.8
Misding Rata
Missing Rim
Figure 9: The performance fluctuation with the changes of missing rate from 0.1 to 0.8 on the Speech dataset. "MeanF" and "MissF" denotes Mean-Filling -Filiig and MissForest, respectively.
In some real scenarios, it possible that the missing rates of training and sets are not equal. In this section, we conduct related experiments on the Speech dataset. In these experiments, we keep the missing rate 0.5 on the training set and change the missing rate from 0.2 0.8 the test set. We visualize the experimental results in Figure T0
60
55
Rerf Fvnl
AUROCNA
AUPRICIN)
50
Parf 100
Payd 3850
May 05/00
DAB-0
6.2
3.3
0.4
3.5
0.6
3.7
0.8
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Missing Rate Tost
Missing Ratt Test
Figure 10: The performance fluctuation with the changes of missing rate from 0.2 to 0.8 on the test of Speech. "MeanF" and "MissF" denotes Mean Sean-Filling and MissForest, respectively.
25

---

With such an experimental setup, the performance of the methods based on Mean- DeilinFiliing and MissForest fluctuates significantly. Our proposed method outperforms all baselines in almost all cases in spite of also showing some degree of performance fluctuation.
The experimental results the Segerstolpe and Usoskin with MCAR are provided in Table 11 The experimental results on the Botnet dataset with MCAR are provided in Table [2] In addition to the missing mechanism MCAR, we also compare ImAD with cenpute-then-deeee
baselines under
missing mechanism MAR and MNAR. Note that we did not employ GAIN [Yoon et al. 2018 under MAR and MNAR because GAIN was proposed under the assumption of MCAR. Instead, we utilize MissOT Muzeelee et al.) 2020) as the imputation method under MAR and MNAR. For all baselines, the experimental results on the Adult dataset under MAR and MNAR are shown in Table [13 and Table 14. respectively, in which, our proposed method outperforms all two-stage baselines in all cases.
Table 11: Detection accuracy (AUROC and AUPRC (%, mean and std)) on Segerstolpe and Usoskin datasets with MCAR. The best result in each case is marked in bold.
Segerstolpe
Usaskin
DI Methods
AD Methods
AUROCTT))
AUPRC(ST]
AUROCTE)
AUPRCTSO
0.2
0.3
0.2
0.2
0.3
0.2
=
I-Forest
94.48(1.93)
91.23(3.22)
95.09/2.18)
91.7713.499
44.49(5.41)
46.57(5.22 26)
41.61(2.29)
Mcan-Filling
Deep SVDD
94.14/1.9 91)
91.5212.46)
94.46(1 5.5
8.)2
78.57(0.15)
78.41)0.34)
80. 80.8640.1 18)
80.61(0.23)
Neutral.
98.66(0.74)
94.93(1 800
98. 15(1 34)
94. 02:22.9 2.98
69.95(2 74)
66. ((4.9)
75.0722 34)
69. 69.6344.600
DPAD
98.78(0.66)
96.02(0.833
98.36(1 34
94.770193
81.95(1.577
80.23(1.14)
83.27(0.89)
82.35(0.84)
I-Forest
94.91 (1.35)
96.68(0.79 (..))
95. 9// 23)
97 97.56(0.59)
45.19 56)
49 6./ 43)
46.97(3.00 3.04)
49. (4)644))
MissForcst
Neutral. AD
Deep SVDD
96. 20,20022000 66)
89 24(1 44)
97.71(1.76) 89.38)2.80)
97.554 400
84.6 34.61137788
50 (1.5500557)
57,49(4 (455)
37,4203 83)
43.611 49)
(2.27)
63.65(2.40)
S0.55 02)
61.05(4.16 16)
55.05 81)
DPAD
97.09(0.74)
95.31(1.36)
94.93(1.05) (05)
93.75(1.79)
81,25(1.9) 1.33)
80.52(1 80)
83.58(1.32) 3.))
82. (2.69()))2)
I-Forest
94.25(0.90)
(2.07(1.82)
96. 14(0.75)
93.94(1.62
40.96(2.022
(2.11(2.12)
46.29(
42.86(1.22)
GAIN
Deep SVDD
06.322119955
92.25(2.40)
69) 56.18/2.00
(3.44)
64. (4..4(4)74))
Neutral. AD
97.52(0.57)
90.10r0.90)
97.52/1 (2)
82)
501 85)
64. 83.8552. 68)
23.334 1.31)
DPAD
98.53(1 23)
98,50( (.5)
97,42(2.52)
97.73(2.07)
83.92(1 29)
81 (1.60)
84. 30..1)
02.96(0.56)
ImAD (Ours)
99.14(0.88)
96.86(0.67)
98.98/
96.85(0.54)
84,95(1 29)
82.94(1.79)
83.61(1.39) 2..48)2.34)
Table 12: Detection performance in terms of AUROC and AUPRC (%, mean and std) on Botnet with MCAR. mr denotes the missing rate. The best result in each case is marked in bold. The results that exhibit an increase with the rising missing rate mr from 0.2 to 0.5 are emphasized by underlining.
AUROC
AUPRC
DI Methods
AD Methods
me
mr
I-Forest
91.71(1.63)
72.08(3.49)
93.72(1.53)
73.72(3.20)
Mcan-Filling
Deep SVDD
50.07(0.55)
57.87(0.95)
19004)))3
(3.60(0.68)
NeutraL AD
72.603386)
48.95(3.52)
63.13(4.28)
49.45(3.04)
DPAD
66.28(0.14)
57.92(0.71)
68.25(0.24)
63.97(0.35)
Forrst
95.72(0.96)
(3.8660.70)
97.25(0.69)
95 68(0.52)
Deep SVDD
96.72(0.87)
(0.94)
96 60.000080 80)
62)6.)))))
MissForest
NeutraL AD
99.04(0.26)
97.27(0.59)
08.92(0.24)
97 60..33)
DPAD
81.98(0.60)
76.74(0.60)
85.03(0.26)
78.25(0.30)
L-Frrstt
16/0/())
90.01(0.73)
(0.61
96. 18/0.44)
Deep SVDD
98.68(0. I))
08.02(0.41)
98.35(0. 14)
97. 07.59(0.46)
GAIN
Neutral AD
97.42(0.33)
99.56(0.27)
96 89(0.36)
99.41(0.35)
DPAD
(9.35(0.71)
97.63(0.01)
(9.33(0.85)
96.92(0.04)
InAD (Ours)
99.71(0.22)
99.53(0.25)
99.68(0.24)
99.58(0.20)
L Limitations and Future Work
Anomaly detection and data imputation are ubiquitous tasks across various data types. In this work, we primarily focus on incomplete tabular data. However, other data types, such image and time- series data, also need to be studied in similar scenarios. Therefore, we will conduct further studies on more data types in future work based ImAD
26

---

Table 13: Detection performance in terms AUROC and AUPRC (% mean and std) on Adult with MAR. The best result in each case is marked in bold.
AUROC
AUPRC
DI Methods
AD Methods
mor 0.2
mir 0.5
mir = 0.2
mir 0.5
I-Forest
60.54(0.92)
61.441107
56.98
38.14(1.00
Deep SVDD
(1.53)6.24)
56.22(7.75)
69)33)))
61.76(3.16)
MissForest
Neutral. AD
52.29(1.51)
96(1)1))
88)
54.47(0.56)
DPAD
63.54(0.27)
65.40(0.39)
68.43(0.39)
67.14(0.26)
I-Forest
45.63/2.93)
41.94(2227)
45 45.37)
44.97(0.76)
Deep SVDD
51.68(4.17)
(9.59(6.95)
53.21(3.95)
33.33))336
MissOT
Neutral AD
52.54(0.78)
47.24(1
50.79(0.76)
47.2%(1.59)
DPAD
55.83(1.04)
50.93(2622)
53 94(1.22)
50.56(1.39)
ImAD (Ours)
77.43(3442
74.61(2.18)
75.07(1.55
70.74(1.21)
Table 14: Detection performance in terms of AUROC and AUPRC (% mean and std) on Adult with MNAR. The best result each case is marked in bold.
AUROC 0.3
AUPRC
DI Methods
AD Methods
0.2
0.2
0.3
I-Forest
60.53(1.40)
60.24(1.05)
56.84(0.62)
57.54(1.07)
Deep SVDD
54.90(7.71)
57.54(4.33)
63.74(2.27)
62.48/(219 19)
MissForest
Neutral.
53.07(1.26)
50.82(2.35)
(5.41((00))
52.52(1.01)
DPAD
63.36(0.44)
63.23(0.22)
(9.19(0.30)
64.50(0.25)
I-Forest
44.78/2.68)
38.62(1.52)
45.72(0.95)
43.23(1.01)
Deep SVDD
45.77(8.47)
50.29(6.28)
54.04(2.57)
50.90(2.43)
MissOT
Neutral.
49.87(1.07)
49.38(1.0) 1.07)
50.20(0.42) 42)
(8.88)
DPAD
54.96(3.19)
54, 10(250))50)
13.49(1.85)
51.98(1.80
InAD (Ours)
73.73(3.57)
72.35(1.53)
71.60(0.74)
68.97(0.31)
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See the abstract part. Guidelines:
The answer NA means that the abstract and introduction do not include the claims
made in the paper.
The abstract and/or introduction should clearly state the claims made, including the
contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
See Appendix K.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: See Section 3.4 and Appendix A,B,C.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
28

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Section 4.1 and Appendix I. Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results. 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All the datasets used are publicly available and we provide the download URLs (see Appendix I) for each dataset. We provide experimental code in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurlPS code and data submission guidelines (https://nips. cc/ wttgiddde/ooeeuubaaeee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not
29

---

reproducible, they including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
httpsccccpbplccccc ceaicc/guiieee Sttp://///uieee/eeeuuuubbiii CodeSubmissss wwwiibbmssssiiiii ssinnsiiccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why. At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: See Section 4.1 and Appendix I.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: See Section 4.2 and 4.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the eтror bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma еrror bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: All details are included in the paper. Guidelines:
30

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper). 9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:p:://eerrpp...ccpppbi/i//ip
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid-
eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal
impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (c.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA]
Question:
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
31

---

Datasets that have been scraped from the Internet could pose safety risks. The authors
should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See the Section 4.1 and Appendix I.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release assets. Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
32

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
33


Reviews:
[{"review_id": "Asp9ElTOmU", "reviewer": "Reviewer_tw7F", "summary": "Paper proposes a unified framework to find anomalies in data with missing attribute values. Instead of relying on impute-then-detect approach, which could lead to imputation bias, authors proposed a multi-objective learning framework in which the imputation and data modeling are done together. The core assumption is that there is a latent dimension in which the normal data lies within a sphere and anomalous data lies in the region outside this sphere, and sampling from this anomalous region will provide anomalous samples in the original space. This is done by learning two-way mappings between the latent space and original space. An imputation map is also learnt for the anomalous data. Details of a neural network based implementation is given. \n\nExperimental results on several benchmark data sets are given to showcase two capabilities: the proposed method can indeed identify anomalies when the data has missing attribute values better than other methods which do not impute, and the proposed method has better performance than impute-then-detect scheme.", "questions": "- How sensitive is the performance to the choice of $r_1$ and $r_2$? I feel that this choice could be very impactful.\n- I am not clear on the need for Sinkhorn distance. When will the samples not be pairwise?", "limitations": "Limitations are discussed, though more discussion on the sensitivity to parameter choices might be included. The paper does not have any immediate societal concerns.", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- This is an interesting paper with an innovative approach. The idea of combining imputation and detection in a joint optimization framework is novel. \n\n- Authors have supported their assumptions with theoretical results.\n\n- The experimental evaluation is detailed and robust and largely supports the claims made in the paper.", "weaknesses": "- Experimental results are only marginally better than other solutions. I must admit that the method consistently outperforms the best existing approach so it might be better to use this approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very pleased and honored to receive your positive evaluation of our work. Our responses to your questions are as follows.\n\n**Response to Question 1:** \n\n&nbsp; &nbsp; &nbsp; &nbsp; As you are concerned, we have explored the influences of constrained radii $r_1, r_2$ for detection performance, and the related results are reported in Appendix G of our manuscript. According to Proposition A.1 in our manuscript, we have $r= \\sigma\\sqrt{F ^ {-1} _ d (p)}$, where $\\sigma$ and $d$ are the variance and dimension of the target distribution, respectively, and $p$ denotes the sampling probability. We set the target distribution as $\\mathcal{D} _ {\\mathbf{z}} \\sim \\mathcal{N}(\\mathbf{0}, 0.5 ^ 2 \\cdot \\mathbf{I} _ d), \\mathcal{D} _ {\\tilde{\\mathbf{z}}} \\sim \\mathcal{N}(\\mathbf{0}, \\mathbf{I} _ d)$, and $p=0.9$. We adjust the dimension $d \\in \\\\{ 4, 8, 16, 32, 64, 128, 256, 512\\\\}$ of the target space and obtain the corresponding $r_1, r_2$ (see following table).\n\n| Latent Dimension(d) &nbsp; | 4    | 8    | 16   | 32   | 64   | 128  | 256  | 512  |\n|--------------------------------------|-------|-------|------|------|------|------|------|------|\n| $ r_1=0.5\\sqrt{F^{-1}_{d}(0.9)} $               | 1.39 | 1.82 | 2.42 | 3.26 | 4.44 | 6.10 | 8.45 | 11.76|\n| $ r_2=\\sqrt{F^{-1}_{d}(0.9)} $             | 2.78 | 3.65 | 4.85 | 6.52 | 8.88 | 12.20| 16.90| 23.52|\n\nAccording to the results (Appendix G of our manuscript), our method is not very sensitive to the changes in the radii $r_1$ and $r_2$, but its performance degrades with the reduction in the latent dimension. This is reasonable since a smaller latent dimension results in more information loss.\n\n**Response to Question 2:**\n\n&nbsp; &nbsp; &nbsp; &nbsp; In the proposed method ImAD, we utilize a projector $\\mathcal{P}:\\mathbb{R} ^ m\\rightarrow\\mathbb{R} ^ d$ to \ntransform $\\mathcal{D} _ {{\\mathbf{x}}}$ (normal data distribution) and $\\mathcal{D} _ {\\tilde{\\mathbf{x}}}$ (pseudo-abnormal data distribution) into $\\mathcal{D} _ {{\\mathbf{z}}}$ and $\\mathcal{D} _ {\\tilde{\\mathbf{z}}}$ respectively. In this process, the samples from data distribution ($\\mathcal{D} _ {{\\mathbf{x}}}$ or $\\mathcal{D} _ {\\tilde{\\mathbf{x}}}$) and target distribution ($\\mathcal{D} _ {{\\mathbf{z}}}$ or $\\mathcal{D} _ {\\tilde{\\mathbf{z}}}$) do not have pairwise relationship and we need to measure the discrepancy between $\\mathcal{P}(\\mathcal{D} _ {{\\mathbf{x}}})$ and $\\mathcal{D} _ {{\\mathbf{z}}}$ using their finite samples. Thus, we use\nSinkhorn divergence to cover this situation. \n\n**Response to limitation:**\n\n&nbsp; &nbsp; &nbsp; &nbsp; The key hyperparameters of our proposed method include latent dimension $d$, constrained radii $r_1, r_2$, learning rate and the trade-off coefficients $\\alpha, \\beta, \\lambda$ in optimization objective. According to the results in Appendix G of our manuscript, our method is not very sensitive to changes in the radii $r_1$ and $r_2$, but its performance degrades with the reduction in the latent dimension $d$. The choice of latent dimension $d$ is based on the data dimension. Typically, high data dimensions correspond to high latent dimensions $d$. According to the ablation study on $\\alpha, \\beta, \\lambda$ in Appendix H and hyperparameters selected via grid search in Appendix I.5, the changes of the three coefficients have a non-trivial impact on detection performance, which indicates that each module corresponding to $\\alpha, \\beta$, or  $\\lambda$ is indispensable for the proposed method. Moreover, while setting $\\alpha, \\beta, \\lambda = 1$ (the same coefficient with Sinkhorn divergence), ImAD achieves good performance in most cases based on the empirical results. The learning rate is set to 0.0001 on most datasets and other choices mainly aim to make the optimization converge faster.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "joNBhDQDVN", "reviewer": "Reviewer_US97", "summary": "This paper introduces ImAD, an end-to-end approach to anomaly detection in the presence of missing data. It addresses the imputation bias observed in the traditional impute-then-detect approaches, where the imputation model trained only on normal data tends to normalize incomplete abnormal samples. The proposed method generates pseudo-abnormal samples in the latent space and uses them in joint learning of the imputor, reconstructor, and projector.", "questions": "- Are the missing rates used for training and test data the same? How would it affect the performance if different values are used? \n- Could you provide more information on how to select the optimal hyperparameters in the given experiments and also in general situations? \n- I am curious to see how the performance would change to different missing rates in comparison with other methods (for a wider range of values). \n- Is the imputor learned by this model only effective for the task of anomaly detection, or can it be applied to other tasks as well?  \n- What are the specific details of the architectures for MLPs used for each main component? How would different architectures affect the performance? \n- Is the method applicable to non-tabular data?", "limitations": "Limited experimental setting and ablation study.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- It proposes a novel approach that combines data imputation and anomaly detection in a single framework, which has not been explored much in previous studies. \n- The idea of generating pseudo-abnormal samples in the latent space and using them for joint learning of the imputor, projector, and reconstructor makes sense and shows strong empirical performance.", "weaknesses": "- Although the authors provide extensive details on the data and the experimental setup, some crucial information is still missing. For instance, it does not explain how the optimal hyperparameters for each method (both the proposed and others) are selected for the results shown in the tables and figures (whether by using a validation set, or referring to the performance on the test set, or using other methods), which can significantly affect the results and generalizability. Additionally, the composition of the training and test set split is not explained. \n- The results from using only two missing rate values (0.2 and 0.5) may not provide enough evidence about its robustness and effectiveness. In the previous study referred as a reference to this setting (Yoon et al., 2018), a broader range from 0.1 to 0.8 was explored. Therefore, certain claims regarding the imputation bias (e.g., the authors state that the detection performance of “impute-then-detect” methods does not decrease with the increasing of missing rate from 0.2 to 0.5 because of the imputation bias issue) are not fully supported empirically.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your reviews and suggestions of our work. Our responses to your questions are as follows.\n\n**Response to Weakness 1 and Question 2:**\n\nSince we are studying unsupervised anomaly detection, there is no validation set during the training stage. As shown in Table 8 of our paper, we have the latent dimension $d$, learning rate $\\eta$, and loss hyperparameters $\\alpha,\\beta,\\lambda$ to tune. Since we did not want to tune these hyperparameters, initially, we just let $d=4, 32, 128$ for low-dimensional, moderate-dimensional, and high-dimensional data casually, let $\\eta=0.0001$, $\\alpha=\\beta=\\lambda=1$. This setting works well in most cases. However, due to the high-diversity of the datasets (they have different sizes, are from different fields, and are with different missing rates and patterns), we have to tune the hyperparameters w.r.t. the test performance slightly. This is actually the convention of unsupervised anomaly detection. This strategy is commonly used in unsupervised learning such as clustering, novelty detection, and representation learning.  Note that we have used grid search w.r.t. the testing set to find the optimal hyperparameters for all baselines to ensure fair comparisons. Tuning hyperparameters for unsupervised learning remains an open problem [1], although automated machine learning has made considerable progress for supervised learning.\n\n    [1] Fan et al. A simple approach to automated spectral clustering. NeurIPS 2022.\n\nIn Appendix I.1, we describe in detail the data sources of all the used datasets as well as the settings for normal and abnormal samples, where the arrhythmia and Speech datasets are from ODDs (Outlier Detection DataSets), with inherent settings for normal and abnormal samples. For the split of the training and test sets, we first set the ratio of normal and abnormal samples in the test set to 1:1, and then use all the remaining normal samples as the training set.\n\n**Response to Weakness 2 and Question 3:**\n\n1. In fact, besides the datasets with synthetic missing values (missing rate=20\\% and 50\\%), our work included real datasets with inherent missing values of which the missing rates are about 10\\% and 80\\%. Please refer to Table 1 and Table 3 in our manuscript.\n\n2. The main experiments in (Yoon et al., 2018) are conducted with 20\\% missing rate only.  The paper only tested one dataset with missing rates ranging from 10\\% to 80\\% in the ablation study.\n\n3. In this rebuttal, we added related experiments on Speech dataset with missing rate $\\text{mr} \\in \\\\{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8\\\\}$ and the results are shown in Figure 1 of the attached PDF, where the detection performance of \"impute-then-detect\" methods does not degrade and some of them even improve with the increasing of missing rate from 0.1 to 0.8. Moreover, our proposed method (ImAD) outperforms all baselines in almost all cases.\n\n**Response to Question 1:**\n\nThis is a very practical question. Yes, they are the same in our experiments, because such a setting is consistent with existing data imputation works [Yoon et al., 2018; Muzellec et al., 2020] and can also facilitate the comparison among baselines.\n\nHowever, the issue you mentioned is meaningful and worth exploring in depth for practical imputation scenarios. We conduct related experiments on the Speech dataset. In these experiments, we keep the missing rate $\\text{mr}=0.5$ on the training set and change the missing rate from 0.2 to 0.8 on the test set. We visualize the experimental results in Figure 2 of the attached PDF. With such an experimental setup, the performance of the methods based on Mean-Filling fluctuates little, and the performance of the methods based on MissForest and GAIN fluctuates significantly. Our proposed method also shows some degree of performance fluctuation.\n\n**Response to Question 4:**\n\nThanks for your question. It is hard to answer this question according to the current results accurately. Since our work is anomaly detection with missing data, our imputer is specially designed for the detector and the overall anomaly detection method is an end-to-end method, very different from the ``impute-then-detect'' pipelines. Therefore, we have to say that the learned imputer in our method is only effective for the task of anomaly detection, currently.\n\n**Response to Question 5:**\n\nThe architectures of each module of our method are provided as follows:\n- **Imputer**: $input \\rightarrow 512 \\rightarrow 128 \\rightarrow 128 \\rightarrow 512 \\rightarrow output$\n- **Projector**: $input \\rightarrow 512 \\rightarrow 256 \\rightarrow 128 \\rightarrow output$\n- **Reconstructor**: $input \\rightarrow 128 \\rightarrow 512 \\rightarrow output$\n\nIn all three modules, we used LeakyReLU as the activation function and didn't use bias terms.\n\nBased on the empirical results, the architecture designed exhibits superior performance in comparison to all baselines. Therefore, we did not further explore the effects of network architecture and this is also not a key contribution to our work.\n\n**Response to Question 6:**\n\nThanks for your question. \n     Anomaly detection and data imputation are ubiquitous tasks across various data types. In this work, we primarily focus on incomplete tabular data since data missing is quite common on tabular data. Anomaly detection with missing data on tabular data has many practical applications such as identifying abnormal users in recommendation systems and discovering abnormal cells in bioinformatics. However, anomaly detection with missing data on other data types such as images may encounter some new questions and challenges, such as in what scenarios we need the method and how to define a meaningful missing pattern for images. Therefore, we are not sure whether our method can be directly applied to non-tabular data with missing values. But it is worth studying in the future.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WS1LaNjvLE", "reviewer": "Reviewer_4yRa", "summary": "This study addresses the challenge of anomaly detection in the presence of missing data, which is common in various fields like recommendation systems and bioinformatics. Traditional methods struggle with missing data, leading to biased imputations and ineffective anomaly detection. The study proposes an integrated approach that combines data imputation with anomaly detection in a unified optimization framework. By generating pseudo-abnormal samples during training, the method mitigates imputation biases and enhances anomaly detection performance. The approach is supported by theoretical guarantees and outperforms baseline methods in experimental evaluations on diverse datasets.", "questions": "N/A", "limitations": "N/A", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper is well-written\n- The proposed method is novel\n- Extensive experiments are conducted to robustly support the claims", "weaknesses": "- There is no report on computational time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your reviews and recognition of our work. Regarding your concern about computational time, we supplement comparisons of theoretical time complexity and experimental time cost between our proposed method and the baselines in the following Table 1 and Table 2.\n\n**The notations used in the complexity analysis (Table 1) are explained as follows:**\n- $n, m$ denote the number of samples of the training phase and inference phase, respectively.\n\n- Missforest is a well-known data imputation algorithm based on random forest ($\\mathcal{O}(t_1 \\cdot v \\cdot n\\log{n})$) where $t_1$ denotes the number of trees, $v$ denotes the number of attributes.\n\n- $T, T_g, T_d, T_{ae}, T_{oc}$ denote the iterations of corresponding methods.\n\n- $\\bar{L}$ and $\\bar{d}$ denote the number of layers of the neural network and the maximum width of the layers of the corresponding models, respectively. \n\n- $t_2$ denotes the number of trees of I-Forest and $t$ is the maximum iterations of the Sinkhorn algorithm.\n\n- $p, \\psi, K$ denote the key parameter of the corresponding methods.\n\n**Table 1: The time complexity of training and inference phase.**\n\n| DI Methods | AD Methods | Time Complexity (Training) | Time Complexity (Inference) |\n|------------|------------|------------------------------|------------------------------|\n|              | I-Forest   | $ \\mathcal{O}(T \\cdot p(t _ 1 \\cdot v \\cdot n\\log{n}) + t _ 2 \\cdot \\psi \\log{\\psi}) $ | $ \\mathcal{O}(p(t _ 1 \\cdot v \\cdot m\\log{n}) +  t _ 2 \\cdot m\\log{\\psi}) $ |\n| MissForest| Deep SVDD | $ \\mathcal{O}(T \\cdot p(t _ 1 \\cdot v \\cdot n\\log{n}) + (T _ {ae} + T _ {oc}) (n\\bar{d} ^ 2\\bar{L} + n )) $ | $ \\mathcal{O}(p(t _ 1 \\cdot v \\cdot m\\log{n}) + (m\\bar{d} ^ 2\\bar{L} + m)) $ |\n|$ \\mathcal{O}(T \\cdot p(t _ 1 \\cdot v \\cdot n\\log{n}))$| NeutraL AD | $ \\mathcal{O}(T \\cdot p(t _ 1 \\cdot v \\cdot n\\log{n}) + T (n\\bar{d} ^ 2\\bar{L} + n\\cdot K)) $ | $ \\mathcal{O}(p(t _ 1 \\cdot v \\cdot m\\log{n}) + (m\\bar{d} ^ 2\\bar{L} + m\\cdot K)) $ |\n|| DPAD       | $ \\mathcal{O}(T \\cdot p(t _ 1 \\cdot v \\cdot n\\log{n}) + T (n\\bar{d} ^ 2\\bar{L} + n ^ 2) ) $ | $ \\mathcal{O}(p(t _ 1 \\cdot v \\cdot m\\log{n}) + (m\\bar{d} ^ 2\\bar{L} + mn)) $ |\n|  | I-Forest   | $ \\mathcal{O}((T _ g + T _ d)n\\bar{d} ^ 2\\bar{L} + t _ 2 \\cdot \\psi \\log{\\psi}) $ | $ \\mathcal{O}(m\\bar{d} ^ 2\\bar{L} + t _ 2 \\cdot m\\log{\\psi}) $ |\n| GAIN      | Deep SVDD | $ \\mathcal{O}((T _ g + T _ d)n\\bar{d} ^ 2\\bar{L} + (T _ {ae} + T _ {oc}) (n\\bar{d} ^ 2\\bar{L} + n)) $ | $ \\mathcal{O}(m\\bar{d} ^ 2\\bar{L} + (m\\bar{d} ^ 2\\bar{L} + m)) $ |\n| $ \\mathcal{O}((T _ g + T _ d)n\\bar{d} ^ 2\\bar{L})$| NeutraL AD | $ \\mathcal{O}((T _ g + T _ d)n\\bar{d} ^ 2\\bar{L} + T (n\\bar{d} ^ 2\\bar{L} + n \\cdot K)) $ | $ \\mathcal{O}(m\\bar{d} ^ 2\\bar{L} + (m\\bar{d} ^ 2\\bar{L} + m\\cdot K)) $ |\n| | DPAD       | $ \\mathcal{O}((T _ g + T _ d)n\\bar{d} ^ 2\\bar{L} + T (n\\bar{d} ^ 2\\bar{L} + n ^ 2) ) $ | $ \\mathcal{O}(m\\bar{d} ^ 2\\bar{L} + (m\\bar{d} ^ 2\\bar{L} + mn)) $ |\n| ImAD (Ours) | -          | $ \\mathcal{O}(T(n\\bar{d} ^ 2 \\bar{L} + t \\cdot n ^ 2)) $ | $ \\mathcal{O}(m\\bar{d} ^ 2\\bar{L} + m) $ |\n\nWe have utilized the Speech and Usoskin datasets to benchmark the time cost of all methods, including ours and baselines. The two datasets exemplify the two distinct categories of tabular datasets used in our experiments. The Speech dataset contains 3,686 instances with 400 attributes and Usoskin dataset contains 610 instances with 25,334 attributes. ALL experiments were conducted on 20 Cores Intel(R) Xeon(R) Gold 6248 CPU with one NVIDIA Tesla V100 GPU, CUDA 12.0. The related results are provided in Table 2.\n\n**Table 2: The time cost (second) on Speech and Usoskin datasets.**\n\n\n| DI Methods | AD Methods | Time (Speech Training)  | Time (Speech Inference) | Time (Usoskin Training) | Time (Usoskin Inference) |\n|------------|------------|:-------------------:|:--------------------:|:-------------------:|:--------------------:|\n| MissForest | I-Forest   | 86.48             | 82.69              | 5648.62           | 5662.65           |\n| MissForest | Deep SVDD  | 109.12            | 82.59              | 5651.65           | 5660.07           |\n| MissForest | NeutraL AD  | 115.17            | 82.59              | 5658.38           | 5660.11           |\n| MissForest | DPAD       | 106.40            | 82.66              | 5652.64           | 5660.09           |\n| GAIN       | I-Forest   | 149.95            | 0.11               | 3664.48           | 9.42              |\n| GAIN       | Deep SVDD  | 172.59            | 0.01               | 3567.51           | 6.84              |\n| GAIN       | NeutraL AD  | 178.64            | 0.01               | 3574.24           | 6.88              |\n| GAIN       | DPAD       | 169.87            | 0.08               | 3568.50           | 6.86              |\n| ImAD (Ours)| -          | 471.43            | 0.02               | 95.04             | 0.04              |\n\nBased on the theoretical analysis of time complexity in Table 1 and empirical results in Table 2, the \"impute-then-detect'' pipelines have high time costs for the dataset like Usoskin with a large number of attributes. In contrast, our proposed method shows significant efficient advantages in the inference phase for both Speech-like and Usoskin-like datasets.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "f5b9F82laO", "reviewer": "Reviewer_aFbK", "summary": "In anomaly detection, where training data consists only of normal instances, conventional missing value imputation approaches may cause imputation bias, meaning that imputations are inclined to make anomalous incomplete instances appear normal. This study addressed this issue by proposing an end-to-end training method that incorporates missing value imputation and anomaly detection into a unified optimization problem. The experimental results demonstrated that the proposed method can mitigate imputation bias, thereby outperforming conventional \"impute-then-detect\" methods in various anomaly detection benchmarks.", "questions": "- How does the imputer network look? The network may not directly process missing values. How were the missing values handled before being fed into the network? This should be clarified.\n- In Table 8, the hyperparameter settings differ between datasets. It is important to explain how they were chosen. It is very inappropriate if the authors chose them based on test performance. Additionally, how is overfitting prevented?\n- I suppose the proposed method may work well when the distribution of each feature is significantly different between normal and anomalous data. It would be great if the authors investigated when the proposed method was more effective compared to \"impute-then-detect\" baselines.\n- In the real world, the mechanism of how missing values appear in data is usually unknown, and we cannot guess it from data alone; it can only be conjectured based on domain knowledge. If we don't know the actual missing mechanism of the data, how do we choose the missing mechanism M M when generating pseudo-abnormal samples? Is the effectiveness sensitive to the suitability of the missing mechanism we choose? For example, what will happen if the actual mechanism was NMAR and we set M to MCAR?\n- As motivated by cases of abnormal user detection in recommendation systems and novel or anomalous cell detection in bioinformatics, where the missing rates can be higher than 30% or even 80%, the effectiveness of the proposed method should be evaluated under such conditions. The benchmark datasets used in the experiments do not cover such conditions.", "limitations": "-", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Anomaly detection in the presence of missing values is a very important problem in practice, while few existing studies have addressed this issue.\n- This study addresses the issue properly using a novel approach.\n- The manuscript is overall well written, providing detailed justification for the approach.", "weaknesses": "- The implementation details are obscure in the manuscript.\n- Discussion on when the proposed method is more effective is needed.\n- Evaluation on realistic scenarios that motivated this study is needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your reviews and recognition of our work. Our responses to your questions are as follows.\n\n**Response to Weakness 1 and Question 1:**\n\nThe imputer is an MLP network, in which its backbone is designed as follows: $input \\rightarrow 512 \\rightarrow 128 \\rightarrow 128 \\rightarrow  512 \\rightarrow output$, and we use LeakyReLU as the activation function and don't use bias term. As depicted in Figure 4 of our manuscript, the missing values in incomplete data are filled with zeros before being fed into the network.\n\n**Response to Question 2:**\n\nAs shown in the table, we have latent dimension $d$, learning rate $\\eta$, and loss parameters $\\alpha, \\beta, \\lambda$ to tune. We have to say that hyperparameter tuning for unsupervised learning is a very challenging task and so far there is no good solution. \n\n- Since different dataset usually have different ambient dimension, we need to use different latent dimension. Therefore, in the experiments, for low-dimensional data, we let $d=4$; for moderate-dimensional data, we let $d=32$; for high-dimensional data, we let $d=128$. These values were selected casually. \n\n- For the learning rate $\\eta$, we just let it be $0.0001$, which works well in most cases, as shown in the table. However, due to the high diversity of the datasets, we need to use other values for some datasets to ensure the convergence speed.\n\n- For $\\alpha, \\beta, \\lambda$, we set all values to 1, which works well in most cases. However, due to the high diversity of the datasets, in some cases, we need to tune them slightly according to the test performance. This strategy is very common in unsupervised learning such as anomaly detection. Almost all papers on unsupervised anomaly detection used this strategy. \n\nIt should be emphasized that to ensure fairness, we have sufficiently tuned the hyperparameters of all compared methods in the experiments.\n\nRegarding overfitting, since our neural network is not very complex, the number of data points in each dataset is relatively large, and we fixed the number of optimization epochs to a small number, the overfitting seems not happened; otherwise, the AUROC score of our method will not be high.\n\n**Response  to Weakness 2 and Question 3:**\n\nAccording to our experimental settings and empirical results, we have the following findings:\n\n- Depending on the experimental settings and results of our manuscript, the detection performance of our proposed method outperforms all baselines in most cases, which indicates our method is more effective compared to \"impute-then-detect\" baselines under unsupervised settings.\n- Moreover, according to some new experimental results (See Figure 1 in the attached PDF), the effect of the imputation bias from normal data on the ``impute-then-detect'' methods gradually diminishes as the missing rate increases. Therefore, our proposed method will be more effective when the missing rate is relatively small, say less than 50\\%.\n- The generated pseudo-abnormal samples are critical to our method and our proposed method will be more effective when they overlap more significantly with real abnormal samples.\n\n**Response to Question 4:**\n\nThis is a very practical and valuable question. In real scenarios, we generally don't know the missing mechanism of incomplete data. In such a situation, our method chooses the simplest missing mechanism (MCAR) based on Occam's Razor. For your second question, we have designed two experiments to answer it.\n\n1. Observing the detection performance on a real incomplete dataset when using different missing mechanisms to generate pseudo-abnormal samples. In this situation, we don't know what the real missing mechanism is for the incomplete data.\n\n2. Observing the detection performance on synthetic incomplete datasets when using different missing mechanisms to generate incomplete data and pseudo-abnormal samples. In this situation, we know the missing mechanism of the incomplete data.\n\n**The experimental results are provided in the following table.**\n\n| Dataset | Missing Mechanism on Normal Data | | |Missing Mechanism on Pseudo-Abnormal Samples| | | |\n|---|---|:-:|:-:|:-:|:-:|:-:|:-:|\n| | | MCAR | MCAR | MAR| MAR| MNAR|MNAR|\n| | |AUROC(%)|AUPRC(%)|AUROC(%)|AUPRC(%)|AUROC(%)|AUPRC(%)|\n| Titanic | Unknown                           | 82.09 | 81.39 | 79.06 | 77.08 | 80.50 | 79.17 |\n| MovieLens1M | Unknown               | 66.32  | 65.34 | 63.14 | 63.39 | 61.44 | 60.91 |\n| Bladder  | Unknown                        | 100.00 | 100.00| 99.95| 99.95| 100.00| 100.00|\n| Seq2_Heart | Unknown                  | 96.62 | 96.40 | 96.79 | 96.60 | 95.56 | 94.41 |\n| Adult    | MCAR                              | 71.19 | 71.50 | 64.11 | 66.44 | 67.28 | 66.72 |\n| Adult    | MAR                                | 65.66 | 67.23 | 74.61 | 70.74 | 71.14 | 69.69 |\n| Adult    | MNAR                             | 70.69  | 69.17 | 68.35 | 68.78 | 71.60 | 68.97 |\n\n\nAccording to the experimental results, on real incomplete data, our method is robust to the setting of missing mechanisms of the mask for the generated pseudo-abnormal samples and has better overall performance on MCAR. Therefore, based on the Occam's Razor and empirical results, we recommend using MCAR as the missing mechanism of generated pseudo-abnormal samples when the real missing mechanism is unknown. Moreover, on synthetic incomplete data, detection performance degrades when using different missing mechanisms generating incomplete normal data and pseudo-abnormal data.\n\n**Response to Question 5:**\n\nIn fact, our work covered the real scenarios in Section 4 of our manuscript. In Table 1 of our manuscript, we introduced four real incomplete datasets, including Titanic (pattern recognition), MovieLens1M (recommendation system), Bladder, and Seq2-Heart (cell analysis), with missing rates of 10.79\\%, 82.41\\%, 86.93\\%, and 88.51\\%, respectively. The related experimental results are reported in Table 3 of our manuscript.\n\nThank you again for the comments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the comments made by all reviewers. We summarize the major work of this rebuttal as follows:

* As requested by reviewer 4yRa, we added time complexity analysis (in the form of $\mathcal{O}(\cdot)$) and running time cost of the compared methods in Table 1 and Table 2 of the attached PDF. On high-dimensional data (e.g. Usoskin with 25000+features), our method is at least 30 times faster than the competitors.

* As requested by reviewer US97, we added the experiments with the missing rate changing from 0.1 to 0.8 and the experiments with different missing rates on the training set and test set. Due to the space limitation, we provided these results in Figure 1 and Figure 2 of the attached PDF. Our method outperformed other methods in almost all cases.

* As suggested by Reviewer aFbK, we added experiments to study the effects of different missing mechanisms of the mask for the generated pseudo-abnormal samples, where previously we only used MCAR. The related results are provided in Table 3 of the attached PDF. Our method is quite robust to the setting of missing mechanisms of the mask for the generated pseudo-abnormal samples.

In addition to this global rebuttal and the attached PDF, we responded to the specific questions of each reviewer separately. Thank you again for the comments and suggestions by all reviewers. We are looking forward to further discussion with you.

Meta-review: This paper proposes a novel unsupervised anomaly detection framework for incomplete data, with a focus on mitigating imputation bias of the common "impute-then-detect" practice. It first generates incomplete pseudo-normal data and then jointly learns an imputation model and a detection model from both normal and pseudo-normal data. Theorectical supports of the designed framework are provided, and experimental results on 11 data sets show the framework outperforms several baseline methods (in terms of AUROC and AURPC) on most data sets. 

All reviewers find the design novel and the paper acceptable. Most reviewers requested more detailed information on the experiments, especially regarding the implementation of network and the choice of hyper-parameters. These requests are overall well addressed in rebuttal. In addition, the requested computational complexity analysis and a few more sensitivity analysis are also provided. 

Overall, this is a well-written paper that presents a novel unsupervised anomaly detection framework for incomplete data, whose effectiveness is justified by theoretical promises and comprehensive empirical evaluations.
Decision: Accept (poster)