Outlier Preserving Distribution Mapping Autoencoders 
Authors: Walter Gerych
Elke Rundensteiner
Emmanuel Agu
Venue:  (2021)

Abstract: State-of-the-art deep outlier detection methods map data into a latent space with the aim of having  outliers far away from inliers in this space. Unfortunately, this often fails as  the divergence penalty they adopt pushes outliers into the same high-probability regions as inliers. We propose a novel method, OP-DMA, that successfully addresses the above problem. OP-DMA   succeeds in mapping outliers to low probability regions in the latent space by leveraging a novel Prior-Weighted Loss (PWL) that utilizes the insight that outliers are likely to have a higher reconstruction error than inliers. Building on this insight,  OP-DMA weights the reconstruction error of individual points by a multivariate Gaussian probability density function evaluated at each point's latent representation. We demonstrate and provide theoretical proof that this succeeds to map outliers to low-probability regions. Our experimental study shows that OP-DMA  consistently outperforms  state-of-art  methods on a rich variety of outlier detection benchmark datasets.

Under review as a conference paper at ICLR 2021
OUTLIER PRESERVING DISTRIBUTION MAPPING AU- TOENCODERS
Anonymous authors Paper under double-blind review
ABSTRACT
State-of-the-art deep outlier detection methods map data into a latent space with the aim of having outliers far away from inliers in this space. Unfortunately, this is shown to often fail the divergence penalty they adopt pushes outliers into the same high-probability regions as inliers. We propose novel method, OP-DMA, that successfully addresses the above problem. OP-DMA succeeds in mapping outliers to low probability regions in the latent space by leveraging a novel Prior- Weighted Loss (PWL) that utilizes the insight that outliers are likely to have a higher reconstruction error than inliers. Building on this insight, explicitly encour- ages outliers to be mapped to low- cwwproobabbility regions of its latent by weighing the reconstruction eтror of individual points by a multivariate Gaussian probabil- ity density function evaluated at each point's latent representation. We formally prove that OP-DMA succeeds to map outliers to low-probability regions. Our ex- perimental study demonstrates that OP-DMA consistently outperforms state-of-art methods on rich variety of outlier detection benchmark datasets.
INTRODUCTION
Background. Outlier detection, the task of discovering abnormal instances in dataset, is critical for applications from fraud detection, error measurement identification to system fault detection (Singh & Upadhyayal 2012). Given outliers are by definition rare, it is often infeasible to get enough labeled outlier examples that are represetnative of all the forms the outliers could take. Consequently, unsupervised outlier detection methods that do not require prior labeling inliers or outliers are frequently adopted (Chandola et al. 2009).
State-of-Art Deep Learning Methods for Outlier Detection. Deep learning methods for outlier detection commonly utilize the reconstruction error of autoencoder model outlier score for outlier detection (Sakurada Yairi 2014] Vu et al.) 2019). However, directly using the reconstruc- tion error as the outlier score has major flaw. As the learning process converges, both outliers and inliers tend to converge to the average reconstruction error (to the same outlier score) - making them indistinguishable (Beggel et al., 2019). This is demonstrated in Figure Tal which shows that the ratio of average reconstruction error for outliers converges to that of the inliers.
To overcome this shortcoming, recent work (Beggel et al. 2019 Perera et al., 2019) utilizes the distribution-mapping capabilities of generative models that encourage data to follow a prior distri- bution in the latent space. These cutting-edge methods assume that while the mapping of inlier points will follow the target prior distribution, outliers will not due to their anomalous nature. In- stead, outliers will mapped to low-probability regions of the prior distribution, making easy to detect them outliers (Beggel et al.) 2019: Perera et al.) 2019).
However, this widely held assumption has been shown to not hold practice (Perera et al., 2019). Unfortunately, shown in Figure [bb both inliers and outliers are still mapped to the same high probability regions of the target prior distribution, making them difficult to distinguish.
Problem Definition. Given given dataset X E R of multivariate observations, let f R.M RN. < M, be a function from the multivariate feature space of X to a latent space f(x) E RN such that f(X) Pz. where Pz is known and tractable prior probability density function. The dataset X IRM is composed as Xo + X,, where Xo and X, are a set outlier and inlier points, respectively. During training, it is unknown whether any given point x X is outlier

---

Under review as a conference paper at ICLR 2021
Outliers over Inters
1.4
Inlier
Average Erro: of
Outlier
1.2
1.0
0.6
20
epoch
(b) Inliers and outliers in latent space of leading distribution-mappig methad
(a) Ablation study
Figure I: Data Set used in (a) and (b) is Inliers taken from MNIST "I"s while outliers are MNIST "O"s, such that the outliers account for roughly 20% of the total data. (a) Left plot shows average reconstruction error of outliers over average reconstruction error of inliers during the training of a standard autoencoder. of the total data. As the plot shows, the ratio of errors for outliers to inliers goes to I, meaning outliers are difficult to distinguish from inliers after training.
(b) The right plot
shows inliers and outliers in the 2-dimensional latent space of Wasserstein Autoencoder popular type of distribution mapping autoencoder). As seen, the outliers are in high-probabiln regions of the latent space and are thus difficult separate from the inliers.
or an inlier. Intuitively, our goal is to find a function f that maps instances of a dataset X into a latent space S with known distribution, such that outliers are mapped to low probability regions and inliers to high probability regions. More formally, we define unsupervised distribution-muppinn outlier detection as the problem of finding a function f with the aforementioned properties of such that we maximize the number of outliers x, E Xo and inliers E X2 for which (2 (Io)) < Pz (f (x:)) holds.
Challenges. To address the open problem defined above, the following challenges exist:
1. Overpowering divergence penalty. Intuitively, distribution mapping methods utilize divergence penalty to achieve a latent space mapping of input data that has a high probabil- ity of following a target prior distribution. While the data overall should follow this prior distribution, a solution must be found to instead maps outliers to low-probability regions of the prior. Having the data match the prior overall, while having outliers mapped low probability regions of the prior creates a conflict, as the two tasks are diametrically op- posed. To achieve such a mapping requires overpowering the divergence penalty in order to map outliers to low probability regions in the latent space.
2. Unknown outlier status. In unsupervised outlier detection, during training points do not have any labels indicating whether they are outliers inliers. This unsupervised scenario, while common in practice (Singh & Upadhyaya] 2012), makes it challenging to design strategies that explicitly coerce outliers to be mapped to low-probability regions.
Our OP-DMA Approach. In this work, we propose the Outlier Preserving Distribution Mapping Autoencoder (OP-DMA). Our core idea is to propose a novel Prior Weighted Loss (PWL) function that solves the two conflicting tasks of mapping the input data to a prior distribution while encourag- ing outliers to be mapped to low probability regions of that prior. This PWL directly addresses the shortcomings of the existing distribution mapping outlier detection methods (Vu et al. 2019 Perera et al. 2019). and to the best of our knowledge is the first unsupervised cost function that explicitly encourages outliers to be mapped to low probability regions.
We assume that outliers will have high reconstruction error during the initial stages of training. which causes the PWL to place them in low-probability (low PDF) regions in the latent space. This way, overcomes the challenge of overpowering the divergence penalty. It succeeds in mapping outliers to low-probabilii, regions (far from the mean of the latent distribution) even though each input point's outlier status is unknown. Our OP-DMA framework is pluggable, meaning off-the- shelf distance-based outlier methods can be flexibly plugged in post-rransformation.
Our key contributions follows:
2

---

Under review as a conference paper at ICLR 2021
1. Propose OP-DMA, a novel distribution mapping autoencoder that effectively separates out- liers from inliers in the latent space without knowing nor making assumptions on the orig- inal distribution of the data in the feature space.
2. Design the Prior-Weighted Loss (PWL., which when coupled with a divergence penalty encourages outliers to be mapped to low-prohability regions while inliers are mapped to high-probability regions of the latent space of an autoencoder.
3. Provide rigorous theoretical proof that the optimal solution for OP-DMA places outliers further than inliers from the mean of the distribution of the data in the latent space.
4. Demonstrate experimentally that OP-DMA consistently outperforms other state-of-ar art out- lier detection methods on a rich variety of real-world benchmark outlier datasets.
Significance: OP-DMA is a versatile outlier detection strategy as it can handle input data that has arbitrary distributions in the feature space, while not making any distance or density assumptions on the data. To the best of our knowledge, we are the first to propose loss function that explicitly encourages outliers to be mapped to low-probability regions while inliers are mapped to high proba- bility regions. Our PWL approach is pluggable, and can easily be incorporated into alternate outlier detectors. Our ideas could also spur further research into various prior weighted loss functions.
2 RELATED WORK
State-of-the the--eterttrrrt art deep outlier detection methods fall into one of three categories: 1) Autoencoders
coupled with classic outlier detectors (Erfani et al., 2016, Chalapathy et al.) 2018). 2) Reconstruc- tion error-based outlier detection methods (Zhou & Paffenroth| 2017 Chen et al., 2017 Sabokrou
iet al.
2018 Xia et al. 2015). or 3) Generative outlier detection methods (Perera et al. 2019 Vu
al.
2019 Liu et al. 2019).
1) Autoencoders coupled with classic outlier detectors project data into a lower dimensional latent that latent representation. These methods make the strict assumption that outliers in the original space will remain outliers in the latent space. Further, they fail to explicitly encourage this in the mapping function.
space before performing outlier detection
2) Reconstruction error-based outlier detection methods utilize the reconstruction error of an au- toencoder network to identify outliers. They typically use the reconstruction error directly as the anomaly score (An & Cho 2015). In more recent work, they try to separate outliers into a sepa- rate low-rank matrix analogous to RPCA (Zhou & Paffenroth] (017) or they introduce a separate discriminator network (Sabokrou et al. 2018). However, as shown in (Beggel et al.) (019), for autoencoders the reconstruction error of outliers often converges that of inliers. This negatively impacts the performance of such reconstruction error methods.
3) Generative outlier detection methods leverage deep generative models (Goodfellow et al. 2014 Kingma & Welling 2013) to generate the latent space such that the distribution of the latent space is encouraged to match a known prior so that thereafter an appropriate outlier method for the prior can be applied (Vu et al. 2019) to the latent space, discriminator can identify outliers in the latent space (Vu et al. 2019) or both the latent space and reconstructed space (Perera et al.) 2019)) However, as discussed in Section I, in practice both inliers and outliers are both mapped to the prior distribution as outliers that are mapped to low- cow-probability regions will generally incur a high cost from the divergence term which matches the latent distribution the prior.
OP-DMA shares characteristics with each of these three categories. However, unlike the other meth- ods in these categories. OP-DMA actively encourages outlier to be mapped low- owwprobability re- gions instead of just assuming that this will be the case. OP-DMA is is generative outlier method that uses the reconstruction error to encourage outliers to be mapped to low-probability regions. Further, it can flexibly be paired with nearly any classic outlier detector after distribution mapping.
PROPOSED APPROACH: OP-DMA
Overview of approach. OP-DMA consists of three main components:
1. A distribution mapping autoencoder (DMA) that OP-DMA utilizes to map a dataset X from the feature space IR into lower dimensional latent space R.N. such that the distribution of
3

---

Under review as a conference paper at ICLR 2021
the encoded data in the lower dimensional latent space has known probability distribution Pz. This step is crucial as it makes easy for OP-DMA to easily identify low probability regions of the latent space (outliers should be mapped here). This be done because after the distribution mapping, we can explicitly calculate the Probability Density Function (PDF) of the latent space long
selected prior distribution with known PDF
2. A novel Probability- Probability-Wiigheedd Loss (PWL) function for distribution mapping that encour- ages outliers to be mapped to low-probability regions the latent space, solving both the challenges of overpowering divergence penalty and unknown outlier status.
3. An traditional outlier detection method is used to identify outliers in the transformed la- tent space. The choice of outlier detection method is flexible as long as it amenable to the prior distribution Pz selected in step I of OP-DMA. For instance, when Gaussian distribution is used for the prior, then OP-DMA utilizes a classical distance- based outlier detection method for step 3. These steps are described in the following subsections and illustrated in Figure? 2
3.1 DISTRIBUTION MAPPING AUTOENCODER (DMA)
In order to use prior-weighting to map outliers to low-probabilip regions of known PDF in a latent space, our distribution mapping method must meet two design requirements:
1. A one-to-one mapping between each original data point, its latent representation and the reconstructed data point must be established so that each data point's reconstructed data point unique and can be determined, and vice versa.
2. The divergence term must impose cost based on how well batch of latent data points match the prior overall, rather than requiring individual data points to have high proba- bility of being a draw from the prior.
To meet these requirements, select the Wasserstein AutoEncoder (WAE) (olstiikhin et al. 2017) as the foundation for our distribution mapping. WAEs are distribution-mapping autoencoders that minimize the Wasserstein distance between the original data and its reconstruction, while mapping the input data to a latent space with a known prior distribution. To see why we base our distribution- mapping technique on this method, consider the WAE objective function for encoder network Q and decoder network G:
Reconstruction Error
Divergence Penalty
(X,Y) = inf Ep. Eq(zxx G(Z))) + P2)
(1)
The first term on the right hand side Equation I corresponds to the reconstruction error between the input data and reconstructed data for cost function The second term D is divergence penalty between the distribution of the latent space and the prior distribution, with a constant weight term that determines how much that divergence is penalized. Let deterministically produce the latent representation Q(X) and output ((Q(())X) (by using Q(X) = 6 6p(x), where μ is some function
OP-DMA Overview
Legend
Original Data
Isplated Outliers
Outlier Inlier
Map Oullirs To
Apply Distance Based
Low Probabilly Regions
Outlie Delecton
Achieve By
Phoo Weighted Lose
+
MMD
I
Divergence
Prior Distr loution
Function
Figure 2: An overview of OP-DMA. Data is mapped to latent space with prior distribution using a Prior-Woor Weighte Loss (PWL), which encourages outliers to be mapped to low-probability regions. This allows for distance-based outlier detection.
4

---

Under review as a conference paper at ICLR 2021
mapping imput data set X Q(X), for instance). It now clear why Wasserstein autoencoders an appropriate choice to model our distribution mapping method, as the reconstruction eπror term Erx_Borzxx/ee(X, G(Z)) in Equation [ represents one-to-one correspondence between input data, its latent representation and the reconstructed output (meeting requirement 1). Additionally, D is a batch-level cost term that would be incurred if the latent representation of batch doesn't t match the prior distribution but doesn't 't require individual points to be mapped to a high probability region of the prior (meeting requirement 2). However, we note that WAEs unfortunately do not encourage outliers in the feature space to remain outliers in the latent space. Consider 'D to be a discriminator network. Then D likely to learn a boundary around the high probability region of the prior distribution. Thus the encoder network Q will be penalized for mapping an outlier to low probability region outside of the boundary found by D as the discriminator D would correctly identify it as a generated point.
((())
3.2 PRIOR-WEIGHTTE WEIGHTED Loss (PWI.): NOVEL LOsS FUNCTION FOR OUTLIER COERCION
We now describe our novel mapping outliers to high probability regions. The key idea is that outliers will initially have higher reconstruction error than inliers during training. This core idea draws from the area of anomaly detection using reconstruction probability (An & Chol 2015). We thus propose the Prior Weighted Loss (PWL), a novel cost term that weights each data point's reconstruction error term Equation I by the point's latent likelihood, (x()(=)). The latent likelihood is the PDF of the latent space's s prior distribution evaluated at its corresponding latent representation.
Loss (PWL) that tackles the above challenge of WAEs
The prior weighted loss e' is defined d' == c(2, G(Q(z))) Pz(Q(())
As the latent likelihood is large in high probability regions and small in low probability regions by definition, points with high reconstruction error that are mapped to high-probability regions will be penalized more than those with high reconstruction error that are mapped to low probability regions. Since outliers assumed to result in high reconstruction error (at least during early training epochs), by reducing the penalty to the network for poorly reconstructed points that have been mapped to low-probability regions of the prior, the network is encouraged map outliers to these low- low-probability regions. We now introduce our OP-DMA objective W as:
Prior weighted loss
Divergence penally
W
inf E (xx.Eazz)e((X,,GZ)) XD(PQ. Pz) Q:Pa= Pz
(2)
Since have significantly modified the reconstruction error term in the Wasserstein autoencoder loss function, a natural question is whether or not OP-DMA still corresponds to an autoencoder. Specifically, will the decoder's output still match the input data to the encoder? If this does not hold, two issues could arise: 1) The latent features learned by the network might be unrelated to the input, and hence useless in cases where it is desirable to use the latent representation in a downstream task. 2) More importantly for our outlier detection task, if the network is no longer encouraged to reconstruct the input, the crucial property that outliers will have higher reconstruction error may no longer hold. In such a case, the "reconstruction error" may be meaningless. Fortunately, we can show that our OP-DMA loss function still corresponds to a Wasserstein divergence between the input and reconstructed distributions (Theorem []. For this, must demonstrate that is that the prior-weighted cost e' meets the requirements of a Wasserstein divergence's 's cost function, namely, that e' c'(x1, x22 2 0 (A 02,32 E supp( (P)), (d (x), 0) (V x E supp( (P)), and [d #2)) 2 O (V E [[P,PZ)) Pz))
Theorem 1. Let W. be a Wasserstein distance. Then We is a Wasserstein distance, with c' the prior-weighted
3.3 UNSUPERVISED STATISTICAL OUTLIER DETECTION METHOO
Intuitively, an ideal mapping would place all inliers within regions where the latent likelihood is greater than some value V, and all outliers into some alternate regions where the latent likelihood is less than that value V. The core result fundamental to our work is thus that this scenario is indeed the optimal solution for the loss function of OP-DMA as stated in Theorem 2
5

---

Under review as a conference paper at ICLR 2021
Dataset
# Features
Datapoints
% Outllers
Dataset
Features
Datapoints
% Outliers
Satellite
36
6435
32%
Lynpho
18
148
4.1%
Pima
8
768
35%
Musk
166
3062
3.2%
NRC
Thyroid
Arrythmia
274
452
15%
Satimage-2
36
5803
1.2%
Breastw
9
638
35%
Cover
10
286048
0.9%
Letter
32
1600
6.23%
Fever
36
5293
0.2%
Cardio
1831
9.6%
MN2ST
784
6365
1%
Table I: Description of real-world datasets' dimensionality, size, and outlier percentage. datasets taken from the standard ODDs database', while RC Flu was taken from the Reality- Commons Social Evolution database?. We also evaluate on the well-known MNIST dataset.
Theorem 2. Let Q be an encoder network such that P(q F2. where D(A, B, F) is the Maximum Mean Discrepancy between A and B, F is the set of bounded continuous finctiions and Pz = (0, E). Let us consider or dataset X as centered random variable, X S - I", X ~ Px. Let (A), A C Q, be outliers and let H Q be the inliers, where Sx(A) Px (x)dxxr a. Further, let c' (a, G(Q(a)) > e' (h, G()(h)) V ((), E X (H). Then, the optimal solution of where
OP-DMA map such that (XQ (A)) mahalanobis and
122 25T
(3)
This important result implies that after transformation with OP-DMA outliers can be separated from inliers using a simple distance metric. This lays solid foundation for simple yet effec- tive outlier detection scheme. Namely, we first transform the dataset to a latent representation with a multivariate Gaussian prior distribution, justified by Theorem 2 Then, Equation B states, outliers can isolated using simple distance-based approach. More specifically, any stan- dard outlier detection method that finds outliers in Gaussian distributions (e.g. EllipticEnvclope method (Rousseeuw & Driessen (999)) can be used to find outliers the latent space.
3.4 PULLING IT ALL TOGETHER: UNSUPERVISED OUTLIER DETECTION USING OP-DMA
OP-DMA, our end-to-end outlier detection approach, now summarized. First, the input data is transformed match a prior distribution with a distribution mapping autoencoder using our novel Prior-Weighted Loss (PWL) (Equation
2 We chose this prior to be multivariate Gaussian dis- tribution with 0 mean and identity covariance, as justified by Theorem [ Then, Elliptic En- velope (Rousseeww & Driessen 1999) is used to identify outliers. The outlier detection process is outlined in Appendix [A.3] We use the unbiased estimator of Maximum Mean Discrepency (MMD) from (Gretton et al. 2012) for the divergence term. For the kernel k of MMD, we use the inverse multiquadraties kernel as in (Tolstikhin al. 2017) and Mean Squared Error (MSE) for c.
EXPERIMENTAL EVALUATION
Compared Methods. We compare OP-DMA to state-of-the distribution mapping outlier detection methods. These include methods that perform outlier detection on the latent space of WAE (Tolstikhin et al.] (017). a VAE (Kingma & Welling, 2013), and an Adversarial Autoencoder (AAE) (Makhzani et al. 2015) all with Gaussian prior but they do not integrate our PWL idea. We test against MO-GAAL (Liu et al., 2019) and ALOCC (Sabokrou et al.] 2018)) two Mtate-of-the-art deep generative outlier detection models. We also test against LOF (Breunig al., 2000) and OC-SVM (Schölkopf et al.) 2001), two popular state-of-the of-the-art non-deep outlier detection methods.
Data Sets. We evaluated on rich variety of real-world data sets from the ODDs benchmark data store (Rayanal 2016). These datasets cover a wide range of dimensionality in the feature space from to 274, and also different outlier contamination percentages from 0.2% to 32%.
http:///odds.cs.soombboookkkkk

---

Under review as a conference paper at ICLR 2021
Methooss
OP-DMA
WAF
VAE
AAE
MO-GAAL
ALOCC
LOF
OC-SVM
Satellit
0.735
0.554
0,310
0.480
0.481
0.706
0.413
0.417
=0.012
=0.009
=0.007
+0.008
+0.001
=0.008
Pima
0.625
0.520
0.23
0.497
0.518
0.517
0.456
0.440
+-00188
+0.020
+0.019
+0.007
+0.002
=0.010
WBC
0.590
0.448
0.268
0.T9
0.468
0.529
0.480
0.199
#0.011
=0.013
=0.011
+0.018
+0,060
=0.007
Arzythmia
0.531
0.601
0.201
0.294
0,518
0.457
0.464
0.254
+0.017
-0.015
+-00100
+0.010
+0.011
=0.020
Breastw
0.951
0.950
0.368
0.479
0.944
0.863
0.292
0.824
#0.014
=0.011
=0.009
+0.007
#0.014
=0.062
Letter
0.182
0.091
0.048
0.10
0.159
0.165
0.488
0.208
+0.001
+0.003
+0.003
+0.002
+0.003
=0.001
Cardio
0.590
0.290
0.221
0.204
0.542
0.432
0.208
0.323
#0.013
=0.012
=0.008
+0.009
+0.032
=0.071
Lympho
0,585
0.443
0.341
0.310
0.719
0.958
0.833
0.150
+0.012
+0.008
+0.011
+0.018
+0.119
=0.083
Mosk
0.32
0.330
0.243
0.228
0.394
0.199
0.069
0.101
=0.007
=0.010
=0.009
+0.025
+0.090
=0.160
Thyroid
0.29
0.173
0.130
0.170
0.225
0.084
0.200
0.090
+0.019
+0002
+-00119
+0.023
+0.027
=0.026
Sat mattmage-2
0.860
0.176
0.148
0.535
0.814
0.818
0.122
0.019
+0.039
=0.013
=0.007
+0.015
+0.049
=0.026
Cover
0.200
0.142
0.069
0.070
0.032
0.050
0.0227
0.1194
+0.002
=0.001
+-00004
+0.001
+0.009
=0.003
Fever
0.854
0.786
0.274
0.485
0.560
0.720
0.459
0.568
=0.031
=0.027
#0.006
+0,014
+0.030
=0.094
MNIST
0.684
0.532
0.614
0.599
0.469
0.571
0.491
0.610
+0.020
#0.042
+0.036
+0.013
+0.141
=0.026
Table Weighted Fl scores with 95% confidence interval for OP-DMA state-of-the-art methods on benchmark outlier detection datasets. Best performing method in bold, second-best underlined. No confidence intervals on LOG and OC-SVM as they are deterministic.
Table [ breaks down the statistics each dataset. We evaluate all methods on their ability to detect subjects who have fever from smartphone sensible data using the MIT Social Evolution dataset (Madan et al., 2011) (RC Fever) D to demonstrate OP-DMA's effectiveness for mobile healthcare. Finally, also evalue the MNIST dataset [ We used all MNIST images of "7"s as inliers, and randomly sampled "O"s as outliers such that "O"'s account for ~ 1% of the data. Since outlier detection unsupervised without any supervised training phase, we perform outlier detection in an unsupervised manner on the entire dataset instead of having to introduce train/test splits. In each dataset, all points are labeled either inlier or outlier as ground truth. We emphasize that these ground truth labels are only used for evaluation but not for training all methods.
Metrics. Due to the large class imbalance inherent to outlier detection, use the Fl score as our performance metric MLazarevicMeMMnuus et al. (008) as commonly used to evaluate outlier detection methods (An & Cho 2015 Zhou & Paffenroth 2017 Zong et al. 2018).
Parameter Configurations of Methods. Encoders and decoders of all methods consist of 3-layer neural networks, where the decoder each pair mirrors the structure of its encoder. The number of nodes in the hidden layer of each network is hyperparameter from (5, 6, 9, 15, 18, 100). The number nodes the latent layer varies from {2, 3, 6, 9, 15). The regularization parameter is chosen such that the reconstruction error on the same order of magnitude the first epoch. We use the standard parameters of MO-GAAL from the authors' code We also use the standard configuration of ALOCC from the authors code 5 except we add an additional dense layer at the beginning of each subnetwork. We do this as ALOCC assumes input to be images of certain shape. The additional dense layer transforms the input data from its original dimensionality
the MMD error for
httt://lyannleeuu.ccm/eeedddde
7

---

Under review as a conference paper at ICLR 2021
0.7
F1 Score
Latent Dim
0.4
0.2
0.4 0.5
Contamination Estimate
Latent Dim
a) OP-DMA Sensitivity to Contamination Est. b) Data in OP-DMA Latent Space
Figure 3: a) Fl score of OP-DMA for various values of the contamination parameter the Satimage-2 dataset. b) Outliers and inliers in OP-DMA's latent space.
into the required shape. For these we use the standard parameters from Scikit-Learn.
Versatile Anomaly Detection. We validate the versatility of our OP-DMA method by showing that our method consistently outperforms state-of-the-an methods on a rich variety of benchmark datasets. As shown in Table 2 OP-DMA outperforms the majority (9/13) of the other methods on the benchmark datasets. We see that OP-DMA's superior performance is not limited to datasets with either high or low percentage of outliers. OP-DMA is the best performing method on the dataset with the largest ratio outliers (Satellite) as well as that with the smallest ratio (Cover).
Experiment 1:
Experiment 2: Sensitivity to Contamination Parameter. The contamination parameter a is used to fit the standard outlier method, Elliptic Envelope, plugged into OP-DMA framework on the encoded data after training. Thus, we test the sensitivity of EllipticEnvelope to the value of the contamination parameter by evaluating the Fl score of outlier detection
the Satellite dataset mapped by OP-DMA. The results (Figure 3 (a)) show that as long as this parameter is significantly underestimated, the Fi-score robust different values of the contamination parameter.
Experiment 3: Verifying that Outliers are Mapped To Low-Probability Regions. We trans- formed data from a multi-modal distribution in R consisting of a mixture of two Gaussians centered at (0,0,0,0) and (5,5,5,5) to standard normal Gaussian in RR.. Outliers in the original space were drawn from a uniform distribution and consisted of 2.4% of the total data. As Figure 3)) shows, outliers are successfully mapped far from the inlier data points. Furthermore, the average value of the prior evaluated at the outlier points is 0.02, while the average for inliers is 0.08, confirming that outliers are mapped to nower-probabiligg regions than inliers.
CONCLUSION
We have introduced OP-DMA, an autoencoder-based solution that unlike prior methods is truly outlier preserving in its distribution mapping method. That is, OP-DMA maps outliers in the feature space to low probability regions in the latent space in which multivariate standard normal Gaussian prior distribution enforced. Outliers are consequently easily identifiable in the latent space. Our experimental study comparing OP-DMA to state-of-the-arr methods a collection of benchmark outlier detection datasets shows that it consistently outperforms these methods on the majority the datasets. We have also demonstrated that there is not significant increase in running time between our method and state-of-th--arr methods.
REFERENCES
Jinwon An and Sungzoon Cho. Variational autoencoder based anomaly detection using reconstruc- tion probability. Special Lecture IE, 2(1):1-18, 2015.
Laura Beggel, Michael Pfeiffer, and Bernd Bischl. Robust anomaly detection in images using ad- versarial autoencoders. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 206-222. Springer, 2019.
8

---

Under review as a conference paper at ICLR 2021
Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 93-104, 2000.
Raghavendra Chalapathy, Aditya Krishna Menon, and Sanjay Chawla. Anomaly detection using one-class neural networks. arXiv preprint 302.06660. 2018.
Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: survey. ACM com- puting surveys (CSUR), 41(3):1-58, 2009.
Jinghui Chen, Saket Sathe, Charu Aggarwal, and Deepak Turaga. Outlier detection with autoencoder ensembles. In Proceedings of the 2017 SIAM international conference on data mining, pp. 90-98.
SIAM, 2017.
Sarah M Erfani, Sutharshan Rajasegarar, Shanika Karunasekera, and Christopher Leckie. High- dimensional and large-scale anomaly detection using a linear one-class svm with deep learning. Puttern Recognition, 58:121-134, 2016.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Arthur Gretton, Karsten M Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola.
A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
Diederik Kingma and Max Welling. arXiv: 2013.
Auto-encoding variational bayes. arXiv preprint
Neda MazzarevccMMMaane JR Renno, Dimitrios Makris, and Graeme A Jones. An object-based comparative methodology for motion detection based on the f-measure. Computer Vision and Image Understanding, 111(1):74-85, 2008
Yezheng Liu, Zhe Li, Chong Zhou, Yuanchun Jiang, Jianshan Sun, Meng Wang, and Xiangnan He. Generative adversarial active learning for unsupervised outlier detection. IEEE Transactions on Knowledge and Data Engineering, 2019.
Anmol Madan, Manuel Cebrian, Sai Moturu, Katayoun Farrahi, et al. Sensing the" health state" of a community. IEEE Pervasive Computing, 11(4):36-45, 2011.
Alireza Makhzami. Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. Adversarial autoencoders. arXiv preprint arXiv: 1.05644, 2015.
Pramuditha Perera, Ramesh Nallapati, and Bing Xiang. Ocgan: One-class novelty detection using gans with constrained latent representations. In Proceedings the IEEE Conference on Computer Vision and Paltern Recognition, pp. 2898-2906, 2019.
Shebuti Rayana. Odds library http://dddd....toonyoooo.eeeed Technometrics, 2016.
Peter Rousseeuw and Katrien Van Driessen. A fast algorithm for the minimum covariance deter- minant estimator. Technometrics, 41(3):212-22 1999.
Mohammad Sabokrou, Mohammad Khalooei, Mahmood Fathy, and Ehsan Adeli. Adversarially learned one-class classifier for novelty detection. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 3379-3388, 2018
Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with nonlinear dimen- sionality reduction. In Proceedings of the MLSDA 2014 2nd Workshop Machine Learning for Sensory Data Analysis, pp. 4-14, 2014.
Bernhard Schölkopf, John C Platt, John Shawe-Taylor Alex J Smola, and Robert Williamson. Estimating the support of high-dimensional distribution. Neural computation, 13(7); 1443-14114 2001.
Karanjit Singh and Shuchita Upadhyaya. Outlier detection: applications and techniques. Interna- tional Journal of Computer Science Issues (IJCSI), 9(1):377 2012.

---

Under review as a conference paper at ICLR 2021
Iyy Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schoelkopf. Wasserstein auto- encoders. arXiv preprint arXiv:1711.01558, 2017.
Ha Son Vu, Daisuke Ueta, Kiyoshi Hashimoto, Kazuki Maeno, Sugiri Pranata, and Sheng Mei Shen. Anomaly detection with adversarial dual autoencoders. arXiv preprint arXiv: 06924, 2019.
Yan Xia, Xudong Cao, Fang Wen, Gang Hua, and Jian Sun. Learning discriminative reconstruc- tions for unsupervised outlier removal. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1511-1519, 2015.
Chong Zhou and Randy Paffenroth. Anomaly detection with robust deep autoencoders. In Pro- ceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 665-676666 674, 2017.
Bo Zong. Qi Song. Martin Rengiang Min, Wei Cheng, Cristian Lumezanu, Dacki Cho, and Haifeng Chen. Innttrraatat Conference on Learning Representations, 2018.
10

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 PROOF OF THEOREM 1
Proof. Since c is a Wasserstein divergence, we know that c(21, 0 (V 21,32 E supp(P)),
(c(z,æ)= = 0) (A z E supp(P)), and E, [e x2)] 2 0 (V- T[P, Pz]). Since (2(2) 2 (V 2),c
will also fulfill the three aforementioned properties of c. Thus, We is Wasserstein divergence.
0
A.2 PROOF OF THEOREM 2
Proof. The Mahalanobis distance of QXX) can itself be expressed as random variable, 8 v /Q(X)=--Q(X)² Let &y be the CDF of 8. Then, &3 - a) P(o s 1 a) P(62 S
(1 === az)
Let - where MT E is the Choleski decomposition of the covariance Σ. Since D(Pq, Pz. F). 0, and D(A, F) iff A B, we thus know that Q(X) (0, E). Thus since (x) is normally distributed and centered, Y is normally distributed with identity covariance. YY-, & is the CDF of of the sum of squares of normally distributed variables with mean 0 and a = 1. Thus, 2g² is the Chi Squared distribution. The inverse Chi Squared CDF will thus give the distance ô such that 1 a percent of the points are
Since 8² Q(X)=='Q(X)*
within d
V So 11-a n/2 1e dt Now, let us assume that for some parameter choice O' for that 2
aP(Q(X S 6) 3, B O. Consequently, (1 a) P(Q > 6) = B, since P(Q(X) > 6) = a and Sx(a) Px )dz = a. Conversely, let us assume that there is a parameter configuration θ such that (P)(((X())))) S 6) 0 and so (1 a) P(Q(X > 6) 0. Since Pz ~ (0, Σ), (2(d)) < (d2) for [|d Ilmahatnnhiis > ||d2] mahalanabis- Thus, since we assume c(a, (()())) d(h, ((Q())) 4 ((A), h (HH), then
Epx Eq (Ip, G(Q(I, = Epx Eq(zix) G(Q(=,, (2)))
> Epx Eq(2 ((Q(z,
= Epx Eo(2) (Tp. ((Q(z,)0)))
Thus, the optimal solution for OP-DMA's cost function is one that maps outliers to regions with a larger Mahalanobis distance than that of inlicrs.
A.3 OP-DMA ALGORITHM
Algorithm 1: Unsupervised Outlier Detection with OP-DMA
Require: Regularization coefficient À
Contamination parameter a Initialized encoder network Q+ and decoder network Ge with random weights + and θ
Dataset X
while O & not converged do Sample (II,,I,,,., ..,,,,,,,,, ]n from X, {21, Æ1, 2n from N/ Z), and {E1, Ei,., (an....,,,) 2n} from
Qo (Z)
Update weights and θ by descending
clus, Ge(z:) X Pz(2)) +
1
Σ hgj
k(zn, 2z) + Σ k( 23)) 23)
2 Σ
k(zn, Z5)
n²
n²
h.j
end
Find D (o)(=)) (xxx) |Dmin (1 a)llDI with Minimum
Covariance Determinant estimator, infy Det(E))
Find estimated mean j from D.nn
return || Q (I:) Il makalanoois = (Qx(z;) - ()) (i) for E D as outlier scores


Reviews:
[{"review_id": "zT9rmPCiFWH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe authors proposed a WAE-based algorithm for outlier detection, aiming at mapping outliers to a low probability region and inliers to a high probability region. The training objective is based on WAE by replacing the reconstruction error with the prior weighted loss. Experiments were performed to show the effectiveness of the proposed algorithm.  \n\n################\n\nStrong points:\n\n. The authors proposed a training objective, which intends to map outliers to a low probability region.\n\n. The authors theoretically showed that under certain conditions the outliers and inliers can be separated by a distance metric.\n\n################\n\nWeak points:\n\n. In the abstract the authors claimed that ``We formally prove that OP-DMA succeeds to map outliers to low-probability regions''. This is not true. This conclusion is only verified in Experiment 3 using a synthetic dataset. A formal proof in Theorem 2 is regarding the separation between outliers and inliers under a distance metric. As the claim is the main contribution of the paper, more theoretical/experimental work is needed to demonstrate that.\n\n. The authors claimed that the reconstruction error can hardly separate outliers from inliers and showed an example in Figure 1. However, such conclusion may directly depend on the percentage of outliers in the training set. As an extreme case, if the training set only includes inliers, then the reconstruction error may work well in the testing phase.  \n\n\n. In the proposed network architecture, is the encoder deterministic or non-deterministic? It seems that the authors adopt the deterministic one but both Q(X) (deterministic) and Q(Z|X) (non-deterministic) are adopted, which is very confusing.\n\n. The authors claimed that the reconstruction error term in Equation 1 ensures the one-to-one correspondence between input, latent, and reconstructed input. This claim is questionable. Consider the loss function as the L2 norm. It is possible that different reconstructed inputs lead to the same reconstruction error. Also, it is unclear why one-to-one correspondence is critical to outlier detection.\n\n. The expression in Equation (2) is incorrect: the optimization problem should not be subject to P_Q = P_Z as the divergence penalty has already been included.\n\n\n. The contamination parameter \\alpha is required as an input in the algorithm, which significantly limits the practical use of the algorithm. In reality, the value of \\alpha can only be estimated, thus inaccurate estimation of \\alpha inevitably degrades the algorithm performance. This is also confirmed in Experiment 2: if \\alpha is underestimated, F1 score decreases significantly.\n\n. In experiment, the authors compared with benchmarks using F1 score. Another commonly used metric in outlier detection is AUROC. Do the authors have any comparison results regarding that?\n\n################\n\nThe paper needs to be proofread. There are many grammar mistakes and typos.\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "uGIvjQjydE4", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Autoencoders suffer from the issue of assigning unusual or outlying samples to regions in the latent space which contain a large amount of nominal samples: i.e. unusual samples are not separated from more normal looking samples. This issue is highly problematic when using an AE for anomaly detection. I this paper the authors present a method to help prevent this using the intuition that samples which are reconstructed poorly are likely to be outlying and incorporating this into the Wasserstein Autoencoder loss. They demonstrate the efficacy of their loss experimentally.\n\nOverall this paper paper suffers from two main problems:\n\n1. Lack of novelty. This slight adjustment to the WAE loss is rather incremental and doesn't incorporate any great new insight or particularly intriguing new concept to the problem, it is a rather small and not-so-interesting adjustment in my opinion.\n\n2. Weak experiments. The authors state\n>State-of-the-art deep outlier detection methods fall into one of three categories: 1) Autoencoders coupled with classic outlier detectors (Erfani et al., 2016; Chalapathy et al., 2018), 2) Reconstruction error-based outlier detection methods (Zhou & Paffenroth, 2017; Chen et al., 2017; Sabokrou et al., 2018; Xia et al., 2015), or 3) Generative outlier detection methods (Perera et al., 2019; Vu et al., 2019; Liu et al., 2019).\n \nWhich is a woefully dated outlook of deep outlier detection, considering predominant  SOTA methods include self-supervised learning (Deep Anomaly Detection Using Geometric Transformations Golan and Yaniv) and deep one class models (Ruff et al. 2018), and more rich probabliistic models (DEEP ANOMALY DETECTION WITH OUTLIER EXPOSURE Hendrycks et al 2019). Representatives from these classes of outlier detection models should really be included. Additionally the authors experiments are predominantly on \"non-deep\" (i.e. low dim non-image) datasets which are of limited interest for deep models. This is further exacerbated by the fact that standard blackbox \"classic\" outlier detection methods are missing (e.g. iForest) and details are missing about other classic methods, e.g. what is the kernel for the OC-SVM.\n\nDue to the above considerations I cannot recommend that this paper is accepted to a top tier ML conference in its current state.", "extra_scores": {}, "extra_text": {}}, {"review_id": "VB74HqoXrLT", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an autoencoder-based outlier detection system. The main idea of the paper is to ensure that outlier points are mapped to areas distant from the inliers in the embedding space. To this end, a novel cost function is introduced, which weighs the reconstruction error based on a prior distribution for the embedding space. This cost function hopes to force inliers to be mapped to the high probability region of the prior distribution and push outliers to low probability regions. Combined with a normal/multivariate prior distribution, this then enables the use of simple distance-based outlier detection methods.\n\nWhile the idea and goal of the paper are sensible specific claims and assumptions are not explained or motivated well. For example, the paper claims that at the beginning of the training, the reconstruction error of outliers will be higher than that of inliers. It is unclear why, for an untrained network, this should be true. This is a core assumption of the paper and yet no detail about this aspect it is given. The fact that a distribution loss over batches is employed might aid in achieving the goal, however, a more substantial discussion and theoretical underpinning of this core claim are needed.\n\nThe complete method is never described in detail. The closest is Section 3.4, where all the components are quickly enumerated. It would have been desirable to see a discussion of the choices made and what properties they provide and possibly what alternatives could be used.\n\nThe theoretical results provided are somewhat confusing. For one, the main paper contains only the theorems and none of the proofs or sketches thereof. This means that the paper is only complete with its appendix. Theorem 2 indicates that the optimal solution for the algorithm is to map outliers to low probability areas. However, this does not guarantee that the embedding being learned will actually do so. It would have been an excellent addition to talk about this and if there are any mechanisms that could be used to check if the learned embedding is indeed performing as desired.\n\nThe experiments demonstrate that the proposed method works very well and better than other methods. However, sadly the paper lacks any meaningful discussion of the results or insights to be gained from them. Good results are encouraging, however, they do not make up for a lack of discussion. For example, the few cases were the proposed method is not performing on par with other methods it would be good to know why that is. Another interesting aspect is what is so special about the Lympho dataset that a handful of methods perform exceptionally well while the others fare much worse. The description of experiment-2 is very unclear, and it is not clear at all what the experiment tries to showcase or what the outcome was.\n\nWhile the datasets chosen provide a good variability in scenarios, an interesting addition would have been synthetic data which explores various aspects, such as types of outliers, types of inlier distributions, etc. Such a study of the properties or behaviour of the proposed method in controlled environments would be a great addition to understand the method in greater detail.", "extra_scores": {}, "extra_text": {}}, {"review_id": "PQvzyW7W5dl", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a solution for outlier detection implicitly based on the idea of robust principal component analysis that decomposes data into inliers and outliers while utilizing a novel prior weighted loss and addressing two challenges: overpowering divergence penalty and unknown outlier status. \n\n##Summary of the main contributions:\n(1) A novel Autoencoders that distinguishes inliers and outliers by optimizing Wasserstein distance\n(2) A novel probability weighting approach to address two aforementioned challenges\n\n##Overall Feedback\nI found the paper easy to follow, which means it is well structured. However, the level of novelty is not high. In other words, the proposed solution combines several existing approaches to tackle the problem and address the challenges. The experimental settings (datasets, experiments and metric) is strong, but more recent baselines can be included, especially those based on auto encoders. So, the key concern is lack of through experimental evaluation to study the effectiveness of the proposed method.\n\n##Strong Points\n(1) The proposed method is technically correct\n\n\n##Weak Points\n(1) The level of novelty is not high\n(2) The proposed method does not work in end-to-end manner. More precisely, the main focus is on the input in an appropriate format for traditional outlier detection method. However, it is often well-observed that joint methods work better in reality.\n(3) Experimental evaluation requires improvement. For instance, more details and elaboration is necessary for Experiment 3 which seems to be important for supporting the claim that the paper makes.\n\n##Suggestions:\n(1) Please elaborate more on the proposed loss function, especially the Autoencoders loss\n(2) Please include more baselines on outlier detection into the experimental evaluation\n\n##Minor comments:\n(1) resolution of figure is not high", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, a data mapping method to a latent space designed for outlier detection is proposed. Outlier detection by latent space mapping has been extensively studied in the literature. Unfortunately, this paper does not fully discuss the relation of the proposed method with a large amount of existing literature and lacks novelty. 
Decision: Reject