Original: Perturbation Learning Based Anomaly Detection
Revised: Perturbation Learning Based Anomaly Detection

Original: Abstract
Revised: Abstract

Original: This paper presents a simple yet effective method for anomaly detection.
Revised: This paper presents a simple yet effective method for anomaly detection.

Original: The main idea is to learn small perturbations to perturb normal data and learn a classiﬁer to classify the normal data and the perturbed data into two different classes.
Revised: The main idea is to learn small perturbations to perturb normal data and learn a classiﬁer to classify the normal data and the perturbed data into two different classes.

Original: The perturbator and classiﬁer are jointly learned using deep neural networks.
Revised: The perturbator and classiﬁer are jointly learned using deep neural networks.

Original: Importantly, the perturbations should be as small as possible but the classiﬁer is still able to recognize the perturbed data from unperturbed data.
Revised: Importantly, the perturbations should be as small as possible but the classiﬁer is still able to recognize the perturbed data from unperturbed data.

Original: Therefore, the perturbed data are regarded as abnormal data and the classiﬁer provides a decision boundary between the normal data and abnormal data, although the training data do not include any abnormal data.
Revised: Therefore, the perturbed data are regarded as abnormal data and the classiﬁer provides a decision boundary between the normal data and abnormal data, although the training data do not include any abnormal data.

Original: Compared with the state-of-the-art of anomaly detection, our method does not require any assumption about the shape (e.g. hypersphere) of the decision boundary and has fewer hyper-parameters to determine.
Revised: Compared with the state-of-the-art of anomaly detection, our method does not require any assumption about the shape (e.g. hypersphere) of the decision boundary and has fewer hyper-parameters to determine.

Original: Empirical studies on benchmark datasets verify the effectiveness and superiority of our method.
Revised: Empirical studies on benchmark datasets verify the effectiveness and superiority of our method.

Original:  Introduction
Revised:  Introduction

Original: Anomaly detection (AD) is an important research problem in many areas such as computer vision, machine learning, and chemical engineering [Chandola et al. , 2009;
Revised: Anomaly detection (AD) is an important research problem in many areas such as computer vision, machine learning, and chemical engineering [Chandola et al. , 2009;

Original: Ramachandra et al. , 2020; Pang et al. , 2021; Ruff et al. , 2021].
Revised: Ramachandra et al. , 2020; Pang et al. , 2021; Ruff et al. , 2021].

Original: AD aims to identify abnormal data from normal data and is usually an unsupervised learning task because the anomaly samples are unknown in the training stage.
Revised: AD aims to identify abnormal data from normal data and is usually an unsupervised learning task because the anomaly samples are unknown in the training stage.

Original: In the past decades, numerous AD methods [Schölkopf et al. , 1999, 2001; Breunig et al. , 2000; Liu et al. , 2008] have been proposed.
Revised: In the past decades, numerous AD methods [Schölkopf et al. , 1999, 2001; Breunig et al. , 2000; Liu et al. , 2008] have been proposed.

Original: For instance, one-class support vector machine (OCSVM)
Revised: For instance, one-class support vector machine (OCSVM)

Original: [Schölkopf et al. , 2001] maps the data into high-dimensional feature space induced by kernels and tries to ﬁnd a hyperplane giving the possibly maximal distance between the normal data and the origin.
Revised: [Schölkopf et al. , 2001] maps the data into high-dimensional feature space induced by kernels and tries to ﬁnd a hyperplane giving the possibly maximal distance between the normal data and the origin.

Original: Tax and Duin [2004] proposed a method called support vector data description (SVDD), which ﬁnds the smallest hyper-sphere encasing the normal data in a high-dimensional feature space.
Revised: Tax and Duin [2004] proposed a method called support vector data description (SVDD), which ﬁnds the smallest hyper-sphere encasing the normal data in a high-dimensional feature space.

Original: SVDD is similar to OCSVM with a Gaussian kernel function.
Revised: SVDD is similar to OCSVM with a Gaussian kernel function.

Original: Classical anomaly detection methods such as OCSVM and SVDD are generally not suitable for large-scale data due to the high computational costs and are not effective to deal with more complex data such as those in vision scenarios.
Revised: Classical anomaly detection methods such as OCSVM and SVDD are generally not suitable for large-scale data due to the high computational costs and are not effective to deal with more complex data such as those in vision scenarios.

Original: To address these issues, a few researchers [Erfani et al. , 2016;
Revised: To address these issues, a few researchers [Erfani et al. , 2016;

Original: Golan and El-Yaniv, 2018; Abati et al. , 2019; Wang et al. , 2019; Qiu et al. , 2021] attempted to take advantage of deep learning [LeCun et al. , 2015; Goodfellow et al. , 2016] to improve the performance of anomaly detection.
Revised: Golan and El-Yaniv, 2018; Abati et al. , 2019; Wang et al. , 2019; Qiu et al. , 2021] attempted to take advantage of deep learning [LeCun et al. , 2015; Goodfellow et al. , 2016] to improve the performance of anomaly detection.

Original: One typical way is to use a deep auto-encoder or its variants[Vincent et al. , 2008; Kingma and Welling, 2013; Pidhorskyi et al. , 2018; Wang et al. , 2021] to learn effective data representation or compression models.
Revised: One typical way is to use a deep auto-encoder or its variants[Vincent et al. , 2008; Kingma and Welling, 2013; Pidhorskyi et al. , 2018; Wang et al. , 2021] to learn effective data representation or compression models.

Original: Auto-encoder and its variants have achieved promising performance in AD.
Revised: Auto-encoder and its variants have achieved promising performance in AD.

Original: In fact, these methods do not explicitly deﬁne an objective for anomaly detection.
Revised: In fact, these methods do not explicitly deﬁne an objective for anomaly detection.

Original: Instead, they usually use the data reconstruction error as a metric to detect anomalies.
Revised: Instead, they usually use the data reconstruction error as a metric to detect anomalies.

Original: There are a few approaches to building deep anomaly detection objectives or models.
Revised: There are a few approaches to building deep anomaly detection objectives or models.

Original: Typical examples include classical one-class learning based approach[Ruff et al. , 2018, 2019; Perera and
Revised: Typical examples include classical one-class learning based approach[Ruff et al. , 2018, 2019; Perera and

Original: Patel, 2019; Bhattacharya et al. , 2021], probability estimation based approach [Zong et al. , 2018;
Revised: Patel, 2019; Bhattacharya et al. , 2021], probability estimation based approach [Zong et al. , 2018;

Original: Pérez-Cabo et al. , 2019; Su et al. , 2019], and adversarial learning based approach [Deecke et al. , 2018;
Revised: Pérez-Cabo et al. , 2019; Su et al. , 2019], and adversarial learning based approach [Deecke et al. , 2018;

Original: Perera et al. , 2019; Raghuram et al. , 2021].
Revised: Perera et al. , 2019; Raghuram et al. , 2021].

Original: For example, Ruff et al.
Revised: For example, Ruff et al.

Original: [2018] proposed deep one-class classiﬁcation (DSVDD), which applies deep neural network to learn an effective embedding for the normal data such that in the embedding space, the normal data can be encased by a hyper-sphere with minimum radius.
Revised: [2018] proposed deep one-class classiﬁcation (DSVDD), which applies deep neural network to learn an effective embedding for the normal data such that in the embedding space, the normal data can be encased by a hyper-sphere with minimum radius.

Original: The deep autoencoding Gaussian mixture model (DAGMM) proposed by
Revised: The deep autoencoding Gaussian mixture model (DAGMM) proposed by

Original: Zong et al.
Revised: Zong et al.

Original: [2018] is composed of a compression network and an estimation network based on the
Revised: [2018] is composed of a compression network and an estimation network based on the

Original: Gaussian mixture model.
Revised: Gaussian mixture model.

Original: The anomaly score is deﬁned as the output energy of the estimation network.
Revised: The anomaly score is deﬁned as the output energy of the estimation network.

Original: Perera et al.
Revised: Perera et al.

Original: [2019] proposed one-class GAN (OCGAN) to learn a latent space to represent a speciﬁc class by adversarially training the auto-encoder and discriminator.
Revised: [2019] proposed one-class GAN (OCGAN) to learn a latent space to represent a speciﬁc class by adversarially training the auto-encoder and discriminator.

Original: The properly trained OCGAN network can well reconstruct the speciﬁc class of data, while failing to reconstruct other classes of data.
Revised: The properly trained OCGAN network can well reconstruct the speciﬁc class of data, while failing to reconstruct other classes of data.

Original: Moreover, some latest works also explore interesting perspectives.
Revised: Moreover, some latest works also explore interesting perspectives.

Original: Goyal et al.
Revised: Goyal et al.

Original: [2020] proposed the method called deep robust one-class classiﬁcation (DROCC).
Revised: [2020] proposed the method called deep robust one-class classiﬁcation (DROCC).

Original: DROCC assumes that the normal samples generally lie on low-dimensional manifolds, and regards the process of ﬁnding the optimal hyper-sphere in the embedding space as an adversarial optimization problem.
Revised: DROCC assumes that the normal samples generally lie on low-dimensional manifolds, and regards the process of ﬁnding the optimal hyper-sphere in the embedding space as an adversarial optimization problem.

Original: Yan et al.
Revised: Yan et al.

Original: [2021] claimed that anomalous domains generally exhibit different semantic patterns compared with the peripheral domains, and proposed the semantic context based anomaly detection network (SCADN) to learn the semantic context from the masked data via adversarial learning.
Revised: [2021] claimed that anomalous domains generally exhibit different semantic patterns compared with the peripheral domains, and proposed the semantic context based anomaly detection network (SCADN) to learn the semantic context from the masked data via adversarial learning.

Original: Chen et al.
Revised: Chen et al.

Original: [2022] proposed the interpolated Gaussian descriptor (IGD) to learn more valid data descriptions from representative normal samples rather than edge samples.
Revised: [2022] proposed the interpolated Gaussian descriptor (IGD) to learn more valid data descriptions from representative normal samples rather than edge samples.

Original: Shenkar and Wolf [2022] utilized contrastive learning to construct the method called generic one-class classiﬁcation (GOCC) for AD on tabular data.
Revised: Shenkar and Wolf [2022] utilized contrastive learning to construct the method called generic one-class classiﬁcation (GOCC) for AD on tabular data.

Original: It is worth noting that classical AD methods such as OCSVM [Schölkopf et al. , 2001] and DSVDD [Ruff et al. , 2018] require speciﬁc assumptions (e.g. hypersphere) for the distribution or structure of the normal data.
Revised: It is worth noting that classical AD methods such as OCSVM [Schölkopf et al. , 2001] and DSVDD [Ruff et al. , 2018] require speciﬁc assumptions (e.g. hypersphere) for the distribution or structure of the normal data.

Original: The GAN-based approaches [Deecke et al. , 2018; Perera et al. , 2019] suffer from the instability problem of min-max optimization and have high computational costs.
Revised: The GAN-based approaches [Deecke et al. , 2018; Perera et al. , 2019] suffer from the instability problem of min-max optimization and have high computational costs.

Original: In this paper, we propose a novel AD method called perturbation learning based anomaly detection (PLAD).
Revised: In this paper, we propose a novel AD method called perturbation learning based anomaly detection (PLAD).

Original: PLAD aims to learn a perturbator and a classiﬁer from the normal training data.
Revised: PLAD aims to learn a perturbator and a classiﬁer from the normal training data.

Original: The perturbator uses minimum effort to perturb the normal data to abnormal data while the classiﬁer is able to classify the normal data and perturbed data into two classes correctly.
Revised: The perturbator uses minimum effort to perturb the normal data to abnormal data while the classiﬁer is able to classify the normal data and perturbed data into two classes correctly.

Original: The main contributions of our work are summarized as follows:
Revised: The main contributions of our work are summarized as follows:

Original: • We propose a novel AD method called PLAD.
Revised: • We propose a novel AD method called PLAD.

Original: PLAD does not require any assumption about the shape of the decision boundary between the normal data and abnormal data.
Revised: PLAD does not require any assumption about the shape of the decision boundary between the normal data and abnormal data.

Original: In addition, PLAD has much fewer hyper-parameters than many state-of-the-art AD methods such as [Wang et al. , 2019; Goyal et al. , 2020; Yan et al. , 2021].
Revised: In addition, PLAD has much fewer hyper-parameters than many state-of-the-art AD methods such as [Wang et al. , 2019; Goyal et al. , 2020; Yan et al. , 2021].

Original: • We propose to learn perturbations directly from the normal training data.
Revised: • We propose to learn perturbations directly from the normal training data.

Original: For every training data point, we learn a distribution from which any sample can lead to a perturbation such that the normal data point is ﬂipped to an abnormal data point.
Revised: For every training data point, we learn a distribution from which any sample can lead to a perturbation such that the normal data point is ﬂipped to an abnormal data point.

Original: • Besides the conventional empirical studies on one-class classiﬁcation, we investigate the performance of our PLAD and its competitors in recognizing abnormal data from multi-class normal data.
Revised: • Besides the conventional empirical studies on one-class classiﬁcation, we investigate the performance of our PLAD and its competitors in recognizing abnormal data from multi-class normal data.

Original: These results show that our PLAD has state-of-the-art performance.
Revised: These results show that our PLAD has state-of-the-art performance.

Original: • Although our experiments are mainly on images and tabular data, PLAD is actually a framework of AD and can also be applied to time series, text, and graph data via changing the network components.
Revised: • Although our experiments are mainly on images and tabular data, PLAD is actually a framework of AD and can also be applied to time series, text, and graph data via changing the network components.

Original:  Proposed method
Revised:  Proposed method

Original: Suppose we have a distribution D of dimension d and any data drawn from D are deemed as normal data.
Revised: Suppose we have a distribution D of dimension d and any data drawn from D are deemed as normal data.

Original: Now we have some training data X
Revised: Now we have some training data X

Original: = { x 1 , x 2 , . . .
Revised: = { x 1 , x 2 , . . .

Original: , x n } randomly drawn from D and we want to learn a discriminative function f from X such that f ( x ) > 0 for any x ∈ D and f ( x ) < 0 for any x / ∈ D .
Revised: , x n } randomly drawn from D and we want to learn a discriminative function f from X such that f ( x ) > 0 for any x ∈ D and f ( x ) < 0 for any x / ∈ D .

Original: This is an unsupervised learning problem and also known as anomaly detection, where any x / ∈ D are deemed as abnormal data.
Revised: This is an unsupervised learning problem and also known as anomaly detection, where any x / ∈ D are deemed as abnormal data.

Original: In contrast to classical anomaly detection methods such as one-class SVM [Schölkopf et al. , 2001], deep SVDD [Ruff et al. , 2018] , and DROCC [Goyal et al. , 2020], in this paper, we do not make any assumption about the distribution D .
Revised: In contrast to classical anomaly detection methods such as one-class SVM [Schölkopf et al. , 2001], deep SVDD [Ruff et al. , 2018] , and DROCC [Goyal et al. , 2020], in this paper, we do not make any assumption about the distribution D .

Original: Inspired from the generative adversarial learning (GAN) [Goodfellow et al. , 2014], we propose to learn a discriminator that is able to recognize fake data (abnormal data) and a generator that is able to fool the discriminator, but the input of the generator is real data (normal data), which is essentially different from GAN.
Revised: Inspired from the generative adversarial learning (GAN) [Goodfellow et al. , 2014], we propose to learn a discriminator that is able to recognize fake data (abnormal data) and a generator that is able to fool the discriminator, but the input of the generator is real data (normal data), which is essentially different from GAN.

Original: To be more precise, we propose to learn perturbations for x such that the perturbed X (denoted by (cid:101)
Revised: To be more precise, we propose to learn perturbations for x such that the perturbed X (denoted by (cid:101)

Original: X
Revised: X

Original: = { ˜ x 1 , ˜ x 2 , . . .
Revised: = { ˜ x 1 , ˜ x 2 , . . .

Original: , ˜ x n } ) are abnormal but quite close to X .
Revised: , ˜ x n } ) are abnormal but quite close to X .

Original: To ensure the abnormality of (cid:101) X , we learn a classiﬁer f from { X , (cid:101) X } such that f ( x ) > 0 for any x ∈ X and f (˜ x ) < 0 for any ˜ x ∈ (cid:101) X .
Revised: To ensure the abnormality of (cid:101) X , we learn a classiﬁer f from { X , (cid:101) X } such that f ( x ) > 0 for any x ∈ X and f (˜ x ) < 0 for any ˜ x ∈ (cid:101) X .

Original: To ensure that (cid:101) X is close to X , the perturbations should be small enough.
Revised: To ensure that (cid:101) X is close to X , the perturbations should be small enough.

Original: Therefore, we propose to solve the following problem
Revised: Therefore, we propose to solve the following problem

Original: [Equation]
Revised: [Equation]

Original: where (cid:96) ( · , · ) denotes some loss function such as cross-entropy and y 1 = · · · = y n = 0 and ˜ y 1 = · · · = ˜
Revised: where (cid:96) ( · , · ) denotes some loss function such as cross-entropy and y 1 = · · · = y n = 0 and ˜ y 1 = · · · = ˜

Original: y n = 1 are the labels for the normal data and perturbed data respectively.
Revised: y n = 1 are the labels for the normal data and perturbed data respectively.

Original: θ denotes the set of parameters of the classiﬁer f and φ ( · , · ) is some distance metric quantifying the difference between two data points.
Revised: θ denotes the set of parameters of the classiﬁer f and φ ( · , · ) is some distance metric quantifying the difference between two data points.

Original: λ is a hyperparameter to control the magnitudes of the perturbations.
Revised: λ is a hyperparameter to control the magnitudes of the perturbations.

Original: However, directly solving (1) encounters the following difﬁculties.
Revised: However, directly solving (1) encounters the following difﬁculties.

Original: • First, the number ( | θ | + dn ) of decision variables to optimize can be huge if n is large, where | θ | denotes the cardinality of the set θ .
Revised: • First, the number ( | θ | + dn ) of decision variables to optimize can be huge if n is large, where | θ | denotes the cardinality of the set θ .

Original: Second, it is hard to use mini-batch optimization because some decision variables (i
Revised: Second, it is hard to use mini-batch optimization because some decision variables (i

Original: e. (cid:101) X ) are associated with the sample indices.
Revised: e. (cid:101) X ) are associated with the sample indices.

Original: Lastly, it is not easy to determine φ because φ relies on the unknown distribution D .
Revised: Lastly, it is not easy to determine φ because φ relies on the unknown distribution D .

Original: For instance, φ ( x , ˜ x ) = (cid:107) x − ˜ x
Revised: For instance, φ ( x , ˜ x ) = (cid:107) x − ˜ x

Original: (cid:107) 2 , namely the squared Euclidean norm, does not work if X is enclosed by a hypersphere or hypercube (data points close to the centroid require much larger perturbations than those far away from the centroid, which implies that x − ˜ x has a non-Gaussian distribution).
Revised: (cid:107) 2 , namely the squared Euclidean norm, does not work if X is enclosed by a hypersphere or hypercube (data points close to the centroid require much larger perturbations than those far away from the centroid, which implies that x − ˜ x has a non-Gaussian distribution).

Original: To overcome these three difﬁculties, we propose to solve the following problem instead minimize
Revised: To overcome these three difﬁculties, we propose to solve the following problem instead minimize

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: subject to ˜ x
Revised: subject to ˜ x

Original: [Equation]
Revised: [Equation]

Original: where 1 = [1 , 1 , . . . , 1] (cid:62) and 0 =
Revised: where 1 = [1 , 1 , . . . , 1] (cid:62) and 0 =

Original: , 0] (cid:62) are d -dimensional constant vectors and (cid:12) denotes the Hadamard product of two vectors.
Revised: , 0] (cid:62) are d -dimensional constant vectors and (cid:12) denotes the Hadamard product of two vectors.

Original: α i and β i are multiplicative and additive perturbations for x
Revised: α i and β i are multiplicative and additive perturbations for x

Original: i
Revised: i

Original: and they are generated from a perturbator g ˜ θ , where ˜ θ denotes the set of parameters to learn.
Revised: and they are generated from a perturbator g ˜ θ , where ˜ θ denotes the set of parameters to learn.

Original: In (2), we hope that the multiplicative perturbation is close to 1 and the additive perturbation is close tobut they relies on the data point x .
Revised: In (2), we hope that the multiplicative perturbation is close to 1 and the additive perturbation is close tobut they rely on the data point x .

Original: Therefore, the perturbation learning is adaptive to the unknown distribution D , which solves the third aforementioned difﬁculty.
Revised: Therefore, the perturbation learning is adaptive to the unknown distribution D , which solves the third aforementioned difﬁculty.

Original: In fact, problem (2) can be reformulated as
Revised: In fact, problem (2) can be reformulated as

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: = g ˜
Revised: = g ˜

Original: [Equation]
Revised: [Equation]

Original: x i ) , i = 1 , 2 , . . . , n .
Revised: x i ) , i = 1 , 2 , . . . , n .

Original: We see that we only need to optimize the parameters θ and ˜ θ and the total number of decision variables is | θ | + | ˜ θ | , which solved the ﬁrst difﬁculty we discussed previously.
Revised: We see that we only need to optimize the parameters θ and ˜ θ and the total number of decision variables is | θ | + | ˜ θ | , which solved the ﬁrst difﬁculty we discussed previously.

Original: On the other hand, θ and ˜ θ are not associated with the sample indices, which solved the second difﬁculty.
Revised: On the other hand, θ and ˜ θ are not associated with the sample indices, which solved the second difﬁculty.

Original: Once f θ and g ˜ θ are learned, we can then use f θ to detect whether a new data point x new is normal (e.g. f θ ( x new ) < 0 . 5 ) or abnormal (e.g. f θ ( x new ) > 0 . 5 ).
Revised: Once f θ and g ˜ θ are learned, we can then use f θ to detect whether a new data point x new is normal (e.g. f θ ( x new ) < 0 . 5 ) or abnormal (e.g. f θ ( x new ) > 0 . 5 ).

Original: We call the method Perturbation Learning based Anomaly Detection (PLAD).
Revised: We call the method Perturbation Learning based Anomaly Detection (PLAD).

Original: In PLAD, namely (3), both the classiﬁer f θ and the perturbator g ˜ θ are neural networks.
Revised: In PLAD, namely (3), both the classiﬁer f θ and the perturbator g ˜ θ are neural networks.

Original: They can be fully connected neural networks, convolutional neural networks (for image data anomaly detection),
Revised: They can be fully connected neural networks, convolutional neural networks (for image data anomaly detection),

Original: [Figure]
Revised: [Figure]

Original: Perturbator I npu t
Revised: Perturbator I npu t

Original: 𝛼 𝛽
Revised: 𝛼 𝛽

Original: Classifier
Revised: Classifier

Original: Additive Perturbation
Revised: Additive Perturbation

Original: MultiplicativePerturbation
Revised: MultiplicativePerturbation

Original: PerturbedSample 𝐱 ෤𝐱 0/ or recurrent neural networks (for sequential data anomaly detection).
Revised: PerturbedSample 𝐱 ෤𝐱 0/ or recurrent neural networks (for sequential data anomaly detection).

Original: Figure 1 shows the motivation of our PLAD and the network architecture.
Revised: Figure 1 shows the motivation of our PLAD and the network architecture.

Original: Compared with many popular anomaly detection methods, our PLAD has the following characteristics.
Revised: Compared with many popular anomaly detection methods, our PLAD has the following characteristics.

Original: • PLAD does not make any assumption about the distribution or structure of the normal data.
Revised: • PLAD does not make any assumption about the distribution or structure of the normal data.

Original: In contrast, one-class SVM [Schölkopf et al. , 2001], deep SVDD [Ruff et al. , 2018], and DROCC [Goyal et al. , 2020] make speciﬁc assumptions about the distribution of the normal data, which may be violated in real applications.
Revised: In contrast, one-class SVM [Schölkopf et al. , 2001], deep SVDD [Ruff et al. , 2018], and DROCC [Goyal et al. , 2020] make speciﬁc assumptions about the distribution of the normal data, which may be violated in real applications.

Original: For example, deep SVDD assumes that the normal data are encased by a hypersphere, which is hard to guarantee in real applications and may require a very deep neural network to transform a non-hypersphere structure to a hypersphere structure.
Revised: For example, deep SVDD assumes that the normal data are encased by a hypersphere, which is hard to guarantee in real applications and may require a very deep neural network to transform a non-hypersphere structure to a hypersphere structure.

Original: • In PLAD, besides the network structures, we only need to determine one hyperparameter λ , which provides huge convenience in real applications.
Revised: • In PLAD, besides the network structures, we only need to determine one hyperparameter λ , which provides huge convenience in real applications.

Original: In contrast, many state-of-the-art AD methods such as [Wang et al. , 2019; Goyal et al. , 2020; Yan et al. , 2021] have at least two key hyperparameters.
Revised: In contrast, many state-of-the-art AD methods such as [Wang et al. , 2019; Goyal et al. , 2020; Yan et al. , 2021] have at least two key hyperparameters.

Original: For example, in DROCC [Goyal et al. , 2020], one has to determine the radius of the hyper-sphere, the step size of gradient-ascent
Revised: For example, in DROCC [Goyal et al. , 2020], one has to determine the radius of the hyper-sphere, the step size of gradient-ascent

Original: and two regularization parameters.
Revised: and two regularization parameters.

Original: In PLAD, we can use gradient-based optimizer such as Adam to solve the optimization.
Revised: In PLAD, we can use gradient-based optimizer such as Adam to solve the optimization.

Original: The time complexity is comparable to that of vanilla deep neural networks (for classiﬁcation or representation).
Revised: The time complexity is comparable to that of vanilla deep neural networks (for classiﬁcation or representation).

Original: On the contrary, many recent advances DROCC [Goyal et al. , 2020] of anomaly detection especially those GAN based methods [Deecke et al. , 2018; Perera et al. , 2019; Yan et al. , 2021] have much higher computational costs.
Revised: On the contrary, many recent advances DROCC [Goyal et al. , 2020] of anomaly detection especially those GAN based methods [Deecke et al. , 2018; Perera et al. , 2019; Yan et al. , 2021] have much higher computational costs.

Original: As shown in the left plot of Figure 1, normal training data can be perturbed to abnormal data by different perturbations.
Revised: As shown in the left plot of Figure 1, normal training data can be perturbed to abnormal data by different perturbations.

Original: Therefore, we propose to learn a distribution from which any perturbations can perturb the normal training data to be abnormal.
Revised: Therefore, we propose to learn a distribution from which any perturbations can perturb the normal training data to be abnormal.

Original: Speciﬁcally, for every x i , there exists a distribution Ω i , such that for any z ∈ Ω i , the perturbation given by ( α , β )
Revised: Speciﬁcally, for every x i , there exists a distribution Ω i , such that for any z ∈ Ω i , the perturbation given by ( α , β )

Original: =
Revised: =

Original: h ( z ) can perturb x i to be abnormal, where h is a nonlinear function modelled by a neural network.
Revised: h ( z ) can perturb x i to be abnormal, where h is a nonlinear function modelled by a neural network.

Original: We can just assume that Ω i is a
Revised: We can just assume that Ω i is a

Original: Gaussian distribution with mean µ i and variance σ 2 i , i.e., Ω i = N ( µ i , σ 2 i ) , because of the universal approximation ability of h .
Revised: Gaussian distribution with mean µ i and variance σ 2 i , i.e., Ω i = N ( µ i , σ 2 i ) , because of the universal approximation ability of h .

Original: We take the idea of variational autoencoder (VAE)
Revised: We take the idea of variational autoencoder (VAE)

Original: [Kingma and Welling, 2013] and minimize
Revised: [Kingma and Welling, 2013] and minimize

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where p ( z ) = N ( 0 , 1 ) , ε = (cid:20) 10 (cid:21) , z = l ˜ θ 1 ( x ) , ( α , β ) =
Revised: where p ( z ) = N ( 0 , 1 ) , ε = (cid:20) 10 (cid:21) , z = l ˜ θ 1 ( x ) , ( α , β ) =

Original: h ˜ θ 2 ( z ) ,
Revised: h ˜ θ 2 ( z ) ,

Original: ˜ θ 1 are the parameters of the encoder l , and ˜ θ 2 are the parameters of the decoder h .
Revised: ˜ θ 1 are the parameters of the encoder l , and ˜ θ 2 are the parameters of the decoder h .

Original: Combing (4) with (3), we have ˜ θ
Revised: Combing (4) with (3), we have ˜ θ

Original: =
Revised: =

Original: { ˜ θ 1 , ˜ θ 2 } and solve minimize θ, ˜ θ
Revised: { ˜ θ 1 , ˜ θ 2 } and solve minimize θ, ˜ θ

Original: [Equation]
Revised: [Equation]

Original: where g ˜ θ = h ˜ θ 2 ◦ l ˜ θ 1 .
Revised: where g ˜ θ = h ˜ θ 2 ◦ l ˜ θ 1 .

Original: The training is similar to that for VAE [Kingma and Welling, 2013] and will not be detailed here.
Revised: The training is similar to that for VAE [Kingma and Welling, 2013] and will not be detailed here.

Original: It should be pointed out that the method (5) is just an extension of the method (3).
Revised: It should be pointed out that the method (5) is just an extension of the method (3).

Original: In (5), we want to learn a distribution for each normal training data point x i such that any perturbations generated from the distribution can perturb x i to be abnormal.
Revised: In (5), we want to learn a distribution for each normal training data point x i such that any perturbations generated from the distribution can perturb x i to be abnormal.

Original: It is expected that (5) can outperform (3) in real applications.
Revised: It is expected that (5) can outperform (3) in real applications.

Original: The corresponding experiments are in Appendix A.4.
Revised: The corresponding experiments are in Appendix A.4.

Original:  Connection with previous works
Revised:  Connection with previous works

Original: The well-known one-class classiﬁcation methods such as OCSVM [Schölkopf et al. , 2001], DSVDD [Ruff et al. , 2018] and DROCC [Goyal et al. , 2020] have speciﬁc assumptions for the embedded distribution while our PLAD does not require any assumption and is able to adaptively learn a decision boundary even if it is very complex.
Revised: The well-known one-class classiﬁcation methods such as OCSVM [Schölkopf et al. , 2001], DSVDD [Ruff et al. , 2018] and DROCC [Goyal et al. , 2020] have speciﬁc assumptions for the embedded distribution while our PLAD does not require any assumption and is able to adaptively learn a decision boundary even if it is very complex.

Original: It is also noteworthy that the idea of DROCC in identifying anomalies is slightly similar to ours, i.e, training a classiﬁer instead of an auto-encoder or embedding model.
Revised: It is also noteworthy that the idea of DROCC in identifying anomalies is slightly similar to ours, i.e, training a classiﬁer instead of an auto-encoder or embedding model.

Original: Self-supervised learning based methods, e.g., GOAD [Bergman and Hoshen, 2020] NeuTraL AD [Qiu et al. , 2021], etc., work through designing an appropriate auxiliary task to help the model to learn useful data features.
Revised: Self-supervised learning based methods, e.g., GOAD [Bergman and Hoshen, 2020] NeuTraL AD [Qiu et al. , 2021], etc., work through designing an appropriate auxiliary task to help the model to learn useful data features.

Original: Both our approach and the self-supervised learning based methods aim to learn a binary classiﬁer on the original data and auxiliary data.
Revised: Both our approach and the self-supervised learning based methods aim to learn a binary classiﬁer on the original data and auxiliary data.

Original: But they are different in the following two points.
Revised: But they are different in the following two points.

Original: First, in self-supervised learning (e.g. contrastive learning) based methods, the auxiliary data are generated by some pre-deﬁned and manual operations (e.g. image rotation and cropping).
Revised: First, in self-supervised learning (e.g. contrastive learning) based methods, the auxiliary data are generated by some pre-deﬁned and manual operations (e.g. image rotation and cropping).

Original: In contrast, in our approach, the auxiliary data are adaptively and dynamically learned from the original data.
Revised: In contrast, in our approach, the auxiliary data are adaptively and dynamically learned from the original data.

Original: Second, self-supervised learning based methods are often designed for some speciﬁc types of data such as image data, whereas it may be difﬁcult to generate auxiliary data for other types of data.
Revised: Second, self-supervised learning based methods are often designed for some speciﬁc types of data such as image data, whereas it may be difﬁcult to generate auxiliary data for other types of data.

Original: In contrast, it is easy to apply our approach to any type of data.
Revised: In contrast, it is easy to apply our approach to any type of data.

Original: Adversarial learning based methods [Malhotra et al. , 2016; Deecke et al. , 2018; Pidhorskyi et al. , 2018; Perera et al. , 2019] are generally constructed with auto-encoder and GAN [Goodfellow et al. , 2014], and the most widely used measure of them to detect anomalies is the reconstruction error.
Revised: Adversarial learning based methods [Malhotra et al. , 2016; Deecke et al. , 2018; Pidhorskyi et al. , 2018; Perera et al. , 2019] are generally constructed with auto-encoder and GAN [Goodfellow et al. , 2014], and the most widely used measure of them to detect anomalies is the reconstruction error.

Original: Compared with them, we exploit the idea of VAE when producing perturbations and the detection decision is made by a classiﬁer, which should be more suitable than reconstruction error for anomaly detection.
Revised: Compared with them, we exploit the idea of VAE when producing perturbations and the detection decision is made by a classiﬁer, which should be more suitable than reconstruction error for anomaly detection.

Original: On the other hand, in these adversarial learning based AD methods, the min-max optimization leads to instabilities in detecting anomalies, while the optimization of PLAD is much easier to solve.
Revised: On the other hand, in these adversarial learning based AD methods, the min-max optimization leads to instabilities in detecting anomalies, while the optimization of PLAD is much easier to solve.

Original: Another interesting work SCADN [Yan et al. , 2021] tries to produce negative samples by multi-scale striped masks to train a GAN, but its anomaly score still relies on reconstruction error and the production of masks has randomness or may be hard to determine in various real scenarios.
Revised: Another interesting work SCADN [Yan et al. , 2021] tries to produce negative samples by multi-scale striped masks to train a GAN, but its anomaly score still relies on reconstruction error and the production of masks has randomness or may be hard to determine in various real scenarios.

Original: Our PLAD learns perturbations adaptively from the data itself, which is convenient and reliable.
Revised: Our PLAD learns perturbations adaptively from the data itself, which is convenient and reliable.

Original:  Experiment
Revised:  Experiment

Original: In this section, we evaluate the proposed method in comparison to several state-of-the-art anomaly detection methods on two image datasets and two tabular datasets.
Revised: In this section, we evaluate the proposed method in comparison to several state-of-the-art anomaly detection methods on two image datasets and two tabular datasets.

Original: Note that all the compared methods do not utilize any pre-trained feature extractors.
Revised: Note that all the compared methods do not utilize any pre-trained feature extractors.

Original: 4.1 Datasets and baseline methods Datasets description
Revised: 4.1 Datasets and baseline methods Datasets description

Original: • CIFAR-10:
Revised: • CIFAR-10:

Original: CIFAR-10 image dataset is composed of 60,000 images in total, where 50,000 samples for training and 10,000 samples for test.
Revised: CIFAR-10 image dataset is composed of 60,000 images in total, where 50,000 samples for training and 10,000 samples for test.

Original: It includes 10 different balanced classes.
Revised: It includes 10 different balanced classes.

Original: • Fashion-MNIST:
Revised: • Fashion-MNIST:

Original: Fashion MNIST contains 10 different categories of grey-scale fashion style objects.
Revised: Fashion MNIST contains 10 different categories of grey-scale fashion style objects.

Original: The data is split into 60,000 images for training and 10,000 images for test.
Revised: The data is split into 60,000 images for training and 10,000 images for test.

Original: • Thyroid: Thyroid is a hypothyroid disease dataset that contains 3,772 samples with 3 classes and 6 attributes.
Revised: • Thyroid: Thyroid is a hypothyroid disease dataset that contains 3,772 samples with 3 classes and 6 attributes.

Original: We follow the data split settings of [Zong et al. , 2018] to preprocess the data for one-class classiﬁcation task.
Revised: We follow the data split settings of [Zong et al. , 2018] to preprocess the data for one-class classiﬁcation task.

Original: • Arrhythmia:
Revised: • Arrhythmia:

Original: Arrhythmia dataset consists of 452 samples with 274 attributes.
Revised: Arrhythmia dataset consists of 452 samples with 274 attributes.

Original: Here we also follow the data split settings of [Zong et al. , 2018] to preprocess the data.
Revised: Here we also follow the data split settings of [Zong et al. , 2018] to preprocess the data.

Original: The detailed information of each dataset is illustrated in Table 1.
Revised: The detailed information of each dataset is illustrated in Table 1.

Original: [Table]
Revised: [Table]

Original: Baselines and state-of-the-arts.
Revised: Baselines and state-of-the-arts.

Original: We compare our method with the following classical baseline methods and state-of-the-art methods: OCSVM [Schölkopf et al. , 2001], isolation forest (IF)
Revised: We compare our method with the following classical baseline methods and state-of-the-art methods: OCSVM [Schölkopf et al. , 2001], isolation forest (IF)

Original: [Liu et al. , 2008], local outlier factor (LOF)
Revised: [Liu et al. , 2008], local outlier factor (LOF)

Original: [Breunig et al. , 2000], denoising auto-encoder (DAE)
Revised: [Breunig et al. , 2000], denoising auto-encoder (DAE)

Original: [Vincent et al. , 2008], E2E-AE and DAGMM [Zong et al. , 2018], DCN [Caron et al. , 2018], ADGAN [Deecke et al. , 2018], DSVDD [Ruff et al. , 2018], OCGAN [Perera et al. , 2019], TQM [Wang et al. , 2019],
Revised: [Vincent et al. , 2008], E2E-AE and DAGMM [Zong et al. , 2018], DCN [Caron et al. , 2018], ADGAN [Deecke et al. , 2018], DSVDD [Ruff et al. , 2018], OCGAN [Perera et al. , 2019], TQM [Wang et al. , 2019],

Original: GOAD [Bergman and Hoshen, 2020], DROCC [Goyal et al. , 2020], HRN-L2 and HRN [Hu et al. , 2020], SCADN [Yan et al. , 2021], IGD (Scratch) [
Revised: GOAD [Bergman and Hoshen, 2020], DROCC [Goyal et al. , 2020], HRN-L2 and HRN [Hu et al. , 2020], SCADN [Yan et al. , 2021], IGD (Scratch) [

Original: [Table]
Revised: [Table]

Original: et al. , 2022], NeuTraL AD [
Revised: et al. , 2022], NeuTraL AD [

Original: [Table]
Revised: [Table]

Original: et al. , 2021], and GOCC [Shenkar and Wolf, 2022].
Revised: et al. , 2021], and GOCC [Shenkar and Wolf, 2022].

Original: 4.2 Implementation details and evaluation metrics
Revised: 4.2 Implementation details and evaluation metrics

Original: In this section, we ﬁrst describe the implementation details of the proposed PLAD method.
Revised: In this section, we ﬁrst describe the implementation details of the proposed PLAD method.

Original: The settings for image and tabular datasets are illustrated as follows:
Revised: The settings for image and tabular datasets are illustrated as follows:

Original: • Image datasets.
Revised: • Image datasets.

Original: For image datasets (CIFAR-10 and Fashion-MNIST), we utilize the LeNet-based CNN to construct the classiﬁer, which is same as [Ruff et al. , 2018] and [Goyal et al. , 2020] to provide fair comparison.
Revised: For image datasets (CIFAR-10 and Fashion-MNIST), we utilize the LeNet-based CNN to construct the classiﬁer, which is same as [Ruff et al. , 2018] and [Goyal et al. , 2020] to provide fair comparison.

Original: And we apply the MLP-based VAE to learn the noise for data.
Revised: And we apply the MLP-based VAE to learn the noise for data.

Original: Since both image datasets contain
Revised: Since both image datasets contain

Original: different classes, it can be regarded as 10 independent one-class classiﬁcation tasks, and each task on CIFAR-10 has 5,000 training samples (6,000 for FashionMNIST) and 10,000 testing samples for both of them.
Revised: different classes, it can be regarded as 10 independent one-class classiﬁcation tasks, and each task on CIFAR-10 has 5,000 training samples (6,000 for FashionMNIST) and 10,000 testing samples for both of them.

Original: Consequently, the choice of optimizer (from
Revised: Consequently, the choice of optimizer (from

Original: Adam [Kingma and Ba, 2015] and SGD), learning rate and hyper-parameter λ could be varies for different classes.
Revised: Adam [Kingma and Ba, 2015] and SGD), learning rate and hyper-parameter λ could be varies for different classes.

Original: The suggested settings of them on each experiment in this paper refer to
Revised: The suggested settings of them on each experiment in this paper refer to

Original: Appendix A.1.
Revised: Appendix A.1.

Original: • Tabular datasets.
Revised: • Tabular datasets.

Original: For tabular datasets (Thyroid and Arrhythmia), we both use the MLP-based classiﬁer and VAE in practice, and we train them by Adam optimizer with learning rate 0.001.
Revised: For tabular datasets (Thyroid and Arrhythmia), we both use the MLP-based classiﬁer and VAE in practice, and we train them by Adam optimizer with learning rate 0.001.

Original: Besides, λ is set to 3 for Thyroid and 2 for Arrhythmia.
Revised: Besides, λ is set to 3 for Thyroid and 2 for Arrhythmia.

Original: For the compared methods in the experiment, we report their performance directly from the following paper [Hu et al. , 2020; Goyal et al. , 2020; Yan et al. , 2021;
Revised: For the compared methods in the experiment, we report their performance directly from the following paper [Hu et al. , 2020; Goyal et al. , 2020; Yan et al. , 2021;

Original: Qiu et al. , 2021; Chen et al. , 2022;
Revised: Qiu et al. , 2021; Chen et al. , 2022;

Original: Shenkar and Wolf, 2022] except for DROCC, which we run the ofﬁcial released code to obtain the results.
Revised: Shenkar and Wolf, 2022] except for DROCC, which we run the ofﬁcial released code to obtain the results.

Original: Due to the limitation of paper length, the details of the our network settings are provided in
Revised: Due to the limitation of paper length, the details of the our network settings are provided in

Original: Appendix A.1.
Revised: Appendix A.1.

Original: Regarding the evaluation metrics, we follow the previous works such as [Ruff et al. , 2018] and [Zong et al. , 2018] to use AUC (Area Under the ROC curve) for image datasets and F1-score for tabular datasets because the anomaly detection for image and tabular datasets has different evaluation criteria.
Revised: Regarding the evaluation metrics, we follow the previous works such as [Ruff et al. , 2018] and [Zong et al. , 2018] to use AUC (Area Under the ROC curve) for image datasets and F1-score for tabular datasets because the anomaly detection for image and tabular datasets has different evaluation criteria.

Original: Moreover, our method does not need pre-training like [Ruff et al. , 2018] and others did, so we train the proposed method 5 times with 100 epochs to obtain the average performance and standard deviation.
Revised: Moreover, our method does not need pre-training like [Ruff et al. , 2018] and others did, so we train the proposed method 5 times with 100 epochs to obtain the average performance and standard deviation.

Original: Note that we run all experiments on NVIDIA
Revised: Note that we run all experiments on NVIDIA

Original: RTX

Revised: RTX

Original: GPU with 32GB RAM, CUDA 11.0 and cuDNN 8.0.
Revised: GPU with 32GB RAM, CUDA 11.0 and cuDNN 8.0.

Original: 4.3 Experimental results on image datasets
Revised: 4.3 Experimental results on image datasets

Original: Table 2 and Table 3 summarize the average AUCs performance of the one-class classiﬁcation tasks on CIFAR-10 and Fashion-MNIST, where we have the following observations:
Revised: Table 2 and Table 3 summarize the average AUCs performance of the one-class classiﬁcation tasks on CIFAR-10 and Fashion-MNIST, where we have the following observations:

Original: • Compared with some classical shallow model-based approaches such OCSVM and IF, the proposed PLAD method signiﬁcantly outperforms them on each one-class classiﬁcation task with a large margin.
Revised: • Compared with some classical shallow model-based approaches such OCSVM and IF, the proposed PLAD method signiﬁcantly outperforms them on each one-class classiﬁcation task with a large margin.

Original: This is mainly due to the powerful feature learning capability of deep neural network.
Revised: This is mainly due to the powerful feature learning capability of deep neural network.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: • PLAD also explicitly outperforms several well-known deep anomaly detection methods such as DAGMM and DSVDD, and consistently obtains the top two AUC scores on most classes of CIFAR-10 and Fashion-MNIST compared to some latest methods such as TQM, DROCC and HRN.
Revised: • PLAD also explicitly outperforms several well-known deep anomaly detection methods such as DAGMM and DSVDD, and consistently obtains the top two AUC scores on most classes of CIFAR-10 and Fashion-MNIST compared to some latest methods such as TQM, DROCC and HRN.

Original: Speciﬁcally, on the class “Automobile” of CIFAR-10 and class “Sandal” of Fashion-MNIST, the AUC improvements of the proposed method exceed 5.9% and 4.7% respectively compared to the runner-up.
Revised: Speciﬁcally, on the class “Automobile” of CIFAR-10 and class “Sandal” of Fashion-MNIST, the AUC improvements of the proposed method exceed 5.9% and 4.7% respectively compared to the runner-up.

Original: We also conduct statistical analysis for our method against other competitors, the detailed analysis refers to Section 4.6.
Revised: We also conduct statistical analysis for our method against other competitors, the detailed analysis refers to Section 4.6.

Original: • It is noteworthy that some deep anomaly detection methods such as DSVDD and DROCC, are mainly based on the assumption that the normal data in the embedding space are situated in a hyper-sphere, while anomalies are outside the sphere.
Revised: • It is noteworthy that some deep anomaly detection methods such as DSVDD and DROCC, are mainly based on the assumption that the normal data in the embedding space are situated in a hyper-sphere, while anomalies are outside the sphere.

Original: Therefore the edge of the hyper-sphere is then the decision boundary learned by the model to identify anomalies.
Revised: Therefore the edge of the hyper-sphere is then the decision boundary learned by the model to identify anomalies.

Original: In contrast, PLAD does not require any assumption about the shape of the decision boundary.
Revised: In contrast, PLAD does not require any assumption about the shape of the decision boundary.

Original: It attempts to learn the perturbation from data itself by neural network and construct the anomalies by enforcing the perturbation to original data, then train the network to distinguish the normal samples and anomalies.
Revised: It attempts to learn the perturbation from data itself by neural network and construct the anomalies by enforcing the perturbation to original data, then train the network to distinguish the normal samples and anomalies.

Original: This is natural to relate PLAD with some adversarial learning based methods such as ADGAN and OCGAN, etc.
Revised: This is natural to relate PLAD with some adversarial learning based methods such as ADGAN and OCGAN, etc.

Original: In contrast to them, the optimization of PLAD is a non-adversary problem and hence is easier to solve.
Revised: In contrast to them, the optimization of PLAD is a non-adversary problem and hence is easier to solve.

Original: [Figure]
Revised: [Figure]

Original: Table 4 shows the average performance on CIFAR-10 and Fashion-MNIST over all 10 classes to provide an overall comparison.
Revised: Table 4 shows the average performance on CIFAR-10 and Fashion-MNIST over all 10 classes to provide an overall comparison.

Original: Note that two more latest methods SCADN and IGD (Scratch) are also compared in the table, though their performance on each single class were not reported in their papers.
Revised: Note that two more latest methods SCADN and IGD (Scratch) are also compared in the table, though their performance on each single class were not reported in their papers.

Original: From the table we observe that PLAD achieves the best average AUCs on both datasets among all compared methods.
Revised: From the table we observe that PLAD achieves the best average AUCs on both datasets among all compared methods.

Original: [Table]
Revised: [Table]

Original: We use t-SNE [Van der Maaten and Hinton, 2008] to visualize the learned embedding features of
Revised: We use t-SNE [Van der Maaten and Hinton, 2008] to visualize the learned embedding features of

Original: PLAD.
Revised: PLAD.

Original: Speciﬁcally, we visualize the training samples of a speciﬁc category, perturbed data for training samples, as well as all test samples.
Revised: Speciﬁcally, we visualize the training samples of a speciﬁc category, perturbed data for training samples, as well as all test samples.

Original: They are divided into three sub-ﬁgures for ease of understanding.
Revised: They are divided into three sub-ﬁgures for ease of understanding.

Original: Figure 2 shows the visualization of the class “Sneaker” in Fashion-MNIST.
Revised: Figure 2 shows the visualization of the class “Sneaker” in Fashion-MNIST.

Original: From this ﬁgure we have the following observations.
Revised: From this ﬁgure we have the following observations.

Original: First, PLAD obviously learns good decision boundary for distinguishing normal data and perturbed data from Figure 2(a).
Revised: First, PLAD obviously learns good decision boundary for distinguishing normal data and perturbed data from Figure 2(a).

Original: Second, Figures 2(b) and 2(c) show that the normal test samples lie in the same manifold as the training samples, while the abnormal samples (abnormal test samples and perturbed samples) are relatively separated.
Revised: Second, Figures 2(b) and 2(c) show that the normal test samples lie in the same manifold as the training samples, while the abnormal samples (abnormal test samples and perturbed samples) are relatively separated.

Original: In other words, PLAD explicitly learns a discriminative embedding space to distinguish normal samples and anomalies.
Revised: In other words, PLAD explicitly learns a discriminative embedding space to distinguish normal samples and anomalies.

Original: More results can be found in Appendix A.2.
Revised: More results can be found in Appendix A.2.

Original: 4.4 Experimental results on non-image datasets
Revised: 4.4 Experimental results on non-image datasets

Original: Table 5 summarizes the F1-scores of each compared method on the Thyroid and Arrhythmia datasets.
Revised: Table 5 summarizes the F1-scores of each compared method on the Thyroid and Arrhythmia datasets.

Original: It can be observed that PLAD signiﬁcantly outperforms several baseline methods such as OCSVM,
Revised: It can be observed that PLAD signiﬁcantly outperforms several baseline methods such as OCSVM,

Original: DAGMM, DSVDD and DROCC with a large margin.
Revised: DAGMM, DSVDD and DROCC with a large margin.

Original: Although NeuTraL AD and GOCC achieve encouraging 76.8% F1-score on Thyroid, it is worth mentioning that they are both methods designed for non-image data.
Revised: Although NeuTraL AD and GOCC achieve encouraging 76.8% F1-score on Thyroid, it is worth mentioning that they are both methods designed for non-image data.

Original: The Arrhythmia dataset seems to be a more difﬁcult anomaly detection task because of its small sample size, which is not conducive to deep learning.
Revised: The Arrhythmia dataset seems to be a more difﬁcult anomaly detection task because of its small sample size, which is not conducive to deep learning.

Original: Surprisingly, the proposed
Revised: Surprisingly, the proposed

Original: PLAD method shows remarkable performance on Arrhythmia, which surpasses 9.2% compared to the runner-up.
Revised: PLAD method shows remarkable performance on Arrhythmia, which surpasses 9.2% compared to the runner-up.

Original: Moreover, the performance of PLAD is also comparable to NeuTraL AD and GOCC on Thyroid, which fully demonstrates its applicability to the anomaly detection task for non-image data.
Revised: Moreover, the performance of PLAD is also comparable to NeuTraL AD and GOCC on Thyroid, which fully demonstrates its applicability to the anomaly detection task for non-image data.

Original: [Table]
Revised: [Table]

Original: 4.5 Experiment of separating anomaly from multi-class normal data
Revised: 4.5 Experiment of separating anomaly from multi-class normal data

Original: It should be pointed out that in real applications, the normal data may contain multiple classes without labels.
Revised: It should be pointed out that in real applications, the normal data may contain multiple classes without labels.

Original: We need to separate anomalies from these multi-class normal data.
Revised: We need to separate anomalies from these multi-class normal data.

Original: In this study, we randomly select 10,000 samples among the training split of CIFAR-10 or Fashion-MNIST to construct a new normal training set, namely, the normal data in multiple classes.
Revised: In this study, we randomly select 10,000 samples among the training split of CIFAR-10 or Fashion-MNIST to construct a new normal training set, namely, the normal data in multiple classes.

Original: Subsequently, we randomly select two samples from the test split of CIFAR-10 or Fashion-MNIST to construct 10,000 anomalous samples using the means of pair-wise samples at the pixel level.
Revised: Subsequently, we randomly select two samples from the test split of CIFAR-10 or Fashion-MNIST to construct 10,000 anomalous samples using the means of pair-wise samples at the pixel level.

Original: The produced anomalous samples are merged with the original test split to form a new test set.
Revised: The produced anomalous samples are merged with the original test split to form a new test set.

Original: Compared with the previous two tasks (Sections 4.3 and 4.4), this one is much more difﬁcult because the decision boundary between the anomalous samples and the normal samples is very complicated.
Revised: Compared with the previous two tasks (Sections 4.3 and 4.4), this one is much more difﬁcult because the decision boundary between the anomalous samples and the normal samples is very complicated.

Original: We run the experiment to compare our method with four baselines including OCSVM, DAGMM, DSVDD, DROCC and report the results in Table 6.
Revised: We run the experiment to compare our method with four baselines including OCSVM, DAGMM, DSVDD, DROCC and report the results in Table 6.

Original: We see that the proposed PLAD method outperforms the baselines signiﬁcantly.
Revised: We see that the proposed PLAD method outperforms the baselines signiﬁcantly.

Original: For example, the improvement over the runner-up DSVDD are 9.1% and 4.4% on CIFAR-10 and
Revised: For example, the improvement over the runner-up DSVDD are 9.1% and 4.4% on CIFAR-10 and

Original: Fashion-MNIST respectively.
Revised: Fashion-MNIST respectively.

Original: The success of PLAD mainly stems from the ability of learning a decision boundary adaptively without any assumption.
Revised: The success of PLAD mainly stems from the ability of learning a decision boundary adaptively without any assumption.

Original: [Table]
Revised: [Table]

Original: 4.6 Statistical analysis of the proposed method
Revised: 4.6 Statistical analysis of the proposed method

Original: To assess whether the results obtained of our method are statistically signiﬁcantly different compared to others, we further conduct statistical analysis for them.
Revised: To assess whether the results obtained of our method are statistically signiﬁcantly different compared to others, we further conduct statistical analysis for them.

Original: Student’s t-test is a common scheme for assessing the signiﬁcance of differences between two groups of data.
Revised: Student’s t-test is a common scheme for assessing the signiﬁcance of differences between two groups of data.

Original: Usually, the difference is said to be signiﬁcant if the p -value obtained in the t-test is less than 0.05.
Revised: Usually, the difference is said to be signiﬁcant if the p -value obtained in the t-test is less than 0.05.

Original: We compare with three baseline methods including DSVDD [Ruff et al. , 2018], DROCC [Goyal et al. , 2020], and HRN [Hu et al. , 2020] on Fashion-MNIST.
Revised: We compare with three baseline methods including DSVDD [Ruff et al. , 2018], DROCC [Goyal et al. , 2020], and HRN [Hu et al. , 2020] on Fashion-MNIST.

Original: Note that we perform t-test experimental by using the results obtained from running their ofﬁcial released codes.
Revised: Note that we perform t-test experimental by using the results obtained from running their ofﬁcial released codes.

Original: Table 7 summarizes the reproduced results and the p -value
Revised: Table 7 summarizes the reproduced results and the p -value

Original: (t-test) of our method against other baseline methods.
Revised: (t-test) of our method against other baseline methods.

Original: We can see that the reproduced results are close to that reported in the related papers [Hu et al. , 2020; Goyal et al. , 2020], and the t-test results of our methods are statistically signiﬁcantly different ( p < 0 .
Revised: We can see that the reproduced results are close to that reported in the related papers [Hu et al. , 2020; Goyal et al. , 2020], and the t-test results of our methods are statistically signiﬁcantly different ( p < 0 .

Original: 01 ) from other comparative methods in most cases except for the comparison with HRN on the “Trouser” and “Dress” classes.
Revised: 01 ) from other comparative methods in most cases except for the comparison with HRN on the “Trouser” and “Dress” classes.

Original: [Table]
Revised: [Table]

Original: 4.7 Extension of PLAD to time-series anomaly detection
Revised: 4.7 Extension of PLAD to time-series anomaly detection

Original: We further conduct experiment to demonstrate the effectiveness of PLAD on the time-series anomaly detection task, the details refer to Table 15 in Appendix A.7 due to the limitation of the paper length.
Revised: We further conduct experiment to demonstrate the effectiveness of PLAD on the time-series anomaly detection task, the details refer to Table 15 in Appendix A.7 due to the limitation of the paper length.

Original:  Conclusion
Revised:  Conclusion

Original: We have presented a novel method PLAD for anomaly detection.
Revised: We have presented a novel method PLAD for anomaly detection.

Original: Compared with its competitors,
Revised: Compared with its competitors,

Original: PLAD does not require any assumption about the distribution or structure of the normal data.
Revised: PLAD does not require any assumption about the distribution or structure of the normal data.

Original: This is the major reason that PLAD outperforms its competitors.
Revised: This is the major reason that PLAD outperforms its competitors.

Original: In addition, PLAD has fewer hyperparameters to determine and has a lower computation cost than many strong baselines such as [Wang et al. , 2019; Goyal et al. , 2020; Yan et al. , 2021].
Revised: In addition, PLAD has fewer hyperparameters to determine and has a lower computation cost than many strong baselines such as [Wang et al. , 2019; Goyal et al. , 2020; Yan et al. , 2021].

Original: Actually, PLAD provides us a framework for anomaly detection.
Revised: Actually, PLAD provides us a framework for anomaly detection.

Original: Different neural networks such as CNN [Krizhevsky et al. , 2012], RNN [Mikolov et al. , 2010], GNN [Scarselli et al. , 2008], and even transformer Vaswani et al.
Revised: Different neural networks such as CNN [Krizhevsky et al. , 2012], RNN [Mikolov et al. , 2010], GNN [Scarselli et al. , 2008], and even transformer Vaswani et al.

Original: [2017] can be embedded into PLAD to accomplish various anomaly detection tasks such as time series anomaly detection.
Revised: [2017] can be embedded into PLAD to accomplish various anomaly detection tasks such as time series anomaly detection.

Original: One limitation of our work is that we haven’t included enough these experiments currently.
Revised: One limitation of our work is that we haven’t included enough these experiments currently.

Original: Note that in this study, we only considered the additive and multiplicative perturbations, i.e., ˜ x
Revised: Note that in this study, we only considered the additive and multiplicative perturbations, i.e., ˜ x

Original: = x (cid:12) g α ˜ θ ( x ) + g β ˜ θ ( x ) , while a more general perturbation can be formulated as ˜ x
Revised: = x (cid:12) g α ˜ θ ( x ) + g β ˜ θ ( x ) , while a more general perturbation can be formulated as ˜ x

Original: = g (cid:48)
Revised: = g (cid:48)

Original: ˜
Revised: ˜

Original: θ ( x ) .
Revised: θ ( x ) .

Original: When we need a very sophisticated perturbator, we may not well approximate g (cid:48)
Revised: When we need a very sophisticated perturbator, we may not well approximate g (cid:48)

Original: ˜
Revised: ˜

Original: θ ( x ) by x (cid:12) g α ˜
Revised: θ ( x ) by x (cid:12) g α ˜

Original: θ ( x )
Revised: θ ( x )

Original: + g β ˜
Revised: + g β ˜

Original: θ ( x ) , because we have to made some restriction for g α ˜ θ
Revised: θ ( x ) , because we have to made some restriction for g α ˜ θ

Original: and g β ˜ θ .
Revised: and g β ˜ θ .

Original: Therefore, future study may also focus on determining the φ for (1), to learn g (cid:48)
Revised: Therefore, future study may also focus on determining the φ for (1), to learn g (cid:48)

Original: ˜
Revised: ˜

Original: θ ( x ) more effectively.
Revised: θ ( x ) more effectively.

Original: Acknowledgments
Revised: Acknowledgments

Original: The work was supported by the research funding T00120210002 of Shenzhen Research Institute of
Revised: The work was supported by the research funding T00120210002 of Shenzhen Research Institute of

Original: Big Data and the Youth program 62106211 of the National Natural Science Foundation of China.
Revised: Big Data and the Youth program 62106211 of the National Natural Science Foundation of China.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information on how to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section.
Revised: [Yes] See Section.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether
Revised: [N/A] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Appendix A.1 Detailed settings of the network architecture, hyperparameter, and optimization CIFAR
Revised: A Appendix A.1 Detailed settings of the network architecture, hyperparameter, and optimization CIFAR

Original: : We use
Revised: : We use

Original: LeNet based classiﬁer with 4 convolutional layers of kernel size 5 and 3 linear layers for CIFAR-10 in this paper.
Revised: LeNet based classiﬁer with 4 convolutional layers of kernel size 5 and 3 linear layers for CIFAR-10 in this paper.

Original: For the perturbator, we use fully-connected network based VAE.
Revised: For the perturbator, we use fully-connected network based VAE.

Original: Note that we do not perform dimension reduction in the perturbator because our aim is to learn the perturbation from the data.
Revised: Note that we do not perform dimension reduction in the perturbator because our aim is to learn the perturbation from the data.

Original: We use LeaklyReLU as the activation function for both the classiﬁer and perturbator.
Revised: We use LeaklyReLU as the activation function for both the classiﬁer and perturbator.

Original: The detailed network structure is shown in Table 8.
Revised: The detailed network structure is shown in Table 8.

Original: Fashion-MNIST 4 : We use LeNet-based classiﬁer with 2 convolutional layers of kernel size 5 andlinear layers for Fashion-MNIST in this paper.
Revised: Fashion-MNIST 4 : We use LeNet-based classiﬁer with 2 convolutional layers of kernel size 5 andlinear layers for Fashion-MNIST in this paper.

Original: The activation function and perturbator are similar to the settings for CIFAR-10.
Revised: The activation function and perturbator are similar to the settings for CIFAR-10.

Original: The detailed network structure is shown in Table 9.
Revised: The detailed network structure is shown in Table 9.

Original: Thyroid 5 and Arrhythmia 6 : For Thyroid and Arrhythmia, we use the same fully-connected network based classiﬁer constructed with single hidden layer.
Revised: Thyroid 5 and Arrhythmia 6 : For Thyroid and Arrhythmia, we use the same fully-connected network based classiﬁer constructed with single hidden layer.

Original: Denoting the dimensionality of input data as d , the size of each layer in the perturbator is then d .
Revised: Denoting the dimensionality of input data as d , the size of each layer in the perturbator is then d .

Original: The detailed network structure is shown in Table 10.
Revised: The detailed network structure is shown in Table 10.

Original: The one-class classiﬁcation of each class can be regarded as an independent task, therefore the desirable settings of parameters for each class would be different.
Revised: The one-class classiﬁcation of each class can be regarded as an independent task, therefore the desirable settings of parameters for each class would be different.

Original: To improve the reproducibility of the paper, we provide some recommended settings for each parameter in Tables 11 and 12, including the hyper-parameter λ , the choice of the optimizer (from Adam [Kingma and Ba, 2015] and SGD), and the learning rate.
Revised: To improve the reproducibility of the paper, we provide some recommended settings for each parameter in Tables 11 and 12, including the hyper-parameter λ , the choice of the optimizer (from Adam [Kingma and Ba, 2015] and SGD), and the learning rate.

Original: [Table]
Revised: [Table]

Original: BatchNorm2d(
Revised: BatchNorm2d(

Original: [Table]
Revised: [Table]

Original: eps
Revised: eps

Original: [Table]
Revised: [Table]

Original: 1e
Revised: 1e

Original: [Table]
Revised: [Table]

Original: , afﬁne
Revised: , afﬁne

Original: [Table]
Revised: [Table]

Original: False)
Revised: False)

Original: , Leaky
Revised: , Leaky

Original: [Table]
Revised: [Table]

Original: ReLU(), MaxPool2d
Revised: ReLU(), MaxPool2d

Original: [Table]
Revised: [Table]

Original: Conv2d(in_channel=16, out_channel=32, kernel_size
Revised: Conv2d(in_channel=16, out_channel=32, kernel_size

Original: [Table]
Revised: [Table]

Original: 5, bias
Revised: 5, bias

Original: [Table]
Revised: [Table]

Original: False, padding
Revised: False, padding

Original: [Table]
Revised: [Table]

Original: BatchNorm2d(32,
Revised: BatchNorm2d(32,

Original: [Table]
Revised: [Table]

Original: 1e
Revised: 1e

Original: [Table]
Revised: [Table]

Original: 4, afﬁne
Revised: 4, afﬁne

Original: [Table]
Revised: [Table]

Original: ), Leaky
Revised: ), Leaky

Original: [Table]
Revised: [Table]

Original: (), MaxPool2d
Revised: (), MaxPool2d

Original: [Table] [Table]
Revised: [Table] [Table]

Original: (in_channel=32, out_channel=64,
Revised: (in_channel=32, out_channel=64,

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: BatchNorm2d(64, eps=1e-4, afﬁne=False), Leaky_ReLU(), MaxPool2d
Revised: BatchNorm2d(64, eps=1e-4, afﬁne=False), Leaky_ReLU(), MaxPool2d

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: Conv2d(in_channel=64, out_channel=128,
Revised: Conv2d(in_channel=64, out_channel=128,

Original: [Table]
Revised: [Table]

Original: _size=5,
Revised: _size=5,

Original: [Table]
Revised: [Table]

Original: , padding
Revised: , padding

Original: [Table] [Table]
Revised: [Table] [Table]

Original: BatchNorm2d(128,
Revised: BatchNorm2d(128,

Original: [Table]
Revised: [Table]

Original: 1e
Revised: 1e

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Linear(
Revised: Linear(

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Linear(128,
Revised: Linear(128,

Original: [Table]
Revised: [Table]

Original: Leaky_
Revised: Leaky_

Original: [Table]
Revised: [Table]

Original: Linear(64,
Revised: Linear(64,

Original: [Table]
Revised: [Table]

Original: bias
Revised: bias

Original: [Table]
Revised: [Table]

Original: Linear(3072,
Revised: Linear(3072,

Original: [Table]
Revised: [Table]

Original: Leaky
Revised: Leaky

Original: [Table]
Revised: [Table]

Original: ) µ : Linear(3072, 3072)
Revised: ) µ : Linear(3072, 3072)

Original: [Table]
Revised: [Table]

Original: σ : Linear(3072, 3072)
Revised: σ : Linear(3072, 3072)

Original: Reparameterzie( µ , σ )
Revised: Reparameterzie( µ , σ )

Original: Linear(3072,
Revised: Linear(3072,

Original: [Table]
Revised: [Table]

Original: Leaky
Revised: Leaky

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: -based Classiﬁer
Revised: -based Classiﬁer

Original: Conv2d(in_channel=1, out_channel=16, kernel_size=5, bias=False, padding=2)
Revised: Conv2d(in_channel=1, out_channel=16, kernel_size=5, bias=False, padding=2)

Original: BatchNorm2d(16, eps=1e-4, afﬁne=False), Leaky_ReLU(), MaxPool2d(2,2)
Revised: BatchNorm2d(16, eps=1e-4, afﬁne=False), Leaky_ReLU(), MaxPool2d(2,2)

Original: Conv2d(in_channel=16, out_channel=32, kernel_size=5, bias=
Revised: Conv2d(in_channel=16, out_channel=32, kernel_size=5, bias=

Original: False, padding=2)
Revised: False, padding=2)

Original: BatchNorm2d(32, eps=1e-4, afﬁne=
Revised: BatchNorm2d(32, eps=1e-4, afﬁne=

Original: False), Leaky_ReLU(), MaxPool2d(2,2)
Revised: False), Leaky_ReLU(), MaxPool2d(2,2)

Original: Flatten()
Revised: Flatten()

Original: Linear(32 × 7 × 7, 128, bias=False)
Revised: Linear(32 × 7 × 7, 128, bias=False)

Original: Leaky_ReLU()
Revised: Leaky_ReLU()

Original: Linear(128, 64, bias=False)
Revised: Linear(128, 64, bias=False)

Original: Leaky_ReLU()
Revised: Leaky_ReLU()

Original: Linear(64, 1, bias=False)
Revised: Linear(64, 1, bias=False)

Original: VAE-based Perturbator
Revised: VAE-based Perturbator

Original: Linear(784, 784)
Revised: Linear(784, 784)

Original: Leaky_ReLU() µ : Linear(784, 784); σ : Linear(784, 784)
Revised: Leaky_ReLU() µ : Linear(784, 784); σ : Linear(784, 784)

Original: Reparameterzie( µ , σ )
Revised: Reparameterzie( µ , σ )

Original: Linear(784, 784)
Revised: Linear(784, 784)

Original: Leaky_ReLU()
Revised: Leaky_ReLU()

Original: Linear(784,784 × 2)
Revised: Linear(784,784 × 2)

Original: [Table]
Revised: [Table]

Original: Input_dim =
Revised: Input_dim =

Original: d
Revised: d

Original: Linear(d, d), ReLU()
Revised: Linear(d, d), ReLU()

Original: Linear(d, 20), ReLU() µ : Linear(d, d); σ : Linear(d, d)
Revised: Linear(d, 20), ReLU() µ : Linear(d, d); σ : Linear(d, d)

Original: Linear(20, 1)
Revised: Linear(20, 1)

Original: Reparameterzie( µ , σ )
Revised: Reparameterzie( µ , σ )

Original: Linear(d, d), ReLU()
Revised: Linear(d, d), ReLU()

Original: Linear(d, d × 2)
Revised: Linear(d, d × 2)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: A.2 Visualization of the learned embedding space by PLAD
Revised: A.2 Visualization of the learned embedding space by PLAD

Original: We further provide the visualization of the learned embedding space by PLAD method on FashionMNIST in Figures 3 and 4.
Revised: We further provide the visualization of the learned embedding space by PLAD method on FashionMNIST in Figures 3 and 4.

Original: Note that we use t-SNE [Van der Maaten and Hinton, 2008] method to process the training samples of each class together with the test set, i.e., to reduce their dimensionality to 3, and they are marked in different colors.
Revised: Note that we use t-SNE [Van der Maaten and Hinton, 2008] method to process the training samples of each class together with the test set, i.e., to reduce their dimensionality to 3, and they are marked in different colors.

Original: We can observe that although the learned decision boundary of PLAD does not based on any assumption, it still adaptively distinguishes the normal samples and anomalies.
Revised: We can observe that although the learned decision boundary of PLAD does not based on any assumption, it still adaptively distinguishes the normal samples and anomalies.

Original: Moreover, the training samples marked in blue and normal test samples marked in green are projected to the same space, which is consistent with our expectation of learning a space that accommodates only normal samples through the training data.
Revised: Moreover, the training samples marked in blue and normal test samples marked in green are projected to the same space, which is consistent with our expectation of learning a space that accommodates only normal samples through the training data.

Original: Of course, the 3-D visualization shown is only for an intuitive understanding of PLAD and may not be the optimal decision boundary, so the visualization on some classes (such as Pullover and Shirt) is not desirable.
Revised: Of course, the 3-D visualization shown is only for an intuitive understanding of PLAD and may not be the optimal decision boundary, so the visualization on some classes (such as Pullover and Shirt) is not desirable.

Original: PLAD may obtain a better decision boundary in a higher dimension.
Revised: PLAD may obtain a better decision boundary in a higher dimension.

Original: A.3 Illustration of the anomalies produced from the multi-class normal data
Revised: A.3 Illustration of the anomalies produced from the multi-class normal data

Original: To better understand the experiment conducted in Section 4.5, we show the anomalies produced from pair-wise samples in Figure 5.
Revised: To better understand the experiment conducted in Section 4.5, we show the anomalies produced from pair-wise samples in Figure 5.

Original: We select part of pair-wise normal samples from CIFARand Fashion-MNIST, then use their means in pixel level to produce anomalies.
Revised: We select part of pair-wise normal samples from CIFARand Fashion-MNIST, then use their means in pixel level to produce anomalies.

Original: We can also observe from this ﬁgure that the anomalies produced in this way contain characteristics from multiple classes, which is a much more difﬁcult anomaly detection task to solve because the decision boundary between the anomalous samples and the normal samples are very complicated.
Revised: We can also observe from this ﬁgure that the anomalies produced in this way contain characteristics from multiple classes, which is a much more difﬁcult anomaly detection task to solve because the decision boundary between the anomalous samples and the normal samples are very complicated.

Original: A.4 Comparison between the AE-based and VAE-based perturbators in PLAD
Revised: A.4 Comparison between the AE-based and VAE-based perturbators in PLAD

Original: We further evaluate the performance of AE-based and VAE-based perturbators.
Revised: We further evaluate the performance of AE-based and VAE-based perturbators.

Original: Speciﬁcally, we build an AE-based perturbator, whose architecture is the same as the VAE-based one.
Revised: Speciﬁcally, we build an AE-based perturbator, whose architecture is the same as the VAE-based one.

Original: Then we run the experiment on CIFAR-10 and Fashion-MNIST following the same experimental setup as mentioned in Appendix A.1.
Revised: Then we run the experiment on CIFAR-10 and Fashion-MNIST following the same experimental setup as mentioned in Appendix A.1.

Original: The experimental results are shown in Table 13.
Revised: The experimental results are shown in Table 13.

Original: We can observe that the
Revised: We can observe that the

Original: VAE-based perturbator generally performs better than the AE-based one in most classes, especially in the “Bird” and “Deer” classes on CIFAR-10, and in the “Shirt” and “Bag” classes on Fashion-MNIST.
Revised: VAE-based perturbator generally performs better than the AE-based one in most classes, especially in the “Bird” and “Deer” classes on CIFAR-10, and in the “Shirt” and “Bag” classes on Fashion-MNIST.

Original: Nevertheless, the AE-based perturbator still achieves remarkable performance on some classes.
Revised: Nevertheless, the AE-based perturbator still achieves remarkable performance on some classes.

Original: For example, it outperforms VAE-based perturbator on “Truck”, “Sandal” and “Ankle boot”.
Revised: For example, it outperforms VAE-based perturbator on “Truck”, “Sandal” and “Ankle boot”.

Original: Overall,
Revised: Overall,

Original: [Figure]
Revised: [Figure]

Original: the average performance of AE-based and VAE-based methods both outperform most competing methods in Section 4.
Revised: the average performance of AE-based and VAE-based methods both outperform most competing methods in Section 4.

Original: [Table]
Revised: [Table]

Original: A.5 Comparison between the CNN-based perturbator and FCN-based perturbator in PLAD
Revised: A.5 Comparison between the CNN-based perturbator and FCN-based perturbator in PLAD

Original: As the above experiment can be seen
Revised: As the above experiment can be seen

Original: [Table]
Revised: [Table]

Original: VAE-based perturbator generally performs better than AEbased one.
Revised: VAE-based perturbator generally performs better than AEbased one.

Original: Therefore, we further evaluate the performance of CNN-based and FCN-based VAE perturbators to provide a more comprehensive analysis.
Revised: Therefore, we further evaluate the performance of CNN-based and FCN-based VAE perturbators to provide a more comprehensive analysis.

Original: Similarly, we experiment on CIFARand Fashion-MNIST.
Revised: Similarly, we experiment on CIFARand Fashion-MNIST.

Original: The encoder of CNN-based perturbator is similar to the classiﬁer, with two differences that it allows bias and contains two hidden layers to produce µ and σ for VAE.
Revised: The encoder of CNN-based perturbator is similar to the classiﬁer, with two differences that it allows bias and contains two hidden layers to produce µ and σ for VAE.

Original: The
Revised: The

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: dimension of hidden layer is set to 128, and the decoder is symmetric to the encoder.
Revised: dimension of hidden layer is set to 128, and the decoder is symmetric to the encoder.

Original: We show the experimental results in Table 14.
Revised: We show the experimental results in Table 14.

Original: We can observe that the performance of CNN-based and FCN-based perturbators are comparable in most cases, except for the more remarkable performance achieved by the FCN-based perturbator on the “Airplane” and “Dog” classes of CIFAR-10.
Revised: We can observe that the performance of CNN-based and FCN-based perturbators are comparable in most cases, except for the more remarkable performance achieved by the FCN-based perturbator on the “Airplane” and “Dog” classes of CIFAR-10.

Original: Nevertheless, the average performance of them are quite close and it should be noted that the CNN-based perturbator may have an encouraging performance in more complex scenarios
Revised: Nevertheless, the average performance of them are quite close and it should be noted that the CNN-based perturbator may have an encouraging performance in more complex scenarios

Original: A.6 Ablation study for PLAD
Revised: A.6 Ablation study for PLAD

Original: We conduct an ablation study on CIFAR-10 to validate the effectiveness of the proposed method, and also discuss the inﬂuence on the hyper-parameter λ to the anomaly detection performance.
Revised: We conduct an ablation study on CIFAR-10 to validate the effectiveness of the proposed method, and also discuss the inﬂuence on the hyper-parameter λ to the anomaly detection performance.

Original: Speciﬁcally, we vary the value of λ in the range of [0 , 0 . 1 , . .
Revised: Speciﬁcally, we vary the value of λ in the range of [0 , 0 . 1 , . .

Original: , 50 , 100] , and show the average AUCs in Figure 6.
Revised: , 50 , 100] , and show the average AUCs in Figure 6.

Original: Note that the performance shown in the dotted line indicates the degradation model of
Revised: Note that the performance shown in the dotted line indicates the degradation model of

Original: PLAD, which drops the perturber, i.e., contains only a simple classiﬁcation network without any perturbation learned.
Revised: PLAD, which drops the perturber, i.e., contains only a simple classiﬁcation network without any perturbation learned.

Original: We have the following observation from this ﬁgure:
Revised: We have the following observation from this ﬁgure:

Original: • The most signiﬁcant AUC gap observed from Figure 6 is whether the perturbator is considered or not.
Revised: • The most signiﬁcant AUC gap observed from Figure 6 is whether the perturbator is considered or not.

Original: Even with a very small value of λ (e
Revised: Even with a very small value of λ (e

Original: g., 0 . 1 ), we can see remarkable performance improvements on some classes such as “Airplane”, “Bird”, and ‘’Deer”.
Revised: g., 0 . 1 ), we can see remarkable performance improvements on some classes such as “Airplane”, “Bird”, and ‘’Deer”.

Original: This indicates that
Revised: This indicates that

Original: [Table]
Revised: [Table]

Original: the proposed perturbator forces the classiﬁer to learn more discriminative decision boundary to distinguish the normal samples and anomalies.
Revised: the proposed perturbator forces the classiﬁer to learn more discriminative decision boundary to distinguish the normal samples and anomalies.

Original: • Generally, we can observe the performance improves as the value of λ increases.
Revised: • Generally, we can observe the performance improves as the value of λ increases.

Original: Yet, as mentioned before, anomaly detection for each class is an independent task, so the desirable range of λ is different.
Revised: Yet, as mentioned before, anomaly detection for each class is an independent task, so the desirable range of λ is different.

Original: For example, λ ∈ [1 , 5] performs well on classes “Automobile”, “Deer”, “Horse”, and “Truck”, while λ ∈ [5 , 20] performs well on classes “Bird”, “Dog”, “Frog”, and “Ship”.
Revised: For example, λ ∈ [1 , 5] performs well on classes “Automobile”, “Deer”, “Horse”, and “Truck”, while λ ∈ [5 , 20] performs well on classes “Bird”, “Dog”, “Frog”, and “Ship”.

Original: • We can also see that the performance commonly degrades to some extent when λ is too large, except on the class ‘Frog”, where λ = 100 even performs better than λ = 50 .
Revised: • We can also see that the performance commonly degrades to some extent when λ is too large, except on the class ‘Frog”, where λ = 100 even performs better than λ = 50 .

Original: Nevertheless, the proposed PLAD method shows robustness to the variation of λ and still achieves a relatively stable performance even against very large values
Revised: Nevertheless, the proposed PLAD method shows robustness to the variation of λ and still achieves a relatively stable performance even against very large values

Original: [Figure]
Revised: [Figure]

Original: v e r a g e
Revised: v e r a g e

Original: [Figure]
Revised: [Figure]

Original: A.7 Extension of PLAD to time-series anomaly detection
Revised: A.7 Extension of PLAD to time-series anomaly detection

Original: In this section, we further conduct an experiment to demonstrate the effectiveness of PLAD on the time-series anomaly detection task.
Revised: In this section, we further conduct an experiment to demonstrate the effectiveness of PLAD on the time-series anomaly detection task.

Original: Speciﬁcally, we compare with ﬁve baseline methods on Epileptic
Revised: Speciﬁcally, we compare with ﬁve baseline methods on Epileptic

Original: Seizure, which is a multivariate time-series dataset [Andrzejak et al. , 2001].
Revised: Seizure, which is a multivariate time-series dataset [Andrzejak et al. , 2001].

Original: Dataset description: Epileptic Seizure 7 consists of the EEG values recorded in 500 patients over 23.5 seconds, with 178 data points contained per second.
Revised: Dataset description: Epileptic Seizure 7 consists of the EEG values recorded in 500 patients over 23.5 seconds, with 178 data points contained per second.

Original: Therefore, the dataset contains 500 × 23 = 11 , 500 sequential data in total, with 5 categories of EEG values (each one has 2, samples).
Revised: Therefore, the dataset contains 500 × 23 = 11 , 500 sequential data in total, with 5 categories of EEG values (each one has 2, samples).

Original: In this paper, we divide the dataset into two categories (epileptic seizure or not).
Revised: In this paper, we divide the dataset into two categories (epileptic seizure or not).

Original: The detection task is to recognize whether the EEG values are abnormal (i.e., is epileptic seizure activity or not).
Revised: The detection task is to recognize whether the EEG values are abnormal (i.e., is epileptic seizure activity or not).

Original: Experimental Settings: We split the Epileptic Seizure dataset into a training set with 6,900 normal samples and a testing set with 4,600 samples (half normal and half abnormal).
Revised: Experimental Settings: We split the Epileptic Seizure dataset into a training set with 6,900 normal samples and a testing set with 4,600 samples (half normal and half abnormal).

Original: For fair comparison, we use a single layer LSTM for all experiments following the settings in [Goyal et al. , 2020] to handle time-series data.
Revised: For fair comparison, we use a single layer LSTM for all experiments following the settings in [Goyal et al. , 2020] to handle time-series data.

Original: For the proposed PLAD method, we simply construct a single layer AE-based perturbator to learn perturbations, and connect a fully-connected (FC) layer after the hidden state of the last time step of the LSTM layer in the classiﬁer to obtain anomaly detection results.
Revised: For the proposed PLAD method, we simply construct a single layer AE-based perturbator to learn perturbations, and connect a fully-connected (FC) layer after the hidden state of the last time step of the LSTM layer in the classiﬁer to obtain anomaly detection results.

Original: We choose
Revised: We choose

Original: Adam as the optimizer with a learning rate of 1 × 10 − 5 and λ = 0 .
Revised: Adam as the optimizer with a learning rate of 1 × 10 − 5 and λ = 0 .

Original: The comparative methods include k -NN, AE [Sakurada and Yairi, 2014], DAGMM [Zong et al. , 2018], DSVDD [Ruff et al. , 2018], and DROCC [Goyal et al. , 2020].
Revised: The comparative methods include k -NN, AE [Sakurada and Yairi, 2014], DAGMM [Zong et al. , 2018], DSVDD [Ruff et al. , 2018], and DROCC [Goyal et al. , 2020].

Original: Each method is run 10 times to obtain the average AUC scores and their standard deviations.
Revised: Each method is run 10 times to obtain the average AUC scores and their standard deviations.

Original: Experimental results Table 15 summarizes the experimental results of each method on the Epileptic Seizure dataset.
Revised: Experimental results Table 15 summarizes the experimental results of each method on the Epileptic Seizure dataset.

Original: We can observe that PLAD signiﬁcantly outperforms most baseline methods, e.g., k -NN, AE, DAGMM and DSVDD, and is competitive against with DROCC.
Revised: We can observe that PLAD signiﬁcantly outperforms most baseline methods, e.g., k -NN, AE, DAGMM and DSVDD, and is competitive against with DROCC.

Original: The p -value v.s.
Revised: The p -value v.s.

Original: DROCC also shows the effectiveness of our PLAD.
Revised: DROCC also shows the effectiveness of our PLAD.

Original: Overall, the study demonstrates the potential and feasibility of PLAD to be extended to the time-series anomaly detection tasks.
Revised: Overall, the study demonstrates the potential and feasibility of PLAD to be extended to the time-series anomaly detection tasks.

Original: [Table]
Revised: [Table]