Original: Perturbation Learning Based Anomaly Detection
Revised: Perturbation Learning Based Anomaly Detection

Original: Abstract
Revised: Abstract

Original: This paper presents a simple yet effective method for anomaly detection.
Revised: This paper presents a simple yet effective method for anomaly detection.

Original: The mainidea is to learn small perturbations to perturb normal data and learn a classiﬁer toclassify the normal data and the perturbed data into two different classes.
Revised: The mainidea is to learn small perturbations to perturb normal data and learn a classiﬁer toclassify the normal data and the perturbed data into two different classes.

Original: The perturbator and classiﬁer are jointly learned using deep neural networks.
Revised: The perturbator and classiﬁer are jointly learned using deep neural networks.

Original: Importantly,the perturbations should be as small as possible but the classiﬁer is still able torecognize the perturbed data from unperturbed data.
Revised: Importantly,the perturbations should be as small as possible but the classiﬁer is still able torecognize the perturbed data from unperturbed data.

Original: Therefore, the perturbed dataare regarded as abnormal data and the classiﬁer provides a decision boundary between the normal data and abnormal data, although the training data do not includeany abnormal data.
Revised: Therefore, the perturbed dataare regarded as abnormal data and the classiﬁer provides a decision boundary between the normal data and abnormal data, although the training data do not includeany abnormal data.

Original: Compared with the state-of-the-art of anomaly detection, ourmethod does not require any assumption about the shape (e.g. hypersphere) of thedecision boundary and has fewer hyper-parameters to determine.
Revised: Compared with the state-of-the-art of anomaly detection, ourmethod does not require any assumption about the shape (e.g. hypersphere) of thedecision boundary and has fewer hyper-parameters to determine.

Original: Empirical studieson benchmark datasets verify the effectiveness and superiority of our method.
Revised: Empirical studieson benchmark datasets verify the effectiveness and superiority of our method.

Original:  Introduction
Revised:  Introduction

Original: Anomaly detection (AD) is an important research problem in many areas such computer vision,machine learning and chemical engineering [Chandola et al. , 2009; Ramachandra et al. , 2020; Panget al. , 2021; Ruff et al. , 2021].
Revised: Anomaly detection (AD) is an important research problem in many areas such computer vision,machine learning and chemical engineering [Chandola et al. , 2009; Ramachandra et al. , 2020; Panget al. , 2021; Ruff et al. , 2021].

Original: AD aims to identify abnormal data from normal data and is usually anunsupervised learning task because the anomaly samples are unknown in the training stage.
Revised: AD aims to identify abnormal data from normal data and is usually anunsupervised learning task because the anomaly samples are unknown in the training stage.

Original: In thepast decades, numerous AD methods [Schölkopf et al. , 1999, 2001; Breunig et al. , 2000; Liu et al. ,2008] have been proposed.
Revised: In thepast decades, numerous AD methods [Schölkopf et al. , 1999, 2001; Breunig et al. , 2000; Liu et al. ,2008] have been proposed.

Original: For instance, one-class support vector machine (OCSVM)
Revised: For instance, one-class support vector machine (OCSVM)

Original: [Schölkopf etal. , 2001] maps the data into high-dimensional feature space induced by kernels and tries to ﬁnd ahyperplane giving possibly maximal distance between the normal data and the origin.
Revised: [Schölkopf etal. , 2001] maps the data into high-dimensional feature space induced by kernels and tries to ﬁnd ahyperplane giving possibly maximal distance between the normal data and the origin.

Original: Tax and Duin[2004] proposed a method called support vector data description (SVDD), which ﬁnds the smallesthyper-sphere encasing the normal data in the high-dimensional feature space.
Revised: Tax and Duin[2004] proposed a method called support vector data description (SVDD), which ﬁnds the smallesthyper-sphere encasing the normal data in the high-dimensional feature space.

Original: SVDD is similar to
Revised: SVDD is similar to

Original: OCSVM with Gaussian kernel function.
Revised: OCSVM with Gaussian kernel function.

Original: Classical anomaly detection methods such as OCSVM and SVDD are generally not suitable forlarge-scale data due to the high computational costs, and are not effective to deal with more complexdata such as those in vision scenarios.
Revised: Classical anomaly detection methods such as OCSVM and SVDD are generally not suitable forlarge-scale data due to the high computational costs, and are not effective to deal with more complexdata such as those in vision scenarios.

Original: To address these issues, a few researchers Erfani et al.
Revised: To address these issues, a few researchers Erfani et al.

Original: Golan and El-Yaniv [2018]; Abati et al.
Revised: Golan and El-Yaniv [2018]; Abati et al.

Original: ; Wang et al.
Revised: ; Wang et al.

Original: [2019]; Qiu et al.
Revised: [2019]; Qiu et al.

Original: [2021] attempted to takeadvantages of deep learning [LeCun et al. , 2015; Goodfellow et al. , 2016] to improve the performanceof anomaly detection.
Revised: [2021] attempted to takeadvantages of deep learning [LeCun et al. , 2015; Goodfellow et al. , 2016] to improve the performanceof anomaly detection.

Original: One typical way is to use deep auto-encoder or its variants[Vincent et al. ,2008; Kingma and Welling, 2013; Pidhorskyi et al. , 2018; Wang et al. , 2021] to learn effectivedata representation or compression models.
Revised: One typical way is to use deep auto-encoder or its variants[Vincent et al. ,2008; Kingma and Welling, 2013; Pidhorskyi et al. , 2018; Wang et al. , 2021] to learn effectivedata representation or compression models.

Original: Auto-encoder and its variants have achieved promisingperformance in AD.
Revised: Auto-encoder and its variants have achieved promisingperformance in AD.

Original: In fact, these methods do not explicitly deﬁne an objective for anomaly detection.
Revised: In fact, these methods do not explicitly deﬁne an objective for anomaly detection.

Original: In stead, they usually use the data reconstruction error as a metric to detect anomaly.
Revised: In stead, they usually use the data reconstruction error as a metric to detect anomaly.

Original: There are a few approaches to building deep anomaly detection objectives or models.
Revised: There are a few approaches to building deep anomaly detection objectives or models.

Original: Typicalexamples include classical one-class learning based approach[Ruff et al. , 2018, 2019; Perera and
Revised: Typicalexamples include classical one-class learning based approach[Ruff et al. , 2018, 2019; Perera and

Original: Patel, 2019; Bhattacharya et al. , 2021], probability estimation based approach [Zong et al. , 2018;
Revised: Patel, 2019; Bhattacharya et al. , 2021], probability estimation based approach [Zong et al. , 2018;

Original: Pérez-Cabo et al. , 2019; Su et al. , 2019], and adversarial learning based approach [Deecke et al. , 2018;
Revised: Pérez-Cabo et al. , 2019; Su et al. , 2019], and adversarial learning based approach [Deecke et al. , 2018;

Original: Perera et al. , 2019; Raghuram et al. , 2021].
Revised: Perera et al. , 2019; Raghuram et al. , 2021].

Original: For example, Ruff et al.
Revised: For example, Ruff et al.

Original: [2018] proposed deep one-classclassiﬁcation (DSVDD), which applies deep neural network to learn an effective embedding for thenormal data such that in the embedding space, the normal data can be encased by a hyper-sphere withminimum radius.
Revised: [2018] proposed deep one-classclassiﬁcation (DSVDD), which applies deep neural network to learn an effective embedding for thenormal data such that in the embedding space, the normal data can be encased by a hyper-sphere withminimum radius.

Original: The deep autoencoding Gaussian mixture model (DAGMM) proposed by Zonget al.
Revised: The deep autoencoding Gaussian mixture model (DAGMM) proposed by Zonget al.

Original: [2018] is composed of a compression network and an estimation network based on Gaussianmixture model.
Revised: [2018] is composed of a compression network and an estimation network based on Gaussianmixture model.

Original: The anomaly scores are described as the output energy of the estimation network.
Revised: The anomaly scores are described as the output energy of the estimation network.

Original: Perera et al.
Revised: Perera et al.

Original: [2019] proposed one-class GAN (OCGAN) to learn a latent space to represent a speciﬁcclass by adversarially training the auto-encoder and discriminator.
Revised: [2019] proposed one-class GAN (OCGAN) to learn a latent space to represent a speciﬁcclass by adversarially training the auto-encoder and discriminator.

Original: The properly trained OCGANnetwork can well reconstruct the speciﬁc class of data, while failing to reconstruct other classesof data.
Revised: The properly trained OCGANnetwork can well reconstruct the speciﬁc class of data, while failing to reconstruct other classesof data.

Original: Moreover, some latest works also explore interesting perspectives.
Revised: Moreover, some latest works also explore interesting perspectives.

Original: Goyal et al.
Revised: Goyal et al.

Original: [2020]proposed the method called deep robust one-class classiﬁcation (DROCC).
Revised: [2020]proposed the method called deep robust one-class classiﬁcation (DROCC).

Original: DROCC assumes thatthe normal samples generally lie on low-dimensional manifolds, and regards the process of ﬁndingthe optimal hyper-sphere in the embedding space as an adversarial optimization problem.
Revised: DROCC assumes thatthe normal samples generally lie on low-dimensional manifolds, and regards the process of ﬁndingthe optimal hyper-sphere in the embedding space as an adversarial optimization problem.

Original: Yan etal.
Revised: Yan etal.

Original: [2021] claimed that anomalous domains generally exhibit different semantic patterns comparedwith the peripheral domains, and proposed the semantic context based anomaly detection network(SCADN) to learn the semantic context from the masked data via adversarial learning.
Revised: [2021] claimed that anomalous domains generally exhibit different semantic patterns comparedwith the peripheral domains, and proposed the semantic context based anomaly detection network(SCADN) to learn the semantic context from the masked data via adversarial learning.

Original: Chen et al.[2022] proposed the interpolated Gaussian descriptor (IGD) to learn more valid data descriptionsfrom representative normal samples rather than edge samples.
Revised: Chen et al.[2022] proposed the interpolated Gaussian descriptor (IGD) to learn more valid data descriptionsfrom representative normal samples rather than edge samples.

Original: Shenkar and Wolf [2022] utilizedcontrastive learning to construct the method called generic one-class classiﬁcation (GOCC) for ADon tabular data.
Revised: Shenkar and Wolf [2022] utilizedcontrastive learning to construct the method called generic one-class classiﬁcation (GOCC) for ADon tabular data.

Original: It is worth noting that classical AD methods such as OCSVM [Schölkopf et al. , 2001]and DSVDD [Ruff et al. , 2018] require speciﬁc assumptions (e.g. hypersphere) for the distribution orstructure of the normal data.
Revised: It is worth noting that classical AD methods such as OCSVM [Schölkopf et al. , 2001]and DSVDD [Ruff et al. , 2018] require speciﬁc assumptions (e.g. hypersphere) for the distribution orstructure of the normal data.

Original: The GAN-based approaches [Deecke et al. , 2018; Perera et al. , 2019]suffer from the instability problem of min-max optimization and have high computational costs.
Revised: The GAN-based approaches [Deecke et al. , 2018; Perera et al. , 2019]suffer from the instability problem of min-max optimization and have high computational costs.

Original: In this paper, we propose a novel AD method called perturbation learning based anomaly detection(PLAD).
Revised: In this paper, we propose a novel AD method called perturbation learning based anomaly detection(PLAD).

Original: PLAD aims to learn a perturbator and a classiﬁer from the normal training data.
Revised: PLAD aims to learn a perturbator and a classiﬁer from the normal training data.

Original: Theperturbator uses minimum effort to perturb the normal data to abnormal data while the classiﬁer isable to classify the normal data and perturbed data into two classes correctly.
Revised: Theperturbator uses minimum effort to perturb the normal data to abnormal data while the classiﬁer isable to classify the normal data and perturbed data into two classes correctly.

Original: The main contributionsof our work are summarized as follows:
Revised: The main contributionsof our work are summarized as follows:

Original: • We propose a novel AD method called PLAD.
Revised: • We propose a novel AD method called PLAD.

Original: PLAD does not require any assumption about theshape of the decision boundary between the normal data and abnormal data.
Revised: PLAD does not require any assumption about theshape of the decision boundary between the normal data and abnormal data.

Original: In addition, PLADhas much fewer hyper-parameters than many state-of-the-art AD methods such as [Wang et al. ,2019; Goyal et al. , 2020; Yan et al.
Revised: In addition, PLADhas much fewer hyper-parameters than many state-of-the-art AD methods such as [Wang et al. ,2019; Goyal et al. , 2020; Yan et al.

Original: We propose to learn perturbations directly from the normal training data.
Revised: We propose to learn perturbations directly from the normal training data.

Original: For every training datapoint, we learn a distribution from which any sample can lead to a perturbation such that the normaldata point is ﬂipped to an abnormal data point.• Besides the conventional empirical studies on one-class classiﬁcation, we investigate the performance of our PLAD and its competitors in recognizing abnormal data from multi-class normaldata.
Revised: For every training datapoint, we learn a distribution from which any sample can lead to a perturbation such that the normaldata point is ﬂipped to an abnormal data point.• Besides the conventional empirical studies on one-class classiﬁcation, we investigate the performance of our PLAD and its competitors in recognizing abnormal data from multi-class normaldata.

Original: These results show that our PLAD has state-of-the-art performance.
Revised: These results show that our PLAD has state-of-the-art performance.

Original:  Proposed method
Revised:  Proposed method

Original: Suppose we have a distribution D of d dimension and any data drawn from D are deemed as normaldata.
Revised: Suppose we have a distribution D of d dimension and any data drawn from D are deemed as normaldata.

Original: Now we have some training data X
Revised: Now we have some training data X

Original: = { x 1 , x 2 , . . .
Revised: = { x 1 , x 2 , . . .

Original: , x n } randomly drawn from D
Revised: , x n } randomly drawn from D

Original: and we wantto learn a discriminative function f from X such that f ( x ) > 0 for any x ∈ D and f ( x ) < 0 for anyx / ∈ D .
Revised: and we wantto learn a discriminative function f from X such that f ( x ) > 0 for any x ∈ D and f ( x ) < 0 for anyx / ∈ D .

Original: This is an unsupervised learning problem and also known as anomaly detection, where anyx / ∈ D are deemed as abnormal data.
Revised: This is an unsupervised learning problem and also known as anomaly detection, where anyx / ∈ D are deemed as abnormal data.

Original: In contrast to classical anomaly detection methods such as one-class SVM [Schölkopf et al. , 2001],deep SVDD [Ruff et al. , 2018] , and DROCC [Goyal et al. , 2020], in this paper, we do not make anyassumption about the distribution D .
Revised: In contrast to classical anomaly detection methods such as one-class SVM [Schölkopf et al. , 2001],deep SVDD [Ruff et al. , 2018] , and DROCC [Goyal et al. , 2020], in this paper, we do not make anyassumption about the distribution D .

Original: Inspired from the generative adversarial learning [Goodfellowet al. , 2014], we propose to learn a discriminator that is able to recognize fake data (abnormal data)and a generator that is able to fool the discriminator, but the input of the generator is real data (normaldata), which is essentially different from GAN.
Revised: Inspired from the generative adversarial learning [Goodfellowet al. , 2014], we propose to learn a discriminator that is able to recognize fake data (abnormal data)and a generator that is able to fool the discriminator, but the input of the generator is real data (normaldata), which is essentially different from GAN.

Original: To be more precise, we propose to learn perturbationsfor X such that the perturbed X (denoted by (cid:101)
Revised: To be more precise, we propose to learn perturbationsfor X such that the perturbed X (denoted by (cid:101)

Original: X
Revised: X

Original: = { ˜ x 1 , ˜ x 2 , . . .
Revised: = { ˜ x 1 , ˜ x 2 , . . .

Original: , ˜ x n } ) are abnormal but quite close to
Revised: , ˜ x n } ) are abnormal but quite close to

Original: X .
Revised: X .

Original: To ensure the abnormality of (cid:101) X , we learn a classiﬁer f from { X , (cid:101) X } such that f ( x ) > 0 for anyx ∈ X and f (˜ x ) < 0 for any ˜ x ∈ (cid:101) X .
Revised: To ensure the abnormality of (cid:101) X , we learn a classiﬁer f from { X , (cid:101) X } such that f ( x ) > 0 for anyx ∈ X and f (˜ x ) < 0 for any ˜ x ∈ (cid:101) X .

Original: To ensure that (cid:101) X is close to X , the perturbations should be small enough.
Revised: To ensure that (cid:101) X is close to X , the perturbations should be small enough.

Original: Therefore, we propose to solve the following problem
Revised: Therefore, we propose to solve the following problem

Original: [Equation]
Revised: [Equation]

Original: where (cid:96) ( · , · ) denotes some loss function such as cross-entropy and y 1 = · · · = y n = 0 and˜
Revised: where (cid:96) ( · , · ) denotes some loss function such as cross-entropy and y 1 = · · · = y n = 0 and˜

Original: y 1 = · · · = ˜ y n = 1 are the labels for the normal data and perturbed data respectively.
Revised: y 1 = · · · = ˜ y n = 1 are the labels for the normal data and perturbed data respectively.

Original: λ is ahyperparameter to control the magnitudes of the perturbations.
Revised: λ is ahyperparameter to control the magnitudes of the perturbations.

Original: θ denotes the set of parameters of theclassiﬁer f and φ ( · , · ) is some distance metric quantifying the difference between two data points.
Revised: θ denotes the set of parameters of theclassiﬁer f and φ ( · , · ) is some distance metric quantifying the difference between two data points.

Original: However, directly solving (1) encounters the following difﬁculties.
Revised: However, directly solving (1) encounters the following difﬁculties.

Original: • First, the number ( | θ | + dn ) of decision variables to optimize can be huge if n is large, where | θ |denotes the cardinality of the set θ .
Revised: • First, the number ( | θ | + dn ) of decision variables to optimize can be huge if n is large, where | θ |denotes the cardinality of the set θ .

Original: • Second, it is hard to use mini-batch optimization because some decision variables (i
Revised: • Second, it is hard to use mini-batch optimization because some decision variables (i

Original: e. (cid:101) X ) areassociated with the sample indices.
Revised: e. (cid:101) X ) areassociated with the sample indices.

Original: • Lastly, it is not easy to determine φ because φ relies on the unknown distribution D .
Revised: • Lastly, it is not easy to determine φ because φ relies on the unknown distribution D .

Original: For instance,φ ( x , ˜ x ) = (cid:107) x − ˜ x
Revised: For instance,φ ( x , ˜ x ) = (cid:107) x − ˜ x

Original: (cid:107) 2 , namely the squared Euclidean norm, does not work if X is enclosed by ahypersphere or hypercube (data points close to the centroid require much larger perturbations than
Revised: (cid:107) 2 , namely the squared Euclidean norm, does not work if X is enclosed by ahypersphere or hypercube (data points close to the centroid require much larger perturbations than

Original: those far away from the centroid, which implies that x − ˜ x has a non-Gaussian distribution).
Revised: those far away from the centroid, which implies that x − ˜ x has a non-Gaussian distribution).

Original: To overcome these three difﬁculties, we propose to solve the following problem instead
Revised: To overcome these three difﬁculties, we propose to solve the following problem instead

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: where 1 = [1 , 1 , . . . , 1] (cid:62) and 0 =
Revised: where 1 = [1 , 1 , . . . , 1] (cid:62) and 0 =

Original: , 0] (cid:62) are d -dimensional constant vectors and (cid:12) denotesthe Hadamard product of two vectors.
Revised: , 0] (cid:62) are d -dimensional constant vectors and (cid:12) denotesthe Hadamard product of two vectors.

Original: α i and β i are multiplicative and additive perturbations for x iand they are generated from a perturbator g ˜ θ , where ˜ θ denotes the set of parameters to learn.
Revised: α i and β i are multiplicative and additive perturbations for x iand they are generated from a perturbator g ˜ θ , where ˜ θ denotes the set of parameters to learn.

Original: In (2),we hope that the multiplicative perturbation is close to 1 and the additive perturbation is close to zerobut they relies on the data point x .
Revised: In (2),we hope that the multiplicative perturbation is close to 1 and the additive perturbation is close to zerobut they relies on the data point x .

Original: Therefore, the perturbation learning is adaptive to the unknowndistribution D , which solves the third aforementioned difﬁculty.
Revised: Therefore, the perturbation learning is adaptive to the unknowndistribution D , which solves the third aforementioned difﬁculty.

Original: In fact, problem (2) can be reformulated asminimize
Revised: In fact, problem (2) can be reformulated asminimize

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: g
Revised: g

Original: [Equation]
Revised: [Equation]

Original: = g
Revised: = g

Original: [Equation]
Revised: [Equation]

Original: x i ) , i = 1 , 2 , . . . , n .
Revised: x i ) , i = 1 , 2 , . . . , n .

Original: We see that we only need to optimize the parametersθ and ˜ θ
Revised: We see that we only need to optimize the parametersθ and ˜ θ

Original: and the total number of decision variables is | θ | + | ˜ θ | , which solved the ﬁrst difﬁculty wediscussed previously.
Revised: and the total number of decision variables is | θ | + | ˜ θ | , which solved the ﬁrst difﬁculty wediscussed previously.

Original: On the other hand, θ and ˜ θ are not associated with the sample indices, whichsolved the second difﬁculty.
Revised: On the other hand, θ and ˜ θ are not associated with the sample indices, whichsolved the second difﬁculty.

Original: Once f θ and g ˜ θ are learned, we can then use f θ to detect whether anew data point x new is normal (e.g. f θ ( x new ) < 0 . 5 ) or abnormal (e.g. f θ ( x new ) > 0 . 5 ).
Revised: Once f θ and g ˜ θ are learned, we can then use f θ to detect whether anew data point x new is normal (e.g. f θ ( x new ) < 0 . 5 ) or abnormal (e.g. f θ ( x new ) > 0 . 5 ).

Original: We call themethod Perturbation Learning based Anomaly Detection (PLAD).
Revised: We call themethod Perturbation Learning based Anomaly Detection (PLAD).

Original: In PLAD, namely (3), both the classiﬁer f θ and the perturbator g ˜ θ are neural networks.
Revised: In PLAD, namely (3), both the classiﬁer f θ and the perturbator g ˜ θ are neural networks.

Original: They can befully connected neural networks, convolutional neural networks (for image data anomaly detection),or recurrent neural networks (for sequential data anomaly detection).
Revised: They can befully connected neural networks, convolutional neural networks (for image data anomaly detection),or recurrent neural networks (for sequential data anomaly detection).

Original: Figure 1 shows the motivationof our PLAD and the network architecture.
Revised: Figure 1 shows the motivationof our PLAD and the network architecture.

Original: Compared with many popular anomaly detection methods,our PLAD has the following characteristics.
Revised: Compared with many popular anomaly detection methods,our PLAD has the following characteristics.

Original: • PLAD does not make any assumption about the distribution or structure of the normal data.
Revised: • PLAD does not make any assumption about the distribution or structure of the normal data.

Original: In
Revised: In

Original: contrast, one-class SVM [Schölkopf et al. , 2001], deep SVDD [Ruff et al. , 2018], and DROC-
Revised: contrast, one-class SVM [Schölkopf et al. , 2001], deep SVDD [Ruff et al. , 2018], and DROC-

Original: C [Goyal et al. , 2020] make speciﬁc assumptions about the distribution of the normal data, which
Revised: C [Goyal et al. , 2020] make speciﬁc assumptions about the distribution of the normal data, which

Original: may be violated in real applications.
Revised: may be violated in real applications.

Original: For example, deep SVDD assumes that the normal data are
Revised: For example, deep SVDD assumes that the normal data are

Original: encased by a hypersphere, which is hard to guarantee in real applications and may require a very
Revised: encased by a hypersphere, which is hard to guarantee in real applications and may require a very

Original: deep neural network to transform a non-hypersphere structure to a hypersphere structure.
Revised: deep neural network to transform a non-hypersphere structure to a hypersphere structure.

Original: [Figure]
Revised: [Figure]

Original: 𝛼 𝛽
Revised: 𝛼 𝛽

Original: Classifier
Revised: Classifier

Original: [Figure]
Revised: [Figure]

Original: Perturbation
Revised: Perturbation

Original: MultiplicativePerturbation
Revised: MultiplicativePerturbation

Original: PerturbedSample 𝐱 ෤𝐱 0/
Revised: PerturbedSample 𝐱 ෤𝐱 0/

Original: • In PLAD, besides the network structures, we only need to determine one hyperparameter λ , which
Revised: • In PLAD, besides the network structures, we only need to determine one hyperparameter λ , which

Original: provides huge convenience in real applications.
Revised: provides huge convenience in real applications.

Original: In contrast, many state-of-the-art AD methods such
Revised: In contrast, many state-of-the-art AD methods such

Original: as [Wang et al. , 2019; Goyal
Revised: as [Wang et al. , 2019; Goyal

Original: et al. ,
Revised: et al. ,

Original: 2020; Yan et
Revised: 2020; Yan et

Original: al.
Revised: al.

Original: have at least two key hyperparameters
Revised: have at least two key hyperparameters

Original: For example,
Revised: For example,

Original: in DROCC
Revised: in DROCC

Original: [Goyal et al.
Revised: [Goyal et al.

Original: , one has to determine the radius of hyper-sphere
Revised: , one has to determine the radius of hyper-sphere

Original: the
Revised: the

Original: step
Revised: step

Original: size of gradient-ascent
Revised: size of gradient-ascent

Original: and two regularization parameters
Revised: and two regularization parameters

Original: • In PLAD, we can use gradient based optimizer such as Adam to solve the optimization.
Revised: • In PLAD, we can use gradient based optimizer such as Adam to solve the optimization.

Original: The
Revised: The

Original: time complexity is comparable to that of vanilla deep neural networks (for classiﬁcation or
Revised: time complexity is comparable to that of vanilla deep neural networks (for classiﬁcation or

Original: representation).
Revised: representation).

Original: On the contrary, many recent advances
Revised: On the contrary, many recent advances

Original: DROCC [Goyal et al. ,
Revised: DROCC [Goyal et al. ,

Original: ] of anomaly
Revised: ] of anomaly

Original: detection especially those
Revised: detection especially those

Original: GAN based
Revised: GAN based

Original: methods
Revised: methods

Original: [Deecke et al. ,
Revised: [Deecke et al. ,

Original: ; Perera
Revised: ; Perera

Original: et al. , 2019; Yan et al. ,
Revised: et al. , 2019; Yan et al. ,

Original: 2021] have much higher computational cost
Revised: 2021] have much higher computational cost

Original: In the left of Figure 1, actually, a normal training data can be perturbed to an abnormal data bydifferent perturbations.
Revised: In the left of Figure 1, actually, a normal training data can be perturbed to an abnormal data bydifferent perturbations.

Original: Therefore, we propose to learn a distribution from which any perturbations canperturb the normal training data to be abnormal.
Revised: Therefore, we propose to learn a distribution from which any perturbations canperturb the normal training data to be abnormal.

Original: Speciﬁcally, for every x i , there exists a distributionΩ i , such that for any z ∈ Ω i , the perturbation given by ( α , β )
Revised: Speciﬁcally, for every x i , there exists a distributionΩ i , such that for any z ∈ Ω i , the perturbation given by ( α , β )

Original: =
Revised: =

Original: h ( z ) can perturb x i to be abnormal,where h is a nonlinear function modelled by a neural network.
Revised: h ( z ) can perturb x i to be abnormal,where h is a nonlinear function modelled by a neural network.

Original: We can just assume that Ω i is a
Revised: We can just assume that Ω i is a

Original: Gaussian distribution with mean µ i and variance σ 2 i , i.e., Ω i = N ( µ i , σ 2 i ) , because of the universalapproximation ability of h .
Revised: Gaussian distribution with mean µ i and variance σ 2 i , i.e., Ω i = N ( µ i , σ 2 i ) , because of the universalapproximation ability of h .

Original: We take the idea of variational autoencoder (VAE)
Revised: We take the idea of variational autoencoder (VAE)

Original: [Kingma and Welling,2013] and minimize
Revised: [Kingma and Welling,2013] and minimize

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where p ( z ) = N ( 0 , 1 ) , ε = (cid:20) 10 (cid:21) , z = l ˜ θ 1 ( x ) , ( α , β ) =
Revised: where p ( z ) = N ( 0 , 1 ) , ε = (cid:20) 10 (cid:21) , z = l ˜ θ 1 ( x ) , ( α , β ) =

Original: h ˜ θ 2 ( z ) ,
Revised: h ˜ θ 2 ( z ) ,

Original: ˜ θ 1 are the parameters of the encoderl , and ˜ θ 2 are the parameters of the decoder h .
Revised: ˜ θ 1 are the parameters of the encoderl , and ˜ θ 2 are the parameters of the decoder h .

Original: Combing (4) with (3), we have ˜ θ
Revised: Combing (4) with (3), we have ˜ θ

Original: =
Revised: =

Original: { ˜ θ 1 , ˜ θ 2 } and solveminimize θ,
Revised: { ˜ θ 1 , ˜ θ 2 } and solveminimize θ,

Original: [Equation]
Revised: [Equation]

Original: ) where g ˜ θ = h ˜ θ 2 ◦ l ˜ θ 1 .
Revised: ) where g ˜ θ = h ˜ θ 2 ◦ l ˜ θ 1 .

Original: The training is similar to that for VAE [Kingma and Welling, 2013] and willnot be detailed here.
Revised: The training is similar to that for VAE [Kingma and Welling, 2013] and willnot be detailed here.

Original: It should be pointed out that the method (5) is just an extension of the method (3).
Revised: It should be pointed out that the method (5) is just an extension of the method (3).

Original: In (5), we wantto learn a distribution for each normal training data point x i such that any perturbations generatedfrom the distribution can perturb x i to be abnormal.
Revised: In (5), we wantto learn a distribution for each normal training data point x i such that any perturbations generatedfrom the distribution can perturb x i to be abnormal.

Original: It is expected that (5) can outperform
Revised: It is expected that (5) can outperform

Original:  Connection with previous works
Revised:  Connection with previous works

Original: The well-known one-class classiﬁcation methods such as OCSVM [Schölkopf et al. , 2001], DSVDD [Ruff et al. , 2018] and DROCC [Goyal et al. , 2020] have speciﬁc assumptions for the embedded distribution while our PLAD does not require any assumption and is able to adaptively learn a decisionboundary even if it is very complex.
Revised: The well-known one-class classiﬁcation methods such as OCSVM [Schölkopf et al. , 2001], DSVDD [Ruff et al. , 2018] and DROCC [Goyal et al. , 2020] have speciﬁc assumptions for the embedded distribution while our PLAD does not require any assumption and is able to adaptively learn a decisionboundary even if it is very complex.

Original: It is also noteworthy that the idea of DROCC in identifyinganomalies is similar to ours, i.e, training a classiﬁer instead of an auto-encoder or embedding model.
Revised: It is also noteworthy that the idea of DROCC in identifyinganomalies is similar to ours, i.e, training a classiﬁer instead of an auto-encoder or embedding model.

Original: Self-supervised learning based methods, e.g., GOAD [Bergman and Hoshen, 2020] NeuTraL AD [Qiuet al. , 2021], etc., work through designing a appropriate auxiliary task to help the model to learnuseful data features.
Revised: Self-supervised learning based methods, e.g., GOAD [Bergman and Hoshen, 2020] NeuTraL AD [Qiuet al. , 2021], etc., work through designing a appropriate auxiliary task to help the model to learnuseful data features.

Original: Both our approach and the self-supervised learning based methods aim to learna binary classiﬁer on the original data and auxiliary data.
Revised: Both our approach and the self-supervised learning based methods aim to learna binary classiﬁer on the original data and auxiliary data.

Original: But they are different in the following twopoints.
Revised: But they are different in the following twopoints.

Original: First, in self-supervised learning (e.g. contrastive learning) based methods, the auxiliary dataare generated by some pre-deﬁned and manual operations (e.g. image rotation and cropping).
Revised: First, in self-supervised learning (e.g. contrastive learning) based methods, the auxiliary dataare generated by some pre-deﬁned and manual operations (e.g. image rotation and cropping).

Original: Incontrast, in our approach, the auxiliary data are adaptively and dynamically learned from the originaldata.
Revised: Incontrast, in our approach, the auxiliary data are adaptively and dynamically learned from the originaldata.

Original: Second, self-supervised learning based methods are often designed for some speciﬁc types ofdata such image data, whereas it may be difﬁcult to generate auxiliary data for other types of data.
Revised: Second, self-supervised learning based methods are often designed for some speciﬁc types ofdata such image data, whereas it may be difﬁcult to generate auxiliary data for other types of data.

Original: Incontrast, it is easy to apply our approach to any types of data.
Revised: Incontrast, it is easy to apply our approach to any types of data.

Original: Adversarial learning based methods [Malhotra et al. , 2016; Deecke et al. , 2018; Pidhorskyi et al. ,2018; Perera et al. , 2019] are generally constructed with auto-encoder and generative adversarialnetworks (GANs)
Revised: Adversarial learning based methods [Malhotra et al. , 2016; Deecke et al. , 2018; Pidhorskyi et al. ,2018; Perera et al. , 2019] are generally constructed with auto-encoder and generative adversarialnetworks (GANs)

Original: [Goodfellow et al. , 2014], and the most widely used measure of them to detectanomalies is the reconstruction error.
Revised: [Goodfellow et al. , 2014], and the most widely used measure of them to detectanomalies is the reconstruction error.

Original: Compared with them, we exploit the idea of VAE whenproducing perturbations and the detection metric is a classiﬁer, which should be more suitable thanreconstruction error for anomaly detection.
Revised: Compared with them, we exploit the idea of VAE whenproducing perturbations and the detection metric is a classiﬁer, which should be more suitable thanreconstruction error for anomaly detection.

Original: On the other hand, in these adversarial learning based ADmethods, the min-max optimization leads to instabilities in detecting anomaly, while the optimizationof PLAD is much easier to solve.
Revised: On the other hand, in these adversarial learning based ADmethods, the min-max optimization leads to instabilities in detecting anomaly, while the optimizationof PLAD is much easier to solve.

Original: Another interesting work SCADN [Yan et al. , 2021] tries to producenegative samples by multi-scale striped masks to train a GAN, but its anomaly score still relies onreconstruction error and the production of masks has randomness or may be hard to determine invarious real scenarios.
Revised: Another interesting work SCADN [Yan et al. , 2021] tries to producenegative samples by multi-scale striped masks to train a GAN, but its anomaly score still relies onreconstruction error and the production of masks has randomness or may be hard to determine invarious real scenarios.

Original: Our PLAD learns perturbations adaptively from the data itself, which isconvenience and reliable.
Revised: Our PLAD learns perturbations adaptively from the data itself, which isconvenience and reliable.

Original:  Experiment
Revised:  Experiment

Original: In this section, we evaluate the proposed method in comparison to several state-of-the-art anomalydetection methods on two image datasets and two tabular datasets.
Revised: In this section, we evaluate the proposed method in comparison to several state-of-the-art anomalydetection methods on two image datasets and two tabular datasets.

Original: Note that all the compared methodsdo not utilize any pre-trained feature extractors.
Revised: Note that all the compared methodsdo not utilize any pre-trained feature extractors.

Original: 4.1 Datasets and baseline methods
Revised: 4.1 Datasets and baseline methods

Original: [Table]
Revised: [Table]

Original: • CIFAR-10:
Revised: • CIFAR-10:

Original: CIFAR-10 image dataset is composed of 60,000 images in total, where 50,000 samplesfor training and 10,000 samples for test.
Revised: CIFAR-10 image dataset is composed of 60,000 images in total, where 50,000 samplesfor training and 10,000 samples for test.

Original: It includes 10 different balanced classes.• Fashion-MNIST:
Revised: It includes 10 different balanced classes.• Fashion-MNIST:

Original: Fashion MNIST contains 10 different categories of grey-scale fashion styleobjects.
Revised: Fashion MNIST contains 10 different categories of grey-scale fashion styleobjects.

Original: The data is split into 60,000 images for training and 10,000 images for test.• Thyroid: Thyroid is a hypothyroid disease dataset that contains 3,772 samples with 3 classes and6 attributes.
Revised: The data is split into 60,000 images for training and 10,000 images for test.• Thyroid: Thyroid is a hypothyroid disease dataset that contains 3,772 samples with 3 classes and6 attributes.

Original: We follow the data split settings of [Zong et al. , 2018] to preprocess the data forone-class classiﬁcation task.
Revised: We follow the data split settings of [Zong et al. , 2018] to preprocess the data forone-class classiﬁcation task.

Original: [Table]
Revised: [Table]

Original: • Arrhythmia:
Revised: • Arrhythmia:

Original: Arrhythmia dataset consists of 452 samples with 274 attributes.
Revised: Arrhythmia dataset consists of 452 samples with 274 attributes.

Original: Here we also followthe data split settings of [Zong et al. , 2018] to preprocess the data.
Revised: Here we also followthe data split settings of [Zong et al. , 2018] to preprocess the data.

Original: The detailed information of each dataset is illustrated in Table 1.
Revised: The detailed information of each dataset is illustrated in Table 1.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Baselines and state-of-the-arts.
Revised: Baselines and state-of-the-arts.

Original: We compare our method with the following classical baselinemethods and state-of-the-art methods: OCSVM [Schölkopf et al. , 2001], isolation forest (IF)
Revised: We compare our method with the following classical baselinemethods and state-of-the-art methods: OCSVM [Schölkopf et al. , 2001], isolation forest (IF)

Original: [Liu etal. , 2008], local outlier factor (LOF)
Revised: [Liu etal. , 2008], local outlier factor (LOF)

Original: [Breunig et al. , 2000], denoising auto-encoder (DAE)[Vincent etal. , 2008], E2E-AE and DAGMM [Zong et al. , 2018], DCN [Caron et al. , 2018], ADGAN [Deeckeet al. , 2018], DSVDD [Ruff et al. , 2018], OCGAN [Perera et al. , 2019], TQM [Wang et al. , 2019],
Revised: [Breunig et al. , 2000], denoising auto-encoder (DAE)[Vincent etal. , 2008], E2E-AE and DAGMM [Zong et al. , 2018], DCN [Caron et al. , 2018], ADGAN [Deeckeet al. , 2018], DSVDD [Ruff et al. , 2018], OCGAN [Perera et al. , 2019], TQM [Wang et al. , 2019],

Original: GOAD [Bergman and Hoshen, 2020], DROCC [Goyal et al. , 2020], HRN-L2 and HRN [Hu et al. ,2020], SCADN [Yan et al. , 2021], IGD (Scratch)
Revised: GOAD [Bergman and Hoshen, 2020], DROCC [Goyal et al. , 2020], HRN-L2 and HRN [Hu et al. ,2020], SCADN [Yan et al. , 2021], IGD (Scratch)

Original: [Chen et al. , 2022], NeuTraL AD [Qiu et al. , 2021],and GOCC
Revised: [Chen et al. , 2022], NeuTraL AD [Qiu et al. , 2021],and GOCC

Original: [Shenkar and Wolf, 2022].
Revised: [Shenkar and Wolf, 2022].

Original: 4.2 Implementation details and evaluation metrics
Revised: 4.2 Implementation details and evaluation metrics

Original: In this section, we ﬁrst describe the implementation details of the proposed PLAD method.
Revised: In this section, we ﬁrst describe the implementation details of the proposed PLAD method.

Original: Thesettings for image and tabular datasets are illustrated as follows:
Revised: Thesettings for image and tabular datasets are illustrated as follows:

Original: • Image datasets.
Revised: • Image datasets.

Original: For image datasets (CIFAR-10 and Fashion-MNIST), we utilize the LeNet-basedCNN to construct the classiﬁer, which is same as [Ruff et al. , 2018] and [Goyal et al. , 2020] toprovide fair comparison.
Revised: For image datasets (CIFAR-10 and Fashion-MNIST), we utilize the LeNet-basedCNN to construct the classiﬁer, which is same as [Ruff et al. , 2018] and [Goyal et al. , 2020] toprovide fair comparison.

Original: And we apply the MLP-based VAE to learn the noise for data.
Revised: And we apply the MLP-based VAE to learn the noise for data.

Original: Sinceboth image datasets contains 10 different classes, it can be regard as 10 independent one-classclassiﬁcation tasks, and each task on CIFAR-10 have 5,000 training samples (6,000 for FashionMNIST) and 10,000 testing samples for both of them.
Revised: Sinceboth image datasets contains 10 different classes, it can be regard as 10 independent one-classclassiﬁcation tasks, and each task on CIFAR-10 have 5,000 training samples (6,000 for FashionMNIST) and 10,000 testing samples for both of them.

Original: Consequently, the choice of optimizer (fromAdam [Kingma and Ba, 2015] and SGD), learning rate and hyper-parameter λ could be varies
Revised: Consequently, the choice of optimizer (fromAdam [Kingma and Ba, 2015] and SGD), learning rate and hyper-parameter λ could be varies

Original: for different classes.
Revised: for different classes.

Original: The suggested settings of them on each experiment in this paper refer to the
Revised: The suggested settings of them on each experiment in this paper refer to the

Original: supplementary material.
Revised: supplementary material.

Original: • Tabular datasets.
Revised: • Tabular datasets.

Original: For tabular datasets (Thyroid and Arrhythmia), we both use the MLP-basedclassiﬁer and VAE in practice, and we uniformly train them with Adam optimizer and learning rateof 0.001.
Revised: For tabular datasets (Thyroid and Arrhythmia), we both use the MLP-basedclassiﬁer and VAE in practice, and we uniformly train them with Adam optimizer and learning rateof 0.001.

Original: Besides, λ is set to 3 for Thyroid and 2 for Arrhythmia.
Revised: Besides, λ is set to 3 for Thyroid and 2 for Arrhythmia.

Original: For the competitive methods in the experiment, we report their performance directly from thefollowing paper
Revised: For the competitive methods in the experiment, we report their performance directly from thefollowing paper

Original: [Hu et al. , 2020; Goyal et al. , 2020; Yan et al. , 2021; Qiu et al. , 2021; Chen et al. ,2022; Shenkar and Wolf, 2022] except for DROCC, which we run the ofﬁcial released code to obtainthe results.
Revised: [Hu et al. , 2020; Goyal et al. , 2020; Yan et al. , 2021; Qiu et al. , 2021; Chen et al. ,2022; Shenkar and Wolf, 2022] except for DROCC, which we run the ofﬁcial released code to obtainthe results.

Original: Due to the limitation of paper length, the details of the our network settings are providedin the supplementary material.
Revised: Due to the limitation of paper length, the details of the our network settings are providedin the supplementary material.

Original: For the selection of evaluation metrics, we follow the previous works such as [Ruff et al. , 2018]and [Zong et al. , 2018] to use AUC (Area Under the ROC curve) for image datasets and F1-score fortabular datasets because the anomaly detection for image and tabular datasets has different evaluationcriteria.
Revised: For the selection of evaluation metrics, we follow the previous works such as [Ruff et al. , 2018]and [Zong et al. , 2018] to use AUC (Area Under the ROC curve) for image datasets and F1-score fortabular datasets because the anomaly detection for image and tabular datasets has different evaluationcriteria.

Original: Moreover, our method does not need pre-training like [Ruff et al. , 2018] and others did, sowe uniformly train the proposed method 5 times with 100 epochs to obtain the average performanceand standard deviation.
Revised: Moreover, our method does not need pre-training like [Ruff et al. , 2018] and others did, sowe uniformly train the proposed method 5 times with 100 epochs to obtain the average performanceand standard deviation.

Original: Note that we run all experiments on NVIDIA
Revised: Note that we run all experiments on NVIDIA

Original: RTX

Revised: RTX

Original: GPU with 32GB
Revised: GPU with 32GB

Original: RAM, CUDA 11.0 and cuDNN 8.0.
Revised: RAM, CUDA 11.0 and cuDNN 8.0.

Original: 4.3 Experimental results
Revised: 4.3 Experimental results

Original: 4.3.1 Experiment on image datasets
Revised: 4.3.1 Experiment on image datasets

Original: Table 2 and Table 3 summarize the average AUCs performance of the one-class classiﬁcation taskson CIFAR-10 and Fashion-MNIST, where we have the following observations:
Revised: Table 2 and Table 3 summarize the average AUCs performance of the one-class classiﬁcation taskson CIFAR-10 and Fashion-MNIST, where we have the following observations:

Original: • Compared with some classical shallow model-based approaches such OCSVM and IF, the proposedPLAD method signiﬁcantly outperforms them on each one-class classiﬁcation task with a largemargin.
Revised: • Compared with some classical shallow model-based approaches such OCSVM and IF, the proposedPLAD method signiﬁcantly outperforms them on each one-class classiﬁcation task with a largemargin.

Original: This is mainly due to the powerful feature learning capability of deep neural network.• PLAD also explicitly outperforms several well-known deep anomaly detection methods suchas DAGMM and DSVDD, and consistently obtains the top two AUC scores on most classes ofCIFAR-10 and Fashion-MNIST compared to some latest methods such as TQM, DROCC andHRN.
Revised: This is mainly due to the powerful feature learning capability of deep neural network.• PLAD also explicitly outperforms several well-known deep anomaly detection methods suchas DAGMM and DSVDD, and consistently obtains the top two AUC scores on most classes ofCIFAR-10 and Fashion-MNIST compared to some latest methods such as TQM, DROCC andHRN.

Original: Speciﬁcally, in the class “Automobile” of CIFAR-10 and class “Sandal” of Fashion-MNIST,the proposed method exceeds 5.9% and 4.7% in terms of AUC compared to the runner-up.
Revised: Speciﬁcally, in the class “Automobile” of CIFAR-10 and class “Sandal” of Fashion-MNIST,the proposed method exceeds 5.9% and 4.7% in terms of AUC compared to the runner-up.

Original: Wealso conduct statistical analysis for our method against other competitors, the detailed analysisrefers to the supplementary material due to the page
Revised: Wealso conduct statistical analysis for our method against other competitors, the detailed analysisrefers to the supplementary material due to the page

Original: limitations.•
Revised: limitations.•

Original: It is noteworthy that some deep anomaly detection methods such as DSVDD and DROCC, aremainly based on the assumption that the normal data in the embedding space are situated in a hyper-
Revised: It is noteworthy that some deep anomaly detection methods such as DSVDD and DROCC, aremainly based on the assumption that the normal data in the embedding space are situated in a hyper-

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: sphere, while anomalies are outside the sphere.
Revised: sphere, while anomalies are outside the sphere.

Original: Therefore the edge of the hyper-sphere is then thedecision boundary learned by the model to identify anomalies.
Revised: Therefore the edge of the hyper-sphere is then thedecision boundary learned by the model to identify anomalies.

Original: In contrast
Revised: In contrast

Original: [Table]
Revised: [Table]

Original: PLAD does not requireany assumption about the shape of the decision boundary.
Revised: PLAD does not requireany assumption about the shape of the decision boundary.

Original: It attempts to learn the perturbation fromdata itself by neural network and construct the anomalies by enforcing the perturbation to originaldata, then train network to distinguish the normal samples and anomalies.
Revised: It attempts to learn the perturbation fromdata itself by neural network and construct the anomalies by enforcing the perturbation to originaldata, then train network to distinguish the normal samples and anomalies.

Original: Of course, this is alsonatural to associate PLAD with some adversarial learning based methods such as ADGAN and
Revised: Of course, this is alsonatural to associate PLAD with some adversarial learning based methods such as ADGAN and

Original: OCGAN, etc.
Revised: OCGAN, etc.

Original: In contrast to them, the optimization of PLAD is a non-adversary problem, becauseit tries to simultaneously minimize the perturbation to produce the anomalies that similar to normalones and minimize the cross-entropy to distinguish them.
Revised: In contrast to them, the optimization of PLAD is a non-adversary problem, becauseit tries to simultaneously minimize the perturbation to produce the anomalies that similar to normalones and minimize the cross-entropy to distinguish them.

Original: Moreover, Table 4 also shows the average performance on CIFAR-10 and Fashion-MNIST over all10 classes to provide an overall comparison.
Revised: Moreover, Table 4 also shows the average performance on CIFAR-10 and Fashion-MNIST over all10 classes to provide an overall comparison.

Original: Note that we further compare with two latest methods
Revised: Note that we further compare with two latest methods

Original: SCADN and IGD (Scratch).
Revised: SCADN and IGD (Scratch).

Original: They do not included in the above two tables because the performanceon each class is not provided in the paper.
Revised: They do not included in the above two tables because the performanceon each class is not provided in the paper.

Original: From this table we can observe that PLAD achieves thebest average AUCs on both datasets among all competitive methods.
Revised: From this table we can observe that PLAD achieves thebest average AUCs on both datasets among all competitive methods.

Original: We also apply t-SNE [Van der Maaten and Hinton, 2008] to visualize the learned embedded spacein Figure 2.
Revised: We also apply t-SNE [Van der Maaten and Hinton, 2008] to visualize the learned embedded spacein Figure 2.

Original: Speciﬁcally, we visualize the training samples of a speciﬁc category, perturbed datafor training samples, as well as all test samples.
Revised: Speciﬁcally, we visualize the training samples of a speciﬁc category, perturbed datafor training samples, as well as all test samples.

Original: They are divided into three sub-ﬁgures for ease
Revised: They are divided into three sub-ﬁgures for ease

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: of understanding.
Revised: of understanding.

Original: Figure 2 shows the visualization of the the class “Sneaker” in Fashion-MNIST.
Revised: Figure 2 shows the visualization of the the class “Sneaker” in Fashion-MNIST.

Original: More results can be found on the supplementary material.
Revised: More results can be found on the supplementary material.

Original: From this ﬁgure we have the followingobservations.
Revised: From this ﬁgure we have the followingobservations.

Original: First, PLAD obviously learns good decision boundary for distinguishing normal dataand perturbed data from Figure 2(a).
Revised: First, PLAD obviously learns good decision boundary for distinguishing normal dataand perturbed data from Figure 2(a).

Original: Second, Figures 2(b) and 2(c) show that the normal testsamples lie in the same manifold as the training samples, while the abnormal samples (abnormal testsamples and perturbed samples) are relatively separated.
Revised: Second, Figures 2(b) and 2(c) show that the normal testsamples lie in the same manifold as the training samples, while the abnormal samples (abnormal testsamples and perturbed samples) are relatively separated.

Original: In other words, PLAD explicitly learns adiscriminative embedded space to distinguish normal samples and anomalies.
Revised: In other words, PLAD explicitly learns adiscriminative embedded space to distinguish normal samples and anomalies.

Original: 4.3.2 Experiment on non-image datasets
Revised: 4.3.2 Experiment on non-image datasets

Original: Table 5 summarizes the F1-scores of each competitive method on the Thyroid and Arrhythmiadatasets.
Revised: Table 5 summarizes the F1-scores of each competitive method on the Thyroid and Arrhythmiadatasets.

Original: It can be observed that PLAD signiﬁcantly outperforms several baseline methods suchas OCSVM, DAGMM, DSVDD and DROCC with a large margin.
Revised: It can be observed that PLAD signiﬁcantly outperforms several baseline methods suchas OCSVM, DAGMM, DSVDD and DROCC with a large margin.

Original: Although NeuTraL AD and
Revised: Although NeuTraL AD and

Original: GOCC achieve encouraging 76.8% F1-score on Thyroid, it is worth mentioning that they are bothmethods designed for non-image data.
Revised: GOCC achieve encouraging 76.8% F1-score on Thyroid, it is worth mentioning that they are bothmethods designed for non-image data.

Original: The Arrhythmia dataset seems to be a more difﬁcult anomalydetection task because of its small sample size, which is not conducive to deep learning.
Revised: The Arrhythmia dataset seems to be a more difﬁcult anomalydetection task because of its small sample size, which is not conducive to deep learning.

Original: Surprisingly,the proposed PLAD method show remarkable performance on Arrhythmia, which surpasses 9.2%compared to the runner-up.
Revised: Surprisingly,the proposed PLAD method show remarkable performance on Arrhythmia, which surpasses 9.2%compared to the runner-up.

Original: Moreover, the performance of PLAD is also comparable to NeuTraL ADand GOCC on Thyroid, which fully demonstrates its applicability to the anomaly detection task fornon-image data.
Revised: Moreover, the performance of PLAD is also comparable to NeuTraL ADand GOCC on Thyroid, which fully demonstrates its applicability to the anomaly detection task fornon-image data.

Original: 4.3.3 Experiment on separate anomaly from multi-class normal data
Revised: 4.3.3 Experiment on separate anomaly from multi-class normal data

Original: It should be pointed out that in real applications, the normal data may contains multiple classeswithout labels.
Revised: It should be pointed out that in real applications, the normal data may contains multiple classeswithout labels.

Original: We need to separate anomaly from these multi-class normal data.
Revised: We need to separate anomaly from these multi-class normal data.

Original: In this study,
Revised: In this study,

Original: [Table]
Revised: [Table]

Original: we randomly select 10,000 samples among the training split of CIFAR-10 or Fashion-MNIST toconstruct a new normal training set, namely, the normal data are in multiple classes.
Revised: we randomly select 10,000 samples among the training split of CIFAR-10 or Fashion-MNIST toconstruct a new normal training set, namely, the normal data are in multiple classes.

Original: Subsequently, werandomly select two samples from the test split of CIFAR-10 or Fashion-MNIST to construct 10,000anomalous samples using the means of pair-wise samples in pixel level.
Revised: Subsequently, werandomly select two samples from the test split of CIFAR-10 or Fashion-MNIST to construct 10,000anomalous samples using the means of pair-wise samples in pixel level.

Original: The produced anomaloussamples are merged with the original test split to form a new test set.
Revised: The produced anomaloussamples are merged with the original test split to form a new test set.

Original: Compared with the previoustwo tasks (Sections 4.3.1 and 4.3.2), this one is much more difﬁcult because the decision boundarybetween the anomalous samples and the normal samples are very complicated.
Revised: Compared with the previoustwo tasks (Sections 4.3.1 and 4.3.2), this one is much more difﬁcult because the decision boundarybetween the anomalous samples and the normal samples are very complicated.

Original: We run the experimentto compare our method with four baselines including OCSVM, DAGMM, DSVDD and DROCCand report the results in Table 6.
Revised: We run the experimentto compare our method with four baselines including OCSVM, DAGMM, DSVDD and DROCCand report the results in Table 6.

Original: We see that the proposed PLAD method outperforms the baselinessigniﬁcantly.
Revised: We see that the proposed PLAD method outperforms the baselinessigniﬁcantly.

Original: Foe example, the improvement over the runner-up DSVDD is 9.1% and 4.4% on
Revised: Foe example, the improvement over the runner-up DSVDD is 9.1% and 4.4% on

Original: CIFAR-10 and Fashion-MNIST.
Revised: CIFAR-10 and Fashion-MNIST.

Original: The success of PLAD mainly stem from the ability of learning adecision boundary adaptively without any assumption.
Revised: The success of PLAD mainly stem from the ability of learning adecision boundary adaptively without any assumption.

Original: [Table]
Revised: [Table]

Original:  Conclusion
Revised:  Conclusion

Original: We have presented a novel method PLAD for anomaly detection.
Revised: We have presented a novel method PLAD for anomaly detection.

Original: Compared with its competitors,
Revised: Compared with its competitors,

Original: PLAD does not require any assumption about the distribution or structure of the normal data.
Revised: PLAD does not require any assumption about the distribution or structure of the normal data.

Original: This isthe major reason for that PLAD outperforms its competitors.
Revised: This isthe major reason for that PLAD outperforms its competitors.

Original: In addition, PLAD has fewer hyperparameters to determine and has lower computation cost than many strong baselines such as [Wang etal. , 2019; Goyal et al. , 2020; Yan et al. , 2021].
Revised: In addition, PLAD has fewer hyperparameters to determine and has lower computation cost than many strong baselines such as [Wang etal. , 2019; Goyal et al. , 2020; Yan et al. , 2021].

Original: Actually, PLAD provides us a framework for anomalydetection.
Revised: Actually, PLAD provides us a framework for anomalydetection.

Original: Different neural networks such as CNN [Krizhevsky et al. , 2012], RNN [Mikolov et al. ,2010], GNN [Scarselli et al. , 2008], and even transformer Vaswani et al.
Revised: Different neural networks such as CNN [Krizhevsky et al. , 2012], RNN [Mikolov et al. ,2010], GNN [Scarselli et al. , 2008], and even transformer Vaswani et al.

Original: [2017] can be embedded into
Revised: [2017] can be embedded into

Original: PLAD to accomplish various anomaly detection tasks such as time series anomaly detection.
Revised: PLAD to accomplish various anomaly detection tasks such as time series anomaly detection.

Original: Onelimitation of our work is that we haven’t included these experiments currently.
Revised: Onelimitation of our work is that we haven’t included these experiments currently.

Original: Note that in this study,we only considered the additive and multiplicative perturbations, i.e., ˜ x
Revised: Note that in this study,we only considered the additive and multiplicative perturbations, i.e., ˜ x

Original: = x (cid:12) g α ˜ θ ( x )+ g β ˜ θ ( xi ) , whilea more general perturbation can be formulated as ˜ x
Revised: = x (cid:12) g α ˜ θ ( x )+ g β ˜ θ ( x ) , whilea more general perturbation can be formulated as ˜ x

Original: = g (cid:48)
Revised: = g (cid:48)

Original: ˜
Revised: ˜

Original: θ ( x ) .
Revised: θ ( x ) .

Original: When we need a very sophisticatedperturbator, we may not well approximate g (cid:48)
Revised: When we need a very sophisticatedperturbator, we may not well approximate g (cid:48)

Original: ˜
Revised: ˜

Original: θ ( x ) by x (cid:12) g α ˜
Revised: θ ( x ) by x (cid:12) g α ˜

Original: θ ( x )
Revised: θ ( x )

Original: + g β ˜
Revised: + g β ˜

Original: θ ( xi ) , because we have to madesome restriction for g α ˜ 
Revised: θ ( x ) , because we have to madesome restriction for g α ˜ θ

Original: θ and g β ˜ θ .
Revised: and g β ˜ θ .

Original: Therefore, future study may also focus on determining the φ for (1),to learn g (cid:48)
Revised: Therefore, future study may also focus on determining the φ for (1),to learn g (cid:48)

Original: ˜
Revised: ˜

Original: θ ( x ) more effectively.
Revised: θ ( x ) more effectively.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section.•
Revised: [Yes] See Section.•

Original: Did you include the license to the code and datasets?
Revised: Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.
Revised: [No] The code and the data areproprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A](d)
Revised: [N/A](d)

Original: Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?
Revised: of GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and
Revised: (c) Did you include the estimated hourly wage paid to participants and

Original: the total amountspent on participant compensation?
Revised: the total amountspent on participant compensation?

Original: [N/A]
Revised: [N/A]