Adversarially learned anomaly detection for time series data
Authors: Alexander Geiger
Alfredo Cuesta-Infante
Kalyan Veeramachaneni
Venue:  (2020)
Keywords: anomaly detection
gan

Abstract: Anomaly detection in time series data is an important topic in many domains. However, time series are known to be particular hard to analyze. Based on the recent developments in adversarially learned models, we propose a new approach for anomaly detection in time series data. We build upon the idea to use a combination of a reconstruction error and the output of a Critic network. To this end we propose a cycle-consistent GAN architecture for sequential data and a new way of measuring the reconstruction error. We then show in a detailed evaluation how the different parts of our model contribute to the final anomaly score and demonstrate how the method improves the results on several data sets. We also compare our model to other baseline anomaly detection methods to verify its performance.

Under review as a conference paper at ICLR 2020
ADVERSARIALLY LEARNED ANOMALY DETECTION FOR TIME SERIES DATA
Anonymous authors Paper under double-blind review
ABSTRACT
Anomaly detection in time series data is an important topic in many domains. However, time series are known to be particularly hard to analyze. Based on the recent developments in adversarially learned models, we propose new approach for anomaly detection in time series data. We build upon the idea to use a combi- nation of a reconstruction error and the output of a Critic network. To this end we propose a cycle-consistent GAN architecture for sequential data and new way of measuring the reconstruction error. We then show in detailed evaluation how the different parts of our model contribute to the final anomaly score and demonstrate how the method improves the results several data sets. We also compare model to other baseline anomaly detection methods to verify its performance.
INTRODUCTION
With recent proliferation of devices collecting temporal observations, there has been increasing demand for anomaly detection in time series. The main goal is to identify any behavior in the time series that unusual, flag and bring it for analysis. In many real world settings a continuous time series collected over long periods is provided, and the goal is to isolate anomalous sub sequences of varied lengths. One usually does not know where those sub sequences may exist, or how long or short each one would be and how many there are.
In a classical setting, it is possible to segment the time series into many subsequences (overlapping or otherwise) of a certain length and apply methods focused on generating anomaly score for each sequence order to show how certain sequences are compared to others. [Chandola et al. (2008) present comparative study for several time series anomaly detection methods. They categorize the techniques into kernel-based, window- based and Markovian techniques.
Practitioners who don't "on't know how segment may resort to less complex techniques, such as simple thresholding, to detect any data points that exceed the normal range (Chandola et al.) 2009). How- ever, many anomalies do not exceed any boundaries for example, they may have values that are purportedly "normal," but are actually unusual at the specific time that they occur. These contextual anomalies naturally harder to identify, since the context of signal is often not clear (Chandola et al. 2009 Ahmad et al. 2017).
In recent years, Deep Learning- based methods have been developed deal with such issues (woon et al. 2017). These methods make use of the increased availability of data in order to learn the underlying structure of time series, and identify unusual changes in behavior using prediction or reconstruction errors (Malhotra et al., 2015). Within this framework - learning a model, predicting and reconstructing sequences, and using reconstruction errors to detect anomalies multiple variants have been developed (Malhotra et al. 2015| Hundman et al. 2018 Goh et al. 2017).
At the same time, recent years have also seen the introduction of adversarially trained networks, which can learn the underlying distributions of data sets and generate impressive synthetic data from this information. Generative Adversarial Networks (GANs), which were introduced by [Goodfellow et al. (2014) in 2014, have been very successful, especially in the area of image processing. Without direct access to real data, Generators in GANs attempt to synthesize real-looking data by implicitly learning the structure of dataset. Seeing this success has motivated
to explore whether GANs can also learn the structure of a time series. To the best of our knowledge, only one other work by L al. (018) uses GANs in time series anomaly detection. Building on this approach, this paper

---

Under review as a conference paper at ICLR 2020
aims to give more thorough insight into this domain, and to demonstrate how adversarially learned networks could be used for anomaly detection in time series data.
Our key contribution is the development of cycle-consistent GAN for sequential data that can be used for anomaly detection. Because we analyze time series data, which naturally comes with short- or long-term dependencies, Memory (LSTM) cells. In order to achieve cycle consistency during training, we use a reconstruc- tion loss for the Encoder and Generator training, and a second Critic network to support the correct bidirectional mappings.
encoding and generating networks are based on Long Short Term
Furthermore, we propose that the point-wise reconstruction error between original time series points and the reconstructed points, which is often used for anomaly detection, regularly fails to give the best error function for time series data. Instead, we introduce two similarity measures, which try to evaluate the local similarity between the original and the reconstructed sequences. We then combine this similarity measure and the Critic output into function that gives robust anomaly scores for the time series.
To provide further insights into anomaly detection with GANs and to demonstrate our proposed model, we provide an evaluation which investigates how each component of our model contributes to anomaly detection performance. Finally, we provide several benchmarks on well-known time series data sets and show how approach exceeds the performance of current state-of-the-an methods.
The paper' is structured as follows: First, we give an overview of related literature in section 2 Next, we introduce our model in section 3 We then describe the anomaly detection method in section and give evaluation of our proposed model in section [
RELATED WORK
Anomaly detection is a broad area of study, and several methods have been developed over the course of years, if not decades. To limit the scope of related work, we focus on instances where generative adversarial networks have been used for anomaly detection.
GANs are most often used to generate images and few studies have used them to detect anomalies in them. For example, |Schlegl et al. (2019) and Deecke et al. (2018) use GAN based architectures to detect anomalies in images.
However, only a few studies have used GANs for time series data. For examplc, Esteban et al. (017)) recurrent conditional GANs to generate medical time series data. More recently, (Luo et al. 2018) used them to impute missing values in multivariate time series.
In order to use GANs for anomaly detection in time series, Li et al. (2018) proposed a GAN model to capture the distribution of a multivariate time series. The Critic is then used to detect anomalies. They also attempt use the reconstruction loss find the inverse mapping from the data space to the latent space. This mapping, which tries to infer 2 from X, done second step after the GAN is trained.
additional anomaly detection method, they
The same two-step approach is used in a more recent preprint by Li et al. (2019) as well as by Schleel al. (2017) detect anomalies in medical images. However, as Zenati et (2018) mention in their paper. this method is impractical for large data sets or real-time applications. They propose a bi- directional GAN for anomaly detection in tabular and image data sets, which allows to train the inverse mapping through an encoding network simultaneously.
The idea of training both encoder and decoder networks was originally developed by Donahue et al. (2017) and Dumoulin et al.] (2017). who show how to get bidirectional GANs by trying to match joint distributions. In the optimum situation, the joint distributions are the same and the Encoder and Decoder must be inverses of each other. A similar cycle consistent GAN was introduced in Zhu jet al. (0077), where two networks try to map into opposite dimensions, such that samples can be mapped from space to the other and vice versa. Also, Zhou al. (2019) using a Encoder and Decoder GAN architecture that allows to use the reconstruction error for anomaly detection in ECG data.
2

---

Under review as a conference paper at ICLR 2020
3 ADVERSARIAL LEARNING FOR TIME SERIES
While our time series anomaly detection model builds heavily on the bidirectional concepts intro- duced in the works mentioned above, we are introducing some changes the loss and objective functions. Therefore we will describe the details of our model in this section.
The standard GAN can be used to generate time series. It consists of two components, the Generator G and the Critic (or Discriminator) Cz. Typically, both are implemented through neural networks. maps z drawn from an assumed latent distribution Pz typically standard multivariate normal distribution, i. e. Pz= (0, 7). The input data domain X is described by the given training samples {<))"..))} x:... E and X represents the possible time series sequences of length t. For convenience of notation we use 2; to imply a time sequence of length t.
The goal of C2 to distinguish between real time series sequences from and the generated time series sequences from ((z), whereas G is trying to fool C2 by generating real-looking sequences. Therefore, we have a saddle-poin soidee-pointppe problem, where G and C2 are competing against each other.
In our implementation makes use of the Wasserstein- I distance when training the Critic network. Formally, let x be the distribution over X, then we have the following problem:
Wasserstein loss, first introduced by Arjovsky et al. (201)). which
min max Vx (C, C,EC2
(1)
with
Vx -
(x) EzyEZ (C(())
where Cx denotes the of I-Lipschitz functions. We LSTM based neural network for 9 and a 1-D convolutional neural network for C₂ in our implementation.
In order to match samples with representations & for the mapping X + Pz. Therefore, we compute another loss term with the intention to minimize the L. norm of the difference between the original and the reconstructed samples:
the latent space, we train an encoding network
Vi2 9) Ex-Px Ilke ((E(x))
This gives the following objective we try to solve:
min max Vx (C,, VI2(E,9)
(4)
G.8 C.EC2
In this objective function, the Critic for the Generator is optimized by approximating the Wasserstein-1 distance, and the Generator and the Encoder optimized to minimize this distance and the additional 1.2 objective function.
To support the correct mapping into the latent distribution Pz, we add second Critic C.. The resulting architecture becomes similar to the one due to the CycleGAN proposal of of et al. al.] (2017). although with different loss. Its purpose is try distinguish between random latent samples ~ Fz and encoded samples {(x) with x ~ Px. Thus, we have the following additional objective using the Wasserstein-1 I loss again:
min max EC, C.EC,
(5)
with
Vz = BgnTy C2(2)) B ByNTy (C= (E(x)))
(6)
Combining all of the objectives given in
2
3 and leads the final MinMax problem:
min (E.0) (c, (C, 9 + Vıa(E, + Vz(C,,
max
In order to enforce the I-Lipschitz constraint during training, we apply gradient penalty regular-
ization term as introduced in Gulrajani et al. (2017), which penalizes gradients not equal to 1.
The full architecture of our model can be seen in Figure T

---

and Y (91, V2..... : Vm J. and let M R be a matrix such that the (i, element is a distance measure between and rj. denoted wk- Then we can construct warp path W (W1, W2..... wx) with max(n, m) < K n + m 1, and K is the length of the warp path. We want to find the warp path W* that defines the minimum Under review as a conference paper at ICLR 2020
C2
Px
1.2
((E(z))
G(=)
g
((x)
Pg
Figure Model architecture
4 ANOMALY DETECTION
Having the cycle-consistent architecture, anomalies. On the one side we can use the reconstruction error by encoding and decoding time series sequences. On the other side we can use the output of the Critic C₂ (x) as direct anomaly measure. In this section, we describe these two methods in more detail and then present a way to combine them effectively to give a robust anomaly score.
now
different methods in order detect
4.1 RECONSTRUCTION ERROR
Encoding time series sequence into the latent space using the Encoder and decoding it back into the original data space allows to reconstruct the sequence due to the cycle consistency our archi- tecture. This part behaves like Autoencoder, where the reconstruction error can be used as an anomaly measure. The reason to use the reconstruction error is the fact that the model should not be able to reconstruct anomalous sequences well as normal sequences and the use of this method is well studied and accepted in the area of anomaly detection (Pimentel et al. 2014 Hundman et al., 2018 Malhotra al., 2015). Typically, the reconstruction series is defined as the point-wise difference between the true value and the reconstructed value, i.e. e, Ij 24, Vr; X. However, we claim that this approach might not always be the best way to define the reconstruction error. Instead, we propose a different method, which is applicable in more cases. Informally, we want to measure the local similarity between the true sequence and the reconstructed sequence and identify regions where the similarity is low. There exist various similar- ity measures for time series (Serrà Arcos 2014). One simple reason why we might not want to use the absolute difference between the points is that two curves could have only small difference but over long period of time, which would not give high point-wise errors. Thus, we propose two other similarity measures that could be used, i.c. Dynamic Time Warping and the simple difference of areas below the two curves, both applied over windows of certain length in order to measure the similarity locally.
used in anomaly detection for time
4.1.1 DYNAMIC TIME WARPING
Dynamic Time Warping (DTW) was first introduced by Bemdt & Clifford 1994). Let us assume that we have two time series X (I) ,2,..., ,,,))
distance between the two curves, subject to boundary conditions the start and end, as well as constraints on continuity and monotonicity. The DTW distance between time series X and Y is defined as:
K
1
W = DTW(X, Y) min
Wk
(8)
K
A=
4

---

Under review as a conference paper at ICLR 2020
4.1.2 AREA DIFFERENCE
For the second, even simpler similarity measure, want to define the similarity of the two curves by simply comparing the areas beneath them. This seems very intuitive yet not often used this context, but will show in our experiments that this approach works very well. Suppose two functions (z) and ( then we define the similarity & over an interval [t, t + +2] as the average difference of areas below the curves:
have
1 A+l 8t 1.-00 [(x) - g(x) dr
-(x)-9(x))
where is the length of the sequence that we want to integrate over.
We stress that there is not the absolute difference f g(x)) inside of the integral, which would denote the area between the two curves. Instead we allow the integral to become negative, thus allowing the areas between the curves to cancel each other out if the curves cross at some point. By doing so we penalize regions where the original and the predicted sequence are just shifted by small number.
Since we are only given fixed samples of the functions, we can use the trapezoidal rule to calculate the definite integral. We apply the similarity measure to a moving window of size (between 50 and 150 worked best in our experiments), which slides over the whole time series. The result is function of similarity at every point in time, which we can then use as an anomaly measure.
4.2 Critic OUTPUT
Next to the reconstruction eπror, we can use the trained Critic C,. During the training process, the Critic has distinguish between real input sequences and synthetic ones. As we use the Wasserstein-1 distance when training the Critic, where the Critic is intuitively trying to assign scores of realness to the samples, the output of the Critic can be seen as a score of how real or fake a se- quence is. Therefore, once the Critic is trained, it can serve as anomaly measure for time series sequences.
We show in experiments that it is indeed the case that the Critic assigns different scores to anomalous regions compared to normal regions. This allows to use thresholding techniques to iden- tify the anomalous regions.
4.3 ANOMALY SCORE
In order to get a robust anomaly detection measure, we can combine the reconstruction error and the Critic output. Different methods have been proposed to combine these two scores. For example, the reconstruction error RE(x) and the Critic output Cz (x) can be merged in a single value a(x) with convex combination, a(x) REF(x) + (1 a Cr (1r) (Li al. 2018 Schlegl al. 2017). In case, we similarly aim to find deviations from the normal state by finding outliers in both scores. To
this end, we compute the mean and standard deviation of the two scores and calculate the respective z-score in order to normalize both. Finally, we then multiply both score vectors element-wise:
a (x)) RE(x) ② (x)
(10)
This way we get very robust anomaly score since high values in both scores are emphasized through the multiplication. Once the score is calculated, thresholding techniques can be applied in order to identify anomalous sequences. In our case, the threshold is calculated over window of certain length (where we use a window size of 1000 time steps in our experiments). While there exist different approaches to define the threshold, deviations from the mean the window. In order to avoid outliers in both directions during the mean calculation, we define the mean only over the values within the 25th and 75th percentile.
simple static threshold defined 4 standard
5

---

Under review as a conference paper at ICLR 2020
5 EXPERIMENTAL RESULTS
5.1 DATA SETS
In order to measure the performance of our model, we evaluate it on multiple time series data sets. In total, we have six data sets from different repositories, all of which contain anomalies.
Spacecraft telemetry: Within the spacecraft domain, we have two data sets provided by NAS The MSL. data set contains 27 signals from the Mars Science Laboratory (with total of 36 anoma- lies), while the SMAP data set contains 55 signals from the Soil Moisture Active Passive satellite (with a total of 69 anomalies). The locations of the anomalies are known. The amount of training samples varies between the signals but is mostly in the range of 1000.
YahooS5:
The YahooS5 data collection contains four different data sets The Al data is based on real production traffic to some Yahoo properties. Data sets A2, A3 and A4 are synthetic data sets. In total, the four data sets contain 371 different signals and the locations of the anomalies are known. The amount of training samples also varies between the signals but is mostly in the range of 1000 as well. Some of the signals contain a large number of anomalies, which decreases the amount samples. We still found in experiments that the model works quite well even on small numbers of training samples.
5.2 DATA PREPARATION
Before the training, we normalize the data and create training sequences by using a rolling window function, which creates sequences of length 100. These sequences are the inputs the network. We move the window by one timestep, such that each point of the time series gets reconstructed 100 times different places of the sequence. We then take the average of these reconstructions obtain a single reconstruction point. Similarly, the Critic gives 100 different scores for each data point. We then take the median of those scores to obtain the Critic value at this points. (In the plots, we also show the 5th and 95th percentile of these 100 values to demonstrate the additional increase of variance in anomalous regions.)
In the case of the Yahoo data sets, we also apply simple detrending function (which subracts the result of a linear least-squares to the signal) before training and testing, as many signals these data sets contain linear trends.
Finally, we train the model only on non-anomalous sequences to ensure that the model does not learn to reconstruct anomalies, which is well-known problem in reconstruction- and prediction- based anomaly detection methods.
5.3 ARCHITECTURE
In our experiments the inputs to our model are time series sequences of length 100 and the latent space 20-dimensional.
Furthermore, we use 1-layer bidirectional Long Short- Term Memory (LSTM) with 100 hidden units as an Encoder, and 2-layer bidirectional LSTM with 64 hidden units each as the Generator. LSTMs, first introduced by Hoccheiter & Schmidhuber (1997), are known for being able to model (long-term) time dependencies in data. Since we analyze time series data that naturally comes with these sequential dependencies, we chose LSTMs for our Encoder and Generator. We also use Dropout for the LSTM weights. We then use a 1-D convolutional net for both Critics, with the intention to capture local temporal features that can determine how anomalous a sequence is.
The model is trained on specific data set for 2000 iterations, with a batch size 64.
Metrics: We measure the performance of different methods with the commonly used metrics Pre- cision, Recall and Fl-Score. In our case, a true positive found when we have overlap between
1 'The spacecraft telemetry data
downloaded here: https::/s3-us-wettt
com/ te
"The YahooS5 data collection can be requested here:
sandoox.
catalog olallogpppo paaallggpppp//aatttyee--ssd

---

Under review as a conference paper at ICLR 2020
1.5
0.5
8
Figure 2: Results for the NASA S-1 signal. The first plot shows the original time series (blue) and the reconstructed one (orange), well the anomalous region (red). The second plot shows the time series and the aggregated output of the Critic (purple), as well as the 25th and 75th percentile of the Critic output. The third plot shows the reconstruction error (orange), the Critic error (purple) and the combined error (black)
predicted anomalous sequence and known sequence. A false positive is any sequence that was predicted but has no overlap with any known sequence, and a false negative is any known sequence that does not overlap with any of the predicted sequences.
5.4 RESULTS
The following section aims to evaluate different model variants in order to show the impact of our proposed model. To this end, we consider multiple variations of our system and check the scores on the data sets. This is going to show the differences between the combination of Critic and re- construction errors compared to single measures alone. Further, we want to show how the proposed similarity measures can affect the performance of anomaly detection. After that, we compare our model to baseline methods in time series anomaly detection. (The full results can be seen in Tables 3 and the appendix)
5.4.1 EVALUATION OF MODEL
As the results Table show, the Critic does indeed have an influence on the anomaly detection scores. On the NASA data sets, these differences are quite significant. For example, we see that in the SMAP data set, the best Fl score is achieved by using only the Critic.
When looking at the outputs of our different models, it becomes clear that the combination enforces the scores in anomalous regions while lowering the scores in normal regions. Figure 2 shows an example of this behavior. We also see this in the Precision and Recall differences, where we observe a greater average difference in the Precision than in the Recall when comparing a combination of both errors with the reconstruction error alone. This is due to the more significant reduction of false positives.
The differences are not as pronounced with the Yahoo data sets, which seems to be a result of the characteristics of those data sets, which are often more simple time series sequences with very clear, short and frequent anomalies. Although the scores are closer, the combination of Critic and reconstruction error gives a better score compared to just the single measures. When looking at the visual outputs of our model, we also continue stable and reliable anomaly score in most cases, as the difference between the scores of anomalous regions compared normal regions is increased.
still observe that in many cases
generally more
In addition to that, our results show that in all but one data set, the similarity measures we proposed perform better than the point-wise reconstruction error. This shows how the use of local similarity

---

Under review as a conference paper at ICLR 2020
measures reconstruction can be efficient approach than the point- wise when dealing with time series.
Data sets
NASA
Yahoo S5
Variation
MSL
SMAP
Total
A1
A2
A3
A4
Total
Critic
0,292
0.706
0.599
0.037
0.01
0.014
0.013
Critic area difference
0.573
0.689
0.655
0.524
0.789
0.945
0.873
0.862
Area difference
0,546
0.604
0.587
0.564
0.759
0.95
0.896
0.869
Critic point difference
0.481
0.567
0.545
0.618
0.75
0.89
0.844
0.829
Point difference
0,466
0.456
0.462
0.62
0.747
0.867
0.843
0.817
Critic DTW
0.503
0.596
0.573
0.612
0.798
0.897
0.826
0.831
DTW
0,434
0.494
0.481
0.603
0.777
0.876
0.832
0.82
Table 1: F1-Scores of different variations of our model.
5.4.2 COMPARISON TO BASELINES
We now compare model to simple LSTM-based prediction, ARIMA-based prediction model and a simple Autoencoder with dense layers (in order to represent a network without LSTMs), A short description of the baseline methods can be found in appendix A.2
As is evident in Table 2 our proposed GAN models perform better on three of the six individual data sets and, when considering all signals across the NASA and YahooS5 data sets, outperform every other method. For the three datasets where they perform better, we see a significant 12% improvement on SMAP, a 15% improvement A3 and 27% improvement in A4.
Data sets
NASA
Yahoo $5
Method
MSL
SMAP
Total
A1
A2
A3
A4
Total
LSTM prediction
0.602
0.521
0.547
0.706
0.783
0.822
0.706
0.764
Arima prediction
0.355
0.452
0.416
0.678
0.898
0.498
0,621
0,621
Dense Autoencoder
0.517
0.629
0.593
0.599
0.206
0.297
0.26
0.29
Table 2: FI-Scores of baseline models
CONCLUSION
In this paper we showed how GANs can be effectively used for anomaly detection in time series data. We proposed a cycle-consistent GAN architecture that allows the encoding and decoding of time series. We further proposed new reconstruction eπror measures based on local similarities that outperform the point-wise reconstruction. We have also shown that a combination of the Critic output and the reconstruction increase the number of true positives. This due to the more robust anomaly scores that our method providing, where effectively increase the distance between the scores of anomalous and anomalous regions. As a result, the anomaly scores are more robust and allow a better detection of anomalous time series sequences.
help reduce the number of false positives, well as might
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Subutai Ahmad, Alexander Lavin, Scott Purdy, and Zuha Agha. Unsupervised real-time anomaly
detection for streaming data. Neurocomputingg. 262:134 147, 2017. ISSN 0925-2312. doi:
URL http: / www. sciencedirect. com/ Online Real-Time Learning Strategies
for Data Streams.
Martín Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Proc. of the 34th Int. Conf, on Machine Learning. ICML, pp. 214-223, 2017.
Donald J Bemdt and James Clifford. Using Dynamic Time Warping to FindPattems in Time Series. In AAAI-94 Workshop onKnowledge Discovery in Databases, Seattle, Washington, 1994. URL
www. aaai org
Varun Chandola, Varun Mithal, and Vipin Kumar. Comparative Evaluation of Anomaly Detection Techniques for Sequence Data. In IEEE Int. Conf. on Data Mining, pp. 743 748, dec 2008, ISBN 978-0-7695-3502-9. doi: 10. N0.100//CDM220088.515
URL http://ieeexploor ieee.
#Tc//oocumet/446111
Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM Com- pututing Surveys, (11) July 2009. ISSN 0360-0300. doi: 011144//551180.55411111 URL
15118
Lucas Deecke, Robert Vandermeulen, Lukas Ruff, Stephan Mandt, and Marius Kloft. Anomaly detection with generative adversarial networks, 2018. URL httpps://ppenreeveew net.
Forum?1d=SlEfy_ll
Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. Adversarial Feature Learning. In IEEE Int. Conf. Learning Representations (ICLR), 2017. URL http:///axxiv..rraabbbo
09782
Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Ar- jovsky. and Aaron Courville. Adversarially Learned Inference. In JEEE Int. Conf. on Learning Representations (ICLR), 2017. URL (RRL//ttp//axxiii org wttg://////....0oo00/. 00704
Cristóbal Esteban, Stephanie L. Hyland, and Gunnar Rätsch. Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs. jun 2017. URL htl/tt////rxxiv.rr////
17766006333
Jonathan Goh, Sridhar Adepu, Marcus Tan, and Zi Shan Lec. Anomaly detection in cyber physical systems using recurrent neural networks. 2017 IEEE 18th International Symposium on High Assurance Systems Engineering (HASE), pp. 140-145, 2017.
lan J Goodfellow, Jean Abaget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. In Advances in neural information processing systems, pp. 2672-2680, 2014. URL http: http://www github........ com/
ggodEell/adversarrt
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved Training of Wasserstein GANs. In Proc. of the 31st Int. Conf. on Neural Informa- tion Processing Systems, pp. 5769-5779, 2017. URL htttps://githut. hww.sttuu.ooo.oeeeeemoeomoooeett hacomm/i///////// training.
[mppovedd__woooe
Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Computation, 9())
1735- 1997. URL https:/ www./wwwwkkmmmmm wtw.:.itpress...o.tottw ponittpressourrnlllss 1162/neco. 1 1997. 99979998..733
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soder- strom. Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding. In Proc. of the 24th ACM SIGKDD Int. Conf. on Knowledge Discov- ery & Data Mining, 2018. ISBN 978- 978-1-4503-5552-0. 1802. 04431 (wttpppst//ttt//d..doooooooo |3044//455444000///////oo/aoooa1
doi: 10.1145/320981993229988
org.///wooroooooo...oo.o.oooooo/ abs/

---

Under review as a conference paper at ICLR 2020
Donghwoon Kwon, Hyunjoo Kim, Jinoh Kim, Sang C. Suh, Ikkyun Kim, and Kuinam J. Kim. A survey of deep learning-base based network anomaly detection. Cluster Computing, 22:949-961, 2017.
Dan Li, Dacheng Chen, Jonathan Goh, and See- KionKiongATr Ng. Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series. In 7th Int. Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applica- lions; on the ACM Knowledge Discovery and Data Mining conference, 2018. URL https: //ithub.comm 4tmmmm//iaa44///mm
ADDttp::/aaxiiiii
04758
Dan Li, Dacheng Chen, Lei Shi, Baihong Jin, Jonathan Goh, and See-Kiong Ng. MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks.
Technical report, 2019. URL.htps::/ http:/ggtttttttttt com www.Liian/55555
Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, and Xiaojie Yuan. Multivariate time series imputation with generative adversarial networks. In Advances in Neural Information Processing Systems, volume 2018-Decem, pp. 1596-1607, 2018.
Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal. Long Short Term Memory Networks for Anomaly Detection in Time Series. In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2015. ISBN 9782875870148. URL
http:/ / //www 16doc. com/ /en/
Pankaj Malhotra, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, and Gautam Shroff. Lstm-based encoder-decoder for multi-sensor anomaly detection. arXiv preprint arXiv:1607.001 2016.
Marco A.F. Pimentel, David A. Clifton, Lei Clifton, and Lionel Tarassenko. A review of elty detection. Signal Processing, 99:215-249, jun 2014. ISSN 01651684. doi: 10.1016/
1.sigrro.003..1..02 URL https:://inkinne http://wwww...eeeeeeeeeeeeeeeeee
Thomas Schlegl, Philipp Seebück, Sebastian M Waldstein, Ursula Schmidt-Erfurth, and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In International Conference on Information Processing in Medical Imaging, volume 10265 LNCS, pp. 146-141. 2017. ISBN 9783319590493.. doi: 00007/97-33-31--9000 URL httpp::/aaaxvv..rr///dff..
12.
Thomas Schlegl, Philipp Seeböck, Sebastian M. Waldstein, Georg Langs, and Ursula Schmidt- Erfurth. f-anogan: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis, 54:30 44, 2019. ISSN 1361-8415. doi: https://oi.oog//0..u
201eiia.019.01.011 URL hhtp::/ http:://wwwwww.nneeeecoccoo rect. cwm/
p11/$13618415 8302640
Joan Serrà and Josep LI. Arcos. An empirical evaluation of similarity measures for time series classification. Knowledge-Basse Based Systems, 67:305 314, 2014. ISSN 0950-7051. doi: https://doi. k00. atticleep11/33557011110000
URL | /twp::://www... www.::/wwiennciire..cccm wtt.://cett.cooccceeeeeeeeeeeeee
Houssam Zenati, Manon Romain, Chuan-Sheng Foo, Bruno Lecouat, and Vijay Chandrasekhar. Adversarially Learned Anomaly Detection. In IEEE Int. Conf. on Data Mining (ICDM), pp- 727-736, nov 2018 ISBN 978-1-5386-915955. doi: 10.1 10L10///DM.2018.00003 URL https: 7/ieeexplore leee. org/ /dcccmntttttttt399/7
Bin Zhou, Shenghua Liu, Bryan Hooi, Xueqi Cheng, and Jing Ye. Beatgan: Anomalous rhythm detection using adversarially generated time series. In Proceedings of the Twenty-Eighth Inter- national Joint Conference on Artificial Intelligence. IJCAI-19, pp. 4433-4439. International Joint Conferences on Artificial Intelligence Organization, 7 2019. doi: 10.24963ficcu.22119661 URL 2019/615
https://doi... org. 10. 24963/1jcca
Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired Image-to-Image Trans- lation Using Cycle- Cecee-Consistent Adversarial Networks. In JEEE Int. Conf. Computer Viston (ICCV), pp. 22511, oct 2017. ISBN 118-1--386-1032--9
doi: 10.1100//CC222017.2244
URL /ttt:://ieexx:: ore. leee. httpe//ee..rrr/ooooooome///22553
10

---

Under review as a conference paper at ICLR 2020
A APPENDIX
A.1
RESULT DETAILS
Data
NASA
Yahoo S5
Variation
MSL
SMAP
Total
A1
A2
A3
A4
Total
P
0.869
0.8
0.806
0.1
0.143
0.192
0.11
Critic
0.176
0.633
0.476
0.022
0.005
0.007
0.007
F1
0.292
0.706
0.599
0.037
0.01
0.014
0.013
0.674
0.598
0.612
0.557
0.689
0.969
0.936
0.885
Critic area difference
R
0.5
0.816
0.705
0.494
0.924
0.921
0.818
0.84
FI
0.573
0.689
0.655
0.524
0.789
0.945
0.873
0.862
P
0.603
0.495
0.518
0.535
0.638
0.968
0.949
0.871
Area difference
R
0.5
0.773
0.676
0.596
0.937
0.932
0.849
0.867
FI
0.546
0.604
0.587
0.564
0.759
0.95
0.896
0.869
P
0.69
0.478
0.513
0.623
0.628
0.967
0.929
0.869
Critic point difference
0.37
0.696
0.581
0.612
0.93
0.824
0.773
0.793
F1
0.481
0.567
0.545
0.618
0.75
0.89
0.844
0.829
0.707
0.357
0,401
0.596
0.618
0.97
0.938
0.863
Point difference
R
0.352
0.633
0.543
0.646
0.943
0.784
0.765
0.777
FI
0,466
0,456
0.462
0.62
0.747
0.867
0.843
0,817
0.752
0.514
0,548
0.636
0.702
0.971
0.936
0.889
Critic DTW
R 0.38
0.71
0.6
0.59
0.924
0.833
0.739
0.781
FI 0.503
0.596
0,573
0.612
0.798
0.897
0.826
0.831
P
0.665
0.402
0.438
0.6
0.664
0.978
0.945
0.879
DTW
R
0.324
0.643
0.533
0.607
0.937
0.794
0.743
0.769
FI
0.434
0.494
0.481
0.603
0.777
0.876
0.832
0.82
Table 3: Precision, Recall and F1 of different variations of our model.
Data sets
NASA
Yahoo $5
Method
MSL
SMAP
Total
A1
A2
A3
A4
Total
P
0.463
0.38
0.406
0.614
0.643
0.93
0.837
0.814
LSTM prediction
R
0.861
0.826
0.838
0.831
0.737
0.611
0.72
Fl
0.602
0.521
0.547
0.706
0.783
0.822
0.706
0.764
P
0.268
0.359
0.324
0.614
0.912
0.908
0.892
0.845
Arima prediction
0.528
0.609
0.581
0.758
0.885
0.343
0.476
0.491
F1
0.355
0.452
0.416
0.678
0.898
0.498
0.621
0.621
P
0.682
0.709
0.701
0.837
0.988
0.687
0.836
Dense Autoencoder
R
0.417
0.565
0.514
0.466
0.115
0.175
0.16
0.176
F1
0.517
0.629
0.593
0.599
0.206
0.297
0.26
0.29
Table 4: Precision, Recall and FI-Scores baseline models
11

---

Under review as a conference paper at ICLR 2020
A.2 BASELINE METHODS
The network consists of two LSTM layers with 80 units each and following Dense layer with one unit which predicts the next time step. We then use a point-wise prediction error between the predicted and the observed time step to identify outliers. The model is similar to the one used by Hundman et (2018).
LSTM prediction:
ARIMA: ARIMA models try to describe autocorrelations in the time series, i.e. how the values of past time steps influence the current and future time steps. We use an ARIMA model to predict the next time steps based on the previous ones. The parameters were selected individually for each data set based on the best performance. We then use a point-wise prediction error to detect outliers.
The Autoencoder model consists of three Dense layers with 60, 20 and 60 units respectively. Again, a point-wise reconstruction error is used in order to detect anomalies. The method is related the one used by Malhotra al. (2016), however we are not using LSTMs in this case since our GAN model with only the reconstruction error is essentially an LSTM-based Autoencoder and this way we include baseline with different layer type as well.
Autoencoder:
12

---

Under review as a conference paper at ICLR 2020
A.3 EXAMPLES
The following examples show how our method performs on several signals. The plots are structured the following: The first plot shows the original time series (blue) and the reconstructed time series (orange), as well as the anomalous region (red). The second plot shows the time series and the aggregated output of the Critic (purple), as well as the 25th and 75th percentile of the Critic output. The third plot shows the reconstruction error (orange), the Critic error (purple) and the combined error (black)
A.3.1
NASA SMAP
0
Figure 3: E-1 Signal
0.5
Women wa 4nth with uM M
0.5
0-
0.5
0
Figure 4: E-9 Signal
13

---

Under review as a conference paper at ICLR 2020
0.5
wil الاامبطط
00.5
0.5
0
0.5
6-
2-
0-
Figure 5: P-1 Signal
A.3.2 NASA MSL
0
-1
0
4
2
0-
Figure 6: S-2 Signal
2-
0-
-1
2-
6-
4-
Figure 7: F-5 Signal
14

---

Under review as a conference paper at ICLR 2020
A.3.3 YAHOO A1
0-
Figure real-32 Signal
0-
6
Figure 9: real-8 Signal
15

---

Under review as a conference paper at ICLR 2020
A.3.4
YAHOO A2
0.5
0
0.5
0.5
6
2-
0-
Figure 10: synth-40 Signal
0.5
0.3
0.5
Figure 11: synth-52 Signal
A.3.5
YAΤIOO A3
0.5
0.5
4-
Figure 12: TS-29 Signal
16

---

Under review as a conference paper at ICLR 2020
0.5
00.5
0.5
6-
-4
2-
0-
Figure 13: TS-1 Signal
A.3.6 YAHOO A4
0.5
www.
0.5
1
0-
0.5
2-
Figure 14: TS-8 Signal
0.5
0.5
0.5
Figure 15: TS-61 Signal
17


Reviews:
[{"review_id": "ryevebv_Yr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper trains a GAN on univariate time series data and uses reconstruction errors in combination with the critic's output to predict anomalous subsequences. The method is applied on two real-world data sets and compared to three simple baselines.\n\nI have several reservations about this manuscript:\n- The methodology isn't very original: The GAN architecture is essentially a slightly modified CycleGAN trained with Wasserstein loss.\n- Many design choices appear to be ad-hoc: there has been no principled selection of the GAN's hyperparameters, nor has their effect on the experimental results been studied. For computing the reconstruction error the authors use the integral over the difference between two time series without taking the absolute value, which is a very unusual choice. (Standard choices would be e.g. L1 or L2 norm.) While this turns out to have worked \"surprisingly well\" on the studied datasets, it is not difficult to construct scenarios where this choice will fail. \n- Another design choice that should be discussed in more detail is the smoothing of the time series data using a moving average filter; in the context of anomaly detection this can have a significant impact, and it may not work equally well for different data sets, so a principled approach for determining the level of filtering is paramount. Same goes for the de-trending, and the actual parameters both of the moving averages and the de-trending functions should be reported.\n- It is not clear to me how exactly the anomaly scores (supposedly for different sequence lengths l?) are used to predict the subsequence(s) containing an anomaly. It seems there is no incentive to keep the predicted subsequences as short as possible, i.e. if the anomaly score indicates there might be an anomaly present, then the safe approach is to just flag the entire sequence as anomalous (it doesn't hurt the sensitivity they way it's computed)?\n- The description of the experiments requires more detail. Are both datasets labelled? After dividing the datasets into rolling chunks of length 100, how many samples do the training and test sets contain? How many of the test set samples contain anomalies?\n- The baselines need to be described in more detail. What method was used, e.g. to select and fit the ARIMA models? What type of reconstruction loss was used? How were the anomalous subsequences predicted?\n- Is there any way to compare the proposed method with Li et al's (who also used a GAN for anomaly detection in time series data)?\n\nDetailed comments:\n- abstract: \"particular hard\" -> \"particularly hard\"\n- p.3, paragraph starting with \"To support...\": shouldn't this be \"E(x) with x ~ P_X\"?\n- p.4 and p.6: broken citations \"?\"\n\n------------------\n\nI acknowledge that I have read the authors' response, but it doesn't change my assessment that several major revisions are needed:\n- further motivating the design choices and comparing them to the state-of-the-art;\n- formalizing (ideally providing a model for) the sort of anomalies that the proposed method aims to detect;\n- discussing limitations of the proposed approach.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "r1xrGN9sFr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a GAN model with cycle-consistent loss function for anomaly detection on timeseries. The loss function is the combination of reconstruction error (L2 norm) and both discriminators (generator and decoder) loss. Once trained, the anomaly score is computed as the mean-var-normalized product of the reconstruction score and the discriminator score. The framework is adapted to timeseries by using LSTM networks for the generator and the encoder and 1D CNN for the discriminators (critics). For the calculation of the reconstruction error, the authors study three approaches: point-wise, dynamic time warping and area difference. The method is applied to two datasets (NASA Spacecraft telemetry, Yahoo traffic) and is compared to simple baselines (LSTM, ARIMA, DenseAE). The results shows an improvement over the baselines. Variations of the models are also studied (with and without critic, different similarity measures for the reconstruction errors). The results show that the discrimination score (critic) does overall provide an improvement over the reconstruction error alone. And for the reconstruction score, the area differences is best, followed by DTW and point-wise.\n\nPROS:\n\n* A somewhat novel approach to anomaly detection in timeseries is proposed that combines GAN discriminator score with a simple area-based similarity measure for the  reconstruction error.\n* A solid adaptation of CycleGAN to timeseries.\n* The simple area-based similarity measure is interesting and seems to be a good match for timeseries reconstruction as slight shifts are not penalized.\n* The paper is well written, well organized and easy to follow.\n* The technical content is sound and the math correct.\n\n\nCONS:\n\n* not really novel (CycleGAN for timeseries) nor DTW as reconstruction error. \n* no comparison to State-of-the-art GAN models for anomaly detection, such as AnoGAN and ADGAN.\n* Too few datasets are used (2) for the experiments, making it hard to draw conclusions as to which variation is better.\n* Baseline approaches are just mentioned by name. A minimal description would be desirable.\n* The ground-truth anomalies and the anomaly threshold are not marked in the plots, making it hard to evaluate them.\n\nOverall the paper proposes a method that improves over baseline but is not compared to other GAN-based SOT models. The novelty is not very high as the main architecture is from CycleGAN and the proposed similarity measures for the reconstruction error are not really novel.\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "Sked-RZaFr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:  The paper propose a cycle gan variants combined with RNN for time series anomaly detection. The setting is assuming training model on a given normal data, then applying the trained model to detect anomalies.  Different detecting criteria are studied in the experiments. \n\n1. It is not clear the advantage of using GANs in anomaly detection of the proposed algorithm, and it is questionable if using GANs is really useful. The authors only provide hand-waving explanation.  For example, the papers says  \" Therefore, once the Critic is trained, it should assign more or less stable scores to the normal sequences and a significantly different score to an anomalous sequence.\" in p5.  I'm doubt if it is true.  The GAN objective only says the score on x~p(x) and g(z) are similar. Also, the score for normal samples can also have a distribution since we only do mean matching in W-1 distance.  I think a stronger analysis with certain assumptions is necessary for making this statement and justifying the proposed algorithm.  Some possible route can be found in \n\nChang et al., Kernel Change-Point Detection with Auxiliary Deep Generative Models, ICLR 2019.  \n\nAlthough their setting is slightly different, where they focus on change point detection.  They provide a testing power lower bound explanation of using the critic of GANs, but they require some early stopping. Otherwise the guarantee won't hold.  I'm wondering if the analysis can be extended to here. Also, if the proposed algorithm requires the early stopping or not? If not, why?  \n\n2. The experiments are not conclusive.  The simple LSTM predictions beats the proposed algorithms in some cases.   Given there are only two datasets, it's hard to say if the proposed algorithm is really better. Also, the author proposed so many combinations, it is also not clear which one should be favored based on the table. \n\n3. Many related works are missing.  In addition to Chang el al (2019) mentioned above, there are many related works of using GANs in time series detection problem, e.g. BeatGAN: Anomalous Rhythm Detection using Adversarially Generated Time Series, and many others. \n\n\nTo summarize, the motivation and the advantage of using GANs is not well justified in addition to experiments.  Also, the authors only study two datasets, and the results are not very conclusive. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a cycle-consistent GAN architecture with measuring the reconstruction error of time series for anomaly detection.

The paper aims to address an important problem, but the current version is not ready for publication. We suggest the authors consider the following aspects for improving the paper:
1. The novelty of the proposed model: motivate the design choices and compare them with state-of-art methods
2. Evaluation: formalize the target anomalies and identify datasets/examples where the proposed model can significantly outperform existing solutions. 


Decision: Reject