Year: 2019

Abstract:
Identifying anomalous samples from highly complex and unstructured data is a crucial but challenging task in a variety of intelligent systems. In this paper, we present a novel deep anomaly detection framework named AnoDM (standing for Anomaly detection based on unsupervised Disentangled representation learning and Manifold learning). The disentanglement learning is currently implemented by β-VAE for automatically discovering interpretable factorized latent representations in a completely unsupervised manner. The manifold learning is realized by t-SNE for projecting the latent representations to a 2D map. We define a new anomaly score function by combining β-VAE’s reconstruction error in the raw feature space and local density estimation in the t-SNE space. AnoDM was evaluated on both image and time-series data and achieved better results than models that use just one of the two measures and other deep learning methods.

heading: 1 INTRODUCTION
text: Detecting anomalies in data flow of modern intelligent systems is an important but challenging problem. Formally speaking, anomaly detection problems can be statistically viewed as identifying outliers having low probabilities from the modelling of data distribution p(x). Practically, since statistical modelling of the data is often difficult, it degenerates to domain description (Tax & Duin, 1999) or supervised prediction (Gornitz et al., 2013) problems in some cases. The exact explanation of an anomalous data point depends on the specific domain of focus. In data centers, it probably indicates an attempt of cyber intrusion. In recognition systems, it could be an adversarial attack. In biomedical information systems, it means possible onset of certain diseases. In Internet of Things (IoT) systems, it may represent a hardware failure or alarming event captured by sensors. An anomalous sample is not always associated with negativity. Sometimes, it leads to novel discoveries in scientific explorations.
However, from the data analytics perspective, anomaly detection is a difficult task due to the following reasons. (1) Many forms of data, e.g., images, text, and other types of sequences, are often highly unstructured and complex. How can these data be well represented and high-level information be extracted by an algorithm? (2) The sample sizes of modern data sets are often extremely large and most of them are unlabelled. Unfortunately, traditional methods do not scale and perform well on these data. (3) When data of multiple modalities are naturally available for same events in a system, a robust and precise algorithm needs to be designed to integrate these information for system diagnosis or decision making. (4) Many intelligent systems, such as IoTs, require real-time detection and reaction of abnormal events to avoid costly and irrevocable damages. Thus, anomaly monitoring algorithms to be designed in these platforms must be highly efficient. In summary, anomaly detection raises challenges in representability, scalability, multimodality, and time complexity.
Deep learning (LeCun et al., 2015) offers great potentials to overcome these challenges. (1) Representation learning mechanisms (such as convolution for images, embedding for discrete symbols, and recurrence for time-series) have been developed in supervised and unsupervised deep models to consider the nature of specific types of input samples and encode them into vectors of continuous values as corresponding latent representations. (2) Most deep learning models are trained using stochastic gradient descent that splits a giant training set into mini-batches. Thus, learning be-
comes unrestricted and blessed by a large sample size. Particularly, stochastic variational inference (Hoffman et al., 2013; Zhang et al., 2018) has successfully enabled scalable learning and inference for deep generative models (DGMs) on a vast amount of unlablled data. (3) The development of deep learning programming packages, such as PyTorch (Paszke et al., 2017) and TensorFlow (Abadi et al., 2016), greatly eases the assembly of multiple network components (corresponding to different modalities) together for multimodal representation learning (Li et al., 2018b). (4) Once a deep model is learned, the inference or encoding step is very efficient, thanks to the highly parallel computing architectures and techniques.
In some applications, if the domain of anomalous and normal samples is well defined, anomaly detection can be reduced to binary classification problems. However, in many situations, either the domain of anomalous samples cannot be fully understood or modelled, or the domain of the normal samples is too complicated to be modelled in one class. DGMs are more suitable than supervised methods in such cases. DGMs are concerned with the joint distribution of visible and latent variables with a hierarchy of stochastic (and deterministic) layers. With proper emphasis on disentanglement of latent representations, DGMs have the potential of dissecting hidden factors that are key to sample generation. Unsupervised disentangled representation learning (Bengio et al., 2013) renders several benefits. (1) It helps better understand our data, providing a path towards explainable AI. (2) It gives a better control on the generation process of novel samples. (3) The disentanglement of latent factors may provide an opportunity to distinguish anomalies based on the landscape of latent space, which is our interest in this paper. It has been shown that the likelihood of a data point p(x) estimated in DGM is not a reliable measure for detecting abnormal samples (Nalisnick et al., 2019). Instead, reconstruction error is widely used as an anomaly score function (An & Cho, 2015).
As a variant of variational autoencoder (VAE) (Kingma & Welling, 2014), β-VAE (Higgins et al., 2017) is designed for unsupervised discovery of interpretable factorized latent representations from raw image data. An adjustable hyperparameter β is introduced to balance the extent of learning constraints (a limit on the capacity of the latent information channel and an emphasis on learning statistically independent latent factors) and reconstruction accuracy. It was demonstrated that βVAE with appropriately tuned value of β (when β > 1) qualitatively outperforms VAE (when β = 1, β-VAE is exactly VAE). Burgess et al. (2018) proposed a modification to the training regime of βVAE by progressively increasing the information capacity of the latent code during training. This modification facilitates the robust learning of disentangled representations in β-VAE, without the previous trade-off in the reconstruction accuracy. Hoffman et al. (2017) introduced a reformulation of β-VAE for 0 < β < 1. They argued that, within in this range, training β-VAE is equivalent to optimizing an approximate log-marginal likelihood bound of VAE under an implicit prior.
Manifold learning is a family of nonlinear dimensionality reduction techniques. The t-distributed stochastic neighbor embedding (t-SNE) (van der Maaten & Hinton, 2008) is an unsupervised manifold learning method primarily used for data exploration and visualization by approximating highdimensional data distribution using a two or three-dimensional map that could preserve local and certain global structures of the data. The use of t-SNE for anomaly detection has been sceptical (van der Maaten & Hinton, 2008). However, no comprehensive investigation has been made in this topic. Taking advantages of both disentangled representation learning (using β-VAE as an implementation) and low-dimensional manifold learning (using t-SNE as an implementation), we propose a novel anomaly detection approach named AnoDM, standing for Anomaly detection based on unsupervised Disentangled representation learning and Manifold learning. We introduce a new anomaly score function by combining: (1) β-VAE’s reconstruction error, and (2) distances between latent representations of test points and training points in t-SNE map. AnoDM is a general framework, thus any disentangled representation learning and manifold learning techniques can be applied. The choice of a lower-level encoding scheme in β-VAE depends on data type of interest. For image data, deterministic convolutional network (CNN) is used in the encoder. In case of time series (sequence) data, we design an improved version of β-VAE by replacing CNN with temporal convolutional network (TCN) (Bai et al., 2018), a generic architecture for convolutional sequence prediction, in the encoder. We incorporate TCN as part of the encoder, because Bai et al. (2018) have shown that TCN outperforms canonical recurrent networks such as LSTMs (Hochreiter & Schmidhuber, 1997) across a range of supervised learning tasks and recommended that CNN should be regarded as the first method to try for sequence modeling tasks. Regarding the decoding architecture, we simply choose CNN, because by choosing a simpler CNN architecture as a part of the decoder, the model can achieve a comparable even better performance but take much less running time.
The contributions of this paper are summarized as follows. (1) We comprehensively explore the capacity of unsupervised disentangled representation learning, using β-VAE as an implementation, for anomaly detection. (2) We thoroughly investigate the potential of manifold learning for outlier identification by taking the disentangled latent representations from β-VAE as input to t-SNE. To the best of our knowledge, this is the first attempt to explore t-SNE for anomaly detection. (3) For sequence anomaly detection, instead of using prevailed recurrent networks (such as LSTM), as a practical contribution, we adopt an improved convolution architecture (TCN) to capture the temporal dependency in the encoder in unsupervised way.

heading: 2 RELATED WORK
text: In the big data era, the development of deep learning models, especially DGMs, flourishes due to the need of modelling and analyzing massive amount of unstructured data (such as images, timeseries, graphs, text, etc.) generated in many application domains. Designing DGM-based solutions for anomaly detection becomes an important topic. Since DGMs, such as VAE (Kingma & Welling, 2014), and deep belief net (DBN) (Hinton et al., 2006; Li & Zhu, 2018a), aim at modelling the joint distribution of visible and latent variables (that is p(x,h)), their likelihood p(x) by marginalizing out h may serve as an abnormality indicator. However, unlike exponential family restricted Boltzmann machines (exp-RBMs) (Li & Zhu, 2018b), exact likelihood is unavailable for most DGMs. Alternatively, reconstruction error serves as an abnormality measure based on the intuition that outof-distribution samples can be reconstructed badly (An & Cho, 2015). Some deep hybrid methods (e.g. VAE+OCSVM (Andrews et al., 2016) and DBN+OCSVM (Erfani et al., 2016)), successfully combine classical one-class support vector machine (OCSVM; or kernel-based support vector domain description (SVDD)) with DGMs by using DGMs to learn latent representations of samples and using OCSVM to detect abnormal data points. However, these methods face the challenge of scalability, because the size of kernel matrices in dual form of SVDD is quadratic of sample size.
Generative adversarial net (GAN) has also been applied for anomaly detection (Schlegl et al., 2017). Since there is no encoder in GAN, Deecke et al. (2019) presented the ADGAN algorithm based on the availability of a good representation of a sample in latent space of its generator by assuming that the generator is able to effectively capture the distribution of the training data. Li et al. (2018a) proposed the GAN-AD method for cyber-physical systems (CPSs). It distinguishes fake data from actual data by taking into consideration of both discrimination loss calculated by the trained discriminator and residual loss between reconstructed and actual test data.
Furthermore, DGM-based algorithms are also devised to detect anomaly problem on sequence data (e.g. LSTM-VAE (Park et al., 2017) and GAN-AD (Li et al., 2018a)). Conventionally, canonical recurrent networks (such as LSTM and GRUs (Cho et al., 2014)) are considered as the dedicated methods for sequence modeling. Some recent studies have also claimed that there was no architecture that could consistently beat LSTM in some typical sequence modelling tasks (Jozefowicz et al., 2015; Greff et al., 2015; Melis et al., 2017). On the other hand, some other researchers insist that CNN (LeCun et al., 1989) should be considered as more appropriate choice for sequences. Inspired by more recent CNN-based sequence modelling (such as machine translation (Gehring et al., 2016; 2017) and language modeling (Dauphin et al., 2016)), Bai et al. (2018) conducted a systematic evaluation of generic convolutional and recurrent architectures for sequence modelling across a broad range of tasks that are commonly used to benchmark recurrent networks, and concluded that convolutional networks, rather than recurrent networks, should be respected as a “natural starting point for sequence modelling tasks”.

heading: 3 METHOD
text: In this paper, we propose a novel generic anomaly detection framework named AnoDM, which the first time combines unsupervised disentangled representation learning (implemented using β-VAE as an example) and low-dimensional manifold learning (currently using t-SNE as implementation) together to detect outliers via effectively taking the advantages of reconstruction at raw feature space and disentangled latent distribution in t-SNE map. Figure 1 shows the architecture of AnoDM which includes two main phases: (1) unsupervised disentangled representation learning and (2) anomaly detector. After β-VAE is learned using unlablled training normal samples, it then can be employed
by the anomaly detector to efficiently obtain latent encoding and reconstructed version of a sample. Once latent embeddings of both training samples (or a representative subset from the training data) and a test sample are obtained, t-SNE is used to map the latent representations of these samples further to the 2-dimensional space (called t-SNE space or map), such that the average distance between the 2D representation of the test sample and its k nearest neighbors from the 2D representations of training samples is calculated. Finally, this distance is combined with the reconstruction error of the test sample to define its anomaly score. The essential parts of this framework are discussed below in details. Full AnoDM approach is given in Algorithm 1 in Appendix E.

heading: 3.1 UNSUPERVISED DISENTANGLED REPRESENTATION LEARNING
text: The unsupervised disentangled representation learning component in our architecture is implemented but not limited by β-VAE (Higgins et al., 2017; Burgess et al., 2018; Hoffman et al., 2017; Mathieu et al., 2019). A self-inclusive description of β-VAE is provided in Appendix B. The objective function to be maximized for β-VAE is defined as (Burgess et al., 2018):
L = Eqφ(z|x)[log pθ(x|z)]− β|DKL(qφ(z|x)||p(z))− C|. (1)
The first term of this objective function corresponds to reconstruction error in the raw feature space. The KL divergence characterizes the discrepancy between approximate posterior and isotropic prior of latent representations. A small discrepancy between them indicates high disentanglement of latent representations in independent variables. C is a hyperparameter which is used to improve the quality of reconstructed images. The loss function of the original β-VAE proposed in (Higgins et al., 2017) does not have this hyperparameter. The value of β trades off reconstruction error and disentanglement. Unlike (Higgins et al., 2017) and (Burgess et al., 2018), we consider β > 0 rather than just β > 1, because it is unnecessary to bound the value of β by 1, β > 0 allows us search for a more appropriate disentanglement. The special case β = 0 could make the model learning very unstable, because the variance of inference distribution loses control. It worth highlighting that other unsupervised disentanglement models can be used as well in AnoDM. For example, Mathieu et al. (2019) interpreted disentanglement as decomposition instead of independence by adding an additional regularization term to reduce the discrepancy between the aggregate posterior and a desired structured
prior. However, designing a properly structured prior could be practically challenging. The adoption of β-VAE in our framework is sufficient to prove the concept that unsupervised disentanglement helps anomaly detection.

heading: 3.2 EFFECTIVITY OF T-SNE ALGORITHM
text: In addition to β-VAE’s reconstruction error, we use the average distance between a test sample and its k-nearest neighbors from the collection of training samples in the t-SNE map to score the outlierness of a test sample. As discussed in Appendix C, t-SNE is significantly influenced by perplexity. The nature of complexity in data distributions makes it impossible to utilize a uniform criteria to define optimal perplexity for all data. Moreover, Wattenberg et al. (2016) mentioned several weaknesses of t-SNE, for examples, (1) it naturally expands dense clusters and contracts sparse ones, evening out cluster sizes, and (2) distances between clusters might not reflect global geometry. However, it is likely that k-nearest neighbors still work for local clumps, because, with proper value of perplexity, local topological information of latent distributions can be preserved by the t-SNE plot. Thus, in t-SNE space, the measure of k-nearest neighbors is more suitable than full density estimation which is very sensitive with the sizes of clusters. Furthermore, as to be shown in Section 4, distancing in the 2D t-SNE map could be more robust than distancing in β-VAE’s latent space where many non-determinative factors may influence the calculation of distances. Finally, it worth clarifying that we do not directly learn a 2D latent representation from β-VAE, because it will bottleneck too much the information flow for reconstruction. Instead, a lower-dimensional representation is learned through t-SNE for density estimation only.

heading: 3.3 DETERMINISTIC OR STOCHASTIC LATENT REPRESENTATIONS FOR T-SNE
text: Either the mean µ or a sample z from the approximate inference distribution q(z|x) can be passed to t-SNE to calculate the k-NN distance of a test sample. There is trivial difference between performances achieved by these two methods in our framework. Generally, the µ-based method achieved slight better performance. The comparison of these two methods can be found in Table 2 in appendices. Furthermore, from the latent representations’ t-SNE maps (see Figure 9 in appendices), one can interestingly see that when β is small (not overly large), the t-SNE maps for both methods are quite similar. As β becomes overly large, some normal classes can still form their own clusters (even though some similar classes, such as classes 3 and 8 in MNIST, tend to mingle together) in µ-based method, but in z-based method all classes are entangled with each other. Same phenomena can be observed on the other datasets (see Figures 10 and 11 in appendices). Therefore, the µ-based method is used in current design of AnoDM.

heading: 3.4 TCN ENCODER FOR UNSUPERVISED SEQUENCE MODELLING
text: Bai et al. (2018) distilled superior design in convolutional network into a simple architecture and referred it as a temporal convolutional network (TCN) with two distinctive characteristics: (1) the convolutions in the architecture are causal, and (2) the architecture can take a sequence of any length and map it to an output vector of fixed length, just as with an RNN. Bai et al. (2018) also explained that TCNs capture significantly longer history than recurrent networks. Inspired by Bai et al. (2018), we replace CNN with TCN in the encoder of β-VAE when evaluating the proposed AnoDM framework on time-series data, while we still use CNN in the decoder, because our preliminary experiments demonstrated that keeping decoder as simpler CNN can help achieve comparable even better results, and take much less computing time. The architecture of TCN used in this paper is depicted in Figure 7 of appendices. In this TCN, we set kernel size to 4 and dilation factors to [1, 2, 4, 8, 16, 32]. In Section 4, the comparison among TCN, CNN, and LSTM encoders in our framework also shows that TCN outperforms CNN and particularly LSTM to a great extent for ECG signal anomaly detection.

heading: 3.5 ANOMALY SCORE FUNCTION IN ANODM
text: In the anomaly detector, the reconstruction error of a test sample in the original feature space and the average distance from its k-nearest-neighbors in training samples within the 2D t-SNE map are
combined as a final anomaly score function:
SβVAE+tSNE(xte) = αDRE(xte) + (1− α)DktSNE(xte), (2) where the first term is defined using normalized squared error (NSE):
DRE(xte) , NSE(xte,x′te) = ‖xte − x′te‖22 ‖xte‖2 , (3)
where xte is a test sample, and x′te is its reconstructed version by sending the stochastic latent encoding through the decoder of β-VAE. The second term in Equation (2) is defined using β-VAE’s deterministic latent encoding (mean from the encoder of learned β-VAE) as input to t-SNE:
DktSNE(x (i) te ) ,
1
k ∑ j∈N(i,k) ‖l(i)te − l (j) tr ‖2, (4)
where l(i)te is the 2D representation of the i-th test sample in t-SNE map, N(i, k) is the set of indices of l(i)te ’s k nearest neighbors from training samples’ 2D representations ltr in t-SNE map. In Equation (2), α ∈ [0, 1] is the combination hyperparameter such that the two terms can effectively complement each other. To allow the anomaly score function to achieve its full potential, α value should be sensitively searched, because the values of DRE and DktSNE can stay at different magnitudes, a very small change of α value may dramatically alter the contributions of these two terms. Alternatively, the distance score in Equation 4 can be normalized by average distance of training samples in t-SNE map, which may alleviate the magnitude difference, thus ease search of optimal value of α. The use of this normalized distance is investigated in Appendix F.

heading: 4 EXPERIMENTS
text: We evaluated our framework on four public image datasets, including MNIST (LeCun et al., 1998), Fashion-MNIST (Xiao et al., 2017), Small-Norb (LeCun et al., 2004) and CIFAR-10 (Krizhevsky, 2009), as well as one collection of ECG heartbeat categorization time-series data named Arrhythmia (Fazeli, 2018). Brief descriptions of these data sets can be found in Appendix G. The detail of β-VAE architecture is given in Table 3 in appendices. The number of epochs was set to 20 for experiments on MNIST and Fashion-MNIST datasets, and 50 for CIFAR-10, Small-Norb, and Arrhythmia datasets; batch size was set to 100 for experiments on all these datasets. When using t-SNE, the dimension of t-SNE map was set as 2 for all datasets, perplexity 30, the learning rate 200, and maximum number of iterations 1000. The value of α in the anomaly score function is searched from set {0.0, 0.005, 0.01, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.95, 0.99, 1.0}. We used k = 1 for calculating k-nearest-neighbors distance in t-SNE map (we also tried to set k to 3 or 5, the results were similar).

heading: 4.1 COMPARISON WITH CAPSNET, GANS, AND VAE
text: We compared AnoDM with state-of-the-art algorithms including a supervised method – CapsNet (Li et al., 2019), and two types of generative models – GANs (including AnoGAN and ADGAN) (Deecke et al., 2019) and β-VAE (implemented by setting α = 1 thus using only reconstruction error as anomaly score). As shown in Table 1, on average, AnoDM achieved either comparable (on MNIST) or better (on Fashion-MNIST and CIFAR-10) performance in terms of receiver operating characteristic curve (auROC). On Fashion-MNIST, CapsNet (prediction-probability-based), as the best benchmark method, obtained an average auROC of 0.765, while AnoDM achieved 0.883. On MNIST, both AnoDM and CapsNet obtained the highest performance. However, CapsNet is a supervised method that takes advantages of class information, while ours is completely unsupervised that is more suitable in many practices as class information is either incomplete or unavailable. Furthermore, by comparing AnoDM with β-VAE that only considers reconstruction error as anomaly score, AnoDM dramatically improved the performance in all cases. In other words, t-SNE makes a prominent contribution to improve β-VAE for anomaly detection problems. However, all generative models did not work well on Small-Norb, mainly because these models used convolution to extract features from image, but convolution is only able to capture translation but not other affine transformations. Although CapsNet learns these transformations as a supervised method, it worth exploring unsupervised learning of affine transformations as a future topic.

heading: 4.2 IMPACT OF BETA TO PERFORMANCE
text: As discussed in Higgins et al. (2017), β (> 1) functions as a controller to encourage most efficient latent representation learning via limiting the capacity of latent information channel. Mathieu et al. (2019) however interpreted the objective of β as tuning a proper level of overlap of encodings by working with another term that regularizes the divergence of the aggregate posterior qφ(z) and the desired prior p(z). The main intuition is that purely increasing β induces too much overlap which actually discourages the disentanglement of data (information which is necessary for expressing desired structure is lost). Higgins et al. (2017) demonstrated that β-VAE with β > 1 leads to interesting results when learning interpretable factorized latent representations on a variety of datasets. Surprisingly, our investigation demonstrates that by setting 0 < β < 1, it actually achieved the state-of-theart results for anomaly detection problems on a range of datasets, such as MNIST, Fashion-MNIST and Arrhythmia. Figure 2 illustrates the impact of values of β and α in our anomaly score function. Interestingly, best performances were achieved when β < 1 and the performances generally degrade as β increases, resonating with Mathieu et al. (2019) that overly large values of β actually causes a mismatch between qφ(z) and p(z) (resulting in inappropriate level of overlap in the latent space). This phenomenon can be further seen in the t-SNE maps of latent embeddings in Figure 3. When β becomes extremely larger than the appropriate value, the anomalous class becomes entangled with its normal neighboring classes and the boundaries between normal classes become unclear. Theoretically, adding the divergence of the aggregate posterior qφ(z) and the desired structured prior p(z) is an effective way to limit the level of overlap when β is too large. However, it is practically challenging to design an appropriate structured prior. Therefore, in our investigation, we focused on exploring the full range of β’s value in β-VAE for the impact of disentanglement to anomaly
detection. Since our framework is quite general, it can be easily extended to other unsupervised disentangled representation learning models for anomaly detection.

heading: 4.3 IMPACT OF BETA TO T-SNE REPRESENTATIONS
text: The t-SNE plots in Figure 3 reflect the impact of β’s value on latent representations in the case of identifying anomalous digit 7 from MNIST. In this example, the best performance (auROC = 0.975) was achieved when β = 0.01. Clearly, as β increases, all latent clusters become less dense, and more anomalous latent data points move to neighboring clusters. Furthermore, Figure 3 also corroborates that, even though in t-SNE maps distances between clusters might not reflect global geometry and cluster sizes might not mirror the true sizes (Wattenberg et al., 2016), using averaged distance from a test sample to its k nearest normal data points represented in t-SNE space to qualify outlierness still is a very effective way for distinguishing anomalous samples when β is tuned properly.

heading: 4.4 EVALUATION OF ANOMALY SCORE FUNCTION
text: In order to better evaluate our anomaly score function, as formulated in Equation (2), we conducted a comprehensive comparison with methods only based on either distance in t-SNE map (DktSNE) or reconstruction-error in raw feature space (DRE). To see the contribution of t-SNE, it is also compared with the method that calculates nearest neighbor distance directly in latent space of β-VAE. Figure 4 displays the ROC curves of these four approaches when assuming anomalous classes are respectively 1 (“Trouser/pants”), 3 (“Dress”), 5 (“Sandal”), and 7 (“Sneaker”) on Fashion-MNIST. It is obvious that AnoDM achieves best results among them by taking advantages of both β-VAE reconstruction and t-SNE embedding. The β-VAE reconstruction reflects whether useful information is captured by the model through recovering the input x; the t-SNE embedding indicates the disentanglement of latent representations z. Both measures effectively complement each other. Besides, comparing the auROCs between t-SNE-based and latent-distance-based score functions, one can clearly see that the former dramatically outperforms the latter. Same conclusion can be drawn for MNIST, CIFAR-10, and Arrhythmia, as displayed in Figures 12, 14, 16, 17, and 18 in appendices. To further show that the optimal values of α are close to 1 (see Figure 2) is due to magnitude difference rather than less usefulness of t-SNE, we replaced the distance score (Equation (4)) with normalized distance score in the weighted final anomaly score function (Equation (2)). We found that the optimal values of α shift to the lower end of the spectrum (see Figure 8). It implies that t-SNE does play a critical role in our framework.

heading: 4.5 ANODM FOR TIME-SERIES
text: As mentioned in Section 3, our method uses a TCN encoder in β-VAE for time-series anomaly detection. Figure 5 displays the comparison among TCN, CNN and LSTM encoders in the AnoDM framework on Arrhythmia. As a special case of LSTM-β-VAE, LSTM-VAE was presented in (Park et al., 2017) for state-of-the-art sequence modelling. For the five classes in Arrhythmia, iteratively one class was treated as anomalous class, while the other classes were used as normal classes. The TCN-encoder-based method outperforms the other two methods significantly in all five cases. Even though the CNN encoder achieved impressive results when detecting anomalous class “S”, “V”, “F” and “Q” respectively, it did not work quite well when class “N” was treated as anomaly. One
possible reason might be that comparing with TCN and LSTM, the performance of CNN is more sensitive on the training sample size. Taking the above case as an example, as class 0 (“N”) accounts for over 80% of training data, when considering it as anomaly, normal training data hence become less sufficient for learning β-VAE. Nevertheless, in TCN-based β-VAE, each hidden unit of the
last deterministic hidden layer before latent encoding at the bottleneck is calculated based on much longer sequence dependency, such that it is less sensitive to the limitation of small sample size. Conclusively, as mentioned in (Bai et al., 2018), TCN should be regarded as a natural starting point for sequence modeling tasks.
As a case study, Figure 6a shows the original EGG signals and reconstructed signals by TCN-based β-VAE when considering class “Q” as anomaly. Normal samples can be reconstructed very well, whereas abnormal samples suffer from larger reconstruction errors. Meanwhile, the corresponding t-SNE plot in Figure 6b displays two distinctive clusters of abnormal samples. The combination of these two measures thus leads to the best performance as seen in Figure 6c.

heading: 5 CONCLUSIONS
text: We propose a new methodology which successfully integrates t-SNE with disentangled representation learning for anomaly detection. This approach achieved state-of-the-art performances on both image data (MNIST, Fashion-MNIST and CIFAR-10) and Arrhythmia time-series data. Specifically, best performance is accomplished when 0 < β < 1 for almost all cases involving β-VAE. We also defined an anomaly score function by effectively taking the advantages of both low-dimensional t-SNE embedding and β-VAE reconstruction. Our algorithm demonstrated that t-SNE plays an essential role for measuring abnormality. This research initiates the research on anomaly detection using unsupervised disentangled representation learning and lower-dimensional manifold learning. Besides, our model uses TCN network as encoding architecture for detecting anomalous time-series data and the experimental results convince us that TCN consistently outperforms CNN and LSTM. As a proof of concept, our current framework automatically inheres advantages of deep learning to address anomaly detection’s issues in representability and scalability as discussed in the beginning of this paper. The extension of our framework to multimodal data is straightforward. It is also possible that a neural t-SNE component could be designed and integrated into the learning of β-VAE to achieve real-time efficiency. Other new well-performing manifold learning methods, such as UMAP (McInnes et al., 2018) which is faster and keeps global topologies, could be employed as replacement of t-SNE.

heading: B BETA-VAE AND BEYOND FOR DISENTANGLED REPRESENTATION LEARNING
text: Higgins et al. (2017) proposed a novel deep generative model, named β-VAE, a modification of VAE by introducing an adjustable hyperparameter β to learn an interpretable disentangled representation of the data generative latent factors. Specifically, β functions as a controller to trade off between the extent of learning constraints and reconstruction accuracy. The constraints impose a limit on the capacity of the latent information channel and an emphasis on learning statistically independent latent factors. Higgins et al. (2017) demonstrated that β-VAE with appropriately tuned β (β > 1) qualitatively outperforms VAE (β = 1) as well as state of the art unsupervised (InfoGAN) and semisupervised (DC-IGN) approaches to disentangled factor learning on a variety of datasets (celebA, faces and chairs).
Higgins et al. (2017) assumed that an image x is generated by the true world simulator using ground truth data generative factors: p(x|v,w) = Sim(v,w), where v is set of conditionally independent factors and w is set of conditionally dependent factors. Therefore, the joint distribution of the data x and a set of generative latent factors z is: p(x|z) ≈ p(x|v,w) = Sim(v,w). The aim of this generative model is then to ensure that the inferred latent factors from qφ(z|x) capture the generative factors v in a disentangled manner. The conditionally dependent data generative factors w can remain entangled in a separate subset of z that is not entangled with v . Considering the prior p(z) is set to be an isotropic unit Gaussian p(z) = N (0, I), a constraint δ is introduced to encourage the matching between qφ(z|x) and p(z) such that the disentangling property in the inferred qφ(z|x) can be realized.
Following the same incentive as in VAE: maximizing the probability of generating real data, while minimizing the distance between the generative and approximate posterior distributions, as formulated below
max φ,θ
Ex∼X [ Eqφ(z|x)[log pθ(x|z)] ] (20)
s.t. DKL ( qφ(z|x)||p(z) ) < δ, (21)
where X = {x1,x2, . . . ,xn} is the training data set. Rewriting this equation as a Lagrangian under the KKT conditions (Kuhn & Tucke, 1951; Karush, 1939), the following equation F(θ, φ, β) is obtained:
F(θ, φ, β) = Eqφ(z|x)[log pθ(x|z)]− β(DKL ( qφ(z|x)||p(z))− δ ) = Eqφ(z|x)[log pθ(x|z)]− βDKL ( qφ(z|x)||p(z) ) − βδ
≥ Eqφ(z|x)[log pθ(x|z)]− βDKL ( qφ(z|x)||p(z) ) . (22)
The objective function to be maximized in β-VAE is thus defined as: Lβ(θ, φ) = Eqφ(z|x)[log pθ(x|z)]− βDKL ( qφ(z|x)||p(z) ) , (23)
where the Lagrangian multiplier β is the regularisation coefficient that constrains the capacity of the latent information channel z and puts implicit independence pressure on the learnt posterior due to the isotropic nature of the Gaussian prior p(z). When β = 1, β-VAE corresponds to the original VAE formulation of Kingma & Welling (2014). When β > 1, it applies stronger constraint which limits the capacity of z and encourages the model to learn the most efficient representation of the
data. Theoretically, a higher β encourages more efficient latent encoding and further encourages the disentanglement. However, a higher β may lead to poorer reconstructions due to the loss of high frequency details when passing through a constrained latent bottleneck.
Burgess et al. (2018) proposed an improvement to β-VAE, by progressively increasing the information capacity of the latent code during training. It facilitates the robust learning of disentangled representations in β-VAE without the previous trade-off in reconstruction accuracy. The objective function to be maximized in β-VAE is redefined as:
Lβ(θ, φ) = Eqφ(z|x)[log pθ(x|z)]− β|DKL(qφ(z|x)||p(z))− C|, (24) where the hyperparameter β controls how heavily to penalise the deviation of DKL ( qφ(z|x)||p(z) ) and a controllable value C. By gradually increasing C from zero to a large value, good-quality reconstruction can be obtained.
Hoffman et al. (2017) had also made a deeper research on β-VAE considering β < 1. They argued that optimizing this partially regularized ELBO is equivalent to performing variational expectation maximization (EM) with an implicit prior r(z) (r(z) ∝ qφ(z)(1−β)p(z)β) that depends on the marginal (aggregate) posterior q(z) , 1N ∑N i=1 q(z|x(i)), and further derived some approximations to examine this prior.
Mathieu et al. (2019) explained that most recent work for learning disentangled representations of data with deep generative models has focused on capturing purely independent factors of variation by employing regularizers explicitly encouraging independence in the representations. They argued that such an approach is not generalisable, and quite restrictive for complex models where the true generative factors are not independent, very large in number, or where a set of true generative factors cannot be well defined. Overly large β is not universally beneficial for disentanglement, Since this in turn causes a mismatch between marginal posterior qφ(z) and the prior p(z). Thus they proposed a generalization of disentanglement in VAE by explicitly separating such a decomposition as two tasks: a) the latent encoding of data should achieved an appropriate level of non-negligible overlap in aggregate encoding qφ(z), and b) the aggregate encoding of data qφ(z) should match the prior p(z) which demonstrates the desired dependency structure between latent variables. By improving the match of qφ(z) and p(z), the overlap is only up to an appropriate level. Mathieu et al. (2019) developed a new objective that incorporates both a) and b) by introducing an additional divergence term D ( qφ(z), p(z) ) . The objective of this improved β-VAE is defined as below:
Lα,β(x) = Eqφ(z|x)[log pθ(x|z)]− βKL(qφ(z|x)||p(z))− αD(qφ(z), p(z)). (25) By appropriately setting β and α, it allows direct control over the level of overlap and the regularization between the marginal posterior and the prior. However, a practical challenge of this method is how to define a proper structured prior when the structure of real hidden factors is poorly known. For this reason, our computational experiment in this paper is based on Burgess et al. (2018)’s β-VAE.
Instead of employing the regularization controller β to disentangle the independent data factors by putting implicit independence pressure in the representation, Hamaguchi et al. (2018) presented a new method to learn disentangled representation by introducing two additional loss functions Lsim (a similarity loss function) and Lact (an activation function). This technique aims to detect trivial events in an image resulting from environmental changes (such as illumination changes, background motions and shadows) by disentangling each image into two kinds of features, specific and common. The functionality of Lsim constrains common features to represent invariant factors between two paired images. Lact encourages activation of common features to avoid a trivial solution. After common features of paired images are separated, the means of common features from two images are fed into event detector (a classifier) for training. As mentioned in Hamaguchi et al. (2018), this method cannot achieve good disentanglement when dealing with complicated scenes, since the activation of the units in the common features are degenerated a certain value.

heading: C T-SNE FOR MANIFOLD LEARNING
text: The t-distributed stochastic neighbor embedding (t-SNE), introduced by van der Maaten & Hinton (2008), is a popular technique for nonlinear dimensionality reduction that is particularly well suited for visualizing similarity of high-dimensional data that lie on several different, but related, lowdimensional manifolds. It is an improvement of stochastic neighbor embedding (SNE) presented
by Hinton & Roweis (2002) with easier optimization and better visualization via replacing the SNE cost function with a symmetrized version and using a Student-t distribution instead of a Gaussian to compute the similarity between two points in the low-dimensional space.
C.1 STOCHASTIC NEIGHBOR EMBEDDING (SNE)
SNE measures similarities (in both original high-dimensional space and mapped low-dimensional space) between points by converting Euclidean distances between data points into conditional probabilities under Gaussian distributions. In original high-dimensional space, for example, the similarity of a pair of data points xi and xj is represented by pj|i under a Gaussian centered at xi. High pj|i means xi and xj are close to each other, and vice versa. pj|i is defined as:
pj|i = exp(−||xi − xj ||2/2σ2i )∑ k 6=i exp(−||xi − xk||2/2σ2i )) , (26)
where σi is the variance of the Gaussian that is centered on data point xi. It is influenced by the perplexity Perp(Pi), which is defined as follows,
Perp(Pi) = 2H(Pi), (27)
where Pi corresponds particular σi and H(Pi) is the Shannon entropy of Pi measured in bits.
For the low-dimensional counterparts yi and yj of the high-dimensional data points xi and xj , the similarity of these two points is denoted as qj|i, and similarly defined as:
qj|i = exp(−||yi − yj ||2)∑ k 6=i exp(−||yi − yk||2) , (28)
where the variance of the Gaussian is set to 1√ 2 . Since the focus is modeling pairwise similarities, both pi|i and qi|i are set to zero.
C.2 COST FUNCTION OF T-SNE
Kullback-Leibler divergence is utilized to measure the difference between the low-dimensional data representation distribution qj|i and high-dimensional data distribution pj|i. The cost function of SNE, which is sum of Kullback-Leibler divergences over all data points, to be optimized by a gradient descent method, is given by
C = ∑ i DKL(Pi||Qi) = ∑ i ∑ j pj|i log pj|i qj|i . (29)
Aiming to alleviate the difficult optimization problem of the cost function above and “crowding problem” (comparing nearby data points, moderate distant data points will not occupy reasonably large area in the low-dimensional map), van der Maaten & Hinton (2008) introduced symmetric SNE and employed a Student t-distribution with one degree of freedom (which is the same as a Cauchy distribution) as the heavy-tailed distribution in the low-dimensional map. The cost function is redefined as:
C = DKL(P ||Q) = ∑ i ∑ j pij log pij qij , (30)
where again, pii and qii are also set to zero, and
pij = exp(−||xi − xj ||2/2σ2)∑ k 6=l exp(−||xk − xl||2/2σ2) , (31) qij = (1 + ||yi − yj ||2)−1∑ k 6=l(1 + ||yk − yl||2)−1 . (32)
The gradient of the Kullback-Leibler divergence between P and the Student-t based joint probability distribution Q is given by
δC δyi = 4 ∑ i (pij − qij)(yi − yj)(1 + ||yi − yj ||2)−1. (33)

heading: D TEMPORAL CONVOLUTIONAL NETWORKS
text: Bai et al. (2018) employed a basic architecture which is essentially same as the time delay neural network proposed by Waibel et al. (1990) to ensure outputs of same length as inputs and no leakage from the future into the past.
TCN = 1D FCN + causal convolutions.
However, since a simple causal convolutions is not able to achieve a long effective history size, Bai et al. (2018) employed dilated causal convolutions (same architecture as WaveNet (van den Oord et al., 2016), please refer to Figure 7a) that enable an exponentially large receptive field. The dilated convolution operation F on element s of the sequence is defined as:
F (s) = (x ∗d f)(s) = k−1∑ i=0 f(i) · xs−d·i, (34)
where x ∈ Rn is a 1-D sequence input, f : 0, . . . , k − 1→ R is a filter, d is the dilation factor, k is the filter size, and s− d · i accounts for the direction of the past. When d = 1, a dilated convolution reduces to a regular convolution. By choosing larger filter sizes k and increasing the dilation factor d, the receptive field of TCN can be enlarged.
Bai et al. (2018) also utilized a generic residual module (He et al., 2015) in place of a convolutional layer. A residual block is defined as:
o = Activation(x + F(x)).
The outputs of a series of transformations F are added to the input x of the block. To tackle with discrepancy of input and output widths in a standard residual block, an additional 1× 1 convolution is used to ensure that element wise addition ⊕ receives tensors of the same shape (see Figure 7b). Bai et al. (2018) further discussed the advantages of TCN (including parallelism, flexible receptive field size, stable gradients, low memory requirement for training and variable length inputs) and its disadvantages (including possibly high memory requirement for evaluation and potential parameter change for a transfer of domain).

heading: E THE FULL ANODM ALGORITHM
text: Algorithm 1: AnoDM Algorithm Result: Anomaly scores of test samples Inputs: Xtr: training samples,Xte: test samples, β > 0: hyperparameter for β-VAE
1 while epoch no more than training iterations do 2 Encoder net mapsXtr into µtr and σtr; 3 Ztr = µtr + σtr , ∼ N (0, I); 4 Decoder net reconstructsXtr toX′tr using Ztr; 5 Update β-VAE’s parameters θ and φ; 6 end 7 ForXtr andXte, obtain µtr and µte respectively using trained β-VAE; 8 Use t-SNE to map µtr and µte to 2D representations ltr and lte; 9 for x(i)te withinXte do
10 DRE(x(i)te ) , NSE(x (i) te ,x ′(i) te ) =
‖x(i)te −x ′(i) te ‖ 2 2
‖x(i)te ‖2 ; // reconstruction error
11 DktSNE(x (i) te ) , 1 k ∑ j∈N(i,k)‖l (i) te − l (j) tr ‖2 ; // N(i, k) is the set of indices of l(i)te ’s
k nearest neighbors from ltr
12 SβVAE+tSNE(x(i)te ) = αDRE + (1− α)DktSNE ; // α ∈ [0, 1] 13 end

heading: F ANOMALY SCORES WITH NORMALIZED K-NN DISTANCE IN T-SNE MAPS
text: Since the normalised reconstruction error in input space and the k-NN distance in t-SNE maps may have very different magnitudes (as mentioned in Section 3.5), some values of α in Equation (2) are close to (but not exactly equal to) 1. The reader may have the intuition that t-SNE is not useful in AnoDM. A simple way, to find out whether large α value is due to magnitude difference or useless of t-SNE, is to replace k-NN distance of a test sample in t-SNE map (Equation (4)) with a k-NN distance normalized by average distance among training samples in the t-SNE map as defined below:
N ktSNE(x (i) te ) ,
DktSNE(x (i) te )
c ∗ DtSNE(xtr) , (35)
DtSNE(xtr) , 1
n ∑ i,j∈{1,2,...,n} ‖x(i)tr − x (j) tr ‖2, (36)
where c > 0 is normalization hyperparameter (we set it to 0.5 in our experiment); xtr is a training sample; n is the total number of training samples in a t-SNE map. The heatmaps in Figure 8 depict performances of AnoDM on Fashion-MNIST (“Dress”class is considered as anomaly) and Arrhythmia (“F” class is treated as anomaly) using normalized k-NN distance in t-SNE maps in combination reconstruction error. By comparing Figure 8 with Figure 2, one can see that the optimal values of α shift upper right corner area to the upper left corner area. It thus implies that, the optimal
value of α is affected by the magnitude difference, and t-SNE indeed plays an essential role in AnoDM.

heading: G DESCRIPTIONS OF DATA SETS
text: • MNIST: It contains a training set of 60000 gray scale digit images of 28 × 28 and a test set of 10000 same resolution gray scale examples from approximately 500 different writers (LeCun et al., 1998).
• Fashion-MNIST: It is a dataset of Zalando’s article images, comprising 70000 MNISTlike labeled fashion images of 28× 28, with 7000 images per category (Xiao et al., 2017). The training set has 60000 images and the test set has 10000 images. The samples come from 10 classes: T-shirt/top, trouser, pullover, dress, coat, sandal, shirt, sneaker, bag, and ankle boot.
• Small-Norb: It contains 24300 96 × 96 grayscale images pairs of 50 toys belonging to 5 generic categories: four-legged animals, human figures, airplanes, trucks, and cars (LeCun et al., 2004). The objects were imaged by two cameras under 6 lighting conditions, 9 elevations , and 18 azimuths. As in (Sabour et al., 2017), the images were resized to 48×48; random 32×32 crops of them were obtained during training process. Central 32×32 patches of test images were used during test.
• CIFAR-10: It consists of 60000 32 × 32 colour images in 10 classes, with 6000 images per class. There are 10000 test images which include exactly 1000 randomly-selected images from each class and 50000 training images (with 5000 images from each class) are randomly grouped into 5 batches (Krizhevsky, 2009).
• Arrhythmia: It is derived from Physionet’s MIT-BIH Arrhythmia Datase which consists of ECG recordings from 47 different subjects recorded at the sampling rate of 360Hz . This dataset is created by five different heart beat categories: “N”, “S”, “V”, “F” and “Q”, in accordance with Association for the Advancement of Medical Instrumentation (AAMI) EC57 standard (Kachuee et al., 2018). The meanings of these categories are explained below.
1. “N”: normal, left or right bundle branch block, atrial escape and nodal escape.
2. “S”: atrial premature, aberrant atrial premature, nodal premature and supra-ventricular premature.
3. “V”: premature ventricular contraction and ventricular escape. 4. “F”: fusion of ventricular and normal. 5. “Q”: paced, fusion of paced and normal and unclassifiable.

Review summaries:
In a paper a new way to compute anomality score (for a test point) is suggested. A paper is purely experimental, based on existing techniques to dimension reduction (beta-VAE and t-SNE). Given trained beta-VAE, latent vectors, obtained for training set, are feed into t-SNE algorithm. The overall anomality score for a test point is combined from 1-NN distances on t-SNE plot and reconstruction error of beta-VNE. 

This paper presents a novel deep anomaly detection model. It combines two existing models: B-VAE and t-SNE. The B-VAE is trained unsupervised and learns an encoder and decoder which provide both an embedding and a reconstruction. Using t-SNE to reduce its dimensionality, the embedding is projected into a 2 dimensional space. An anomaly score function is defined that combines the reconstruction error and the distance in t-SNE space to the K nearest neighbor(s). Experiments are conducted with several image datasets (MNIST,FMNIST,CIFAR10,SmallNORB) and one timeseries dataset (Arrhythmia). For the image sets, the B-VAE model is implemented with a CNN, while for timeseries, a TCN is used. Comparisons are conducted showing the approach to be beat other SOT unsupervised methods, AnoGAN and ADGAN, by 63% and 22% respectively for MNIST and 8% and 2% for FMNIST (in terms of error reduction). For CIFAR-10 and FMNIST it is even demonstrated to beat a supervised SOT method CapsNET. Another experiment shows that t_SNE dramatically improves the performance over B-VAE alone. For the timeseries, the approach is not compared to other SOT approaches as the authors only provide an experiment showing that TCN beats CNN and LSTM for the implementation of the B-VAE. In addition the authors study the effect of the various parameters of the system, in particular the effect of the B in B-VAE and of alpha, the mixing factor between reconstruction error and kNN distance in t_SNE. 3D plots give a good idea on how to select optimal values for the various datasets. The impact of B is also shown on the t-SNE map for MNIST. Finally an ablation studies compares on MNIST the performance of the approach with t-SNE alone, reconstruction alone, and latent distance. On average over 4 digits taken as anomaly, the proposed approach dramatically outperforms the others.