## 2 Background
### Deep Anomaly Detection  
Detecting anomalies in data has been a major scientific concern for over 100 years; research in _outlier detection_ in the statistical community dates back to the 19 th century. In the modern world, awash in data from innumerable sensors, anomaly detection is a common need in a wide variety of major systems and infrastructure. Per [PERSON] et al. [5], AD algorithms can be classified into three categories: supervised, semi-supervised, and unsupervised. The _supervised_ algorithms resemble general supervised classification algorithms; examples of normal and anomalous behavior are collected, and a discriminative model for them is learned. As classifiers assume that the normal and anomalous classes both map to regions in the extracted feature space, a large amount of anomalous data is needed to adequately cover this classification region. In reality, however, anomalous data are often difficult or expensive to come by, and such data are almost always scarcer than observations of normal behavior. This also means that a class imbalance problem further complicates training the AD. In datasets with very complex behaviors, it may become extremely difficult to adequately separate normal and anomalous instances. _Semi-supervised_ AD algorithms, on the other hand, are distinct from general semi-supervised learning. This class of algorithms assumes that _only_ normal examples are available in the training data, due to the rarity or expense of observing anomalies. A semi-supervised AD is thus trained to model normal behavior, and deviations from that model are considered anomalies. One-class classifiers are commonly used in this class of problems. Finally, _unsupervised_ anomaly detection presumes that no labels are available at all. An assumption is made that anomalies are relatively rare, and so models that capture the great majority of data points should represent normal behavior. Deviations from these models are again declared to be anomalies.  
The type of anomaly to be detected also varies. The greatest volume of literature focuses on _point_ anomalies, for which values are highly distinct from normal behavior. However, if \"normal\" behavior is not a constant phenomenon (as commonly happens in a time series), one also encounters _contextual_ anomalies, values that are within the overall range of normal behaviors but are significantly different then observations around that particular time [5].  
In the past several years, deep learning has been proposed for AD. Deep Learning (DL) methods have the ability to learn extremely complex tasks, through several levels of representation acquired via individually simple non-linear modules. Moving deeper into the model, each representation is thus at a higher level of abstraction. Studies also indicate that DL is superior to \"shallow\" learning algorithms (decision trees, support vector machines, etc.) in time-series forecasting [8; 9] and modeling sequential data [10]. For these reasons, DL seems to be a good match for the normal model. Some studies of deep AD approaches have illustrated a considerable superiority over traditional methods in a range of real-world problems [1; 7]. However, recent investigations show that deep neural networks are trained to be interpolators [11] and perform poorly on out-of-distribution data [12].