# Autoencoder-Based Semantic Novelty Detection: Towards Dependable AI-Based Systems
## 3 State of the Art in Novelty Detection  
Novelty detection as a classification task to determine whether newly observed data differ from the available training and test data has been widely used in different fields. Different approaches of novelty detection have been investigated [7; 21; 22]. Novelty detection is closely related to anomaly detection, which typically refers to the detection of  
Figure 3: Continuous monitoring architecture as a part of dependability cage.  
samples during inference that do not conform to an expected normal behavior [23]. If enough diverse training data is available any anomaly is a novelty, but not any novelty is necessarily an anomaly. Hence, novelty detection is the more general and safer approach for us. Note that automotive novelty detection is also known as corner case detection.  
One well-known classification of novelty detection approaches was proposed in [7], consisting of five taxonomies, i.e., probabilistic, distance-based, domain-based, reconstruction-based, and information theoretic approaches. In our work, we focus on the taxonomy of reconstruction-based approaches, which are often used on safety-critical applications and cover approaches based on diverse neural networks (NNs) such as MLP, SVM, and auto-associator, as well as oscillatory and habituation-based networks [24].  
Auto-associator approaches, particularly referring to autoencoder approaches, have gained our attention in the research because of their strong ability to implicitly learn the underlying characteristics of an input dataset without any prior knowledge or assumptions [24]. Autoencoder approaches automatically learn data abstraction on different implicit representation levels by attempting to regenerate the system output as the same as the originally given input, such as camera images, which have been investigated in different studies [25; 26; 27].  
For example, [PERSON] et al. identified three critical properties of autoencoder-based novelty detection: (a) production of the same output vector with infinite input vectors, (b) output-constrained hyperplane when using bounded activation functions, and (c) hyperplane located in the vicinity of training pattern by the minimizing error function [28]. [PERSON] et al. found that kernel regression mapping is better suited than least squares mapping for autoencoder networks of novelty detection [25]. [PERSON] et al. [8] focused on the idea of setting the threshold as some function of the largest reconstruction error for any single training data point, and based on this, [PERSON] et al. [9] selected a high percentile of the cumulative distribution of losses on training datasets as their threshold to make it more robust in detecting novel instances with different input data structures. Their experimental setting--handwritten digits versus (novel) letter images--serves as starting point for our work. To improve the criterion of novelty detection, [PERSON] et al. investigated autoencoders on different levels of error and determined an optimal real multiple of the standard deviation of the average reconstruction error as a better threshold. Their evaluation was based on an application for document classification [29].  
As already introduced in the previous sections, the aim of this study is to realize a quantitative monitor as part of the dependability cage by using a new improved autoencoder-based approach. The autoencoder-based approach by [PERSON] et al. [9] is used as a reference in our study, for which details are introduced in Sections 5-8. By introducing a new semantic novelty detection approach using a new autoencoder architecture, we will improve the results of [PERSON] et al.