# Autoencoder-Based Semantic Novelty Detection: Towards Dependable AI-Based Systems
## 4 Research Hypothesis of This Work  
To realize the previously mentioned task of quantitative monitoring, we propose an autoencoder-based approach of semantic novelty detection during real-time system operation to classify a current situation into (a) tested and known or (b) novel and unknown. In the case of (b), the corresponding raw data will be recorded for further development of the monitored autonomous system. Moreover, the classification may also lead to a (a) safe or (b) unsafe state decision in the system and a corresponding fail-operational reaction. It can be concluded that from the viewpoint of safety it is important to detect all novel situations. Otherwise, the system decides to operate in an unlearned environment, which could lead to a safety-critical situation. In Section 3, an autoencoder-based novelty detection approach by [PERSON] et al. [9] was briefly introduced, in which, as compared to the images of the MNIST handwritten digits dataset [12], the images of letters with nonstandard fonts from the notMNIST-dataset [30] were successfully classified as novel. To compare our results with their approach, we reproduced the results of [PERSON] et al., as described in Section 5. We focused particularly on their example of detecting letter images as novel. Our reproduced results showed that [PERSON]'s approach works well in identifying novel data that have different input data structures as compared with the MNIST-dataset, which is used to train the autoencoder. In this paper, we argue that their success depends on major structural differences in the input data, i.e., images of digits and letters. Therefore, we modified the experiment of [PERSON] et al. such that we randomly excluded some classes of digits from the training set of the autoencoder to determine whether the autoencoder could still detect these unknown digits as novel. Therefore, in Section 5, we present a slightly modified dataset, containing seven classes of digits for training, and the other three classes of digits as well as nine classes of letters remain as novel data. We expect that [PERSON]'s approach will fail once the novel data have a minor structural, but a major semantic difference. Consequently, images will be detected as known instead of novel, producing a high rate of false negative errors. In the subsequent section, [PERSON]'s approach is taken as a reference and referred to as the \"naive\" autoencoder-based novelty detection, for which failure regarding false negative errors is further proven in Section 6. Additionally, to compare the autoencoder-based approach, [PERSON]'s, as well as our improved approach, with other AI-based approaches for novelty detection, a normal classification neural network for novelty detection is introduced in Section 5, which is named as a simple classification approach in this study and serves as a lower bound benchmark solution in our evaluation. To minimize the false negative errors, we improve autoencoder-based novelty detection in two steps. The first step is based on a semantic topology, which is called semi-semantic novelty detection. In this approach, the semantics of the monitored system's output, which is a classification system, is considered. We force the autoencoder to reflect the semantics in its topology by applying three rules (cf. Section 7). In the second step, to realize a so-called fully semantic novelty detection, we reuse the topology of semi-semantic novelty detection, but the novelty detection criterion is replaced by a semantic loss instead of the original mean squared error (MSE). With this, our research hypothesis (HYPO) for upcoming sections are as follows:  
**Hypothesis 1 (H1). **_\"Naive\" autoencoder-based novelty detection does not work well for similar structural input data with a different semantic interpretation._  
**Hypothesis 2 (H2). **_Autoencoder-based semi-semantic novelty detection works better than the \"naive\" approach._  
**Hypothesis 3 (H3).