# Complex Background Reconstruction for Novelty Detection
## 1 Introduction  
Novelty detection tackles an important unsupervised learning problem where novelty samples are not known a priori and the majority of the training dataset consists of \"normal\" data [1]. This problem has been widely applied in many areas, including abnormality detection [2, 3], intruder detection [4], biomedical data processing [5], imbalance learning [6], vehicle tracking [7], and specific sign detection [8]. Different from other machine learning tasks, methods for one-class novelty detection are trained on only one class (i.e., the normal class) and aim to determine whether the given sample is a novelty during the inference stage. The challenges associated with anomaly detection tasks encompass several key aspects. (1) Lack of manual supervision: Anomaly detection often operates in an unsupervised or semi-supervised manner, meaning that there is limited availability of labeled data specifically indicating anomalies. This scarcity of labeled data hampers the ability to train models effectively. (2) Limited novelty samples: Novelty or anomaly samples are typically rare and occur infrequently in real-world data. The small number of available anomaly samples makes it challenging for models to learn and generalize effectively. (3) Imbalanced training data: In most cases, the training dataset consists primarily of instances from the normal class, making it imbalanced. This imbalance can lead to model bias towards the majority class and the poor detection of anomalies. (4) Unknown anomaly patterns: Anomalies can take various forms, and their characteristics may not be well-defined or known in advance. This uncertainty about the nature and structure of anomalies adds complexity to the task. In recent years, novelty detection tasks have received extensive attention. According to the recent literature, existing models for novelty detection can be generally divided into twobranches. The first branch is to extract latent features of an image as the input to traditional novelty detection algorithms such as one-class SVM [9]. Another one is to reconstruct the image using the deep generative networks [10], which are trained on samples from the normal class, and use the reconstruction error as a measure for novelty sample detection. The reconstruction approach using deep generative networks focuses on learning potential information for the retaining of normal samples, but, for the novelty samples, it will have large reconstruction errors. In this strategy, existing methods improve the performance by making image reconstruction better and choosing more appropriate distance metrics. Most of the existing methods ignore the effects of the image background. Specifically, the one-class novelty detection results for the CIFAR10 dataset are comparatively weaker for all methods. A complex background can affect the result of image reconstruction, which will affect the choice of distance metrics. However, background information can also help in object detection. In natural surroundings, the foreground and the background are strongly correlated. For instance, the sky background naturally correlates with birds and airplanes with a higher probability, whereas the ground background relates to terrestrial animals and vehicles. Hence, the background, especially a complex one, subtly indicates certain foreground elements, with distinctive backgrounds potentially indicating novel samples. Furthermore, owing to the expansive and non-discriminatory nature of neural networks, background information and foreground information exhibit a certain diffusion effect, resulting in the inclusion of some foreground information within the background. Hence, this paper proposes a method for complex background reconstruction, thereby facilitating the detection of novel targets. Inspired by this, we offer improvements from the following aspects. First, we use the Maximum Mean Discrepancy (MMD) to make the distribution of reconstructed samples similar to that of the samples in the training set.