# Complex Background Reconstruction for Novelty Detection
## 2 Related Work  
In this section, we review the related work on novelty detection, MMD, and metric learning. **Novelty Detection:** The one-class novelty detection task has attracted considerable attention in recent years. With the success of neural networks and deep learning, great progress has been made in novelty detection. According to the recent literature, approaches to novelty detection can be generally divided into two branches. One is to extract the latent features of the image as the input to a traditional anomaly detection algorithm such as one-class SVM (OC-SVM) [12]. [PERSON] [13] and [PERSON] and [PERSON] [14] propose to leverage the mean squared error to conduct novelty detection. [PERSON] et al. [15] leverage a self-adaptive and self-organizing paradigm for novelty detection. Deep One-Class Classification [16] jointly trains a deep neural network while optimizing a data-enclosing hypersphere in the output space, which firstly introduces the fully deep one-class classification objective for unsupervised anomaly detection. Another approach is to reconstruct the image using a deep generative network. Since generative adversial networks (GANs) [17] have shown a strong ability to extract deep features, several works apply GANs in novelty detection. AnoGAN [2] hypothesizes that the latent vector mapped from the input of the GAN represents the high-dimensional distribution of the data. The detection process is based on the reconstruction error and the error between the intermediate discriminator feature of the test image and the reconstructed image. Meanwhile, in ADGAN [18], which shares a similar framework with AnoGAN, the network is optimized based on both the latent vector and the generator, and only the reconstruction error is considered as the anomaly score while inferencing. [PERSON] et al. [1] attempt to de-noise noisy samples of the given class, and the discriminator's prediction in the image space is used to quantify the reconstruction error. [PERSON] et al. [19] propose an augmented time-regularized generative adversarial network to generate effective artificial samples for novelty detection. [PERSON] et al. [20] leverage an encoder-decoder reconstruction network and a CNN-based discrimination network to recognize noisy novel samples. [PERSON] et al. [10] exploit an isometric adversarial auto-encoder to obtain stable detection results. Figure 1: The result of our image reconstruction is compared with that of other methods. The left side shows normal samples (automobile) and the right side shows novel samples. For normal samples, our method has a better reconstruction effect than other methods (using \(L2\) loss) on the whole image, because it not only learns the features of the target as in the previous method, but also the distribution of the background. Reconstructing background information can reduce the interference of the background information in reconstructed targets. Nevertheless, the aforementioned approaches primarily concentrate on extracting and generating features from the foreground, neglecting the significant correlation that it shares with the background, especially for complex background cases. These methods depend solely on foreground information to detect novel targets. Highlighting the aforementioned concerns, this paper posits that complex backgrounds, with their implicit association with the foreground, can be leveraged to enable the detection of novel targets. By integrating both the foreground and background, the accuracy of novelty detection can be significantly enhanced. **MMD**: MMD (Maximum Mean Discrepancy) is a statistical hypothesis aiming to measure the distance between two probability distributions. [PERSON] et al. [21] first proposed the criterion to replace the hard minimax objective function used in generative adversarial network training. The generative moment matching network (GMMN) [22; 23] is a generative model that replaces the discriminator in the GAN with a two-sample test based on kernel MMD.