# Complex Background Reconstruction for Novelty Detection
## 3 Methodology
### Reconstruction Network (RNet)  
In previous work [14], samples were typically reconstructed using auto-encoders and by generating normal samples against network training. The essential goal of reconstruction is to classify different object categories by calculating the reconstruction error between \(x\) and \(\hat{x}\). By reviewing the previous work [28], we find that most of the methods do not perform well on some datasets (e.g., CIFAR-10). There exist studies showing that making the distribution of real data similar to that of reconstructed data can benefit image generation [11]. Inspired by these studies, we still adopt the strategy of deep reconstruction networks, but we replace the discriminator in GAN with a two-sample test based on MMD. Specifically, our RNet consists of three subnetworks: (1) the encoding subnetwork \(E\) to map the image \(x\) to the latent representation \(z\), (2) the reconstruction subnetwork \(G\) to reconstruct \(\hat{x}\) by decoding \(z\), and (3) the discrimination subnetwork \(D\). In the reconstruction network, we employ a classical GAN architecture to extract background features. In the encoder part, we utilize the Maximum Mean Discrepancy (MMD) as a measure of dissimilarity between the original and generated images. This choice is motivated by the fact that, compared to other distance measures between distributions, such as the Kullback-Leibler (KL) divergence, which either require density estimation (either parametric or nonparametric) or space partitioning/bias correction strategies, MMD can be easily estimated as an empirical mean that converges to the true value of the MMD. Thus, MMD exhibits superior performance in the scenario of unknown image distributions. Moreover, MMD is equivalent to finding the Reproducing Kernel Hilbert Space (RKHS) function that maximizes the expectation difference between the two probability distributions, indicating its strong solution coherence. In the reconstruction part, we utilize the 1-norm to promote sparse features. **Adversarial Loss:** Following the trend in current novelty detection methods [2], we use the feature distribution loss for adversarial learning. Studies have shown that feature matching loss can reduce the instability of GAN training [29]. MMD distance was first proposed for the two-sample test problem, which aims to determine whether two given distributions are the same. It uses kernel embedding \(\phi(\mathbf{x})=k(\cdot,\mathbf{x})\) associated with a characteristic kernel \(k\), where \(\phi\) is infinite-dimensionaland \(\left\langle\phi(\mathbf{x}),\phi(\mathbf{y})\right\rangle_{\mathcal{H}}=k(\mathbf{x},\mathbf{y})\). Given two distributions \(\mathbb{P}\) and \(\mathbb{Q}\), and a kernel \(k\), the squared MMD distance is defined as  
\[M_{k}^{2}(\mathbb{P},\mathbb{Q})=\left\|\mathbf{\mu}_{\mathbb{P}}- \mathbf{\mu}_{\mathbb{Q}}\right\|_{\mathcal{H}}^{2}=\mathbb{E}_{\mathbf{x},\mathbf{x}^{ \prime}\sim\mathbb{P}}\big{[}k(\mathbf{x},\mathbf{x}^{\prime})\big{]}\\ +\mathbb{E}_{\mathbf{y},\mathbf{y}^{\prime}\sim\mathbb{Q}}\big{[}k(\mathbf{y},\mathbf{y}^{\prime})\big{]}-2\mathbb{E}_{\mathbf{x}\sim\mathbb{P},\mathbf{y}\sim\mathbb{ Q}}[k(\mathbf{x},\mathbf{y})]. \tag{1}\]  
where \(\mu\) and \(\mathbb{E}[\cdot]\) denote the mean value of the distribution and the expectation of the kernel function, respectively. The kernel \(k(\mathbf{x},\mathbf{y})\) measures the similarity between two samples \(\mathbf{x}\) and \(\mathbf{y}\). \(\mathbf{x}\) and \(\mathbf{x}^{\prime}\) represent two random variables subjected to \(\mathbb{P}\), and \(\mathbf{y}\) and \(\mathbf{y}^{\prime}\) represent random variables subjected to \(\mathbb{Q}\). With respect to the square of the MMD distance, we have the theorem below. **Theorem 1**. : _Given a kernel \(k\), if \(k\) is a characteristic kernel, then \(M_{k}^{2}(\mathbb{P},\mathbb{Q})=0\) if \(\mathbb{P}=\mathbb{Q}\)._  
Theorem 1[30] shows that the more similar the two distributions are, the smaller the MMD distance between them is.