# Complex Background Reconstruction for Novelty Detection
## 5 Experiment
### Ablation Study  
In order to investigate the effectiveness of each subnetwork of our method, we conduct several further ablation studies. \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline
**Normal Class** & **OC-SVM** & **KDE** & **VAE** & **GAN** & **AnoGAN** & **DSVDD** & **OCGAN** & **Ours** & **Ours** \\  & & & & & & & & **(MSE)** & **(MNet)** \\ \hline AIRPLANE & 61.6 & 61.2 & 70.0 & 70.8 & 67.1 & 61.7 & **75.7** & **89.6** & **86.7** \\ AUTMOBILE & 63.8 & **64.0** & 38.6 & 45.8 & 54.7 & **65.9** & 53.3 & 47.3 & **70.5** \\ BIRD & 50.0 & 50.1 & **67.9** & **66.4** & 52.9 & 50.8 & 64.0 & **71.5** & 63.4 \\ CAT & 55.9 & 56.4 & 53.5 & 51.0 & 54.5 & 59.1 & **62.0** & **62.1** & **60.1** \\ DEER & 66.0 & 66.2 & **74.8** & 72.2 & 65.1 & 60.9 & 72.3 & **83.5** & **78.4** \\ DOG & **62.4** & **62.4** & 52.3 & 50.5 & 60.3 & **65.7** & 62.0 & 61.2 & **64.7** \\ FROG & 74.7 & **74.9** & 68.7 & 70.7 & 58.5 & 67.7 & 72.3 & **88.8** & **86.2** \\ HORSE & **62.6** & **62.6** & 49.3 & 47.1 & 62.5 & **67.3** & 57.5 & 56.0 & **65.7** \\ SHIP & 74.9 & 75.1 & 69.6 & 71.3 & 75.8 & 75.9 & **82.0** & **89.5** & **82.4** \\ TRUCK & **75.9** & **76.0** & 38.6 & 45.8 & 66.5 & 73.1 & 55.4 & 50.8 & **76.8** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Average AUCs in % one-class novelty detection for CIFAR-10 dataset. The top three results are marked in **bold**. \begin{table}
\begin{tabular}{l c} \hline \hline
**Experiment** & **Average AUC \%** \\ \hline (a) \(\text{RNet}+\text{self-looping}\) & 70.03 \\ (b) \(\text{MNet}+\text{self-looping}\) & 68.41 \\ (c) \(\text{RNet}+\text{MNet}\) & 71.73 \\ (d) \(\text{RNet}+\text{MNet}+\text{self-looping}\) & 73.48 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Ablation experiment for our method performed on CIFAR-10. \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline  & **CoP** & **REAPER** & **OutlierPursuit** & **LRR** & **DPCP** & **R-Graph** & **ALOCC** & **Ours** & **Ours** & **Ours (MNet)** \\ \hline AUC\({}_{1}\) & 90.5 & 81.6 & 83.7 & 90.7 & 78.3 & **94.8** & **94.2** & **94.2** & **94.1** \\ AUC\({}_{3}\) & 67.6 & 79.6 & 78.8 & 47.9 & 79.8 & 92.9 & **93.8** & **93.8** & **94.1** \\ AUC\({}_{5}\) & 48.7 & 65.7 & 6  
**How important is MNet? ** We firstly use MSE instead of MNet to measure the reconstruction error. The penultimate columns of Tables 1 and 2(a) show the results of the method without MNet. The result shown in Table 2(a) is also slightly greater than the one of OCGAN but is inferior to ours. We conjecture that the reason is that the metric network extracts discriminative characteristics of the target, especially for objects with high complexity. However, the comparison between the last two columns of Table 1 indicates that MSE is superior to MNet when some classes (e.g., deer, birds) are selected as the normal classes and MNet performs better when others (e.g., car, truck) are. We conduct an analysis on this result. First, we calculate the variance of each sample by class. The variance represents the dispersion level of the data. The larger the variance, the more dispersed the data distribution is. As can be seen from Tables 1 and 4, for the three classes with the most discrete image distribution, TRUCK, AUTOMOBILE, and HORSE, our method with the MNet function outperforms the state-of-the-art methods. However, the results obtained using the MSE function are reduced by 33.2%, 28.2%, and 16.8% compared to the state-of-the-art methods. At the same time, for the other seven classes with a relatively concentrated image distribution, our method can achieve very good results by applying both the MSE function and MNet function. Therefore, compared with the MSE function, our proposed MNet function can more stably achieve excellent results on different discrete-level datasets. Some different discrete-level samples are shown in Figure 4.