# Extended Autoencoder for Novelty Detection with Reconstruction along Projection Pathway
## 3 Preliminaries of RaPP
### Results  
In this section, our extended autoencoders for AE, VAE, and AAE are referred to as XAE, XVAE, and XAAE, respectively. Table 2 summarizes the RaPP performance of the standard autoencoders and our extended autoencoders trained for 100 epochs with novelty datasets. In Table 2, the best score for each dataset is indicated with underlining. The extended autoencoders provided the best results for all datasets except the MI-F and MI-V datasets. Given that MI-F and MI-V are derived from the same dataset, the extended autoencoder achieved the best results among six of the seven datasets. Thus, this empirical result demonstrates that the extended autoencoder is generally suitable for detecting novelties using the RaPP method. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Standard Autoencoders**} & \multicolumn{3}{c}{**Extended Autoencoders**} \\ \cline{2-7}
**Dataset** & **AE** & **VAE** & **AAE** & **XAE** & **XVAE** & **XAAE** \\ \hline MI-F & 0.643 & 0.655 & 0.639 & 0.637 & 0.653 & 0.646 \\  & (0.048) & (0.066) & (0.052) & (0.058) & (0.060) & (0.050) \\ MI-V & 0.902 & 0.901 & 0.896 & 0.890 & 0.896 & 0.894 \\  & (0.012) & (0.016) & (0.013) & (0.015) & (0.017) & (0.010) \\ EOPT & 0.629 & 0.593 & 0.628 & 0.630 & 0.597 & 0.631 \\  & (0.018) & (0.007) & (0.018) & (0.019) & (0.006) & (0.023) \\ NASA & 0.717 & 0.723 & 0.719 & 0.729 & 0.735 & 0.727 \\  & (0.031) & (0.027) & (0.025) & (0.028) & (0.027) & (0.024) \\ RARM & 0.647 & 0.651 & 0.640 & 0.656 & 0.648 & 0.673 \\  & (0.072) & (0.046) & (0.065) & (0.050) & (0.051) & (0.056) \\ STL & 0.819 & 0.818 & 0.817 & 0.824 & 0.821 & 0.821 \\  & (0.053) & (0.051) & (0.058) & (0.049) & (0.049) & (0.056) \\ OTTO & 0.829 & 0.838 & 0.831 & 0.833 & 0.838 & 0.829 \\  & (0.010) & (0.012) & (0.011) & (0.011) & (0.011) & (0.012) \\ SNSR & 0.988 & 0.988 & 0.989 & 0.988 & 0.989 & 0.990 \\  & (0.003) & (0.003) & (0.003) & (0.003) & (0.003) & (0.003) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The AUROC (area under the receiver operating characteristic curve) of RaPP with novelty datasets. The best cases are underlined for each dataset, and the number of in parenthesis indicates the standard deviation. However, note that these results do not include the best cases since the hyperparameters including the number of hidden layers has not been optimized for each dataset. In this respect, we have investigated the performance of RaPP by varying the number of hidden layers with MI-F and MI-V datasets, and the results are presented in Tables 3 and 4. The extended autoencoder denoted as XAE achieves the best score when the number of hidden layers is set to 2 on MI-F and 6 on MI-V, respectively. Table 5 summarizes the RaPP performance with the MNIST and F-MNIST image datasets. The extended autoencoders required only two epochs to achieve performance similar to that of standard autoencoders, which are trained for 100 epochs. For the F-MNIST dataset, the extended AAE (trained for only two epochs) outperformed all standard autoencoders trained for 100 epochs, and for the MNIST dataset, the proposed model showed almost the same performance as that of standard autoencoders. Of course, since the execution time of a single epoch can vary depending on autoencoder methods, it is necessary to check the time. Figure 7 shows the execution time in seconds of a single training epoch. As seen in the figure, during a single epoch, each of the extended autoencoders take approximately twice as long as its corresponding standard autoencoder; in other words, the time required for two epochs of the extended autoencoder is almost the same as the time required for four epochs of the standard autoencoder. After all, in the training process, the extended autoencoders outperform the standard autoencoders within a short training time for both MNIST and F-MNIST datasets.