# Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering
## 4 Experimental Study
### Experimental Result  
The experiment was carried out on a Thinkpad W510, 1 st gen i7 and 8 GB Ram. First, we have made an experiment on 20 benchmark datasets to compare Algorithm 1 and Algorithm 3. As a result of applying Algorithm 1, we get 2 dimensional features consisting of the reduced dimension and the reconstruction errors as well as a finalThreshold which represents the novelty threshold. If algorithm 1 is used without algorithm 2, each point that is exceeding the finalThreshold will be defined as a novelty. However, the synergy of using both algorithm 1 and algorithm 2 will result in grouping points that exceed the finalThreshold as a novelty group. Table 2 shows that the AUC score of noveltydetection is growing when using Algorithm 1 and Algorithm 2 together (Algorithm 3) instead of using Algorithm 1 alone. Second, we have made experiments on a total of 20 benchmark datasets to compare our proposed algorithms with state-of-the-art algorithms. Extensions of RapidMiner, which is proposed by OC-SVM, eta-SVM, Robust SVM and Robust PCA Mennatallah et al. was used and other algorithms were implemented in Python with Keras, which is a high-level neural networks API, written in Python and capable of running on top of TensorFlow. The Deep autoencoder model is set to match the same for DAE-DBC and DAE-Kmeans algorithms. Deep autoencoder was trained by the Adam algorithm [38] and learning rate was 0.001 to minimize mean squared error. Batch size was 2 and number of epochs to train model was 50. In implementation of algorithm 1, original CSV data file is loaded into DataFrame in pandas Python package. DataFrame is a 2-dimensional labeled data structure with columns of potentially different types. After building first autoencoder model, decoded output result and reconstruction errors are stored in one-dimensional arrays. The data with a lower reconstruction error is stored into new DataFrame. Train the 2 nd autoencoder model by the data obtained and the final compressed data including reconstruction errors and low dimensional representation from second AE model is written into CSV file. In the implementation of algorithm 2, the compressed data file is loaded into the DataFrame. Clusters' labels and distances between points in the compressed representation are stored in one-dimensional arrays. Also, the count of the novelty in each cluster is stored in a dictionary data structure. The final novelty labels will be appended into the original CSV data file. Experimental results show that we can use the DAE-DBC method on various dimensions of data due to the fact that it has given higher AUC than other methods on most of the data. A total of 13 methods were tested on 20 benchmark datasets, the highest values of AUC scores are marked in bold in Table 3. The first is the DAE-DBC algorithm 0.7829, the second is the PCA-DBC algorithm 0.6924, the third is OC-SVM algorithm 0.6865 by the average AUC score are higher than other algorithms. The proposed DAE-DBC method shows the AUC score of less than 0.8 on Vertebral, Optdigits, Letter, Arrhythmia and Annthyroid datasets. Comparing the above datasets with the datasets shown in the high AUC score, the average values between some of the features have high variance. Testing \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **Algorithm 1** & **Algorithm 3** \\ \hline Annthyroid & 0.5866 & **0.6308** \\ Breastw & 0.8962 & **0.9767** \\ Arrhythmia & **0.9181** & 0.6966 \\ Cardio & 0.7837 & **0.8661** \\ Glass & 0.4580 & **0.7195** \\ Ionosphere & 0.7440 & **0.7813** \\ Letter & 0.5957 & **0.6030** \\ Lympho & 0.7500 & **0.8709** \\ Mnist & 0.7315 & **0.7358** \\ Musk & 0.9266 & **0.9978** \\ Optdigits & **0.6108** & 0.4972 \\ Pendigits & 0.7861 & **0.8466** \\ Pima & 0.6423 & **0.6758** \\ Satellite & 0.6696 & **0.7299** \\ Satimage & 0.9204 & **0.9861** \\ Suttle & 0.9427 & **0.9695** \\ Thyroid & **0.9554** & 0.9077 \\ Vertebral & **0.4000** & 0.5000 \\ Wbc & **0.8221** & 0.8179 \\ Wine & **0.8487** & **0.8487** \\ \hline \hline \end{tabular}
\end{table}