# Deep Morphological Anomaly Detection Based on Angular Margin Loss
## 5 Experimental Results and Discussion
### Statistical Analysis of the Results Using the Wilcoxon Signed-Rank Test  
To analyze the experimental results by a statistical method, we performed the Wilcoxon signed-rank test and reported it in Table 4. The Wilcoxon signed-rank test is a popular nonparametric statistical test for matched or paired data. This statistical test is based on both difference scores and the magnitude of the observed differences. In this paper, we define the difference score as follows:  
\[\text{difference score}=AUROC_{i}^{\text{related}}-AUROC_{i}^{\text{proposed}},\]  
where \(AUROC_{i}^{\text{related}}\) and \(AUROC_{i}^{\text{proposed}}\) are AUROC values of the compared algorithm and the proposed algorithm on \(i\)th class in the certain dataset. In Wilcoxon signed-rank test, the hypotheses are concerned by the population median of the difference scores. In this paper, we consider a one-sided test which is as follows:  
**Hypothesis 1 (H1)**.: _The median difference is zero._  
**Hypothesis 2 (H2)**.: _The median difference is negative._  
The test static for the Wilcoxon signed-rank test \(W\) is defined as the smaller value among \(W_{+}\) (the sum of the positive ranks) and \(W_{-}\) (the sum of the negative ranks). To determine whether the observed test static \(W\) supports H1 or H2, we have to choose the \(W_{\text{critical}}\). Therefore, if the \(W\) is less than \(W_{\text{critical}}\), we reject H1, in favor of H2. In contrast, if the \(W\) exceeds the \(W_{\text{critical}}\), we do not reject H1. In this paper, to set the appropriate \(W_{\text{critical}}\), we choose the level of significance \(\alpha\) to 0.05, which is the most common value for the Wilcoxon signed-rank test.  
For the MVTec dataset, the proposed algorithm yields statistically highest performance than the other algorithms. Moreover, in MNIST and Fashion-MNIST, the proposed algorithm has better performance than several algorithms. Because MNIST and Fashion-MNIST are relatively easier datasets than the MVTec dataset, it exhibits a similar performance than [33]. However, notably, the proposed algorithm is developed to address the morphological anomaly detection problem. Therefore, in the Wilcoxon signed-rank test results of the MVTec dataset, it is clearly demonstrated that the proposed algorithm achieved significantly higher performance than [33] in the morphological anomaly detection task.  
\begin{table}
\begin{tabular}{c c c c c c} \hline
**Class** & **[PERSON] et al. [11]** & **[PERSON] et al. [35]** & **[PERSON] et al. [13]** & **[PERSON] et al. [33]** & **Proposed** \\ \hline T-shirt & 82.4 (0.5) & 91.0 (0.2) & 86.6 (1.3) & **89.6 (0.2)** & 88.4 (0.1) \\ Trouser & 95.7 (0.7) & 98.0 (0.1) & 97.8 (0.4) & 98.4 (0.3) & **98.6 (0.3)** \\ Pullover & 81.2 (0.9) & 86.2 (0.3) & 84.0 (0.8) & 87.5 (0.2) & **88.2 (0.2)** \\ Dress & 90.4 (0.6) & 91.9 (0.2) & 93.0 (0.8) & 92.7 (0.4) & **92.8 (0.2)** \\ Coat & 86.1 (0.8) & 87.4 (0.1) & 88.5 (0.5) & **89.9 (0.3)** & 87.9 (0.1) \\ Sandal & 86.6 (1.1) & 90.8 (0.4) & 91.8 (1.5) & 90.6 (0.5) & **92.3 (0.4)** \\ Shirt & 79.8 (0.5) & 80.3 (0.8) & 81.5 (0.6) & 83.0 (0.6) & **84.0 (0.2)** \\ Sneaker & 93.3 (0.2) & 97.8 (0.3) & 97.7 (0.4) & 98.2 (0.1) & **98.5 (0.1)** \\ Bag & 80.1 (0.8) & 85.6 (0.1) & 86.7 (1.4) & 85.9 (0.1) & **86.5 (0.1)** \\ Ankle boot & 95.1 (0.4) & 96.5 (0.4) & **98.0 (0.5)** & 97.6 (0.1) & 97.8 (0.2) \\ \hline Mean & 87.1 & 90.6 & 90.6 & 91.3 & **91.5** \\ \hline \end{tabular}
\end{table}
Table 3: Comparison of mean and variance AUROC performances for various algorithms on the Fashion-MNIST dataset.