## 5 Experiments
### Results
#### 5.3.3 Distance Measure Effectiveness  
We also examined the effectiveness of our proposed model in distance measurement, by using the AUC (area under the curve) ROC (receiver operating characteristic) curve. The AUC-ROC curve is commonly composed of two performance measures: the true-positive (FPR) rate and the false-positive rate (FPR) rate. The equations related to these two performance measures of the AUC-ROC curve are shown as follows:  
\[FPR(P^{*}) :=\int_{p^{*}}^{1}f_{0}(p)dp, \tag{10}\] \[TPR(P^{*}) :=\int_{p^{*}}^{1}f_{1}(p)dp.\]  
where the \(f_{0}(p)\) is denoted by the probability of a density function for the predictions \(p(x)\) produced by our proposed model. The negative pairs are labeled as 0, and \(f_{1}(p)\) is the probability from the positive pair that is labeled as 1. The given discrimination threshold \(P\) is the integrals of the tails of these distributions according to the true-positive rate and false-positive rate. Based on the two parameters TPR and FPR, the AUC-ROC curve is defined as follows:  
\[AUC-ROC=\int_{0}^{1}TPR(FPR)D(FPR). \tag{11}\]  
where the AUC measures the entire two-dimensional area underneath the entire ROC curve (i.e., integral calculus) from (0,0) to (1,1). For example, a model whose predictions are 100% wrong has an AUC of 0.0, while a model whose predictions are 100% correct has an AUC of 1.0. Using this concept, we demonstrated the result predicted by the saved weights of one-shot and five-shot, respectively, of the learned model on the test set, including 5-way, 10-way, and 15-way. These AUC-ROC curves are shown in Figures 13 and 14, which illustrate that the \(\beta\) value at 0.8 provided the best performance when it was tested on one-shot N-way learning. This result also confirms that the hybrid multi-loss function can reduce the distance between the same classes, and enlarge the distance between different classes. Additionally, it does not change the attribute of the feature in the feature space, so the optimization of this layer will never negatively affect the deeper network layers. This hybrid loss function can also compute classification accuracy with a learned distance threshold on distances. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{2}{c}{1-shot} & & & \\ \hline Ref & **Method** & **5-way** & **10-way** & **15-way** \\ \hline
[53] & Matching network & 85 \(\pm\) 2.2\% & 84 \(\pm\) 2.4\% & 76 \(\pm\) 2.6\% \\
[54] & Prototypical network & 86 \(\pm\) 1.7\% & 82 \(\pm\) 1.7\% & 81 \(\pm\) 1.9\% \\
[52] & Siamese network & 82 \(\pm\) 2.5\% & 69 \(\pm\) 2.3\% & 64 \(\pm\) 2.6\% \\  & Task-aware SNN & 88 \(\pm\) 2.2\% & 86 \(\pm\) 2.2\% & 82 \(\pm\) 2.4\% \\ \hline \multicolumn{2}{c}{5-shot} & & & \\ \hline Ref & **Method** & **5-way** & **10-way** & **15-way** \\ \hline
[53] & Matching network & 89 \(\pm\) 2.1\% & 86\(\pm\) 2.1\% & 78\(\pm\) 2.3\% \\
[54] & Prototypical network & 89 \(\pm\) 1.2\% & 85\(\pm\) 1.4\% & 82 \(\pm\) 1.5\% \\
[52] & Siamese network & 85 \(\pm\) 2.0\% & 72 \(\pm\) 2.2\% & 69 \(\pm\) 2.7\% \\  & Task-aware SNN & 91\(\pm\) 1.8\% & 88\(\pm\) 2.1\% & 83 \(\pm\) 2.1\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of classification performance of different few-shot learning approaches for Andro-Dumpsys datasets. Figure 14: AUC-ROC curves under the N-way N-shot. Figure 13: One-shot performance with hyperparameter changing. As shown, our result is on a set of points in the true positive rate-false positive rate plane. The results achieved an AUC-ROC equal to 0.92, 0.91, and 0.80, respectively, under the 5-way, 10-way, and 15-way on the one-shot learning. We further conducted the AUC-ROC on the five-shot learning. Our proposed model also obtained better performance than the generic SNN, with 95.6, 90.7, and 86.8% at 5-way, 10-way, and 15-way, respectively. As expected, the accuracy of both one-shot and five-shot dropped as the number of N-way increased with higher intra-class variance.