# Face Spoofing Detection with Stacking Ensembles in Work Time Registration System
## 4 Experimental Evaluation
### Experimental Results
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Our Methods** & **Small dataset** & **Spoofing Data** & **Crowd-collected** & **CelebA** \\ \hline Bezel (B) & 70.99 \(\pm\) 4.22 & 57.67 \(\pm\) 3.05 & 37.63 \(\pm\) 0.69 & 47.05 \(\pm\) 0.50 \\ \hline Smartphone CNN (S) & 81.35 \(\pm\) 3.83 & 86.78 \(\pm\) 2.63 & 90.27 \(\pm\) 0.59 & 89.26 \(\pm\) 0.47 \\ \hline Context (C) & 77.98 \(\pm\) 4.21 & 91.95 \(\pm\) 1.86 & 90.47 \(\pm\) 1.05 & 98.22 \(\pm\) 0.17 \\ \hline Image analysis CNN (N) & 85.64 \(\pm\) 4.59 & 91.03 \(\pm\) 1.98 & 98.20 \(\pm\) 0.36 & 94.71 \(\pm\) 0.24 \\ \hline
**Other Methods** & **Small dataset** & **Spoofing Data** & **Crowd-collected** & **CelebA** \\ \hline EfficientNetV2 (E) & 83.21 \(\pm\) 7.91 & 91.48 \(\pm\) 3.80 & 88.53 \(\pm\) 0.95 & 91.50 \(\pm\) 0.44 \\ \hline ConvNeXt (X) & 54.71 \(\pm\) 6.47 & 56.03 \(\pm\) 4.60 & 90.47 \(\pm\) 1.05 & 90.37 \(\pm\) 4.05 \\ \hline MobileNet & 71.73 \(\pm\) 2.33 & 73.86 \(\pm\) 0.89 & 90.83 \(\pm\) 0.19 & 87.96 \(\pm\) 0.34 \\ \hline Depth prediction & 53.25 \(\pm\) 3.33 & 41.27 \(\pm\) 5.25 & 89.37 \(\pm\) 7.01 & 85.75 \(\pm\) 8.27 \\ \hline ViT (V) & 52.52 \(\pm\) 6.04 & 56.56 \(\pm\) 10.46 & 87.92 \(\pm\) 0.65 & 75.49 \(\pm\) 0.55 \\ \hline Swin Transformer (T) & 50.32 \(\pm\) 3.43 & 59.48 \(\pm\) 5.54 & 88.07 \(\pm\) 0.67 & 58.75 \(\pm\) 0.32 \\ \hline
**Ensembles** & **Small dataset** & **Spoofing Data** & **Crowd-collected** & **CelebA** \\ \hline C,N,E (stacking) & 91.18 \(\pm\) 3.95 & 96.35 \(\pm\) 1.69 & 98.89 \(\pm\) 0.22 & 98.94 \(\pm\) 0.17 \\ \hline S,C,N,E (stacking) & 92.29 \(\pm\) 3.16 & 96.77 \(\pm\) 1.46 & 98.93 \(\pm\) 0.22 & 99.14 \(\pm\) 0.11 \\ \hline S,C,N,X,E (stacking) & 92.15 \(\pm\) 3.36 & 96.85 \(\pm\) 1.51 & 99.00 \(\pm\) 0.17 & 99.19 \(\pm\) 0.10 \\ \hline B,S,C,N,X,E (stacking) & 92.46 \(\pm\) 3.17 & 96.82 \(\pm\) 1.54 & 98.99 \(\pm\) 0.19 & 99.18 \(\pm\) 0.12 \\ \hline B,S,C,N,X,E,V (stacking) & 92.62 \(\pm\) 3.05 & 96.88 \(\pm\) 1.48 & 98.99 \(\pm\) 0.17 & 99.20 \(\pm\) 0.12 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Prediction accuracy [%] of the evaluated methods and ensembles in five runs of a 5-fold cross-validation (a total of 25 training/test pairs). At the bottom, the best-performing ensembles of 3, 4, 5, 6, and 7 models are presented. The complete results of 240 different stacking and voting ensembles are presented in the Appendix A. * For the ANOVA test: \(f=116.36\), \(p=1.73\times 10^{-145}\), which is significantly below the accepted threshold. Therefore, the null hypothesis can be rejected. * For the Kruskal-Wallis test: \(h=476.42\), \(p=4.80\times 10^{-96}\), which is also significantly below the accepted threshold. Therefore, the null hypothesis can be rejected. This proves that the tested systems differ and that the differences in the results are statistically significant. The conclusion is that the stacking ensemble increases the accuracy of the system. Adding the ConvNeXt network had a bigger impact on prediction accuracy than further adding other networks, which might be a result of the individual accuracy of those networks being lower than that of ConvNeXt. Only for the Small dataset, including low-accuracy models (ConvNeXt, Vision Transformer-ViT) in the ensemble significantly increases the obtained accuracy. The results indicate that the system is effective in recognizing spoofing attempts while simultaneously displaying high accuracy in correctly identifying authentic faces. The diversity of datasets used to evaluate the system's performance shows that it is effective in various cases, with different backgrounds and appearances of the people in front of the camera, as well as that it performs significantly better than other systems when using a small dataset for its training. As part of further testing, all possible ensemble combinations of the seven methods (Bezel, Smartphone Detection CNN, Context, Image Analysis CNN, EfficientNetV2, ConvNeXt, and ViT) were evaluated in an attempt to find the optimal one, separately for stacking and voting ensembles.