## 4 Results
### Convolutional Neural Network (CNN) Result
Both the learning loss and testing loss for each optimizer are covered in the graph. The two zoom-in areas are \((x_{1},x_{2})=(0,1000)\), \((y_{1},y_{2})=(3,4)\); \((x_{1},x_{2})=(0,300)\), \((y_{1},y_{2})=(0,2)\). VGG serves as the foundation for modern convolutional neural networks and has shown excellent performance in our in-air signature dataset. Figure 13 shows the loss curve for VGG with respect to each of the optimizers. The largest train loss optimizer for the VGG is Adadelta with a value of \(1.51\times 10^{-1}\), and the smallest train loss optimizers for the VGG are Adam, Adamax, NAdam, and RAdam, with a value of \(0.00\times 10^{0}\). The largest test loss optimizer for the VGG is Adadelta, with a value of \(6.63\times 10^{-2}\), and the smallest train loss optimizers for the VGG are Adam, AdamW, Adamax, NAdam, and RAdam, with a value of \(0.00\times 10^{0}\). The best train score optimizer for VGG is Adamax, with a score of \(99.0646\%\) and the worst train score optimizer for VGG is Adadelta, with a score of \(81.4840\%\). The best test score optimizer for VGG is Adamax, with a score of \(99.5205\%\), and the worst test score optimizer for VGG is Adadelta, with a score of \(88.2750\%\). The combination of the VGG and Adamax for image classification is popular in research; for example, in the medical study by [PERSON] et al. [56], the VGG-16 structure is used for bladder cancer diagnosis. The combination of VGG-16 and the Adamax optimizer can successfully detect bladder cancer images with an accuracy of \(0.98\). Figure 12: The loss curve of AlexNet with respect to 10 different optimizers. Both the learning loss and testing loss for each optimizer are covered in the graph. The zoom-in area is \((x_{1},x_{2})=(0,100)\), \((y_{1},y_{2})=(0,4)\). ResNet is one of the most powerful convolutional neural networks in the decades and it works as the foundation for variations of modern CNNs. Figure 14 shows the loss curve for ResNet with respect to each of the optimizers. The largest train loss optimizer for the ResNet is ASGD, with a value of \(5.41\times 10^{-2}\), and the smallest train loss optimizer for the ResNet is RMSprop, with a value of \(3.10\times 10^{-7}\). The largest test loss optimizer for the ResNet is ASGD, with a value of \(3.42\times 10^{-2}\), and the smallest train loss optimizer for the ResNet is NAdam, with a value of \(2.72\times 10^{-6}\). The best train score optimizer for ResNet is Adamax, with a score of 99.7636%, and the worst train score optimizer for ResNet is Adadelta, with a score of 85.6623%. The best test score optimizer for ResNet is Adagrad, with a score of 99.8514%, and the worst test score optimizer for ResNet is Adadelta, with a score of 96.3550%. In the study by [PERSON] et al. [57], the comparison between different CNN structures in combination with various optimizers was conducted for weed detection, in which the combination of the ResNet and Adagrad outperformed other optimizers with an accuracy of 0.98. The research can be used as statistical support for the performance of the ResNet and Adagrad combination. Figure 13: The loss curve of VGG with respect to 10 different optimizers. Both the learning loss and testing loss for each optimizer are covered in the graph. The zoom-in area is \((x_{1},x_{2})=(0,100)\), \((y_{1},y_{2})=(0,4)\).