## 4 Results
### Convolutional Neural Network (CNN) Result
The  
\begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multicolumn{7}{c}{**Model Performance with Respect to Optimizers**} \\ \hline \(l=1\times 10^{-4}\) & \multicolumn{1}{c}{**\& Cross-Entropy**} & \multicolumn{6}{c}{Optimizers**} \\ \hline
**Model** & **C Criteria** & **Adadelta** & **Adagrad** & **Adam** & **AdamW** & **Adamax** & **ASGD/Fult** & **NAAdam** & **RAdam** & **RMSprop** & **SGD** \\ \hline \multirow{4}{*}{**LeNet**} & Train Loss & \(3.71\times 10^{9}\) & \(3.57\times 10^{9}\) & \(7.15\times 10^{-8}\) & \(1.65\times 10^{-6}\) & \(0.00\times 10^{9}\) & \(3.33\times 10^{9}\) & \(1.91\times 10^{-7}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(3.52\times 10^{9}\) \\ \cline{2-11}  & Train Score (\%) & \(5.0468\) & \(28.7778\) & \(96.1396\) & \(97.9845\) & \(93.0550\) & \(12.0159\) & \(98.0236\) & \(97.0636\) & \(97.9309\) & \(24.3936\) \\ \cline{2-11}  & Test Loss & \(3.69\times 10^{9}\) & \(3.32\times 10^{9}\) & \(8.01\times 10^{-6}\) & \(4.26\times 10^{-5}\) & \(1.91\times 10^{-7}\) & \(2.38\times 10^{9}\) & \(1.53\times 10^{-5}\) & \(2.15\times 10^{-7}\) & \(4.08\times 10^{-5}\) & \(1.86\times 10^{9}\) \\ \cline{2-11}  & Test Score (\%) & \(3.8445\) & \(12.1927\) & \(98.4664\) & \(98.4008\) & \(92.9885\) & \(34.8241\) & \(96.1205\) & \(\mathbf{98.4814}\) & \(96.4250\) & \(46.2164\) \\ \hline \multirow{4}{*}{**AlexNet**} & Train Loss & \(1.31\times 10^{9}\) & \(2.13\times 10^{-3}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(0.22\times 10^{-2}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(3.89\times 10^{-2}\) & \(9.13\times 10^{-3}\) \\ \cline{2-11}  & Train Score (\%) & \(55.5794\) & \(98.0123\) & \(97.7586\) & \(97.7295\) & \(98.8518\) & \(98.1736\) & \(97.8955\) & \(97.4246\) & \(97.4186\) & \(97.8732\) \\ \cline{2-11}  & Test Loss & \(2.51\times 10^{-1}\) & \(6.72\times 10^{-3}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(1.67\times 10^{-7}\) & \(4.10\times 10^{-3}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(6.04\times 10^{-3}\) \\ \cline{2-11}  & Test Score (\%) & \(70.4927\) & \(98.2100\) & \(98.9527\) & \(98.7150\) & \(97.4223\) & \(97.4283\) & \(98.5796\) & \(98.9955\) & \(98.1380\) & \(99.2946\) \\ \hline \multirow{4}{*}{**VGG**} & Train Loss & \(1.51\times 10^{-1}\) & \(1.00\times 10^{-3}\) & \(0.00\times 10^{9}\) & \(8.13\times 10^{9}\) & \(7.79\times 10^{-3}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(7.67\times 10^{-1}\) & \(8.05\times 10^{-3}\) \\ \cline{2-11}  & Train Score (\%) & \(81.4840\) & \(98.0579\) & \(97.8546\) & \(98.1414\) & \(99.0646\) & \(94.1918\) & \(98.2399\) & \(97.5366\) & \(97.8327\) & \(98.1264\) \\ \cline{2-11}  & Test Loss & \(6.63\times 10^{-2}\) & \(1.03\times 10^{-3}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(2.29\times 10^{-3}\) & \(0.00\times 10^{9}\) & \(0.00\times 10^{9}\) & \(7.87\times 10^{-7}\) & \(2.74\times 10^{-3}\) \\ \cline{2-11}  & Test Score (\%) & \(88.2750\) & \(99.4282\) & \(90.0191\) & \(99.1255\) & \(\mathbf{99.3205}\) & \(98.3523\) & \(99.1636\) & \(98.1818\) & \(98.4050\) & \(99.3759\) \\ \hline \multirow{4}{*}{**ResNet**} & Train Loss & \(5.95\times 10^{-2}\) & \(4.29\times 10^{-3}\) & \(2.02\times 10^{-4}\) & \(4.41\times 10^{-4}\) & \(2.96\times 10^{-6}\) & \(5.41\times 10^{-2}\) & \(1.32\times 10^{-5}\) & \(7.04\times 10^{-5}\) & \(3.10\times 10^{-7}\) & \(1.13\times 10^{-3}\) \\ \cline{2-11}  & Train Score (\%) & \(85.6623\) & \(99.6332\) & \(99.1150\) & \(99.0900\) & \(99.75636\) & \(97.6668\) & \(99.1091\) & \(98.9427\) & \(99.0100 superior performance of the combination of AlexNet and Adamax can be supported by the study conducted by [PERSON] et al. [55]. The study compares the performance of AlexNet with the assistance of Adam and Adamax on 20 audio files. In this experiment, the combination of AlexNet and Adamax shows a greater performance than the normal Adam optimizer. Figure 11: The loss curve of LeNet with respect to 10 different optimizers.