# Real UAV-Bird Image Classification Using CNN with a Synthetic Dataset
## 4 Result and Discussion
### Training and Test Results
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Exp 2.1 Synthetic Images Test**} & \multicolumn{3}{c}{**Exp 2.2 Real Images Test**} \\
**Model** & **F-Score** & **Acc** & **AUC** & **F-Score** & **Acc** & **AUC** \\
**Optimizer** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** \\ \hline CDNTS + AlexNet & **90** & **90** & **89.9** & 77 & 75 & 75 \\ SGD & & Lr: 0.1, Bs: 256 & & & Lr: 0.001, Bs:256 & \\ CDNTS + SqueezeNet & 84 & 84 & 83.7 & 91 & 89 & 87.8 \\ Adam & & Lr: 0.01, Bs:256 & & & Lr: 0.01, Bs:32 & \\ CDNTS + SqueezeNet & 79 & 76 & 75.6 & **91** & **90** & **88.9** \\ RMSProp & & Lr: 0.001, Bs:128 & & Lr: 0.01, Bs:256 & \\ CDNTS + VGG16 & 88 & 88 & 87.4 & 62 & 58 & 57.7 \\ SGD & & Lr: 0.001, Bs:32 & & & Lr: 0.001, Bs:32 & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Experiment 2: CDNTS and deep learning-based networks’ F-score, accuracy, and AUC values of test results. It shows the test success of networks trained using by CDNTS layer. Values that styled bold represent the most successful test listed. Lr: Learning Rate, Bs: Batch Size. models consisting of combinations of hyperparameters. All models trained in experiment 2.1 and experiment 2.2 were tested separately with synthetic- and real-image test datasets. The positive effect of the CDNTS layer was observed in all networks (see Table 3). The F-score, accuracy, and AUC values obtained from the results of experiment 2 were considerably improved in comparison to experiment 1. The increment in AUC values from the two experimental setups, as shown below, can be considered as evidence of the effect of the CDNTS layer on the synthetic image test dataset. The results listed in Table 2 are test results obtained without adding the CDNTS network. The results listed in Table 3 are the test results obtained by adding the CDNTS network suggested by us. When the results of Tables 2 and 3 are compared, the increase in the success of the test results in Table 3, which listed the effect of adding the CDNTS layer, is seen. The increases in AUC performance with the synthetic-image dataset for the AlexNet network were from 55% to 89.9%, while those for the SqueezeNet network were 74% to 83.7%, and those for the VGG16 network were 70% to 87.4%. A similar increment in the AUC values was also shown when testing with real-image datasets, which can be also considered as evidence of the effect of the CDNTS and are shown as follows: for the AlexNet network, an increase from 55% to 75% was found, and an increase from 57% to 88.9% was found for the SqueezeNet network. The network with the best result values in the synthetic-image test experiments was the AlexNet network SGD optimizer with a 90% F-score, 90% accuracy, and 89.9% AUC, where the detected learning rate and batch size were 0.1 and 256, respectively. The network with the best result values in the real-image test experiments was the SqueezeNet network RMSprop optimizer with a 91% F-score, 90% accuracy, and 88.9% AUC as shown in Table 3, where the detected learning rate and batch size were 0.01 and 256, respectively. The precision and recall graph of the most successful networks for synthetic- and real-image test data in experiment 1 and experiment 2 is shown in Figure 9. It was observed that the precision and recall plots of experiment 2, consisting of CDNTS and a network layer, were more stable than experiment 1. The CDNTS layer works similarly to a feature extraction layer while maintaining the general outline of the pictures, preserving their structural features. While creating and collecting synthetic- and real-image test data, pictures that could be encountered in daily life were created and selected. In the backgrounds of the test images, houses, trees, roofs, etc., were included. The CDNTS layer gave successful results on these test images. The corners of the roofs, branches of trees, and walls of the houses could resemble bird and UAV shapes after the CDNTS layer process. This kind of situation could cause the failure of networks.