# Real UAV-Bird Image Classification Using CNN with a Synthetic Dataset
## 4 Result and Discussion  
### Training and Test Results  
We examined the AI training using synthetic data under two methods (experiments 1 and 2). In both methods, three different networks were trained to determine the most successful network and optimizer with a combination of four optimizers, two dropout values, three batch size values, and three learning rate values. Therefore, a total of 432 trained combinations of models were obtained, including 216 different trained models in experiment 1, and 216 different trained models in experiment 2. Experiment 1 consisted of training deep learning-based classification networks without a CDNTS layer, while experiment 2 consisted of training deep learning-based classification networks with a CDNTS layer. Each model was trained and later tested in comparison with a real-test dataset and synthetic-test dataset, and the 432 combinations are presented in Figure 8.  
The test accuracy results and training accuracy results of experiments 1 and 2 were compared. It was reported in experiment 2 that the added CDNTS layer affected the training time that defined average cost of training time is 36.7 ms and 75.9 ms/image. The CDNTS layer with an attached network in experiment 2 gave improved results for real and synthetic datasets. The training models of all hyperparameter combinations were tested separately with synthetic-image and real-image test data. The results of experiments with two subtests--experiments 1.1, 1.2, 2.1, and 2.2--are reported in Tables 2 and 3. Figure 7: CDNTS layer, nearest three-point selection process schematic representation. The synthetic-image data and real-image data results of experiment 1 without the CDNTS layer are tabulated in Table 1. The F-score, accuracy, and AUC test results were less than 55% for the synthetic- and real-image test data with the AlexNet network, and 57% to 80% with the SqueezeNet and VGG16 networks. In experiment 1.2, using real-image data for testing, the SqueezeNet and Adadelta optimizer yielded a noteworthy result but a poor score of 72% for the AUC. The most successful test results in experiment 1 was in the synthetic image test dataset amounting to a 74% AUC value from SqueezeNet and the Adam optimizer. The evaluation of the results obtained in experiment 1 shows that significant performance was not achieved in general. The reason for this failure was the low realism of the synthetic images, insufficient depth perception, and insufficient color contrast [49]. In experiment 2, consisting of the training and testing of CDNTS layer and DL networks, the training dataset consisting of synthetic images was trained with all \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Exp 1.1 Synthetic Images Test**} & \multicolumn{3}{c}{**Exp 1.2 Real Images Test**} \\
**Model** & **F-Score** & **Acc** & **AUC** & **F-Score** & **Acc** & **AUC** \\
**Optimizer** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** \\ \hline AlexNet & \textless{}55 & \textless{}55 & \textless{}55 & \textless{}55 & \textless{}55 & \textless{}55 & \textless{}55 \\ for All & & Lr: for all, Bs: for all & & Lr: for all, Bs: for all & \\ SqueezeNet & 71 & 70 & 70 & **76** & **73** & **72** \\ AdaDelta & & Lr: 0.1, Bs:256 & & Lr: 0.001, Bs:128 & & \\ SqueezeNet & **80** & **74** & **74** & 63 & 58 & 57 \\ Adam & & Lr: 0.01, Bs:32 & & & Lr: 0.1, Bs:128 & \\ VGG16 & 76 & 70 & 70 & 69 & 64 & 63 \\ SGD & & Lr: 0.01, Bs:32 & & & Lr: 0.001, Bs:32 & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Experiment 1: Deep learning-based networks’ F-score, accuracy (Acc), and AUC values of the test results. It shows the test success of networks trained without using the CDNTS layer. Values that styled bold represent the most successful test listed. Lr: Learning Rate, Bs: Batch Size. Figure 8: Schematic representation of combinations of networks.