## 3 Results and Discussion
### Evaluation with COCO API  
The DOTA dataset was randomly divided into two as a training and test with 90% and 10% ratios, respectively. However, there is a difference in the distribution of object scales for the training and test groups. Moreover, for the independent large-scale image set produced from Pleiades satellite images, most of the objects are in the medium range (Table 3). To evaluate the converge rates of the models on the training data, the performance metrics were also calculated for the DOTA training set, in addition to the test data. The performances of all trained models were examined with the COCO metric API, except for the first training attempts of SSD and Faster R-CNN as their learning rate was low (Table 4). According to the COCO metrics, the Faster \begin{table}
\begin{tabular}{c c c} \hline \hline
**Calculated for** & **Metric Name** \\ \hline AP for [ IoU = 0.50:0.95 ] area = all | maxDets = 100 ] & 1. Metric \\ AP for [ IoU = 0.50 ] area = all | maxDets = 100 ] & 2. Metric \\ AP for [ IoU = 0.75 ] area = all | maxDets = 100 ] & 3. Metric \\ AP for [ IoU = 0.50:0.95 ] area = small | maxDets = 100 ] & 4. Metric \\ AP for [ IoU = 0.50:0.95 ] area = medium | maxDets = 100 ] & 5. Metric \\ AP for [ IoU = 0.50:0.95 ] area = large | maxDets = 100 ] & 6. Metric \\ AR for [ IoU = 0.50:0.95 ] area = all | maxDets = 1 ] & 7. Metric \\ AR for [ IoU = 0.50:0.95 ] area = all | maxDets = 100 ] & 8. Metric \\ AR for [ IoU = 0.50:0.95 ] area = all | maxDets = 100 ] & 9. Metric \\ AR for [ IoU = 0.50:0.95 ] area = small | maxDets = 100 ] & 10. Metric \\ AR for [ IoU = 0.50:0.95 ] area = medium | maxDets = 100 ] & 11. Metric \\ AR for [ IoU = 0.50:0.95 ] area = large | maxDets = 100 ] & 12. Metric \\ \hline \hline \end{tabular}
\end{table} 
Table 2: COCO performance metrics with calculation rules. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Data Set/Object Scale** & **Small** & **Medium** & **Large** \\ \hline DOTA Training Set & 0.06 & 0.52 & 0.42 \\ DOTA Test Set & 0.03 & 0.28 & 0.69 \\ Large Scale Image Set & 0.10 & 0.76 & 0.14 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The ratios (%) of data sets according to the object scale. R-CNN model provided the best results when considering the mean of the precision for different IoU values. Yolo-v3 provided promising results for 0.5 IoU and above, while Faster R-CNN is better if 0.75 IOU and above is desired. For metrics 4, 5, and 6, Faster R-CNN provided the best AP result for different IOUs in small, medium, and large objects for the DOTA test set. However, in the large-scale image set, the Yolo-v3 model provided better results for small and medium objects. The reason for these results is that the architectures have different structures to learn different attributes from the training data. The seventh, eighth, and ninth metrics provide information about the recall rates for all object sizes according to the detection number per image. Similarly, the Faster R-CNN provided better results according to these metrics. When the AR results were investigated according to metrics 10, 11, and 12, it was revealed that the recall rates of Yolo-v3 are worse than the SSD for large-scale image sets. In addition, the SSD is also ahead of the Faster R-CNN for small and medium aircrafts. The fact that the DOTA training and test performances are similar for the three architectures indicates that the models can successfully learn the object characteristics from the DOTA dataset. However, when these results were compared with the results from the large-scale Pleiades image set, there is a big performance gap. The main reasons behind the performance gap are that the dimensions of the aircrafts inside the large-scale image set are distributed differently than the DOTA dataset and large-scale image sets contain different types of aircraft (Figure 9). With the COCO metric API, precision-recall curves were plotted according to the object size, and the differences between these curves provides valuable insights about the detection efficiencies of models.