# Drone vs. Bird Detection: Deep Learning Algorithms and Results from a Grand Challenge
## 6 Alexis Team
### Experimental Evaluation  
In this section, the performance of the approach is evaluated and experiments are reported to show the impact of some hyperparameters such as the tile size and data selection and augmentation strategies, which can have a significant impact. The AP was used, which is the standard metric for object detection, for the evaluation.  
In the following experiments, two different data sampling strategies to better measure the generalization performance of their approach were used. The first data sampling strategy used is the standard way of splitting data: all images were sampled into training (85% of the images), validation (5%) and test (10%) sets. This strategy was named \"known sequences\", since images were sampled without taking the video sequence of the Drone vs. Bird dataset into account (meaning that images in the test set could come from the same sequence as images of the training set). For the second strategy, which is called \"unknown sequences\", contain exclusively 12 selected sequences for the test set and the images from the remaining sequences were selected for training and validation. This is a more realistic way of measuring the generalization performance of the approach by decreasing the odds of overfitting the test sequences.  
[PERSON] team first evaluated the impact of tile resolution on a seen and unseen sequence. Results of this experiment are reported in Table 1. From the results, a tile resolution of 480 pixels seems to be a good compromise in terms of detection performances and processing time. The performance was evaluated on a PC equipped with an Intel Core i7-7820X CPU, a Samsung 970 EVO PLUS SSD and a NVIDIA GeForce RTX 2080 Ti GPU.  
Subsequently, the impact of employing synthetic data and style augmentation during training was evaluated. The results in Table 2 are reported for a test set filled with images from an unseen sequence. In the first case, the performance of the approach when trained only with images from the Drone vs. Bird (DvB) Challenge was evaluated. The impact of adding the synthetic dataset and style augmentation was then evaluated. One can see that for that data split, both strategies helped this approach to generalize better to images of the unseen sequence.  
Finally, the average performance for 5 different data splits for both seen and unseen sequences are reported in Table 3. For the experiments, a tile size of 480 pixels and an overlap of 100 pixels between each tile were used.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Metrics/Tile Size (px)** & **320** & **480** & **640** \\ \hline Known sequence AP (\%) & 91.3 & 92.9 & 89.1 \\ Unknown sequence AP (\%) & 56.1 & 57.8 & 44.3 \\ Processing time per image (ms) & 195 & 169 & 125 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Impact of tile resolution on detection performances and processing time per image evaluated on images from a seen and an unseen sequence.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Training** & **AP (\%)** \\ \hline DvB & 52.2 \\ DvB + synthetic & 55.6 \\ DvB + synthetic + style augmentation & 57.8 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance of different training strategies evaluated on images from an unknown sequence. The performance is reported when training only with images from the Drone vs. Bird (DvB) Challenge and when the synthetic dataset and style augmentation are added.