# Building Ensemble of Resnet for Dolphin Whistle Detection
## 3 Experimental Results  
The protocol used in our experiments mirrored that proposed in [28]. However, we used the validation set described in Section 2.3.2 to learn which data augmentation methods to apply and the weights of the weighted sum rule. After choosing the weights based on the validations set, we used the subdivision of the training and testing set described in [28]. We wish to stress that the validation set was extracted from the training set, so there was no overfitting on the test set. We gauged the performance of the model on the distinct test set by calculating the same performance indicators used in [28]. The true positive rate and the false positive rate was used to ascertain precision/recall. These metrics were used to generate the receiver operating characteristic (ROC) curves and evaluate the corresponding area under the ROC curve (AUC):  
\[Precision=TP/\left(TP+FP\right) \tag{1}\]  
Figure 3: Illustration of the SA method described in Section 2.3.3, with time on the \(x\)-axis and frequency in hertz on the \(y\)-axis. The **left** image showcases the original spectrogram. The **right** image presents the spectrogram after SA. Figure 2: Spectrograms illustrating the RS method described in Section 2.3.3, with time on the \(x\)-axis and frequency in hertz on the \(y\)-axis. The **left** image showcases the original spectrogram. The **center** image presents the spectrogram after applying the random shift. The **right** image demonstrates the filled version of the spectrogram. \[Recall=TP/(TP+FN) \tag{2}\]  
\[True\ Positive\ Rate=TP/(TP+FN); \tag{3}\]  
where \(TP\) indicates true positives, \(TN\) indicates true negatives, \(FP\) indicates false positives, and \(FN\) indicates false negatives. In Table 1, we present a comparison between the baseline ResNet50 and the proposed data augmented ResNet50 (named ResNet50_DA). ResNet50(x)_DA indicates the combination of \(x\) ResNet50_DA networks using the sum rule. Figure 4 reports the ROC curve for ResNet50(1) vs. ResNet50(10)_DA. We acknowledge that the performance increase recorded in Table 1 may not appear high compared to the baseline. However, our results are interesting because many of the misclassified samples are unclassifiable by humans. Thus, we are likely already very close to the maximum performance (\(AUC=1\) not obtainable). Furthermore, our results create a new baseline on an available dataset that can be repeated for testing other methods. The plot of the ROC curve in Figure 4 clearly shows that our proposed approach outperforms ResNet50(1). It is important to note that we obtained a true positive rate of 0.9 and a false positive rate of 0.02. Moreover, it is clear that the ResNet50(10)_DA improves ResNet50(1). The number of false positives of the standalone networks was more then two times the number of false positives of the ensemble. In Table 2, we present a comparison between our proposed method and two other approaches using the same dataset with the same testing protocol, reporting a full set of performance indicators (accuracy, AUC, precision, and recall). Clearly, the proposed ensemble performed better than the methods reported in the literature, although with \begin{table}
\begin{tabular}{c c} \hline \hline
**ResNets** & **AUC** \\ \hline ResNet50(1) & 0.960 \\ ResNet50(1)\_DA & 0.964 \\ ResNet50(5)\_DA & 0.972 \\ ResNet50(10)\_DA & **0.973** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison (Area under the ROC curve) of baseline ResNet50(1) with the proposed augmented ensembles of ResNet50s (ResNet50(x)_DA (bold indicates best performance). Figure 4: ROC curve (ratios of the true positives on the \(y\)-axis and false positives on the \(x\)-axis). The light blue represents our proposed ensemble, and the dark blue represents ResNet50(1), a single network. higher computational costs. We do not believe this is a problem, considering that the current computing power of GPUs and the developments expected in the coming years will reduce the considerations of such costs.