# Spectrogram Classification Using Dissimilarity Space
## Experimental Results  
The approach proposed in this paper is tested, along with some comparison canonical approaches, using a stratified ten-fold cross validation protocol and the classification accuracy as the performance indicator. Tests were performed on two datasets:  
* _BIRDZ_, which was also used as a control and as a real-world audio dataset in [23]. The real-world tracks were obtained from the Xeno-canto Archive ([[http://www.xeno-canto.org/](http://www.xeno-canto.org/)]([http://www.xeno-canto.org/](http://www.xeno-canto.org/))) and cover 11 widespread North American bird species. Thus, the dataset contains 11 classes: (1) Blue Jay, (2) Song Sparrow, (3) Marsh Wren, (4) Common Yellowthroat, (5) Chipping Sparrow, (6) American Yellow Warbler, (7) Great Blue Heron, (8) American Crow, (9) Cedar Waxwing, (10) House Finch, and (11) Indigo Bunting. _BIRDZ_ is composed of five types of spectrograms: constant frequency, frequency modulated whistles, broadband pulses, broadband with varying frequency components, and strong harmonics, for a total of 2762 bird acoustic events with 339 detected \"unknown\" events corresponding to noise and other unknown species vocalizations. Including the \"unknown class\", _BIRDZ_ has 3101 samples for 12 classes. * _CAT_, which was first presented in [27; 45]. This dataset is composed of 10 balanced classes with about 300 samples per class: (1) Resting, (2) Warning, (3) Angry, (4) Defence, (5) Fighting, (6)-Happy, (7) Hunting mind, (8) Mating, (9) Mother call, and (10) Paining. The samples have an average duration of about 4s and were collected by the author from different sources: Kaggle, Youtube and Flickr. _CAT_ has a total of 2962 samples. In Tables 2 and 3, the performance of the four tested clustering algorithms is reported using different values of \(kc\) (i.e., the number of clusters per class). As a baseline for comparison, the classification accuracy is also reported for the following well-known CNN models, each fine-tuned on the problem (for 30 epochs, using a batch size of 30, and a learning rate of 0.0001, no freezing):  
* Googlenet [46], VGG16 and VGG19 [47], all pretrained on ImageNet [48];
* GoogleNetP365, a GoogleNet model pretrained on Places365 [49]. Moreover, in Tables 2 and 3, the accuracy obtained by the following fusion approaches are reported:  
* KAll, fusion by sum rule of the four SVMs trained using the dissimilarity space built with all tested values for \(kc\) = 15, 30, 45, 60;
* ALL, fusion by average rule of the four approaches KAll (one for each clustering method);
* eCNN, fusion by sum rule of the four CNNs;
* ALL+eCNN, fusion by sum rule between ALL and eCNN;
* ALL+GoogleNet, fusion by sum rule between ALL and GoogleNet;
* ALL+GoogleNetP365, fusion by sum rule between ALL  
From the results reported in Tables 2 and 3, the following conclusions can be drawn:  
1. KAll outperforms each stand alone method based on a single value of \(kc\);
2. ALL outperforms each KAll in both datasets;
3. Performance of ALL is similar to that obtained by GoogleNet;
4. The ensemble ALL based on our dissimilarity space is a feasible representation for spectograms and achieves a performance that is comparable to the CNNs. 5. In both datasets, the best performance is obtained by ALL+eCNN, (even though the improvement with respect to eCNN is negligible). 6. ALL+GoogleNet strongly outperforms ALL and Googlenet; this light ensemble, which uses only one CNN, is our recommended method. The proposed approach based on the representation of animal sound in a dissimilarity space has two main advantages: (1) it produces a compact representation on the signal (ranging from 15 to 60, depending on the number of clusters for the single space, to 150 for the KAll ensemble); (2) it generates a high diversity of classification results with respect to the baseline CNNs, which can be exploited to improve the performance in an ensemble method (i.e., ALL+GoogleNet).