# An Ensemble of Convolutional Neural Networks for Audio Classification
## 6 Conclusions  
In this paper, we presented the largest study conducted so far that investigates ensembles of CNNs using different data augmentation techniques for audio classification. Several data augmentation approaches designed for audio signals were tested and compared with each other and with a baseline approach that did not include data augmentation. Data augmentation methods were applied to the raw audio signals and their visual representations using different spectrograms. CNNs were trained on different sets of data augmentation approaches and fused via the sum rule. Experimental results clearly demonstrate that ensembles composed of fine-tuned CNNs with different architectures maximized performance on the tested three audio classification problems, with some of the ensembles obtaining results comparable with the systems, including on the ESC-50 dataset. To the best of our knowledge, this is the largest, most exhaustive study of CNN ensembles applied to the task of audio classification. Our best ensemble, FusionGlobal is composed of 63 networks with nine different architectures and seven different trainings for every network. Four of the networks were obtained using different data augmentation strategies and three different methods to create a spectrogram. It is worth noting that the same ensemble is competitive with the state of the art on the three datasets. To be more precise, we reach 96.82% on the BIRDZ dataset, 90.51% on the CAT dataset, and 88.65% on ESC-50. This work can be expanded further by investigating which augmentation methods (spectrogram augmentation vs. signal augmentation) work best for classifying different kinds of sounds. We also plan to apply transfer learning using spectrograms instead of natural images. A systematic selection of augmentation approaches, e.g., by iteratively evaluating an increasing subset of augmentation techniques (as is typical when evaluating different features), would require an enormous amount of time and computation power. An expert-based approach that utilizes the knowledge of environmental scientists would be the best way of handling this challenge. This study could also be expanded by including more datasets, which would provide a more comprehensive validation of the proposed fusions. Furthermore, there is a need \begin{table}
\begin{tabular}{c c c} \hline \hline
**Paper** & **Method** & **Advantages** \\ \hline \multirow{3}{*}{Ours} & Ensemble of networks with different data augmentation and different spectrograms. & Great performances over a variety of datasets and easy to implement but requires large computational power. \\ \cline{2-3}  & Multi-stage sequential learning with knowledge transfer from a large audio dataset. & State of the art on ESC-50. However, it requires large computational power, and the model is pretrained on a large audio dataset. \\ \hline \multirow{2}{*}{[54]} & Pretrained model with multi-channel features. & Simple model with performances similar to ours on ESC-50. \\ \hline \multirow{2}{*}{[21]} & CNN model with potential compression. & Performance close to the state of the art on ESC-50 and suitable for mobile devices. \\ \hline \multirow{3}{*}{[16]} & CNN with filter banks learned using convolutional Restricted Boltzmann Machines + fusion with gammatone spectral coefficient and Mel energies. & Performance close to ours on ESC-50 \\ \cline{1-1} \cline{2-3}  & CNN with mix-up and data augmentation. & Very light network for mobile devices that performs very well, but not as much as the state-of-the art models. \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Comparisons among the different approaches of audio classification. to investigate the impact on performance when different CNN topologies and parameter settings in the retuning step are combined with different types of data augmentation. The technique proposed here is not exclusively applicable to environmental sound classification but can be applied to many related audio tasks.