# An Ensemble of Convolutional Neural Networks for Audio Classification
## 2 Audio Image Representations  
Since the input to a CNN is in the form of a matrix, the following four methods were used to map the audio signals into spectrograms:  
1. The Discrete Gabor Transform (DGT): this is a short-time Fourier transform (STFT) with a Gaussian kernel as the window function. The continuous version of DGT can be defined as the convolution between the product of the signal with a complex exponential and a Gaussian, as: \[G(\tau,\omega)=\frac{1}{\sigma^{2}}\int_{-\infty}^{+\infty}\mathbf{x}(t)e^{i\omega t }e^{-\pi\sigma^{2}(t-\tau)^{2}}\,dt,\] (1) where \(\mathrm{s}(t)\) is the signal, \(\omega\) is a frequency, and \(i\) is the imaginary unit. The width of the Gaussian window is defined by \(\sigma^{2}\). The discrete version of DGT applies the discrete convolution rather than the continuous convolution. The output \(G(\tau,\,\omega)\) is a matrix, where the columns represent the frequencies of the signal at a fixed time. The DGT implementation used in this study (see, [35]) is available at [[http://lftat.github.io/doc/gabor/sgram.html](http://lftat.github.io/doc/gabor/sgram.html)]([http://lftat.github.io/doc/gabor/sgram.html](http://lftat.github.io/doc/gabor/sgram.html)) (accessed on 6 January 2021).
2. Mel spectrograms (MEL) [36]: these spectrograms are computed by extracting the coefficients relative to the compositional frequencies with STFT. Extraction is accomplished by passing each frame of the frequency-domain representation through a Mel filter bank (the idea is to mimic the non-linear human ear perception of sound, which discriminates lower frequencies better than higher frequencies). Conversion between Hertz (\(f\)) and Mel (\(m\)) is defined as: \[m=2595\log_{10}(1+700f).\] (2)  
Figure 1: Proposed system. Many ensembles are generated by fine-tuning five pre-trained convolution neural networks (CNNs) separately on four possible audio representations combined with six main data augmentation methods. The best combinations are experimentally derived. The arrows indicate one possible combination of methods for building ensembles.  
The filters in the filter bank are all triangular, which means that each has a response of 1 at the center frequency, which decreases linearly towards 0 until it reaches the center frequencies of the two adjacent filters, where the response is 0.
3. Gammatone (GA) band-pass filters: this is a bank of GA filters whose bandwidth increases with the increasing central frequency. The functional form of Gammatone is inspired by the response of the cochlea membrane in the inner ear of the human auditory system [37]. The impulse response of a Gammatone filter is the product of a statistical distribution (Gamma) and a sinusoidal carrier tone. This response can be defined as: \[h_{i}(t)=\left\{\begin{array}{ll}a\cdot t^{n-1}e^{-2\pi B_{i}t}cos(2\pi \omega_{i}t)+\phi&,\;t\geq 0\\ 0&,\;t<0\end{array}\right.\] (3) where \(\omega_{i}\) is the central frequency of the filter and \(\phi\) its phase. Gain is controlled by the constant \(a\), and \(n\) is the order of the filter. \(B_{i}\) is a decay parameter that determines the bandwidth of the band-pass filter.
4. Cochleagram (CO): this mapping models the frequency selectivity property of the human cochlea [38]. To extract a cochleagram, it is first necessary to filter the original signal with a gammatone filter bank (see, Equation (3) above). The filtered signal must then be divided into overlapping windows. For each window and every frequency, the energy of the signal is calculated.  
Each of the four spectrograms is then mapped to a gray-scale image using a linear transformation that maps the minimum value to 0 and the maximum value to 255, with the value of each pixel rounded to the closest smaller integer.