## 5 Discussion  
Interpreting the presented results, we conclude that the use of CWTFT scalograms returns the best results for audio scene and event classification applications. This is supported by our previous experiments, which were performed using the SINS database [45; 46] and the experiments conducted in this work. This can be justified by the fact that scalograms possess excellent time and frequency localization. Furthermore, another advantage is that it also separates audio sources upon the wavelet computation process. Using an FFT-based wavelet transform also returns favourable time duration requirements, which exceeded that of cepstral and spectral features.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **MAlexNet-33** & **SqueezeNet** & **MobileNet-v2** & **NasNet Mobile** & **ShuffleNet** \\ \hline Number of Layers & 33 & 68 & 155 & 913 & 173 \\ Depth & 8 & 18 & 53 & N/A & 50 \\ Type & Series Network & DAG & DAG & DAG & DAG \\ Network Size & 14.33 MB & 3.07 MB & 9.52 MB & 19.44 MB & 3.97 MB \\ Activation Function & Leaky ReLU & & Clipped ReLU & ReLU & ReLU \\  & (0.01) & Fire ReLU & (C: 6) & ReLU & ReLU \\ Weighted F1-score & 87.92\% & 84.48\% & 86.85\% & 83.38\% & 86.91\% \\ Training time & 178 min & 273 min & 599 min & 1668 min & 792 min \\ Epochs & 30 & 30 & 30 & 30 & 30 \\ Loading time average & 1.10 s & 1.04 s & 1.32 s & 2.59 s & 1.62 s \\ Execution time average & 0.0148 s & 0.0159 s & 0.0338 s & 0.1345 s & 0.0348 s \\ \hline \hline \end{tabular}
\end{table}
Table 9: Detailed Comparison with other Compact Neural Networks.  
There are three main discoveries found in this study:  
**Hypothesis 1:**_The Leaky ReLU activation function returned higher performance for multi-level classification as opposed to the traditional ReLU in the majority of cases._  
**Verification of Hypothesis 1:** This is true on a case-by-case scenario. This can be explained by the presence of the dying ReLU problem in feature sets, which is ameliorated through the small parameter added through the Leaky ReLU. However, it is important to note that the presence of the dying ReLU problem could depend on several factors, including the nature of the data being trained. In cases where this does not occur, replacing the activation function to a Leaky ReLU may not return any advantages.  
**Hypothesis 2:**_Decreasing the number of fully connected and convolutional layers throughout the network also slightly decreases the performance._  
**Verification of Hypothesis 2:** Generally, convolutional layers represent high level features within the network. Accordingly, fully connected layers flatten and combine these features. Hence, reducing the number of these layers negatively affects the performance of the network.  
**Hypothesis 3:**_Decreasing parameters, weight factors, and biases within the fully connected and convolutional layers helps decrease the size of the network more, compared to when these layers are removed completely._  
**Verification of Hypothesis 3:** Both convolutional and fully connected layers contribute to the high and low-level features from which the network learns, and are therefore essential. However, since pre-trained models are originally trained on very large data, large parameters, weight, and bias factors are often not necessary for the smaller dataset by which transfer learning is being implemented for. This explains the maintenance of the system performance despite decreasing the parameters for these layers accordingly. Based on our experiments, scaling the parameters uniformly across fully connected layers returns the best performance.