## 3 Analysis and Discussion
### Analysis of AFEM Setting
#### 3.3.3 Different Weight Initialization  
Parameter learning in neural networks is a class of non-convex optimization problems. When gradient descent is used to optimize the network parameters, the initial value setting is important and significantly affects the generalization ability and optimization efficiency of the network. In this subsection, we will provide more possible initialization settings for the linear transform layer of AFEM. Here, five different frequency warping methods [85] (initialization values) are selected. 1. [label=1]
2. Liner warping. After reaching the peak, it linearly decreases in a triangular fashion like Mel-warping. However, unlike Mel-warping, the spacing between the points corresponding to the peak is equal. 3. Gaussian warping. It is created by applying a one-dimensional Gaussian kernel function to Mel-warping (Equation (1)). It peaks at the same time as Mel-warping. Figure 10: Two layer normalization. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Log-Domain** & **Batch-Norm** & **Layer-Norm** & **Accuracy \%** \\ \hline ✓ & ✓ & ✓ & 89.2 \\ ✓ & ✓ & \(\times\) & 88.7 \\ \(\times\) & ✓ & ✓ & 84.5 \\ ✓ & \(\times\) & ✓ & 86.0 \\ \hline \hline \end{tabular} _Note._ ✓ means that the method is used; \(\times\) means none.  
\end{table} 
Table 5: Effect of different normalization settings on the \(M_{1}\) model’s performance (validation set). However, unlike Mel-warping, it gradually decreases in a Gaussian fashion rather than a triangular fashion. \[f(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-y)^{2}}{2\sigma^{2}}} \tag{9}\]  
where \(\mu\) means the sample mean and \(\sigma\) is the sample variance. 3 Barr warping. A non-linear transformation describing the human ear's perception of frequencies in terms of psychoacoustic scales, and the equidistance corresponds to a frequency scale of equal distance on perception. Here, the following equation transforms the frequency \(f\) to the bark scale. \[Bark=6\sin^{-1}\Bigl{(}\frac{f}{600}\Bigr{)} \tag{10}\]
4 Equivalent rectangular bandwidth scale (ERB). It simulates the perception of sound from the human ear using a rectangular frequency bandpass filter or a bandstop filter for psychoacoustic measurements. Here, the following equation calculates the \(f\) conversion to the ERB scale. \[ERB(f)=9.265\times\lg\Bigl{(}1+\frac{f}{24.7\times 9.265}\Bigr{)} \tag{11}\]
5 GammaTone warping. Similar to Mel-warping, an audio signal is discriminated by simulating the response of the human ear cochlea to frequencies. Here, the following frequency expression is used to calculate this. \[H(f) = c[R(f)\otimes S(f)] \tag{12}\] \[= \frac{c}{2}(n-1)!(2\pi b)^{-n}\biggl{\{}e^{i\phi}[1+i(f-f_{0})/b] ^{-n}+e^{-i\phi}[1+i(f+f_{0})/b]^{-n}\biggr{\}}\]  
where \(c\) is the proportionality constant, \(n\) is the filter order, and \(b\) is the time decay coefficient. \(f_{0}\) and \(\phi\) (radian) are the frequency and phase of the carrier wave, respectively. As shown in Figure 11, compared with Mel-warping, each warping decreases differently after reaching the peak point, and the corresponding log features also vary. There is little change among Figure 11b,d,f. In contrast, Figure 111 has greater differences from the other features. Additionally, in Table 6, there is a 1.1% to 2.1% increase in accuracy relative to the \(M_{0}\) model for models with six different initialization settings, which indicates that the features extracted by them are more suitable for the task at hand. \begin{table}
\begin{tabular}{c c c c} \hline
**No.** & **Initialization Method** & **Accuracy \%** & **Increase \%** \\ \hline
1 & Mel & 89.2 & 1.8 \\
2 & Linear & 88.9 & 1.5 \\
3 & Gauss & 89.0 & 1.6 \\
4 & Brak & 89.4 & 2.1 \\
5 & ERB & 88.6 & 1.1 \\
6 & GammaTone & 89.0 & 1.6 \\ \hline \end{tabular} _Note_: The increase is relative to the network (3 layers 32 cells) with hand-crafted features as input on the \(M_{0}\) model.  
\end{table} 
Table 6: The effect of different initialization methods on the \(M_{1}\) model.