# Speech Emotion Recognition Using Deep Learning Transfer Models and Explainable Techniques
## 4 Late-Fusion Model Design with VGGish and YAMNet  
### Features of VGGish and YAMNet Models  
Figure 4 shows the structure of the VGGish model developed by Google that is used for speech recognition and classification in this study. It is designed as a CNN by modifying an existing visual geometry group (VGG) network for image classification in speech data processing.  
The model above, designed to extract features from 2D data rather than 1D data in the time domain, uses a convolutional layer that learns the spatial patterns and features of an input image through weights and batch normalization. An activation function layer, which allows learning highly complex patterns by applying nonlinear features, is used to output the spatial feature vectors of the audio images for analysis and classification. A global pooling layer is not applied to preserve the spatial feature information in the time-frequency domain of the input spectrogram. The number of channels in the fully connected layer is designed to be large enough to generate a feature map larger than the input data. The output information is synthesized and used for classification, utilizing the location characteristics of the time-frequency domain that can be obtained for each pooling layer.  
Figure 3: Applying Gaussian data selection to a speech spectrogram.  
In VGGish, feature maps were extracted for each pooling layer, except for the first pooling layer. The size of the feature map extracted from the network is [24 \(\times\) 16 \(\times\) 128/12 \(\times\) 8 \(\times\) 256/6 \(\times\) 4 \(\times\) 512], respectively. Each feature map aggregates information that can be obtained from each layer by adjusting and fusing the number of channels.  
Similar to VGGish, YAMNet, as shown in Figure 5, is an audio classification model developed by Google. It is a transfer learning model that learns from a large-scale dataset to perform 512 different speech recognition tasks, including animal sounds, cars, music, nature, and human conversations. This model is based on a CNN and consists of a convolution layer that can extract spatial patterns and features of multiple images. It also contains a pooling layer, which increases computational efficiency by reducing the spatial dimensions, as well as an activation function layer. Using a global pooling layer before the classification layer, the model focuses on the channel characteristics of the extracted feature map rather than the time-frequency positional characteristics of the feature map input from the previous layer. It is designed to identify the overall features of the input spectrogram image and does not rely on the positional features. Figure 6 shows the classification process of combining the feature maps obtained from each transfer learning model. Here, feature maps in VGGish were obtained by each pooling layer except for the first pooling layer.  
Figure 4: VGGish network structure for video game genre identification using spectrograms and headphones (VGGish), incorporating rectified linear unit (ReLU).  
Figure 5: Diagram illustrating the network structure of YAMNet (yet another mobile network) and its integration with the genetic algorithm (GA).