# A Feature Fusion Model with Data Augmentation for Speech Emotion Recognition
## 5 Experiment Results
### MHA-CRNN and Feature Fusion Model  
1. A key parameter of the multi-head attention mechanism is the number of attention heads, which determines how many linear transformations and self-attention operations are performed on the input, and the core the determination of the best num_ heads. To verify the effect of num_heads on model performance, four values of 1, 4, 8, and 16 are selected for the experiments, and the size of each head is 128, 256, 512 and 1024, to be consistent with the input vector dimension before transformation. We use the IEMOCAP on four types of samples--neutral, happy, sad, and angry--with a total of 5531 data, and the training set, validation set, and test set in the ratio of 7:1:2. We conducted a five-fold crossover experiment, and reported the UAA for the five experiments. A comparison of the experimental results is shown in Figure 8.  
The model achieves the highest recognition rate when the number of attention heads is 8, and the lowest when the number of attention heads is 1. When the number of multiple attention heads is 1, it is equivalent to the original self-attention model. It can be seen  
Figure 8: Comparison of the number of different attention heads in the MHA-CRNN structure. Figure 7: Classification performance of feature subsets obtained using two feature selection methods. that the multi-head attention mechanism has achieved better results than the single head self-attention mechanism. When the number of heads is 8, the highest recognition accuracy is 65.5%, and when the number of heads is 16, the lowest is 63.02%, but both are higher than the recognition accuracy of 62.54% obtained by single head self-attention mechanism. Moreover, increasing the number of attention heads does not always lead to performance improvement. As the number of attention heads increases from 4 to 16, the experimental results show an increasing and then decreasing trend, and the performance is optimal at eight heads. This indicates that more heads are not better, and that there is a saturation value for dividing the attention subspace (number of heads), and that too many or too few heads can affect performance. The number of attention heads determines the number of spatial vectors for linear transformations of the input. The larger the number, the more attention analysis can be performed on the input from more scales, but too many spatial transformations will affect the retention of the original feature information of the input, so choosing the appropriate number of attention heads is an important step in adjusting the attention parameters. Based on the above experiments, after determining that the optimal number of attention heads is equal to 8, the influence of the size of attention head (head_size) on the performance of the model is further explored. Four attention head sizes are selected for the experiment. The attention head size determines the length of the linear mapping of the input vector in the multi-head attention mechanism. Finally, the length of the output of the multi-head attention mechanism is equal to the size of the attention head multiplied by the number of heads. The experimental comparison of different attention head sizes is shown in Figure 9.  
Here, the input of the multi-head attention mechanism layer is [19; 1024], When the number of heads is 8, the head size is 128 so that the input and output dimensions of the multi-head attention mechanism layer are equal. When head size is larger than 128, the later Bi-LSTM will obtain longer input of time-series sentiment information. As shown in Figure 9, the classification results obtained when the number of attention heads is 256, 512, and 1024 are all better than the case when the head size is 128. However, the increase of attention head size does not always lead to performance improvement, and either too large or too small will lead to performance variation. This proves that choosing the right attention head size is also an important step in tuning the attention parameters.