## 5 Experiments and Results
### Experiment II: Comparison with Different Models  
To verify the generalization of the MSFL proposed in this paper, a comparative experiment was conducted between MSFL and other baseline models on the categorical corpora. For the baseline models, five models were selected, where the MSFL_ST and MTL_DNN were performed by us, and other models were selected from research from the past three years, retaining their experimental results from the literature. Note that MSFL_ST is set to verify the generalization ability in MTL, and the results of the MTL-DNN model are obtained by reproducing the literature [41]. The experimental results are shown in Table 6. Since not all studies in multilingual SER use the same corpora as this study, some models show incomplete corpus recognition results in Table 6. To further extend our comparison scope, the model performance is evaluated from a macro perspective and we explored the contribution of MTL on emotion categories in each corpus from a microscopic perspective. From a macro perspective, our evaluation is first started by benchmarking the performance of the ST model over the MT model (i.e., MSFL_ST and MSFL). As shown in Table 6, the emotion recognition performance of each corpus improved with an average improvement of 4.49%. Additionally, the performance of MSFL also outperformed in EMO-DB, CASIA, and SAVEE, which shows significant improvement of 3.36%, 5.36%, and 4.75%, respectively. This proves that introducing MTL can improve the multilingual SER effect using language recognition as an auxiliary task. Then, the performances among the other models were compared, including MT_DNN, MT-SHL-DNN, CAbilS, and Ensemble Model. Except for the result in the Ensemble Model on EMO-DB, MSFL outperforms the other models on three corpora, where the average improvement is 3.37% compared with MTL-DNN. The results prove the robustness and effectiveness of the MSFL model proposed in this paper. From a micro perspective, two comparisons based on the confusion matrix are performed. The first one compares the confusion matrixes of MSFL_ST and MSFL to demonstrate the contribution of MTL for each emotion category. The second one compares the confusion matrix of MSFL and MTL_DNN to demonstrate that the model proposed in this paper outperforms the MTL_DNN in each emotion category. Figures 4-6 show the confusion matrix for MSFL_ST, MTL_DNN, and MSFL on EMO-DB, CASIA, and SAVEE, respectively. The confusion matrix describes the accuracy and the degree of confusion between four emotion categories. The x-axis in the confusion matrix illustrates the predicted labels, and the y-axis illustrates the ground-truth labels. Values on the diagonal indicate the proportion of correct predictions, while the remaining values indicate the \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Models** & **EMO-DB** & **CASIA** & **SAVEE** & **Average** \\ \hline MSFL\_ST & 81.93\% & 79.05\% & 71.14\% & 77.37\% \\ MTL\_DNN & 81.14\% & 82.30\% & 72.03\% & 78.49\% \\ MT-SHL-DNN (2019) [42] & 82.34\% & - & - & - \\ CAbilS (2020) [39] & 75.61\% & 64.33\% & - & - \\ Ensemble Model (2021) [35] & 89.75\% & - & 69.31\% & - \\ MSFL (our proposed) & 85.29\% & 84.41\% & 75.89\% & 81.86\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Accuracy of different models for SER on discrete corpora. proportion of incorrect predictions (i.e., confusion). For the first comparison, except for the reduction of neutral accuracy in SAVEE, the recognition accuracy of almost all emotions in the three corpora on MSFL outperform MSFL_ST, especially emotions in CASIA, are all improved. The reason for the exception of neutral accuracy in SAVEE is the unbalanced sample size, where the number of neutral samples is twice that of other emotions. This leads to more biased recognition results in identifying neutral when conducting monolingual ST experiments, and especially sadness in SAVEE is extremely easy to confuse as neutral.