## 4 Experiment and Results
### Comprehensive Analysis of Weighting Schemes  
To further enhance the model's performance, various weighting schemes were explored, as follows:  
1. Equal Weighting Scheme: This scheme assigns equal importance to both discrete and dimensional emotion tasks. While simple to implement, it might not always yield the best results but serves as a baseline to identify improvements from other schemes;
2. Task-Specific Weighting Scheme: Weights are assigned based on the importance of each task or the dataset's characteristics. This scheme leverages domain knowledge or prior research findings to assign appropriate weights for discrete and dimensional tasks;
3. Dynamic Weighting Scheme: This approach updates the weights during training based on the model's performance on each task. By allocating more resources to tasks with higher error rates or lower performance, the model can adapt to the difficulty of each task, potentially leading to improved overall performance. Dynamic weighting is particularly beneficial in scenarios where the importance of tasks may vary during training. As highlighted in Table 8, this scheme produced strong results, especially when paired with Wav2 Vec 2.0, achieving a WA of 72.66% and a UA of 73.72%;
4. Joint Weighting Scheme: Similar to dynamic weighting, joint weighting adjusts the weights dynamically at each training epoch. However, it includes the weight in the gradient computations during backpropagation, allowing the model to learn the optimal balance between tasks. This method can enhance overall performance by fine-tuning the importance of each task during the training process. This scheme demonstrated the highest performance for HuBERT, achieving a WA of 70.37% and a UA of 70.27%.  
The results from applying these weighting schemes to the joint model architecture using raw PCM values with Wav2 Vec 2.0 and HuBERT feature encoders are summarized in Table 6. The dynamic weighting scheme achieved the best performance for Wav2 Vec 2.0, with a WA of 69.31%, a UA of 68.37%, and a mean CCC of 0.3736. For HuBERT, the joint weighting scheme outperformed others, with a WA of 70.37%, a UA of 70.27%, and a mean CCC of 0.3477.  
Based on these experiments, the proposed joint model architecture, incorporating multiple feature encoders and attention mechanisms, significantly improves the performance of Speech Emotion Recognition tasks. By effectively leveraging both discrete and dimen  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Reference** & **Proposed Method** & **Weighted Average (WA)** & **Unweighted Average (UW)** \\ \hline ([PERSON] et al., 2021) [37] & LSTM & 64.3\% & 61.1\% \\ ([PERSON] et al., 2021) [38] & SUPERB & - & 67.62\% \\ ([PERSON] et al., 2021) [39] & Wav2 Vec 2.0 & - & 67.2\% \\ ([PERSON] et al., 2022) [35] & Multi-channel & 69.80\% & 71.05\% \\ ([PERSON] et al., 2022) [40] & HuBERT Base & 69.9\% & - \\  & HuBERT Large & 74.2\% & - \\ ([PERSON] et al., 2024) [36] & BERT-RoBERTa & 73.95\% & 74.27\% \\ ([PERSON] et al., 2024) [41] & Wav2 Vec 2.0 & - & 74.0\% \\ Proposed Method (ours) & Parallel Model with Wav2 Vec 2.0 & 72.66\% & 73.72\% \\  & (Dynamic Weighting) & & \\ Proposed Method (ours) & Parallel Model with HuBERT 2.0 & 71.03\% & 70.38\% \\  & (Dynamic Weighting) & & \\ \hline \hline \end{tabular}
\end{table}
Table 8: Baseline performance on other models.  
sional emotion representations, the joint model achieves higher accuracy and robustness compared to baseline models and the existing approaches. The exploration of different loss functions and weighting schemes further enhances the model's adaptability and performance, demonstrating its potential for advanced emotion recognition applications.