## 3 Experimental Results  
In this section, we provide detailed classification results obtained using the proposed audiovisual emotion recognition model, specifically observing the influence of implementing the federated averaging described in Section 2.2. Firstly, let us observe the performance of the proposed model without the accompanied federated learning considering the experimental setup described in Section 2.3. The results of classification accuracy are presented in Table 2 whereas detailed results of unimodal video and audio classifiers, without applied sequential voting strategy, are presented in Tables 3 and 4, respectively. It can be noticed that the proposed model for the video modality provides relatively good performance, achieving classification accuracy in the range of 78.70-87.94 (%) on a single frame of test video signals. Similarly, we can notice from Table 3 that the achieved average precision, recall and F1-score are relatively high, following the results of classification accuracy. However, detailed analysis leads us to the observation that there are differences among clients and among emotions. The best performance is achieved in the case of Client 3, whereas classification accuracy in the case of Client 2 is about 9% less in the case of a single spectrogram. After applying the proposed sequential voting strategy, the accuracy significantly increased. The achieved performance is in the range of 85.71-91.66 (%). As we use a lightweight model with only 0.37 million trainable parameters, pre-trained on the ImageNet, there is not so much space for further improvement in the case of the video modality. Also, the expected tradeoff between possible improvement and an increase in complexity led us to the decision not to implement federated averaging to the video modality. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Classification Accuracy (\%)**} \\ \cline{2-6}  & **Audio** & \multicolumn{4}{c}{**Video**} \\ \cline{2-6}  & **Single Spectrogram** & **Voting** & **Single Frame** & **Voting** & **Multimodal** \\ \hline Client 1 & 45.95 & 57.14 & 82.04 & 88.10 & 89.29 \\ \hline Client 2 & 48.44 & 61.90 & 78.70 & 85.71 & 91.66 \\ \hline Client 3 & 44.08 & 55.95 & 87.94 & 91.66 & 92.86 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Classification accuracies of the proposed AVER model without federated learning. The obtained results for the classification accuracy in the case of the audio modality (Table 2) indicate that the proposed classifier could not generalize well as the one in the case of the video modality. Similar behavior was also observed in the literature [15; 17]. By observing Table 4, it can be noticed that the average results for precision, recall, and F1-score are in accordance with the results for classification accuracy. This motivated us to apply federated learning framework to the clients in the case of the audio modality in order to provide better generalization. These results are presented in Tables 5 and 6. The performance of the video modality in Table 6 is omitted as we do not apply federated learning for this modality. For the purpose of multimodal inference, we consider the performance of the video modality from Table 2.  
In order to compare the achieved accuracy improvement using the federated learning model, we define classification accuracy gain as an improvement of the proposed federated model over the non-federated model as:  
\[Gain_{f}[\%]=CA_{f}-CA_{nf}, \tag{3}\]  
where \(CA_{f}\) represents the classification accuracy of the observed client achieved using federated learning, whereas \(CA_{nf}\) represents the classification accuracy of the same client achieved by training the model without client federation. The achieved gains in the case of the audio modality and multimodal inference are presented in Figure 3.