# A Novel Multi-Server Federated Learning Framework in Vehicular Edge Computing
## 6 Experimental Results
### Simulation Results
#### 6.4.1 Performance Evaluation of the Proposed Server-Level Aggregation Methods  
To benchmark our proposed MS-FL framework against the vanilla FL setup [5]--which does not share knowledge between servers--we adopted the same shard-based non-IID distribution from [5]. Specifically, we sorted the dataset by class label (43 categories), divided it into 96 shards of 222 samples each, and assigned two shards to each of the 48 clients. We then ran 500 training rounds and reported training loss, training accuracy, test accuracy, F1-score, precision, and recall for our MS-FL aggregation methods. For comparison, we include the results for the baseline FL model without any server-level evaluation or aggregation--referred to as the \"Without Aggregation\" (WA) model.  
Table 3 summarizes the performance metrics, average test recall, test accuracy, test precision, and test F1-score for the proposed aggregation methods and vanilla FL in a multi-server FL framework with three servers. The results, averaged across the servers after 500 training rounds, indicate that SPAA outperforms all other methods across all evaluation metrics, achieving the highest test recall (75%), test accuracy (84%), precision (78%), and F1-score (76%). This performance underscores the effectiveness of SPAA's statistical weighting and aggregation technique in enhancing model performance. DWAA and SA also deliver strong results, surpassing BA and significantly outperforming WA in all metrics, highlighting the benefits of performance-based aggregation methods. Although SPAA, DWAA, and SA exhibit slightly higher runtimes than WA due to additional evaluation and aggregation steps, the substantial gains in accuracy and recall justify the increased computational cost, making SPAA a highly effective method for MS-FL in environments with data heterogeneity.  
Figure 4 presents comparisons of training accuracy and training loss, between the proposed aggregation methods and vanilla FL [4] over 500 training rounds. The results are averaged across three servers for each method, providing a comprehensive view of model performance in an FL setting. The proposed methods, particularly SPAA and DWAA, demonstrate improved training accuracy compared to vanilla FL (i.e., WA). Additionally, SPAA, DWAA, and SA exhibit better results in terms of training loss compared to vanilla FL.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Differed Aggregation Methods at Server Level**} & \multicolumn{5}{c}{**Performance Evaluation Parameters**} \\ \cline{2-6}  & **Recall** & **Test Accuracy** & **Precision** & **F1-Score** & **Total Run Time (ms)** \\ \hline SPAA & 75\% & 84\% & 78\% & 76\% & 36,211 \\ \hline DWAA & 74\% & 83\% & 77\% & 75\% & 36,211 \\ \hline SA & 74\% & 82.5\% & 75.5\% & 74.3\% & 36,175 \\ \hline BA & 72.7\% & 82.9\% & 73.5\% & 71.5\% & 36,148 \\ \hline WA & 49\% & 62.6\% & 38\% & 41\% & 28,042 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Performance evaluation on GTSRB dataset after 500 training rounds for MS-FL.  
To isolate SPAA's contribution, we disable its z-score penalty--setting every penalty to 1--so that SPAA reverts to a loss-weighted DWAA where weights are proportional to raw loss values. We then compare the average test accuracy curves to reveal the impact of the statistical penalty. As Figure 5 shows, SPAA with the z-score penalty delivers slightly higher accuracy at both 50 and 200 rounds--particularly in the early phases--indicating faster convergence. While the absolute difference is modest in our benign setting, we expect the penalty to provide substantially larger benefits under adversarial or high-noise conditions.  
Figure 4: MS-FL training on GTSRB dataset (shard-based non-IID) over 500 rounds with a CNN: (a) average training accuracy; (b) average training loss (log scale on the y-axis).