# Hierarchical Federated Learning-Based Intrusion Detection for In-Vehicle Networks
## 4 Results and Evaluations
### Performance Results and Analysis  
We evaluate the performance of the proposed IDS within an H-FL architecture using two datasets, each with three non-IIDlevels, resulting in a total of 24 scenarios: 12 for dataset 1 and 12 for dataset 2. Figure 8 shows the average F1-score performance of the ANN model and the proposed cascaded IDS across different non-IID levels in the car hacking dataset [29]. As depicted in Figure 8, the ANN classifier's performance improved at all non-IID levels except when (\(\mu=0.5\)), where it slightly decreased after applying the H-FL model. In contrast, the F1-score for the cascaded multistage IDS increased across all non-IID levels, as shown in Figure 8. Notably, the H-FL improved the F1-score in 10 out of 12 scenarios in the car hacking dataset. This indicates that deploying the model in an H-FL architecture significantly enhanced performance, enabling it to more effectively identify both known and unknown attacks by not only learning from participating clients' data but also from a broader range of client data and evolving threats. The reasoning behind this is that an edge-FL approach, when applied to a limited number of vehicles with specific data and attack scenarios, confines the learning process to the vehicles participating within that particular edge-FL network. This restriction can limit the model's ability to generalize to diverse attack patterns not represented in the local FL process. In contrast, H-FL aggregates models from multiple edge servers across different areas into a central server, then redistributes the global model back to the edges and ultimately to all participating vehicles. This process enables vehicles to leverage insights from a broader range of data and attack scenarios across various regions, enhancing the IDS's adaptability to new and varied threats, and ultimately increasing robustness and accuracy in attack detection across the network. Figure 8 also reveals a notable difference between Edge 1 and Edge 2, primarily due to the high imbalance in data distribution when (\(\mu=0.3\)), compared to (\(\mu=0.5\)) and (\(\mu=0.7\)). Across all non-IID distributions, the H-FL approach consistently outperforms the edge-FL approach, with percentage increases ranging from 0.57% to 10.63%, highlighting a significant performance improvement with the H-FL method. The increase may vary based  
Figure 8: Average F1-score results non-IID levels across communication rounds for car hacking dataset [29]. on the clients selected under each edge and non-IID level, as demonstrated by the Edge 2 results in Figure 7(b). These results underscore the critical impact of the participating clients' data on performance. Detailed F1-score performance is available in Figures 9 and 10, which further illustrate the F1-score performance of the ANN model and the proposed cascaded IDS across various non-IID levels. Table 5 shows the average recall and precision of our IDS in an H-FL environment. The results indicate an increase in both recall and precision after deploying the H-FL model at nearly all non-IID levels. The best results are obtained when the data are distributed across clients with a low level of imbalance, such as at \((\mu=0.7)\). Figure 11 demonstrates the distributed loss across communication rounds. Results show that the model has different convergence speeds across non-IID levels. We also evaluated  
Table 6 shows the average training time in seconds (S) for both the ANN and LSTM models, as well as the total training time for one round. The results indicate that training time depends on the number of training data, with Client 6 having the highest training time of 75.35 s due to its large training dataset, and the inverse being true for clients with smaller datasets. We also measured the network load by calculating the size of weights transferred between the server and edges, as well as between vehicles and edges. The weights size is 0.966 MB.