# Federated Learning for Secure In-Vehicle Communication
## 5 Results Analysis
### Federated Learning-Based Intrusion Detection Performance
#### 5.1.1 Per-Round Performance Evaluation  
In the following, we represent the history of accuracy and loss achieved by the two aforementioned approaches as a function of the number of training epochs. During the evaluation phase, the global model is assessed after each training round by aggregating the evaluation results from all participants. The goal is to track the improvement of the model accuracy over successive rounds.  
Figure 8 illustrates the accuracy history of FL-IVN-IDS-1E, FL-IVN-IDS-5E, FL-based LSTM-1E and LSTM-5E as a function of the number of rounds. We can notice that:  
* Our FL-IVN-IDS always learns faster and performs better than FL-based LSTM in both cases with one and five epochs and gives higher accuracies. We can notice that when only one epoch is applied using both approaches, the global accuracy of FL-IVN-IDS-1E and LSTM-1E are the same and equal to 98.2% but when the number of rounds is lower than 10, we can notice that FL-IVN-IDS-1E model has good convergence than FL-based LSTM-1E model. LSTM-1E model performs well in learning patterns in some rounds but does not generalize in others due to sensitivity to training data. It could be due to high variance and poor convergence.
* We can observe that when five epochs are applied using both approaches, the accuracy is higher than that used by using one epoch and achieved higher accuracy of 100% with very lower number of rounds. This can be explained by the fact that the training for more epochs allows the model to learn useful patterns rather than just noise in the dataset. However, the accuracy of our FL-IVN-IDS-5E model always increases with the number of rounds. On the other hand, when the number of rounds is between 10 and 21, the tain in LSTM-5E is not stable due to poor convergence.  
Figure 9 presents the loss history of FL-IVN-IDS-1E, FL-IVN-IDS-5E, FL-based LSTM-1E, and LSTM-5E across federated learning rounds. We can notice that:  
* Among all models, FL-IVN-IDS-5E achieves the lowest loss, demonstrating superior convergence and stability over multiple rounds. When only one epoch is applied using both approaches, the global loss of FL-IVN-IDS-1E and FL-based LSTM-1E are the same and equal to 0.05% but when the number of rounds is lower than 10, we can notice that our FL-IVN-IDS-1E model has good convergence and decrease faster than FL-based LSTM-1E model.  
Figure 8: Accuracy Training History of Distributed Learning for Per-Round Evaluation.  
when five epochs are applied using both approaches, the loss is lower than that used by using one epoch and achieved a global loss equal to \(10^{-5}\) of our model FL-IVN-IDS-5E lower than the loss of LSTM-5E that equal to 0.0085%. This can be explained by the fact that the training for more epochs allows the model allows the model to run more optimization iterations, adjusting its parameters and more and more reducing the loss.
* Overall, having a loss value that is better when using 5 epochs than when using 1 epoch indicates that the model was undertrained earlier and required more iterations before it could reach a better region of the loss landscape. This means that the model is still in the learning stage, being optimized by more training without overfitting. The loss of our FL-IVN-IDS-5E model always decreased with the number of rounds. On the other hand, the tail in LSTM-5E is not stable due to poor convergence when the number of rounds is between 10 and 21.