# An Improved Federated Learning-Assisted Data Aggregation Scheme for Smart Grids
## 8 Evaluation  
We conducted an assessment of the honest-but-curious version of our system and provided a comprehensive overview of its performance in various aspects. Our scheme offers user validation of aggregated results, ensuring the privacy of users' sensitive data throughout and after the execution process. Additionally, users have the flexibility to exit the system while ensuring the integrity and accuracy of the final outcome at any stage of the system's operation. Through our evaluation, we aimed to demonstrate the effectiveness and reliability of our system, highlighting its capabilities in maintaining data security, supporting user validation, and accommodating user exits while preserving result correctness. Considering that the accuracy of the model in federated learning (FL) depends on two crucial factors--the number of participating users during training and the magnitude of the local gradient held by each user--we conducted our experiments with careful attention to detail. We meticulously recorded the user count and gradient size in each experiment, aiming to thoroughly examine the correlation between these variables, the model's accuracy, and the system's overhead. Figure 5 presents the classification accuracy and execution time in relation to various gradients in our experimental trials. The system's computation overhead and accuracy exhibit variations contingent upon the number of users and the magnitude of the gradient. As illustrated in Figure 5a, an augmentation in the number of local gradients possessed by users corresponds to an elevated accuracy in the model's outcome. However, it also results in greater computation overhead per round. Once the number of gradients surpasses a certain threshold, the model's accuracy tends to stabilize. Likewise, Figure 6 demonstrates that solely considering changes in the number of users, an elevation in user count enhances the computation overhead while improving the model's accuracy. An increase in both the number of users and gradients unavoidably amplifies the computation overhead. Consequently, it is advisable to select an appropriate number of users and gradients, taking into account the computational overhead, rather than simply striving for more. Figure 7 depicts the experimental evaluation of the accuracy of results obtained from the user verification server. During the verification stage, each user is tasked with validating the aggregated results received from the server and making a decision to either accept or reject them before updating their local model. Our analysis involves a diverse set of users, and we compare their verified server aggregation results with the ground truth. As illustrated in the figure, the validated data precisely align with the actual data, underscoring the confidence in the accuracy of the aggregated results generated by the server after undergoing the rigorous validation process. This validates the effectiveness of our approach, as the verification stage ensures the reliability of the server's aggregated outcomes and confirms that they remain consistent with the ground truth. Figure 5: No dropout, \(|\mathcal{U}|=100\). Figure 6: No dropout, \(|\mathcal{G}|=1000\). Figure 7: Verification accuracy. Figures 8 and 9 provide insight into the computation and communication overhead incurred by users during the execution of the system, measured in terms of runtime and data volume. As depicted in Figure 8, it is apparent that the computation overhead escalates as the number of local gradients possessed by users increases. Nevertheless, if only the number of users increases without any change in the local gradients, the computation overhead remains constant. These overheads primarily relate to the number of user gradients, meaning that an increase in the number of user gradients leads to an overall rise in the total overhead. Figure 9 demonstrates that the communication overhead of users follows a similar pattern as the computation overhead.