## 4 Experiments and Evaluation
### Key Finding  
In general, our findings help to build robust and privacy-preserving approaches for FL by shedding light on the effectiveness of noise in minimizing MIAs while acknowledging the inherent trade-off with model performance. This understanding enables practitioners to make informed decisions that emphasize data protection and meaningful insight, even in difficult situations, such as the analysis of sensitive payroll data. Table 3 provides important information on how noise multipliers affect the performance of a DPFL model and privacy guarantees. The performance of both data sets was comparatively consistent at all noise levels. This implies that the model's capacity to learn well was not considerably impacted by the addition of noise for privacy protection. * Opinion Data: At the end of the training, the accuracy and F1 score showed a slight improvement for several noise levels. This analysis indicates that the model's ability to detect minor details in the data can improve at certain noise levels. It could be very instructive to take a closer look at these noise levels and the kinds of particular insights that they might provide. Figure 5: Payroll data: epsilon (\(\epsilon\)) and Renyi differential privacy in relation with round number and noise multipliers. Figure 6: Payroll data: inference attack F1 score success rates of un-noisy and noisy model. Lower rates indicate stronger privacy against attacks. Colors/bars represent different noise levels. * Payroll Data: There were clear relationships in the F1 score for the payroll data. It dropped quickly at first, but as training continued, it started to gradually increase and slow down. This relationship raises two important ideas, such as the following:
* Initial Difficulties: The model's initial inability to detect nuances in the payroll data may have been impeded by higher noise levels created by smaller multipliers, which is what caused the F1 score to drop quickly. * Improvement Potentials: The modest increase and gradual decrease observed in the F1 score show that the model is still learning despite noise. It is possible that the F1 score will also show a significant increase with more training iterations (beyond 30 rounds) while still offering respectable privacy guarantees. It is recommended to investigate how prolonged training affects the F1 score. Table 3 also provides information on the relationship between noise levels and privacy guarantees in DPFL. We use RDP as a metric to quantify these guarantees, with higher values representing stronger privacy protection. Evaluating the RDP values between training rounds (1 to 30) for each noise level indicates a consistent behavior. RDP values frequently drop from the beginning (Start) to the end (End) of the training. This represents a gradual weakening of privacy assurances as the model learns from the injected noise. This observation is consistent with the inherent privacy-utility trade-off in DPFL, in which improved utility (model performance) often comes at the cost of decreased privacy. Two important measures that quantify the trade-off between privacy and performance in DPFL for both opinion and payroll data are epsilon (\(\epsilon\)) and RDP, as can be seen in Table 3. Epsilon (\(\epsilon\)) shows a continuous sequence with smaller values, denoting higher privacy assurances. For both data sets, it begins at a low value and increases over training rounds. This is consistent with the intrinsic trade-off in DPFL: larger noise multipliers, which introduce less noise into the training set, provide better privacy at first (lower epsilon (\(\epsilon\))), but they may also result in more privacy loss over time (higher epsilon (\(\epsilon\))). On the other hand, weaker initial privacy (higher epsilon (\(\epsilon\))) is produced by smaller noise multipliers (which introduce more noise), but they may also result in a less gradual loss of privacy. * Opinion Data: Epsilon (\(\epsilon\)) often increased gradually during the training period, indicating the inherent trade-off between privacy and performance.