The Impact of Differential Privacy on Group Disparity Mitigation (2024)

Abstract: The performance cost of differential privacy has, for some applications, been shown to be higher for minority groups; fairness, conversely, has been shown to disproportionally compromise the privacy of members of such groups. Most work in this area has been restricted to computer vision and risk assessment. In response, we evaluate the impact of differential privacy on fairness across four diverse tasks, focusing on how attempts to mitigate privacy violations and between-group performance differences interact: Does privacy inhibit attempts to ensure fairness? To this end, we train (\varepsilon,\delta) -differentially private models with empirical risk minimization and group distributionally robust training objectives. Consistent with previous findings, we find that differential privacy increases between-group performance differences in the baseline setting; more interestingly, differential privacy reduces between-group performance differences in the robust setting. We explain this by interpreting differential privacy as regularization.

## introduction classification tasks in computer vision and natural language processing face the challenge of balancing performance with the need to prevent discrimination against protected demographic subgroups, satisfying fairness principles. in some tasks, we train our classifiers on private data and therefore also need our models to satisfy privacy guarantees. privacy-preserving algorithms, however, tend to disproportionally affect members of minority classes @xcite . e.g., @xcite , show the performance cost of differential privacy @xcite in face recognition is higher for minority groups, suggesting that privacy and fairness are fundamentally at odds @xcite . in this paper, we evaluate two hypotheses at controls the level of privacy (rather than its scope). note that it also follows directly that if ε = 0 and δ = 0, absolute privacy is required, leading θ to be independent of the data. several authors have shown that differential privacy comes at different costs for minority subgroups @xcite @xcite . the more private the model is required to be, the larger group disparities it will exhibit. 4 this happens because differential privacy distributes noise where it is needed to reduce the influence of individual examples. since outlier examples are likely to have disproportional influence on output distributions @xcite , they are also disproportionally affected by noise injection in differential privacy. @xcite show that, in fact, a (ε, 0)private and fully fair model -using equalized odds as the definition of fairness -will be unable to learn anything. to see this, remember that a fully private model is independent of the data and unable to learn from correlations between input and output. if θ is, in addition, required to be fair, it is thereby required to be fair for all distributions, which prevents θ from encoding any prior beliefs about the output distribution. note this finding generalizes straight-forwardly to equalized risk, and even to approximate fairness (since even for finite distributions, we can define a ∆ > 0, such that preserving absolute privacy would lead to a constant θ). theorem 1. for sufficiently small values of ∆, a fully (ε, 0)-private model θ that is also ∆-fair, will have trivial performance. proof. this follows directly from the above. while we do not strictly require an absolute privacy in our experiments (setting δ = 10 -5 ), intuitively, privacy compromises fairness by adding more noise to data points of minority group members than to those of majority groups. fairness, on the other hand, leads to over-sampling or overattending to data points of minority group members, more likely compromising their privacy. @xcite show, however, that it is possible to learn somewhat private and tion d is a mixture of a discrete number of groups, 199 d g for @xmath0, we define the worst-case loss as the 200 maximum of the group-specific expected losses: in group dro -in contrast with erm -we exploit per-batch gradient where its norm exceeds a predetermined clipping bound c, and by (ii) adding gaussian noise n characterized by a noise scale σ to the aggregated per-sample gradients. we control this influence with a privacy budget ε, where lower values for ε indicates a more strict level of privacy. dp-sgd has remained popular, among other things because it generalizes to iterative training procedures (mcmahan et al., 2018), and supports tighter bounds using the rényi method (mironov, 2017). differential privacy generally comes at a performance cost, leading to privacy-preserving models performing worse compared to their non-private counterparts (alvim et al., 2011). however, we follow kerrigan, slack, and tuyls (2020) and finetune the private models, which are first pretrained (without differential privacy) on a large public dataset. this protocol generally seems to provide a better trade-off between accuracy and privacy (kerrigan, slack, and tuyls, 2020), leading to betterperforming, yet private models. the only exception to this setup is the volatility forecasting task, where our models were trained from scratch, as those rely on praat audio features. ## related work fair machine learning early work on mitigating group-level disparities included oversampling @xcite and undersampling @xcite , as well as instance weighting @xcite . other proposals modify existing training algorithms or cost functions to obtain fairness @xcite . in the context of large-scale deep neural networks, group dro is a particularly interesting approach to mitigating group-level disparities (creager, jacobsen, and zemel, 2021). see @xcite and corbett-davies and goel (2018) for interesting discussions of how fairness has been measured. more recent alternatives to group dro include invariant risk minimization (arjovsky et al., 2020), spectral decoupling @xcite and adaptive risk minimization @xcite . we ran experiments with both invariant risk minimization and spectral decoupling, but they performed much worse than group dro.