Pruning has a disparate impact on model accuracy
Abstract
Network pruning is a widely-used compression technique that is able to significantly scale down overparameterized models with minimal loss of accuracy.
This paper shows that pruning may create or exacerbate disparate impacts.
The paper sheds light on the factors to cause such disparities, suggesting di ff erences in gradient norms and distance to decision boundary across groups to be responsible for this critical issue.
It analyzes these factors in detail, providing both theoretical and empirical support and proposes a simple, yet e ff ective, solution that mitigates the disparate impacts caused by pruning
Introduction
As deep learning models evolve and become more powerful, they also become larger and more costly to store and execute.
The trend hinders their deployment in resource-constrained platforms, such as embedded systems or edge devices, which require e ffi cient models in time and space.
To address this challenge, studies have developed a variety of techniques to prune the relatively insignificant or insensitive parameters from a neural network while ensuring competitive accuracy [1, 5, 7, 30, 31, 32, 41].
When a model needs to be developed to fit given and certain requirements in size and resource consumption, a pruned model which is derived from a large, rigorously-trained, and (often) over-parameterized model, is regarded as a de-facto standard.
That is because it performs incomparably better than a same-size dense model which is trained from scratch, when the same amount of e ff ort and resources are invested.
In spite of its strengths, pruning has been showed to induce or exacerbate disparate e ff ects in the accuracy of the resulting reduced models [20, 19].
Intuitively, the removal of model weights a ff ects the process in which the network separates di ff erent classes, which can have contrasting consequences for di ff erent groups of individuals.
This paper further shows that the accuracy of the pruned models tends to increase (decrease) more in classes that had already high (low) accuracy in the original model, leading to a “the rich get richer” and “the poor get poorer” e ff ect.
This Matthew e ff ect is illustrated in
Figure 1.
The figure shows the accuracy of a facial recognition task on di ff erent demographic groups for several pruning rates (indicating the percentage of parameters removed from the original models).
Notice how the accuracy of the majority group (White) tends to increase while that of the minority groups tends to decrease as the pruning ratio increases.
Following these observations, we shed light on the factors to cause such disparities.
The theoretical findings suggest the presence of two key factors responsible for why accuracy disparities arise in
[Figure]
pruned models: (1) disparity in gradient norms across groups, and (2) disparity in Hessian matrices associated with the loss function computed using a group’s data.
Informally, the former carries information about the groups’ local optimality, while the latter relates to model separability.
We analyze these factors in detail, providing both theoretical and empirical support on a variety of settings, networks, and datasets.
By recognizing these factors, we also develop a simple yet e ff ective training technique that largely mitigates the disparate impacts caused by pruning.
The method is based on an alteration of the loss function to include components that penalize disparity of the average gradient norms and distance to decision boundary across groups.
These findings are significant: Pruning is a key enabler for neural network models in embedded systems with deployments in security cameras and sensors for autonomous devices for applications where fairness is an essential need.
Without careful consideration of the fairness impact of these techniques, the resulting models can have profound e ff ects on our society and economy .
Related work
Fairness and network pruning have been long studied in isolation.
The reader is referred to the related papers and surveys on fairness [4, 8, 11, 17, 24] and pruning [1, 5, 7, 30, 31, 32, 33, 41] for a review on these areas.
The recent interest in assessing societal values of machine learning models has seen an increase of studies at the intersection of di ff erent properties of a learning model and their e ff ects on fairness.
For example, Xu et al.
[39] studies the setting of adversarial robustness and show that adversarial training introduces unfair outcomes in term of accuracy parity [42].
Zhu et al.
[44] show that semisupervised settings can introduce unfair outcomes in the resulting accuracy of the learned models.
Finally, several authors have also shown that private training can have unintended disparate impacts to the resulting models’ outputs [3, 13, 34, 36, 43] and downstream decisions [29, 35].
Network compression has also been shown to have a profound impact towards the model fairness.
For example, several works observed empirically that network compression may amplify unfairness in di ff erent learning tasks [27, 19, 20, 22].
Most of the focus has been on vision tasks and in identifying the set of Pruning Identified Exemplars (PIEs), the samples that are impacted most under the compression scheme and conclude that PIEs belongs to low frequency groups (those observed at the tail of the data distribution).
Blakeney et al.
[6] further investigate how bias could be evaluated and mitigated in pruned neural networks using knowledge distillation while Hosseini et al.
[21] observed empirically that knowledge distillation processes may produce unfair student models.
The impact of network compression towards fairness has also been assessed in natural language tasks.
For example, Du et al.
[10] and Xu et al.
[37] empirically measure the robustness of compressed large language models, while Ahia et al.
[2] look into how compression schemes a ff ects data-limited regimes.
Finally, Xu and Hu [38] investigate ways to improve fairness in generative language models by compressing them.
We also note that, concurrently to this work, Good et al.
[14] studied the relative distortion in recall for various classes. They show that pruning has a Matthews e ff ect on the recall for various classes and propose an algorithm to attenuate such an e ff ect.
This paper builds on this body of work and their important empirical observations and provides a step towards a deeper theoretical understanding of the fairness issues arising as a result of pruning.
It derives conditions and studies the causes of unfairness in the context of pruning as well as it introduces mitigating guidelines.
Problem settings and goals
The paper considers datasets D consisting of n datapoints ( x i , a i , y i ), with i ∈ [ n ], drawn i.i.d.
from an unknown distribution Π .
Therein, x i ∈ X is a feature vector, a i ∈ A with A =
[ m ] (for some finite m ) is a demographic group attribute, and
y i ∈ Y is a class label.
For example, consider the case of a face recognition task.
The training example feature x i may describe a headshot of an individual, the protected attribute a i may describe the individual’s gender or ethnicity, and y i represents the identity of the individual.
The goal is to learn a predictor f θ : X → Y , where θ is a k -dimensional real-valued vector of parameters that minimizes the empirical risk function:
[Equation]
where ℓ : Y × Y → R + is a non-negative loss function that measures the model quality.
We focus on analyzing properties arising when extracting a small model f ¯ θ with ¯ θ ⊂ ⋆ θ of size | ¯ θ | =
¯ k ≪ k .
Model
f ¯ θ is constructed by pruning the least important values or filters from vector ⋆ θ (i.e., those with smaller values in magnitude) according to a prescribed criterion, such as an ℓ p norm [25, 31].
The paper focuses on understanding the fairness impacts (as defined next) arising when pruning general classifiers, such as neural networks.
Fairness The fairness analysis focuses on the notion of excessive loss , defined as the di ff erence between the original and the pruned risk functions over some group a ∈ A :
[Equation]
where D a denotes the subset of the dataset D containing samples ( x i , a i , y i ) whose group membership a i = a .
Intuitively, the excessive loss represents the change in loss (and thus, in accuracy) that a given group experiences as a result of pruning.
Fairness is measured in terms of the maximal excessive loss di ff erence , also referred to as fairness violation :
[Equation]
defining the largest excessive loss di ff erence across all protected groups.
(Pure) fairness is achieved when ξ ( D ) = 0, and thus a fair pruning method aims at minimizing the excessive loss di ff erence.
The goal of this paper is to shed light on why fairness issues arise (i.e., R ( a ) > 0) as a result of pruning,
why some groups su ff er more than others (i.e., R ( a ) >
R ( a ′ )), and what mitigation measures could be taken to minimize unfairness due to pruning.
We use the following notation: variables are denoted by calligraph symbols, vectors or matrices by bold symbols, and sets by uppercase symbols.
Finally, ∥ · ∥ denotes the Euclidean norm and we use f θ ( x ) to refer to the model’ soft outputs.
All proofs are reported in Appendix A.
Fairness analysis in pruning: Roadmap
To gain insights on how pruning may introduce unfairness, we start with providing a useful upper bound for a group’s excessive loss.
Its goal is to isolate key aspects of model pruning that are responsible for the observed unfairness.
The following discussion assumes the loss function ℓ ( · ) to be at least twice di ff erentiable, which is the case for common ML loss functions, such as mean squared error or cross entropy loss.
Theorem 1.
The excessive loss of a group a ∈ A is upper bounded by 2 :
[Equation]
where g ℓ a = ∇ θ J ( ⋆ θ ; D a ) is the vector of gradients associated with the loss function ℓ evaluated at ⋆ θ and computed using group data D a , H ℓ a = ∇ 2 θ J ( ⋆ θ ; D a ) is the Hessian matrix of the loss function ℓ , at the optimal parameters
vector ⋆ θ , computed using the group data D a (henceforth simply referred to as group hessian ), and λ ( Σ ) is the maximum eigenvalue of a matrix Σ .
The bound above follows from a second order Taylor expansion of the loss function, Cauchy-Schwarz inequality, and properties of the Rayleigh quotient.
Notice that, in addition to the di ff erence in the original and pruned parameters vectors, two key terms appear in Equation (9): (1) The norms of the gradients g ℓ a and (2) the maximum eigenvalue of the Hessian matrix H ℓ a for a group a .
Informally, the former is associated with the groups’ local optimality while the latter relates to the ability of the model to separate the groups data.
As we will show next these components represent the main sources of unfairness due to model pruning.
The following is an important corollary of Theorem 1.
It shows that the larger the pruning, the larger will be the excessive loss for a given group.
Corollary 1.
Let ¯ k
and ¯ k ′ be the size of parameter vectors ¯ θ and ¯ θ ′ , respectively, resulting from pruning model
f ⋆ θ , where ¯ k < ¯ k ′ (i.e., the former model prunes more weight than the latter one).
Then, for any group a ∈ A ,
[Equation]
where ˜ R ( a , ω ) is the excessive loss upper bound computed using pruned model parameters ω (Eq.
The corollary above indicates that the excess risk for a group increases as the pruning regime increase.
Building on this result, the paper illustrates next why unfairness can become more significant as the pruning regime increases.
The next sections analyze the e ff ect of gradient norms and the Hessian to unfairness in the pruned models.
The theoretical claims are supported and complemented by analytical results.
These results use the UTKFace dataset [40] for a vision task whose goal is to classify ethnicity.
The experiments use a ResNet-18 architecture and the pruning counterparts remove the P % parameters with the smallest absolute values for various P .
All reported metrics are normalized and an average ofrepetitions.
While the theoretical analysis focuses on the notion of disparate impacts under the lens of excessive loss, the empirical results report di ff erences in accuracy of the resulting models.
The empirical results thus reflect the setting commonly adopted when measuring accuracy parity [42].
We report a glimpse of the empirical results, with the purpose of supporting the theoretical claims, and extended experiments, as well as additional descriptions of the datasets and settings, are reported in Appendix C.
Why disparity in groups’ gradients causes unfairness?
This section analyzes the e ff ect of gradients norms on the unfairness observed in the pruned models.
In more detail, it shows that unbalanced datasets result in a model with large di ff erences in gradient norms between groups (Proposition 1), it connects gradients norms for a group with the resulting model errors in such a group (Proposition 2), and connects these concepts with the excessive loss (Theorem 1) to show that unfairness in model pruning is largely controlled by the di ff erence in gradient norms among groups.
[Figure]
Gradient norms and group sizes.
The section first shows that imbalanced datasets lead a model to have imbalanced gradient norms across groups.
The following result assumes that the training converges to a local minima.
Proposition 1.
Consider two groups a and b in A with | D a | ≥ | D b | .
Then (cid:13)(cid:13)(cid:13) g ℓ a (cid:13)(cid:13)(cid:13) ≤ (cid:13)(cid:13)(cid:13) g ℓ b (cid:13)(cid:13)(cid:13) .
That is, groups with more data samples will result in smaller gradients norms than groups with fewer data samples and vice-versa.
Figure 2 illustrates Proposition 1.
The plot shows the relation between groups sizes | D a | and their associated gradient norms ∥ g ℓ a ∥ on the UTK dataset and settings described above.
Notice the strong trend between decreasing group sizes and increasing gradient norms for such groups.
These theoretical considerations can be used to explain why underrepresented groups are often subject to larger performance impacts after network pruning [20].
These groups tend to exhibit large gradient norms at convergence, relative to other groups, thus, by
Theorem 1, they are also subject to larger excessive losses due to pruning.
Gradient norms and accuracy.
Next, the section shows a strong connection between the gradient norms of a group and its associated accuracy.
The following assumes the models adopt a cross entropy loss (or mean squared error for regression tasks, as shown Appendix A).
Proposition 2.
For a given group a ∈ A , gradient norms can be upper bounded as:
[Equation]
The above relates gradient norms with an error measure of the classifier to a target label multiplied by the gradient of the predictions.
For example, in a classification task with cross entropy loss, ℓ ( f θ ( x ) , y ) = − (cid:80) z ∈Y f z θ ( x ) y z , where f z θ ( x ) represents the z -th element of the output associated with the soft-max layer of model f θ , and y is a one-hot encoding of the true label y , with y z representing its z -th element, then,
[Equation]
[Figure]
[Equation]
[Figure]
A similar observation holds for mean square error loss, as illustrated in Appendix A.
The observation above sheds light on the correlation between the prediction error of a group and its model gradients.
This relation is emphasized in Figure 3, which illustrates that the gradient norm for a given group increases as its prediction accuracy decreases.
Proposition 2 allows us to link the gradient norms with the group accuracy of the resulting model, which, together with the result above will be useful to reason about the impact of gradient norms on the disparities in the group excessive losses.
The role of gradient norms in pruning.
Having highlighted the connection between gradients norms of a group with the accuracy of the pruned model on such a group, this section provides theoretical intuitions on the role of gradient norms in the disparate group losses during pruning.
From Theorem 1, notice that the excessive loss is controlled by term
∥ g ℓ a ∥ × ∥ ¯ θ − ⋆ θ ∥ .
As already noted in Corollary 1, the term ∥ ¯ θ − ⋆ θ ∥ regulates the impact of pruning on the excessive loss, as the di ff erence between the pruned and non-pruned parameters vectors directly depends on the pruning rate.
For a fixed pruning rate, however, notice that groups with di ff erent gradient norms will have a disparate e ff ect on the resulting term.
In particular, groups with very small gradients norms (those generally associated with highly accurate predictions) will be less sensitive to the e ff ects of the pruning rate.
Conversely, groups with large gradient norms will be a ff ected by the pruning rate to a greater extent, with larger pruning rates, typically reflecting in larger excessive losses.
These observations of the factors of disparity, accuracy, and group size, can also be appreciated empirically in Figures 4a and 4b.
The plots report accuracy (a) and gradient norms (b) on the
UTKFace datasets for a variety of pruning rates.
Consider group White (containing 42% of the total samples) and Others (containing 7% of the total samples).
The unpruned model has high accuracy on the former group and small gradient norms.
The accuracy of this group is insensitive to various pruning rates and even increases at large pruning regimes.
In contrast, group Others has much lower accuracy and larger gradient norms in the unpruned model.
As the pruning rate increase, their accuracies drastically drop.
As a result, in high pruning regimes, this minority group exhibits poor accuracy and very high gradient norms.
Notice that the empirical results apply to much more complex settings than those which can be analyzed formally, thus they complement the theoretical observations.
[Figure]
Why disparity in groups’ Hessians causes unfairness?
Having examined the properties of the groups gradients and their relation to unfairness in pruning, this section turns on analyzing how the Hessian associated with the loss function for a group is linked to the unfairness observed during pruning.
In more detail, it connects the groups’ Hessian to the distance to the decision boundary for the samples in that group and their resulting model errors (Theorem 2), it illustrates a strong positive correlation between groups’ Hessian and gradient norms, and links these concepts with the excessive loss (Theorem 1) to show that unfairness in model pruning is controlled by the di ff erence in maximum eigenvalues of the Hessians among groups.
Group Hessians and accuracy.
The section first shows that groups presenting large Hessian values may su ff er larger disparate impacts due to pruning, when compared with groups that have smaller
Hessians.
It does so by connecting the maximum eigenvalues of the groups Hessians with their distance to decision boundary and the group accuracy.
The following result sheds light on these observations.
It restricts its attention to models trained under binary cross entropy losses, for clarity of explanation, although an extension to a multi-class case is directly attainable.
Theorem 2.
Let f θ be a binary classifier trained using a binary cross entropy loss.
For any group a ∈ A , the maximum eigenvalue of the group Hessian λ ( H ℓ a ) can be upper bounded by:
[Equation]
[Table]
[Equation]
[Table]
[Equation]
The proof relies on derivations of the Hessian associated with model loss function and Weyl inequality.
In other words, Theorem 2 highlights a direct connection between the maximum eigenvalue of the group Hessian and (1) the closeness to the decision boundary of the group samples, and (2) the accuracy of the group.
The distance to the decision boundary is derived from [9].
Intuitively this term is maximized when the classifier is highly uncertain about the prediction: f ⋆ θ ( x ) → 0 . 5, and minimized when it is highly certain f ⋆ θ ( x ) → 0 or 1, as showed in the following proposition.
Proposition 3. Consider a binary classifier f θ ( x ) .
For a given sample x ∈ D, the term f
⋆ θ ( x )(1 − f
⋆ θ ( x )) is maximized when f ⋆ θ ( x ) = 0 .
5 and minimized when f ⋆ θ ( x ) ∈ { 0 , 1 } .
[Figure]
Observe that a group consisting of samples that are far from the decision boundary will have smaller Hessians and, thus, be less subject to a drop in accuracy due to model pruning.
These results can be appreciated in Figure 5.
Notice the inverse relationship between maximum eigenvalues of the groups’
Hessians and their average distance to the decision boundary.
The same relation also holds for accuracy: the higher the
Hessians maximum eigenvalues, the smaller the accuracy.
This is intuitive as samples which are close to the decision boundary will be more prone to errors due to small changes in the model due to pruning, when compared with samples lying far from the decision boundary.
Correlation between group Hessians and gradient norms.
This section observes a positive correlation between maximum eigenvalues of the Hessian of a group and their gradient norms.
This relation can be appreciated in Figure 6.
While mainly empirical, this observation is important as it illustrates that both the Hessian λ ( H ℓ a ) and the gradient ∥ g ℓ a ∥ terms appearing in the upper bound of the excessive loss R ( a ) reported in Theorem 1 are in agreement.
This relation was observed in all our experiments and settings.
Such observation allows us to infer that it is the combined e ff ect of gradient norms and group Hessians that is responsible for the excessive loss of a group and, in turn, for the exacerbation of unfairness in the pruned models.
[Figure]
The role of the group Hessian in pruning.
Having highlighted the connection between Hessian for a group with the resulting accuracy of the model on such a group, this section provides theoretical intuitions on the role of the Hessians in the disparate group losses during pruning.
In Theorem 1, notice that the excessive loss is controlled by term ∥ H ℓ a ∥ × ∥ ¯ θ − ⋆ θ ∥ 2 .
As also noted in the previous section, the term ∥ ¯ θ − ⋆ θ
∥ regulates the impact of pruning on the excessive loss as the di ff erence between the pruned and non-pruned parameter vectors directly depends on the pruning rate.
Similar to the observation for gradient norms, with a fixed pruning rate, groups with di ff erent Hessians will have a disparate e ff ect on the resulting term.
In particular, groups with small
Hessians eigenvalues (those generally distant from the decision boundary and highly accurate) will be less sensitive to the e ff ects of the pruning rate.
Conversely, groups with large Hessians eigenvalues will be a ff ected by the pruning rate to a greater extent, typically resulting in larger excessive losses.
These observations can further be appreciated empirically in Figures 4a (for accuracy) and 4c (for maximum group Hessian eigenvalues) on the UTKFace datasets for a variety of pruning rates.
Mitigation solution and evaluation
The previous sections highlighted the presence of two key factors playing a role in the observed model accuracy disparities due to pruning: the di ff erence in gradient norms, and the di ff erence in
Hessians losses across groups.
This section first shows how to leverage these findings to provide a simple, yet e ff ective solution to reduce the disparate impacts of pruning.
Then, the section illustrates the benefits of this mitigating solution on a variety of tasks, datasets, and network architectures.
6.1 Mitigation solution
To achieve fairness, the aforementioned findings suggest to equalize the disparity associated with gradient norms ∥ g ℓ a ∥ and Hessians λ ( H ℓ a ) across di ff erent groups a ∈ A .
For this goal, we adopt a constrained empirical risk minimization approach:
[Equation]
where g ℓ = ∇ θ J ( θ ; D ) and H ℓ = ∇ 2 θ J
( θ ; D ) refer to the gradients and Hessian associated with loss function ℓ , respectively, and are computed using the whole dataset D .
The approach (7) is a common strategy adopted in fair learning tasks, and the paper uses the Lagrangian Dual method of Fioretto et al.
[12] which exploits Lagrangian duality to extend the loss function with trainable and weighted regularization terms that encapsulate constraints violations (see Appendix C for additional details).
A shortcoming of this approach is, however, that requires computing the gradient norms and Hessian matrices of the group losses in each and every training iteration, rendering the process computationally unviable, especially for deep, overparametrized networks.
To overcome this computational burden, we will use two observations made earlier in the paper.
First, recall the strong relation between gradient norms for a group and their associated losses.
This aspect was noted in Proposition 2.
That is, when the losses across the groups are similar, the gradient norms across such groups will also tend to be similar.
Next, Theorem 2 noted a positive correlation between model errors (and thus loss values) for a group and its associated Hessian eigenvalues.
Thus, when the losses across the groups are similar, the group Hessians will also tend to be similar.
This intuition
[Figure]
is also complemented by the strong correlation between group Hessians and gradient norms reported in Section 5.
Based on the above observations
[Figure]
we propose a simpler version of the constrained minimizer (7) defined as minimize θ
J ( θ ; D ) such that: J ( θ ; D a ) =
J ( θ ; D ) ∀ a ∈ A , (8) that substitutes the gradient norms and max eigenvalues of group Hessians equality constraints with proxy terms capturing the group J ( θ ; D a ) and population J ( θ ; D ) losses.
[Figure]
The impact of such proxy terms in the fairnessconstrained problem above can be appreciated, empirically, in Figure 7.
The plots, that use the
UTK-Face dataset, with Ethnicity as protected group, show an original unfair model (top) and a fair counterpart obtained through problem (8) (bottom).
Both top and bottom sub-figures use an unpruned model.
The top sub-figure shows the performance of an original unpruned model trained by minimizing the empirical risk function while the bottom one shows the e ff ect of solving Problem (8), i.e., it constrains the empirical risk function with the various group loss terms.
Notice how enforcing balance in the group losses also helps reducing and balancing the gradient norms and group’s average distance to the decision boundary.
As a consequence, the resulting model fairness is dramatically enhanced (bottom-left subplot).
6.2 Assessment of the mitigation solution
Datasets, models, and settings.
This section analyzes the results obtained using the proposed mitigation solution with ResNet50 and VGG19 on the UTKFace dataset [40], CIFAR-10 [23], and
SVHN [26] for various protected attributes.
The experiments compare the following four models:
• No Mitigation : it refers to the standard pruning approach which uses no fairness mitigation strategy.
• Fair Bf Pruning : it applies the fairness mitigation process (Problem (8)) exclusively to the original large network, thus before pruning.
• Fair Aft Pruning : it applies the mitigation exclusively to the pruned network, thus after pruning.
• Fair
Both : it applies the mitigation
both to the original large network and to the pruned network.
The experiments report the overall accuracy of resulting models as well as their fairness violations, defined here as the di ff erence between the maximal and minimal group accuracy.
The reported metrics are the average of 10 repetitions.
Additional details on datasets, architectures, and hyper-parameters adopted, as well as additional and extended results are reported in Appendix C.
E ff ects on accuracy.
The section first focuses on analyzing the e ff ects of accuracy drop due to applying the proposed mitigation solution for fair pruning.
Figure 8 compares the four models on the
UTK-Face dataset using a ResNet50 architecture.
The left subplots use ethnicity as protected group and class label, with |Y| = 5, while the right subplots use age as protected group and class label,
[Figure] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
with |Y| = 9.
Notice that, as expected, all compared models present some accuracy deterioration as the pruning rate increases.
However, notably, the deterioration of the models that apply the fair mitigation steps are comparable to (or even improved) those of the " No mitigation " model, which applies standard pruning.
A similar trend can be seen in Figure 9 that reports results on CIFAR (left) and SVHN (right).
Both use the ten class labels as protected attributes.
These results clearly illustrate the ability of the mitigating solution to preserve highly accurate models.
A comparison of the “full” (Equation 7) and “relaxed” (Equation 8) versions of the proposed mitigation solutions is provided in Table 1.
We note that while the "full" version leads to fairer results, the reduction in the various groups accuracy is often insubstantial.
We also note that the running time of the "full" version is largely (over an order magnitude) longer than the relaxed counterpart.
This is due to the calculation of gradient norms and the Hessian terms associated with each group.
E ff ects on fairness.
The section next illustrates the ability of the proposed solution to achieve fair pruned models.
Table 2 illustrates the results for the UTKFaces dataset with ethnicity as class labels and age as protected attributes for a CNN with two convolutional layers and three linear layers and prune amounts: 30%, 50%, 70%, and 90%.
Notice how Fair Aft pruning and Fair both achieve relatively lesser fairness violations compared to the No mitigation and the Fair bf Pruning methods in most cases.
Next, the second and fourth subplots presented in Figures 8 and 9 illustrate the fairness violations obtained by the four models analyzed on di ff erent datasets and settings.
We make the following observations: First, all the plots exhibit a consistent trend in that the mitigation solution produces models which improve the fairness of the baseline, " No mitigation " model.
Observe that, as already illustrated in Figure 7, the fair models tend to equalize the gradient norms and group Hessians components (and thus the distance to the decision boundary across groups).
Thus, the resulting pruned models also attain better fairness, when compared to their standard counterparts.
Next, notice that " Fair Aft Pruning " often achieves better fairness violations than " Fair Bf Pruning ", especially at high pruning regimes.
This is because the former has the advantage to apply the mitigation solution directly to the pruned model to ensure that the resulting model has low di ff erences in gradient norms and group Hessians.
The presentation also illustrates the application of the mitigation strategies both before and after pruning ( Fair Both ) which shows once again the significance of applying the mitigation solution over the pruned network.
[Table]
Finally, it is notable that " Fair Aft Pruning " achieves good reductions in fairness violation.
Indeed, pre-trained large (non-pruned) fair models may not be available and the ability to retrain these large models prior to pruning may be hindered by their size and complexity.
Discussion and limitations
This section discusses three key messages found in this study.
First, we notice that pruning a ff ecting model separability and distance to the decision boundary is related to concepts also explored in robust machine learning [15, 28].
Not surprisingly, some recent literature in network pruning has empirically observed that pruning may have a negative impact on adversarial robustness [16].
These observations raise questions about the connection between pruning, robustness, and fairness, which we believe is an important direction to further investigate.
Next, although the solution proposed in Problem (8) allows it to be adopted in large models, the size of modern ML models (together with the amount of hyperparameters searches) may hinder retraining such original massive models from incorporating fairness constraints.
Notably, however, the proposed mitigation solution can be used as a post-processing step to be applied during the pruning operation directly.
The previous section shows that the proposed method delivers desirable performance in terms of both accuracy and fairness.
Finally, we notice that the results analyzed in this paper pertain to losses that are twice di ff erentiable.
Lifting such an assumption will be an interesting and challenging future research avenue.
Ethical considerations.
The analyses and solutions reported in this paper should not be intended as an endorsement for using the developed techniques to aid facial recognition systems.
We hope this work creates further awareness of the unfairness caused by pruning mechanisms in ML systems in the context of models that could be deployed in energy-e ffi cient devices, such as smart cameras or access control systems, etc.
Conclusion
This work observed that pruning, while e ff ective in compressing large models with minimal loss of accuracy, can result in substantial disparate accuracy impacts.
The paper examined the factors causing such disparities both theoretically and empirically showing that: (1) disparity in gradient norms across groups and (2) disparity in Hessian matrices associated with the loss functions computed using a groups’ data are two key factors responsible for such disparities to arise.
By recognizing these factors, the paper also developed a simple yet e ff ective retraining technique that largely mitigates the disparate impacts caused by pruning.
As reduced versions of large, overparameterized models become increasingly adopted in embedded systems to facilitate autonomous decisions, we believe that this work makes an important step toward understanding and mitigating the sources of disparate impacts observed in compressed learning models and hope it will spark awareness in this important area.
Acknowledgments
This research is partly funded by NSF grants SaTC-1945541, SaTC-2133169, and CAREER-2143706.
F. Fioretto is also supported by a Google Research Scholar Award and an Amazon Research Award.
The views and conclusions of this work are those of the authors only.
A Missing Proofs
Theorem 1.
The excessive loss of a group a ∈ A is upper bounded by 3 :
[Equation]
where g ℓ a = ∇
⋆ θ J ( ⋆ θ ; D a ) is the vector of gradients associated with the loss function ℓ evaluated at ⋆ θ and computed using group data D a , H ℓ a = ∇ 2 ⋆ θ J
( ⋆ θ ; D a ) is the Hessian matrix of the loss function ℓ , at the optimal parameters
vector ⋆ θ , computed using the group data D a (henceforth simply referred to as group hessian ), and λ ( Σ ) is the maximum eigenvalue of a matrix Σ .
Proof.
Using a second order Taylor expansion around ⋆ θ , the excessive loss R ( a ) for a group a ∈ A can be stated as:
[Equation]
The above, follows from the loss ℓ ( · ) being at least twice di ff erentiable, by assumption.
By Cauchy-Schwarz inequality, it follows that
[Equation]
In addition, due to the property of Rayleigh quotient we have:
[Equation]
The upper bound for the excessive loss R ( a ) is thus obtained by combining these two inequalities.
□
Proposition 1.
Consider two groups a and b in A with | D a | ≥ | D b | .
Then (cid:13)(cid:13)(cid:13) g ℓ a (cid:13)(cid:13)(cid:13) ≤ (cid:13)(cid:13)(cid:13) g
[Equation]
b (cid:13)(cid:13)(cid:13) .
Proof.
By the assumption that the model converges to a local minima, it follows that:
[Equation]
Thus, g ℓ a = − | D b | | D a | g b .
Hence ∥ g ℓ a ∥ = | D b | | D a | ∥ g ℓ b ∥ ≤ ∥ g ℓ b ∥ , because | D a | ≥ | D b | .
□
Proposition 2.
For a given group a ∈ A , gradient norms can be upper bounded as:
[Equation]
The above proposition is presented in the context of cross entropy loss or mean squared error loss functions.
These two cases are reviewed as follows
Cross Entropy Loss.
Consider a classification task with cross entropy loss: ℓ ( f ⋆ θ ( x ) , y )
= − (cid:80)
z ∈Y f z
⋆ θ ( x ) y z , where f z ⋆ θ ( x ) represents the z -th element of the output associated with the softmax layer of model
f ⋆ θ , and y is a one-hot encoding of the true label y , with y z representing its z -th element, then,
[Equation]
where the third equality is due to that the gradient of the cross entropy loss reduces to f ⋆ θ ( x ) − y .
Mean Squared Error.
Next, consider a regression task with mean squared error loss ℓ ( f ⋆ θ ( x ) , y ) =
( f ⋆ θ ( x ) − y ) 2 .
Using the same notation as that made above, if follows:
[Equation]
where the third equality is due to that the gradient of the mean squared error loss w.r.t.
f ⋆ θ ( · ) reduces to 2( f ⋆ θ ( x ) − y ).
Theorem 2.
Let f θ be a binary classifier trained using a binary cross entropy loss.
For any group a ∈ A , the maximum eigenvalue of the group Hessian λ ( H ℓ a ) can be upper bounded by:
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation] [Equation]
Proof.
First notice that an upper bound for the Hessian loss computed on a group a ∈ A can be derived as:
[Equation]
where H ℓ x represents the Hessian loss associated with a sample x ∈ D a from group a .
The above follows Weily’s inequality which states that for any two symmetric matrices A and B , λ
( A + B ) ≤ λ ( A ) + λ ( B ).
Next, we will derive an upper bound on the Hessian loss associated to a sample x .
First, based on the chain rule a closed form expression for the Hessian loss associated to a sample x can be written as follows:
[Equation]
The next follows from that
[Equation]
Applying the Weily inequality again on the R.H.S. of Equation 12, we obtain:
[Equation]
The statement of Theorem 2 is obtained combining Equations 13 with 11.
□
Proposition 3. Consider a binary classifier f θ ( x ) .
For a given sample x ∈ D, the term f
⋆ θ ( x )(1 − f
⋆ θ ( x )) is maximized when f ⋆ θ ( x ) = 0 .
5 and minimized when f ⋆ θ ( x ) ∈ { 0 , 1 } .
Proof.
First, notice that f ⋆ θ ( x ) ∈ [0 , 1], as it represents the soft prediction (that returned by the last layer of the network), thus f ⋆ θ ( x ) ≥ f 2 ⋆ θ ( x ).
It follows that:
[Equation]
In the above, it is easy to observe that the equality holds when either f ⋆ θ ( x ) = 0 or f ⋆ θ ( x ) = 1.
Next, by the Jensen inequality, it follows that:
[Equation]
The above holds when f ⋆ θ ( x )
= 1 − f
⋆ θ ( x ), in other words, when f ⋆ θ ( x ) = 0 .
Notice that, in the case of binary classifier, this refers to the case when the sample x lies on the decision boundary.
□
B Dataset and Experimental Settings B.1 Datasets
The paper uses the following datasets to validate the findings discussed in the main paper:
• UTK-Face [40].
A large-scale face dataset with a long age span (range from 0 toyears old).
The dataset consists of over 20,000 face images with annotations of age, gender, and ethnicity.
The images cover large variations in pose, facial expression, illumination, occlusion, resolution
etc.
The experiments adopt the following attributes for classification (e.g., Y ) and as protected group ( A ): ethnicity , age bins , gender .
• CIFAR-10 [23].
This dataset consists of 60,000 32 × 32 RGB images in 10 classes, with 6,000 images per class.
The 10 di ff erent classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks.
• SVHN [26] Street View House Numbers (SVHN) is a digit classification dataset that contains 600,000 32 × 32 RGB images of printed digits (from 0 to 9) cropped from pictures of house number plates.
B.2 Architectures, Hyper-parameters, and Settings
The study adopts the following architectures to validate the results of the main paper:
• ResNet18 : An 18-layer architecture, with 8 residual blocks.
Each residual block consists of two convolution layers.
The model has ∼ 11 million trainable parameters.
• ResNet
This model contains 48 convolution layers, 1 MaxPool layer and a AvgPool layer.
ResNet50 has ∼ 25 million trainable parameters.
• VGG-
This model consists of 19 layers (16 convolution layers,
3 fully connected layers, 5 MaxPool layers and 1 SoftMax layer).
The model has ∼ 143 million parameters.
Hyperparameters for each of the above models was performed over a grid search (for di ff erent learning rates =
over a cluster of NVIDIA RTX
A6000 with the above networks using the UTKFace dataset.
The models with the highest accuracy
[Figure] [Figure]
were chosen and employed for the assessment of the mitigation solution in Sec. 6.2.
The running time required for all sets of experiments which include mitigation solutions was about ~3 days .
The training uses the SGD optimizer with a momentum of 0 .
9 and weight_decay of 1 e − 4 .
Finally, the
Lagrangian step size adopted in the Lagrangian dual learning framework [12] is set to 0 .
All the models developed were implemented using Pytorch 3.0.
The training was performed using
NVidia Tesla P100-PCIE-16GB GPUs and 2GHz Intel Cores.
The model was run for 100 epochs for the CIFAR-10 and SVHN and 40 epochs for UTK-Face dataset.
Each reported experiment is an average of 10 repetitions.
In all experiments, the protected group set coincides with the target label set: i.e., A = Y .
C Additional Experimental Results C.1 Impact of pruning on fairness
This section shows and a ffi rms the impact of pruning towards accuracy disparity through VGGnetwork.
The same training procedures as employed with ResNet18 in Fig 1 were followed.
Each demographic group’s accuracy is shown before and after pruning on the UTK-Face dataset in two cases: when ethnicity is a group attribute as in Figure 10, and when gender is a group attribute as in Figure 11.
A consistent message is that under a higher pruning rate, the accuracies are more imbalanced across di ff erent groups, emphasizing the negative impact of pruning on fairness.
C.2 Correlation of gradient / hessian norm and average distance to the decision boundary
This subsection elaborates the impact of gradient norms and group Hessians towards the fairness issues shown in Figures 10 and 11.
In Section 4, it has been shown that the group with a larger gradient norm before pruning will be penalized more than the groups with a smaller gradient norm.
Figures 13 and 12 show the gradient norm of each demographic group for UTK-Face dataset under two choices of protected attributes for VGG 19 networks.
The results indicate that a group penalized less will have a smaller gradient norm with respect to those of the other groups.
In addition, Section 5 supports that Hessian norm is another factor.
More precisely, the groups with a larger Hessian norm will be penalized more (drop much more in accuracy) than groups with a smaller
Hessian norm.
Evidence is provided for the claim on VGG19 in Figures 12 and 13.
These results on
VGG19 again confirm the theoretical findings.
Finally, in Section 5, a positive correlation between gradient norms and Hessian groups is shown in
Theorem 2, and a negative correlation between Hessian groups and distance to the decision boundary
[Figure] [Figure]
is shown in Proposition 3.
These important results again are supported by the results in Figuresand 13.
C.3 Impact of group sizes to gradient norm
This section presents additional empirical results to support Theorem 1, stating that the group with more samples will tend to have a smaller gradient norm.
In these experiments, run on a ResNet network, one group is chosen and upsampled 1 × , 5 × , 10 × , and 20 × times.
Note that by increasingly upsampling it, the group becomes the majority group in that dataset.
A group with more samples is expected to end up with a smaller gradient norm when the training convergences.
UTK-Face with gender Since the UTK-Face is balanced with regard to gender (Female / Male), the number of samples in Female, and Male groups is upsampled in turn.
Figure 14 reports the respective gradient norms at the last training iteration when upsampling Females (left) and Males (right.)
Note how the Male group, initially with no upsampling, has a larger gradient norm than the Female group (right sub-plot).
However, if the number of Male samples is increased enough, its gradient norm becomes smaller than that of the Female group.
UTK-Face with age bins Similar experiments are performed with UTK-Face on nine age bin groups.
Three age bins are randomly chosen, 0, 2, 4, and the number of samples for each group is upsampled in turn.
The gradient norms of nine age bin groups are shown in Figure 15, where the upsampled groups are highlighted with dotted thick lines.
The results echo that if a group’s number of samples is increased enough, its gradient norm at convergence will be smaller than the other 8 age bin groups.
[Figure] [Figure]