Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The work studies effect of pruning neural networks on their losses for specific groups and the differences between them, termed as unfairness. It shows that gradient norm and Hessian of the loss function are important quantities controlling the upper bound on group-specific losses. As a way to mitigate unfairness from pruning, the effect of adding constraints to the learning problem that penalise the two quantities is explored. Some of the claims are validated through experiments on a face images used to train an ethnicity detection model.
strengths_and_weaknesses: Strengths
- Provides clear experimental evidence that backs the claim of relation between excessive losses from pruning and properties of the network such as gradient norm and the max eigenvalue of the Hessian.
- Concepts and results are presented clearly.
- Studies a practically relevant problem.

Weaknesses
- The main claim on unfairness from pruning is not well-motivated by the theoretical results. The discussion centres on unfairness that is difference in excessive losses across groups while the result analyzes excessive losses for each group separately.
- Missed related work which makes a similar observation that pruning may cause unfairness.
- Experimental results are not presented as clearly as possible. Error bars can be reported and correlation coefficient can be reported in plots where the claim is that the plotted quantities are correlated.

The evidence presented in this work that pruning causes unfairness is an important one because of the prevalence of pruning in practice. Also, the observations on gradient norm and max eigenvalue of the Hessian are interesting. However, the presentation of the results seems to overstate the significance of the theoretical results. I think that these can be addressed through minor writing edits.

---

After the response

The response adequately clears my concerns on the proposed relaxation for the solution and related work. The response to the over-interpretation of the theoretical result in the paper is adequate. The promised change to the text after Corollary 1 is good. I would strongly encourage the authors to propagate this change to the rest of the paper e.g. the text in the heading of Sections 4 and 5 can be changed to may cause. 

The work presents concrete evidence for the relation between two training properties during pruning and unfairness which is an important problem due to prevalence of pruning. Thus I am increasing the score to 7, Accept. The motivation from theory needs improvement which needs to be adequately acknowledged in a revision. The presentation should be improved also including error bars in figures.
questions: Major points to address in the response

1. There is an unacknowledged jump in reasoning from Corollary 1 to the main claim of the work that pruning increases unfairness. Corollary 1 shows that the upper bound on excessive loss for *a given* sensitive group may increase due to pruning. But it does not guarantee that *relative* excessive loss may increase. Also, the results are on the upper bound of the excessive loss and not the actual value of the excessive loss. The difference in actual loss across groups need not be smaller than the difference between upper bound. The theoretical results may motivate investigating gradient norms and the Hessian in experiments but the motivation is not as clear as presented. Thus the claims for studying disparate excessive loss in the rest of the results are overstated. The claims should be stated more carefully.

2. Please discuss the work by Hooker et al. 2020 which seems to have the same conclusion. Please discuss relationship between the findings.
Hooker et al. 2020, Characterising Bias in Compressed Models. https://arxiv.org/abs/2010.03058

Another related work is Ahia et al. 2021 which posits that pruning affects infrequent sentences more. This relates to Proposition 1’s claim that the proportion of data points from the group is a potential source of disparate impact of pruning.
Ahia et al. 2021, The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation. https://aclanthology.org/2021.findings-emnlp.282/

3. Validating the relaxed version of the mitigation method eq. (8) is a bit unsatisfactory. Partly because it is not directly related to the analysis. Consider solving the optimization problem in eq. (7) instead of the relaxed version on a smaller dataset and model. This will help in validating the constraints motivated by the theoretical analysis instead of simplifying the problem. Perhaps only the gradient norm penalty, which is relatively quicker to estimate than max eigenvalue of the Hessian, can be tested in experiments.

Secondly, the relaxed version is an existing fairness method which is not discussed. It is also considered in the domain generalization problem where one approach is to equalize losses across domains that are groups in this case. In this sense, the contribution can be rephrased as taking an existing method and showing its relevance for fairness of pruning.
See Donini et al. 2018 https://papers.nips.cc/paper/2018/hash/83cdcec08fbf90370fcf53bdd56604ff-Abstract.html, and Williamson and Menon 2019 https://proceedings.mlr.press/v97/williamson19a.html for the fairness method, and Section 2.4 of Krueger et al. 2021 https://proceedings.mlr.press/v139/krueger21a.html for the domain generalization method.

4. Finally, please address the ethical concern of using the UTKFace dataset as the only exemplar in the work.


Minor points or suggestions (no response is being expected)

Handling one failure mode (disparity due to pruning, here) in isolation risks unintended consequences on other failure modes. Please consider adding a note that the question of how the proposed solution impacts fairness on other attributes, robustness, and interpretability of the pruned network.

Given the discrepancy between the theoretical results and claims, the empirical results become essential to the significance of the work. Consider repeating the analyses in Figures 2 to 6 on more models and datasets as considered in Figure 9.

In the line 88, fairness wrt pruning is taken to be zero excessive loss \xi(D) =0. While terming this as fair pruning is ok, it can be commented that the pruned network can still be unfair in terms of differences in losses between groups.

Clarify the accuracy metric used in Figure 3.

Mention the discrepancy between loss metrics (like squared loss) used in the results and accuracy metric (0-1 loss) used to evaluate them in the experiments.

Proposition 3 can be a remark as it is a known fact.

Add error bars specifically in Figure 8 and 9 where different methods are compared. Ideally error bars should be reported in other figures also to convey robustness of the findings.

I think a more natural way to visualise Figure 1 would be a multiple line plot of accuracy vs pruning percentage with separate lines for each group.

Report the R^2 value for the correlation between the two variables shown in Figure 2, 3 and 6 which is easier to interpret.

It makes more sense to normalize the gradient norm plot in Figure 8 since two different models are compared.

Include a reference to Weyl’s inequality e.g. ones in https://terrytao.wordpress.com/tag/weyl-inequalities/.
Weily’s -> Weyl’s in Appendix.

Consider using the first person in sentences as it is less ambiguous, The paper uses -> we use.
limitations: Some of the limitations are presented. Please discuss limitations including relevance of theoretical results to differences in excessive loss, reliance on assumptions like being able to find the local minima. Also, mention the effect of other pruning methods, and uncertainty around effect on other properties of the pruned model than fairness. Prominently write the assumptions in one place and reference them in the results like in Proposition 1 which does not state the assumptions.

Please consider clarifying why ethnicity detection task from face images was used as an example to illustrate effects of pruning. Since it is a controversial application of machine learning with damaging consequences (with or without fairness concerns), it becomes imperative to motivate its inclusion in the paper. In my personal opinion (disagreements are welcome), the inclusion of facial recognition application should not appear as an endorsement of the technology.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper has studied two factors causing disparity of network pruning, that is differences of gradient norm and Hessian matrix cross groups. It further introduced a simple training strategy to mitigate this disparity. With extensive experiments on UTK-Face, CIFAR-10, and SVH datasets,  this strategy has shown to mitigate the disparity well and keep comparable pruning performance to those without migration.
strengths_and_weaknesses: Strengths:
1. This paper is well written and the idea is clear and easy to follow. 
2. It is novel to explain the fairness with gradient norms and Hessian matrices across groups theoretically.
3. The proposed method is simple yet efficient to mitigate the disparity across groups. 
Please look at the Questions for the weaknesses.
questions: 1. It is unclear to me how to do the pruning with mitigation. Does the mitigation after pruning need retraining the model? 
2. In Figure 7, although the fairness among groups is improved,  the pruning accuracy of all groups except “Others” equally decreases with that. The unfair model (top) seems to be an unpruned model. Is the fair model (bottom) the one before pruning or after pruning? If it is after pruning, please clarify the pruning rate. 
3. If I understand correctly, all the experiments have the class labels as the protected groups. How does this mitigation strategy work when they are different? For example, the model on the UTK-Face dataset uses ethnicity as class labels and uses age as protected groups. 
4. In Proposition 2 and Theorem 2, $||f_{\theta}(x) -y||$ is for the error instead of the accuracy, $(f_\theta(x)(1-f_\theta(x)))$ and the distance to the decision boundary seems negative proportional. The larger  $(f_\theta(x)(1-f_\theta(x)))$, the smaller the distance to the decision boundary is. 
5. In line 94, what is the soft output? Is it the output after softmax or before it? 
limitations: The proposed mitigation solution uses the loss disparity cross groups as the fairness proxy. Although much simpler, it is not so novel that it can be proposed without knowing the gradient norms and Hessian matrices as the cause of the disparity.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the relationship between pruning and group disparities in model accuracy. Using the UTKFace dataset and ResNet and VGG models, the authors show that increased levels of pruning show an increase in disparities in accuracy between groups. That is, groups that had lower accuracy than average had their accuracies get lower, while groups with higher than average accuracy have their accuracies improve. The authors relate these changes theoretically to the gradient norm and Hessian and propose mitigation techniques with these insights. 
strengths_and_weaknesses: The paper’s main strengths are its originality, clarity, and quality. The effect of pruning on group disparity metrics has not been studied in the past, and this paper does a very thorough theoretical and empirical treatment of the effect. In particular, the insights about the relationship of the gradient norm and Hessian eigenvalues to fairness violation were well done. The paper is systematic in its approach and exposition, with an easy-to-follow motivation.

The main weakness of the paper is that the mitigation results presented are difficult to interpret. While the results showing the relationship between pruning and fairness violation are quite clear, the results shown in figure 9 are difficult to read and understand. For example, consider the fairness violation plot in the bottom left of Figure 9. Here there is no smooth behavior relative to the pruning fraction, despite the fact that the theoretical results seem to indicate that there should be. It looks as though there may be some statistical uncertainty in these experiment results. 

Even without these mitigation results, however, the theoretical treatment and empirical results on pruning are interesting and could be a paper on their own. 

questions: In equation 3, why is the maximum chosen for fairness violation? Was the average, standard deviation, etc. also considered? Does max have desirable theoretical properties that the others don’t? Using the maximum implies a worst case fairness violation, but it could be interesting to understand the behavior of, say, the average fairness violation as well. This may be worth discussing in the text. 

Rather than having the two bars per group in figure 2, would it be possible to made a 2D plot of group size vs gradient norm instead? For this figure, it’s not crucial to know which groups are which, but instead to see the correlation between group size and gradient norm more directly. It can be inferred from the bars, but a scatter plot might convey the results more clearly. 

Given that a relationship between gradient norms and loss is shown, what is the relationship between fairness violation and accuracy? Is there an inherent tradeoff? In interpreting the results of figure 9, I found myself wondering whether having a lower accuracy automatically meant having a higher fairness violation. It was hard to tell from the plots themselves, but I think this is worth some discussion.

limitations: The primary limitation of the work is that it only considers pruning in the context of a single dataset and task (with multiple models). So, some of the empirical results may not generalize to, say, a BERT model performing sentiment classification. That being said, the theoretical results are not task dependent.
review_title: null

Meta-review: metareview: The paper hits two important angles in current ML: first, offering theoretical predictions about the behavior of networks in the context of pruning, and second, providing guidance on how differences in data distributions will be affected by pruning. Examples include face image analysis with labels for race, these should be treated carefully so as not to pass on errors.

Ethics reviewers would prefer that face recognition not be the central example, but serve more as a supporting example. If that modification is possible within the NeurIPS timeframe if the paper is accepted, I would like to see that happen, but I would not necessarily hold back on publication for that sole reason. At the very minimum there should be a discussion of potential harms of disparate quality and availability in facial recognition technology, and how the current work relates to those. For example, do we *want* facial recognition to work well? Can we spell out the possible harms of disparate quality, such as increased risk of false identification?

The reviewers have engaged with author comments and have specific recommendations for improvements in clarity and evaluation strength.
confidence: Certain
Decision: Accept