Paper: Understanding Unfairness via Training Concept Influence (ICLR-2024)

Reviewer comment (Theory):
There are no theoretical guarantees about the methods used to generate counterfactual samples.

Author response:
We thank the reviewer for considering our work to be "well-written," "intriguing," "practical," "intuitive," "easy to understand", and  "reasonable design."

We address the following concern.

### **1. Theoretical Understanding**
We have dedicated an entire section on the theoretical insights in Appendix D. We did not include it in the main paper due to the page limit.

We briefly summarize here why intervening training data attributes using CIF framework can improve fairness. For simplicity, we focus on accuracy disparity as the fairness measure.

We base the analysis on the data generation model adopted in [1] to capture the impact of data patterns generated with different frequencies and the impact of label errors. This setup is a good fit for understanding how counterfactual data interventions can change the data frequency of different groups (majority group with higher frequency vs. minority group with lower frequency) and provides insights for CIF. 

Intervening labels $Y$ is relatively straightforward. If we are able to intervene on a training label of a disadvantaged group from a wrong label to a correct one, we can effectively improve the performance of the model for this group. Therefore the label intervention can reduce the accuracy disparities. Our analysis also hints that 
the influence function is more likely to identify samples from the disadvantaged group with a lower presence in the data and mislabeled samples. This is because, for a minority group, a single label change would incur a relatively larger change in the influence value.

Intervening sensitive attributes $A$ improves fairness by ``balancing" the data distribution. In the experiments, we show that the influence function often identifies the data from the majority group and recommends them to be intervened to the minority group. In the analysis, we also show that this intervention incurs positive changes in the accuracy disparities between the two groups and therefore improves fairness. 

Please let us know if there is still anything unclear.


[1] Feldman, Vitaly. "Does learning require memorization? a short tale about a long tail." Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 2020.

Reviewer comment (Theory):
Much of the discussion seems to assume a relatively limited class of machine learning problems, such as binary classification, which may restrict the generality of the work.

Author response:
We thank the reviewer for the concern. Although we test binary classification models in our experiments, our method is flexible enough to be extended to multi-class fairness - the only difference is the definition of fairness, which is easy to plug in.

We admit that extending to the regression or generative models would require significant change, but almost all works in influence function (we mean general influence function, not even the influence function for fairness) only target the classification models, and therefore we think our scope is reasonable.

Reviewer comment (Writing):
The introduction of the override operator is not particularly obvious. For example, the statement "override() operator counterfactually sets the value of the concept variable to a different c0" sounds similar to the do() operation, although the paper attempts to distinguish between the two.

Author response:
We thank the reviewer for the clarification. As the reviewer already noticed our explanation in Section 2.3, we will try our best to further clarify.

The "override(.)" operator is conceptually similar to the do(.) operator in the causal inference; but it is broader. In general, it represents the change of data attributes (feature $X$, label $Y$, or sensitive attribute $A$) of training samples. Since $X$, $Y$, and $Y$ are interdependent, we need to find out, for example, how the change of $A$ would impact $X$. If we already know the underlying causal relationship between variables, then it is the same as the do(.) operator. However, it would only work on synthetic data.

In practice, given non-synthetic data, the causal relationship is unclear (and discovering it is a notoriously hard problem in causal inference), which is our focus where the traditional causal methods are not applicable. To this end, we propose a set of empirical methods to estimate it (i.e. our W-GANs).

Reviewer comment (Experiments):
The methods do not appear to have a comparison benchmark, even a naive one, which could help readers understand the contribution over any existing approaches in this context.

Author response:
We thank the reviewer for raising the concern. 

To the best of our knowledge, the only prior works that study influence function on fairness are [1, 2]. However, both are not comparable to our work because they only consider removing or reweighing whole training samples while we decompose the fairness influence into features, labels, and sensitive attributes. We do not know other works that also study the fairness influence function of modifying features, labels, and sensitive attributes *independently*. If the reviewer knows, please let us know.

In addition, the "Removal" baseline in our experiment considers the effect of removing a training sample, which can be thought of as the method commonly used in the literature [1,2].

Furthermore, we have compared our mitigation to the baseline of in-processing mitigation with fair training [3] in Figure 5.  

Last but not least, our work is mainly an explainable AI tool and the primary goal is to explain fairness to AI practitioners. The main objective is not to outperform other bias mitigation methods but rather attributing and explaining the observed fairness.

[1] Wang, Jialu, Xin Eric Wang, and Yang Liu. "Understanding instance-level impact of fairness constraints." International Conference on Machine Learning. PMLR, 2022.

[2] Li, Peizhao, and Hongfu Liu. "Achieving fairness at no utility cost via data reweighing with influence." International Conference on Machine Learning. PMLR, 2022.

[3] Agarwal, Alekh, et al. "A reductions approach to fair classification." International conference on machine learning. PMLR, 2018.

Reviewer comment (Writing):
The paper uses the term "fairness" throughout without an explicit definition early on. At times, the term appears more qualitative (e.g., "Influence Functions for Fairness"; "They can all contribute to unfairness"), while at other times it seems quantitative (e.g., "fairness can improve"; "fairness becomes worse"). Although the meaning may be clear to those embedded in the fair ML literature, clarifying the definition early would help fix ideas for readers.

Author response:
We thank the reviewer for clarification. We have included all precise mathematical definitions of fairness in Appendix C, Table 1's middle column. We understand it is easy to omit them. We included them in the Appendix because they are the standard group fairness definitions and we thought no special mention is needed. We hope it also answers Question (2).

Therefore, when we say "fairness improves," it simply means the measured unfairness, i.e. the gap between different groups, becomes smaller. They are indeed the standard phrases in the fairness literature, as the reviewer pointed out, and we do not invent anything new. We will clarify it.

We hope it is also clear that our fairness definition is mathematical rather than social which is not up to subjective interpretations.

Reviewer comment (Writing):
Clarification is needed on the statement, "we show that the influence function often identifies the data from the majority group and recommends them to be changed to the minority group." Presumably, this refers to re-weighting of the training data, but as currently written it reads as if it might mean direct manipulation of observed data values.

Author response:
"We show that the influence function often identifies the data from the majority group and recommends them to be changed to the minority group." It does not mean reweighting training samples but rather resampling training samples. It means our method mostly recommends counterfactually changing training samples from a disadvantaged group (e.g. female) to an advantaged group (e.g. male), i.e. had those female applicants were to become male applicants, the fairness of the application decisions would improve. It is similar to resampling to increase female representation in the training data to reduce gender bias.