Paper: Fair Feature Importance Scores for Interpreting Tree-Based Methods and Surrogates (ICLR-2024)

Reviewer comment (Evaluation):
The paper lacks a discussion on how practitioners should use the proposed metric.

Author response:
Thanks for the question and for giving us an opportunity to clarify this. FairFIS can be used in the same way that MDI (FIS) is used for decision trees and tree-based ensembles.  Also, FairFIS can be interpreted similarly to MDI (FIS) for decision trees, except that FairFIS can be both positive and negative.  A positive FairFIS score indicates that there are other biased features in the model and that this positively scored feature can help in reducing the impact from other biased features. On the other hand, features with negative FairFIS scores directly contribute to the bias of the model.  If a feature is not used in the model (e.g. not split upon in the tree), the FairFIS will be zero just as with FIS.  Thus, the sign of FairFIS indicates whether a feature improves or worsens model bias and the magnitude of FairFIS indicates the relative importance of the feature's contribution to the model fairness or bias.  In our paper, we use our FairFIS metric to interpret features in several case studies in Section 3.2.  We look forward to adding this discussion to our manuscript.

Reviewer comment (Experiments):
In all the datasets that the authors employed, the number of features is small. Thus, it would be easy to fit p models where p is the number of features and each model is missing one feature, and then analyze how fairness metrics vary across models. This idea is similar to leave-one-covariate-out (LOCO) inference, see https://www.stat.cmu.edu/~ryantibs/talks/loco-2018.pdf.

Author response:
Thank you for this comment and great suggestion. First, the datasets we study in our paper are standard benchmarks in fairness; these include many one-hot-encoded features which often measure in the hundreds and hence the number of features is not as small as it appears.  Next, LOCO inference or the related feature occlusion importance score are indeed very interesting approaches. To the best of our knowledge, however, these have not been proposed or studied in the context of fairness.  Feature occlusion importance requires refitting many models and is hence computationally expensive; our approach, on the other hand, is simple and fast.  We look forward to investigating LOCO / feature occlusion in the context of fairness in the future and to comparing it with our FairFIS metric. We will also add a discussion about this to our related works.

Reviewer comment (Experiments):
One could fit two different models to the data and compare the disparities of the models with the tree-based method proposed by https://arxiv.org/pdf/1707.00046.pdf. The authors could analyze FairFIS for each model and see if they reach similar conclusions about which features contribute towards disparities as they would with the method from that paper.

Author response:
Thank you so much for this suggestion and reference.  This is an interesting method and as you suggest, we can use to compare two models with and without biased features and to further validate our metric and surrogates. We would like to highlight, however, the many simulation studies (many further studies are in the appendix) as well as case studies that we include in the paper to validate our approach.  Nonetheless, we look forward to investigating this model comparison framework in future work and will add a discussion about this method to our paper.

Reviewer comment (Presentation):
In Figure 2, it is unclear what the error bars represent. Are they confidence intervals? What are the groups? The legend should be improved because it is mentioned that they are G1, G2 etc. only on page 7.

Author response:
The error bars are the standard error.  We will also improve the clarity of the figure legend.

Reviewer comment (Theory):
It seems that the authors are considering $\Sigma=I$ in the main paper, and $\Sigma\neq I$ in the Appendix. Clarification is needed on whether this is correct.

Author response:
In the simulations presented in the main paper, the features are uncorrelated.  We include simulations with correlated features in the Appendix.

Reviewer comment (Presentation):
Proposition 1 could be shortened. It is taking up a lot of space and it is pretty clear how these metrics need to be computed.

Author response:
Thanks for the suggestion regarding Proposition 1

Reviewer comment (Writing):
There are a couple of typos in the paper.

Author response:
Thanks for mentioning the typos and we look forward to fixing these.

Reviewer comment (Theory):
The paper's approach to addressing fairness through the proposed algorithm is not adequately substantiated. The algorithm requires users to define a set of "protected" features, then computes bias based on this selection. This methodology rests on the assumption that the chosen protected features are inherently unbiased—a claim not demonstrated in the paper. To bolster the argument that the FairFIS algorithm genuinely addresses fairness, the authors should either provide a rigorous formal definition of what constitutes a "protected feature" or temper their claims regarding the algorithm's ability to ensure fairness.

Author response:
Thank you for the question. We would like to clarify that in terms of fairness, protected attributes are those traits that are prohibited by law from being the subject of discrimination. These include gender, race, age, and etc.  These protected attributes are prohibited from being used for decision-making and hence cannot be used in the training of any machine learning model.  We look forward to clarifying this in our paper.

Reviewer comment (Novelty):
I have some concerns about the motivations of this paper. The authors claim that they want to interpret how a feature influences the prediction bias of one model. However, I feel that this problem could be solved by performing counterfactuals over the features that we want to explain and evaluating how the prediction bias gets changed. The authors need to provide more justifications for why this strategy is not satisfactory.

Author response:
Counterfactual explanations are a really interesting approach to understand a feature's contribution to fairness; thanks for this suggestion.  To the best of our knowledge, these have not been explored before in the context of fairness.  In this paper, our goal was to develop a global feature importance score.  Notice that counterfactual explanations are local in nature and hence cannot provide a global metric for fair feature importance. As a result, counterfactual explanations could be more appropriate for individual fairness (local) rather than group fairness (global) which we study in this paper. We look forward to adding a discussion of this to our related works section.

Reviewer comment (Experiments):
Related to the above point, the baseline comparison in the experiments is very simple and lacks many critical baseline methods, in particular, the state-of-the-art feature importance score over general models. As mentioned above, we adapt such baseline methods by measuring how perturbing the target feature impacts the prediction bias. It would be essential to consider the adaptations of such a baseline and perform an empirical comparison.

Author response:
From what you are describing, it sounds like you are referring to feature permutation scores. If so, this is a great idea and has been studied for how feature permutation impacts accuracy. To the best of our knowledge, however, these have not yet been applied in the context of fairness. While there are many advantages, feature permutations for accuracy are known to have a few weaknesses including expensive computation, poor performance for correlated features, and they often create features that are outside of the domain of the original data.  Feature permutations for fairness would likely inherit these properties and hence we believe a thorough examination of these and their properties would be needed before we should use them as a baseline for comparison.  Our FairFIS approach, on the other hand, is simple, computationally fast, and inherits some of the nice properties of tree-based feature importance scores. The idea of feature permutation scores will be useful to include in our discussion and we look forward to exploring this further in future work.

Reviewer comment (Experiments):
Interpreting the influence of a feature over the model prediction bias through the decision tree surrogate is also problematic. I am not sure whether the feature that causes the most significant prediction bias for the surrogate model can really cause the same amount of bias in the original model. The authors need to verify this somehow, say through performing counterfactuals over the target features on the original models.

Author response:
Thank you for this comment. We would like to draw your attention to a figure in the appendix where we validate our surrogate model by comparing the accuracy-based feature importance scores from a tree-based surrogate of a deep learning model with scores from epsilon-Layerwise Relevance Propagation (LRP). In the future, we plan to investigate this further and include comparisons with other feature importance scores.

Reviewer comment (Novelty):
**Framing:** The paper claims to be the first to consider fairness and explainability of ML models. For instance, the paper notes that "we have no current way of understanding how a feature affects the fairness of the model’s predictions". However, there is already non-negligibly amount of work that focuses on fairness and explainability both. Consider for example [this blogpost](Explaining Measures of Fairness) showing how to use SHAP to understand model unfairness. On a more academic side, I would suggest that the paper factors papers 1-4 (and related references therein) in the related work section so that the readers can better frame its contributions.

Author response:
Thank you for bringing these interesting papers to our attention.  While this literature tells us important information about the features, they do not provide a global score or metric. Furthermore, the methods inspired by SHAP tend to be very computationally expensive, especially with large numbers of features. On the other hand, FairFIS is fast, easy to compute, and provides a global metric for fair feature interpretation. We look forward to adding further discussion of these papers to our related works and contextualizing them with our proposed method.

Reviewer comment (Theory):
**Motivation:** The paper needs to motivate the design choices in a better way. I am not sure that "trees have a popular and easy-to-compute intrinsic feature importance score known as mean decrease in impurity" is the most important reason to focus on trees. There exist plenty of methods like SHAP, LIME and Integrated Gradients for explaining all kinds of other ML models. Similarly, why consider the mean decrease in impurity (MDI) score? There are other methods like TreeSHAP which seem to offer better theoretical properties. Also, the proposed FairFIS metric has a rather interesting choice in comparing a parent and the child node. The paper explains why the the computation of the style of Eq. 1 was not considered, but does not mention what are the pros and cons of the computation of FairFIS? How can we ascertain that this choice corresponds to how humans would expect model explanations to behave?

Author response:
Our motivation was to develop a simple, fast, easy-to-use, and easy-to-interpret metric for trees and tree-based ensembles.  Since the MDI score for trees is the most widely used and the fastest, we opted to focus on this.  While SHAP, TreeSHAP, LIME and many other model-specific methods can be useful, they are typically computationally expensive, which is why we chose to employ the MDI approach for trees. Computationally, FairFIS has many benefits. Just as in FIS, we can automatically calculate the change in bias for each split of the tree and sum over all of the splits that contain feature $j$. This is fast computationally and a major motivation for our work.

Reviewer comment (Writing):
**Writing:** I think the writing should also be improved before the paper is ready for publication. Currently, the paper tends to simply provide information without first giving motivation and reasoning. Consider for instance the experiment in Section 3.1, where the paper directly dives into the details of the experiment without explaining why it was set up in this way, what metrics should the users watch out for, and what kind of values of should they expect to see.

Author response:
Thank you very much for this comment and we look forward to improving this aspect of the paper. Please see our point three for further discussion on how we set up this model.