SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Fair Effectiveness-Cost Trade-Off The classifier is fair if the protected subgroups have the same effectiveness-cost distribution, or equivalently for each cost budget  $c$ , their aggregate effectiveness is equal:
$$
\max  _ {c} | \operatorname {e c d} (c; A, G _ {0}) - \operatorname {e c d} (c; A, G _ {1}) | = 0.
$$
The left-hand side represents the two-sample Kolmogorov-Smirnov statistic for the empirical cumulative distributions (ecd) of the protected subgroups. We say that the classifier is fair with confidence  $\alpha$  if this statistic is less than  $\sqrt{-\ln(\alpha / 2)\frac{|G_{p,0}| + |G_{p,1}|}{2|G_{p,0}||G_{p,1}|}}$ .
The last definition takes a micro viewpoint and extends the notion of burden [32] from literature to the case where not all individuals may achieve recourse. The mean recourse cost of a group  $G$ ,
$$
\bar {\mathbf {r c}} (A, G) = \frac {1}{| G |} \sum_ {x \in G} \mathbf {r c} (A, x)
$$
considers individuals that cannot achieve recourse through  $A$  and have a recourse cost of  $c_{\infty}$ . To exclude them, we denote as  $G^{*}$  the set of individuals of  $G$  that can achieve recourse through an action in  $A$ , i.e.,  $G^{*} = \{x \in G | \exists a \in A, h(a(x)) = 1\}$ . Then the conditional mean recourse cost is the mean recourse cost among those that can achieve recourse:
$$
\bar {\mathsf {r c}} ^ {*} (A, G) = \frac {1}{| G ^ {*} |} \sum_ {x \in G ^ {*}} \mathsf {r c} (A, x).
$$
If  $G = G^{*}$ , the definitions coincide with burden.
Equal (Conditional) Mean Recourse The classifier is fair if the (conditional) mean recourse cost for the protected subgroups is the same:
$$
\bar {\mathbf {r c}} ^ {*} (A, G _ {0}; A) = \bar {\mathbf {r c}} ^ {*} (A, G _ {1}).
$$
Note that when the group  $G$  is the entire dataset of affected individuals, and all individuals can achieve recourse through  $A$ , this fairness notion coincides with fairness of burden [35, 32].
3 Fairness-aware Counterfactuals for Subgroups
This section presents FACTS (Fairness-aware Counterfactuals for Subgroups), a framework that implements both the micro and the macro viewpoint, and all respective fairness definitions provided in Section 2.3 to support auditing of the "difficulty to achieve recourse" in subgroups. The output of FACTS comprises population groups that are assigned (a) an unfairness score that captures the disparity between protected subgroups according to different fairness definitions and allows us to rank groups, and (b) a user-intuitive, easily explainable counterfactual summary, which we term Comparative Subgroup Counterfactuals (CSC).
Figure 2 presents an example result of FACTS derived from the adult dataset [1]. The "if clause" represents the subgroup  $G_{p}$ , which contains all the affected individuals that satisfy the predicate  $p =$  (hours-per-week = FullTime) ∧ (marital-status = Married-civ-spouse) ∧ (occupation = Adm-clerical). The information below the predicate refers to the protected subgroups  $G_{p,0}$  and  $G_{p,1}$  which are the female and male individuals of  $G_{p}$  respectively. With blue color, we highlight the percentage  $\mathrm{cov}(G_{p,i}) = |G_{p,i}| / |D_i|$  which serves as an indicator of the size of the protected subgroup. The most important part of the representation is the actions applied that appear below each protected subgroup and are evaluated in terms of fairness metrics. In this example, the metric is the Equal Cost of Effectiveness with effectiveness threshold  $\phi = 0.7$ . For  $G_{p,0}$  there is no action surpassing the threshold  $\phi = 0.7$ , therefore we display a message accordingly. On the contrary, the action  $a = \{ \text{hours-per-week} \to \text{Overtime}, \text{occupation} \to \text{Exec-managerial} \}$  has effectiveness  $0.72 > \phi$  for  $G_{p,1}$ , thus allowing a respective  $72\%$  of the male individuals of  $G_{p}$  to achieve recourse. The unfairness score is "inf", since no recourse is achieved for the female subgroup.
If hours-per-week=FullTime, marital-status=Married-civ-spouse, occupation=Adm-clerical: Protected Subgroup  $=$  'Male',  $1.87\%$  covered Make hours-per-week=Overtime, occupation=Exec-managerial with effectiveness  $72.00\%$  Protected Subgroup  $=$  Female,  $1.80\%$  covered No recourses for this subgroup. Bias against Female' due to Equal Cost of Effectiveness (threshold  $= 0.7$  ).Unfairness score  $=$  inf.
Figure 2: CSC for a highly biased subgroup in terms of Equal Cost of Effectiveness with  $\phi = 0.7$ .
Method overview Deploying FACTS comprises three main steps: (a) Subgroup and action space generation that creates the sets of groups  $\mathcal{G}$  and actions  $A$  to examine; (b) Counterfactual summaries generation that applies appropriate actions to each group  $G\in \mathcal{G}$ ; and (c) CSC construction and fairness ranking that applies the definitions of Section 2.3. Next, we describe these steps in detail.
(a) Subgroup and action space generation Subgroups are generated by executing the fp-growth [11] frequent itemset mining algorithm on  $D_0$  and on  $D_{1}$  resulting to the sets of subgroups  $\mathcal{G}_0$  and  $\mathcal{G}_1$  and then by computing the intersection  $\mathcal{G} = \mathcal{G}_0\bigcap \mathcal{G}_1$ . In our setting, an item is a feature-level predicate of the form "feature-operator-value" and, consequently, an itemset is a predicate  $p$  defining a subgroup  $G_{p}$ . This step guarantees that the evaluation in terms of fairness will be performed
between the common subgroups  $\mathcal{G}$  of the protected populations. The set of all actions  $A$  is generated by executing fp-growth on the unaffected population to increase the chance of more effective actions and to reduce the computational complexity. The above process is parameterizable w.r.t. the selection of the protected attribute(s) and the minimum frequency threshold for obtaining candidate subgroups. We emphasize that alternate mechanisms to generate the sets  $A$  and  $\mathcal{G}$  are possible.
(b) Counterfactual summaries generation For each subgroup  $G_{p} \in \mathcal{G}$ , the following steps are performed: (i) Find valid actions, i.e., the actions in  $A$  that contain a subset of the features appearing in  $p$  and at least one different value in these features; (ii) For each valid action  $a$  compute  $\mathrm{eff}(a, G_{p,0})$  and  $\mathrm{eff}(a, G_{p,1})$ . The aforementioned process extracts, for each subgroup  $G_{p}$ , a subset  $V_{p}$  of the actions  $A$ , with each action having exactly the same cost for all individuals of  $G_{p}$ . Therefore, individuals of  $G_{p,0}$  and  $G_{p,1}$  are evaluated in terms of subgroup-level actions, with a fixed cost for all individuals of the subgroup, in contrast to methods that rely on aggregating the cost of individual counterfactuals. This approach provides a key advantage to our method in cases where the definition of the exact costs for actions is either difficult or ambiguous: a misguided or even completely erroneous attribution of a cost to an action will equally affect all individuals of the subgroup and only to the extent that the respective fairness definition allows it. In the setting of individual counterfactual cost aggregation, changes in the same sets of features could lead to highly varying action costs for different individuals within the same subgroup.
(c) CSC construction and fairness ranking FACTS evaluates all definitions of Section 2.3 on all subgroups, producing an unfairness score per definition, per subgroup. In particular, each definition of Section 2.3 quantifies a different aspect of the difficulty for a protected subgroup to achieve recourse. This quantification directly translates to difficulty scores for the protected subgroups  $G_{p,0}$  and  $G_{p,1}$  of each subgroup  $G_{p}$ , which we compare accordingly (computing the absolute difference between them) to arrive at the unfairness score of each  $G_{p}$  based on the particular fairness metric.
The outcome of this process is the generation, for each fairness definition, of a ranked list of CSC representations, in decreasing order of their unfairness score. Apart from unfairness ranking, the CSC representations will allow users to intuitively understand unfairness by directly comparing differences in actions between the protected populations within a subgroup.
This section presents the experimental evaluation of FACTS on the Adult dataset [1]; more information about the datasets, experimental setting, and additional results can be found in the appendix. The code is available at: https://github.com/AutoFairAthenaRC/FACTS. First, we briefly describe the experimental setting and then present and discuss Comparative Subgroup Counterfactuals for subgroups ranked as the most unfair according to various definitions of Section 2.3.
Table 1: Unfair subgroups identified in the Adult dataset.
Subgroup 1	Subgroup 2	Subgroup 3
rank	bias against	unfairness score	rank	bias against	unfairness score	rank	bias against	unfairness score
Equal Effectiveness	2950	Male	0.11	10063	Female	0.0004	275	Female	0.32
Equal Choice for Recourse (φ = 0.3)	Fair	-	0	12	Female	2	Fair	-	0
Equal Choice for Recourse (φ = 0.7)	6	Male	1	1	Female	6	Fair	-	0
Equal Effectiveness within Budget (c = 5)	Fair	-	0	2806	Female	0.056	70	Female	0.3
Equal Effectiveness within Budget (c = 10)	2350	Male	0.11	8518	Female	0.0004	226	Female	0.3
Equal Effectiveness within Budget (c = 18)	2675	Male	0.11	9222	Female	0.0004	272	Female	0.3
Equal Cost of Effectiveness (φ = 0.3)	Fair	-	0	Fair	-	0	1	Female	inf
Equal Cost of Effectiveness (φ = 0.7)	1	Male	inf	12	Female	2	Fair	-	0
Fair Effectiveness-Cost Trade-Off	4065	Male	0.11	3579	Female	0.13	306	Female	0.32
Equal (Conditional) Mean Recourse	Fair	-	0	3145	Female	0.35	Fair	-	0
Experimental Setting The first step was the dataset cleanup (e.g., removing missing values and duplicate features, creating bins for continuous features like age). The resulting dataset was split randomly with a 70:30 split ratio and was used to train and test respectively a logistic regression model (consequently used as the black-box model to audit). For the generation of the subgroups and the set of actions we used fp-growth with a  $1\%$  support threshold on the test set. We also implemented various cost functions, depending on the type of feature, i.e., categorical, ordinal, and numerical. A detailed description of the experimental setting, the models used, and the processes of our framework can be found in the supplementary material.
Unfair subgroups Table 1 presents three subgroups which were ranked at position 1 according to three different definitions: Equal Cost of Effectiveness  $(\phi = 0.7)$ , Equal Choice for Recourse  $(\phi = 0.7)$  and Equal Cost of Effectiveness  $(\phi = 0.3)$ , meaning that these subgroups were detected to have the highest unfairness according to the respective definitions. For each subgroup, its rank, bias against, and unfairness score are provided for all definitions presented in the left-most column. When the unfairness score is 0, we display the value "Fair" in the rank column. Note that subgroups with exactly the same score w.r.t. a definition will receive the same rank. The CSC representations for the fairness metric that ranked the three subgroups of Table 1 at the first position are shown in Figure 3.
Subgroup 1 is ranked first (highly unfair) based on Equal Cost of Effectiveness with  $\phi = 0.7$ , while it is ranked much lower or it is even considered as fair according to most of the remaining definitions (see the values of the "rank" column for Subgroup 1). The same pattern is observed for Subgroup 2 and Subgroup 3: they are ranked first based on Equal Choice for Recourse with  $\phi = 0.7$  and Equal Cost of Effectiveness with  $\phi = 0.3$  accordingly, but much lower according to the remaining definitions. This finding provides a strong indication of the utility of the different fairness definitions, i.e., the fact that they are able to capture different aspects of the difficulty in achieving recourse.<sup>1</sup>
Subgroup 1
If age  $= (41.0, 50.0]$ , marital-status  $= \text{Never-married}$ , race  $= \text{White}$ , relationship  $= \text{Not-in-family}$ : Protected Subgroup = 'Male', 1.34% covered No recourses for this subgroup. Protected Subgroup = 'Female', 1.47% covered Make marital-status  $= \text{Married-civ-spouse}$ , relationship  $= \text{Married}$  with effectiveness 70.49% Bias against 'Male' due to Equal Cost of Effectiveness (threshold = 0.7). Unfairness score = inf.
Subgroup 2
If workclass=Private, hours-per-week=FullTime, marital-status=Married-civ-spouse, occupation=Adm-clerical, race=White:  
Protected Subgroup = 'Male', 1.04% covered  
Make hours-per-week=OverTime, occupation=Exec-managerial with effectiveness 70.00%  
Make hours-per-week=OverTime, occupation=Prof-specialty with effectiveness 70.00%  
Make hours-per-week=BrainDrain, occupation=Exec-managerial with effectiveness 70.00%  
Make hours-per-week=BrainDrain, occupation=Prof-specialty with effectiveness 70.00%  
Make Workclass=Self-emp-in, occupation=Exec-managerial with effectiveness 70.00%  
Make Workclass=Self-emp-in, hours-per-week=OverTime, occupation=Exec-managerial with effectiveness 80.00%  
Make Workclass=Self-emp-in, hours-per-week=OverTime, occupation=Sales with effectiveness 70.00%  
Make Workclass=Self-emp-in, hours-per-week=BrainDrain, occupation=Exec-managerial with effectiveness 70.00%  
Protected Subgroup = 'Female', 3.51% covered  
Make Workclass=Self-emp-in, hours-per-week=OverTime, occupation=Exec-managerial with effectiveness 74.51%  
Make Workclass=Self-emp-in, hours-per-week=BrainDrain, occupation=Exec-managerial with effectiveness 74.51%  
Bias against 'Female' due to Equal Choice for Recourse (threshold = 0.7). Unfairness score = 6
Subgroup 3
If age=(41.0, 50.0], occupation=Sales:  
Protected Subgroup 'Male', 1.18% covered  
Make occupation Craft-repair with effectiveness 0.00%  
Make occupation=Adm-clerical with effectiveness 0.00%  
Make occupation=Tech-support with effectiveness 19.23%  
Make occupation=Prof-specialty with effectiveness 28.21%  
Make occupation=Exec-managerial with effectiveness 39.74%  
Protected Subgroup 'Female', 1.56% covered  
Make occupation=Craft-repair with effectiveness 0.00%  
Make occupation=Adm-clerical with effectiveness 0.00%  
Make occupation=Tech-support with effectiveness 0.00%  
Make occupation=Exec-managerial with effectiveness 6.94%  
Make occupation=Prof-specialty with effectiveness 6.94%  
Make age=(50.0,90.0] with effectiveness 6.94%  
Make age=(50.0,90.0], occupation=Prof-specialty with effectiveness 6.94%  
Make age=(50.0,90.0], occupation=Craft-repair with effectiveness 6.94%  
Make age=(50.0,90.0], occupation=Adm-clerical with effectiveness 6.94%  
Make age=(50.0,90.0], occupation=Exec-managerial with effectiveness 6.94%  
Bias against 'Female' due to Equal Cost of Effectiveness (threshold = 0.3). Unfairness score = inf.
Figure 3: Comparative Subgroup Counterfactuals for the subgroups of Table 1.
What is more, these different aspects can easily be motivated by real-world auditing needs. For example, out of the aforementioned definitions, Equal Cost of Effectiveness with  $\phi = 0.7$  would be suitable in a scenario where a horizontal intervention to support a subpopulation needs to be performed, but a limited number of actions is affordable. In this case, the macro viewpoint demonstrated in the CSC Subgroup 1 (top result in Figure 3) serves exactly this purpose: one can easily derive single, group-level actions that can effectively achieve recourse for a desired percentage of the unfavored
Additional examples, as well as statistics measuring this pattern on a significantly larger sample, are included in the supplementary material to further support this finding.
subpopulation. On the other hand, Equal Choice for Recourse  $(\phi = 0.3)$ , for which the CSC of a top result is shown in the middle of Figure 3, is mainly suitable for cases where assigning costs to actions might be cumbersome or even dangerous/unethical. This definition is oblivious to costs and measures bias based on the difference in the number of sufficiently effective actions to achieve recourse between the protected subgroups.
Another important observation comes again from Subgroup 1, where bias against the Male protected subgroup is detected, contrary to empirical findings from works employing more statistical-oriented approaches (e.g., [34]), where only subgroups with bias against the Female protected subgroup are reported. We deem this finding important, since it directly links to the problem of gerrymandering [17], which consists in partitioning attributes in such way and granularity to mask bias in subgroups. Our framework demonstrates robustness to this phenomenon, given that it can be properly configured to examine sufficiently small subgroups via the minimum frequency threshold described in Section 3.
There exist numerous challenges and constraints when attempting to determine costs associated with specific counterfactuals. The cost functions involved are intricate, dataset-specific, and frequently necessitate the expertise of a domain specialist. These cost functions may depend on individual characteristics or human interpretation of the perceived difficulty of specific changes, potentially giving rise to concerns like breaches of user privacy (as the expert may require access to all individual user characteristics [25]) or the risk of malicious specialists manipulating cost functions to conceal existing biases.
We acknowledge the difficulties of finding the dataset-dependent cost functions and proceed to implement various "natural" cost functions tailored to different feature types (e.g.,  $L_{1}$  norm, exponential distances). These are only suggestions to the expert who is going to use our framework, are susceptible to change, and can be tailored for the specific datasets. Our primary focus is to identify potential biases towards specific subpopulations and recognizing the aforementioned inherent difficulties in defining costs, we introduce fairness metrics that remain independent of cost considerations. We recognize that a single metric, whether cost-dependent or not, cannot identify all forms of bias that may be present within a model's predictions. We do not aim to replace the existing statistical measures for bias detection, rather than complement them by focusing on the bias of achieving recourse.
It is essential to note that FACTS, like other resource generation algorithms, may be susceptible to errors, as demonstrated in prior research [31]. Outcomes produced by FACTS can vary based on the chosen configuration, including hyperparameters and metric selection. These variations can either obscure biases present within a classifier or indicate their absence. It is crucial to emphasize that since FACTS is an explainable framework for bias assessment within subgroups, its primary purpose is to serve as a guidance tool for auditors to identify subgroups requiring in-depth analysis of fairness criteria violations. Responsible usage of our framework necessitates transparency regarding the chosen configuration. Therefore, we recommend running FACTS with different hyperparameters and utilizing as many metrics as possible. Appendix G provides a comprehensive discussion of FACTS' limitations and responsible disclosure guidelines.
In this work, we delve deeper into the difficulty (or burden) of achieving recourse, an implicit and less studied type of bias. In particular, we go beyond existing works, by introducing a framework of fairness notions and definitions that, among others, conceptualizes the distinction between the micro and macro viewpoint of the problem, allows the consideration of a subgroup as a whole when exploring recourses, and supports oblivious to action cost fairness auditing. We complement this framework with an efficient implementation that allows the detection and ranking of subgroups according to the introduced fairness definitions and produces intuitive, explainable subgroup representations in the form of counterfactual summaries. Our next steps include the extension of the set of fairness definitions, focusing on the comparison of the effectiveness-cost distribution; improvements on the exploration, filtering, and ranking of the subgroup representations, via considering hierarchical relations or high-coverage of subgroups; a user study to evaluate the interpretability of fairness judgments, and the development of a fully-fledged auditing tool.
This work has been funded by the European Union's Horizon Europe research and innovation programme under Grant Agreement No. 101070568 (AutoFair).
[1] K. Bache and M. Lichman. UCI machine learning repository, 2013.
[2] Reuben Binns. Fairness in machine learning: Lessons from political philosophy. In Conference on fairness, accountability and transparency, pages 149-159. PMLR, 2018.
[3] Emily Black, Samuel Yeom, and Matt Fredrikson. Fliptest: fairness testing via optimal transport. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 111-121, 2020.
[4] Naama Boer, Daniel Deutch, Nave Frost, and Tova Milo. Just in time: Personal temporal insights for altering model decisions. In 2019 IEEE 35th International Conference on Data Engineering (ICDE), pages 1988-1991. IEEE, 2019.
[5] Miranda Bogen and Aaron Rieke. Help wanted: An examination of hiring algorithms, equity, and bias. Analysis & Policy Observatory, 2018.
[6] Nadia Burkart and Marco F Huber. A survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Research, 70:245-317, 2021.
[7] Lee Cohen, Zachary C Lipton, and Yishay Mansour. Efficient candidate screening under multiple tests and implications for fairness. arXiv preprint arXiv:1905.11361, 2019.
[8] Giandomenico Cornacchia, Vito Walter Anelli, Giovanni Maria Biancofiore, Fedelucio Narducci, Claudio Pomo, Azzurra Ragone, and Eugenio Di Sciascio. Auditing fairness under unawareness through counterfactual reasoning. Information Processing & Management, 60(2):103224, 2023.
[9] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214–226, 2012.
[10] Vivek Gupta, Pegah Nokhiz, Chitradeep Dutta Roy, and Suresh Venkatasubramanian. Equalizing recourse across groups. arXiv preprint arXiv:1909.03166, 2019.
[11] Jiawei Han, Jian Pei, and Yiwen Yin. Mining frequent patterns without candidate generation. In Weidong Chen, Jeffrey F. Naughton, and Philip A. Bernstein, editors, Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, May 16-18, 2000, Dallas, Texas, USA, pages 1-12. ACM, 2000.
[12] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
[13] Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. Model-agnostic counterfactual explanations for consequential decisions. In AISTATS, volume 108 of Proceedings of Machine Learning Research, pages 895–905. PMLR, 2020.
[14] Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. A survey of algorithmic recourse: contrastive explanations and consequential recommendations. ACM Computing Surveys, 55(5):1-29, 2022.
[15] Amir-Hossein Karimi, Bernhard Scholkopf, and Isabel Valera. Algorithmic recourse: from counterfactual explanations to interventions. In FAccT, pages 353-362. ACM, 2021.
[16] Amir-Hossein Karimi, Julius Von Kugelgen, Bernhard Scholkopf, and Isabel Valera. Algorithmic recourse under imperfect causal knowledge: a probabilistic approach. Advances in neural information processing systems, 33:265-277, 2020.