SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 9: Example of three unfair subgroups in SSL
Subgroup 1	Subgroup 2	Subgroup 3
rank	bias against	unfairness score	rank	bias against	unfairness score	rank	bias against	unfairness score
Equal Effectiveness	1630.0	Black	0.076	70.0	Black	0.663	979.0	Black	0.151
Equal Choice for Recourse (φ = 0.3)	Fair	Fair	0.0	12.0	Black	1.0	12.0	Black	1.0
Equal Choice for Recourse (φ = 0.7)	13.0	Black	3.0	Fair	Fair	0.0	Fair	Fair	0.0
Equal Effectiveness within Budget (c = 1)	Fair	Fair	0.0	195.0	Black	0.663	1692.0	White	0.138
Equal Effectiveness within Budget (c = 2)	2427.0	Black	0.111	126.0	Black	0.663	3686.0	White	0.043
Equal Effectiveness within Budget (c = 10)	2557.0	Black	0.076	73.0	Black	0.663	1496.0	Black	0.151
Equal Cost of Effectiveness (φ = 0.3)	Fair	Fair	0.0	1	Black	inf	1	Black	inf
Equal Cost of Effectiveness (φ = 0.7)	1	Black	inf	Fair	Fair	0.0	Fair	Fair	0.0
Fair Effectiveness-Cost Trade-Off	3393.0	Black	0.111	443.0	Black	0.663	2685.0	Black	0.151
Equal (Conditional) Mean Recourse	3486.0	Black	0.053	1	Black	inf	1374.0	White	0.95
Table 10: Example of three unfair subgroups in Ad Campaign
Subgroup 1	Subgroup 2	Subgroup 3
rank	bias against	unfairness score	rank	bias against	unfairness score	rank	bias against	unfairness score
Equal Effectiveness	319.0	Female	0.286	Fair	Fair	0.0	467.0	Male	0.099
Equal Choice for Recourse (φ = 0.3)	5.0	Female	1.0	2.0	Female	4.0	Fair	Fair	0.0
Equal Choice for Recourse (φ = 0.7)	Fair	Fair	0.0	1	Female	4.0	Fair	Fair	0.0
Equal Effectiveness within Budget (c = 1)	Fair	Fair	0.0	Fair	Fair	0.0	Fair	Fair	0.0
Equal Effectiveness within Budget (c = 5)	Fair	Fair	0.0	Fair	Fair	0.0	345.0	Male	0.099
Equal Cost of Effectiveness (φ = 0.3)	1	Female	inf	Fair	Fair	0.0	Fair	Fair	0.0
Equal Cost of Effectiveness (φ = 0.7)	Fair	Fair	0.0	Fair	Fair	0.0	Fair	Fair	0.0
Fair Effectiveness-Cost Trade-Off	331.0	Female	0.286	Fair	Male	0.0	547.0	Male	0.099
Equal (Conditional) Mean Recourse	Fair	Fair	0.0	Fair	Fair	0.0	1	Male	inf
Subgroup 1
If PREDICTOR RAT ARRESTS VIOLENT OFFENSES = 1, PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT VICTIM BATTERY OR ASSAULT = 1:
Protected Subgroup = 'Black', 1.04% covered
No recourses for this subgroup
Protected Subgroup = 'White', 1.00% covered
Make PREDICTOR RAT ARRESTS VIOLENT OFFENSES = 0, PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT VICTIM BATTERY OR ASSAULT = 0 with effectiveness 72.73%
Make PREDICTOR RAT ARRESTS VIOLENT OFFENSES = 0, PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT VICTIM BATTERY OR ASSAULT = 0 with effectiveness 72.73%
Make PREDICTOR RAT ARRESTS VIOLENT OFFENSES = 0, PREDICTOR RAT NARCOTIC ARRESTS = 2, PREDICTOR RAT VICTIM BATTERY OR ASSAULT = 0 with effectiveness 72.73%
Bias against 'Black' due to Equal Cost of Effectiveness (threshold = 0.7). Unfairness score = inf
Subgroup 2
If PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-0.2, -0.1], PREDICTOR RAT UW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0:
Protected Subgroup = 'Black', 2.51% covered
Make PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-8.200999999999999, -0.3], PREDICTOR RAT UUW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-0.1, 0.1], PREDICTOR RUT UW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (0.1, 0.3], PREDICTOR RAT UUW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-8.200999999999999, -0.3], PREDICTOR RAT UUW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (0.3, 7.3], PREDICTOR RAT UUW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-0.3, -0.2], PREDICTOR RAT UW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-0.1, 0.1], PREDICTOR RUT UW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-0.2, -0.1], PREDICTOR RAT UW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (0.1, 0.3], PREDICTOR RAT UOW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 2, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-8.200999999999999, -0.3], PREDICTOR RAT UUW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (0.3, 7.3], PREDICTOR RAT UW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Protected Subgroup = 'White', 2.87% covered
Make PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-8.200999999999999, -0.3], PREDICTOR RAT UUW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 57.14%
Make PREDICTOR RAT NARCOTIC ARRESTS  $= 0$  , PREDICTOR RAT TREND IN CRIMINAL ACTIVITY  $= (0.3,7.3]$  , PREDICTOR RAT UW ARRESTS  $= 0$  , PREDICTOR RAT VICTIM SHOOTING INCIDENTS  $= 0$  with effectiveness  $0.00\%$
Make PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-0.3, -0.2],
PREDICTOR RAT UUW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-0.2, -0.1], PREDICTOR RAT UW ARRESTS = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness
Make PREDICTOR RAT NARCOTIC ARRESTS = 1, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (0.1, 0.3], PREDICTOR RAT UUAR REMST = 0, PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 0.00%
Make PREDICTOR RAT NARCOTIC ARRESTS  $= 1$  , PREDICTOR RAT TREND IN CRIMINAL ACTIVITY  $= (0.3,7.3]$  , PREDICTOR RAT UUW ARRESTS  $= 0$  , PREDICTOR RAT VICTIM SHOOTING INCIDENTS  $= 0$  with effectiveness  $0.00\%$
Bias against 'Black' due to Equal(Conditional) Mean Recourse. Unfairness score = inf
Subgroup 3
If PREDICTOR RAT GANG AFFILIATION = 1, PREDICTOR RAT NARCOTIC ARRESTS = 2, PREDICTOR RAT TRENDE IN CRIMINAL ACTIVITY = (-8.20099999999999, -0.3], PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0:
Protected Subgroup = 'Black', 1.18% covered
Protected Subgroup  $=$  'White'，  $1.00\%$  covered
Make PREDICTOR RAT GANG AFFILIATION = 0, PREDICTOR RAT NARCOTIC ARRESTS = 0, PREDICTOR RAT TREND IN CRIMINAL ACTIVITY = (-8.200999999999999, -0.3], PREDICTOR RAT VICTIM SHOOTING INCIDENTS = 0 with effectiveness 31.82%
Bias against 'Black' due to Equal Cost of Effectiveness (threshold = 0.3). Unfairness score = inf
Figure 6: Example of three Comparative Subgroup Counterfactuals in SSL; ref. Table 9
If age = 45-54, area = Unknown, parents = 1:  
Protected Subgroup = Male', 1.22% covered  
Make age=55-64, area=Rural with effectiveness 30.77%.  
Protected Subgroup = 'Female', 1.13% covered  
No recourses for this subgroup.  
Bias against Female due to Equal Cost of Effectiveness (threshold=0.3). Unfairness score = inf.
If age = 55-64, area = Unknown, homeowner = 1, income = Unknown, parents = 0, politics = Unknown, religion = Unknown:  
Protected Subgroup = 'Male', 2.53% covered  
Make homeowner=0, parents=1 with effectiveness 100.00%.  
Make homeowner=0, parents=1, religion=Christianity with effectiveness 100.00%.  
Make homeowner=0, parents=1, religion=Other with effectiveness 100.00%.  
Make area=Urban, parents=1, religion=Christianity with effectiveness 93.91%.  
Make area=Urban, parents=1, religion=Other with effectiveness 93.91%.  
Make homeowner=0, income=<100K, parents=1 with effectiveness 100.00%.  
Make area=Rural, parents=1, religion=Other with effectiveness 100.00%.  
Make area=Rural, parents=1, religion=Christianity with effectiveness 100.00%.  
Make homeowner=0, income=<100K, parents=1, religion=Christianity with effectiveness 100.00%.  
Make homeowner=0, income=<100K, parents=1, religion=Other with effectiveness 100.00%.  
Protected Subgroup = 'Female', 2.33% covered  
Make homeowner=0, parents=1 with effectiveness 100.00%.  
Make homeowner=0, parents=1, religion=Christianity with effectiveness 100.00%.  
Make homeowner=0, parents=1, religion=Other with effectiveness 100.00%.  
Make homeowner=0, income=<100K, parents=1 with effectiveness 100.00%.  
Make homeowner=0, income=<100K, parents=1, religion=Christianity with effectiveness 100.00%.  
Make homeowner=0, income=<100K, parents=1, religion=Other with effectiveness 100.00%.  
Bias against Female due to Equal Choice for Recourse (threshold=0.7). Unfairness score = 4.
If ages = 55–64, income = <100K, religion = Unknown:  
Protected Subgroup = 'Male', 1.02% covered  
Make religion=Christianity with effectiveness 0.00%.  
Make religion=Other with effectiveness 0.00%.  
Protected Subgroup = 'Female', 1.08% covered  
Make religion=Christianity with effectiveness 9.86%.  
Make religion=Other with effectiveness 9.86%.  
Bias against Male due to Equal Conditional Mean Recourse. Unfairness score = inf.
Figure 7: Example of three Comparative Subgroup Counterfactuals in Ad Campaign; ref. Table 10
E Further Discussion on Experiments
E.1 Model Agnostic Results
We note that our method is model-agnostic and does not depend on the model class. Nonetheless, we have considered two additional models, XGBoost and a Neural Network, for the Adult dataset. We provide an example CSC for Subgroup 1 of Figure 3, one for the output of each model (XGBoost in Figure 8 and NN in Figure 9)
Subgroup 1  
If age=(41.0, 50.0], marital-status=Never-married, race=White, relationship=Not-in-family: Protected Subgroup = 'Male', 1.13% covered No requasses for this subgroup. Protected Subgroup = 'Female', 1.14% covered No requasses for this subgroup. No bias
Figure 8: Output of FACTS on an XGBoost model, for the same subgroup as Subgroup 1 of Figure 3.
Subgroup 1  
If age=(41.0, 50.0], marital-status=Never-married, race=White, relationship=Not-in-family: Protected Subgroup = Male', 1.21% covered No requurses for this subgroup. Protected Subgroup = 'Female', 1.44% covered Make marital-status=Married-civ-spouse, relationship=Married with effectiveness 72.41%. Bias against 'Male' due to Equal Cost of Effectiveness (threshold = 0.7). Unfairness score = inf.
Figure 9: Output of FACTS on a simple NN model, for the same subgroup as Subgroup 1 of Figure 3.
E.2 Generalization to multiclass demographic groups
The generalization of the algorithm to multiple protected groups (comprising multiclass demographic groups) is a straightforward process. For instance, we can compare the burden of each protected group against all other groups using a one-vs-all approach. An example of this methodology applied to the Adult dataset and its multi-valued race attribute is illustrated in Figure 10.
If capital-gain=0,marital-status=Divorced: Protected Subgroup  $=$  Asian-Pac-Islander'，12.95% covered Make capital-gain=15024,marital-status=Married-civ-spouse with effectiveness 100.00%. Make capital-gain=7688,marital-status=Married-civ-spouse with effectiveness 75.00%. Make capital-gain=7298,marital-status=Married-civ-spouse with effectiveness 75.00%. Protected Subgroup  $=$  'Amer-Indian-Eskimo'，20.37% covered Make capital-gain=15024,marital-status=Married-civ-spouse with effectiveness 100.00%. Make capital-gain=7688,marital-status=Married-civ-spouse with effectiveness 81.82%. Make capital-gain=7298,marital-status=Married-civ-spouse with effectiveness 72.73%. Protected Subgroup  $=$  'Black'，15.15% covered Make capital-gain=15024,marital-status=Married-civ-spouse with effectiveness 98.90%. Protected Subgroup  $=$  'Other'，12.97% covered Make capital-gain=15024,marital-status=Married-civ-spouse with effectiveness 100.00%. Make capital-gain=7688,marital-status=Married-civ-spouse with effectiveness 76.92%. Make capital-gain=7298,marital-status=Married-civ-spouse with effectiveness 69.23%. Protected Subgroup  $=$  'White'，16.96% covered Make capital-gain=15024,marital-status=Married-civ-spouse with effectiveness 99.41%. Make capital-gain=7688,marital-status=Married-civ-spouse with effectiveness 73.90%. Make capital-gain=7298,marital-status=Married-civ-spouse with effectiveness 70.43%. Bias against 'Black' due to Equal Choice of Recourse.Unfairness score = 2.
Figure 10: Output of FACTS for Adult for a multi-valued protected attribute (race); all protected subgroups other than 'Black' have three options (actions) to achieve recourse with an effectiveness of at least 0.7, while 'Black' has a single option. Therefore we conclude bias against the 'Black' using the Equal Choice of Recourse.
E.3 Runtime and statistics
In Table 11, we present the runtime of FACTS for all datasets. We also provide statistics on the output generated by fp-growth for all these datasets, offering preliminary insights into scalability.
Additionally, we include the runtime for the Adult dataset, with sex serving as the protected attribute, for various combinations of minimum support threshold and the number of bins in the numerical attributes (see Table 13). These results encompass both the total runtime of FACTS and the time taken by FACTS in the assessment of fairness step. In Table 14 we report the corresponding statistics from fp-growth for the same combinations as presented in 13. It is worth noting that our primary contribution is conceptual in nature, focusing on the formalization of recourse unfairness notions, rather than technical, i.e., how to efficiently determine a good set of actions to explore.
Table 11: Runtime for all datasets (min-support threshold: 0.01 for Adult, COMPAS, SSL, Ad campaign; 0.04 for German credit)
Adult (sex)	Adult (race)	COMPAS	SSL	Ad campaign	German credit
Runtime (seconds)	1,875	2,208	11	1,807	860	689
Table 12: Statistics on the output of fp-growth for all datasets
Adult (sex)	Adult (race)	COMPAS	SSL	Ad campaign	German credit
# male (white) subgroups (affected)	27,510	27,767	1431	7,836	1,658	244,777
# female (non-white) subgroups (affected)	28,176	24,664	3,984	7,801	1,700	241,287
# actions (unaffected)	57,008	56,394	2,895	17,270	3,476	83,631
# common subgroups	13,300	18,692	95	6,552	1,651	36,970
# valid actions	97,413	112,879	188	86,782	4,274	117,143
# subgroups	12,556	15,672	88	6,551	1,280	31,509
Table 13: Runtime for Adult (sex) while varying min-support threshold and number of bins
(min-support threshold, # bins)	(0.1, 10)	(0.05, 10)	(0.01, 10)	(0.05, 5)	(0.05, 20)
Runtime (seconds)	35	178	3,539	265	155
Runtime to assess fairness (seconds)	8	10	17	10	10
Table 14: Statistics on the output of fp-growth for Adult (sex) while varying min-support threshold and number of bins
(min-support threshold, # bins)	(0.1,10)	(0.05,10)	(0.01,10)	(0.05,5)	(0.05,20)
# of male subgroups (affected)	997	3,017	25,569	4,017	2,309
# of female subgroups (affected)	933	3,024	26,173	4,021	2,397
# of common subgroups	453	1,583	12,093	1,969	1,126
# of actions (unaffected)	2,828	7,777	60,006	10,709	5,961
# of all valid actions	351	3,120	87,687	4,348	2,538
# of subgroups	283	1,235	10,975	1,706	887
F Comparison of Fairness Metrics
The goal of this section is to answer the question: "How different are the fairness of recourse metrics". To answer this, we consider all subgroups and compare how they rank in terms of unfairness according to 12 distinct metrics. The results justify our claim in the main paper that the fairness metrics capture different aspects of recourse unfairness. For each dataset and protected attribute, we provide (a) the ranking analysis table, and (b) the aggregated rankings table.
The first column of the ranking analysis table shows the number of the most unfair subgroups per metric, i.e., how many ties are in rank 1. Depending on the unit of the unfairness score being compared between the protected subgroups (namely: cost, effectiveness, or number of actions), the number of ties can vary greatly. Therefore, we expect to have virtually no ties when comparing effectiveness percentages and to have many ties when comparing costs. The second and third columns show the number of subgroups where we observe bias in one direction (e.g., against males) and the opposite (e.g., against females) among the top  $10\%$  most unfair subgroups.
The aggregated rankings table is used as evidence that different fairness metrics capture different types of recourse unfairness. Each row concerns the subgroups that are the most unfair (i.e., tied at rank 1) according to each fairness metric. The values in the row indicate the average percentile ranks of these subgroups (i.e., what percentage of subgroups are more unfair) when ranked according to the other fairness metrics, shown as columns. Concretely, the value  $v$  of each cell  $i$ ,  $j$  of this table is computed as follows:
1. We collect all subgroups of the fairness metric appearing in row  $i$  that are ranked first (the most biased) due to this metric.
2. We compute the average ranking  $a$  of these subgroups in the fairness metric appearing in column  $j$ .
3. We divide  $a$  with the largest ranking tier of the fairness metric of column  $j$  to arrive at  $v$ .
Each non-diagonal value of this table represents the relative ranking based on the specific metric of the column for all the subgroups that are ranked first in the metric of the respective row (all diagonal values of this table are left empty). A relative ranking of  $v$  in a specific metric  $m$  means that the most unfair subgroups of another metric are ranked lower on average (thus are fairer) for metric  $m$ .
F.1 Comparison on Adult for protected attribute gender
The number of affected individuals in the test set for the adult dataset is 10,205. We first split the affected individuals on the set of affected males  $D_{1}$  and the set of affected females  $D_{0}$ . The number of subgroups formed by running fp-growth with support threshold  $1\%$  on  $D_{1}$  and on  $D_{0}$  and computing their intersection is 12,880. Our fairness metrics will evaluate and rank these subgroups based on the actions applied.
Tables 15 and 16 present the ranking analysis and the aggregated rankings respectively on the gender attribute, on the Adult dataset. Next, we briefly discuss the findings from these two tables; similar findings stand for the respective tables of the other datasets, thus we omit the respective discussion.
It is evident from Table 15 that the different ways to produce ranking scores by different definitions can lead to considerable differences in ties, i.e., the number of subgroups receiving the same rank (here only rank 1 is depicted). The "Top  $10\%$  " columns demonstrate interesting statistics on the protected subgroup for which bias is identified: while it is expected that mostly bias against "Female" will be identified, subgroups with reverse bias (bias against "Male") are identified, indicating robustness to gerrymandering, as hinted in Section 4 of the main paper.
Table 16 is produced to provide stronger evidence on the unique utility of the various presented definitions (see footnote 1 of the main paper: "Additional examples, as well as statistics measuring this pattern on a significantly larger sample, are included in the supplementary material to further support this finding.") In particular, in this table, for all subgroups that are ranked first in a definition, we calculate their average relative (normalized in [0, 1]) ranking in the remaining definitions. Given this, a value close to 1 means very low average rank and a value close to 0 means very high rank. Consequently, values away from 0 indicate the uniqueness and non-triviality of the different definitions and this becomes evident from the majority of the values of the table.
Table 15: Ranking Analysis in Adult (protected attribute gender)
# Most Unfair Subgroups	# Subgroups w. Bias against Males (in Top 10% Unfair Subgroups)	# Subgroups w. Bias against Females (in Top 10% Unfair Subgroups)
(Equal Cost of Effectiveness(Macro), 0.3)	1673	56	206
(Equal Cost of Effectiveness(Macro), 0.7)	301	26	37
(Equal Choice for Recourse, 0.3)	2	54	286
(Equal Choice for Recourse, 0.7)	6	31	50
Equal Effectiveness	1	39	1040
(Equal Effectiveness within Budget, 5.0	1	41	616
(Equal Effectiveness within Budget, 10.0)	1	6	904
(Equal Effectiveness within Budget, 18.0)	1	22	964
(Equal Cost of Effectiveness(Micro), 0.3)	1523	10	226
(Equal Cost of Effectiveness(Micro), 0.7)	290	38	27
Equal(Conditional Mean Recourse)	764	540	565
(Fair Effectiveness-Cost Trade-Off, value)	1	61	1156
Table 16: Aggregated Rankings in Adult (protected attribute gender)
(Equal Cost of Effectiveness (Macro), 0.3)	(Equal Cost of Effectiveness (Macro), 0.7)	(Equal Choice for Recourse, 0.3)	(Equal Choice for Recourse, 0.7)	Equal Effectiveness	(Equal Effectiveness within Budget, 5.0)	(Equal Effectiveness within Budget, 10.0)	(Equal Effectiveness within Budget, 10.0)	(Equal Cost of Effectiveness (Micro), 0.3)	(Equal Cost of Effectiveness (Micro), 0.7)	(Equal/Conditional Mean Recourse)	(Fair Effectiveness-Cost Trade-Off, value)
(Equal Cost of Effectiveness(Macro), 0.3)	1.0	0.836	1.0	0.214	0.509	0.342	0.285	0.3	1.0	0.441	0.237
(Equal Cost of Effectiveness(Macro), 0.7)	0.634	-	0.864	0.686	0.358	0.602	0.464	0.407	0.738	0.293	0.481	0.307
(Equal Choice for Recourse, 0.3)	0.018	1.0	-	-	0.001	0.006	0.001	0.01	0.017	1.0	0.105	0.001
(Equal Choice for Recourse, 0.7)	1.0	0.364	0.857	-	0.814	0.528	0.813	0.81	1.0	0.882	0.451	0.34
Equal Effectiveness	0.018	1.0	0.214	1.0	-	0.003	0.0	0.0	0.017	1.0	0.058	0.0
(Equal Effectiveness within Budget, 5.0)	0.018	1.0	0.857	1.0	0.006	-	0.004	0.006	0.017	1.0	1.0	0.006
(Equal Effectiveness within Budget, 10.0)	0.018	1.0	0.214	1.0	0.0	0.002	-	0.0	0.017	1.0	0.047	0.0
(Equal Effectiveness within Budget, 18.0)	0.018	1.0	0.214	1.0	0.0	0.003	0.0	-	0.017	1.0	0.058	0.0
(Equal Cost of Effectiveness(Micro), 0.3)	0.238	1.0	0.857	1.0	0.136	0.452	0.263	0.215	1.0	1.0	0.462	0.155
(Equal Cost of Effectiveness(Micro), 0.7)	0.611	0.279	0.864	0.771	0.336	0.621	0.449	0.402	0.7	-	0.465	0.295
Equal(Conditional) Mean Recourse	0.996	1.0	1.0	1.0	0.723	0.946	0.875	0.777	0.997	1.0	-	0.83
(Fair Effectiveness-Cost Trade-Off, value)	0.018	1.0	0.214	1.0	0.0	0.002	0.0	0.0	0.017	1.0	0.047	-
F.2 Comparison on Adult for protected attribute race
The number of affected individuals in the test set for the adult dataset is 10,205. We first split the affected individuals on the set of affected whites  $D_{1}$  and the set of affected non-whites  $D_{0}$ . The number of subgroups formed by running fp-growth with support threshold  $1\%$  on  $D_{1}$  and on  $D_{0}$  and computing their intersection is 16,621. Our fairness metrics will evaluate and rank these subgroups based on the actions applied.
Table 17: Ranking Analysis in Adult (protected attribute race)
# Most Unfair Subgroups	# Subgroups w. Bias against Whites (in Top 10% Unfair Subgroups)	# Subgroups w. Bias against Non-Whites (in Top 10% Unfair Subgroups)
(Equal Cost of Effectiveness(Macro), 0.3)	1731	0	295
(Equal Cost of Effectiveness(Macro), 0.7)	325	7	51
(Equal Choice for Recourse, 0.3)	1	2	391
(Equal Choice for Recourse, 0.7)	2	10	60
Equal Effectiveness	1	6	1433
(Equal Effectiveness within Budget, 1.15	1	50	24
(Equal Effectiveness within Budget, 10.0	1	3	1251
(Equal Effectiveness within Budget, 21.0)	1	0	1423
(Equal Cost of Effectiveness(Micro), 0.3)	1720	0	294
(Equal Cost of Effectiveness(Micro), 0.7)	325	7	51
Equal(Conditional Mean Recourse)	2545	53	1316
(Fair Effectiveness-Cost Trade-Off, value)	2	0	0
F.3 Comparison on COMPAS