Auditing Predictive Models for Intersectional Biases
Authors: Kate Boxer
Edward McFowland III
Daniel B. Neill
Venue: Accepted by TMLR ()

Abstract: Predictive models that satisfy group fairness criteria in aggregate for members of a protected class, but do not guarantee subgroup fairness, could produce biased predictions for individuals at the intersection of two or more protected classes. To address this risk, we propose Conditional Bias Scan (CBS), an auditing framework for detecting intersectional biases in the outputs of classification models that may lead to disparate impact. CBS aims to identify the subgroup with the most significant bias against the protected class, compared to the equivalent subgroup in the non-protected class. The framework can audit for predictive biases using common group fairness definitions that can be represented as conditional independence statements (separation and sufficiency) for both probabilistic and binarized predictions. We show through empirical evaluations that this methodology has substantially higher bias detection power compared to similar methods that audit for subgroup fairness. We then use this approach to detect statistically significant intersectional biases in the predictions of the COMPAS pre-trial risk assessment tool and a model trained on the German Credit data.

Published in Transactions on Machine Learning Research (01/2026)
Auditing Predictive Models for Intersectional Biases
Kate S. Boxer Deportment of Computer Science
kb145@mpu.edd
New York University
Edward McFowland III
emefoulladd@hbs.c edu
Technology and Operations Management Horvard Business School
Daniel B. Neill
daniel.neill
Deportment of Computer Science New York University
Reviewed OpenReview:
htt tp /t envevii nne
1JTh LHMS m0
Abstract
Predictive models that satisfy group fairness criteria in aggregate for members of protected class, but do not guarantee subgroup fairness, could produce biased predictions for individuals at the intersection of two or more protected classes. To address this risk, we propose Conditional Bias Scan (CBS), an auditing framework for detecting intersectional biases in the outputs of classification models that may lead to disparate impact. CBS aims to identify the subgroup with the most significant bias against the protected class, compared the equivalent subgroup in the non-protecd non-protected class. The framework can audit for predictive biases using common group fairness definitions that be represented conditional independence statements (separation and sufficiency) for both probabilistic and binarized predictions. We show through empirical evaluations that this methodology has substantially higher bias detection power compared to similar methods that audit for subgroup fairness. We then use this approach to detect statistically significant intersectional biascs in the prodictions of the COMPAS pre-trial risk assessment tool and a model trained on the German Credit data.
1 Introduction
Predictive models are increasingly used assist in high-stakes decisions and, therefore, can have significant impacts individuals' lives and livelihoods. However, recent studics have revealed numerous models whose predictions contain biases, in the form of group fairness violations, against disadvantaged and marginalized groups (Angwin et al. 2016 Obermeyer et al. 2019). When auditing predictive model for bias, typical group fairness definitions (Mitchell et al.) 2021) rely on univariate measurements of the difference between the distributions of predictions or outcomes for individuals in a "protected class", typically defined by sensitive attribute such as race or gender, as compared to those the non-protected class. Since these approaches only detect biases for prodetermined subpopulation at an aggregate level, c.g., a bias against Black individuals, they may fail to detect biases that adversely affect subset of individuals in protected class, e.g., Black females.
While it is possible to define a specific multidimensional subgroup and then audit a classifier for biases impacting that subgroup, this approach relies on defining a fixed subgroup and therefore limited to static queries. From a computational perspective, this approach does not scale to the exponential number of multidimensional subgroups; from a statistical perspective, it fails to account for the risk of identifying many false positives if many subgroups are tested for bias. To detect whether there are any subgroups within given protected class that are adversely impacted by predictive biases, need way to search over

---

Published in Transactions on Machine Learning Research (01/2026)
the combinatorial number of subgroups of the protected class, and to account for this search process when deciding which observed biases are statistically significant, rather than separately auditing each subgroup-
To address these issues, we present Conditional Bias Scan (CBS), a novel and flexible framework for bias detection. Given a classifier's probabilistic predictions or binarized recommendations based on those predictions, CRS discovers systematic biases impacting any subgroups of a predefined subpopulation of interest (the protected class). More precisely, CBS aims to discover subgroups of the protected class for whom the classifier's predictions recommendations systematically deviate from the corresponding subgroup of individuals who not part of the protected class. Subgroups are defined by a non-empty subset of attribute values for cach observed attribute, excluding the sensitive attribute which determines whether or not individuals belong to the protected class. We constrain subgroups to categorical representations of the non- non-sensitive attributes for explainability purposes. This requires discretization of continuous variables prior to the scanning step of our methodology, as discussed in detail below. To provide concrete example, the binary sensitive attribute could be defined by whether or not an individual is female, and CBS would search over all other attributes (e-g., race/el ennicty and income) to identify subgroups with gender disparity between female and non-femall individuals. If CBS returns subgroup consisting of Black or Hispanic individuals whose yearly income is $0-$9,999 or $10,000-$25,000 this would indicate that the most significant gender disparity identified by the scan affected individuals with this combination of vace/ethnicit and low income. We provide formal notation for a sensitive attribute and subgroup at the start of Section 2
The detected subgroups can represent both intersectional and contextual biases. In this paper, as in much of the algorithmic fairness literature, we refer to intersectional biases when individuals of subgroup are members of two or more protected classes. We refer to contertual biases for forms of subgroup biases that may only be present for certain decision situations (Runyan 2018 In Section 5.4 we discuss the distinct, yet related topic of intersectionality theory (Crenshaw 1991a b Collins 2008) in sociology, which describes how
individuals" different social positions and identities interact to influence their social and material conditions. In particular, an individual at the intersection of several marginalized groups may be impacted by multiple historical and continuing systems of power and oppression including structural racism, sexism, and class stratification.
The contributions of our rescarch include:
A methodological framework that can flexibly accommodate multiple group fairness definitions and can effectively detect intersectional and contextual biases.
computationally efficient detection algorithm to audit classifiers for fairness violations in the exponentially many subgroups of a given subpopulation.
Detailed empirical evaluations demonstrating substantially improved bias detection accuracy as compared to similar methods that audit for subgroup fairness, along with real-world case studs on COMPAS and German Credit datassts.
Given the overarching objective of understanding the full scope of predictive biases that a model produces for all the sensitive subgroups of given target population, there is a need for expanded measurements of predictive bias and improved methods for searching for these biases within all sensitive subgroups that could be adversely affected by predictive bias. Without auditing tools that can accurately identify these biases, any predictive bias definition will be limited to evaluating a small, static set of subgroups, and there will presumably be some form of intersectional or contextual bias that goes undetected. Therefore, CBS is an important step towards understanding the full scope of biases that a predictive model might produce.
2 Methods
X,,, We define the dataset D = (A, ((,,,,,, Y,, Pr,w))
for n individuals indexed as 1. The sensitive attribute, A,, a binary variable representing whether individual belongs to the protected class. X; =X!..... ........XX. ..r) are other covariates for individual i, excluding A,. Y, is individual i's observed binary outcome, P. E [0, 1] is the classifier's probabilistic prediction of individual i's outcome, and
2

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: Table of scan types for CBS for different group fairness definitions. Scans that correspond well-known fairness definitions (c.g., false positive error rate balance) noted in the table. Shading is used to distinguish between the four main categories of scans (separation for predictions, separation for
recommendations, sufficiency for predictions, and sufficieccy for recommendations). The notation 1 refers to conditional independence from membership in the protected class (4).
Predictions (P [0,
Recommendations (Poin E {0. 13)
Poin
Poin
B = LAXLLA
Pr(Poin = =
Pr(Pbin = 0 0 Y = 1,X)L
==1
Balance for Positive Class
True Positive Rate
False Negative Rate
Separation
E[P Y 0, LA
Pr(Poin bin = =0 XYLA
Pr(Pain = =0 0
==0
Balance for Negotšve Class
False Positive
True Negutive
Pr 1
Pr(Y 1| Poin 1, XYALA
Pr(Y Poin 0, LLA
==1
Positive Predictive Value
False Omiission Rate
Sufficiency
Pr(Y = P P)X)
Pr(Y = 01 Poin = 1,
Pr(Y = 0| Poin = 0, LA
==0
False Discovery Rate
Negatioc Predictive Value
Punii 1] is the binary recommendation correspouding to P.. For example, is common to define Pa,ii = 1(2: 0.5).
Given these data, the CBS framework searches for subgroups of the protected class, defined by subset of values for each covariate XI...., for whom some group fairness definition (contained in Table is violated with respect to A. Therefore, CBS returns subgroup S represented as the Cartesian product, S = 5t x S2 x x... ... x 5th, where Ş X for = 1,.... ,,...,,. m. Each fairness definition in Table ! is in the form of a conditional independence relationship between an individual's membership in the protected class, A,, and their value an event variahle, Is conditioned on their value conditional voriable, C₁ and their covariates, X5: |[[4] Cr, A,J We define the null hypothesis, Ho, that A (C, and use CBS search for subgroups with statistically significant violations of this conditional independence relationship, correctly adjusting for multiple hypothesis testing, allowing us to reject Ho in favor of the alternative hypothesis H1 that 1 y A XC.
The CBS framework has four sequential sleps:
(1) Given fairness definition, CBS chooses I Ponn and C (Y,P, Section 2.1 maps different group fairness criteria to particular choices of event variable I and conditional variable C.
(2) CBS estimates the expected value of I; for each individual in the protected class under the null hypothesis Ho that and A are conditionally independent given C' and X. These expectations are denoted as L, and Section 2.2 describes how to estimate
(3) CBS uses a novel multidimenssioaal subset scan to search for subgroups, S, where, for E S, the observed I, deviates systematically from its expectation 1, in the direction of interest. This step to detect S* is described in Section 2.3
(4) The final step evaluate stafistical significance of the detected subgroup S in Section 2.4 uses permutation testing to adjust for multiple hypothesis testing and determine if S deviation between protected and non-protected class is statistically significant.
P. bim sometimes referred binary prediction. We "recommendation" distinguish Paben from the probabilistic prediction P..
3

---

Published in Transactions on Machine Learning Research (01/2026)
Note that, for Step 2 covariates X can be represented as continuous categorical variables; prior to Step (3) X must be discretized and represented as categorical variables so that interpretable subgroups can be produced by the scan.
2.1 Define (1,C): Overview of Scan Types
Many of the group fairness criteria proposed in the fairness literature fall into two categories of statistical lairness called sufficiency and separation. Sufficieccy is focused on equivaleney in the rate of outcome (for comparable individuals with the same prediction recommendation) regardless of protected class membership (Y 1 A P. X),, whereas separation is focused on equivalency of the expected prediction or rocommendation (for comparable individuals with the same outcome) regardless of protected class membership (P 1 A Y. X)..
The choice between separation and sufficiency determines whether outcome Y the event variable of interest or the conditional variable C, where bias is defined as E[ X, = 1] + EE C, X, = 0]. The combination of fairness metric (sufficiency or separation) and prediction type (contimuous prediction or binary recommendation) produces four classes of fairness scans, defined by the chosen values for I and C:
For separation for predictions, we define 1 = P and C = Y.
For separation for recommendations, we define I = Phin and C = Y.
For sufficiency for predictions, we define I Y and C = P.
For sufficiency for recommendations, we define I = and C Poin
also perform "value-conditional" scans by restricting the value of the conditional variable, C. For example, to scan for subgroups with increased false positive rate (FPR), we restrict the data to individuals with Y = 0, and perform a scparation scan for recommendations (setting T= Pain and C Y). All of the scan options for CBS are shown in Table Each scan in Table can detect bias in either direction, e.g., searching for subgroups with increased FPR (i.e., positive direction bias) or decreased FPR (i.e., negative direction bias).
Depending on the particular bias of interest,
2.2 Generate Expectations of the Event Variable
Once have defined the event variable I and conditional variable C, as discussed in Section 2.1 we wish to detect fairness violations by assessing whether there exist subgroups of the protected class where E[ C, X, A = 1] differs systematically from I[ C, X, = Oj. For each individual in the protected class, 4 C, A; = 1 is observed but I, C, X,, = is unobserved. Thus we calculate an estimate f. Err, [T, C,, X,, A, = 1], under the null hypothesis, Ho: 1. A (C,X), and compare î, to the observed 4. Using the estimated î and observed I, we aim to determine which subgroups in the protected class have the largest deviations in as compared to what we would expect if there was bias, î. The method to generate Î borrows from the literature causal inference in observational settings, where propensity score reweighting is used to account for the selection of individuals into a "treatment" condition (here, membership in the protected class) given their observed covariates X.
The method to estimate i consists of the following steps:
Step 1: Train a predictive model using all the individuals in the data to estimate Pr(A = x
Step 2:
Use this model to produce the probabilitics, p,a Pr(A, = 1| and the corresponding propensity
x score weights, w T- for each individual i in the non-protected class (A, 0). Intuitively, individuals the non-protected class whose attributes X; are more similar to individuals in the protected class have higher weights wyA. This weighting scheme is used in the literature produce causal effect estimates that can be interpreted as the average treatment effect on treated individuals (ATT) under typical causal inference assumptions of positivity and strong ignorability.
4

---

Published in Transactions on Machine Learning Research (01/2026)
Step 3: Estimate EH [I C.X]:
Case A (Binary event variable): For all sufficiency scans and separation scan for recommenda- tions, where the event variable, L, is binary, we train a model using only data for individuals in the non-protected class (A, = 0) to estimate En, [ C, X] by weighting each individual in the non-protected class by wz" The trained model is used to estimate the expectations 1-=EH,,44 [4; Ci X.] for each individual in the protected class (A; = 1) under the null hypothesis, Ho. of I L A XC,
Case B (Real-valued event variable): For the separation scan for predictions, where the event variable, I, is a real-valued variable, the probabilistic predictions P. rather than a binary event variable, we use a similar but modified process to estimate [ XT, where / = P and C = Y. For each individual i in the non-protected class, we create two training records containing the same covariates, X,, but different labels and associated weights:
i. For the first record, we set the label, 1,eme, ; equal to 1, and set the weight to wpPP P.
ii. For the second record, we set the label, 1.omp equal to O, and set the weight to w (1 (1). Note, for separation scan for predictions, P: is the observed prediction for individual i. We create dataset that includes both records, described in [ and i above, for each individual in the non-protected class and their associated weights, and use this concatenated data set to train a model that estimates EH /omp C, by weighting each individual i in the non-protected class by either w P, or wrA - P.) as described above. This approach is consistent with other CBS variants and enforces the desired constraint 0 < f, S 1, unlike alternative approaches such as using regression models to predict í.
For valne-conditional scans, such as the FPR, FDR, and balance for positive class scans shown in Table | CBS andits for biases in the subset of data where C == 2, for 2 E (0, 13. Dataset D is filtered before Stee to only include individuals where C' -==a z. For example, for the value-conditional scan for FPR, we filter the data to only include individuals where C 0 (or equivalently, Y ===0 0).
A probabilistic model can be used to estimate Pr(A - 1 ||X) X) in Step 1 and probabilistic model that allows for weighting of instances during training can be used to estimate EH, [ C, X] in Step 3, For Sections 3 and as well as Appendices B.3 and B.4, we use logistic regression to estimate Pr(A = 1 X)) and weighted logistic regression to estimate EH, [X, When estimating EH, PP (the realized expectation of [H, C, X)) for sufficiency scan for prodictions, we transform the conditional variable, P,, to its corresponding log-odds, log 3_ prior to training, since we expect log y (the larget of the logistic regression) to be approximately P. for well-calibrated classificrs.
log 1-P:
Accurate estimates of î are essential for CBS to accuratcly detect the subgroup in the protected class with the most deviation between the observed T and estimated î under the null hypothesis of no bias. The method described above has the limitation of only producing accurate estimates of I when both the model for Pr(A- = 1 x and EH,, [7] C, X] are well-specified. Given the consistency of our findings for the COMPAS well as other checks performed to examine T (such as the calibration curve plots for both Pr(A = 1 | X) and EH,, (7 | C, X] for our COMPAS case study included in Appendix C.I.I,, we believe that the method above suffices for COMPAS. However, we find that logistic regression is insufficient in estimating î for the German Credit Data, due to the smaller dataset size and highly-correlated predictors. Thus we use more flexible model a gradient boosting classifier with Platt scaling-in in our German Credit Data experiments predictions are well-calibrated when computing propensity scores and when estimating T. In Appendix C.2.3 we include calibration curve plots for the models used to estimate Pr(A- = X) and EH, C] for our German Credit Data case study. We encourage others using CBS to be aware of this limitation, pay special consideration to estimates of 1, and if necessary, employ methods from the causal inference literature on doubly robust estimation [Tmbens] 2004 Schuler & Rose 2017) or methods from the computer science literature for model calibration when producing estimates of I.
case study in Section 4 with other researchers' findings about COMPAS,
have
Appendix C.2 to cnsure that CBS
We note that both discreto-valued and continuous-va valued covariates, X, can be used for estimating i, for both the propensity model Pr(AA = 1 X) and the model of [T C) However, continuous- covariates,
5

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: Null and alternative hypotheses, Ho and (S),,, and corresponding log-likelihood ratio score functions, F(S), used mcasure subgroup's degree of anomalousness (comparing the event variable its expectation / under Ho) for all four variants of CBS. Shading is used to distinguish between Gaussian and Bernoulli scans and provide consistency with Table I Over-estination (under-estimatiion bias means that the expectations I; are larger (smaller) than I; Note that the free parameters in F(S) are and for Gaussian and Bernoulli scan respectively. Derivations of F(S) can be found in Appendix A.2
Scan Types
Hypotheses and Bias Constraints
F(S)
A N(O,)), We Dr A o(H,0)
(i)
2/r
2 iEg 4.)
181/
Predictions
where
A. log 14 loog
max
20
1-f:
Scparation
Over-estimation:
0, vi S, and = 0, vi & S.
Finder-estimation:
0, S. and μ = 0, Vi vigs
Gaussian Distribation
Recommendations
Ho
odds(1.)
1, 1- E Di
Hi(S):
odds(I;)
max Eies log(1)
Predictions
log(gh - 1
Sufficiency
Over-estination:
< 1, Vi C S, and q 1, Vi 2 S. > 1, vi E S, and q 1, vi S.
Recommendations
Under-estimation:
Bernoull Distribution
X, must be discretized or removed prior to the step described below in Section 2.3 which requires that all scan dimensions, i.e. the covariates, X, are discrete-vllued.
2.3 Detect the Most Significant Subgroup S*
Given the observed event variables I; and the expectations f, of the event variable under the mull hypot (I L A C, X) for the protected class, where the procedure for calculating is described in Section 2.2 define score function measuring subgroup bias, F S > Rgo, that can be efficiently optimized over exponentially many subgroups with the goal of identifying S* arg maxş F(S).
To do so, we follow the literature on spatial and subset scan statistics Kulldorff 1997 Neil 2012) by defining the general form of the various score functions, F(S), for the CBS scans as a log- likelihood ratio (LLR) test statistic:
(Pr(D (S)) (Pr(D) Ho)
F(S) = log
(1)
Here the denominator represents the likelihood of seeing the observed values of event variable I for subgroup S' of the protected class under the null hypothesis Ho of no bias. The numerator represents the likelihood of seeing the observed values of for subgroup S of the protected class under the alternative hypothesis H1 (S), where the J values are systematically increased or decreased as compared to 7;-
For the alternative hypothesis, (S), to represent a deviation from Ho. H, contains a free parameter (g or p) that is determined by maximum likelihood estimation. enttaatnt bias (I, > Í.) or over-estimation bias (T, f.) can be detected using different constraints for & or p. These constraints, well as all

---

Published in Transactions on Machine Learning Research (01/2026)
hypotheses (Ho and H1) and corresponding score functions, F(S), for all scans are shown in Table 2
As shown in Table 2 when / is a probabilistic prediction (i.e., for separation scan for predictions), the hypotheses are in the form of a difference of log-odds between I and f sampled from a Gaussian distribution. Here, the free parameter lr in H1 represents a mean shift (n * 0) of the Gaussian distrribution. For all other scans, under Ho, each observed 4 is assumed corresponding expectation I,. Under H1, the free parameter q represents multiplicative increase or decrease (q * 1) of the odds of I as compared to I.
be drawn from a Bernoulli distribution centered at the
As shown in the rightmost column of Table 2 the various score functions all aggregate the deviations from Ho for each instance in a subgroup, and thus the log-likelihood ratio score F(S) scales linearly with subgroup size |S| for a given amount of deviation between observed and expected values. This dependence on [S] prevents the scan from assigning disproportionatel high log- log-likelihood scores to subgroups with few instances that have large, chance deviations from the null hypothesis over lavoring the true, larger subgroups of interest. For example, a subgroup with very few instances where there is a large deviation in, for example, the false positive rate between individuals in the protected class and those in the non-protected class, would not be favored over larger subgroup with less pronounced deviation in its false positive rate because of this scaling effect that controls for subgroup cardinality.
As in Zhang & Neill 2016). a complexity penalty term can be added to the expressions for F(S) shown in the rightmost column of Table 2 The complexity penalty equal to a prespecified scalar times the total number of attribute values included in subgroup S, summed across all covariates X ;.... Xmm Note that there is no penalty for given attribute if all attribute values are included, since this is equivalent to ignoring the attribute when defining subgroup S. The penalty term results in more interpretable subgroups by encouraging the scan either to ignore an attribute (i.e., all values of that attribute are included in the subgroup) or to choose a smaller number of attribute values to include in the subgroup.
We now consider how CBS can efficiently maximize F(S) over subgroups S of the protected class, returning the discovered subgroup S* and the corresponding score F(S*), where we aim identify
S* = arg max F(S).
The scan procedure for CBS takes as inputs a dataset D1 = XX) consisting of the observed event variable It, the estimated expectation of 4, under the mull hypothesis 1;, as calculated in Section 2.2) and the covariates X1, for each individual in the protected class (A, = 1), along with several parameters: the type of scan (Gaussian Bernoulli), the direction of bias to scan for (over- or under-estimation bias), complexity penalty, and number of iterations. It then searches for the highest-scoring subgroup (consisting of a non-empty subset of values VJ for each covariate X starting with random initialization on each iteration, and proceeding by coordinate ascent.
The coordinate ascent step identifies the highest-scoring non-empty subset of values Vj for a given covariate X conditioned on the current. subsets of values V-1 for all other attributes. As shown in McFowland III et al. (2023), each individual coordinate ascent step can provably find the optimal subset of attribute values while evaluating only X of the X subsets of values, where |X*| is the arity of covariate X This efficient subroutine follows from the fact that the score functions above satisfy the additive linear-time subset scanning property (Neill 2012 (Spcakman et al. 2016). The coordinate ascent step is repeated with different, randomly selected covariates until convergence to local optimum of the score function, and multiple random restarts enable the scan to approach the global optimum. McFowland III et al. (2023) provide sufficient conditions under which this routine will identify the global optimum in the large-sample limit; empirically, the approach converges to near-optimal subgroups while requiring only low-order polynomial time.
For an in-depth, self-contained description of the scan algorithm, including pseudocode (Algorithm 2 in Appendix |. an analysis of its computational complexity (Appendix (A.1.3), and how it exploits an additive property of the score functions to achieve linear-time efficiency for each scan step (Appendix A.1.1). see Appendix A.1 Derivations for F(S) are provided in Appendix A.2
7

---

Finally, we note that this procedure does not account for additional multiple testing issues which could result if we run several conditional bias scans (e.g., with different choices of the protected class or different group fairness definitions) but wish to bound the total type I crror rate across all scans. An additional Bonferroni Published in Transactions on Machine Learning Research (01/2026)
Algorithm 1 Semi-Synthetic Data, D
1: Require: (X1, = X
2: Randomly pick protected class attribute A
3: XA= = X
wy ~ (.(,0..22 where Vj = 1 to (
5: for i == 1 to nrows( x do
6: Li (=)=) wj xxX ) !.
b where eiruc ~
Otrue); (N(
L Ltue represents the true log-odds positive outcome for row i.
7: Yi ~ (ornoulli(((AA710g
b a is the sigmoid function,
is the true outcome for row b where epredict ~ (predctt))))
8: L ppredict_ L + predict
L I.redict represents the predicted log-odds for row i, gredict represents non-systematic errors (random noise) in the predictive model.
9:
P. = RRo((Lnrdee
b a is the sigmoid function,
Ps is the predicted probability of a positive outcome for row i.
10:
Papen = 1.1:=1(PP2 0.55
Min the predicted recommendation for row i based thresholding P..
11: end for
12: return Doorr = (X A, true predict Y, Posn)
return covariates, protected class attribute, true log-odds, predicted log-odds, outcomes, probabilities, and recom- mendations
2.4 Permutation Testing to Evaluate the Statistical Significance of S*
As described in Section 2.3 the scan step returns the detected subgroup S that maximizes the score function for a given scan type and parameterization, 5* = arg maxş F(S) as shown in Equation 2 The statistical significance (p-value) of the discovered subgroup S can be obtained by permatation testing, which correctly adjusts for the multiple testing resulting from searching over subgroups.
To do so, generate large number of simulated datasets under the null hypothesis Ho- For each null dataset, we generate new estimates for i, described in Section 2.2, and perform the same CBS scan, described in Section 2.3 (maximizing the log-likelihood ratio score over subgroups, exactly as performed for the original dataset). We then compare the maximum score F(S*) found for the true dataset to the distribution of maximum scores F(S*) found for the simulated datasets. To generate each simulated datasct under the null hypothesis, we copy the original dataset and randomly permute the values of A; (whether or conditionally
not each individual is a member of the protected class), thus testing the null hypothesis that independent of the event variable The detected subgroup is significant at level a if its score exceeds the 1 quantile of the F(S*) values for the simulated datasets. For a given dataset, the score threshold for significance at a fixed level a .05 will differ for different choices of the sensitive attribute and protected class. Thus, if CBS is used to audit a classifier for possible biases against multiple protected classes, a separate permutation test must be performed for each protected class value.
This permutation testing approach computationally expensive, multiplying the runtime by the total number of datascts (original and simulatod) on which the CBS scan is performed, but has the benefit of bounding the overall false positive rate (family-wise type I error rate) of the scan while maintaining high detection power. In comparison, the simpler approach of Bonferroni correction also bounds the overall false positive rate, and requires much less runtime, but suffers from dramatically reduced detection power.
correction (dividing the p-value threshold for statistical significance by the number of CBS runs, not the number of subgroups) can be applied in this case.
8

---

Published in Transactions on Machine Learning Research (01/2026)
3
Evaluation
Given the lack of gold standard approaches for evaluating subgroup bias auditing methods, we evaluate the CBS framework through semi-syuthetic simulations with the following steps:
Step (A):
Given a real-world dataset with attributes X, randomly select a protected class and generate semi-synthetic dataset, referred to Pairr where the predictions, recommen- dations, and outcomes are conditionally independent of A given X \ A, i.c., there are sufficiency or separation violations (defined in Section 2.1 pertaining to protected class A.
Step (B):
Take the unmodified semi-synthetic data, Poorr and inject signal consistent, with a separation or sufficiency violation or base rate shift into subgroup of protected class A, referred to as Stios; to generate datasel. Dmn-
Step (C):
Run CBS and benchmark methods to detect violations pertaining to protected class A for Dinj and measure the accuracy of the detected subgroups, referred to as S compared to the known (injected) biased subgroup, Sbias-
We will discuss cach step, Step (A) Step (C) in detail below, including the motivating questions that underpin our design choices.
Step (A) Generate a semi-symthetic dataset: Using COMPAS data date described in Section 4 we use Algorithm to generate a semi-synthetic datasct, D Specifically, we randomly select an attribute and value to define the protected class A and remove that altribute from X, as shown in Lines 23 In Line | we draw a weight. from a Gaussian distribution, wj ~ (O, 0.2), for each attribute-value of the covariates, excluding A. We use these weights, Wj; in Line [ to produce the true log-odds, Lirue, of a positive outcome (Y; 1) for each row i by a linear combination of the attribute values with these weights. Note, in Line G of Algorithm I we add dtrue ~ (O, trrue) to each row's true log-odds, T.frue, representing variation bet rows that arises from external factors (not included in the scan attributes). and is incorporated into the predictive model 3 Given the true log-odds I,rue of Y; = for each row i, in Line 7 we draw each outcome Y, from a Bernoulli distribution with the corresponding probability, expit (L)rre) which we refer to as the true probabilities. Next, in Lines [ we set each row's predicted probability P; == expit(L..ue + G)), where 6, ~ (0(,ddrrd represents non-systematii errors (random noise) in the predictive model. Finally, we threshold the probabilities to produce recommendations Pa 1(P, 2 0.5} for each row i in Line 10
Tpredict)
Algorithm 1 returns the randomly seleeted protected class attribute, A, the original covariates, excluding the protected class membership, XT. the synth true labels, Y, the predicted probabilities, P, and thresholded probabilities to create recommendations, Pr,ann Importantly, since is conditionally independent of the outcomes Y, predictions P and recommendations Pvin given the observed covariates X, by design, this dataset, Dyair, contains no signals indicating separation or sufficiency violations for a subgroup of protectod class A.
We use default values of Otrue 0.6 and Tpredict = 0.2 for Algorithm [ and examine sensitivity to these parameters in Appendix B.4 see Appendix B.2 for a discussion of the impact of Otrue on suffeiency-bbssed fairness definitions.
Step (B) Inject signal: We randomly select a subgroup, Shass of the protected class A of Doar into which we will inject biases or base rate shifts to create Dinj- We pick Spis by randomly choosing two attributes (Abia = 2) and then independently including or excluding each value of those attributes with probability piias 0.5. (This process is repeated until the resulting subgroup is non-cmpty.)
We designed the evaluation to address three key questions about the performance of the four CBS variants and benchmark methods:
We use the covariates from COMPAS maintain realistic covariate correlations, but do not use the predictions outcomes. Rudin et al. (020)) that COMPAS relies on up 137 variables collected from a questionnaire, and we expect that some of these additional variables are correlated with outcomes.
9

---

Published in Transactions on Machine Learning Research (01/2026)
(Q1): How well do they detect biases represented systematic differences, in the form of injected signals Msep and Msuf; between the predicted and true probabilities for the event variable in subgroup Sbios of the protected class A?
(Q2): How do they respond to an injected base rale shift, i.e., an equal shift by ō in the predicted and true probabilities for the event variable I for subgroup Spias of the protected class A?
(Q3): How do the answers to the first two questions, (Q1) and (22) vary based on the size of the biased subgroup Soine: which is controlled by the values chosen for Tibias and phias?
To address Q1) we inject a bias signal into subgroup Shins of the protected class A, keeping the corresponding subgroup of the non- poo-peeteteed class unchanged, in one of two ways, corresponding to separation and sufficieccy violations respectively:
bias Msep-
To produce Danj from Daiir for each row of Sbias: increase the predicted probability P by psep and then recompute the model's recommendation Pipin by thresholding P. at 0.5. When Meep > 0, this creates a signal which is consistent with separation violations in the positive direction for subgroup Spias of protected class A in Dinj- Detection results of CBS and benchmark methods for the injected signal of Usep to Spias are in the leftmost plot of Figure I
bias !
To produce Diny from fair, for each row of Stias, we reduce the true probability by Psuf and then redraw the outcome Y. When psuf > 0, this creates a signal which is consistent with sufficiency violations in the negative direction for subgroup Sheas of protected class A in Dins- Detection results of CBS and benchmark methods for the injected signal of Mauf to Shiae are in the rightmost plot of Figure 1
Both of these injected signals result in a bias where P and Poi overestimate the outcomes (Y) for the given subgroup Ssias of the protected class A in Dinj-
To address
we inject a base rate shift into subgroup Smiss of the protected class A, keeping the corresponding subgroup of the non-protected class unchanged:
Q22)
shift 6:
To produce Dinj from Djair, for each row of Spass we increase both the true probabilities and the predicted probabilities of Sbias by 6, then redraw outcomes Y; and recompute recommendations Pubin. For positive 6, this creates a higher base rate of a positive outcome for subgroup Smice of the protected class A in Dinj, as compared to the corresponding subgroup of the non-protccted class, while maintaining well-calibrated predictions. Detection results CBS and benchmark methods for the injected base rate shift of to Shias are in Figure 2
Importantly, the signals for Meeps Heaf, and 8 are created by uniform shift in the true and predicted probabilities, which corresponds to a non-umiform shift in the true and predicted log-odds. This is distinct from the modeling assumption made by CBS, which assumes (under the alternative hypothesis that bias is present) constant additive shift in the true or predicted log-odds. By injecting signal in this way, we ensure that our method is robust to non-additive shifts in log-odds. For simulation results that inject bias represented as additive shifts in log-odds, please see Appendix [.4 We observe high consistency between those additional results and the ones presented here.
To address (33) we run three experiments (Neep 0.50, Paaf 0.50, and d 0.25) while varying the size of Sbias in one of two ways:
Abiaa::
We vary the number of attributes, Ahias, that the attribute-values can be chosen from, between 1 and 4, when randomly selecting Sbias. Detection results of CBS and benchmark methods when varying Thias with different types of injected signal are in the top row of Figure 3
Phias: We vary the probability, Poice: that each value of the chosen attributes is included in Swas Detoction results of CBS and benchmark methods when varying Poias with different types of injected signal are in the bottom of Figure] 3
10

---

Published in Transactions on Machine Learning Research (01/2026)
Together, the different shifts of Mseps Houf, and along with variations in the parameters Riias and Poias used to select Shias result. in 1,344 distinct processes that each generate a distinct dataset Dinj from the base semi-synthetic dataset, Dj fairs generated in Step (A)
Step (C) Run CBS and benchmark methods and measure the accuracy of the detected subgroups: We compare the four variants of CBS to the benchmark methods, GerryFair (Kearns et al. 2018) and MultiAccuracy
Boost (Kim al. 2019a described in Section 5 For more information about the benchmark methods and
the modifications made to make them more comparable to CBS for these simulations, see Appendix [B.1
After injecting bias into or shifting the base rates of Shias for the protected class A of Daair to create Dinj. as described in Step (B) we run four variants of CBS (separation scan for recommendations in the positive direction; separation sean for predictions in the positive direction; sufficiency scan for recommendations in the negative direction; and sufficiency scan for predictions in the negative direction) and GerryFair and MultiAccuracy Boost for Dinj for the randomly selected protected class attribute, A. (Note: We use the same set tings for CBS as described in Section | with the exception of running all scans with all conditional variable values rather than as value--ondiiional scans.) Then, we measure the accuracy of detected subset for each CBS scan and each benchmark method, S*, compared to the ground truth subgroup, Stias, using the following metric:
accuracy(S*) SSpiss 0 S* Sbias St
(3)
Equation 8 is the Jaccard similarity between the injected and detected subscts, Stias and S respectively. This accuracy measure penalizes both falsely detected unbiased instances and undetected instances affected by bias, making it appropriate for applications where both types of error should be minimized.
Finally, to generate confidence intervals for these simulations, we repeat the full set of experiments 100 times and report the average accuracy of each method across runs, specifically:
For Step (A) we generate 100 distinct semi-synthetic datasets, Dfair-
For each of these 100 datasets of Drair: we perform the same 1,344 experiments described in [Step [B] each defined by unique configuration of injected bias. This produces 1,344 distinct biased datasets, Dinj. per ffair-
For each of the resulting Dinj. we evaluate the accuracy of the detected subgroup S discovered by the four CBS scans and two benchmark methods, described in Step (C)
This results in performance accuracy scores for all 1,344 experiments, repeated across 100 independently generated Daai datasets, which are then averaged by grouping distinct experiment configurations across all 100 versions of Dear to obtain confidence intervals.
3.1 Simulation Results
In Figure [ which addresses Q1) we observe that all four variants of CBS are able to detect the injected signal, Maep or Peaf, in subgroup Soias of the protected class A, with higher detection accuracy (defined in Equation 3) than Gerry Fair or MultiAccuracy Boost. Sufficiency scans had highest detection accuracy for shifts in true probabilities (Hsuf); as shown the rightmost plot of Figure 1 and separation scans had highest detection accuracy for shifts in predicted probabilities (Maep), as shown in the leftmost plot of Figure [ Scans for predictions generally outperformed scans for recommendations, due to the loss of information from binarization of the probabilistic predictions.
Interestingly, sufficiency sean for predictions (but not for recommendatiins) converged to perfect detection accuracy for psep. while separation scans did not converge to perfect detection accuracy for Hsuf. Sufficiency scan for predictions conditioned on a real-valued variable (P.) rather than a binary variable (P. or Yi), allowing for more flexible modeling of E P.X] and thus greater sensitivity to shifts in predicted probabilities.
11

---

Published in Transactions on Machine Learning Research (01/2026)
1.0
0.8
Accurad
0.6
Accuracy
2.66
0.4
0.2
0.2
0.0
0.0
Usep
Psuf
Separation
Suffiriaryy
Separation for Predictions
Suffiriency Predictions
Gemyfair features for Felse Pasitives
Recurmendeiions
for Recenmeendiators
Multicccrrcyy Heest
Figure 1: Average accuracy (with 95% CI) as a function the amount bias injected into subgroup Spiae of the protected class A, for four variants of CBS, GerryFair, and MultiAccuracy Boost. Left: increasing predicted probabilities by signal Msep- Right: decreasing true probabilities by signal pauf-
14
69
Separation Scar Recantmantt mandators
6.0
for Recom mendators
See aeceratio Soar
6.4
for Przdi clions
Multoccuracy Boost
62
Gerryaa
Fihha Pondivees
00
-3.00
-5.75
-5.0
-0.25
0.30
0.25
0.55
0.55
1.0
5
Figure 2: Average accuracy (with 95% CT) as a function of the base difference between protected and non-protected class for subgroup Ssias: for four variants of CBS, GerryFair, and MultiAccuracy Boost. Note that predictions are well calibrated, Msep Hauf O, for this set of results.
In Figure 2 which addresses Q22 shifting the base rate by for subgroup Sbias of the protected class A results in separation scans detecting a base rate shift when d O, while sufficiency scans and competing methods not sensitive to this shift. This finding aligns with previous rescarch proving that differences in base rates between two populations will result in higher false positive rate for the population with a higher base rate when using well-calibrated classifier (Chouldechova. 2017).
Interestingly, as shown in Figure 2 sufficiency scan for recommendations detects a base rate shift for & In this case, E|| Poin; X] is lower for instances of S in the protccted class than for instances with negative recommendations (Prin 0) in the non-protected class. Thus conditioning on the binary indicator Pibin for this simulation is not sufficient to capture this decrease in the true probabilities, while conditioning on the real-valued prediction P; allows sufficiency scan for predictions to extrapolate reasonably well to these
cases.
In Figure which addresses contained in the top row of Figure CBS has similar detection accuracy results to the simulations shown in Figures 1 and 2 with separation scans and sufficiency scan for predictions having higher bias detection accuracy when Msep 0.50, and sufficiency scans having higher bias detection accuracy when uluf 0.50,
Q3)
we observe that, when varying Tixas which can be observed in the plots
12

---

Published in Transactions on Machine Learning Research (01/2026)
Haep 0.50
When Rvn! = 0.50
when 6 0.25
1.0
3.0
1.0
20
15
30
35
4.0
15
28
25
35
1.0
Phiee
Piius
Poles
Phiac
Separation
Recommendasiors
rscommeendation
for Predictions
Sufficinccy Predictions
Multiacuracyy
False Posidves
Figure 3: Average accuracy (with 95% CI) for biases and base rate shifts injected into subgroup Stios of the protected class, for CBS, GerryFair, and MultiAccuracy Boost, as a function of varying parameters Thiias (top row) and Phiae (bottom row). Left: increasing predicted probabilities by Peep = 0.50. Center: decreasing true probabilitics by Hsuf 0.50. Right: base rate difference 0.25, for Meep Msaf 0.
compared to compeling methods across all settings of Wiias- Interestingly, as observed in the lower left plot of Figure when Msep = 0.50 and Phiae approaches (i.e., more individuals in the protected class are included in S GerryFair has improved bias detection accuracy, approaching that of CBS, but it performs poorly for values of Pbias closer O. This suggests that CBS is better detecting smaller, more subtle subgroups Sbias than the competing methods.
All fixed hyper-parameter choices for these simulations moderate values which align with non-edge cases. Additional robustness checks for varying ayper-parameter choices for these simulations are described in A Appendix B.4 For estimates of compute power needed for the simulations see Appendix B.5
4 Case Study of COMPAS
The COMPAS algorithm is used in various jurisdictions across the United States as decision support tool to predict individuals' risk of recidivism. It is commonly used by judges when deciding whether an arrested individual should be released prior to their trial (Angwin et al. 2016b). Following the initial investigation by ProPublica about fairness issues the risk scores generated by the COMPAS algorithm (Angwin al., 2016b), ProPublica's COMPAS dataset has been used as benchmark in the fairness literature. We follow many of the processing decisions made in the initial ProPublica analysis, including removing traffic offenses and defining recidivism
arrest within two years of the initial arrest for defendant (Larson et al. 2016 Larson & Roswell 2017). After preprocessing the initial data set, we have 6,172 defendants, their gender, race, age (Under 25 or 25+), charge degree (Misdemeanor or Felony), prior offenses (None, 1 to 5, or Over 5), predicted recidivism risk score 8;, where s, E (1,2,... 10), and whether they were re-arrested within two years of the initial arrest, where Y; = if individual was re-arrested and Y, otherwise. Given that COMPAS only provides risk scores and not predicted probabilities of reoffending, we define each defendant i's predicted probability of reoffending using maximum likelihood estimation:
13

---

Published in Transactions on Machine Learning Research (01/2026)
Σ أ- 1(Y, = 1 =
P.
(4)
Σj=1 1 8.)
Defendants with COMPAS risk scores of 5+ (s; 2 5) are considered "high risk" since the COMPAS documentation stipulates careful consideration by supervision agencies for these defendants Larson et al,, 2016). Therefore we define the COMPAS recommendations as Pr,in = 1(s, 2 5).
We chose the parameters for each of the four variants of CBS scans (values of the event and conditioning variables, / and C, respectively, and direction of bias) in order to search for systematic biases in COMPAS predictions and recommendations which disadvantage defendants for a given protected class A. For the separation scans, we detect positive deviations for the protected class attribute A in the E(P Y = X), listed as FPE in Table 3 and Pr(Poin = 1 Y y =0, X)) listed as FPR in Table 3 i.c., increase in prodicted risk and increase in FPR for non-reoffending defendants, respectively. For the sufficiency scans, we detect. negative deviation for the protected class A in the Pr(Y = 1 P.X), listed as CAL in Table 3 and Pr(Y =1 Poin X), listed as PPV in Table 3 i.c., decreased probability of reoffending conditional on predicted risk and on being flagged as high-risk, respectively. These choices were made to ensure our ability to verify our findings based on previous research on COMPAS, which commonly focus on similar fairness violations to those used in our case study. With that said, we strongly encourage auditing for predictive biases that affect rcoffending defendants and low-risk defendants as well, using CBS to audit an algorithmic risk assessment tool in practice. For example, auditing for the increased probability of being flagged high-risk for reoffending defendants could help to uncover subpopulations that are over-prosccuted in comparison to other populations of reoffending defendants. Therefore, expanding the fairness definitions used to audit pre-trial risk assessment tools for biases could have beneficial findings.
For all scans, we use all attributes except for the sensitive attribute when calculating the probability of being a member of the protected class A (for the propensity score weighting in Step 1 and Step 2f Section 2.2 and when generating the predicted values î under the null hypothesis of no bias, Ho. in Step 3 of Section 2.2 All scans were run for 500 iterations with a penalty equal to 1.
4.1 COMPAS Results
Table 3 contains the detected subgroups S", and their associated log-likelihood ratio scores F(S*), and corresponding indicators of statistical significance, found by each of the four variants of CBS, for various choices of the sensitive class attribute: Black, white, female, male, younger (under the age of 25) defendants, older (age 25+) defendants, and defendants with no priors. Please see Section 2.4 for the permutation test procodure used to determine statistical significance of CBS's detected subgroups. For the full set of rcsults for all CBS scans when treating each attribute value as the sensitive class attribute, please see Table 5 in Appendix C.1.2 For a discussion of the benchmark methodologies results for COMPAS, please reference Appendix C.1.3
Below are summaries of some of the statistically significant results that CBS found in COMPAS predictions and recommendations displayed in Table 3
Racial bias in COMPAS. Table 3 shows that the separation scans identify statistically significant biases negatively impacting a subgroup of Black defendants, while the sufficiency scans do not. These results support and complement the previous findings by ProPublica (Angwin et al., (016b) and follow-up analyses (Chouldechova] (217)), which concluded that COMPAS has large error rate disparities which negatively impact Black defendants (corresponding to large log- likeliiod ratio scores, F(S*), for separation scans), and that its predictions are well-calibrated for Black defendants (corresponding to small and statistically insiguificant log-likelihood ratio scores for sufficiency scans).
However, CBS's detected subgroup for the two separation scans adds a useful finding to this discussion: the large FPR disparity of COMPAS against Black defendants is even more significant in the intersectional subgroup of Black males found by CBS's separation scans. Non-rooffending Black male defendants have an FPR of 0.44, compared to non-reoffending non-Black male defendants' FPR of
14

---

Published in Transactions on Machine Learning Research (01/2020)
Ohserved Metric
Observed Metric for Complement
Lng-Likelihood
Scan
Detected
for Sensitive
Ratio
Metric
Type
Subgroup (S*)
Detected Subgroup
Detected Subgroup
(F(S*)
(Num. of Defendants)
(Num. of Defendants)
Sonsitive Attribute: Black Defendants compared to Non-Black Defendants
Scp, Pred.
Males
42.4
FPE
0.45 (1168)
0.35 (1433)
Scp. Roc.
Males
102.3
FPR
0.44 (1168)
0.19 (1433)
Suff. Pred.
Femalcs
2.21
CAL
0.37 (549)
0.34 (626)
Suff. Rec.
Age 25+ with 0-5 priors
0.37
PPV
0.50 (581)
0.52 (404)
Sensitive Attribute: White Defendants compared to Non-White Defendants
Sep. Pred.
0.0
FPE
Females under age 25
Sep. Rec.
2.01
FPR
0.71 (31)
0.56 (70)
with priors
Suff. Pred.
Under age 25
2.36
CAL
0.49 (347)
0.58 (1000)
Suff. Rec.
Females under age 25
0.41
PPV
0.39 (57)
0.47 (110)
Sensitive Attribute: Female Defendants compared to Male Defendants
Sep. Pred.
White
1.51
FPE
0.38 (312)
0.35 (969)
Sep. Rec.
White
12.5
FPR
0.29 (312)
0.20 (969)
Suff. Pred.
Under age 25
18.7
CAL
0.38 (246)
0.60 (1101)
Suff. Rec.
Under age 25
13.2
PPV
0.44 (167)
0.68 (699)
Sensitive Attribute: Male Defendants compared to Female Defendants
Sep. Pred.
Asian
0.63
FPE
0.30 (22)
0.22 (1)
Scp. Rec.
Asian and Hispanic
22.5
FPR
0.21 (286)
0.05 (57)
Suff. Pred.
Native Americans age 25
31.4
CAL
0.14 (7)
1.00 (2)
Suff. Rec.
Native Americans age 25+
14.1
PPV
0.25 (4)
1.00 (2)
Sensitive Attribute: Defendants under age compared Defendants age 25+
Sep. Pred.
All defendants under age 25
128.2
FPF
0.51 (593)
0.37 (2770)
Sep. Rec.
All defendants under age 25
159.3
FPR
0.53 (403)
0.25 (1583)
Suff. Pred.
0.0
CAL
Suff. Rec.
0.0
PPV
Sensitive Attribute: Defendants age 25+ compared to Defendants under age 25
Sep. Pred.
0.0
FPE
Sep. Rec.
Asians arrested on felony charges
0.74
FPR
0.20 (10)
0.00 (1)
Suff. Pred.
Males with 0-5 priors
92.7
CAL
0.35 (2867)
0.59 (1041)
Suff. Rec.
Males with 0-5 priors
53.0
PPV
0.52 (772)
0.67 (641)
Sensitive Attribute: Defendants with no priors compared Defendants with 14 priors
Sep. Pred.
FPE
Sep. Rec.
0.0
FPR
Suff. Pred.
All defendants with no priors
111.6
CAL
0.29 (2085)
0.54 (4087)
Suff. Rec.
All defendants with no priors
51.0
PPV
0.46 (553)
0.67 (2198)
Table 3: Select results from CBS scans run on COMPAS data. Sep. Pred. is short for scparation scan for predictions in the positive direction where the metric FPE stands for E[P|Y =0, X]. Sep. Rec. is short for separation scan for recommendations in the positive direction where the metric FPR, ie. false positive rate, is Pr(Ppmn = Y = X)X). Suff. Pred. short for sufficiency scan for predictions in the negative direction where the metric CAL, i.c. calibration, is Pr(Y scan for recommendations in the negative direction where the metric PPV, i.e. positive predictive value, is Pr(Y Pbin X). The third column contains the log- likeliood ratio, F(S*) defined in Equation I for the detected subgroup, S*, listed in the second column. Note, bold scores of F(S*) ) are statistically significant with p-value <.05 measured by permutation testing, as described in Section 2.4 For example, for the separation sean for recommendations with Black defendants as the sensitive attribute (sccond row), Black males had a false positive rate of 0.44 (n = 1168) compared to 0.19 for non-Black males (n = 1433). Please reference Table 5 in Appendix C.1.2 for the comprehensive set of results for COMPAS.
P,X). Suff. Rec. is short for sufficiency
0.19, whereas non-reoffending Black defendants have an FPR of 0.42, compared to non-reoffending non-Bl oon-Black defendants' FPR of 0.20.
15

---

Published in Transactions on Machine Learning Research (01/2026)
CBS does not find any high-scoring or statistically significant subgroups for white defendants, suggesting that COMPAS predictions do not disadvantage white defendants compared to non-white defendants or subgroups of white defendants compared to their non-white counterparts.
As shown in Table D of Appendix C.1.2 sufficiency scans find that Asian defendants arrested on misdemeanor charges have lower rate of reoffending compared to non-Asian defendants with comparable COMPAS risk scores, and that Hispanic defendants flagged as high-risk by COMPAS have a lower rate of reoffending compared to non-Hispanic defendants flagged as high-risk.
Gender bias in COMPAS. While male and female defendants have equal false positive rates overall, separation scan for recommendations detects a statistically significant gender bias: non-reoffending white female defendants have a higher false positive rale than non-reoflending white male defendants (0.29 vs 0.20). Separation scan for predictions detects the same gender bias but to a lesser degree: zon-reoffending white females have an expected risk of 0.38, compared to non-rcoffonding white males with an expected risk of 0.35. Sulficiency scans for both recommendations and predictions detect a statistically significant over-estimation bias for females under the age of 25. 44% of females under the age of 25 who are flagged "high-risk" by COMPAS reoffend, compared to 68% recidivism rate for males under the age of 25 who are flagged as "high-risk" by COMPAS. For both separation and sufficiency scans, thresholding the risk scores to create recommendations results in larger deviations between the subgroups of females and males found by the scans, thereby exacerbating the underlying biases present in the COMPAS risk scores that adversely impact white female defendants and younger female defendants, respectively.
Lastly, separation scan for recommendations finds that non-reoffending Asian and Hispanic male defendants have a statistically significant higher false positive rate of being flagged high-risk (0.21) in comparison to non-reoffending Asian and Hispanic female defendants (0.05) showing that the COMPAS risk scores have intersectional gender biases that adversely impact different subgroups of male and female defendants.
Age bias in COMPAS. Previous research argucs that COMPAS relics hcavily on the assumption that younger defendants are more likely to reoffend (Rudin et al. 2020) when computing risk scores. Younger defendants have a higher reoffending rate compared older defendants (0.56 vs. 0.46), and thus, well- calibrated predictions and recommendations would result. in younger defendants having higher FPR than older defendants. Our separation scans identify noo-reoofen defendants under age 25 as the subgroup with the largest. FPR disparity: these defendants have a 53% FPR and average predicted probability of reoffending of 51%, as compared to non-reoffending defendants of age 25+, who have a 25% FPR and average predicted probability of reoffending of 37%. On the other hand, our sufficiency scans identify a large subgroup bias within the protected class of defendants age 25+: older male defendants with to priors have lower rate of reoffending, as compared to younger malc defendants with 0 to priors, both for flagged high-risk defendants (sufficiency scan for recommendations) and for defendants with similar risk scores (sufficiency scan for predictions). This finding highlights the scenario described in Section that CBS is designed to detect: predictions are well-calibrated between older and younger defendants, in aggregate, but not for the detected subgroup of older males with 0 to 5 priors
Risk overestimation for defendants with no priors in COMPAS. Sufficiency scans find that defendants with no priors are disadvantaged by COMPAS because their rate of reoffending is statistically significantly lower than defendants with priors who are assigned similar risk scores by COMPAS. Specifically, the rate of reoffending for defendants with no priors who are flagged high-risk by COMPAS is 0.46 compared to 0.67 rate of offending for defendants with 1+ priors who are flagged as high-risk by COMPAS. Relatedly, sufficiency scan for predictions shows that defendants with no priors assigned similar risk scores to defendants with 1+ priors have a lower rate of reoffending, representing a miscalibration of risk scores for defendants with no priors. While not a demographic hiss per se, this finding highlights a systematic miscalibration in COMPAS's risk scores that penalizes defendants with no prior convictions.
For our German Credit Data case study, see Appendix
C.2
16

---

Published in Transactions on Machine Learning Research (01/2026)
5 Related Work
5.1 Auditing for Subgroup Biases
Bias Scan (Zhnn & Neil 2016 uses multidimensional subset scan to search exponentially many subgroups of data, with the goal of identifying the subgroup with the most significantly miscalibrated probabílistic predictions compared to the observed outcomes. Bias Scan lacks the functionality of traditional group fairness auditing techniques to define a protected class and to determine whether those individuals are impacted by biased predictions, and is thus limited to asking, "Which subgroup in a dataset has the most miscalibrated predictions?" In contrast, given a protected class A, CBS aims to identify biases impacting A or any subgroup of A. CBS searches for subgroups within the protected class A with the most significant. deviation in their predictions and observed outcomes as compared the predictions and observed outcomes for the corresponding subgroup of the non-protectod class (c.g., racial bias against Black females as compared to non-Black females). Since Bias Scan solely focuses on the deviation between the predictions and observed outcomes within subgroup, it would be unable to detect the subgroup with the most significant deviations between the protected and non-protected class unless this subgroup also displays significant miscalibration of predictions. Furthermore, CBS generalizes to separation- and sufficiency-base based group fairness metrics, and to probabilistic and binarized predictions. To enable this new functionality, CBS deviates from Bias Scan in substantial ways, including novel preprocessing and estimation techniques (see Section 2.2) and new hypotheses and score functions (see Section 2.3
GerryFair Kearns et al. 2018 and MultiAccuracy Boost Kim et al. 2019a). the two benchmark mot used in Section 3 use an auditor to iteratively detect subgroups while training correcting a classifier to guarantee subgroup fairness. GerryFair's auditor relies on linear regressions trained to predict differences between the predictions and the observed global error rate of dataset. MultiAccuracyy Boost iteratively forms subgroups by evaluating rows with predictions above and below threshold to determine which predictions to adjust. CBS's methodology for forming subgroups more complex because it does not assume linear relationship between covariates and the difference between the predictions and baseline error rate. For more details about these benchmark methods, reference Appendix [.1 Unlike CBS, these methods provide limited fairness definitions for auditing, and do not return interpretable subgroups that are defined by discrete autribute values of the covariates, but rather identify all rows that have a fairness violation on a given iteration. Since both methods incorporate the predictions in forming subgroups and enable auditing, they are comparable to CBS. In Section 3 we show that CBS has substantially higher bias detection accuracy than GerryFair and MultiAccuracy Boost.
There is other rescarch for subgroup bias auditing which is not directly comparable to CBS. For example, Chouldechova & G'Sell (2077) use recursive partitioning algorithu to find subgroups where the false positive rate disparity between individuals in the protected and non-protected class differs between two predictive models. In addition to this framework providing limited fairness metrics for auditing, this work is formulated to mcasure pairwise disparitics between two models' predictive performance, whercas CBS separately audits each predictive model's results, making this work ill-suited as a benchmark for CBS.
5.2 Learning Fair Classifiers
Several recent quantitative research papers (Bose & Hamilton 2019 Foulds et al. 2020 Subramanian et al, 2021) have proposed methods for learning fair classifiers (as opposed to auditing classifiers) with respect to intersectional and/or contextual biases. In the machine learning literature, Bose & Hamilton (2019) use filtered embeddings to train debiased graph embeddings; Foulds et al. (2020) propose new definitions of intersoctional bias and use regularization to train fair classifiers; and SSbramaniia et al. (2021) propose a classifier trained with bias-constraints and also extend a post-hoc debiasing method called iterative millspace projection (INLP) to address intersectional bias
17

---

Published in Transactions on Machine Learning Research (01/2026)
5.3
Subgroup Discovery
We present a novel subgroup discovery algorithm search for predictive bias. Subgroup discovery is a rich research domain. Herrera et al. (2011) provide comprehensive overview of subgroup discovery, covering various fundamental topics including sampling of search algorithms and quality measurements. K Klögenn
1999) provides a condensed and select overview of the topic of subgroup discovery. Lastly, Leman et al.
2008 present a framework for multi-target attribute subgroup discovery. While this work is significantly
different from CBS regarding framing, quality measurements, search algorithms, etc., it provides a useful overview of various considerations of subgroup discovery pertaining to model's outputs for a given data distribution.
5.4 Intersectionality
The concept of intersectionality has a rich history (Crenshaw 1991a b Collins 2008).
Given the importance of intersectional biases, we provide concise resources for the original conceptualizations of "interscctionality.. In the sociology literature, intersectionality theory (Crenshaw) 1991a b Collins 2008) describes how individuals' different social positions and identitics interact to influence their social experiences, actions, and outcomes. In particular, an individual at the intersection of several minoritized groups may be impacted by multiple historical and continuing systems of power and oppression such as structural racism, sexism, income and wealth disparities, etc.
Throughout this paper we are intentional in distinguishing between intersectional and contextual biases, as defined in Section [ is important to note that while an intersectional bias detected by CBS could be representative of the distinct type of discrimination described by intersoctionality theory in some instances, others they might not perfectly align. Therefore, an intersectional bias detected by CBS cannot interchangeably be used to refer to instances of sociological intersectionality without further analysis and contextualization.
6 Limitations
Our CBS framework is designed to audit a classifier's predictions and recommendations for biases with respect to subgroups of protected class, whereas competing methods provide mechanisms for both anditing and correcting classifiers. Combining auditors with correction and training presents the challenge of how to quantify the inherent trade-offs between performance and fairness when correcting for subgroup biases. Additionally, dosigning auditors that are linked to correction and training mothods reinforces the framing that the primary solution to subgroup biases is to correct the models. Given that fairness often context-specific, ideas of fairness could differ between stakeholders, and upstream biases exist in data sources used in many socio-technical settings, designing an optimally fair model not always feasible. We endorse exploring larger policy shifts (not limited to model correction) to address biases that auditing tools like CBS might unearth that are correlated with broader societal issues.
CBS is designed to detect biases in the form of group fairness violations represented as conditional independence relationships. While CBS is casily generalizable to other objectives that can be represented as group-level conditional independence relationships, is less generalizable other fairness definitions such as individual and counterfactual fairness [wookk et al. 2012: Kusner et al. 2017).
Our technique for estimating the expectations i under the null hypothesis of no bias has the limitation (which is commonly cited in the average treatment effects literature) of only being reliable when using well-specified models for estimating the propensity scores of protected class membership and for estimating f. Given the consistency of our COMPAS results in Section 4 with other researchers' findings about COMPAS, and the calibration analysis we present in Appendix C.1.1 the process of estimating seems to model the COMPAS data well. With that said, we encourage users of CBS check estimates of î and if necessary, employ procedures common in the econometric literature [Imbens 2004 Schuler & Rose 2017) or calibration methods within the computer science literature.
18

---

Published in Transactions on Machine Learning Research (01/2026)
There are various limitations to permutation testing, some of which are discussed in Berger 2000 For CBS specifically, if i is poorly estimated during permutation testing, this could result in higher type II errors where CBS is more likely to erroneously fail to reject the null hypothesis Ho of no bias.
Our simulations in Section g account for bias in the form of shifts in the predicted and true probabilities (separately and jointly) which produces predictive and aggregation biases for a prescribed set of covariate attribute values in the protected class. We provide additional simulations with signal and base rate shifts represented as shifts in the true and predicted log-odds in Appendix [B.4] In real-world scenarios, the generative process of bias might differ from the assumptions made in simulations. Future research could determine and (if necessary) improve CBS's robustness to different generative schemas of bias. While this is a limitation of our simulations, the results of CBS for COMPAS, which is a real-world application where the biases present are not a result of our generative process, are in line with other research about biases in COMPAS and the U.S. criminal justice system at large (Chouldechova & G'Sell 2017 Everett et al.) 2011 Rudin et al., 2020). Additionally, we provide a discussion of the benchmark methodologies' results for COMPAS in Appendix C.1.3 to highlight that CBS has various advantages an auditor in this real-world application (not restricted by the assumptions used in Section 3 compared to the benchmark methodologies'
auditor results.
In relation to our case study of COMPAS presented in Section + there have been various critiques of the COMPAS data that range from questioning the accuracy of the sensitive attributes (specifically race), noting missing features in the ProPublica dataset that the COMPAS creators claim are important for score calculations, and most importantly, a lack of evaluation of the biases that exist in the outcome variable of whether defendant is rearrested within two years of arrest (Fabris et al. 2022 Given that certain types of individuals are arrested at higher rate than others, the outcome variable of re-arrest most likely under- and wver-represents certain subpopulations of defendants. Given the various issues pertaining to COMPAS, our case study results in Section 4 may contain findings that do not perfectly align with the underlying distribution of crime or reoffense rates of the true population that COMPAS purports to capture.
Critically, while we use COMPAS as case study in Section 4 because of its familiarity and supporting research, we want to emphasize the importance of alternative framings of the evaluation of automated decision support tools in the criminal justice systems, such as examining the risks that the system poses to defendants
rather than the risk of the defendants to public safely (Mitchell el al. 2021
Meyer el al. 2022
Green 2020
7 Conclusion
In summary, CBS is a flexible framework that works with group-level fairness definitions that can be represented as conditional independence statements (separation and sufficiency) to detect intersectional and contextual biases within subgroups of the protected class while overcoming some of the issues that arisc when only considering fairness violations in aggregate for a single protected attribule value. CBS can discover intersectional and contextual biases in COMPAS scores and German Credit Data, and outperforms similar methods that audit classifiers for subgroup fairness.
Acknowledgments
This work was partially supported by the National Science Foundation's Program on Fairness in Artificial Intelligence in Collaboration with Amazon under Grant No. ITS-2040898.
References
Julia Angwin, Jeff Larson, and Lauren Kirchner. Machine bias, May 2016a. URL
Julia Angwin, Jels Larson, Surya Mattu, and Lauren Kirchner. Machine bias. In Ethics of Data and Analylics, pp. 254-264. Auerbach Publications, 2016b.
Vance W Berger. Pros and cons of permutation tests in clinical trials. Statistics in medicine, 19(10): -3288, 2000.
19

---

Published in Transactions on Machine Learning Research (01/2026)
A. J. Bose and W. Hamilton. Compositional fairness constraints for graph embeddings. In Proc. 36th International Conference on Machine Learning, 2019.
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 522):153-163, 2017.
Alexandra Chouldechova and Max G'Sell. Fairer and more accurate, but for whom? arXiv preprint WrXiu:707.00044 2017.
P. H. Collins. Black Feminist Thought: Knowledge, Consciousness,, and the Politics of Empowerment. Routledge, 2008.
K. Crenshaw. Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review, 4346):1241-1299, 1991a.
K. Crenshaw. Race, gender, and sexual harassment. Stonford Law Review, 1991b.
Datahub.io. Datahub.io's credit g, 2019. URL http://///aaaaatioo/a/aaeeeeeear 1i
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214 226, 2012.
Bethany G Everett, Richard G Rogers, Robert A Hummer, and Patrick M Krueger. Trends in educational attainment by race/et aace/etnnicit, nativity, and sex in the united states, 1989-2005, Ethnic and racial studies, 34(9))1543-1566 2011.
Alessandro Fabris, Stefano Messina, Gianmaria Silvello, and Gian Antonio Susto. Algorithmic fairness datascts: the story so far. Data Mining and Knowledge Discovery, 366))2074-2215, 2022.
J. R. Foulds, R. Islam, K. N. Keya, and S. Pan. An intersectional definition of fairness. In Proc. 36th IEEE International Conference on Data Engincering, pp. 1918-1921, 2020.
Ben Green, The false promise of risk assessments: epistemic reform and the limits fairness. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pp. 594-606, 2020.
Franciso Herrera, Cristóbal José Carmona, Pedro González, and María José Del Jesus. An overview on subgroup discovery: foundations and applications. Knowledge and Information Systems, 29:495-525 525,, 2011.
Hans Hofmann. Statlog (German Credit Data). UCI Machine Learning Repository, 1994. DOI:
https://doi.. htt///oi.oo/////////////// wtt.ooogg//......... 10.24432/ 24.22443//5N777 C5NC77.
Guido W Imbens. Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and statistics, 86(1):4-22 29, 2004.
Faisal Kamiran and Toon Calders. Classifying without discriminating. In 2009 2nd international conference on computer, control and communication, pp. 1 1-6. TEEE, 2009.
Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International Conference on Machine Learning, pp. 2564-2572. PMLR, 2018.
Michael P Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAT/ACM Conference on AI, Ethics, and Sociely, pp. 247 254. ACM, 2019a.
Michael P. Kim, Amirata Ghorbani, and James Zou. Multiaccuracyboost https://giihub...mmmiiiaaattt MultiiAccuacyyBost, 2019b.
Willi Klösgen. Subgroup patterns. Handbook of Data Mining and Knowledge Discovery, 1999.
20

---

Published in Transactions on Machine Learning Research (01/2026)
Martin Kulldorff. A spatial scan statistic. Communications in Statistics- Statistics-heery and methods, 26(6):1481-1496 1997.
Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. Advances in neural
information processing systems, 30, 2017.
Jeff Larson and Marjorie Roswell. Compas-annllsssscoompps analysis.iyybb at master analysis, Jun 2017. URL httpes//ggtaab...co//ppppplll//p analysis
propublica/compas.
Jeff Larson, Julia Angwin, Lauren Kirchner, and Surya Mattu. How we analyzed the compas recidivism algorithm, May 2016. URL
aeddivism-algorie
Dennis Leman, Ad Feelders, and Arno Knobbe. Exceptional model mining. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Antwerp, Belgium, September 15- 19, 2008, Proceedings, Part 19, pp. 16. Springer, 2008.
Edward McFowland III, Sriram Somanchi, and Daniel B Neill. Efficient discovery of heterogeneous quan- tile treatment effects in randomized experiments via anomalous pattern detection. ar preprint WiX::1803.091555 2023.
Mikaela Meyer, Aaron Horowitz, Erica Marshall, and Kristian Lum. Flipping the script criminal justice ice risk assessment: An actuarial model for assessing the risk the federal sentencing system poses to defendants. arXiv preprint arXt::2205.1550: 2022.
Shira Mitchell, Eric Potash, Solon Barocas, Alexander D'Amout and Kristian Lum. Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8:141-163, 2021.
Seth Neel, William Brown, Adel Boyarsky, Arnab Sarker, Aaron Hallac, Michael Kearns, Aaron Roth, and http://ittuub.ccoallggggoog
Z. Steven Wu. Gerryfair: Auditing and learning for subgroup fairness. penn/GerryFair 2019.
Daniel B Neill. Fast subset scan for spatial pattern detection. Journal of the Royal Statistical Society: Series B (Stalistical Methodology), 74(2):337-366 2012.
Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used manage the health of populations. Science, 366(6464)):44-4433, 2019.
Cynthia Rudin, Caroline Wang, and Beau Coker. The age of secrecy and unfairness in recidivism prediction. Harvard Data Science Review, 2(1):1, 2020.
Anne Sisson Runyan. What is intersectionality and why is it important? Academe, 104(6):10 14, 2018.
Megan S Schuler and Sherri Rose. Targeted maximum likelihood estimation for causal inference in observational studies. American journal of epidemiology, 185(1): 2017.
Skyler Speakman, Sriram Somanchi, Edward McFowland III, and Daniel Neill. Penalized fast subset scanning. Journal of Computational and Graphical Statistics, 25(2):382-4044 2016.
S. Subramanian, X. Han, T. Baldwin, T. Cohn, and L. Frermann. Evaluating debiasing techniques for intersectional biases. In Proc. Conf. on Empirical Methods in Natural Language Processing, pp. 2492 24922 2021.
Zhe Zhang and Daniel B Neill. Identifying significant predictive bias in classifiers. arxiv preprint arviv 161 108292, 2016.
21

---

Published in Transactions on Machine Learning Research (01/2026)
A Methods Appendices for Section 2
A.1 Fast Subset Scanning for Conditional Bias Scan
In this section, we explain the fast subset scanning (FSS) algorithm used by CBS, with the goal of identifying the subgroup, S of the protected class, A, with the most biased predictions or recommendations (Neill 2012). Specifically, we will: (1) introduce FSS using a simplified example, for illustrative purposes, to highlight the computational difficulties inherent in subset scanning: (2) formally define the additive property of the score functions for CBS that enables computationally feasible subset scanning in Appendix A.1.1 and (3)
provide an implementation of FSS for CBS in the form of pseudocode in Appendix [A.1.2
Let assume a dataset consisting of only individuals in the protected class (A= 1), denoted as Q {(x", [) [(X] 1, f,));z that contains values of the event variable 7,, estimates i, of the expected value of the event variable under the null hypothesis of no bias, and a single categorical covariate attribute X for each individual i in protected class A. For concreteness, we perform a sufficiency scan for predictions, therefore, the event variable T; is the observed binary outcome Y; for individual i, and the corresponding í; is the estimated Pr(Y;- = 1 Pt, Xi) under the null hypothesis, Ho. that Y 1 A (P.X). S refers to a subgroup of Q which in our simple example is a non-empty subset of values for attribute X'. Since our event variable is binary, we use the Bernoulli likelihood function, shown in the bottom half of Table 2 to represent the hypotheses in the score function, F(S), used to determine the level of anomalousness of a subgroup S of Q.
In the worst-case scenario, X' would be a categorical variable where each row of Q holds distinct value, meaning that there are n unique values for X If we were to score all of the possible S C Q using F(S), this method would have a runtime of O(2"), which would be computationally infcasible. To overcome this compulational barrier, FSS relies on its score functions, F(S), being a part of an efficiently optimizable class of functions with the goal of finding the most anomalous subset S = arg maxsgq F(S) without the need to evaluate all of the subsets of Q. The property that determines if function
part of this class that enables fast subset scanning is called Additive Linear-Time Subset Scamning (ALTSS) (Speakman
al., 2016) and is formally defined in Section A.1.1 Informally, if F(S) can be represented as an additive set. function over all instances i E S when conditioning on the free parameter (q for the Bernoulli distribution or μ for the Gaussian distribution, where q and are defined in Table 2), it satisfies the ALTSS property (Speakman
et al. 2016
To explore how FSS exploits the ALTSS property for computationally efficient subset scanning, assume that the categorical covariate X for each individual can only be equal to one of four values, X! E fa. c. FSS constructs a subset for each distinct attribute value of X such that S. {i E Q X! = a), S. (i (i E Q: X! = b), S. (i E Q X = c), Sa {i E Q : X!! d). Since function for the Bernoulli distribution for F(S), F(S) is concave function of the free parameter q, and for illustrative purposes, we will assume that max, F(S) > 0 for all subsets Sg, Sp, Se and Sg. Therefore, for each subset Sa, S,, S. and Sa, F(S) is a function over the domain of g. where as q increases from -x, F(S) eventually equals and then the global maximum for F(S) for that given subset, and then starts decreasing until again reaches a point where F(S)= = 0, and then remains negative as g approaches co. FSS identifies three g values for each subset, S E {Sa. So, Se, Sa}:
are using the likelihood
1. The first value of where F(S) = 0 as increases from -oo to oc, which we will refer to as quin-
2. The second value of q where F(S) = 0 as increases from oo to oo, which we will refer to as Imar-
The value of q for arg max, F(S), which we will refer to as QULE-
Each distinct qmin and qmaz value for subsets (S., St, Se, Sa) is a value of q where the score function F(S) becomes negative or positive for al least one of these four subsets. By sorting all of the distinct ymin and Imaa values across all the subsets (Sa, S₁, Se, Sa) in ascending order, we construct a list of q values, {q(1)..... Amm) } where each pair of adjacent values, 9() and 1(++),, represents an interval of the q domain, (4()), ((x+1))) ) for which each subset S E (Sa, Sc, Sa} has either F(S)= for the entire interval or F(S) < 0 for the entire interval. For each interval, (q0s) (k41)), we perform the following:
22

---

Published in Transactions on Machine Learning Research (01/2026)
1. Find the midpoint of the interval (average of (()) and Q(++))), which we refer to q qitdd
2. Create a new subset 5y S by aggregating all subsets S E {Sa, So, Sex Su} where the subset's 's amin < giid and the subset's qmar > qi i.e., F(S) > O when q= qiid and therefore for the entire interval (g(K), ((++1)). Since the score function is additive, conditioned on q, we know that a subset S will make a positive contribution to the score F(S if and only if F(S) > 0 for that value of q. Thus, we know that the highest scoring subset Sk Siugregate for that interval [g(k): 9(++)) contains all and only those subsets S with F(S) > at q = 9
aggregate = arg max, F(Spesregal)) ) and the corresponding 3. Find the maximum likelihood estimate of 9, MMLE score
The aggregate subset, Sk Siggregute with the highest score for F(S) using its associated aggregate is the most MMLE anomalous subset when considering subsets formed by combinations of different. Altribule-values of X1.
For our simplified example, there are at most 8 distinct Qmin or Qmaz values from the four subsets (Sa, So, Se, Sa), and thus at most 7 distinct intervals (q(a). Q(++))) that must be considered. For given interval, we need to evaluate only a single subset S and thus, only 7 of the 15 non-empty subsets of (Sa, So Se, Sa}. More generally, if is the arity (number of attribute values) of categorical attribute X1, at most 2n of the 2 non-empty subsets of attribute values must be evaluated to identify the highest-scoring subgroup.
The scenario where the covariates consist of single categorical attribute is simplified example, where only a single iteration of FSS is needed to find the optimal subset, S" of Q. When there are two more attributes for the covariates, multiple iterations of FSS must be performed to approach the optimal subset. On each iteration the following is performed:
We define an initial subset, Stemp where:
(a) If it is the first iteration, all of the attribute values for each attribute are included in Stemp- (b) Otherwise, a random subset of attribute values for each attribute are chosen to be included Stepp
For each attribute Xt, random order, we construct subsets by partitioning Stomp by the distinct attribute values of X form intervals across the domain of q for F(S), and then assemble and score the subsets for each interval (as described above). Stemp is updated as higher scoring subsets using F(S) are found. Therefore, when attribute is evaluated, Stemp contains only rows of Q that fit the found criteria (in the form of attribute values) from previously evaluated attributes, excluding the attribute currently under consideration. This iterative ascent procedure is repeated until convergence.
Multiple iterations are performed with the final detected subset being the subset with the highest score using F(S) found across all iterations, S For the psoudocode of FSS for CBS, please see Algorithm 2 The returned results from FSS are: (1) the detected subset, S*, in the form of attributo-values that form the criteria for the subgroup in the protected class with the highest score F(S); (2) the parameter q or p that maximizes F(S*); and (3) and the score F(S*) given the parameter q μ.
A.1.1 Formal Definition of Additive Linear-Time Subset Scanning Property (ALTSS)
Below we provide a formal definition of the Additive Linear-Time Subset Scanning Property. The score functions, F(S), used to evaluate subgroups are a log-likelihood ratio formed from two different hypotheses whose likelihoods are modeled by likelihood functions for either the Bernoulli distribution or Gaussian distribution, both of which satisfy the Additive Linear-time Subset Scanning Property (Speakman et al.
2016 Zhang& & Neill 2016).
Definition A.1 (Additive Linear-time Subset Scanning Property). A function, F S + R>o, that produces a score for a subsct S C D, where D is a set of data and 0 = arg max, F(S| satisfics the Additive Linear-time Subset Scanning Property if F(S 6) = Ex,es F(s; ()) where s; is a subset of and Vs,, S, where 8; # 8j, we have 8, (88- = 0.
23

---

Published in Transactions on Machine Learning Research (01/2026)
We refer to the score functions, F(S), contained in the rightmost column of Table 2 F(S 1t) for the score functions that use the Gaussian likelihood function to form hypotheses and F(S g) for the score functions that use the Bernoulli likelihood function to form hypotheses. F(S g) contains a summation, (l log q - log(gī, - I + 1)), that is the sum of individuua-speecffi values derived from I,, f, and q- Given that each individual is distinct, F(S] g) = Eies F(s, 9q); where 8, is the subset of S that contains only individual i, satisfies the ALTSS property. Similarly, F(S (u) contains a summation, Eies 4,, that is the sum of undivduul--peccfti values A; derived from 4, 4,, and μ. Therefore F(S| p)) = En.es F(s; p))
where 82 is the subset of S that contains only individual i, satisfies the ALTSS property.
A.1.2 Pseudocode of Fast Subset Scan Algorithm for Conditional Bias Scan
Algorithm 2 is the pseudocode for the Fast Subset Scan (FSS) algorithm used in the CBS framework Neill 2012). The FSS algorithm aims to find the subgroup, S with the most anomalous signal for the observed event. variable, I (i.e., the highest score F(S*)) in a dataset, D. For CBS, this signal is in the form of a bias (according to one of the fairness definitions in Table [ against members of subgroup S of the protected class (A = 1). The dataset passed to the FSS algorithm by CBS contains only individuals i in the protected class, and FSS compares their values of the event variable 1, to the estimated expectalions f, under the null hypothesis of no bias. The method for estimating 7, is performed prior to running the FSS scan step and is described in detail in Section 2.2
At the initialization of FSS, placeholder variables are created that will hold the highest- (most anomalous) subset (S*), and S corresponding information (6", Score"), found across all iterations (Lines D 3). At the beginning of an iteration, random subset is picked (set of attribute-values) as the starting subset, Stemp with the exception of the first iteration where the starting subset includes all attribute values, shown in the if-else statement starting on Line 5 For each iteration of this algorithm, we repeatedly choose a random attribute to scan (i.c., we scan over subsets of its attribute values) as shown in Lines convergence (i.e., when all altributes have been scanned without increasing the score F(Stemp))
14 15 until
For each attribute Xtemp to be scanned, for each of its attribute values, Xtemp, we score the subset Sxcem, containing only the records with the given value of that attribute (Xtemp (eemp,); and matching subsct Stemp on all other attributes in X. We write this as Sx,emp +- Srelaged n (i- E D Xtemp Xtemp:), in Line 18 where Syskeped is the relaxation of subset Stemp to include all values for attribute Xtemp- Along with scoring this vitibuutevaluee subset Sx Sxtemey we find the two values of 0 where F(Sx = Omin: and mmaz. and the f that maximizes F(S P(SX..mp)) ), OMLE, with the exception of attribute-value subsets Sxamp, that are not positive for any value of A. This is shown in the for-loop in Lines 16
Line 21 states that Omin; and Omaz; must be adjusted according to the direction of the scan lo enforce that the found parameters Omin, and 6 man,, adhere to the restrictions set by the direction of the scan. The constraints necessary for the scans to detect biases in the positive and negative directions are fully specified in Table 2 For positive scans that have score functions that utilize the Gaussian likelihood function to form hepotheses, Ominn = max (0, Ominn.)) and for negative scans that utilize the Gaussian likelihood function, 0maz, min(0, 0man;). For positive scans that have score functions that utilize the Bernoulli likelihood function to form hypotheses, minn. max(1, minn and for negative scans that utilize the Bernoulli likelihood function, Omoz. min(1, mmaz.). Attribute-value subsets SXtem,, should not be considered when choosing subsets for Suggregate for positive scans where 0men, < 00 or Omaz; 1 for scans using the Gaussian likelihood function or Bernoulli likelihood function in F(S), respectively. Conversely, attribute-value subsets SXLamp should not be considered when choosing subsets for Saseregate for negative scans where 6mnnn > or 6 for scans using the Gaussian likelihood function or Bernoulli likelihood function in F(S), respectively.
We sort the 0min, and mmaz, values found across all the attribute values of the attribute we are scanning in ascending order in Line 23 These form a list of intervals over the domain of 0. For each interval, we calculate a midpoint of that interval, and aggregate all the altribule-value subsets that have a positive score, F(S), when 0 equals the midpoint of that interval in Lines 33 If the aggregated subset of attribute values with the maximum score across all the intervals is greater than the score of Stemp: we update Stemp and all of its accompanying information (ftemp: Scoretemp) to equal the maximum-scoring subset of aggregated Aatribute-values across all the intervals and its accompanying information, as shown in Lines 43 48 Therefore,
24

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm Fast Subset Scan (FSS) for Conditional Bias Scan
Require: Witers > O, (Xi, li, 12) Vi D where A; = direction {positive, negative)
S
Score* -00
0" +- -00
4: for f-1 1.... 1.... Niters do
5: if = 1 then
6:
Stemp + all attributo-values for cach attribute
7: else
Stemp f- random nonempty subset of valuues for each attribute X
end if
10:
Otemp f- arg max, (F(Seemp 8))
11:
Scoretemp <- F(S temmp)
12:
Battributes f number of attributes in X
13:
Dssaaned 4-
D mark all attributes as un- scanned
14:
while necannedd Rattributes do
15:
tempp +- randomly selected attribute that is marked as unscanned
16:
for Xtemp, E X Xtemp do
for all attribute-values in Xtemp
17:
Sxcemps +-
Stepp
Srelazed n {i D: Xtemp = Xtemp;
b see Appendix A.1.2 for defi- nition of Stelazed
temp
18:
Oming Oman: f (F Xcmpp (0) 00)
exception noted in Ap- pendix A.1.2
19:
OMLE = arg max (F (F(X 0))
20:
Score; t F(Stemp, (MLLE)
21:
Adjust Oming and Omaz, depending on the direction of scan
explained in text of Ap- pendix A.1.2
22:
end for intervals f- {fmin Omnn VXtempe E Xtemp) ascending order
23:
all values of where F(S) VXtempp Xtemps indexed by 8(k) below
24:
Scoreinterval x- -00
25:
Sinterval t 0
26:
Ointerval t- -00
not to be confused with Interrvlls
27:
for 1.. 1
do
S Snegregate 0 0(x)+0(++1)
28:
gmid
29:
30:
for Xcemp, E Xtemp do if Score; > O and 6min. gmid and Oman. > @mid then Saggregate +- S Sk
31:
Saggregate U USS SXeemp:
32:
33:
end if end for gaggregate arg max (S Saggregaae 10)) k
34:
35:
36:
Scoreneerege aggregate + (SSggrrggtee &
gaggregate)
37:
if Scor aggrogate Scoreinterval then
Scoreintervol < Score aggregattee
38:
Sinterval 4- 5 gaggregate
39:
40:
Oitternat f- 4
aggregate
41:
end if
42:
end for
25

---

Published in Transactions on Machine Learning Research (01/2026)
43:
if Scoreremp Scor Cinterval then
44:
Scoretemp + Scorenttereal
45:
Stemp Sinterval
46:
Oemmp - interval
47:
Uscanned f-
mark all attributes as un- scanned
48:
end if
49:
T Recanned < usconned +1
D mark attribute Xtemp as scanned
50:
end while
51:
if Score* Scoretemp then
52:
Score" t- Scoretemp
53:
5 t- Stemp
54:
fr t teemp
55:
end if
56: end for
57: return 5", Score", ,
Stemp is continuously updated as higher scoring subsets are found as we scan over all the attributes and their attribute values.
At the end of an iteration, if the found subset., Stemp, has higher score than the global maximum scoring subset S*, then S and its accompanying information (0", Score") are replaced with Stemp and Stemp 's accompanying information, as shown Lines 55 Once all the iterations have completed, the subset with the maximum score found across all iterations is returned, S", with its score, Score = F(S* 0*), and accompanying 8" paramoter Line 57
McFowland III al. (0023) show that similar multidimensional scan algorithm, used for heterogeneous treatment effect estimation, will converge with high probability to a near-optimal subset when run with multiple iterations.
A.1.3 Computational Complexity of the Fast Subset Scan Algorithm
The computational complexity of the fast subset scan can be expressed as the product of the number of iterations (Niters), the number of covariate attributes (m), the nunber of passes through the attributes required for convergence (Z), and the time needed to process a single attribute (Algorithm Lines 16 49 Z generally small (no more than 10-20). Optimizing over all non-empty subsets of values for given attribute, conditional on the current subsets of values for all other attributes, requires aggregation of data for the subsets Sx Sxcamp, (Line 17/. computing and sorting the Oming and 0maz, values (Lines 18 23 and scoring Aggregation requires pass through the data and is thus O(n) operation, where is the number of data elements. The remaining steps require O([V] log [V]) time, where [V] is the arity (number of altribute values) for the current attribute Xtemp- Thus the total complexity can be written as (ZZt + V log [V)), where lo [
the aggregated subsets (Lines) 27 42
is averaged over all attributes.
A.2 Derivation of Score Functions in Table 2
In this section, we provide derivations of the Gaussian and Bernoulli log- likelihood ratio score functions F(S) shown in Table 2
A.2.1 Derivation of Gaussian Score Function
For the Gaussian score function, we compare the mull hypothesis Ho: A N(0, to the alternative hypothesis H, (S): A, ~ N(u, o), Vi E S, and A, N(O, o), & S. Here, P the maxinum likelihood value of the free parameter (additive shift in log-odds) for the alternative hypothesis HS)), conditional on the constraints for z listed in Table 2 Specifically, for over-estimation bias, μ is constrained to be less than 0,
26

---

Published in Transactions on Machine Learning Research (01/2026)
while for under-estimation bias, j is constrained to be greater than 0. We can then write F(S) as follows, where D represents a data set and S represents subgroup, S S D, as defined in Section 2
Pr(D HS)S)
F(S) log
Pr(D Ho)
max, Pr(D H(()):
log
Pr(D Ho)
max, Iliss oY2 exp
(AI-)(²
20
Tlgs
0v2n
exp
20²
A
= log
IlIED ov2n
exp
20
(A,-P)
max 4 TLES 0V26 exp
20
log
Ilis oV exp
A2
20²
A? - (A; - -113
log max
exp
20²
iES
A (A + 11² 2/11i)
max
202
2/ (Σ iEs A.) - 111122
max
20
The maximum likelihood value of a can be obtained by setting the first derivative of F(S) with respect p to zero and enforcing the constraints p listed above:
EiEs As < and we are searching for over-estimation bias;
iss
if Eies Ai > 0 and we are scarching for under-ostimation bias;
otherwise.
Finally, plugging in this value of into the equation for F(S), we obtain:
Eins ).)
il Eies Ai O and we are searching for over-estimation bias;
20 1ST
F(S)
Σ
iic 2.)²
if Eies A > 0 and we are searching for under-estinnation bias;
20 20
otherwise.
A.2.2 Derivation of Bernoulli Score Function
4 For the Bernoulli score function, compare the null hypothesis Ho: odds(L.) 1.. to the alternative
hypothesis HS)::: odds([,) = q , Vi S, and odds(I,) = - Vi & S. Here, q is the maximum likelihood value of the free parameter (multiplicative shift in odds) for the alternative hypothesis Hi)),, conditional the constraints for q listed in Table 2 (As discussed in Section 2.2 is the estimate of Pr(l, 1 X;, C.) under the condition that 1 A (C,X). As defined in Section 2.1 the variables represented by I and C change depending on the sean type.) Equivalently, we can write the null hypothesis as Ho: 1; ~ Bernoulli and the alternative hypothesis as H, (S): 1 ~ Bernoulli -T. - Vi E S, and 1 Bernoulli(i., Vi & S. For over-estimation bias, g is constrained to be less than 1, and for under-estimation bias, g is constrained to
27

---

Published in Transactions on Machine Learning Research (01/2026)
be greater than 1. We can then write:
Pr(DI Hi)S)
F(S) = log
Pr(D Ho)
maxa Pr(D Hi(S):))
=
log
Pr(D Hu)
max, Ihes
(1t af
"
1-h 1-1,
T
(I:)
(1-
1-1
g4, (f,-1
(44-1141)
=
log
ILLD (I)) (1 (1-
1-1,
max, Ilies
at.
1: 1-h.
1-4
g1.
1,1-1
= log
(1)) 1,
Ilis
1
1-
1-4
log max
II
ạ
1
qi. - I, +
91. 4 +1
iES
max Σ
(1 log(g) - log(gl; - -1++11 1)).
iE
As in the Gaussian case, the maximum likelihood value of q can be obtained by setting the first derivative of F(S) with respect to g to zero, obtaining the equation:
q
Σ4=Σ
gl. 1,+1
Unlike the Gaussian case, there is no closed form solution. However, the right-hand side of the equation an increasing function of 9, enabling to obtain the value of q that satisfies the equation by binary search. Let q" denote the (unconstrained) value of found in this way. Then, to enforce the constraints above, we observe that x² > if and only if Eies Ti > Eies ls and similarly that g² < 1 if and only if Eies Ii < <Eiss' i. Thus we can write:
- log(g (oog())
1)) if DiEsh< Eies 4
and we are searching for over-estimation bias;
F(S)-
Lies(4 - +, 1)) if Eies Eies f
and we are scarching for under-estimation bias; otherwise.
0
A.3
Conditional Bias Scan Framework Parameters
Table contains all the parameter settings needed to run Conditional Bias Scan.
B
Evaluation Appendices for Section 3
B.1
Adaptations of the Benchmark Methods for Evaluation
Both GerryFair and MultiAccuracy Boost provide implementations of their methods GitHub (Neel et al, Kim et al. 2019b). Our goal was to use their provided codc with minimal changes as benchmarks in Sections 3 and However, GerryFair and MultiAccuracy Boost do not provide the functionality to indicate whether to audit for bias in the positive direction (under-estimation bias) or bias in the negative direction (over-ostimation bias). For example, when auditing for the subgroup, S* of a protected class, A, with the most anomalous deviation in the false positive rate compared to the analogous subgroup in the non-protectod class, FerryFair and MultiAccuracy, Boost, in their native form, do not allow specify if the deviation
2019
28

---

Published in Transactions on Machine Learning Research (01/2026)
Table Table with all parameters needed run Conditional Bias Scan.
Parameter Attribute Valucs
Soctions for Reference
Parameter
Purpose
Binary attribute which defines whether each individual is member of the protected class. We wish to identify any biases that are present in the classifier's predictions recommendations that impact the protected class.
Membership in Protected Class Indicator Variable (A)
Separation for recommendations: Separation scan for predictions; Sufficiency sean for recommendations; Sufficiency scan for predictions
Scan Type
The subcategory of the scan type.
2.1
The event of interest for the scan. The abstracted event variable must be defined either the outcome, prediction, recommendation variable.
Event Variable (7)
Y; P; Poin
2.1
The conditional variable for the The abstracted conditional variable must defined either outcome, prediction, or recommnnndation variable. For calue-conditional scans, this the value on which we are conditioning the conditional variable (C). Defining a field value results in scans that detect different forms of fairness violations.
Conditional Variable (C)
Y; P; Phin
2.1
Field value (z) of Conditional Variable
None; 0; 1
2 2.2
23
A.1
(C' (C)=a)
List of Attributes for Forming Subgroups (X)
List of attributes to scan over to form subgroups.
A.1
detecting under-estimatiin hias (positive direction) or over-estimation hias (negative direction).
Specifies whether
Direction of Blas
Positive; Negative
2.1 2.3.
A.1
List attributes used for conditioning when producing f. In this paper use the same attributes to form subgroups and produce f. This does not necessarily have be case CBS.
List of Attributes
for Estimating / (X)
2.2
applications
The non-negative integer-valued scalar penalty that is subtracted from the score function for each subgroup depending on the subgroup's ss total number of included values for each covariate X X not including covariates for which all values included. Specifies the number of iterations the fast subset scanning algorithm.
Subgroup Complexity Penalty
2.3
(default value:
Scan Iterations
2.3
A.1
(default value: 500)
The table above lists the parameter, purpose of the parameter, possible values of the parameter, when applicable, and the sections in our paper where this parameter is described in further detail.
should be an incrcase (or decrease) of the false positive rate for subgroup S This lack of functionality makes the results from CBS substantially different than those returned by GerryFair and MultiAccuracy Boost.
For GerryFair's auditor, given the type of error rate to audit (false negative rate or false positive rate), they train four lincar regressions using the features (X) as dependent variables with the following four sets of labels:
1. Two linear regressions with the zero set labels.
29

---

Published in Transactions on Machine Learning Research (01/2026)
2. One linear regression with the labels set to measurement that assigns positive costs for predictions that deviate in the posilive direction (when the predictions are greater than the observed global error rate), and negative costs otherwise.
3. One linear regression with the labels set to a measurement that assigns positive costs for predictions that deviate in the negative direction (when the predictions are less than the observed global error rate), and negative costs otherwise.
They use the predictions from the linear regressions to flag a subset of data where the predictions from the linear regression trained with the zero set labels are greater than the values predicted by the linear regression trained with the costs representing deviations of the predictions from the observed baseline error rate metric of interest labels. Two linear regressions are used to estimate deviations of the predictions from the observed error rate baseline, and therefore they form two subgroups: (1) subgroup with rows that are estimated to have predictions that are greater than the baseline for the metric of interest; and (2) subgroup with rows that are estimated to have predictions that are less than the baseline for the metric of interest. The original GerryFair implementation uses a custom hcuristic to decide which subgroup has more significant biases and returns that subgroup accordingly. The subgroup with the rows that are estimated to have predictions that, are greater than the metric of interest more closely aligns with the concept of auditing for bias in the positive direction auditing for under-cstimation bias. Since CBS provides the functionality of auditing for biases of specific direction, we add an option to GerryFair that allows the auditor to parameterize which direction of bias they are interested in, specifically by overriding GerryFair's automated process described above for deciding which subgroup from the two subgroups that represent over- and under-estimation bias has the largest deviation from the bascline, making GerryFair's 's results more comparable to CBS.
For each simulation, we ran GerryFair two times, once to detect bias the form of systematic increases the false positive rate, and once to detect bias in the form of systematic increases in the false negative rate. In each case, we allow GerryFair to use all covariates (X) to make the predictions used to form subgroups, including the protected class category. This resulted in two result sets for GerryFair for each simulation. We present the result set in Section 3 that had the highest overall accuracy for most of the simulations, which is the GerryFair setup for detecting increased false positive rate for subgroups of the protected class A. GerryFair returns a subgroup that could contain individuals in both the protected class and the non-protected class. To have the accuracy measurements for GerryFair and CBS be comparable, we filter the subgroup returned by GerryFair to only include individuals in the protected class before calculating the subgroup's accuracy.
MultiAccuracy Boost is iterative algorithm where, on each iteration, it audits for subgroup with inaccuracies and then corrects that subgroup's predicted log-odds. More specifically, for each iteration:
1. A custom hcuristic is calculated for all rows of data, similar to an absolute residual, where larger values represent a larger deviation between the observed labels and predictions.
2. The residuals of all the rows' predictions and observed outcomes are calculated.
3. The full data is split into training and holdout set.
4. Three partitions of data are created for the training data, hold out data, and the full dataset:
(a) A partition containing all the rows.
(b) A partition containing all the rows with predictions greater than 0.50.
(c) A partition containing all the rows with predictions less than or equal to 0.50.
5. For each of the partitions of data constructed in Step 1
(a) A ridge regression classifier (using a 1.0) is trained using the respective partition in the training data, with the covariates X and the sensitive attribute A as features and the custom heuristic calculated in Step [ as labels.
(b) The ridge regression classifier used to make predictions for the respective partition in the holdout data.
30

---

Published in Transactions on Machine Learning Research (01/2026)
(c) If the average of the predictions multiplied by the residuals for the partition set in the hold out data is greater than 10-1, then the predicted log-odds for the respective partition in the full dataset is shifted by the predictions multiplied by 0.1.
(d) If the predicted log-odds are updated, the iteration terminates and no other partitions of data are evaluated for that iteration.
The steps above slightly modified for the scenario of a classifier that produces singular probability of a posilive oulcome whereas the original MultiAccuracy Boost was designed for was a bivariate outcome vector from an Inception-eeNNetvvl model. To make MultiAccuracy Boost audit for bias in one direction, when calculating whether a partition of the data's predicted log-odds should be updated using the holdout data to remove an inaccuracy, we override the residuals that are negative with O. In effect, we only consider rows with negative outcomes when deciding which partition of predictions have inaccuracies that need to be corrected on a given iteration. This was the Ieast invasive modification we could make to MultiAccuraccy Boost to have it solely consider bias in the positive direction when deciding which subgroup's 's predieted log-odds to updale. When using this slight adaptation, we see an increase in the overall average accuracy for the simulations by approximately 8% for MultiAccuracy Boost compared to a version of MultiAccuracy Boost without the modification intended to account for directional bias.
Since the auditor and correction method are functioning in tandem, we run all iterations of the algorithm and log each subgroup (i.c., partition) that was detected as needing a correction to its predicted log-odds and its associated score calculated in Step be. After the algorithm terminates, we find the partition with the highest score and return its associated partition in the full data set. The decision to return the partition with the highest score across all the iterations of MultiAccuracy Boost in the simulations is motivated by the fact that MultiAccuracy Boost's auditor has no thcoretical guarantecs of detecting the most inaccurate partition on a specific iteration of the algorithm. Similarly to GerryFair, MultiAccuracy Boost detects a subgroup that contains members of the protected class and non-protected class. We filter all the individuals in the returned subgroup to only contain individuals who are part of the protected class before calculating the accuracy of the returned partition.
One distinction between these methods and CBS is that their anditors were intended to be used in conjunction with another process to improve classifier or predictions. Therefore, their auditors were designed to have the level of detection accuracy necessary to discern which subgroups or partitions of data need to be corrected, either by modifying the classifier or by post-processing their predicted log-odds. Given that both methods suggest that they can be used for auditing purposes, they are appropriate choices as benchmarks for CBS, but it is important to note that CBS was specifically designed to have a high accuracy for bias detection, whereas that was not necessarily an explicit intention of GerryFair or MultiAccuracy Boost.
B.2
Explanation of the Additive Term (crre) for the True log-Odde used in the Generative Model for the Semi-Synthetic Data
For the evaluation simulations described in Section 3 when producing the true log-odds that are used to determine the outcomes, Y, and predicted values, P or Poin: we add term to each row's true log-odds, I of a value drawn from a Gaussian distribution <:rue ~ NO(0, otrae) where Strue = 0.6. We add this term to the true log-odds to ensure that when the true probabilities (expit(L(rue) for the rows of Shine in the protected class A are injected with Peuf, this results in a violation of the fairness definition for sufficiency.
To illustrate this further, for the remainder of this section we will focus on sufficiency scan for predictions, but our explanation below is applicable for sufficiency scan for recommendations as well. Sufficiency implies that the outcomes Y are conditionally independent of membership in the protected class A given the predictions P and covariates X, that is, Y 1 A (P, Assume that we have predictions that are independent of the outcome conditional on the covariates, Y 1 P X. Since the outcome is independent of the predictions conditions ional on the covariates, the definition of sulficiency simplifies to Y 1 A X. This simplification of sufficiency reduces sufficiency scans to finding the subgroup in the protected class with the largest base rate difference from its corresponding subgroup the non-protected class regardless of that subgroup's 's predictions. Therefore, it is not evaluating sufficiency violations because these base rate differences are independent of the predictions. Consequentially, when there is no base rate difference between the protected and non-protected
31

---

Published in Transactions on Machine Learning Research (01/2026)
when Meep 0.50
when Peuf 0.50
1.0
1.0
0.8
0.8
Accuracy
0.6
Accuracy
0.4
0.2
0.2
0.0
-0.2
0.0
Separation Scan for Recommeendeloos
Suffirienry Sran for Recenimendators
Separation Sran for Predictions
Suffiriency Sran for Predictions
Multiaccuraty #oest
Gemyfair us all features for Felse Pusitives
Figure 4: Average accuracy (with 95% CT) for biases injected of Meep 0.50 or Psuf = 0.50 into subgroup Sniss of the protected class A, for CBS, GerryFair, and MultiAccuracy Boost, as a function of varying base rate difference å between protected and non-protected class for subgroup Soias Left: increasing predicted probabilities by μsep = 0.50. Right: decreasing true probabilities by flaf = 0.50.
class conditional on the covariates, (Y X)), in order for sufficiency be violated, Y + A (P,x), we must also have Y P X. This is formally stated in Theorem B.1
Theorem B.1. have violations of the sufficiency definition, Y (P,xX) when there are no base rate differences between the protected class and non-protected class conditional on the covariates, Y LA the predictions and oulcomes must be conditionally dependent given the covariales, Y P|
Proof. Let assume that (i) there are no base rate differences between protected and protected class conditional on the covariates, Y 1 A X; (ii) outcomes are independent of the predictions conditional on the covariates, Y J. X;; and (iii) violations of the sufficiency definition exist, Y y A (P, We will show that these three statements lead to a contradiction. First, (Y I. P and (Y 1 LAIX) together imply that Y (P, A) X. Furthermore, using the weak union axiom for conditional independence, implies that Y 1 XP, which contradicts (iii). Since these three statements cannot all be true, we know that base rate differences (i) and violations of sufficiency (iii) together imply that the outcomes cannot be independent of the predictions conditional on the covariates, Y P
L (P, X
To ensure that Y L Pl PIX, the predictions P must carry information about the outcomes Y that is not carried in X. By adding the term grue to the true log-odds for each row, given that the predicted log-odds (and the corresponding predicted probabilitics P; and binarized recommendations Pr,sn) and the outcomes are both derived from the true log-odds, this ensures that 1 P X in the evaluation simulations because P carries information about Y, in the form of the added row-wise terms (drawn from Gaussian distribution), that are captured in Y, but are not captured in X.
B.3 Additional Evaluation Simulations
This section includes additional evaluation sinulations and their accompanying results conducted to further examine CBS's performance.
Firstly, we investigated the case where we have both an injected bias signal (Meep 0.50 or Peaf = 0.50) and an injected base rate shift 8 in subgroup Sbias of the protected class A (Figure 4 We examined the extent to which positive and negative shifts for 5 either help harm the detection accuracy of the various methods.
32

---

Published in Transactions on Machine Learning Research (01/2026)
1.0
1.0
0.8
0.8
0.6
Accuracy
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.5
1.0
3.5
K
Pep
Separation
Sufficiorry for Recenmmendiatoos
Separation Sran
Suffiriency Predictions
Gemyyair features for Felse Pasitives
Recunmenddsions
for Predictions
Multicccrrcyy Hoest
Figure 5: Average accuracy (with 95% CI) as a function the amount bias injected into subgroup Spiae of the protected class A, for four variants of CBS, GerryFair, and MultiAccuracy Boost. Left: increasing predicted log-odds by Keepp Right: decreasing true log-odds by plags
Thus we run two separate sets experiments, each defined by differing initial setups injecting a fixed and constant bias signal into subgroup Syias for protected class of: (1) Peep 0.50, and (2) Psuf 0.50. Then, for both experiments, (1) and (2), we vary the injected base rale shift from -0.50 to +0.50. A positive base rate shift (ō 0) means Spias in the protected class A has higher base rate, while negative base rate shift (8 0) means Spies in the protected class A has lower base rate, compared to Spice in the non-protected class.
In the leftmost plot of Figure where a separation violation injected with a signal of Psep = 0.50, we can observe, as expected, that Shins is detected with near-perfect accuracy with separation scans for recommendations and predictions, as well sufficiency scan for predictions. These results are in line with those displayed in Figure 1 where the injected signal of Ussp for Spias is increased but the base rate difference is unchanged (ō 0) for Sbias in protected class A compared to Sbias in the non-protected class. Additionally, in the leftmost, plot of Figure we see that the detection accuracy for the sulficiency scan for recommendattions of Spiae increases as the base rate difference, 6, for Suias decreases. This is in line with the results contained in Figure 2 where conditioning on the binary recommendation, Pe not sufficient in these simulations to capture the decrease true probabilities, and, therefore, negative base rate shifts (ō < 0) for Spies are detected as sufficiency violations.
In the rightmost plot of Figure 4 where sufficiency violation is injected with signal of Psuf 0.50, the sufficiency scans for recommendations and predictions detect Spiae with near-perfect accuracy independent of the base rate shift injected into Spias- This is both expected and consistent with the results contained in Figure [ Additionally, as is documented in prior research and we show empirically in Figure 2 for well-calibrated models, base rate difference in Spias for the protected class A will result in separation violations (Chouldechova 2017). This consistent with the results in the rightmost, plot of Figure 4 which show that as we inject a positive basc rate shift (8 > 0) into subgroup Sbias of protected class A, the separation scans detect Spias with near-perfect accuracy.
Lastly, the method use in Section 3 for injecting bias signal or shifting the base rate of the affected subgroup Sbies in the protected class A involves incrcasing or decrcasing the true probabilities and predicted probabilities. CBS is designed to detect a constant, additive shift in the true and/or /o predicted log-odds (not the true and/or predicted probabilities) for a subgroup, Smas in the protected class A in comparison that subgroup the non-protected class (as shown the alternative hypotheses contained in Table 2 Therefore, our main simulations are designed to ensure that CBS is robust to injected biases and base rate shifts that do not take the same form CBS's modeling assumptions, specifically, do not take the form
33

---

Published in Transactions on Machine Learning Research (01/2026)
1.0
0.
Separdtion Scar Rocam mandaters
0.0
for Recom mondadons
See sratratoo Star
64
for Prediction claama
MuBlcccuacy doost
63
Eerylar using all feetu is
Figure 6: Average accuracy (with 95% CI) as function of the base rate difference 8' between protected and non-protected class for subgroup Swias: for four variants of CBS, GerryFair, and MultiAccuraccy Boost. Note that predictions are well calibrated, P'app = P'ss O.
of additive shifts in the true and/or predicted log-odds. With that said, for comparison purposes, we also examine injected biases and base rate shifts that take the form of shifts the true and/or predicted log-odds. and 0 can be directly compared to Figures and 2 respectively. Specifically, we perform the following simulations:
The resulting Figures S
We increase the predicted log-odds, L.predcct, by Usep for Soice in the protected class A. Note, this shift is performed prior to the predicted probabilities, P., being drawn for all the data.
We decrease the true log-odds, I,tue, by H'eus for Spiss in the protected class A. This shift is performed after predicted probabilities have been drawn for all the data. After the true log-odds, T,rre, have been decreased by V'nf for Spins in the protected class A, outcomes Y, are redrawn specifically for the rows of Shius in the protected class.
We simultaneously shift the true and predicted log-odds, I,true and T.predict by 8 for Spias in the protected class A. Outcomes are redrawn for Spios in the protected class A after the shift by is performed.
Figure 5 we observe that the injected signals for p'ep and Paayy (represented as shifts the predicted and true log-odds, Lpredict and L respectively) have an effect on CBS's detection accuracy that nearly identical to the predicted and true probability shifts (Meep and Meuf respectively) shown in Figure] Similarly, in Figure 0
that the base rate shift created by simultaneously shifting the true and predicted log-odds, Igrue and predict by 8' for Ssias in the protected class A has an effect on CBS's detection accuracy that is nearly identical to the simultaneous shift of the true and predicted probabilities of Spias in the protected class A by as shown in Figure 2 Therefore, we can conclude that CBS performs well for both constant additive shift in the true and/or predicted log-odds (consistent with its modeling assumptions), as shown in Figures and 6 and also achieves high detection power for non-additive shifts as shown in Section 3 Figures and 2
B.4 Robustness Analyses of Evaluation Simulations for Parameters Osrue and o predict
In this section, we examine the robustness of our results in Section 3 by varying the parameters Opeddict and Otrue from their default values of 0.2 and 0.6 respectively.
First, examine the impact of varying a predict- Recall that each predicted log-odds, I. I.predict, is drawn from Gaussian distribution centered al the true log-odds, with standard deviation predditt. Thus Opredict can be interpreted as the average amount of random error in the classifie''s predictions as compared to the true log-odds values. We run three separate sets of experiments with different initial setups:
(1) We alter Shies in the protected class A by injecting a bias of Msep 0.50 (separation violation).
We alter Smiss in the protected class A by injecting bias of psuf 0.50 (sufficiency violation).
34

---

Published in Transactions on Machine Learning Research (01/2026)
when Peep 0.50
when Prof 0.50
when d 0.25
1.3
3.4
000
3.5
1.50
006
0.5
633
0.25
Separation Su.fficienty Recommendations
Recomimendat ons
Separation Snan
Predictions
Sufficiency Fredictions
Muttaccurzcy Boost
Gerryfair using all features Positives
Figure 7: Average accuracy (with 95% CI) for biases and base rate shifts injected into subgroup Smies of the protected class A, for CBS, GerryFair, and MultiAccuracy Boost, as function of varying parameter Ppeedict Left: increasing predicted probabilities by Psep 0.50. Center: decreasing true probabilities by Menf = 0.50. Right: base rate difference 8 = 0.25, for Peep = Hauf =
when = 0.50
when Mauf 0.50
when 5 = 0.25
34
0.
2.6
64
0.2
Uiru
Orru
Separatian Sran for Retommmendat ons
Sufficienct Sran for Recorimendations
Separation Sran for Fredictions
Suffirianry Sran for Predictions
#ultiaccuracy fleest
Geryyfiir asing all features for Falce Positives
Figure 8: Average accuracy (with 95% CI) for biases and base rate shifts injected into subgroup Sbias of the protected class A, for CBS, GerryFair, and MultiAccuracy Boost, as function of varying parameter Otrue- Left: increasing predicted probabilities by Psep 0.50. Center: decreasing true probabilities by Msuf = 0.50. Right: base rate difference 8 = 0.25, for Psep Mauf
0.
(3) We create a positive base rale difference for Spine in the protected class A of 8 = 0.25.
CBS and the competing methods while varying predict between 0 and 2. Accuracies are averaged over 100 semi-synthetic datasets for each experiment. The experiments where Psep = 0.50 and Msuf = 0.50 analyze the robustness to predict of the evaluation simulations for [Q1) whereas the experiments where = 0.25 analyze the robustness to Opredica of the evaluation simulations for (Q2)
For each initial setup. 1
(3)
In Figure 7 we observe that large amounts of noise predict harm the accuracy of the separation scans for injocted biases Psaf 0.50 which shift the true probabilitics in subgroup Spias for the protected class. When Opeedict is large, biases Msep = 0.50, which is expected given this scan's initial lower accuracy detection with recommendations with a moderate value of noise in the rocommendations.
see a reduction in accuracy for the sufficiency scan for recommendations for injected
Second, we examine the impact of varying Otrue- Recall that each individual's true log-odds a deterministic (linear) function of their covariate values X, plus a term, ę(ree, drawn from a Gaussian distribution centered at with standard deviation of dtrue- Thus the paramoter Orrue represents the variation between individuals' true log-odds based on characteristies other than the covariate values X, used by CBS. Moreover, since each individual's predicted log-odds is drawn from Gaussian distribution centered at the true log-odds, these
35

---

Published in Transactions on Machine Learning Research (01/2026)
characteristics are assumed to be known and incorporated into the classifier, thus creating the dependency Y / PIX when Stree > 0. In other words, Otree represents the average amount of signal in the predictions P (for predicting the outcome Y) that is not already present in the covariates X.
We run three separate sets of experiments, with the same initial setups described in 11) 3 where we alter
Sbias in the protected class by injecting a bias of Msep = 0.50, injecting a bias of Msuf = 0.50, and creating a base rate difference of 6 = 0.25 respectively. Similarly, for each setup (1)
we run CBS and the competing methods while vary true between 0 and 2 for cach experiment. Accuracies are averaged over 100 semi-syuthetic datasets for each experiment. The experiments where Meep 0.50 and Mawf 0.50 analyze the robustness to true of the evaluation simulations for [Q1) whereas the experiments where 8 = 0.25 analyze the robustness to Orrue of the evaluation simulations for [Q2)
33
In Figure [ we observe that small values of Irue harm the accuracy of the separation scans for injected bias f 0.50 while making them more likely to detect basc rate shifts > in subgroup Saaas for the protected class. Most interestingly, when Trree small, we see a substantial reduction in accuracy for the sufficiency scans for injected bias Msep 0.50. This reduced performance for truue R 0 follows from our argument in Section B.2 above: Otrue = 0 implies Y 1 P X, and if we also have no base rate difference between the protected and non-protected classes (Y 1 A X), this implies Y 1 A P.X. In other words, even if a bias is injected into the predicted probabilities (and recommendations) in subgroup Smas for the protected class, the sufficiency-bast based definition of fairness is not violated, and thus the injected bias cannot be accurately detected.
B.5 Estimates of Compute Power
For all of the experiments in Section 3 Appendix B.3 and Appendix B.4 with the exception of the experiments displayed in Figure 5 and Figure G we used a university's high-performance computing (HPC) services. We completed all these simulations with 100 jobs that used one node, one core (CPU), and 7 GB of memory each. Each of these jobs performed 1,344 CBS runs, and each job was alive for approximately days. To perform the experiments displayed in Figure 5 and Figure D as well additional robustness checks, we used 15 shared, university compute servers rumning CentOS with 16-64 cores (CPU) and 16-256 GB of memory. Each server performed 15-120 runs of CBS concurrently, and ran for approximately 9 days. We estimate that to run all of the simulations and robust checks (1,344 CBS runs in lotal) for a single data set using shifts in the predicted and true probabilities for injecting bias and base rate shifts, this would take approximately 9 days. We estimate that to run all of the simulations and robustness checks (1,504 CBS runs in total) for single data set using shifts in the predicted and true log-odds for injecting bias and base rale shifts, this would take approximately 32.5 hours. Lastly, to an individual CBS scan for the COMPAS data (150 iterations), it takes on average approximately 90 seconds. A single run of CBS takes a similar runtime for the German Credit Data.
[.4
C Case Studies Appendices for Section 4
C.1 Case Study of COMPAS Appendices
C.1.1
Calibration Curve Plots for Estimating for COMPAS
The method presented in Section 2.2 to estimate the event variable î for individuals in the protected class under the null hypothesis Hn: I L A (C,X) relies on two models being well-specifidd: (1) the propensity score model for estimating Pr(A = 1| X); and (2) the outcome model for estimating ī i= E C, X]. Figure 9 contains the calibration curves for all propensity score models for all CBS runs on the CÓMPAS data; note that this curve is computed using all data for fitting Pr(A= = X), which includes all instances in the entire dataset D. Figure I0 contains the calibration curves for all outcome models for all CBS runs on the COMPAS data; note that this curve is computed using all data for fitting î = IE[ C, X], which only includes the individuals in the non-protected class (A, = 0) and, depending on the scan, may have further restrictions on the conditional variable C. For our COMPAS case study, we used logistic regression for all of these models.
36

---

Published in Transactions on Machine Learning Research (01/2026)
COMPAS's Propensity Score Models with Quantile Binning (5 bins)
1.0
Mean of Observed Event Variable (0) per bin
0.9
0.8
0.6
0.5
0.4
0.2
0.1
0.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Mean Predicted Probability per bin
Figure 9: Calibration curves for all propensity score models used to estimate Pr(A = 1 1 X) when generating Í, for all CBS runs on the COMPAS data represented in Table
COMPAS's Outcome Models with Quantile Binning (5 bins)
10
Mean of Observed Event Variable () per bin
0.8
0.7
0.5
0.4
0.3
0.2
0.0
0.1
0.2
0.3
0.5
0.6
0.7
0.8
10
Mean Predicted Probability per bin
Figure 10: Calibration curves for all outcome models used to estimate score- weightdd data from the non-protected class, for all CBS runs on the COMPAS data represented in
= E[I C, using propensity
Table
37

---

Published in Transactions on Machine Learning Research (01/2026)
C.1.2
Full Results of COMPAS Case Study
Table contains the full set COMPAS results for CBS.
Observed Metric
O Osserved Metric for Complement Detected Subgroup
Log-Likelihood
Scan
Detected Subgroup (8*)
for Sensitive
Ratio
Metric
Type
(F(S*))
Detected Subgroup
(Num. of Defendants)
(Nun. of Defendants)
Sensitive Attribute: Black Defendants compared to Non- Black Defendants
Sep. Pred.
Males
42.4
FPE
0.45 (1168)
0.35 (1433)
Sep. Rec.
Males
102.3
FPR
0.44 (1168)
0.19 (1433)
Suff. Pred.
Females
2.21
CAL
0.37 (549)
0.34 (626)
Suff. Rec.
Age 25+ with 0-5 priors
0.37
PPV
0.50 (581)
0.52 (404)
Sensitive Attribute: White Defendants compared to Non- White Defendants
Sep. Pred.
0.0
FPE
Females under age 25
Sep. Rec.
2.01
FPR
0.71 (31)
0.56 (70)
with no priors
Suff. Pred.
Under age 25
2.36
CAL
0.49 (347)
0.58 (1000)
Suff. Rec.
Females under age 25
0.41
PPV
0.39 (57)
0.47 (110)
Sensitive Attribute: Native American Defendants compared to Non-Native American Defendants
Scp. Pred.
All Native American defendants
0.45
FPE
0.49 (6)
0.39 (3357)
Sep. Roc.
All Native American defendants
0.53
FPR
0.50 (6)
0.30 (3357)
Suff. Pred.
Native American defendants
0.14
CAL
0.45 (11)
0.46 (6161)
Suff. Rec.
0.0
PPV
Sensitive Attribute: Asian Defendants compared to Aslan Defendants
Sep. Pred.
0.0
FPE
Sep. Rec.
0.0
FPR
Suff. Pred. Arrested on misdemeanor charges
3.16
CAL
0.00 (12)
0.38 (2190)
Suff. Rec.
6+ priors
0.11
PPV
0.00 (1)
0.76 (965)
Sensitive Attribute: Hispanic Defendants compared to Non-Hispanie Defendants
Scp. Pred.
0.0
FPE
Sep. Rec.
0.0
FPR
Suff. Pred.
All Hispanic defendants
0.26
CAL
0.37 (509)
0.46 (5663)
Suff. Rec.
All Hispanic defendants
2.48
PPV
0.56 (141)
0.63 (2610)
Sensitive Attribute: Male Defendants compared Female Defendants
Sep. Pred.
Asian
0.63
FPE
0.30 (22)
0.22 (1)
Sep. Rec.
Asian and Hispanic
22.5
FPR
0.21 (286)
0.05 (57)
Suff. Pred.
Native Americans age 25
31.4
CAL
0.14 (7)
1.00 (2)
Suff. Rec.
Native Americans age 25+
14.1
PPV
0.25 (4)
1.00 (2)
Sensitive Attribute: Female Defendants compared to Male Defendants
Sep. Pred.
White
1.51
FPE
0.38 (312)
0.35 (969)
Sep. Rec.
White
12.5
FPR
0.29 (312)
0.20 (969)
Suff. Pred.
Under age 25
18.7
CAL
0.38 (246)
0.60 (1101)
Suff. Rec.
Under age 25
13.2
PPV
0.44 (167)
0.68 (699)
Sensitive Attribute: Defendants under age 25 compared to Defendants age 25+
Sep. Pred.
All defendants under age 25
128.2
FPE
0.51 (593)
0.37 (2770)
Sep. Roc.
All defendants under age 25
159.3
FPR
0.53 (403)
0.25 (1583)
Suff. Pred.
0.0
CAL
Suff. Rec.
0.0
PPV
Sensitive Attribute: Defendants age 25+ compared to Defendants under age 25
Sep. Pred.
0.0
FPF
Sep. Rec.
Asians arrested felony charges
0.74
FPR
0.20 (10)
0.00 (1)
Suff. Pred.
Males with 0-5 priors
92.7
CAL
0.35 (2867)
0.59 (1041)
Suff. Rec.
Males with 0-5 priors
53.0
PPV
0.52 (772)
0.67 (641)
Continued on next page...
38

---

Published in Transactions on Machine Learning Research (01/2026)
Ohserved Metric for Sensitive
Ohserved Metric for Complement
Log- Likelihood
Scan Type
Detected Subgroup (S*)
Ratio
Metric
Detected Subgroup
Detected Subgroup (Num. of Defendanta)
(F(5*)
(Num. of Defendants)
Sensitive Attribute: Defendants with no priors compared to Defendants with 1+ priors
Scp, Pred.
0.0
FPE
Scp. Roc.
0.0
FPR
Suff. Pred.
All defendants with no priors
111.6
CAL
0.29 (2085)
0.54 (4087)
Suff. Rec.
All defendants with no priors
51.0
PPV
0.46 (553)
0.67 (2198)
Sensitive Attribute: Defendants with 1-5 priors compared to Defendants with no or 6 priors
Sep. Pred.
Under age 25
3.28
FPF
0.54 (227)
0.49 (366)
Sep. Rec.
Under age 25
12.6
FPR
0.64 (227)
0.47 (366)
Suff. Pred.
Black defendants of age 25+
2.17
CAL
0.42 (1038)
0.55 (1328)
Suff. Rec.
Male defendants of age 25+
26.8
PPV
0.54 (595)
0.70 (981)
Sensitive Attribute: Defendants 6+ prions compared to Defendants with 0-5 priors
Sep. Pred.
All defendants with 6+ priors
83.9
FPE
0.54 (349)
0.38 (3014)
Sep. Rec.
All defendants with 6+ priors
126.9
FPR
0.66 (349)
0.26 (3014)
Suff. Pred.
0.0
CAL
Suff. Rec.
Asian
0.42
PPV
0.00 (1)
0.83 (6)
Sensitive Attribute: Defeudants arrested on felony charges compared to Defendants arrested on misdemeanor charges
Sep. Pred.
White females
2.45
FPE
0.42 (139)
0.34 (173)
Sep. Rec.
White females
9.56
FPR
0.38 (139)
0.21 (173)
Suff. Pred.
0.0
CAL
Suff. Rec.
0.0
PPV
Sensitive Attribute: Defendants arrested on misdemeanor charges compared to Defendants arrested on felony charges
Sep. Pred.
0.0
FPE
Sep. Rec.
Native Americans with 1-5 priors
1.67
FPR
1.00 (2)
0.00 (1)
Suff. Pred.
Females
3.51
CAL
0.26 (491)
0.41 (684)
Suff. Rec.
All defendants arrested
10.7
PPV
0.55 (736)
0.66 (2015)
on misdemeanor charges
Table 5: Full results from CBS scans run COMPAS data. Sep. Pred. is short for separation scan for predictions in the positive direction where the metric FPE stands for E[P Y = O, X]. Sep. Rec. is short for separation scan for recommendations in the positive direction where the metric FPR, L.e. false positive rate, is Pr(Pain = Y| = 0. X). Suff. Pred. is short for sufficiency scan for predictions in the negative direction where the metric CAL, Le. calibration, is Pr(Y = X). Suff. Rec. is short for sufficiency is Pr(Y = 1 Poin 1). The third column contains the log- likelihood ratio, F(S*) defined in Equation [ for the detected subgroup, S*, listed in the second column. Note, bold scores of F(S*) are statistically significant with p-value <.05 measured by permutation testing, as described Section 2.4 For example, for the separation scan for recommendations with Black defendants as the sensitive attribute (sccond row), Black males had false positive rate of 0.44 (n = 1168) compared to 0.19 for non-Black males (n 1433).
for recommendations in the negative direction where the metric PPV, i.e. positive predictive value,
C.1.3 Discussion of COMPAS Results for Benchmark Methodologies
Our ovaluation of CBS, GerryFair, and MultiAccuraccy Boost (Section 3) uses semi-synthetic data that maintains the covariate distribution of COMPAS. The evaluation simulations follow framework that employs certain generative assumptions for injecting bias into subgroups. The limitations of these generative assumptions used in our framework are discussed in detail in Section 6 In this Appendix, we provide the results of the benchmark methodologies (GerryFair and MultiAccuracy Boost) run on the original COMPAS data, and compare these results to the CBS results for the COMPAS case study in Section 4 We include these results to highlight the differences between CBS and the benchmark methodologics on non-synthetic dataset, showing the benefits of CBS in a setting without the generative model assumptions used in Section 3
We ran GerryFair and MultiAccuracy Boost using the same COMPAS data, preprocessing steps, and setup described in Section 4 We report two sets of results: (1) the results of these methodologies (GerryFair and MultiAccuracy Boost) with their out-of-the-box settings; and (2) the results when using the minimum modifications needed to adapt these methods for under-estimation and over-estimation bias, described in Appendix [B.]] We include both of these results to display the methodologics' default functionality, which we assume is the intended setting for practitioners, and to obtain a set of results for COMPAS data that can be used to contextualize the differences between these benchmark methodologies and CBS in a woal-world setting.
39

---

Published in Transactions on Machine Learning Research (01/2026)
GerryFair and MultiAccuracy Boost provide demonstration code that uses probabilities as the predictive output to be audited, and therefore we use the same P. calculated for each defendant based on their COMPAS risk score, as described in Section 4
GerryFair Results: When running GerryFair to detect intersectional biases in false positive rates, with race, sex, and the indicator variable of whether defendants are under the age of 25 marked as sensitive attributes, the detected subgroup consists of all defendants aged 25+ who are not Black or Native American. This subgroup is systematically advantaged rather than disadvantaged: non-reoffending defendants in the detected subgroup have an average predicted risk E(P Y = 0) = 0.32, while non-reoffending defendants not included in this subgroup have an average predicted risk E(P = 0) = 0.45. When modified to perform directional scan to search for a systematically disadvantaged subgroup, GerryPair detects a subpopulation consisting of three distinct, marginal groups- all defendants under 25, all Black defendants, and all Native American defendants-rathur than an intersoctional or contextual subgroup.
MulliAccuracy Boost Results: MultiAccuracy Boost chooses between three partitions of data on each iteration of the algorithm, where the chosen partition has its probabilities adjusted. When rumning MultiAceuracy Boost with its default settings on COMPAS, the highest scoring partition is found on the first iteration. This partition consists of all defendants in the initial iteration that had higher probabilities (P > 0.50), and therefore each of those defendants' probabilities gets adjusted depending on MultiAccuracy Boost 's custom residual heuristic metric (see Appendix B.I]. Given that there are large overlaps in the covariate spaces of the partition that gets its predictions adjusted and the other partitions, the best way to describe this partition's 's covariate space is based on the coefficients of the classifier used to model the custom residual heuristic metric, as described in Appendix B.1 where larger values contribute to larger adjustments needed to the probabilities of the defendants in the detected subgroup. The factors that are associated with defendants in this partition needing larger adjustments to their probabilities include defendants with no priors and Hispanic defendants. We note that this algorithm is stochastic, but these covariates consistently show a positive association with larger values of the adjustment hcuristic.
When running Multi MultiAccuracy Boost using the modifications described in Appendix B. to detect directional bias, the highest scoring partition is found on the first iteration of the algorithm. We find that the factors that estimate the level of adjustments needed to the defendant's probabilities include defendants with no priors, Hispanic and Female defendants, defendants of age 25+, and defendants arrested on misderneanor charges.
Discussion: There are several takcaways to highlight about the results of GerryFair and MultiAccuracy Boost for COMPAS:
GerryFair's original implementation of its auditor does not allow the user to select between detection of over-estimation bias and detection of under-estimation bias. This results in a detected subgroup of non-rcoffending defendants that advantagod rather than disadvantaged, benefiting from lower predicted risk.
With our modification to detect directional bias, GeiryFair finds a large subpopulation consisting of all Black defendants, all Native American defendants, and all defendants under the age of 25. The results of CBS for separation scans for predictions (Appendix C.1.2) show some similarities with GerryFair's results that is, for cach of the three protected classes included in GerryFair's 's results, the subgroups detected by CBS within the protected class also have posilive scores. The major distinction is that GerryFair is not detecting intersectional or contextual subgroups within the protected class, such as the subgroup of Black males detected by CBS. In contrast, CBS identifies that non-rcoffending Black male defendants have a higher predicted risk compared to non-reoffending non-Black male defendants, and that this identified racial disparity is more significant than the disparity between all non-reoffending Black defendants and all non-reoffending non-BBack defendants.
More generally, GerryFair appears to lack the flexibility of CBS to specify a single protected class and scarch for intersoctional or contextual subgroups within that protected class for whom bias is present. In the given example, it identifies some individuals using characteristics unrelated to race, and the marginal subgroups of all Black defendants who did not reoffend and all Native American defendants
40

---

Published in Transactions on Machine Learning Research (01/2026)
who did not reoffend respectively. This is consistent with our evaluation results in Section 3 in which GerryFair was able to reliably detect marginal biases (for simulation parameter Phias low power to detect smaller, more subtle subgroup biases.
1) but had
The results of MultiAccuracy Boost suggest that while MultiAccuracy Boost provides black-box auditor tool, its auditor does not provide interpretable results. This is because the algorithm forms subgroups based only prediction thresholding, which results in these subgroups having overlapping covariate spaces. This, in combination with the method's inability to audit for specific biases for specified protected class attributes, results in the algorithm neglecting to find important intersectional biases. This evident from the factors that describe over-cstimation bias being defendants of age 25+, defendants with no priors, Hispanic and female defendants, which somewhat aligns with CBS's results for sufficiency sean for predictions for COMPAS, but does not have the capabilities to also find more subtle biases such the subgroup of Asian defendants arrested on misdemeanor charges affected by evtreestination bias.
In summary, we believe that the above results demonstrate the advantages of CBS as compared to competing mothods, as an auditor for detecting intersoctional and contextual biases in a real-world context.
C.2 Case Study of German Credit Data
We present the results of using CBS to audit for prodictive bias in algorithmically-genee risk scores for customers in the German Credit Data (Hofmann 1994). This dataset contains information about 1,000 customers from a German financial institution. Each row of the dataset represents a customer. For each customer, various pieces of demographic, socioeconomic, and financial information are available, well as a label generated by the financial institution indicating whether each customer is "good" (trustworthy for credit) "bad" (untrustworthy for credit) customer. This dataset is often used in the fair machine learning literature to evaluate the predictive bias of models estimating credit risk. This is also the context we assume for these data. We include these appendices to demonstrate the use of CBS for an additional dataset. This case study also provides an example of running CBS on a notably smaller data set: the German Credit Data is less than one sixth of the size of the COMPAS data terms of rows. Below we provide the same set of results those shown for COMPAS above.
C.2.1 Preprocessing of German Credit Data
We use a publicly available version of the German Credit Data that has mapped the keys in the original Statlog data file to thoir decoded categorics [Datahub.io 2019).
We follow the feature selection and preprocessing methods documented Kamiran Calders (2009), which is one of the first publications that used these data for fair machine learning research. For each customer, we use the following information:
Whether the customer is under age 26 or age 26+
Whether the customer owns, rents, or lives in their housing for free.
The customer's gender and marital status. These were initially coded as one variable. For CBS we create two separate categories for gender and marital status. Additionally, we create two high-level categories for marital status: single or married (rrried/seppatted] seppaated//divoreed hetvvorred/widowwwe (i.e., "non-single").
The customer's credit history. We recode this category to the following schema: previously delayed credit/ critical credit/ credit/other /other existing credit or no credit/all credit paid. This involved combining the "no credit/ all credit paid", "all paid", and "existing credit paid" categories because of their overlap. Additionally, we combine previously delayed credit and critical credit/ other existing credit categorics because lack of clear differences between the categorics. The main motivation of these simplifications was to ensure that each category was not overlapping and thus to increase interpretability. We note that there is lack of granularity specifying if the customer has never
41

---

Published in Transactions on Machine Learning Research (01/2026)
had credit before has no credit because they have paid off all their previous credit for most of the customers in the data set. This is why we see a correlation between customers being labeled as untrustworthy for credit and customers in the category of "no credit/all paid".
Whether a customer is considered a trustworthy or untrustworthy customer for credit by the financial institution. An untrustworthy customer is coded as a positive outcome and a trustworthy customer as a negative outcome for consistency with the COMPAS case study's outcome label.
Unlike COMPAS, which provides both an algorithmically-geneaaa for each row, the German Credit Data only provides the label of whether a customer trustworthy or untrustworthy for credit, which is commonly used as an outcome variable. To produce the equivalent of an algorthmiialll-geeeenaeedd risk score for each customer, which we will subsequently audit for predictive bias, we train a logistic regression model using credit history, age (under 26 or age 26+), and housing ownership as predictors and the binary indicator of whether the customer is trustworthy or untrustworthy for credit as the label. We use this model to produce the predicted probability that each customer is untrustworthy for credit. These predicted probabilities, and the corresponding binarized recommendations as to whether each customer is predicted high-risk or low-risk of being untrustworthy for credit, are the prodictive risk scores that we audit with CBS. This modeling approach an example of "fairness through unawareness" because it does not use the two sensitive attributes (gender and marital status) predictors in training to produce its predictions and recommendations. We will examine whether the predictions and recommendations produced by this model still contain predictive biases, as identified by CBS.
risk score and an observed outcome
C.2.2 Scans for the German Credit Data
We preprocessed the outcome variable (whether a customer is trustworthy or untrustworthy for credit) in similar fashion to the COMPAS outcome variable. A positive outcome represents a less desirable real-world result. For the German Credit Data, this means that a positive outcome represents an observed untrustworthy customer for credit. Therefore, we run the same scans in terms of conditional variables and direction for the German Credit Data that we ran for COMPAS. For the separation scans, we detect positive deviations for the protected class attribute in |E(P| = X) and Pr(Pban = 1 Y = 0, X), i.e., increase in average predicted risk for trustworthy customers and increase in FPR (probability of being predicted high-risk for trustworthy customers), respectively. For the sufficiency scans, we detect negative deviation for the protected class in Pr(Y = 1| P) and Pr(Y 1 Poin = X), i.c., decreased probability of being untrustworthy customer conditional on predicted risk and conditional on being predicted as high-risk, respectively. For the separation and sufficiency scans for recommendations, we threshold the probability risk-scores by 0.5 to construct recommendations: Prin = 1(P 2 0.5}. Given the smaller dataset size (as compared to COMPAS) and highly-correlated predictor variables, we found that logistic regression was inadequate for computing propensity scores and for the outcome model (predicting the probabilities i using data from the non-protected class). Thus we use a more flexible model a gradient boosting classifier with Platt scaling, ensure that our predictions are well-calibrated when computing propensity scores and when estimating Í. All scans were run for 500 iterations with a penalty equal to 1.
C.2.3 Calibration Curve Plots for Estimating i for German Credit Data
The method presented in Section 2.2 to estimate the event variable î for individuals in the protected class under the null hypothesis Ho: I 1 A (C,X) relies on two models being well-specified: (1) the propensity score model for estimating Pr(A 1 X)); and (2) the outcome model for estimating f = E | X]. Figure 11 contains the calibration curves for all propensity score models for all CBS runs on the German Credit Data; note that this curve is computed using all data for fitting Pr(A = 1 X), which includes all instances in the entire dataset D. Figure 12 contains the calibration curves for all outcome models for all CBS runs on the German Credit Data; note that this curve is computed using all data for fitting î = E[1 C, which only includes the individuals in the non-protected class (A, = 0) and, depending on the scan, may have further restrictions on the conditional variable C. For our German Credit Data case study, we used gradient boosting classifiers with Platt scaling for all of these models.
42

---

Published in Transactions on Machine Learning Research (01/2026)
German Credit Data's Propensity Score Models with Quantile Binning (5 bins
1.0
Mean of Observed Event Variable (0) per bin
0.9
0.7
0.1
0.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Mean Predicted Probability per bin
Figure 11: Calibration curves for all propensity score models used to estimate Pr(A 1 X) when generating Í, for all CBS runs on the German Credit Data represented in Table
German Credit Data's Outcome Models with Quantile Binning (5 bins)
10
Mean of Observed Event Variable (0) per bin
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.0
0.3
0.4
0.5
0.6
0.7
Mean Predicted Probability per bin
Figure 12: Calibration curves for all outcome models used to estimate - E[7 | C, using propensity score-west weightee data from the non con-prrtected class, for all CBS runs on the German Credit Data represented
in Table
43

---

Published in Transactions on Machine Learning Research (01/2026)
C.2.4 Results of German Credit Data Case Study
Table contains the full set of German Credit Data results for CBS. We observe that the statistically significant biases detected by separation scams are those corresponding to subpopulations with higher base rates (i.e., higher probability of being labeled "untrustworthy" for credit): customers with all paid or no previous credit, younger customers, and customers who have free housing or rent their housing. For sufficiency scans, we detect only a single stalistically significant bias: conditional on predicted risk, older female customers with all paid or no previous credit who own their housing are significantly less likely to be labeled as "untrustworthy" than older female customers with all paid or no previous credit who rent or have free housing.
As described Appendix C.2.1 we purposely excluded the gender and marital stalus leat ures when modeling the risk scores. Since the exclusion of sensitive features alone does not guarantce that model will produce predictions without predictive biases, we examine gender biases detected in the logistic regression model's risk scores. It is notable that sufficiency scan for recommendations identifies subgroup of female customers who own or rent their housing, have critical, previously delayed, or other existing credit, and are aged 26 or older who are flagged as high-risk for credit. This subgroup has a lower rate of being untrustworthy for credit (0.12) compared to the equivalent group of male customers predicted as high-risk for credit, where the rate being untrustworthy for credit is 0.19. This scan additionally detects that male customers who have free housing and are predicted as high-risk have lower rate of being untrustworthy for credit (0.37) compared female customers who have free housing and are predicted as high-risk (0.58). Although neither of these detected subgroups is statistically significant, they do represent deviations, in the form of miscalibrated predictions, that disadvantage subgroup of customers based on their gender compared to the opposite gender. This suggests that removing gender and marital status as predictors may not be sufficient to fully remove gender-related subgroup biases in the model predictions.
44

---

Published in Transactions on Machine Learning Research (01/2026)
Observed Metric for Sensitive
Observed Metric for Complement Detected Subgroup
Log-L Log-Likelhhood Ratio (F(S*)
Scan
Detected Subgroup (S*)
Metric
Type
Detected Subgroup
(Num. of Defendants)
(Num. of Defendants)
Sensitive Attribute: Customers under age compared to Customers age 26+
Scp, Pred.
All customers under age 26
13.5
FPE
0.41 (110)
0.26 (590)
Sep. Rec.
0.0
FPR
Suff. Pred.
All customers under age 26
0.07
CAL
0.42 (190)
0.27 (810)
Suff. Rec.
0.0
PPV
Sensitive Attribute: Customers of age 26 compared Customers under age 26
Sep, Pred.
0.0
FPE
Sep. Rec.
FPR.
Suff. Pred. Single customers who their housing
42.4
CAL
0.22 (366)
0.36 (42)
Suff. Rec.
PPV
Sensitive Attribute: Female Customers compared to Male Customers
Sep. Pred.
0.0
FPE
Sep. Rec.
All Female Customers Owns or rents their housing
0.08
FPR
0.11 (201)
0.03 (499)
Suff. Pred.
7.31
CAL
0.12 (66)
0.19 (234)
with critical, previously delayed
other existing credit of age 26
Suff. Rec.
0.0
PPV
Sensitive Attribute: Male Customers compared Female Customers
Sep. Pred.
FPE
Sep. Rec.
Customers who have housing
8.39
FPR.
1.00 (2)
0.00 (1)
under age 26
Suff. Pred.
Customers who have housing
CAL
0.37 (89)
(19)
Suff. Rec.
All Male customers
0.02
PPV
0.46 (24)
0.48 (44)
Sensitive Auributee: Single Customers compared to Non-Single Customers
Sep. Pred.
0.0
FPE
Sep. Rec.
Customers who have free housing
9.19
FPR
1.00 (2)
0.00 (1)
under age 26
Suff. Pred.
Customers who have free housing
4.92
CAL
0.38 (85)
0.52 (23)
Suff. Rec.
0.0
PPV
Sensitive Attribute: Non-Single Customers compared Single Customers
Sep. Pred.
FPE
Sep. Rec.
Customers who rent their housing
FPR
0.42 (74)
(35)
Suff. Pred.
CAL
Suff. Rec.
All Non-Single Customers
0.54
PPV
0.45 (56)
0.58 (12)
Sensitive Attribute: Customers with all paid or no previous credit
compared to Customers with critical, previously delayed or other existing credit
Sep. Pred.
All customers with all paid no previous credit
86.5
FPF
0.35 (397)
0.20 (303)
Sep. Rec.
0.0
FPR
Suff. Pred.
Single customers of age 26+ who own their housing
1.55
CAL
0.28 (189)
0.15 (177)
Suff. Rec.
0.0
PPV
Sensitive Attribute: Customers with critical, previously delayed other existing credit compared Customers with all paid or no previous credit
Sep. Pred.
FPE
Sep. Rec.
0.0
FPR
Suff. Pred.
Customers who own their housing of age 26
8.80
CAL
0.16 (267)
0.29 (340)
Suff. Rec.
0.0
PPV
Continued next page...
45

---

Published in Transactions on Machine Learning Research (01/2026)
Observed Metric for Sensitive
Observed Metric for Complement Detected Subgroup
Log-Likelihood
Scan
Detected Subgroup (S*)
Ratio
Metric
Type
Detected Subgroup (Num. of Defendants)
(F(S*)
(Num. of Defendants)
Sensitive Attribute: Customers with free housing compared to Customers who rent or own their housing
Scp, Pred.
All customers who have free housing
12.9
FPE
0.39 (64) 0.67 (3)
0.28 (636) 0.32 (107)
Sep. Roc.
Customers under age 26
3.02
FPR
Suff. Pred.
0.0
CAL
Suff. Rec.
0.0
PPV
Sonsitive Attribute: Customers who rent compared to Customers with free housing own their housing
Sep, Prod.
All customers who rent their housing
5.62
FPE
0.38 (109)
0.27 (591)
Sep. Rec.
0.0
FPR.
Suff. Pred.
Female customers
1.91
CAL
0.41 (95)
0.33 (215)
Suff. Rec.
0.0
PPV
Sensitive Attribute: Customers who own their housing compared to Customers with free housing or rent their housing
Sep. Pred.
0.0
FPE
Sep. Rec.
0.0
FPR
Suff. Prod.
Female customers of age 26 + with all paid or no previous credit
81.2
CAL
0.33 (93)
0.50 (42)
Suff. Rec.
0.0
PPV
Table Full results from CBS scans on German Credit Data. Sep. Pred. short for separation scan for predictions the positive direction where the metric FPE stands for B[P Y 0. Sep. Rec. short for separation scan for recommendations in the positive direction where the metric FPR, Le. false positive rate, Pr(Puin = 1 Y 0, 0,X). Suff. Pred. is short for sufficiency scan for predictions in the negative direction where the metric CAL, i.e. calibration, short for sufficiency scan for recommendations is Pr(Y'= 1| Poin X). The third column contains the log-likelihood ratio, F(S*) defined in Equation [| for the detected subgroup, 5", listed the second column. Note, bold scores of P(S") are statistically significant with p-value <.05 measured by permutation test.ing, as described in Section 2..4 Some subgroups are not included for binary sufficiency and binary separation ion scans because the limited range of the predicted risk score prevented auditing with CBS. These are denoted with a s_" the Log- Likellehood Ratio (P(S")) column. We note that the three lowest-scoring subgroups for sufficiency scan for predictions had higher obscrved rates in the detectod group vs. comparison group. These observed rates were still lower than expected, resulting in small but non-zero scores, given the systematic differences in other predictors between protected and non- non-protected class. "Non-single" is short for the marital status attribute "Married/ /divorced /separated/ wwidowed".
Pr(Y P.). Suff. Rec. the negative direction where the metric PPV, i.c. positive predictive value,
C.2.5 German Credit Data Results for Benchmark Methodologies
We use the same setup described in Appendix C.1.3 for rumning the benchmark methodologies with their default settings and with the modifications to account for directional bias. Additionally, we use the same data and risk scores described in the other sections of Appendix C.2
GerryFair Results: When running GerryFair with its default settings of detecting positive or negative deviations in the false positive rate in comparison to the global false positive rate with marital status and gender marked as sensitive attributes, Gerry Fair detects a subgroup of single male customers with a slightly decreased average predicted risk for credit of 0.27 for trustworthy customers comparison the global average predicted risk score of 0.29 for trustworthy customers. This is negative deviation in the false positive rate. The German Credit dataset contains no single females. When running GerryFair to detect positive deviations in the false positive rate, it detects a subgroup of credit-trustwortly married married/divorced /divorced / /separated/ wwdowed customers (i.e., "non-single") who have a slightly increased average predicted risk of 0.30 in comparison to the global expected risk score of 0.29 for all trustworthy customers.
Multi Boost Results: The MultiAccuracy Boost results, both for its default settings and when accounting for over-estimation bias, found no noteworthy associations between the coefficients of the predictors used to estime the custom residual heuristic used in MultiAccuracy Boost. This further substantiates our claim that MultiAccuracy Boost does not have the capabilities to be easily used, in terms of interpretability, as an auditing tool for subgroup predictive biases.
46


Reviews:
[{"review_id": "b9EyrRdEa9", "reviewer": "Reviewer_3jt3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes Conditional Bias Scan (CBS), a framework for auditing intersectional biases. The core methodology uses a well-known Fast Subset Scan (FSS) and derives the score function from the log-likelihood test w.r.t. Gaussian and Bernoulli distributions (Neill, 2012). Then the paper provides extensive empirical evaluation and analysis of the framework on the COMPAS dataset, yielding qualitative analysis results that align with prior works, and yielding some new insights.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes.", "audience_yn": "Yes", "audience_explanation": "Yes. The issue of intersectional bias is of large interest to the ML fairness community.", "requested_changes": "**Making the problem setting and writing a bit clearer**\n- The definition of subgroup, \"a non-empty subset of attribute values for each observed attribute, excluding the sensitive attribute\", should be highlighted. Does this imply that the paper only deals with discrete attributes? Because if some attributes are continuous, then each individual would constitute a separate group, right?\n- I think this is a misunderstanding on my part, but I'm a bit confused about the setting. So.. there is a single protected attribute $A \\in \\\\{0,1\\\\}$ and $m$ other covariates, $X^j$ for $j \\in [m]$. And the goal is to identify subgroups (e.g., individuals with $X^1 = 1$ and $X^5 = 3$) that are unfairly impacted w.r.t $A$? But then, one of the examples the authors mentioned is \"Black and female\". My first understanding of this is that there are multiple sensitive attributes, and we are looking for a subset of sensitive attributes (e.g., $A^1$ (ethnicity, black or not black), $A^2$ (gender, female or not female)) that is most impacted by unfairness w.r.t. our chosen criterion (separation, sufficiency).\n   - My other possible understanding is that there is indeed only a single binary sensitive attribute $A$, and we are looking for a subset of \"binarized\" attributes (e.g., $X^1 = 1$ or not; $X^5 = 3$ or not) that \"contribute\" the most to the unfairness w.r.t. $A$.\n   - Anyhow, maybe I missed it, but the current wordy & \"flowy\" writing style makes it for me too difficult to comprehend what the precise problem formulation is. I am not saying that you should rewrite everything (I personally like such flowy style), but I feel that a bit more rigorous/terse yet-to-the-point problem formulation (sub)section would help a lot.\n\n**Table 2**\n- For clarity, although elementary, I think it would help if you could show the derivations of the $F(S)$'s in the Appendix.\n- Also, aren't some of them available in closed-form solutions? For instance, with my computations, it seems that $F(S) = \\frac{1}{2 \\sigma^2 |S|} \\left( \\sum_{i \\in S} \\Delta_i \\right)^2$ for Gaussian scenario.", "broader_impact_concerns": "None. The authors have sufficiently addressed this in their Section 6.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "k6Alw2Qwgk", "reviewer": "Reviewer_Z978", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The  paper introduces Conditional Bias Scan (CBS), a new auditing framework for detecting intersectional and subgroup biases in predictive models. The experiments conducted on semi-synthetic data and case studies on real COMPAS show that CBS provides higher bias detection power than existing methods (GerryFair, MultiAccuracy Boost) and reveals meaningful intersectional biases\n\nStrengths \n- The paper is well-written with clear and meaningful contribution.\n- The paper presents an extensive simulation and evaluation (100 semi-synthetic datasets, 1344 experiments) and the proposed auditing framework is compared with too previous frameworks.\n- The proposed auditing framework is used on COMPAS dataset (and German credit in the appendix), which reveals interesting intersectional biases (real-world impact).\n\nWeaknesses\n- The competitors are old: GerryFair (Kearns et al., 2018) and MultiAccuracy Boost (Kim et al., 2019a).\n- “Continuous-valued covariates, X, must be discretized or removed prior to the scan step” (Section 2.2), which may lead to a loss of information.\n- The proposed algorithms lack a formal analysis of their computational complexity.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The experiments conducted on both semi-synthetic data and real-world datasets show the efficiency of the proposed method in comparison with the baseline.", "audience_yn": "Yes", "audience_explanation": "Machine learning researchers, particularly those working on fairness and responsible AI.", "requested_changes": "See the weaknesses.", "broader_impact_concerns": "There is no ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "blCNafwxLY", "reviewer": "Reviewer_JpWq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a new framework, the Conditional Bias Scan (CBS), for detecting intersectional and contextual subgroup biases in predictive models. CBS formulates various group fairness notions (separation, sufficiency, prediction-based or recommendation-based fairness) as conditional independence tests and uses fast subset scanning to efficiently search over exponentially many subgroups. The method estimates counterfactual expectations under a null “no-bias” model using propensity reweighting, computes log-likelihood ratio scores for candidate subgroups, and evaluates significance through permutation tests. The paper includes extensive semi-synthetic evaluations, a detailed COMPAS case study, and an additional experiment on the German Credit dataset.\n\nStrengths\n\n* Clear motivation and strong relevance: The paper addresses a central limitation of current fairness audits (attention to only marginal protected attributes) by proposing a principled approach for discovering intersectional biases.\n\n* Methodological novelty: The unification of group fairness criteria into the CBS framework, combined with multidimensional fast subset scanning, is technically original and builds meaningfully on prior work (e.g., Bias Scan).\n\n* Comprehensive empirical evaluation: The semi-synthetic experiments systematically vary bias type, strength, subgroup size, and noise parameters. Detection accuracy comparisons against GerryFair and MultiAccuracy Boost consistently favor CBS.\n\n* Compelling case study: Application to COMPAS reproduces known disparities (e.g., FPR gaps for Black defendants) while revealing more refined intersectional patterns (e.g., Black male non-reoffenders).\n\n* Transparent limitations discussion: The authors appropriately acknowledge assumptions regarding propensity estimation, fairness definitions, and data issues in COMPAS.\n\nWeaknesses / Points for Improvement\n\n* Overstated or imprecise claims. Several statements imply guarantees that exceed what the methods or theory provide: (1) CBS does not reliably identify “the most biased subgroup”; the search is stochastic with only asymptotic optimality guarantees (2) Claims of “significantly higher detection power” are descriptive rather than statistically validated. (3) The framework does not cover “most” group fairness definitions, only those expressible as conditional independence of the form $I\\perp A\\mid (C,X)$. These should be softened to accurately reflect the scope of guarantees.\n\n* Strong modeling assumptions for estimating $\\hat I$. The null expectation $\\hat I$ relies on well-specified (and well-calibrated) propensity score and outcome models. Although the paper notes limitations, no diagnostics (e.g., calibration curves, cross-validated predictive performance, sensitivity analyses) are shown for COMPAS. Estimation error in $\\hat I$ may materially influence scan results and type I/II error rates.\n\n* Permutation testing approximation. Permutation tests treat the estimated $\\hat I$ as fixed across permutations. This neglects uncertainty in the propensity and expectation models; re-estimating them under each permutation would be more principled, though computationally expensive. At minimum, the approximation should be explicitly justified.\n\n* Limited generality of empirical evaluation. The synthetic experiments rely on a single generative family (linear log-odds with Gaussian noise). While varied bias signals are tested, alternative data-generating processes (non-linearities, heteroscedasticity, feature interactions) are not explored. Likewise, only two real datasets are analyzed, both with known limitations.\n\n* Interpretability concerns. Although CBS encourages concise subgroups via a complexity penalty, the paper does not provide guidance on penalty selection or discuss potential instability of subgroup composition under noise or perturbed covariates.\n\n* Missing relevant context. The paper cites intersectionality theory but does not deeply engage with the conceptual gap between sociological intersectionality and statistically defined subgroups. Additional discussion would help prevent overinterpreting statistically detected subgroups as socially meaningful intersectional categories.\n\nOverall Evaluation\n\nThis is a high-quality and timely contribution to fairness auditing. The methodological development is rigorous, the empirical work is extensive, and the case studies provide useful insights beyond traditional fairness metrics. The primary weaknesses relate to scope overstatements and unexamined modeling assumptions rather than fundamental flaws. With modest revisions to clarify guarantees, add diagnostic checks for $\\hat I$, and temper generality claims, this paper would make a strong contribution to the literature on bias detection and subgroup fairness.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Partially. The core technical claims are supported by solid evidence, but several broader or stronger claims are only weakly supported, e.g.,\n\n* “CBS identifies the subgroup with the most significant bias.”\nThe experiments illustrate that CBS finds high-scoring subgroups, but the algorithm only guarantees approximate or asymptotic optimality. No experiments compare the discovered subgroups with true global optima on real data.\n\n* “CBS has significantly higher detection power.”\nThe semi-synthetic evaluation supports comparative performance, but the paper does not perform statistical significance tests across methods or explore a wide diversity of generative models. Results are convincing within the tested scenarios, but generality is overstated.\n\n* “Robust empirical evaluation.”\nThe simulations are detailed, but robustness claims are limited by:\none generative family (linear log-odds + Gaussian noise),\nlimited real-world datasets (COMPAS and German Credit),\nno stress testing under model misspecification or dependence violations.\nThus the evidence is informative but not “robust” in the broad sense.\n\n*  Claims about “reliability” of subgroup detection.\nReliability depends on accurate estimation of $\\hat I$, but the paper presents no calibration or diagnostic analyses for propensity scores or outcome models on COMPAS. The method may underperform if these models are mis-specified.\n\n* Interpretation of sociological “intersectionality.”\nThe paper meaningfully detects statistical subgroups, but equating these with sociological intersectionality is a conceptual leap. Evidence does not support that CBS captures the structural dynamics described in intersectionality theory.\n\n* Complexity and interpretability claims.\nWhile CBS applies a complexity penalty, the paper does not show experiments on subgroup stability or interpretability trade-offs.", "audience_yn": "Yes", "audience_explanation": "Yes. At least some of TMLR’s audience would be interested in this paper.\nThe work addresses an active and important topic in ML (auditing models for subgroup and intersectional bias) which is highly relevant to researchers in fairness, responsible AI, and model evaluation. CBS extends existing subgroup-fairness methods with a more general and efficient framework, and the findings on COMPAS and German Credit provide practical insights that many TMLR readers would care about. Overall, the problem, methodology, and results fall squarely within areas of ongoing interest in the TMLR community.", "requested_changes": "Clarify and temper overstated claims about optimality and reliability.\nClaims such as “CBS identifies the subgroup with the most significant bias” and “reliably detects intersectional biases” should be revised to reflect the stochastic, approximate nature of the scan and its dependence on model specification.\n\nProvide clearer justification and diagnostics for estimating $\\hat I$.\nSince CBS relies heavily on accurate propensity and outcome models, the paper should include basic diagnostics (e.g., calibration plots or performance metrics) to support their adequacy in the real-data experiments.\n\nExplicitly discuss the limitations of the permutation-testing approximation.\nThe paper should acknowledge that $\\hat I$ is held fixed across permutations and explain the implications for valid $p$-values.\n\nReframe broad statements about generality and robustness.\nGeneral claims (“most fairness definitions,” “robust evaluation”) should be aligned with the actual scope of the experiments and fairness notions covered.", "broader_impact_concerns": "Not sure, sorry", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)