Beyond Prediction: Managing the Repercussions of Machine Learning Applications
Authors: Aline Weber
Blossom Metevier
Yuriy Brun
Philip S. Thomas
Bruno Castro da Silva
Venue: NeurIPS 2025 poster ()
Keywords: Classification algorithms
Repercussion-aware optimization
Data-driven guarantees
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Machine learning models are often designed to maximize a primary goal, such as accuracy. However, as these models are increasingly used to inform decisions that affect people's lives or well-being, it is often unclear what the real-world repercussions of their deployment might be—making it crucial to understand and manage such repercussions effectively. Models maximizing user engagement on social media platforms, e.g., may inadvertently contribute to the spread of misinformation and content that deepens political polarization. This issue is not limited to social media—it extends to other applications where machine learning-informed decisions can have real-world repercussions, such as education, employment, and lending. Existing methods addressing this issue require prior knowledge or estimates of analytical models describing the relationship between a classifier's predictions and their corresponding repercussions. We introduce Theia, a novel classification algorithm capable of optimizing a primary objective, such as accuracy, while providing high-confidence guarantees about its potential repercussions. Importantly, Theia solves the open problem of providing such guarantees based solely on existing data with observations of previous repercussions. We prove that it satisfies constraints on a model's repercussions with high confidence and that it is guaranteed to identify a solution, if one exists, given sufficient data. We empirically demonstrate, using real-life data, that Theia can identify models that achieve high accuracy while ensuring, with high confidence, that constraints on their repercussions are satisfied.

Beyond Prediction: Managing the Repercussions of Machine Learning Applications
Aline Weber" University of Massachusetts Amherst, MA 01003, USA al ilinweebercs. umass. edu
Blossom Metevier" University of Massachusetts Amherst, MA 01003, USA
bmetevier@umase edu
Yuriy Brun University of Massachusetts Amherst, MA 01003, USA brunêcs. umass. edu
Philip S. Thomas
Bruno Castro da Silva University of Massachusetts Amherst, MA 01003, USA bsilvaêcs. umass edu
University of Massachusetts Amherst, MA 01003, USA pthomas@cs umass. edu
Abstract
Machine learning models are often designed to maximize primary goal, such as accuracy. However, as these models increasingly used to inform decisions that affect people's 's lives or well-being, is often unclear what the real-world repercus- sions of their deployment might be be-making it crucial to understand and manage such repercussions effectively. Models maximizing user engagement on social media platforms, e.g., may inadvertently contribute the spread of misinformation and content that deepens political polarization. This issue is not limited to social me- dia-it it extends to other upplications where machine learning-informed decisions can have real-world repercussions, such as education, employment, and lending. Existing methods addressing this issue require prior knowledge estimates of analytical models describing the relationship between a classifier's predictions and their corresponding repercussions. We introduce THEIA, novel classification algo- rithm capable of optimizing a primary objective, such as accuracy, while providing high-confidence guarantees about its potential repercussions. Importantly, THEIA solves the open problem of providing such guarantees based solely on existing data with observations of previous repercussions. We prove that it satisfies constraints on model's 's repercussions with high confidence and that it is guaranteed to identify a solution, one exists, given sufficient data. We empirically demonstrate, using real-life data, that THEIA can identify models that achieve high accuracy while ensuring, with high confidence, that constraints their repercussions are satisfied.
Introduction
Machine learning (ML) models are widely applied to real-life tasks, ranging from high-stakes applications such lending, hiring, and criminal sentencing, everyday applications, such as product recommendations and personalized advertisements. These models are often designed with the primary goal of maximizing the accuracy of their predictions. However, in applications where such models inform decisions that have an impact on people's lives or well-being, it is crucial to effectively manage the potential real-life repercussions of their deployment.
Models designed to maximize user engagement on social media, for example, have been shown to con- tribute to political polarization and the spread of misinformation [32 15]. This issue extends beyond
Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

social media, affecting various applications where ML-informed decisions have real-worls repercus- sions, such as in education, employment, and lending. Previous work addressing this problem requires access to analytical models describing the relationship between a classifier's predictions and its reper- cussions [48]]]]]]]]]] 27). Zhu et al. [8], for example, assumes access to a model of how posts influence user opinions in order to optimize post-recommendation strategies that minimize polarization. However, this relationship is often difficult to characterize analytically. Designing classification algorithms that can mitigate repercussions without knowledge of such models remains open problem.
We introduce THEIA, novel classification algorithm that addresses this open problem. THEIA does not require analytical models of the repercussions of model's 's predictions, Instead, it operates under the weaker assumption of having access only data containing observations of the repereussions of a previously deployed classifier.
Social media example. As running example, consider social media platform using classification algorithm to aid in predicting which posts to present to a user to maximize engagement. These predictions may have unintended repercussions, such as increasing political polarization. Taking repercussions into account is important- the platform may need (or be required) to maximize user engagement while managing the possible repercussions of the posts it presents. This could mean imposing constraints on metrics related to political polarization (e.g., changes in engagement or sentiment shift) after posts are presented to users ] Unfortunately, existing methods for this problem require analytical models of how social media posts affect political polarization. Constructing such models-
-.gg e.g., through causal modeling techniques- challenging: many complex factors (e.g.. social, economic) influence the impact of the presented posts on political polarization. THEIA, by contrast, works without a model describing the relationship between predictions and their repercussions: it can ensure that constraints on the possible repercussions are satisfied with high confidence, provided the social media platform can collect data regarding the political polarization resulting from the posts presented (or recommended) by previously deployed classifier.
Contributions. We present THEIA, a novel method capable of managing the repercussions caused by a model when the analytical relationship between model predictions and their repercussions is not known. We prove that 1) the probability that THEJA returns a solution that satisfies constraints on possible repercussions is at least (1 8). where 8 is user-specified confidence level; and 2) THEIA consistent; intuitively, identifies and returns a model that satisfies the constraints (if one exists), with high confidence as more data is observed. We empirically analyze THEIA'S 's performance in two rial-life settings, while varying both the amount of training data and the amount of repercussions that classifier's i predictions have. We show that THEIA can identify accurate solutions while ensuring, with high confidence, that constraints on its potential repercussions are satisfied.
2 Problem formulation
THEIa's goal is to identify a high-accuracy predictive model while ensuring, with high confidence, that the repercussions of its deployment satisfy user-specified constraints. THEIA is designed to achieve this using only existing data collected during the deployment of previous model, including empirically observed repercussions of its decisions. Concretely, we assume access to dataset where each 7th data point contains feature vector X₁, a label Y,, and a prediction Y,S made by previously deployed stochastic classifier B. We call 33 the behavior model, defined as 8(x, f)= We assume 3 was trained by the same user who now seeks to improve it, so 3 is known, by construction, in our setting.
Consider a model trained to predict the type of content that will maximize user's time spent on platform. These predictions can be used to decide which posts to show to a particular user. One possible repercussion of deploying this model could be, e.g., how many times that user later interacted with extremist content online -a downstream effect of the content they were shown based on the model's predictions. More generally, we define a repercussion as a real-valued, instance-specific quantity that can be empirically observed after deploying model and using its predictions to make decisions. Let R. be the real-valued random variable representing the repercussion associated with the ithh instance, observed after the model 3 makes a prediction Y In the example above, Re is the empirical observation of how many times the 7h user interacted with extremist content after being
For more examples of important real-life applications where THEIA can be applied, see Appendix A

---

shown posts selected based on the model's prediction P.a. We adopt the convention (w.loo.g.) that smaller values of Rº correspond to more favorable repercussions. We append R. to each data point and define the dataset sequence of n independent and identically distributed (i.i.d.) data points: D=-X,, Y, P,, )= We denote an arbitrary data point in D by suppressing the subscripts.
Our goal is to construct classification algorithm that takes as input D and outputs new model To that is as accurate as possible while enforcing high-confidence constraints on repercussions If deploying model results in repereussions that satisfy all user-specified constraints, such repereussions are referred to as accepiable, and the model is deemed repercussion-awarr The form of the new model is (2 ú) Pr(V =ÿ) X=#), where my is parameterized by a vector 0 E θ (e.g., the weights of neural network), for some feasible set θ and where ye is the prediction made by my given X. Like R.. R is the empirically observed repercussion if the model outputs the prediction Y:Te.
Quantifying repercussions. As stated previously, we assume the repercussions of interest are measurable. Specifically, we assume there are k repercussion objectives, 9j θ - R,j E I..... k. that take as imput the parameters 9 of a classifier and return a real-valued measurement of its repercussions. We adopt the convention (w.l.o.g.) that a classifier is repercussion -warre iff g, (,) - 0 for all j. To simplify notation, we first investigate the setting with a single repercussion objective (i.e., A = 1), and later show how to enforce multiple repercussion objectives (Algorithm 3
In this work, we investigate repercussion objectives aimed ensuring that the repercussions of new model To will not exceed a threshold т. Specifically, we consider cases where each repercussion objective is based on a conditional expected value of the form
g(0) =
-T,
where E R is a tolerance and YYX,Y) is a Boolean conditional; see below for more details. Appendix D shows how THEIA can enforce repercussion objectives beyond this form. These include, e.g., high-confidence guarantees that repercussions are approximately equal across different demographic groups, as well as objectives that, instead of controlling the conditional expected value, enforce high-confidence constraints on the repercussion variance, median, or conditional value risk.
To help interpret (T. consider a platform aiming to maximize user engagement while mitigating polarization. Assume that sentiment scores are used as a proxy for polarization. Users are often segmented into groups based on factors such as age or political affiliation; c(X, Y). in this case, could be a group membership variable. The platform wishes to ensure that the sentiment resulting from predictions made by a new model Th will be lower than T. R is the sentiment score after showing a post to a use and r is the repercussion threshold the platform does not want to exceed. This could be application-specife (e.g, r=0.3 implies moderate
ptttimmen//ollrizationn 7 could also be the current average sentiment of a particular group: 7= _ Ei= (Y)[(, where [.] is the mi
Iverson bracket and is the number of people in the group. Requiring that the sentiment induced by Ty less than can be modeled by a constraint E[R x)= S T. This can be expressed in the form (I by rewriting it as { c(X, Y)=1J- -T. Then, g()) < 0 iff the future sentiment (im- pacted by ne's 's predictions) is at most T. Additional constraints can be added to include other groups.
Algorithmic properties of interest. We wish to ensure that g < 0 since this implies that 0 (the model returned by a classification algorithm) has acceptable repercussions. However, this is often not possible, as it requires extensive prior knowledge of how predictions influence repercussions. Instead, we aim to design an algorithm that is capable of reasoning about its confidence that g()) I O using only available data. That is, we wish to construct a classification algorithm, a, where al a(D) E θ is the model returned by a when given dataset D as input, that satisfies constraints of the form
Pr(g(a(D)) s 0) 2 2-88 8,
(2)
where 5 E (0, 1) is the admissible probability of returning a model with unacceptable repercussions. Algorithms satisfying (2) are called Seldonian 42 Notice it may be impossible to enforce all constraints simultaneously or there may be insufficient data to satisfy the constraints with the required confidence. THEIA reasons about its own uncertainty and determines when this is the case.
"Our algorithm works with other performance objectives, not just accuracy. See Appendix D for a detailed discussion how THEIa can address other settings and objectives.
Many indicators of polarization have been proposed 7 45 31
We use sentiment scores example.
3

---

In such situations, it proactively notifies the user that no repercussion-awee aware solution can be provided, returning "No Solution Found" (NSF), rather than producing a model it does not trust This can be achieved by letting NSF E ② and (NSF) = 0.
Our goal, then, is to design classification algorithm with two properties: 1) the algorithm satisfies (2) and 2) the algorithm is consistent, i.c., if a nontrivial repercussion-aware model exists, the probability that the algorithm returns a solution other than NSF converges to one as the amount of training data increases. In Section 4 we prove that our algorithm, THEIA, satisfies both properties.
3 Enforcing repercussion constraints
Recall that a repercussion-awee aware algorithm must ensure with high confidence that the repercussion objective satisfies g(0) I O, where 0 is the returned model and g(0) E| [[[
r. Since THEDA only has access to data collected from a previously deployed model, B, the only available samples are of R the repercussions of 3's predictions. Below, we show (1) how one can construct i..d. estimates of R using samples collected using 3: (2) how confidence intervals can be used to derive high-confidence upper bounds on g0); and (3) the pseudocode for our algorithm.
Deriving estimates of repercussions. Recall that the distribution of empirically observed reper- cussions in D results from I's predictions. However, our goal is to evaluate the repercussions of a different model, me. This is challenging: given empirically observed repercussions from predictions of a previously deployed model B, how to estimate the repercussions oa were used instead? One solution is to run Ta on held-out data. However, this would only produce predictions yTo, not their corresponding repercussions; i.e., each sample's repereussion would still be in terms of B, not ma.
We solve this problem using off off-policy evaluation methods from the RL literature, which use data collected under one rocision-making model to estimate what would have happened under a different one. Specifically, we importance sampling M obtain new random variable, Rm constructed using data from (9, such that Re is an unbiased estimator of R under the standard assumptions of importance sampling (see Appendix B). For each instance in D, the estimator fi weights the
observed repercussions Rd based on how likely the prediction 8 is under my. my would make the label 9 more likely, R is given a larger weight; otherwise, R is given a smaller weight. Formally, the estimator is ft"*=mo(X. Y )/, y Theorem establishes that this estimator is unbiased.
Theorem 1. R is an unbiased estimator of RT E E [( - R |c(X, Appendix
Proof. See
Bounds on repercussions. We now discuss how to derive high-confidence upper bounds on g()) using unbiased estimates of g(0) and confidence intervals. While different confidence intervals for the mean can be used, we consider Student's (-test [40] and Hoeffding's 's inequality (20). Consider vector
of m i.i.d. samples (zz) of a random variable Z; let the sample mean be = 1 Em Z, the sample
m
standard deviation be o (Z1, Zm
m 1 (Z, Zy, and 8 E (0,1) be a confidence level.
Property 1. E Zg is normally distributed, then Pr (E|Z.) 2 Z - o(Zg.......)
t1 -5, mmmm) 15mmmm))
vmn
1 5, where i 11-8,m-1 is the 1 1-8 quantile of the Student's 's & distribution with m-1 1 degrees of freedom. Proof. See the work of Student (40).
"This is particularly important one of the groups lacks sufficient data to satisfy its corresponding constraint with high confidence. In such cases, THEIA abstains from returning model and instead outputs NSF, rather than producing model it cannot trust. This built-in safeguard prevents the algorithm from favoring well-represented groups at the expense of those with less data, helping to avoid unintended disparities in performance treatment. "Importance sampling estimators, in general, may suffer from high variance. In our setting, two properties prevent this in practice. First, although importance sampling can require data exponential in the horizon, our classification setting involves single decision per instance (predicting label), making variance independent of horizon length. Second. THEIA actively rejects candidates that diverge too far from the current model: such candidates produce large importance ratios, which yield wide confidence intervals for the estimated repercussions and cause them to fail THEIA's 's repercussion awareness test (Alg. I lines 4 1-.). Thus, models that could lead to high variance naturally filtered out during the search since they induce unreliable repercussion estimates.

---

Property can be used to obtain high-confidence upper bound for the mean of Z: vm 5m-mm-1-- Let 9 be a vector of i.i.d. and unbiased estimates of (0).. Once computed (using importance sampling), these are provided to Ussert to derive high- confidence upper bound on g()): Pr(E[At*]c(( Y)] Usimu (G)) 2 1-ô. Our strategy for deriving high-confidence upper bounds for repereussion objectives is general and other confidence intervals can be used. Student's t-test may be used and holds exactly if the distribution of Σ Zi is normal. In Appendix we describe a bound based on Hoeffding's inequality (20), which replaces the normality assumption with the weaker assumption that Ò is bounded, resulting in different upper bound, Ultett--
Ustest (Z,,.., 7m) = + d(Z
C
Complete algorithm. Algorithm| 1 provides pseudocode for THEIA. has three main steps. First, the dataset 1) is split between D. and D (line 1). In the second step, candidate selection (line 3), De used to find and train a model, called the candidate solution, 0z. The cost of candidate solution (line 3) is computed by Algorithm 2 (detailed in Appendix G). which leverages the high-confidence upper bounds introduced above In the repercussion-awarenes awareness test (lines 4-9), D, is used to compute unbiased estimates of g(O)) using the importance sampling method described above. These estimates are used to calculate a --)-cfndneeeeee upper bound, U, on g(O.), using Hoeffding's inequality or Student's t-test (line 8). Then, U is used to decide whether 0g or NSF is returned (line 9).
Algorithm 1 THEIA
Input: 1) 1) ((X,, Y, P.P. R,)) confidence level 6; 3) tolerance value r; 4) behavior model B3 and 5)
Bound E (Hoeff, ttest]..
Output: Model 0, or NSF.
I: De, D, partition(D);;
2: nD, Length(Dj)
x- ()
3: 0g f arg mingee cost(0, De, r, Bound, IDD))
for E (I, nD,) do
5: Let (X,, Y, 5, Jta)) ) be the j data point in Dy
6: if c(X,, Yj) True then g.append *e. (X, (X,, 9. R2 - end if
7: end for
8: if Bound is Hoeff then = ( else Bound is ttest then Urrest end if
9: U 2 0 then return NSF else return 0g
4 Theoretical results
This section shows that 1) THEIA is guaranteed to satisfy the probabilistic constraints defined in ( and 2) given reasonable assumptions about the repercussion objectives, THEIA is consistent. Recall we wish to compute confidence intervals to bound ((0.)." where 0, is the model returned by candidate
selection. We assume that the requirements related to Student's t-test (Property D Hoeffding's inequality (Appendix C are satisfied. Let Avg(Z)= 1 <nz _ Z; be the average of size n2 vector Z.
Assumption 1. If Bound is Hoeff, then for all E {1,..., k). each estimate in 9; (in Algorithm 3 is bounded in some interval (ay,by) by}. If Bound is ttest, then each Avg(ū;) is normally distributed.
Theorem 2. Let (9)))=1 be sequence of repercussion objectives, where 9j O - R, and let (6,))=1 be corresponding sequence of confidence levels, where each 65 E (0, 1). Assumpption 1 holds, then for all j {L,...,), k),, Pr(g)(a(D)) < <0) 2 d5. Proof. See Appendix
THEIA satisfies Theorem - if the solutions it produces satisfy (1....,k), k). Pr(g;(((D)) 0) 2 T &y, where a Algorithm Because Algorithm is an extension of Algorithm [ to multiple constraints, it suffices to show that Theorem holds for Algorithm 3
i.c., if Vj
Next, we show that THEIA is consistent: when repercussion-aweer model exists, the probability that
THEIA returns a solution other than NSF converges to as the amount of training data goes to infinity.
'THELA mitigates potential high-varianee issues by penalizing candidate models that diverge substantially from 3. as these yield wide confidence intervals unlikely to pass the repercussion awareness test (App. Our experimental results are consistent with this we did not observe high- variance issues, even in low-data regimes. THeLA works with any confidence intervals for the mean.
5

---

Theorem 3 (Consistency guarantee). If Assumptions 7 hold (see Appendix F for 20. then lim Pr(a(D) + NSF, g(a( S 0) Proof. We extend the proof strategy of Metevier et al. 229 from the contextual bandit setting to the supervised learning setting with high-confidence constraints. The detailed proof for this theorem is provided Appendix
Theorem Brelies on mild assumptions (Assumptions 20:
1) the function used to evaluate models is smooth; 2) at least one repereussion-awanl model exists that is not on the scceptable-repercussiod boundary; and 3) a model's sample performance converges to it expected value given enough data.
5 Empirical evaluation
We empirically investigate three research questions: RQ1: Does THEIA enforce repercussion con- straints with high probability, while existing algorithms do not? RQ2:
What is the cost (e.g., in terms of accuracy) of enforcing such constraints? RQ3: How does THEIA perform when predictions have little influence on the repercussions of interest relative to factors outside of the model's control?
In our experiments, we examine two real-life use cases of THEIA focused on ensuring fair repercus- sions, i.e., that deploying the model will benefit different demographic groups approximately equally Recent research has shown that ML models can impact different demographic groups differently [ST. In our first experiment (EXP-1). 1). classifier makes predictions about whether youth in the U.S. foster care system are likely to get a job. These predictions may affect a person's 's life if, c.g., they influence financial aid decisions. EXP-1 uses two data sources from the National Data Archive on Child Abuse and Neglect [. which include financial, educational, and well-being data on youth over time and dur- ing their transition from foster care to adulthood. The feature vector X contains five attributes related to the job and educational status of a person in foster care, including their race. The goal is to predict a binary label, Y. denoting whether person has full-time job after leaving the program. The behavior model, 33, is a logistic regression classifier. We explore our research questions in a setting where clas- sifier predictions may have different levels of repercussions. To do so, consider parameterized definition of a measurable repercussion. Let R." be the observed repercussion on person i's life if a classifier s outputs the prediction 8: given X,. Here, (a,, 0) = Pr((." = X,=x). We define R. as
a.p. (1 (22.0.5) if X = 0 a (1=
R
33)
if XT 1,
where XT¹ is the race of person i, and a regulates whether a model's repercussions are strongly affected by its predictions (high prediction-repereussii repercussion dependency) or if they have little to no repercussions (low prediction-red reprrrcusoo-reerussiioooo dependency). As a goes to zero, predictions have no repercussions. We vary a from 0 to 1 in increments of 0.1, and ve is defined as the behavior model, 3. Intuitively, Equation 3 captures the idea that if a youth is predicted to get a job, they are more likely to receive financial aid leading to less severe observed repercussions. That is, R. decreases when D:" 1. Notice that both the mean and variance of the repercussion resulting from a given prediction vary by race. This reflects the fact that youth from different racial backgrounds may experience systematically different outcomes, c.g., due to structural or social biases. This form aligns with the empirical findings of Chetty et al. [10], who show, using real-world data, that the distribution of financial outcomes (i.e., repercussions) differs across racial groups even when starting from similar conditions (e.g., having the same prediction P:7) ). Notice that Equation 3 characterizes the setting we investigate and is used solely to generate challenging, parameterized datasets with varying levels of cnddiction-reprruussiii dependency. THEIA does not have access to it, nor does it depend on or assume its particular form.
In our second experiment (EXP-2), a bank's lending decisions are informed by a classifier predicting repayment success. Such decisions have repercussions clients' lives, affecting their financial well-being, savings rate, or debt-to-income ratio after a lending decision. EXP-2 uses real-life financial information for 250,000 clients who requested loans [44). The feature vector X contains attributes such as client's 's age. monthly income, debt-to-income ratio, number of open loans, number of dependents, and various delinquency measures. The goal predict a binary label Y, denoting
$ *Our notion of repercussion fairness differs qualitatively from standard static fairness definitions. Static fairness metrics rely on performance metrics like false positive rate and can be determined by evaluating a model on a validation set. Repercussion fanneresssionffareesss.. by contrast, is more general and requires deploying the model and empirically observing/meassuingg its repercussions.

---

whether a client will miss any payments by more than 90 days. THEIA'S primary objective is to approve loan requests for clients unlikely to miss payments while accounting for the possible repercussions of lending decisions across age groups (attribute X"): younger individuals who are often not yet financially stable, for example, may experience a greater impact on their financial well- being from receiving new loan than more senior individuals. In this experiment, R is the observed repercussion on client i's financial well well-being if a classifier 36 outputs the prediction Y; given X, For a thorough discussion, see Appendix [H] Importantly, in EXP-2 there exists a non-linear relationship between predictions and their repercussions due to the Law of Diminishing Marginal Utility of Income: as income increases (e.g., through an approved loan), the benefits of further increasing decrease non-linearly [28]. Furthermore, this experiment models a challenging scenario where distribution shift causes the repereussions of ML models to change between the deployment of the baseline classifier (behavior model) THEIA aims to improve upon and the time when THEIA searches for repercussion-avee -aware model. Specifically, it models a scenario where significant societal changes have altered the relationship between predictions and their repercussions, reflecting, for example, how loan decisions may impact clients differently over time. This allows us to empirically evaluate THEIA's 's robustness in cases where statistical properties of the data change after the training data was collected.
In both E and EXP-2, we wish to guarantee that the repercussions of the new model, ITa are smaller (i.e., less severe) than those of the current model, B. In particular, THEIA'S 's goal is to identify an accurate predictive model that is repercussion-awann
ensuring its repercussions are not 2). To do so, define two repercussion objectives, Du and 91- Let t E {0, 1 and ge (0)))=EE(R" E =t] Tt, where T.= n. is the average observed repercussion caused by 3 on people of race (or age group) X"=t and where n= Ei=1 Xx- The confidence level & for all objectives is 0.1.
detrimental to particular race (EXP-1) 1) age group (EXP-2).
11 Ei=1 [X]==
100%
100%
Failure hate Canstraint
Failure Rate of Constraint
80%
Fatteam of Faitram wim Eqennee Palmss Cosstrants with
ineam roble
60%
60%
40%
40%
Masimses debrainiiie
with PC
20%
CSA with Coims Theie
20%
0%%
10a
10a
1.0
10a
10a
Amount of Training Data (n)
Amount of Training Data (n)
Figure 1: Failure rates w.r.t. repercussion constraint [Left:: EXP- Right: EXP-2]. Black lines show the maximum admissible probability of failing to enforce the constraints (6g= -60%).
RQ1: Managing repercussions. We first investigate if THEIA can effectively satisfy repercussion constraints while existing algorithms fail to do so. A key novelty of THEIA is its ability to ensure reper- cussion fairness using only data from a previously deployed classifier, without relying on analytical models describing how predictions affect repercussions. While bodel-based approaches exist, ours is, to the best of our knowledge, the first capable of satisfying repercussion constraints in the fully model- free setting. As such, there are no direct baselines that operate solely on data collected from a previ- ously deployed classifier. We therefore compare THETA to well-established fairness methods that make assumptions aligned with ours and that also rely only on observed data. In particular, we compare with 1) Fairlearn (2). 2) Fairness Constraints [46]. and 3) quasi-Seldonian algorithms (QSA) (4). We consider five classic fairness constraints: demographic parity (DP), equalized odds (EqOdds), disparate impact (Dislmp), equal opportunity (EqOpp), and predictive equality (PE)
sate--o-thheann model-free
We also compare THEIA to a simple baseline: logistic regression (LR).
We first examine how often each algorithm fails to satisfy repercussion constraints. Let the failure rate be the probability algorithm returns solution violating constraint. We measure this by evaluating the solutions returned by each algorithm on a larger dataset not used during training. In EXP-1, we also investigate how varying the peredicciooneeeeerneseeiei dependency, o, influences failure rates. We focus on one representative experiment with a=0.9 due to space constraints. Similar behavior observed for other a values. See Appendix H for full results and implementation details.
Figure I shows each algorithm's 's failure rate, as a function of training data size, in each experiment (EXP-I and EXP-2). We computed failure rates and standard errors over 500 trials. Figure depictss
depicts
We task each competing method with enforcing the constraints analyzed in those methods' original papers.
7

---

failure rates with respect to a representative repercussion constraint; complete results are in Ap- pendix H. Notice that the solutions returned by THEIA always satisfy both repereussion constraints [o] This is consistent with THEIA's 's theoretical guarantees, which ensure with high probability that solutions it returns satisfy all constraints. Other methods, by contrast, either 1) always fail to satisfy both constraints; or 2) fail to satisfy one constraint while only occasionally satisfying the other.
RQI: Our experiments show that, with high probability, THEIA identifies repercussion-awan models satisfying all user-specified constraints, while alternative methods fail to do so.
RQ2: The cost of managing repercussions. Depending on the problem. there may be trade-off between satisfying repercussion constraints and optimizing accuracy. In Appendix D we show how THEIA can satisfy repercussion constraints while bounding accuracy loss. We now investigate the impact that enforcing repereussion constraints has on accuracy. Figure 2 shows the accuracy of classifiers returned by different algorithms as function of n. in EXP- I; the results of EXP- (Appendix H) follow the same pattern. We bounded accuracy loss via an additional constraint requiring that returned models have an accuracy of at least 75%. Under low-data regimes, THEIA'S accuracy is approximately 90%, whereas competing methods (with no repercussions guarantees) exhibit slightly better performance above 90%. As increases, THEIA'S accuracy reaches that of the other techniques. Importantly, although competing methods may occasionally achieve slightly higher accuracy than ours, they consistently return models that violate the constraints the repercussions. THEIA, by contrast, ensures that all constraints are satisfied with high probability and always returns models with accuracy above the specified threshold (see Figure
1
100%
100%
Probaliility Solution
faikkam hankkam Expetl Construints
80%
Accuracy
60%-
60%
0SA
40%
0SA EgOrds
CSA EgOpp
40%
054
054 mp
20%
0%
0%
10a
10a
10a
Amount of Training Data (n)
Amount of Training Data (n)
Figure 2: [Left:: EXP-1] Accuracy of the models returned by algorithms (subject to different constraints) function of n. [Right: EXP-1] Probability that these algorithms return a solution.
trade-off might also exist between the amount of available training data and the confidence that models satisfying all constraints can be identified. Recall that some methods (including THEIA) may choose not to return a solution if they cannot confidently ensure that all constraints are satisfied. We now study how often each method identifies candidate solution as function of Figure 2 shows that THEIA starts returning models that satisfy all constraints, with high confidence, when given a num- ber of samples corresponding just 3.1% of the available data, and it is capable of consistently return- ing models (with 90% probability) when given a number of samples corresponding to just 12.5% of the available data. As increases, the probability of THEIA returning models increases rapidly. Although Fairlearn, Fairness Constraints, and LR return model regardless of the amount of training data, these models never satisfy both constraints (see Figure D. QSA often returns candidate models with less training data than THEIA; these models, however, also fail to satisfy both constraints simultaneously.
While there is cost to managing a model's repercussions depending on the setting, THEIA suc- ceeds in its primary objective: ensuring, with high confidence, that repercussion constraints are satisfied without requiring unreasonable amounts of data, and while limiting accuracy loss.
RQ2:
RQ3: Varying prediction-repereustie dependeney. Finally, we investigate THELA'S performance (in terms of failure rate, probability of returning solutions, and accuracy) in settings with varied levels of prediction-repeet reppercoonnrreerussiionn dependency. These include challenging cases where predictions have little influence on the observed repercussions, relative to other factors outside of the model's 's control.
We first study THELA'S 's failure rate for different values of Figure shows that THEIA never returns solutions that fail to satisfy the repercussion constraints, independent of a, confirming empirically that
TTheEA returns a model only when it can ensure, with high confidence, that all constraints are satisfied. In EXP-1, 1, this requires just =11.% of the available data; in EXP-2, it requires only =z 0.1%.
8

---

its -robbabbiii guarantees hold in settings with wide range of qualitatively different observed repercussion characteristics. Next, we investigate how often THELa identifies and returns a model for various values of o. If predictions have little to no influence on observed repereussions (i.e., low o), it becomes difficult to distinguish minor repereussions from noise-making making harder for THEIA to confidently identify models that satisfy all repercussion constraints. As expected, this reduces the likelihood of returning a solution (see Figure 3b). THEIA returns models for all > given sufficient data, but this probability decreases noisy repereussions by reasoning about its uncertainty - noise inflates confidence intervals, making it less likely that it will have sufficient confidence in its predictions. It returns model only if enough data is available to offset this effect; otherwise, it proactively warns the user that no repercussion-aweee aware model can be identified. Lastly, investigate how the amount of condiction-reprrusssie affects the accuracy of THEIA's 's models. Figure 3c shows model accuracy for various values of a, as a function of The accuracy trade-off is more evident when THEIA must satisfy challenging repercus- sion objectives, as a approaches zero. In these cases, accuracy decreases from 95% to 90%. Impor- tantly, however, even though trade-off exists, our method remains successful bounding accuracy while ensuring, with high confidence, that all constraints on the model's repercussions are satisfied.
approaches zero. This is by design: THEIA naturally handles
dependency
RQ3:
Our experiments confirm that TIEEA performs well in wide range of settings, with various dependency. Although managing repercussions may impact accuracy
levels of and the probability of finding solutions, these unavoidable trade-offs do not affect THEIA'S 's high confidence guarantees. In our experiments, all returned solutions satisfy both constraints.
100%
100%
100%
Probabi By of Saluton
80%
80%
80%
60%
60%
Accuracy
60%
Macimu Admiedibe
Fail
40%
40%
40%
20%
20%
20%
0%
0%
10a
10a
10*
102
10a
10
10a
107
10
Amount Training
Amount Training Data (n)
Amount of Training Data [n)
(a) Failure rate.
(h) Prob. of returning a solution.
(c) Accuracy.
Figure 3: [EXP-1] THEIA'S performance under various levels of prediction-reppeccusion dependency.
Related work
Our paper addresses the side effects problem, i.e., how to ensure that predictive models are effective without causing unintended consequences 26 37). Below, we discuss relevant work related to ours.
Counterfactual prediction. Counterfactual prediction has been studied in the off off-policy policy evaluation (OPE) [34] [41] and causal inference literature [33]. THEIA uses importance sampling to construct unbiased estimators of model's I's repercussions without having to deploy it. This type of counterfactual reasoning aligns with how importance sampling is used in OPE. Recent work has shown that counterfactual modeling is required if ML-informed decisions act as risk-mitigating interven- tions 27 [2], Methods addressing this setting include, e.g., techniques to manage long term fairesss; importantly, they assume a known analytical relationship between predictions and long-term impact. Constructing these models is challenging: complex factors (social, economic, etc.) may influence, e.g., how financial decisions affect various demographic groups. We, by contrast, investigate a novel model- free method that reasons about the repercussions of predictive models based only on existing data.
Dynamics modeling in content delivery settings. A substantial body of literature focuses on model- ing the dynamics of systems that optimize personalized content delivery. Social media platforms, for instance, use algorithms to determine which posts to present to users maximize engagement (30] 47 16 66. These algorithms typically assume access to a model of how recommendation system Tommasel et al. [43 model the dynamics of echo chambers to construct friend recommendation system that mitigates echo chamber effects by enhancing recommendation diversity, We extend the state of the art by introducing model- free technique that, with high confidence, mitigates repercussions such the exacerbation of existing echo chambers. Carroll et al. [8] investigate a setting similar to ours. They optimize an RS so it does not result in a manipulative or undesirable influence user preferences. This is achieved by training
(RS) may affect user preferences and content exposure 24 23 &

---

dynamics model of how user preferences evolve under new RS policies, and using it to identify "safe" RS policies. THEIA, by contrast, can identify repereussion-avee aware models using a model-free OPE approach and is supported by strong formal guarantees on user-defined notions of repercussions.
Fairness over time. Liu et al. [27] showed that classifiers' predictions that appear fair with respect to static fairness criteria can negatively impact the long-term wellness of the community it aims to protect. Many works addressed the challenge of identifying classifiers that optimize measures of fairness over time [21 22 27
13 [9 Existing methods assume access to a model relating decisions
and their repercussions on different populations (27 47 17]. We showed that it possible to achieve
this same goal using only existing data, without having to learn models titiss--hiiee -which is often challenging or infeasible. Appendix [
discussses how this setting may be mapped our mathematical framework.
Seldonian algorithms. THEIA extends the existing body of work on Seldonian algorithms [42], which provide high-confidence guarantees on user-defined metrics of interest, such fairness and
safety, and have shown strong performance in real-world applications [42 (2)]. They also provide
straightforward way for users define multiple constraints on metries of interest [42]. THEIA is
the first classification Seldonian algorithm capable of providing high-confidence guarantees that constraints on the repercussions of deploying a predictive model will be satisfied.
7 Conclusions
We introduced THEIA, a novel approach to mitigating the real-life repercussions of deploying ML models. Unlike existing methods, it does not rely on complex analytical models describing the relation- ship between a classifier's predictions and their repercussions. THEIA, by contrast, is the first method that can provably provide high-confidence guarantees on a model's 's repercussions using only available data. Importantly, it can reason about its own uncertainty: if there is insufficient data to satisfy all constraints with the required confidence, it proactively notifies the user that no safe solution can be provided, rather than returning a model it does not trust. A promising direction for future research is ex- tending THEIA to settings with complementary assumptions about the statistical properties of training data, such as those where the relationship between predictions and repercussions evolves over time.
8 Acknowledgments
This work is supported by the National Science Foundation under grant no. CCF-2210243.
References
[1] Administration on Children, Youth & Families. National Data Archive on Child Abuse and Neglect (NDACAN). 2021. URL httpps//:s//w httpa///a/aaanaaaaaa...mmaa wwn.nndannaa.. .hss gov/
[2] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach.
A reductions approach to fair classification. In 35th International Conference on Machine Learning, pages 60-69, 2018.
[3] Jason Altschuler, Victor-Emmanuel Brunel, and Alan Malek. Best arm identification for contaminated bandits. Journal of Machine Learning Research, 20(91):1-39, 2019.
[4] Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint /606. 06665,66666560 2016.
[5] Robert Bartlett, Adair Morse, Richard Stanton, and Naney Wallace. Consumer-lending discrim- ination in the FinTech era. Journal of Financial Economics, 145(():30-56 2021.
[6] Elisabetta Biondi, Chiara Boldrini, Andrea Passarella, and Marco Conti. Dynamics opinion polarization. IEEE Transactions Systems, Man, and Cybermetics: Systems, 2023.
[7] Dennies Bor, Benjamin Seiyon Lee, and Edward J Oughton. Quantifying polarization across political groups on key policy issues using sentiment analysis. arXiv preprint arXiv:2302.00775. 2023.
10

---

[8] Micah D Carroll, Anca Dragan, Stuart Russell, and Dylan Hadfield-Menell Estimating and penalizing induced preference shifts in recommender systems. In International Conference on Machine Learning, pages 2708. PMLR, 2022.
[9] Yash Chandak, Scott Niekum, Bruno Castro da Silva, Erik Learned-Miller, Emma Brunskill, and Philip S Thomas. Universal off-policy evaluation. Advances in Neural Information Processing Systems, 34:27475-27490 2021.
[10] Raj Chetty, Nathaniel Hendren, Maggie Jones, and Sonya R Porter. Race and economic opportunity in the United States: An intergenerational perspective. The Quarterly Journal of Economics, 135(2):71--78,, 2020.
[11] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2):153- 2017.
[12] Amanda Coston, Alan Mishler, Edward H Kennedy, and Alexandra Chouldechova. Counter- factual risk assessments, evaluation, and fairness. In Proceedings of the 2020 conference fairness, accountability, and Iransparency, pages 582-593, 2020.
[13] Alexander D' Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, D Sculley, and Yoni Halpern. Fairness is not static: Deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference on Fairness, Accounlability, and Transparency, pages 525- 534, 2020.
[14] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. Proceedings of the 3rd Innovations Theoretical Computer Science Conference, pages 214-226, 2012.
[15] Miriam Fernandez, Alejandro Bellogín, and Iván Cantador. Analysing the effect of recom- mendation algorithms on the spread of misinformation. In Proceedings of the 16th ACM Web Science Conference, 2024.
[16] Noah E Friedkin and Eugene Johnsen. Social influence and opinions. Journal of mathematical sociology, 15(3-4):193-2 1990.
[17] Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge. Wenwu Ou, et al. Towards long-term faimess in recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pages 445 453, 2021.
[18] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems 29, pages
2016.
[19] Hoda Heidari, Vedant Nanda, and Krishna Gummadi. On the long-term impact of algorithmic decision policies: Effort unfairness and feature segregation through social learning. In 36th International Conference Machine Learning, pages 2692-2701, 2019.
[20] Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.
[21] Lily Hu and Yiling Chen. A short-term intervention for long-term fairness in the labor market. In Proceedings of the 2018 World Wide Web Conference, pages 1389-1398, 2018
[22] Lily Hu and Yiling Chen. Welfare and distributional impacts of fair classification. arXiv preprint arXiv:1807.01134 2018
[23] Ray Jiang. Silvia Chiappa, Tor Lattimore, András György, and Pushmeet Kohli. Degenerate feedback loops in recommender systems. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 383-390, 2019.
[24] Dimitris Kalimeris, Smriti Bhagat, Shankar Kalyanaraman, and Udi Weinsberg. Preference amplification in recommender systems. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 805-815, 2021.

---

[25] Ramtin Keramati, Christoph Dann, Alex Tamkin, and Emma Brunskill. Being optimistic to be conservative: Quickly learning a CVaR policy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4436 4443, 2020.
[26] Victoria Krakovna, Laurent Orseau, Ramana Kumar, Miljan Martic, and Shane Legg. Penalizing side effects using stepwise relative reachability. arXiv preprint arXiv:1806.01 01118806011888 186, 2018.
[27] Lydia T Liu, Sarah Dean, Esther Rolf. Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In 35th International Conference on Machine Learning, pages 31 3150-3158, 2018.
[28] A. Marshall. Principles of Economics: An Introductory Volume. Macmillan, 1920.
[29] Blossom Metevier, Stephen Giguere, Sarah Brockman, Ari Kobren, Yuriy Brun, Emma Brun- skill, and Philip Thomas. Offline contextual bandits with high probability fairness guarantees. In Advances in Neural Information Processing Systems 32, 2019.
[30] Cameron Musco, Christopher Musco, and Charalampos E Tsourakakis. Minimizing polarization and disagreement in social networks. In Proceedings of the 2018 world wide web conference, pages 369-378, 2018.
[31] Christopher Musco, Indu Ramesh, Johan Ugander, and R Teal Witter. How to quantify polariza- tion in models of opinion dynamics. arXiv preprint arXiv:2110.1 10.1911,, 2021.
[32] Royal Pathak, Francesca Spezzano, and Maria Soledad Pera. Understanding the contribu- tion of recommendation algorithms misinformation recommendation and misinformation dissemination on social networks. ACM Transactions on the Web, 17(4):1-26, 2023.
[33] Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016.
[34] Doina Precup, Richard S. Sutton, and Sanjoy Dasgupta. Off-policy temporal-different learning with function approximation. In 18th International Conference on Machine Learning, pages 417-424, 2001.
[35] Anton Proskurnikov and Roberto Tempo. A tutorial on modeling and analysis of dynamic social networks. part i. Annual Reviews in Control, 43:65-79, 2017.
[36] Ingo Rechenberg and Manfred Eigen. Exolutionnsttrategi:: Optimierung technischer systeme nach prinzipien der biologischen evolution. Frommann-Flolzboog Stuttgart, 1973.
[37] Sandhya Saisubramanian, Ece Kamar, and Shlomo Zilberstein. A multi-objective approach to mitigate negative side effects. In Proceedings of the twenty-ninth international conference on international joint conferences artificial intelligence, pages 354-361, 2021.
[38] Hans-Paul Schwefel. Numerische optimicrung von computer-modellen mittels der evolution- sstrategie. 1977.
[39] Pranab K. Sen and Julio M. Singer. Large Sample Methods in Statistics: An Introduction with Applications. Chapman & Hall, Boca Raton, FL., 1993.
[40] Student. The probable error of mean. Biometrika, 6(1):1-25, 1908.
[41] Philip S Thomas. Safe reinforcement learning. PhD thesis, University of Massachusetts Libraries, 2015.
[42] Philip S Thomas, Bruno Castro da Silva, Andrew Barto, Stephen Giguere. Yuriy Brun, and Emma Brunskill. Preventing undesirable behavior of intelligent machines. Science, 366(6468); 999-1004, 2019.
[43] Antonela Tommasel, Juan Manuel Rodriguez, and Daniela Godoy. I want to break free! recommending friends from outside the echo chamber. Proceedings of the 15th ACM Conference Recommender Systems, pages 23-33, 2021.
12

---

[44] Will Cukierski Credit Fusion. Give Me Some Credit Dataset, 2011. URL https://wwi
kaggle. www.coomcccpp.eee wwt.pppeii.iiin/iiiieeeeeeeeeaaa
[45] Muheng Yang, Xidao Wen, Yu-Ru Lin, and Lingjia Deng. Quantifying content polarization on twitter. In 2017 IEEE 3rd international conference collaboration and internet computing (CIC), pages 299- 308. IEEE, 2017.
[46] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In 20th Artificial Intelligence and Statistics, pages 962-970, 2017.
[47] Xueru Zhang, Mohammad Mahdi Khalili, and Mingyan Liu. Long-term impacts of fair machine learning. Ergonomics in Design, 28(3):7-11, 2020.
[48] Liwang Zhu, Qi Bao, and Zhongzhi Zhang. Minimizing polarization and disagreement in social networks via link recommendation. Advances in Newral Information Processing Systems, 34: 2072-2084, 2021.
13

---

Appendix
A Other motivating examples
In our paper we discussed two important real-life examples where our method could be applied. First, we introduced motivating problem where social media platform wished to take into account the repercussions of presenting users with different posts. Secondly, in our empirical evaluation, we considered the repereussions of providing financial aid to youth in foster care. Here, discuss three additional examples of possible applications of THEIA in real-life settings:
Consider university that has a 1-on-1 tutoring program. However, the university has limited resources and cannot offer tutoring for all students. To select which students should participate in the program, the university' s decision is based on GPA predictions. This can have repereussions in students' lives: receiving tutoring (or not) can influence the chances of student graduating from college.
Consider police department deciding which crime prevention strategy to use in each district of a city, based on predictions about crime recidivism. This could have repercussions the form of, e.g., changes in the average incarceration rate after this decision.
Assume that medical decisions are influenced by predictions of whether person qualifies for high-risk care management. These predictions may have a repereussion on a person's health; e.g., the severity of chronic illnesses after receiving the treatment.
B Proof of Theorem
Proof. At high level, we start with E[ c(X, Y)] and apply a series of transformations, standard probabilistic identities, and general properties of the setting (see below) to arrive at R c(X, Y) To simplify notation, throughout this proof we let C' c(X,Y) Also, for any random variable Z, let supp(Z) Z) denote the support of Z (e.g.. if 7 is discrete, then supp(Z) {= Pr(Z 2) > 0}). To begin, substitute the definition of RT and expand this expression using the definition of expected value:
(x 99)) R C
E E =E
(4)
3(X,80)
To 1)
Σ Y= Y-,, Y ="
(5)
3(x, 9
Using the chain rule repeatedly, we can rewrite the joint probability in 5) as follows:
Y
(6)
(7)
=
PrXX=2,y=1()))
(8)
Furthermore, recall that our algorithm addresses a classification problem. In standard machine learning settings, the predicted label depends only on the input features provided to the model. As result, conditioning the label's probability on any additional variables not seen by the model does not change that probability. That is, Pr(Y ==) Y=y,C) Pr(r " X=x), which is the definition of ((x, v)).
We perform this substitution and simplify by canceling out the 3 terms:
Y-ylCC (z, 9)
E EL [C] = Σ Pr ( (R== Y-v, 8 8-- C.
(9)
3(x, Ú)
(s..s.rr)saupp(XYY)) X, XYY
= - Pr R²=r| Yo 8 89-0. c) Pr (YXx=, Tz D)..
(10)
14

---

Note that (e( 1) can be rewritten as "x Using the multiplication rule of probability. we can combine this term with the Pr(X=x,Y=y|C) term (TO) obtain the joint probability Pr(X=z, yy =))) Furthermore, notice that in the setting we investigate, predictions' repercussions do not depend on how a prediction was made. That is, does not matter, e.g., if a neural network or a random forest determined which posts to show to a user. Formally, this means that Vx, y/, r, Pr(R"=r|X=z,
=r Y=y, P= P=)=ğ). For this rea- son, we can substitute Pr( R X=x, Y=y. y PD= C) for
Y=y, P===ū, C).
We substitute these terms into 10 and apply the multiplication rule of probability once more:
=
(11)
Exupp(XX)X
=E Y=y, R
(12)
Finally, we make the standard assumption in the importance sampling literature that predictions made by can, in principle, be evaluated using the available data. This means that predictions occurring
that are possible under Ty (the model being evaluated) have a non-zero probability under 3 (the classifier used to generate training data). In other words, for all æ and y, (2 y) > 0 implies that ((x,, y) > 0. This assumption trivially satisfied in our setting: we consider predictive models 0 E θ that naturally satisfy this condition, such as standard stochastic classifiers that assign non-zero probability to all outputs (e.g., Softmax layers neural networks). Under this assumption, supp(Y"*) S supp(P'"). and so supp( R²*)) S supp( R So, we can rewrite (12) as
y R =r(=r))
(13)
(2.8.ū.)) (...B..))emupp((.. (X.X, v, Y,Sou,
By the definition of expectation, this is equivalent to [ Therefore, we have shown that C]..
C Bounds on repercussions using Hoeffding's inequality
This section focuses on how one can use unbiased estimates of g(0) together with Hoeffding's inequality (20] to derive high-confidence upper bounds on g(0). Given vector of m i.i.d. samples (Z)) of random variable Z, let Z = i Em Z; be the sample mean, and let 8 E (0, 1) be confidence level.
Property (Hoeffding's Inequality). Pr [a, b]) then
In(1/6)
Pr
E Z > 2 - (b- v)v
2 1 1
(14)
2m
Proof. See the work of Hoeffding (20).
L
Property can be used to obtain high-confidence upper bound for the mean of Z:
log(1/6)
Uneett (1,, Zz, ..., Zm) = - (b (b-a))
(15)
(2m)
Let ý be vector of i.i.d. and unbiased estimates of g(f). Once these are procured (using importance sampling as described in Section B}. they can be provided to Unaatt derive high-confidence upper bound on y0))
Pr
E
n
v(x.))
Unoate (0)
(16)
Notice that using Hoeffding's inequality to obtain the upper bound requires the assumption that ý bounded.
15

---

D Extensions of THEIA
In this section we discuss how THEIA can be extended to provide similar high-confidence guarantees for the regression setting and for repercussion objectives beyond the form assumed in ①
D.1 Repercussion-awarrene guarantees in the regression setting
In our problem setting, we study pppercussion-wwarcee in the classification setting, in which the labels Y produced by a model are discrete. However, our method can also be applied in the regression setting, where a (stochastic) regression model produces continuous predictions Y, instead of discrete labels. To use THEIA in this setting, one may adapt Algorithm [2s0 that uses a loss function suitable for regression; e.g., sample mean squared error. Furthermore, notice that the importance sampling technique described in Section 3 is still applicable in the regression setting, requiring only minor changes so that it can be used in such a continuous setting. In particular, the importance sampling technique we described can be adapted by replacing summations with integrals, probability mass functions with probability density functions, and probabilities with probability densities. By doing so, all results presented in our work (e.g., regarding the unbiasedness of the importance sampling estimator) carry to the continuous case. Notice, finally, that in order to apply THEIA in the regression setting, the behavior model, 8, and the new candidate model, Tg. must be stochastic regression models this is similar the assumption we made when addressing the classification setting (see the discussion in Section 2)
D.2 Alternative definitions of repereussion objectives
Until now, we have assumed that the repercussion objectives take the form of (I) However, this can be restrictive when the user's 's notion of repercussion-awareeess requires a different definition of repercussions. Below, we discuss how users of THEIa may construct other repercussion objectives, and how our formulation (shown in (T) is related to the definitions introduced in the work of Liu al. 27
Connections to selayed-impace fairness and the work of Liu et al. (27]. Consider the lending scenario described in Liu et al. [27], in which bank's objective is to maximize loan repayments, and its lending decisions are informed by classifier that predicts repayment success. For simplicity, the population of loan applicants consists of two mutually exclusive groups, A and B (e.g.. based on race or gender). The repercussions of lending decisions are multi-fold payment defaults not only reduce the bank's profit, but worsen the financial situation of the borrowers, whereas successful repayments lead to profit for the bank and an increase in the borrowers' financial welfare.
In their work, Liu et al. [27] define long-term improvement and long-term stagnation as repercussion objectives that do not cause long-term harm. Specifically, they use Aμ, to represent the difference in repercussions (such as changes in financial welfare) for group j E {A, B} between a previously deployed model and a new model. If Ap, O, then the new model has resulted in long-term harm.
To enforce long-tern fairness in our framework, we assume that the training dataset provided has data instances of the form (X, Y, y R²), where X is a real-valued vector describing information about loan applicant, Y indicates whether a loan should have been approved, T E (A, B} is the group indicator, ÿ is the prediction previous classifier 3 made given X, and R is the resulting change in credit score of the individual, which Liu et al. (27] use as a proxy for financial well-being in their work. To map this example to our framework, we can set 5 to be group T's average credit score R
under the current model (i.e., under the behavior model, (3) and E
T = j] to be the expected
credit score of group j under the new model, ma. Then, Apy-
E n T
Enforcing general definitions of repercussion objectives To enforce repercussion objectives beyond (1. the importance sampling technique introduced in Section 3 can be combined with techniques presented in the work of Metevier et al. (29. As a concrete example, consider the lending scenario described in the previous section. Assume that instead of long-term improvement or stagnation, the bank is interested in ensuring (with high probability) that the repercussions of classifier's 's predictions are approximately equal for loan applicants in groups and B. This can be represented by the repercussion objective grair (0)
E
A] E
R R T
B]
16

---

To satisfy with the repereussion objective Stair in place of g. the techniques in this paper and the bound-propagation methods introduced by Metevier al. [29] can be combined. At a high-level. THEIA would first compute (as before) unbiased estimates of E R [T A] and E [R [T = B] using the importance sampling technique described in Section [ Then, it would use the bound- propagation methods introduced by Metevier et al. 29 to obtain high-confidence upper bounds on (0)
The discussion above corresponds to just one example of how to deal with alternative repercussion objectives: in this particular example, Ilair- The same general idea and techniques can also be used to tackle alternative repercussion objectives that users of THEIA may be interested in IT] For example, in settings where one wishes to minimize repercussions (rather than merely constrain them), repereussion threshold can be chosen based on target value deemed sufficiently low- thus ensuring that any returned model also achieves near-optimal repercussion values. All other parts of the algorithm would remain the same-e, e.g., the algorithm would still split the dataset into two, identify a candidate solution, and check whether passes the www.rrcssionawwaaeenen test.
Beyond conditional expectation. In Section we assume that g is defined in terms of the conditional expected value of the repercussion objectives. However, other forms of repereussion awareness might be more appropriate for different applications. For example, conditional value at risk (25) might be appropriate for risk-sensitive applications, and the median might be relevant for applications with noisy data 3. Chandak al. [9] introduce off off-policy evaluation methods that produce estimates and high- confideneee bounds for different distributional parameters of interest, including value at risk, conditional value at risk, variance, median, and interquantile range. These techniques can be combined with ours to obtain high-confidence upper bounds for metrics other than the conditional expected value of 1e
E
Proof of Theorem 2
This section proves Theorem which is restated below.
Theorem
Let (g)) be a sequence of repercussion objectives, where 9j 0 - R, and
let (6,) be corresponding sequence of confidence levels, where each d3 (0, 1). If
Assumption holds, then for all E (l,....k).
Pr())(a(D)) S 0) 2 83.
(17)
In what follows, we consider the same conditions as in Theorem 1 which we restate below for completeness:
Conditioning the probability of a label on variables not observed by the classification model does not affect that probability.
The repercussions of a prediction do not depend on how a prediction was made- it does not matter, e.g., if a neural network random forest determined which posts to show to user.
Predictions made by me can, in principle, be evaluated using the available data. Specifically, predictions that are possible under тe (the model being evaluated) have zenzzero probability of occurring under 3 (the classifier used generate training data).
We first provide three lemmas that will be used when proving Theorem 2
Lemma 1. Let 9j be the estimates of g constructed in Algorithm 3 and let Dj. be subdataset of Dy such that a data point (X,Y, 88, R²) is only in Ds. if o(X, is true. Then, for all E O the elements in 9j are i.i.d. samples from the conditional distribution of 93 given c(X,Y).
"This statement holds assuming that the repercussion objective of interest satisfies the requirements for the gound-propagaan technique to be applicable; for example, that the repercussion objective can expressed using elementary arithmetic operations (e.g.. addition and subtraction) over base variables for which we know unbiased estimators (29]. In the case of the repercussion objectives discussed in this paper, we can obtain unbiased estimates of the relevant quantities using importance sampling, discussed in Section 3
17

---

Proof. To obtain Oj. each data point in D/. is transformed into an estimate of g using the importance sampling estimate xx(X, 80) R (Algorithm 3 lines 5-8). Since each element of فJ B(X.92)
computed from a single data point in Dr.- and the points in D are conditionally independent given c(X, Y), it follows that each element of 9, is conditionally independent given c(X, Y). So, each element of 0j can be viewed as an i.i.d. sample from the conditional distribution of 93 given ((X,Y).
Lemma 2. Let ûj be the estimates of g constructed in Algorithm 3 It follows from Theorem that for all 0 E O, each element in 9j an unbiased estimate of g
Proof. We begin by considering the expected value of any element in .y:
na (X, 83)
R (xxx)
E
R c(X,Y)
=E
(18)
3(X, Ya)
33x,, Yo)
Remo c(X,Y) y)]
(19)
E [ |c(X, Y)]
(20)
(0)
(21)
Expression (20) follows from Theorem estimates of (0).
Therefore, for all 0 E θ the elements of وj are unbiased
Let Og be the model returned by candidate selection in Algorithm 3 (line 2), and let U, be the value of U at iteration j of the for loop (lines 4-10).
Lemma 3. If Lemmas 7 and 2 hold, then the upper bounds Uj calculated in Algorithm satisfy vj E (1....,k) k). Pr(g, (0 U,) 55-
Proof. We begin by noting that by Lemma]) I the data points used to construct each each (1-6,) (1-6,)- bound, i.e., the data points in each タj. are (conditionally) i.i.d. Because 0. E θ, by Lemma 2 we know that each element in ÔJ is an unbiased estimate of gj (0.). Therefore, Hoeffding s inequality or Student's 1-test can applied to random variables that are (conditionally) i.i.d 12 and unbiased estimates of (0.). Moreover, under Assumption 1 when Bound is Hoeff, the requirements of Hoeffding's s inequality are satisfied (Property [ and when Bound is ttest, the requirements of Student's 's f-test satisfied (Property [ Therefore, the upper bounds calculated in Algorithm satisfy Pr(g)(0.) > U,)= I 85.
Proof of Theorem 2
Proof. To show Theorem 2 we prove the contrapositive, i.e., Vj k), Pr(s, 0) s 6y.
Consider the event Vj f1,...,k, (a()) 0. When this event occurs, it is always the case that a(D) * NSF (by definition, g(SF)) = 0). That is, a nontrivial solution was returned by the algorithm, and for all j. U, < 0 (line 11 of Algorithm B. Therefore, (22) (shown below) holds.
Pr(g)(((D) 0) (a(() 0,U, <))
(22)
s (Pr(o;((DD)) > U,)
(23)
Pr(gg U2)
(24)
(25)
Expression 23
is a result of the fact that the joint event in (2 implies the event (g, (a()) > U,).
We substitute 8g for a(D) in (24) because the event Vj E {I...., k), (_...,)),((((())) (a())) 0 implies that
nontrivial solution, or a solution that is not NSF, was returned: a(D) = tc. Lastly, (25) follows from
Lemma 3 This implies that Pr(g,
(((9)(((()) (a(D) 0) s 5, Vj E {1,..., k). completing the proof.
"aSmpllss that are conditionally i..dd given some event E' can be viewed as i.i.d. samples from the conditional distribution. Applying the confidence intervals to these samples provides high-come conficoonedence bounds on the conditional expected value given the event E, which is precisely what we aim to bound.
18

---

F Proof of Theorem 3
This section proves Theorem restated below. We build upon the proof strategy introduced by Mete- vier et al. [29] extending it from the contextual bandit setting to the supervised learning setting with high-confidence constraints. Extending their proof to our setting involves the following changes:
1. Changes related to the output the function used to calculate the utility of a model: Metevier et al. (29] consider utility function that returns the sample reward of policy. Instead, our utility function (Algorithm 2 outputs the sample loss of a model.
2. Changes due to the form of the constraints: The form our repereussion constraint differs from the more general form of the constraints considered by Metevier et al. [29]. This results in a simplified argument that our algorithm is consistent.
We present the complete proof, with these changes incorporated, below.
Theorem Assumptions
4 hold, then lim Pr(a(D) + NSF, 3(a(D))=0)==1
We begin by providing definitions and assumptions necessary for presenting our main result. To simplify notation, we assume that there exists only a single constraint and note that the extension of this proof to multiple constraints straightforward. before, we consider the same conditions as in Theorem 1 which we restate below for completeness:
Conditioning the probability of a label on variables not observed by the classification model does not affect that probability.
The repercussions of prediction do not depend on how a prediction was made it does not matter, e.g., if a neural network or random forest determined which posts to show to user.
Predictions made by Tа can, in principle, be evaluated using the available data. Specifically, predictions that are possible under Ty (the model being evaluated) have non non-zero probability of occurring under 3 (the classifier used to generate training data).
Recall that the logged data, D, is random variable. To further formalize this notion, let (Ω, E, E,p) be a probability space on which all relevant random variables are defined, and let D,, Ω - D random variable, where D is the set of all possible datasets and D, De U Dy. y. We will discuss convergence as n -> Co. (w)(w) a particular sample of the entire set of logged data with n data points, where E f2
Definition 1 (Piecewise Lipschitz continuous). We say that function f M + E metric space (M, d) is piecewise Lipschitz continuous with Lipschitz constant and with respect to a countable partition, M2,,,,,,... of M if is Lipschitz continuous with Lipschitz constant K on all metric spuces in (M,
Definition (6-covering). If (M,d) d) a metric space, a set C M a d-covering of (M, d) if and only if max min d(z, < 8.
yEM
Let D.) denote the output of a call to Algorithm 2 and let c(0) == Lmax + g(0). The next assumption ensures that e and are piecewise Lipschitz continuous. Notice that the 8-covering requirement is straightforwardly satisfied if θ is countable or θ C I for any positive natural number m.
The feasible set of policies, θ equipped with metric, de, such that for all De(w) there exist countable partitions O Oc = (Of, O.....,, and O ( Or,,.,, where cl-) and c(., D.(w)) are piecewise Lipschitz continuous with respect to O and O respectively with Lipschitz constants K and K. Furthermore, for all i N and all 0 there exist countable 8-covers of O and O
Assumption
Intuitively, Assumption states that (/) the cost function used to evaluate classifiers is smooth: similar classifiers have similar costs/performances: and (2) each classifier can be described by set of velled parameters, as is the case with all parametric supervised learning algorithms.
19

---

Next, we assume that repercussion-ane aware model, 8", exists such that g(#)) is not precisely on the boundary of acceptable repercussions. This can be satisfied by models that are arbitrarily close to such boundary.
Assumption 3.
There exists an > { and a 8 E θ such that g( S cc
Intuitively, Assumption that the space classifiers not degenerate: at least one repercussion- aware model exists such that if we perturb its parameters infinitesimally, it would not arbitrarily no longer satisfy the repercussion objective. Next, assume that the sample loss, é, converges almost surely to f, the actual expected loss.
Assumption 4. AO E o, E(0, D.) -h, ((0)-
Intuitively, Assumption 4 states that the sample performance of a classifier converges to its true expected performance given enough data. This is similar to the usual assumption, e.g.. in the regression setting, that model's sample Mean Squared Error (MSE) converges to its true MSE given sufficient examples.
We prove Theorem by building up properties that culminate with the desired result, starting with a variant of the strong law of large numbers:
Property 3 (Khintchine Strong Law of Large Numbers). Let {X.) be independent and identi- cally distributed random variables. Then ( E X,) is a sequence of random variables that converges almost surely to E[X1]. E[x1] exists, i.e., +E== X, a.s E[X1].
Proof. See Theorem 2.3.13 of Sen and Singer [39
Next, we show that the average of the estimates of g(0) converges almost surely to g(0):
Property 4. If Lemmas I and hold, then v0 O, Avg(ū) 6.s. g0)).
Proof. Recall that if Lemmas and 2 hold, estimates in 9 are i.i.d and each estimate in g is an unbiased estimate of g()). Also, recall that if ng is the number of elements in 9. Avg(ğ) 1 Rig i=1 9i. Then, by Property we have that Avg(g) g()).
In this proof, we consider the set θ C O, which contains all models that not repercussion-awarre and some that are repereussion-aware but fall beneath a certain threshold: @ = 10 E θ g()) >
-E/2). At high level, we will show that the probability that the candidate model, Oc. viewed as a random variable that depends on the candidate dataset De. satisfies 0, & θ converges to one as n + 0o, and then that the probability that 0g returned also converges to one as n oc.
First, we will show that the upper bounds U (constructed in candidate selection, i.c., Algorithm 2 and U (constructed in the repercussion-awarte awareness test, i.c., Algorithm D converge g(0) for all E O. To clarify notation, we write U (0, Dz) and ((,, Dy) to emphasize that each depends on and the datasets D. and Dy, respectively.
Property 5. If Assumption holds, then it follows Property that for all E O, (0, D.) as, g(0) and (0, Dy)
g(0).
Proof. Given Assumption [T. Hoeffding's inequality and Student's f-test construct high-confidence upper bounds on the mean by starting with the sample mean of the unbiased estimates (in our case, Avg(g)) and then adding an additional term (a constant in the case of Hoeffding's inequality). Thus, UO, Dy) can be written as Avg(ū) + Zn. where Zn, is a sequence of random variables that converges (surely for Hoeffding's 's inequality, almost surely for Student's t-test) zero. So, Zn _ 0, and we need only show that Avg(ğ) as, g(0), which follows from Property 4 We therefore have that U 3.5, g())
The same argument can be used when substituting U+ D.) for U(B, Dy). Notice that the only difference between the method used to construct confidence intervals in the repercussion-awarenn test (that is, U+) and in Algorithm 2 (that is, U/ is the multiplication of Z,, by constant X. This still results in sequence of random variables that converges (almost surely for Student's (-test) to zero.
20

---

Recall that we define e(0, D.) to be the output of Algorithm 2 repercussion-ane aware model # and data D.. a(8", Dz) converges almost surely to E(0*). the expected loss of 0*
Below, we show that given
Property 6. If Assumptions Ladd hold, then it follows from Property 5 that
an, P(O*).
Proof. By Property 5 we have that U (0*) " g(0)). By Assumption 3 we have that g(0*) < -c. Now, let
A= (w E Ω: lim U+(O,, De(w)) = g(0)))
(26)
So, w is in A almost surely, i.e., Pr(w A) = I. Consider any w A. From the definition of limit and the previously established property that g(0*) S -G, we have that there exists an no such that for all n 2 no,, Algorithm 2 will return , Algorithm for values smaller than n2).
Recall that (0"0", - g(0*) means that - U+
De) (this avoids the discontinuity of the if statement in
Furthermore, we have from Assumption that E(00, De) -S E(0*). Let
B {w Ω: lim i(00, De(w)) {(0"))-
(27)
From Assumption 4 we have that w is in B almost surely, i.e., Pr(w E B) = 1, and thus by the countable additivity of probability measures, Pr(w E (An B)) = 1.
Consider now any w E (AnB). We have that for sufficiently large n,, Algorithm 2 will return E(0", D&)
(since w E A), and further that É(O', Da)) - e(0*) (since w E B). Thus, for all w (An ( B), the output of Algorithm 2 converges to {(0*), ), i.e., '(0', Da(w)) + E((*). Since Pr(w E (An B)) = 1, we conclude that e(0*, D.(w)) u.s. E(0))
We have now established that the output of Algorithm Seriithhmmoneeee converges almost surely to E(0*) for the 0* assumed to exist in Assumption 3
We now establish a similar result for all O-that the output of Algorithm converges almost surely to c(0) (recall that c(0) is defined as mmx + (()))
Property 7. It follows from Property 5 that for all 0 E θ, E(0, Dz) 0.S. c(0).
Proof. By Property we have that U+ D2) = g()). If θ, then have that g(0) -E/2. We now change the definition of the set A from its definition in the previous property to a similar definition suited this property. That is, let:
A = (w E Ω: lim U (0, = _(0)).
(28)
n-0
Recall that (+, D2) AS, g(f) means that Pr(limm U+ (+, = g(0) 1. So, w is in A almost surely, i.e., Pr(w A)= = Consider any w E A. From the definition of a limit and the previously established property that g(0) > -E/2, we have that there exists an no such that for all n 2 no Algorithm all w E A, the output of Algorithm 2 converges almost surely l + g(6; that is, c(0, De(w)) A5,
2 will return mmax + + (0, (w(w)) By Property 5 U (0, (c()) as g(0). So, for
lmax + g and since c(f) = <mux + (0),, we therefore conclude that č(0, De(w)) 25 c(0). O
By Property 7 and one of the common definitions of almost sure convergence,
v0 e o, Ve > 0, Pr lim inf(w E Ω |č(0, Dn(w)) c(0)) < <)) = 11
Because is not countable, it is not immediately clear that all 0 E θ converge simultancously to their respective c(0). We show next that this is the case due to our smoothness assumptions.
Property 8. If A'Assmpption 2 holds, then it follows from Property 7 that Ve' > 0.
Pr lim inf(w E Q V9 G, 16(0, (=)w)) c(0) < <)) =1
(29)
n-0
21

---

Proof. Let C()) denote the union of all the points in the d-covers of the countable partitions of è assumed to exist by Assumption Since the partitions are countable and the 8-covers for each region are assumed to be countable, we have that C(6) is countable for all 8. Then by Property we have convergence for all 0 E C(6) simultaneously:
7 for all 6,
A6 > 0, Ve 0, Pr lim inf(w eQ: v0 e C(8), 12(0, De(w)) - c(0) < < =1 (30) 72020
Now, consider a 0 & C(o). By Definition 2 and Assumption 2 3 0' E θ d(,, 0') S Ó. Moreover, because and are Lipschitz continuous on o and @ (by Assumption 2 respectively, we have that Ic(0) - c(0')| < Kб and lē(0, De(w)) - '(6'. De(w)) - K8. So, ((0, De(w)) - c(0) I
|ē (c(w)) c(0') + K8 - |é(0", De(w)) c(0') + 6(K + K). This means that for all d 0:
(v0 E ((0), (e(w)) c()) < = (vo 6, (0 Do(w)) c(0) < S + K))
Substituting this into (30 we get:
A5 > 0, Vc > Pr
lim inf(w E EQ: :2:00 E 6, |c D2(w)) - c(0) <c+ 5(K + +R11) = n-0o0
Now, let 6= = </(K< + K) and e' 2e. Thus, we have the following:
Ve' 0, Pr lim inf(w E Q v0 E G, 10 Da(w)) c(0) < <(}) 1.
x-+00
So. given the appropriate assumptions, for all 0 E θ, we have that 6(6, De(w)) -6, c(0) and that 6(0", (w)) _ f((*). Due to the countable additivity property of probability measures and Property 8 we have the following:
Pr V0 E θ, lim a(0, De(w)) = =c(0) lim a(0", Da(w)) =((00)) 1, (31) n--00
where Pr(A, B) denotes the joint probability of A and B.
Let H denote the set of w E such that (3T) is satisfied. Note that maxx defined as the value always greater than (()) for all 0 O and g()) 2 - for all 0 θ. So, for all w H, for sufficiently large n, candidate selection will not define 0g to be in θ. Since w is in H almost surely (Pr(w E H) = 1), we therefore have that lom Pr(O. & 6) =1
The remaining challenge is to establish that, given 9g E θ, the probability that the repercussion-
awareness test returns 6. rather than NSF converges to one as n + co. By Property S we have that
UOO,, Dy) a.s. g(Oc). Furthermore, by the definition of θ, when 0₂ & Ô we have that g(o)) E/2. So, Ul,,, Dj) converges almost surely to a value less than -E/2. Since the repercussion- aawaeeesss test returns 0_ rather than NSF if U (6,, Dy) < E/4 and U(e,, D2) converges almost surely to value less than -2/2, it follows that the probability that Ufe,, Dy) S E/4 converges to one. Hence, given that 0g & O, the probability that 0g is returned rather than NSF converges to one.
We therefore have that 1) the probability that 0g & θ converges to one - Do and 2) given that Og & Ô the probability that f. returned rather than NSF converges to one. Since 0g & Ô implies that 0. is repercussion-wwant
these two properties imply that the probability that repercussion-aware model returned converges to one as - DC.
G Full Algorithm
corresponds to the cost function used in candidate selection (line 3 of Algorithm [ When given a candidate model, 0, to evaluate, the cost function first uses the training set, D.. to estimate whether the candidate model is likely to pass the repercussions-awaree by using D. to compute an upper bound, U+, on g(l.). If U+ < -E/4, a small negative constant 13
Algorithm 2
test. This is done
We consider -E/4, instead of 0 as the threshold in THEIA to ensure consistency. Appendix F discusses this in more detail.
22

---

Algorithm 2 cost
Input: 1) the vector 0 that parameterizes model π; 2) D. - {(X,, Y;, 4.8 )=)); 3) confidence
level ó; 4) tolerance value т; 5) the behavior model 3; 6) Bound E {Hoeff, ttest); and 7) the number of data points in Dy, denoted D., Output: The cost π.
1: I <- -()
2: for i [l,...,, m} do
if o(X, Y,) is True then @append ma(X,, B(X.,P,)
R² -T) end
4: end for
5:
2; n length(()
6: if Bound is Hoeff then
7: a, b- upper and lower bounds of
8: U+ = EE 9.) + x(baa)/
1og(1/6)
(2mD,)
9: else if Bound is ttest then U+ (2x=1.)) 9.) + a())
yaD, t 118n0
end if
lmax ma E(0', De)
11: if U return E(0, Dz) else return (Lmax + U+)
Algorithm THEIA with Multiple Constraints
Input: 1) D {(X,,, V R. ) 2) the number of repercussion constraints, k; 3) a sequence
of Boolean conditionals (cy) such that for j fl,..., k). cy(,,, Y,) indicates whether the event
associated with the data point (X,, Y. Y R occurs; 4) confidence levels 5 (6));-1 where each 6, E (0, 1) corresponds to repercussion constraint gj: 5) tolerance values T = (ry } where each 7j is the tolerance associated with repercussion constraint g5 6) the behavior model |3; and 7)
Bound E {Hoeff, ttest].
Output: Model 0g NSF.
De: Dy + partition( D)
2: 0g < arg minage cost(O, De. k. 5, 8, Bound, length( Dj))
3: U ()
4: for (1...., k} do
5: 9j + ()
6: for E {1, (I,..., n. do
7: if cX, Yt) is True then و append Toe (X., R -r;) end if 8,)
8: end for
9: if Bound is Hoeff then Uappend(Unoarts (ū;)) else U.append(Uitear(( (())))((()) end
10: end for
11: fjj E (l....,,), < 0 then return 9g else return NSF
Algorithm determines that 0 is likely to pass the repercussion-awareet test, and the cost associated with the loss is returned. Otherwise, the cost of defined as the sum of U+ and the maximum loss that can be obtained on dataset Dr (lines 10-11). This discourages candidate selection from returning models unlikely to pass the repercussion-awarrees
test.
Notice that Algorithm 2 uses importance sampling ratios between two models currently de- ployed classifier, 83, and a candidate model, Ti to compute an unbiased estimate of the repercussion objective, If nothing is known in advance about the relationship between these models, the resulting ratios can be extremely small large, leading to high-variance estimators. However, the candidate models evaluated by THEIA are not arbitrary. THEIA is designed to reject candidates that diverge too much from the current model: such models produce large importance ratios, which lead to wide confidence intervals and typically fail THEIA's 's repercussion-awareee These candidate models, therefore, are filtered out during the search. In particular, wide confidence intervals result in large upper bounds, U+,, on the repereussion objective. These bounds are likely
test (lines 4-9, Algorithm [
23

---

Algorithm 4 cost with Multiple Constraints
i, R,)) 3) the number of repercussion constraints, k; 4) sequence of Boolean conditionals (cy) such that for j (1,.., ki), Cj( (X; Y.) indicates whether the event associated with the data point (X,, V, 8 R occurs; 5) confidence levels = {6y 1 where each dg E (0, 1) corresponds with constraint 95' 6) tolerance values (Ty 1= where each Tj is the tolerance associated with repereussion constraint 9j: 7) the behavior model 3: 8) Bound E (Hoeff, ttest); and 9) the number of data points in D denoted r.
Input: 1) the vector 0 that parameterizes the model n; 2) D. - {(X,, Yo, 8
Output: The cost associated with the model ma.
for
k} do
2: Øj 4 ()
3: for (I, m) do
4:
cx(,, Yi) is True then و append
R - -T)) PP)
end if
5:
end for
6:
Let A = 2; n4 length(0j)
7:
if Bound is Hoeff then
8:
Let a, b be the lower and upper bounds of gj
= 1 (o)))) + A(b A(b-a) alb log(1/6,) (2nD,)
9:
ng,
else if Bound is ttest then o( Vun,
UJ 1 n5
"Ay ( +
1-83 1-5,
12: end if
end for
14: fmax maxo E
15: ifjj (1...., k,,KK,,U E/4 then return ((0, De)
else return
(muxx Σイ U
to exceed the threshold 5. causing the algorithm to assign the maximum possible cost, fmax to high-variance candidates - effectively eliminating them from consideration (line 11, Algorithm 2 In practice, thus, this mechanism serves as way to implicitly constrain importance sampling ratios and control the variance of THEIA's estimates of the repercussion objective.
Furthermore, notice that in Algorithm 2 instead of calculating a high-confidence upper bound on g(0) using Unoett or Uttest, we calculate an inflated upper bound U+, Specifically, we inflate the width of the confidence interval used compute the upper bound (lines 5-9). This is mitigate the fact that multiple comparisons are performed on the same dataset (D.) during the search for candidate model, which often leads candidate selection overestimate its confidence that the model it picks will pass the repercussion -eruussioon-warensssss test. Our choice to inflate the confidence interval in this way, i.e., considering the size of the dataset D, used the repercussion-awrres scaling constant X, is empirically driven and was first proposed for other Seldonian algorithms (42
test and the use of
Algorithm 3 shows THEIA with multiple constraints. The changes relative Algorithm I relatively small: instead of considering only single constraint, the repercussion-awareen over all k constraints and only returns the candidate model if all & high-confidence upper bounds are at most zero. Similarly, the cost function, Algorithm 4 changes relative to Algorithm 2 in that when predicting the outcome of the repercussion epperuusionnawwrrenes awareness test includes this same loop over all
test loops
constraints.
H
Experiments
Recall that in our experiments, we evaluate THEIA in two real-life problems: EXP-1, involving predictions about youth in the U.S. foster care system, and EXP-2, loan-repayment setting. In both, the goal to identify accurate models while ensuring acceptable repereussions for all demographic groups.
24

---

In EXP-1, we consider a classifier that predicts whether youth in foster care will have job after leaving the program. Intuitively, Equation B captures the idea that if youth is predicted to get a job, they are more likely to receive financial aid aid-leading to less severe observed repercussions.
That is, R," decreases when y: 1. Notice that both the mean and variance of the repereussion resulting from a given prediction vary by race. This reflects the fact that youth from different racial backgrounds may experience systematically different outcomes, e.g., due to structural social biases. This form aligns with the empirical findings of Chetty et al. [10], who show, using real-world data, that the distribution of financial outcomes (i.e., repercussions) differs across racial groups even when starting from similar conditions (e.g.. having the same prediction Y,").
In experiment EXP-2, we tasked THEIA with assisting banks and clients in making responsible financial decisions. In this experiment, bank's lending decisions informed by a classifier predicting loan repayment success. These decisions can have repercussions on clients' lives, affecting their financial well-being, savings rate, or debt-to-income ratio after lending decision is made. In this setting, repercussions are not deterministic and can be influenced by various unobserved environmental factors. To account for this variability, our experimental setup incorporates noise models with different means and variances. This approach reflects that repercussions may differ across clients of different ages due unobserved factors such as their professional stability or the likelihood of having a support network capable of assisting in case of economic distress. Younger individuals, e.g.. who are often not yet financially stable, may experience a greater impact on their financial well-being from receiving a new loan than more financially stable senior individuals. Importantly, this setting involves complex non- relationships between predictions and their repercussions due to the Law of Diminishing Marginal Utility of Income: as income increases (e.g., due to an approved loan), the benefits of further increasing it decrease. This relationship is known in economics to be non-linear (28).
Furthermore, this setting presents challenging scenario where distribution shift causes the repercus- sions of ML models to change between the deployment of the baseline classifier (behavior model) THEIA aims improve upon and the time when THEIA searches for a repercussion-awart model, Specifically, it models scenario where significant societal changes have altered the relationship between predictions and their repercussions, reflecting, for example, how loan decisions may impact clients differently over time. This allows us to empirically evaluate THEIA's 's robustness in cases where statistical properties of the data change after the training data was collected.
In all experiments, our implementation of THEIA used ES [36 38 to search over the space of candidate models and the ttest concentration inequality. We partitioned the dataset D into D. and D using a stratified sampling approach where D. contains 60% of the data and D contains 40% of the data.
Experiments were conducted on a computer cluster containing 50 computer nodes with 28 cores (2 processors, 14 cores each- 56 cores with RAM, 200GB local SSD disk, and 50 compute nodes with 28 cores (2 processors, 18 cores each- 72 cores with hyper- threading) Xeon Gold 6240 CPU @ 2.60GHz, 192GB RAM, and 240GB local SSD disk. Each node had 3GB of allocated memory. The 500 trials of each experiment were run in parallel, and the total running time was less than 12 hours.
Xeon E5-2680 v4 @ 2.40GHz, 128GB
We now present the complete set of results for RQI and RQ2: does THEIA enforce repercussion constraints with high probability, while existing algorithms fail to do so; and what is the cost (e.g.. in terms of accuracy) of enforcing repercussion constraints. Figure 4 shows the complete failure rate plots for both constraints across both experiments (EXP-1 and EXP-2). These experiments fully support the conclusions discussed in the main text. Specifically, they confirm that the solutions returned by THEIA always satisfy both repercussion constraints; other methods, by contrast, either (1) consistently fail to satisfy both constraints or (2) satisfy one but not the other.
Recall that in the main text, we presented results on EXP-1 regarding the accuracy of classifiers returned by different algorithms and the probability with which each of them returns a solution as a function of the amount of training data. In Figure we present this analysis for EXP-2. In particular, [egure]]shows on the left, the accuracy of classifiers returned by different algorithms, and on the right, the probability of returning solution, accuracy matches or exceeds that of competing methods while consistently satisfying all constraints (as shown in Figure 4. Furthermore, THELA starts returning models that satisfy all constraints with high confidence when given a number of samples corresponding to just 0.1 of the available data.
function of n. for EXP-2 Notice that THEIA'S 's
25

---

100%
Failure Rate of Constraint
Failure Rate of Constraint!
80%-
80%
60%
Pebkidblly of
sill Putkas Famroa Donatrano
60%
40%
OMA with butkks
40%
CMA wit P
20%
20%
0%
0%
10a
10a
10
10
10
Amount of Training Data [n]
Amount Training Data
Fallure Rate of Constraintt ?
100%
Failure Rate of Constraint. 2
100%
80%
80%
60%
60%
Pede of
40%
Polstire ciracee
40%
20%
20%
0%
0%
10a
10a
102
10a
Amount of Training Data [n]
Amount
Training Data (n)
Figure 4: Failure rates of various methods w.r.t. repercussion constraints as a function of n [Left: EXP-1; Right: EXP-2]. Black lines show the maximum admissible probability of failing to enforce the constraints (ōg = 51 10%).
100%
LR
Probabiliity Solution
80%
Perream with DP
80%-
Terream with Eqpids
Accuracy
60%
QSA with OF
Tammss Conalranta
60%
QSA with Secidd
QSA with Eqpip
40%
QSA with
OSA with Usamp
20%
Thela
0%
0%
10a
102
100
Amount of Training Dala (n)
Amount Training Data (n)
Figure 5: [Left: EXP-2] Accuracy of the models returned by algorithms (subject to different constraints) function of n. (Right: EXP-2] Probability that these algorithms return solution as a function n.
consistently returns models (with 100% probability) when given number of samples corresponding to just 0.4% of the available data. Note that although Fairlearn and logistic regression always return a model regardless of training data size, these models never satisfy both constraints. These results demonstrate that even in a challenging setting with non-linear prediction-repercussio dependencies that are affected by distribution shift, THELA still significantly outperforms all competing methods.
Finally, recall that in the main text we evaluated THEIA in EXP-1 under one representative value of a (a 0.9). We now show full results for wide range of values of further support the observation that THEIA is robust with respect to various pediitiion-rperccusiio dependency levels. In particular, we investigate the performance of THEIA and competitors in terms of failure rate, probability of returning a solution, and accuracy, for different values of a and as a function of n. Notice that Fig- ures 6 and 7 present results consistent with the observations made Section 5 that is, the qualitative behavior of all considered algorithms remains the same for all values of a, further supporting our dependency
observation that THEIA outperforms competitors both under high and when classifiers' predictions have little to influence the repercussions.
26

---

-20%
3375
3075
40%
4050
40%
40%
050
0%.
Ancunt Thanang X
Aerount Thoning Data tn!
Arooutt Trating Daca
ATSum olaangg Dota
0 0.1
00%
60%
48%
45%
27%
20%
30"
3P
30
32'
30
1.00%
100%
150
60%
60%
507
66%
20%
25%
307
301
361
32
35'
25
30
Aroun: Thaning Data
Teiling Data
Thating
Atount Thanng Dalatni
8 0.3
20.0
40%
40%
40%
45%
Anoun: Thannn
Amrount Teaing Data
Arouui Trakirg Daca 90
ATeant elttann Dala
0.4
30%
60%
60%
20%
28%
25%
22"
30"
32
127
131
22'
32
20"
Figure 6: Algorithms' performances w.r.t. failure rate (first and second columns), probability of returning a solution (third column), and accuracy (fourth column), as function of and for different values of a. The black horizontal lines indicate the maximum admissible probability of failure, 6 60=0%= All plots use the following legend: with DP .... QSA with EqOdds ...... QSA with EqOpp QSA with PE ....... QSA with DisImp
THEIA
LR QSA
Fairlear with DP
Fairlearn with EqOdds
Fairness Constraints.
27

---

d
3305
305
10%
4050
4010
40%
0%.
Ancun: Thannng Data
Aerount Traning Data
Arourtt Trating Dica
0.6
00%
60%
66%
45%
27%
25%
32"
3P
10
32'
30
1.00%
100%
150
90%
60%
50%
66%
20%
25%
30
361
32
35'
15'
30
AToum: Thaning
Arrzzd Teiking Data
Aroutt Thatinng
ATount Thanng Dalatni
8 0.8
20:5
40%
40%
40%
45%
Anoun: Thannng
Amrunnt Tf Teaing Data In
Arouut Traking Daca
ATeam oltanig Dala
0.9
04%
60%
20%
28%
25%
20"
32"
3P
23"
32
13'
22'
12
1.0
1.0%
100%
332%
105%
KB%
65%
45%
20%
25%
15/
Taaking
Arruzte Tekking Data
Aroutt Taaning
Amount Thannn Datatei
Figure 7: Algorithms' performance w.r.t. failure rate (first and second columns), probability of
returning solution (third column), and accuracy (fourth column), as a function of n and for different values of The black horizontal lines indicate the maximum admissible probability
of failure, & - 10%. All plots use the following legend: with DP QSA with EqOdds _..... QSA with EqOpp QSA with PE ....... QSA with
THEIA
LR
QSA
DisImp - Fairlearn with DP ---- Fairleam with EqOdds
Fairness Constraints.
28

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims are supported by theoretical analyses (Section 4 and empirical evaluation (Section and Appendix H).
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss the keys assumptions (and when they satisfied) required for our method to identify, with high confidence, repercussion-aware solutions. We also investigate the performance and robustness of our approach with respect increasing levels of noise.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
29

---

Answer: [Yes]
Justification: We present our theory using a set of four assumptions and three theorems, with complete proofs provided in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We describe the experiments in detail the main paper, and add more specific details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
30

---

reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The data for EXP-1 can be obtained directly from the NDACAN webpage. The data for EXP-2 can also be obtained online, using the reference provided. The code is not yet publicly available. We are in the final stages of developing a library that will be made publicly available to the community, and expect release it in the coming months. The paper contains all the necessary details for implementing the proposed method. Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/ cubblc/guides////
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
wwtpc//guiiis/eoeeeee Cooeee//iSeSSSSSSSeeeoeee www.:Suubmmiiiiooooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The main details are located in Section of the main paper and Section of the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Details be found in Section 5
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
31

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Details can be found in Appendix H
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httpe:///euiippcceeee httpr//s.....bb...thhhcccccaaaad
Answer: [Yes]
Justification: We not present experiments involving human subjects. Furthermore, comply with all policies outlined by the sources from which the datasets used in EXP-1 and
EXP-2 were obtained.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: In Section we discuss the conditions under which the guarantees provided by our algorithm do not necessarily hold. This could lead to models that are not repercussion- aware if THEIA is deployed even when its requirements not satisfied.
32

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
33

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.coo//aaaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
34

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The paper does not make use of LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
35


Reviews:
[{"review_id": "FZgfIRMeST", "reviewer": "Reviewer_aGjM", "summary": "This paper proposed THEIA, a new classification algorithm that seeks to manage the real-world consequences of deploying machine learning models. Rather than relying on causal models or predefined analytical relationships between predictions and their effects, THEIA uses observational data from a previously deployed classifier to ensure, with high probability, that user-specified constraints on repercussions are satisfied. The authors provide formal guarantees and evaluate the approach in two real-world scenarios involving employment outcomes and lending decisions.", "questions": "While THEIA offers a valuable framework for making conservative decisions based on observational data, its reliance on correlational estimates inherently limits its scope. In particular, it cannot provide causal guarantees or account for unobserved confounding. There are well-known cases where results based solely on correlation are indistinguishable from misleading or spurious ones. It would be helpful for the authors to comment on how THEIA handles such scenarios, or whether safeguards could be incorporated to detect when observational estimates may be unreliable.", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "+ The paper addresses an increasingly important problem, how to reason about the real-world effects of ML models.\n+ THEIA operates without requiring access to a causal relationships.\n+ The algorithm includes strong theoretical backing, including high-confidence guarantees and consistency under mild assumptions.\n- The method fundamentally relies on observational associations, and cannot distinguish causation from correlation. This limits its ability to draw meaningful conclusions when confounding is present.\n- There is no mechanism for detecting whether the estimated repercussions actually reflect causal effects, which may be problematic in practice.\n- THEIA assumes that consequences under a new policy can be estimated using data collected from a previous policy—this stationarity assumption may not hold.", "was_revised": true, "final_justification": "My concerns remain and score stays the same. After reading the rebuttals, it appears that the model heavily depends on the underlying unknown causal models. If the correlations perfectly account for the confounders, then the model may perform well; otherwise, the reliability of the model is questionable.", "rebuttal": "We thank the reviewer for their time and thoughtful feedback. We appreciate their recognition of the novelty and importance of our work, which addresses a key gap in the ML literature and solves an important open problem in the field. We also appreciate their positive comments on the strength and technical soundness of our theoretical contributions.\n\nBelow, we address the reviewer’s questions and comments.\n\n---\n### **Is Theia affected by unobserved confounders?** \nAs pointed out by the reviewer, unobserved confounding—cases where unobserved variables could simultaneously influence both predictions *and* repercussions—could affect the conclusions drawn by Theia. For example, an unobserved variable that simultaneously caused predicted probabilities and associated repercussions to increase could lead Theia to believe that higher predictions caused higher repercussions. \n\nIn the specific setting we study, however, this concern does not arise due to two key assumptions:\n\n- **(1)** Since we are addressing a classic classification problem, predictions $\\widehat Y = \\beta(X)$ depend only on the feature vector $X$, which is observed and available by definition since it serves as input to the prediction model (e.g., a neural network). There are no additional unobserved variables (e.g., $U$) influencing the model's output, beyond its input variables.\n\n- **(2)** The behavior model $\\beta$, which produces the prediction probabilities, is known (as stated on line 71) since it is the model Theia aims to improve upon. \n\nUnder these conditions, no unobserved variables *jointly* influence both $\\widehat Y$ and the repercussions $R$—and so there is no confounding along the prediction pathway. In other words, although $\\widehat Y$ and $R$ *may* be statistically associated (since both $\\widehat Y$ and $U$ influence $R$), $U$ is not a confounder since it does not influence $\\widehat Y$. All non-causal paths from $\\widehat Y$ to $R$ remain blocked by the collider at $R$.\n\nThat said, the reviewer is correct in noting that Theia provides high-confidence guarantees under these assumptions, but not causal guarantees. Causal guarantees would require access to a full causal model—which we assume is unavailable. In fact, our primary motivation is to develop an algorithm that offers meaningful, high-confidence guarantees about repercussions *precisely* in the absence of such models.\n\nWe will extend Section 2 (Problem Formulation) to discuss the points above and further clarify how potential issues related to unobserved confounders are avoided in our setting.\n\n---\n### **How do unobserved variables impact Theia?**\nUnobserved factors that influence repercussions act as noise rather than confounders—in particular, they lead to increased variance in observed repercussions. Theia’s high-confidence bounds are designed to account for this: if the data is too noisy to satisfy user-defined constraints with confidence $1-\\delta$, the algorithm returns *No Solution Found*. This mechanism prevents Theia from returning a model it does not trust, thus acting as a safeguard against noisy observations.\n\nBeyond introducing and formally characterizing the mechanism that safeguards Theia against the effects of unobserved variables, we also empirically evaluated Theia's behavior in such scenarios (Research Question 3). Specifically, we evaluated Theia in challenging settings where predictions had limited influence on repercussions compared to unobserved external factors. Figure 3 shows that Theia returns *No Solution Found* more frequently when $\\alpha$ approaches zero, which corresponds to the scenario described above: one where unobserved factors make repercussions too noisy, leading to higher uncertainty in the estimated repercussions of a candidate model.\n\nThese empirical results support Theia’s robustness in handling scenarios with unreliable repercussion observations and its ability to operate effectively even when noise arises due to unobserved factors.\n\n---\n### **How does Theia perform when repercussions are non-stationary?**\n\nDesigning ML systems that are robust to sudden shifts in dynamics is indeed important, as acknowledged in our Conclusions. However, addressing non-stationarity is not the focus of this paper. *Prior to our work, no methods existed that could satisfy repercussion constraints with high confidence without access to models—**even in stationary cases**.* As shown in our experiments, state-of-the-art competitors consistently fail to tackle such cases, while Theia always succeeds at identifying solutions satisfying all relevant constraints. \n\nThe reviewer correctly points out that in some problems, stationarity may not hold exactly. That said, many real-world problems of interest are either stationary or stable enough for Theia’s assumptions to be valid and its guarantees useful. Our case studies—based on real-world data from the U.S. foster care system and bank lending decisions—illustrate exactly the kind of high-stakes applications where Theia performs effectively and offers meaningful guarantees. Providing formal high-confidence guarantees in these settings, without relying on models, had remained an open problem in the literature, which we solved.\n\nThat said, for completeness, we evaluated Theia’s performance under non-stationary conditions. Appendix H (lines 913–919) describes a challenging experiment where distribution shift alters the relationship between predictions and repercussions—modeling, for instance, how societal changes may cause lending decisions to affect individuals differently over time. Figures 4 and 5 show that in this experimental setting, even though no models are available, Theia remains robust and capable of producing repercussion-aware models even when standard stationarity assumptions are relaxed.\n\nThe above-mentioned empirical analyses of Theia in non-stationary scenarios are presented in the Appendix, since addressing such settings was not the primary focus of our paper. We appreciate the reviewer’s question and will update Section 5 to mention this point and direct readers to the relevant Appendix section for a more detailed discussion and accompanying experimental results.\n\n---\nWe hope the comments and discussion above address your main questions. If so, we would sincerely appreciate your consideration in revisiting your score. If any concerns remain, or if there are additional points you believe we should address to strengthen the work, we would be happy to do so. Thank you again for your thoughtful and constructive review."}, {"review_id": "KX9SoNCECJ", "reviewer": "Reviewer_fNCw", "summary": "This paper introduces a new Seldonian algorithm, Theia, for producing classification models that can satisfy constraints on repercussions derived from a previous model’s performance. The algorithm works by using a historical behavior model and data about the repercussions of that model (for example, levels of polarization induced by a recommendation algorithm). The algorithm allows specification of a probability, $\\delta$, that the repercussion constraint is violated, as well as a tolerance value $\\tau$ to control the tightness of the constraint. The paper uses importance sampling to construct an unbiased estimator of the new model’s repercussions given the historical model’s data. The paper also provides several theoretical guarantees about Theia, including that it satisfies the probabilistic constraint specified and that it is consistent, meaning that it will return a solution satisfying the constraints given enough training data. Through several empirical results, the paper shows that the algorithm does indeed behave as specified across two different settings with repercussions.", "questions": "- Does the accuracy loss in figure 2 occur only for the group from whom the algorithm is attempting to minimize repercussions? Or is it dataset-wide? I am wondering whether the impacted group that one is hoping to help actually ends up suffering in a different way because of lower accuracy.\n\n- In the case where multiple groups have constraints that are trying to be satisfied, what might affect Theia’s efficacy in satisfying those constraints? For example, if I have two groups with constraints, and one has significantly more data, will that one have its constraint satisfied more reliably? Are there ways to mitigate such disparities?", "limitations": "I have discussed some limitations in the weaknesses above.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "This paper’s strengths are its quality, clarity, and thoroughness. The authors have introduced a clear and useful framework for satisfying constraints given historical data on model behavior. The method is novel and will likely be very useful to the fairness community. The empirical results show a clear advantage over other fairness approaches in terms of satisfying the constraints, and the experiments are quite thorough, comparing against several other approaches. The paper lays out several research questions and answers all of them clearly and concisely. This paper would be a valuable addition to the Neurips community.\n\nIn terms of weaknesses, the primary item missing from this paper, in my opinion, is more discussion of the groupwise impacts of Theia’s approach. I have some specific questions below regarding this. Given that we know that models already struggle for accuracy on groups that are most likely to have these repercussions because they are smaller or tend to be less represented in datasets, I am curious as to whether Theia’s efficacy varies across demographic groups in a similar manner. Lack of experiments in this regard is not necessarily a dealbreaker for the paper, but I think it would be strongly improved by some discussion or consideration of the possibility that the performance of the Theia algorithm itself may be in some way biased or unfair. \n\nOverall, though, this is a very well done and thorough paper, and I thank the authors for their wonderful contribution.", "was_revised": true, "final_justification": "I maintain my high opinion of this paper after discussion", "rebuttal": "We thank the reviewer for their positive comments on the novelty and importance of our work, the thoroughness of our empirical evaluation, and the relevance of our proposed framework—the first to provide high-confidence guarantees about potential repercussions based only on historical data. We also appreciate the reviewer’s comment that our paper would be a valuable contribution to the NeurIPS community.\n\nBelow, we address the reviewer’s questions.\n\n---\n### **Theia's groupwise accuracy**\nThe reviewer correctly pointed out that models satisfying repercussion constraints may lead to different accuracy levels across demographic groups. They asked whether, in our experiments, Theia’s accuracy empirically varied across groups. To investigate this, we conducted additional experiments using the real-world dataset from EXP-1 and found that the accuracy of Theia’s models differed by less than 1%, on average, between the two demographic groups.\n\nThat said, in some applications, it may be important to ensure not only that repercussion constraints are satisfied with high confidence, but also that other potential undesirable behaviors—such as large differences in accuracy across groups—are explicitly and formally controlled. Theia fully supports this capability. In fact, as noted in line 289, EXP-1 already makes use of an additional constraint unrelated to repercussions; in particular, a requirement that selected models achieve at least 75% accuracy with high confidence.\n\nConcretely, one way to address the reviewer’s concern within our framework is to include an additional constraint enforcing, with high confidence, that the difference in accuracy across groups does not exceed a specified threshold $\\varepsilon$:\n\n$g_{acc}(\\theta) = |\\Pr(Y=\\widehat Y | T = A) - \\Pr(Y=\\widehat Y | T = B)| - \\varepsilon$,\n\nwhere $\\Pr(Y=\\widehat Y | T = i)$ can be estimated by evaluating the accuracy of a candidate model $\\theta$ on instances from group $i$.\n\nThis type of constraint is naturally supported by Seldonian algorithms (of which Theia is an instance). For further details, please see [42].\n\n\n---\n### **Satisfying constraints under different group sizes**\n\nThe reviewer correctly notes that the amount of available data may differ across demographic groups, which can make it harder to confidently satisfy constraints for underrepresented groups. Theia is specifically designed to handle this scenario in a robust and principled way: it returns a solution only if *all* constraints are satisfied with high confidence. \n\nIn particular, Theia is capable of reasoning about its own uncertainty. If any group lacks sufficient data to satisfy its corresponding constraint with high confidence, Theia abstains from returning a model, opting instead to return *No Solution Found*, rather than producing a model it does not trust. This built-in safeguard ensures that the final model returned by Theia does not favor well-represented groups at the expense of those with less data, thus helping to prevent unintended disparities in performance or treatment.\n\n---\nWe hope the comments above have addressed your main comments and questions. If any points remain unclear, we would be glad to clarify further. Once again, we sincerely thank the reviewer for their time and thoughtful feedback."}, {"review_id": "1oVaIqz6DQ", "reviewer": "Reviewer_DLhw", "summary": "This work presents THEIA, a framework to optimize beyond accuracy and consider repercussions of machine learning applications in the optimization. The algorithm satisfied a constrain called Seldonian, meaning that if no solution can be found with the present data, it will return NSF instead of returning a solution that itself does not trust.", "questions": "Please also refer to the Weaknesses.\n\n1. When will THEIA fails due to the high variance of the importance sampling and what is the time complexity comparing to existing fairness algorithms?\n2. How can the repercussion be mathematically defined beyond fairness? The authors should provide some examples of how repercussions g is defined for other applications. \n3. When the algorithm return NSF, how do you know if it is not the case when the constraints are too strict or the hyper-parameters such as \\beta are ill-selected?\n4. Can THEIA handle objectives that are not limited to the expected behavior? Such as controlling the tail distribution?", "limitations": "I think the Weaknesses reflect the limitations of this work.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 1, "originality": 2, "strengths_and_weaknesses": "Strengths\n\nThe Seldonian constraint and the ability that the algorithm is able to return NSF when the solution is not feasible is a very practical and useful feature.\n\nWeaknesses\n\n1. Importance sampling is hard to be controlled and may have a high variance. The authors did not provide a good justification of how to get over it; instead they just mention in the footnote that the high variance situation was not observed by them. It weakens the value of this work.\n2. The authors overloaded the term repercussion and position THEIA as a general solutuion. However, throughout the paper, the only repercussion demonstrated is the fairness constraints. The claim does not match the results presented in this work. There are many repercussions that a real ML application should consider such as long-term behavior shift, polarization of comments in social media, legal constraints, causality, interpretability, etc. It is the main reason why I give a low score for this work.\n3. As this paper is mainly comparing fairness constraints, there is a lack of comparison against existing fair learning baselines. The authors claim in the footnote that repercussion fairness is different from static fairness but did not compare clearly what is the different and also did not compare with baselines that can perform repercussion fairness.\n4. The different condition that cause BSF is underexplored. It is common that a fair learning algorithm can’t achieve high accuracy and low fairness metrics at the same time. However, the authors did not provide concrete discussions and empirical results with high-level causes on why it is not achievable.", "was_revised": true, "final_justification": "The authors have provided the rebuttal that partially addresses my questions, and thus I decided to increase my rating.", "rebuttal": "We thank the reviewer for their feedback and appreciate their recognition of the practical importance of Theia’s ability to return *No Solution Found* when constraints cannot be satisfied with high confidence, rather than producing models it does not trust.\n\nBelow, we address the reviewer’s questions and constraints.\n\n----\n### **Importance sampling variance**\nThe reviewer is correct that importance sampling estimators may suffer from high variance. However, in our setting, two key properties ensure that this concern does not materialize in practice:\n1. In general decision-making settings, importance sampling may require data exponential in the horizon. The classification setting we address, by contrast, involves a *single decision* per instance: determining its label. This makes ours an ideal setting for using importance sampling, since variance is independent of the decision horizon.\n2. Theia was actively designed to incorporate mechanisms that control potential variance issues (see Footnote 4 and App.G, lines 857–869). In particular:\n   * To estimate a model’s repercussions, Theia computes the ratio of the probability the candidate model assigns to a label to the probability assigned to the label by the deployed model.\n   * The concern raised by the reviewer would be justified if nothing were known in advance about the relationship between these models. Then, the resulting ratios could be extremely small or large, leading to high variance.\n   * However, the candidate models evaluated by Theia are *not* arbitrary. Theia is designed to actively reject candidates that diverge too much from the current model. In particular, such candidates produce large importance sampling ratios, which lead to wide confidence intervals on their repercussions, causing them to fail Theia’s repercussion-awareness test (lines 4–9 of Alg.1 and 4–11 of Alg.3).\n   * In other words, models that could lead to high-variance estimates are naturally filtered out during the search process since they induce unreliable repercussion estimates.\n\n**In practice, the two key properties above serve as effective safeguards, constraining importance sampling ratios and thereby controlling variance.** The effectiveness of these variance-control methods is evident in all our experiments: Theia consistently identifies models satisfying *all* constraints with high confidence, requiring at most 12% of the available data in the worst case.\n\n---\n### **Do all repercussions have to be fairness-related?**\nOur empirical evaluation touches on fairness because we required Theia to satisfy repercussion constraints across different groups. However, the *repercussion objectives themselves* were **not** fairness-related. In EXP-1, the objective was to improve the longer-term repercussions of ML-driven decisions on foster care youth—e.g., how such decisions might shape their future opportunities. In EXP-2, the goal was to increase, with high confidence, the future financial well-being of clients. These are meaningful, application-specific metrics—not fairness measures per se.\n\nFairness arises only when we ask Theia to improve these repercussion metrics uniformly across all groups. The exact same experiments could be conducted without that requirement. In this case, however, Theia might return models that improve repercussions on average but risk harming specific subgroups, thus raising concerns about trustworthiness.\n\nImportantly, although Theia’s formulation supports arbitrary group-specific constraints (lines 99–118), these do not need to be fairness-related. One could require, e.g., that repercussions improve for a single group while requiring them to worsen for others. The framework is flexible and fairness-agnostic.\n\n---\n### **How can repercussion be defined beyond fairness?**\nThe reviewer asked for examples of how repercussions can be defined beyond fairness. In addition to the running example of political polarization and the two scenarios used in our experiments, App.A provides further examples of non-fairness-related domains where Theia could be applied:\n\n* Ensuring that a university’s policy for assigning 1-on-1 tutoring, based on students’ GPAs, does not reduce overall graduation rates. Let $\\tau$ be the current probability that students graduate under the existing policy. The dataset could include $\\widehat Y_i$, indicating whether the $i$-th student received tutoring, and $R_i$, an indicator of whether they graduated. Then, the repercussion constraint can be defined as $g(\\theta) = E[R^{\\pi_\\theta}] - \\tau$.\n* Ensuring that a new crime prevention strategy, implemented by a police department based on predicted recidivism risk, does not lead to higher average incarceration rates. Let $\\tau$ denote the current incarceration rate prior to deployment of the new strategy. Then, the repercussion objective $g$ could enforce that the probability of incarceration (i.e., the expected value of an indicator variable denoting whether a person was incarcerated) does not exceed $\\tau$ with high confidence.\n* The third example, related to a potential healthcare application, can be modeled similarly—by defining a repercussion objective that compares expected patient outcomes (e.g., a metric of chronic illness severity after treatment) under a new decision policy to a baseline level of care.\n\n---\n### **Difference between static and repercussion fairness**\nThe reviewer asked us to clarify the difference between repercussion fairness and standard static fairness definitions studied in the literature. As discussed in footnote 6, these differ qualitatively. Static fairness metrics—such as equalized odds—are defined in terms of performance metrics like false positive and true positive rates. They can be easily/directly computed by evaluating a model on a validation set and comparing its predictions to the true labels.\n\nRepercussion fairness is significantly more general. It concerns the *actual downstream impact of model decisions after deployment*. Quantifying it requires observing the real-world consequences of using the model—for example, by measuring a client’s debt-to-income ratio one year after the model has been deployed. Such metrics cannot be inferred from standard performance evaluations alone and go beyond what is captured by static fairness metrics.\n\n---\n### **Comparison to model-free baselines for repercussion fairness**\nThe reviewer asked why we did not compare to baselines that ensure repercussion fairness. As discussed in Section 5, while model-based approaches exist, Theia is, to the best of our knowledge, the *first* method capable of satisfying repercussion constraints in the model-free setting. There are no existing baselines that operate exclusively on data collected from a previously deployed classifier without access to a model of the environment. For this reason, we compare Theia to well-established, state-of-the-art model-free methods that share our core assumptions and also rely solely on observed data.\n\n---\n### **Causes for *No Solution Found***\nUnlike most existing methods, Theia can reason about its own uncertainty and return *No Solution Found* when it cannot guarantee, with the desired level of confidence, that all constraints will be satisfied. The reviewer asked whether Theia could be extended with explainable AI capabilities to help users understand why no models could be certified as repercussion-aware with high confidence.\n\nRecall that to determine if a candidate solution is repercussion aware, Theia checks if a high-confidence upper bound on its repercussion passes a safety test. A first step towards understanding *why* no solutions were found could be to check if the estimated *average* repercussion of a model (i.e., a point estimate) would pass the safety test. Intuitively, this point estimate provides an *optimistic* assessment of a model's repercussion, since it does not account for uncertainty. If even this optimistic estimate fails the test, it suggests that the failure is not due to data scarcity or estimator variance, but rather that no model can satisfy the constraints. By contrast, if the point estimate would pass the test but its associated confidence interval is too wide—resulting in an upper bound that fails—the likely issue is insufficient data to meet the desired confidence level.\n\nWhile incorporating such diagnostic capabilities into Theia is beyond the scope of this work, it is a promising future extension. In this paper, we focused on introducing the first classification algorithm capable of providing high-confidence guarantees on repercussion constraints using only historical data—thus solving a key open problem in the literature.\n\n---\n### **Can Theia handle objectives beyond expected repercussions?**\nThis is a great question. Theia can be readily extended to enforce repercussion objectives beyond those involving expectations. Repercussion objectives can be defined over other statistics of the repercussion distribution to enforce high-confidence guarantees, e.g, on repercussion variance, median, or conditional value at risk. All that is required is a confidence interval for the statistic of interest. If the user wishes, e.g., to impose constraints on the CVaR of the repercussions, they could directly leverage the concentration inequality introduced by Brown (2007). Similarly, Chandak et al. [9] proposed methods for producing high-confidence bounds for various other distributional parameters, e.g., median and interquantile range. These can be readily integrated into Theia to support the extensions suggested by the reviewer.\n\n*Brown, D. B. Large deviations bounds for estimating conditional value-at-risk. Operations Research Letters, 2007.*\n\n---\nWe hope these clarifications address your concerns. If so, we would sincerely appreciate it if you would consider revisiting your score. If anything remains unclear, we welcome further feedback and would be happy to continue the discussion. Thank you again for your time and engagement."}, {"review_id": "ZD4bAzlJTi", "reviewer": "Reviewer_mwhg", "summary": "This paper presents a novel algorithm, THEIA to manage the real-world repercussions of machine learning models. The primary contribution is an approach that optimizes for a main objective, such as accuracy, while simultaneously ensuring with high, user-specified confidence that constraints on a model's repercussions are met using only existing data from a previously deployed classifier. The method leverages importance sampling to estimate the potential repercussions of a new model and provides theoretical guarantees that it will satisfy the specified constraints. The authors empirically demonstrate THEIA's effectiveness on two real-world case studies, showing it can successfully enforce fairness-related repercussion constraints without a significant loss in accuracy.", "questions": "See strength and weekness", "limitations": "N/A", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength\n1. The authors conduct a comprehensive empirical evaluation on real-world datasets, demonstrating that THEIA successfully identifies accurate models that satisfy repercussion constraints while various state-of-the-art fairness algorithms fail to do so. The experiments also robustly test the algorithm's performance across different data sizes and in challenging settings where the model's predictions have varying levels of influence on the observed repercussions.\n2. The paper tackles the critical and previously open problem of managing the real-world repercussions of ML models without requiring pre-existing analytical models that describe these effects. This approach significantly enhances practicality, as it relies only on observational data from a previously deployed model, which is more readily available than a formal causal model.\n3. The proposed algorithm, THEIA, is supported by formal proofs which guarantee that any solution it returns will satisfy the user-defined repercussion constraints with high probability. \n\nWeakness\n1. The work's primary contribution is in problem formulation, as it applies existing Seldonian and off-policy evaluation techniques to the domain of repercussions, which may be viewed as an incremental methodological step.\n2. The assumption that repercussions are instance-specific and independent may be unrealistic for systemic, user-level impact across multiple instances and population-level impacts across users are very common, limiting the framework's applicability to a narrower class of problems.\n3. The method's reliance on importance sampling is known to suffer from high variance, potentially requiring impractically large datasets to yield reliable estimates and raising concerns about sample efficiency.", "was_revised": true, "final_justification": "I have read authors' rebuttal and will remain my initial evaluation", "rebuttal": "We thank the reviewer for their positive comments on the novelty and importance of our work, the thoroughness of our empirical evaluation, and the relevance of our proposed framework—the first to provide high-confidence guarantees about potential repercussions based only on historical data. We also appreciate the reviewer’s comments on how our method solves, in a principled way, a critical and previously open problem in the literature.\n\nBelow, we address the reviewer’s questions and concerns.\n\n---\n### **On our contributions and methodological novelty**\n\nWhile Theia builds on ideas from Seldonian algorithms and off-policy evaluation, combining these concepts to create a general-purpose framework capable of managing the repercussions of real-world ML model, in a principled way, is far from straightforward. Formalizing a setting where this combination is not only feasible but also provably effective required addressing significant technical challenges. For example, establishing consistency guarantees—i.e., proving that Theia will *always* identify and return a repercussion-aware model if one exists, given sufficient data—required a complex four-page-long proof. Without such guarantees, Theia would be only a heuristic, likely unsuitable for high-stakes applications that require rigorous safety assurances.\n\nBeyond this core contribution, we also discuss many principled and readily implementable ways to significantly expand the applicability of our framework. These include adaptations to regression problems and support for enforcing constraints on a wide range of properties of the repercussion distribution—such as conditional value at risk, variance, or median—all of which are paramount in risk-sensitive settings.\n\nOur proposed approach solves an important open problem in the literature. Theia is the first method capable of providing high-confidence guarantees on the real-world repercussions of ML decisions using only historical data. Until now, such guarantees were only attainable through complex analytical models that precisely characterize how predictions influence downstream outcomes. Constructing these models (e.g., via causal inference) is often impractical or infeasible. By eliminating this dependency, Theia fills a critical gap and enables the development and deployment of ML systems that are not only effective but also provably capable of satisfying user-defined constraints on repercussions with high confidence.\n\n---\n### **Instance-level versus group-level repercussions** \nThe reviewer asks whether focusing on instance-based repercussions may be potentially limiting in scenarios where one wishes to enforce constraints at the group level. This is a great question.\n\nUnder our formulation, group-level repercussions can be directly computed from instance-level information. In particular, the objective introduced in Equation 1 is designed *precisely* to model repercussion constraints at the group level, where $c(X_i,Y_i)$ is a group membership variable indicating to which group the $i$-th instance belongs.\n\nFinally, note that settings where only group-level repercussion information is available typically arise when predictions made by an ML model are not targeted at individuals, but instead influence decisions with collective consequences—e.g., a decision to change the speed limit on a major street. These settings can still be handled by our framework: the group can be treated as an instance, described by group-level features, and the repercussion reflects the average effect of the model’s prediction on all individuals within that group.\n\n---\n### **Importance sampling variance**\nThe reviewer is correct that importance sampling estimators, in general, may suffer from high variance. However, in our setting, two key properties ensure that this concern does not materialize in practice:\n1. In general decision-making settings, importance sampling may require data exponential in the horizon. The classification setting we address, by contrast, involves a *single decision* per instance: determining its label. This makes ours an ideal setting for using importance sampling, since variance is independent of the decision horizon.\n2. Theia was actively designed to incorporate mechanisms that control potential variance issues (see Footnote 4 and Appendix G, lines 857–869). In particular:\n   * To estimate a model’s repercussions, Theia computes the ratio of the probability the candidate model assigns to a label to the probability assigned to the label by the deployed model.\n   * The concern raised by the reviewer would be justified if nothing were known in advance about the relationship between these models. Then, the resulting ratios could be extremely small or large, leading to high variance.\n   * However, the candidate models evaluated by Theia are *not* arbitrary. Theia is designed to actively reject candidates that diverge too much from the current model. In particular, such candidates produce large importance sampling ratios, which lead to wide confidence intervals on their repercussions, causing them to fail Theia’s repercussion-awareness test (lines 4–9 of Algorithm 1 and 4–11 of Algorithm 3).\n   * In other words, models that could lead to high-variance estimates are naturally filtered out during the search process since they induce unreliable repercussion estimates.\n\n**In practice, the two key properties above serve as effective safeguards, constraining importance sampling ratios and thereby controlling variance.** The effectiveness of these variance-control methods is evident in all our experiments: Theia consistently identifies models that satisfy all constraints with high confidence, using on average less than 1.5% of the available data—and no more than 12% even in the worst case.\n\n---\nWe hope these clarifications address your concerns. If so, we would sincerely appreciate it if you would consider revisiting your score. If anything remains unclear, we welcome further feedback and would be happy to continue the discussion. Thank you again for your time and engagement."}]

Author rebuttal:

Meta-review: meta review forthcoming (in touch with sac)
Decision: Accept (poster)