Title: Beyond Prediction: Managing the Repercussions of Machine Learning Applications
Venue: NeurIPS 2025
Accepted: yes
Keywords: Classification algorithms

Repercussion-aware optimization

Data-driven guarantees
Abstract: Machine learning models are often designed to maximize a primary goal, such as accuracy. However, as these models are increasingly used to inform decisions that affect people's lives or well-being, it is often unclear what the real-world repercussions of their deployment might be—making it crucial to understand and manage such repercussions effectively. Models maximizing user engagement on social media platforms, e.g., may inadvertently contribute to the spread of misinformation and content that deepens political polarization. This issue is not limited to social media—it extends to other applications where machine learning-informed decisions can have real-world repercussions, such as education, employment, and lending. Existing methods addressing this issue require prior knowledge or estimates of analytical models describing the relationship between a classifier's predictions and their corresponding repercussions. We introduce Theia, a novel classification algorithm capable of optimizing a primary objective, such as accuracy, while providing high-confidence guarantees about its potential repercussions. Importantly, Theia solves the open problem of providing such guarantees based solely on existing data with observations of previous repercussions. We prove that it satisfies constraints on a model's repercussions with high confidence and that it is guaranteed to identify a solution, if one exists, given sufficient data. We empirically demonstrate, using real-life data, that Theia can identify models that achieve high accuracy while ensuring, with high confidence, that constraints on their repercussions are satisfied.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel algorithm, THEIA to manage the real-world repercussions of machine learning models. The primary contribution is an approach that optimizes for a main objective, such as accuracy, while simultaneously ensuring with high, user-specified confidence that constraints on a model's repercussions are met using only existing data from a previously deployed classifier. The method leverages importance sampling to estimate the potential repercussions of a new model and provides theoretical guarantees that it will satisfy the specified constraints. The authors empirically demonstrate THEIA's effectiveness on two real-world case studies, showing it can successfully enforce fairness-related repercussion constraints without a significant loss in accuracy.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n1. The authors conduct a comprehensive empirical evaluation on real-world datasets, demonstrating that THEIA successfully identifies accurate models that satisfy repercussion constraints while various state-of-the-art fairness algorithms fail to do so. The experiments also robustly test the algorithm's performance across different data sizes and in challenging settings where the model's predictions have varying levels of influence on the observed repercussions.\n2. The paper tackles the critical and previously open problem of managing the real-world repercussions of ML models without requiring pre-existing analytical models that describe these effects. This approach significantly enhances practicality, as it relies only on observational data from a previously deployed model, which is more readily available than a formal causal model.\n3. The proposed algorithm, THEIA, is supported by formal proofs which guarantee that any solution it returns will satisfy the user-defined repercussion constraints with high probability. \n\nWeakness\n1. The work's primary contribution is in problem formulation, as it applies existing Seldonian and off-policy evaluation techniques to the domain of repercussions, which may be viewed as an incremental methodological step.\n2. The assumption that repercussions are instance-specific and independent may be unrealistic for systemic, user-level impact across multiple instances and population-level impacts across users are very common, limiting the framework's applicability to a narrower class of problems.\n3. The method's reliance on importance sampling is known to suffer from high variance, potentially requiring impractically large datasets to yield reliable estimates and raising concerns about sample efficiency.\n\nQUESTIONS:\nSee strength and weekness\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents THEIA, a framework to optimize beyond accuracy and consider repercussions of machine learning applications in the optimization. The algorithm satisfied a constrain called Seldonian, meaning that if no solution can be found with the present data, it will return NSF instead of returning a solution that itself does not trust.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nThe Seldonian constraint and the ability that the algorithm is able to return NSF when the solution is not feasible is a very practical and useful feature.\n\nWeaknesses\n\n1. Importance sampling is hard to be controlled and may have a high variance. The authors did not provide a good justification of how to get over it; instead they just mention in the footnote that the high variance situation was not observed by them. It weakens the value of this work.\n2. The authors overloaded the term repercussion and position THEIA as a general solutuion. However, throughout the paper, the only repercussion demonstrated is the fairness constraints. The claim does not match the results presented in this work. There are many repercussions that a real ML application should consider such as long-term behavior shift, polarization of comments in social media, legal constraints, causality, interpretability, etc. It is the main reason why I give a low score for this work.\n3. As this paper is mainly comparing fairness constraints, there is a lack of comparison against existing fair learning baselines. The authors claim in the footnote that repercussion fairness is different from static fairness but did not compare clearly what is the different and also did not compare with baselines that can perform repercussion fairness.\n4. The different condition that cause BSF is underexplored. It is common that a fair learning algorithm can’t achieve high accuracy and low fairness metrics at the same time. However, the authors did not provide concrete discussions and empirical results with high-level causes on why it is not achievable.\n\nQUESTIONS:\nPlease also refer to the Weaknesses.\n\n1. When will THEIA fails due to the high variance of the importance sampling and what is the time complexity comparing to existing fairness algorithms?\n2. How can the repercussion be mathematically defined beyond fairness? The authors should provide some examples of how repercussions g is defined for other applications. \n3. When the algorithm return NSF, how do you know if it is not the case when the constraints are too strict or the hyper-parameters such as \\beta are ill-selected?\n4. Can THEIA handle objectives that are not limited to the expected behavior? Such as controlling the tail distribution?\n\nLIMITATIONS:\nI think the Weaknesses reflect the limitations of this work.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new Seldonian algorithm, Theia, for producing classification models that can satisfy constraints on repercussions derived from a previous model’s performance. The algorithm works by using a historical behavior model and data about the repercussions of that model (for example, levels of polarization induced by a recommendation algorithm). The algorithm allows specification of a probability, $\\delta$, that the repercussion constraint is violated, as well as a tolerance value $\\tau$ to control the tightness of the constraint. The paper uses importance sampling to construct an unbiased estimator of the new model’s repercussions given the historical model’s data. The paper also provides several theoretical guarantees about Theia, including that it satisfies the probabilistic constraint specified and that it is consistent, meaning that it will return a solution satisfying the constraints given enough training data. Through several empirical results, the paper shows that the algorithm does indeed behave as specified across two different settings with repercussions.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper’s strengths are its quality, clarity, and thoroughness. The authors have introduced a clear and useful framework for satisfying constraints given historical data on model behavior. The method is novel and will likely be very useful to the fairness community. The empirical results show a clear advantage over other fairness approaches in terms of satisfying the constraints, and the experiments are quite thorough, comparing against several other approaches. The paper lays out several research questions and answers all of them clearly and concisely. This paper would be a valuable addition to the Neurips community.\n\nIn terms of weaknesses, the primary item missing from this paper, in my opinion, is more discussion of the groupwise impacts of Theia’s approach. I have some specific questions below regarding this. Given that we know that models already struggle for accuracy on groups that are most likely to have these repercussions because they are smaller or tend to be less represented in datasets, I am curious as to whether Theia’s efficacy varies across demographic groups in a similar manner. Lack of experiments in this regard is not necessarily a dealbreaker for the paper, but I think it would be strongly improved by some discussion or consideration of the possibility that the performance of the Theia algorithm itself may be in some way biased or unfair. \n\nOverall, though, this is a very well done and thorough paper, and I thank the authors for their wonderful contribution.\n\nQUESTIONS:\n- Does the accuracy loss in figure 2 occur only for the group from whom the algorithm is attempting to minimize repercussions? Or is it dataset-wide? I am wondering whether the impacted group that one is hoping to help actually ends up suffering in a different way because of lower accuracy.\n\n- In the case where multiple groups have constraints that are trying to be satisfied, what might affect Theia’s efficacy in satisfying those constraints? For example, if I have two groups with constraints, and one has significantly more data, will that one have its constraint satisfied more reliably? Are there ways to mitigate such disparities?\n\nLIMITATIONS:\nI have discussed some limitations in the weaknesses above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed THEIA, a new classification algorithm that seeks to manage the real-world consequences of deploying machine learning models. Rather than relying on causal models or predefined analytical relationships between predictions and their effects, THEIA uses observational data from a previously deployed classifier to ensure, with high probability, that user-specified constraints on repercussions are satisfied. The authors provide formal guarantees and evaluate the approach in two real-world scenarios involving employment outcomes and lending decisions.\n\nSTRENGTHS AND WEAKNESSES:\n+ The paper addresses an increasingly important problem, how to reason about the real-world effects of ML models.\n+ THEIA operates without requiring access to a causal relationships.\n+ The algorithm includes strong theoretical backing, including high-confidence guarantees and consistency under mild assumptions.\n- The method fundamentally relies on observational associations, and cannot distinguish causation from correlation. This limits its ability to draw meaningful conclusions when confounding is present.\n- There is no mechanism for detecting whether the estimated repercussions actually reflect causal effects, which may be problematic in practice.\n- THEIA assumes that consequences under a new policy can be estimated using data collected from a previous policy—this stationarity assumption may not hold.\n\nQUESTIONS:\nWhile THEIA offers a valuable framework for making conservative decisions based on observational data, its reliance on correlational estimates inherently limits its scope. In particular, it cannot provide causal guarantees or account for unobserved confounding. There are well-known cases where results based solely on correlation are indistinguishable from misleading or spurious ones. It would be helpful for the authors to comment on how THEIA handles such scenarios, or whether safeguards could be incorporated to detect when observational estimates may be unreliable.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-07"}