Finding Regions of Heterogeneity in Decision-Making via Expected Conditional Covariance

1 Introduction . Understanding heterogeneity in decision-making is an established problem in medicine ( Birkmeyer et al. , 2013 ; Corallo et al. , 2014 ; De Jong et al. , 2006 ) , consumer choice ( Ortega et al. , 2011 ; Scarpa et al. , 2005 ) , and law ( Kang et al. , 2012 ; Kleinberg et al. , 2018 ; Arnold et al. , 2018 ) . In the context of medicine , this is referred to as the study of practice variation ( Atsma et al. , 2020 ; Cabana et al. , 1999 ) , where it is often observed that doctors , facing the same clinical context , will make different decisions . Likewise , in a legal context , judges often differ in their leniency in their decisions regarding bail ( Kleinberg et al. , 2018 ) , juvenile incarceration ( Aizer and Doyle Jr , 2013 ) , the use of alternatives to incarceration ( Di Tella and Schargrodsky , 2013 ) , and incarceration length ( Kling , 2006 ) . In some scenarios this variation may be justified : The best medical treatment may not be obvious . In others , it may be grossly unfair , as in the case of racial bias in bail decisions ( Arnold et al. , 2018 ) . ∗Equal contribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we tackle the question of how to find and characterize this variation in the first place . In particular , we present a learning algorithm for identifying a “ region of heterogeneity ” , defined as a subset of all contexts ( e.g. , patients , cases ) for which the identity of the decision-maker substantially affects the decision . In medicine , a better understanding of treatment variation can inform the development and dissemination of clinical guidelines . In the legal domain , characterizing the cases where judges vary most in their leniency may help with investigating potential issues of fairness . We formalize characterizing the region of heterogeneity as a causal inference problem : We want to characterize examples where changing the decision-maker would have resulted in a different decision . The challenge is two-fold : First , we only observe a single decision-maker per example , so we can not directly observe how ( for instance ) multiple judges would have decided the same case . Second , our data on individual decision-makers is often scarce . For instance , in Section 5 , we consider a medical dataset with more than 400 doctors , each of whom has fewer than 9 samples on average . We will refer to decision-makers as “ agents ” , and our contributions are as follows : In Section 2 , we propose an objective defined in terms of counterfactual decisions across different agents , and show that this objective can be identified from observational data . Moreover , this objective does not require the use of agent-specific statistical models , making it amenable to our sparse setting . In Section 3 , we give an iterative algorithm to identify regions of disagreement by maximizing this objective and provide intuition ( in the form of a generalization bound ) for the factors that drive its performance . In Section 4 , we use a semi-synthetic dataset , derived from crowd-sourced recidivism predictions , to demonstrate that our algorithm recovers the correct region of heterogeneity accurately , even when there are many agents . Finally , in Section 5 , we apply our algorithm to a real-world healthcare dataset and confirm that it recovers intuitive regions of variation in first-line diabetes treatment . We conclude with a discussion of related work and implications . Our code is available at https : //github.com/clinicalml/finding-decision-heterogeneity-regions . Our algorithm does not determine whether variation is inherently good or bad or how it should be addressed . Rather , more careful study with domain experts would be required to determine if variation can ( or should be ) reduced and how . In addition , false discovery of variation is possible and could have a negative impact . We expect that validation on independent datasets would be required in real-world applications , using the regions identified by our method as plausible hypotheses to test . 2 Characterizing Heterogeneity from a Causal Perspective . 2.1 Notation . Let the data be drawn from a distribution P ( X , A , Y ) , where X is a random variable representing context ( or features ) , A is a discrete agent , and Y ∈ { 0 , 1 } is the binary decision . The spaces of all X and A are denoted as X andA with realized values as lower case x and a , respectively . Indicator variables 1 [ · ] are one if the statement inside the brackets is true and zero otherwise . For a subset S ⊆ X , 1 [ x ∈ S ] is sometimes written as a function S ( x ) , where S : X → { 0 , 1 } . A subset S may have several disjoint regions . E [ Y |X ∈ S ] denotes the average Y across samples in S . For instance , if S = { X : X0 < 10 } , then E [ Y |X0 < 10 ] is a scalar average of Y among samples with X0 < 10 . 2.2 Heterogeneity as a Causal Contrast . Our conceptual goal is to identify a region S ⊆ X where different agents tend to make different decisions even when faced with the same context . We can formalize this in the language of potential outcomes from the causal inference literature ( Pearl , 2009 ; Hernán and Robins , 2020 ) , which for clarity we will refer to as potential decisions : In particular , we denote Y ( a ) to be the potential decision of agent a . The fundamental challenge of causal inference is that we do not observe all potential decisions { Y ( a ) : a ∈ A } for each sample , but only a single decision Y . With this in mind , we will make the following assumptions , standard in the literature on causal effect estimation . Assumption 1 ( Causal Identification Assumptions ) . ( i ) Consistency : Y = y , A = a =⇒ Y ( a ) = y , and ( ii ) No Unmeasured Confounding ( NUC ) : For all a ∈ A , Y ( a ) ⊥ A | X . Consistency links the potential Y ( a ) to the observed Y , and NUC says that there are no unobserved factors that influence both the assignment of agents and the decision itself . For instance , the quasirandom assignment of cases to judges conditioned on features X satisfies NUC ( Kleinberg et al. , 2018 ) . NUC may be violated if key aspects of the case ( e.g. , misdemeanor vs. felony ) are omitted as features . For instance , misdemeanor and felony cases may be seen by different judges and have different decision processes , but this variation is not due to agent preferences . Given these assumptions , we propose a causal measure of agent-specific bias , defined as a contrast between potential decisions . Definition 1 ( Conditional Relative Agent Bias ) . For an agent a ∈ A and a subset S ⊆ X , the conditional relative agent bias is defined as E [ Y ( a ) − Y ( π ( x ) ) | A = a , X ∈ S ] ( 1 ) where Y ( a ) is the potential decision of agent a , and Y ( π ( x ) ) B ∑ a′ E [ Y ( a′ ) | x ] π ( a′ | x ) denotes the expected potential decision under the agent assignment distribution π ( a′ | x ) B P ( A = a′ | X = x ) . Note that under Assumption 1 , Y ( π ( x ) ) = E [ Y | x ] ,2 but here we emphasize its causal interpretation as the expected decision of a random agent . Equation ( 1 ) represents the relative difference between the decision of an agent ( on their particular distribution of cases in the region ) and the potential decision of a random agent . In particular , Equation ( 1 ) can be written as follows under Assumption 1 E [ Y ( a ) − Y ( π ) | A = a , X ∈ S ] = ∫ x∈S E [ Y ( a ) − Y ( π ) | X = x ] p ( x | A = a , X ∈ S ) dx , ( 2 ) where we shorten Y ( π ( x ) ) to Y ( π ) . This is the average difference ( over p ( x | a ) , restricted to those x in the set S ) of the conditional expected difference between Y ( a ) and Y ( π ) . For example , suppose that the agent a is a judge who is particularly lenient on bail decisions for felony arrests ( the region S ) , and Y = 1 denotes granting bail . Then imagine the following counterfactual : Take the felony cases that are assigned to this judge and reassign each individual case , described by x , to a random judge a′ , proportionally to p ( a′ | x ) . We may then observe , on average , that the bail rate would decrease , because most judges are less lenient than judge a , corresponding to a positive value of Equation ( 1 ) . Equation ( 1 ) has the additional advantage of being easy to estimate : Under Assumption 1 , it can be rewritten3 as E [ Y−E [ Y | X ] | A = a , X ∈ S ] , the expected residual in predicting ( using the conditional expectation E [ Y | X ] ) the decision of an agent a across the context x typically seen by that agent . 2.3 Formalizing a Causal Objective . Our primary goal is to discover a region S where substantial heterogeneity exists across agents . To do so , we define an aggregate objective across a group G of agents , where G ( a ) ∈ { 0 , 1 } is an indicator function for membership . Q ( S , G ) B ∑ a : G ( a ) =1 P ( A = a | X ∈ S ) E [ Y ( a ) − Y ( π ) | A = a , X ∈ S ] , ( 3 ) We now show that this quantity can be identified and estimated from observational data without requiring agent-specific statistical models , before discussing the interpretation of this objective . Theorem 1 ( Causal Identification ) . Under Assumption 1 , Q ( S , G ) can be identified as Q ( S , G ) = ES [ Cov ( Y , G | X ) ] = ES [ ( Y − E [ Y | X ] ) G ] , ( 4 ) where ES [ · ] B E [ · | X ∈ S ] and Cov ( Y , G | X ) is the conditional covariance . Theorem 1 and other theoretical results are proven in Appendix A . The result follows from proving that the agent-specific bias ( Definition 1 ) is identifiable using the expected conditional covariance between Y and the binary indicator 1 [ A = a ] . With this in mind , we optimize the following objective , where the set S is constrained to be at least a certain size β and S is a hypothesis class of functions S . max S∈S , G Q ( S , G ) s.t. , P ( S ) ≥ β , ( 5 ) Interpretation : Intuitively , this objective measures the disagreement between the agents in the group G ( a ) = 1 and the overall average E [ Y | X ] on the region S . Hence , the choice of group is important for interpreting the objective : If G ( a ) = 1 for all agents , the objective will be zero for any set S , as can be seen from Equation ( 4 ) , applying the definition of the conditional expectation . 2See Proposition A1 for a short proof , and Proposition A2 for the derivation of Equation ( 2 ) . 3See Proposition A3 in Appendix A.1 . Accordingly , we seek a region S for which the partially maximized objective L ( S ) B maxG Q ( S , G ) is large : This partial maximization is obtained by taking G ( a ) = 1 whenever the conditional relative agent bias of agent a ( on the set S ) is non-negative . Thus , Equation ( 5 ) can be re-written as max G Q ( S , G ) = ∑ a∈A P ( A = a | X ∈ S ) |E [ Y ( a ) − Y ( π ) | A = a , X ∈ S ] |+ , ( 6 ) where |x|+ B max ( x , 0 ) , and this objective becomes an average over agents who have a positive bias . This population objective is also equivalent ( up to a constant factor ) to the ( weighted ) sum of the magnitude of each agent ’ s conditional relative agent bias . See Proposition A4 in Appendix A.1 . Lack of Overlap : We have not made the overlap or positivity assumption that P ( A = a | x ) > 0 for all x , a . While this assumption is required to identify conditional causal effects E [ Y ( a ) − Y ( a′ ) | X ] ( Nie and Wager , 2017 ; Wager and Athey , 2018 ; Shalit et al. , 2017 ) , it is not required for identifying our causal contrast . Our problem only requires each context has a positive probability of being seen by more than one decision maker . For instance , suppose that S contains both misdemeanors and felonies and there are four judges a0 , . . . , a3 . If judges a0 and a1 make bail decisions exclusively for felonies while judges a2 and a3 make bail decisions exclusively for misdemeanors , our measure captures disagreement between a0 and a1 and between a2 and a3 even though comparisons between a0 and a2 or other pairs are impossible to make . Thus , we have chosen to compare Y ( a ) to the decisions of viable alternative agents , weighted by their probability p ( a′ | x ) of being selected for such a case .

Summary: The authors are concerned with identifying regions in a covariate space. A region is characterised by having high variation in (potential) treatment outcome, as compared to the expected (potential) outcome. A method such as this could be used to identify variation of practice in the legal or medical domain (as indicated by the authors).   This is a relevant topic which the authors rightfully relate to the causal literature. Specifically, the authors chose the potential outcomes model, where the treatment is now a clinician or judge; the outcome is binary (amounting to a decision, e.g. bail or no bail); and the condition is an individual. Contrasting potential outcomes, the authors are interested in learning subsets of X, rather than Y(a).