Defining Expertise: Applications to Treatment Effect Estimation
Authors: Alihan Hüyük
Qiyao Wei
Alicia Curth
Mihaela van der Schaar
Venue: ICLR 2024 poster (2024)
Keywords: expertise
model selection
balancing representations
treatment effect estimation
Primary area: causal reasoning

Abstract: Decision-makers are often experts of their domain and take actions based on their domain knowledge. Doctors, for instance, may prescribe treatments by predicting the likely outcome of each available treatment. Actions of an expert thus naturally encode part of their domain knowledge, and can help make inferences within the same domain: Knowing doctors try to prescribe the best treatment for their patients, we can tell treatments prescribed more frequently are likely to be more effective. Yet in machine learning, the fact that most decision-makers are experts is often overlooked, and “expertise” is seldom leveraged as an inductive bias. This is especially true for the literature on treatment effect estimation, where often the only assumption made about actions is that of overlap. In this paper, we argue that expertise—particularly the type of expertise the decision-makers of a domain are likely to have—can be informative in designing and selecting methods for treatment effect estimation. We formally define two types of expertise, predictive and prognostic, and demonstrate empirically that: (i) the prominent type of expertise in a domain significantly influences the performance of different methods in treatment effect estimation, and (ii) it is possible to predict the type of expertise present in a dataset, which can provide a quantitative basis for model selection.

Published as a conference paper at ICLR 2024
DEFINING EXPERTISE:
APPLICATIONS TO TREATMENT EFFECT ESTIMATION
Alihan Hüyük, Qiyao Wei, Alicia Curth, Mihaela van der Schaar University of Cambridge
ABSTRACT
Decision-makers are often experts of their domain and take actions based on their domain knowledge. Doctors, for instance, may prescribe treatments by predicting the likely outcome of each available treatment. Actions of an expert thus naturally encode part of their domain knowledge, and can help make inferences within the same domain: Knowing doctors try to prescribe the best treatment for their patients, we can tell treatments prescribed more frequently are likely to be more effective. Yet in machine learning, the fact that most decision-maker overlooked, and "expertise" " is seldom leveraged as an inductive bias. This is especially true for the literature only assumption made about actions is that of overlap. In this paper, we argue the type of expertise the decision-makers of domain
experts is often
treatment effect estimation, where often the
that pnpertise-particulaae are likely to have-can be informative in designing and selecting methods for treatment effect estimation. We formally define two types of expertise, predictive and prognostic, and demonstrate empirically that: (i) the prominent type of exper- tise domain significantly influences the performance of different methods in treatment effect estimation, and (ii) it is possible to predict the type of expertise present in a dataset, which can provide quantitative basis for model selection.
INTRODUCTION
Those responsible for making important decisions are often experts of their domain, and they take actions based primarily on domain knowledge. We rely doctors, for instance, to make treatment decisions for patients. When a patient visits the clinic, their doctor may try to predict the likely outcome of each possible treatment and follow the best treatment route for the patient (Centor 2007). Another example is teaching: A good teacher would assess the different levels of understanding within their classroom and may choose tailor their explanations for those who are struggling the most with the lesson (Rosenshine] 2012). When experts take informed actions based on domain knowledge, they naturally impart part of their knowledge to those actions, hence expert actions can in turn be informative in making inferences within the same domain. For instance, knowing that doctors aim to prescribe the best treatment for their patients, we can infer that a treatment prescribed more frequently is likely to be more effective than alternatives. Similarly, considering that teachers focus their explanations on students who need the most help, we can identify more accurately which students might be struggling to understand the lecture. Yet in some fields of machine learning, the fact that most prcision-maker tend to be domain experts is often overlooked as a potential source of information. Expertise is seldom formalized as an assumption and leveraged
inductive bias.
Perhaps the most prominent machine learning problem in which the notion of expertise arises quite naturally, while its potential an inductive bias is usually neglected, is that of personalized treatment effect estimation. Estimating the effects of actions, treatments, or interventions is a central concern in numerous domains, and as such, plethora of machine learning methods has been proposed for estimating treatment effects based on observational data collected by decision-maker (e.g. Bica et al.) 2020a| Curth & van der Schaar 2021a). These methods become susceptible to confounding when happen to be experts and assign treatments based on factors that influence the outcomes of their assignments. As consequence, treatment effects can generally only be identified if all such confounding factors are recorded in data, and when this is the case, correcting for the resulting shift in covariates across treatment groups is considered a major challenge of the setting (Johansson al. 2016) Shaait et al.) 2017; Assaad et al. 2021). Surprisingly, despite often being one of the principal reasons behind confounding, the expertise of decision-maken almost never formalized as
those
"Atthors contrihuted equally. Correspondence to: Qiyao Wei <qw281 @cam. www.ac.uu. .k>

---

Published as a conference paper at ICLR 2024
an assumption in the literature treatment effect estimation. Rather, the standard and often the ouly-assumption made regarding a decision-maker's 's policy is that of overlap. This assumption states that all treatments must have non-zero probability of being assigned to any individual so that the resulting data has enough variability to identify treatment effects (Rubin 2005). As far as expertise is concerned, the overlap assumption means that the decision-maker cannot have perfect knowledge of their domain, allowing them to consistently take the same action given the same situation. Then, by far the most common approach in the machine learning literature for addressing the effects of confounding is to try and remove the imbalances in data created by the decision-make maker's s policy-fon for instance, by learning balancing representations according to which the decision-maker's actions appear to be taken randomly with no expertise (Johansson et al. 2016; Shalit et al. 2017: Bica et al. 2019).
In this paper, we argue that methods for treatment effect estimation should consider making more specific assumptions regarding sxpertise-beyone just overlap, which only rejects the possibility of perfect expertise. Specifically, identifying in what manner the pecision-makers of a domain typically exercise their expertise, should inform the design of methods in that domain. To this end, we distinguish two types of expertise: (i) predictive expertise, where actions are based on treatment effects only, and (ii) prognostic expertise, where actions based on all potential outcomes more generally[" For instance, the doctors in our earlier example happened to have predictive expertise as their goal was prescribe treatments with the largest benefit, whereas the teachers had prognostic expertise as they aimed to identify students with an overall weaker understanding of the subject, rather than just predicting how much the student could benefit from focused explanations. Our argument, then, is that the prominent type of expertise within a domain matters when designing and selecting models. For instance, we will see in our experiments that balancing representations could be more suitable approach in domains with predominantly prognostic expertise (e.g. education) than in domains with predominantly predictive expertise (e.g. healthcare). This insight has practical value because model selection is fundamental issue in treatment effect estimation: The ground- truth treatment effects are almost never observed, which makes conventional model validation infeasible Curth & van der will Schaar (023). ) Reasoning about expertise thus offers one solution. In experiments,
also see that it is possible to estimate the amount of predictive vs. prognostic expertise that the deci- sion-maker collecting dataset had, which can even provide quantitative basis for model selection.
Conceptually, we introduce the idea that a decision-maker's 's policy should not always be considered nuisance that causes covariate shift, but rather the fact that it often may high ex- pertise could be leveraged as an inductive bias. + Technically, we provide definition of what it means for policy to have expertise (Sec. 3): Expertise is the extent which variations in a policy's ' actions coincide with variations in treatment effects-for for predictive expertise-or potential outcomes-for prognostic expertise. We show theoretically that high expertise leads to greater shift in covariates and poor overlap. making it even more critical to leverage it as an inductive bias. Empirically we demon- strate that: (i) the type and the amount of expertise present in a dataset significantly influences the performance of different methods for treatment effect estimation [Sec./4.1 and (ii) it may be possible to classify datasets according to what type of expertise they reflect and thereby identify what methods might be more or less suitable for a given dataset- we propose pipeline that does this (Sec. 4.2)
Contributions
2 PROBLEM SETUP: TREATMENT EFFECT ESTIMATION
We consider the standard static setup for treatment effect estimation (e.g. Curth & van der Schaar 2021a). This setup assumes a data-generating process where, at each round of decision making, a new subject arrives with features X E X. These features are distributed according to E A(X') such that X ~ a, where A(X) is the set of all distributions over X. Each subject is intrinsically associated with two potential outcomes (as in Rubin 2005): Yo ) is the baseline outcome that would be realized without treatment and Y1 E y is the outcome that would be realized if the subject is to be treated. These outcomes distributed according to po E A()))* and P ( respectively such that
Yo ~ Po X) and Y1 ~ (X).. After the subject's 's arrival, a decision-maker assigns them a treatment A (0, 1]. where A = 0 denotes no treatment- which
call the "negative" greatmentwwiie AT = 1 denotes that the subject is sreated-which we call the "positive" treatment. These assignments are made according to some policy र E A((0, 1))* such that A π (X). Once treatment is as- signed, the corresponding potential outcome is realized and observed, which we denote as Y YAT.
'Tii terminology is inspired by the medical literature; predictive biomarkers are informative of the treatment effect and prognostic biomarkers are of outcomes regardless of treatment (Ballman) 2015 Sechidis al. 2018
2

---

Published as a conference paper at ICLR 2024
The treatment effect estimation problem Suppose we given observational dataset D {z', a',y'''' Hi= generated by the process we have just described, consisting of subjects x' ~ a, actions (i.e. treatment decisions) a² TTz2), and factual outcomes y ~ Pa. (z+). Then, the treatment effect estimation problem is to determine, based on this observational dataset D, the conditional average treatment effects (CATE): r(x) E Yo
z). Generally, this can only be achieved under the overlap assumption-thar is [a] > 0 for all x E X, E {0,1] (Rubin) 2005). Our setup implicitly assumes no hidden confounding as features {x²) } are fully observed in D and a' ((x)).
DEFINING EXPERTISE
Before we can provide quantitative evidence in support of our main argument- is expertise can act as an inductive bias and inform model selection in treatment effect estimation we first need to define formally what we refer to as expertise. This section first states mathematical definitions of prog- nostic and predictive expertise, and then discusses the motivation behind these definitions (Section 3.1 as well as their implications for the treatment effect estimation problem (Section 3.2). Later in Sec- tion we return back to our main argument and present empirical results based on our definitions here.
What we want to capture expertise is to what extent the actions of a decision-maker maker are informed by how a subject's ss features shape their potential outcomes we call this general type of expertise prognostic expertise. When a policy ı has high prognostic expertise, it should be possible to explain variations in its actions A mostly through the variability of potential outcomes Yo. Y1 yhethe the policy should take different actions under different circumstances only because the potential outcomes are also different. In a similar vein, we aim to capture predictive expertise, where actions are specifically informed by the treatment effect (rather than the potential outcomes more generally). When a policy т has high predictive expertise, variations in AA should mostly be due to the variability of Y1 Yo-
As our intuitive understanding of expertise is tightly linked the variability of actions under different policies, we start by quantifying action variability as the entropy of actions A for a given policy m:
H[A"] = - - 2
P[A" a} log, P(A" = =a}
(1)
This entropy can be interpreted as a measure of the inherent "randomness" or "uncertainty" actions AT have. Whenever A is observed, this randomness is removed or the uncertainty is wooolvddhenee entropy [[]T] can also be interpreted as the amount of information carried by the observations of AA.
Since expertise has intrinsically to do with how much of this action variability is due to different subjects having different potential outcomes, what we consider next is the entropy of A conditioned on potential outcomes: H[A"]Yo, Yo. Y1). If variations in AA were to be entirely due to variations in Yo. Yi, then A would appear deterministic conditioned on Yo. Yi 77tthht is [|[A")).. Yi] O. We want to capture this as the case with maximal prognostic expertise. Conversely, if treatment decisions are not informed by potential outcomes at all, which would be the case with no prognostic expertise, then the action variability would stay the same regardless of whether we condition on Yo. Yh-that is H[[AT]Y, Y1] = [. Generalizing these two extremes to a continuum: The higher the expertise is, the less variable actions should appear to be conditioned on specific pair of potential outcomes, and the lower [AAAA Yo, Y1] should be relative to HI [A Hence, we define prognostic expertise as follows:
Definition 1 (Prognostic expertise). The prognostic expertise of policy is defined as Epprag = ||4 X1] [H[[A"]
(2)
Notice that prog E [0, 1]. where E for the maximal-expertise case and Ep = 0 for the no-expertise case. Based on our earlier interpretations of entropy, we can conceptualize prognostic expertise in different ways. In those interpretations, actions had an inherent randomness or uncertainty, and if one were to somehow observe both potential outcomes, a portion of this randomness would disappear-og or part of the uncertainty would be resolved. In this hypothetical scenario, prognostic ex- pertise would be the portion of randomness regarding actions that remains that is still not resolved-even after observing both potential outcomes. Alternatively, Egng is the proportion of information carried by A that also informative of Yo, Yi (cf. mutual information).
part of the uncertainty
Similar to prognostic expertise, we define predictive expertise as the proportion of action variability that is due to variations in the treatment effect Y₁ Yo specifically (and not just Yo, Y1 more generally):
Definition 2 (Predictive expertise), The predictive expertise of policy nt is defined as Fipeed 1 H 1 Yo) [[]]
(3)
3

---

Published as a conference paper at ICLR 2024
Finally, when a policy has unit expertise, meaning any variation in its actions A is entirely due to the variability of potential outcomes (or the treatment effect), we call that policy a perfect expert:
Definition 3 (Perfect prognostie/prcdictive expert), Policy is perfect prognostic expert if Eprog 1. Similarly, policy x is a perfect predictive expert if Eiree = 1.
3.1 DISCUSSION ON EXPERTISE
Why two types of expertise? Because actions can be informed by outcomes in two distinct ways: First, treatments can be assigned based on treatment effects (ef. predictive expertise). This may happen when the goal is to achieve the best possible outcome for each subject, which is often the case in health- care (e.g. Graham et al. 2007; Caye et al. 2019). Alternatively, treatments can be assigned according to the outcome that subjects would attain regardless of treatment (cf. prognostic expertise). This may happen in cases of self-selection-foo instance, in education, students who choose to attend optional lectures might be those who already would have had higher grades even if they did not attend those lec- tures (waak et al., 2 or when the goal is to equalize outcomes across subjects for instance, in social planning, institutions that receive the most funding might be the ones that need it the most, and not necessarily the ones that would benefit the most (Ladd & Yinger 1994, Betts& & Roemer 1999). In our experiments, we will evaluate the performance of different treatment effect estimation methods for treatment effect estimation under varying amounts of both expertise types (see Figure in Section (41)).
Are the two types of expertise related? Yes, predictive expertise implies prognostic expertise. This is because the treatment effect Yi Yo is a function of potential outcomes Yo, Y1, hence actions in- formed by Y; Yu are also indirectly informed by Yo, Y1. However, the converse is not necessarily true: A policy might have prognostic expertise but completely lack predictive expertise at the same time.
Can expertise be measured? Not directly types of expertise oracle measures, meaning they cannot be computed in practice as only one of the potential outcomes would be observed. But, they can be estimated via models
demonstrate with experiments (see Figure 5 in Section 4.2).
How does expertise differ from optimality? Optimality is tied to a specific success measure- as in utility functions in economics (Kapteyn 1985) or reward functions in control/reinforcemmntt learning (e.g. |Sttton & Barto 2018 Holt et al. 2023 2024| Sun et al. 2024)- -nn entails high performance w.r.t. that success measure. In contrast, expertise expresses the idea that any variability in treatment assignments is solely motivated by what the potential outcomes (or the treatment effect) could be ne-innependeet of any particular success measure. Dependence on single success measure can be problematic: Policies with factually incorrect information (lacking expertise) might lead to high performance some measures by coincidence. Conversely, actual experts might appear to be performing poorly according to one measure only because they are trying to optimize another measure.
As an illustration of such cases, consider an environment with features X (X XB: Xc) such that X; ((,,,,0, 1)) for all i E {A, B, C),, where US) is the uniform distribution over set S, and outcomes Yn = and Y = X X so that the treatment effect is (2x) = x' +. One goal in this environment could be to maximize realized outcomes with success measure E[Y*]. Then, consider two policies: (i) a misinformed policy Fmis(() = [ 2c > 0} which tries to maximize outcomes by assigning the treatment whenever its effect is positive, but has incorrectly identified an irrelevant variable X in place of XB. and (ii) a risk-averse policy Tris
1 {IA + TB > }} which has correct information, but instead of maximizing outcomes, tries to avoid adverse negative outcomes hence assigns the treatment only when its effect is overwhelmingly positive. In this scenario, the misinformed policy still happens to perform better than the risk-averse policy in maximizing outcomes: E| > ]. However, its actions are varied based on variables that should ideally not be a consideration decision-main making, and its performance comes the expense of individuals with features z who would have benefited from the treatment as r(x:) > 0 but did not receive it as IC & O. Appropriately, the (predictive) expertise of the risk-averse policy happens to be higher, Fired > Fpra
Are experts more desirable than optimal policies? Sometimes yes, for instance in healthcare! Since treatments might have different effects for different patients, variability is needed in which treatments are prescribed to which patients. However, this variability should occur only with respect to variables that are clinically relevant and not with respect to arbitrary characteristics. In this context, policies like Tri could be more desirable: It might fall short of maximizing outcomes but varies its ac- tions only when needed. Policies like Tii.s on the other hand, possibly have unwanted variation where subjects are treated preferentially based on factors that should not be relevant to the outcome of interest.
4

---

Published as a conference paper at ICLR 2024
3.2 IMPLICATIONS OF EXPERTISE FOR TREATMENT EFFECT ESTIMATION
Before we move onto investigating, practically, how expertise can act as an inductive bias and inform model selection in treatment effect estimation, it would be insightful first understand the implica- tions of having high expertise for the treatment effect estimation problem itself. Below, we show that high expertise necessarily leads to poor overlap, which overall makes the problem more challenging.
Characterizing this mathematically requires us to first differentiate between two distinct sources of action variability. To that end, suppose X ~ B(1/2), where Bp)) the Bernoulli distribution with suc- cess probability p. and consider: (i) uniform policy uung(( [a] = 1/2, which assigns treatments uni- formly at random for all subjects, and (ii) preferential policy ( = 1 {a = =t), which assigns half of the subjects one treatment and the other half the other treatment. Both policies have the same action variability, H[[4"w] = H[[A = 1. However, while the uniform policy is highly stochastic, the preferential policy behaves in a completely deterministic way given subject X. We capture the difference between these two policies by defining the notion of in-context action variability:
Definition 4 (In-context action variability). The in-context action variability of policy ι is defined as
C = -I[AA|X] H[[A]
(4)
Notice that C' E [0, 1]. and the uniform policy has unit in-context variability, C.T = 1, while the preferential policy has zero in-context variability, CF = O. In-context action variability is useful to define because it is related directly to the overlap assumption. For it to hold, the in-context variability has be non-zero, CR = 0 would imply that the overlap assumption is violated. On the other extreme, when treatments are assigned uniformly at random (as in a randomized controlled trial), the in-context variability would reach its maximal value. As such, in-context action variability can be thought of more generally as a measure of the amount of overlap the decision-maker policy has or how "difficult" it is to estimate treatment effects from data collected by a particular policy. Keeping this interpretation in mind, what we show next is that the total expertise and in-context action variability of a policy is bounded meaniin-meaning
meaning high expertise would lead to low in-context action variability hence poor overlap
Proposition 1 (Boundedness of expertise and in-context action variability). For all ı E A((0, 1))*²
Eproy + C 1 1
and
Epred + C" 1
(5)
Proof. Appendix C
(.) Supepaare sogee
This is why leveraging expertise as an inductive bias is so crucial: Cases where more information can be gained through expertise (i.e. cases with high expertise) happen to align with cases where treatment effect estimation is particularly hard due to poor overlap (Figure D According to one extreme of Proposition [ any perfect pppert-predictvve or prognostic-mus be making deterministic decisions with respect to subject features-violatii the overlap as- sumption necessary for non non-parametric treatment effect estimation:
Secnarba www dea bolk
Frepention
Worst Case
Cabe where Ienenging expertise
Expertise (Ei or E
Figure 1: The higher the expertise of a policy, the lower its in-context action variability, hence its overlap, has to be (Prop. []. When expertise is high, leveraging it becomes criti- cal as overlap would low, making CATE estimation more challenging.
Proposition 2 (Determinism of perfect experts). If policy is perfect expert- that is either Eipog = 1 or = C =0.
Now, a natural question one might ask next is: Should we still care about treatment effect estimation when the decision-maker 's policy is already a perfect expert? Possibly "Yes!" for two reasons:
(i) As we have stressed earlier, being expert does not necessarily imply optimal performance with respect to any arbitrary success measure. In particular, a decision-maker might be a perfect expert because they optimizing one success measure, while estimating potential outcomes or the treatment effect accurately might allow us to optimize another success measure of interest.
(ii) Even if an expert were to be readily optimal with respect to desired objective, one might still be interested in formalizing their policy in mathematical terms aemmsrrtthrr than being reliant on the in- ternal thought process of human human-therebb extracting the knowledge that might otherwise only
be known (or intuitive) to the expert. In fact, this happens to be the main goal of inverse decision
modeling (Bica al. 2020b Qin al. 202 Jarrett et al. 2021 Hüyük et al. 2021 2022a b).
PI this paper, we continue to assume overlap and only ever consider policies with high expertise rather than perfect expertise- having poor overlap does not mean that the overlap assumption violated.
5

---

Published as a conference paper at ICLR 2024
4 APPLICATIONS TO TREATMENT EFFECT ESTIMATION
Having established the necessary language for discussing expertise (and action variability) in a quan- titative manner, we can finally return to our main practical argument: apperise--prticularlyy the rype of expertise present in dataset-can can act as an inductive bias and inform the selection of methods for treatment effect ciimation-ooliiiss should not necessarily be viewed as nuisances only (as in balane- ing representations), This section provides two empirical demonstrations in support of our argument:
(i) Section 4.1 we analyze the operating characteristics of various treatment effect estimation methods under different related scenarios. Our analysis reveals an important insight: Learning balancing representations can indeed hurt performance in treatment effect estimation when Decision-maker have predictive expertise. Under purely prognostic expertise however (without any predictive expertise), balancing representations can select against features that affect outcomes despite being unrelated to the treatment theecty improving performance.
(ii) In Section 4.2
we estimate the amount of predictive vs. prognostic expertise a secision-maker has, which not only enables one to evaluate the expertise of different decision-makers, but also to predict which treatment effect estimation methods might perform hetter for a given dataset.
Simulation environment As is common in the treatment effect estimation literature, we need con- struct a synthetic data-generating mechanism to ensure that potential outcomes are known by design (Curth al., 2021 Hüyūk et al. 2024). Inspired by the simulator in Crabbé al. (2022), and similar to them, we start with covariates X E Rd from real-world datasets. We designate each covariate as either being prognostic, predictive, or irrelevant such that X = (Xpreg Xpred= X E Rd X ddr.
Then, we generate potential outcomes such that Ya = (Wprog X 'mmg) + (Waa X Xpre)) +7 where Wpmn E Rdme and Wp, w1 E R.qe are weights with components sampled independently from the uniform dis- tribution over 11,, and 1Jo 7/1 are noise variables sampled from the normal distribution with μ O and a = 0.1. According to this process: (i) the treatment effect F(z) = (wi wa Ipred) depends only on predictive variables and not on prognostic variables, and (īi) neither of the two potential outcomes
depend on irrelevant variables. (A full description of our simulator can be found in Appendix E
Decision-making policies Using Figure I as map, vary the expertise of policies in this environment on two distinct axes (Fig- ure :): (i) from "best case scenario" to the high-expertise scenario (Best>Exxerry and (ii) from "worst case scenario" to the high- expertise scenario (Worst-Expperl
Hoi -operr (rual Mandt- (Saa)
Consider the (soft) predic- tive expert sof that is more likely to assign the positive treatment if the treatment effect is positive: [x (f())/33), where (x) = 1/(1 + e e=²) 3) is the sigmoid function and the temperature parameter / E R+ controls the in-context action variability. Then:
E442
0202
114
0.1
11.
05
Prodeced Bepariia (A
(i) For Best~>xxpert Expert, we simply vary the parameter 3. At the limit 3 + 00, Tage becomes equivalent to the random policy Trane (z) [a] 1/2 (i.e. the "best case scenario"). ). As / gets smaller and smaller, saf gains more and more expertise.
Figure 2: During our simulations, we increase the expertise: (i) by de- creasing / in Top (Best=>xxpert), which also decreases the in-context action variability (i.e. the overlap). and (ii) by decreasing d in Tair (Worst-Expeers as seen in Fig. D.
(ii) For Expst- we fix 3 1/4-and with it also the in-context action variability- but we slowly replace the pre- dictive variables in Xpred that an expert would take into consideration with irrelevant variables from Xir instead. Letting d E {O..... min dpred: d be the number of predictive variables that are replaced, we construct misspecified policies (x) = a((w, Wn, wwea /8) where ( Note that Tiis becomes equivalent to Toz when d = O.
2'pred pred = ((Iim))sds
We also consider policies without any predictive expertise that take actions based of prognostic and irrelevant variables: π (zon-ppa(())) = o (WW /3)) where d'yrog = ((Iar) \ids (F ) )4+1:dmm)) Similar to before, for Best-+xxpert, we fix d = min dhpro, d //2 and vary B, and for Worst -xpst=Experr, we fix = 1/4 and vary d. Despite lacking predictive expertise, Tnon-pred has prog- nostic expertise (as long d + dpng). hence why we will refer to this case the prognostic setting.
combination
Benchmark algorithms Below, we review the existing strategies for treatment effect estimation and consider them through expertise lens. In particular, highlight the inductive biases regarding ex- pertise that are implicitly encoded in different methods, which as we will discuss, arise as a by by-product of unrelated design motivations. (A full description of all algorithms can be found in Appendix FF)

---

Published as a conference paper at ICLR 2024
Potential outcome predictors: The methods estimate CATE through prediction of the negative
and the positive outcomes separately. Many popular CATE estimators in the machine learning
literature (e.g. Johansson et al. 2016 Hassanpour & Greiner 2019: Künzel et al. 2019: Curth & van der Schaar 2021b) fall into this class. In our experiments, we consider TARNet of Shalit et al. (2017) as a representative of this class. TARNet first learns shared representation & X R. which is then taken as input by two separate output prediction heads f R x {0,1) + y. Finally, the treatment effect is estimated f(x) = f()(2), 1) f(((2), 0). By focussing on potential outcome prediction only, these methods are essentially agnostic to any specialized structure that the treatment effect estimation problem has, including policies and any relationship they might have with the potential outcomes (or the treatment effect) such as expertise. Being neutral method in terms of how policies are treated, we use TARNet as our baseline method ("Baseline").
Propensity-weigee weighted predictors: One potential issue that potential outcome predictors are ignorant of is the possibility of covariate shift shift-that is the distribution of covariates in the training set for each potential outcome predictor, [((x)), 0) and [()(,,, 1), not being equal to the marginal distri- bution a of features X. One straightforward correction for covariate shift is importance weighting, which in the CATE estimation case is also known as inverse propensity weighting (IPW, e.g. Wager & Athey 2018: Hassanpour & Greiner 2019; AAssad et al. 2021; Dorn & Guo 2022). IPW can be applied to the same architecture as TARNet (i.e. Baseline) ("Propensity") " In TARNet, functions f. are trained according to some loss function L = E, lly - f(o(2') ). a II, where II denotes an arbitrary distance metric. IPW corrects for shifts in covariates by re- weighting the contribution of individual data points to this loss function: L' = E; Ilys (dbxx)), a²) /
Now notice that IPW requires policy π to be known (or estimated) beforehand, and given a policy it does not leverage any relationship between π and the potential outcomes Yo. Y1. In other words, IPW is indifferent to any expertise the decision-maker s policy might have (but corrects for the possible effects of covariate shift). However, there is one caveat: IPW is extremely sensitive to the stochasticity of policy since propensity scores π [a] appear in denominators more deter- ministic policies with smaller propensity scores for certain actions lead to estimates with higher variance (Cortes et al., 20100). Having high expertise might mean policies with less in-context vari- ability in their actions (according to Proposition I hence less stable estimates when IPW is used.
Balancing representations:
While IPW requires a policy to be fully specified beforehand and ignores any relationship between that policy and the potential outcomes. balancing representations
(e.g. JJohanssnnnon et al., 2016 Yao et al. 20188 |ica et al., 2019| Du et al. [02 Huang et al.) 2022) actively try to disentangle the from each other. For instance, CFRNet proposed by (Shalit et al. (017)) ("Balancing") uses the same architecture as TARNet, except that the bias caused by policy т is addressed by training func- tion é to generate representations that are predictive of potential outcomes Yo, Y1 but not of actions A7, so that actions a' appear to be random with respect to learned representations 2² d(x²). A typical loss function for learning such representations would be L ExE,ly' f(2" a')ll
maker's policy and the estimates for potential outcomes
hoo==0 - {2x ) =1l= This approach originates from domain adaptation (Ganin et al. 2016). where it was proposed avoid the variability issues of importance eeporrance-weighted estimators. In
experiments however, we will see that, when policies have predictive expertise, balancing repre- sentations actively remove the information carried by that expertise, which not always desirable.
Action-predictive representations: Finally, in stark contrast with balancing representations, we consider a final strategy that learns function & so that representations 2' 6(x²) are actually predictive of actions a². This strategy encodes predictive expertise as an inductive bias in the sense that it assumes that policies and outcome predictors can be represented in a joint space R from which it is easier to learn function ff then from the original space. Such a strategy is implemented albeit motivated from different angle
DragonNet in Shi et al. (2019) The loss function for DragonNet looks like L = E,Ily'' - f(2", a a')ll + log g(x;) (a.)) where function g : R ((10, 1}) is trained jointly with functions Φ, f to predict action distributions.
Performance metric As our main metric of performance, we consider precision in estimation of het- erogeneous effects (PEHE) that is (xx(((((()) - 7(X))?)1/2 for an estimator (xx). In the main pa- per, we focus on a single simulation environment, additional experiments can be found in Appendix D
Shh et al. (2019) consider average treatment effect estimation that is the estimation of B - Yo). In this context, policy plays a different, special, role: It is sufficient for adjustment (Rosenbaum & Rubin 1983)
which is what Shi et al. (2019) propose to exploit. Note that ot is not sufficient for adjustment in CATE estimation
unless nta (x) TE
xr] is function of (()) alone.
7

---

Published as a conference paper at ICLR 2024
4.1 PERFORMANCE UNDER DIFFERENT EXPERTISE SCENARIOS
When varying policy x as in Figure 2 as we move away from the "best case scenario" in Figure to the case with high expertise and eventually to the "worst case scenario" (Best=<Expert->> Worst,, the treatment effect estimation problem gets generally harder- see Figure 3 In order to compensate for this inherent change in the difficulty of estimating treatment effects, in this subsection, we measure the performance of all methods relative to Baseline.
Boo- Bot--llaper Cual
Wonnt-aloeent Real
Figures 4a and 4b the PEHE improvement of different meth- ods over Baseline for the setting with predictive expertise. Three observations stand out: First, Action-predictive achieves better and better performance over Baseline as the decision-maker expertise increases. This is because Action- action-predictive learns variables that predictive of actions, and when the expertise is high, these variables happen to be good predictors of the treatment effect as well. Second, the performance of Balancing degrades relative to Baseline as the expertise increases. This is because more relevant features becoming predictive of the actions forces Balancing to exclude more of those features from its representation space 4 Third, the relative performance of Propensity appears to degrade more consistently Best than Worst Experr This supports our earlier intuition that Propensity is mostly sensitive to in-context action variability (and overlap) rather than expertise that Best=>xxpee reduces the in-context action variability whereas Worst=Expper does not (Figure
D0
022 12.3
Produture Drpurtac Upuaal
Figure 3: As Best -xxpert (i.e. away from the "best case scenario" treatment effect estimation gets gen- erally harder and the performance of Baseline degrades. Similarly, as Worst=EExper (i.c. away from the "worst case scenario") the perfor- mance Baxeline improves instead.
remember
Figures 4c and 4d show the PEHE improvement of the same methods but for the prognostic setting. In general, we see that the performance of Action-predictiv becomes worse than Baseline. For Action-predictive, learning variables that influence actions the most, which happen to be prognostic or irrelevant variables, is no longer directly informative in estimating the treatment effect (but can still be informative in predicting potential outcomes). For Balancing the other hand, having non-predietive variables especially irrelevant variables determine actions, most of the time, helps
12.0
PERE
kiter Wr/tte
Prigzork Cigrti
Drggedic
(a) Bestr=xxpert
(b) Worst->Expert for predictive expertise
(c) Best-+Expert for prognostic expertise
(d) Worst-EXppert for prognostic expertise
for predictive expertise
Figure4 4: Performance improvements over Baseline. For predictive expertise, Action-predictive is able to improve more and more upon Baseline by exploiting the increasing expertise (both as Expert and Worst- Expert). contrast, Balancing gets worse with increasing expertise since information discards about the policy becomes more correlated with the treatment effects. However, for prognostic expertise, we observe that having non-predictive variables determine actions can help Balancing select against those variables when forming representations, improving performance upon Boseline-ti in most configurations opposed to Action-predictes
Cronseras Prodass (s
Greed lztk Prr Bayalin Faa)
(a) Best-Exppett for predictive expertise
(c) Best-B Tont->Exper Expert for predictive expertise for prognostic expertise for prognostic expertise
(b) Wxrst=Expert
(d) Worst->Expen
Figure 5: Estimated predictive/prognosti expertise. Balancing fails completely since, by design, it removes all information that would have been predictive of expertise. Propensiry more sensitive changes in expertise as Best-Expprr than Worst-EExpert (especially for predictive expertise) since the in-context action variability hence the magnitude of propensity scores-changee as Best- Expert while it stays the same as Worst Worst->Eage Expert.
"This is related to information loss in representations as analyzed theoretically in Johansson al. 2019) for domain adaptation. Their results can potentially be adapted to our setting to provide generalization bounds.
8

---

Published as a conference paper at ICLR 2024
Balancing eliminates those variables from its representations, which regularizes the estimation of treatment effects. However, Balancing still suffers if the policy depends only on prognostic variables.
Finally, perhaps the most important insight we gain from results in Figure 4 that, average, differ- ent methods perform the best under predictive vs. prognostic expertise. While predicciree may be more suitable for domains with high predictive expertise, such healthcare, Balancing may be more suitable for domains with high prognostic (and importantly low predictive) expertise, such as edu- cation. In the next section, we demonstrate that the prominent type of expertise present in a dataset can be identified by estimating the relative amounts of predictive and prognostic expertise reflected in that dataset with one of our benchmark algorithms. This provides quantitative basis for deciding whether to use Action-predee predictive or Balancing when estimating treatment effects given a particular dataset.
4.2 ESTIMATING EXPERTISE
Figure 5 shows that, although expertise is oracle metric meaning it cannot be computed directly using observable data-it can be estimated by plugging potential outcome predictions, which all of our methods learn, into (2) and (3) Specifically, given a partition of the outcome space y 5 .U)..
the predictive expertise can be estimated by treating outcome predictions (Va) as discrete variables:
Epredd = 1- - -EE (0.1) sclk
log2
1 - / [..( (0.1)
lita==al log2
|
(6)
|#:0] -
Prognostic expertise can be estimated in a similar manner as well. All methods except Balancing per- form well in this task. This is no surprise as one of the training goals of Balancing is to remove all in- formation regarding the decision-maker maker's policy, including any indicators of its expertise. We see once again that Propensity is more sensitive to changes in expertise as Bext->Expert than Worst-> (since Best->Expert affects the magnitude of propensity scores whereas Worst -xpeet does not).
Notice that Action- prriii-preeiitive performs the best in estimating expertise universally across all contig- urations, hence it can be used to identify whether a dataset has predominantly predictive expertise or prognostic expertise, and according to the identified expertise type, choose between Action- predictive or Balancing for estimating treatment effects. Given a dataset D, we first estimate both Then,
Estimate Egve and Eyee using Achoos prediene
Dattoun TD
yos
Eeimase ris) using Ralowwing
Estimote 7(x) uxing Articc-rrrredeeitee
Figure 6: Flow diagram of Expertise-informed. informed. First, the dominant type of expertise present in dataset D is identified using Action Action-prediion predictive. Then, a suitable method for CATE estimation is selected accordingly.
Epred and Epng using Action-predictive we determine which treatment effect estimation method use based on the ratio Egre// Eprog If this ratio is larger than a threshold of 1/2, we use Action prreici-eeeieivve for treatment effect estimation, otherwise we use Balancing. We call this pipeline "Expertise-nformeer
Table 1: PEHE of various methods averaged across predictive, prognostic, and all dotasets. By accurately identifying the type expertise present datasets, and then selecting between action-predictivs and Bal- ancing for CATE estimation accordingly, Expertise-ine in- formed achieves the host-of-both-workee performance.
(b) Table T reports the performance of all our methods, including this new pipeline, averaged across the datasets gener- ated by all the policies we have considered so far that is Tagfhmis and Tred with varying {33 d) as Best Best-xxpertt and Experr We see that, by accurately identifying the prominent type of expertise present in dataset, and selecting the most suitable method between Action- iirionn-reeiiivve and Balancing for that dataset, Expertise- -inrrise---oooeeddde achieves the best-of-both-wordee performance.
Predictive Datasets
Proguostic
All
Method
Datasets
Datasets
Baseline
0.784 (0.130)
14833 (0.258)
1.134 (0.194)
Propensity Balancing Action-preedsetve
0.786 (0.126)
1.511 (0.259)
1.149 (0.193)
0.936 (0.131)
1.499 (0.243)
1.188 (0.187)
0.751 (0.128)
1.495 (0.259)
1.123 (0.194)
Expertise informed
0.751 (0.128)
1.439 (0.243)
1.096 (0.185)
5 CONCLUSION
In this paper, we introduced a technical language that defines what expertise should be conceptualized as in treatment effect estimation. Using this language, we have provided an initial demonstration of an important phenomenon: Encoding the intuition that decision makers are usually experts of their domain-particulain that they often have certain type of expertise-can act as an inductive bias in estimating treatment effects. In our experiments, we were able to identify what type of expertise is present in given dataset and select the most suitable treatment effect estimation method for that dataset accordingly. We hope that the definitions we have provided, along with our empirical demonstrations, will encourage future research to capture the same intuition with new approaches.

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
We have described the details of our experimental setup in Appendix for the simulation environ- ment and in Appendix for the benchmark algorithms. Moreover, the code for reproducing our main experimental results can be found at httpp:::/// pettps///gtthtie com/ httptuu....m////yyyeeeeaaee Eeteeeixxxperiiiee coomvvanders
We have provided rigorous proofs in Appendix of Propositions I and ( well the additional Propositions and 4 that appear in Appendix B which refer to the assumptions we have made in the main paper as needed.
and https: /tt:://gttttte
aaabaabbaa Expert
ACKNOWLEDGMENTS
We would like to thank Ioana Bica, Krzysztof Kacprzyk, Kasia Kobalczyk, Max Ruiz Luyten, Julianna Piskorz, and Andrew Rashbass for their comments, suggestions, and generous feedback. AH, QW and AC gratefully acknowledge funding from the United States Office of Naval Research (ONR), GlaxoSmithKline, and AstraZeneca respectively.
REFERENCES
Assaad, S., Zeng, S., Tao, C., Datta, S., Mehta, N., Henao, R., F., & Carin, L.., "Counterfactual rep- resentation learning with balancing weights," in International Conference on Artificial Intelligence and Statistics, 2021.
Ballman, K. V., "Biomarker: Predictive or prognostic?" Journal of Clinical Oncology: Official Journal of the American Society of Clinical Oncology, vol. 33, no. 33, pp. 3968-3971, 2015.
Betts, J. R. & Roemer, J. E., Equalizing opportunity through educational finance reform. Public Policy Institute California, 1999.
Bica, I., Alaa, A. M., Jordon, J., van der Schaar, M., "Estimating counterfactual treatment outcomes over time through adversarially balanced representations," in International Conference on Learning Representations, 2019.
Bica, I,, Alaa, A., & van der Schaar, M., "Time series deconfounder: Estimating treatment effects over time the presence of hidden confounders," in International Conference on Machine Learning. 2020.
Bica, I., Jarrett, D., Hüyük, A., & van der Schaar, M., "Learning "what-iff" explanations for sequential decision-makiee making," in International Conference on Learning Representations, 2020.
Caye, A., Swanson, J. M.. Coghill, D., & Rohde, L. A., "Treatment strategies for ADHD: An cvidence-based guide to select optimal treatment," Molecular Psychiatry, vol. 24, no. 3, pp. 390-408, 2019.
Centor, R. M., "To be a great physician, you must understand the whole story," Medscape General Medicine, vol. 9, no. 1, p. 59, 2007.
Cortes, C., Mansour, Y., & Mohri, M., "Learning bounds for importance weighting," in Conference on Neural Information Processing Systems, 2010.
Crabbé, J., Curth, A., Bica, I., & van der Schaar, M., "Benchmarking heterogeneous treatment effect models through the lens of interpretability." in Conference on Neural Information Processing Systems, 2022.
Curth, A. & van der Schaar, M., "On inductive biases for heterogeneous treatment effect estimation," in Conference on Neural Information Processing Systems, 2021.
Curth, A. & van der Schaar, M., "Nonparametric estimation of heterogencous treatment effects: From theory to learning algorithms," " in International Conference on Artificial Intelligence and Statistics, 2021.
10

---

Published as a conference paper at ICLR 2024
Curth, A. & van der Schaar, M., "In search of insights, not magic bullets: Towards demystification of the model selection dilemma in heterogenous treatment effect estimation," in International Conference Machine Learning, 2023.
Curth, A., Svensson, D., Weatherall, J., & der Schaar, M., "Really doing great estimating CATE? critical look at ML benchmarking practices in treatment effect estimationn" in Conference on Neural Information Processing Systems, 2021.
Dorn, & Guo, K., "Sharp sensitivity analysis for inverse propensity weighting via quantile balanc- ing," Journal of the American Statistical Association, vol. 118, pp. 2645-2657, 2022.
Du, X., Sun, L., Duivesteijn, W. Nikolaev, A., & Pechenizkiy, M., "Adversarial balancing based representation learning for causal effect inference with observational data," Data Mining and Knowledge Discovery, vol. 35, no. 4, pp. 1713-1738, 2021.
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. "Domain-adversaria training neural networks," The Journal of Machine Learning Research, vol. 17, no. 1, pp. 2096-2030, 2016.
Graham, A. J., Shrive, F. M., Ghali, W. A., Manns, B. J., Grondin, S. C., Finley, R. J., & Clifton, J., "Defining the optimal treatment locally advanced esophageal cancer: A systematic review and decision analysis," The Annals of Thoracic Surgery, vol. 83, pp. 1257-1264, 2007.
Hassanpour, N. & Greiner, R., "Counterfactual regression with importance sampling weights," in International Joint Conference on Artificial Intelligence, 2019.
Holt, S., Hüyük, A., Qian, Z., Sun, H., & van der Schaar, M., "Neural laplace control for continuous- time delayed systems," in International Conference Artificial Intelligence and Statistics, 2023.
Holt, S., Hüyük, A., & van der Schaar, M., "Active observing in continuous- time control." in Conference on Neural Information Processing Systems, 2024.
Huang, Y., Leung, C. H., Ma, S., Wu, Q., Wang, D., & Huang. Z., pooderately-balancee representa- tion learning for treatment effects with orthogonality information," in Pacific Rim International Conference on Artificial Intelligence, 2022.
Hüyük, A., Jarrett, D., & van der Schaar, M., "Explaining by imitating: Understanding decisions by interpretable policy learning," in International Conference on Learning Representations, 2021.
Hüyük, A., Jarrett, D.. & van der Schaar, M., "Inverse contextual bandits: Learning how behavior evolves over time," in International Conference on Machine Learning, 2022.
Hüyük, A., Zame, W. R., & van der Schaar, M., "Inferrnng lexicographically ordered rewards from preferences." Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
Hüyük, A., Qian, Z., & van der Schaar, M., "Adaptive experiment design with synthetic controls," in International Conference Artificial Intelligence and Statistics, 2024.
Jarrett, D., Hüyük, A., & van der Schaar, M., "Inverse decision modeling: Learning interpretable representations of behavior," International Conference on Machine Learning, 2021.
Johansson, F., Shalit, U., & Sontag, D., "Learning representations for counterfactual inference," in
International Conference Machine Learning, 2016.
Johansson, F. D., Sontag, D., & Ranganath, R., "Support and invertibility in domain-invarian representations, in International Conference on Artificial Intelligence and Statistics, 2019.
Kapteyn, A., "Utility and economics." " De Economist, vol. 133, no. 1, pp. 1-20, 1985.
Künzel, S. R., Sckhon, J. S., Bickel, P. J., & Yu, B., "Metalearners for estimating heterogencous treatment effects using machine learning," Proceedings of the National Academy Sciences. vol.
116, no. 10, pp. 4156-4165, 2019.
Kwak, D. W., Sherwood, C., & Tang, K. K., "Class attendance and learning outcome," Empirical Economics, vol. 57, pp. 177-203, 2019.
11

---

Published as a conference paper at ICLR 2024
Ladd, H. F. & Yinger, J., "The case for equalizing aid," National Tax Journal, vol. 47, no. 1, pp. 211-224, 1994.
Newman, D., "Bag of words data set," UCI Machine Learning Respository, 2008.
Pearl, J., Causality, Cambridge University Press, 2009.
Qin, Y., Imrie, F., Hüyük, A., Jarrett, D., van der Schaar, M. el al., "Closing the loop in medical decision support by understanding clinical decision-making: A case study on organ transplantation." in Conference on Neural Information Processing Systems, 2021.
Rosenbaum, P. R. & Rubin, D. B., "The central role of the propensity score in observational studies for causal effects," Biometrika, vol. 70, no. 1, pp. 41-55, 1983.
Rosenshine, B., "Principles of instruction: Research- based strategies that all teachers should know." American Educator, vol. 36, no. 1, p. 12, 2012.
Rubin, B., "Causal inference using potential outcomes: Design, modeling, decisions," Journal of the American Statistical Association, vol. 100, no. 469, pp. 322-331, 2005.
Schwab, P.. Linhardt, L., Bauer, S., Buhmann, J. M., & Karlen, W., "Learning counterfactual representations for estimating individual dose-response curves," in Proceedings of the AAAI Conference Artificial Intelligence, 2020.
Sechidis, K., Papangelou, K., Metcalfe, D., Svensson, D., Weatherall, J., Brown, G., "Distinguish- ing prognostic and predictive biomarkers: An information theoretic approach." Bioinformatics, vol. 34, no. 19, pp. 3365-3376, 2018.
Shalit, U., Johansson, D., & Sontag, D., "Estimating individual treatment effect: Generalization bounds and algorithmms," in International Conference on Machine Learning. 2017.
Shi, C., Blei, D., & Veitch, V., "Adapting neural networks for the estimation of treatment effects," in Conference on Neural Information Processing Systems, 2019.
Sun, H., Hüyük, A., Jarrett, D., & van der Schaar, M., "Accountability in offline reinforcement learning: Explaining decisions with a corpus of examples," in Conference on Neural Information Processing Systems, 2024.
Sutton, R. S. & Barto, A. G., Reinforcement learning: An introduction. MIT Press, 2018.
Wager, S. & Athey, S., "Estimation and inference of heterogeneous treatment effects using random forests," Journal of the American Statistical Association, vol. 113, no. 523, pp. 1228-1242, 2018.
Weinberg, C. R., "Can DAGs clarify effect modification?" Epidemiology, vol. 18, no. 5, 569, 2007.
Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R., Ozenberger, B. A., Ellrott, K., Shmule- vich, I., Sander, C., & Stuart, M., "TTe Cancer Genome Atlas Pan-Cancer analysis project," Nature Genetics, vol. 45, no. 10, pp. 1113-1120, 2013.
Yao, L., Li, S., Li, Y., Huai, M.. Gao. J., & Zhang, A., "Representation learning for treatment effect estimation from observational data," in Conference on Neural Information Processing Systems, vol. 31, 2018.
12

---

Published as a conference paper at ICLR 2024
A FURTHER DISCUSSION
Why entropy over statistical measures? Our definitions of expertise are based on the concept of entropy. An alternative approach could have been to define expertise using statistical measures such as variance. Although variance, similar to entropy, is also indicative of how "random" a random variable is, the two quantities measure fundamentally different things: Variance quantifies the amount of spread around a mean while entropy quantifies uncertainty. When defining expertise, our aim was capture the uncertainty of outcomes when the actions of packsion-mmaer are known vs. unknown to an outside observe (the bigger the difference between the two cases, we say the larger the expertise is). Hence, a more direct measure uncertainty was naturally more suitable our aim and use case.
On a more technical level, attempting to define expertise through variance would have had two immediate shortcomings:
(i) First, variance is not well defined for categorical variables, such as binary actions in our work, unless we assign numerical values to each category. Even if we were to assign such numeric values (for instance, A = 0 for the negative treatment and A = for the positive treatment), the resulting variance would be sensitive to our assignments (for instance, the variance of A would have increased if we were to represent the negative treatment as A = -1 instead of A" O, which should not be a meaningful change as far as expertise is concerned).
(ii) Variance depends on the scale of variables whereas entropy does not. For instance, the variance 2Y would be double the variance of Y, while their entropies would be the same: H[[] H[ Such sensitivity to scale is undesirable when defining expertise- the fact that outcomes are recorded twice as large in a dataset (maybe due to a change of units) should have no effect on expertise.
Why not use the graphical framework of Pearl?
We refrained from using the graphical framework Pearl (2009) when defining expertise because it is specifically not well-equipped to distinguish between prognostic and predictive variables. This is because directional acyclic graphs (DAGs) are notoriously bad at representing effect modification (i.e. predictive variables) natively. There have been proposals to extend the graphical framework to better depict effect modifiers WWennerrg (2007). but the best of our knowledge, no solution has been well established in the literature to this data.
Expertise in terms of mutual information It should be mentioned that the expertise definitions we arrived at in Section 3 hhpppen to be exactly equal to the mutual information between actions and outcomes: For predictive expertise, Epmu = I[A;; Y; - Yo)/H[[7] and for prognostic expertise, Firng ((A [/[AA)
Expertise in experimental data Experimental data constitute a noteworthy extreme when viewed through the lens of expertise. Particularly a randomized controlled trial, when the propensity scores are uniform across treatment, there would be no expertise, and appropriately, both the predictive and the prognostic expertise would be equal to zero for datasets collected through randomized controlled trial (this can be inferred from Proposition [ a uniform policy Tumi/ would have " = 1 hence Exree 0 and Expor = 0). Consequently, expertise an inductive bias would of course be less helpful in estimating treatment effects using such datasets, however in that case, the datasets would already be ideal for treatment effect estimation with no confounding bias (the best case scenario in Figure I This is why we mainly focused on the high expertise, low overlap setting (the amber region in Figure D
While taking advantage of expertise would not be possible for datasets collected through a randomized controlled trial, estimating expertise (as in Section 4.2) can still act as a data-driven way to determine whether the data we have is effectively randomized or not: The closer both Epred and Eprog are the closer the data would be to trial data, in which case we might prefer conventional supervised learning methods over algorithms specialized for treatment effect estimation.
B THEORY OF EXPERTISE
We have mentioned two fairly straightforward facts regarding expertise without formally stating and proving them: (i) the fact that predictive expertise implies prognostic expertise (in Section (.T). and (ii) the fact that having zero in-context action variability implies that the overlap assumption violated (in Section 3.2). In this section, we formally state these facts as Proposition and Proposition respectively with accompanying formal proofs in Appendix
13

---

Published as a conference paper at ICLR 2024
Proposition 3. Eprdd I for all A((0.1))*
Proposition 4, C 0 implies that π (x))[a] for some X, {0,1}.
C PROOFS OF PROPOSITIONS
Proof of Proposition 1
We unify the proof by letting Z denote (Yo, Y1) in the case of progmostic
expertise and Y, Yn in the case of predictive expertise. Then, for both types of expertise, we have
E" +
H ||
[[A
[| HI[A"]
A /
H
A4)
HI |X 2] H[A*]
(8)
S H[A
HA
H[AA| H[A**]
(9)
=1
where 7 is by definitions of E" and C holds since Z 1 A | according our problem setup, and 9 is because conditioning never increases entropy.
Proof of Proposition| 2
This proposition is corollary of Proposition I If policy л is perfect expert such that E" 1. then Ex + + < 1 implies that C" (since C E [0, 1).
Proof of Proposition 3
We have
Eprd
H[A Yol H[[*
S H[A Yo. Y1, Y1 - Yo) H[AA]
(10)
1 - H[A" vi) H[[A"]
(11)
Prrpog where (T0) is because conditioning never increases entropy, and 11 holds since Yi - Yo is deterministic function of (Yo, Y1) hence Yi Yo JL A" (Y, Yi.
Proof of Proposition 4
C = / = 0, then
Σ aEx,
P(A" X x) log P(A" = a|X = x}}
Σ aEX alz] (- log (x)
(12)
First, notice that both a[z] and [x) log [x) are non-negative for all x E X, E {0, 1}.
hence each summand in (12) must be equal zero (for the overall sum also be equal to zero). Now,
consider some x² E X' such that a<xx > 0. This would imply that - -T(x") [a] log T(x" {[ = 0
for all a E (0, 1}, which in turn would imply that x (xx") [a] or क (<=
for all (0, 1).
Since it cannot be the case that (2 = π ((x" [1] = 1, there exists some a" E {0, such that (a" [a*] O.
D ADDITIONAL EXPERIMENTS
In the main paper, we focused on single simulation environment based on the covariate from the TCGA dataset (Weinstein al. 2013 Schwab al., 2020). this section, we repeat our experiments for two more environments: (i) Linear News, which is based the covariates from the News dataset (Newman 2008), and (ii) Non-linear TCGA, which is still based on the TCGA dataset, but in which, outcomes have non-linear relationship to features. In Non-linear TCGA, outcomes are generated as
Y, (Mlppg p 32 /2
(Ppoa)? ///2 + 7l
(13)
The results for Linear News are given in Figures 7 and Table 2 while the results for TCGA are given in Figures
9 I0, and Table 3 These results mostly support the same conclusions as the still achieves the best-of- both- worlds performance in Linear News, it fails to do so in Non-linear TCGA. This is because, Non-linear TCGA, Balancing no longer happens to be the best performing method still
main results in Figures 5 and Table T One notable exception that, while Expertise-infon
under prognostic expertise, which Expertise-informer relies on. Otherwise, correctly identifies the type of expertise present in datasets as it still matches the performance of Action predictvee under predictive expertise and of Balancing under prognostic expertise. Regarding why Balancing does not perform as well in Non-linead TCGA, this might be a side effect of the fact that we implemented all algorithms specifically linear outcomes in mind (see Appendix F for details).
14

---

Published as a conference paper at ICLR 2024
0.0
Adoe-
podd
Pag Sew.he
(a) Expe-+ for predictive expertise
(b) Worst-> Expert for predictive expertise
(c) Best- Expert for prognostic expertise
(d) Expst-Expeett for prognostic expertise
Figure 7: Performance improvements Baseline in the Linear News environment.
Jalue prabitune
018
123
(a) Expert for predictive expertise
(b) Worst Expert for predictive expertise
(c) Best- Best->Expert Expert
(d) Worst Exper- for prognostic expertise
for prognostic expertise
Figure 8: Estimated predictive/prognostic expertise in Linear News environment.
+MC
085
Topenct
300
(a) Best->Expent for predictive expertise
(b) Worst-Exper
(c) Best Expert for prognostic expertise
(d) Worst->xxpen
for predictive expertise
for prognostic expertise
Figure 9: Performance improvements over Baseline in the Non-linear TCGA environment.
Tambee
Tanler
d Falecie
Ealecae
Edlic
(a) Best- Expert for predictive expertise
(b) -xpsttExxpert for predictive expertise
(c) Best Expert for prognostic expertise
(d) Worst Worst-Exxerrt for prognostic expertise
Figure 10: Estimated predictive/prognostic expertise the Non-linear TCGA environment.
Table 2: PEHE of various methods averaged across datasets from the Linear News environment.
Table 3: PEHE of various methods averaged across datasets from the Non-linear TCGA environment.
Predictive Datasets
Proguostic
Predictive
Proguostic
Method
Datasets
Datasets
Method
Datasets
Datasets
Datasets
0.495 (0.082)
(0.155)
(0.1 18))
Raseline
(0.0699
(0.062)
Propensity
0.786 (0.126)
1.511 (0.259)
1.149 (0. (0.193)
Propensity
0.408 (0.069)
0.315 (0.057)
0.361 (0.063)
Balancing
(0.083)
Balancing
Action-prodictive
0.465 (0.085)
0.839 (0.154)
0.652 (0.119)
Action-predictive
0.392 (0.069)
0.305 (0.057)
0.349 (0.063)
Expertise informed
0.465 (0.085)
(0.148)
(0.117)
Expertise informed
(0.069)
0.320 (0.055)
0.356 (0.062)
15

---

Published as a conference paper at ICLR 2024
E DETAILS OF THE SIMULATION ENVIRONMENT
In the environments based on the TCGA dataset, we used the measurements from the 100 most variable genes- are all continuous features and the data is log-normalized with each feature scaled in the [0, 1J interval. Meanwhile, the News dataset consists of 10,000 randomly samples news items, with 2858 word counts. Similar to Shalit et al. (201)). we perform principal component analysis (PCA) and use the first 100 principal continuous features as covariates. In the case of both dataset, we decompose the feature space as (Xpmg- Xprvt, Xir) E 1R-0040x200 that is dpmg dprd 40 and dir = 20. During this decomposition, we choose which features are prognostic, predictive, or irrelevant completely at random.
We consider variety of policies in our environments, namely Rzaft and mis as the predictive settings, and Toon-prrd as the prognostic settings, In the predictive setting, for Exper+ we fix d and vary / E {0.25, 0.28, 0.33, 0.40, 0.50, 0.66, 1, 10), and for Worst->Expert, we fix B 0.25 and vary d E {0, 2, 4, ..., dar = 30) (Figure 2) In the prognostic setting, for Best Dest--Expert, we fix d d 2 10 and vary 3 E {0.25, 0.28, 0.33, 0.40, 0.50, 0.66, 1, 2, 10], and for Exper, we fix B = 0.25 and vary d E (0, 2, 2...2....dde din = 20}.
F DETAILS OF THE BENCHMARK ALGORITHMS
We benchmark across four models: TARNet (i.e. Baseline), TARNet with IPW (i.e. Propensity), CFRNet (i.e. Balancing), and DragonNet (i.e. Action-predictive) We use the PyTorc implementa- tions of these models provided in the python package CATENets (Curth & van der Schaar 2021a). All models have representation network (i.e. function Φ) with one hidden layer with 100 hidden units, and linear prediction layers (i.e. function f) for potential outcomes as well as action predictions when applicable. We use dense layers with the ReLU activation function. All models are trained using the Adam optimizer with learning rate 0.001, batch size 1024, and early stopping on validation set, where we employ a standard vran-validation split of -0%% We used a virtual machine with six 6-Core Intel Xeon E5-2690 v4 CPUs, one Tesla V100, and 1 110BB of RAM to run all experiments. For all experiment results, we average over seeds to obtain error bars.
Computing the ground-truth expertise well as estimating it using any one of our benchmark algorithms would normally require knowing the marginal distributions of potential outcomes Yo, Y1. Since do not have access to these marginal distributions, we rely on numerical approximation instead. Whenever we need to compute an expertise measure for a dataset D, we first build a histogram of the observed outcomes {v'} } D by letting numpy automatically determine which bins to use: y = 3i1 U... U Vr. Then, we discretize outcome predietionssobservatoon (v6) and (vt) according those bins, which leaves us with approximate categorical distribution of potential outcomes Yo, Yi to compute the expertise with as follows:
1h -0-aa to wi e
b ka'=aw of E>/a
Eprog 1 Eae
log2
Keg E)/,1
(14)
Eoe(o,1)
bo-'=ol
log,
Ria'=a
Epred = 1 Enc (0, []((())())))
Ie =a Ei
log2
Kia iin'zan v -WjE Feyt M0E
(15)
Exe(0.1)
1:=0==0
log,
=a
16


Reviews:
[{"review_id": "qkO3uhkiBK", "reviewer": "Reviewer_7Wu5", "summary": "The paper discusses the notion that expert decision-makers can implicitly use their domain knowledge when choosing actions, like prescribing treatments. These actions, in turn, can reveal insights about the domain. For example, frequently prescribed treatments might be more effective. However, current machine learning methods may fail to capitalize on the concept of \"expertise\" as a guiding factor. Specifically, in the context of estimating treatment effects, the prevailing assumption is simply that treatments overlap without considering expert behavior. The paper proposes that recognizing two types of expertise: (1) predictive and (2) prognostic, possessed by decision-makers may enhance the methodology and selection of models for estimating treatment effects. The authors show that understanding the predominant expertise in a domain may significantly impact the performance of these models and that it is possible to identify the type of expertise in a dataset to inform model choice.", "questions": "1. The paper proposed two novel concepts to define the expert knowledge: (1) Prognostic expertise: $\\mathbb{E}^{\\pi}_{prog} = 1 - \\mathbb{H}[A^{\\pi}|Y_0, Y_1] / \\mathbb{H}[A^_{\\pi}]$. (2) Predictive expertise: $\\mathbb{E}^{\\pi}_{pred} = 1 - \\mathbb{H}[A^{\\pi}|Y_1 - Y_0] / \\mathbb{H}[A^_{\\pi}]$. The rationale for selecting entropy over other statistical measures, such as the variance of actions, is not immediately clear. Can you elucidate the benefits of employing entropy in this context? What are the specific advantages of entropy in capturing the nuances of expert decision-making, and under what conditions does it outperform simpler measures like variance?\n\n2. The consideration of domain expert knowledge in personalized decision-making problems is crucial but challenging, especially given that such expertise may not always be accurate or easily discernible in real data sets. How might one effectively balance the integration of domain knowledge with data-driven insights to mitigate the risk of relying on potentially inaccurate expertise? Is there a methodology within your framework that utilizes data-driven approaches to validate and calibrate the contributions of domain expertise, particularly through the lens of prognostic and predictive expertise measures?\n\n3. In the context of clinical trials, where treatment assignments are typically randomized and propensity scores are uniformly distributed. Then, the application of expert knowledge is less straightforward. Considering such scenarios, how do the proposed prognostic and predictive expertise metrics maintain their utility? Can these metrics still provide meaningful information for personalized treatment recommendations when the baseline assumption of expert influence on treatment assignment is removed? How might these metrics be adapted or interpreted in a randomized trial environment to enhance personalized medicine approaches?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper presents the following advantages: \n\nFirstly, it effectively communicates its proposed concepts through relatable examples, such as those from the medical and teaching fields, introducing and distinguishing between predictive and prognostic expertise. \n\nSecondly, it addresses an intriguing problem that may often be overlooked in causal inference: the incorporation of domain knowledge. By shedding light on this issue, the paper emphasizes the importance of considering expert behavior in the analysis, which could lead to more accurate and informed causal inferences.", "weaknesses": "Please see more details in the Questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BtlwK9OVPi", "reviewer": "Reviewer_ScD8", "summary": "THis paper extends methods for estimating causal treatment from observational data in the presence of expert actions, e.g. the treatment decisions of a clinician.  There is value in the information available from the expert's actions;  What can we learn about the effectiveness of treatment from their actions as domain experts? As mentioned in the  paper \"These methods become susceptible to confounding... and treatments (are assigned) based on factors that infuend the outcomes...\" This is related to \"confounding by indication\" [ Salas, M., Hotman, A., Stricker, B.H.: Confounding by indication: an example of variation in the use of epidemiologic terminology. American journal of epidemiology 149(11), 981–983 (1999)]. Confounding occurs because the expert's action conditions both the treatment applied and the outcome\n\nThe paper makes a distinction between \"predictive\" expertise (knowledge of likely outcome,  specifically of Y_1 - Y_0, as occurs in healthcare) and \"prognostic\" expertise. (knowledge of potential outcomes, Y_1, Y_0 as occurs in education) Prognostic expertise implies predictive, since Y_1 - Y_0 can be determined by knowlege of Y_1 and Y_0. \n\nFor this to work one must take into account \"overlap\" - the variability due to the decision-maker's imperfect knowledge; equivalently the possibility of perfect expertise.  The paper makes the point that one needs the additional distinction between \"predictive\" and \"prognostic.\"", "questions": "In this conference venue why not draw upon the ML literature on causality (e.g. Pearl's work) and its use of structural equation models? I presume this touches on a long standing discussion that encompasses much more than just the claims of the paper, but such a comparison would open the field to a larger audience, and possibly, by recourse to additional domain knowledge elucidate why it is so that \"Cases where more information can be gained through expertise (i.e. cases with high expertise) happen to align with cases where treatment effect estimation is particularly hard.\" (But other methods may be able to identify the causal effect.)", "limitations": "", "rating": 5, "confidence": 2, "soundness": 4, "presentation": 2, "contribution": 1, "strengths": "The paper shows that it is possible to take expertise into account and exploit the value of a decision-maker's inductive bias. It concludes that the type of expertise affects the performance of methods for estimating treatment effect (because such methods rely on the difference Y_1 - Y_0,?) , and that the difference in type of expertise may be evident as a way to distinguish datasets.", "weaknesses": "The prognostic - predictive distinction on which the paper depends is so subtle that it is hard to see how it is of any consequence.  This may be just a lack of clarity and familiarity with the current economic literature on causality, but it seems to rely on a strong claim that characteristically clinical treatment decisions would be ignorant of outcomes, but only of the _difference_ in outcomes, a claim whose reasons are obscure.  Both types of expertise are defined in terms of expectations of actions conditioned on outcomes as in Equations (2) and (3) -- definitions that leave this question of the distinction open. \n\nSuch formulation of expertise  as '.. to what extent the actions of a decision-maker are informed by what a subject’s potential outcomes\" seem to contradict the basic notion that experts \"assign treatments based on factors that influence the outcomes.\" What influences what?  In simple terms, the decision-maker's action depends on their state of information -- the features known to them at the time they make their decision. One presumes that thinking of actions influenced by subsequent outcomes is a convention in the economics literature on causality.  This may be necessary for the methods of analysis but it is far from intuitive. \n\nIn summary, the significance of the paper relies on a questionable distinction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q1CvJ0enp4", "reviewer": "Reviewer_obyv", "summary": "This paper introduces the idea that selection bias due to expertise might be informative in designing and selecting methods for the conditional treatment effect (CATE) estimation.\n\nThey differentiate between two types of expertises, which are formally defined using the concept of entropy for actions induced by a certain policy. On the one hand, they define *prognostic expertise* where actions are based on all potential outcomes. It is given as one minus the relation of the entropy conditional on the potential outcomes and the total entropy. On the other hand, *predictive expertise* refers to the case where actions are based on the treatment effects, and is defined analogously as one minus the relation of the entropy conditional on the treatment effect and the total entropy. Note that these definition bound the expertise between zero and one with zero being no expertise and one being perfect expertise.\n\nThey draw connections from expertise to the validity of the overlap assumptions. In particular, the higher the expertise of a policy, the lower is the overlap.\n\nThey perform multiple experiments using semi-synthetic data and give intuitive explanations for the different performance of state-of-the-art CATE estimation methods.\n\nBased on the theoretical definition of expertise, they propose an estimator for the predictive and prognostic expertise and a pipeline, called \"expertise-informed\" to that automatically chooses a suitable CATE estimator based on the dominant expertise type.", "questions": "*Definition of Expertise*\n\n1. The entropy is defined with log to base 2. Do you decide on this because the actions are binary, i.e. $A\\in$ { 0,1}?\n\n2. Could you formally prove that $C^{\\pi} =0$ implies the violation of the overlap assumption?\n\n3. The comment (i) after Proposition 2 is unclear to me. Could you provide an example for that?\n\n*Application*\n\n4. Is it possible to give more formal arguments how the discussed CATE estimators (TARNet, IPW, CFRNet, DragonNet) are related to the entropy definition of expertise?\n\n5. Are the results sensitive to the choice of network based CATE estimators in the analysis?\n\n*Estimating Expertise*\n\n6. Could you please provide a more detailed description of your pipeline process, in particular whether the same samples are used for estimating expertise and treatment effect? Do you anticipate pre-testing problems in this case?\n\n7. How sensitive are the outcomes of the pipeline to the threshold 1\\2?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "1. The paper seems to be novel in introducing the idea that a specific type of selection bias, namely allocation done by experts, is informative.\n\n2. The paper is very well-written. It provides a great intuition why the idea of formally introducing expertise is fundamental and how it is related to other quantities of interest such as optimality or overlap assumption.\n\n3. The proofs are very detailed and easy to follow.", "weaknesses": "*Definition of Expertise*\n\n1. The relationship between in-context action variability and the overlap assumption is just intuitively explained. A formal proof for this relationship is missing.\n\n2. In Figure 1 the axis tick values are missing. (The ticks are also confusing as for $C^{\\pi}=0$, the expertise is somewhere between ticks.)\n\n*Application*\n\n3. The authors explain in great detail why some CATE estimator perform intuitively better under some type of expertise than others. A formal proof for this intuition is however missing.\n\n*Estimating Expertise*\n\n4. It is unclear in the proposed pipeline whether the expertise is estimated on same samples used for training or on some hold-out set and what the consequences of this choice have on the outcomes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KTvvbnQggZ", "reviewer": "Reviewer_uYNJ", "summary": "This paper studies the role of expertise in treatment effect estimation. They formalize predictive (actions dependent on treatment effect) and prognostic expertise (actions dependent on potential outcomes) and argue that knowledge of the type of expertise provides a useful inductive bias for selection of a treatment effect estimation method. They further show that the type of expertise present in a dataset can be estimated observationally with a plug-in method, suggesting a meta-algorithm whereby the type of expertise is estimated and the more appropriate method chosen. Experiments are conducted with sweeps over the amount of each type of expertise in the data. The results have implications particularly for the appropriateness of methods that learn balancing representations, as these are shown to worse than other methods in cases with predictive expertise (as the balancing removes information about the treatment effect from the representation) and better than other methods in cases with prognostic expertise (as the shifts present are unrelated to the treatment effect).", "questions": "* As a suggestion, it would interesting to understand the relationship between the issues identified with balancing representations for treatment effect estimation in the presence of predictive expertise with the consistency issues identified with balanced representations for domain adaptation (see Johansson 2019 https://proceedings.mlr.press/v89/johansson19a.html).\n* Please elaborate on the relationship the expertise measures and mutual information and the motivation for defining new measures given the relationship between them.\n* I don’t follow the argument in section 3.1 regarding the relationship between the two types of expertise. It seems odd to say that predictive expertise implies prognostic expertise, because a policy with predictive expertise (dependent only on the treatment effect) would be insensitive to a constant shift in the potential outcomes while a policy with prognostic expertise would be sensitive to it.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "* This work offers a novel perspective and analytic framework for reasoning about expertise in treatment effect estimation problems. I found it particularly insightful in the context of reasoning about the causes of covariate shift and overlap violations across treatment strata and the implications for estimation. \n* The paper is well-written and clear, with high-quality visualizations.\n* The experiments are well-designed and convincing in supporting the theoretical claims with empirical evidence.", "weaknesses": "The paper takes a somewhat narrow lens with respect to the breadth of relevant prior work considered. It largely focuses on a particular lineage of treatment effect estimation methods that use neural networks for representation learning, balancing, and fitting potential outcome models. The work would be stronger if it could be contextualized better in the full landscape of work on causal inference for treatment effect estimation across fields, including statistics, epidemiology, and econometrics. \n\nIt would be particularly prudent to try to understand how this work fits into the broader landscape of approaches that aim to learn doubly-robust estimators that only require one of either the treatment or outcome models correctly and do not require any notion of balancing beyond overlap. For example, see the Augmented Inverse Propensity Weighted Estimator (e.g., Glynn and Quinn 2010) and Targeted Maximum Likelihood Estimation (Schuler and Rose 2017)\n\nReferences\n* Glynn, Adam N., and Kevin M. Quinn. \"An introduction to the augmented inverse propensity weighted estimator.\" Political analysis 18.1 (2010): 36-56.\n* Schuler, Megan S., and Sherri Rose. \"Targeted maximum likelihood estimation for causal inference in observational studies.\" American journal of epidemiology 185.1 (2017): 65-73.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)