Individualised Dose-Response Estimation using Generative Adversarial Nets
Authors: Ioana Bica
James Jordon
Mihaela van der Schaar
Venue:  (2020)
Keywords: individualised dose-response estimation
treatment effects
causal inference
generative adversarial networks

Abstract: The problem of estimating treatment responses from observational data is by now a well-studied one. Less well studied, though, is the problem of treatment response estimation when the treatments are accompanied by a continuous dosage parameter. In this paper, we tackle this lesser studied problem by building on a modification of the generative adversarial networks (GANs) framework that has already demonstrated effectiveness in the former problem. Our model, DRGAN, is flexible, capable of handling multiple treatments each accompanied by a dosage parameter. The key idea is to use a significantly modified GAN model to generate entire dose-response curves for each sample in the training data which will then allow us to use standard supervised methods to learn an inference model capable of estimating these curves for a new sample. Our model consists of 3 blocks: (1) a generator, (2) a discriminator, (3) an inference block. In order to address the challenge presented by the introduction of dosages, we propose novel architectures for both our generator and discriminator. We model the generator as a multi-task deep neural network. In order to address the increased complexity of the treatment space (because of the addition of dosages), we develop a hierarchical discriminator consisting of several networks: (a) a treatment discriminator, (b) a dosage discriminator for each treatment. In the experiments section, we introduce a new semi-synthetic data simulation for use in the dose-response setting and demonstrate improvements over the existing benchmark models.

Under review as a conference paper at ICLR 2020
INDIVIDUALISED DOSE-RESPONSE ESTIMATION USING GENERATIVE ADVERSARIAL NETS
Anonymous authors Paper under double-bling review
ABSTRACT
The problem of estimating treatment responses from observational data is by now a well-studied one. Less well studied, though, is the problem of treatment response estimation when the treatments are accompanied by continuous dosage parameter. this paper, we tackle this lesser studied problem by building on a modification of the generative adversarial networks (GANs) framework that has already demonstrated effectiveness in the former problem. Our model, DRGAN, is flexible, capable of handling multiple treatments each accompanied by dosage parameter. The key idea is to use a significantly modified GAN model to generate entire dose-response curves for each sample in the training data which will then allow us to use standard supervised methods to learn an inference model capable of estimating these curves for a new sample. Our model consists of 3 blocks: (1) a generator, (2) a discriminator, (3) an inference block. In order to address the challenge presented by the introduction of dosages, we propose novel architectures for both our generator and discriminator. We model the generator as multi- task deep neural network. treatment space (because of the addition of dosages), we develop hierarchical discriminator consisting of several networks: (a) a treatment discriminator, (b) a dosage discriminator for each treatment, In the experiments section, we introduce a new semi-synthetic data simulation for use in the dose-response setting and demonstrate improvements over the existing benchmark models.
order to address the increased complexity of the
INTRODUCTION
Most of the methods developed in the causal inference literature focus on learning the effects of binary or categorical treatments (Bertsimas et al.) 2017 Alaa et al. 2017 Alaa & van der Schaar 2017 Athey & Imbens 2016 Wager & Athey| [2018] Yoon et al. 2018p. These treatments, though, are often administered at a certain dosage which can take on continuous values (such as vasopressors (Döpp-Zemel & Groeneveld 2013)). In medicine, using high dosage of drug can lead to toxic effects while using a low dosage can result in no effect on the patient outcome (Wang et al. (017). Moreover, the dosage levels used when choosing between multiple treatments for patient crucial for the decision (Rothwell et al. 2018).
While admissible dosage intervals for drugs are often determined from clinical trials (Cook et al. 2015), these trials often have a small number of patients and use simplistic mathematical models to assign dosage levels patients that do not take into account patient heterogeneity (Ursino al., 2017). After drugs are approved through clinical trials, observational data collected about different treatment dosages prescribed to diverse set of patients offers us the opportunity learn individualized responses. As the relationships between treatment dosage efficacy, toxicity and patient features become more complex, estimating dose-response from observational data becomes particularly important in order to identify optimal dosages for each patient. Fortunately, there is wealth of observational data available in the medical domain from electronic health records (Henry et al. 2016))
Learning from observational data already presents significant challenges in the binary treatment setting. As explained by Spirtes (2009). in an observational treatment-effect dataset, only the factual outcome is present (i.e. the outcome for the treatment that actually given) the counterfactual outcomes are not observed. This problem is exacerbated in the dose-response setting in which the number of counterfactuals is longer even finite. Moreover, the treatment assignment is non-random

---

Under review as a conference paper at ICLR 2020
DRGAN Discriminttr
Factual onteome Counerfictual Hockgropogatior
GANITE inistimnnttrr
Preimn cusome
Palimn onteome
Patienfouttoome
for treammeett
Parlent onfcoene treatrient
Dasgy
Dosage
Treatment
Tiratment
DRGAN Conmerfactual
GANITE Counteriactual
Ganerator
Generator
Figure I: Comparison of DRGAN and GANITE highlighting the key difference between the two dif- ferent problems they address (dose-response estimation vs. standard rreatmen-rrespone estimation).
and instead is assigned according to the features associated with each sample. Due to the continuous nature of the dosage parameter, adjusting for the bias in the dosage assignments is significantly more complex than for binary (or even multiple) treatments. Thus, standard methods for adjusting for treatment selection bias cannot be easily extended to handle bias in the dosage parameter.
In this paper we address the problem of dose-response estimation from observational data by building on the framework introduced in GANITE (Yoon et al. 2018). The key idea is to modify the GAN framework (Goodfellow et al. 2014) to generate the unobserved counterfactual outcomes from a standard treatment-effect dataset. Already, GANITE presents a significant modification to the original GAN framework rather than the discriminator discriminating between entirely real or entirely fake samples, the discriminator is attempting pick out the real component from vector containing the real (factual) outcome from the dataset and the fake (counterfactual) outcomes generated by the generator. We also inherit this key difference from standard GAN, but in addition must make further modifications to the original GANITE framework in order to address the dosage problem.
naive attempt to extend Yoon et al. 20188) to the dosage setting might involve trying to define discriminator that takes as input an entire dose-response curve for each treatment from the generator (with the outcome for the observed treatment-dosage pair replacing the generated one) and that tries to determine the factual treatment to assume prior knowledge of the functional form of the dose-response curves and so will have access to the generated dose-response curves only by evaluating them at given points (and so "entire" dose-response curves cannot be passed to the discriminator); (2) substituting the generator output for the factual treatment-dosage pair with the factual outcome will almost always create discontinuity in the response curve and thus the factual treatment-dosage pair would be very easy to identify.
pair. This fails for two reasons: (1) we do not wish
We overcome these two hurdles by defining discriminator that, rather than acting on the entire dose-response curves, acts on finite set of points from each curve, as shown in Fig. From among the chosen points, the discriminator will then attempt to pick out the factual one. To ensure that the entire dose-response curve is well-estimated, we sample the set of points randomly each time an input would be passed the discriminator. If we were to fix a set of points in advance to compare for all treatments and samples then only the outcomes associated with these dosage levels would be well estimated. As our discriminator will be taking as input set of random dosage-outcome pairs, we need to condition its behaviour to be like that of a function on set. In particular, we draw on ideas from Zaheer et al. (2017) to ensure that the discriminator acts as a function on sets and its output does not depend on the order in which the elements of the set are given as input.
In addition, we model the generator as multi task deep network capable of taking dosages as an input; this gives us the flexibility to learn heterogeneous dose-nes curves for the different treatments. We also develop hierarchical discriminator which breaks down the job of the discriminator into determining the factual treatment and determining the factual dosage using separate networks. We show in the experiments section that this approach significantly improves performance and is more stable than using single network discriminator.
Our contributions in this paper are 3-fold: (1) we propose DRGAN, a significantly modified GAN framework, capable of dose-resme response estimation, (2) propose novel architectures for each of our networks, (3) we propose a new semi-synthetic data simulation for use in the dose-response setting. We show, using semi-synthetic experiments, that our model outperforms existing benchmarks.
2

---

Under review as a conference paper at ICLR 2020
2
RELATED WORK
Methods for estimating the outcomes of treatments with an exposure dosage parameter that only employ observational data make use of the generalized propensity score (GPS) (Imbens] [2000] Imai & Van Dyk, 2004] Hirano Imbens (004) or build on top of balancing methods for multiple treatments. Schwab et al. (2019) developed a neural network based method to estimate counterfactuals for multiple treatments and continuous dosages. The proposed Dose Response networks (DRNets) in Schwab al. (2019) consist of a three level architecture with shared layers for all treatments, multi-task layers for each treatment and additional multi-task layers for dosage sub-intervals. More specifically, for each treatment w, the dosage interval (aw. bw] is subdivided into E' equally sized sub-intervals and a multi-task head is added for each sub-interval. Their model architecture extends the one in (Shalit et al., (2017) by adding the multi-task heads for the dosage strata. However, the main advantage of using multi-task heads for dosage intervals would be the added flexibility in the model to learn potentially very different functions over different regions of the dosage interval. DRNets does not determine the dosage intervals dynamically and thus much of this flexbility is lost. We demonstrate in our experiments that DRGAN outperforms both GPS and DRNets.
For a discussion of works that address treatment-responss estimation without a dosage parameter, see Appendix A Note that for such methods we cannot treat the dosage as an additional input due the bias associated with its assignment.
3 PROBLEM FORMULATION
We consider receiving observations of the form (x', vf) for i I..., .N, where, for each i, these are independent realizations of the random variables (x,Ty,Y)) Yy). We refer to X as the feature vector lying in some feature space X, containing pre-treatment covariates (such as age, weight and lab test results). The treatment random variable, Ty, is fact a pair of values T, = (W,, D. Dy) where W E W corresponds to the rype of treatment being administered (e.g. chemotherapy radiotherapy) which lies in the discrete space of l treatments, W = [w],..., w }. and D, corresponds to the dosage of the treatment (e.g. number of cycles, amount of chemotherapy, intensity of radiotherapy), which, for a given treatment w lies in the corresponding treatment's dosage space, Da., which in the most general case some continuous space (e.g. the interval 1)). We define the set of all treatment- dosage pairs to be T = {w, d) w E W, d E Dw).
Following Rubin's 's potential outcome framework Rubin 1984), we assume that for all treatment- dosage pairs, (w, d). there is a potential outcome Y(w. d) y' (e.g. 1-year survival probability). The observed outcome is then defined be Y, Y(W,. Dy). We will refer to the unobserved (potential) outcomes as counterfactuals.
The goal of dose-response estimation is to derive unhiased estimates of the potential outcomes for a given set of input covariates:
1lt, x) = (t) x
(I)
for each t E T E X. We refer to p(-) as the individualised dose-response function. In general, this quantity is not the same as [[[[XX x, T, /) (which can be easily estimated from observational data) in the presence of selection bias which often presents itself in observational datasets. In order for these two quantities to be equal, we must make the following common assumption.
Assumption 1. (Unconfoundedness) The treatment assignment, Tr, and potential outcomes, Y Y(w, d).
are conditionally independent given the covariates X, i.e.
{Y (w, df|w E W, d, D 11 X
This assumption is also commonly referred to as no hidden confounding.
In addition, in order to make μa(-) identifiable we must also assume that any treatment-dosage pair could be assigned any given sample.
Assumption 2. (Overlap) For each x E N' such that p(x) > O, we have 1 > p(t(x) > 0 0ffor each t t T.
3

---

Under review as a conference paper at ICLR 2020
4 DOSE-RESPONSE GAN
We propose estimating ii by first training a generator to generate dose-response curves for each sample within the training dataset. The learned generator then be used to train inference network using standard supervised methods. We build on the idea presented in Yoon et al. (0088) using a modified GAN framework to generate potential outcomes conditional on the observed features, treatment and factual outcome. Several changes must be made to both the generator and discriminator architectures and learning paradigms in order produce model capable of handling the dose-response setting.
4.1 COUNTERFACTUAN GENERATOR
Our generator, G X' xxx yT takes features, x E X' factual outcome, yf E Y, received treatment and dosage, f, = (wy.d), dy) T. and some noise, Z (typically multivariate uniform or Gaussian), as inputs. The output will be dose-response curve for each treatment (as shown in Fig. D. so that the output a function from to y, i.e. (1,,
- y. We can then write
(t) = Gi
(3)
to denote our generated counterfactual outcome for the treatment-dosage pair t. We will write Yer(t) = G y, (t) (i.e. the random variable induced by G).
While the job of the counterfactual generator is to generate outcomes for the treatment-dosage pairs which were not observed. Yoon et al. (2018) demonstrated that the performance of the counterfactual generator is improved by adding supervised loss term that regularises its output for the factual treatment (in our case treatment-dosage pair). We define the supervised loss, Cs. to be
Gs(G) = E (Y, C(X,T),Y,,2)10
where the expectation is taken over X, Ky Yy and Z.
4.2 COUNTERFACTUAL DISCRIMINATOR
As noted in Section 1 our discriminator will act on random set of points from each of the generated dose-response curves. Similar Yoon et al.) (2018), we define discriminator, D, that will attempt to pick out the factual treatment-dosage pair from among the (random set of) generated ones.
Formally, let The E Z+ be the number of dosage levels we will compare for treatment w E W For each w E W, let Di,. (DT:... D...) be random subse of D of size Thws where for the factual treatment, Wy, Dw, contains nw, - 1 random elements along with Dy. We define Y. (D.r, W (D., x m e to be the vector of dosage-outcome pairs for treatment w where
(Y Wf and Dj D.
(5)
(e D.")) else
and will write Y wYar)ee. We will write dz. yw and ý to denote realisations of D., Y... and y.
Our discriminator, D X' x (Dw x v)" + 10, "h will take the features x E X together with the (random) of generated outcomes ý E yo" nu, and output probability for each treatment-dosage pair indicating the discriminator's belief that that pair is the factual one.
As in the standard GAN framework, define minimax game by defining the value function be
Σ (TT,=(w.a)) log D wid X. +) (ag(1-Dw,eee (X, 83) (6)
C(D.G) E
Σ NEW WED
where the expectation is taken over T,, Y and (Dw. w E W), Dw.d corresponds to the discriminator output for treatment-dosage pair (w, d).
In practice we set all n. to be the same. The default setting is 5 in the experiments. In practice when D., [0, each cach D is sampled independently and uniformly from [0, 1]. Note that for each training iteration, D., is resampled (see Section
4

---

Under review as a conference paper at ICLR 2020
D D
Sweet you
( w. d)
y
X Foetoms
Der
Facnal rrsomeent
Foxtaal deiege
y Ja
Juy
Counterfactual generator
G
10
Trastnsst
Dw
D
Dosage ganerrte
x
U1
Z
W
LW
Generner doage -otoome pais fo tratesos
Coapr dnapr-ntooee pais acuineett Iaputiouape
() w d,)
Ss
CG
Balqpopappiics
Figure 2: Overview of our model for the setting with two treatments (wy corresponds to the factual treatment and Wef to the counterfactual treatment). The generator is used to generate an output for each dosage level in each D.., these outcomes together with the factual outcome, Vf, are used to create the set of dosage-outcome pairs, y, which passed directly the treatment discriminator. Each dosage discriminator receives only the part of y corresponding to that treatment, i.c. Yw. These discriminators are combined (Eq. II define D which is used to give feedback to the generator.
The minimax game then given by
mjn L(D, G) ACs(G),
where À is used control the trade-off between L and Ls (we set À in the experiments).
The task of the discriminator (i.e. picking out the factual dosage from Σ ΣØ=1 Thyy treatment-dosage pairs) becomes increasingly difficult we increase nue or k because the dimension of the discrimi- nator output space, > fww increases. Although we control Rw., if we set it too low, then the set y.. may not well-represent the dose-response curve, particularly the dose-response curve is complex. In practice we found that even for moderate settings of and only 2 treatments, modelling the discriminator as a single function resulted in poor performance. In order to overcome this problem, we introduce novel hierarchical discriminator which involves a treatment discriminator with output dimension and several dosage discriminators, one for each treatment, with output dimensions Thp-
First observe that the probability P((W,, Dy) = (w, d Dus Y) can be written as P(W, = w X, D.,, Y) x IP(D, = d = w, X, D,, Y).
(8) We can therefore break down the discriminator into hierarchical model by learning one discriminator, Dw, that outputs P(W, = w XX D... Y) which we will refer as the treatment discriminator, and then discriminator, Dw. for each treatment, w E W. that outputs P(D, = d = w, X, Dww Y) which we will refer as the dosage discriminator for treatment
The treatment discriminator, Dw X Iww x 2)" - [0, 1]*, takes the features, x, and generated potential outcomes, y, and outputs a probability for each treatment, W1, ..., Wk. Writing Dw, to denote the output of Dw corresponding to treatment w, we define the loss, Lw. to be
=w log (X, Y) + I(W, log(1 DD (X, ry)
(Dw: G) -E
Σ NEW
(9)
where, again, the expectation is taken over X, Wy, Dy, Ý and (Dw heew
Then, for each E W. D. x (D "r - + [0, 1 is map that takes the features. x, and generated potential outcomes, Dar. corresponding to treatment and outputs a probability for each dosage level, di ..., dic.. in given realisation of D... Writing D to denote the output of D m corresponding to dosage level Dy, we define the loss of each dosage discriminator to be
Tha
La(D wi G) = --E L(W,
I I(n,- - D. log D (X, Y +I(D, my) log(1 - (D/( Y Y.))
(X,
(10)
5

---

Under review as a conference paper at ICLR 2020
where the expectation is taken over X, Dw, Y Wy and Dy. The I(w,=w) term ensures that only samples for which the factual treatment is w are used to train dosage discriminator Dw (otherwise there would be no factual dosage for that sample).
We define the overall discriminator DH X x II Imew (D.: r)n + 10, 1122 TM by defining its output corresponding to the treaament-dosage pair (w, d.))) as
Did (x, (x, (x,
(11)
Instead of the minimax game in Eq. 7 the generator and discriminator are trained according to the minimax game defined by seeking G D that solve
G arg min C(DH: G) + ACs(G)
D = Di x D
G
Di = arg min
D = arg min LD Vw E W (12) D.
Dw
Fig. depicts generator and hierarchical discriminator. Pseudo-code for algorithm can be found in Appendix
4.3 INFERENCE NETWORK
Once we have learned the counterfactual generator, we can use it only access (generated) dose- response curves for all samples in the dataset. To generate dose-response curves for new sample we use the counterfactual generator along with the original data to train an inference network, I: X +. Details of the loss and pseudo-code can be found in Appendix
5 ARCHITECTURE
In this section, describe in detail the novel architectures that adopt to model each of the functions G, D, Dw, ..., D Dws which draws from the ideas in Zaheer al. (2017). The inference network, I, has the same architecture as the generator, but does not receive wf. d Uy inputs.
5.1 GENERATOR ARCHITECTURE
f(w, d)
We adopt a multi-task deep learning model for G by defining function g X' x y Z - H for some latent space (typically R' for some 1) and then for each treatment w E W we introduce multitask "head", Iw
W
Tesemenr head
Thoitnos boal
hand
x D., - y taking inputs from H and dosage, d, to produce an outcome i(w, d) E y Given observations, (x, (r)) a noise vector z, and target treatment-dosage pair, (w, d), we define
Shared layer
G(x, f Us. (t) = (g U/, z).
(13)
Each of g: :........ Iw, are modelled as fully connected networks. Fig. 3 depicts our generator architecture.
10
Figure Generator architecture.
5.2 DISCRIMINATOR ARCHITECTURES
As noted in Section I our discriminators need to act as functions of sets (of randomly selected dosage- outcome pairs). While we could require that our discriminators try to learn this during training, by enforcing them be functions of sets through their architecture, the discriminators (they no longer need to "rule out" functions which are not functions of sets). This results in better performing discriminators, which in turn improves the performance of the generator.
reduce the complexity of learning
In practice, the treatment discriminator receives all of the sets (i.e. one set for each treatment) output
of dosage-outeome pairs and outputs probability for each treatment (i.e. there is corresponding to each set). In order to define such function, we treat each input set as a vector but require that the outputs be invariant to (i.e. should not depend on) the ordering of the set vector.

---

Under review as a conference paper at ICLR 2020
Each dosage discriminator receives the set corresponding outputting a probability for each element in the set. In order to define such a function, we consider the imput and output vectors but then require that if we permute the elements of the input vector, the output should be permuted in the same way. We formalise the required notions permutation invariance and permutation equivariance (Zaheer al. 2017) in the following subsection.
given treatment and is tasked with
5.2.1 PERMUTATION INVARIANCE AND PERMUTATION EQUIVARIANCE
The notions of what it means for function to be permutation invariant and permutation equivariant with respect to (a subset of) its inputs are given below in definitions and respectively. Let U, V, C be some spaces. Let E Z+.
Definition 1. A function f t x V + C is permutation invariant with respect to the space 4 if for every u = (ui,.., Um) E U every
and every permutation, o, of {I....,m m] we have
f(u1, Um v) f(uo(1)).... (a())) v).
(14)
Definition 2. A functinn f :u x VV + cm is permutation equivariant with respect to the space u for every (U1 um) UTT every u E V and every permutation, a, of (f, m} we have (M.(1):..... "mm)))..))))) = u)... (u, (u,
(15)
where fu, v) is the jth element of (u,
To build up functions that are permutation invariant and permutation equivariant we make the following observations: (1) the composition of any function with a permutation invariant function is permutation invariant, (2) the composition of two permutation equivariant functions is permutation cquivariant.
Zaheer et al. (201)) provide several possible building blocks to use to construct invariant and cquivariant deep networks. The basic building block we will use for invariant functions will be layer of the form
[in =
(16)
where 1; a vector of of dimension I, Φ any function Φ 2 + R& for some (in this paper we use standard fully connected layer) and a is some non-linearity.
The basic building block for equivariant functions is defined in terms of an equivariance input, u. and an auxiliary input, v. by
fequi (u, v) o (AIm u + (1m (1. (T)))
(17)
where I, is the m x m identity matrix, À and are scalar parameters and θ is vector of weights.
5.2.2 HIERARCHICAL DISCRIMINATOR ARCHITECTURE
In the case of the hierarchical discrim- inator, want the treatment discrim- inator, Dw. to be permutation invari- with respect y.. for each treat- ment, w E W. To achieve this we define a function h Iww x )/ - H. and require that this function be permutation invariant with respect to each of the spaces (D x )))".. We then concatenate the out- put of h2 with the features x and pass these through fully connected net- work h2 X Hu + [0, 1 so that )w hz(x, h (J)). (18)
Yw
Yiry
Equivariant layer
h1
Innitan loya
Inveroni byer
Irvariand
Equivariant layer
P(4 -.-. Pl
X
Fully counected layes
Equivariant imput
P()1) P(w2) ... P(w ((w))
Auxiliary input
(a) Treatment Discriminator
(b) Dosage Discriminator
Figure 4: Architecture of our discriminators.
To construct h, we concatenate the outputs of several invariant layers of
7

---

Under review as a conference paper at ICLR 2020
the form given in Eq- [66] that each individually act the spaces (Dw x ))m That is, for each treatment, w E W we define a map h (Dw x ) - H by substituting Yw for u in Eq. (16). We then define H wew H. and h (5) (A (y ar )...),... 120 (5 (Fw,,))
We want each dosage discriminator, Di to be permutation equivariant with respect to y.. To achieve this each D will consist of two layers of the form given in Eq. (17) with the equivariance input, u, to the first layer being y and to the second layer being the output of the first layer and the auxiliary input, v, the first layer being the features, x, and then no auxiliary input to the second layer.
Diagrams depicting the architectures of the treatment discriminator and dosage discriminators can be found in Fig. 4 a) and Fig. (b) respectively.
6 EVALUATION
The nature of the treatment-effects estimation problem in even the binary treatments setting does not allow for meaningful evaluation on real-world datasets. While there are well-established benchmark synthetic models for use in the binary (or multiple) case, no such models exist for the dosage setting. We propose our own semi-synthetic data simulation to evaluate our model against several benchmarks.
6.1 EXPERIMENTAL SETUP
Semi-synthetic data generation: We simulate data as follows. We obtain features, x, from a real dataset (in this paper we use TCGA (Weinstein et al. 2013), News (Johansson et al. 2016 Schwab al. 2019)) and MIMIC III (Johnson et al. 2016 We consider treatments each accompanied by dosage. Each treatment, w E W, is associated with set of parameters, vi", v2 v² For each run of the experiment, these parameters are sampled randomly by first sampling vector, u. from (0,, 1) and then setting v = u ///l||u]]] where || || is the standard Euclidean norm. The shape of the dose-response curve for each treatment, Sw (x, d). given Table I along with a closed-form expression for the optimal dosage. We add
(0, noise to the outcomes.
We assign factual treatment-dosage pairs to each sample by first sampling a dosage, du, for each treatment from beta distribution, d Beta ax 3w)) The parameter 2 controls the dosage selection bias and the parameter 3w is set to 3. a-1
d= + 2 - a, with d% being the optimal dosage for each treatmen This setting of 3.r ensures that the mode of the Beta distribution is the optimal dosage. Once we have sampled dosage for each treatment, we assign a treatment according to wy Xx ~ Categorical (softmax [xf((x dw.)) where a higher x will result in a stronger selection bias, and 0 results in the treatments being assigned completely randomly. The factual treatment-dosage pair is then given by (uy, dwwy). Unless otherwise specified, we set K = 2 and a = 2.
We consider 3 different shapes for Sw to demonstrate learning heterogeneous dose-response curves. The first curve can be broken down into two terms, linear (in d) increasing term (v ) x2(v) ) xxd and quadratic (in d) decreasing term - -12(v xxl. This first term could, for example, represent the improved efficacy of higher dosages of chemotherapy in reducing the size of tumour, while the quadratic term could represent the increasing toxicity of chemotherapy the dosage increases. This type of trade-off presents itself in many other settings where there are both costs and rewards.
For metrics, we use Mean Integrated Square Error (MISE), Dosage Policy Error (DPE) and Policy Error (PF) Siila 2016 Schwab et al. 2019). Details can be found in Appendix
We compare against two benchmarks: Generalized Propensity Score (GPS) IImeess 2000) and Dose Reponse Networks (DRNet) (Schwab et al. 2019). For DRNets, we compare against both the standard model architecture described by Schwwb et al. 2019 as well as with Wasserstein regularization (DRN-W).
Benchmarks:
Details of each dataset along with suggested interpretations of the synthetic treatments and outcomes for each datasct can be found in Appendix G
'When a = 1, 3gg =1 and Beta(o, 3w) reduces to uniform distribution. See Appendix H
'If the optimal dosage is 0, we sample d from 1 Beta(n, 3w) where Sig is set as though d = 1. This results in the dosage heing sampled symmetrically for di 0 and d",, 1.
8

---

Under review as a conference paper at ICLR 2020
Treatment
Dose-Resperr Response
Optimal dosage
fi(x,)) C(vv))( xd -
di
(v))
2(v
2
f2(x, d) = (
dj 2(v=)
(v5) x
fa(x.d) C((V)= 12d(d b) where b = 0.75 (v
$ if 2 0.75
3
if < 0.75
Table 1: Dose response curves used to generate semi-synthetic outcomes for patient features In the experiments, we set C' = 10. vi v² V3 are the parameters associated with each treatment w.
As baseline for comparison, we also a standard multilayer perceptron (MLP) that takes input the patient features, the treatment and dosage and estimates the patient outcome and multitask variant (MLP-M) that has designated head for each treatment. See Appendix for details of the benchmark models and their hyperparameter optimisation.
6.2 SOURCE OF GAIN
Before comparing against the benchmarks, we investigate how each component of our model affects performance. We start with baseline model in which both the generator and discriminator consist of single fully connected network. One time, we add in the following components (cumulatively until we reach our full model): (1) the supervised loss in Eq. 4+ Ls). (2) multitask heads in the generator (+ Multitask), (3) hierarchical discriminator (+ Hierarchical) and (4) invariance/equivarianee layers in the treatment and dosage discriminators (+Inv/Eqv). We report the results in Table 2 for TCGA and News for 3 error metries (MISE, DPE and PE), computed over runs.
TCGA
News /DP PE
MISE
/DPE
VPE
MISE
Baseline
4.18 + 0.32
2.06 = 0.16
1.93 + 0.12
6.17 0.27
6.97
0.27
6.20 0.21
Ls
3.37 + 0.11
1.14 0.05
0.84 0.05
4.51
0.12
4.40 +
Multitask
3.15 0.12
0.85 0.05
0.67 = 0.05
0.11
0.11
4.31 0.11
Hierarchical
2.54 + 0.05
0.36 + 0.05
0.45 = 0.05
4.07 0.05
0.11
4.17 + 0.12
Inv/Bgvv
1.89 1 0.05
0.31 + 0.05 0.25 1 0.05
3.71 1: 0.05
4.14
3.90 0.05
Table 2: Source of gain analysis for our model. Metrics are reported Mean = Std.
We see that the addition of each component results performance improvement for our model, with the final row (which corresponds to our full model) demonstrating the best performance across both datasets and for all metrics.
To further demonstrate the advantages of our hierarchical discriminator, in Fig. we investigate how our hierarchical discriminator compares with single network discriminator (all other components are included in both models, see Appendix B for details of the single discriminator) when we vary the hyperparameter nhw on TCGA. Similar results for News can be found in Appendix
DRGAN HerrtNaal
ORDAN enroncall
ORGAN Hlerarokol
DRBAN Singe
DRGAN -Singl
DRGAN -Siglaa
MISS
10
4
[
5
2
3 5 0 11 13 15 17 Number of dosage samples
3
5
9
11
13
15
17
19
3 5
9
11 13
15 17 19
Number of dosage samples
Number of dosage samples
(a) v MISE
(b) /DPE
(c) /PE
Figure 5: Performance of single vs. hierarchical discriminator when increasing the number of dosage
samples (n.,) on TCGA dataset.

---

Under review as a conference paper at ICLR 2020
The performance of the single discriminator causes significant performance drops around the across all metrics. As previously noted, this is due to the dimension of the output space (which for The 9 is 27) being too large. Conversely, more stable performance even when fhw = 19. We investigate in Appendix [.1 the hyperparameter X.
that hierarchical discriminator shows much
6.3 BENCHMARKS COMPARISON
We now compare DRGAN against the benchmarks on our semi-synthett datasets, For Mimic, due the low number of samples available, we use only treatments 2 and 3. We report /MISE and PE in Table 3 with results for / DDE given in Appendix L.3 We see that DRGAN demonstrates a statistically significant improvement over every benchmark across all 3 datasets, confirming that DRGAN is able to learn response-curver on top of very different underlying patient features.
TCGA
News
MIMIC
Method
MIIS
VPE
VMISE
/PE
VMISEE
/PE
DRGAN
1.89 1: 0.05
0.25 1 0.05
3.71 1: 0.05
3.90 1 0.05
2.09 + 0.12
0.32 1: 0.05
DRNet
4.14
0.67 + 0.05
4.98 0.12
4.17 0.11
1.5=0.12
1.44 0.05
DRN-W
3.12
0.63 + 0.05
5.07 + 0.12
1.56 0.12
1.17 0.12
1.37 = 0.05
GPS
4.83 0.01
1.60 0.01
6.97 0.01
21.1 0.05
20.2 = 0.01
MLP-M
3.96 = 0.12
1.20
5,17
5.82 0.16
4.16
1.59 = 0.05
MI.P
4.31 0.05
0.97 + 0.05
5.48 + 0.16
6.45 + 0.21
5.34 0.16
1.65 = 0.05
Table Performance of individualized treatment-dose response estimation on three datasets. Bold indicates the method with the best performance for each dataset.
In Appendix 1.4 we compare DRGAN with DRNET and GPS for an increasing number of treatments.
6.4 TREATMENT AND DOSAGE SELECTION BIAS
In this section, we assess the robustness of each method varying treatment and dosage bias. We report results for MISE on TCGA here. For the other metrics see Appendix 1.2 Fig. 6(a) shows the performance of the 4 methods for к between 0 (no bias) and 10 (strong bias). Fig. b(b) shows the
performance for a between (no bias) and (strong bias). We see that our model shows consistent performance, significantly outperforming the benchmark methods across the entire ranges of K and a.
DROAN
DRNET
DRGAN
DRNET
MISE
3
2
7
10
Trealment assignment
Dosage assignment bias
(a) Treatment selection
(b) Dosage selection
Figure 6: Performance of the 4 methods on datasets with varying bias levels.
CONCLUSION
In this paper we proposed novel framework for estimating dose-response curves from observational data. Our method modified the GAN framework, introducing a novel hierarchical discriminator for in the dose-response setting. We also proposed novel architectures for the networks involved in our model and introduced new semi-synthetic data simulation for use as a benchmark in this setting. On this data demonstrated significant improvements the benchmarks.
10

---

Under review as a conference paper at ICLR 2020
REFERENCES
Ahmed M Alaa and Mihaela van der Schaar. Bayesian inference of individualized treatment effects using multi-task gaussian processes. In Advances in Neural Information Processing Systems, pp. 1424-3432, 2017.
Ahmed M Alaa, Michael Weisz, and Mihaela Van Der Schaar. Deep counterfactual networks with propensity-dropout. arXiv preprint arXiv:/706.0596 2017.
Susan Athey and Guido Imbens. Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 113(27):7333-736) 2016.
James Bergstra and Yoshua Bengio. Random search for Machine Learning Research, 13(Febb:281-335, 2012.
optimization. Journal of
Dimitris Bertsimas, Nathan Kallus, Alexander M Weinstein, and Ying Daisy Zhuo Personalized diabetes management using electronic medical records. Diabetes Care, 40(2):210-217, 2017.
Hugh A Chipman, Edward George, Robert E McCulloch, al. BART: Bayesian additive regression trees. The Annals of Applied Statistics, 4(1):266 298, 2010.
Natalie Cook, Aaron R Hansen, Lillian L Siu, and Albiruni R Abdul Razak. Early phase clinical trials to identify optimal dosing and safety. Molecular Oncology, 9(5)9977-1077, 2015.
Richard K Crump. Joseph Hotz, Guido W Imbens, and Oscar A Mitnik. Nonparametric tests for treatment effect heterogeneity. The Review of Economics and Statistics, 90(3):389-405, 2008
Donna Döpp-Zemel and AB Johan Groeneveld. High-dose norepinephrine treatment: determinants of mortality and futility in critically ill patients. American Journal of Critical Care, 22(1):22-32, 2013.
Douglas Galagate. Causal Inference with a Continuous Treatment and Outcome: Alternative Estimators for Parametric Dose-Response function with Applications. Phl) thesis, 2016.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, pp. (672-2680, 2014.
J Henry, Yuriy Pylypchuk, Talisha Searcy, and Vaishali Patel. Adoption of electronic health record systems among US non-federal acute care hospitals: 2008-2015. ONC Data Brief, 35:1-9, 2016.
Keisuke Hirano and Guido W Imbens. The propensity score with continuous treatments. Applied Bayesian Modeling and Causal Inference from Incomplete-Dade -Diaa Perspectives, 226164:73-84, 2004.
Kosuke Imai and David A Van Dyk. Causal inference with general treatment regimes: Generalizing the propensity score. Journal of the American Statistical Association, 99(467):854 866, 2004.
Guido W Imbens. The role of the propensity score in estimating dose-response functions. Biometrika, 87(3):706-710, 2000.
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In International Conference on Machine Learning, pp. 3020-3029, 2016.
Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Lco Anthony Celi, and Roger Mark. Mimic-iii, a freely accessible critical care database. Scientific Data, 3:160035, 2016.
Nathan Kallus. Recursive partitioning for personalization using observational data. In Proceedings of the 34th International Conference on Machine Learning-Voil Volume 70, pp. 1789-1798. JMLR. org, 2017.
Sheng Li and Yun Fu, Matching on balanced nonlinear representations for treatment effects estimation. In Advances in Neural Information Processing Systems, pp. 929-939, 2017.
11

---

Under review as a conference paper at ICLR 2020
Min Qian and Susan A Murphy. Performance guarantees for individualized treatment rules. Annals of Statistics, 39(2): 180, 2011.
Peter M Rothwell, Nancy R Cook, Michael Gaziano, Jacqueline F Price, Jill FF Belch, Maria Carla Roncaglioni, Takeshi Morimoto, and Ziyah Mehta. Effects of aspirin on risks of vascular events and cancer according to bodyweight and dose: Analysis of individual patient data from randomised trials. The Lancet, 392(10145):387-999) 2018.
Donald Rubin. Bayesianly justifiable and relevant frequency calculations for the applies statistician. The Annals of Statistics, pp. 1151-1172, 1984.
Patrick Schwab, Lorenz Linhardt, Stefan Bauer, Joachim M Buhmann, and Walter Karlen. Learning counterfactual representations for estimating individual dose-response curves. arXiv preprint arXiv:1902.00983 00981. 2019.
Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: General- ization bounds and algorithms. In Proceedings of the 34th International Conference on Machine Learning-Volie Volume 70, pp. 3076-3085. JMLR. org, 2017.
Claudia Shi, David Blei, and Victor Veitch. Adapting neural networks for the estimation of treatment effects. arXiv preprint arXiv:1906.02120) 2019.
Ricardo Silva. inpervaaiooallineeeentiooo priors for dose-response learning. In Advances in Neural Information Processing Systems, pp. 1561-1569, 2016.
Peter Spirtes. A tutorial on causal inference. 2009.
J Stoehlmacher, DJ Park, W Zhang, D Yang. S Groshen, $ Zahedy, and HJ Lenz. A multivariate anal- ysis of genomic polymorphisms: Prediction of clinical outcome to S-FU/OXaliplatin combination chemotherapy in refractory colorectal cancer. British Journal of Cancer, 91(2):344, 2004.
Moreno Ursino, Sarah Zohar, Frederike Lentz, Corinne Alberti, Tim Friede, Nigel Stallard, and Emmanuelle Comets. Dose-finding methods for Phase clinical trials using pharmacokinetics in small populations. Biometrical Journal, 59(4)8804 825, 2017.
Stefan Wager and Susan Athey, Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523) 2018.
Kyle Wang, Michael J Eblan, Allison M Deal, Matthew Lipner, Timothy M Zagar, Yue Wang, Panayiotis Mavroidis, Carrie B Lee, Brian Jensen, Julian Rosemman, et al. Cardiac toxicity after radiotherapy for stage III non-small-cell lung cancer: Pooled analysis of dose-escalation trials delivering 70 to 90 Gy. Journal of Clinical Oncology, 35(13):1387, 2017.
John N Weinstein, Eric A Collisson, Gordon Mills, Kenna R Mills Shaw, Brad A Ozenberger, Kyle Ellrott, Ilya Shmulevich, Chris Sander, Joshua M Stuart, Cancer Genome Atlas Research Network, et al. The cancer genome atlas pan-cancer analysis project. Nature Genetics, (5(10):1113, 2013.
Liuyi Yao, Sheng Li, Yaliang Li, Mengdi Huai, Jing Gao, and Aidong Zhang. Representation learning for treatment effect estimation from observational data. Advances in Neural Information Processing Systems, pp. 2633-2643, 2018.
Jinsung Yoon, James Jordon, and Mihaela van der Schaar. GANITE: Estimation of individual- ized treatment effects using generative adversarial nets. International Conference on Learning Reprexentations (ICLR), 2018
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan R Salakhutdinov, and Alexander J Smola. Deep sets. In Advances in Neural Information Processing Systems, pp. 3391-3401, 2017.
12

---

Under review as a conference paper at ICLR 2020
APPENDIX
A EXPANDED RELATED WORKS
Most methods for performing causal inference in the static setting focus on the scenario with two or multiple treatment options and no dosage parameter. The approaches taken by such methods to estimate the treatment effects involve either building separate regression model for each treatment (Stoehlmacher et al. 2004, Qian & Murphy 2011 |Bertsimas et al. 2017) or using the treatment as feature and adjusting for the imbalance between the different treatment populations. The former does not generalise to the dosage setting due to the now infinite number of possible treatments available. In the latter case, methods for handling the selection bias involve propensity weighting (Crump
et al.] [2088] Alaa et al. 2017: [Sh et al. 2019), building sub-populations using tree based methods (Chipman et al. 2010
Athey & Imbens| 2016 Wager & Athey 2018 Kallus (017) or building
balancing representations between patients receiving the different treatments (Johansson al. 2016
Shalit et al. 2017] Li Fu
2017 Yao et al. 2018). An additional approach involves modelling the
data distribution of the factual and counterfactual outcomes (Alaa & van der Schaar 2017 Yoon
ct al. 2018
Silva 2016 leverages observational and interventional data estimate the effects of discrete dosages for single treatment. In particular, [Silaa 2016) uses observational data to construct a non-stationary covariance function and develop a hierarchical Gaussian process prior build distribution over the dose response curve. Then, controlled interventions are employed to learn non- parammetttii affine transform to reshape this distribution. The setting in Silva (016)) differs significantly from ours as we do not assume access to any interventional data.
13

---

Under review as a conference paper at ICLR 2020
B SINGLE DISCRIMINATOR MODEL
In the paper we developed a hierarchical discriminator and demonstrated that it performs significantly better than the single discriminator setup that we now describe in this section.
B. SINGLE DISCRIMINATOR
In the single model, we will aim to learn a single discriminator, D, that outputs P((W,, D2) = (w, d))x, D.., Y) for each W and d E D.,. We will write (.) to denote the output of D that corresponds to the treatment-dosage pair (w, d). We define the loss, Lp, to be
DD(D:G) = E Σ
Σ NEW deD...
T(T,=(w,d)) log D mdd (X Y) + +I (Ty log(1 D (x, Y))
(19)
where the expectation is taken over X, (Dw Y, y and Dj and we note that the dependence on G is through Y. Our single discriminator will be trained minimise this loss directly. The generator GAN-loss, Lg. is then defined by
GG = Lp LD(D':DDD G
(00)
where D the optimal discriminator given by minimising Lp. The generator will be trained to minimise La ALs.
B.2 SINGLE DISCRIMINATOR ARCHITECTURE
In the case of the single discriminator, we want the output of D corresponding each treatment E W, iee (D":.1 ..... D (0:,w) ), to be permutation equivariant with respect to y and permutation invariant with re- spect to each y, for E W To achieve this, we first define a function f ( x x))" Hg and require that this function be permutation invari- ant with respect to each of the spaces (De, x ")n=. For each treatment, task head, fw : x Hs x (D x ")" - [0. 1["",, and require that each of these functions be permuta- tion equivariant with respect to their corresponding input space (D b X 7 but they can depend on the features, x E X, and invariant latent representa- tion coming from arbitrarily. Writing fú to denote the jth output of fw. the output of the discriminator given input features, x, and generated outcomes, y. is defined by
ye' Invorint bype
Invarinnt layor
Invartant layer
E W. we introduce multi-
X
fly)
Expaxetton
lnnt
Equciaa lank boyer
layer
Pl.1 Pi
Pu P((W,, (mr,
=x. (
(21) Figure 7: Overview of the single discrimina- tor architecture.
To construct the function f. we concatenate the outputs of several invariant layers of the form given in Eq. on the spaces (D,r x 3/"",, That is, for each treatment, w W we define a map fi (Dw x "> + Hy by substituting yw for in Eq. (16). We then define Hs IIwew Hy
(16) that each individually act
and f(ý) (J (F ).... ful (F,,)).
Each fw will consist of two layers of the form given in Eq. (17) with the equivariance input, u, to first layer being y.. and to the second layer being the output of the first layer and the auxiliary input,
v, to the first layer being the concatenation of the features and invariant representation, i.e. (x, (5))
and then no auxiliary input to the second layer.
diagram depicting the architecture of the single discriminator model can be found Fig.
14

---

Under review as a conference paper at ICLR 2020
C COUNTERFACTUAL GENERATOR PSEUDO-CODE
Algorithm Training of the generator in DRGAN
1: Input: dataset C {(x', fty, wÿ) i 1, ................................ N}. batch size Rem., number of dosages per treatment Rg, number of discriminator updates per iteration nD, number of generator updates per iteration ng., dimensionality of noise 22 learning rate
2: Initialize: fg. 8y: (( keew
3: while G has not converged do Discriminator updates
4: for i 1, .,.., 7 do
5: Sample (x1, (W1, d1). v1), (Xumsr (Wams: dama)) Vroms) from C
6: Sample generator noise Zj (o) ("1,..., 2h. from Unif([O, 1)mx) for j I,.... Rmbb
7:
for w E W do
8:
for j I.... 7mb do
9:
Sample D (d ..... d independently and uniformly from (Dw))
10:
Set ÿ according to Eq. 5
11:
Calculate gradient of dosage discriminator loss
nd (d log D (X), + Ld, log(1 D (xx, F(n))
Iw
Von.
Σ
[5:m,==0) k=1
12:
Update dosage discriminator parameters Ow Ow aghe
Set ý, = (y w
14:
Calculate gradient of treatment discriminator loss
9w + Vow
[wy== log Dw(x,,3,) + [my log(1 - Dw(xx,Jj))
j=1 WEW
15:
Update treatment discriminator parameters Ow + 8y + agw
Generator updates
for 1,.., ng do
17:
Sample (x1, (wi, di), v1). (Xmmer (Wame: an ). Dmmm) from C
18:
Sample generator noise 2j (2) ..., 2bt from Unif([0, =] for 1,... Rmb
19:
Sample ) from (l for j = 1, ..., Rmb
20:
Set according to Eq. Calculate gradient of generator loss
21:
Rd
9G f-
Aic
tw buy==,d,==a =d (x,, D' (x,,
j=1 NEW
(w log(1 (DW D' (xx
22:
Update generator parameters fG + 0g + agg
23:
Output: G
15

---

Under review as a conference paper at ICLR 2020
D INFERENCE NETWORK
To generate dose-response curves for new samples, we learn an inference network, I: X x This inference network is trained using the original dataset and the learned counterfactual generator. As with the training of the generator and discriminator, we train using random set of dosages, D... The loss is given by
y
LT =
Σ WEW deDx
Σ (V d) I(X, (w, d)))²
(22)
where (w,, d) is Yy Tf (w, d) or given by the generator if Ty (w, d). The expectation is taken over X, f. Y,. Z and D..
D.1 PSEUDO-CODE FOR TRAINING THE INFERENCE NETWORK
Algorithm 2 Training of the inference network in DRGAN
1: Input: dataset = [(x', Cy 2/) i 1, .,., J, trained generator G, batch size Ylm number of dosages per treatment nha dimensionality of noise learning rate
2: Initialize: 8,,
3: while I has not converged do
4: Sample (x1, (W1, d1), 91), (Xamor (Wome> damm): Unms) from
5: Sample generator noise 2 = (2) 21 from Unif([0, 1)xx) for j 1,.... Rmmb
6: for j L..., 7mmb do
7: for w E W do
8: Sample D = (d
dw
independently and uniformly from (Dw.)
9: Set y according to Eq.
10:
Calculate gradient of inference network loss
(Fa): - I(x,, (w, ))2
+
Ve,
WEW 1=1
Update inference network parameters 0₁ 01 + ag!
12:
Output:
16

---

Under review as a conference paper at ICLR 2020
E BENCHMARKS
We use the publicly available GitHub implementation DRNet provided by [Schwab et al. (2019): Moreover, we also used GPS implementation similar com/ d909b /rnett which uses the causa ddr f R package Galagate 2016). More spcifically, the GPS implementation uses a normal treatment model, linear treatment formula and a 2-nd degree polynomial for the outcome. Moreover, for the TCGA and News datasets, we performed PCA and only used the 50 principal components as input to the GPS model to reduce computational complexity.
to the one from httpmnttpp::gggttat
Hyperparameter optimization: The validation split of the dataset is used for hyperparameter optimization. For the DRNet benchmarks we use the same hyperparameter optimization proposed by Schwab et al. (2019) with the hyperparameter search ranges described in Table [ For DRGAN, we use the hyperparameter optimization method proposed in GANITE Yoon al. 2018), where use the complete dataset from the counterfactual generator to evaluate the MISE on the inference network. We perform random search (Bergstra & Bengio 2012) for hyperparameter optimization over the search ranges in Table 5 For fair comparison, for the MIP-M model we used the same
architecture used in the inference network of DRGAN. Similarly, for the MIP model we the same architecture as for the MLP-M, but without the multitask heads.
Hyperparameter
Search range
Batch size
32, 64, 128 24, 48, 96, 192 2, 0.0, 0.2
Number of units per hidden layer
Number of hidden layers
Dropout pereentage
Imbalance penalty weight"
0.1, 1.0, 10.0
Fixed
Number of dosage strata E
5
Table 4: Hyperparameters search range for DRNet. *; For the DRNet model using Wasserstein regularization only.
Hyperparameter
Search range
Batch size
64, 128, 256 32, 64, 128 16, 32, 64, 128
Number of units per hidden layer
Size of invariant and equivariant representations
Fixed
Number of hidden layers per multitask head Number of dosage samples
Optimization
Adam Moment Optimization
Table Hyperparameters search range for DRGAN.
17

---

Under review as a conference paper at ICLR 2020
F METRICS
The Mean Integrated Square Error (MISE) measures how well the models estimates the patient outcome across the entire dosage space:
MISE
Σ WEW
Di (v'(w,u) u) Suc, w du
(23)
In addition to this, we also compute the mean dosage policy error (DPE) (Schwab al,, 2019) to assess the ability of the model to estimate the optimal dosage point for every treatment for each individual:
DPE
11 Σ NEW
Σ
N
(w, 5 (w, )
(24)
N
where d is the true optimal dosage and d' is the optimal dosage identified by the model. The optimal dosage points for a model are computed using SciPy's implementation of Sequential Least SQuares Programming.
Finally, we compute the mean policy error (PE) (Schwab et al. 2019) which compares the outcome of the true optimal treatment-dosage pair to the outcome of the optimal treatment-dosage pair as selected by the model:
N Σ E=1
V' ((W""dh))= - y di,)
PE
(25)
N
where w" is the true optimal treatment and úi is the optimal treatment identified by the model. The optimal treatment-dosage pair for model is selected by first computing the optimal dosage for each treatment and then selecting the treatment with the best outcome for its optimal dosage.
Each of these metrics computed held out test-set.
18

---

Under review as a conference paper at ICLR 2020
G DATASET DESCRIPTIONS
TCGA: The TCGA dataset consists of gene expression measurements for cancer patients (Wein- stein et al. 2013). There are 9659 samples for which we used the measurements from the 4000 most variable genes. The gene expression data was log-normalized and each feature was scaled in the [0. 1] interval. Moreover, for each patient. the features x were scaled to have norm We give meaning to our treatments and dosages by considering the treatment as being chemother- seyy/ndiithhrayymmmuuteeen
and their corresponding dosages. The outcome can thought of as the risk of cancer recurrence (Schwab et al. 2019).
News: The News dataset consists of word counts for news items. We extracted 10000 samples each with 2858 features. As in (Johansson et al.] 2016]] [Schwab et al. (019), we give meaning to our treatments and dosages by considering the treatment as being the viewing device (e.g. phone, tablet etc.) used to read the article and the dosage being the amount of time spent reading it. The outcome can be thought as user satisfaction.
The Medical Information Mart for Intensive Care (MIMIC IIII (Johnson et al. 2016) database consists of observational data from patients in the ICU. We extracted 3000 patients that receive antibiotics treatment and used features clinical covariates measured during the day of ICU admission. Again, the features were scaled in the [o, 1] interval. In this setting, we can considered treatments the different antibiotics and their corresponding dosages.
MIMIC III:
For a summary description of the datasets, see table 6 The datasets are split into 64/16/20% for training, validation and testing respectively. The validation dataset is used for hyperparameter optimization.
TCGA
News
MIMIC
Number of samples Number of fcatures
9659
10000
3000
4000
2858
9
Number of treatments 3"
3
2
Table Summary description of datasets. *; for our final experiment in Appendix I.4 we increase the number of treatments in TCGA to and 9.
H DOSAGE BIAS
In order to create dosage-assignment bias in our dataset, we assign dosages according to d Beta(a, B. The selection bias is controlled by the parameter a 2 1. When we set 8. a-1 +2-a (which ensures that the mode of distribution is dr,), ) and d as follows
write the variance of d in terms of
a²aa + 2a a²
Var(dw)
d=
ca
(26)
2)2
da
We see that the variance of our Beta distribution therefore decreases with resulting in the sampled dosages being closer to the optimal dosage, thus resulting in higher dosage-selection bias. In addition note that the Beta(1 1) distribution is in fact the uniform distribution, corresponding to the dosages being sampled independently of the patient features, resulting in no selection bias when =
19

---

Under review as a conference paper at ICLR 2020
1 ADDITIONAL RESULTS
I.1 INVESTIGATING HYPERPARAMETE SENSITIVITY (Ou AND A)
Here present additional results for investigation of the hyperparameters fww and A. Fig. reports each of the 3 performance metrics as we increase the number of dosage samples, fw. used to train the discriminators the News dataset. As with the TCGA results in the main paper see that the single discriminator suffers a significant performance decrease when Rew is set too high.
17.5
UHIAAN -Tanecncio
14
DNGAN- Hartrrcaa
DRIAAN -taradhccal
15.0
ORGAN Sinn
ORGAN Singln
DRG4NN--100 -ingle
12.6
IDRE
10
10.0
7.5
8.0
3 5
9 11
13 15 17
19
3 5 9 11 13 15 17 19
5 11 13 15 17 19
Number of dosage samples
Number of dosage samples
Number of dosage samples
(a) v /MIS
(b) vDPE
(c) /PE
Figure 8: Performance metrics when increasing the number of dosage samples on News dataset.
Fig. and Fig. 10 report each of the performance metrics when we increase X, the hyperparameter that trades off between the GAN loss, C, and the supervised loss, Ls. The results shown are on the TCGA and News datasets, respectively. As we would expect, for 0, the performance is significantly worse than for À = 1, since this corresponds to no supervised loss (this result is also backed up by our source of gain experiments). In addition, slowly degrades, this is because the model starts to behave more and more like MLP-M since the supervised loss becomes dominant.
increase À we see performance
3.5
2.0
3.0
MIIS
1.5
1.0
1.0
2.0
0.5
0.5
1.5
DRBAN
ORGAN
DRGAN
10
(a) MISE
(b) / DPE
(c) VPE
Figure 9: Performance metrics when increasing on TCGA dataset.
5.0
6.0
5.0
(MISE
5.5
4.8
4.5
8.0
4.0
4.5
4.0
DRGAN
ORGAN
DRGAN
3.5
9 10
0
1 2
10
(a) v MISE
v DPE
(c)
PP
Figure 10: Performance metrics when increasing À on News dataset.
20

---

Under review as a conference paper at ICLR 2020
1.2 ADDITIONAL RESULTS ON SELECTION BIAS
In Fig. I1 we report the DPE and PE for our treatment and dosage bias experiment from Section 6.4 of the main paper.
DRGAN
DRNET
3.0
DRGAN MLP-M
DRNET OPS
MLPMM
OPS
2.0
2.0
YDPE
1.5
1.8
0.5
0.5
7
10
4
5
Trealment ass gnnment bias
Dosage assignment bias
(a) Treatment selection bias
(b) Dosage selection bias
2.0
1.5
1.0
1.0
0.5
0.0
7
10
Trealment assignment
Dosage assignment bias
(c) Treatment selection
(d) Dosage selection
Figure 11: Additional performance metrics of the 4 methods datasets with varying bias levels on
TCGA dataset.
1.3
DOSAGE POLICY ERROR FOR BENCHMARK COMPARISON
Table we report the Dosage Policy Error (DPE) corresponding to Section 6.3 in the main paper.
TCGA VDPE
News VDPF
MIMIC VDPE
Methods
DRGAN
0.31 + 0.05
4.14 0.11
0.51 + 0.05
DRNet
0.51 + 0.05*
4.39 +.111
0.52 1 0.05
DRN-W
0.50 - 0.05*
4.21
0.53 + 0.05
GPS
1.38 + 0.01*
6.00
1.41 0.12
MLP-M
0.92 1 0.05*
0.16
0.77 0.05
MLP
1.04 - 0.05*
5.12
0.05"
Table 7: Performance of individualized treatment-dose response estimation on three datasets. Bold indicates the method with the best performance for each dataset. *, performance improvement is statistically significant.
21

---

Under review as a conference paper at ICLR 2020
1.4 VARYING THE NUMBER OF TREATMENTS
In our final experiment, we increase the number of treatments by defining or 6 additional treatments. The parameters ví 2 V3 defined in exactly the same way for 3 treatments. The outcome shapes for treatments 4 and 7 are the same as for treatment 1, similarly for 5, 8 and and for 6, and 3. In Table 8 we report MISE, DPE and PE on the TCGA dataset with 6 treatments (TCGA-6) and with 9 treatments (TCGA-9). Note that we use 3 dosage samples for training DRGAN in this experiment.
TCGA
TCGA VDPE
Method
MISE
VDPE
PE
MISE
PE
DRGAN
2.37 + 0.12
0.43 = 0.05
0.32 + 0.05
2.79 +: 0.05
0.51 + 0.05
0.54 : 0.05
DRNET
4.09 + 0.16
0.52 = 0.05
0.71 0.05
4.31
+ 0.12
0.59 + 0.05
0.74 + 0.05
GPS
6.62 + 0.01
2.04 = 0.01
2.61
7.58
0.01
3.14
2.91 + 0.01
Table 8: Performance of DRGAN and the benchmarks when we increase the number of treatments in the dataset to 6 and 9, Bold indicates the method with the best performance for each dataset.
22


Reviews:
[{"review_id": "SJxMxb4S5S", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nTreatment Response Estimation \nDifferent from Average treatment effect because of multiple treatment and dosage parameters the problem gets complicated \nPreviously, other papers had tried to estimate ITE using distribution matching, propensity dropout  and matching on balancing scores.\nMost relevant benchmark DrNet(https://arxiv.org/pdf/1902.00981.pdf)\nHere, authors use GANs to perform the task of Dose-response estimation instead of traditional methods. They use a counterfactual generator and counterfactual discriminator for the task. Generator is modeled as a multitask neural network and the discriminator is modeled as hierarchical network consisting of treatment and dosage discriminator.\nOverall, the model outperforms other baselines such as DRNet and GPS indicating improved dose-response estimation capabilities.\nPros:\nSignificant empirical advance? Did they solve a standing open problem?\nSignificant performance improvement   in defined metrics compared to previous methods such DrNet. This indicates the proposed methods are better for dose-response estimation.\nAny good practical outcome (code, algorithm, etc)?\nNovel neural network architectures used with counterfactual generator and counterfactual discriminator are interesting and these can be extended to other tasks.\nAre the experiments well executed?\nGood experiments in general. Source of gain experiment(section 6.2) provides deeper insight into why the model might perform better compared to other baselines.\n\nUseful for the community in general?\nYes it’s useful for the community as this is a challenging problem and estimating dose-response from observational data(EHR) is tricky.\n\nCons:\nAny missing baselines? Missing datasets?\nMissing baselines: TARNET, MLP, GANITE, kNN, BART. These baselines were evaluated in DrNet paper.\nAny odd design choices in the algorithm not explained well?\nWhy did authors use GAN with Vanilla loss function while it has been shown that there are other loss functions which can help in improving the performance? Perhaps evaluating the GAN with other loss function can provide further insight to the stability of the model\n\nQuality of writing?\nThe overall flow of the paper is a little hard to follow. \n\nIs the model better in terms of stability?\nCompared to DrNet Paper, is the model easy to train? Since the model uses a GAN loss function which are known for being hard to train, it would be good to evaluate the model in terms of difficulty of training.\n\nIs there sufficient novelty in what they propose? Minor variation of previous work?\nWhile this is a very interesting task, this paper falls under incremental improvement as it improves GANITE framework to solve dose-response estimation. As authors use GAN for counterfactual estimation which is not novel(GANITE) and estimating dose-response estimation is not novel(DrNet). \nSpecific Points\nIn comparison to DrNet Paper(Learning Counterfactual Representations for Estimating Individual Dose-Response Curves)\nSquare root of MISE is different for the datasets. \nIs it because of different hyperparameters?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "SJlmIlnecH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a generative adversarial net model for heterogeneous dose-response causal effect estimation. The idea is to generate counterfactual dose-response curves using the generator that can fool the discriminator that tries to distinguish between factual data and counterfactual data. Factual data along with counterfactual data generated by the GAN can then be used for heterogeneous causal effect estimation. \n\nTo tailor to this specific heterogeneous dose-response curve estimation problem, the paper makes various architectural design modification from a regular GAN, such as using factual predictive performance as regularization, multitask for treatment option and dosage in the generator, hierarchical decomposition of the dose and response prediction in the discriminator, and permutation invariance when predicting the dosage-response curves for various treatment in the discriminator. These design choices are reasonably explained and the empirical gains due to these design choices are well documented (Table 2).\n\nA few concerns:\n\n1. It is unclear why using a GAN that seeks to produce samples that are indistinguishable from the factual ones will be the right way to synthesize counterfactual samples. Intuition and further justification need to be provided. Related to this concern, do the authors also implicitly reason under the consistency assumption (the observed/factual samples are the realizations potential outcomes of the corresponding treatment-dosage pairs in the potential outcome framework) ? If so, the authors should also reflect that in their list of assumptions.\n\n2. It is also unclear what kind of theoretical guarantees the proposed method could deliver. Since the setting considered by the paper is to formally estimate the dosage-response effect, such guarantees, while potentially challenging to derive, are nonetheless crucial to better understand the circumstances under which the proposed method is reliable/unreliable.  This is especially the case given the elusive use of GAN to generate counterfactual outcomes as mentioned in the previous concern.\n\n3. It will also be interesting to see whether the proposed method can be degenerated to handle simpler cases such as heterogeneous treatment effect estimation of binary treatment.  Or alternatively, when there is only one treatment option, where when treated there is a dosage-response curve while when not treated the dosage is zero. Related to this question, is this also the way to parameterize whether a subject receives a treatment or not in the model?\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "S1l8cmXhYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces Dose Response Generative Adversarial Network (DRGAN) that is aimed at generating entire dose-response curve from observational data with single dose treatments. This work is an extension of GANITE (Yoon et al., 2018) for the case of real-valued treatments (i.e., dosage). The proposed model consists of 3 blocks: (1) a generator, (2) a discriminator, and (3) an inference block. In this paper, GANITE’s generator and discriminator architectures are modified to be able to handle real-valued treatments.\n\nThis paper should be rejected due to the following arguments:\n\t- Since the paper is based on GANITE, I’m going to start my review by pointing out the problems/questions I have about GANITE. First, let me briefly summarize my understanding of this method: \n\t\t+ Given (x, t, yf), the generator G tries to estimate counterfactual outcomes (ycf). \n\t\t+ Given (x, (yf, ycf)), the discriminator D tries to figure out which outcome was factual.\n\t\t+ Yoon et al. (2018) claims that optimizing a GAN with this G and D, the generator will become better and better at estimating accurate counterfactual outcomes.\nLooking at the objective function in Eq. (4), it is not clear why D should learn to distinguish factual from counterfactual outcome as opposed to learning the treatment selection bias as t.log(D(x ,y))+... would suggest. Yoon et al. (2018) assume the former while the latter seems more plausible. Of course, a by-product of learning the treatment selection bias is distinguishing yf from ycf, however, this doesn’t mean that G did a good job at generating an accurate estimation of ycf. \nIn summary, it seems that the adversarial training designed in GANITE and consequently DRGAN provide no advantage in terms of accurate estimation of counterfactuals, which in turn, nullifies the entire claimed contribution of these two works. I will, however, read the rebuttal carefully and am willing to modify the score if the authors address this major concern.\n\nReferences:\n\t- Yoon, J., Jordon, J., & van der Schaar, M. (2018). GANITE: Estimation of individualized treatment effects using generative adversarial nets.\n\n\n********UPDATE after reading the rebuttal********\nThank you for your response. \nStill, there are multiple issues with this reasoning that you need to address.\n\n$X$ is way more expressive than the added binary input that you described in the example. While the binary input is sampled from a Uniform distribution (either ½ or ⅔), $Pr(X|t=1)$ and $Pr(X|t=1)$ distributions are often Gaussian (certainly not Uniform) and therefore, embed a lot more information about which instances should receive which treatment. Therefore, $X$ is way more informative about selection bias than your binary input is about the image being fake/real. Do not underestimate this!! \n\nIt is true that the discriminator reaches its maximum loss when the generator’s estimation of the counterfactual is spot on. That is, it simply predicts the administered treatment according to the treatment selection bias. You need to show, however, that this maxima is global. Otherwise, there could be infinite maximas with the same loss; e.g., why not simply $Y^{factual} = Y^{counterfactual}$ does not reach this maxima? \n\nYou also need to show that the added information to $X$ by concatenating it with the factual outcome plus the generator’s estimation of the counterfactual outcome do improve prediction of the administered treatment -- i.e., minimizing the discriminator loss.\n\nTo summarize, I am not yet convinced that either GANITE or DRGAN do what they claim.\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses the problem of estimating treatment responses involving a continuous dosage parameter.  The basic idea is to learn a GAN model capable of generating synthetic dose-response curves for each training sample, which then facilitates the supervised training of an inference model that estimates these curves for new cases.  For this purpose, specialized architectures are also proposed for the GAN, which involves a multi-task generator network and a hierarchical discriminator network.  Empirical results demonstrate improvement over existing methods.

While there is always a chance that reviewers may underappreciate certain aspects of a submission, the fact that there was a unanimous decision to reject this work indicates that the contribution must be better marketed to the ML community.  For example, after the rebuttal one reviewer remained unconvinced regarding explanations for why the proposed method is likely to learn the full potential outcome distribution.  Among other things, another reviewer felt that both the proposed DRGAN model, and the GANITE framework upon which it is based, were not necessarily working as advertised in the present context.
Decision: Reject