Title: TRANSFER LEARNING FOR ESTIMATING CAUSAL EF-FECTS USING NEURAL NETWORKS

Abstract: We develop new algorithms for estimating heterogeneous treatment effects, combining recent developments in transfer learning for neural networks with insights from the causal inference literature. By taking advantage of transfer learning, we are able to efficiently use different data sources that are related to the same underlying causal mechanisms. We compare our algorithms with those in the extant literature using extensive simulation studies based on large-scale voter persuasion experiments and the MNIST database. Our methods can perform an order of magnitude better than existing benchmarks while using a fraction of the data.

1. INTRODUCTION:

The rise of massive datasets that provide fine-grained information about human beings and their behavior provides unprecedented opportunities for evaluating the effectiveness of treatments. Researchers want to exploit these large and heterogeneous datasets, and they often seek to estimate how well a given treatment works for individuals conditioning on their observed covariates. This problem is important in medicine (where it is sometimes called personalized medicine) (Henderson et al., 2016; Powers et al., 2018) , digital experiments (Taddy et al., 2016) , economics (Athey and Imbens, 2016) , political science (Green and Kern, 2012) , statistics (Tian et al., 2014) , and many other fields. Although a large number of articles are being written on this topic, many outstanding questions remain. We present the first paper that applies transfer learning to this problem.

In the simplest case, treatment effects are estimated by splitting a training set into a treatment and a control group. The treatment group receives the treatment, while the control group does not. The outcomes in those groups are then used to construct an estimator for the Conditional Average Treatment Effect (CATE), which is defined as the expected outcome under treatment minus the expected outcome under control given a particular feature vector (Athey and Imbens, 2015) . This is a challenging task because, for every unit, we either observe its outcome under treatment or control, but never both. Assumptions, such as the random assignment of treatment and additional regularity conditions, are needed to make progress. Even with these assumptions, the resulting estimates are often noisy and unstable because the CATE is a vector parameter. Recent research has shown that it is important to use estimators which consider both treatment groups simultaneously (Künzel et al., 2017; Wager and Athey, 2017; Nie and Wager, 2017; Hill, 2011) . Unfortunately, these recent advances are often still insufficient to train robust CATE estimators because of the large sample sizes required when the number of covariates is not small.

In this paper, we show how these difficulties in estimating the CATE can sometimes be overcome through the use of transfer learning. In particular, we provide several strategies for utilizing ancillary datasets that are related to the causal mechanism under investigation. Examples of such datasets include observations from: experiments in different locations on different populations, different treatment arms, different outcomes, and non-experimental observational studies. We show that, by transferring information from these ancillary datasets, CATE estimators can converge to better solutions with fewer samples. This is particularly important for CATE estimation, as the cost of collecting additional data is quite high and often requires real-world data collection. Our contributions are as follows:

1. We introduce the new problem of transfer learning for estimating heterogeneous treatment effects.

2. MLRW Transfer for CATE Estimation adapts the idea of meta-learning regression weights (MLRW) to CATE estimation. By using a learned initialization, regression problems can be optimized much more quickly than with random initializations. Though a variety of MLRW algorithms exist, it is not immediately obvious how one should use these methods for CATE estimation. The principal difficulty is that CATE estimation requires the simultaneous estimation of outcomes under both treatment and control, but we only observe one of the outcomes for any individual unit. Most MLRW transfer methods optimize on a per-task basis to estimate a single quantity. We show that one can overcome this problem with clever use of the Reptile algorithm (Nichol et al., 2018) .

3. We provide several additional methods for transfer learning for CATE estimation: warm start, frozen-features, multi-head, and joint training.

4. We apply our methods to difficult data problems and show that they perform better than existing benchmarks. We reanalyze a set of large field experiments that evaluate the effect of a mailer on voter turnout in the 2014 U.S. midterm elections (Gerber et al., 2017) . This includes 17 experiments with 1.96 million individuals in total. We also simulate several randomized controlled trials using image data of handwritten digits found in the MNIST database (LeCun, 1998) . We show that our methods, MLRW in particular, obtain better than state-of-the-art performance in estimating CATE, and that they require far fewer observations than extant methods.

5.:

We provide open source code for our algorithms.1 

2. CATE ESTIMATION:

We begin by formally introducing the CATE estimation problem. Following the potential outcomes framework (Rubin, 1974) , assume there exists a single experiment wherein we observe N i.i.d. distributed units from some super population,

(Y i (0), Y i (1), X i , W i ) ⇠ P. Y i (0) 2 R denotes the potential outcome of unit i if it is in the control group, Y i (1) 2 R is the potential outcome of i if it is in the treatment group, X

i 2 R d is a d-dimensional feature vector, and W i 2 {0, 1} is the treatment assignment. For each unit in the treatment group (W i = 1), we only observe the outcome under treatment, Y i (1). For each unit under control (W i = 0), we only observe the outcome under control. Crucially, there cannot exist overlap between the set of units for which W i = 1 and the set for which W i = 0. It is impossible to observe both potential outcomes for any unit. This is commonly referred to as the fundamental problem of causal inference.

However, not all hope is lost. We can still estimate the Conditional Average Treatment Effect (CATE) of the treatment. Let x be an individual feature vector. Then the CATE of x, denoted ⌧ (x), is defined by

⌧ (x) = E[Y (1) Y (0)|X = x].

Estimating ⌧ is impossible without making further assumptions on the distribution of

(Y i (0), Y i (1), X i , W i ).

In particular, we need to place two assumptions on our data. Assumption 1 (Strong Ignorability, Rosenbaum and Rubin (1983) ) Then there exists constant 0 < e min , e max < 1 such that for all x 2 Support(X), 0 < e min < e(x) < e max < 1.

EQUATION

In words, e(x) is bounded away from 0 and 1.

Assumption 1 ensures that there is no unobserved confounder, a random variable which influences both the probability of treatment and the potential outcomes, which would make the CATE unidentifiable.

The assumption is particularly strong and difficult to check in applications. Meanwhile, Assumption 2 rectifies the situation wherein a certain part of the population is always treated or always in the control group. If, for example, all women were in the control group, one cannot identify the treatment effect for women. Though both assumptions are strong, they are nevertheless satisfied by design in randomized controlled trials. While the estimators we discuss would be sensible in observational studies when the assumptions are satisfied, we warn practitioners to be cautious in such studies, especially when the number of covariates is large (D'Amour et al., 2017) .

Given these two assumptions, there exist many valid CATE estimators. The crux of these methods is to estimate two quantities: the control response function,

µ 0 (x) = E[Y (0)|X = x],

and the treatment response function,

µ 1 (x) = E[Y (1)|X = x].

If we denote our learned estimates as μ0 (x) and μ1 (x), then we can form the CATE estimate as the difference between the two ⌧ (x) = μ1 (x) μ0 (x). The astute reader may be wondering why we don't simply estimate µ 0 and µ 1 with our favorite function approximation algorithm at this point and then all go home. After all, we have access to the ground truths µ 0 and µ 1 and the corresponding inputs x. In fact, it is commonplace to do exactly that. When people directly estimate µ 0 and µ 1 with their favorite model, we call the procedure a T-learner (Künzel et al., 2017) . Common choices of models include linear models and random forests, though neural networks have recently been considered (Nie and Wager, 2017) .

A practitioner of deep learning might find the T-learner quite trivial. After all, it amounts to using neural networks to fit two quantities, µ 0 and µ 1 . However, it is important to note that the T-learner is a baseline method. We use it in this paper only to ease exposition, especially as it relates to transfer learning. The T-learner has many drawbacks (Athey and Imbens, 2015) . It is almost always an inefficient estimator. For example, it will often perform poorly when one can borrow information across the treatment conditions. For these reasons, more sophisticated learners such as the S, X, T, R, and Y learners are almost always used instead of the T-learner (Hill, 2011; Athey and Imbens, 2016; Nie and Wager, 2017; Künzel et al., 2017; Stadie et al., 2018) . Although much of our exposition will focus on transfer learning in the context of the T-learner, in practice we extend the discussed methods to these other more advanced learners, as shown in the Evaluation section. Descriptions of these more advanced estimators are given in the appendix.

3. TRANSFER LEARNING:

In this section, we consider a scenario wherein one has access to many related causal inference experiments. The goal is to use the results from some old experiments to obtain faster training with less data on other new experiments. Since direct transfer between different populations is wrought with difficulty, we will instead achieve transfer by using previous experiments to help find an initialization for new experiments which leads to faster optimization. 2We consider two kinds of algorithms. First, there are transfer algorithms that sit on top of existing CATE estimators. These transfer algorithms take a CATE estimation strategy, such as the S-learner, and provide a recipe for transforming it into a transfer learning CATE estimator. The second class of algorithms does not sit on top of existing CATE estimation strategies. Instead, they are built from the ground up to take advantage of transfer learning. These algorithms are joint training and MLRW.

Across all experiments, the input space X is the same. Let i index an experiment. Each experiment has its own distinct outcome when treatment is received, µ 1,i (x), and when no treatment is received, µ 0,i (x). Together, these quantities define the CATE ⌧ •,i (x) = µ 1,i (x) µ 0,i (x), In standard CATE estimation, we define a strategy that takes x as input and outputs predictions μ0,i (x) and μ1,i (x). 

Warm start method:

Frozenfeatures method In transfer learning, the hope is that we can transfer knowledge between experiments. The model parameters that allowed us predict µ 0,i (x), µ 1,i (x) and ⌧ •,i (x) from experiment i should help us predict µ 1,j (x), µ 0,j (x), and ⌧ •,j (x) from experiment j.

X 1 1 X 1 0 L 1 π ϕ 1 0 π ϕ 1 1 μ1 1 μ1 0 π θ 0 0 π θ 0 1 X 0 1 X 0 0 L 0 π ϕ 0 0 π ϕ 0 1 μ0 1 μ0 0 π θ 1 0 π θ 1 1 backprop backprop warm start random initialization X 1 1 X 1 0 L 1 π ϕ 1 0 π ϕ 1 1 μ1 1 μ1 0 π γ 0 π γ 1 X 0 1 X 0 0 L 0 π ϕ 0 0 π ϕ 0 1 μ0 1 μ0 0 π γ 0 π γ 1 backprop backprop random initialization warm start warm start Multihead method X 1 1 X 1 0 L 1 π ϕ 1 0 π ϕ 1 1 μ1 1 μ1 0 π γ 0 π γ 1 X 0 1 X 0 0 L 0 π ϕ 0 0 π ϕ 0 1 μ0 1 μ0 0 π γ 0 π γ 1

Let ⇡:

✓ be a generic expression for a neural network parameterized by ✓. Parameters will have two subscripts. The index on the left indicates if their neural network predicts treatment or control (0 for control and 1 for treatment). The index on the right is for the experiment. For example, ✓ 0,2 parametrizes ⇡ ✓0,2 (x) to predict µ 0,2 (x), the outcome under control for Experiment 2. All of the transfer algorithms described here are presented in full detail as pseudo-code in the appendix.

3.0.1. ALGORITHMS THAT EXTEND EXISTING CATE ESTIMATORS TO TRANSFER LEARNING:

These algorithms extend existing CATE estimation techniques to the transfer learning setting. The following exposition is largely motivated by transfer learning with the T-Learner as a base CATE estimator. This is only for ease of exposition. The discussed procedures can extend to other, more complicated, CATE estimators such as the R, X, Y, and S learners. Warm start: Experiment 0 predicts ⇡ ✓0,0 (x) = μ0,0 (x) and ⇡ ✓1,0 (x) = μ1,0 (x) to form the CATE estimator ⌧•,0 = μ1,0 (x) μ0,0 (x). Suppose ✓ 0,0 , ✓ 1,0 are fully trained and produce a good CATE estimate. For experiment 1, the input space X is identical to the input space for experiment 0, but the outcomes µ 0,1 (x) and µ 1,1 (x) are different. However, we suspect the underlying data representations learned by ⇡ ✓0,0 and ⇡ ✓1,0 are still useful. Hence, rather than randomly initializing ✓ 0,1 and ✓ 1,1 for experiment 1, we set ✓ 0,1 = ✓ 0,0 and ✓ 1,1 = ✓ 1,0 . We then train ⇡ ✓0,1 (x) = μ0,1 (x) and ⇡ ✓1,1 (x) = μ1,1 (x). See Figure 1 and Algorithm 8 in the appendix. Frozen-features: Begin by training ⇡ ✓0,0 and ⇡ ✓1,0 to produce good CATE estimates for experiment 0. Assuming ✓ 0,0 and ✓ 1,0 have more than k layers, let 0 be the parameters corresponding to the first k layers of ✓ 0,0 . Define 1 analogously. Since we think the features encoded by ⇡ i (X) would make a more informative input than the raw features X, we want to use those features as a transformed input space for ⇡ ✓0,1 and ⇡ ✓1,1 . To wit, set z 0 = ⇡ 0 (x) and

z 1 = ⇡ 1 (x). Then form the estimates ⇡ ✓0,1 (z 0 ) = μ0,1 (x) and ⇡ ✓1,1 (z 1 ) = μ1,1 (x).

During training of experiment 1, we only backpropagate through ✓ 0,1 and ✓ 1,1 and not through 0 and 1 . See Figure 1 and Algorithm 9 in the appendix. Multi-head: In this setup, all experiments share base layers that are followed by experiment-specific layers. The intuition is that the base layers should learn general features, and the experimentspecific layers should transform those features into estimates of µ j,i (x). More concretely, let 0 and 1 be shared base layers for estimating µ 0,• (x) and µ 1,• (x) respectively. Set z 0 = ⇡ 0 (x 0 ) and z 1 = ⇡ 1 (x 1 ). The base layers are followed by experiment-specific layers 0,i and 1,i . Let

✓ j,i = [ j , j,i ]. Then ⇡ ✓j,i (x) = ⇡ j,i ⇡ j (x) = ⇡ j,i (z j ) = μj,i (x).

Training alternates between experiments: each ✓ 0,i and ✓ 1,i is trained for some small number of iterations, and then the experiment and head being trained are switched. Every head is usually trained several times. See Figure 1 Algorithm 10 in the appendix.

SF Reptile transfer for CATE estimators: Pick your favorite CATE estimator. The goal is to learn an initialization for that CATE estimator's weights that leads to fast convergence on new experiments. More concretely, starting from good initializers ✓ 0 and ✓ 1 , one can train neural networks ⇡ ✓0 and ⇡ ✓1 to estimate µ 0,i (x) and µ 1,i (x) much faster and with less data than starting from random initializations. To learn these good initializations, we use a transfer learning technique called Reptile. The idea is to perform experiment-specific inner updates U (✓) and then aggregate them into outer updates of the form

✓ new = ✏ • U (✓) + (1 ✏) • ✓.

In this paper, we consider a slight variation of Reptile. In standard Reptile, ✏ is either a scalar or correlated to per-parameter weights furnished via SGD. For our problem, we would like to encourage our network layers to learn at different rates. The hope is that the lower layers can learn more general, slowly-changing features like in the frozen features method, and the higher layers can learn comparatively faster features that more quickly adapt to new tasks after ingesting the stable lower-level features. To accomplish this, we take the path of least resistance and make ✏ a vector which assigns a different learning rate to each neural network layer. Because our intuition involves slow and fast weights, we will refer to this modification in this paper as SF Reptile: Slow Fast Reptile. Though this change is seemingly small, we found it boosted performance on our problems. See Algorithm 11.

3.0.2. TRANSFER LEARNING ALGORITHMS THAT DO NOT EXTEND CATE ESTIMATION STRATEGIES:

Joint training: All predictions share base layers ✓. From these base layers, there are two heads per experiment i: one to predict µ 0,i (x) and one to predict µ 1,i (x). Every head and the base features are trained simultaneously by optimizing with respect to the loss function

L = P i k (μ 0,i (x) µ 0,i (x)) k + k (μ 1,i (x) µ 1,i (x)

) k and minimizing over all weights. This will encourage the base layers to learn generally applicable features and the heads to learn features specific to predicting a single µ j,i (x). See Algorithm 6. MLRW transfer: In this method, there exists one single set of weights ✓. There are no experimentspecific weights. Furthermore, we do not use separate networks to estimate µ 0 and µ 1 . Instead,

⇡:

✓ is trained to estimate one µ i,j (x) at a time. We train ✓ with SF Reptile so that in the future ⇡ ✓ requires minimal samples to fit µ i,j (x) from any experiment. To actually form the CATE estimate, we use a small number of training samples to fit ⇡ ✓ to µ 0,i (x) and then a small number of training samples to fit ⇡ ✓ to µ 1,i (x). We call ✓ meta-learned regression weights (MLRW) because they are meta-learned over many experiments to quickly regress onto any µ i,j (x). The full MLRW algorithm is presented as Algorithm 5.

4. EVALUATION:

We evaluate our transfer learning estimators on both real and simulated data. In our data example, we consider the important problem of voter encouragement. Analyzing a large data set of 1.96 million potential voters, we show how transfer learning across elections and geographic regions can dramatically improve our CATE estimators. To the best of our knowledge, this is the first successful demonstration of transfer learning for CATE estimation. The simulated data has been intentionally chosen to be different in character from our real-world example. In particular, the simulated input space is images and the estimated outcome variable is continuous.

4.1. GOTV EXPERIMENT:

To evaluate transfer learning for CATE estimation on real data, we reanalyze a set of large field experiments with more than 1.96 million potential voters (Gerber et al., 2017) . The authors conducted 17 experiments to evaluate the effect of a mailer on voter turnout in the 2014 U.S. Midterm Elections. The mailer informs the targeted individual whether or not they voted in the past four major elections (2006, 2008, 2010, and 2012) , and it compares their voting behavior with that of the people in the same state. The mailer finishes with a reminder that their voting behavior will be monitored. The idea is that social pressure-i.e., the social norm of voting-will encourage people to vote. The likelihood of voting increases by about 2.2% (s.e.=0.001) when given the mailer.

Each of the experiments targets a different state. This results in different populations, different ballots, and different electoral environments. In addition to this, the treatment is slightly different in each experiment, as the median voting behavior in each state is different. However, there are still many similarities across the experiments, so there should be gains from transferring information.

In this example, the input X is a voter's demographic data including age, past voting turnout in 2006, 2008, 2009, 2010, 2011, 2012, and 2013 , marital status, race, and gender. The treatment response function μ1 (x) estimates the voting propensity for a potential voter who receives a mailer encouraging them to vote. The control response function μ0 estimates the voting propensity if that voter did not receive a mailer. The CATE ⌧ is thus the change in the probability of voting when a unit receives a mailer. The complete dataset has this data over 17 different states. Treating each state as a separate experiment, we can perform transfer learning across them. Being able to estimate the treatment effect of sending a mailer is an important problem in elections.

x outcome µ 0 µ 1 ⌧ A

We may wish to only treat people whose likelihood of voting would significantly increase when receiving the mailer, to justify the cost for these mailers. Furthermore, we wish to avoid sending mailers to voters who will respond negatively to them. This negative response has been previously observed and is therefore feasible and a relevant problem-e.g., some recipients call their Secretary of State's office or local election registrar to complain (Mann, 2010; Michelson, 2016) .

Evaluating a CATE estimator on real data is difficult. The primary difficulty is that we do not get to observe the true CATE for any unit, due to the fundamental problem of causal inference. By definition, only one of the two outcomes is observed for any unit. One could use the original features and simulate the outcome features, but this would require us to create a response model. Instead, we estimate the "truth" on the real data using linear models (version 1) or random forests (version 2). We then construct the data based on these estimates. For a detailed description, see Appendix A.2. We then ask the question: How do the various methods perform when they have less data than the entire sample?

RESULTS:

We evaluate all the algorithms discussed in section 3 on the GOTV dataset. For the algorithms in section 3.0.1 that require a base CATE estimator, we use the Y learner because we found it delivered the best performance. 3 For baselines, we compare against the non-transfer Y-learner and the S learner with random forests. 4 In previous work, state of the art results on this problem have been achieved with both non-transfer tree-based estimators such as S-RF (Künzel et al., 2017; Green and Kern, 2012) and neural-network-based learners such as the R and Y-learners (Nie and Wager, 2017; Stadie et al., 2018) .

The best estimator is MLRW. This algorithm consistently converges to a very good solution with very few observations. Looking at Tables 1, 2 , and 3, we observe that MLRW is the best performing transfer learner for GOTV version 1 in 8 out of 17 trials. In GOTV version 2, it is the best in 11 out of 17 trials. In Figure 2 , its average performance is dominant over all other algorithms. We hypothesize that this method does best because it does not try to artificially bottleneck the flow of information between outcomes and experiments. MLRW also seems more resilient to data-poisoning when it encounters outlier data, though we did not concretely test against this. We also observe that multi-head, frozen-features, and SF all generally improve upon non-transfer baselines. The faster learning rate of these algorithms indicates that positive transfer between experiments is occurring. Warm start, however, does not work well and often even leads to worse results than the baseline estimators. This is consistent with prior findings on warm start (Finn et al., 2017) . 

4.2. MNIST EXAMPLE:

In the previous experiment, we observed that the MLRW estimator performed most favorably, and transfer learning significantly improved upon the baseline. To confirm that this conclusion is not specific to voter persuasion studies, we intentionally consider a very different type of data. Recently, (Nie and Wager, 2017) introduced a simulation study wherein MNIST digits are rotated by some number of degrees ↵; with ↵ furnished via a single data generating process that depends on the value of the depicted digit. They then attempt to do CATE estimation to measure the heterogeneous treatment effect of a digit's label.

Motivated by this example, we develop a data generating process using MNIST digits wherein transfer learning for CATE estimation is applicable. In our example, the input X is an MNIST image. We have k data-generating processes which return different outcomes for each input when given either treatment or control. Thus, under some fixed data-generating process, µ 0 represents the outcome when the input image X is given the control, µ 1 represents the outcome when X is given the treatment, and ⌧ is the difference in outcomes given the placement of X in the treatment or control group. Each data-generating process has different response functions (µ 0 and µ 1 ) and thus different CATEs (⌧ ), but each of these functions only depends on the label presented in the image X. We thus hope that transfer learning could expedite the process of learning features which are indicative of the label. See Appendix A for full details of the data generation process. In Figure 3 , we confirm that a transfer learning strategy outperforms its non-transfer learning counterpart, even on image data. We also see that MLRW performs well, though in this case multi-head is competitive. We also see that several of the transfer methods are worse than non-transfer baselines.

Figure 3 : MNIST task. The baseline is the S-learner. All transfer CATE estimators for this task are built on top the S-learner, rather than the Y-learner, because we found it delivered better performance for this problem.

5. RELATED WORKS AND DISCUSSION:



5.0.1. RELATED WORKS:

In this paper, we proposed the problem of transfer learning for CATE estimation. One immediate question the reader may be left with is why we chose the transfer learning techniques we did. We only considered two common types of transfer: (1) Basic fine tuning and weights sharing techniques common in the computer vision literature (Welinder et al., 2010; Saenko and Darrell, 2010; Bourdev et al., 2011; Donahue et al., 2014; Koch, 2015) , (2) Techniques for learning an initialization that can be quickly optimized (Finn et al., 2017; Ravi and Larochelle, 2017; Nichol et al., 2018) . However, many further techniques exist. Yet, transfer learning is an extensively studied and perennial problem (Schmidhuber, 1992; Bengio et al., 1992; Thrun, 1996; Thrun and Pratt, 1998; Taylor and Stone, 2009; Silver et al., 2013) . In (Vinyals et al., 2016) , the authors attempt to combine feature embeddings that can be utilized with non-parametric methods for transfer. (Snell et al., 2017) is an extension of this work that modifies the procedure for sampling examples from the support set during training. (Andrychowicz et al., 2016) and related techniques try to meta-learn an optimizer that can more quickly solve new tasks. (Rusu et al., 2016) attempts to overcome forgetting during transfer by systematically introducing new network layers with lateral connections to old frozen layers. (Munkhdalai and Yu, 2017) uses networks with memory to adapt to new tasks. We invite the reader to review (Finn et al., 2017) for an excellent overview of the current transfer learning landscape. Though the majority of the discussed techniques could be extended to CATE estimation, our implementations of (Rusu et al., 2016; Andrychowicz et al., 2016) proved difficult to tune and consequently learned very little. Furthermore, we were not able to successfully adapt (Snell et al., 2017) to the problem of regression. We decided to instead focus our attention on algorithms for obtaining good initializations, which were easy to adapt to our problem and quickly delivered good results without extensive tuning.

On the topic of using neural networks to improve causal inference algorithms, a flurry of relevant work exists (Ramachandra, 2018; Magliacane et al., 2017; Johansson et al., 2016; Louizos et al., 2017; Alaa et al., 2017; Shalit et al., 2017; Nie and Wager, 2017) . We found that these papers either did not allow us to better estimate the CATE, or else provided worse performance than the baseline methods we did consider in this paper. Extending transfer to other causal inference algorithms is an ongoing and interesting area of research.

5.0.2. CLOSING REMARKS:

We are left with several open questions. Can transfer learning still be applied to CATE estimation when the experiment input spaces differ? How should one properly deal with missing and incomplete data? Do there exist better methods for interpretability, highlighting which features are most important for transfer and why? Can these techniques be extended to causal models outside of CATE estimation? How can one properly encode causal relationships into a neural network? Answering these questions would have a positive impact on fields such as causal inference, deep learning, and reinforcement learning.

A APPENDIX: SIMULATION STUDIES AND APPLICATION:

A.1 MNIST SIMULATION For our MNIST simulation study (Section 4.2), we used the MNIST database (LeCun, 1998) which contains labeled handwritten images. We follow here the notation of Nie and Wager (2017) , who introduce a very similar simulation study which is not trying to evaluate transfer learning for CATE estimation, but instead emulates a RCT with the goal to evaluate different CATE estimators.

The MNIST data set contains labeled image data (X i , C i ), where X i denotes the raw image of i and C i 2 {0, . . . , 9} denotes its label. We create k Data Generating Processes (DGPs), D 1 , . . . , D k , each of which specifies a distribution of (Y i (0), Y i (1), W i , X i ) and represents different CATE estimation problems.

In this simulation, we let W i = 0 if the image X i is placed in the control, and

W i = 1 if the image X i is placed in the treatment. Y i (W i ) quantifies the the outcome of X i under W i .

To generate a DGP D j , we first sample weights in the following way, m j (0), m j (1), . . . , m j (9)

iid ⇠ Unif( 3, 3), t j (0), t j (1), . . . , t j (9)

iid ⇠ Unif( 1, 1), p j (0), p j (1), . . . , p j (9)

iid ⇠ Unif(0.3, 0.7),

and we define the response functions and the propensity score as

µ j 0 (C i ) = m j (C i ) + 3C i , µ j 1 (C i ) = µ j 0 (C i ) + t j (C i ), e j (C i ) = p j (C i ). To generate (Y i (0), Y i (1), W i , X i ) from D j ,

we fist sample a (X i , C i ) from the MNIST data set, and we then generate Y i (0), Y i (1), and W i in the following way:

" i iid ⇠ N (0, 1) Y i (0) = µ 0 (C i ) + " i Y i (1) = µ 1 (C i ) + " i W i ⇠ Bern(e(C i )).

During training, X:

i , W i , and Y i are made available to the convolutional neural network, which then predicts ⌧ given a test image X i and a treatment W i . ⌧ is the difference in the outcome given the difference in treatment and control.

Having access to multiple DGPs can be interpreted as having access to prior experiments done on a similar population of images, allowing us to explore the effects of different transfer learning methods when predicting the effect of a treatment in a new image.

A.2 GOTV DATA EXAMPLE AND SIMULATION:

In this section, we describe how the simulations for the GOTV example in the main paper were done and we discuss the results of a much bigger simulation study with 51 experiments which is summarized in Tables 1, 2, and 3.

A.2.1 DATA GENERATING PROCESSES FOR OUR REAL WORLD EXAMPLE:

For our data example, we took one of the experiments conducted by Gerber et al. (2017) . The study took place in 2014 in Alaska and 252,576 potential voters were randomly assigned in a control and a treatment group. Subjects in the treatment group were sent a mailer as described in the main text and their voting turnout was recorded.

To evaluate the performance of different CATE estimators we need to know the true CATEs, which are unknown due to the fundamental problem of causal inference. To still be able to evaluate CATE estimators researchers usually estimate the potential outcomes using some machine learning method and then generate the data from this estimate. This is to some extend also a simulation, but unlike classical simulation studies it is not up to the researcher to determine the data generating distribution. The only choice of the researcher lies in the type of estimator she uses to estimate the response functions. To avoid being mislead by artifacts created by a particular method, we used a linear model in Real World Data Set 1 and random forests estimator in Real World Data Set 2. Specifically, we generate for each experiment a true CATE and we simulate new observed outcomes based on the real data in four steps.

1. We first use the estimator of choice (e.g., a random forests estimator) and train it on the treated units and on the control units separately to get estimates for the response functions, µ 0 and µ 1 . 2. Next, we sample N units from the underlying experiment to get the features and the treatment assignment of our samples (X i , W i ) N i=1 . 3. We then generate the true underlying CATE for each unit using ⌧

i = ⌧ (X i ) = µ 1 (X i ) µ 0 (X i ).

4. Finally we generate the observed outcome by sampling a Bernoulli distributed variable around mean µ i .

Y obs i ⇠ Bern(µ i ), µ i = ⇢ µ 0 (X i ) if W = 0, µ 1 (X i ) if W = 1.

After this procedure, we have 17 data sets corresponding to the 17 experiments for which we know the true CATE function, which we can now use to evaluate CATE estimators and CATE transfer learners.

A.2.2 DATA GENERATING PROCESSES FOR OUR SIMULATION STUDY:

Simulations motivated by real-world experiments are important to assess whether our methods work well for voter persuasion data sets, but it is important to also consider other settings to evaluate the generalizability of our conclusions.

To do this, we first specify the control response function, µ 0 (x) = E[Y (0)|X = x] 2 [0, 1], and the treatment response function,

µ 1 (x) = E[Y (1)|X = x] 2 [0, 1].

We then use each of the 17 experiments to generate a simulated experiment in the following way:

1. We sample N units from the underlying experiment to get the features and the treatment assignment of our samples (X i , W i ) N i=1 . 2. We then generate the true underlying CATE for each unit using ⌧

i = ⌧ (X i ) = µ 1 (X i ) µ 0 (X i ).

3.. Finally we generate the observed outcome by sampling a Bernoulli distributed variable:

around mean µ i .

Y obs i ⇠ Bern(µ i ), µ i = ⇢ µ 0 (X i ) if W = 0, µ 1 (X i ) if W = 1.

The experiments range in size from 5,000 units to 400,000 units per experiment and the covariate vector is 11 dimensional and the same as in the main part of the paper. We will present here three different setup. Simulation LM (Table 1 ): We choose here N to be all units in the corresponding experiment. Sample

0 = ( 0 1 , . . . , 0 d ) iid ⇠ N (0, 1) and 1 = ( 1 1 , . . . , 1 d )

iid ⇠ N (0, 1) and define,

µ 0 (x) = logistic x 0 , µ 1 (x) = logistic x 1 .

Simulation RF (Table 2 ): We choose here N to be all units in the corresponding experiment.

1. Train a random forests estimator on the real data set and define µ 0 to be the resulting estimator, 2. Sample a covariate f (e.g., age), 3. ample a random value in the support of f (e.g., 38), 4. Sample a shift s ⇠ N (0, 4).

Now define the potential outcomes as follows:

µ 0 (x) = trained Random Forests algorithm µ 1 (x) = logistic (logit (µ 0 (x) + s ⇤ 1 f v ))

Simulation RFt (Table 3 ): This experiment is the same as Simulation RF, but use only one percent of the data, N = #units 100 .

B PSEUDO CODE FOR CATE ESTIMATORS:

In this section, we will present pseudo code for the CATE estimators in this paper. We present code for the meta learning algorithms in Section C. We denote by Y 0 and Y 1 the observed outcomes for the control and the treated group. For example, Y 1 i is the observed outcome of the ith unit in the treated group. X 0 and X 1 are the features of the control and treated units, and hence, X 1 i corresponds to the feature vector of the ith unit in the treated group. M k (Y ⇠ X) is the notation for a regression estimator, which estimates x 7 ! E[Y |X = x]. It can be any regression/machine learning estimator, but in this paper we only choose it to be a neural network or random forest.

These algorithms first appeared in (Künzel et al., 2017; Stadie et al., 2018) . We reproduce them here for completeness.

Algorithm 1 T-learner 1: procedure T-LEARNER(X, Y obs , W ) 2: μ0 = M 0 (Y 0 ⇠ X 0 ) 3: μ1 = M 1 (Y 1 ⇠ X 1 ) 4:

⌧ (x) = μ1 (x) μ0 (x) 5: end procedure M0 and M1 are here some, possibly different machine learning/regression algorithms.

Algorithm 2 S-learner

1: procedure S-LEARNER(X, Y obs , W ) 2: μ = M (Y obs ⇠ (X, W )) 3: ⌧ (x) = μ(x, 1) μ(x, 0) 4: end procedure M (Y obs ⇠ (X, W )) is the notation for estimating (x, w) 7 ! E[Y |X = x, W = w] while treating W as a 0,1-valued feature. Algorithm 3 X-learner 1: procedure X-LEARNER(X, Y obs , W, g) 2: μ0 = M 1 (Y 0 ⇠ X 0 ) . Estimate response function 3: μ1 = M 2 (Y 1 ⇠ X 1 ) 4: D1 i = Y 1 i μ0 (X 1 i )

. Compute imputed treatment effects 5:

D0 i = μ1 (X 0 i ) Y 0 i 6: ⌧1 = M 3 ( D1 ⇠ X 1 )

. Estimate CATE for treated and control 7:

⌧0 = M 4 ( D0 ⇠ X 0 ) 8: ⌧ (x) = g(x)⌧ 0 (x) + (1 g(x))⌧ 1 (x)

. Average the estimates 9: end procedure g(x) 2 [0, 1] is a weighing function which is chosen to minimize the variance of ⌧ (x). It is sometimes possible to estimate Cov(⌧0(x), ⌧1(x)), and compute the best g based on this estimate. However, we have made good experiences by choosing g to be an estimate of the propensity score, but also choosing it to be constant and equal to the ratio of treated units usually leads to a good estimator of the CATE.

Under review as a conference paper at ICLR 2019

Algorithm 4 Y-Learner Pseudo Code 1: if W i == 0 then 2: Update the network ⇡ ✓0 to predict Y obs i 3: Update the network ⇡ ✓1 to predict Y obs i + ⇡ ⌧ (X i ) 4: Update the network ⇡ ⌧ to predict ⇡ ✓1 (X i ) Y obs i 5: end if 6: if W i == 1 then 7: Update the network ⇡ ✓0 to predict Y obs i ⇡ ⌧ (X i ) 8:

Update the network ⇡ ✓1 to predict Y obs i 9:

Update the network for i < numexps do 9:

⇡ ⌧ to predict Y obs i ⇡ ✓0 (X i ) 10: end if

Sample X 0 and X 1 : control and treatment units from experiment i 10:

for j = [0, 1] do . j iterating over treatment and control 11:

Let U 0 (✓) = ✓ 12:

for k < inneriters do 13:

L = k⇡ U k (✓) (X j ) µ j (X j )k 14:

Compute r ✓ L.

15::

Use ADAM with r ✓ L to obtain U k+1 (✓).

16:

U k (✓) = U k+1 (✓) 17:

end for 18:

for p < N do 19:

✓ p = ✏ p • U k (✓ p ) + (1 ✏ p ) • ✓ p . 20:

end for Sample X 0 and X 1 : control and treatment units from experiment i 28:

Sample X: test units from experiment i.

29::

for j = [0, 1] do . j iterating over treatment and control 30:

for k < innteriters do 31:

L = k⇡ U k (✓) (X j ) µ j (X j )k 32:

Compute r ✓ L.

33::

Use ADAM with r ✓ L to obtain U k+1 (✓).

34:

U k (✓) = U k+1 (✓) 35:

end for 36: 1 be the outcome under control and treatment for experiment i. 2: Let numexps be the number of experiments. 3: Let ⇡ ⇢ be a generic expression for a neural network parameterized by ⇢. 4: Let ✓ be base neural network layers shared by all experiments. 5: Let (i) 0 be neural network layers predicting µ

μj = ⇡ U k (✓) (X)

(i) 0 in experiment i. 6: Let (i) 1 be neural network layers predicting µ (i) 1 in experiment i. 7: Let ! (i) 0 = h ✓, (i) 0 i

be the full prediction network for µ 0 in experiment i.

EQUATION

i be the full prediction network for µ 1 in experiment i.

9: Let ⌦ = S 1 j=0 S numexps i=1 ! (i)

j be all trainable parameters. 10: Let numiters be the total number of training iterations 11: for iter < numiters do 12: L = 0 13: for i < numexps do 14: Sample X 0 and X 1 : control and treatment units from experiment i 15:

for j = [0, 1] do . j iterating over treatment and control 16:

L (i) j = k⇡ ! (i) j (X j ) µ j (X j )k 17: L = L + L (i) j 18:

end for 19:

end for 20:

Compute r ⌦ L = @L @⌦ =

P i P j @L (i) j @! (i) j 21:

Apply ADAM with gradients given by r ⌦ L.

22::

for i < numexps do 

C.3 T-LEARNER TRANSFER CATE ESTIMATORS:

Here, we present full pseudo code for the algorithms from Section 3 using the T-learner as a base learner. All of these algorithms can be extended to other learners including S, R, X, and Y . See the released code for implementations.

Algorithm 7 Vanilla T-learner (also referred to as Baseline T-learner)

1: Let µ 0 and µ 1 be the outcome under treatment and control.

2: Let X be the experimental data. Let X t be the test data. 3: Let ⇡ ✓0 and ⇡ ✓1 be a neural networks parameterized by ✓ 0 and ✓ 1 . 4: Let ✓ = ✓ 0 [ ✓ 1 . 5: Let numiters be the total number of training iterations. 6: Let batchsize be the number of units sampled. We use 64. 7: for i < numiters do 8:

Sample X 0 and X 1 : control and treatment units. Sample batchsize units. 9:

L 0 = k⇡ ✓ (X 0 ) µ 0 (X 0 )k 10: L 1 = k⇡ ✓ (X 1 ) µ 1 (X 1 )k 11: L = L 0 + L 1 12:

Compute r ✓ L = @L @✓ .

13:

Apply ADAM with gradients given by r ✓ L. 14: end for

15: μ0 = ⇡ ✓0 (X t ) 16: μ1 = ⇡ ✓1 (X t ) 17: return CATE estimate ⌧ = μ1 μ0

Algorithm 8 Warm Start T-learner 1: Let µ i 0 and µ i 1 be the outcome under treatment and control for experiment i. 2: Let X i be the data for experiment i. Let X i t be the test data for experiment i. 3: Let ⇡ ✓0 and ⇡ ✓1 be a neural networks parameterized by ✓ 0 and ✓ 1 . 4: Let ✓ = ✓ 0 [ ✓ 1 . 5: Let numiters be the total number of training iterations. 6: Let batchsize be the number of units sampled. We use 64. 7: for i < numiters do 8: Sample X 0 0 and X 0 1 : control and treatment units for experiment 0. Sample batchsize units. 9:

L 0 = k⇡ ✓0 (X 0 0 ) µ 0 (X 0 0 )k 10: L 1 = k⇡ ✓1 (X 0 1 ) µ 1 (X 0 1 )k 11: L = L 0 + L 1 12:

Compute r ✓ L = @L @✓ .

13:

Apply ADAM with gradients given by r ✓ L. 14: end for 15: for i < numiters do 16:

Sample X 1 0 and X 1 1 : control and treatment units for experiment 1. Sample batchsize units. 17:

L 0 = k⇡ ✓0 (X 1 0 ) µ 0 (X 1 0 )k 18: L 1 = k⇡ ✓1 (X 1 1 ) µ 1 (X 1 1 )k 19: L = L 0 + L 1 20:

Compute r ✓ L = @L @✓ .

21::

Apply ADAM with gradients given by r ✓ L. 22: end for 23: μ0 = ⇡ ✓0 (X 1 t ) 24: μ1 = ⇡ ✓1 (X 1 t ) 25: return CATE estimate ⌧ = μ1 μ0

D FULL RESULTS:

Below, we include the full results for the GOTV and MNIST experiments. In particular, we use show results for transfer CATE learners with S, T, X, and Y base learners. We also provide full tables of more comprehensive results for all methods and all train-test splits. 

:

The software will be released once anonymity is no longer needed. We can also provide an anynomized copy to reviewers upon request.

By faster optimization, we mean that starting from the learned initialization will allow the problem to be solved in fewer optimization epochs and with less data than starting with random weights.

See Tables1, 2, and 3 and Figures5, 6, and 7 for full results with the X, Y, S, and T-learners as base CATE estimators.

We use the S learner because the Y learner is not compatible with random forests.

Figures and Tables:

0)) ? W |X. Assumption 2 (Overlap) Define the propensity score of x as, e(x) := P(W = 1|X = x).

Figure 1: Warm start, frozen-features, and multi-head methods for CATE transfer learning. For these figures, we use the T-learner as the base learner for simplicity. All three methods attempt to reuse neural network features from previous experiments.

Figure 2: Social Pressure and Voter Turnout, Version 1 (real data as a linear model) and Version 2 (real data as random forest). Our transfer learning algorithms far exceed the previous state of the art non-transfer baselines, which are represented here as S-RF and Baseline (Y-NN).

This process describes training the Y-Learner for one step given a data point (Y obs i , Xi, Wi) C EXPLICIT TRANSFER LEARNING ALGORITHMS FOR CATE ESTIMATION C.1 MLRW TRANSFER FOR CATE ESTIMATION Algorithm 5 MLRW Transfer for Cate Estimation.

outcome under treatment and control for experiment i. 2: Let numexps be the number of experiments. 3: Let ⇡ ✓ be an N layer neural network parameterized by ✓ = [✓ 0 , . . . , ✓ N ]. 4: Let ✏ = [✏ 0 , . . . , ✏ N ] be a vector, where N is the number of layers in ⇡ ✓ . 5: Let outeriters be the total number of training iterations. 6: Let inneriters be the number of inner loop training iterations. 7: for oiter < outeriters do 8:

28: return CATE estimate ⌧ = μ1 μ0 5/17/2018 joint_training

Figure 4: Joint Training -Unlike the Multi-head method which differentiates base layers for treatment and control, the Joint Training method has all observations and experiments (regardless of treatment and control) share the same base network, which extracts general low level features from the data.

Figure 5: Social Pressure and Voter Turnout, Version 1. Our results far exceed the previous state of the art, which are represented here as S-RF, T-RF, and the baseline method for S-NN and T-NN. Our new methods are Y-NN and the transfer learning methods: warm, frozen, multi-head, joint, SF Reptile, and MLRW.

Figure 6: Social Pressure and Voter Turnout, Version 2. Our results exceed the previous state of the art results, which are represented here as S-RF, T-RF, and the baseline method for S-NN and T-NN. Our new methods are Y-NN and the transfer learning methods: warm, frozen, multi-head, joint, SF Reptile, and MLRW.

Figure 7: MNIST task

MSE in percent for different CATE estimators.

Table content: <table><tr><td>11.21 20.9</td><td>13.49 18.56</td><td>13.67 18.96</td><td>10.4 15.53</td><td>6.63 9.87</td><td>5.15 5.58</td><td>30.81 18.08</td><td>8.59 10.75</td><td>8.55 12.02</td><td>7.38 9.39</td><td>2.65 3.64</td><td>20.81 6.78</td><td>6.71 10.31</td><td>7.25 18.98</td><td colspan="2">39.96 32.81 0.94* 111.42</td><td>9.85 10.15</td><td>9.2 18.45</td><td>11.57 16.7</td><td>0.97 43.82</td><td>35.43 9.29</td><td>9.58 5.08</td><td>4.11 7.72</td><td>1.12 29.01</td><td colspan="2">9.91 11.68 2.75* 1.57</td></tr><tr><td>9</td><td>14.11</td><td>14.38</td><td>3.5</td><td>5.9</td><td>6.77</td><td>31.06</td><td>6.05</td><td>6.53</td><td>6.58</td><td>2.32</td><td>16.3</td><td>5.69</td><td>6.15</td><td>16.61</td><td>5.03</td><td>13.8</td><td>7.27</td><td colspan="2">7.52 1.15*</td><td>1.26</td><td>12.98</td><td>1.2</td><td>17.45</td><td>13.13</td><td>1.15</td></tr><tr><td>8.43</td><td>12.57</td><td>13.66</td><td>19.23</td><td>4.88</td><td>5.43</td><td>8.64</td><td>4.34</td><td>5.82</td><td>5.99</td><td>1.95</td><td>40.03</td><td>6.26</td><td>8.93</td><td>12.25</td><td>3.34</td><td>9.81</td><td>13.11</td><td>10.96</td><td>1.02</td><td>19.11</td><td>8.48</td><td>8.19</td><td>15.78</td><td colspan="2">8.55 1*</td></tr><tr><td>7.63</td><td>16.53</td><td>17.11</td><td>14.56</td><td>6.77</td><td>5.22</td><td>7.73</td><td>7.12</td><td>7.6</td><td>6.79</td><td>2.36</td><td>27.94</td><td>10.18</td><td>6.81</td><td colspan="2">10.81 0.85*</td><td>16.33</td><td>6.25</td><td>8.66</td><td>7.21</td><td>9.4</td><td>7.21</td><td>1.35</td><td>13.25</td><td>4.75</td><td>1.85</td></tr><tr><td>15.95 7.82 20.14 6.62 46.15 17.24 9.88 10.65 44.77</td><td>19.13 13.18 7.62 13.27 15.66 15.58 11.44 16.3 11.34</td><td>20.04 13.56 7.79 13.62 16 15.99 11.76 16.65 11.54</td><td>18.95 5.19 9.33 28.98 3.04 10.03 4.88 16.34 7.91</td><td>frozen 6.74 6.17 2.75 5.76 5.68 6.27 4.23 7.17 4.15</td><td>multi head 3.3 3.05 2.34 3.83 3.6 4.89 4.56 8.47 5.87</td><td>SF 4.85 38.65 7.54 55.92 11.36 3.98 50.76 7.74 5.72</td><td>baseline 6.84 5.29 3.77 6.86 5.94 7.19 4.6 8.08 5.61</td><td>warm 7.29 6.44 4.08 7.25 6.74 7.17 6.03 8.8 5.63</td><td>frozen 6.53 6.73 4.49 6.35 7.23 6.61 5.99 7.59 5.79</td><td>multi head 2.73 2.34 1.34 2.11 2.49 2.3 1.97 2.73 1.94</td><td>SF 20.72 27.71 8.54 20.18 23.06 15.35 14.27 13.05 9.37</td><td>baseline 22.76 5.34 5.98 5.84 5.16 10.37 10.9 7.26 10.25</td><td>warm 23.46 7.2 5.75 6.41 5.21 11.56 12.21 8.77 8.93</td><td>frozen 6.63 14.04 10.73 19.57 17.94 14 13.67 18.83 14.36</td><td>multi head 1.19 11.38 84.13 174.18 1.87 19.55 62.12 22.7 9.67</td><td>SF 18.83 10.72 10.3 10.61 10.11 12.5 21.37 11.12 8.27</td><td>baseline 19.52 8.25 4.68 5.06 6.6 11.5 10.78 12 10.26</td><td colspan="2">warm 20.06 8.54 5.57 1* frozen 1.54 2.1 6.37 10.77 9.34 11.36 13.16 8.03 3.3 1.11* 46.07 27.63 5.47 7.48</td><td>multi head 0.92 2.21 1.26 15.86 1.19 20.47 1.76 2.75 3.96</td><td colspan="2">SF 5.68 12 16.57 0.9* baseline 1.31 5.24 38 5.54 3.49 31.43 6.8 1.23* 8.2 4.8 2.61 7.5 1.07* 1.32</td><td>warm 48.91 1.26 5.79 3.61 29.43 2.71 3.94 12.87 21.76</td><td colspan="2">joint 13.75 5.81 3.46 1.05* 1 1.41 4.12 1.94* 3.46 12.22 1.97 1.26 9.58 14.96 1.03* 2.05 0.9*</td></tr><tr><td colspan="5">s-rf t-rf R-NN S-NN</td><td/><td/><td/><td colspan="2">T-NN</td><td/><td/><td/><td colspan="2">X-NN</td><td/><td/><td/><td colspan="2">Y-NN</td><td/><td/><td/><td colspan="3">joint MLRW</td></tr></table>

MSE in percent for different CATE estimators.Method RFt-1 RFt-2 RFt-3 RFt-4 RFt-5 RFt-6 RFt-7 RFt-8 RFt-9 RFt-10 RFt-11 RFt-12 RFt-13 RFt-14

Table content: <table><tr><td>83.37 4.26 4.31 31.76</td><td>5.09 9.29 11.64 4.28</td><td>12.48 14.64 13.61 12.02</td><td>54.17 23.86 3.02 18.98</td><td>17.25 64.22 24.94 11.24</td><td>1.53 0.79 0.74 1.59</td><td>5.97 19.67 11.64 8.68</td><td>1.44 2.37 2.85 1.78</td><td>3.76 5.44 3.13 3.49</td><td colspan="2">1.47 1.68 1.85 2.27 0.46* 0.55* 0.66 0.8</td><td>27.64 17.96 14.08 4.49</td><td>45.59 16.03 4.79 18.63</td><td>37.59 18.62 6.87 32.52</td><td>132 434.25 125.88 168.92</td><td>1.82 99.38 14.67 28.43</td><td>6.42 7.21 4.89 2.34</td><td>23.89 25.5 4.67 17.1</td><td>19.27 12.92 8.1 15.73</td><td>0.7 33.11 1.34 13.64</td><td>9.55 0.6 3.93 0.49</td><td>11.87 11.33 4.51 2.64</td><td>0.54 2.01 0.61 2.13</td><td>23.48 1.4 1.15 17.72</td><td>33.71 26.33 11.99 147.45 0.24* 0.57 1.32 0.8</td></tr><tr><td>8.94</td><td>6.52</td><td>18.99</td><td>86.89</td><td>27.02</td><td>1.07</td><td>9.11</td><td>5.29</td><td>5.5</td><td>1.63</td><td>0.53</td><td>13.87</td><td>30.02</td><td>40.65</td><td>83.75</td><td>18.46</td><td>7</td><td>80.71</td><td>50.45</td><td>0.81</td><td>0.41</td><td>3.37</td><td>0.54</td><td>0.74</td><td>11.97 0.4*</td></tr><tr><td>22.65 2.74 2.3 5.07 4.78 8.3 9.77 38.73 23.94</td><td>3.54 7.04 4.39 6.4 8.09 6.41 11.44 6.33 6.05</td><td>13.3 13.55 8.41 11.9 14.11 15.48 14.42 12.54 10.19</td><td>62.08 17.68 5.22 19.91 14.43 20.53 18.51 49.63 8.99</td><td>frozen 2.49 51.39 37.47 49.5 43.17 29.19 21.81 27.25 61.34</td><td>multi head 1.4 0.57 0.73 1.48 1.54 1.27 1.1 1.63 0.88</td><td>SF 0.84 68.31 4.12 10.77 17.28 20.06 10.22 1.08 8.95</td><td>baseline 14.66 2.08 0.95 1.38 2.24 7.3 3.17 1.08 0.88</td><td>warm 17.64 2.76 1.38 2.31 1.99 8.64 4.32 1.37 0.85</td><td>frozen 2.57 1.69 1.12 1.55 1.81 1.6 1.52 1.86 1.43</td><td>multi head 0.69 0.55 0.34 0.49 0.63 0.64 0.51 0.64 0.5</td><td>SF 18.04 9.54 7.33 6.61 22.11 9.21 20.2 10.65 15.06</td><td>baseline 43.37 14.55 9.48 19.35 15.68 20.91 16 41.72 6.99</td><td>warm 69.41 19.81 12.99 24.12 20.28 37.07 19.64 39.37 9.36</td><td>frozen 12.41 221.49 215.01 131.46 94.89 328.53 199.48 41.14 398.33</td><td>multi head 12.24 0.79 3.22 12.46 2.87 12.11 21.38 1.86 1.48</td><td>SF 3.98 11.03 4.33 5.31 5.42 4.67 14.03 5.99 7</td><td>baseline 60.44 15.03 4.93 7.08 9.51 21.22 14.7 16.83 8</td><td colspan="2">warm 30.46 13.02 5.11 9.95 9.21 13.12 9.95 12.8 0.33* frozen 1.72 0.87 2.47 1.8 5.44 1.35 19.37 4.48 37.98</td><td>multi head 5.95 11.12 0.41 2.94 2.49 24.73 3.44 58.63 12.99</td><td>SF 3.98 14.56 12.85 4.3 3.66 9.78 11.13 3.03 5.78</td><td colspan="2">baseline 0.88 10.92 0.28 1.32 1.89 11.35 8.42 1.6 0.31* warm 8.03 0.48 0.36 0.71 2.53 1.21 1.77 5.66 0.41*</td><td>joint 102.83 38.47 133.88 15.31 37.32 410.04 38.28 14.96 0.31* 1.25 0.21* 0.27* 0.35* 0.29* 0.33 0.61* 5.02 0.61</td></tr><tr><td colspan="5">t-lm s-rf t-rf R-NN S-NN</td><td/><td/><td/><td colspan="2">T-NN</td><td/><td/><td/><td colspan="2">X-NN</td><td/><td/><td/><td colspan="2">Y-NN</td><td/><td/><td/><td colspan="2">joint MLRW</td></tr></table>

MSE in percent for different CATE estimators.

Table content: <table/>

Peer review (iclr2019):
 The authors propose methods to address a novel task of transfer learning for estimating the CATE function. The methods proposed are several different neural network architectures and training strategies, which vary in how they leverage information from one task to another: learning joint representations and then different task-specific "heads", warm-starting from one task to another, or aggregate gradient updates across experiments. Experimental evaluation is always challenging for CATE tasks. In this paper the evaluation of the method is done using both a synthetic setting based on MNIST, and a real-world experimental dataset from a large Get Out The Vote experiment. The task itself is of high interest, and should definitely be a subject of research. The paper is mostly clearly written and technically competent. My main concerns are as follows: 1. The degree of novelty is not very high. While the task is new, I believe the technical innovation is not profound. The techniques are mostly straightforward re-use of features. The multi-head architecture has been introduced in the single experiment setting by Shalit et al. (2017). The meta-learner approach is directly adapted from Nichol et al. (2018). Also, for this task, I would have liked to see more engagement with the many other transfer learning / domain adaptation methods out there: In the very least, re-weighting based methods (which incidentally bear strong connections to propensity-score re-weighting commonly used in causal inference), as well as at least some well-known deep-learning transfer-learning methods such as Bengio "Deep learning of representations for unsupervised and transfer learning." (2012), Ganin et al. "Domain-adversarial training of neural networks."(2016), Long, Mingsheng, et al. "Unsupervised domain adaptation with residual transfer networks." (2016), or similar work. 2. The experimental evaluation can be improved in two ways: 2.a. I would like to see more baselines. For example, in the GoTV task, how well does RF do when the "ground truth" is a linear model, and vice-versa? Also, Causal Forest (Wager & Athey 2017) and BART (Chipman et al. 2011) have emerged as two very strong and widely successful methods for CATE estimation. Both can be trivially adapted to the transfer learning task, e.g. by learning each task separately or by learning them jointly with an indicator feature for each task. 2.b. In the GoTV task, the original data is obtained from an RCT. That means that evaluation can be done using a much more powerful approach than simulating potential outcomes: one can instead evaluate the policy risk implied by the CATE estimator, as done in Shalit et al. (2017) using the LaLonde dataset. Overall I think this paper introduces an interesting problem and some valid approaches, but can be significantly improved both in the methods it presents and the experimental setup.