Continual Lifelong Causal Effect Inference with Real World Evidence
Authors: Zhixuan Chu
Stephen Rathbun
Sheng Li
Venue:  (2021)
Keywords: continual learning
incremental learning
causal effect inference
representation learning
treatment effect estimation

Abstract: The era of real world evidence has witnessed an increasing availability of observational data, which much facilitates the development of causal effect inference. Although significant advances have been made to overcome the challenges in causal effect estimation, such as missing counterfactual outcomes and selection bias, they only focus on source-specific and stationary observational data. In this paper, we investigate a new research problem of causal effect inference from incrementally available observational data, and present three new evaluation criteria accordingly, including extensibility, adaptability, and accessibility. We propose a Continual Causal Effect Representation Learning method for estimating causal effect with observational data, which are incrementally available from non-stationary data distributions. Instead of having access to all seen observational data, our method only stores a limited subset of feature representations learned from previous data. Combining the selective and balanced representation learning, feature representation distillation, and feature transformation, our method achieves the continual causal effect estimation for new data without compromising the estimation capability for original data. Extensive experiments demonstrate the significance of continual causal effect inference and the effectiveness of our method.

Under review as a conference paper at ICLR 2021
CONTINUAL LIFELONG CAUSAL EFFECT INFERENCE WITH REAL WORLD EVIDENCE
Anonymous authors Paper under double-blind review
ABSTRACT
The era real world evidence has witnessed an increasing availability of observa- tional data, which much facilitates the development of causal effect inference. Al- though significant advances have been made to overcome the challenges in causal effect estimation, such as missing counterfactual outcomes and selection bias, they only focus on source-specific and stationary observational data. this paper, we investigate new research problem of causal effect inference from incrementally available observational data, and present three new evaluation criteria accordingly, including extensibility, adaptability, and accessibility. We propose Continual Causal Effect Representation Learning method for estimating causal effect with observational data, which are incrementally available from non-stationary data distributions. Instead of having access to all seen observational data, our method only stores limited subset of feature representations learned from previous data. Combining the selective and balanced representation learning, feature represen- tation distillation, and feature transformation, our method achieves the continual causal effect estimation for new data without compromising the estimation ca- pability for original data. Extensive experiments demonstrate the significance of continual causal effect inference and the effectiveness of our method.
INTRODUCTION
Causal effect inference is a critical research topic across many domains, such as statistics, computer science, public policy, and economies. Randomized controlled trials (RCT) are usually considered as the gold-standard for causal effect inference, which randomly assigns participants into a treatment or control group. As the RCT conducted, the only expected difference between the treatment and control groups is the outcome variable being studied. However, in reality, randomized controlled trials are always coneuuinu and expensive, and thus the study cannot involve many subjects, which may be not representative of the real-world population the intervention would eventually tar- get. Nowadays, estimating causal effects from observational data has become an appealing research direction owing to large amount of available data and low budget requirements, compared with RCT (Yao et al.] 2020). Researchers have developed various strategies for causal effect inference with observational data, such as tree-based methods (Chipman al. 2010
Wager & Athey 2018), representation learning methods (Johansson et al., 2016)] Li & Fu 2017] Shalit et 2017 Chu
al.) 2020), adapting Bayesian algorithms (Alaa & van der Schaar 2017). generative adversarial
nets Yoon et al. 2018). variational autoencoders (Louizos et al. (017) and so on.
Although significant advances have been made overcome the challenges in causal effect estima- tion with observational data, such as missing counterfactual outcomes and selection bias between treatment and control groups, the existing methods only focus on source-specific and stationary ob- servational data. Such learning strategies assume that all observational data are already available during the training phase and from the only one source. This assumption is unsubstantial prac- tice due to two reasons. The first one is based on the characteristics of observational data, which are incrementally available from non-stationary data distributions. For instance, the number of elec- tronic medical records in one hospital is growing every day, or the electronic medical records for one disease may be from different hospitals or even different countries. This characteristic implies that cannot have access to all observational data one time point and from single source. The second reason is based on the realistic consideration of accessibility. For example, when the new observational are available, if want refine the model previously trained by original data, maybe

---

Under review as a conference paper at ICLR 2021
the original training data are no longer accessible due to variety of reasons, e.g.. legacy data may be unrecorded, proprietary, too large to store, or subject to privacy constraint (Zhang et al. 2020). This practical concern of accessibility is ubiquitous in various academic and industrial applications. That's what it boiled down to: in the era of big data, we face the new challenges in causal inference with observational data: the extensibility for incrementally available observational data, the adapt- ability for extra domain adaptation problem except for the imbalance between treatment and control groups in one source, and the accessibility for huge amount of data.
Existing causal effect inference methods, however, are unable to deal with the aforementioned new challenges, i.e., extensibilityy adaptability, and accessibility. Although is possible to adapt existing causal inference methods to address the new challenges, these adapted methods still have inevitable defects. Three straightforward adaptation strategies are described as follows. (1) If we directly apply the model previously trained based on original data to new observational data, the performance on new task will be very poor due to the domain shift issues among different data sources; (2) If we utilize newly available data to re-train the previously learned model, adapting changes in the data distribution, old knowledge will be completely or partially overwritten by the new one, which can result in severe performance degradation old tasks. This is the well-known forgetinnii problem (McCloskey & Cohen 1989; French 1999); (3) To overcome the catastrophic forgetting problem, we may rely on the storage of old data and combine the old and new data together, and then re-train the model from scratch. However, this strategy is memory inefficient and time-consuming. and it brings practical concerns such as copyright or privacy issues when storing data for a long time (Samet et al. 2013). Our empirical evaluations in Section 4 demonstrate that any of these three strategies combination with the existing causal effect inference methods is deficient.
To address the above issues, we propose a Continual Causal Effect Representation Learning method (CERL) for estimating causal effect with incrementally available observational data. Instead of hav- ing access to all previous observational data, we only store a limited subset of feature representations learned from previous data. Combining the selective and balanced representation learning, feature representation distillation, and feature transformation, our method preserves the knowledge learned from previous data and update the knowledge by leveraging new data, so that it can achieve the continual causal effect estimation for new data without compromising the estimation capability for previous data. To summarize, our main contributions include:
Our work is the first to introduce the continual lifelong causal effect inference problem for the incrementally available observational data and three corresponding evaluation criteria, i.e., extensibility, adaptability, and accessibility.
We propose a new framework for continual lifelong causal effect inference based on deep representation learning and continual learning.
Extensive experiments demonstrate the deficiency of existing methods when facing the incrementally available observational data and our model's 's outstanding performance.
BACKGROUND AND PROBLEM STATEMENT
Suppose that the observational data contain n units collected from d different domains and the d-th dataset Da contains the data {(x. y, l/ E X, E Y, E T} collected from d-th domain, which contains ny units. Let X denote all observed variables, Y denote the outcomes in the observational data, and T is a binary variable. Let Disd = (D1, D2...., Dd) be the set of combination of d dataset, separately collected from d different domains. For d datasets {D,, D2, ..., Da). they have the common observed variables but due to the fact that they are collected from different domains, they have different distributions with respect to X, Y. and T in each dataset. Each unit in the observational data received one of two treatments. Let l. denote the treatment assignment for unit i; i 1, 1,..., n. For binary treatments, 4 = 1 is for the treatment group, and I, = 0 for the control group. The outcome for unit is denoted by zi when treatment t is applied to unit é; that is, yi is the potential outcome of unit i in the treatment group and y is the potential outcome of unit i in the control group. For observational data, only one of the potential outcomes is observed. The observed outcome called the factual outcome and the remaining unobserved potential outcomes are called counterfactual outcomes.
In this paper, we follow the potential outcome framework for estimating treatment effects (Rubin 1974 Splawa-Neyman et al. 1990). The individual treatment effect (ITE) for unit is the difference
2

---

Under review as a conference paper at ICLR 2021
between the potential treated and control outcomes, and is defined as ITE, vi vo- The average treatment effect (ATE) is the difference between the mean potential treated and control outcomes, which is defined as ATE 1/2 - (vi --).
The success of the potential outcome framework is based on the following assumptions (Imbens & Rubin 2015), which ensure that the treatment effect be identified. Stable Unit Treatment Value Assumption (SUTVA):: The potential outcomes for any units do not vary with the treatments assigned other units, and, for each unit, there are no different forms versions of each treatment level, which lead to different potential outcomes. Consistency: The potential outcome of treatment t equal the observed outcome if the actual treatment received treatment assignment is not deterministic, i.e.,P(TT = x|X = 2) > O, for all t and r. Ignorability: Given covariates, treatment assignment is independent the potential outcomes, i.e., (y1, yo) A/..
Positivity: For any value of x,
The goal of our work is to develop a novel continual causal inference framework, given new available observational data Da, to estimate the causal effect for newly available data Da as well as the previous data D1:(d-1) without having access to previous training data in D1:(d-1).
THE PROPOSED FRAMEWORK
The availability of "real world evidence" is expected to facilitate the development of causal effeet in- ference models for various academic and industrial applications. How to achieve continual learning from incrementally available observational data from non-stationary data domains is new direc- tion in causal effect inference. Rather than only focusing on handling the selection bias problem, we also need take into comprehensive consideration three aspects of the model, i.e., the extensibility for incrementally available observational data, the adaptability for various data sources, and the accessibility for a huge amount of data.
We propose the Continual Causal Effect Representation Learning method (CERL) for estimating causal effect with incrementally available observational data. Based on selective and balanced rep- resentation learning for treatment effect estimation, CERL incorporates feature representation dis- tillation to preserve the knowledge learned from previous observational data. Besides, aiming at adapting the updated model to original and new data without having access to the original data, and solving the selection bias between treatment and control groups, we propose one representation transformation function, which maps partial original feature representations into feature repre- sentation space and makes the global feature representation space balanced with respect to treatment and control groups. Therefore, CERL can achieve the continual causal effect estimation for new data and meanwhile preserve the estimation capability for previous data, without the aid of original data.
3.1 MODEL ARCHITECTRRE
To estimate the incrementally available observational data, the framework of CERL is mainly com- posed of two components: (1) the baseline causal effect learning model is only for the first available observational data, and thus we don't need to consider the domain shift issue among different data sources. This component is equivalent continual causal effect learning model is for the sequentially available observational data, where we need handle more complex issues, such as knowledge transfer, catastrophic forgetting, global rep- resentation balance, and memory constraint. We present the details of each component as follows.
the traditional causal effect estimation problem; (2) the
3.1.1 THE BASELINE CAUSAL RFFECT LEARNING MODEL.
We first describe the baseline causal effect learning model for the initial observational dataset and then bring in subsequent datasets. For causal effect estimation in the initial dataset, it can be trans- formed into the traditional causal effect estimation problem. Motivated by the empirical success of deep representation learning for counterfactual inference (Shalit et al. 2017): Chu et al., (2200), we propose to learn the selective and balanced feature representations for treated and control units, and then infer the potential outcomes based on learned representation space.
Learning Selective and Balanced Representation. Firstly, adopt deep feature selection model that enables variable selection in one deep neural network, i.c., 9wn : X - R, where denotes the original covariate space, R denotes the representation space, and wi are the learnalle parameters in
3

---

Under review as a conference paper at ICLR 2021
function g. The elastic net regularization term Zou & Hastie, 2005) is adopted in model, i.e,, Lw, = w2 + ||wil|2. Elastic net throughout the fully connected representation layers assigns larger weights to important features. This strategy can effectively filter out the irrelevant variables and highlight the important variables.
Due to the selection bias between treatment and control groups and among the sequential different data sources, the magnitudes of confounders may be significantly different. To effectively eliminate magnitudes between treatment and control
the imbalance caused by the significant difference groups and among different data sources, we propose to use cosine normalization in the last repre- sentation layer. In the multi-layer neural networks, we traditionally use dot products between the output vector of the previous layer and the incoming weight vector, and then input the products to the activation function. The result of dot product unbounded. Cosine normalization uses cosine similarity instead of simple dot products in neural networks, which can bound the pre-activation between 1 and 1. The result could be even smaller when the dimension high. As a result, the variance can be controlled within a very narrow range (Luo et al. 2018 Cosine normalization is de- anvar ) where Tnorm is the normalized pre-activation, fined as r = a (Fnorm) = o cos(w, x)) = o |
w is the incoming weight vector, x is the input vector, and a is nonlinear activation function.
adopt integral probability metrics (IPM) when learning the representation space to balance the treatment and control groups The IPM measures the diver- gence between the representation distributions of treatment and control groups, so we want to min- imize the IPM to make two distributions more similar. Let P(g(z)|t denote the empirical distributions of the representation vectors for the treatment and control groups, respectively. We adopt the IPM defined in the family of 1-Lipschitz functions, which leads to IPM being the Wasserstein distance (Sriperumbudur et al. 2012, Shalit al.) 2017). In partieu- lar, the IPM term with Wasserstein distance is defined as Wass(P,
Motivated by Shalit al. (017),
1) and Q(g(=)]t
0)
inf gg(z) [[((()))
P(g(s))d(g(z)) where 7 denotes the hyper--paameete controlling the trade-off between Wass P. Q) and other terms in the final objective function. K = (*|Q(((((x)) = P(g(=)) de- fines the set of push-forward functions that transform the representation distribution of the treatment distribution P to that of the control Q and g(z) E {9(x).
Inferring Potential Outcomes. We aim to learn a function hg R X + Y that maps the rep- resentation vectors and treatment assignment to the corresponding observed outcomes, and it can be parameterized by deep neural networks. To overcome the risk of losing the influence of T on R. he, (gwe, (æ), t) is partitioned into two separate tasks for treatment and control groups, respec- tively. Each unit is only updated in the task corresponding to its observed treatment. Let ý ho, (Dwn, (Ir), I) denote the inferred observed outcome of unit i corresponding to factual treatment 4- We minimize the mean squared error in predicting factual outcomes: Ly-
(04-0
Putting all the above together, the objective function of our baseline causal effect learning model is: L = Ly + aW (P,((P aWasss(P Q) + XLoyy where and İ denote the hyper-parameters controlling the trade-off among W Wass(P. P, Q). Lw. and Ly in the objective function.
3.1.2 THE SUSTAINABILITY OF MODEL LEARNING
By far, we have built the baseline model for causal effect estimation with observational data from a single source. To avoid catastrophic forgetting when learning new data, we propose to preserve a subset of lower-dimensional feature representations rather than all original covariates. We also can adjust the number of preserved feature representations according to the memory constraint.
After the completion of baseline model training. we store subset of feature representations Ri (9w, (1:]æ: E D1) and the corresponding (Y,7) E D1 as memory M1. The size of stored representation vectors can be reduced to satisfy the pre-specified memory constraint by a herding algorithm (Welling 2009 Rebuffi al. 2017). The herding algorithm can create representative set of samples from distribution and requires fewer samples to achieve a high approximation quality than random subsampling. We run the herding algorithm separately for treatment and control groups to store the same number of feature representations from treatment and control groups. At this point, we only store the memory set Mj and model gwy; without the original data (D1).
4

---

Under review as a conference paper at ICLR 2021
3.1.3 THE CONTINUAL CAUSAL EFFECT LEARNING MODEL
For now, we have stored memory M1 and baseline model. To continually estimate the causal effect for incrementally available observational data, we incorporate feature representation distillation and feature representation transformation to estimate causal effect for all seen data based on balanced global feature representation space. The framework of CERL is shown in Fig.
Feature Representation Distillation. For next available dataset D2 (a,,,,))z E X, e Y, E Т) collected from second domain, we adopt the same selective representation learning Gwg X - x>
R2 with elastic net regularization (Treg) on new parameters W2- Because we expect model can estimate causal effect for both previous and new data, we want the new model to inherit some knowledge from previous model. In continual learning, knowledge distillation (Hin- [on et al. 2015: Li & Hoiem 2017) is commonly adopted to alleviate the catastrophic forgetting, where knowledge is transferred from one network to another network by encouraging the outputs of the original and new network to be similar. However, for the continual causal effect estima- tion problem, we focus more on the feature representations, which are required to be balanced between treatment and control, and among different data domains. Inspired by |Hou al. (2019); Dhar et al. (201);; Iscen et al. (2020). we propose feature representation distillation to encourage the representation vector {9w. (x){æ
D2} based on baseline model to be similar to the repre- sentation vector {>))<>) E D2} based on new model by Euclidean distance. This feature dis- tillation can help prevent the learned representations from drifting too much in the new feature representation space. Because we apply the cosine normalization to feature representations and B)). the feature
||A 2|| (A (A B) ||A|1 + ||B|| 2AT 2(1 representation distillation is defined (F) cos (gwi (I), Gwo (I)), where x E D2.
14
Minday
Monding
X
Av
R₂ T
lis.
Rg
YL.1g
T
7PM
IPM
Figure 1: The blue part is baseline causal effect learning model for the first observational data. After baseline model training, store a subset of feature representations R₁ into M₁ by herding algorithm. The green part helps to map RS to transformed feature representations RS compatible with new feature representations space R2- Then the red part is used for continual causal effect estimation based on feature distillation and balanced global feature representation learning for R1 and R2.
Feature Representation Transformation. We have previous feature representations R₁ stored in Mj and new feature representations R2 extracted from newly available data. R1 and R2 lie in different feature representation space and they are not compatible with each other because they are learned from different models. In addition, we cannot learn the feature representations of previous data from the new model Iwz, as we no longer have access to previous data. Therefore, to balance the global feature representation space including previous and new representations between treatment and control groups, a feature transformation function is needed from previous feature representations R1 to transformed feature representations R1 compatible with new feature representations space Rg. We define a feature transformation function as Φ1-22 Rr - R1. We also input the feature representations of new data D2 learned from old model, i.c., 9w, (z). to get the transformed feature representations of new data, i.e., @1-+2(9w, (z)). To keep the transformed space compatible with the new feature representation space, we train the transformation function Φ1-22 by making the ( (x)) and Gun. (I) similar, where x E D2. The loss function is defined as LFr(*) = 1 cos (01= (z)), Iw., (z)), which is used to train the function 61-22 to transform feature representations between different feature spaces. Then, we can attain the transformed old feature representations R Φ1-22 RR), which is in the same space R2-
Balancing Global Feature Representation Space. We have obtained a global feature representa- tion space including the transformed representations of stored old data and new representations of new available data. We adopt the same integral probability metrics as baseline model to make sure that the representation distributions are balanced for treatment and control groups in the global fea-
5

---

Under review as a conference paper at ICLR 2021
ture representation space. In addition, we define potential outcome function ha, R2) R2)xT
Y. Let 1j hag (0)->2(1;);))) 1), where ri E and o ho, (Iw, (rz). 1), where 2;j E D2 denote the inferred observed outcomes. We aim to minimize the mean squared error predicting factual outcomes for global feature representations including transformed old feature representations and new feature representations: LG R. -0 y. )2 + R2 2ー=1 (0) v,?), where n, is the number of units stored in M1 by herding algorithm, y E M1, and vn E D22
In summary, the objective function of our continual causal effect learning model is L La + a (P.s(P..C) () + ALw; + 3LFD + 5LFT. where a, X, B, and denote the paper-erreeters con- trolling the trade-off among W aasss(P P. Q), Tuwg LED, LFT, and Lg in the final objective function.
3.2 OVERVIEW OF CERL
In the above sections, we have provided the baseline and continual causal effect learning models. When the continual causal effect learning model for the second data is trained, we can extract the R2 = (gw, (z)lx= E D2} and R1 ( E M1}. We define a new memory set as M2 {R2. Y2, T2) U (M1), where 61-2((M) includes R₁ and the corresponding (Y, T) stored in Mj. Similarly, to satisfy the pre-specified memory constraint, M2 can be reduced by conducting the herding algorithm to store the same number of feature representations from treatment and control groups. We only store the new memory set M2 and new model Jwg. which are used to train the following model and balance the global feature representation space. It is unnecessary to store the original data (D1 and D2) any longer.
We follow the same procedure for the subsequently available observational data. When we obtain the new observational data Da. we can train (9wz) and Φ6-1-ed Ry-1 - Ry-1 based on the continual causal effect learning model. Besides, the new memory set is defined as: Ma {Ra, Yd, Ta} U (Pu-1-a(Ma-Jl) So far, our model (9w()w)) can estimate causal effect for all seen observational data regardless of the data source and it doesn't 't require access to previous data. The detailed procedures of our CERL method are summarized in Algorithm in Section B of Appendix.
4 EXPERIMENTS
We adapt the traditional benchmarks, i.e., News (Johansson et al., 2016 Schwab et al. 2018 and BlogCatalog (Guo et al. 2020) to continual causal effect estimation. Specifically, we consider three scenarios to represent the different degrees of domain shifts among the incrementally available ob- servational data, including the substantial shift, moderate shift, and no shift. Besides, we generate series of synthetic datasets and also conduct ablation studies to demonstrate the effectiveness of our model on multiple sequential datasets. The model performance with different numbers of preserved feature representations, and the robustness to hyperparameters are also evaluated.
4.1 DATASET DESCRIPTION
We utilize two semi-synthetic benchmarks for the task of continual causal effect estimation, which are based on real-world features, synthesized treatments and outcomes.
News, The News dataset consists of 5000 randomly sampled news articles from the NY Times c It simulates the opinions of media consumers on news items. The units are different news items represented by word counts x; NY and outcome (xx;) E R the news item. The intervention & E {0, 1} represents the viewing device, desktop (t = 0) or mobile (t = 1). We extend the original dataset specification in Johansson et al.) (2016); Schwab et al. (018)) to enable the simulation of incrementally available observational data with different degrees of domain shifts. Assuming consumers prefer to read certain media items on specific viewing devices, we train a topic model on large set of documents and define 2(x) the topic distribution of news item x. We define one topic distribution of a randomly sampled document as centroid of for mobile and the average topic representation of all document as centroid =$ for desktop. Therefore, the reader's 's opinion of news item x on device is determined by the similarity between 2(x) and 2f. ie., v(z;) = Cf + t, =(x) 2f) + c, where C 60 is scaling factor and N(0,1). 1).
https://archiveg iss uci edu /ml / /tt///aaaasstttt htt.:////////aaaaae//////rrrrrre

---

Under review as a conference paper at ICLR 2021
Table I: Performance on two sequential data and M=500. We present the mean value of v VEPEHE and EATE on test sets from two datasets. The standard deviations are tiny. Lower is better. Under no domain shift scenario, the three strategies and CERL have the similar performance, because the previous and new data are from the same distribution. Under substantial shift and moderate shift scenarios, CFR-A performs well on previous data, but significantly declines on new dataset; straxtegy CFR-B shows the catastrophic forgetting problem; CERL has similar performance to strategy CFR-C, while CERL does not require access to previous data. Besides, the larger domain shift leads worse performance of CFR-A and CFR-B. CERL has remained stable against shift.
News
BlogCatalg
Previous data
New data
Previous data
New data
Strategy
v /CPEHE
<ATE
/EPEHE
CATE
/EPEHE
CATE
/CEEHE
EATE
Substantial
CFR-A
2.49
0.80
3.62
1.18
9.92
4.25
13.65
6.21
shift
CFR-B
3.23
1.06
t
2.71
0.91
14.21
6.98
t
9.77
4.11
CFR-C
2.51
0.82
2.70
0.92
9.93
4.24
9.77
4.12
CERL
2.55
0.84
2.71
0.91
9.96
4.25
9.78
4.12
Moderate shift
CFR-A
2.58
0.85
3.06
1.02
9.89
4.22
11.26
5.03
CFR-B
2.98
0.99
t
2.65
0.92
12.35
5.67
9.83
4.18
CFR-C
2.56
0.85
2.63
0.90
9.88
4.21
9.81
4.16
CERL
2.59
0.86
2.66
0.92
9.90
4.24
9.82
4.17
No shift
CFR-A
2.58
0.87
2.62
0,88
9.86
4.20
9.85
4.19
CFR-B
2.60
0.88
2.60
0.87
9.85
4.18
9.83
4.18
CFR-C
2.58
0.87
2.59
0.87
9.84
4.18
9.83
4.18
CERL
2.59
0.87
2.60
0.87
9.85
4.19
9.83
4.18
Besides, the intervention defined by = 1[a)) (. 88 ( where 10 indicates
an expected selection bias. In the experiments, 50 LDA topics are learned from the training corpus and 3477 bag-of-words features are the dataset. To generate two sequential datasets with different domain shifts, we combine the news items belonging to LDA topics from 1 to 25 into first dataset and the news items belonging to LDA topics from 26 to 50 into second dataset. There is no overlap of the LDA topics between the first dataset and second dataset, which is considered as substantial domain shift. In addition, the news items belonging to LDA topics from to 35 and items belonging to from 16 to 50 used to construct the first dataset and second dataset, respectively, which is regarded as moderate domain shift. Finally, randomly sampled items from 50 LDA topics compose the first and second dataset, resulting in no domain shift,, because they are from the same distribution. Under each domain shift scenario and each dataset, we randomly sample 60% and 20% of the units as the training set and validation set and let the remaining be the test set.
BlogCatalog, BlogCatalog (Guo et al. 2020) is a blog directory that manages the bloggers and their blogs. In this semi-synthetic dataset, each unit is blogger and the features are bag-of-words representations of keywords in bloggers' descriptions collected from real-world source. We adopt the same settings and assumptions to simulate the treatment options and outcomes as we do for the News dataset. 50 LDA topics are learned from the training corpus. 5196 units and 2160 bag-of- words features are in the dataset. Similar to the generation procedure of News datasets with domain shifts, we create two datasets for each of the three domain shift scenarios. Under each domain shift scenario and each dataset, we randomly sample 60% and 20% of the units as the training set and validation set and let the remaining be the test set.
4.2 RESULTS AND ANALYSIS
Evaluation Metrics.
We adopt two commonly used evaluation metrics. The first one is the error of ATE estimation, which is defined as EATE |ATE- ATE|, where ATE is the true value and ATE is an estimated ATE, The second one is the error of expected precision in estimation of heterogencous effect (PEHE) Hill (201 1), which is defined as εPEнE: Σn_ (ITE, ITE,)², where ITE, is the true ITE for unit é and ITE, is an estimated ITE for unit i.
We employ three strategies to adapt traditional causal effect estimation models to incrementally available observational data: (A) directly apply the model previously trained based on original data to new observational data; (B) utilize newly available data to fine-tune the previously learned model;
7

---

Under review as a conference paper at ICLR 2021
Instrumental 2
Confonders C
Adjustment: A
Data 1
Data 2
Data
Data
Data
Train
Model
Model 2
Model 3
Model 4
Model 5
Evaluate
Inolevant
Trestment
Outcome
T
Data 1
Data
Data
Data
Data 5
Figure 2: The types of variables.
Figure 3: The work flow of task.
Table Performance on two sequential data and M 10000. We present the mean value of /CPEHR and ATE on test sets from two datasets. The standard deviations are tiny. Lower is better.
Previous data
New data
Strategy
FPENE
<TTE
/FBBEE
CFR-A
1.47
0.35
2.51
0.73
CFR-B
1.82
0.47
t
1.63
0.45
CFR-C
0.36
1.62
CERL
1.49
0.37
1.63
0.44
CERL (w/o FRT)
1.71
0.43
f
1.63
0.44
CHRL (w/o herding)
1.57
0.40
1
1.63
0.44
CERL (w/o cosine norm)
0.38
7
1.65
(C) store all previous data and combine with new data to re-train the model from scratch. Among these three strategies, (C) is expected be the best performer and get the ideal performance with respect to ATE and PEHE, although it needs to take up the most resources (all the data from previous and new dataset). We implement the three strategies based on the counterfactual regression model (CFR) (hallit et al. 2017). which representative causal effect estimation method.
As shown Table [ under no domain shift scenario, the three strategies and our model have the similar performance on the News and BlogCatalog datasets, because the previous and new data from the same distribution. CFR-A, CFR-B, and CERL need less resources than CFR-C. Under substantial shift and moderate shift scenarios, find strategy CFR-A performs well on previous data, but significantly declines on new dataset; strategy CFR-B shows the catastrophic forgetting problem where the performance on previous dataset is poor; strategy CFR-C performs well on both previous and new data, but re-trains the whole model using both previous and new data. However, there a memory constraint barrier accessing previous data, the strategy CFR-C cannot be conducted. Our CERL has similar performance to strategy CFR-C, while CERL does not require access to previous data. Besides, by comparing the performance under substantial and moderate shift scenarios, the larger domain shift leads to worse performance of CFR-A and CFR-B. However, no matter what the domain shift is, the performance of our model CERL consistent with ideal strategy CFR-C.
4.3 MODEL EVALUATION
Synthetic Dataset.
Our synthetic data include confounders, instrumental, adjustment, and irrele- vant variables. The interrelations among these variables, treatments, and outcomes are illustrated in Figure 2 We totally simulate five different data sources with five different multivariate normal distributions represent the incrementally available observational data. In each data source, we ran- domly draw 10000 samples including treatment units and control units. Therefore, for five datasets, they have different selection bias, magnitude covariates, covariance matrices for variables, and number of treatment and control units. To ensure robust estimation of model performance, for each data source, we repeat the simulation procedure 10 times and obtain 10 synthetic datasets. The details of data simulation provided in Section of Appendix.
Results. Similar to the experiments for News and BlogCatalog benchmarks, we still utilize two sequential datasets to compare model with CFR under three strategies the more complex
8

---

Under review as a conference paper at ICLR 2021
synthetic data. As shown in Table 2 the result is consistent with the conclusions News and Blog- Catalog. Our model's 's performance demonstrates its superiority over CFR-A and CFR-B. CERL is comparable with CFR-C, while it does not need to have access to the raw data from previous dataset. Besides, we also conduct three ablation studies to test the effectiveness of the important components in CERL, i.e., CERL (w/o FRT), CERL (w/o herding), and CERL (w/o cosine norm). CERL (w/ FRT) is the simplified CERL without the feature representation transformation, which is based on traditional continual learning with knowledge distillation and integral probability metrics, In CERL (w/o FRT), we do not store and transform the previous feature representation into new feature space, and only utilize the knowledge distillation realize the continual learning task and balance the bias between treatment and control groups with each new data. CERL (w/o herding) adopts random subsampling strategy to select samples into memory, instead of herding algorithm. CERL (w/o co- sine norm) removes the cosine normalization in the last representation layer. Table P the performance becomes poor after removing anyone in the feature representation transformation, herding, cosine normalization modules compared to the original CERL. More specifically, after removing the feature representation transformation, VCPENE and CATE increase dramatically, which demonstrates that the knowledge distillation always used in continual learning task is not enough for the continual causal effect estimation. Also, using herding to select a representative set of samples from treatment and control distributions is crucial for the feature representation transformation.
shows that
CERL. Performance Evaluation. As illustrated in Figure 3 the five observational data are incre- mentally available in sequence, and the model will continue to estimate the causal effect without having access to previous data. We further evaluate the performance of CERL from three perspec- tives, i.e., the impact of memory constraint, effeteness of cosine normalization, and its robustness to hyper-parameters
As shown in Figure 4 (a) and (b), as the model continually learns a new dataset, every time when finishing training one new dataset, we report the ViPERE and CATE on test sets composed of previous data and new data. Our model with memory constraints has a simi- lar performance to the ideal situation, where all data are available to train the model from scratch. However, our model can effectively save memory space, c.g., when facing the fifth dataset, our model only stores 1000, 5000, or 10000 feature representations, but the ideal situation needs to store 5 x 10000 50000 observations with all covariates. For the cosine normalization, we perform an ablation study of CERL (M=5000, 5 datasets), where we remove cosine normalization in the repre- sentation learning procedure. We find the /CPEHE increases from 1.80 and 1.92 and CATE from 0.55 to 0.61. Next, we explore the model's sensitivity to the most important parameter a and 6, which controls the representation balance and representation transformation. From Fig. (c) and (d), we observe that the performance is stable over a large parameter range. In addition, the parameter / for feature representation distillation is set (Rebuffi et al.) 2017: Iscen et al.) 2020).
Staring
Storing
10000
10000
36-
Mi-0000
5000
Ears
M-1000
1000
15
1.4
Dataset
Dataset
Figure 4: Performance of CERL under different settings.
5 CONCLUSION
It is the first time to propose the continual lifelong causal effect inference problem and the cor- responding evaluation criteria. As the real world evidence is becoming more prominent, how to integrate and utilize these powerful data for causal effect estimation becomes new research chal- lenge. To address this challenge, we propose the Continual Causal Effect Representation Learning method for estimating causal effect with observational data, which are incrementally available from non-stationary data distributions. Extensive experiments demonstrate the superiority of our method over baselines for continual causal effect estimation.
9

---

Under review as a conference paper at ICLR 2021
REFERENCES
Ahmed M Alaa and Mihaela van der Schaar. Bayesian inference of individualized treatment effects using multi-task gaussian processes. In Advances Neural Information Processing Systems, pp. 2017.
Hugh A Chipman, Edward George. and Robert E McCulloch. Bart: Bayesian additive regression trees. The Annals of Applied Statistics, (1):266-2998 2010.
Zhixuan Chu, Stephen L Rathbun, and Sheng Li. Matching in selective and balanced representation space for treatment effects estimation. arXiv preprint arXiv:2009.0660 06828, 2020.
Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa. Learning without memorizing. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 5138-5146, 2019.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135, 1999.
Ruocheng Guo, Jundong Li, and Huan Liu. Learning individual causal effects from networked observational data. In Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 232-240, 2020.
Johanna Hardin, Stephan Ramon Garcia, and David Golan. A method for generating realistic corre- lation matrices. The Annals of Applied Statistics, pp. 1733-1762, 2013.
Jennifer L Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):217-240, 2011.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint arXiv:1503.02533 2015.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 83 831-839, 2019.
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sci- ences. Cambridge University Press, 2015.
Ahmet Iscen, Jeffrey Zhang, Svetlana Lazebnik, and Cordelia Schmid. Memory-efficient incremen- tal learning through feature adaptation. arXiv preprint arXiv:2004.00713 2020.
Daniel Jacob, Wolfgang Karl Härdle, and Stefan Lessmann. Group average treatment effects for observational studies. arXiv preprint weI.vv1911..020 2019.
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual infer- ence. In International conference on machine learning. pp. 0020-3029, 2016.
Peter Langfelder, Bin Zhang, and Steve Horvath. Defining clusters from hierarchical cluster tree: the dynamic tree cut package for Bioinformatics, 24(5):71 2008
Sheng Li and Yun Fu. Matching on balanced nonlinear representations for treatment effects estima- tion. In Advances in Neural Information Processing Systems, pp. 929-939, 2017.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions pattern analysis and machine intelligence, 40(12):2935-290) 2017.
Christos Louizos, Uri Shalit, Joris M Mooij. David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. In Advances Neural Information Processing Systems, pp. 6446-6456, 2017.
Chunjie Luo, Jianfeng Zhan, Xiaohe Xue. Lei Wang, Rui Ren, and Qiang Yang. Cosine normaliza- tion: Using cosine similarity instead of dot product in neural networks. International Confer- on Artificial Neural Networks, pp. 382-391. Springer, 2018.
10

---

Under review as a conference paper at ICLR 2021
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Peter M Robinson. Root-n-consistent semiparametric regression. Econometrica: Journal of the Econometric Society, pp. 931-954, 1988.
Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies.
Journal of educational Psychology, 66(5):688, 1974.
Saced Samet, Ali Miri, and Eric Granger. Incremental learning of privacy-preserving bayesian networks. Applied Soft Computing. 1365)33657-3667 2013.
Patrick Schwab, Lorenz Linhardt, and Walter Karlen. Perfect match: simple method for learning representations for counterfactual inference with neural networks. arXiv preprint arXiv:/810.00655 2018.
Uri Shalit, Fredrik Johansson, and David Sontag. Estimating individual treatment effect: gener- alization bounds and algorithms. In International Conference on Machine Learning, pp. 3076- 3085. PMLR, 2017.
Jerzy Splawa-Neyman, Dorota M Dabrowska, and TP Speed. On the application of probability theory to agricultural experiments. essay on principles. section 9. Statistical Science, pp. 465- 472. 1990.
Bharath K Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf, Gert RG Lanck- riet, et al. On the empirical estimation of integral probability metrics. Electronic Journal of Statistics, 6:1550- 1590, 2012.
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 1228622) 2018. doi: 10. 002.0080001621449200..11111
URL https: //ois wrg/10...100o 1ttp:/11...00001112111
2017.1319639
Max Welling, Herding dynamical weights to learn. In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1121-1128, 2009.
Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. A survey on causal inference. arXiv preprint arXiv:2002.027700 2020.
Jinsung Yoon, James Jordon, and Mihaela der Schaar. Ganite: Estimation of individualized treatment effects using generative adversarial nets. In International Conference on Learning Rep- resentations, 2018
Bin Zhang and Steve Horvath. A general framework for weighted gene co-expression network analysis. Statistical applications in genetics and molecular biology, 4(1), 2005.
Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, and C-C Jay Kuo. Class-incremental learning via deep model consolidation. In The IEEE Winter Conference on Applications of Computer Vision, pp. 1131-1140, 2020.
Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology), 67(2):301-320 2005.
11

---

Under review as a conference paper at ICLR 2021
A SIMULATION PROCEDURE
Our synthetic data include confounders, instrumental, adjustment, and irrelevant variables. The in- terrelations among these variables, treatments, and outcomes are illustrated in Figure The number of observed variables in the vector X = (CT, ZT, IT, AT) is set to 100, including 35 confounders in C. 35 adjustment variables in A, 10 instrumental variables
Z, and 20 irrelevant variables in I. The model used to generate the continuous outcome variable Y in this simulation is the partially linear regression model, extending the ideas described in Robinson (1988); Jacob et al. (2019): Chu et al. 20200)
((CTT(,ATTTII AT))
where are unobserved covariates, which follow a standard normal distribution (0, 1) and EldC, A,T] = O. T ind. (ernuuli(e((CT.22T)) and eo((CT, (T)T is the propensity score, which represents the treatment selection bias based on their own confounders C' and instrumental variables Z. Because we aim to simulate multiple data sources {D,: = 1, D). the vector of all observed covariates X (CT, ZT, 1T, AT)T sampled from different multivariate normal dis- tribution with mean vector MC: 12 1T, and v9x and different random positive definite covariance matrices Ed
For each data source, except for the different magnitude of mean vector and structure of covariance matrix, the simulation procedure is the same. Let D be the diagonal matrix with the square roots of the diagonal entries of on its diagonal, i.e., D = v /iog(o)) then the correlation matrix is given as:
R = Dー
(2)
We use algorithm in Hardin et al. (2013) to simulate positive definite correlation matrices consist- ing of different types of variables. Our correlation matrices are based on the hub correlation structure which has a known correlation between hub variable and each of the remaining variables (Zhang & Horvath 2005 Langfelder et al., 2008). Each variable in one type of variables is correlated the hub-variable with decreasing strength from specified maximum correlation to minimum correlation, and different types of variables are generated independently or with weaker correlation among vari- able types. Defining the first variable as the hub, for the ith variable (i = 2, 3, ..., n),, the correlation between it and the hub-variable in one type of variables is given as:
i 2 1 = Pmax
1
(Pmax - Pmin);
(3)
d 2
where Pmax and Pmin are specified maximum and minimum correlations, and the rate y controls rate at which correlations decay.
After specifying the relationship between the hub variable and the remaining variables in the same type of variables, we use Toeplitz structure to fill out the remainder of the hub correlation matrix and get the hub-Toeplitz correlation matrix Reype for other type of variables. Here, R the n matrix having the blocks Rz, Rc, RA. and R, along the diagonal and zeros at off-diagonal elements. This yields a correlation matrix with nonzero correlations within the same type and zero correlation among other types. The amount of correlations among types which can be added to the positive- definite correlation matrix Ri is determined by its smallest cigenvalue.
The function 7 r((CT, .T))T) describes the true treatment effect function of the values of adjust- ment variables A and confounders C; namely ([(CCT,AT)) = (sin ((CT, x 62))² where b, represents weights for every covariate in the function, which is generated by uniform(0, 1). The variable treatment effect implies that its strength differs among the units and is therefore condi- tioned on C' and A. The function g((CT, AT)T) can have influence on outcome regardless of treatment assignment. It is calculated via a trigonometric function to make the covariates non- linear, which is defined as (((CT, AT)T) (cos ((CT, AT)T x by))2. Here, by represents a weight for each covariate in this function, which is generated by uniform (0, 1). The bias is attributed to unobserved covariates which follow a random normal distribution (O, 1). The treatment assign-
ment T' follows the Bernoulli distribution, i.e., T ind. Bernolli(eo((CT, ZT)T)) with probability
12

---

Under review as a conference paper at ICLR 2021
(0 ( where en (C IT) represents the propensity score, which is the Co((CT, ZT)T)
cumulative distribution function for standard normal random variable based confounders C' and instrumental variables Z, i.c., a = sin ((CT, ZTT x ba), where b, is generated by uniform (0, 1).
We totally simulate five different data sources with five different multivariate normal distributions to represent the incrementally available observational data. In each data source, we randomly draw 10000 samples including treatment units and control units. Therefore, for five datasets, they have different selection bias, magnitude of covariates, covariance matrices for variables, and number of treatment and control units. To ensure a robust estimation of model performance, for each data source, repeat the simulation procedure 10 times and obtain 10 synthetic datasets.
B ALGORITHM 1
Algorithm Continual Causal Effect Representation Learning
Data: Given incrementally available observational data from D₁ Da
if {z, U, t E Dj then
*** Train baseline causal effect model he, (Dwn ) ***
W1, 01 OPTIMIZE Ly + oW aWass (a, Q) + XLwa
R1 (9w, (x) Di}
M1 = HERDING R1 Ti]
else
for {ar, v, E D2, Da do
*** Train continual causal effect model ho, (gma) #**
wa, 0a P5-1-d OPTIMIZE(LG + aW aasssssP..) (s, Q) + NWwa + 3LLFD + ÖLFT)
Ra-1 (d-1-a(
Rd = (r)l E Dal
M3 = HERDING ( Ya. Ta) U (R Va- E Ma-1, To-1 Ma-1))
end
end
13


Reviews:
[{"review_id": "YeMiIpcp4Wo", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose to estimate individualized treatment effects in a setting they describe as continual learning, batches of data becoming available over time. The technical proposal is to store a feature representation of the presently available data to be updated with new data in the future -- a technique that, together with other regularization methods for selection bias and variable selection, is shown to perform well on synthetic experiments. \n\nCausal inference is increasingly relevant as machine learning assists decision-making, and an important aspect of this process is to handle information streams over time and update accordingly, just as humans would be expected to. In the context of causality though, I believe this requires a more detailed motivation. Causal effects by their very definition exhibit invariance to interventions on observed variables, and if properly estimated, these should not vary between well-behaved environments. With enough data a priori there should not be large changes in estimation in new environments. This may certainly change if the underlying causal mechanisms change and is a problem that has been studied as data fusion, yet this line of research is not mentioned in the paper.\n\nWriting needs to improve substantially. The use of the words extensibility, adaptability, and accessibility is grammatically incorrect, and the context in which they are used does not clarify the meaning of these ideas either. As I understand it, adaptability refers to domain adaptation yet how domains differ, how to merge them depending on their differences, or even an investigation of these problems is not discussed. Accessibility refers to the amounts of data one needs to deal with. This does not strike me as a problem in causal inference where datasets are typically small. Perhaps examples of applications where each of these aspects is a concern would be helpful.\n\nMore importantly, the notation is counter-intuitive and inconsistent. For instance, \\mathcal X is defined as the set of all observed variables, but then a specific realization x is written to be an element of \\mathcal X (as if \\mathcal X was a measurable space of possible realizations of a random variable). Both interpretations cannot be correct. The notation \\mathcal X_d is similarly ambiguous, does this mean each domain has different sets of variables or that the spaces in which they are defined differ? \n\nThe experiments are underwhelming and certainly do not back up the claims made in the introduction. Most benchmarks for individualized treatment effects are semi-synthetic (it would be advisable to stick to these data generating processes while modifying them to highlight specific features of your problem). Many more competing algorithms could be evaluated.", "extra_scores": {}, "extra_text": {}}, {"review_id": "NZZBckgfEwe", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The work adopts a continual lifelong learning paradigm for causal effect estimation. In five synthetic datasets, the proposed method achieves similar results to CFR-C, a strong baseline which requires to store all data and needs to be trained from scratch.\n\nI have some concerns about the motivation of this work. A simple question about the claim on the original data: why does the performance of the model trained continually on the original data matter? It is not difficult to save a checkpoint of the model trained on original data and use it for the original data whenever you want, given the fact the neural network based causal inference models are small. Or you can let the model do the inference and save the results periodically. Similarly, I could not get why “catastrophic forgetting” is an issue in this problem. If storing the original data would cause a copyright or privacy issue, I wonder why storing a model or feature representations trained by such data can avoid these issues. The memory issue of storing data is also not likely to happen in real-world since collecting data is often more complicated and expensive than storing them.\n\nIn section 2, there is an issue in the strong ignorability assumption. It would be great if the authors can clarify. The notation \\mathcal{X}_d assumes that the feature space can change over domains, but the ignorability needs a unified feature space. So, the ignorability assumption actually relies on varying observed covariates space when $d$ changes.\n\nRegarding Table 1, could the authors report the exact setting for the CERL model which achieves the reported numbers? For example, the number of stored feature representations. This is because in  Figure 4, it shows M=10000 has higher PEHE and \\epsilon_ATE than the one reported in Table 1.\n\nSince training from scratch with all data is expensive, it would be interesting to show a comparison of storage space, RAM memory and runtime between CERL and CFR-A, B and C.\n\n\n\n\n\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "8F-ugub_lDS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThis paper considers adopting continual learning on the problem of causal effect estimation. The paper combines methods and algorithms for storing feature representation and representative samples (herding algorithm), avoiding drifting feature representation when new data is learned (feature representation distillation), balanced representation by regularization, etc. Consequently, the paper presents a system that makes use of existing methods as a loss function (the sum of losses and regularization terms). \n\nIt is difficult to observe the novelty of the method---there is no apparent challenge arose in combining these methods as they can just be simply added. Further, the development of this framework for causal inference is mostly orthogonal to the problem of causal effect estimation (and the use of regularization does not add any novelty.) In addition to the lack of novelty, the paper has many non-negligible mistakes in describing causal inference. For example, consider a phrase \"selection bias\" between \"treatment and control groups\" in the first sentence of the second paragraph. \"Treatment and control groups\" usually are used to describe two groups under the context of RCT. Further, the bias we are talking in the paper is a \"confounding bias\" not a \"selection bias\". In Section 2, \"Each unit ... received ... one of ... treatments\" is also relevant to RCT not observational data. There are many other places the authors mentioning \"selection bias\".\n\nQuestion:\nWhat is \"Real world evidence\"?, which is used in the title, abstract, conclusion, and at the beginning of section 3? Is it just observational data?", "extra_scores": {}, "extra_text": {}}, {"review_id": "1j9Tuxyd_cf", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I think the problem proposed in the paper is interesting and the method is moderately novel. I did not find the paper's writing very helpful. The method seems reasonably motivated but the discussion around it does not provide enough intuition about the influence of the different pieces on the final prediction, and no ablation experiment are provided.\n\nMy main issue with the paper is the limited experimental evidence.\n\n1. Without any real data  (at least semi-synthetic) experiments, I do not think there is a good way to evaluate the significance of the problem and the proposed method.\n\n2. The single synthetic experiment in the paper is  not sufficient to evaluate the effectiveness of the method. I believe at a minimum, different synthetic experiments with different structural models, dataset shifts and rate of shift over time should be considered.\n\n3. No ablation studies are done and no real baseline is considered. For example, a non-trivial baseline would be to do use a traditional continual learning objective with distillation with a new treatment-control population balancing objective with each new dataset. The value of the proposed method's representation-herding would then become clear.\n\n\nIf the authors address these issues, I'm happy to change my score.\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors consider the problem of causal inference from multiple conditionally ignorable models that yield different observed data distributions.  This problem is distinct from transportability (which assumes some types of causal invariance across domains, and aims to move causal conclusioned learned in one context to another using this invariance).  The authors adapt a machine learning approach from (Shalit et al, 2017).

Because the authors describe an algorithm rather than a model, it was a bit difficult to understand what assumptions tie the different observed data distributions together (I am guessing there is a way to formulate a 'global model' tying all datasets together in terms of the algorithm hyperparameters but the authors do not discuss this).

The authors evaluate their method via a simulation study.  Moreover, in response to reviewer criticism, the authors uploaded additional results from semi-synthetic data.

Some of the concerns of reviewers were about novelty and scope of evaluation (in addition, some complained about writing and notation).

Decision: Reject