Continual Lifelong Causal Effect Inference with Real World Evidence

1 INTRODUCTION . Causal effect inference is a critical research topic across many domains , such as statistics , computer science , public policy , and economics . Randomized controlled trials ( RCT ) are usually considered as the gold-standard for causal effect inference , which randomly assigns participants into a treatment or control group . As the RCT is conducted , the only expected difference between the treatment and control groups is the outcome variable being studied . However , in reality , randomized controlled trials are always time-consuming and expensive , and thus the study can not involve many subjects , which may be not representative of the real-world population the intervention would eventually target . Nowadays , estimating causal effects from observational data has become an appealing research direction owing to a large amount of available data and low budget requirements , compared with RCT ( Yao et al. , 2020 ) . Researchers have developed various strategies for causal effect inference with observational data , such as tree-based methods ( Chipman et al. , 2010 ; Wager & Athey , 2018 ) , representation learning methods ( Johansson et al. , 2016 ; Li & Fu , 2017 ; Shalit et al. , 2017 ; Chu et al. , 2020 ) , adapting Bayesian algorithms ( Alaa & van der Schaar , 2017 ) , generative adversarial nets ( Yoon et al. , 2018 ) , variational autoencoders ( Louizos et al. , 2017 ) and so on . Although significant advances have been made to overcome the challenges in causal effect estimation with observational data , such as missing counterfactual outcomes and selection bias between treatment and control groups , the existing methods only focus on source-specific and stationary observational data . Such learning strategies assume that all observational data are already available during the training phase and from the only one source . This assumption is unsubstantial in practice due to two reasons . The first one is based on the characteristics of observational data , which are incrementally available from non-stationary data distributions . For instance , the number of electronic medical records in one hospital is growing every day , or the electronic medical records for one disease may be from different hospitals or even different countries . This characteristic implies that one can not have access to all observational data at one time point and from one single source . The second reason is based on the realistic consideration of accessibility . For example , when the new observational are available , if we want to refine the model previously trained by original data , maybe the original training data are no longer accessible due to a variety of reasons , e.g. , legacy data may be unrecorded , proprietary , too large to store , or subject to privacy constraint ( Zhang et al. , 2020 ) . This practical concern of accessibility is ubiquitous in various academic and industrial applications . That ’ s what it boiled down to : in the era of big data , we face the new challenges in causal inference with observational data : the extensibility for incrementally available observational data , the adaptability for extra domain adaptation problem except for the imbalance between treatment and control groups in one source , and the accessibility for a huge amount of data . Existing causal effect inference methods , however , are unable to deal with the aforementioned new challenges , i.e. , extensibility , adaptability , and accessibility . Although it is possible to adapt existing causal inference methods to address the new challenges , these adapted methods still have inevitable defects . Three straightforward adaptation strategies are described as follows . ( 1 ) If we directly apply the model previously trained based on original data to new observational data , the performance on new task will be very poor due to the domain shift issues among different data sources ; ( 2 ) If we utilize newly available data to re-train the previously learned model , adapting changes in the data distribution , old knowledge will be completely or partially overwritten by the new one , which can result in severe performance degradation on old tasks . This is the well-known catastrophic forgetting problem ( McCloskey & Cohen , 1989 ; French , 1999 ) ; ( 3 ) To overcome the catastrophic forgetting problem , we may rely on the storage of old data and combine the old and new data together , and then re-train the model from scratch . However , this strategy is memory inefficient and time-consuming , and it brings practical concerns such as copyright or privacy issues when storing data for a long time ( Samet et al. , 2013 ) . Our empirical evaluations in Section 4 demonstrate that any of these three strategies in combination with the existing causal effect inference methods is deficient . To address the above issues , we propose a Continual Causal Effect Representation Learning method ( CERL ) for estimating causal effect with incrementally available observational data . Instead of having access to all previous observational data , we only store a limited subset of feature representations learned from previous data . Combining the selective and balanced representation learning , feature representation distillation , and feature transformation , our method preserves the knowledge learned from previous data and update the knowledge by leveraging new data , so that it can achieve the continual causal effect estimation for new data without compromising the estimation capability for previous data . To summarize , our main contributions include : • Our work is the first to introduce the continual lifelong causal effect inference problem for the incrementally available observational data and three corresponding evaluation criteria , i.e. , extensibility , adaptability , and accessibility . • We propose a new framework for continual lifelong causal effect inference based on deep representation learning and continual learning . • Extensive experiments demonstrate the deficiency of existing methods when facing the incrementally available observational data and our model ’ s outstanding performance . 2 BACKGROUND AND PROBLEM STATEMENT . Suppose that the observational data contain n units collected from d different domains and the d-th dataset Dd contains the data { ( x , y , t ) |x ∈ X , y ∈ Y , t ∈ T } collected from d-th domain , which contains nd units . Let X denote all observed variables , Y denote the outcomes in the observational data , and T is a binary variable . Let D1 : d = { D1 , D2 , ... , Dd } be the set of combination of d dataset , separately collected from d different domains . For d datasets { D1 , D2 , ... , Dd } , they have the common observed variables but due to the fact that they are collected from different domains , they have different distributions with respect to X , Y , and T in each dataset . Each unit in the observational data received one of two treatments . Let ti denote the treatment assignment for unit i ; i = 1 , ... , n. For binary treatments , ti = 1 is for the treatment group , and ti = 0 for the control group . The outcome for unit i is denoted by yit when treatment t is applied to unit i ; that is , y i 1 is the potential outcome of unit i in the treatment group and yi0 is the potential outcome of unit i in the control group . For observational data , only one of the potential outcomes is observed . The observed outcome is called the factual outcome and the remaining unobserved potential outcomes are called counterfactual outcomes . In this paper , we follow the potential outcome framework for estimating treatment effects ( Rubin , 1974 ; Splawa-Neyman et al. , 1990 ) . The individual treatment effect ( ITE ) for unit i is the difference between the potential treated and control outcomes , and is defined as ITEi = yi1 − yi0 . The average treatment effect ( ATE ) is the difference between the mean potential treated and control outcomes , which is defined as ATE = 1n ∑n i=1 ( y i 1 − yi0 ) . The success of the potential outcome framework is based on the following assumptions ( Imbens & Rubin , 2015 ) , which ensure that the treatment effect can be identified . Stable Unit Treatment Value Assumption ( SUTVA ) : The potential outcomes for any units do not vary with the treatments assigned to other units , and , for each unit , there are no different forms or versions of each treatment level , which lead to different potential outcomes . Consistency : The potential outcome of treatment t is equal to the observed outcome if the actual treatment received is t. Positivity : For any value of x , treatment assignment is not deterministic , i.e. , P ( T = t|X = x ) > 0 , for all t and x. Ignorability : Given covariates , treatment assignment is independent to the potential outcomes , i.e. , ( y1 , y0 ) ⊥ t|x . The goal of our work is to develop a novel continual causal inference framework , given new available observational data Dd , to estimate the causal effect for newly available data Dd as well as the previous data D1 : ( d−1 ) without having access to previous training data in D1 : ( d−1 ) . 3 THE PROPOSED FRAMEWORK . The availability of “ real world evidence ” is expected to facilitate the development of causal effect inference models for various academic and industrial applications . How to achieve continual learning from incrementally available observational data from non-stationary data domains is a new direction in causal effect inference . Rather than only focusing on handling the selection bias problem , we also need to take into comprehensive consideration three aspects of the model , i.e. , the extensibility for incrementally available observational data , the adaptability for various data sources , and the accessibility for a huge amount of data . We propose the Continual Causal Effect Representation Learning method ( CERL ) for estimating causal effect with incrementally available observational data . Based on selective and balanced representation learning for treatment effect estimation , CERL incorporates feature representation distillation to preserve the knowledge learned from previous observational data . Besides , aiming at adapting the updated model to original and new data without having access to the original data , and solving the selection bias between treatment and control groups , we propose one representation transformation function , which maps partial original feature representations into new feature representation space and makes the global feature representation space balanced with respect to treatment and control groups . Therefore , CERL can achieve the continual causal effect estimation for new data and meanwhile preserve the estimation capability for previous data , without the aid of original data . 3.1 MODEL ARCHITECTURE . To estimate the incrementally available observational data , the framework of CERL is mainly composed of two components : ( 1 ) the baseline causal effect learning model is only for the first available observational data , and thus we don ’ t need to consider the domain shift issue among different data sources . This component is equivalent to the traditional causal effect estimation problem ; ( 2 ) the continual causal effect learning model is for the sequentially available observational data , where we need to handle more complex issues , such as knowledge transfer , catastrophic forgetting , global representation balance , and memory constraint . We present the details of each component as follows .

Summary: This paper considers adopting continual learning on the problem of causal effect estimation. The paper combines methods and algorithms for storing feature representation and representative samples (herding algorithm), avoiding drifting feature representation when new data is learned (feature representation distillation), balanced representation by regularization, etc. Consequently, the paper presents a system that makes use of existing methods as a loss function (the sum of losses and regularization terms).