Time Series Counterfactual Inference with Hidden Confounders
Authors: Guangyu Li
Jiahao Chen
Samuel A Assefa
Yan Liu
Venue:  (2021)
Keywords: Time Series Analysis
Counterfactual Inference
Differential Equations.

Abstract: We present augmented counterfactual ordinary differential equations (ACODEs), a new approach to counterfactual inference on time series data with a focus on healthcare applications. ACODEs model interventions in continuous time with differential equations, augmented by auxiliary confounding variables to reduce inference bias. Experiments on tumor growth simulation and sepsis patient treatment response show that ACODEs outperform other methods like counterfactual Gaussian processes, recurrent marginal structural networks, and time series deconfounders in the accuracy of counterfactual inference. The learned auxiliary variables also reveal new insights into causal interventions and hidden confounders.

Under review as a conference paper at ICLR 2021
TIME SERIES COUNTERFACTUAL INFERENCE WITH HIDDEN CONFOUNDERS
Anonymous authors Paper under double-blind review
ABSTRACT
We present augmented counterfactual ordinary differential equations (ACODEs), a new approach to counterfactual inference on time series data with focus on healthcare applications. ACODEs model interventions in continuous time with differential equations, augmented by auxiliary confounding variables to reduce inference bias. Experiments on tumor growth simulation and sepsis patient treat- ment response show that ACODES outperform other methods like counterfactual Gaussian processes, recurrent marginal structural networks, and time series de- confounders the accuracy of counterfactual inference. The learned auxiliary variables also reveal new insights into causal interventions and hidden confounders.
INTRODUCTION
Decision makers want know how to produce desired outcomes and act accordingly, which requires causal understanding of cause and effect. In this paper, we consider applications in healthcare, where time series data past features and outcomes are widely available. Causality in time series have been long studied in statistics (Box et al., 2008), and allows more powerful analysis than methods on time-independent data, like instrumental variable regression (Stock & Trebbi 2003). However, temporal causality in statistics and econometrics focuses mainly on passively discovering time lag structure (Eichler 2012). In contrast, decision-making applications need concrete interventions, which is more amenable to an interventionist approach to causality [Woodward] 2005 Pearl (209)) To give one example, electronic health records (EHR) in healthcare provide an accessible history of patient's s disease progression over time, together with their treatment records and their results. To identify effective treatments, a doctor may want to ask counterfactual questions (Johansson et 2016), like "Would this patient have lower blood sugar had she received different medication?" Through such counterfactual analysis, medical professionals may hope to discover new cures and improve existing treatments. Similar situations arise in other use cases, For example, user interface designer may want to ask "Would the user have clicked on this ad had been in a different color?", substantiating their answer from counterfactual inference clickstream data other user behaviors.
Counterfactual inference in time series has studied, assuming that all possible causal variables are observed (Soleimani al., 2017 [Schulam & Saria 2017 Lim 2018). In practice, however, this assumption of perfect observability is not testable and too strong for many real-world scenarios (Bica) al. 2020). For example, there are many ways in general to treat cancer, but each patient requires their own bespoke treatment plan based on unique characteristics of each case such as drug resistance and toxic response VVlachostergios
Faltas 2018, Kroschinsky et al. 2017 Bica et al., 2020). However, these factors are also likely to be unmeasurable in practice, or otherwise not recorded in EHRs. Detecting these hidden confounding variables therefore crucial to avoid bias in the estimation of treatment effects.
The challenge introduced by confounders in counterfactual inference was first studied the static setting. [Wang& & Blei (2019) developed a two-step method that estimates confounders with latent factor models, then infers potential outcomes with bias adjustment. However, confounders time series can have their own dynamics, and can themselves be affected by the history of interventions. Subsequently, Bica et al. (2020) introducing recurrent neural networks (RNNs) into the factor model to estimate the dynamics of confounders. However, this method only works in discrete time setting with a fixed time step, due to how RNNs are structured. In this paper, we consider the continuous time setting, which is more flexible in practice and provides more insights of the underlying mechanisms

---

Under review as a conference paper at ICLR 2021
(Chen et al. 2018 Rubanova et al. 2019). The continuous-time setting is particularly important for healthcare, where there are many iie-inrying treatments, seggularly-sampllee or partially observed time series (Soleimami et al. 2017).
The classical modeling approach to dynamics uses ordinary differential equations (ODEs) d(æx(t)//dt = f()(t)), encoding domain expertise of underlying mechanisms in the explicit specifi- contrast,Chen
cation of f. In contrast, [Chen et al. (2018) introduced the concept of neural ODEs by parameterizing with neural networks, thus allowing dynamics to be described by arbitrarily complicated func- tions. Several extensions handle even more complicated issues like irregular sampling or switching dynamics (Jia & Benson 2019 Kidger al., 2020). However, these methods cannot be directly applied to time series counterfactual inference, as they focus on initial value problems, which cannot describe interventions without explicit modification of f (Kidger et al.) 2020). Furthermore, these existing methods can only handle hidden variables by explicitly describing their dynamics and interdependency with interventions, thus limiting their utility when confounders exist.
Our contributions. We propose augmented counterfactual ODEs (ACODEs) to predict how continuous-time time time series will evolve under a sequence of interventions. Our method augments the observed time series with additional dimensions to represent confounders. We then construct counter- factual ODEs based on the neural ODE framework to model the effects of incoming interventions. The ACODE model has three key features. First, it allows for the presence of confounders that can reduce the prediction bias. Second, the ACODE can continuously incorporate incoming interventions using neural ODEs and support inreuldrly-sampleed time series. Third, it demonstrates state-of-the-an performance against competitive baselines for counterfactual inference in both simulation of tumor growth and real-world time series of sepsis patients treatment response. Moreover, the ACODE provides an interface between machine learning and dominant modelling paradigm described in differential equations, which allows for well-understood domain knowledge be applied to time series counterfactual inference. To the best of our knowledge, this represents the first method for counterfactual inference with confounders in the continuous-ting setting.
RELATED WORK
Time series counterfactual inference stems from causal inference (Pearl 2009 Eichler 2012). A large body of pioneering work in causal inference focus on causal relations such as structural causal models (Pearl 2019)) and Granger causality (Fichler 2007). Counterfactual inference, the other hand, tocus on estimating the effects of actionable interventions, which is a pervasive problem healthcare (Hoover) 2018). In literature, the difference between the counterfactual outcomes if an intervention had been taken or not is defined as the causal effect of the intervention (Pearl 2009). Originated from the literature observational studies (Shadish et al. 2002). Rubin's potential outcome framework has been popular language to formalize counterfactuals and intervention effect estimate (Rubin) 2005 Imbens& & Rubin 2015).
The problem of hidden confounders in counterfactual inference was first studied in the static setting. Wang & Bleij (2019) developed theory for adjusting the bias introduced by the presence of hidden confounders in the observational data. They found out that the dependencies in these multiple confounders can be used to infer latent variables and act as substitutes for the hidden confounders. In this paper, we are interested in considering hidden confounders in time series setting which is much more complicated than in the static setting. Not only because the hidden confounders may evolve over time, but also because they might be affected by previous interventions. On the other hand, most existing work on time series counterfactual inference including counterfactual Gaussian processes (CGP) (Schulam & Saria 2017) and recurrent marginal structural networks (RMSNs) (Lim) 2018) assume there is no hidden confounders, i.e. all variables affecting the intervention plan and the potential outcomes are observed, which is not testable in practice and not true in many cases. Recently, BBca et al.) (2020) applied the idea of latent factor models from Wang & Blei (2019) to the deconfounding of time series. However, their proposed method is based on recurrent neural networks, which works only with discrete and regularly-spaced time series.
Differential equations have been introduced into causal and counterfactual inference in previous studies. Rubenstein et al. (2018) showed that equilibrium states of first-order ODE system can be described with deterministic structural causal model, even with non-constant interventions. This
2

---

Under review as a conference paper at ICLR 2021
line of literature is centering around casual relations, which is a different focus from this work. On the other hand, differential equations with incoming information is a well-studied mathematical problem in the field of rough amalysis, which is referred controlled differential equations rough differential equations. These approaches directly integrate with respect to incoming processes (Friz & Victoir 2010:
Lyons et al. 2007).
Neural Ordinary Differential Equations Neural ODEs (Chen et al. 2018 are family of time titeoouus-tteeeeeeee work based differential equations. The state at any time 4 is given by integrating ODE forward in time:
continuous-
models. Starting from an initial state z(to), it evolves following neural net-
(z((t) = f(z(t),t;0), z(t+)= = =(to) =(to)+ + L.' dz( dt
dt
dt
where f is a neural network parametrized by A. Given the initial state, states at any desired time stamps can be evaluated with numerical ODE solver:
20, 21 .... N ODESolve(fo, =to)), (tu, t1, .... fx))
(2)
More importantly, Chen et al. (018) proposed to use the adjoint method to compute the gradient with respect the parameters 0 as long as f is uniformly Lipschitz continuous in (t) and continuous in t. This allows ODE solvers to be used as black box building block in large models.
3 PROBLEM FORMULATION
Consider multivariate time series xtt) and continuous-time time-dependent interventions a(t). The observational data consists of multiple realizations of above mentioned time series and interventions. Given that realizations are independent to each other, we only consider one realization in following part for simplicity. In realization up time t, we observe N time series data points and their timestamps {æ;, 44 J along with continuous-time interventions {a(s) S l}. We would like to infer the potential outcome under future interventions given all historical information for any potential intervention plan {a(s) s >> t}. We will abuse the notation of asr and {a(s) > t in following sections. Our goal is to infer the following distribution:
p(z(a>))aas: {x,,t] (/11))
(3)
where w(aze)) denotes the potential outcome of future time series under future intervention ast. Although we cannot directly model this objective distribution, we can instead fit a re- gression model to estimate p astr, [e,,4)]11 1) from observational data (Rubin) 1978).
For cases without hidden confounders, this lead to unbiased estimation of potential outcome p(z(a>)))ast {x,.te) p(=></a>> act, [x, 1. under certain assumptions, including sequential strong ignorability (Fitzmaurice et al. 2008):
z =aaz) a(t) Tst for Vazr
(4)
This condition holds if there are hidden confounders, which cannot be tested in practice since counterfactual outcomes are never observed in practice. With the presence of hidden confounders, the above assumption is longer valid and
p(z(a>:))age (4)))) # / age, fæ,, t, 1,)
(5)
Consequently, existing methods which infer conditional distribution p( aasts acti (x,, L. from observed data would result in biased estimation of potential outcome.
4 AUGMENTED COUNTERFACTUAL ORDINARY DIFFERENTIAL EQUATIONS
To address the problem, the key is to reduce inference bias caused by the presence of hidden confounders and capture underlying temporal dynamics and the intervention effects, We propose a two-step method, called augmented counterfactual ordinary differential equations (ACODEs), which first lifts the time series into an augmented space with additional dimensions and then models the augmented time series with neural network parameterized counterfactual differential equations.
3

---

Under review as a conference paper at ICLR 2021
Latent States =(1)
Mzu
E
Interventions a(t)
Augmented Time Series T
14
16
11
Auxiliary Variables
Time Series Data
233
Time
02 03 f2 4
13 1x
Figure The ACODE model with CDE-RNN encoder and CDE decoder. The CDE-RNN en- coder first runs Backwards-in-tie (zol{ & LN ace). Given a sample of žy and intervention process (t), we can generate latent state any point of interest, and further generate augmented time series observations.
produce approximate posterior the initial latent state
The proposed method first lifts the time series observations by introducing k auxiliary variables u, R into Rt, resulting in augmented time series 3y augmented space, we can safely assume
Augmented time series
IIn this
p(æ(aa:))ass, (ā,, ()) p( age, (0;, t;)
(6)
The insights of introducing auxiliary variables u, come from two aspects. The first one centered around bias- variance trade off. Basically, modeling in the augmented space reduce estimation bias at the cost of higher variance (Robins et al. 2000). The second one is about the dimensionality want approximate the
of the space where underlying temporal dynamics work. Basically, underlying temporal dynamics with learnable mappings. Given the presence of hidden confounders, the true temporal dynamics work in space with higher dimensionality comparing with the space of time series observations. Therefore, additional dimensions could make it casier to approximate the true temporal dynamics (Dupont al. 2019).
In general, auxiliary variables u, serve purely as mathematical component without interpretable mechanistic meaning and are initialized with all zero vectors. However, in some cases, the learned auxiliary variables Ut can provide interpretable insights about hidden confounders. We will show this later in the experiment on tumor growth simulation. Further, we may also leverage the domain knowledge hidden confounders and initialize hidden confounders based on their dependency with observed covariates and interventions. This is especially useful in fields where we have mechanistic understanding of hidden confounders thought they cannot be directly measured. For example, patient's 's cardiac contractility (the heart's ability to squeeze blood), stroke volume, or systemic vascular resistance not unobserved, but be inferred with domain knowledge.
Latent Counterfactual Differential Equations Starting from augmented time series, the proposed method models the intervention effects with latent counterfactual differential equations. Specifically, we assume there are latent states representing the state of time series. Latent states evolves controlled by both the baseline progress and intervention effects.
z(t) = =to) + [' fa(z(s);6)dds + f f((2(s),
Baseline progress
a(t) pl (()(())2(())
(8)
eq. 7 represents counterfactual differential equations (CDEs), where both fz and fo are neural networks parameterized by 02 and 0,, respectively. Unlike previous methods such as (Rubanoval et al. 2019), the proposed counterfactual differential equations continuously incorporate incoming interventions, without interrupting the differential equation. Therefore, the same techniques as for Neural ODEs. Given an initial latent state 2p, the generation process for continuous value time series is summarized in Algorithm
solve the CDEs using
4

---

Under review as a conference paper at ICLR 2021
Algorithm I Generation process of the latent CDE model.
Input: A distribution of initial latent state p(z)); timestamps of interest {4 (4)/= continuous-úime intervention process a-g--
Output: Time series observations and their timestamps 4())) corresponding latent states ((2)
Sample 20 p(zo).
2: Compute 21, ..... NN fo for age, (t))
3: for fori= No
4: Compute pz:; DT. (2(;;;0m))
5: Sample 24 ((H Ez.)
6: end for
7: return {a., 2, J
Algorithm 2 The CDE- RNN- encoder for general cases.
Input: Time series observations and their timestamps (( (,) ) i= the continuous- time interventions
process agt-
Output: Hidden states and their timestamps ((h;, (:))
1: Set ho 0
2: for i= 1, N do
3: Update h' ODESlove(gh, Fa. (4--1, h; {a(s) S <8<4))
4: Update h2 = RNNCell (1, 2;))
5: end for
6: return {
We use variational autoencoder framework for model training and counterfactual inference. This requires estimating the approximate posterior g(z0 {æ,, t; J.2 act)). Inspired by Rubanova et al. (2019), we use RNN together with CDE and propose the CDE-RNN model to incorporate time series observations {æ;, t4 J.M during encoding. The proposed CDE-RNN, summarized in Algorithm 2 would an effective way to handle irregularly-same sampled time series. To get the approximate posterior of initial latent state zo at time point lg. run the CDE-RNN encoder backwrads-in-time from Iy to in- Then we represent the approximate posterior with Gaussian random variables depending on the final hidden state of an CDE-RNN:
(200 1 agi) -N Ezo)
(9)
where 1o Ezo 9s (CDE-RNN,((2,, 4. ag:))
(10)
Here 92 a neural network mapping the final hidden state of the CDE-RNN encoder into the mean and variance of the approximate posterior of zo. Following autoencoders framework, we jointly learn both the CDE-RNN encoder and CDE decoder by maximizing the evidence lower bound (ELBO):
LELBO (6, Φ) =E = ag. po (2), xN))
DkLlq(zo) (x,, 1 agi) [|{()))]
(11)
Although with incoming interventions, the whole model is still ODE-based sequence-to-seeqeeee model. Therefore. we use the adjoint-based backpropagation described in Chen et al. (2018) for training. The overall learning procedure of ACODE is summarized in Algorithm
5 EXPERIMENTS
We evaluate the proposed method with two experiments, including realistic tumor growth simulation (Geng et al. 2017) and a real-world large scale dataset of ICU patients with sepsis, which is extracted from MIMIC-III dataset (Johnson et al.) 2016). Through experiments,
answer the following questions: (1) How the performance of the proposed model for time series counterfactual inference in the presence of hidden confounders, compared to existing state-of-the-a methods? (2) How would the number of auxiliary variables affect the performance in the presence of hidden confounders? (3)
Do the learned auxiliary variables provide any insight of hidden confounders?
5

---

Under review as a conference paper at ICLR 2021
Algorithm 3 Learning process of ACODE with variational inference.
Input: A set of time series along with continuous-time intervention process D; initial value of parameter (0, Φ).
1: while not converged do
2: Choose a time series with timestamps {( t.) E and continuous time intervention process ast D.
3: Initialize auxiliary variables u, with all zero vectors for i = 1,2,..., N.
4: Augment time series æ; = m: for i 1.2,...,
5: Initialize hidden state hx with all zero vectors.
6:
for == N-1, do
7:
Update h ODESlove(gh, 9a. ((1,), hit1, {a(s) S t41))
8:
Update h; = RNNCell(h, @;)
9:
end for
10:
Compute 1o Σno = 92 (ho; Φ=))
11:
Sample 20
p(zo)-
12:
Compute 21, ZN OD 20, age, (4))
for ..., N do
14:
Compute uxx Eg, = 0z)
Sample ã; (H,,, Exe))
16:
end for
Compute the gradient of LELBO (0, Φ) as shown in Equation
18:
Update (0, d) with Adam optimizer.
19:
end while
Baselines We compared the proposed method with 3 competitive baselines in the time series counterfactual inference task, including Counterfactual Gaussian Process (CGP) (Schulam & Saria 2017), Recurrent Marginal Structural Network (RMSN) (Lim 2018) and Time Series Deconfounder with RMSN (TSD-RMSN) (Bica al. 2020). To demonstrate the effectiveness of augmented space in ACODE, we remove all auxiliary variables (k=0) from ACODE for ablation comparisons.
Performance Criterion We compute root mean square error (RMSE) and normalized root mean square (NRMSE) for each time series averaging across inference time horizon. For each experiment setting, we repeat 10 times and compute the standard deviation as a measure inference variance.
Implementation Details For baselines only work with discrete-time interventions, we discretize the continuous-timg intervention process with the same timestamps as time series observations {(x,,t)) (.) Ji=1. i.e. {a, a, [( For baselines designed for regularly-spaced time series, like RMSN, TSD-RMSN, we use linear interpolation as the bridge to transfer trreuularlyssapple time series to its regularly-spacer counterpart, and vice versa. We Gaussian distribution with diagonal covariance for the distribution latent state z and time series observation æ. All neural network mappings are parametrized with 3-layr MIPs methods, we use a similar amount of parameters for fair comparison. We randomly split each dataset into the craining/validation
ReLU activation. For all neural network based
set, and choose hyperparameters e.g. the number of hidden factors of TSD-RNN based on the validation set.
5.1 TUMOR GROWTH SIMULATION
To show the effectiveness of the proposed method, we first evaluate on a simulated environment with fully control the hidden confounders the pharmacokinetic- aaammookneicc-haammotymmmm model of tumor growth under the effects of chemotherapy and radiotherapy proposed by Geng et al. (2017). The tumor volume after days since diagnosis is modeled follows:
(PK-PD)
V(t)= = (1 plog V(t
K
B.C(1)
(a,d(() + + +
v((-1) -1 (12)
1)
Chematherapy
Radiotherapy
Noise
Tumar growth

---

Under review as a conference paper at ICLR 2021
3.00
CGP
Type Patient Type Patient Type
2.75
RMSN
TSD
2.50
ACODE (k=1)
2:25
ACODE (k=3)
NRMSE
2:00
ACODE (K=5)
1.75
1.50
1.25
1.00
0.75
3
Counterfactusl Inference Time Horizon (days)
Figure 2: Normalized RMSE curve of coun- terfactual inference for treatment response tumor growth.
Figure 3: Visualization of learned auxiliary variables sequence u, for all three types of patients.
where parameter set (K, p. Se, Or, Br,er ) are sampled as descrihed in Geng et al. (2017) Radio- therapy and chemotherapy prescriptions on the tumor size (tt). Specifically, we assume the chemotherapy prescriptions and radiotherapy prescriptions have probabilities po(() and pa(() respectively that are a functions of the tumor diameter:
modeled Bernoulli random variables which depend
(t) o Ye (D(t) - 0.)
Pa(t) ( 7id (D(t) 0a)
D
(D())-0a)
(13)
max
where D(t) is the average tumor diameter over the past 15 days, a(.) is the sigmoid activation function, 0g and f₂ are constant parameters, and controls the degree of time-dependent confounding. In the previous study on the tumor growth (Lim 2018). the prior means of 3. and a, are adjusted according three patient types accounting for patient heterogeneity due to genetic features. The patient group corresponds to different parameter setting and thus affects the tumor growth and subsequently treatment plan. this experiment, we treat the tumor size as time series observation <tt), patient types the hidden confounder, treatment plan of chemotherapy and radiotherapy as the intervention att). . Our task is to predict tumor growth progress under various treatment plans, without any information about patient types.
Following experimental set-up Lim (2018), we simulated data with 10000 patients for training, 1000 for validation, and 1000 for testing. We set the number of auxiliary variables in the augmented space E {0, 1, 3,3,55 5), and evaluate treatment response inference on tumor size with normalized root mean square errors (NRMSE) and standard deviations.
Figure shows the counterfactual inference accuracy on tumor size across time horizon days. We observe that, methods considering hidden confounders including the proposed ACODE (k 0) and TSD-RMSN outperform other baselines that assume all confounders are observed such as CGP, RMSN and ACODE (k- 0). Further, with the representation power of neural networks and differential equations, ACODE excels all other competitive baselines in term of inference accuracy, especially for long-term inference. Although, auxiliary variables the ACODE can effectively reduce the inference bias caused by hidden confounders, too many auxiliary variables may introduce unnecessary variance without much help on inference accuracy. we can see ACODE (k: 5) has similar accuracy comparing with ACODE (k- 3), but suffers from higher variance.
So far we have been treating the auxiliary variables u, as mathematical auxiliary component. Since we force the model to learn the system mechanism in the augmented space &. [:: we would like to know whether there is insight learned by auxiliary variables u, Therefore, we randomly choose 1500 patients and project time series of auxiliary variables {u, 1a learned by ACODE (k 3) on the two dimensional plane using t-SNE (Maaten & Hinton 2008), as shown in Figure 3 As we see, the learned auxiliary variables can be clustered into three groups corresponding three
1.N
7

---

Under review as a conference paper at ICLR 2021
Table 1: Average RMSEx10 102 and standard error with 10 runs for the inference of sepsis patients.
Methods
White blood cell count
Blood pressure
Oxygen saturation
CGP
2.53 + 0.05
9.31 + 0.06
1.21 + 0.04
RMSN
2.91 = 0,05
10.29 + 0.05
1,74 = 0.03
TSD-RMSN
2.48 = 0.06
9.20 0.12
1.17 + 0.08
ACODE (k = 0)
2.57 = 0.04
9.40
1.27 = 0.05
ACODE (k ==11
2.47 0.05
9.35 0.07
1.15 + 0.05
ACODE (k
2.36 = 0.07
8.94 + 0.08
1.09 = 0.06
ACODE (k= 10)
2.43 = 0.11
9.18 = 0.13
1.06 = 0.09
different patient types. This pattern demonstrates the potential of ACODE to provide insights of hidden confounders via learned auxiliary variables u,
5.2 INTENSIVE CARE OF PATIENTS WITH SEPSIS
Next, we evaluate the proposed method in real-world scenario without full understanding of hidden confounders - electronic health records of sepsis patients in ICU with three treatment options: antibiotics, vasopressors, and mechanical ventilator. We use the electronic health records extracted from Medical Information Mart for Intensive Care (MIMIC III) database (Johnson et al., (016). Follow the same pipeline Bica al. (2020) we extracted 25 patient covariates consisting of lab tests and vital signs for each patient. In this experiment, we would like to infer the effects of antibiotics, vasopressors, and mechanical ventilator on three patient covariates: white blood cell count, blood pressure, oxygen saturation. Specifically, we extract patient records up to 50 days from MIMIC III database for training and testing, and infer treatment response in 24 hours. The hidden confounders include comorbidities and lab test that recorded in MIMIC III database but not used by counterfactual inference models. fact, there might be other hidden confounders, given that is real-world scenario.
As we can see in T the proposed ACODE model outperforms other competitive baselines. Also, there is clear performance gap between methods considering hidden confounders like ACODE (k >> 0) and TSD-RMSN, and methods ignoring them like CGP and RMSN. Among all ACODE baselines, with the increasing amount of auxiliary variables, ACODE gets lower RMSE the cost of higher variance. This pattern is consistent with what we observed in the tumor growth simulation. The proper number of auxiliary variables k* varies for individual applications and need be tuned as hyperparameter.
SUMMARY
We proposed the augmented controlled ordinary differential equations (ACODES) novel neural network based model for time series counterfactual inference with the presence of hidden confounders. The proposed method introduces auxiliary variables and lifts time series into an augmented space to reduce the inference bias caused by the hidden confounder. With the representation power of neural networks and differential equations, it can effectively capture underlying temporal dynamics and intervention effects from observational data. Empirically, we show that ACODES outperform existing methods on inference accuracy in both simulations and real-world applications, and showed its potential to provide insight of hidden confounders via auxiliary variables. With the increasing amount of data we collect everyday, ACODE would empower us answer "what if" questions regarding time series and discover insights of underlying mechanisms behind time series observations and possible interventions. For the future work, it would be worth to explore using the ACODE as the interface between machine learning and dominant modelling paradigm described in differential equations, and incorporate well-understood domain knowledge into time series counterfactual inference.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Ioana Bica, Ahmed M Alaa, and Mihaela van der Schaar. Time series deconfounder: Esti- mating treatment effects over time in the presence of hidden confounders. In Daumé, Hal, III and Aarti Singh (eds.), Proceedings of the International Conference on Machine Learn- ing, pp. 3954,3954, 2020. URL /ttp://pss//pppeeeiii.s.ooa.a/ap 2734484 2134 htt.6e22ccccccccbb2222eee
George E. P. Box, Gwilym M. Jenkins, and Gregory C. Reinsel. Time series analysis: Forecasting and control. Wiley Series in Probability and Statistics. John Wiley & Sons, Hoboken, NJ, edition, 2008. doi: 10. 10.10/2/978) 111119919999
Ricky Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Bengio, Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31, pp. 6571- 6583, Curran Associates, 2018. URL /ttpsttp//pppprrspppp..///apeeee ordinary differential.. equations.po
neural
Emilien Dupont, Arnad Doucet, and Yee Whye Teh. Augmented neural ODEs. In H. Wallach, H. Larochelle, A. Beygelzimer, Alché Buc, Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, pp. 3140-31411000 3150. Curran Associates, 2019. URL
Michael Eichler. Granger causality and path diagrams for multivariate time series. Journal of Econometrics, 137(2):334-353, April 2007.
doi: j0.0006/)jeccommm.200. 2005.066.03..
Michael Eichler. Causal Inference in Time Series Analysis, chapter 22, pp. 327-354, John Wiley & Sons, Chichester, UK, 2012. doi: 10.1002/978 10.1002/9981111999977777 19945710.ch22..
Garrett Fitzmaurice, Marie Davidian, Geert Verbeke, and Geert Molenberghs. Longitudinal data analysis. Chapman & Hall/CRC, New York, edition, 2008. doi: 10.1201/9781420001557
Peter K Friz and Nicolas B Victoir. Multidimensional stochastic processes rough paths: theory and applications, volume 120 of Cambridge Series in Advanced Mathematics. Cambridge University Press, Cambridge, UK, 2010.
Changran Geng, Harald Paganetti, and Clemens Grassberger. Prediction of treatment response for combined chemo-and radiation therapy for non-small cell lung cancer patients using a bio- mathematical model. Scientific Reports, 7:13542, 2017. doi: 10. 10.1038/441998001-111 13646-2.
Kevin D Hoover. Causality in economics and econometrics, pp. 1446-1457. Palgrave Macmillan UK, London, 2018. doi: 10.
2227.
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, Cambridge, UK, 2015.
Junteng Jia and Austin R Benson. Neural jump stochastic differential equations. In H. Wal- lach, H. Larochelle, A. Beygelzimer, F. d' Alché Buc, E. Fox, and R. Garnett (eds.). Advances in Neural Information Processing Systems, volume 32, pp. 1843-9854.. Curran Associates, 2019. URL differential -ew.alaosuggaati..pp......
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In Proceedings of The 33rd International Conference on Machine Learning, vol- ume 48, pp. 3020-3029, Proceedings of Machine Learning Research, 2016. URL http //proceedings.mi.z
Alistair E W Johnson, Tom J Pollard, Lu Shen, H Lehman Li- Li-Wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. MIMIC- III, freely accessible critical care database. Scientific Data, 3:160035, 2016. doi: 10.1038.
2data.2016.35.
Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. arXiv:23005.08822 2020.

---

Under review as a conference paper at ICLR 2021
Frank Kroschinsky, Friedrich Stölzel, Simone von Bonin, Gernot Beutel, Matthias Kochanek, Michael Kiehl, Peter Schellongowski, and Intensive Care in Hematological and Oncological Patients (iCHOP) Collaborative Group. New drugs, new toxicities: severe side effects of modern targeted and immunotherapy of cancer and their management. Critical Care, 21(1):89, 2017. doi: 10. 186/ 113054-017-167-11
Bryan Lim. Forecasting treatment responses over time using recurrent marginal structural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Dasa-Bianchi. and R. Garnett (eds.). Advances Neural Information Processing Systems, volume 31, pp. 7483-74887 7493., Curran Associates, 2018. doi: 0..555/33277557.32275555
Terry J Lyons, Michael Caruana, and Thierry Lévy. Differential equations driven by rough paths: École d' d'Éíé de Probabilités de Saint-Flowd XXXIV 2004, volume 1208 of Lecture Notes in Mathematics. Springer- Verlag, Berlin Heidelberg, 2007. doi: 10.1007/978---550-77220
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 1-2579-2605 November 2008. CCLLhttp::/ http:///mmm...ro//p///////////
Waaderrmatten08 ff
Judea Pearl. Causal inference in statistics: An overview. Statistics Surveys, 3:96-146, 2009, doi: 10.1214/09-S5057
Judea Pearl. Causal and counterfactual inference. Technical Report R-485, UCLA Cognitive Systems Laboratory, October 2019.
James M Robins, Andrea Rotnitzky, and Daniel O Scharfstein. Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In Statistical models in epidemiology, the environment, and clinical trials, volume 116 of IMA Volumes in Mathematics and its Applications, pp. 1-94. Springer, New York, 2000. doi: 10. 1-61--124--3.1.
Yulia Rubanova, Ricky Tian Qi Chen, and David K Duvenaud. Latent ordinary dif- ferential equations for irregularl-sampped time series. In H. Wallach, H. Larochelle, A. Beygelzimer, d' Alché Buc. E. Fox, and Garnett (eds.). Advances in Neural Information Processing Systems, volume 32, 5320-5330. Curran Associates, 2019. diferentiaa /tt.a:ac.//ppeeaa....cppp//////a http:///ppeeeetttaaaaa.caaaaaaae eer.aaCoons--oo--rrepaaaaaaaaaap
Paul K Rubenstein, Stephan Bongers, Joris Joris M Mooij. and Bernhard Schölkopf. From deterministic odes to dynamic structural causal models. Proceedings of the Conference on Uncertainty in Artificial Intelligence, pp. 43, 2018. URL htl.:t/g///aaaa.orr/aaaa////////
/tterr//3..dd.
Donald Rubin. Bayesian inference for causal effects: The role of randomization. The An- nals statistics, 6(1):34-58, 1978. hrallhtppps///pppoocccccllddooop
1176344064)
Donald B Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469):322-331, 2005. doi: 10.1198/
0162145040000018880
Peter Schulam and Suchi Saria. Reliable decision support using counterfactual models. In I. Guyon, U. Luxburg, S. Bengio, H. Wallach, R. Fergus. Vishwanathan, and Garnett (eds.), Advances in Neural Information Processing Systems, volume 30, pp. 1697-1708. Curran As- sociates, 2017.
support-us wow.raaooooooottttoaaaaaaaoeeeee
William R Shadish, Thomas D Cook, and Donald Thomas Campbell. Experimental and quasi- experimental designs for generalized causal inference. Houghton Mifflin, Boston, 2002.
Hossein Soleimani, Adarsh Subbaswamy, and Suchi Saria. Treatment-response models for counterfactual reasoning with continuous-time continuous-valued interventions. In Proceed- ings of the Conference on Uncertainty in Artifical Intelligence, pp. 266, 2017. URL http? 7/
10

---

Under review as a conference paper at ICLR 2021
James H. Stock and Francesco Trebbi. Retrospectives: Who invented instrumental variable regression? Journal of Economic Perspectives, 17(3):777-94, September 2003. doi: 10.125709953003366244166
Panagiotis Vlachostergios and Bishoy M Faltas. Treatment resistance in urothelial carcinoma: an evolutionary perspective. Nature Reviews Clinical Oncology, 15(8):495-509, 2018. doi:
10.1138//41571-018--00
Yixin Wang and David Blei. The blessings of multiple causes. Journal of the American Statistical Association, 114(58))1574-1598 2019. doi: 10. 10800011221459 2019. 1686987.
James Woodward. Making Things Happen: Theory of Causal Explanation. Oxford Studies in Philosophy of Science. Oxford University Press, Oxford, UK, 2005.


Reviews:
[{"review_id": "uo--LjItJu", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors present an approach to counterfactual inference for time series based on ordinary differential equations to accommodate for the continuous-time setting. \n\nIt seems more accurate to refer to the proposed approach as one for counterfactual inference with confounders in the irregularly-spaced time series rather than continuous-time setting.\n\nIt may be good for the authors to explicitly point out the similarities and differences of the proposed approach relative to Bica et al, 2020, because on the surface, it seems that the proposed approach simply replaces the RNN in Bica et al, 2020 with the neural ODE model of Chen et al, 2018.\n\nFor the first experiment on tumor growth simulation data, for which samples can be obtained at regularly sampled times it is not clear that 1) the linear interpolation for RMSN and TSD-RMSN is necessary (please clarify, because the details of the experiment are quite vague) and 2) assuming samples are regular, why would the proposed approach be better than TSD-RMSN?\n\nThe results in Figure 3 are very difficult to evaluate without context about the three groups of patients and whether they will exhibit similar clustering structure should they have clustered from data.\n\nThe results in Table 1 are underwhelming considering the error bars of the results relative to the simpler TSD-RMSN. More so, when results for different number of hidden confounders (D_z in Bica et al, 2020) are considered, which they should provided that they are also presented for ACODE for completeness.", "extra_scores": {}, "extra_text": {}}, {"review_id": "iBogl4Yetkw", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed to solve an interesting problem: how do we perform counterfactual inference for time series data? The paper follows a study of the problem in the static setting: in the first step, the paper fit an augmented time series $u_t$ as additional confounders, and then perform inference based on the augmented time series. The experiments presented appear promising to the time series inference problem.\n\nStrengths:\n+ This is a very important problem that needs careful investigation\n+ The application of this method to healthcare data is very important\n+ The paper is well written and easy to follow\n+ The execution of the experiments\n\nPoints for Improvement\nI would love to hear some clarification questions from the authors\n1. The paper introduces the auxiliary variables $u_t$ and assume they plus $x_t$ satisfy the sequential ignorability. How do we ensure the learned $u_t$ satisfy the condition? I am wondering whether there are metrics (for example, in static case, it would be something similar to a conditional independence test) for the sequential case. And whether this constraint should be reflected in the learning process, for example, adding a regularization function?\n\n2. The paper argues that we should consider the continuous case interventions. However, in the applications/experiments, it appears the interventions and problem settings are all discrete time spaced. For example, the tumor growth function is formulated as V(t) = f * V(t-1), which looks like discrete time points. I would love to hear more discussions on the choice between discrete and continuous, and why we should consider the continuous case. \n\n3. What are the exact number of parameters used by each of the method? I would assume to learn the effect of a continuous time process $a_t$, we need much more parameters, however, the paper said we use similar parameters, so would love to know more the size of the parameters. Maybe the proposed method can utilize more parameters better, while the other methods don't benefit from adding more parameters.", "extra_scores": {}, "extra_text": {}}, {"review_id": "esRvuQBUglL", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a new method, called augmented counterfactual ordinary differential equations (ACODs), to do counterfactual inference on time series data in healthcare. This is done by modelling interventions in continuous time with differential equations augmented by auxiliary confounding variables to reduce bias. They demonstrate the proposed method on tumor growth simulation and sepsis patient treatment response. \n\nPros:\n+ The proposed method can predict how a continuous-time time series will evolve under a sequence of interventions. \n+ The proposed method is naturally constructed upon the neural ODE framework by additionally modelling the effects of incoming interventions. \n\nConcerns:\n- The major concern in this paper is that the authors did not deal with the identifiability issue of the proposed method for counterfactual inference. It is widely acknowledged that identifiability plays a pivotal role in counterfactual inference, without which any counterfactual quantities cannot be guaranteed. In particular in the methods based on neural networks, it is extremely difficult to address the identifiability issue. In light of this, instead it might be a better idea to provide some counterfactual bounds (Pearl, 2009) for the quantities of interest. \n- From the experimental results, it seems that the proposed method does not have an obvious advantage in comparison with the baselines. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "UjVZaRthubG", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\nSummary:\n \nIn this manuscript, the authors propose a novel way of performing counterfactual inference in time-series in the presence of hidden confounders. For this, they employ neural ODEs as a latent time-series model, which they augment with additional latent variables. They test their approach on synthetic and real-world data and demonstrate improved performance in comparison to the state-of-the-art.\n\n##########################################################################\nReasons for score: \n \nOverall, I vote for weak rejection (5) in its current form. This is mainly due to the following reasons:\n1.\tMarginal theoretical contribution \n2.\tAnalysis of the method is lacking. How critical is the introduction of dynamic variables for the performance? How much is the performance increasing by only increasing the number of parameters of a vanilla neuralODE? How significant are the results, e.g., in figure 2? \n\n##########################################################################\nPros: \n1.\tThe paper is well and concisely written and is understandable to a broad audience.\n2.\tThe authors apply an interesting model to an interesting problem. \n3.\tTo the best of my knowledge, neuralODEs have not been applied to counterfactual inference before, but it is hard to keep track of those two fast-evolving fields (neural ODEs and counterfactuals). I believe, however, that this approach is very promising. The idea of introducing additional dynamic variables to neuralODEs to account for the bias introduced by hidden confounders is also novel. \n4.\tThe presence of clusters in the inferred auxiliary variable states is an exciting finding.\n \n##########################################################################\nCons: \n1.\tThe theoretical contribution is marginal. The authors only slightly extend the model without further studying the implications of the model. \n\n2.\tI would have expected to see the following analysis: \nA)\tAuxiliary states vs. history-dependent (non-Markovian) neuralODE\nB)\tAuxiliary states vs. additional parameters of neuralODE\nThis is due to the following reasons: A) In prior work, a similar approach has been employed in various prior works to model hidden confounders, e.g. [Nodelman, U., Shelton, C. R., & Koller, D. (2012). Expectation-Maximization and Complex Duration Distributions for Continuous-Time Bayesian Networks. UAI], or see “Mori-Zwanzig” formalism, in order to model non-Markovian behavior. I would have liked to see some discussion on auxiliary variables vs. non-Markovian neuralODEs (f(x,u,t) vs f(x_1,…x_t,t))?. What is the better approach? To introduce auxiliary variables into the neuralODE, or to extend the neuralODE also to have the time-series’ history as input? B) My kneejerk reaction is that decreased bias and increased variance can be expected when introducing auxiliary variables, solely because this increases the number of parameters of the neuralODE. I would suggest performing a dedicated experiment that allows for disentangling the number of parameters from the presence of auxiliary variables.\n3) Further, the results in fig. 2 should include the variance of results. The curves are all close together, and it would be interesting to see whether the improvement is significant.\n4) How would I determine k in practice?\n##########################################################################\nQuestions during the rebuttal period: \n\nPlease address and clarify the cons above. \n \n#########################################################################\nSome typos: \n•\tAlgorithm 1,2,3 “ODESlove”\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: There are some interesting ideas raised on continuous-time models with latent variables in machine learning. However, the reviewers argue, and I agree, that the connection to causal models as typically required in applications about the effects of interventions is not addressed with as much care as it might have been needed.
Decision: Reject