SyncTwin: Transparent Treatment Effect Estimation under Temporal Confounding
Authors: Zhaozhi Qian
Yao Zhang
Ioana Bica
Angela Wood
Mihaela van der Schaar
Venue:  (2021)
Keywords: treatment effect
interpretability
healthcare
causal inference

Abstract: Estimating causal treatment effects using observational data is a problem with few solutions when the confounder has a temporal structure, e.g. the history of disease progression might impact both treatment decisions and clinical outcomes. For such a challenging problem, it is desirable for the method to be transparent --- the ability to pinpoint a small subset of data points that contributes most to the estimate and to clearly indicate whether the estimate is reliable or not. This paper develops a new method, SyncTwin, to overcome temporal confounding in a transparent way. SyncTwin estimates the treatment effect of a target individual by comparing the outcome with its synthetic twin, which is constructed to closely match the target in the representation of the temporal confounders. SyncTwin achieves transparency by enforcing the synthetic twin to only depend on the weighted combination of few other individuals in the dataset. Moreover, the quality of the synthetic twin can be assessed by a performance metric, which also indicates the reliability of the estimated treatment effect. Experiments demonstrate that SyncTwin outperforms the benchmarks in clinical observational studies while still being transparent.

Under review as a conference paper at ICLR 2021
SYNCTWIN: TRANSPARENT TREATMENT EFFECT ESTIMATION UNDER TEMPORAL CONFOUNDING
Anonymous authors Paper under double-blind review
ABSTRACT
Estimating causal treatment effects using observational data is a problem with few solutions when the confounder has a temporal structure, e.g. the history of disease progression might impact both treatment decisions and clinical outcomes. For such a challenging problem, is desirable for the method to be transparent - the ability to pinpoint a small subset of data points that contribute most to the estimate and to clearly indicate whether the estimate is reliable or not. This paper develops a new method, SyneTwwin, overcome temporal confounding in transparent way. SyncTwwi estimates the treatment effect of a target individual by comparing the outcome with its synthetic twin, which is constructed to closely match the target in the representation of the temporal confounders. SyncTwin achieves transparency by enforcing the synthetic twin only depend on the weighted combination of few other individuals in the dataset. Moreover, the quality of the synthetic twin can be assessed by a performance metric, which also indicates the reliability of the estimated treatment effect. Experiments demonstrate that SyncTwwin outperforms the benchmarks in clinical observational studies while still being transparent.
INTRODUCTION
Estimating the causal individual treatment effect (ITE) on patient outcomes using observational data (observational studies) has become a promising alternative to clinical trials as large-scale electronic health records become increasingly available (Booth & Tannock 2014). Figure [illustrates a common setting in medicine and it will be the focus of this work (DiPietro) 2010): an individual may start the treatment at some observed time (black dashed line) and we want to estimate the ITE on the outcomes over time after the treatment starts (shaded area). The key limitation of observational studies is that treatment allocation is not randomised but typically influenced by prior measurable static covariates (e.g. gender, ethnicity) and temporal covariates (e.g. all historical medical diagnosis and conditions, squares in Figure TT. When the covariates also modulate the patient outcomes, they lead to the confounding bias in the direct estimation of the ITE (Psaty et al. 1999).
Although a plethora of methods overcome the confounding bias by adjusting for the static covariates YYonn et al. 2018 Yao al., 2018
Louizos et al., 2017: Shalit et 20.| [i & Fu [017]] Alaa &
Outcomes
van der Schaar 2017 Johansson al. 2016), few existing works
lake advantage of the temporal covariates that are measured irreg-
ularly over time (Figure 1 (Bica al.] 2020 Lim et al.) 2018
Schulam & Saria 2017 Roy et al. 2017p. Overcoming the con- founding bias due to temporal covariates especially important for medical research as clinical treatment decisions are often based on the temporal progression of a disease. Transparency highly desirable in such challenging problem.
Treatment
Temporal Covariates
Time
Figure Illustration of treated individual. Yellow dots represent the outcomes under no treatment.
Although transparency a general concept, we will focus on two spe- cific aspects (Arrieta et al. 2020). (1) Explainability: the method should estimate the ITE of any given individual (the target individual) based on a small subset of be quantified (e.g using weight
other individuals (contributors) whose amount of contribution between 0 and 1). Although the estimate of different target individuals may depend on different contributors, the method can always shortlist the few contributors for the expert to understand the

---

Under review as a conference paper at ICLR 2021
1. Encode
2. Synthetize
3. Estimate Synthetic twin
[N]
Target individual
[Sp]
Covariates
ITE
(treated)
Latent Variable
Unobserved Confounder
Contributors
(control)
Og
Vi
TET
UT
Covariates
Rep. Vectors
Weights
Outcomes
Time
Outcomes
Figure 2: A: Illustration SyneTTwin (shaded area: time points after the treatment starts). 1. Temporal
covariates are encoded as representation vectors. 2. The synthetic twin of a treated target individual is constructed as the weighted average of the few contributors from control group. 3. The difference hetween observed outcome and the synthetic twin outcome estimates ITE. B: the DAG of the data generating model (Sec. 2
rationale for each estimate. (2) Trustworthines:: the method should identify the target individuals whose ITE cannot be reliably estimated due to violation of assumptions, lack of data, other failure modes. Being transparent about what the method cannot do improves the overall trustworthiness because guides the experts only use the method when it deemed reliable.
Inspired by the well-established Synthetic Control method in Statistics and Econometrics (Abadie al., 2010 Abadie 2019), we propose SyncTwwn, transparent ITE estimation method which deals with temporal confounding. Figure A illustrates the schematics of SyncTTwin. SyncTwwin starts by encoding the irreularly-measune target individual, SwnnTwwn selects and weights few contributors from the control group based on their representation vectors and the sparsity constraint. Sync proceeds to construct a synthetic twin whose representation vector and outcomes are the weighted average of the contributors. Finally, the ITE estimated as the difference between the target individual's and the Synthetic Control's outcomes after treatment. The difference in their outcomes before treatment indicates the quality of the synthetic twin and whether the model assumptions hold. If the target individual and synthetic twin do not match in pre- treetmeetett outcomes, the estimated ITE should not be considered trustworthy.
temporal covariates as representation vectors. For each treated
Transparency of SyncTwin. SyncTwin achieves explainabiliiyy by selecting only a few contributors for each target individual. It achieves trustworthiness because quantifies the confidence should put into the estimated ITE as the difference between the target and synthetic pre-treatment outcomes.
2 PROBLEM SETTING
We consider clinical observational study with N individuals indexed by i [N] (1...., N). Let E {0, 1) } be the treatment indicator with a; = 1 if i started to receive the treatment at some time and a; = if never initiated the treatment. We realign the time steps such that all treatments were initiated at time
0} be the set of the treated and the control respectively. Denote No = [Zol and N₁ = [T, as the sizes of the groups. The time = 0 is of special significance because it marks the initiation of the treatment (black dashed line in Figure D. We call the period < 0 the pre-treatment period and the period > the treatment period (shaded area in Figure
0. Let I (i E [N] a; 1} and Io = (i E [N]
Temporal covariates are observed during the pre-treatment period only and may influence the treatment decision and the outcome. Let X, [xs))=((S)) be the sequence of covariates X R which includes S, N observations taken times E T {tts { where all tis E R and tis < 0. Note that X may also include static covariates whose values are constant over time. To allow the covariates to be sampled different frequencies, let m,, {0, 1) be the masking vector with Misd = 1 indicating the 7th element in Xi, is sampled.
The outcome of interest is observed both before and after the treatment. In many cases, the researchers are interested in the outcomes measured at regular time intervals (c.g. the monthly average blood pressure). Hence, let T- =(- {-M, M,..., -1} and T (0... ..., H 1} be the observation times
2

---

Under review as a conference paper at ICLR 2021
before and after treatment initiation. In this work, we focus real-valued outcomes Yar E IR observed at t E T UT+. We arrange the outcomes after treatment into a H H-dimensional vector denoted as y; bYseETT E BR.". Similarly define pre- treattettent outcome vector y, = 1tskkT-- E R.
Using the potential outcome framework (Rubin 2005), y,r (a:) E IR denote the potential outcome at time in a world where i received the treatment as indicated by a,. Let y;(1) = a (1))eT E RR and y (1)= = [y E TRM, similarly for (x) and y; (0). The individual treatment effect (TTE) defined as T; = (1(1) - (0) E TR. Under the consistency assumption (discussed later in details), the factual outcome is observed y:(a;) y,, which means for any E [N] only the unobserved counterfactual outcome y, (1 a.) needs to be estimated in order to estimate the ITE. To simplify the notations, we focus on estimating the ITE for the treated, i.e. i y;(() (0) for
i E Z, though the same approach applies to the control E Io and new units $ [N] without loss of generality (A.5).
SyncTwwn relics on the following assumptions. (1) Consistency, also known as Stable Unit Treatment
Value Assumption (Rubin 1980); y,,(a,) Y Vi E NV. t E T T+. (2) No anticipation,
(0), also known as causal systems (Abbring & Van den Berg. 2003 Dash 2005): Yır = =yx(1
vt T-_i [N]. (3) Data generating model: the assumed directed acyelic graph is visualized B PPaal 2009). where we introduce two variables c, E _K and v; E IR" in addition to the previously defined ones. The latent variable c; is the common cause of Yae and Xis, and it indirectly influences
in Figure 2
through Xi.- As we show later, SyncTTwin tries to learn and construct synthetic twin that has the same C₁ as the target. The variable V; is an unobserved confounder. Although SwncTTwin, like all other ITE methods, works better without unobserved confounders (i.c. v; = 0, Vi E [N]. we develop a unique checking procedure in Equation (4) to validate there exists V; * O. We also demonstrate that under certain favourable conditions, SyncTTwin can overcome the impact of the v,, To establish the theoretical results, we further assume (0) follows a latent factor
model with C, V; as the latent "factors" (Bai & Ng. 2008
y. = q. c, + u. v. + Ex. UTT
(1)
where q: E U, UV weight vectors and Eu is the white noise. We require the weight vectors to have | T UT+ (Xu) 2017), which does not reduce the expressiveness of the model. We further require the dimensionality of the latent factor be smaller than the number of time steps before or after treatment, i.e. K < min(M. H). Furthermore, let Q = [4 E RRAX and IR. "x denote the matrices that stack all the weight vectors q s' before and after treatment as rows respectively. The latent factor model assumption may seem restrictive but as we show in Appendix (A.4fi is applicable to many scenarios. In the simulation study (3.I) we further show Sync'Twin performs well even when the data is not generated using model (T) but instead using a set of differential equations. We compare our assumptions with those used in the related works in Appendix A.3
Q =
RELATED WORK
3.1 SYNTHETIC CONTROL
Similar to Sync Twin, Synthetic control (SC) (Abadie 2019) and its extensions (Athey et al. 2018
Amjad et al. 2018) estimate ITE based on Synthetic Control outcomes. However, when applied to temporal confounding, SC will flatten the temporal covariates [xtlee[[s) into a fixed-sized (high- dimensional) vector x, and use it to construct the twin. As a result, SC does not allow the covariates to be variable-length or sampled at different frequencies (otherwise x, 's dimensionality will vary across individuals). contrast, SyncTwin can gracefully handle these irregularities because it constructs the twin using the representation vectors. Moreover, the covariates x; may contain observation noise and other sources of randomness that do not relate to the outcome or the treatment. Enforcing the target and the twin to have similar X, will inject these irrelevant noise to the twin, a situation we call over-match (because it resembles over-fit). Over-match undermines ITE estimation as we show in the simulation study in Section 5.1 Finally. SC assumes (O) q x, + u, v; + Eve. i.e. the flattened covariates x; linearly predicts Yzr (0), which is a special case of our assumption [ and unlikely to hold for many medical applications.
3

---

Under review as a conference paper at ICLR 2021
3.2 COVARIATE ADJUSTMENT WITH DEEP LEARNING
In the static setting, the covariate adjustment methods fit two functions (deep neural networks) to predict the outcomes with and without treatment i.e. ya(0) [x(x; and (1) = fi (xx) (Johansson
et al. 2016; Shalit et al.] 2017] Yao et al., 2018 Yoon et al. 2018). The ITE is then estimated
as f2 = (x., - fo(x.) ). Under this framework. various methods have been proposed address
temporal confounding (Lim ct al. 2018 Bica et al. 2020). However, these methods generally lack transparency because the black-box neural networks cannot easily pinpoint the contributors for each prediction. Moreover, the prediction accuracy before treatment cannot directly measure the confidence or trustworthiness for the predictions after treatment because the network very nonlinear and non-stationary. Lastly, Bica et al. (2020) and Lim et al. (2018) are applicable to more general setting where the treatment can be turned on and off over time whereas SyneTTwin assumes the outcomes will continue to be influenced by the treatment after the treatment starts.
Works with similar terminology. Several works in the literature use similar terms such as "twin" while most of them are not related to SwncTwin. We discuss these works in Appendix A.6
4 TRANSPARENT ITE ESTIMATION VIA SYNCTWIN
To explain when and why SyncTTwin gives valid ITE estimate, let us assume that we have learned
a representation è; that approximates the latent variable c,, Vi E [N] in Equation 1 For target individual i E IT let b; = [b,gljEz, R. be vector of weights, each associated with control individual. A synthetic twin can be generated using b; as
é = Σ bu JEIJ
(a) = Σ be) = Ebaa VETTUTY
(2)
jEI
JEI
where è; is the synthetic representation and (0) is the synthetic outcome under no treatment. The last equality follows from the consistency assumption. Let (0) = [8 (0))eTT+ be the post- treatment synthetic outcome vector, and similarly y, = ())(e) The ITE of i can be estimated
as
+ = y:(1 - (0 = y: Σ biyys:
JEZ
where again the last equality follows from the consistency assumption. We should highlight that y; and yj. Vj E Io in the equation above are the observed outcomes. Hence, b; is the only free parameter that influences the ITE estimator îz. The following two distances are central to the training and inference procedure:
== Ed|l,
- - yi yT
where || is the vector norm and || the vector -norm.
Minimizing di to construct synthetic twins. df indicates how well the synthetic twin matches the target individual representations. Intuitively, we should seek to construct a twin who closely matches the target by minimizing df. This intuition is verified in Proposition (proved in A.1.1).
Proposition 1 (Bias bound on ITE with no unobserved confounders). Suppose that v, = O, Vi E [N] and de = 0 for some i I (v; and df are defined in Equation [ and respectively), the absolute value of the expected difference in the true and estimated ITE of is bounded by:
E Birs]l < I II1 Σ bicc call < T
Σ JET
||c, -,يl + ||c, a:ll)
(5)
JEIn
Here show that when de is minimized at zero and there is unobserved confounder, the bias on the ITE estimate only depends on how close the learned representation č is to the true latent variable We will use use representation learning uncover the latent variable in the next section.
Using d to measure trustworthiness. By definition, dă indicates how well the synthetic pre- treatment outcomes match the target individual's outcomes. Intuitively, matching the outcomes before treatment is prerequisite for a good estimate the ITE after treatment (Equation and 3). We formalize this intuition in Proposition 2, which is proved in Appendix A.I.T
4

---

Under review as a conference paper at ICLR 2021
Proposition 2 (Trustworthiness of SyncTwwn under hidden confounders). Suppose that all the outcomes are generated by the model in Equation D with the unobserved confounders equal to zero s.t. V, 0. vi E [N], and that we reject the estimate fz if the pre-treatment error de on T is larger than 1/1++ ,, the post-treatment ITE estimation error on T+ is below 6.
Here we show that if we would like to ensure the ITE error to fall below a pre-specified threshold we should reject the estimate f; when the distance dØ > T 1/|T* assuming no unobserved confounder. In other words, di can be used as an evaluation metric to access whether the estimated ITE trustworthy.
Situation with unobserved confounders. In presence of the unobserved confounders v; # O, SyncTwin cannot guarantee to correctly estimate the ITE. However, dī can still indicate whether v, has significant impact on the pre-treatment outcomes, i.e. the unobserved confounders may exist but only weakly influence the outcomes before treatment. We discuss unobserved confounders in detail in Appendix 1.2
4.1 LEARNING TO REPRESENT TEMPORAL COVARIATES
In this section, we show how SwnnTTwin learns the representation è; as a proxy for the latent variable C using sequence-fo-sequence architecture depicted in Figure 3 (A) and discussed below.
Architecture. Sync'Twin is agnostic to the exact choice of architecture as long as the network translates the covariates into single representation vector (encode) and reconstructs the covariates from that representation (decode). For this reason, we use the well-proven sequence-to-sequeee architecture (Seq2Seg) (Sutskever al., 2014) with encoder similar to the one proposed in Bahdanau et al. 2015) and a standard LST decoder
Moanddeeooooeeeeee (Hochreiter & Schmidhuber 1997).
The encoder first obtains sequence of representations at each time step using a recurrent neural network. Instead of using the bi-directional LSTM as in Bahdanau et al. (201)). we use a GRU-D network because
designed encode preguuaarll--sampled temporal observations (Che al. 2018). This gives us the sequence h, = GRU-D(h,, X;s, mis: 4,), Vs E [S.]. Since our goal to obtain a single representation vector rather than sequence of representations, we aggregate the sequence of h,s using the same attentive pooling method as in Bahdanau et al.) (2015). The final representation vector C is obtained as: č₁ = E.e(s,] Qig where =r h //K is the attention weight and E R the attention parameter (Vaswani et al. 2017).
The decoder uses the representation è; to reconstruct Xis at time tıs Vs E [St]. Since the timing information tis may be lost in é due to aggregation, we reintroduce it the decoder by first obtaining a sequence of time representations
Kn + Wo lis. where Ois, ko, Wo E KR and then concatenating each with è; to obtain: e;s ē; @ Ois E RR2K" Reintroducing timing information during decoding is a standard practice in Seq2Seq models for irregular time-serics (Rubanova et al.) 2019 Li & Marlin| 2020). Furthermore, using time representation O, instead time values t. inspired by the success of positional encoding in the self-attention architecture (Vaswani et al. 2017 Gehring et al.) 2017). .. The decoder then applies a LSTM autoregressively on the time-aware representations C₁2 decode g., LS.M(g..-1). c. Vs [S,], where gis RRK. Finally, it uses linear layer to obtain the reconstructions: X; = k, W Wiss. where k, RD, W, E R
Loss functions. We train the networks with the weighted sum of the supervised loss C, and the reconstruction loss L, (Figure A):
C.(D))
Σ iEDo
D)) Σ Σ
[|( x m (6)
SED +E[S.]
where Do S Io. D1 I m, is the masking vector (Section 2). represents element-wiss product and Q R l a trainable parameter and || I|| is the L2 norm. Intuitively, the supervised loss L. ensures that the learned representation è; to be a linear predictor of the outcomes under no treatment (0). Here a linear function (0) == Q غ is used to be consistent with the data generating model M. Using nonlinear function here might lead smaller L. but it will not uncover the latent variable c; desired. We justify the supervised loss in Proposition 3 below and present the proof and detailed discussions in Appendix Appendix/A.1.I]
Proposition 3 (Error bound on the learned representations). Suppose that v; = 0, Vi E [N] (v; is defined in Equation 1). the total error on the learned representations for the control, i.e., the first term
5

---

Under review as a conference paper at ICLR 2021
(A) Training
(B) Inference optimization step M TT)
(0
yj (0)
Di
Do
Encoder
X
(X Mj T5)
Encoder
Decoder
Dr
X3 M, T:)
Ci
(X, M,, Tr)
D
L
Encoder
Decoder
B
Weights
C
Loss
Figure 3: Illustration the loss functions. (A) The representation networks are trained using and in Equation Note that the supervised loss L, only applies to the control. (B) Validation and inference involve
optimizing the matching loss L. in Equation Note the encoder needs to be fixed during optimization.
in the upper bound of the absolute value of the expected difference in the true and estimated ITE bounded as follows:
(R.H.S of Equation 5
Σ
1c - 5 3C.
Σ 11
JEZn
jEZn
where Cg the supervised loss in Equation 6 and Ey the white noise in Equation
4.2 CONSTRUCTING SYNTHETIC TWINS
Constraints. We require the weights b, in Equation to satisfy two constraints (1) positivity: bij 2 2'vi E [N], j E Io. and (2) sum-to-one: Σjer, bi = 1, vi E [N]. The constraints are needed for three reasons. (1) The constraints reduce the solution space of b, and serve as a regularizer. Regularizing vital because the dimensionality of b, R. can easily exceed ten thousand in observational studies. (2) The constraints encourage the solution to be sparse by fixing the L -norm of b, be one i.c. |b./l1 transparency. (3) Finally, the constraints ensure that the synthetic twin in Equation is the weighted average of the contributors. Therefore the weight b,, directly translates into the "contribution" or "importance" of to i, further improving the transparency.
(Tibshirani] 1996). Better sparsity leads fewer contributors and better
Matching loss. The matching loss finds weight b, so that the synthetic twin and the target individual match in representations, as depicted in Figure (B).
Cm Di) = Σ
||
Σ b,rū
(8)
IED
JED.
where again D, C Io and D1 C IT We use the GumbellSoftmax reparameterization detailed in Appendix A.9 optimize L. under the constraints (Jang et al. 2016 Maddison al., 2016).
4.3 TRAINING, VALIDATION AND INFERENCE
As is standard machine learning, we perform model training, validation and inference (testing) on three disjoint datasets. On high level, we train the encoder and decoder on the training data using the loss functions described in Section hyper-parametts of the encoder and decoder. Finally, we fix the encoder and optimize the matching loss L,, on the testing data to find the weight b, which leads to the ITE estimate using Equation The detailed procedure is described in A.8 The hyperparamter sensitivity is studied in A.13
The validation data then used to validate and tune the

---

Under review as a conference paper at ICLR 2021
5 EXPERIMENTS
5.1 SIMULATION STUDY
In this simulation study, we evaluate SyncTTwi on the task of estimating the LDL cholesterol-lowering effect of statins, a common drug prescribed to hypercholesterolacmii patients. We simulate the ground truth ITE using the widely adopted Pharmacokinetic Pharmcokinetc--PPaaaadyyyy Pharmacodynamic model the literature (Faltaos et al. [0066] Yokote et al., 2008 Kim et al. 2011).
dp: dt
dd: dt
dye
de k Y+-
k
k- Pri Pi
h h-d:; dt
= k. de + doo
(9)
where y, is the LDL cholesterol level (outcome) and a, is the indicator of statins treatment. The interpretation of all other variables involved are presented in Appendix AA.
Data generation. Following our convention, the individuals are enrolled at 0, the covariates are observed in T = [-S,0), 0), where E (15, 25, 45J, and the ITE is to be estimated in the period T- = [0, 4] We start by generating k for each individual from the following mixture distribution:
ktt = g. fe: g: ò:en + (1 - 6:) 6, sid Bern(p);
üd N( E,,, n 1,2 (10)
ein
where ft E R are the Chebyshev polynomials, Bern(p) is the Bernoulli distribution with success probability p and (nn, Σn) the Gaussian distribution. To introduce confounding, we vary p for the treated and the control: of confounding bias. After that, the variables Pr. de, y, are obtained by solving Equation using scipy (Virtanen et al. 2020) and adding independent white noise N(0, 0.1) to the solution. The temporal variables defined above give the covariates irregular sampling by creating masks ma maa~Ber((n)) where probability m E (0.3, 0.5, 0.7,
Po. vi E To and p 1, Vi It, where po controls the degree
[k.". yy, Pt. de). Finally, introduce
Benchmarks. From the Synthetic Control literature, we considered the original Synthetic Control method (SC) (Abadie al., 2010), Robust Synthetic Control
SyncTwin
SC
(RSC) (Amjad et al. 2018) and MC-NNM (Athey al. 2018). From the deep learning literature,
Treated
Treated
compared against Counterfactual Recurrent Network (CRN) (Bica et al. 2020) and Recurrent Marginal Structural Network (RMSN) (Lim et al. 2018), which are the -tate-of-the-ar methods estimate ITE under temporal confounding. In addition, we included a modified version of the CERNet, which was origi- nally developed for the static setting (Shalit et al. 2017). To allow the CFRNet to handle temporal co- variates, we replaced its fully-connected encoder with the encoder architecture used by Swin (Section 4.1. We also included the counterfactual Gaussian Process (CGP) (Schulam & Sarial 2017) and One-netrest Neighbour Matching (INN) (Stuart) 2010) as baselines. The implementation details of all benchmarks are available in Appendix A.7 We also included two ablated versions of SyncTwwin. SyncTwin-C, is trained only with reconstruction loss and SyncTwn--C, only with supervised loss.
Control
Control
Figure 4: Heatmap of the weights b, learned by SyncTwwi and SC. Each row represents b;
Main results. We evaluate the mean absolute error (MAE) on ITE estimation: N, Σ.V. ||7, A//1- In lable@tth parameter Po controls the level of confounding bias (smaller Po. larger bias). Additional results for different sequence length S and sampling irregularity m are shown in Appendix 11 SyncTwin achieves the best or equally- performance in all cases. The full SyncTwin with both loss functions also consistently outperforms the versions trained only with L, or L.. As discussed in Section 4 (and Appendix I.I.I). training with only reconstruction loss L, leads to significant performance degradation. It worth highlighting that the data generating model used in this simulation (9) is not the same as SyncTwine Twin's assumed latent factor model II] This implies that SyncTwin may still achieve good performance when the assumed model II does not exactly hold.
SC, RSC and MC-NNM underperform because their assumption that the flattened covariates X; linearly predict the outcome is violated (Section B. Furthermore, Table shows the synthetic twin created by SC matches the target covariates x, consistently better than SyncTwine yet produces worse ITE estimates. This suggests that matching covariates better may not lead to better ITE estimate
7

---

Under review as a conference paper at ICLR 2021
Table I:
absolute error on ITE under different levels of confounding bias Po. m and S are used, Estimated standard deviations are shown in the parentheses. The best performer in bold.
No 200
No 1000
Method
= 0.1
0.25
0.5
= 0.1
Pb 0.25
Po 0.5
SyncTwin-Fuu Full
0.324 (.038)
0.144 (.012)
0.119 (.008)
0.141 (.012)
0.106 (.006)
0.093 (.005)
SyncTwin--e
0.353 (.039)
0.170 (.015)
0.139 (.010)
0.256 (,026)
0.145 (.012)
0.101 (.006)
SyncTwin-La
0.336 (.039)
0.170
0.120
0.144 (.012)
0.113 (.007)
0.127 (.010)
SC
0.340 (.041)
0.151 (.024)
0.149 (.018)
0.258 (.050)
0.166 (.034)
0.214 (.036)
RSC
0,837 (.044)
0.360 (.020)
0.321 ((018)
0.310 (.016)
0.298 (.014)
0.302 (.014)
MC-NNM
1,160 (.059)
0.612 (.03))
0.226 (.011)
0.527 (,029)
0.159 (.008)
0.124 (.006)
CFRNet
0,895 (.077)
0,411 (.037)
0.130
0,411 (.038)
0.075
0.106 (.007)
CRN
1.045 (.064)
0.546
0.360 (.024)
0.864 (.052)
0.767 (.040)
0.357 (.021)
RMSN
0,390 (.031)
0.362 (.028)
0.332 (.026)
0.447 (.041)
0.386 (.034)
0.385 (.032)
CGP
0.660 (.043)
0.610 (.039)
0,561 (.035)
0.826 (.056)
0.693 (.047)
0.602 (,038)
INN
1.866 (.099)
1.721 (.091)
1.614 (.078)
2.446 (.131)
1.746 (.106)
1.384 (.083)
Table 2: Mean absolute error between observed covariates x, and synthetic twin's covariates &, matches the covariates better yet produces worse ITE estimate (Table suggesting it is over-matching. The average distance between any two individuals is 0.95, much larger than all values reported in the table.
No 200
No 1000
Method
Po 0.1
Po 0.25
Po 0.5
Po 0.1
Po 0.25
po 0.5
SynllTwin-Full SyncTwin-Cr
0.343 (.029)
0.203 (.014)
0.179 (.011)
0.469 (.037)
0.223 (.015)
0.175 (.012)
0.321 (.028)
0.192 (.015)
0.182 (.015)
0.250 (.019)
0.190 (.013)
0.195 (.013)
SC
0,236 (.027)
0.117 (.014)
0.111 (.011)
0.155 (.025)
0.110 (.019)
0.128 (.020)
Outcome before after treetment
Treatment starts
(mmmolt))
Target Individual
The Synthetic Twin
Time (years)
Distance between treatment autcomes: dl
Number Individuals contributing synthetic
The median distance between the target individuel and the synthetic twin: 0.24
average individuais out 24557 total contribute to the synthetic twin
2000
3000
Frequency
11500-
2000
The median distance between
two individuo's 0.76
1000
1000
0
0.0
0.5
1.0
2.0
10
20
40
dr
Number of individuals
Figure 5; Illustration of the transparency of SyncTwin. Top: the outcomes (LDL) before and after treatment of a target individual and its synthetic twin. Bottom left: histogram of distance d" (Equation 4p. Bottom right: histogram of number of contributors used construct the synthetic twin.
because the covariates are noisy and method might over-match (Section 3). In addition, Figure 4 visualizes the weights b, of SyncTwwn and SC in a heatmap. We can clearly that SyncTTwin produces sparser weights because SC needs to use more contributors to construct the twin that (over- matches x;- Quantitative evaluation of the sparsity is provided in Appendix A.12 These findings verify our belief that constructing twins in the representation space (SyneTwin) rather than in the high-dimensional observation space (SC) leads to better performance and transparency.
8

---

Under review as a conference paper at ICLR 2021
5.2 EXPERIMENT ON REAL DATA
Purpose of study, We present an clinical observational study using SyncTwin to estimate the LDL Cholesterol-twwering effect of statins in the first year after treatment (Ridker & Cook 2013).
Data Source. We used medical records from English National Health Service general practices that contributed anonymised primary care electronic health records to the Clinical Practice Research Datalink (CPRD), covering approximately 6.9 percent of the UK population (Herrett et al. 2015). CPRD was linked to secondary care admissions from Hospital Episode Statistics, and national mortality records from the Office for National Statistics. We defined treatment initiation as the date of first CPRD prescription and the outcome of interest was measured LDL cholesterol (LDL). Known risk factors for LDL were selected as temporal covariates measured before treatment initiation: HDL Cholesterol, Systolic Blood Pressure, Diastolic Blood Pressure, Body Mass Index, Pulse, Creatinine, Triglycerides and smoking status. Our analysis is based on subset of 125,784 individuals (Appendix A.15) which was split into three equally- sized subsets for training, validation and inference, each with 17,371 treated and 24,557 controls.
Evaluation. We evaluate our models using the average treatment effect on the treated group (ie, ATT = E(Tiaa 1)) to directly correspond to the reported treatment effect in randomised clinical trials, e.g. The Heart Protection Study reported an a change of -1226 mmol/L
(SD=0.06) in LDL
cholesterol for participants randomised to statins versus placebo (Group et al.) 2007 2002). We use the sample average on the testing set to estimate the ATT as ΣiEDi Tit [D]] where DJ are the individuals in the testing set who received the treatment. Sync'Twin estimates the ATT to be -125 mmol/L (SD 0.01), which is very close to the results from the clinical trial. In comparison, CRN and RMSN estimate the ATT to be -0.72 mmol/L (SD 0.01) and -0.83 mmol/L (SD 0.01) respectively. Other benchmark methods either cannot handle tregularly-measune size of the dataset. Our result suggests SyncTwin is able to overcome the confounding bias in the complex real-world datasets.
covariates or do not scale to the
visualize the outcomes before and after the treatment and compare them with the synthetic twin in order to sense-check the estimate. The individual shown in Figure 5 (top) (top) has sensible ITE estimate because the synthetic twin matches its pre-treatment outcomes closely over time. In addition to visualization, we can calculate the distance d (Equation 4 to quantify the difference between the pre-treament outcomes. From Figure (bottom left) we can see in most cases the distance is small with median of 0.24 mmol/L (compared to the population average distance 0.76 mmol/L). This means if the expert can only tolerate an error of 0.24 mmol/L on ITE estimation, half of the estimates (those with d < 0.24 mmol/L. can be accepted (Section 4. The estimates are also explainable due to the sparsity of SyneTwin. As shown in Figure (bottom right) average only 15 (out of 24,557) individuals contribute to the synthetic twin.
Transparent ITE estimation. For each individual,
6 CONCLUSION
In this work, we present SyncTwwn, an transparent ITE estimation method that deals with temporal confounding and has a broad range of applications in clinical observational studies and beyond. Combining the Synthetic Control method and deep representation learning, SyncTwin achieves transparency and strong performance in both simulated and real data experiments.
REFERENCES
Alberto Abadie. Using synthetic controls: Feasibility, data requirements, and methodological aspects. Journal of Economic Literature, 2019.
Alberto Abadic and Javier Gardeazabal. The economic costs of conflict: A case study of the basque country. American economic review, 93(1):113 13-1 2003.
Alberto Abadie, Alexis Diamond, and Jens Hainmueller. Synthetic control methods for comparative case studies: Estimating the effect of california's 's tobacco control program. Journal of the American statistical Association, 1054450) 2010.
Jaap H Abbring and Gerard J Van den Berg. The nonparametric identification of treatment effects in duration models. Econometrica, 71(5): 1491111711 2003.

---

Under review as a conference paper at ICLR 2021
Ahmed M Alaa and Mihaela der Schaar. Bayesian inference of individualized treatment effects using multi-task gaussian processes. In Advances in Neural Information Processing Systems, pp. 3424-3432, 2017.
Muhammad Amjad, Devavrat Shah, and Dennis Shen. Robust synthetic control. The Journal of Machine Learning Research, 19(1):002-852 2018.
Alejandro Barredo Arrieta, Natalia Díaz-Rodriguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcfa, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58:82-115 115, 2020.
Susan Athey, Mohsen Bayati, Nikolay Doudchenko, Guido Imbens, and Khashayar Khosravi. Matrix completion methods for causal panel data models. Technical report, National Bureau of Economic Research, 2018.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, 2015.
Jushan Bai and Serena Ng. Large dimensional factor analysis. Now Publishers Inc, 2008
Ioana Bica, Ahmed M Alaa, James Jordon, and Mihacla van der Schaar. Estimating counterfactual treatment outcomes over time through adversarially balanced representations. In International Conference on Learning Representations, 2020.
CM Booth and IF Tannock. Randomised controlled trials and population-based observational research: partners in the evolution of medical evidence. British journal of cancer, 1 110(3)5511555 2014.
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):1-12, 2018.
Denver Dash. Restructuring dynamic causal systems in equilibrium. In AISTATS. Citeseer, 2005.
Barbra A Dickerman, Xabier Gareía-Albéniz, Roger W Logan, Spiros Denaxas, and Miguel A Hernán. Avoidable flaws in observational analyses: an application to statins and cancer. Nature Medicine, 25 16010161 2019.
Natalie A DiPietro. Methods in epidemiology: observational study designs. Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy, 30(10):973-984, 2010.
Dumitru Erhan, Pitrre-Antoine Manzagol, Yoshua Bengio, Samy Bengio, and Pascal Vincent. The difficulty of training deep architectures and the effect of unsupervised pre-training. Artificial Intelligence and Statistics, pp. 153-160, 2009.
Dumitru Erhan, Aaron Courville, Yoshua Bengio, and Pascal Vincent. Why does unsupervised pre-training help deep learning? In Proceedings of the Thirleenth International Conference on Artificial Intelligence and Statistics, pp. 201-208, 2010.
Demiana William Faltaos, Saik Urien, Valérie Carreau, Marina Chauvenet, Jean Sebastian Hulot, Philippe Giral, Eric Bruckert, and Philippe Lechat. Use of an indirect effect model to describe the ldl cholestenol-lowering effect by statins in hypercholesterolaemii patients. Fundamental & clinical pharmacology, 20(3):321-330 2006.
Steven E Finkel. Causal analysis with panel data. Sage, 1995.
Jared C Foster, Jeremy MG Taylor, and Stephen J Ruberg. Subgroup identification from randomized clinical trial data. Statistics in medicine, 30(24):8877-8880, 201
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. Convolutional sequence to sequence learning. arXiv preprint arXiv:1705.03/22 2017.
GPy. GPy: A gaussian process framework in python. http: com/ SheffieldMM/ GPy since 2012.
10

---

Under review as a conference paper at ICLR 2021
Heart Protection Study Collaborative Group et al. Mrc/bhf heart protection study: randomised trial of cholesterol-lowering with simvastatin in 20,536 high-risk individuals. Lancet, 360(9326):7-22) 2002.
Heart Protection Study Collaborative Group et al. Randomized trial of the effects of cholesterol- lowering with simvastatin peripheral vascular and other major vascular outcomes in 20,536 people with peripheral arterial disease and other high-risk conditions. Journal of vascular surgery, 45(4))645-654, 2007.
Harshad Hegde, Neel Shimpi, Aloksagar Panny, Ingrid Glurich, Pamela Christie, and Amit Acharya. Mice vs ppea: Missing data imputation in healthcare. Informatics in Medicine Unlocked, 17: 100275, 2019.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. In Advances in Neural Information Processing Systems, pp. 15663-15674 2019.
Emily Herrett, Arlene M Gallagher, Krishnan Bhaskaran, Harriet Forbes, Rohini Mathur, Tjeerd Van Staa, and Liam Smeeth. Data resource profile: clinical practice research datalink (cprd). International journal of epidemiology, 44(3):827-836, 2015.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint 1611v:1611.00 144, 2016.
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In International conference machine learning, pp. 0000002000000 3000000009,,-0 2016.
Fredrik Johansson, Nathan Kallus, Uri Shalit, and David Sontag. Learning weighted representations for generalization across designs. arXiv preprint 1xXiv:1802.08855/ 2018.
Nathan Kallus. Deepmatch: Balancing deep covariate representations for causal inference using adversarial training. arXiv preprint arXiv: 05664, 2018.
Jimyon Kim, Byung-Jin Ahn, Hong-Seok Chae, Seunghoon Han, Kichan Doh, Jeongeun Choi, Yong K Jun, Yong W Lee, and Dong-Seok Yim. A population Marmaaokinetic-hhaaacodyyi model for simvastatin that predicts low- densienit iipoproeii-cholesteeee reduction in patients with primary hyperlipidaemia. Basic & clinical pharmacology & toxicology, 109-3):156-163, 2011.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.
Cheng-Xias -ian Steven Li and Benjamin M Marlin. Learning from irregularly time series: A missing data perspective. International Conference on Machine Learning. PMLR, 2020.
Sheng Li and Yun Fu. Matching balanced nonlinear representations for treatment effects estimation. In Advances Neural Information Processing Systems, pp. 929-939, 2017.
Bryan Lim, Ahmed M Alaa, and Mihaela van der Schaar. Forecasting treatment responses over time using recurrent marginal structural networks. In Advances Neural Information Processing Systems, pp. 7483-7493, 2018.
Christos Louizos, Uri Shalit, Joris Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. In Advances in Neural Information Processing Systems, pp. 6446-6456, 2017.
Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint 1oX 2016.
Judea Pearl. Causality. Cambridge university press, 2009.

---

Under review as a conference paper at ICLR 2021
Bruce M Psaty, Thomas D Koepsell, Danyu Lin, Noel S Weiss, David S Siscovick, Frits R Rosendaal, Marco Pahor, and Curt D Furberg. Assessment and control for confounding by indication in observational studies. Journal of the American Geriatrics Sociery, 47(6):749-7544 1999.
Paul M Ridker and Nancy R Cook. Statins: new american guidelines for prevention of cardiovascular discase. The Lancet, 382(9900))117221177 2013.
Jason Roy, Kirsten J Lum, and Michael Daniels. A bayesian nonparametric approach to marginal structural models for point treatments and continuous survival outcome. Biostatistics, 18(1): 32-47, 2017.
Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. Latent ordinary differential equations for prregularly-smmpler time series. In Advances in Neural Information Processing Systems, pp. 5320-5330, 2019.
Donald Rubin. Randomization analysis of experimental data: The fisher randomization test comment. Journal of the American Statistical Association, 75(371):591-555 1980.
Donald B Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100x469):322-331, 2005.
Peter Schulam and Suchi Saria. Reliable decision support using counterfactual models. In Advances in Neural Information Processing Systems, pp. 1697-1708, 2017.
Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: general- ization bounds and algorithms. In International Conference on Machine Learning, pp. 3076-3085 PMLR, 2017.
Elizabeth A Stuart. Matching methods for causal inference: review and a look forward. Statistical science: a review journal of the Institute of Mathematical Statistics, 25(1):1, 2010.
Ilya Sutskever, Oriol Vinyals, and Quoc Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp. 3104-3112, 2014.
Timo Teräsvirta, Dag Tjøstheim, Clive William John Granger, al. Modelling nonlinear economic time series. Oxford University Press Oxford, 2010.
Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1))267-288 1996.
Madeleine Udell and Alex Townsend. Nice latent variable models have log-rank. arXiv preprint arXiv:1705.07474, 1, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, CJ Carey, Îlhan Polat, Yu Feng, Eric W. Moore, Jake Vand erPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, lan Henriksen, E. A. Quintero, Charles R Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272, 2020. doi:
Yiqing Xu. Generalized synthetic control method: Causal inference with interactive fixed effects models. Political Analysis, 25(1):57-76, 2017.
Liuyi Yao, Sheng Li, Yaliang Li, Mengdi Huai, Jing Gao, and Aidong Zhang. Representation learning for treatment effect estimation from observational data. In Advances in Neural Information Processing Systems, pp. 2633-2643, 2018.
12

---

Under review as a conference paper at ICLR 2021
Koutaro Yokote, Hideaki Bujo, Hideki Hanaoka, Masaki Shinomiya, Keiji Mikami, Yoh Miyashita, Tetsuo Nishikawa, Tatsuhiko Kodama, Norio Tada, and Yasushi Saito. Multicenter collaborative randomized parallel group comparative study of pitavastatin and atorvastatin in japanese hyperc- holesterolemic patients: collaborative study on hypercholesterolemiia drug intervention and their benefits for atherosclerosis prevention (chiha study). Atherosclerosis, 201(2):345-352, 2008
Jinsung Yoon, James Jordon, and Mihaela van der Schaar. Ganite: Estimation of individualized treatment effects using generative adversarial nets. In International Conference on Learning Representations, 2018.
13

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 THEORETICAL RESULTS
A.1.1 SITUATION WITH NO UNOBSERVED CONFOUNDERS
Proposition 1 Bias bound on ITE with no unobserved confounders. Suppose that = 0, vi e [M] and de 0 for some i I (v, and di are defined in Equation I and respectively), the absolute value of the expected difference in the true and estimated ITE of is bounded by:
[ B[ri]| S S
Σ JEZO
buc
c,ll S TT
Σ jEIp
lc, 8,رl + ||c -0 -cil)
(1I)
Proof.
We start the proof by observing
E
B(r,]
Σ
E (0)
E[y
TE
Σ
q
bijc c;)
r
JELL
(12)
Σ
||
Σ b c,ll
1ET+
jEIn
|T+II
big c:ll
jEZn
where the first equation follows from the definition of ITE in Section 4 The second equation follows
from Equation D and 2 together with the fact that v, = 0, Vi E [MT. The third line follows from
Sehhwa- inequality. The fourth line uses the fact that ||q:]] = 1. By definition, d, = 0 implies Σ bijC; = c. Continuing the proof,
11 Σ bije call = Σ
by ē2) - (c. a, -g)
JEI
JEIn
<
Σ
bi - -ē + ||c - d:1l
(13)
jEZp
Σ
||c, - اارl + ||c. c||,,
JEZ0
where the second line follows from the triangular inequality and the third line relies on ΣjEI, bij we prove the inequalities in Equation
1
and bij- 2 O, Vj E To. Combining inequality 12 and
13
Justification for the matching loss and df. Proposition I presents justification for minimizing df (or the matching loss L.). Essentially, when the synthetic representations are matched with the target (df 0), the bias in ITE estimate is controlled by how close the learned representations is to the true latent variable e up to an arbitrary linear transformation A. An important implication that the learned representation does not need be equal to c, instead the learning algorithm only needs to identify up to linear transformation. Of course, Proposition | also implies that [E[A.] [[r]] < ΣjEto c, ا, + ||c. - -,,l when taken to be the identity matrix instead of the minimizer.
Proposition 2 Trustworthiness of SyncTwwi under no hidden confounders. Suppose that all the outcomes are generated by the model in Equation I with the unobserved confounders equal to zero s.t. = 0, vi E [N], and that we reject the estimate f, if the pre-treate treettetmeett error de on T is larger than I/IT+ + the post-treatment ITE estimation error on T is below 8.
Proof. As a reminder, Q = /q.ITT- and Q = |qikeT+ denote the matrix that stacks all the weight vectors q's before and after treatment as respectively where each q satisfies that ||g, 1 in
14

---

Under review as a conference paper at ICLR 2021
Equation 1 noise error.
The enror de in Equation can be decomposed into representation and white
d
18; yr 11
Σ b - -y; 11
JEZn
Σ
c + + (up(Te
(Q + E.)l1
jETo
||Q
biy c))l1 +1| Σ
byjš Ex) lla
(14)
JETo
Σ
||lll||
Σ
bijc - -cil +
Σ
DigEy - &ill
TET
JEIo
JEZo
bi cill +
Σ
bigEy - Eill
JE
JETO
We can not estimate the error from the representation and white noise on the last line of Equation 14 Conservatively, we can say the representation error itself is larger or equal to d such that
ITI
Σ jEZO
buc
c: d.
i.e.,
byc cill 2 d
(15)
JEIn
The post-treatment error is upper bounded as follows,
[Bf. - B|Ti]| = [0)] - [0)]
Σ 1
bije - c:)l
IE
jEIn
< T
|II Σ
bijC - c.ll
jEZn
sup
E
E(Ti]]-
E
Using Equation
15
we have
sup [
E [T.]] 2 d |T+ 1/1T
Conservatively, we reject the estimate 7 sup; [[ E [T:
is larger than 8. That is when
d > 8 1/
Why does d indicate the trustworthiness of the estimation? Proposition 2 shows that we can control the estimation error be below certain threshold by rejecting the estimate if its error d' during the pre-treatment period is larger than [TT- Alternatively, we can rank the estimation trustworthiness for the individuals based on d' alone. This helpful when the user willing to accept a percentage of estimations which are deemed most trustworthy. We note that this proposition only holds under the assumption that the outcomes over time are generated by the model stated in Equation [ representation. However, the model assumes that the outcomes over time are linear functions of the same representation. This is the reason why the pre-treatment error can be used to assess the post-tred ses-treetment error. We parameterize our neural network model according to Equation T it is a not good fit the data, the model should have large estimation error before treatment. The users should also use their domain knowledge to check if the model holds for their data, i.c., if there is any factor starting affect the outcomes in halfway and causes the representation change time.
The outcomes generated by such model can be nonlinear and complicated due to the
15

---

Under review as a conference paper at ICLR 2021
Proposition 3 Error bound the learned representations. Suppose that v; = 0, Vi [N] (v; is defined in Equation [ the total error on the learned representations for the control, i.., the first term in the upper bound of the absolute value of the expected difference in the true and estimated ITE (R.H.S of Equation 11 is bounded as follows:
1c, - s 3Ls
||
(16)
JEZo
JEIo
where Cg is the supervised loss in Equation and Ey is the white noise in Equation
Proof. We start the proof from the definition of the supervised loss.
C,
[Qē, - (0)l
JEIn
Σ
[Qēē - (Qc, + E,)I
jETn
(17)
1) -c,
Σ
[
-c]]
qt
[ qi
Cj
Σ
116/11
[9t]
jETn
MET
JETO
2
|
Σ JEZp
||ss|l
JETp
where 3 denotes the square root of the element of the matrices
q
q
Vt E T_, with the
q:
smallest absolute value. The first and second equations follow from Equation 6 [nnd] [ Let 8 denotes
the constant 1/(3v[T-1). Arranging the terms in inequality 17
and we prove Proposition
Justification for the supervised loss. Proposition B provide justification for the supervised loss E. By optimizing the supervised loss, SyncTTwin learns the representation è₁ that is close to the latent variable c,, which also reduces the bias bound on ITE in Proposition
Rationale for the reconstruction loss. Although the bias bounds we developed so far do not include the reconstruction loss Le, we believe is useful in real applications. Our reasoning follows from the fact that unsupervised or semi-supervised loss often improve the performance of deep neural networks (Erhan al., 2009; 2010, Hendrycks et al., 2019). addition, the reconstruction loss ensures the representation č retains the information from the temporal covariates as required in the DAG (Figure 2). our simulations (Section(5.] [.TI). we found that ablating the reconstruction loss leads to consistently worse performance (though the magnitude is somewhat marginal).
Can we estimate the ITE as A; -1) Q ē.? No, this is because Lg is based on the factual outcome (0) of the control group E To only. For treated individuals i I,, the predictor Q c; can be biased for their counterfactual outcomes (0). Hence, C. only used to learn a good representation é; for downstream procedures, and not to directly predict counterfactual outcomes.
A.1.2 SITUATION WITH UNOBSERVED CONFOUNDERS
In general, the unobserved confounders make it hard to provide good estimates for the ITE. The matching in pre-enrollment outcomes d (Equation 4h validates if the unobserved confounders v; create significant enror in the pre-treatment period. Using the same derivation of Theorem we can see that:
d
||Q
Σ byc, -;) +U)
by v,) + Elb
(18)
JEIn
JEI.
where Q and U- are unknown but fixed matrices relating to the data generating process and is term only depending on the white noise.
As shown in Proposition [ the matching in representations encourages the first term involving c; to be small. Hence, a large value in d implies that the remaining term involving the unmeasured confounders v, is big, which leads to large estimation error. It is worth pointing out that small
16

---

Under review as a conference paper at ICLR 2021
value of de does not guarantee there is no unobserved confounders hypothesis cannot test empirically. For instance, consider the weights U = O. It follows that the second term in Equation there exists unobserved confounders but they do not impact the outcomes before treatment (Equation 1. In summary. dx does not prove or disprove the existence of unobserved confounders; it only indicates their impact on the pre-treatment outcomes. Our assumption is small relaxation of the standard no unmeasured confounders assumption by allowing linear effect from some unmeasured confounders. More conservatively, we can assume there is no unmeasured confounders by setting all the v; to O, vi E [N] in Equation
18 will always be zero even if v; * 0 -
A.2 COMPARISON OF THE TEMPORAL COVARIATES ALLOWED IN THE RELATED WORKS
As introduced in Section SyncTTwin is able to handle temporal covariates sampled at different frequencies, i.e. the set of observation times T; and mask m2 can be different for different individuals. In comparison, Synthetic Control (Abadie et al. 2010), robust Synthetic Control (Amjad et al. 2018), and MC-NNM (Athey et al. covariates, i.e. 7, -2, -L} vi E [N. and my = Vi E [N], E T,, In other words, the temporal covariates [<.s) X,
2018) are only able to handle regularly-sampled
R PXS: has matrix form.
The deep learning methods including CRN (Bica et al. 2020) and RMSN (Lim et al. 2018) have the potential to handle tregulaaly-mcasureed used. However, the architectures proposed in the original papers only apply to regularly-sample case and no simulation real data experiments were conducted for the more general irregular cases.
measured variable-length covariates when suitable architecture is
A.3 COMPARISON OF THE CAUSAL ASSUMPTIONS IN THE RELATED WORKS
Table 3: Comparison the causal assumptions in the related works. The definitions Consistency, Sequential overlap, and No unobserved confounder are given in A.3 in bold. The data generating model (D.G.M) in Equation contains the one in Equation 9 as special case.
Approach
Ref
Consistency
D.G.M
Sequential Overlap
No unobserved conf.
SC
Abadie 2019
Yes
Equation
19
RSC
Amjad et al.
20188
Yes
Equation
19
MC-NNM
Athey el a
2018
Yes
Equation
19
CRN
Bica ct al.
(022
Yes
Yes
Yes
RMSN
Lim ct al.
2018
Yes
Yes
Yes
SyncTwwn
This work
Yes
Equation
A.3.1 SYNTHETIC CONTROL
As shown in Table 3 Synthetic control (Abadie et al. 2010 Abadie 2019) and its variants (Athey et al. 2018 Amjad et al. 2018) rely on two causal assumptions: (I) consistency: (a(a)) Yu. and (2) data generating assumption (linear factor model):
(19) where x; vec(X.) E IR D vec is the vectorization operation; u, E R and qu E Dx varying variables and v, E R. is a latent variable. Ear is an error term that has mean zero and satisfies Eú IL ars; X,, u, V, for V k,r,,5 s,t.
(0) q x, + v. Vi [N], TET UT+
is worth highlighting that the data generating assumption of Synthetic Control is a special case of the more general assumption of SyncTwin in Equation [ To see this, let c, x; vec(X,) in Equation [ i.e. we use the flattened temporal covariates directly as the representation. Further let do 6o(C;, las) c; Ds D(s 1)) and 6is where c(a b) takes slice of vector c. between index
a and b. The result is exactly Equation 19
Why does Synthetic Control tend to over-match? Both SyneTwii and Synthetic Control estimate the treatment effects using a weighted combination of control outcomes (Equation 3 However, Synthetic Control finds weight by in a different way by directly minimizing
Cx =||x
Σ bir,,
17

---

Why does SyncTwin not explicitly require overlap? The overlap assumption is commonly made in treatment effect estimation methods. We first give very brief review of why two importance classes of methods need overlap. (1) For methods that rely on propensity scores, overlap makes sure that the propensity scores are not zero, which enables various forms of propensity weighting. (2) For methods that rely on covariate adjustment, overlap ensures that the conditional expectation IFl; a:] is Under review as a conference paper at ICLR 2021
Since X; contains the observation noise and other random components that do not relate to the outcomes, the weights bij that minimize Lg tend to over-match, i.c. they capture the irrelevant randomness in x, In contrast, SyncTTwin finds biy based on the learned representations rather than x, (Lm. Equation 6). Since é; has much lower dimensionality than X,, the reconstruction loss L, encourages the Seq2Seq network learn é; that only retains the signal in X; but not the noise. Meanwhile, the supervised loss encourages è; to only retain the information that predicts the outcomes. As a consequence, we expect the weights based on C; to be less prone to over-match. Moreover, since the relationship between C; and X, is nonlinear (as captured by the decoder network), the weights bi that minimize f. will generally not minimize the Synthetic Control objective Lz. therefore avoiding over-match.
A.3.2 COUNTERFACTUAL RECURRENT NEURAL NETWORKS
As shown Table 3 CRN (Bica et al. 2020) and RMSN (Lim et al. 2018) makes the following
three causal assumptions. (1) Consistency: Yı (ax) yu- (2) Sequential overlap (aka. positivity):
Pr(au = 2-,, Xar) > whenever Pr Xa) + 0. (3) No unobserved confounders: (0), (1) 11. as Xit, ar,t-1- In summary, CRN makes the same consisteney assumption as
SyncTwwnn However, SyncTwwin does not assume sequential overlap or no unobserved confounders
while CRN does not make assumptions on the data generating model.
The sequential overlap assumption means that the individuals should have non-zero probability to change treatment status at any time 2 0 given the history. This assumption is violated in the clinical observational study setting we outlined in Section [ where the treatment group will continue to be treated and cannot switch to the control group after they are enrolled (and similarly for the control group). While the sequential overlap assumption allows these methods to handle more general situations where treatment switching do occur, their performance is negatively impacted in the "special" (yet still widely applicable) setting we consider in this work.
While CRN makes strict memobserved-coofoundee assumption, SyncTwwin allows certain types of unobserved confounders to occur. In particular, the latent factor v; in Equation Can be unobserved confounders. Being less reliant on b--unobserved-confounnde assumption is important for medical applications because it is hard to assume the dataset captures all aspects of the patient health status. SyncTwin' 's ability to handle unobserved confounders V; relies on the validity of its data generating assumption, which we discuss next.
well-defined, i.c. the conditioning variables (X,,a.) a.) have non-zero probability. In comparison, SyncTwwn relies on neither the propensity scores nor the explicit adjustment of covariates, and hence it does not make overlap assumption explicitly. However, as discussed in Proposition I SyneTwwin requires the synthetic twin to match the representations de ~ O, which implies è₂ ~ Σjer, Ōirū; for some b.j- - the target individual should be in or close to the convex hull formed by the controls in the representation space. This condition has similar spirit to overlap (but very different mathematically)) When overlap is satisfied there tend to be control individuals in the neighbourhood of the treated individual, making easier to construct matching twins. Conversely, if overlap is violated, the controls will tend to far away from the treated individual, making it harder to construct good twin.
A.4 THE GENERALITY OF THE ASSUMED DATA GENERATING MODEL
SyncTwwi assumes that the outcomes are generated by a latent factor model (Teräsvirta et al. 2010) with the latent factors c learnable from covariates X, and the latent factors v, that are unobserved confounders. We assume the dimensionality of c; and v; to be low compared with the number of time steps. Despite its seemingly simple form, the assumed latent factor model is very flexible because the factors are in fact latent variables.
The latent factor model is widely studied in Econometrics. In many real applications, the temporally observed variables naturally have a low-rank structure, thus can be described as a latent factor model (Abadie & Gardeazabal 2003. Abadie et al., 2010). The latent factor model also captures many
18

---

Under review as a conference paper at ICLR 2021
of well-studied scenarios as special cases (Finkel 1995) such as the conventional additive unit and time fixed effects (y,x(0) = 9. + c.). Last but not least, It has also been shown that the low-rank latent factor models can well approximate many nonlinear latent variable models (Udell & Townsend 2017].
Latent factor models in the static setting are very familiar in the deep learning literature. Consider deep feed-forward neural network that uses a linear output layer to predict some real-valued outcomes y E RD in the static setting (notations used in this example are not related the ones used in the rest of the paper). Denote the last layer of the neural network as h-1 E IR KKx; it is easy to see that latent factor model i.e. y Ah-1 + b, where h- is the latent
the neural network corresponds factor. Note that this holds true for arbitrarily complicated feed-forwar networks as long as the output layer is linear.
A.S ESTIMATING ITE FOR CONTROL AND NEW INDIVIDUALS
We have been focusing on estimating ITE for treated individual E Zi. The same approach can estimate the ITE for a control individual without loss of generality. After obtaining the representation c for i Ty. SyncTwii can use the treatment group E In to construct the synthetic twin by optimizing the matching loss Equation g The checking and estimation procedure remains the same.
SyncTwwin can also estimate the effect of a new individual i [N.. The same idea still applies, but this time we need to construct two synthetic twins: one from the control group and one from the treatment group. The ITE estimation can be obtain using the difference between the two twins.
SyncTTwin also easily generalizes to the situation where there are A > 1 treatment groups each receiving a different treatment. In this case, the treatment indicator a, [0, ,,.... A]. For target individual in any of the treatment groups, SynnTTwin can construct its twin using the control group To. The remaining steps are the same as the single treatment group case.
A.6 UNRELATED WORKS WITH SIMILAR TERMINOLOGY
Several recent works in the deep learning ITE literature employ similar terminologies such as "matching" (Johansson et al. 2018 Kallus 2018). However, they are fundamentally different from SyncTwin because they only work for static covariates and they try match the overall distribution of the treated and control group rather than constructing synthetic twin that matches one particular treated individual.
The Virtual Twin method (ootter et al. 2011 is designed for randomized clinical trials where there is no confounding (temporal or static). As result, cannot overcome the confounding bias when the problem is to estimate causal treatment effect from observational data.
A.7 IMPLEMENTATION DETAILS OF THE BENCHMARK ALGORIHTM
Synthetic control. We used the implementation of Synthetic Control in the R package Synth (1.1-5). The package is available httptttp///ccctt R- R-prooject org wwtp/acaaaeeeasnncc..cc.cc.c
Robust Synthetic Control. We used the implementation actp:/m with the original paper (Amjad
al. 2018) at fnttpps gtttbb..tbbb..t//// com/ Sucre _ststscchttttthllll_aeaaoa control We optimized the hyperparameters on the validation set using the method described in Section 3.4.3 Amjad et al. (2018).
The best hyperparameter setting was then applied to the test set.
MC-NNM. We used the implementation in the R package So ft Impute (1.4) available at ht 7/CRAN. prrooeettt wwwooor//a... httg:/accaaaessss ft Impute] The regularization strength À is tuned on validation set using grid search before applied to the testing data.
Counterfactual Recurrent Network and Recurrent Marginal Structural Network. We used the implementations by the authors BBca et al. (2020); Lim et al. (2018) at httpt::////b tbukket] are e master/ The networks were trained on the training dataset. We experimented different hyper- paremeerr settings on the validation dataset, and applied the best setting to the testing data. We also found that the results are not sensitive to the hyperparameters.
brg/mvdschaae htt////foooaaahhaaammma
19

---

Under review as a conference paper at ICLR 2021
Counterfactual Gaussian Process. We used the implementation with GPy GGP since 2012), which able to automatically optimize the hyperparameters such as the kernel width using the validation data.
One-nearest neighbour. We used our own implementation. Since no parameters need to be learned tuned, the algorithm was directly applied on the testing dataset.
Search range of
1. Synthetic control: hyperparameters are optimized by Synth directly.
2. Robust Synthetic control: num_sc (1,2,3,4,5}
3. MC-NNM: C (3,4,5,8, 10)
4. Counterfactual Recurrent Network: max_alpha E (0.1, 0.8, 1), hidden dimension H
{32, 64, 128}
5. Recurrent Marginal Structural Network: hidden dimension H {32, 64, 128]
6. Counterfactual Gaussian Process: hyperparameters are optimized by GPy directly,
A.8 DETAILED TRAINING, VALIDATION AND INFERENCE PROCEDURE
As is standard in machine learning, we perform model training. validation and inference (testing) on three disjoint datasets, Dfr, D and D We use D and D to denote the control and the treated in the training data and use similar notations for validation and testing data.
Training. On the training dataset D we learn the representation networks by optimizing L = X, XpCs, where L, and L, are the loss functions defined in Equation 6 The hyperparameter X, and A, controls the relative importance between the two losses. We provide an ablation study in Section and perform detailed analysis hyperparameter importance in Appendix AA.13 The objective " can be optimized using stochastic gradient descent. In particular, we used the ADAM algorithm with learning rate 0.001 (Kingma & Ba 2014).
Validation. Since we never observe the true ITE, we cannot evaluate the error of ITE estimation, ||: 12 As a standard practice (Bica et al. 2020). we rely on the factual loss on observed outcomes: L = EE (0) (:) where (0) is defined as in Equation and obtained
as follows. We obtain the for all i D and then optimize the matching loss L., (28 DD to find weights b.dh It is important to keep the encoder fixed throughout the optimization; otherwise might overfit to Dea, Finally, (0) EEDDE b (0).
Inference. The first steps of the inference procedure are the same as validation. We start by obtaining the representation è; for all i Dee and then obtain weights by= by optimizing the matching loss L. (Dbs, D(*) while keeping the encoder fixed. Using weights b the ITE for any E DJ can be estimated as 6 -1) Σjeri bijy,, (i5y,(0)) (0) according Equation 3. Similarly, obtain C, y (0) according to Equation 2 The expert can check dy to evaluate the trustworthiness of 7.
Table 4: Parameters for each component of the architecture and the loss function for training each parameter.
Component
Parameters
Loss function
Reference
Attentive Encoder
GRU-D parameters.
L. C
Section
Decoder
LSTM parameters, ko, wo. k1, W1
Section
Linear outcome prediction Weights
Q
L.
Section 4.
Lm
Section 4.2
A.9 OPTIMIZING THE MATCHING LOSS
8 To ensure the three constraints discussed in Section 4.2 while also allowing basedd learning algorithm, reparameterize b, = Gumbel-Softmant fm (z.). r), where z; E No, f.. is a masking function that sets the element žμ Inf satisfy constraint (3). Gumbel-SSftmax((- is the Gumbel softmax function
Here we present a way to optimize the matching loss L. in Equation
20

---

Under review as a conference paper at ICLR 2021
Algorithm 1: SyncTTwin training procedure.
Input: Training data set: Di Dfr
Input: Hyperparameters: Ar, App
Input: Encoder, Decoder, Q
Input: Training iteration maz itr, batch size batch size, Optimizer
Output: Trained Encoder, Decoder and Q
Randomly initialize Encoder and Decoder; set Q
for itr E (0, maz itr] do
Randomly draw mini-batch of control units Do C De with batch_size samples.
Randomly draw mini-batch of treated units D1 C Dir with batch_size samples.
Evaluate training loss C (Do. D1) = C, D1) + XpC,(Do) (defined in Equation
Calculate the gradient of f (Do, D1) via back propagation.
Update all encoder, decoder parameters and Q using the Optimizer
Algorithm 2: SyncTTwin inference procedure.
Input: Testing data set: Dic Die
Input: Trained Encoder
Input: Training iteration mar_itr. batch size batch_size, Optimizer
Output: Estimated ITE F,, Vi E Dí
Initialize size (DIT1 by (DD"1 matrix B 0 as the weight matrix.
Use Encoder to get representation č; vi e D U U for ilr E (0, max itr] do
Randomly draw mini-batch of treated units D1 C D with batch_size samples Evaluate matching loss C., (Dfc D2) (defined in Equation 8
Calculate the gradient of L.n (DDe D1) via back propagation.
Update B using the Optimizer while keeping the Encoder fixed.
Use weight matrix B to obtain f; Vi Dfe using Equation 3
21

---

Under review as a conference paper at ICLR 2021
with temperature arper-parameter T (Jang al. 2016 It is straightforward to verify that b₃ satisfies
the three constraints while the loss Lm remains differentiable with respect to Zg. We use the Gumbel softmax function instead of the standard softmax function because Gumbel softmax tend to produce sparse vector by. which is highly desirable as we discussed in Section 4
The memory footprint to directly optimize 2m is (((Pal + (Pil) x (Pol), which can be further
reduced O (Dol) if we use stochastic gradient decent with a mini-batch DB S Do U U....
A.10 THE SIMULATION MODEL
In Equation 9] Re is the LDL cholesterol level (outcome) and It is the dosage of statins. For each individual in the treatment group, one dose of statins (10 mg) is administered daily after the treatment starts, which gives dosage 4 0 if < tu and I, otherwise. K, H and D50 constants fixed to the values reported in Faltaos et al. (2006). KPn E R is individual-specifii time varying variable that summarizes a individual's 's physiological status including serum creatinine, uric acid, serum creatine phosphokinase (CPK), and glycaemia. Pt and Dr are two intermediate temporal variables both affecting Rr.
A.11 ADDITIONAL SIMULATION RESULTS
Table 5: Mean absolute error ITE with varying irregular m. S = 25 and Po 0.5 are used in all cases. Estimated standard deviations shown in the parentheses. The best performer is in bold. * did not finish within 48h.
No 200
No 1000
Method
m = 0.7
m = 0.5
m = 0.3
771 = 0.7
m = 0.5
m = 0.3
Twin-Full
0.129 (.008)
0.142 (.009)
0.190 (.012)
0.109 (.006)
0.116 (.006)
0.141 (.008)
SyncTwin-- Cr
0.158 (.012)
0.176 (.014)
0.245 (.017)
0.125 (.007)
0.133 (.008)
0.175 (,011)
SyncTwin-C.
0.129 (:009)
0.152 (.010)
0.234 (.016)
0.139 (.009)
0.134 (.009)
0.172 (.012)
SC
0.155 (.017)
0.201 (.015)
0.326
0.145 (.015)
0.215
0.359 (.026)
RSC
0,414 (.021)
(.028)
0.639 (.043)
0.495 (.028)
*
MC-NNM
0.363 (.020)
0.556 (.031)
0.898 (.050)
0.174 (.010)
0.332 (.02))
0.556 (.036)
CFRNet
0.317 (.030)
0.303 (.018)
0.481 (.030)
0.143 (.009)
(.012)
0.255 (,018)
CRN
0.300 (.020)
0.364 (.026)
0.424
0.416.00277
0.456.0020
0.677 (,040)
RMSN
0,327 (.025)
0.338 (.025)
0.391 (.026)
0.381 (.031)
0.400
0.471 (.031)
CGP
0.568 (.037)
0.553 (.037)
0.631 (.045)
0.605 (.039)
0.626 (.039)
0.689 (.044)
INN
1.584 (:080)
1.725 (.098)
1.703 (.096)
1.455 (.083)
1.680 (.088)
1,531 (.089)
Table 6: Mean absolute error on ITE under different lengths the temporal covariates S. m and pa are used in all cases. Estimated standard deviations are shown in the parentheses. The best performer is in bold. did not finish within 48 hours.
No 200
No = 1000
Method
S 15
S 25
S 45
S = 15
S 25
= 45
SynnTTin-Full
0.123 (.008)
0.119 (.008)
0.112 (.007)
0.097 (.005)
0.093 (.005)
0.085 (.004)
SyncTwin-Cr
0.136 (.010)
0.139 (.010)
0.139 (.010)
0.114 (.006)
0.101 (.006)
0.098 (.006)
SC
0.139 (.018)
0.149 (.018)
0.138 (.021)
0.190 (.029)
0.214 (.036)
0.215 (.044)
RSC
0,348 (.023)
0.321 (.018)
(.011)
0.302 (.014)
MC-NNM
0,454 (.023)
0.226 (.011)
(.008)
0.139 (,007)
0.124 (.006)
0.109 (.005)
CFRNet
0.126 (.006)
0.130
0.143 (.008)
0.113 (,006)
0.106 (.007)
0.095 (.005)
CRN
0.283 (.019)
0.360
0.422 (.022)
0.387 (.024)
0.357 (.021)
0.426 (.023)
RMSN
0.331 (.024)
0.332 (.026)
0.399 (.024)
0.363 (.029)
0.385 (.032)
0.447 (.029)
CGP
0,561 (.036)
0,561 (.035)
(.035)
0.578 (.037)
0.602 (.038)
0.611 (.038)
INN
1.356 (.072)
1.614 (.078)
1.575 (.078)
1.322 (.072)
1.384 (.083)
1.744 (.098)
Table 5Jsooss the results under irregularly-maastne measured covariates with varying degree of irregularity m (smaller m, more irregular and fewer covariates are observed). For methods that are unable to
22

---

Under review as a conference paper at ICLR 2021
deal with irregular covariates, we first impute the unobserved values using Probabilistic PCA before applying the algorithms (Hegde et al. 2019). Syne'Twin achieves the best performance in all cases. Furthermore, SyncTwin's performance deteriorates more slowly than the benchmarks when sampling becomes more irregular (larger m). This suggests that the encoder network in Swnn is able to learn good representations even from highly tregularly-measunr results under various lengths of the observed covariates S (smaller S, shorter sequences are observed). Again SyncTTwin achieves the best performance in all cases, As expected, SyncTTwin makes smaller error when the observed sequence is longer. Note that this is not the case of CRN and RMSN- their performance deteriorates when the observed sequence longer. This might indicate that these two methods are less able to learn good balancing representations (or balancing weights) when the sequence is longer.
sequences. Table (6) shows the
A.12 SPARSITY COMPARED WITH SYNTHETIC CONTROL
In Figure we have shown visualy that Synn produces sparser solution than SC. To quantify the differences, report the Gini index Eig biy( (1 b1,) ()))N1), entropy (Ei bo,bbb,,gg((()//1 the number of contributors used construct the twin CE 1[b, > 0)/N1)) in the simulation study. All three metries reflect the sparsity of the learned weight vector (smaller more sparse). Table shoos that SyncTTwin achieve sparser results that SC in all metrics considered. The full and ablated versions of SwinTwwin have similar sparsity because the sparsity is regulated in the matching loss, which all versions share. It is worth pointing out that RSC and MC-NNM do not produce sparse weights and the weights do not need to be positive and sum to one (Amjad et al., 2018 Athey et al. 2018).
and
Table 7: Sparsity metrics of the learned b, Estimated standard deviations are shown in the parentheses. Here Pu = 0.5, m 1, S = 25. The worst performer is italicized
No 200
No
1000
Method
Gini
Entropy
N Control
Gini
Entropy
N Matched
Full
0.213 (.016)
0.409 (.030)
1.755 (.069)
0.242 (.017)
0.483 (.034)
1,830 (.073)
SyncTwin-Lr
0.214 (.017)
0.407 (.033)
1.780 (.075)
0.267 (.018)
0.548 (.037)
1.930 (.080)
SyncTwin-L.
0.213 (.016)
0.409 (.030)
1.760 (.068)
0,249 (.018)
0.500 (.037)
1.930 (.083)
SC
0.792 (.009)
1,871 (.035)
6,125 (.135)
0.862 (.006)
2.274 (.029)
7.059 (.110)
RSC
1.903 (.084)
2.311 (.964)
A.13 SENSITIVITY OF HYPER-PARAMETERS
It is beneficial to understand the network's sensitivity each hyper-parameter as effectively
optimize them during validation. In addition to the standard hyper-parameters in deep learning (e.g. learning rate, batch size, etc.), SyncTTwin also includes the following specific hyper-parameters: (1) T.
the temperature of the Gumbel-softmax function Appendix(A.9 [A.9 (2) Ap in the training loss ~ (since only the ratio between Ap and X, matters, we keep X, and search different values of Xp) and (3)
H, the dimension of the representation č;
Here we present a sensitivity analysis on the hyper- hyper-parameters H, Ap and 7 using the simulation
framework detailed in Section [..] Here we present the results for No = 2000 and S = 15 although these results generalize to all the simulation settings we considered. The results are presented in Figure where we can derive two insights.
Firstly, the hyper-parameter during validation. This is understandable because is the temperature parameter of the Gumbel softmax function and it directly controls the sparsity of matrix B. In comparison, hyper-parameter H and Xp do not impact the performance in significant way. Therefore we recommend to use H 40 and X = as the default.
very important to the performance and need to be tuned carefully
Secondly, we observe that the validation loss e closely tracks the error on ITE estimation (which is not directly observable in reality). These results support the use of ee to validate models and perform hyper-parameter optimization.
23

---

Under review as a conference paper at ICLR 2021
MAF
lamods
0.500
0.250
0.325
100
Figure 6: The sensitivity of loss defined in Section 4.33 The left panel shows the results for various choices of H; the middle panel shows Ap; and the right panel shows The y-axis is shown in log scale.
on the mean absolute error of ITTE estimation and the validation
A. 14 COMPUTATION TIME
420
Wal-cick
2008
Figure 7: The wall-clock time of the simulation study under different settings. For each setting, 10 independent simulation conducted. The har shows the average wall-clock time and the line range captures the 95% confidence interval.
In figure present the wall-clock computation time (in seconds) of SyncTTii under various simulation conditions - with the control group size No = (200, 1000, 2000) and the length of pre-enrollment period (15, 25, 45). The simulations were performed Core(TM) i5-8600K CPU @ 3. 3.60GHz and Nvidia(R) GeForce(TM) RTX 2080 Ti GPU. All simulations finished within 10 mins. As we expect, the computation time increases with respect to No and S as more data need to be processed. However, 10-fold increase in No only approximately doubled the computation time, suggesting that SyneTwin scales well with sample size. In comparison, S seems to affect the computation time more because the encoder and decoder need to be trained on longer sequences.
server with a Intel(R)
A.15 ADDITIONAL RESULTS IN THE CPRD STUDY
We the treatment and the control group in the CPRD experiment are selected based the selection criterion Figure We have followed all the guidelines listed in Dickerman et (2019) to make sure the selection process does not increase the confounding bias. The summary statisties of the treatment and control groups are listed below. We can clearly see selection bias as the treatment group contains much higher proportion of male and people with previous cardiovascular or renal diseases.
24

---

Under review as a conference paper at ICLR 2021
Table 8: The summary statistics treatment control groups
Treatment Group
Control Group
male
59%
51%
Median age
61
60
Median Townsend Index
8
8
% CVD
16%
9%
Renal disease
16%
12%
% Atrial Fibrillation
4%
4%
A.16 COHORT SELECTION CRITERION IN THE CPRD STUDY
25

---

Under review as a conference paper at ICLR 2021
Number of available People CPRD
17,251,881
Reason exclusion: Remove patients whose marked as unacceplabla, gender is not male or female, oulside England. 5,253,960
Number people 11,997,921
Reason for exclusion: Remeving prople who exited betore the start the study age 40, and who startd after the study exil age 65
Study entry date the
The date of months after the individusl
registered general practice
The date that the individual turned 30 years of age
The date that the data practioe to standard (UTS)
Study exist dite was the eariliest of:
The date deregistration the
The individual's dealh
The date that the individual turned 95 years of age
The last contect date the proactice with CPAD
The administrate end (Nov
= 9,368.221
Number of 2,619,700
Reason for exclusion: Remove palients with data quality issue.
people for whom
available;
people who have death recorded Office
National Statistics (ONS) before study entry:
people entry
N=30,626
Number of people 2,589,
Reason for exclusion: Removing people with prevalent cardiovascular discase (CVD) beafre sludy entry:
N=205.863
Number of people 2,363,191
Resson
Participants detected
exposures for SBP: total cholesterol, HDL, SBP, DBP, BMI, Pulse, Creatinine, Triglycerides smoking status between study enlry
Number people 125,784
2,257.0077
group 73,671
Treatment group 52,113
Figure 8: Flowehart for selection of eligible individuals from CPRD for the observational study the treatment effect of statins. Numbers represent unique individuals in each group.
26


Reviews:
[{"review_id": "ejV8jDumtVb", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper provides an approach for treatment effect estimation when the observational data is longitudinal (with irregular time stamps) and consists of temporal confounding variables. The proposed method can be categorized under the matching methods, in which, in order to estimate the counterfactual outcomes, a subset of the subjects in the opposite treatment arm (i.e., contributors) is selected and weighted. The proposed method is designed such that it achieves explainability (by identifying a few contributors) and trustworthiness (by checking if the estimated outcome is reliable).\n\nPros:\n- The paper presents a simple, yet reasonable solution to the task at hand. \n- The paper is easy to read and understand.\n- Figures were very helpful in understanding the paper; especially Figure 2.\n\nCons:\n- There are lots of design choices that are not motivated. Specifically, in section 4.1, the paragraph on Architecture includes too many design choices and heuristics that lack proper explanation and / or intuition. For example, it is unclear why there is need for a new time aware representation $o_{is}$ while we already do have one, i.e., $h_{is}$? Also, $o_{is}$ is mistakenly called a *projection* (in linear algebra terms).\n- There are many missing details that are crucial for understanding the algorithm. For example, in Equation (5), it is not stated which parameters are optimized by each loss. I’m guessing $h$, $r$, and $Q$ are found by $\\mathcal{L}_s$; and $\\[k_0, k_1\\]$, $\\[s-0, s_1\\]$, and $g$ are found by $\\mathcal{L}_r$; right? And how are they optimized? Do you alternate between the two objectives? How about $\\mathcal{L}_m$ in Equation (6)?\n- The authors claim in section 3.1, paragraph 1, lines -6 to -4 that their model does not over-match however, $\\mathcal{L}_r$ exactly does that.\n-To achieve trustworthiness, the authors state that they minimize $d_i^y$ in Equation (4); however, this term does not show up in any of the objective functions --- see Equations (5) and (6). Also, there should be $M$ values of $d_i^y$; are you aggregating these values into one (e.g., via averaging) or …?\n\n\nMinor:\n- Section 2, paragraph 2, line 3: $t_{is} < 0$ is only for pre-treatment samples. Here, the discussion is on both pre- and post- treatment.\n- Section 2, paragraph 5, line -4,-3: $v_i = 0$ and $v_i \\neq 0$, did you mean empty set $\\emptyset$ instead of $0$?\n- Section 3.1, paragraph 1, line 1: Synthetic Control should be title-case.\n- Equation (2), the subscript for $\\widetilde{c}$ should have been $j$, not $i$.\n- Section 4.3, paragraph 1, line -2: $M$ in $\\mathcal{L}_M$ should be lower-case.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Yi26qTuCX0L", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Summary: \nThe authors develop a method to estimate the effect of a static treatment on outcomes over time under temporal confounding, with an emphasis on making the method transparent. The authors define transparency as (1) the ability to explain the estimate for a given individual as a weighted compination of a small number of other individuals, and (2) trustworthiness meaning the model should be able to identify cases for which it cannot give a reliable estimate due to a violation of one of the main causality assumptions. \n\n## Main comments:\nI would significantly increase the rating if the authors address the following comments. \n \n(1) The authors tackle an interesting challenge. I am a bit concerned though that it trivially reduces to ITE estimation with a high dimensional outcome. It seems that the main novelty here is that the authors use a specific architecture that allows for varying lengths of pre-treatment variables, and hence confounders. My concrete questions are (a) suppose for a second that the outcome is measured for a single time period, why does this problem not reduce to the typical ITE estimation? Conditional on all the time varying confounders would simply be equivalent to the typical ITE estimation with a high dimensional set of pre-treatment variables. \n\n(2) The authors only model y(0) for individuals who received the treatment, and define the ITE to be the difference between the observed outcome and \\hat{y}(0), and similarly for control individuals. This means that in order to use this model i.e., in order to obtain the ITE estimate for a new test patient, we need to first observe the outcome under t = 1 or t = 0. This makes the model not useful for making treatment decisions: it would only be useful to validated that some chosen treatment choice was good/bad. Can the authors clarify what the intended use for this model would be?\n\n(3) d_i^y is simply a test of whether or not we're able to model the outcomes under t = 0. I do not follow the logical jump from \"there is a large estimation error\" to there is additional hidden confounding or the data generating model is not right. There are several other things that could cause that (e.g., the model can be well specified, and no hidden confounding but errors arise due to finite sample analysis). To be more concrete, equation 15 in A.1.2 assumes that \n(a) the SEM is correctly defined. \n(b) Q, U are correct\nEither of these might be violated in practice. In fact, they will likely be violated in practice. The flip side is true, one can get a small estimation error and still have hidden confounding. In general, the assumption of no hidden confounders is statistically untestable (meaning it is impossible to design a statistical test that will answer this question). Can the authors highlight what I am missing here?\n\n## Minor comments: \nAddressing these would be helpful but not consequential for the rating\n(1) The authors make a statement that in general the error in ITE should be bigger than the error in y --> this is not true, for example it does not hold in situations where there is a large bias in opposite directions for estimates in \\hat{y}(1), and \\hat{y}(0). My comment here assumes the authors are operating in the \"usual\" setting where we don't observe the treatment and hence the outcome under treatment for a new test case, and need to estimate the ITE by estimating both potential outcomes. \n\n(2) In eq 5 m_{is} is not defined anywhere\n\n(3) The authors don't make a clear case (empirically or in the writing) as to why their chosen approach is transparent. This b vector can still be very high dimensional even with an imposed sparsity. In a sense, all what the authors do with this b vector can be done using a simple kernel approach. Would the authors be able to explicitly state how is their model uniquely equipped for transparency?\n\n(4) It is somewhat odd to have a causal graph where there is no arrow between the treatment and the outcome. After doing a bit of mental gymnastics, I arrived at the conclusion that this graph is assuming do t= 0, which makes sense but is highly unconventional, and confusing. It would be helpful if the authors follow the conventional notation for causal graphs, which does not condition on a particular treatment assignment (e.g., see graphs in publications by Pearl, Eric Tchetgen Tchetgen, Ilya Shpitser, Tyler VanderWeele) \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "YbMd6HQ0SM9", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I'm going to keep this review short because I thought this was a really well-motivated and executed paper. The paper builds on the 'synthetic control' approach that is popular in econometrics which uses a weighted sum of the outcomes of individuals in the control group as an estimate of the control potential outcome, which can then be compared to a given treatment outcome to estimate the individual treatment effect. The step is the choice of weights: Sync twin uses the same prediction approach, but constructs the weights for the weighted sum using hidden representations from a recurrent network. This lets them work with irregularly spaced observations & missing covariates in a natural manner. \n\nMy only substantive complaint is in the framing of the paper: the introduction describes this temporal setting as more challenging than the static setting, but given that the paper is not about dynamic treatments (which are indeed far harder to deal with), the fact that you see multiple observations for any given individual makes the problem easier not harder... To be clear - this is not bad thing, but rather than saying, \"there are many methods for the static setting, but few are able to address temporal confounding\" (which isn't really true - these approaches could easily be adapted to the single treatment temporal setting using the appropriate recurrent architectures) - instead emphasize the fact that the static approaches don't take advantage of multiple observations from the same individuals. Multiple observations is a *stronger* assumption than the static setting---albeit a very natural one in this domain that you leverage for trustworthiness, etc.---so it shouldn't be presented as though its a weaker assumption / harder setting... \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "RszXAePFYc6", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- pros\n    - The problem is well motivated. Estimating the causal effect with time-series data is a practical and important problem. Transparency is a desirable property. \n    - The idea connects works in econometrics to modern ML. \n- cons\n    - High level points\n        - The paper proposes to use the number of donor units as a metric for explainability. It is not clear why this matric is good a priori. It is possible that the network selects an arbitrary set of units to construct the control.  \n        - For trustworthiness, It is quite confusing how synctwin is supposed to identify individuals whose ITE cannot be reliably estimated. This paper did not provide any formal procedures or guarantees on when identification is possible.\n        - Low rank is a key assumption for synthetic control. A discussion on whether this assumption is required for synthetic twins to work would be useful.\n        - The experiment section could use more clarification. In the synthetic experiment, the paper compares against a number of benchmarks (including robust synthetic control ) for model performance. However, to show that the method uses fewer contributors, the paper only compared against synthetic control, even though robust synthetic control is a more natural comparator, as it explicitly isolates a small subset of contributors during its data pre-processing.\n        - In the real world dataset, 1) while the train. test split is a common practice in ML, it doesn't generalize to inference. Namely, when the estimand is the ATT, we should not use just 1/3 of the data. 2) It is unclear how the ATT is measured.  Is it the average difference at the end of the trial? If so, it may not be a good metric to evaluate the time series model, since it's only the difference in the point estimate. \n    - minor details\n        - in figure 2, is there a missing arrow between the treatment and the outcome?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "LJAFzvESgGR", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I enjoyed reading this paper and think it tackles an important problem. The paper works with a particular model of the data generation and propose to do synthetic control in representation space which ensures that the considered data generation/model are not inflexible.\n\nI list a few issues here; they are mainly about the discussion of the assumptions.\n\nThere is very little discussion about identification guarantees in the paper. While A.1 provides some justification in this regard, I would've liked to see a paragraph about the identification guarantees for the proposed algorithm.\n\nIt seems to me that the functional linearity assumption of the model is what allows the method to not require sequential overlap. Could the authors expand on this?\n\nI believe that the key requirement of the method is the linearity of the control outcome in the latent c_i even when it is a non-linear function of the covariates; I think this helps the method not face serious non-identification issues. Could the authors clarify this intuition?\n\nI liked the idea of using the control outcome as a way to understand whether there is unobserved confounding as per the modelling assumption. The authors should however clearly state that no-unobserved-confounding is an assumption they also require to estimate effects; effect estimation is in generally impossible if v_i exists and is unobserved. \n\nI liked the experimental section also. The consistently better performance (even over the ablated models) provided some good evidence for the merit of the method. The real experiment (and the transperancy discussion) provided further strong evidence in favor of synctwin.\n\nFinally, it would've been nice to see one more synthetic experiment but seeing that the data generation considered in the paper is a biologically informed model (not a toy one), I'm happy with them for now.\n\nI'm happy to increase my score if the discussion around assumptions is sufficiently clarified.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: 
The paper developed a method that estimates treatment effect with
longitudinal observational data under temporal confounding. It extends
the idea of the synthetic control method and offers flexibility and
ease of estimation. However, some major concerns remain after the
discussion among the reviewers. In particular, the proposed method
lacks a clear use case. Moreover, some arguments around
``trustworthiness`` (detecting unreliable ITE estimates) and ``avoid
over-matching`` need to be refined. The error bound for
``trustworthiness`` can not detect hidden confounding. For overlap
issues, the rejection of units with larger error could be overly
conservative because the error bound may often be too loose. Regarding
``avoid over-matching``, while SyncTwin uses a low-dimensional
representation as opposed to the whole x vector for matching, it is
unclear whether SyncTwin can avoid over-match. It is possible that
using low-dimensional representation makes it easier to find a match
in the data and may still over-match. Finally the paper would benefit
from proper causal identification results.
Decision: Reject