Learning Treatment Effects in Panels with General Intervention Patterns
Authors: Vivek Farias
Andrew A Li
Tianyi Peng
Venue: NeurIPS 2021 Oral (2021)
Keywords: Causal Inference
Treatment Effect
Low-Rank Matrix Estimation
Panel Data
Synthetic Control

Abstract: The problem of causal inference with panel data is a central econometric question. The following is a fundamental version of this problem: Let $M^*$ be a low rank matrix and $E$ be a zero-mean noise matrix. For a `treatment' matrix $Z$ with entries in $\{0,1\}$ we observe the matrix $O$ with entries $O_{ij} := M^*_{ij} + E_{ij} + \mathcal{T}_{ij} Z_{ij}$ where $\mathcal{T}_{ij} $ are unknown, heterogenous treatment effects. The problem requires we estimate the average treatment effect $\tau^* := \sum_{ij} \mathcal{T}_{ij} Z_{ij} / \sum_{ij} Z_{ij}$. The synthetic control paradigm provides an approach to estimating $\tau^*$ when $Z$ places support on a single row. This paper extends that framework to allow rate-optimal recovery of $\tau^*$ for general $Z$, thus broadly expanding its applicability. Our guarantees are the first of their type in this general setting. Computational experiments on synthetic and real-world data show a substantial advantage over competing estimators. 

Learning Treatment Effects in Panels with General Intervention Patterns
Vivek Farias
Andrew A. Li Tepper School of Business Carnegie Mellon University Pittshurgh, PA 15213 aal 1@cmu. edu
Sloan School of Management
MIT
Cambridge, MA 02139
Divekf@mit..cm
Tianyi Peng Department of Aeronautics and Astronautics
MIT Cambridge, MA 02139 tianyi@mit, edu
Abstract
The problem of causal inference with panel data is a central econometrie question. The following matrix and E be zero-mean noise matrix. For 'treatment" matrix Z with entries in {0, 1} we observe the matrix O with entries Oig == Mi + Faj + Zi, where Ti are unknown, heterogenous treatment effects. The problem requires we estimate the average treatment effect T" := Eij TizZi// E.j Zij. The synthetic control paradigm provides an approach estimating 7" when Z places support on single row. This paper extends that framework to allow rate-optimal recovery of 7* for general Z, thus broadly expanding its applicability. Our guarantees are the first of their type in this general setting. Computational experiments on synthetic and real-world data show substantial advantage over competing estimators.
fundamental version of this problem: Let M be low rank
1 Introduction
Consider that we are given sequence of T observations each of distinct 'units'. In an econometric context, unit might correspond to geographic region with the associated sequence of observations corresponding to some economic time series of interest. In e-commerce, unit may correspond to customer, with the associated sequence of observations corresponding to site-visits for that customer over time. For each unit, some subset of its observations are potentially impacted by an intervention. This may correspond a new economic policy the econometric example, or the application of new type of promotion in the e-commerce context. The question hand is to estimate the "average treatment effect' of this intervention. This a core question in econometrics.
The problem above can be formalized as follows: Let M R=xT be fixed, unknown matrix and be a zero-mean random matrix; we refer M + E as the 'counterfactual" matrix with each row corresponding to distinct 'unit'. A known 'treatment' matrix Z with entries in {0, 1} encodes observations impacted by an intervention. Specifically, we observe a matrix O with entries Oi Mr + Fij + Tij Zij where the Tıj are unknown, heterogenous treatment effects Our goal is to recover the average treatment effect T" = Ei Ziz/ E.₃ Zij-
Observable covariates on each unit may also be available; we suppress this aspect here for clarity.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

The 'synthetic control' paradigm 2 has been transformative approach to addressing this problem. That setup addresses the special case of this problem where a single unit (say, the first row) is treated; i.e. Z has support on portion of the first row. The approach is conceptually simple: find linear combination of the untreated units (i.e. all rows other than the first) that approximate the first row on its untreated entries. One then uses the same linear combination to impute counterfactual values ie. the entires of M in the support of Z. Under the assumption that M is low rank (and other assumptions), one can show that this approach recovers r"
like the e-commerce setup alluded to earlier require we allow for more general inter- vention patterns, i.e. allow for general Z as opposed to the single row (or block) support required in synthetic control. A tantalizing possibility, first raised by [3] is treating entries of M in the support of Z as missing and applying matrix completion techniques to impute these counterfactual values. This viewpoint has the benefit of making no assumptions on the treatment effects and yields algorithms that do not require any special structure for Z. However, it is unclear that counterfactual recovery- which is equivalent to matrix recovery with general missingness patterns is actually possible via such a method, so that recovery guarantees are typically unavailable.
Applications
1.1 This Paper
Succinetly, we develop an estimator that recovers the average treatment effect under provably minimal assumptions on Z. This is made possible by de-biasing identity and an extension of the entry- wise uncertainty quantification analysis of 4 to general non-random missingness patterns that is of independent interest.
In providing context for our contributions, it worth asking what one can hope for in this problem. In addition to requiring that M have low rank (say r), it is clear that cannor in general expect to recover 7* absent assumptions on the treatment matrix Z. For instance, we must rule out the existence of a rank matrix M', distinct from M for which M' M +-Z for some y * 0, else identifying r" impossible even if E is identically zero (we could rule this out if Z were not in the tangent space of M"). Separately, unlike matrix completion, treated entires. If the heterogeneity in treatment effects is too large, however, it is unclear that these observations are of much value. Thus, in addition benign assumptions on M and E, we make set of assumptions on (A) the projection of Z onto the tangent space of M and (B) the heterogeneity in treatment effects. Against this backdrop, we make the following contributions:
actually observe M TT + Eij on
Rate Optimal Estimator: We construct an estimator that achieves rate optimal guarantees for the recovery of the average treatment effect 7* (Theorem I with general treatment patterns. We show under additional assumptions that our estimator is asymptotically normal (Theorem 2).
Minimality of Assumptions: Should the conditions we place on the projection of Z onto the tangent space of M be violated by an amount that grows small with problem size, we show no algorithm can recover -" even with homogeneous treatments (Proposition 2 Our assumptions on heterogeneity are also shown to be minimal, and satisfied by extant models in the synthetic control literature.
General Treatment Patterns Work:
We show that our assumptions on Z are satisfied by general treatment patterns, e.g., requiring that at least constant fraction of entries are not treated or matrices Z with rank polynomially larger than the rank of M
Empirical Performance: We show both for synthetic and real data that our estimator provides material improvement in empirical performance relative to available alternatives, including matrix completion based estimators and, where applicable, state-of-the-art synthetic control estimators.
1.2 Related Literature
The synthetic control literature pioneered by [ 2 has grown to encompass sophisticated learning and inferential methods; see [6] for review. techniques to learn the linear combination of untreated units that yields a synthetic control. [10) [1 I2 consider instead the use of principal component regression techniques.
9 consider variety of regularized regression
(13] propose alternative
approaches to imputing counterfactuals by averaging across both untreated units (rows) and time
(untreated columns). [14 155 address inferential questions that arise synthetic control with the latter providing a permutation test that is generally applicable.

---

Matrix completion methods present a means to allow for inference with general treatment patterns. 3 are among the first to study this, but provide no guarantees on recovering the average treatment effect. Alternatively methods that do provide guarantees on the recovery of treatment effects via matrix completion tend to make strong assumptions: [66 effectively assume that has support on block (so that traditional synthetic control techniques could also apply), [17] make stationarity assumptions on M and require it to be zero-mean and [18] assume that Z has i.i.d entries (wherein a trivial estimator of the average treatment effect is also applicable).
distinct (and common) setting for treatment effect estimation presents us with a single observation per unit and a rich set of observed covariates on these units. These observed covariates roughly ensure exchangeability of units across interventions within covariate strata. ML has been broadly used in this setting; eg. [99 21 In contrast for our problem, one could view these covariates as latent.
Our estimation procedure begins with first computing 'rough' estimate of the treatment effect via natural convex estimator; [22 23 24] are empirical studies that have uses this estimator. Crucially, we provide a new de-biasing technique that allows for recovery guarantees and exhibits significant performance improvement relative to this rough estimate. It is also worth noting that this convex estimator also finds application in the related problem of panel data regression; see 25 of the art methods there effectively require that Z is dense.
26
State
Whereas work matrix completion with non-standard observation patterns 228 this is by and large not obviously useful or applicable to our problem. Instead, we build on a recent program to bridge convex and non-convex formulations for matrix completion [ and Robust- PCA 5 That work has provided a pioneering analysis of entry-wise guarantees and uncertainty quantification for convex matrix completion estimators wherein entries remain missing at random. Our work here may be viewed as extending that program to a broad class of non-random missingness patterns, a contribution of important independent interest.
29 30 31] exists
2 Model and Algorithm
We begin by formally defining our problem; we in fact present a generalization to the problem described in the previous section, allowing for multiple treatments. Let M E Rn be a fixed rank-r matrix with singular value decomposition (SVD) denoted by M U- where U² V' E Raxr have orthonormal columns, and Σ- E R2x" is diagonal with diagonal entries σi 2- .... o, of Fmax := di, 7min == o, and x == Fmax//mii be the condition number of M There are l treatments that can be applied to each entry, and for each treatment m E {1...., .... we are given a treatment matrix 7,m E {0, | which encodes the entries which have received the mh treatment (0 meaning no treatment, and I meaning being treated) We then observe a single matrix of outcomes:
0 M
+ E
Tm Zm
m=1
(o is the Hadamard or 'entrywise" product). where each Tm E R is an unknown matrix of treatment effects, and E E [ is (possibly heterogeneous) random noise matrix. Finally, let T" E R* be the vector of average treatment effects, whose mhth value is defined as F := (Tm: Zm) 2m)//l /|| Zm l and let 5m Tm Zm
Zm be associated 'residual' matrices. Our problem is to estimate r* after having observed O and Zi...., Zk-
It is worth noting that the representation above is powerful: for instance, it subsumes the setting where the intervention on any entry associated with a (0, 1) --vauud covariate vector, and the treatment effect observed on that entry is some linear function of this covariate vector plus idiosyneratic noise. Recovery of 7" is then equivalent to recovering covariate dependent heterogeneous treatment effects.
Our problem also subsumes the synthetic control setting where k = 1 and 21 must place support on a single row; the focus of our later analysis will be the case where Zi allowed be general.
Noote that in contrast to the previous section, are now assuming square matrices (i.e. n T in the notation of the previous section). This purely to simplify the notation for rectangular n-bb--I matrix, all of our theoretical guarantces hold if one swaps n with min(n, T}.
We allow for multiple treatments to be applied to entry.
3

---

The assumptions that we will need to impose in order to state meaningful results can be divided into two groups. The first are assumptions on M and E that are, by this point, canonical in the matrix completion literature:
Assumption 1 (Random Noise). The entries of E are independent, mean-zero, sub-Gaussian random variables with sub- Gaussian norm bounded by a: that is, Eigllls I for every i.j E [m]. Assumption 2 (Incoherence). M j-inncherent:
||0 12, $ /ur/n and ||V" & vpr/n,
where || 122.00 denotes the maximum lz-norm of the rows of a matrix.
In addition to these standard conditions on M and which we will assume throughout this paper, we will also need to impose conditions the relationship between M and the Zm. 's. Loosely speaking, these conditions preclude treatment matrices Zm that can 'disguised'' within M in the sense that their projections onto the tangent spaces of M* are large. Specifically, the formal statements relate to a particular decomposition the linear space of n x n matrices, R"** T + T where T is the tangent space of M in the manifold consisting of matrices with rank no larger than rank (M*):
T A A BV A, B E [=xxaan(M"))
Equivalently, the orthogonal space of T*, denoted I* is the subspace of guxa whose columns and rows are orthogonal, respectively, to the spaces U and V*. Let Pr-. (-) denote the projection operator onto T
Pr.. (A) -I- -1--""
We will defer the formal statements of the additional conditions to the next section, but suffice to say for now that they assume Pr. to be sufficiently large (we will show that lower bounds on the size of Pr-1 (Zm), in the the precise form of our own conditions, are nearly necessary). In the remainder of this section, we will outline the core contribution of this paper, which an estimator for T' with provably rate-optimal guarantee.
2.1
A De-biased Convex Estimator
Our estimator is constructed in two steps, stated as Eqs. [a and (Ib) below:
(M, E
argmin MER"X" FER
0 M g(M,T) = 2
7m
+ A M II.
(1a)
m=1
== P=
(Ib)
In Eq. 1b define by D E bx the Gram matrix with entires Dim = (Z),, PT. (Zm))
)} and
by A E R the 'error' vector with components A >(Zz, ŪVT). where have let M ÚΣVT be the SVD of M, and let T' denote the tangent space of M
The first step, Eq. (Ta. is a natural convex optimization formulation that we use to compute a 'rough' estimate of the average treatment effects. The objective function's 's first term penalizes choices of M and T which differ from the observed O, and the second term seeks to penalize the rank of M using the nuclear theoretical guarantees, encodes the relative weight of these two objectives.
(convex) proxy. The tuning parameter À >> O, which will be specified in our
After the first step, having (M, 7) as a minimizer of Eq. la). we could simply use as our estimator for-*. r* However, a brief analysis of the first-order optimality conditions for (Ta) yields simple, but powerful decomposition of f x that suggests a first-order improvement to f via de- biasiinn::
Lemma 1. Suppose (M, f) is minimizer of [aa. Let M = =ÔEV be the SVD of M, and let f denote the tangent space of M. Denote E. = Fr + Σm 5m a 7mm Then,
D(--T") = A + A2 + A3
(2)
where A 2,
22,23 E R are vectors with components
A = M/Zm A. (Pp- (Zm). E), A?, = (Zm, (M
'Our definition implicitly assumes that D is invertible. We view this as a natural assumption on (the absence of) collinearity in treatments.

---

DA' + + by Eq. and note that D-'A' is entirely a function of observed quantities. Thus, it known and removable. The second step of of our algorithm, Eq. {T)), does exactly this. The resulting de-biased estimator, denoted rd, the subject of this paper. As an aside, it is worth noting that while de-biased estimators for high-dimensional inference have received considerable attention recently, our de-biasing procedure is algorithmically distinct from existing notions of de-biasing, including those for problems closely related to our own (e.g. matrix completion [32 18 and panel data regression (27).
Consider this error decomposition, i.e.
Our main results characterize the error - T'. The erux of this can be gleaned from the second and third terms of Eq. (2) If I is sufficiently 'close' T", then A becomes negligible (because Pr-1 (M*) = 0). Showing closeness of ī and T is the main technical challenge of this work. The remaining error, contributed by A², can then be characterized the (independent) entries of E and the residual matrices 6n which we show to be min-max optimal.
particular 'weighted average' of
To conclude this section, we prove Lemma for single treatment (k- = 1); the complete proof is straightforward generalization, completed in Appendix (See the online full version for Appendix details).
Proof of Lemmal 7 for k = 1. Since = 1, we suppress redundant subscripts. Consider the first-order
optimality conditions of Iaa
2.0 (2.0- M - - =
(3a)
M - -iZ -(OOT + +W),
(3b)
51
(3c)
PW) = W
(3d)
(W is called the 'dual certificate' in the matrix completion literature 33 34 35 and Eq. (3a), we have
Combining Eq. (3b)
(2,0- - FZ -0 =0 -0 (Z,A(UT +w)
=0
= = (2,00T)) = (Z,AW) (4)
Next, applying Pr. (.) to both sides of Eq. 3b) and using Eq. (3d
P0. - M +Z) = XW
P Py- + 8 7) (f- -f--)))pa(2)=
(5)
where the implication is by definition: 0 = M + E + + 2. Finally, substituting XW from Eq. (5) into Eq.
we obtain
(Z, = - (Z,Pr (M") + Pp. (E+ (f- (Z))
= (+-r))]| = = x (Z. ( (Z,Pz_
+
This is equivalent to Eq. 2). completing the proof.
3 Theoretical Guarantees
Summarizing so far, our estimator is constructed in two steps: solve the convex program in Eq. (lap to obtain an initial estimate (M, +). then de-bias according to Eq. (lb). While we have presented this estimator in setting that allows for multiple treatments (i.e. k 2 1), for the sake simplicity our results here (Theorems | and ( are restricted the single treatment (i.e. == 1) setting. Recall that the work on synthetic control, which we seek to extend, is for single treatment and a particular form of Z, (support on a single row); our results in this section are for a single treatment but general Z1- To ease notation, we will from here on suppress treatient-speciffe subscripts (7,, 71, etc.)
As mentioned earlier, our results require a set of conditions that relate the treatment matrix Z the langent space T" of M*
httpps://rrxyy
5

---

Assumption 3. There exist positive constants Cr, , Cr such that
(a) zv - + ZT < (1 Cr, (1-C. log(n)) Z
(b) (2, U- 1| (2) - (1 Cr, log(n)) P (2)
Assumption is is nearly necessary for identifying =" (in a manner made formal by Proposition 2 below). These conditions are mild enough to allow for various treatment patterns that occur in practice, and broadly expand on the set of patterns possible in the synthetic control literature (as we discuss in Section B.T..
Let 8 T = Z be the matrix of treatment effect 'residuals." Our first result establishes a bound on the eтror rate of rd. Note that 8 is a zero-mean matrix and is zero outside the support of Z. Thus the requirement below that ||ô]| S Cso n//ri is mild. It is trivially met in synthetic control settings. is also easily seen as met when å has independent, sub-Gaussian entries. Finally as it turns out, the condition can also admit random sub-gaussian matrices with complex correlation patterns; see [36]
Theorem (Optimal Error Rate). Suppose
ay S C1 x² C1 log"(n) and ||ô1| Coavn/l Then for
Omin any C2 > O, for susficientlly large n, with probability Onn we have
xarn log (n) ( (6), PT- (7-(2))
-- S Ce log(n) max
|
11
Tmin
Here, Ce is a constant depending (polynomially) on Cr, C2 Cray Cray Co (where Cr and Or, are the constants appearing in Assumption 3
To begin parsing this result, consider a 'typical' scenario in which a, K, 0(1), and Omin (n)). Then Theorem I
implies that
(6)
This is minimax optimal (up to log n factors), as shown below:
Proposition 1 (Minimax Lower Bound). For any estimator f. there exists an instance with O,K, =- O(1), and miin = (n)), on which, with probability at least 1/3,
| -"1 2 max (a/ FF
(z)
Finally, it is worth considering some special cases under which Eq. (6) reduces further to
17d - +'| Ô (l/1)
(7)
which is the optimal rate (up to log n) achieviable even when M and 8 are known. Any of the following are, alone, sufficient to imply Eq. 77
Independent 5: Independent, sub-gaussian d.y with O(1) sub-gaussian norm
Synthetic control and block Z: [|8]| = Ō(1, Z consists of an x e block that is sufficiently
sparse: Vle(EE + +c) = (n). For comparison, state-of-the of-thee-rt synthetic control results [3] 37 require the sparser condition Ec(E + c) O(n) (though that condition enables asymptotic normality results). Panel data regression: The conditions imposed in (27). the most notably that Z be sufficiently dense: P| (2)ll (n²)). This recovers their error guarantee (up log factors); see Appendix D
Our second main result establishes asymptotic normality for our estimator. This naturally requires some additional control over the variability of 8. We consider the setting in which the 8, To are independent variables.
Theorem 2 (Asymptotic Normality). Suppose each dij is a mean-zero, independent random variable with Gau-Ganssian norm ||ō,g|le2 (1). Assume = = ===== =O(1) and Twin M(n). Then
with probability (11nn²).
(E + (5 Z), P (Z))
+0 (og (m)) )
dd
(Z)
n
Consequently.
- (/--")/V!/2 ( NN(11) V= Σ provided that 11/22 = (((log(n)//))
Pr- Var(Err + + /
Pro. (Z))
85

---

Asymptotic normality is of econometric interest, as it enables inference. Specifically, inference can be performed using a 'plug-in' estimator V/ for V-, gotten by substittingg T for T and O - M - -4 for + (6 o 2), where Md is de-biased estimator for M* (see Appendix Lp. This is a common procedure in the literature (e.g., see 18] and it is straightforward to show that V-= ~ V,.
The proofs of Theorems and 2 are inspired by recent developments on bridging convex and non-convex formulations for matrix completion 4 and Robust-PCA [5]. Whereas that work assume random, independent missingness pattern, our proof extends that program to deal with deterministic treatment patterns Z. As such, this analysis is likely of interest, in its own right, as complement to the matrix completion literature [38 39 4 5. Broadly, we must address the issue that constructing a dual certificate to analyze the quality of our convex estimator directly is hard. Instead 4 show the existence of such certificate non- connconstructielly by studying non-convex estimator and showing that a (fictitious) gradient descent algorithm applied to that estimator recovers suitable dual certificate. We effectively extend that program to deterministic pattems Z, and provide entry-wise recovery guarantees on M in this setting that are of independent interest.
Proof Technique:
3.1 Applicability of Treatment Conditions
Having stated our results, we return to our assumptions on (Assumption) 3 and discuss the extent to which they allow for treatment patterns that occur in practice. First however, we establish that Assumption is necessary, Consider the proposition below, which establishes that should either of the condition of Assumption 3 be violated by an amount that grows negligible with n, then identification rendered impossible so that no estimator can recover T"
Proposition 2. Let n be any positive even integer. There exists matrix Z E {0, 1|mXa and a pair of rank-1 and μ = 2 (incoherent) matrices M1 E Rax* with SVDs denoted by M; = =U,,,V and Ta being the tangent space of Mi, such that all three of the following statements hold:
1. || |] + U
Z 1 i= 1.2
2. < 22 U.V. |||||| 11 (2)1l = (2)1 i i= 1,2
3. M1 + 7 = M₂
We next discuss various treatment patterns that are admissible under Assumption 3
1. Rank grows faster than Assumption 3 holds if Ei= oz(2)² S (1-C) log l) IZIB/ I Loosely speaking, this requires that the rank of Z be strictly higher than r, and that less than 1/ 1/ of its 'mass' lie in its first r components. Put another way, Z must be sufficiently different from any rank-r approximation. One common setting where this occurs is when there is sufficient randomness in generating Z, such as the case where the entries of Z are drawn independently, which is the canonical scenario in the matrix completion literature (e.g.. 35 38 39)
Maximal number of ones in a row and columm: Let é and e denote the maximum number of ones in row and column, respectively. of Z. Assumption 3 holds if f + c < (1 (1-6 C/ log n. n/(r²p) p). When r. (11) this allows f. Onn) and generalizes the sparse block patterns studied in the literature (e.g. 40 13), where 7 is a two-by-two block matrix with exactly one block equal to one.
3. Single now column (Synthetic Control): Consider the case when Z is supported on sin- gle row (or column, equivalently), as in synthetic control. Assumption 3 holds if V S (1 C1-C/log n - -rr//n /r//n) ||=]]?, where T is the non-zero row of Z. This will easily hold, if allowing a negligible perturbation either 21 the row space of M It is also interesting to note that the iden- tification assumption made in the canonical paper 2 (i.e., To L (E/o V-V-TT > is bounded away from zero, where To m i), together with To 22nn/ (n)( and pr = Onn/ log(n))
(also implicitly assumed in 2 for an optimal gaurantee), together imply Assumption
We end this section by drawing a connection to the literature on panel data regression with interactive fixed effects 225 36 27 That literature studies estimators similar in spirits to ours (e.g.. 22 analyzed the performance of the convex estimator and a heuristic de-biasing approach). However,
6 Also see a further discussion for mathematical equivalent forms of Assumption 3 based on basis transforma- tions and the generalization to k > in Appendix C
7

---

those approaches are only known to work if P| (Z)lee = 2n This is of course a substantially stronger assumption than Assumption 3 and rules out sparse treatment patterns (as in synthetic control). In summary, our approach also has the potential to broaden the scope of problems addressed via panel data regression.
4 Experiments
We conducted a set of experiments on semi-synthetic datasets (the treatment is introduced artificially and thus ground-truth treatment-effect values are known) and real datasets (the treatment is real and ground-truth treatment-effect values are unknown). The results show that our estimator 7d is more accurate than existing algorithms and its performance is robust to various treatment patterns.
The following four benchmarks were implemented: (i) Synthetic Difference-in--ifffeeee
(SDID)
[3] (ii) Matrix- Saae--Compllooooo with Nuclear Norm Minimization (MC-NNM) [3] (iii) Robust Synthetic
Control (RSC) [00] (iv) Ordinary Least Square (OLS); Selects a, 6 R" . e E minimize
|10 110-al' - 16 TZ fl.. where E I is the vector of ones. It is worth noting that SDID and RSC
one apply to traditional synthetic control patterns (block and stagger below).
Semi-Synthetic Data (Tobacco). The first dataset consists of the annual tobacco consumption per capita for 38 states during 1970-2001, collected from the prominent synthetic control study [2] (the treated unit California is removed), Similar to (3], we view the collected data as M and introduce artificial treatments. We considered two families of patterns that are common in the economics literature: block and stagger 3 Block patterns model simultaneous adoption of the treatment, while stagger patterns model adoption at different times. In both cases, treatment continues forever once adopted. Specifically, given the parameters (m,. m2), set of mi rows of Z are selected uniformly random. On these rows, Zi = 1 and only if j 2 t,, where for block patterns, t = m2, and for stagger patterns, f is selected uniformly from values greater than m2-
To model heterogenous treatment effects, let Tij r' d; where 8; is i.d and 6; (0)
characterizes the unit-specific effect. Then the observation is O = M + T Z. We fix T" = 0g = M*/5 through all experiments, where M the mean value of M The hyperparameters for all algorithms were tuned using rank r ~ 5 (estimated via the spectrum of M²).
Next, we compare the performances of the various algorithms an ensemble of 1,000 instances with mi ~ Uni[1, n1) m2 = Uni patterns (matching the year 1988, where California passed its law for tobacco control). The results are reported in the first two rows of Table in terms the average normalized error |r - 1/7"".
[ni[1,,)) n22) for stagger patterns and my ~ Uni[1, 5), m2 = 18 for block
Note that the treatment patterns here are 'home court' for the SDID and RSC synthetic control methods but our approach nonetheless outperforms these benchmarks. One potential reason that these methods do not leverage all of the available data for learning counterfactuals: MC-NNM and SDID ignore treated observations. RSC ignores even more: addition does not leverage some of the unfreated observations in M* on treated units (i.e. observations Oij for j < 4 on treated units).
Table 1: Comparison of our algorithm (De-biased Convex) to benchmarks on semi-synthetic datasets (Block and Stagger correspond to Tobacco dataset; Adaptive pattern corresponds to Sales dataset). Average normalized error I T "I/T" is reported.
Pattern
De-biased Convex
SDID
MC-NNM
RSC
OLS
Block
0.15 (#0.13)
0.23 (=0.19)
0.27 (+0.24)
0.30 (+0.26)
0.38 (+:036)
Stagger
0.10 (+0.20)
(10.18)
0.15 (+0.16)
0.20 (+ 0.27)
0.18 0.19)
Adaptive
0.02 (4-0.02)
0.13 (+0.10)
0.20 (4-0.18)
Semi-Synthetic Data (Sales). The second dataset consists of weekly sales of 167 products over 147 weeks, collected from a Kaggle competition [41. this application, treatment corresponds to
"Parenthetically, the assumption Pm (Z)Ile (n²)) greatly simplifies analysis in 27 since global bound on -" can be easily obtained. One of the main technical innovations paper, building recent advances in the matrix completion literature, is to conduct a refined 'local' analysis without the assumption on the density of Z.
See Appendix
E for estimating row-specific treatment effects
8

---

various 'promotions' of a product ge.g. price reductions, advertisements, etc.). We introduced an artificial promotion Z, used the collected data as M (M* 12170), and the goal was estimate the average treatment effect given O = M + T Z and Z (T follows the same generation process as above with -" a /5).
Now the challenge in these settings is that these promotions are often decided based on previous sales. Put another way, the treatment matrix Z is constructed adaptively. We considered simple model for generating adaptive patterns for Z: Fix parameters (a, 6). If the sale of product reaches its lowest point among the past weeks, then we added promotions for the following b weeks (this models a common preference for promoting low-sale products). Across our instances, (a, b) was generated according E Uni[5, 25 Uni (5, 25]. This represents treatment pattern where it is unclear how typical synthetic control approaches (SDID, RSC) might even be applied.
The rank of M is estimated via the spectrum with ~ 35. See Table for the results averaged over 1,000 instances. The average of T 7 r"//7" is ~ 5.3% for our algorithm, versus 27.6% for MC-NNM. We conjecture that the reason for this is that highly structured missing-ness patterns are challenging for matrix-completion algorithms; we overcome this limitation by leveraging the treated data as well. Of course, there is a natural trade-off here: if the heterogeneity in å were on the order of the variation in M (so that | Z|I > o, (M))) then it is unclear that the treated data would help (and it might, in fact, hurt). But for most practical applications, the treatment effects we seek to estimate are typically small relative to the nominal observed values.
Real Data. This dataset consists of daily sales and promotion information of 571 drug stores 942 days, collected from Rossmann Store Sales dataset [42]. The promotion dataset Z is binary (1 indicates promo is running that specific day and store). The real pattern is highly complex (see Fig. I and hence synthetic-control type methods (SDID, RSC) again do not apply. Our goal here is estimate the average increase of sales T* brought by the promotion.
202
Test Error
De-biased Convex
(12.44)
0.04 (10.002)
MC-NNM
-4..4 (10.988)
0.07 (1.0.002)
402
OLS
-45.8 (11.24)
0.18 (110.003)
Figure 1: Left: The promotion pattern of the real data. Right: Estimation of T and test errors.
The hyperparameters for all algorithms test set J2 consisting of 20% of the treated entires is randomly sampled and hidden. The test error is then calculated by [| + 7Z
tuned using rank 70 (estimated via cross validation).
Po - O) where Ô the mean-value of O. Fig. [ shows the results averaged over 100 instances. Our algorithm provides superior test error. This is potentially a conservative measure since captures error in approximating both M* and 7"; the variation contributed by M* to observations is substantially larger that that contributed by r". Now whereas the ground-trute -tuth for r" is not known here, the negative treatment effects estimated by MC-NNM and OLS seem less likely since store-wise promotions are typically associated with positive effects on sales.
Asymptotic Normality. The normality of our estimator is also verified, where the prediction from Theorem 2 is precise enough to enable inferential tasks such constructing confidence intervals (CIs) for 7": our 95% CIs typically had "true" coverage rates in the range of 93-96% for a synthetic set of instances described in Appendix B See Fig.
Conclusion
Motivated by the extremely important econometric problem of estimating treatment effects from panel data, studied natural formulation of this problem one of recovering unknown quantity

---

NIO, 1)
942
030
030
Density
22
50
100
150
200
na/m
0.5
0.916
0.957
0.956
0.942
9.1
0.953
0,946
0.954
0.939
0112
2
0.946
0.947
0,945
0.957
0.94
0.934
0.949
0.944
305
0.00
Figure 2: Evaluation of our distributional characterization of 7 synthetic ensemble where 5 and
E follow i.i.d Gaussian distribution. Left: Empirical Distribution of (r 1-"--" T")/V, with = 100, overlaid with the (O, 1) density function predicted by Theorem [ Right: Coverage rates of 95% confidence intervals (the 'correct' coverage rate is 0.95) for different sizes (n1, n2) with r = 10. See Appendix B for the data generation processes in details.
that has been added an unknown low-rank matrix at a subset of its entries. We proposed an estimator based on solving, and de- de-biasing, a natural regularized least-squares objective. We built on recent techniques for establishing entry-wise guarantees that leverage connection between convex and non-convex formulations, and proved that our estimator is order-optimal with nearly minimal conditions on the underlying low-rank matrix and the pattern of 'treated' entries.
One exciting direction for further work is to consider the correlation between noises (as opposed to the independent noise assumption). In fact, our main guarantee for the error rate (Theorem 1) is restricted to independent noise (the results incorporate a deterministic E naturally, see Appendix B and Appendix J). On the other hand, our asymptotic normality result (Theorem 2) indeed requires independent noise. Generalizing this result to correlated noise appears to be more involved and may require further technical innovations.
6
Ackowledgement
We thank all anonymous reviewers for their constructive comments. Farias and Peng were partially supported by NSF Grant CMMI 1727239.
References
[1] Alberto Abadie and Javier Gardeazabal. The economic costs of conflict: A case study of the basque country. American economic review, 93(1): 13-11131111 132, 2003.
[2] Alberto Abadie, Alexis Diamond, and Jens Hainmueller. Synthetic control methods for compar- ative case studies: Estimating the effect of california's 's tobacco control program. Journal of the American statistical Association, 105(490):4335505, 2010.
[3] Susan Athey, Mohsen Bayati, Nikolay Doudchenko, Guido Imbens, and Khashayar Khosravi. Matrix completion methods for causal panel data models. Journal of the American Statistical Association, pages 1-41, 2021.
[4] Yuxin Chen, Yuejie Chi, Jianqing Fan, Cong Ma, and Yuling Yan. Noisy matrix completion: Understanding statistical guarantees for convex relaxation via nonconvex optimization. arXiv preprint arXiv: 07799999,,, 2019.
[5] Yuxin Chen, Jianqing Fan, Cong Ma, and Yuling Yan. Bridging convex and nonconvex optimization robust pea: Noise, outliers, and missing data, arXiv preprint arXiv:2001 005484, 2020.
10

---

[6] Alberto Abadie. Using synthetic controls: Feasibility, data requirements, and methodological aspects, Journal of Economic Literature, 2019.
[7] Nikolay Doudchenko and Guido W Imbens. Balancing, regression, difference-in -ineeneei-ndifffeenees and synthetic control methods: A synthesis. Technical report, National Bureau of Economic Research, 2016.
[8] Kathleen Li and David R Bell. Estimation of average treatment effects with panel data: Asymptotic theory and implementation. Journal of Econometrics, 197(1):65-75, 2017.
[9] Eli Ben-Michacl, Avi Feller, and Jesse Rothstein. The augmented synthetic control method. Journal of the American Statistical Association, (just-accepted): 2021.
[10] Muhammad Amjad, Devavrat Shah, and Dennis Shen. Robust synthetic control. The Journal of Machine Learning Research, 19(1):802 252,22522220 2018.
[11] Muhammad Amjad, Vishal Misra, Devavrat Shah, and Dennis Shen. mrsc: Multi-dimeensonal robust synthetic control. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 3(2):1-27, 2019.
[12] Anish Agarwal, Devavrat Shah, Dennis Shen, and Dogyoon Song. On robustness of principal component regression. Journal of the American Statistical Association, (just-accepted):1 2021.
[13] Dmitry Arkhangelsky, Susan Athey, David A Hirshberg, Guido W Imbens, and Stefan Wager. Synthetic difference in differences, 2019.
[14] Kathleen Li. Statistical inference for average treatment effects estimated by synthetic control methods. Journal of the American Statistical Association, 115(532):2068 2008-208,, 2020.
[15] Victor Chernozhukov, Kaspar Wüthrich, and Yinchu Zhu. An exact and robust conformal inference method for counterfactual and synthetic controls. Journal of the American Statistical Association, (just-accepted):1 1-44, 1-44, 2021.
[16] Jushan Bai and Serena Ng. Matrix completion, counterfactuals, and factor analysis of missing data. arXiv preprint arXiv:1910.06677 2019.
[17] Ruoxuan Xiong and Markus Pelger. Large dimensional latent factor modeling with missing observations and applications to causal inference. arXiv preprint 1rXiv:1910.08273, 2019.
[18] Yuxin Chen, Jianqing Fan, Cong Ma, and Yuling Yan. Inference and uncertainty quantification for noisy matrix completion. arXiv preprint arXiv:1906.0411 041999 2019.
[19] Susan Athey and Guido Imbens. Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 1 113(27):7353- 7360, 2016.
[20] Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523): 1242, 2018.
[21] Sören R Künzel, Jasjeet S Sekhon, Peter Bickel, and Bin Yu. Metalearners for estimating heterogeneous treatment effeets using machine learning. Proceedings of the national academy of sciences, 16611))4156-4165, 2019.
[22] Ruoxuan Xiong, Susan Athey, Mohsen Bayati, and Guido W Imbens. Optimal experimental design for staggered rollouts. Available SSRN, 2019.
[23] Laurent Gobillon and Thierry Magnac. Regional policy evaluation: Interactive fixed effects and synthetic controls. Review of Economics and Statistics, 98(3):535 5511, 2016.
[24] Laurent Gobillon and François- Chhrlles Wolff. The local effects of an innovation: Evidence from the french fish market. Ecological Economics, 171:106594, 2020.
[25] Jushan Bai. Panel data models with interactive fixed effects. Econometrica, 1229 2009.

---

[26] Hyungsik Roger Moon and Martin Weidner. Dynamic linear panel regression models with interactive fixed effects. Econometric Theory, 33(1):158-195, 2017.
[27] Hyungsik Roger Moon and Martin Weidner. Nuclear norm regularized estimation of panel regression models, arXiv preprint arXiv:1810.1098) 2018.
[28] Sourav Chatterjee. A deterministic theory of low rank matrix completion. JEEE Transactions on Information Theory, 2020.
[29] Simon Foucart, Deanna Needell, Reese Pathak, Yaniv Plan, and Mary Wootters. Weighted matrix completion from non-random, non-uniform sampling patterns. arXiv preprint arXh::1910.13980. 2019.
[30] Guangean Liu, Qingshan Liu, and Xiaotong Yuan. A new theory for matrix completion. In Advances in Neural Information Processing Systems, pages 785-794, 2017.
[31] Olga Klopp, Karim Lounici, and Alexandre B Tsybakov. Robust matrix completion. Probability Theory and Related Fields, 169(1-2):233564, 2017.
[32] Dong Xia. Confidence interval of singular vectors for high-dimensional and low-rank matrix regression. arXiv preprint arXiv:1805.0900 09001, 2018.
[33] John Wright, Arvind Ganesh, Shankar Rao, Yigang Peng, and Yi Ma. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Advances in neural information processing systems, pages 2080-2088, 2009.
[34] Benjamin Recht. A simpler approach to matrix completion. Journal of Machine Learning Research, 12(12), 2011.
[35] Emmanuel J Candès and Benjamin Recht. Exact matrix completion via convex optimization. Foundations of Computational mathematics, 9(6): 17,, 2009.
[36] Hyungsik Roger Moon and Martin Weidner. Linear regression for panel with unknown number of factors as interactive fixed effects. Econometrica, 83(4):1543-157)) 2015.
[37] Anish Agarwal, Abdullah Alomar, Romain Cosson, Devavrat Shah, and Dennis Shen. Synthetic interventions. arXiv preprint arXiv:2006.0766 2020.
[38] Emmanuel Abbe, Jianqing Fan, Kaizheng Wang, and Yiqiao Zhong. Entrywise eigenvector analysis of random matrices with low expected rank. arXiv preprint arXiv: 2017.
[39] Cong Ma, Kaizheng Wang, Yuejie Chi, and Yuxin Chen. Implicit regularization in nonconvex statistical estimation: Gradient descent converges linearly for phase retrieval, matrix completion, and blind deconvolution. Foundations of Computational Mathematics, pages 1-182, 2019.
[40] Yiqing Xu. Generalized synthetic control method: Causal inference with interactive fixed effects models. Political Analysis, 25(1):57-76, 2017.
[41] Predict future sales. https://wwa. www.le..g.....m
http://////oootttteeeaaaaaaaaaaa cocdict-future-see /oeevveeevvvvveee Accessed: 2021-11-5.
[42] Rossmann store sales. http://///////eeeeeooooooooo//ea 2021-05-26.
Accessed:
[43] Ji Chen, Dekai Liu, and Xiaodong Li. Nonconvex rectangular matrix completion via gradient descent without 22,00 regularization. IEEE Transactions on Information Theory, 2020.
[44] Rahul Mazumder, Trevor Hastie, and Robert Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. Journal of machine learning research, 11(Aug)2287-2322. 2010.
[45] Guido W Imbens and Jeffrey Wooldridge. Recent developments in the econometrics of program evaluation. Journal of economic literature, 47(1):5-86, 2009.
12

---

[46] Ali Taylan Cemgil. Bayesian inference for nonnegative matrix factorisation models. Computa- tional intelligence and neuroscience. 2009, 2008.
[47] Vivek Farias, Andrew A Li, and Tianyi Peng. Near-optimal entrywise anomaly detection for low-rank matrices with sub-exponential noise. arXiv preprini arXiv:2006. 13126, 2020.
[48] Yi Yu, Tengyao Wang, and Richard Samworth. A useful variant of the Mavis-kahan theorem for statisticians. Biometrika, 102(2):315-222 2015.
[49] Mark Rudelson, Roman Vershynin, et al. Hanson-wright inequality and sub- uub-gaussian concentra- tion. Electronic Communications in Probability, 18, 2013.
13


Reviews:
[{"review_id": "x6jAEJ89HHg", "reviewer": "Reviewer_gN9T", "summary": "The paper studies learning treatment effects in panel data with heterogenous treatment effects, with a de-biased estimator that achieves rate-optimal recovery.", "questions": "", "limitations": "", "rating": 9, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-written and clearly organized. This paper not only extends the synthetic control paradigm to the much broader class, in the sense of robust matrix completion, the paper also delivers novel theoretical results with unknown and deterministic missing patterns. Thus, the reviewer believes that this would be a nice contribution to the Neurips community and also provides new technical tools that could be helpful to other problems. Some minor points:\n\n1. The paper presents the problem under the setting of multiple treatment effects that allows for $k \\geq 2$. However, the main theorems are delivered assuming $k = 1$. Although I believe this is definitely reasonable to be left for future work, I wonder whether there are technical barriers or it could be generalized without much work?\n2. I wonder whether assumption 3 itself is enough for showing the multiple treatment effect case. It is possible that different treatment effects (say $Z_1, Z_2$) cancels in the tangent space of $M^\\star$. Would that be a problem?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The theoretical results are only for $k=1$.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "WvCJmiIn8uT", "reviewer": "Reviewer_BWiu", "summary": "The paper considers the problem of average treatment effect estimation in panel data. The authors consider a setup where the observed outcomes form a matrix with a special structure: it is the sum of a counterfactual matrix and a treatment matrix, where the counterfactual matrix is assumed to be a “noisy observation” of a low rank matrix. The paper proposes a debiased estimator, shows that it is rate optimal, derives its asymptotic normality and demonstrate its performance through numerical experiments. ", "questions": "", "limitations": "", "rating": 10, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The proposed estimator is new. The debiasing technique is very interesting and inspiring. In terms of theory, as far as I know, this is the first rate optimality result in this set up.  \n\nQuality: As far as I checked, all claims are well supported. \n\nClarity: Overall, the paper is well written and I enjoyed reading it. The messages are delivered in a very clear way. I enjoyed a lot reading the proof of Lemma 1. It is very helpful for readers to understand the key idea behind the debiasing technique. \n\nSignificance: Overall, I think the paper makes a strong and important contribution. The problem this paper considers is very important. I believe rate optimality is also a very difficulty goal especially under very general assumptions. The paper provides a strong solution to the problem. Both the upper bound and the lower bound are very nice results. I believe the results and proof techniques are not only important in the field of panel data, but also for other matrix estimation/completion problems in general. The proposed estimator appears to perform very good empirically as well. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work. ", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "UqnXzacmp0", "reviewer": "Reviewer_KTHy", "summary": "This paper looks at the problem of learning treatment effects from observational data. Recently, there has been a flurry of work in establishing theoretical guarantees for low-rank matrix recovery in presence of a Gaussian noise, where de-biasing-based estimators have been analyzed. This paper takes that problem to a next level, in the context of learning treatment effects. Here, the observed matrix is a low-rank matrix added with noise and also treatment matrices. The goal is to recover the effect of average treatment. The authors propose a two-step estimator. In the first step, they try to learn a low-rank matrix and the treatments using a convex optimization program, which is standard in the matrix recovery literature. Then they also use a de-biasing step to improve the performance of their estimation of the treatment matrix.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Finally, they show that their estimator works well in practice for both real and synthetic datasets. They heavily built upon the prior work on low-rank matrix recovery and the new contributions are theoretically and practically justified. I think this is a good paper and vote for acceptance.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "34OGzQbPced", "reviewer": "Reviewer_55D9", "summary": "This paper considers a general problem of applying multiple treatments (heterogenous) to multiple units across multiple time-steps, where the outcome is of this treatment has a linear relationship. Given the treatment decision matrix and the outcome matrix, the goal of the algorithm is to obtain the treatment effect (or more specifically a sufficient statistic that can be used to derive the treatment effect given the model). In particular, the outcome matrix O for n units across T steps is determined by a linear system M* + E + sum_{k} treatment_effect_matrix_k * decision_matrix_k. The algorithm sees the outcome matrix O and knows the decision_matrix_k for each of the k treatments and it needs to recover the treatment_effect_matrix_k for each of the k treatments. The paper identifies assumptions under which such an estimator is possible, shows that when violating one of the assumptions no algorithm can obtain such an estimator and that they have good empirical performance on semi-synthetic datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The problem/model considered in this paper is very general and captures many important problems as special cases: single treatment effect estimation, synthetic control estimator, etc. Thus, as far as I can tell, the problem setting is original. The paper identifies assumptions under which an unbiased estimator is possible. In particular, the important assumption is Assumption 3 (ASsumption 1 is independence of entries of the noise and ASsumption 2 is that of incoherence both are not outrageous assumptions). They show that when this assumption is made an algorithm based on solving convex program can be used to obtain an unbiased estimator for the problem. Moreover, they also show that if this is violated then no algorithm can recover the required statistic. This constitutes a reasonably complete characterization of the problem in my opinion. Although once the assumptions are made, the recovery/proof is pretty straightforward, in my opinion the original contribution of the paper is to identify these set of tracatable assumptions.\n\nQuality/Clarity: The paper is well-written and easy to read. I was able to understand the main ideas of the paper, the proof very easily. I also think the paper does a detailed job (as allowed by the page limit) on experimenting this with semi-synthetic datasets. Overall the results in this paper look complete to me; identify assumptions the estimator, show that its necessary in theory (with some cavets) and experiment the usefulness in some reasonable empiricial setting.\n\nSignificance: The paper adds to the literature of estimating treatment effects of which there are many different lines of work. Synthetic control (a special case of their model) has found usefulness in doing counter factual estimation. Their work also can be conctrued to fall in the line of work on linear casual models (although a simpler version since the main challenges in causal inference is to deal with confounders or correlated noise). The particular setting itself seems reasonably practical in large scale settings like recommender systems where one tends to apply multiple treatments to multiple experimental units simultaneously. One could argue if the linearity assumption between outcome and treatment holds but upto that caveat, the setting and the method is significant/practical.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "One of the limitations of this paper is that of independent noise E. As stated above its not an outrageous assumption, but at the same time by far its the biggest limitation of this model. If we treat the outcome matrix as n*T where n are the units in the system and T is the time-steps, then the noise across the time-steps is rarely independent. In fact, they would be systematic and positively correlated. A lesser issue, but still similar is that across experimental units. The noise across different customers/experimental units will also be correlated. For instance, across two \"similar\" users, it is likely that the noise (typically used to include all the unmeansurable parts of the system) would also be correlated in a systematic way. Acknowledging this limitation in the paper is useful to the reader/a practioner who may potentially try to use this. Of course, making a reasonable non-independence assumption to make the problem tractable is challenging and is reasonable to assume that is beyond the scope of the current paper.\n\nRegarding societal impact, I think the paper is primarily foundational. So as a stand alone paper does not have negative societal impact. But based on application it could have impact. For instance, if the \"treatment\" in question is on human subjects, it may require ethics considerations. But that is largely beyond the scope of the current paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper considers the problem of learning treatment effects with panel data and provides an algorithm with tight theoretical guarantees as well as good empirical performance. All the reviewers agree that the results in this paper are a substantial improvement over existing results for this problem. On the technical side, this problem is closely related to matrix completion but at the same time requires new technical results, which the authors do, building upon recent advances in understanding the convex relaxation formulation of matrix completion through a nonconvex viewpoint. Overall, I feel that this paper has both interesting results as well as techniques that are more broadly interesting.
Decision: Accept (Oral)