Efficient Online Estimation of Causal Effects by Deciding What to Observe

1 Introduction . Statistical and causal modeling typically proceed from the assumption that we already know which variables are ( and are not ) observed . However , this perspective fails to address the difficult data collection decisions that precede such modeling efforts . Doctors must select a set of tests to run . Survey designers must select a slate of questions to ask . Companies must select which datasets to purchase . Whether or not we model these decisions , they pervade the practice of data science , influencing both what questions we can ask and how accurately we can answer them . One might ask , why not collect everything ? The answers are two-fold : First , data acquisition can be expensive . In a medical setting , blood tests can cost anywhere from tens to thousands of dollars . Running every test for every patient is infeasible . Likewise , space on surveys is limited , and asking every conceivable question of every respondent is infeasible . Second , in many settings , we lack complete control over the set of variables observed . Instead , we might have access to multiple data sources , each capturing a different subset of variables . Such data fusion problems pervade economic modeling and public health , and present interesting challenges : ( i ) efficiently estimating ( or even identifying ) a population parameter of interest often requires intelligently combining data from multiple sources ; ( ii ) data collection is often iterative , with tentative conclusions at each stage informing choices about what data to collect next . In this paper , we formalize the sequential problem of deciding , at each time , which data source to query ( i.e. , what to observe ) in order to efficiently estimate a target parameter . We propose online moment selection ( OMS ) , a framework that applies the generalized method of moments ( GMM ) [ 17 ] both to estimate the parameter and to decide which data sources to query . This framework can be 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . applied to estimate any statistical parameter that can be identified by a set of moment conditions . For example , OMS can address ( i ) any ( regular ) maximum likelihood estimation problem [ 13 , Page 109 ] ; and ( ii ) estimating average treatment effects ( ATEs ) using instrumental variables ( IVs ) , backdoor adjustment sets , mediators , and/or other identification strategies . Our strategy requires only that the agent has sufficient structural knowledge to formulate the set of moment conditions and that each moment can be estimated using the variables returned by at least one of the data sources . Interestingly , the optimal decisions which lead to estimates with the lowest mean squared error ( MSE ) depend on the ( unknown ) model parameters . This motivates our adaptive strategy : as we collect more data , we better estimate the underlying parameters , improving our strategy for allocating our remaining budget among the available data sources . We first address the setting where the cost per instance is equal across data sources ( Section 4 ) . First , we show that any fixed policy that differs from the oracle suffers constant asymptotic regret , as assessed by MSE . We then overcome this limitation by proposing two adaptive strategies—explorethen-commit ( OMS-ETC ) and explore-then-greedy ( OMS-ETG ) —both of which choose data sources based on the estimated asymptotic variance of the target parameter . Under OMS-ETC , we use some fraction of the sample budget to explore randomly , using the collected data to estimate the model parameters . We then exploit the current estimated model , collecting the remaining samples according to the fraction expected to minimize our estimator ’ s asymptotic variance . In OMS-ETG , we continue to update our parameter estimates after every step as we collect new data . We prove that both policies achieve zero asymptotic regret . To overcome the non-i.i.d . nature of the sample moments , we draw upon martingale theory . To derive zero asymptotic regret , we show uniform concentration of sample moments and a finite-sample concentration inequality for the GMM estimator with dependent data . Next , we adapt OMS-ETC and OMS-ETG to handle heterogeneous costs over the data sources ( Section 5 ) and prove that they still have zero asymptotic regret . Finally , we validate our findings experimentally 1 ( Section 6 ) . Motivated by ATE estimation in causal models encoded as directed acyclic graphs , we generate synthetic data from a variety of causal graphs and show that the regret of our proposed methods converges to zero . Furthermore , we see that despite being asymptotically equivalent , OMS-ETG outperforms OMS-ETC in finite samples . Finally , we demonstrate the effectiveness of our methods on two semi-synthetic datasets : the Infant Health Development Program ( IHDP ) dataset [ 19 ] and a Vietnam era draft lottery dataset [ 2 ] . 2 Related Work . Many works attempt to identify and estimate causal effects from multiple datasets . [ 4 , 21 ] study the problem of combining multiple heterogeneous datasets and propose methods for dealing with various biases . Other works study causal identification when observational and interventional distributions involving different sets of variables are available [ 28 , 40 ] . [ 12 ] introduce estimators of the ATE that efficiently combine two datasets , one where confounders are observed ( enabling the backdoor adjustment ) and another where mediators are observed ( enabling the frontdoor adjustment ) . [ 29 ] derive an estimator for the ATE in linear causal models with multiple confounders , where the confounders are observed in different datasets . Another related line of work addresses finding optimal adjustment sets for covariate adjustment [ 18 , 33 , 45 , 36 ] . While these works take for granted the collection of available datasets , we focus on the problem of deciding which data to collect . Our work shares motivation with active learning , where the learner strategically chooses which ( unlabeled ) samples to label in order to learn most efficiently [ 35 , 26 ] . [ 8 ] design algorithms for actively collecting samples in a manner that minimizes the learner ’ s variance . In settings where there is a cost associated with collecting each feature , active feature acquisition methods incrementally query feature values to improve a predictive model [ 22 , 34 , 20 ] . [ 47 ] propose an active learning criterion to find the most informative questions to ask each respondent in a survey . In the context of causal inference , [ 37 ] study active structure learning of causal DAGs by finding cost-optimal interventions , [ 11 ] demonstrate that for ATE estimation , actively deconfounding data can improve sample complexity . [ 44 ] propose strategies for acquiring missing confounders to efficiently estimate the ATE . 1The code and data are available at https : //www.github.com/acmi-lab/ online-moment-selection . Others have studied moment selection and IV selection from batch data . [ 1 ] introduce consistent moment selection procedures for the GMM setting with some incorrect moments . [ 6 ] propose an information-based lasso method for excluding invalid or redundant moment conditions . [ 41 ] propose a variable selection framework that uses lasso regression to decide which covariates to include . [ 14 ] propose four statistical criteria—including estimation efficiency and non-redundancy—for selecting among a set of candidate IVs . [ 9 ] develop an IV selection criteria based on asymptotic MSE and develop it for the GMM and generalized empirical likelihood estimators . [ 7 ] develop conservative confidence intervals for structural parameters when the off-diagonal entries of the covariance matrix of the empirical moments are unknown ( e.g. , when the moments are obtained from different datasets ) . By contrast , we are interested in selecting the data sources for the moments in an online setting . Previous works address learning from adaptively collected data . [ 25 ] propose methods for adaptive experimental design , where at each step , the experimenter must decide the treatment probability using past data in order to efficiently estimate the ATE . [ 23 , 24 ] propose a doubly-robust estimator for off-policy evaluation with dependent samples . [ 46 ] provide regret bounds for learning an optimal policy using adaptively collected data , where the probability of selecting an action is a function of past data . [ 48 , 49 ] study statistical inference for OLS and M-estimation with non-i.i.d . bandit data . While these settings are different from ours , some of the theoretical tools ( e.g. , martingale asymptotics and uniform martingale concentration bounds ) are similar . 3 Preliminaries . In the GMM framework [ 17 , 30 ] , we estimate model parameters by leveraging moment conditions that are satisfied at the true parameters θ∗ . A moment condition is a vector g ( Xt , θ ) such that E [ g ( Xt , θ ∗ ) ] = 0 . We estimate θ∗ by minimizing the objective Q̂T : θ̂T = arg min θ∈Θ Q̂T ( θ ) , where Q̂T ( θ ) = [ 1 T T∑ t=1 gt ( θ ) ] > Ŵ [ 1 T T∑ t=1 gt ( θ ) ] , Θ is the parameter space , gt ( θ ) : = g ( Xt , θ ) , and Ŵ is some ( possibly data dependent ) positive definite matrix . In this work , we use the two-step GMM estimator , where the one-step estimator θ̂ ( os ) T is computed with Ŵ : = I ( identity ) and the two-step estimator with Ŵ : = [ Ω̂T ( θ̂ ( os ) T ) ] −1 , where Ω̂T ( θ ) = [ 1 T ∑T t=1 gt ( θ ) gt ( θ ) > ] . Let V be the set of variables of interest and ψ a collection of subsets of V , each corresponding to the specific variables observable via one of the available data sources . Our methods are applicable whenever the target parameter can be identified by a set of moment conditions such that each moment relies on variables simultaneously observable in at least one data source . The selection vector , denoted by st ∈ { 0 , 1 } |ψ| , is the binary vector indicating the data source selected at time t. Assumption 1 . The agent queries one data source at each step : ∑|ψ| i=1 st , i = 1 , i.e. , st is one-hot . We can handle the querying of multiple sources by adding the union of their variables to ψ . In our setup , the moment conditions can be written as gt ( θ ) = m ( st ) ⊗ g̃t ( θ ) ∈ RM , where ⊗ is the Hadamard product , m : { 0 , 1 } |ψ| → { 0 , 1 } M is a fixed known function such that m ( st ) determines which moments get selected , and g̃t ( θ ) are i.i.d . across t. For concreteness , we instantiate our setup with a simple example : Example 1 ( Instrumental Variable ( IV ) graph ) . Consider a linear IV causal model ( Figure 2a ) with instrument Z , treatment X , outcome Y , and the following data-generating process : X : = αZ + η , Y : = βX + , 6⊥⊥ η , ⊥⊥ Z , η ⊥⊥ Z . The target parameter is the ATE β . For ψ = { { Z , X } , { Z , Y } } , the moment conditions are gt ( θ ) = [ st,1 st,2 ] ︸ ︷︷ ︸ =m ( st ) ⊗ [ Zt ( Xt − αZt ) Zt ( Yt − αβZt ) ] ︸ ︷︷ ︸ =g̃t ( θ ) = [ st,1Zt ( Xt − αZt ) ( 1− st,1 ) Zt ( Yt − αβZt ) ] , where θ = [ β , α ] > and { Zt , Xt , Yt } are i.i.d . For some known function ftar : Θ→ R , let β∗ : = ftar ( θ∗ ) be the target parameter ( e.g. , the ATE ) . In practice , we estimate the target parameter by plugging-in the GMM estimate : β̂ = ftar ( θ̂ ) . Let Ht represent the history or the data collected until time t with H0 = { } and spaceHt . A data collection policy π consists of a sequence of functions πt : Ht−1 → { 0 , 1 } |ψ| with st = πt ( Ht−1 ) . Thus st can be dependent on data collected until time ( t− 1 ) and so the sample moments gt ( θ ) are not i.i.d . Definition 1 ( Selection ratio ) . The selection ratio , denoted by κ ( π ) t , encodes the fraction of samples collected from each data source until time t : κ ( π ) t = 1 t ∑t i=1 st ∈ ∆|ψ|−1 ( standard simplex ) . We use θ̂ ( π ) t and θ̂ ( os ) t to denote the two-step and one-step GMM estimators , respectively , that use the data Ht . To reduce clutter , we use ∆ψ : = ∆|ψ|−1 , ctr ( ∆ψ ) = [ 1 |ψ| , 1 |ψ| , . . . , 1 |ψ| ] ( center of the simplex ) , and might drop the superscript π from κt , θ̂t , and β̂t . ‖.‖ denotes the spectral and l2 norms for matrices and vectors , respectively , and N ( θ ) : = { θ′ : ‖θ′ − θ‖ ≤ } ( -ball around θ ) .

Summary: Given multiple data sources, each related to a subset of variables, authors study the problem of deciding which data source to sample from at each time step. Then, by using the samples, the goal is to estimate the model parameters as good as possible. The proposed framework is based on the generalized method of moments. Their contributions are as follows:  1. In Section 4, it is assumed that each instance has the same cost across all data sources. Then, authors first show that any fixed policy that differs from the optimal one will attain a constant asymptotic regret. To overcome this issue, two adaptive methods are proposed OMS-ETC and OMS-ETG. Here, OMS-ETC resembles the classical explore-then-commit algorithm in the context of bandits, where there is an exploration step to estimate the model parameters and then the model commits to a data source that minimizes the asymptotic variance of the target parameter. In contrast, OMS-ETG keeps updating the model parameters as more data is gathered. Under certain conditions, authors show that both adaptive methods will attain a zero asymptotic regret.  2. In Section 5, both adaptive methods are modified as to attain zero regret in the case where instances have different costs across data sources.  3. Experiments are provided in Section 6, where authors depict the behavior of the regret for different causal DAGs. Semi-synthetic dataset are also provided to showcase the usability of their method.