Answering Complex Causal Queries With the Maximum Causal Set Effect

1 Introduction . Recent advances in machine learning technology have made it possible to non-parametrically estimate many parameters present in complex structural causal models ( SCMs ) . Specifically , such estimating technology has rapidly advanced for three major causal inference settings : the many causes setting , the many moderators setting , and the many mediators setting . All three settings represent a situation in which a particular causal query can be stated in terms of a large number of combinations of different variables . Specifically , a researcher could estimate a different treatment effect associated with each of the many different possible combinations of causes [ Imbens , 2000 , Wang and Blei , 2019 , Li et al. , 2019 , Wang et al. , 2018 , Zheng et al. , Forthcoming ] , a different conditional treatment effect for each of the many different combinations of moderators [ Green and Kern , 2012 , Athey and Imbens , 2016 , Grimmer et al. , 2017 , Wager and Athey , 2018 , Künzel et al. , 2019 ] , and a different mediated effect for each of the many different combinations of mediators [ Zhou and Yamamoto , 2020 , Daniel et al. , 2015 ] . Such causal queries are complex in the sense that they require summarizing the combined influence of a large number of causal variables . The main challenge for applied researchers in such settings is that standard causal inference algorithms are designed to provide a different estimate associated with each of the many causal variables rather than a single number summarizing the combined influence of all the causal variables together . Consider , for example , the setting of inferring the causal effect of actors on a film ’ s box office performance . Wang and Blei [ 2019 ] provide a framework for estimating the average treatment effect associated with every actor on a film ’ s performance . While certainly useful for making predictions ∗PHD Candidate , Interdisciplinary Doctoral Program in Statistics and Political Science , https : //zmarkovich.github.io/ 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . about which actors a director should cast , an economist studying the film industry might prefer a single number which summarizes the general importance of actors in general for a film ’ s box office success . As discussed in the next section , such settings are common in scientific research , suggesting the need for novel causal estimands to parameterize the predictions of such theories in the context of a particular SCM . Contribution The contribution of this paper is threefold . First , it introduces the notion of a complex causal query and argues that existing causal estimands are of limited utility to applied researchers in the face of such queries . Second , it defines a novel estimand – the Maximum Causal Set Effect ( MCSE ) – which can be used to provide an interpretable answer to such complex queries . Finally , the paper introduces an estimator for this estimand . The estimator is based on techniques proposed in the double Q-learning literature [ Hasselt , 2010 ] and is asymptotically consistent and conservative in finite samples under assumptions that are standard in the causal inference literature . It is also asymptotically normal and amenable to non-parametric bootstrap techniques , facilitating classical statistical inference about the MCSE . 2 Setting and Previous Work . 2.1 Problem Overview . Standard approaches to causal inference [ Pearl , 2009 ] typically begin with the researcher specifying an SCM and then defining a causal query which can be answered based on the assumed SCM . Under certain assumptions about the SCM , it may be possible to estimate the answer to that causal query using the conventional tools of statistical inference . The standard tools of causal inference are designed with settings in mind where the predictions of a scientific theory take the form of a simple causal query . Such queries are stated in terms of some low dimensional causal variable t and some outcome Y . For example , a question like how much does a medical procedure reduce the risk of disease , represents a simple causal query because it is defined in terms of a single unidimensional treatment . Such queries can be easily quantified using conventional statistical estimands because they are directly formulated in terms of a small number of theoretically motivated variables . This paper instead focuses on situations where a scientific theory makes diffuse predictions about the importance of a large number of causal variables , defying the stylization of simple causal queries . Such queries are common in scientific research . For example : • Genome Wide Association Studies ( GWAS ) – GWAS attempt to quantify the causal effect of a huge number of individual genotypes on the likelihood that some trait is expressed [ Stephens and Balding , 2009 , Visscher et al. , 2017 ] . • Personality – psychologists are often interested in the effect certain personality traits ( such as extraversion or neuroticism ) might have on life outcomes [ Pervin , 2003 ] , but such traits are only observed by the researcher as responses to a large number of survey questions . • Text – language is complex and multi-faceted and the causal effect of the wording of a document on a user ’ s response requires an assessment of the contribution of many different topics or words together [ Fong and Grimmer , 2016 , Egami et al. , 2018 , Fong and Grimmer , forthcoming ] . • Complex medical treatments – many medical treatments can not be reduced to a single low dimensional representation . For example , radiation exposure is observed as a high dimensional vector [ Nabi et al. , 2017 ] and medical researchers might also wish to understand the combined importance of many procedures using electronic medical records [ Gottesman et al. , 2013 ] . Such causal queries are complex because they require estimating the joint influence of many causal variables . The SCM undergirding such complex queries can take many forms . Three major examples are : ( a ) the many causes setting where the researcher wishes to understand the joint influence of many treatments ( b ) many moderators setting where the researcher wishes to understand how effect of a binary treatment varies based on many variables ( c ) the many mediators setting where the researcher wishes to model how a causal effect can be decomposed into many different pieces . These SCM ’ s are Note : Figure visualizes SCMs corresponding to complex causal queries in the case of ( a ) many causes , ( b ) many moderators , and ( c ) many mediators . ( a ) visualizes the case where treatment types t0 . . . tn each influence y described in Wang and Blei [ 2019 ] . ( b ) visualizes the case where the causal effect of t0 on y is directly modified by t1 . . . tn as described in VanderWeele and Robins [ 2007 ] . ( c ) visualizes the case where the effect of t0 on y is mediated by t1 . . . tn as described by Zhou and Yamamoto [ 2020 ] . visualized in Figure 1 in the form of directed acylical graphs ( DAGs ) . The unifying trait of a complex causal query is that it asks about the importance of many arrows present in each DAG . Techniques developed in the context of simple causal queries can not be readily used to answer complex ones . While the standard tools of causal inference can be used to estimate causal effects corresponding to every combination of causal variables in SCM ’ s like those visualized in Figure 1 , they do not provide applied researchers with a single unambiguous estimate with which to summarize the joint causal effect of many such variables . 2.2 Previous Work . The only existent proposal for addressing the challenge presented by complex causal queries in the machine learning literature is to dimension reduce the relevant causal variables and then focus on a simple causal query defined in terms of that latent trait [ Fong and Grimmer , 2016 , forthcoming , Nabi et al. , 2017 ] . This strategy has only been proposed in the many causes setting , but could also be extended to the many moderators or many mediators cases as well . Such a strategy is inherently reductive and risks understating the magnitude of causal effects because it disregards all variation in the treatment types that is not accounted for in the latent trait . Additionally such latent traits are often scale invariant and so may lack a scientifically meaningful interpretation . 2.3 Assumptions and Notation . We assume that the researcher observes a set of N independent ( ti , Yi , xi ) triplets where Yi is the outcome , and ti is a length K vector indicating the treatment type received by unit i , and xi is a length J vector representing a set of background covariates that causal effects should be adjusted for . Additionally , let T denote the support of the distribution of ti . We also assume that the researcher has knowledge of the population distribution of ti : g ( t ) . In many settings , the empirical distribution of ti will be the most logical choice , but other choices may be reasonable as well if the population distribution is known to the researcher , as might be the case when conducting survey research or if the treatment types were experimentally randomized . Finally , we assume that the researcher has specified some SCM and has specified a simple causal query , τ ( T ′ , T ′′ ) , which is defined in terms of two subsets : T ′ , T ′′ ⊆ T . In the many causes case , τ ( T ′ , T ′′ ) might take the form : τ ( T ′ , T ′′ ) ≡ E ( E ( Yi|do ( t ) ) |t ∈ T ′ ) − E ( E ( Yi|do ( t ) ) |t ∈ T ′′ ) where do ( · ) represents some causal intervention [ Pearl , 2009 ] . This estimand represents the average effect of receiving a set of treatments contained in T ′ rather than T ′′ . In the many moderators case on the other hand let the zeroth element of the treatment types vector recieved by unit i , ti,0 ∈ { 0 , 1 } , denote the level of some binary treatment received by unit i . Similarly , let the remaining elements of ti be denoted ti , −0 and indicate the level received by unit i on the many moderators . Then a possible choice for τ ( T ′ , T ′′ ) might be : τ ( T ′ , T ′′ ) ≡ E ( E ( Yi|do ( ti,0 = 1 ) ) − E ( Yi|do ( ti,0 = 0 ) ) |ti , −0 ∈ T ′ ) − E ( E ( Yi|do ( ti,0 = 1 ) ) − E ( Yi|do ( ti,0 = 0 ) ) |ti , −0 ∈ T ′′ ) which represents the difference in average treatment effects between units with moderators contained in T ′ rather than T ′′ . Defining such estimands in the many mediators case requires more cumbersome notation , but can be accomplished in terms of an average of different combinations of path specific effects [ Daniel et al. , 2015 , Zhou and Yamamoto , 2020 ] . 3 The Maximum Causal Set Effect . The challenge for applied researchers in the presence of such complex causal queries is that a different value of τ ( T ′ , T ′′ ) can be defined for every distinct pair of sets T ′ , T ′′ ⊆ T , leaving the analyst without a single unambiguous causal estimand to summarize their findings . In this section , we define a causal quantity of interest which overcomes this challenge by focusing on the contrast between two sets T Maxq and T Minq which maximize τ ( T ′ , T ′′ ) . To avoid choosing sets T Maxq and T Minq which correspond to unrepresentative edge cases , we require that the sets be of a researcher specified size : q . Formally , let the set of subsets of T such that the probability that ti is in T is at least q be defined as : Tq ≡ { T ′ ⊆ T : P ( ti ∈ T ′ ) ≥ q } where P ( ti ∈ T ′ ) = ∫ T g ( t ) 1 { t ∈ T } dt . We then define MCSEq as : MCSEq = max T ′ , T ′′∈Tq τ ( T ′ , T ′′ ) = τ ( T Maxq , T Minq ) We refer to T Maxq as the maximum causal set and T Minq as the minimum causal set . For many applications , the MCSE will have an intuitive and scientifically meaningful interpretation . In the actors example , it might be used to answer a question like what is the expected difference in box office performance between a film cast with one of the 10 % best performing casts rather than one of the bottom 10 % worst performing casts ? Similarly , in the genetics example , it might answer the question , what is the difference in the efficacy of some drug for patients with one of the top 10 % most treatment enhancing sets of genes rather than one of the bottom 10 % most treatment diminishing sets of genes ?

Summary: This paper proposes a single-number measure called the maximum causal set effect to provide a simple and informative answer to complex causal queries that feature many causal variables. The measure is roughly the biggest difference alternative treatments can make, among treatments that appear sufficiently often in the population. An estimator for this quantity is developed and some useful statistical properties of the estimator are established. Empirical results are also presented, including an interesting application to real data from political science.