Identification of Partially Observed Linear Causal Models: Graphical Conditions for the Non-Gaussian and Heterogeneous Cases

In causal discovery , linear non-Gaussian acyclic models ( LiNGAMs ) have been studied extensively . While the causally sufficient case is well understood , in many real applications the observed variables are not causally related . Rather , they are generated by latent variables , such as confounders and mediators , which may themselves be causally related . Existing results on the identification of the causal structure among the latent variables often require very strong graphical assumptions . In this paper , we consider partially observed linear models with either non-Gaussian or heterogeneous errors . In that case we give two graphical conditions which are necessary for identification of the causal structure . These conditions are closely related to sparsity of the causal edges . Together with one additional condition on the coefficients , which holds generically for any graph , the two graphical conditions are also sufficient for identifiability . These new conditions can be satisfied even when the number of latent variables is very large . We demonstrate the validity of our results on synthetic data . 1 Introduction . In the standard causal discovery problem , we are given non-experimental data and aim to learn the direct causal relations between the observed variables [ 1 , 2 ] . But in many applications , we do not believe that all causal variables relevant to the observed system have been measured . While some of the observed variables may interact directly , others might interact indirectly via latent mediators , and still others could be generated by latent common causes ; indeed , any pair of observed variables may stand in all three relations at once . Further , the relevant latent variables may be causally related themselves . For example , responses to psychometric questionnaires are usually thought of as noisy views of various traits , and the researcher is predominately interested in the causal relations between these hidden traits and their hierarchical structure . Similarly , in financial markets , stock returns may be causally related , but may also be confounded or mediated by a complicated network of unmeasured economic and political factors . It is natural to ask what conditions are both necessary and sufficient for the identification of such partially observed causal structures from observational data . Various sufficient conditions have been proposed ; however , these conditions are rather restrictive , and are not in general necessary for identification of the full causal structure . ∗The work presented in this article was started while JA was at CMU . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we consider the case of linear causal models in which the overcomplete mixing matrix from the noise terms to the measured variables is identifiable up to permutation and scaling of columns . This is possible , for example , in the case of independent non-Gaussian noise [ 3 ] , or when given access to heterogeneous domains in which the variances of the noise terms change independently across domains but the causal graph and weights remain constant ( see Theorem 1 of our paper ) . We provide necessary and sufficient conditions under which the latent causal structure can be uniquely identified up to trivial indeterminicies . 2 Problem setup . Suppose some causal variables V = { V1 , . . . , Vp } follow a linear structural equation model ( SEM ) V = FV + ε , ( 1 ) where V : = ( V1 , . . . , Vp ) T is a vector of causal variables , F is a causal adjacency matrix that can be permuted ( by simultaneous row and column permutations ) to strictly lower-triangular form , and ε = ( ε1 , . . . , εp ) T is a vector of independent noise variables . In this paper we consider two settings for εi : 1 ) all εi are mutually independent and non-Gaussian ; or 2 ) there are multiple domains , εi are uncorrelated within each domain , and their variances change independently across domains . We will make the second assumption technically precise Section 3 . We seek necessary and jointly sufficient conditions for identifiability of F ( up to trivial indeterminacies ) in the case where only some subset of V ( which we call X ) is measured . Thus F may encode observed-observed interactions , latent confounding , latent-latent interactions , and latent mediation or intermediate confounding . Our results identify F from the equivalence classM , as defined in Section 2.2 , of mixing matrices induced by ( 1 ) . This equivalence classM is identifiable if , for example , the errors are non-Gaussian or if their distribution changes over time or between domains . 2.1 Notation . For any matrix A and index sets J and K , we write AJK to denote the submatrix of A with columns indexed by J and rows indexed by K. Observe that AJK = IKAI J . Thus , FJK describes the direct effect of { Vj : j ∈ J } on { Vk : k ∈ K } . ( Remember : causes are up-stream of their effects . ) The graph induced by ( 1 ) has edges Vi → Vj whenever Fij 6= 0 . We write Pa ( Vi ) : = { Vj : Vi ← Vj } and Ch ( Vi ) : = { Vj : Vi → Vj } , respectively , to denote the parents and children of Vi . We say that ( V1 , ... , Vk ) constitutes a directed path from V1 to Vk if Vi → Vi+1 for every i ∈ { 1 , ... , k − 1 } . Trivially , for every Vi , ( Vi ) is a directed path from Vi to itself ; we say that a directed graph is acyclic ( a DAG ) if ( Vi ) is the only such path . We write Anc ( Vi ) : = { Vj ∈ V : Vj has a directed path to Vi } and Desc ( Vi ) : = { Vj : Vi has a directed path to Vj } , respectively , to denote the ancestors and descendants of Vi . For DAGs , notice that Anc ( Vi ) ∩Desc ( Vi ) = { Vi } , but Vi 6∈ Pa ( Vi ) ∪ Ch ( Vi ) . We assume that only some subset X ⊆ V is observed , with the remaining L = V − X being latent . We use Vi to denote a generic variable , observed or latent , whileXi ∈ X denotes an observed variable and Lj ∈ L denotes a latent variable . When it is clear from context , we occasionally suppress the distinction between a variable Vi and its index i . 2.2 Identification and minimality . Since F induces a DAG , we can always solve ( 1 ) to express the causal variables in terms of the independent noise terms : V = Mε , ( 2 ) where M : = ( I− F ) −1 ( 3 ) is the mixing matrix with Mij being the net effect of εi on Vj . This net effect is calculated by multiplying causal weights along paths and summing across paths . Notice that if Mij 6= 0 , then Vj ∈ Desc ( Vi ) . Because L is hidden , let us explicitly write X in terms of ε : X = MX ε . ( 4 ) In both the non-Gaussian and heterogeneous settings we consider in this paper , MX is identifiable up to permutation and scaling of columns ; that is , we can identify the equivalence class M = { MXDP : DP ∈ DPp } , ( 5 ) where DPp : = { DP ∈ Rp×p : D is full rank diagonal and P is a permutation matrix } . We argue this for both settings individually in Section 3 . We say that an adjacency matrix F generatesM if ( I−F ) −1X ∈M . Of course , in partially observed systems , the adjacency matrix that generatesM is not unique . However , some of these matrices are sparser than others . In causal discovery , as in model selection more broadly , we tend to prefer the “ simplest ” model that adequately fits the data [ 4 , 5 ] . As a result , without prior knowledge , a partially observed linear causal model can not be identified if the population distribution can be written in terms of an equally sparse or sparser model ; after all , we would never select a complicated model if a simpler model fits just as well . It is therefore natural to recast the question of identifiability to a question of maximal sparsity . Let the ` 0 “ norm ” of a matrix ‖ · ‖0 denote the number of non-zero entries in that matrix . Then we say that a causal adjacency matrix F is minimal with respect toM if F generatesM and ‖F̂‖0 ≥ ‖F‖0 for any F̂ 6= F that generatesM . Let F denote the class of minimal adjacency matrices that generate M. Clearly , since M is identifiable , so is F . We say that an adjacency matrix F is identified up to trivialities if F = { ( DP ) −1FDP : DP ∈ DPp with ( DP ) XX = I } . ( 6 ) The only indeterminacy remaining in F amounts to re-indexing and re-scaling the latent factors . A word of caution is in order . Because the adjacency matrix that generatesM is not unique in the partially observed case , it is only possible to talk about identification with respect to some selection principle . Throughout this work we use minimality as such a selection principle – indeed we define identification in terms of it . As justification , in Section 5.1 , we describe one class of non-minimal adjacency matrices which are pathological and whose exclusion is desirable ; further , in Section 6 , we show that existing works make assumptions even stronger than minimality ; further still , in Section 7 , we show that popular model selection criteria like BIC favor minimal graphs . Nevertheless , just as BIC is not always the most sensible criterion for model selection , so minimality is not always the most sensible principle for an identification theory . For example , Figure 1 shows a non-minimal graph which is not pathological . Thus , if a practitioner believes the true partially observed causal model to be non-minimal , they should content themselves with partial identification ( c.f . [ 6 ] ) . In Sections 4 and 5 , we express identification up to trivialities in terms of two local graphical conditions , which are much easier to check than ( 6 ) . But first , we return to the identifiability ofM . 3 Sufficient conditions for identification ofM The main results of Sections 4 and 5 rely on the identifiability ofM , which is theoretically guaranteed in the two settings we consider in this paper . In the first setting , εi are assumed to be independent and non-Gaussian . Then according to Theorem 3 by Eriksson and Koivunen [ 7 ] , M is identifiable from the distribution of X . The task of estimatingM from X is known as Overcomplete Independent Component Analysis ( OICA ) [ 3 ] , and in practice this task is known to be computationally difficult [ 8 ] . In the second setting , εi are uncorrelated from each other with changing variances across multiple domains ( or over time ) and MX has full row rank ( which is always the case for acyclic models ) . Note that in this setting , while the components of ε are mutually independent within each domain , they are not necessarily mutually independent across domains because their variances may be dependent across domains . This setting is expected to apply to a number of nonstationary scenarios including brain signal analysis , and the following theorem establishes the corresponding identifibiality ofM . Besides complementing Theorem 3 of Eriksson and Koivunen [ 7 ] as an alternative foundation for our identification work , the identifiability ofM in this setting may be of independent interest in the fields of blind source separation and system identification . Theorem 1 . Suppose we have observed X generated according to the mixing procedure ( 4 ) in a number of domains , t = 1 , 2 , ... , T , where MX has full row rank . Assume that εi are uncorrelated in each domain and that their variances in domain t , denoted by σ2ti , change independently across domains in the sense that S , whose ( t , i ) -th entry is σ2ti , has full column rank . Further assume that each |X | columns of MX are linearly independent and that p ≤ 2|X | − 2 . Then if X admits a model

Summary: This paper is about statistical causal inference. More specifically, this paper is about causal discovery that infers causal graphs. The data used is observational data without intervention. All the variables are continuous.  This paper considers linear causal models with hidden variables. This setup can be written using an overcomplete mixing matrix as in (1). This overcomplete mixing matrix is known to identifiable up to the permutation and signs of the columns if the hidden variables are non-Gaussian and independent [4]. This is known in the field of independent component analysis. When the number o the hidden variables are the same as the observed variables, it is known that changing noise variance assumption is sufficient to make the overcomplete mixing matrix identifiable even when the hidden variables are Gaussian [16]. A contribution of this paper is Theorem 1 that extends the identifiability result [16] to overcomplete cases.   Based on the two identifiability results, they further consider the sufficient and necessary conditions of the linear causal models after the overcomplete mixing matrix has been identified. They aim to identify the underlying causal model beyond the Markov equivalence class.  They further assume the sparseness of the causal adjacency matrix F, i.e., the sparsest causal model is true if multiple causal models give the same overcomplete mixing matrix M.