Identification of Partially Observed Linear Causal Models: Graphical Conditions for the Non-Gaussian and Heterogeneous Cases

In causal discovery , linear non-Gaussian acyclic models ( LiNGAMs ) have been studied extensively . While the causally sufficient case is well understood , in many real applications the observed variables are not causally related . Rather , they are generated by latent variables , such as confounders and mediators , which may themselves be causally related . Existing results on the identification of the causal structure among the latent variables often require very strong graphical assumptions . In this paper , we consider partially observed linear models with either non-Gaussian or heterogeneous errors . In that case we give two graphical conditions which are necessary for identification of the causal structure . These conditions are closely related to sparsity of the causal edges . Together with one additional condition on the coefficients , which holds generically for any graph , the two graphical conditions are also sufficient for identifiability . These new conditions can be satisfied even when the number of latent variables is very large . We demonstrate the validity of our results on synthetic data . 1 Introduction . In the standard causal discovery problem , we are given non-experimental data and aim to learn the direct causal relations between the observed variables [ 1 , 2 ] . But in many applications , we do not believe that all causal variables relevant to the observed system have been measured . While some of the observed variables may interact directly , others might interact indirectly via latent mediators , and still others could be generated by latent common causes ; indeed , any pair of observed variables may stand in all three relations at once . Further , the relevant latent variables may be causally related themselves . For example , responses to psychometric questionnaires are usually thought of as noisy views of various traits , and the researcher is predominately interested in the causal relations between these hidden traits and their hierarchical structure . Similarly , in financial markets , stock returns may be causally related , but may also be confounded or mediated by a complicated network of unmeasured economic and political factors . It is natural to ask what conditions are both necessary and sufficient for the identification of such partially observed causal structures from observational data . Various sufficient conditions have been proposed ; however , these conditions are rather restrictive , and are not in general necessary for identification of the full causal structure . ∗The work presented in this article was started while JA was at CMU . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we consider the case of linear causal models in which the overcomplete mixing matrix from the noise terms to the measured variables is identifiable up to permutation and scaling of columns . This is possible , for example , in the case of independent non-Gaussian noise [ 3 ] , or when given access to heterogeneous domains in which the variances of the noise terms change independently across domains but the causal graph and weights remain constant ( see Theorem 1 of our paper ) . We provide necessary and sufficient conditions under which the latent causal structure can be uniquely identified up to trivial indeterminicies . 2 Problem setup . Suppose some causal variables V = { V1 , . . . , Vp } follow a linear structural equation model ( SEM ) V = FV + ε , ( 1 ) where V : = ( V1 , . . . , Vp ) T is a vector of causal variables , F is a causal adjacency matrix that can be permuted ( by simultaneous row and column permutations ) to strictly lower-triangular form , and ε = ( ε1 , . . . , εp ) T is a vector of independent noise variables . In this paper we consider two settings for εi : 1 ) all εi are mutually independent and non-Gaussian ; or 2 ) there are multiple domains , εi are uncorrelated within each domain , and their variances change independently across domains . We will make the second assumption technically precise Section 3 . We seek necessary and jointly sufficient conditions for identifiability of F ( up to trivial indeterminacies ) in the case where only some subset of V ( which we call X ) is measured . Thus F may encode observed-observed interactions , latent confounding , latent-latent interactions , and latent mediation or intermediate confounding . Our results identify F from the equivalence classM , as defined in Section 2.2 , of mixing matrices induced by ( 1 ) . This equivalence classM is identifiable if , for example , the errors are non-Gaussian or if their distribution changes over time or between domains . 2.1 Notation . For any matrix A and index sets J and K , we write AJK to denote the submatrix of A with columns indexed by J and rows indexed by K. Observe that AJK = IKAI J . Thus , FJK describes the direct effect of { Vj : j ∈ J } on { Vk : k ∈ K } . ( Remember : causes are up-stream of their effects . ) The graph induced by ( 1 ) has edges Vi → Vj whenever Fij 6= 0 . We write Pa ( Vi ) : = { Vj : Vi ← Vj } and Ch ( Vi ) : = { Vj : Vi → Vj } , respectively , to denote the parents and children of Vi . We say that ( V1 , ... , Vk ) constitutes a directed path from V1 to Vk if Vi → Vi+1 for every i ∈ { 1 , ... , k − 1 } . Trivially , for every Vi , ( Vi ) is a directed path from Vi to itself ; we say that a directed graph is acyclic ( a DAG ) if ( Vi ) is the only such path . We write Anc ( Vi ) : = { Vj ∈ V : Vj has a directed path to Vi } and Desc ( Vi ) : = { Vj : Vi has a directed path to Vj } , respectively , to denote the ancestors and descendants of Vi . For DAGs , notice that Anc ( Vi ) ∩Desc ( Vi ) = { Vi } , but Vi 6∈ Pa ( Vi ) ∪ Ch ( Vi ) . We assume that only some subset X ⊆ V is observed , with the remaining L = V − X being latent . We use Vi to denote a generic variable , observed or latent , whileXi ∈ X denotes an observed variable and Lj ∈ L denotes a latent variable . When it is clear from context , we occasionally suppress the distinction between a variable Vi and its index i . 2.2 Identification and minimality . Since F induces a DAG , we can always solve ( 1 ) to express the causal variables in terms of the independent noise terms : V = Mε , ( 2 ) where M : = ( I− F ) −1 ( 3 ) is the mixing matrix with Mij being the net effect of εi on Vj . This net effect is calculated by multiplying causal weights along paths and summing across paths . Notice that if Mij 6= 0 , then Vj ∈ Desc ( Vi ) . Because L is hidden , let us explicitly write X in terms of ε : X = MX ε . ( 4 ) In both the non-Gaussian and heterogeneous settings we consider in this paper , MX is identifiable up to permutation and scaling of columns ; that is , we can identify the equivalence class M = { MXDP : DP ∈ DPp } , ( 5 ) where DPp : = { DP ∈ Rp×p : D is full rank diagonal and P is a permutation matrix } . We argue this for both settings individually in Section 3 . We say that an adjacency matrix F generatesM if ( I−F ) −1X ∈M . Of course , in partially observed systems , the adjacency matrix that generatesM is not unique . However , some of these matrices are sparser than others . In causal discovery , as in model selection more broadly , we tend to prefer the “ simplest ” model that adequately fits the data [ 4 , 5 ] . As a result , without prior knowledge , a partially observed linear causal model can not be identified if the population distribution can be written in terms of an equally sparse or sparser model ; after all , we would never select a complicated model if a simpler model fits just as well . It is therefore natural to recast the question of identifiability to a question of maximal sparsity . Let the ` 0 “ norm ” of a matrix ‖ · ‖0 denote the number of non-zero entries in that matrix . Then we say that a causal adjacency matrix F is minimal with respect toM if F generatesM and ‖F̂‖0 ≥ ‖F‖0 for any F̂ 6= F that generatesM . Let F denote the class of minimal adjacency matrices that generate M. Clearly , since M is identifiable , so is F . We say that an adjacency matrix F is identified up to trivialities if F = { ( DP ) −1FDP : DP ∈ DPp with ( DP ) XX = I } . ( 6 ) The only indeterminacy remaining in F amounts to re-indexing and re-scaling the latent factors . A word of caution is in order . Because the adjacency matrix that generatesM is not unique in the partially observed case , it is only possible to talk about identification with respect to some selection principle . Throughout this work we use minimality as such a selection principle – indeed we define identification in terms of it . As justification , in Section 5.1 , we describe one class of non-minimal adjacency matrices which are pathological and whose exclusion is desirable ; further , in Section 6 , we show that existing works make assumptions even stronger than minimality ; further still , in Section 7 , we show that popular model selection criteria like BIC favor minimal graphs . Nevertheless , just as BIC is not always the most sensible criterion for model selection , so minimality is not always the most sensible principle for an identification theory . For example , Figure 1 shows a non-minimal graph which is not pathological . Thus , if a practitioner believes the true partially observed causal model to be non-minimal , they should content themselves with partial identification ( c.f . [ 6 ] ) . In Sections 4 and 5 , we express identification up to trivialities in terms of two local graphical conditions , which are much easier to check than ( 6 ) . But first , we return to the identifiability ofM . 3 Sufficient conditions for identification ofM The main results of Sections 4 and 5 rely on the identifiability ofM , which is theoretically guaranteed in the two settings we consider in this paper . In the first setting , εi are assumed to be independent and non-Gaussian . Then according to Theorem 3 by Eriksson and Koivunen [ 7 ] , M is identifiable from the distribution of X . The task of estimatingM from X is known as Overcomplete Independent Component Analysis ( OICA ) [ 3 ] , and in practice this task is known to be computationally difficult [ 8 ] . In the second setting , εi are uncorrelated from each other with changing variances across multiple domains ( or over time ) and MX has full row rank ( which is always the case for acyclic models ) . Note that in this setting , while the components of ε are mutually independent within each domain , they are not necessarily mutually independent across domains because their variances may be dependent across domains . This setting is expected to apply to a number of nonstationary scenarios including brain signal analysis , and the following theorem establishes the corresponding identifibiality ofM . Besides complementing Theorem 3 of Eriksson and Koivunen [ 7 ] as an alternative foundation for our identification work , the identifiability ofM in this setting may be of independent interest in the fields of blind source separation and system identification . Theorem 1 . Suppose we have observed X generated according to the mixing procedure ( 4 ) in a number of domains , t = 1 , 2 , ... , T , where MX has full row rank . Assume that εi are uncorrelated in each domain and that their variances in domain t , denoted by σ2ti , change independently across domains in the sense that S , whose ( t , i ) -th entry is σ2ti , has full column rank . Further assume that each |X | columns of MX are linearly independent and that p ≤ 2|X | − 2 . Then if X admits a model

Summary: This paper provides two graphical conditions for identifying a linear causal model under non-Gaussian error or heterogenous errors across domains when a subset of variables are latent. Specifically, the paper introduces necessary (‘unique minimal bottleneck’ criterion and strong non-redundancy) and sufficient conditions (bottleneck faithfulness) for the identifiability. These conditions indeed form an identifiability condition.