Statistical Undecidability in Linear, Non-Gaussian Causal Models in the Presence of Latent Confounders

1 Introduction . Spirtes et al . [ 2000 ] develop the elements of causal discovery from observational data in the linear Gaussian setting . They show that when functional relationships between variables are linear and acyclic and noise terms are independent and Gaussian , it is possible to converge to the Markov equivalence class of the graph generating the data . Although some non-trivial causal information can be recovered , causal orientation is in general not identified : two linear Gaussian models may differ in causal orientation and nevertheless generate the exact same distribution over the observed variables . Identifiability fails even when causal faithfulness is satisfied and no hidden variables are present . For this reason , it was a significant advance when Shimizu et al . [ 2006 ] showed that , when functional relationships between variables are linear and acylic and noise terms are independent and non-Gaussian , all causal orientations can be uniquely identified from observational data , even without assuming faithfulness . However , that early result depends on the absence of unobserved confounders . Since then , the LiNGAM framework , as it came to be called , has been extended to accommodate the presence of hidden variables [ Hoyer et al. , 2008 , Salehkaleybar et al. , 2020 ] . Salehkaleybar et al . [ 2020 ] prove that if , in addition to the usual LiNGAM assumptions , we assume causal faithfulness , then causal ancestry relationships between observed variables are identified even in the presence of ∗Research Group Leader `` Ethics and Epistemology of Machine Learning '' at the Cluster of Excellence `` Machine Learning for Science '' , University of Tübingen . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . unobserved confounders . In other words : if two faithful , confounded LiNGAM models generate the same distribution over the observed variables , then for every pair of observed variables X , Y , the models must agree on whether X is causally upstream of Y , Y is upstream of X , or neither is upstream of the other . Note that the models do not have to agree on which variables are direct causes of which others , only on which variables are ancestors of which others . Moreover , although all models generating the same distribution over the observed variables must agree on the causal ancestry relations between them , they may disagree on the strength of the causal effects ( Figure 2 ) . These identifiability results are exciting theoretical developments . However , identifiability is a weak criterion and on its own does not entail the existence of a consistent discovery algorithm . Moreover , distinctions ought to be made between degrees of identifiability . For example , uniform decidability ( elsewhere known as uniform consistency ) requires that one be able to determine a priori a sample size at which the chance of identifying the true orientation is at least 1− α , no matter which causal model is generating the data . Unfortunately , it is easy to show that there is no uniformly consistent algorithm for determining the direction of a causal edge , even in unconfounded LiNGAMs ( see Example 1 in Genin and Mayo-Wilson [ 2020 ] ) . Although one could strengthen the LiNGAM assumptions to make uniform decidability feasible , these assumptions would probably be implausibly strong .2 Decidability in the limit ( elsewhere known as pointwise consistency ) requires only that for each causal model that might be generating the data , there be some sample size by which the chance of outputting the true orientation is at least 1 − α . Note that this is compatible with all kinds of short-run behavior . For example , Kelly and Mayo-Wilson [ 2010 ] show that in the unconfounded linear Gaussian setting , even in situations in which causal orientation is identifiable , it is possible to force any consistent discovery procedure to “ flip '' its judgement about whether X causes Y or vice versa . That means that for any such procedure , it is possible to find a model in which the procedure flips between being highly likely to output the correct causal orientation at a smaller sample size and highly likely to output the incorrect orientation at a larger sample size , with the number of such flips bounded only by the number of variables in the model . Genin and Mayo-Wilson [ 2020 ] introduce a new notion of decidability that is intermediate between uniform decidability and decidability in the limit . Statistical decidability requires that a discovery procedure converge to the true orientation as sample sizes increase and , at every sample size , output an incorrect orientation with probability at most α . Note that statistical decidability rules out flipping behavior , but is consistent with suspending judgement at arbitrarily large sample sizes . Genin and Mayo-Wilson [ 2020 ] show that , in the absence of unobserved confounders , causal orientation is decidable in the LiNGAM setting and therefore , flipping can be avoided . That demonstrates that statistical decision procedures may exist even when uniform decision procedures do not . 2In the linear Gaussian setting , Robins et al . [ 2003 ] show that uniform consistency is not feasible under standard assumptions . Zhang and Spirtes [ 2003 ] propose a strengthening of these assumptions to ensure uniform decidability , but Uhler et al . [ 2013 ] argues that this rules out a topologically large set of models . Bühlmann et al . [ 2014 ] give conditions strong enough the ensure uniform decidability in the general setting of additive structural equation models . Table 1 sums up : if we have a uniform decision procedure we know that the output of the method is probably correct at every sample size and we know a priori how large a sample we need for it to be informative . If we have a decision procedure , we know that the output of the method is probably correct at every sample size , but we do not know a priori how large a sample we need for it to be informative . And if we have only a limiting decision procedure , we can never be sure that the output is correct , although we know it will be probably correct eventually . Crucially , mere identifiability alone does not entail that a problem is decidable in any of the above three senses . Therefore , the results of Salehkaleybar et al . [ 2020 ] leave open the possibility that the causal ancestry relationship is not even decidable in the limit . The first main result of this paper is that the ancestry relationship is indeed decidable in the limit—there exist consistent procedures for learning the causal ancestry relationship between observed variables . Unfortunately , flipping returns in the confounded LiNGAM setting , even when we assume causal faithfulness . Although consistent procedures exist for learning causal orientation , consistent decision procedures do not . Table 2 sums up these results . However , that result suggests several adjustments to the LiNGAM framework that could recover decidability . These are discussed in Section 7 . 2 Technical Preliminaries . We first introduce notation for manipulating matrices . Suppose A is an n× p matrix and U , V are subsets of { 1 , . . . , n } , { 1 , . . . , p } , respectively . Let A [ U ; V ] be the result of dropping all rows from A that are not in U and all columns that are not in V . Let A ( U ; V ) be the result of dropping all rows from A that are in U and all columns that are in V. We write A ( i , j ) for A ( { i } ; { j } ) and Aij for A [ { i } ; { j } ] . We say A has pairwise linearly independent columns iff no two columns of A are proportional . LetM be a set of statistical models . We assume there is a function P : M 7→ PM that maps each model inM to a probability measure over a space Ω of observed outcomes , although we often do not distinguish between a random vector and the probability measure induced by its distribution function . Henceforth , we assume Ω = Rp . We lift P ( · ) to sets of models in the obvious way : if A ⊆M , let P [ A ] = { P ( M ) : M ∈ A } . Let P = P [ M ] . Let P0 ⊆ P be the probability measures in P absolutely continuous with Lebesgue measure on Ω . Let P0 [ M ] = P0 ∩ P [ M ] . If A ⊆ Rnd , let ∂A be the boundary of A in the usual topology on Rnd . The weak topology on P is defined by letting a sequence of Borel measures Pn converge weakly to P , written Pn ⇒ P iff Pn ( A ) → P ( A ) , for every A such that P ( ∂A ) = 0 . A collection of random vectors ( Xn ) converges in distribution to X iff the probability measures induced by the Xn converge weakly to the measure induced by X . We write cl ( · ) for the closure operator in the weak topology . We say that a set is locally closed iff it is the intersection of an open and a closed set . Although every open set and every closed set is also locally closed , the converse is not true : there are properly locally closed sets which are neither open nor closed . In metrizable topologies such as the topology of weak convergence , every open set , and therefore every locally closed set , is a countable union of closed sets . For any natural number k , let P kM be the k-fold product measure of PM with itself . This measure describes the probabilities of events in Rkd when we take k iid samples from PM . If the measures Pn converge weakly to P , the product measures P kn also converge weakly to P k ( see Theorem 2.8 in Billingsley [ 1986 ] ) . We define a question Q to be a countable set of disjoint subsets ofM . The elements of Q are called answers . For all M ∈ ∪Q , let Q ( M ) denote the unique answer in Q containing M. The answer to question Q is identified iff P ( M ) ̸= P ( M ′ ) whenever Q ( M ) ̸= Q ( M ′ ) . Given a question Q , we define a method λ = ⟨λn⟩n∈N to be a sequence of measurable functions λn : Ωn → Q ∪ { M } , where λn maps samples of size n to answers to the question ; a method may also take the value M to indicate that the data do no fit any particular answer sufficiently well , and so we callM the uninformative answer . We require that ∂λ−1n ( A ) has Lebesgue measure zero for all n and every answer A in the range of λn . A method is ( pointwise ) consistent for Q if for all ϵ > 0 and M ∈ ∪Q , there is n such that P kM ( λk = Q ( M ) ) > 1 − ϵ for all k ≥ n. We say that Q is decidable in the limit iff there is a consistent method for Q. Dembo and Peres [ 1994 ] give the following sufficient condition for limiting decidability . For a proof , see their Corollary 2 . Theorem 2.1 ( Dembo and Peres [ 1994 ] ) . Q is decidable in the limit if { P ( A ) : A ∈ Q } is disjoint and each P ( A ) is a countable union of sets closed in the weak topology . Given some α > 0 , say that a method λ is an α-decision procedure for Q if ( 1 ) λ is consistent for Q and ( 2 ) PnM ( M /∈ λn ) ≤ α for all M ∈ ∪Q and all sample sizes n. In other words : an α-decision procedure outputs a false hypothesis with probability at most α . A question is statistically decidable ( or simply decidable ) if there is an α-decision procedure for α > 0 . We give a simple necessary condition for statistical decidability . Theorem 2.2 . Q is statistically decidable only if P0 ( A ) ∩ clP ( B ) = ∅ for all A , B ∈ Q . Proof of Theorem 2.2 . Suppose for a contradiction that PM ∈ P ( A ) ∩ cl ( P ( B ) ) is absolutely continuous with Lebesgue measure and λ is an α-decision procedure for Q . Since λ is consistent for Q , there must be n such that PnM ( λ −1 n ( A ) ) > 1 − α . Since ∂λ−1n ( A ) has Lebesgue measure zero and PnM is absolutely continuous with Lebesgue measure on Rnp , PnM ( ∂λ−1n ( A ) ) = 0.3 Therefore , there are ( Mi ) in B such that PnMi ( A ) → P n M ( A ) . But then there is some Mj ∈ B such that PnMj ( Mj /∈ λn ) > α . Contradiction . It is worth introducing some intuitive language for questions Q with only one ( usually non-exhaustive ) answer A ⊆M . We say that A is statistically verifiable iff Q = { A } is decidable . Say that A is statistically refutable iff Q = { M \ A } is decidable . For partial converses of Theorems 2.1 and 2.2 , see Genin and Kelly [ 2017 ] . Essentially , the converses hold straightforwardly if all distributions are assumed absolutely continuous with Lebesgue measure . The fundamental result of this paper is that the question of whether ( observed ) X is an ancestor of ( observed ) Y or vice-versa , while decidable in the limit , is not decidable . The former follows from Corollary 6.7 and an appeal to Theorem 2.1 . The latter is proven by showing that you can approximate to an arbitrary degree of precision the distribution over ( X , Y ) generated by a faithful LiNGAM in which X → Y by a sequence of distributions ( Xm , Ym ) generated by faithful but confounded LiNGAMs in which Xm ← Ym ( see Figure 3 and Lemma 6.2 ) . Undecidability follows by appeal to Theorem 2.2 .

Summary: This paper is about statistical causal inference. More specifically, this paper is about the causal discovery that infers causal graphs. Many methods focus on investigating the identifiability of various models for causal discovery and proposing estimation methods for the identifiable models. This paper points out that the identifiability does not necessarily imply that desirably consistent estimations methods exist.   There are many types of consistency, including uniform consistency and point-wise consistency. This paper investigates which consistency can be achieved for linear non-Gaussian acyclic models with hidden common causes, confounded LiNGAM (Hoyer et al., 2008). This analysis is based on a consistency concept named statistical decidability proposed in Genin and Mayo-Wilson (2020).   Their main result is summarized in Table 1. Genin and Mayo-Wilson (2020) showed that LiNGAM with latent confounders (Shimizu et al., 2006) is (statistically) decidable with/without the faithfulness assumption. Faithfulness is a common assumption in causal discovery (Spirtes et al., 1993). The causal ordering of variables, ancestral relationships, of confounded LiNGAM has been shown in identifiable if faithfulness is assumed (Hoyer et al., 2008; Salehkaleybar et al., 2020). If faithfulness is not assumed, it is not decidable. The definitions of decidability are on pages 2 and 3. Theorem 1.1 and 1.2 say how one could check whether such consistency hold. Their very main result is Theorem 6.1 in the final section. All the other sections are written to prepare for giving the theorem.  ---- I updated the score and confidence.