SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lingjing Kong $^{1}$ , Biwei Huang $^{2}$ , Feng Xie $^{3}$ , Eric Xing $^{1,4}$ , Yuejie Chi $^{1}$ , and Kun Zhang $^{1,4}$
$^{1}$ Carnegie Mellon University  
 $^{2}$ University of California San Diego  
 $^{3}$ Beijing Technology and Business University  
 $^{4}$ Mohamed bin Zayed University of Artificial Intelligence
Identifying latent variables and causal structures from observational data is essential to many real-world applications involving biological data, medical data, and unstructured data such as images and languages. However, this task can be highly challenging, especially when observed variables are generated by causally related latent variables and the relationships are nonlinear. In this work, we investigate the identification problem for nonlinear latent hierarchical causal models in which observed variables are generated by a set of causally related latent variables, and some latent variables may not have observed children. We show that the identifiability of causal structures and latent variables (up to invertible transformations) can be achieved under mild assumptions: on causal structures, we allow for multiple paths between any pair of variables in the graph, which relaxes latent tree assumptions in prior work; on structural functions, we permit general nonlinearity and multi-dimensional continuous variables, alleviating existing work's parametric assumptions. Specifically, we first develop an identification criterion in the form of novel identifiability guarantees for an elementary latent variable model. Leveraging this criterion, we show that both causal structures and latent variables of the hierarchical model can be identified asymptotically by explicitly constructing an estimation procedure. To the best of our knowledge, our work is the first to establish identifiability guarantees for both causal structures and latent variables in nonlinear latent hierarchical models.
Classical causal structure learning algorithms often assume no latent confounders. However, it is usually impossible to enumerate and measure all task-related variables in real-world scenarios. Neglecting latent confounders may lead to spurious correlations among observed variables. Hence, much effort has been devoted to handling the confounding problem. For instance, Fast Casual Inference and its variants [Spirtes et al., 2000, Pearl, 2000, Colombo et al., 2012, Akbari et al., 2021] leverage conditional independence constraints to locate possible latent confounders and estimate causal structures among observed variables, assuming no causal relationships among latent variables. This line of approaches ends up with an equivalence class, which usually consists of many directed acyclic graphs (DAGs).
Later, several approaches have been proposed to tackle direct causal relations among latent variables with observational data. These approaches are built upon principles such as rank constraints [Silva et al., 2006a, Kummerfeld and Ramsey, 2016, Huang et al., 2022], matrix decomposition [Chandrasekaran et al., 2011, 2012, Anandkumar et al., 2013], high-order moments [Shimizu et al., 2009, Cai et al., 2019, Xie et al., 2020, Adams et al., 2021, Chen et al., 2022], copula models [Cui et al., 2018], and mixture oracles [Kivva et al., 2021]. Pearl [1988], Zhang [2004], Choi et al. [2011], Drton et al. [2017], Zhou et al. [2020], Huang et al. [2020] extend such approaches to handle tree structures
(b)
Figure 1: Examples of latent hierarchical graphs satisfying Conditions 2.4.  $\mathbf{x}_i$  denotes observed variables and  $\mathbf{z}_j$  denotes latent variables. The node size represents the dimensionality of each variable. We note that our graph condition permits multiple paths between two latent variables (e.g.,  $\mathbf{z}_1$  and  $\mathbf{z}_4$  in Figure 1b), thus more general than tree structures.
where only one path is permitted between any pair of variables. Recently, Huang et al. [2022] propose to use rank-deficiency constraints to identify more general latent hierarchical structures. A common assumption behind these approaches is that either the causal relationships should be linear or the variables should be discrete. However, the linearity and discrete-variable assumptions are rather restrictive in real-world scenarios. Moreover, these approaches only focus on structure learning without identifying the latent variables and corresponding causal models. We defer a detailed literature review to Appendix A.
In this work, we identify the hierarchical graph structure and latent variables simultaneously for general nonlinear latent hierarchical causal models. Specifically, we first develop novel identifiability guarantees on a specific latent variable model which we refer to as the basis model (Figure 2). We then draw connections between the basis-model identifiability and the identification of nonlinear latent hierarchical models. Leveraging this connection, we develop an algorithm to localize and estimate latent variables and simultaneously learn causal structures underlying the entire system. We show the correctness of the proposed algorithm and thus obtain identifiability of both latent hierarchical causal structures and latent variables for nonlinear latent hierarchical models. In sum, our main contributions are as follows.
- Analogous to independence tests in PC algorithm [Spirtes et al., 2000] and rank-deficiency tests in Huang et al. [2022], we develop a novel identifiability theory (Theorem 3.2) as a fundamental criterion for locating and identifying latent variables in general nonlinear causal models.
- We show structure identification guarantees for latent hierarchical models admitting continuous multi-dimensional (c.f., one-dimensional and often discrete [Pearl, 1988, Choi et al., 2011, Huang et al., 2022, Xie et al., 2022]) variables, general nonlinear (c.f., linear [Pearl, 1988, Huang et al., 2022, Xie et al., 2022]) structural functions, and general graph structures (c.f., generalized trees [Pearl, 1988, Choi et al., 2011, Huang et al., 2022]).
- Under the same conditions, we provide identification guarantees for latent variables up to invertible transformations, which, to the best of our knowledge, is the first attempt in cases of nonlinear hierarchical models.
- We accompany our theory with an estimation method that can asymptotically identify the causal structure and latent variables for nonlinear latent hierarchical models and validate it on multiple synthetic and real-world datasets.
2 Nonlinear Latent Hierarchical Causal Models
In this section, we introduce notations and formally define the latent hierarchical causal model. We focus on causal models with directed acyclic graph (DAG) structures and denote the graph with  $\mathbf{G}$ , which consists of the latent variable set  $\mathbf{Z} := \{\mathbf{z}_1, \dots, \mathbf{z}_m\}$ , the observed variable set  ${}^1\mathbf{X} := \{\mathbf{x}_1, \dots, \mathbf{x}_n\}$ , and the edge set  $\mathbf{E}$ . Each variable is a random vector comprising potentially multiple dimensions, i.e.,  $\mathbf{x}_i \in \mathbb{R}^{d_{x_i}}$  and  $\mathbf{z}_j \in \mathbb{R}^{d_{z_j}}$ , where  $d_{x_i}$  and  $d_{z_j}$  stand for the dimensionalities of  $\mathbf{x}_i \in \mathbf{X}$  and  $\mathbf{z}_j \in \mathbf{Z}$ , respectively. Both latent variables and observed variables are generated recursively by their latent parents:
$$
\mathbf {x} _ {i} = g _ {x _ {i}} \left(\operatorname {P a} \left(\mathbf {x} _ {i}\right), \varepsilon_ {x _ {i}}\right) \quad \mathbf {z} _ {j} = g _ {z _ {j}} \left(\operatorname {P a} \left(\mathbf {z} _ {j}\right), \varepsilon_ {z _ {j}}\right), \tag {1}
$$
<sup>1</sup>We refer to leaf variables in the graph as observed variables for ease of exposition. The theory in this work is also applicable when some non-leaf variables happen to be observed, which we discuss in Appendix C.2
where  $\mathrm{Pa}(\mathbf{z}_j)$  represents all the parent variables of  $\mathbf{z}_j$  and  $\varepsilon_{z_j}$  represents the exogenous variable generating  $\mathbf{z}_j$ . Identical definitions apply to those notations involving  $\mathbf{x}_i$ . All exogenous variables  $\varepsilon_{x_i}, \varepsilon_{z_j}$  are independent of each other and could also comprise multiple dimensions. We now define a general latent hierarchical causal model with a causal graph  $\mathbf{G} := (\mathbf{Y}, \mathbf{E})$  in Definition 2.1.
Definition 2.1 (Latent Hierarchical Causal Models). The variable set  $\mathbf{Y}$  consists of observed variable set  $\mathbf{X}$  and latent variable set  $\mathbf{Z}$ , and each variable is generated by Equation 1.
In light of the given definitions, we formally state the objectives of this work:
1. Structure identification: given observed variables  $\mathbf{X}$ , we would like to recover the edge set  $\mathbf{E}$ .
2. Latent-variable identification: given observed variables  $\mathbf{X}$ , we would like to obtain a set of variables  $\hat{\mathbf{Z}} := \{\hat{\mathbf{z}}_1, \dots, \hat{\mathbf{z}}_m\}$  where for  $i \in [m]$ ,  $\mathbf{z}_i$  and  $\hat{\mathbf{z}}_i$  are identical up to an invertible mapping, i.e.,  $\hat{\mathbf{z}}_i = h_i(\mathbf{z}_i)$ , where  $h_i$  is an invertible function.
Definition 2.1 gives a general class of latent hierarchical causal models. On the functional constraint, the general nonlinear function form (Equation 1) renders it highly challenging to identify either the graph structure or the latent variables. Prior work Pearl [1988], Choi et al. [2011], Xie et al. [2022], Huang et al. [2022] relies on either linear model conditions or discrete variable assumptions. In this work, we remove these constraints to address the general nonlinear case with continuous variables. On the structure constraint, identifying arbitrary causal structures is generally impossible with only observational data. For instance, tree-like structures are assumed in prior work [Pearl, 1988, Choi et al., 2011, Huang et al., 2022] for structural identifiability. In the following, we present relaxed structural conditions under which we show structural identifiability.
Definition 2.2 (Pure Children).  $\mathbf{v}_i$  is a pure child of another variable  $\mathbf{v}_j$ , if  $\mathbf{v}_j$  is the only parent of  $\mathbf{v}_i$  in the graph, i.e.,  $\mathrm{Pa}(\mathbf{v}_i) = \{\mathbf{v}_j\}$ .
Definition 2.3 (Siblings). Variables  $\mathbf{v}_i$  and  $\mathbf{v}_j$  are siblings if  $\mathrm{Pa}(\mathbf{v}_i) \cap \mathrm{Pa}(\mathbf{v}_j) \neq \emptyset$ .
Condition 2.4 (Structural Conditions).
i Each latent variable has at least 2 pure children.
ii There are no directed paths between any two siblings.
Intuitively, Condition 2.4-i allows each latent variable to leave a footprint sufficient for identification. This excludes some fundamentally unidentifiable structures. For instance, if latent variables  $\mathbf{z}_1$  and  $\mathbf{z}_2$  share the same children  $\mathbf{X}$ , i.e.,  $\mathbf{z}_1 \rightarrow \mathbf{X}$  and  $\mathbf{z}_2 \rightarrow \mathbf{X}$ , the two latent variables cannot be identified without further assumptions, while pure children would help in this case. The pure child assumption naturally holds in many applications with many directly measured variables, including psychometrics, image analysis, and natural languages. We require fewer pure children than prior work [Silva et al., 2006a, Kummerfeld and Ramsey, 2016] and place no constraints on the number of neighboring variables as in prior work [Huang et al., 2022, Xie et al., 2022].
Condition 2.4-II avoids potential triangle structures in the latent hierarchical model. Triangles present significant obstacles for identification – in a triangle structure formed by  $\mathbf{z}_1 \rightarrow \mathbf{z}_2 \rightarrow \mathbf{z}_3$  and  $\mathbf{z}_1 \rightarrow \mathbf{z}_3$ , it is difficult to discern how  $\mathbf{z}_1$  influences  $\mathbf{z}_3$  without functional constraints, as there are two directed paths between them. We defer the discussion of how to use our techniques to handle more general graphs that include triangles to Appendix C.1. Condition 2.4-II is widely adopted in prior work, especially those on tree structures [Pearl, 1988, Choi et al., 2011, Drton et al., 2017] (which cannot handle multiple undirected paths between variables as we do) and more recent work [Huang et al., 2022]. To the best of our knowledge, only Xie et al. [2022] manage to handle triangles in the latent hierarchical structure, which, however, heavily relies on strong assumptions on both the function class (linear functions), distribution (non-Gaussian), and structures (the existence of neighboring variables).
3 Identifiability of Nonlinear Latent Hierarchical Causal Models
This section presents the identifiability and identification procedure of causal structures and latent variables in nonlinear latent hierarchical causal models, from only observed variables. First, in Section 3.1, we introduce a latent variable model (i.e., the basis model in Figure 2), whose identifiability
2We are concerned about representation vectors corresponding to each individual variable. Thus, identifiability up to invertible transforms suffices. Generally, stronger identifiability (e.g., component-wise) necessitates additional assumptions, e.g., multiple distributions [Khemakhem et al., 2020], sparsity assumptions [Zheng et al., 2022].
Figure 2: The basis model.  $\mathbf{v}_1$  and  $\mathbf{v}_2$  are generated from nonlinear functions  $g_{1}$  and  $g_{2}$  which can be distinct and non-invertible individually. We also admit dependence between  $\mathbf{z}$  and  $\mathbf{s}_2$ , as indicated by the dashed edge.
underpins the identifiability of the hierarchical model. In Section 3.2, we establish the connection between the basis model and the hierarchical model. Leveraging this, in Section 3.3, we show by construction that both causal structures and latent variables are identifiable in the hierarchical model.
3.1 Basis Model Identifiability
We present the data-generating process of the basis model in Equation 2 and illustrate it in Figure 2:
$$
\mathbf {v} _ {1} = g _ {1} (\mathbf {z}, \mathbf {s} _ {1}) \quad \mathbf {v} _ {2} = g _ {2} (\mathbf {z}, \mathbf {s} _ {2}), \tag {2}
$$
where  $\mathbf{v}_1\in \mathcal{V}_1\subset \mathbb{R}^{d_{v_1}}$ $\mathbf{v}_2\in \mathcal{V}_2\subset \mathbb{R}^{d_{v_2}}$ $\mathbf{z}\in \mathcal{Z}\subset \mathbb{R}^{d_z}$ $\mathbf{s}_1\in S_1\subset \mathbb{R}^{d_{s_1}}$  , and  $\mathbf{s}_2\in S_2\subset \mathbb{R}^{d_{s_2}}$  are variables consisting of potentially multiple components. The data-generating process admits potential dependence between  $\mathbf{z}$  and  $\mathbf{s}_2$  and dependence across dimensions in each variable. We denote the entire latent variable  $\mathbf{c}\coloneqq [\mathbf{z},\mathbf{s}_1,\mathbf{s}_2]\in \mathcal{C}$  and the mapping from  $\mathbf{c}$  to  $(\mathbf{v}_1,\mathbf{v}_2)$  as  $g:\mathcal{C}\to \mathcal{V}_1\times \mathcal{V}_2$
Below, we introduce notations and basic definitions that we use throughout this work.
Indexing: For a matrix  $\mathbf{M}$ , we denote its  $i$ -th row as  $\mathbf{M}_{i,j}$ , its  $j$ -th column as  $\mathbf{M}_{i,j}$ , and its  $(i,j)$  entry as  $\mathbf{M}_{i,j}$ . Similarly, for an index set  $\mathcal{I} \subseteq \{1,\dots,m\} \times \{1,\dots,n\}$ , we denote  $\mathcal{I}_{i,j} := \{j:(i,j) \in \mathcal{I}\}$  and  $\mathcal{I}_{i,j} := \{i:(i,j) \in \mathcal{I}\}$ .
Subspaces: We denote a subspace of  $\mathbb{R}^n$  defined by an index set  $S$  as  $\mathbb{R}_S^n$ , where  $\mathbb{R}_S^n := \{\mathbf{z} \in \mathbb{R}^n : \forall i \notin S, z_i = 0\}$ .
Support of matrix-valued functions: We define the support of a matrix-valued function  $\mathbf{M}(\mathbf{x}) : \mathcal{X} \to \mathbb{R}^{m \times n}$  as  $\operatorname{Supp}(\mathbf{M}) := \{(i,j) : \exists \mathbf{x} \in \mathcal{X}, \text{s.t.} \mathbf{M}(\mathbf{x})_{i,j} \neq 0\}$ , i.e., the set of indices whose corresponding entries are nonzero for some  $\mathbf{x} \in \mathcal{X}$ .
In Theorem 3.2 below, we show that the latent variable  $\mathbf{z}$  is identifiable up to an invertible transformation by estimating a generative model  $(\hat{p}_{\mathbf{z},\mathbf{s}_2},\hat{p}_{\mathbf{s}_1},\hat{g})$  according to Equation 2. We denote Jacobian matrices of  $g$  and  $\hat{g}$  as  $\mathbf{J}_g$  and  $\mathbf{J}_{\hat{g}}$ , and their supports as  $\mathcal{G}$  and  $\hat{\mathcal{G}}$ , respectively. Further, we denote as  $\mathcal{T}$  a set of matrices with the same support as that of the matrix-valued function  $\mathbf{J}_g^{-1}(\mathbf{c})\mathbf{J}_{\hat{g}}(\hat{\mathbf{c}})$ .
Assumption 3.1 (Basis model conditions).
i [Differentiability & Invertibility]: The mapping  $g(\mathbf{c}) = (\mathbf{v}_1, \mathbf{v}_2)$  is a differentiable invertible function with a differentiable inverse.
ii [Subspace span]: For all  $i \in \{1, \ldots, d_{v_1} + d_{v_2}\}$ , there exists  $\{\mathbf{c}^{(\ell)}\}_{\ell=1}^{\left|\mathcal{G}_{i,:}\right|}$  and  $\mathbf{T} \in \mathcal{T}$ , such that  $\text{span}(\{\mathbf{J}_g(\mathbf{c}^{(\ell)})_{i,:}\}_{\ell=1}^{\left|\mathcal{G}_{i,:}\right|}) = \mathbb{R}_{\mathcal{G}_{i,:}}^{d_c}$  and  $[\mathbf{J}_g(\mathbf{c}^{(\ell)})\mathbf{T}]_{i,:} \in \mathbb{R}_{\hat{\mathcal{G}}_{i,:}}^{d_c}$ .
iii [Edge Connectivity]: For all  $j_z \in \{1, \ldots, d_z\}$ , there exist  $i_{v_1} \in \{1, \ldots, d_{v_1}\}$  and  $i_{v_2} \in \{d_{v_1}, \ldots, d_{v_1} + d_{v_2}\}$ , such that  $(i_{v_1}, j_z) \in \mathcal{G}$  and  $(i_{v_2}, j_z) \in \mathcal{G}$ .
Theorem 3.2. Under Assumption 3.1, if a generative model  $(\hat{p}_{\mathbf{z},\mathbf{s}_2},\hat{p}_{\mathbf{s}_1},\hat{g})$  follows the data-generating process in Equation 2 and matches the true joint distribution:
$$
p _ {\mathbf {v} _ {1}, \mathbf {v} _ {2}} (\mathbf {v} _ {1}, \mathbf {v} _ {2}) = \hat {p} _ {\mathbf {v} _ {1}, \mathbf {v} _ {2}} (\mathbf {v} _ {1}, \mathbf {v} _ {2}), \forall (\mathbf {v} _ {1}, \mathbf {v} _ {2}) \in \mathcal {V} \times \mathcal {V}, \tag {3}
$$
then the estimated variable  $\hat{\mathbf{z}}$  and the true variable  $\mathbf{z}$  are equivalent up to an invertible transformation.
Interpretation. Intuitively, we can infer the latent variable  $\mathbf{z}$  shared by the two observed variables  $\mathbf{v}_1$  and  $\mathbf{v}_2$ , in a sense that the estimated variable  $\hat{\mathbf{z}}$  contains all the information of  $\mathbf{z}$  without mixing any information from  $\mathbf{s}_1$  and  $\mathbf{s}_2$ . The identifiability up to an invertible mapping is extensively employed in identifiability theory [Casey and Westner, 2000, Hyvarinen and Hoyer, 2000, Le et al., 2011, Theis, 2006, von Kugelgen et al., 2021, Kong et al., 2022].
Discussion on assumptions. Assumption 3.1-i assumes the mapping  $g$  preserves latent variables' information, guaranteeing the possibility of identification. Such assumptions are universally adopted in the existing literature on nonlinear causal model identification [Hyvarinen et al., 2019, Khemakhem et al., 2020, Kong et al., 2022, Lyu et al., 2022, von Kugelgen et al., 2021].
Assumption 3.1-ii guarantees that the influence of  $\mathbf{z}$  changes adequately across its domain, as discussed in prior work [Lachapelle et al., 2022, Zheng et al., 2022]. This eliminates situations where the Jacobian matrix is partially constant, causing it to insufficiently capture the connection between the observed and latent variables. This condition is specific for nonlinear functions, and a counterpart can be derived for linear cases [Zheng et al., 2022].
Assumption 3.1-iii is a formal characterization of the common cause variable  $\mathbf{z}$  in the basis model. This assumption excludes the scenario where some dimensions of  $\mathbf{z}$  only influence one of  $\mathbf{v}_1$  and  $\mathbf{v}_2$ , in which case these dimensions of  $\mathbf{z}$  are not truly the common cause of  $\mathbf{v}_1$  and  $\mathbf{v}_2$  and should be modeled as a separate variable rather than part of  $\mathbf{z}$ . This is equivalent to causal minimality, which is necessary for casual structure identification for general function classes with observational data [Peters et al., 2017].
Comparison with prior work. In contrast to similar data-generating processes in prior work [Lyu et al., 2022, von Kugelgen et al., 2021], Theorem 3.2 allows the most general conditions. First, Theorem 3.2 does not necessitate the invertibility for  $g_{1}(\cdot)$  and  $g_{2}(\cdot)$  individually as in [Lyu et al., 2022, von Kugelgen et al., 2021]. The invertibility of  $g_{1}$  and  $g_{2}$  amounts to asking that  $\mathbf{z}$ 's information is duplicate over  $\mathbf{v}_{1}$  and  $\mathbf{v}_{2}$ . In contrast, Assumption 3.1-i only requires that  $\mathbf{z}$ 's information is stored in  $(\mathbf{v}_{1},\mathbf{v}_{2})$  jointly. Moreover, generating functions  $g_{1}(\cdot)$  and  $g_{2}(\cdot)$  are assumed identical in [von Kugelgen et al., 2021], as opposed to being potentially distinct in Theorem 3.2. Additionally, Theorem 3.2 allows for dependence between  $\mathbf{z}$  and  $\mathbf{s}_2$  which is absent in [Lyu et al., 2022].
These relaxations expand the applicability of the basis model. For example, the distinction between  $g_{1}$  and  $g_{2}$  is indispensable in our application to the hierarchical model, as we cannot assume each latent variable generates its many children and descendants through the same function. Our proof technique is distinct from prior related work, which can be of independent interest to the community.
3.2 Local Identifiability of Latent Hierarchical Models
In this section, we build the connection between the basis and hierarchical models. Specifically, we show in Theorem 3.4 that a careful grouping of variables in the hierarchical model enables us to apply Theorem 3.2 to identify latent variables and their causal relations in a local scope. To this end, we modify Assumption 3.1 in basis models and obtain Assumption 3.3 for hierarchical models.
Assumption 3.3 (Hierarchical model conditions).
i [Differentiability]: Structure equations in Equation 1 are differentiable.
ii [Information-conservation]: Any  $\mathbf{z} \in \mathbf{Z}$  and exogenous variable  $\varepsilon$  can be expressed as differentiable functions of all observed variables, i.e.,  $\mathbf{z} = f_z(\mathbf{X})$  and  $\varepsilon = f_{\varepsilon}(\mathbf{X})$ .
iii [Subspace span]: For each set  $\mathbf{A}$  that  $d$ -separates all its ancestors  $\operatorname{Anc}(\mathbf{A})$  and observed variables  $\mathbf{X}$ , i.e.,  $\mathbf{X} \perp \operatorname{Anc}(\mathbf{A})|\mathbf{A}$ , for any  $\mathbf{z}_{\mathbf{A}} \in \mathrm{Pa}(\mathbf{A})$ , there exists an invertible mapping from a set of ancestors of  $\mathbf{A}$  containing  $\mathbf{z}_{\mathbf{A}}$  to the separation set  $\mathbf{A}$ , such that this mapping satisfies the subspace span condition (i.e., Assumption 3.1-ii).
iv [Edge connectivity]: The function between each latent variable  $\mathbf{z}$  and each of its children  $\mathbf{z}'$  has a Jacobian  $\mathbf{J}$ , such that for all  $j \in \{1, \dots, d_z\}$ , there exists  $i \in \{1, \dots, d_{z'}\}$ , such that  $(i, j)$  is in the support of  $\mathbf{J}$ .
Theorem 3.4. In a latent hierarchical causal model that satisfies Condition 2.4 and Assumption 3.3, we consider  $\mathbf{x}_i\in \mathbf{X}$  as  $\mathbf{v}_1$  and  $\mathbf{X}\backslash \mathbf{v}_1$  as  $\mathbf{v}_2$  in the basis model (Figure 2). With an estimation model  $(\hat{p}_{\mathbf{z},\mathbf{s}_2},\hat{p}_{\mathbf{s}_1},\hat{g})$  that follows the data-generating process in Equation 2, the estimated  $\hat{\mathbf{z}}$  is a one-to-one mapping of the parent(s) of  $\mathbf{v}_1$ , i.e.,  $\hat{\mathbf{z}} = h(Pa(\mathbf{v}_1))$  where  $h(\cdot)$  is an invertible function.
The proof can be found in Appendix B.2.