SlowGuess/NeurIPS_2023_OCR huggingface dataset:
provably isolates content from style. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 16451-16467, 2021. URL https://proceedings.neurips.cc/paper/2021/bitstream/8929c70f8d710e412d38da624b21c3c8-AAbstract.html. (page 5)
[37] Julius Von Kugelgen, Abdirisak Mohamed, and Sander Beckers. Backtracking counterfactuals. In Conference on Causal Learning and Reasoning, pages 177-196. PMLR, 2023. (page 7)
[38] Quanhan Xi and Benjamin Bloem-Reddy. Indeterminacy in generative models: Characterization and strong identifiability. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pages 6912–6939. PMLR, 25–27 Apr 2023. URL https://proceedings.mlr.press/v206/xi23a.html. (page 4, 15, 16)
[39] Kevin Xia, Yushu Pan, and Elias Bareinboim. Neural causal models for counterfactual identification and estimation. ArXiv preprint, abs/2210.00035, 2022. URL https://arxiv.org/abs/2210.00035. (page 1)
[40] Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. Causalvae: Disentangled representation learning via neural structural causal models. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 9593-9602. Computer Vision Foundation / IEEE, 2021. doi: 10.1109/CVPR46437.2021.00947. URL https://openaccess.thecvf.com/content/ CVPR2021/html/Yang_CausalVAE_Disentangled_Representation_Learning_via_Neural_Structural_Causal_Models_CVPR_2021_paper.html. (page 1)
[41] Matej Zečević, Devendra Singh Dhami, Petar Velicković, and Kristian Kersting. Relating graph neural networks to structural causal models. ArXiv preprint, abs/2109.04173, 2021. URL https://arxiv.org/abs/2109.04173. (page 2)
A Theory of causal normalizing flows 15
A.1 Identifiability results 15
A.2 Extension to real-world settings 15
B The multiple representations of SCMs 19
B.1 Illustrative example 19
B.2 Non-linear SCM representations 20
C Do-operator: interventions and counterfactuals 21
C.1 Definition and algorithms 21
C.2 Interventions in previous works 23
D Experimental details and extra results 24
D.1 Ablation: Time complexity 26
D.2 Ablation: Base distribution 27
D.3 Ablation: Flow architecture 27
D.4 Comparison:Extra non-linear SCMs 28
E Details on the fairness use-case 29
A Theory of causal normalizing flows
A.1 Identifiability results
First, we provide a more detailed explanation on the connection between the results from §3 and those from the work of Xi and Bloem-Reddy [38]. We consider it important to clarify that the definition of identifiability that we use is the same as [38, Def. 2]. Specifically, this definition is one better suited for deep learning models, which is concerned with recovering the variables  $\mathbf{u}$  and one parametrization that perfectly matches the original generator. In other words, with this definition we aim to recover one parametrization of a neural network which provides the generator function, but not the exact parametrization of the generator that generated the data.
We also want to clarify that Thm. 1 from the main paper corresponds to [38, Prop. 5.2], which we rewrote (without changing its content) to plain English and to match our particular setting. We now provide the proof for Cor. 2:
Proof. Assume that we have a flow  $T_{\theta}$  that does indeed isolate the exogenous variables, meaning that the  $i$ -th output of the flow,  $T_{\theta}(\mathbf{x})_i$ , is related with the true exogenous variable,  $\mathbf{u}_i$ , by an invertible function that only depends on it.
As explained in §3, this means that for a variable  $\mathbf{u} \sim P_{\mathcal{M}}$ , we have that  $T_{\pmb{\theta}}(\mathbf{f}(\mathbf{u})) \sim P_{\pmb{\theta}}$  and  $T_{\pmb{\theta}}(\mathbf{f}(\mathbf{u})) = \pmb{h}(\mathbf{u}) = (h_1(\mathbf{u}_1), h_2(\mathbf{u}_2), \ldots, h_d(\mathbf{u}_d))$ .
But we know the true generator, whose  $i$ -th exogenous variable is given by  $\mathbf{u}_i = f_i^{-1}(\mathbf{x}_{\mathrm{pa}_i},\mathbf{x}_i)$  (the inverse of  $f_{i}$  w.r.t  $\mathbf{u}_i$ ) and, putting all together,
$$
T _ {\boldsymbol {\theta}} (\mathbf {x}) _ {i} = h _ {i} (\mathbf {u} _ {i}) = h _ {i} \left(f _ {i} ^ {- 1} \left(\mathbf {x} _ {\mathrm {p a} _ {i}}, \mathbf {x} _ {i}\right)\right), \tag {9}
$$
which is a function of only the parents of  $\mathbf{x}_i$  and  $\mathbf{x}_i$  itself.
If we call  $\mathbf{u} = \mathbf{f}^{-1}(\mathbf{x})\coloneqq (f_1^{-1}(\mathbf{x}_{\mathrm{pa}_1},\mathbf{x}_1),f_2^{-1}(\mathbf{x}_{\mathrm{pa}_2},\mathbf{x}_2),\ldots ,f_d^{-1}(\mathbf{x}_{\mathrm{pa}_d},\mathbf{x}_d))$  the inverse of the SCM  $\mathcal{M}$  that writes  $\mathbf{u}$  as a function of  $\mathbf{x}$  (see App. B for an example), then it is clear that
$$
\nabla_ {\mathbf {x}} T _ {\boldsymbol {\theta}} (\mathbf {x}) = \nabla_ {\mathbf {x}} \left(\mathbf {h} \circ \mathbf {f} ^ {- 1}\right) (\mathbf {x}) = \nabla_ {\mathbf {u}} \mathbf {h} (\mathbf {u}) \cdot \nabla_ {\mathbf {x}} \mathbf {f} ^ {- 1} (\mathbf {x}) = \mathbf {D} \cdot \nabla_ {\mathbf {x}} \mathbf {f} ^ {- 1} (\mathbf {x}) \equiv \mathbf {I} - \mathbf {A}, \tag {10}
$$
where  $D$  is a diagonal matrix and  $A$  the adjacency matrix of the causal graph induced by  $\mathcal{M}$ .
Similarly,  $T_{\theta}^{-1}(\pmb{h}(\mathbf{u})) = \mathbf{x} = \mathbf{f}(\mathbf{u})$  and  $T_{\theta}^{-1}(\pmb{h}(\mathbf{u}))_i = \mathrm{x}_i = f_i(\mathbf{u}_{\mathrm{an}_i}, \mathbf{u}_i)$ , which again implies that:
$$
\nabla_ {\tilde {\mathbf {u}}} T _ {\boldsymbol {\theta}} ^ {- 1} (\tilde {\mathbf {u}}) \equiv (\boldsymbol {I} - \boldsymbol {A}) ^ {- 1} = \boldsymbol {I} + \sum_ {n = 1} ^ {\operatorname {d i a m} (\boldsymbol {A})} \boldsymbol {A} ^ {n}, \tag {11}
$$
where we call  $\tilde{\mathbf{u}} = h(\mathbf{u})$  the variable  $\mathbf{u}$  recovered by  $T_{\theta}$ , and where we have omitted  $\pmb{h}$  as its Jacobian matrix is diagonal. Note that the infinite sum above vanishes at  $n = \mathrm{diam}A$  since  $A$  is triangular with diagonal zero. Q.E.D.
A.2 Extension to real-world settings
A.2.1 Discrete data
In this section, we describe how to extend the results presented in the main text for the case where one observed variable,  $\mathbf{x}_i$ , is discrete. To this end, we restate the more general data-generative process assumed by Xi and Bloem-Reddy [38], which we used for the theoretical part of the manuscript.
Following the notation of the manuscript, say that we have a data-generating process without recursions, that is, we have a function  $\mathbf{f}$  that maps  $\mathbf{u}$  to  $\mathbf{x}$ . Let us assume, without loss of generality, that only the  $i$ -th observed variable is discrete, and let us focus on the way this variable is generated, dropping the subindex  $i$  along the way to avoid clutter. Now, Xi and Bloem-Reddy [38] additionally
consider the existence of a fixed noise distribution  $P_{\varepsilon}$  and mechanism  $g$ , such that the observed variable  $\mathbf{x}_i$  is generated as,
$$
\mathbf {u} := \left(\mathrm {u} _ {1}, \mathrm {u} _ {2}, \dots , \mathrm {u} _ {d}\right) \sim P _ {\mathbf {u}}, \quad \varepsilon \sim P _ {\varepsilon}, \quad \mathrm {x} _ {i} = g \left(f _ {i} \left(\mathrm {u} _ {\text {a n} _ {i}}, \mathrm {u} _ {i}\right), \varepsilon\right), \tag {12}
$$
with  $\varepsilon \perp \mathbf{u}$ , and where they study the noiseless case under the following assumption: if for two generative processes with  $\varepsilon_{a} \stackrel{\mathrm{d}}{=} \varepsilon_{b}$ , then  $g(f_{i}(\mathbf{u}_{a}), \varepsilon_{a}) \stackrel{\mathrm{d}}{=} g(f_{i}(\mathbf{u}_{b}), \varepsilon_{b})$  if and only if  $f_{i}(\mathbf{u}_{a}) \stackrel{\mathrm{d}}{=} f_{i}(\mathbf{u}_{b})$ , where  $\stackrel{\mathrm{d}}{=}$  denotes equal in distribution.
Just as we do with the rest of variables, we also make the assumption that the observed variable  $\tilde{\mathbf{x}}_i$  is the transformation of a continuous exogenous variable,  $\mathbf{u}_i$ , with a function  $\tilde{f}_i$  that fulfils our assumptions (i.e., that  $\tilde{f}_i$  is a diffeomorphism) that has undergone a quantization process, i.e.,  $\mathbf{x}_i = f_i(\mathbf{u}_{\mathrm{an}_i},\mathbf{u}_i)\coloneqq \lfloor \tilde{f}_i(\mathbf{u}_{\mathrm{an}_i},\mathbf{u}_i)\rfloor$ . Therefore, it is clear that  $f_{i}$  is no longer bijective, as we are clamping real numbers into integers, and that the observational distribution of  $\mathbf{x}_i$  is discrete.
We take advantage of the noise assumption above, and dequantize the observed variable  $\mathbf{x}_i$  by assuming an additive noise mechanism such that  $\mathbf{x}_i \coloneqq f_i(\mathbf{u}_{\mathrm{an}_i}, \mathbf{u}_i) + \varepsilon$ , with  $\varepsilon$  distributed between the unit interval with any continuous distribution (we take in our experiments  $P_{\varepsilon} = \mathcal{U}(0,1)$ ). With this process: i) we have made  $\tilde{\mathbf{x}}_i$  again a continuous random variable, as the sum of independents discrete and continuous random variables is a continuous random variable; and ii) the original distribution of the noiseless observed variables is always recoverable  $P(\tilde{\mathbf{x}}_i = c) = P(c \leq \mathbf{x}_i \leq c + 1)$ .
More importantly, all the theoretical insights from the work of Xi and Bloem-Reddy [38] can still be used, working with the noisy case rather than the noiseless one. Indeed, as for their analysis they assume a single  $\mathbf{u}$  in the domain of the generator, we can merge the generator and noise mechanism  $g\circ f_i:\mathbb{R}\to \mathbb{R}$  (rather than  $g\circ f_{i}:\mathbb{R}\times [0,1]\to \mathbb{R})$  , by mapping the non-injective part of  $\mathbf{u}$  to  $\varepsilon$  itself, i.e., by using the function  $(g\circ f_i)(\mathbf{u}_{\mathrm{an}_i},\mathbf{u}_i) = \lfloor \tilde{f}_i(\mathbf{u}_{\mathrm{an}_i},\mathbf{u}_i)\rfloor +F_\varepsilon^{-1}(\tilde{f}_i(\mathbf{u}_{\mathrm{an}_i},\mathbf{u}_i) - \lfloor \tilde{f}_i(\mathbf{u}_{\mathrm{an}_i},\mathbf{u}_i)\rfloor)$  where  $F_{\varepsilon}^{-1}$  is the quantile function of  $P_{\varepsilon}$ . This new function is a diffeomorphism almost everywhere, as it is a composition of a.e. diffeomorphisms, and we have effectively replaced the noise variable by the floating part of  $\tilde{f}_i(\mathbf{u}_{\mathrm{an}_i},\mathbf{u}_i)$  before quantization. Moreover, note that if the noise is uniformly distributed,  $P_{\varepsilon} = \mathcal{U}(0,1)$ , we have that  $g\circ f_{i} = \tilde{f}_{i}$  (if  $\mathbf{x}_i$  were discretized by taking its integer part).
In short, by adding noise to discrete variables while keeping them recoverable, we can learn a mapping between continuous variables that learns a version of the generator function before the observed values were somehow discretized. Importantly, the observed discrete distribution is always recoverable, independently of whether we learn the (unknown and unrecoverable) underlying continuous distribution before being discretized.
A.2.2 Partial knowledge
In this section, we explain how to expand our framework to settings in which we have partial information about the causal graph of  $\mathcal{M}$ . That is, we know the causal ordering  $\pi$  (so we know that half of the causal relationships, the upper diagonal), and we are certain about some other causal relationships (edges on  $A$ ), but not all of them.
To this end, first let us first introduce the way we deal with partial knowledge, and then clarify the theoretical implications that it has with respect to the theory introduced in §5.
The method Similar to §4, let us motivate the method with an illustrative example. Suppose that we are given an SCM such as the one in Fig. 6a, where we know all relationships but the one between  $\mathbf{x}_2$  and  $\mathbf{x}_3$ . Note that, in this case, we lack even information about the causal ordering. Indeed, there are three possible outcomes: i) the edge  $\mathbf{x}_2 \to \mathbf{x}_3$  could exist (Fig. 6b); ii) the edge  $\mathbf{x}_3 \to \mathbf{x}_2$  could exist (Fig. 6c); or iii) both could exist simultaneously (Fig. 6d), and hence there is a confounder between them. However, we do not know which of the three options is the correct one.
Let us switch now to Fig. 7. To solve the original problem (Fig. 7a, one natural approach is to group the nodes with unknown relationships—assuming that all unknown edges may exist—and maximize the observed likelihood (Fig. 7b). This, effectively, is equivalent to applying an ANF to the known relationships, and using a general-Jacobian NF to learn the joint of the block variables. However, if we want to keep using exclusively ANFs (Fig. 7c), we can learn the joint distribution within the blocks with an ANF using a fixed ordering (which it can always do, as it is a universal density approximator [25]). The only subtle detail here is that, in that case, we need to increase the
Figure 6: Example of an SCM with partial knowledge about the causal graph (a) and possible outcomes: (b) in the actual SCM only the edge  $\mathbf{x}_2 \rightarrow \mathbf{x}_3$  exists; (c) only the edge  $\mathbf{x}_3 \rightarrow \mathbf{x}_2$  exists; (d) both edges exist (and therefore there exists a confounder between them).
Figure 7: Illustrative example (same as in Fig. 6) applying our method for partial information. First, we apply Tarjan's algorithm [35] to find the SCCs of the graph (rectangles) and build a new DAG where each node is a subset of the original nodes. If, for the SCCs, we use an NF with a general Jacobian matrix (b), we keep the individual edges and treat each SCC as a block. If we instead use an ANF (c), we pick an arbitrary order within each SCC, and merge the individual edges into SCC-wide edges. Red represents intra-SCC edges, and green inter-SCC edges. See App. A.2.2 for more details.
granularity of all inter-block edges from node- to block-wise relationships, assuming that an edge exists if it exists for at least one of the elements of the block. To see why this is necessary, assume that the real graph of the example is Fig. 6c, yet we use an ANF with the ordering  $\pi = (1234)$  and the graph  $A$  without inter-block modifications (i.e., the adjacency matrix of Fig. 6b). In that case, we would have that  $\mathbf{x}_4$  depends on  $\mathbf{x}_3$  through  $\mathbf{x}_2$ . However, a causally consistent NF w.r.t.  $A$  would not be able to model that dependency, and thus  $\mathbf{x}_4$  would depend on  $\mathbf{u}_1$ ,  $\mathbf{u}_2$ , and  $\mathbf{u}_4$  but not on  $\mathbf{u}_3$ . With this approach, the ANF can model every case from Fig. 6, and it would need to remove the extra spurious relationships through optimization.
Therefore, to reuse our existing results from §5, the method that we have adopted is the one from Fig. 7c, which can be described in the following steps:
1. Run Tarjan's algorithm [35] to group all nodes by their SCCs (note that, unlike in the given example, there could be more than one cluster of unknown relationships).
2. Choose an ordering that is consistent with the known inter-SCC edges, fixing the edges within the SCCs.
3. Move from node-edges to SCC-edges. In practice, this means introducing edges between every pair of edges of two SCCs, if there exists at least one edge between them.
4. Solve the MLE problem with an ANF as in the main manuscript.
The theory Very conveniently, the method described above fits almost-perfectly into the already-covered theory in §3. To see this, note that our identifiability theory and following results required a fixed ordering, but it does not need to be the causal one: we can always find an equivalent SCM following the selected ordering by applying KR transports as described in §3. Therefore, the technical
implications of both Thm. 1 and Cor. 2 still apply to this fixed ordering, we only need to re-state their implications with respect to the true causal data-generating process  $\mathcal{M}$ .
To this end, we only need to note that all possible graphs, once reduced into a DAG using the partition of SCCs (as in Fig. 7c), are exactly equal. In other words, every possible graph shares the same causal dependencies between SCCs with the other possible graphs. If we start treating them like block, calling  $\{\mathbf{x}_i\} \subset S_i \subset \{1,2,\dots,d\}$  the SCC of the  $i$ -th node, and  $\mathrm{pa}_{S_i}$  and  $\mathrm{an}_{S_i}$  the parents and ancestors of every node in the SCC  $S_i$ , then it is clear that we can write for every graph  $\tilde{A}$  its observed variables as  $\mathbf{x}_i = \tilde{f}_i(\mathbf{x}_{\mathrm{pa}_{S_i}}, \tilde{\mathbf{u}}_{S_i})$  and, more importantly, we can write its "exogenous" variables as a function of the true ones, i.e.,  $\tilde{\mathbf{u}}_{S_i} = \mathrm{KR}(\mathbf{u}_{S_i})$ , where KR is the Knothe-Rosenblatt transport [19, 31]. Note also that it does not depend on the data, i.e.,  $\tilde{\mathbf{u}}_{S_i} \perp \mathbf{x}|\mathbf{u}_{S_i}$ .
Theorem 3 (Identifiability - Partial knowledge). If an element of  $\mathcal{F} \times \mathcal{P}_{\mathbf{u}}$ , and another from  $\tilde{\mathcal{F}} \times \mathcal{P}_{\mathbf{u}}$  where  $\mathcal{F}$  and  $\tilde{\mathcal{F}}$  are TMI maps with different intra-SCC orders (see above), generate the same observational distribution, then the two processes differ by an invertible, SCC-wise transformation of the variables  $\mathbf{u}$ .
Proof. Call  $\mathcal{M}$  and  $\tilde{\mathcal{M}}$  the elements from  $\mathcal{F} \times \mathcal{P}_{\mathbf{u}}$  and  $\tilde{\mathcal{F}} \times \mathcal{P}_{\mathbf{u}}$ , respectively.
W.l.o.g. pick  $\mathcal{M}$ , and apply a KR transport to write it down as another element of  $\tilde{\mathcal{F}} \times \mathcal{P}_{\mathbf{u}}$ , call it  $\hat{\mathcal{M}}$ , with identical observational distribution as both  $\mathcal{M}$  and  $\tilde{\mathcal{M}}$ .
Using Thm. 1, we know that the elements of  $\tilde{\mathcal{M}}$  and  $\hat{\mathcal{M}}$  differ by an invertible, component-wise transformation  $h$ . Moreover, we can write  $\hat{\mathbf{u}}_{S_i}$  as a function of  $\mathbf{u}_{S_i}$ , as argued above, where  $\{i\} \subset S_i \subset \{1,2,\dots,d\}$  are the indexes of the SCC that contains  $\mathbf{u}_i$ . Putting it all together:
$$
\tilde {\mathbf {u}} _ {i} = h _ {i} \left(\hat {\mathbf {u}} _ {i}\right) = h _ {i} \left(\mathrm {K R} _ {i} \left(\mathbf {u} _ {\boldsymbol {S} _ {i}}\right)\right), \tag {13}
$$
and, in vectorial form, for each SCC  $S_{i}$
$$
\tilde {\mathbf {u}} _ {\boldsymbol {S} _ {i}} = \boldsymbol {h} _ {\boldsymbol {S} _ {i}} \left(\mathrm {K R} _ {\boldsymbol {S} _ {i}} \left(\mathbf {u} _ {\boldsymbol {S} _ {i}}\right)\right) = \left(\boldsymbol {h} _ {\boldsymbol {S} _ {i}} \circ \mathrm {K R} _ {\boldsymbol {S} _ {i}}\right) \left(\mathbf {u} _ {\boldsymbol {S} _ {i}}\right) \tag {14}
$$
Corollary 4 (Causal consistency - Partial knowledge). If a causal NF  $T_{\theta}$ , with partial knowledge of the causal graph, SCC-wise isolates the exogenous variables of an SCM  $\mathcal{M}$ , then  $T_{\theta}$  is causally consistent with the true data-generating process,  $\mathcal{M}$ , with respect to each SCC.
Proof. The proof is identical to the one for Cor. 2, but using arguments with respect to the reduced graph after grouping all nodes in their respective SCCs.
Specifically, we can write using Thm. 3 the output of the flow as a function of the exogenous variables,  $T_{\theta}(\mathbf{x})_i = h(\mathbf{u}_{S_i})$ , and using the true causal generator, we have
$$
T _ {\boldsymbol {\theta}} (\mathbf {x}) _ {i} = \boldsymbol {h} \left(\mathbf {u} _ {\boldsymbol {S} _ {i}}\right) = \boldsymbol {h} \left(f _ {\boldsymbol {S} _ {i}} \left(\mathbf {x} _ {\text {p a} _ {\boldsymbol {S} _ {i}}}, \mathbf {x} _ {\boldsymbol {S} _ {i}}\right)\right), \tag {15}
$$
and hence the gradients agree with those from  $\mathcal{M}$ , when looking at the reduced graph. Q.E.D.
Thm. 3 and Cor. 4 provide analogues to those results from the main manuscript. It is important to note, however, that causal consistency refers to the causal relationships between SCCs as a whole, i.e., not at the causal relationships between individual nodes. The reason behind this is the same as why we had to introduced spurious edges in Fig. 7c: as we fix an ordering within each SCC, we may not be able to model indirect dependencies of nodes of one SCC to another unless we artificially introduce shortcuts. As such, every result from the main paper holds, if we treat each SCC (or block) as a whole. That is, when we reason about SCCs instead of individual nodes (note that if the whole causal graph is known every SCC contains a single node), we can safely talk about SCC-identifiability, causal SCC-consistency, and we can perform interventions and compute counterfactuals on SCCs.
(c) Compacted.
Figure 8: Example of the linear SCM  $\{\mathrm{x}_1\coloneqq \mathrm{u}_1;\mathrm{x}_2\coloneqq 2\mathrm{x}_1 + \mathrm{u}_2;\mathrm{x}_3\coloneqq 3\mathrm{x}_2 + \mathrm{u}_3\}$  written (a) in its usual recursive formulation; (b) without recursions, with each step made explicit; (c) without recursions, as a single function; and (d) writing  $\mathbf{u}$  as a function of  $\mathbf{x}$ . The red dashed arrows show the influence of  $\mathbf{u}_1$  on  $\mathbf{x}_3$  for all equations from  $\mathbf{u}$  to  $\mathbf{x}$ , with the compacted version exhibiting shortcuts (see §4). Note that in the linear case we have  $A\coloneqq G\neq 0$ , and that  $G_{1},G_{2},G_{3}\preceq G + I$  are any three matrices such that their product equals  $G^{2} + G + I$ .
B The multiple representations of SCMs
B.1 Illustrative example
In this section, we delve a bit deeper into the illustrative example from the main paper (§4), and write down the maths to understand how are these different models equivalent. For convenience to the reader, we have replicated Fig. 4 in the appendix as Fig. 8.
Recursive SCM As explained in the manuscript, the usual way of describing an SCM  $\mathcal{M}$  is by providing its recursive equations. In our example (Fig. 8a), we have a linear SCM of the form
$$
\left\{ \begin{array}{l} \mathrm {x} _ {1} = \mathrm {u} _ {1} \\ \mathrm {x} _ {2} = 2 \mathrm {x} _ {1} + \mathrm {u} _ {2} \\ \mathrm {x} _ {3} = 3 \mathrm {x} _ {2} + \mathrm {u} _ {3} \end{array} , \right. \tag {16}
$$
which we can compactly write as  $\mathbf{x} = G\mathbf{x} + I\mathbf{u}$ . However, the recursive equations are not the most convenient ones, as they entail solving the system iteratively according to its causal dependencies.
Unrolled SCM Instead, we can write the equations as a function from  $\mathbf{u}$  to  $\mathbf{x}$  directly. To do this, we can proceed and unroll the equations:
$$
\left\{ \begin{array}{l} \mathrm {x} _ {1} = \mathrm {u} _ {1} \\ \mathrm {x} _ {2} = 2 \mathrm {x} _ {1} + \mathrm {u} _ {2} \\ \mathrm {x} _ {3} = 3 \mathrm {x} _ {2} + \mathrm {u} _ {3} \end{array} \right. \Rightarrow \left\{ \begin{array}{l} \mathrm {x} _ {1} = \mathrm {u} _ {1} \\ \mathrm {x} _ {2} = 2 \mathrm {u} _ {1} + \mathrm {u} _ {2} \\ \mathrm {x} _ {3} = 3 (2 \mathrm {x} _ {1} + \mathrm {u} _ {2}) + \mathrm {u} _ {3} \end{array} \right. \Rightarrow \left\{ \begin{array}{l} \mathrm {x} _ {1} = \mathrm {u} _ {1} \\ \mathrm {x} _ {2} = 2 \mathrm {u} _ {1} + \mathrm {u} _ {2} \\ \mathrm {x} _ {3} = 3 (2 \mathrm {\Delta u} _ {1} + \mathrm {\Delta u} _ {2}) + \mathrm {\Delta u} _ {3} \end{array} \right., \tag {17}
$$
which we can write as a multi-step function:
$$
\left\{ \begin{array}{l} \mathrm {z} _ {1} ^ {1} = \mathrm {u} _ {1} \\ \mathrm {z} _ {2} ^ {1} = \mathrm {u} _ {2} \\ \mathrm {z} _ {3} ^ {1} = \mathrm {u} _ {3} \end{array} \right. \Rightarrow \left\{ \begin{array}{l} \mathrm {z} _ {1} ^ {2} = \mathrm {z} _ {1} ^ {1} \\ \mathrm {z} _ {2} ^ {2} = 2 \mathrm {z} _ {1} ^ {1} + \mathrm {z} _ {2} ^ {1} \\ \mathrm {z} _ {3} ^ {2} = \mathrm {z} _ {3} ^ {1} \end{array} \right. \Rightarrow \left\{ \begin{array}{l} \mathrm {x} _ {1} = \mathrm {z} _ {1} ^ {2} \\ \mathrm {x} _ {2} = \mathrm {z} _ {2} ^ {2} \\ \mathrm {x} _ {3} = 3 \mathrm {z} _ {2} ^ {2} + \mathrm {z} _ {3} ^ {2} \end{array} , \right. \tag {18}
$$