Targeted Reduction of Causal Models

Authors: armin  kekić
bernhard  schölkopf
michel  besserve
Venue: ICLR_cc_2024_Workshop_AI4DiffEqtnsInSci
Published: 2024-02-09 19:56:58

Abstract: Why does a phenomenon occur? Addressing this question is central to most scientific inquiries and often relies on simulations based on differential equations. As scientific models become more intricate, deciphering the causes behind phenomena in high-dimensional spaces of interconnected variables becomes increasingly challenging. We introduce Targeted Causal Reduction (TCR), a method for condensing complex intervenable models into a concise set of causal factors that explain a specific target phenomenon. 
We propose an information theoretic objective to learn TCR from interventional data of simulations, establish identifiability for continuous variables under shift interventions and present a practical algorithm for learning TCRs. Its ability to generate interpretable high-level explanations from complex models is demonstrated on toy and mechanical systems, illustrating its potential to assist scientists in the study of complex phenomena in a broad range of disciplines.

Main paper: 
 introduction: 
 Numerical models are indispensable in science for simulating real-world systems and generating etiological explanations-identifying the causes of specific phenomena. General circulation models, for example, shed light on the causes of global warming (Grassl, 2000) , while com-putational brain models explore the origins of neurological pathologies (Breakspear, 2017; Deco and Kringelbach, 2014) . These examples illustrate the increasing complexity of numerical scientific models, designed to faithfully capture the large number of mechanisms at play in these systems. However, this complexity comes at a cost: expanding parameter spaces and heightened computational demands. This trend, in turn, impacts the ability to generate high-level explanations, understandable by scientists and decision makers (Reichstein et al., 2019; Safavi et al., 2023) .Effective human explanations are often based on understanding a few causal relations between a limited number of variables. While the simulation of complex systems might rely on numerous simple mechanisms, extracting overarching causal relations between fewer relevant high-level variables remains largely unaddressed. In particular, while causal representation learning tries to explain data based on a learned latent causal graph (Wendong et al., 2023; Squires et al., 2023; von Kügelgen et al., 2023a) , it currently has theoretical and practical limitations. CRL largely relies on preserving all information in the data to provide recoverability guaranties for the latent causes, while the idea of a highlevel representation is precisely to discard irrelevant data.In contrast, Causal Model Reduction (CMR), which aims to map a low-level causal model to a simpler high-level model with fewer or lower-dimensional variables, embraces the purpose of eliminating irrelevant information. However, existing CMR approaches, such as causal abstractions (Geiger et al., 2023; Zennaro et al., 2023) and Causal Feature Learning (CFL) (Chalupka et al., 2016) are not well-suited to causally describe many scientific models: they use discrete variables and typically rely on hard interventions, disconnecting causal variables from their parents. The following example shows, however, that simpler high-level causal models for continuous variables and soft interventions are natural and useful in domains such as physics.Consider a system of point masses connected by springs shown in Fig. 1a , where each mass is influenced by random external forces. Its trajectory under the intervention of external forces can be accurately predicted by simulating the coupled equations of motion of individual point masses. However, if we are only interested in a particular macroscopic "target" variable of this system: the horizontal speed of the system's center of mass at the end of an experiment, a key result from classical physics is that its motion will depend only on the sum of all horizontal components of external forces applied over time. We thus obtain a form of CMR: a much simpler system that accurately accounts for the effect of interventions in the system on the target variable.This highlights the core elements needed for CMR in scientific models: (1) there is a clearly defined macroscopic target variable, (2) continuous low-level variables are reduced to a smaller set of continuous high-level variables, and (3) low-level interventions are soft: exerted forces modify the future trajectory but do not suppress the influence of other factors such as the past state of the system. Moreover, it is the combination of many low-level soft interventions across time that corresponds to a relevant high-level intervention. These three aspects are commonly found characteristics of studied real-world systems, motivating the development of CMR algorithms adapted to this setting.In this paper, we introduce Targeted Causal Reduction (TCR), depicted in Fig. 1b , a novel approach designed to simplify complex low-level models into high-level models, focused on explaining causal influences on an observable target variable 𝑌 . The key signal we use for learning are interventions applied to the low-level variables, which are mapped to high-level interventions in a way that captures the causal influences on 𝑌 in a concise and interpretable way.We formulate this learning objective as a Kullback-Leibler divergence between the fitted high-level interventional model and the reduced low-level interventional distribution, leading to a practical learning algorithm for the case of linear reductions. Applications to high-dimensional synthetic and scientific models demonstrates accuracy and interpretability of our approach. We refer to the appendix for more related work (A) and proofs (C). 
background 2.1 structural causal models: 
 Causal dependencies between variables can be described using Structural Causal Models (SCM) (Peters et al., 2017) .Notation. We use boldface for column vectors, and 𝒊 𝑆 for the subvector of 𝒊 restricted to the components in set 𝑆.Definition 2.1 (SCM). An 𝑛-dimensional structural causal model is a triplet M = (G, S, 𝑃 𝑼 ) consisting of:• a joint distribution 𝑃 𝑼 over exogenous variables {𝑈 𝑗 } 𝑗 ≤𝑛 , • a directed graph G with 𝑛 vertices, • a set S = {𝑋 𝑗 𝑓 𝑗 (Pa 𝑗 , 𝑈 𝑗 ), 𝑗 = 1, . . . , 𝑛} of structural equations, where Pa 𝑗 are the variables indexed by the set of parents of vertex 𝑗 in G, such that for almost every 𝒖, the system {𝑥 𝑗 𝑓 𝑗 (pa 𝑗 , 𝑢 𝑗 )} has a unique solution 𝒙 = 𝒈(𝒖), with 𝒈 measurable.The unique solvability condition is included in this definition because we consider a general class of SCMs by allowing cycles, that is, G may not be a DAG. Moreover, we allow hidden confounding through the potential lack of independence between the exogenous variables {𝑈 𝑗 }. See Bongers et al. (2021) for a thorough study of these models. Under these conditions, the distribution 𝑃 𝑼 entails a well-defined joint distribution over the endogenous variables 𝑃( 𝑿).Interventions in SCMs involve replacing one or more structural equations, potentially modifying exogenous distributions, and adding or removing arrows in the original graph to reflect changes in dependencies between variables. An intervention transforms the original model M = (G, S, 𝑃 𝑼 ) into an intervened model M (𝒊) = (G (𝒊) , S (𝒊) , 𝑃 (𝒊) 𝑼 ), where 𝒊 is the vector parameterizing the intervention. The base probability distribution of the unintervened model is denoted 𝑃 (0) M ( 𝑿) or simply 𝑃 M ( 𝑿) and the interventional distribution associated with M (𝒊) is denoted 𝑃 (𝒊) M ( 𝑿). Classical 𝑑𝑜-interventions set a structural equation to a constant, removing all influences of the parents on the intervened variable. This can be problematic for studying how the influence of low-level variables is propagated to the target, since for simultaneous interventions, the effects of some interventions can be masked by others. The probability of such masking increases as the number of low-level variables grows. Soft interventions, on the other hand, modify an equation while keeping the set of parents unchanged. This is more appropriate in our setting, since it propagates the information from all interventions to the target simultaneously.Large classes of soft interventions can be designed to match domain knowledge (Besserve and Schölkopf, 2022) . Notably, shift interventions modify the structural equation of endogenous variable 𝑙 through shifting it by a scalar parameter 𝑖{𝑋 𝑙 𝑓 𝑙 (Pa 𝑙 , 𝑈 𝑙 )} ↦ → {𝑋 𝑙 𝑓 𝑙 (Pa 𝑙 , 𝑈 𝑙 ) + 𝑖} . (1)These can be combined to form multi-node interventions with vector parameter 𝒊.We use the term scientific model to refer to a generative model that relies on a set of equations to represent a phenomenon. What distinguishes such models from generative models in machine learning is their decomposability into elementary functions, encoding domain knowledge about the mechanisms being investigated. Simulations based on the numerical solution of scientific models can often be expressed as SCMs; This notably includes Ordinary (ODE) (Mooij et al., 2013) and Stochastic Differential Equations (SDE) (Hansen and Sokol, 2014) . A simulator can thus be seen as a low-level causal model, from which samples of unintervened and intervened distributions can be generated. This forms the basis of the causal framework for learning high-level explanations for simulators developed in Sec. 3.1.We consider as CMR any (possibly approximate) mapping from a low-level SCM L to a simpler high-level SCM H . An example is CFL (Chalupka et al., 2015 (Chalupka et al., , 2016)) , which achieves a CMR by merging values of a large observation space to yield discrete high-level variables taking values in a small finite set. Consider:• L has a vector of endogenous variables 𝑿 with range X and a set of interventions I, • H has a vector of endogenous variables 𝒁 with range Z and a set of interventions J .Starting from the distribution of the low-level model 𝑃 L ( 𝑿), a deterministic mapping 𝜏 : X → Z generates a joint distribution on the high-level variables that is the push-forward distribution of 𝑃 L ( 𝑿) by 𝜏, denoted 𝜏 # [𝑃 L ( 𝑿)] such that𝜏( 𝑿) ∼ 𝜏 # [𝑃 L ( 𝑿)] .The low-level interventional distributions can be pushed forward to the high-level in the same way.A general framework for CMR is based on the notion of exact transformation, which ensures interventional consistency by matching the push-forward low-level distributions to the high-level ones.Definition 2.2 (Exact transformation (Rubenstein et al., 2017) ). A map 𝜏 : X → Z is an exact transformation from L to H if it is surjective, and there exists a surjective intervention map 𝜔 : I → J such that for all 𝒊 ∈ I𝜏 # [𝑃 (𝑖) L ( 𝑿)] = 𝑃 ( 𝜔 (𝑖) ) H (𝒁) .The set of possible 𝜏 can be restricted to constructive transformations, where high-level variables depend only on nonoverlapping subsets of low-level variables. This eases interpretability of CMR and comes with characterization results (Beckers and Halpern, 2019; Geiger et al., 2023) .Definition 2.3. 𝜏 : X → Z is a constructive transformation between model L and H if there exists an alignment map 𝜋 relating indices of each high-level endogenous variable to a subset of indices of low-level endogenous variables such that for all 𝑘 ≠ 𝑙, 𝜋(𝑘) ∩ 𝜋(𝑙) = ∅ and for each component 𝜏 𝑘 of 𝜏 it exists a function τ𝑘 such that for all 𝒙 in X,𝜏 𝑘 (𝒙) = τ𝑘 (𝒙 𝜋 (𝑘 ) ) .The intervention map 𝜔 of constructive exact transformations are required to be constructive as well, such that acting on high-level variable 𝑘 depends only on low-level interventions acting on variables in 𝜋(𝑘) (see App. B). 
theoretical analysis: 
 As described in Fig. 1b , we consider endogenous variables of a low-level model gathered in a (high-dimensional) random vector 𝑿. A target scalar variable 𝑌 = 𝜏 0 ( 𝑿) quantifies a property of interest of this model, and can be thought of as quantifying the presence or magnitude of a phenomenon in the data, using detector 𝜏 0 . To generate a high-level causal explanation of this phenomenon, we learn a highlevel SCM with a fixed causal structure, where the known effect variable 𝑌 is caused by 𝑛 learned independent high-level variables 𝑍 𝑘 . The low-level variables 𝑿 are approximately mapped to the high-level variables 𝒁 using a constructive transformation and an associated constructive interventional map with the same alignment 𝜋.Our reduction framework has the following elements:( It is not always possible to achieve an exact transformation that guarantees consistency of low-and high-level models for almost all interventions. As a consequence, we allow for the consistency between models to be approximate. To ensure that this approximation is as accurate as possible, we minimize the expected KL divergence between the pushforward by the transformation 𝜏 of the low-level interventional distributions that we denote 𝑃 (𝒊) 𝜏 (𝑌 , 𝒁) = 𝜏 # [𝑃 (𝒊) L ( 𝑿)], and the corresponding interventional distribution of the highlevel model 𝑃 ( 𝜔 (𝒊) ) , leading to the consistency lossL cons = E 𝒊∼𝑃 (𝒊) KL 𝑃 (𝒊)𝜏 (𝑌 , 𝒁) ∥𝑃 ( 𝜔 (𝒊) ) (𝑌 , 𝒁) . (4)Other losses have been previously suggested to enforce consistency. Beckers et al. (2020) propose to take a maximum over interventions, whereas we take the expectation in our loss, thus focusing the CMR on the average performance rather than the worst case. EQUATIONand is an upper bound of the causal relevance lossL rel = E 𝒊∼𝑃 (𝒊) KL 𝑃 (𝒊) (𝑌 ) ||𝑃 ( 𝜔 (𝒊) ) (𝑌 ) ≤L cons . (6)Reparametrization invariance (see Def. D.1) refers to transformations of the pairs (𝜏, 𝑓 𝜸 ) that leave the composition 𝑓 𝜸 • 𝜏 invariant. In the 𝑛 = 1 linear setting (see Sec. 3.3) , this corresponds to invariance by multiplicative rescaling. This guarantees that equivalent high-level causal descriptions are treated equally by the loss.We call Eq. ( 5) a Cause-Mechanism Decomposition because the first term quantifies the cause consistency and the second term can be thought of as the mechanism consistency. This latter term assesses the similarity between the outputs of the learned high-level mechanism 𝑃 (0) (𝑌 |𝑧) and the corresponding conditional distribution computed by push-forward of the low-level variables 𝑃 (𝒊) 𝜏 (𝑌 |𝑧). Since we prevent the high-level mechanism from being intervened on, only its unintervened conditional appears in the expression.Lastly, the causal relevance loss L 𝑟 𝑒𝑙 assesses whether the variations of the target 𝑌 due to low-level interventions are well-captured by high-level interventions, on average over the prior 𝑃(𝒊). Its upper bounding by L cons ensures that by optimizing for consistency, we also indirectly promote effective "explanations" of the variations in the target density resulting from low-level interventions. We can thus choose 𝑃(𝒊) to make the most relevant interventions more likely according to domain knowledge, such that optimizing the loss will steer towards a solution capturing the most domainrelevant variations of the target.We further constrain the setting to be able to study the solution minimizing L cons analytically and get insights into the properties of TCR.Notation. When a vector, say 𝝉 𝑘 , is associated to a highlevel SCM component 𝑘 of a constructive transformation with alignment 𝜋, τ𝑘 indicates the restriction of 𝝉 𝑘 to components in 𝜋(𝑘). The number of elements in a set 𝑆 is #𝑆.Tau map. To maximize interpretability, we assume a linear 𝜏-map, represented as a vector 𝝉 such that:𝑿 ↦ → 𝑌 𝒁 =         𝝉 ⊤ 0 . . ., 𝝉 ⊤ 𝑛         𝑿 = τ⊤ 0 𝑿 𝜋 (0) , . . . τ⊤ 𝑛 𝑿 𝜋 (𝑛) ⊤ .Omega map. We focus on shift interventions and map the vector 𝒊 of low-level interventions on the nodes in 𝜋(𝑘) to a scalar shift intervention on the mechanism of each 𝑍 𝑘 . We assume each map 𝜔 𝑘 to be linear with vector 𝝎 𝑘 such that𝜔 𝑘 (𝒊) = 𝝎 ⊤ 𝑘 𝒊 = ω⊤ 𝑘 𝒊 𝜋 (𝑘 ). Because high-level causes are root nodes, intervening amounts to shifting the marginal distribution from 𝑃 (0) (𝑍 𝑘 ) to 𝑃 ( 𝜔 𝑘 (𝒊) ) (𝑍 𝑘 ) = 𝑃 (0) (𝑍 𝑘 -𝜔 𝑘 (𝒊)).Choice of alignment 𝜋. There are potential degrees of freedom for 𝜋, and users may want to incorporate domain knowledge as well as interpretability constraints to reduce the variables included in ∪ 𝑘≠0 𝜋(𝑘). In practice, we learn the distribution of the low-level variables among the 𝜋(𝑘) using regularization (see Sec. 4).High-level mechanism. We use an interpretable affine high-level causal mechanism 𝑓 𝛾 , such that𝑌 𝑘 𝛼 𝑘 𝑍 𝑘 + 𝑅 0 + 𝛽 , 𝛼 1 , . . . , 𝛼 𝑛 , 𝛽 ∈ R . (7)Choice of prior 𝑃(𝒊). The solutions minimizing the loss of Eq. ( 4), may depend on the choice of the prior 𝑃(𝒊), and in particular on which variables are actually intervened on. Let Ω denote the subset of indices of low-level variables that are intervened on with non-zero probability. The components of 𝒊 whose index does not belong to Ω thus take value 𝑖 = 0 with probability one. We provide identifiability guaranties under two kinds of assumptions. If we assume the low-level model is linear Gaussian of the form 𝑿 Ω → 𝑿 𝜋 (0) , we can show the existence and uniqueness of the solution.Proposition 3.4. Assume the low-level SCM follows𝑿 𝐴𝑿 + 𝑼 + 𝒊 , 𝒊 ∼ 𝑃(𝒊) , 𝑈 𝑘 ∼ N (𝜇 𝑘 , 𝜎 2 𝑘 ), 𝜎 𝑘 > 0 , such that 𝑿 and 𝐴 take the block forms𝑿 = 𝑿 𝜋 (0) 𝑿 Ω , 𝐴 = 𝐴 00 𝐴 0Ω 0 𝐴 ΩΩ .Given an arbitrary choice of linear scalar target of the form 𝑌 = 𝜏 ⊤ 0 𝑿 = τ⊤ 0 𝑿 𝜋 (0) and under Assum. 3.2 or Assum. 3.3, there is a unique linear 1-cause TCR (up to a multiplicative constant) satisfying L cons = 0. It is given byEQUATION)Moreover, let 𝑛 𝑚𝑎𝑥 be the maximum number 𝑛 such that a linear 𝑛-cause TCR can achieve L cons = 0. If there are no cancellations2 among causal pathways from each node in supp( ω1 ) of Eq. ( 10) towards 𝑌 , then the 𝑛 𝑚𝑎𝑥 -cause TCR is unique up to rescaling and permutation of the causes.This result provides guaranties for having a unique groundtruth solution in case exact transformations can be achieved. The main assumption is the absence of feedback influences from the target set 𝜋(0) to candidate causes. However, cycles and confounding are allowed in the low-level model, contrary to the learned high-level model. The 1cause solution is easiest to obtain. The study of simple SCMs (App. D.2 and App. D.3) provides some insights on the form of the analytical solution. Additional results show that we lose identifiability of the TCR if we drop the assumption that not all variables in 𝜋(1) are intervened on (see App. D.4). The 𝑛-cause solution is essentially a partition of the 1-cause solution that enforces independence between them.Under the same model assumptions, the resulting constructive transformation can be associated with a constructive causal abstraction, as shown in Proposition D.3. This corresponds to a particular case of low soft abstraction introduced by Massidda et al. (2023, Def. 9 ).Example 3.5 (Linear chain). To illustrate the solutions in Prop. 3.4, we consider a linear chain𝑋 1 → 𝑋 2 → 𝑋 3 𝑋 Ω → 𝑋 4 𝑋 𝜋 (0)with adjacency 𝐴 𝑖 𝑗 = {1 for 𝑗=𝑖+1; 0 else} and targetEQUATION)τ0 puts all its weight on the direct parent of target 𝑋 4 because it mediates all causal influences. In contrast, ω1 puts weight on all variables in Ω because interventions on any of them influence 𝑋 4 
linear tcr algorithm: 
 In this section, we introduce an algorithm to learn a linear targeted causal reduction with shift interventions.Input 𝜆: learning rate, 𝑃(𝒊): intervention prior, Simulate(𝜽, 𝒊, 𝑛 sim ): function returning 𝑛 sim paths, 𝑁 ite : No. epochs, 𝐵, 𝐵 𝒊 : simulation/intervention batch size.Initialize 𝝉 1 , 𝝎, 𝜸 for 𝑚 = 1..𝑁 ite do 𝑋, 𝑌 ← [] for 𝑙 = 1..𝐵 𝒊 do 𝒊 𝑙 ← 𝑆𝑎𝑚 𝑝𝑙𝑒(𝑃(𝒊)) 𝑋 𝑙 = (𝒙 1 , .., 𝒙 𝐵 ); 𝑌 𝑙 ← 𝑆𝑖𝑚𝑢𝑙𝑎𝑡𝑒(𝜽, 𝒊 𝑙 , 𝐵) 𝑋 ← [𝑋 [:], 𝑋 𝑙 ]; 𝑌 ← [𝑌 [:], 𝑌 𝑙 ]; 𝐼 ← [𝐼 [:], 𝒊 𝑙 ] 𝐿 tot ← 𝐶𝑜𝑚 𝑝𝑢𝑡𝑒𝐿𝑜𝑠𝑠(𝑋, 𝑌 , 𝐼, 𝝉 1 , 𝜔 1 , 𝜸) ∇ 𝜸 , ∇ 𝝉 ← 𝐶𝑜𝑚 𝑝𝑢𝑡𝑒𝐿𝑜𝑠𝑠𝐺𝑟𝑎𝑑𝑖𝑒𝑛𝑡 (𝐿 tot ) (𝜸, 𝝉 1 , 𝝎 1 ) ← (𝜸 -𝜆∇ 𝜸 , 𝝉 1 -𝜆∇ 𝝉 1 , 𝝎 1 -𝜆∇ 𝝎 1 ) Output Estimated parameters (𝝉 1 , 𝝎 1 , 𝜸).Gaussian approximation of consistency loss. Since the KL divergence is challenging to compute in non-parametric settings, we make a Gaussian assumption on the densities. This allows us to obtain an analytic expression for the loss based on second order statistics (see expression in App. E.1).Overlap loss. To ensure differentiability of the reduction maps we do not implement the alignment 𝜋 explicitly, but encourage non-overlapping reduction maps via the regularizerEQUATIONwhere | • | is the element-wise absolute value.Balancing loss. Minimizing the Gaussian approximation of the consistency loss together with overlap regularization ( 12) there is nothing preventing the solution from attributing all non-zero weights in the 𝜏-and 𝜔 maps to one high-level variable while ignoring all others. In order to prevent such a collapse, we minimize stark differences between the high-level variables through the balancing termEQUATION)where 𝛼 𝑘 is the coefficient in the linear high-level mechanism corresponding to variable 𝑍 𝑘 .Gathering the losses, we get the total objective minimize 𝜸,𝝉,𝝎EQUATIONThe learning procedure is described in Algorithm 1. 
toy examples: linear gaussian low-level causal models: 
 Linear low-level causal models. We first test TCR by sampling from a linear Gaussian low-level model, rather than a simulation. We construct linear models of the form shown in Prop. 3.4 by drawing the non-zero entries in the adjacency matrix uniformly from the interval [-1, 1]. We learn a targeted causal reduction with two high-level variables: the target 𝑌 and its single cause 𝑍. Fig. 2a compares the learned 𝝉 1 and 𝝎 1 to the analytical solutions ( 9) and (10). We observe that, for these low-level models meeting the linear Gaussian assumption in Section 3, the learning algorithm converges to the global optimum.Two-branch model. To investigate the behavior of TCR with multiple high-level variables, we consider a low-level model with two branch causal structure (Fig. 2b ). With regularization for overlap (12) and balancing ( 13), the learned high-level variables correspond to the two branches. Within each branch, the reduction behaves as described for the linear chain in Ex. 3.5, where 𝜏 focusses on the direct parent of the target and 𝜔 is spread across all variables in the chain.Comprehensive experimental details are given in App. F.For a simulation based on an ODE system, we learn a targeted reduction of a ball moving in a double well potential under linear friction, as shown in Fig. 3 . The state vector 𝑿 encodes the 𝑥-position and velocity in 𝑥-direction of the ball at each time steps of the simulation.As shift-interventions, we apply small random shifts of the ball's velocity at each simulation time step, mimicking an applied external force. Initially, the ball starts on the left-hand side of the potential and starts oscillating. Since the ball experiences friction, it ends up in either the left or right minimum of the potential. The friction is relatively strong, such that, depending on the initial conditions and applied shift interventions, the ball either stays in the left well or crosses the middle hump once and stays in the right well (see Fig. 3 (f)). We learn a simple TCR with a single cause 𝑍 that explains the target 𝑌 . Further details about the nonlinear ODE system and training are given in App. F.2.The learned TCR parameters are shown in Fig. 3(c, d ). The 𝝉 1 and 𝝎 1 parameters for velocity are such that the larger the velocity is to the right, the higher 𝑍 and therefore the higher the predicted target 𝑌 , where positive 𝑌 correspond to the right well and negative to the left. Similarly, for the position parameter: the more negative the position just before the critical point of the ball crossing the hump, the higher the probability of predicting to stay in the left well. This corresponds to the correct dynamics of the system and also identifies the main drivers that influence the outcome 𝑌 . Fig. 3(e) shows how the learned TCR separates the phase space into simulations with enough momentum to the right to make it over the hump (pink) and those without (turquoise).Note that TCR does not focus on the part of 𝑿 which best predicts the final state of the system-like the position just before the end of the simulation. It rather highlights the variables which have the most impact on the target when they are intervened on, emphasizing the decisive time span when the ball either crosses the middle hump or stays in the left well.We simulate a two-dimensional system of four point masses with different weights connected by springs to their respective nearest neighbors, similar to the motivating example introduced in Sec. 1. Initially, the masses are arranged in a rectangle in space such that the springs are at rest length. The masses have a random initial velocity, as shown in Fig. 4(a) .As interventions, we apply random shifts to the velocities in 𝑥-and 𝑦-direction of each mass. The target of the simulation is the center of mass speed in (1, 1)-direction. We learn a TCR with two high-level causes. The full experimental details are given in App. F.3.While the velocities of the individual masses are coupled, the center of mass velocities in 𝑥-and 𝑦-direction of the system as a whole are independent, since the system is freely moving in space. The learned TCR correctly identifies these as the two independent causes of the target, with variable 𝑍 1 corresponding to the 𝑦-direction and 𝑍 2 to the 𝑥-direction. On average, each mass receives a similar shift in velocity through the applied interventions. However, since the masses are different, the shifts correspond to different contributions to the momentum of the system as a whole impacting the target. This is reflected in the relative weights of the learned maps being proportional to the weight of each point mass, as shown in Fig 4(f) .A second experimental setting with two groups of interconnected masses is shown in App. F.4.2, demonstrating a TCR learning independent causes along the mass index. 
discussion: 
 We introduce a novel approach for understanding complex simulations by learning high-level causal explanations from low-level models. Our Targeted Causal Reduction (TCR) framework leverages interventions to obtain simplified, highlevel representations of the causes of a target phenomenon.We formulate the intervention-based consistency constraint as an information theoretic learning objective, which favors the most causally relevant explanations of the target. Under linearity and Gaussianity assumptions, we provide analytical solutions and study their uniqueness, which provides insights into TCR's governing principles. One key assumption to obtain identifiability is that the leaf node, the target, is observed. However, this is to the best of our knowledge the first identifiability proof for a general class of CMR for which the high-level variables are continuous and partially unknown. Notably, the 𝑛-cause TCR provides a form of causal independent component analysis akin to the work of Wendong et al. (2023) but in a non-invertible setting and with a one dimensional target. We provide an algorithm for linear TCR and show it can effectively uncover the key causal factors influencing a phenomenon of interest.We demonstrate TCR on both synthetic models and scientific simulations, highlighting its potential for addressing the challenges posed by increasingly complex systems in scientific research.While we develop a CMR framework to learn high-level explanations for simulations, the simulation itself does not have to be explicitly formulated as a causal model and the causal relationships between variables in 𝑿 do not have to be known a priori. The only additional element needed to learn TCR is a notion of shift-interventions. We think that most scientific simulations based on differential equations naturally allow for a reasonable notion of shift interventions.Limitations and future work. To foster interpretability and tractability, we made Gaussian approximations and used linear 𝝉 and 𝝎 maps. While this has clear benefits, this may be too limiting for some complex simulations, and future work should explore more flexible approaches. Additionally, our method relies on performing a large number of interventions in simulation runs, which represents an additional cost in the context of large-scale simulation. How to make the algorithm scale to this setting is left to future work.Armin Kekić 1 Bernhard Schölkopf 1 Michel Besserve 11 Max Planck Institute for Intelligent Systems, Tübingen, GermanyDesiderata for CMR have been addressed theoretically by several works, in particular in the context of CFL (Chalupka et al., 2015 (Chalupka et al., , 2016)) , and subsequently with the notion of exact transformations (Rubenstein et al., 2017) and a more strongly constrained subclass: causal abstractions (Beckers and Halpern, 2019 ). An alternative framework for composing abstractions of finite models has been proposed by Rischel and Weichwald (2021) . However, only few works have addressed how to build high-level representations from the low-level system data only. A line of works focuses on language models (Geiger et al., 2021 (Geiger et al., , 2023)) , where high-level variables and interpretations are readily available.We start from the opposite direction and develop a general approach to build the high-level abstraction from the ground-up. Such a construction is done in CFL (Chalupka et al., 2015 (Chalupka et al., , 2016)) , where high-dimensional microscopic variables are turned into discrete high-level variables. Zennaro et al. (2023) addressed this question in the context of finite and discrete domains, by minimizing the maximum Jensen-Shannon divergence over a finite set of perfect intervention distributions. In contrast with most works on CMR, our framework is fully compatible with imperfect (soft intervention) at the low level, which are more realistic and interpretable perturbations of many real-world systems than hard interventions. Soft interventions have been used for language model alignment (Geiger et al., 2024) , and their theoretical compatibility with the abstraction framework has been investigated by Massidda et al. (2023) . Our approach aims at approximating an exact transformation, and is thus a relaxation of this setting.Other theoretical frameworks for approximate abstractions have been proposed (Beckers et al., 2020; Rischel and Weichwald, 2021) . Our work differs by providing an explicit loss well-suited to continuous causal models, that can be optimized efficiently and provide interpretable outcomes thanks to a cause-mechanism decomposition, a lower bound, and analytic solutions.The way we relax constraints on low-level interventions shares also similarities with the views of Zhu et al. (2024) who consider stochastic low-level do-interventions sampled according to the observational distribution, our work is instead focused on soft-interventions, for which we impose a prior distribution, reflecting the relative importance that we put on them. Our optimization objective is averaged over this prior, such that it plays a role in the final solution.Our approach also relates to the search for optimal interventional or counterfactual manipulations to steer the output of a system to a particular value or distribution (Amos et al., 2018; Besserve et al., 2020) or to best explain an observation (Budhathoki et al., 2022; von Kügelgen et al., 2023b) . We are in a way also selecting particular manipulations, but through the choice of dimensionality reduction 𝜔, such that they are interpretable at a high level.Finally, our approach relates to several works in causal representation learning, which have addressed identifiability of latent causal models from observational data, with (Wendong et al., 2023) or without (Squires et al., 2023; von Kügelgen et al., 2023a) assumptions on the latent causal graph. In contrast to those works, TCR does not assume an injective mapping of the mapping of the observations to the latent variables, such that the high-level model typically losses information relative to the low-level model.Methods for the numerical approximations of scientific models is a broad area spanning multiple fields. We provide here a few elements based on a 1D example to justify how these models relate to SCMs. The Euler method (Euler, 1794) , can be used to approximate a 1D ODE of the form 𝑥(𝑡 0 ) = 𝑥 0 , 𝑑 𝑥 𝑑𝑡 = 𝐹 (𝑥(𝑡)) , with 𝐹 smooth real-valued function, using a discretized time grid with time step Δ𝑡. The finite difference approximation of the derivative𝑥(𝑡 + Δ𝑡) -𝑥(𝑡) ≈ Δ𝑡 • 𝑑𝑥 𝑑𝑡, leads to the iterative numerical scheme for the approximation x(𝑘Δ𝑡):x(𝑡 0 ) = 𝑥 0 , x(𝑡 0 + (𝑘 + 1)Δ𝑡) = x(𝑡 0 + 𝑘Δ𝑡) + Δ𝑡 • 𝐹 ( x(𝑡 0 + 𝑘Δ𝑡)) , 𝑘 > 0 .This scheme is called explicit because future values depend explicitly on past ones. In that case, one may define the 𝑁 low-level endogenous variables as 𝑿 = [ x(𝑡 0 + Δ𝑡), . . . , x(𝑡 0 + 𝑁Δ𝑡)] ⊤ . They can be seen as pertaining to a chain SCM with structural equations𝑋 1 𝑥 0 + Δ𝑡 • 𝐹 (𝑥 0 ) 𝑋 𝑘+1 𝑋 𝑘 + Δ𝑡 • 𝐹 ( 𝑋 𝑘 ) , 𝑘 > 1.Because the ODE describes deterministic dynamics, the corresponding SCM is deterministic as well, i.e. exogenous variables can be taken as trivial zero constants. However, if we turn this ODE into the following 1D SDE𝑋 (𝑡 0 ) = 𝑥 0 , 𝑑𝑋 = 𝐹 (𝑋 (𝑡))𝑑𝑡 + 𝜎 𝑊 • 𝑑𝑊 ,where 𝑊 is a standard Brownian motion, then the Euler-Murayama method generalizes the previous approximation (Sauer, 2013) , and leads to an updated SCM with structural equation𝑋 1 𝑥 0 + Δ𝑡 • 𝐹 (𝑥 0 ) + 𝑈 1 𝑋 𝑘+1 𝑋 𝑘 + Δ𝑡 • 𝐹 ( 𝑋 𝑘 ) + 𝑈 𝑘+1 , 𝑘 > 1 ,where the exogenous variables 𝑈 𝑘 represent the increments of the scaled Brownian motion 𝜎 𝑊 • 𝑊 between successive time steps, and are thus jointly independent Gaussian due to fundamental properties of Brownian motion.This approach generalizes to explicit numerical schemes for multivariate ODE and SDEs where the state variable 𝑿 is an element of R 𝑛 . As an illustration, we can take the following class of SDE models 𝑛×𝑛) , and 𝑾 a 𝑛-dimensional standard Brownian motion. This leads to the scheme𝑿 (𝑡 0 ) = 𝒙 0 , 𝑑 𝑿 = 𝑭( 𝑿 (𝑡))𝑑𝑡 + 𝝈 𝑊 • 𝑑𝑾 , with 𝑭 : R 𝑛 → R 𝑛 , 𝝈 𝑊 = R (𝑿 1 𝒙 0 + Δ𝑡 • 𝑭(𝒙 0 ) + 𝑼 1 𝑿 𝑘+1 𝑿 𝑘 + Δ𝑡 • 𝑭( 𝑋 𝑘 ) + 𝑼 𝑘+1 , 𝑘 > 1 ,where the 𝑼 𝑘 are now multivariate Gaussian variables, whose components may or may not be independent depending on the choice of the matrix 𝝈 𝑊 . If the exogenous components are independent, the variables can be described by a standard SCM as introduced in the main text. If the exogenous components are dependent, the variables can be described by a more general notion of SCM, allowing hidden confounding (Bongers et al., 2021) .Further generalization to numerical schemes for Stochastic Partial Differential Equations (SPDEs) using finite difference approximations for partial derivatives with respect to other variables than time are also possible (Millet and Morien, 2005) .In the context of the main text example, we assume each point mass is submitted to a fluid friction force opposing its movement with fixed coefficient 𝜆. Masses are moreover intervened on via additional external forces { 𝒇 𝑘 }. Finally, internal forces are exerted on mass 𝑘 by other point masses of the system, summing up to 𝒈 𝑘 . Newton's second law applied to individual masses results in the following system of 2D vector equations𝑚 𝑘 𝑑𝒗 𝑘 𝑑𝑡 = -𝜆𝒗 𝑘 (𝑡) + 𝒇 𝑘 (𝑡) + 𝒈 𝑘 (𝑡) .We can approximate each equation to iteratively estimate the 𝑥 and 𝑦 components of the speed of individual point masses in the system, using a small time-step Δ𝑡, such that we get the discrete time estimates v𝑥,𝑘 [𝑛] ≈ 𝑣 𝑥,𝑘 (𝑛Δ𝑡) and v𝑦,𝑘 [𝑛] ≈ 𝑣 𝑦,𝑘 (𝑛Δ𝑡) satisfying𝑚 𝑘 • v𝑥,𝑘 [𝑛 + 1] (1 -Δ𝑡𝜆) • 𝑚 𝑘 • v𝑥,𝑘 [𝑛] + Δ𝑡 • 𝑓 𝑥,𝑘 (𝑛Δ𝑡) + Δ𝑡 • 𝑔 𝑥,𝑘 (𝑛Δ𝑡) , 𝑚 𝑘 • v𝑦,𝑘 [𝑛 + 1] (1 -Δ𝑡𝜆) • 𝑚 𝑘 • v𝑦,𝑘 [𝑛] + Δ𝑡 • 𝑓 𝑦,𝑘 (𝑛Δ𝑡) + Δ𝑡 • 𝑔 𝑦,𝑘 (𝑛Δ𝑡).Here, 𝑓 represents external forces, 𝜆 is a viscous damping coefficient, and 𝑔 denotes internal forces. We consider 𝒊 to be the vector of all components of external forces, and the target variable to be the final horizontal speed of the center of mass at iteration 𝑁. From the physics of freely moving systems of points, it is clear that the target variable can be predicted by considering only the horizontal dynamics of the center of mass. More precisely, we integrate the sum of external forces over the time span of the experiment, and use the last intervened time point 𝑛 𝑓 to predict the final outcome of the simulation, leading to the reduction𝑍 ( 𝜔 (𝒊) ) 1 = ∑︁ 𝑘 𝑚 𝑘 𝑣 𝑥,𝐺 [𝑛 𝑓 ] = ∑︁ 𝑘 𝑚 𝑘 𝑣 𝑥,𝑘 [𝑛 𝑓 ] = Δ𝑡 𝑛 𝑓 ∑︁ 𝑛=0 (1 -Δ𝑡𝜆) (𝑛 𝑓 -𝑛) ∑︁ 𝑘 𝑓 𝑥,𝑘 (𝑛Δ𝑡)) , 𝑌 = 𝑣 𝑥,𝐺 [𝑁] (1 -Δ𝑡𝜆) ( 𝑁 -𝑛 𝑓 ) 𝑍 1 + 𝑁 ∑︁ 𝑛=𝑛 𝑓 +1 ∑︁ 𝑘 𝑓 𝑥,𝑘 (𝑛Δ𝑡)) .To make the notation compatible with that used in our TCR framework, we can gather all speed variables in a highdimensional vector 𝑿 and all external force variables in a vector 𝒊, the high-level causal model is thus generated by a linear 𝜏-map and linear 𝜔 map for shift interventions, taking the form of the exact transformation𝑍 1 = 𝝉 ⊤ 1 𝑿 + 𝝎 ⊤ 1 𝒊 , 𝑌 = 𝝉 ⊤ 0 𝑿 + 𝝎 ⊤ 0 𝒊 𝑓 (𝑍 1 ) .where the term 𝝎 ⊤ 0 𝒊 accounts for interventions happening between discrete times 𝑛 𝑓 + 1 and 𝑁 and thus affect 𝑌 without being mediated by 𝑍 1 . In our framework, only interventions mediated by the cause, reflected in the term 𝝎 ⊤ 1 𝒊, are accounted for in the high-level model.We complete the main text definition to include the constraint on the intervention map 𝜔 Definition B.1. (𝜏, 𝜔) : (X → Z, I → J ) is a constructive (𝜏 -𝜔)-transformation between model L and H if there exists an alignment map 𝜋 mapping each high-level endogenous variable to a subset of low-level endogenous variables such that for all 𝑘 ≠ 𝑙, 𝜋(𝑘) ∩ 𝜋(𝑛) = ∅ and we have both• for each component 𝜏 𝑘 of 𝜏 there exists a function τ𝑘 such that for all 𝒙 in X, 𝜏 𝑘 (𝒙) = τ𝑘 (𝒙 𝜋 (𝑘 ) ) ;• for each component 𝜔 𝑘 of 𝜔 there exists a function ω𝑘 such that for all 𝒊 in I, 𝜔 𝑘 (𝒊) = ω𝑘 (𝒊 𝜋 (𝑘 ) ) .We first reformulate Proposition 3.1 more formally as follows.Proposition C.1. The consistency loss is positive, invariant to invertible reparametrizations as defined in Definition D.1, and vanishes if and only if the transformation is exact for almost all interventions. It admits the following decomposition:EQUATION)and is an upper bound of the causal relevance lossEQUATION)Proof. Positivity of the loss comes from the positivity of the KL-divergence. Taking the expectation of this divergence with respect to 𝑃(𝒊) thus must be positive too.Invariance to reparameterizations. We assume a reparametrization 𝜌 designed according to the framework introduced in Appendix 𝐷.1. By invariance of the KL divergence to invertible transformations, we have equality between the KL associated to the two different reductions (𝜏, 𝜔) and (𝜌 • 𝜏, 𝜓 • 𝜔):KL 𝑃 (𝒊) 𝜏 (𝑌 , 𝒁)||𝑃 ( 𝜔 (𝒊) ) H,𝜸 (𝑌 , 𝒁) = KL ρ# [ 𝑃 (𝒊) 𝜏 (𝑌 , 𝒁)] || ρ# [𝑃 ( 𝜔 (𝒊) ) H,𝜸 (𝑌 , 𝒁)] = KL 𝑃 (𝒊) 𝜌•𝜏 (𝑌 , 𝒁)||𝑃 ( 𝜓•𝜔 (𝒊) ) H,𝜸 ′ (𝑌 , 𝒁) .The transformation (𝜌, 𝜓) thus leaves L cons invariant.Cause-mechanism decomposition. Under our setting (see Sec. 3.1), the interventional distribution of the high-level causal model factorizes as𝑃 ( 𝜔 (𝒊) ) (𝑌 , 𝒁) = 𝑃 (0) 𝑌 | 𝒁 𝑃 ( 𝜔 (𝒊) ) ( 𝒁) .The pushforward (by reduction) of the interventional distribution of the low-level model factorizes as𝑃 (𝒊) (𝑌 , 𝒁) = 𝑃 (𝒊) 𝑌 | 𝒁 𝑃 (𝒊) (𝒁) , with 𝑃 (𝒊) ( 𝒁) = 𝜏 1,# [𝑃 (𝒊) ( 𝑿 𝜋 (1) )] and 𝑃 (𝒊) 𝑌 | 𝒁 = 𝜏 # 𝑃 (𝒊) (𝑿𝜋(0) ,𝑿 𝜋 (1) ) 𝜏 1,# [ 𝑃 (𝒊) ( 𝑿 𝜋 (1) ) ].Thus, the KL divergence can be decomposed asKL 𝑃 (𝒊) (𝑌 , 𝒁)||𝑃 ( 𝜔 (𝒊) ) (𝑌 , 𝒁) = ∫ Y ∫ Z 𝑃 (𝒊) (𝑌 , 𝒁) log 𝑃 (𝒊) (𝑌 , 𝒁) 𝑃 ( 𝜔 (𝒊) ) (𝑌 , 𝒁) 𝑑 𝒁𝑑𝑌 = ∫ Y ∫ Z 𝑃 (𝒊) 𝑌 | 𝒁 𝑃 (𝒊) (𝒁) log 𝑃 (𝒊) 𝑌 | 𝒁 𝑃 (𝒊) (𝒁) 𝑃 (0) 𝑌 | 𝒁 𝑃 ( 𝜔 (𝒊) ) ( 𝒁) 𝑑 𝒁𝑑𝑌 = KL 𝑍 𝑃 (𝒊) ( 𝒁) ||𝑃 ( 𝜔 (𝒊) ) (𝒁) + E 𝑧∼ 𝑃 (𝒊) (𝒁 ) KL 𝑌 𝑃 (𝒊) 𝑌 | 𝒁 = 𝑧 ||𝑃 (0) 𝑌 | 𝒁 = 𝑧 = KL 𝑍 𝑃 (𝒊) ( 𝒁) ||𝑃 ( 𝜔 (𝒊) ) (𝒁) + KL 𝑌 ,𝑍 𝑃 (𝒊) 𝑌 , 𝒁 ||𝑃 (0) 𝑌 | 𝒁 𝑃 (𝒊) ( 𝒁) .We call the first term cause consistency loss, as it matches the definition of a consistency loss but for cause variables only. The second term can be thought of as a mechanism consistency loss, where we use the ground truth low-level cause distribution to probe the similarity of the outputs of the "true" (in fact, the conditional distribution) and approximate mechanism. Our interpretability choice prevents the high-level mechanism from being intervened on, so a single stochastic map (i.e. a Markov kernel) must fit at best all the sampled experimental conditionals.We may ask the question of causal relevance of high-level causes. One way to quantify this is to assess whether the variations of the target due to low-level interventions are well captured by high-level interventions, which can be measured by a KL divergence on the target's marginalL rel = E 𝒊∼ 𝑝 (𝒊) KL 𝑌 𝑃 (𝒊) (𝑌 ) ||𝑃 ( 𝜔 (𝒊) ) (𝑌 ) .Note: In the Gaussian 1D case, the formula for the causal relevance loss isL rel = 1 2 E 𝒊∼ 𝑝 (𝒊)        (𝜇 𝑌 + 𝛼𝝎 ⊤ 𝒊 -𝜇 (𝒊) 𝑌 ) 2 𝜎 2 𝑌 + 𝜎 2 (𝒊) 𝑌 𝜎 2 𝑌 -ln 𝜎 2 (𝒊) 𝑌 𝜎 2 𝑌 -1       .Interestingly, we can break down this term usingKL 𝑃 (𝒊) (𝑌 , 𝒁)||𝑃 ( 𝜔 (𝒊) ) (𝑌 , 𝒁) = KL 𝑌 𝑃 (𝒊) (𝑌 ) ||𝑃 ( 𝜔 (𝒊) ) (𝑌 ) + E 𝑦∼ 𝑃 (𝒊) (𝑌 ) KL 𝑌 𝑃 (𝒊) 𝒁|𝑌 = 𝑦 ||𝑃 ( 𝜔 (𝒊) ) 𝒁|𝑌 = 𝑦where both terms are positive by positivity of the KL divergence. As a consequence,KL 𝑌 𝑃 (𝒊) (𝑌 ) ||𝑃 ( 𝜔 (𝒊) ) (𝑌 ) = KL 𝑃 (𝒊) (𝑌 , 𝒁)||𝑃 ( 𝜔 (𝒊) ) (𝑌 , 𝒁) -E 𝑦∼ 𝑃 (𝒊) (𝑌 ) KL 𝑌 𝑃 (𝒊) 𝒁|𝑌 = 𝑦 ||𝑃 ( 𝜔 (𝒊) ) 𝒁|𝑌 = 𝑦 ≤ KL 𝑃 (𝒊) ( 𝑌 , 𝒁)||𝑃 ( 𝜔 (𝒊) ) (𝑌 , 𝒁) = L cons .so the minimized consistency loss is an upper bound to causal relevance. 𝑿 𝐴𝑿 + 𝑼 + 𝒊 , 𝒊 ∼ 𝑃(𝒊) , 𝑈 𝑘 ∼ N (𝜇 𝑘 , 𝜎 2 𝑘 ) , 𝜎 𝑘 > 0 ,such that 𝑿 and 𝐴 take the block forms𝑿 = 𝑿 𝜋 (0) 𝑿 Ω , 𝐴 = 𝐴 00 𝐴 0Ω 0 𝐴 ΩΩ .Given an arbitrary choice of linear scalar target of the form 𝑌 = 𝜏 ⊤ 0 𝑿 = τ⊤ 0 𝑿 𝜋 (0) and under Assum. 3.2 or Assum. 3.3, there is a unique linear 1-cause TCR (up to a multiplicative constant) satisfying L cons = 0. It is given byEQUATION)Moreover, let 𝑛 𝑚𝑎𝑥 be the maximum number 𝑛 such that a linear 𝑛-cause TCR can achieve L cons = 0. If there are no cancellations1 among causal pathways from each node in supp( ω1 ) of Eq. ( 10) towards 𝑌 , then the 𝑛 𝑚𝑎𝑥 -cause TCR is unique up to rescaling and permutation of the causes.Proof. Part 1: 1-cause TCR.Overview. We exploit the positive definiteness of the KL loss and its continuity with respect to 𝒊. Since the variables are jointly Gaussian, continuity is obvious from the analytical expression of the KL for Gaussian variables and continuity of the shift operation applied to the parameters of the Gaussian. We exploit the cause-mechanism decomposition and the lower-bound by L cons to progressively identify necessary conditions on parameters to have L cons = 0 and finally check those conditions are sufficient.Preliminaries. Let 𝑁 0 denote the size of 𝜋(0) and 𝑁 1 be the size of 𝜋(1). The SCM is assumed uniquely solvable (Definition 2.1), such that 𝒙 = 𝐴𝒙 + 𝒖 has a unique solution for almost all values of 𝒖. Since 𝑼 has full support, this implies that 𝐼 𝑁 -𝐴 is invertible. The low-level variables then satisfy𝑿 (𝒊) = (𝐼 𝑁 -𝐴) -1 (𝑼 + 𝒊)where, due to the block triangular form of 𝐴,(𝐼 𝑁 -𝐴) -1 = (𝐼 𝑁 0 -𝐴 00 ) -1 , (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 0 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 .In the assumed model all low-level variables are either in 𝜋(0) or in Ω. Since the CMR is constructive we have 𝜋(1) ⊂ Ω.Without loss of generality, we can impose 𝜋(1) = Ω by setting the unused components of 𝝉 1 and 𝝎 1 to zero. For an arbitrary interventional setting 𝒊, this leads to the mapping of the low-level variable to the high-level cause variable, which we denote ) ), to satisfy𝑍 (𝒊) 1 = 𝜏 1 ( 𝑿 (𝒊EQUATIONMoreover, because we assume also shift interventions in the high-level model, the cause 𝑍 1 in this model has an interventional distribution satisfying𝑃 ( 𝜔 1 (𝒊) ) (𝑍 1 ) = 𝑃 (0) 𝑍 1 -𝝎 ⊤ 1 (𝒊) .Necessary conditions. We are looking for solutions satisfying L cons = 0. By positivity of the KL divergence, this implies that for almost all 𝒊, the distribution of 𝑍 (𝒊) 1 matches the learned high-level interventional distribution of high-level cause 𝑍 1 , which satisfies𝑃 ( 𝜔 1 (𝒊) ) (𝑍 1 ) = 𝑃 (0) 𝑍 1 -𝝎 ⊤ 1 (𝒊) .Matching unintervened distributions of 𝑍 1 . If Assum. 3.2 holds, the prior 𝑃(𝒊 Ω ) has density with respect to the Lebesgue measure with support including a neighborhood of 𝒊 Ω = 0. By continuity of the KL divergence with respect to the intervention parameters, a solution making the consistency loss vanish needs to have the KL divergence term vanish for 𝒊 Ω = 0 (otherwise we could find a neighborhood of 𝒊 Ω = 0 such that the KL divergence does not vanish, by continuity of the KL divergence, and L cons would be non-vanishing, contradicting our assumption).Alternatively, Assum. 3.3 also obviously implies vanishing of the KL divergence for the unintervened setting.This vanishing of the KL divergence entails, again by positivity, that its terms, the two unintervened densities, are equal, such that we get, using Eq. ( 17)EQUATIONwhich entails the following constraints on the variance and mean of the high-level causeEQUATIONandEQUATIONMatching interventional distributions of 𝑍 1 . For the same reasons, under Assum. 3.2, we can further match the interventional distributions in an open set included in the interior of the support of 𝑃(𝒊), such that for all 𝒊 in this open set the following distributions are the same𝑃 ( 𝜔 1 (𝒊) ) (𝑍 1 ) = N ( τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 ( 𝝁 Ω + 𝒊 Ω ), τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ τ1 ) and (𝜏 1 ) # 𝑃 (𝒊) ( 𝑿) = N (𝜇 𝑍 1 + 𝝎 ⊤ 1 𝒊, 𝜎 2 𝑍 1 ) = N ( τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 𝝁 Ω + 𝝎 ⊤ 1 𝒊, 𝜎 2 𝑍 1 ) .Indeed, otherwise the KL would not vanish in a neighborhood of non-zero measure and would contradict the assumption that L cons vanishes.This implies that for all 𝒊 in this open neighborhoodτ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 ( 𝝁 Ω + 𝒊 Ω ) = τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 𝝁 Ω + ω⊤ 1 𝒊 Ω , which simplifies to τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 𝒊 Ω = ω⊤ 1 𝒊 Ω .Since this equality between two linear functions of 𝒊 𝜋 (1) is valid on an open set of the vector space of 𝒊 𝜋 (1) , these functions must be equal (we can reparameterize 𝒊 to show that the linear maps must match on a basis of the space, so they are equal). This is valid if and only if, in addition to Eqs. (20-21),EQUATIONis verified.Alternatively, we obtain the same result by replacing Assum. 3.2 by Assum. 3.3. Indeed, the finite distribution over interventions imposes that the KL term inside the expectation must vanish for each of them (including the unintervened distribution). As long as the collection of finite interventions vectors forms a rank #Ω = #𝜋(1) = 𝑁 1 family, we can choose a subset of 𝑁 1 such vectors {𝒊 1 Ω , . . . , 𝒊 𝑁 1 Ω } such that it forms a linearly independent family. It can be used to build the matrix equalityEQUATIONwhere the matrix 𝒊 1 Ω , . . . , 𝒊 𝑁 1 Ω is invertible. By right-multiplying Eq. ( 23) by this inverse, we obtain Eq. ( 22) again.Matching distributions of 𝑌 . We can move on to check the implication of consistency of the effect's conditional. It entails for almost all of 𝒊 𝑃 (𝒊) (𝑌 |𝑍) 𝑃 (𝒊) (𝑍) = 𝑃 ( 𝜔 (𝒊) ) (𝑌 |𝑍) 𝑃 (𝒊) (𝑍) = 𝑃 (0) (𝑌 |𝑍) 𝑃 (𝒊) (𝑍) .Introducing 𝑇 = 𝝉 ⊤ 0 𝝉 ⊤ 1the left-hand side is obtained by using(𝑌 , 𝑍) ∼ N (𝑇 𝜇 𝑿 , 𝑇 Σ 𝑿 𝑇 ⊤ ) .And the right-hand side by using 𝑌 = 𝑓 (𝑍 1 ) + 𝑅 0 .Fitting first only the marginals of 𝑌 , we obtain necessary conditions. We have𝑃 (𝒊) (𝑌 ) = 𝑃 ( 𝜔 (𝒊) ) (𝑌 )where for the left-hand side𝑌 = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝑼 𝜋 (0) + τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 (𝑼 𝜋 (1) + 𝒊)and for the right-hand side 𝑌 ∼ 𝑓 # [𝑃 ( 𝜔 (𝒊) ) (𝑍 1 )] * 𝑃(𝑅 0 ) .Given the affine mechanism assumption of Eq. ( 7), 𝑓 (𝑍) = 𝛼𝑍 + 𝛽 and under Assum. 3.2, equality of marginal distributions entails the following equality for all 𝒊 Ω in an open neighborhood of 0 (otherwise L rel ≤ L cons would not vanish)τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝝁 𝜋 (0) + τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 ( 𝝁 𝑼 1 + 𝒊 Ω ) = 𝛼 τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 (𝜇 Ω + 𝒊 Ω ) + 𝛽 + 𝝁 𝑅 0 which requires (setting 𝒊 = 0) 𝛽 + 𝝁 𝑅 0 = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝝁 𝜋 (0). We can fix 𝝁 𝑅 0 to zero to avoid redundancy of additive constants, such thatEQUATIONand consistency of non-zero shift interventions additionally entail for all 𝒊 Ω in the support of 𝑃(𝒊 Ω )τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 𝒊 Ω = 𝛼 τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 𝒊 Ω .Since one can always choose a linearly independent family of vectors 𝒊 Ω within the open neighborhood of zero for which this equality holds, this yieldsEQUATIONSimilarly as above, the same conclusion can be drawn if we replace Assum. 3.2 by Assum. 3.3.Sufficiency of the constraints. We have derived the expressions of the TCR parameters in Eqs. (22, 24, 25) from necessary conditions for matching the marginals of high-level variables, 𝑃(𝑍 1 ) and 𝑃(𝑌 ), to their corresponding pushforward distributions of the low-level variables, (𝜏 1 ) # [𝑃( 𝑿)] and(𝜏 0 ) # [𝑃( 𝑿)]. Now what remains is to check the same for conditional distributions to show those conditions are sufficient. Indeed, this implies that the joint high-level distributions of (𝑌 , 𝑍 1 ) and (𝜏 0 ( 𝑿), 𝜏 1 ( 𝑿)) are matching.Let us first note that given that the low-level model, the 𝜏 maps and the high-level mechanisms are linear or affine, and that the low-level exogenous variables are Gaussian, the exogenous high-level variable will necessarily be Gaussian as well to satisfy the consistency constraints.Let us now compute the covariance matrix of the low-level variables.𝑐𝑜𝑣( 𝑿) = (𝐼 𝑁 0 -𝐴 00 ) -1 (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 0 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ 𝑈 (𝐼 𝑁 0 -𝐴 00 ) -⊤ 0 (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ 𝐴 ⊤ 0Ω (𝐼 𝑁 0 -𝐴 00 ) -⊤ (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ .Because the exogenous covariance is block diagonal, we get=        (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ 𝐴 ⊤ 0Ω (𝐼 𝑁 0 -𝐴 00 ) -⊤ +(𝐼 𝑁 0 -𝐴 00 ) -1 Σ 𝜋 (0) (𝐼 𝑁 0 -𝐴 00 ) -⊤ (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ 𝐴 ⊤ 0Ω (𝐼 𝑁 0 -𝐴 00 ) -⊤ (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤       .Then, if we denote 𝑌 = 𝜏 0 ( 𝑿) 𝑐𝑜𝑣(( 𝑌 , 𝑍 1 )) = 𝑇 𝑐𝑜𝑣( 𝑿)𝑇 ⊤ and we can derive its conditional mean and covariance𝜇 𝑌 | ẑ1 = 𝜇 𝑌 + τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ 𝜋 (1) (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ τ1 τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ τ1 -1 ẑ1 -𝜇 𝑍 1 .Thus using the above equation𝜇 𝑌 | ẑ1 = 𝜇 𝑌 + 𝛼 ẑ1 -𝜇 𝑍 1 = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝝁 𝜋 (0) + τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 𝝁 Ω + 𝛼 ẑ1 -𝛼 τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1𝝁 Ω , which further simplifies with the same equation to𝜇 𝑌 | ẑ1 = 𝜇 𝑌 + 𝛼 ẑ1 -𝜇 𝑍 1 = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝝁 𝑼 0 + 𝛼 ẑ1 . Moreover, var( 𝑌 | ẑ1 ) = 𝜎 2 𝑌 -𝝉 ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 0Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ τ1 τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ τ1 -1 τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ Ω (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ 𝐴 ⊤ 0Ω (𝐼 𝑁 0 -𝐴 00 ) -⊤ τ0again, using the above equation this leads to the simplificationvar( 𝑌 | ẑ1 ) = 𝜎 2 𝑌 -𝛼 τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ 𝜋 (1) (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ 𝐴 ⊤ 0Ω (𝐼 𝑁 0 -𝐴 00 ) -⊤ τ0 = 𝜎 2 𝑌 -𝛼 τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 Σ 𝜋 (1) (𝐼 𝑁 1 -𝐴 ΩΩ ) -⊤ τ1 𝛼 = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 Σ 𝜋 (0) (𝐼 𝑁 0 -𝐴 00 ) -⊤ τ0 .For the high-level distribution we get 𝑃(𝑌 |𝑧) = N (𝛼𝑧 + 𝛽, 𝜎 2 𝑌 | 𝑍 ) where we can identify all parameters, including the mean and variance of the Gaussian exogenous variable 𝑅 0 , with the above Eqs (22, 24, 25) .Part 2: 𝑛-causes case (𝑛 > 1)We now assume (𝜏 ′ 𝑘 , 𝜔 ′ 𝑘 , 𝜋 ′ ) 𝑘=1..𝑛 such that the loss vanishes, but no such solution for 𝑛 + 1 causes. Properties of exact 𝑛-cause solutions Then such a solution can be linked to the 1-cause solution, which is guaranteed to exist according to our set of assumptions. Indeed, the existence of the n-cause solution implies that the pushfoward interventional distribution of the low-level causal model by 𝜏, 𝑃 satisfies𝑃 (𝒊) (𝑌 |𝒁 = 𝒛) ∼ N ∑︁ 𝑘 𝛼 𝑘 𝑧 𝑘 + 𝛽, 𝜎 2 𝑌 | 𝑍 and 𝑃 ( 𝜔 (𝒊) ) ( 𝒁) ∼ 𝑘 𝑃 (0) (𝑍 𝑘 -𝜔 𝑘 (𝒊 𝑘 ))If we define the aggregate cause Z = 𝑘 𝛼 𝑘 𝑧 𝑘 = 𝑘 𝛼 𝑘 𝜏 𝑘 (𝑥), then we can rewrite the above model as𝑃 (𝒊) (𝑌 | Z = z) ∼ N z + 𝛽, 𝜎 2 𝑌 | 𝑍 and 𝑃 ( 𝜔 (𝒊) ) ( Z) ∼ 𝑘 𝑃 (0) ( Z - ∑︁ 𝑘 𝜔 𝑘 (𝒊 𝑘 ))which implies that concatenating the 𝜏 𝑘 with multiplicative coefficient 𝛼 𝑘 leads to a valid 1-cause TCR, and must thus match the expressions we have found for it, up to a multiplicative constant.Moreover, the interventional consistency of the 𝑛-causes, which do not influence each other according to the assumed high-level causal graph, entails that any low-level intervention 𝒊 affects only high-level variable 𝑍 𝑘 through is components in 𝜋(𝑘).We define 𝐴 1..𝑛,1..𝑛 by reordering the indices of Ω according to the assumed alignment 𝜋. Consistency then implies (using the above 1 cause solution proof)EQUATION)This implies that τ⊤ 𝑘 (𝐼 𝑘 𝑁 𝑘 -𝐴 1..𝑛,1..𝑛 ) -1 𝑘 𝑗 = 0 for all 𝑗 ∈ 𝜋(𝑙), 𝑙 ≠ 𝑘 ,where (.) 𝑘 𝑗 indicates the matrix block corresponding to indices in 𝜋(𝑘) × 𝜋( 𝑗). Because the non-vanishing coefficients of τ𝑘 reflect the influences along the causal pathways from nodes of 𝜋(𝑘) to 𝑌 , the above entails that (𝐼 𝑘 𝑁 𝑘 -𝐴 1..𝑛,1..𝑛 ) -1must vanish on the support of τ𝑘 . Indeed, otherwise Eq. ( 27) would indicate that causal pathways from nodes in 𝜋(𝑘) to 𝜋(0) cancel each other, which is forbidden by our assumptions.We thus deduce that any off-diagonal block element of (𝐼 𝑘 𝑁 𝑘 -𝐴 1..𝑛,1..𝑛 ) -1 whose row component belongs to the support of any 𝜏 𝑘 and whose column component belongs to the support of any 𝜔 𝑘 must vanish. Indeed, otherwise the causes would influence each other. In essence, this means that any node influencing the target must not influence any node though some causal pathway in another group with the same property.Identifiability of the 𝑛-cause solution We assume 𝑛 = 𝑛 𝑚𝑎𝑥 and consider a second 𝑛-cause solution, which we denote: (𝜏 ′ 𝑘 , 𝜔 ′ 𝑘 , 𝜋 ′ ) 𝑘=1..𝑛 . If 𝜋 ′ = 𝜋 up to a permutation of the order of the causes and removal of low-level variables that do not belong to the support of neither any 𝜔. Then identifiability of the corresponding 1-cause solution implies that each 𝜏 ′ 𝑘 is identified with each 𝜏 𝑘 up to a multiplicative constant, because they both match the components of the 1-cause 𝜏 on their (identical) support 𝜋(𝑘) The same goes for 𝜔 ′ 𝑘 and 𝜔 𝑘 . This corresponds to the conclusion of the Proposition. Otherwise, 𝜋 ′ ≠ 𝜋 even up to a permutation of the order of the causes and removal of low-level variables that do not belong to the support of neither any 𝜏 nor 𝜔. There should be an overlap between supports of omegas and taus of one cause of one solution with two different causes of the other solution. Without loss of generality, because of the block-diagonal structure of (𝐼 𝑘 𝑁 𝑘 -𝐴 1..𝑛,1..𝑛 ) -1 entailed by Eqs. (26-27) for both solutions, this overlap implies that 𝜋(𝑘) for at least one 𝑘 can be further partitioned and reordered into two subgroups, such that the corresponding diagonal block of (𝐼 𝑘 𝑁 𝑘 -𝐴 1..𝑛,1..𝑛 ) -1 can be turned in a block diagonal submatrix. This can be used to build a new alignment 𝜋 ′′ for 𝑛 + 1 causes, and its associated tau and omega maps such that Eq. ( 26) will be again satisfied, leading to interventional consistency of the (n+1)-causes. Moreover, because the exogenous variables in Ω are assumed independent, the block diagonal structure of the newly defined matrix (𝐼 𝑘 𝑁 𝑘 -𝐴 1..𝑛+1,1..𝑛+1 ) -1 entails that the covariance of the 𝑛 + 1 high-level cause variable will be diagonal, ensuring mutual independence between high-level causes. This procedure exhibits the existence of an (𝑛 + 1)-cause TCR, contradicting the original assumption that 𝑛 = 𝑛 𝑚𝑎𝑥 . This case is thus excluded. □In order to study invariance properties of TCR, we define transformations compatible with a class of reductions. Let 𝜌 : is also continuous invertible. Among this class of transformations, we define an invertible reparametrization of a TCR as follows.Z 1 × • • • × Z 𝑛 → Z 1 × • • • × Z 𝑛 beDefinition D.1. An invertible reparametrization of a reduction for the class T of 𝜏-maps and the class {H 𝜸 } 𝜸∈Γ satisfies the following properties.• it is compatible with the class of 𝜏-maps as follows: for any map 𝜏 ∈ T , we have ρ • 𝜏 ∈ T , • it is compatible with the high-level model class {H 𝜸 } as follows: for any model parameter 𝜸, the unintervened and intervened distributions 𝑃 H,𝜸 (𝑌 , 𝒁) are such that there exist a parameter 𝜸 ′ and a map between high-level interventions 𝜓 : J → J such that the joint distributions of the transformed variables (𝑌 , 𝜌( 𝒁)) is compatible with unintervened and intervened distributions ofH 𝜸 ′ , in the sense that ρ# [𝑃 ( 𝒋 ) H,𝜸 (𝑌 , 𝒁)] = 𝑃 ( 𝜓 ( 𝒋 ) )H,𝜸 ′ (𝑌 , 𝒁) .Whenever 𝜋(0) is a singleton, 𝜏 0 is univariate and the target 𝑌 essentially corresponds (up to trivial rescaling) to a single low-level variable. We elaborate on the interpretation of Proposition 3.4 in this context.Let us set τ0 = 1 and fix the target index such that 𝜋(0) = {𝑁 } without loss of generality. Then the DAG constraints entail 𝐴 00 = 0 and the structural equations take the form where 𝒂 01 is a column vector of coefficients of the low-level mechanism linking the target 𝑌 to its causes in 𝜋(0). Then the unique linear 1D TCR, up to a multiplicative constant, making the consistency loss vanish is given by τ1 = 𝒂 01 (30)EQUATION)This solution is easily interpretable: τ1 identifies the ground truth mechanism linking 𝑿 𝜋 (0) to the target, while ω1 traces the contribution of interventions on each endogenous variable to the target. Indeed, this contribution is given by the "reduced form" map between exogenous values and endogenous values (see proof of Proposition 3.4 for more insights)𝒊 ↦ → (𝐼 𝑁 -1 -𝐴 11 ) -1 𝒊 ,and by composing this mapping with mechanism 𝒂 01 we get the (shift) influence of interventions on the target𝒊 ↦ → 𝒂 ⊤ 01 (𝐼 𝑁 -1 -𝐴 11 ) -1 𝒊 = ω⊤ 1 𝒊 .The mismatch between ω1 and τ1 is due to the internal causal structure of the submodel described by eq. ( 28). Indeed, if there are no causal links within this subsystem, 𝐴 11 is a zeros matrix and ω1 = (𝐼 𝑁 -1 ) -⊤ τ1 = τ1 = 𝒂 01 , otherwise, the two maps will be different. The discrepancy between the vectors thus reflects the fact that the causal explanation links high-level endogenous variables and interventions on them by potentially complex low-level interactions that do not necessarily have a simple high-level interpretation. This justifies regularizing the consistency loss with an homogeneity loss in order to focus on explanations that exhibit congruent 𝜏 and 𝜔 maps.In the case of a chain SCM𝑿 1 → • • • → 𝑿 𝑁 -1 → 𝑿 𝑁 = 𝑌the above linear setting gets the additional constraints (using a causal ordering of the variables) that the target's mechanism is sparse 𝒂 ⊤ 01 = [0, . . . , 0, 𝑎 𝑁 ] and the structure matrix of 𝑿 𝜋 (1) is subdiagonal𝐴 11 =            0 0 . . . 0 0 𝑎 2 0 . . . 0 0 0 𝑎 3 . . . 0 0 0 0 . . . 0 0 0 0 . . . 𝑎 𝑁 -1 0           and as a consequence, the solution writesEQUATION)This solution is in line with our experimental results:• τ1 has all its weight on the parent of the target.• ω1 has a non-sparse distribution over the chains, decaying in the upstrean direction. This reflects that structure coefficients of 𝐴 11 are selected with absolute value inferior to one, such that the influence of ancestor nodes on the target decays with their distance to it on the graph.Transposing the chain example to the case of Proposition D.2, we can take the case were the direct parent 𝑋 𝑁 -1 of the target is left unintervened. In such a case, τ1 may put its weight on both 𝑋 𝑁 -1 and its direct parent 𝑋 𝑁 -2 , Proposition 3.4 provides two example solutions for different choices of 𝜋(1), including or excluding 𝑋 𝑁 -1 . In the most extreme case of dissimilarity between 𝝉 1 and 𝝎 1 , solution including 𝑋 𝑁 -1 in 𝜋(1) puts all 𝝉 1 's weight on 𝑋 𝑁 -1 , while 𝝎 1 has no weight on it (because it is unintervened). As a consequence, 𝝎 1 and 𝝉 1 are orthogonal and the associated homogeneity loss vanishes. In contrast, the unique solution excluding 𝑋 𝑁 -1 from 𝜋(1) have a larger cosine similarity and will thus be preferred by the homogenity-regularized loss. such that 𝑿, 𝐴 and 𝑃 take the block forms𝑿 =        𝑿 𝜋 (0) 𝑿 𝑆 𝑿 Ω        , 𝐴 =        𝐴 00 𝐴 0𝑆 0 0 𝐴 𝑆𝑆 𝐴 𝑆Ω 0 0 𝐴 ΩΩ       , with 𝜋(0) of size 𝑁 0 , and 𝜋(1) of size 𝑁 1 = 𝑁 -𝑁 0 and 𝑆 of size 𝑠. Then we know from the Proposition 3.4 that there is already a valid solution using 𝜋 as alignment. The only difference is that variables in 𝑆 are unintervened, which does not affect the ability of the solution to achieve L cons = 0. That solution would be compatible with interventions on 𝑆, but since 𝑆 is unintervened, we do not have uniqueness guarantees for this choice of 𝜋.Alternatively, if we choose 𝜋 ′ (0) = 𝜋(0) ∪ 𝑆 and 𝜋 ′ (1) = 𝜋(1) \ 𝑆 = Ω, then, we can again apply Propostion D.2, and see that it provides a different solution with this alignment, which is compatible with the given problem (constructive transformation with constraint on the mapping 𝜏 0 ). Importantly, the key indeterminacy is for the map 𝜏 1 , which will either put all its weight on elements in 𝑆 (direct parents of 𝜋(0)), or alternatively, put all its weights on elements in Ω. There is an additional, but trivial, indeterminacy for the map 𝜔 1 : indeed, since 𝑋 𝑆 is unintervened (as part of 𝜋(0)), the weights in 𝝎 1 associated to these coefficients may take arbitrary values (since their associated component in 𝒊 remains zero). We do not consider these trivial indeterminacies (which do not affect the mapping 𝜔 1 on its domain, i.e. the support of the prior 𝑃(𝒊)) by forcing the weights of 𝝎 1 associated to unintervened variables to zero. .Given an arbitrary choice of linear scalar target of the form 𝑌 = 𝜏 ⊤ 0 𝑿 = τ⊤ 0 𝑿 𝜋 (0) , under the conditions of Proposition 3.4, the unique linear 1-cause TCR (up to a multiplicative constant) is associated to a 1-cause constructive abstraction given by τ1 = 𝐴 ⊤ 01 (𝐼 # 𝜋 (0) -𝐴 00 ) -⊤ τ0 (34) ω1 = (𝐼 # 𝜋 (1) -𝐴 11 ) -⊤ τ1 ,τ𝑈,0 = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 , (36)and τ𝑈,1 = τ⊤ 1 (𝐼 𝑁 1 -) -1 .(37)Proof. To have a valid constructive causal abstraction, we need to verify the existence of an additional constructive map 𝜏 𝑈 for exogenous variables such that for all realizations 𝒖 of 𝑼.𝜏(L (𝒊) (𝒖)) = H ( 𝜔 (𝒊) ) (𝜏 𝑈 (𝒖))where L (𝒊) (.) and H ( 𝒋 ) (.) denote the mappings from endogenous to exogenous variable for the low-and high-level intervened models, respectively.Using L (𝒊) (𝑼) = 𝑿 (𝒊) = (𝐼 𝑁 -𝐴) -1 (𝑼 + 𝒊)we get 𝑿 (𝒊) 𝜋 (1) = (𝐼 𝑁 1 -) -1 (𝑼 𝜋 (1) + 𝒊 𝜋 (1) ) and 𝑿 (𝒊) 𝜋 (0) = (𝐼 𝑁 0 -𝐴 00 ) -1 𝑼 𝜋 (0) + (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 01 𝑿 (𝒊) 𝜋 (1) . Applying the 𝜏 0 map we get the solution of Prop. 3.4 𝑌 (𝒊) = τ⊤ 0 𝑿 (𝒊) 𝜋 (0) = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝑼 𝜋 (0) + τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝐴 01 𝑿 (𝒊) 𝜋 (1) = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 𝑼 𝜋 (0) + τ⊤ 1 𝑿 (𝒊)𝜋 (1) moreover applying the 𝜏 1 map to the variables in 𝜋(1) we get τ⊤ 1 𝑿 (𝒊) 𝜋 (1) = τ⊤ 1 (𝐼 𝑁 1 -) -1 (𝑼 𝜋 (1) + 𝒊 𝜋 (1) ) = τ⊤ 1 (𝐼 𝑁 1 -𝐴 ΩΩ ) -1 𝑼 𝜋 (1) + ω⊤ 1 𝒊 𝜋 (1)So by defining the vectors τ⊤ 𝑈,1 = τ⊤ 1 (𝐼 𝑁 1 -) -1 and τ⊤ 𝑈,0 = τ⊤ 0 (𝐼 𝑁 0 -𝐴 00 ) -1 we get a valid constructive abstraction linking the low-level map 𝒖 ↦ → L (𝒊) = 𝜏(L (𝒊) (𝒖)). □As the KL divergence is hard to estimate in the non-parametric setting, we make a Gaussian approximation of this loss to get an analytical, differentiable expression. Using the general formula for two n-dimensional Gaussian densities 𝑃 and 𝑄 Code is available at: https://github.com/akekic/targetedcausal-reduction.git.Causal pathways cancel if the linear coefficients quantifying the influence of a node on 𝑌 along different directed paths of the low-level SCM sum to zero. Assuming no cancellations is akin to assuming no faithfulness violations and generically satisfied(Sprites et al., 2001, Theorem3.2).Causal pathways cancel if the linear coefficients quantifying the influence of a node on 𝑌 along different directed paths of the low-level SCM sum to zero. Assuming no cancellations is akin to assuming no faithfulness violations and generically satisfied(Sprites et al.,2001, Theorem 3.2). 
 
 Appendix:  


Decision: Thanks to reviewer who shared feedback and score. Since there was only one review for this paper, I went through the paper and can agree with reviewer on the good organization as well as novelty of the approach. The paper proposes a causal inference tool on top of a numerical model simulating a dynamical systems (in this case ODEs considered only) to better interpret the cause of some behaviors/dynamics.  I appreciate detailed information authors provided in appendix and also discuss the limitations and future works. I was also wondering if authors could address whether their approach can be integrated on a ML-based solver of ODEs or PDEs and if not, what are the challenges. I vote accept for this

Reviews: review: title: title: A method for condensing complex intervenable models into a concise set of causal factors for explaining a specific target phenomenon
main_review: main_review: In this paper, the authors proposed a novel approach to understanding complex simulations by learning high-level causal explanations from low-level models.
It is a well-organized paper containing an algorithm and diagrams to help the readers understand the proposed scheme. Moreover, it contains the limitations and future work parts.
However, a comparison with existing methods is unavailable. Comparison with the latest and existing work will highlight the benefits of the proposed method and help the readers understand different aspects of the proposed method.