Causal Abstraction Inference under Lossy Representations
Authors: Kevin Muyuan Xia
Elias Bareinboim
Venue: ICML 2025 poster (2025)
Keywords: causality
causal inference
causal abstractions
neural networks
deep learning
representation learning
neural causal models
Primary area: general_machine_learning->causality
TL;DR: We generalize causal abstraction models to handle cases where high-level interventions may be ambiguous, and this allows for practical causal inferences in high-dimensional settings.

Abstract: The study of causal abstractions bridges two integral components of human intelligence: the ability to determine cause and effect, and the ability to interpret complex patterns into abstract concepts. Formally, causal abstraction frameworks define connections between complicated low-level causal models and simple high-level ones. One major limitation of most existing definitions is that they are not well-defined when considering lossy abstraction functions in which multiple low-level interventions can have different effects while mapping to the same high-level intervention (an assumption called the abstract invariance condition). In this paper, we introduce a new type of abstractions called projected abstractions that generalize existing definitions to accommodate lossy representations. We show how to construct a projected abstraction from the low-level model and how it translates equivalent observational, interventional, and counterfactual causal queries from low to high-level. Given that the true model is rarely available in practice we prove a new graphical criteria for identifying and estimating high-level causal queries from limited low-level data. Finally, we experimentally show the effectiveness of projected abstraction models in high-dimensional image settings.

Causal Abstraction Inference under Lossy Representations
Kevin Xia
Elias Bareinboim'
Abstract
example, a human can easily parse the object in an image as a dog or car instead of interpreting it as a collection of pixel values. Combining these two modes of reasoning vital for building more advanced AI systems.
The study of causal abstractions bridges two inte- gral components of human intelligence: the abil- ity to determine cause and effect, and the abil- ity to interpret complex patterns into abstract concepts. Formally, causal abstraction frame- works define connections between complicated low-level causal models and simple high-level ones. One major limitation most existing def- initions is that they are not well-defined when considering lossy abstraction functions in which multiple low-level interventions can have differ- ent effects while mapping to the same high-level intervention (an assumption called the abstract invariance condition). In this paper, we introduce a new type of abstractions called projected ab- stractions that generalize existing definitions to accommodate lossy representations. We show how construct projected abstraction from the low-level model and how it translates equivalent observational, interventional, and counterfactual causal queries from low to high-level. Given that the true model is rarely available practice we prove new graphical criteria for identifying and estimating high-level causal queries from limited low-level data. Finally, experimentally show the effectiveness of projected abstraction models in high-dimensional image settings.
Causal inference is often studied under the semantics of structural causal models (SCMs) (Pearl, 2000). An SCM models reality with a collection of mechanisms and exoge- nous distributions. Each SCM induces a collection of distri- butions categorized into three successively more descriptive layers known as the Ladder of Causation or Pearl Causal Hi- erarchy (PCH) (Pearl & Mackenzie, 2018; Bareinboim et al., 2022). These three layers refer to the observational (C1), interventional (L2), and counterfactual (C3) distributions. In many causal inference tasks, the goal is to infer a quantity from higher layer using data from lower layers, problem known as cross-layer inference. It is understood that is generally impossible infer higher layer information with- out additional assumptions result known as the Causal Hicrarchy Theorem CHT (Bareinboim al., 2022)), so understanding the necessary assumptions for performing inferences is a key component of any causal inference task.
Existing works on causal abstractions have made significant progress in defining abstraction principles, proving insight- ful properties, and learning abstraction functions practice (Rubenstein et al., 2017; Beckers & Halpern 2019; Beckers et al., 2019; Geiger et al., 2023; Massidda et al., 2023; Zen- naro et al., 2023; Felekis et al., 2024). Causal abstractions are typically studied by comparing a high-level model MиH и- defined over high-level variables VH, with its low-level counterpart Mr, defined over VL. An abstraction function 7 maps from VL L to VH, and Mи is formally defined as abstraction of ML if it satisfies key properties with re- spect such commutativity with interventions. More recently, this notion has been relaxed to only enforcing prop- erties between distributions of Ma and Mr from the PCH (Xia & Bareinboim, 2024). For example, rather than saying MH is full abstraction of MM, one can say that M, is an abstraction of Mr specifically for interventional quantities in C2 or for single causal effect P( do(x)) E C2. Xia & Bareinboim (2024) also shows the synergy between causal abstraction theory and representation learning (Bengio et al., 2013), which has shown great success in many deep learning applications by mapping high-dimensional data like images or text to simpler representation spaces. These definitions of
1. Introduction
The ability to determine cause and effect, and the ability to interpret complex patterns into abstract concepts, are two integral components of human intelligence. From the causality perspective, causal reasoning is vital in planning courses of actions, determining blame and responsibility, and generalizing across changing environments. From the abstraction perspective, humans generally grasp better in- tuition when understanding something high-level. For
"CausalAI Lab, Columbia University. Correspondence to: Kevin Xia
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Causal Abstraction Inference under Lossy Representations
causal abstractions have accomplished formalizing a broad topic of human intelligence into mathematical language.
X <
L[x]]
H[r(x,)) 1]
One particular limitation of existing definitions of abstrac- tions is known as the Abstract Invariance Condition (AIC), which states, informally, that two values cannot be ab- stracted together they have different downstream impacts. This is illustrated in Fig. I. For example, a nutritionist may have collected data on two types of cholesterol, HDL and LDL, and are studying their impact on heart disease (Stein- berg. 2007; Truswell, 2010). They would like to abstract the two together by summing them as total cholesterol (TC). However, this violates the AIC, as it is known that HDL decreases rate of heart disease while LDL increases it, so the sum is ambiguous (a lossy representation). Nonethe- less, it may still be desirable have a consistent formalism in which these kinds of ambiguous abstractions are well- defined, since in many practical settings (where represen- tation learning or dimensionality reduction is needed), the AIC is clearly violated or is impossible to verify.
Mr L
*
H(xm]
X t X2
M L[x]
All Metsol
Figure 1: An illustration of AIC violations. On the low level, two different interventions may be performed (e.g.. X <- Xi and X < -x)). However, after applying the abstraction function T to obtain the high-level model, both interventions are mapped the same result (7(x) = -(x2) x1)). If Mr behaves differently under x1 compared to X2, Mı
cannot stay consistent with both models.
ened to P(x)) as the probability of X taking the values x under the distribution P(X).
We utilize the basic semantic framework of structural causal models (SCMs) (Pearl, 2000), following the presentation in Bareinboim et al. (2022).
In this paper, we study this extension of causal abstractions, which we later define as projected abstractions, referring to the idea that an abstraction that violates the AIC results in a loss of information that is then characterized in the exogenous space. The proposed formalism generalizes ab- stractions both on the SCM and on the PCH level to allow for mathematically consistent abstractions even with AIC violations. Projected abstractions have many uses in prac- tice, resulting in tractable causal inference and high-quality causal sampling even in the presence of extreme dimension- ality reduction, result which we show in the experiments.
Definition 1 (Structural Causal Model (SCM)). An SCM M is a 4-tuple (U, F. P(U)), where U is a set of ex- ogenous variables (or "latents") that are determined by fac- tors outside the model; V is a set {V1, V2...., Vnn of (en- dogenous) variables of interest that are determined by other variables in the model that is, in U V; F is set of functions (fvi fV,, Jv.. } such that each fv, is map- ping from (the respective domains of) Uv, U Pav; to Vi, where Uv. S U. Pay, S VV Vi, and the entire set F forms a mapping from U to V. That is, for i = 1, 1..... n, each fv. E F is such that v; < fv. (pay, uy;): ): and P(U) is probability function defined over the domain of U.
To summarize, in Sec. 2, we generalize abstractions to set- tings which the AIC does not hold and provide an algorithm for constructing the high-level model. In Sec. 3, we show how to perform causal inference from data within this class of abstractions when the true model is not observed. In See. 4, we empirically demonstrate the power of abstrac- tions at performing causal inference in high-dimensional image settings. All proofs can be found in App. A. Ap- pendices can be found in the full technical report, Xia & Bareinboim (2025).
Each M induces a causal diagram G, where every V/2 E V is a vertex, there is a directed arrow (V, - V,) for every Vi E V and V, E Pay, and there is dashed-biiirrctted arrow (V, a---- V,) for every pair V, V2 E V such that Uv, and Uv, are not independent (Markovianity is not assumed). Our treatment is constrained to recursive SCMs, which implies acyclic causal diagrams, with finite discrete domains over endogenous variables V.
1.1. Preliminaries
Counterfactual (and also interventional and observational) quantities can be computed from SCM M as follows:
We now introduce the notation and definitions used through- out the paper. We use uppercase letters (X) to denote random variables and lowercase letters (x) to denote cor- responding values. Similarly, bold uppercase (X) and lowercase (x) letters denote sets of random variables and values respectively. We use Dx to denote the domain of X and Dx Dx, ... x Dx, for the domain of We denote P(X = x) (often short-
Definition (Layer 3 Valuation (Bareinboim et al., 2022, Def. 7)). An SCM M induces layer La(M), a set of distributions over V, each with the form P(Y.) = P(Y Y2/xal...) such that
p. (y 32(xa))))))
(I)
X =
f 1 [[1]ss](u) =y: (u) = =y2.... .2....].. dP(u)
'See App. C Ex. 7 for more concrete explanation.
2

---

Causal Abstraction Inference under Lossy Representations
where Y (xxx) (u) is evaluated under Fx; == (vv V, E V x U {fx x: E X2). Le is the subset of L3 for which all x; are equal, and Li is the subset for which all X; = 0.
3. 7 is composed of subfunctions TC, for each C, E C such that V = 7(v2) (Tc, (ct) : C2 E C) where
TC, (c,) VHr if and only if E De. We also apply the same notation for any WL S VL such that W
is a union of clusters in C (i.e. 7(WL) = (cc)
Each Y, corresponds to a set of variables in world where the original mechanisms fx are replaced with constants x; for each X E X;; this is also known as the mutilation procedure. This procedure corresponds to interventions, and we use subscripts to denote the intervening variables (e.g. Xx) or subscripts with brackets when the variables are indexed (e.g. Y I)x))). For instance, P(yz: V/a)) is the probability of the joint counterfactual event Y = y had X been z and Y = y had X been x².
C, E C, E WL).
Finally, we state the AIC formally below.
Definition 5 (Abstract Invariance Condition (AIC)), Let P(UL)) be an SCM and т
M₁ = (U,, constructive abstraction function rel- Dv, + Dv be a ative to C and ID. The SCM Mr. is said to satisfy the abstract invariance condition (AIC, for short) with respect to if, for all v1, V2 E Dv, such that T(V1) =(22), Vu E Duz C, E C, the following holds:
We use the notation (MM to denote the set of Li dis- tributions from M. We use Z to denote a set of quanti- ties from Layer 2 (i.e.
and Z(M) denotes those same quantities induced by SCM M (i.e. Z(M) (V..(V
(P((a.)))
TC. (lthipaa..yy:Vee
uy):
(2)
= TC. (
The theory of causal abstractions developed in this paper build on the foundations of constructive abstraction func- tions, under which individual distributions of the PCH are well-defined between low and high-level models.
where pay and p are the values corresponding to Vi and V2.
Definition 3 (Inter/Intravariable Clusterings (Xia & Barein- boim, 2024, Def. 5)). Let M be an SCM over V.
A table summarizing the notation can be found in App. A. detailed explanations of these definitions can be found in App. A.2, and additional useful definitions from prior work can be found in App. A.3.
1. A set C is said to be an intervariable clustering of V if C = {C1, C2... ) is a partition of a subset of V. C is further considered admissible w.r.t. M if for any
C, e C and any V E C, no descendent of V outside of C2 is an ancestor of any variable in C, That is, there exists a topological ordering of the clusters of C relative the functions of M.
2. Abstractions under AIC Violations
The abstract invariance condition (AIC) states, in words, that two low-level values cannot map to the same high-level value if they have different downstream effects. This is a critical property that must hold for existing definitions of abstractions to be well-defined. In this paper, will use the following running example to illustrate the key points.
2. A set ID is said to be an intravariable clustering of variables V w.r.t. C if ID- = (De. C, E C), where Dc.- = (D), Dē..... D is a partition (of size m²) of the domains of the variables in C,, Dc. (recall that Dc, is the Cartesian product Dy, x Dv, X X Dv, for C [V1, V2,... Vi 1. so elements of De take the form of tuples of the value settings of Ct).
Example 1. For concreteness, consider a setting in which different insurance companies (2) offer various insurance plans (X), which affect whether an insurance claim is approved (Y). For simplicity, suppose there are two in- surance companies (2) and 22) that offer three insurance plans (r1, 22, and æa), and the claim is either approved (Y 1) or not approved (Y = 0). Suppose the true model M = =M, (U,. VL, Fr, P(U,.)) is described as
Definition& 4 (Constructive Abstraction Function (Xia & Bareinboim, 2024, Def. 6)). A function : Dv, + DV is said to be a constructive abstraction function w.r.t. in- ter/intravariable clusters C and D iff
1. There exists bijective mapping between Va and C such that each Vit E VI ir corresponds to C, E C;
U, = (U2, UZ. UR U U U
Vz = (2,X,Y}
2. For each VH,, E V H, there exists bijective mapping between Dvn. and Dc, such that each uHi E DVn.. corresponds to DC E Mec and
fb(u2)
uz
FL
= uix
(3)
(1, u
= uf
3

---

Causal Abstraction Inference under Lossy Representations
P(Uz = 21) 0.5 )(UX)= = (-1 - 0044;
(b)
Z
X
Y
Z
X
Y
P(UL)= P(US- 1) = 0.9, P =1)=
)(U =(= + -0.1==0,;;22-- -1:1;,0---04
X
P(U = 1) = 0.9
X"
The interpretation of the model is as follows: Insurance plans Ti and 23 are very effective, with 0.9 probability of claim acceptance, while 22 is very ineffective at only 0.1 probability. Insurance company 21 is more reputable than 22 and is more likely to offer plan 21 over Ø2, while company 22 prefers to offer plan 222 over 211-
X
Y
Z
X
Y
Z
Y
Z
X
Y
Suppose an important factor of consideration not shown in the model is that 21 and I2 are cheaper insurance plans, while va is more expensive. A data scientist who is studying this model may choose to abstract the different plans away, categorizing them simply as "cheap" and "expensive" plans. Formally, they would study a set of higher-level variables
Figure 2: Comparison between (a) full SCM projections and (b) partial SCM projections. When X is fully projected away, its function is subsumed by its child's function fr. When X is partially projected, it split into observed por- tion X and unobserved portion X". The role of X" is preserved, while X is subsumed into the function fr.
VH = (Z XH, YH). where Zμ = Z, Yn Y, and X" has domain Dx, {Ic, σE} corresponding to cheap and expensive plans respectively. There exists an abstraction function 7 Dv + Dv such that T maps 2 and ø2 to sc (cheap) and maps 2y to TE (expensive). We will some- times use the notation Xr, to describe X to disambiguate from XH. and we will use the notation Z and Y instead of ZH and Yh since the variables are the same on both levels.
Even when the AIC does not hold, it does not necessarily mean that these lossy transformations should not be used. Representation learning and dimensionality reduction are often performed to improve tractability or interpretability at the cost of some lost information. Hence, it would still be desirable to perform causal inferences in the high-level space even under AIC violations. To address the issue of different low-level quantities matching the same high-level quantity, one can reinterpret the high-level quantity as dis- tribution over its corresponding low-level quantities, where the randomness in the distribution results from the lost infor- mation from the abstraction (i.e., a hard intervention on the high-level translates to a soft intervention on the low-level).
This immediately brings the AIC into question. If the data scientist is interested in the causal effect of cheap plans on claim acceptance (i.e., P(Yxg=ze = 1)), whether xc refers to I1 or I2 is ambiguous. To witness, note that
P(Yx.=z = 1) = 0.9
(4)
P(Yx,=2 =2 1) = 0.1.
(5)
Since T(x1) = xx2))= = Ic. but P(Y.,) * P(Y..), the AIC is clearly violated, leaving the intervention on se am- biguous.
2.1. Projected Abstractions
The discussion on relaxing the AIC begins with the concept of SCM projections (Lee & Bareinboim, 2019), which can be viewed as a primitive form of abstraction. An SCM M projected
Fundamentally, the issue with AIC violations is clear: for- mal definitions of abstractions expect equality between low-level and corresponding high-level quantities, but it is not well-defined when one high-level quantity corresponds to multiple differing low-level quantities. In practice, the AIC can be a difficult restriction. Generally, is assumed to be true whenever abstractions are applied, but it is difficult to verify given that the true SCM and functions are rarely available in real-world settings. The assumption also likely to be incorrect when applying abstractions naïvely, for example, by performing representation learning or di- mensionality reduction without taking the AIC into account. By definition, dimensionality reduction is a lossy transfor- mation of the original data, and the AIC is violated if any of the lost information is relevant for downstream functions.
subset of variables W C V is function-
ally identical SCM defined over W, where the functions of V \ App. A Def. 4 for the full definition and App. C Ex. 8 for an example). In the context of constructive abstraction func- tions, the act of projecting away a variable can be viewed as excluding the variable from all intervariable clusters. This brings the first major insight in addressing AIC violations. In general, when reducing the granularity of a variable, some parts of the variable deemed less important are abstracted away while others are retained. While by definition, SCM projections only allow for entire variables to be included or excluded, one could conceive of SCM projections in which variables are only partially projected away (see App. C Ex.
W are subsumed by other downstream functions (see
4

---

Causal Abstraction Inference under Lossy Representations
for example). Formally, partial SCM projections can be defined as follows.
Observe the intuition from constructing these functions from the perspective of projected abstractions. f& behaves iden- tically to ft except the output remaps the value of Xr, to the corresponding X (i.e. f% = =(ff))) )). However, due to the AIC violation, / is unable to disambiguate between 21 and T22 if X3 = Ic. The solution is to introduce a new exogenous variable X which represents information in Xg that is not captured in X and disambiguates between 21 and 22- f then uses both X3 and X to mimic the behavior of Xr, It is clear that Xr. can be constructed as 8(X nI Xx).. defined as
Proposition 1 (Partial SCM Projection). Let V be set of variables and W C V be subset. For each W; E W, let 6, Dwy x Dw + Dw, be a surjective function mapping new variables Wi and W.' to W;. WP and W are called the observed and unobserved projections of W respectively. Denote 8(W", W") = W. where W {W. : W; E W] and W = {W? : W E W). For any SCM M = (U.V F. P(U),, there exists an SCM
M = (U' = UUW V = W F(U'), such that,
for all u E Du. x W. and x E Dx.
IH IC, X
W = MMM (u, x", 2"),
M
(6)
8(0H x²)) =
T2 TH = IC, Xi = T2
(10)
where 5( 6( wT) (u),,, 8(x², x, Z" V X", and x are the corresponding values from w M' is called a partial SCM projection of M over W
33 FH
FE
which matches Eq. 9. Indeed, MH = (Un = U, U {X#], VH. = 1 fHl eH J, P(UH)) is a partial SCM projection (and also projected abstraction) of Mr over VH. The graph corresponding to Mr is clearly the top graph of Fig. 2(b), but note that through Eq. 8, there now a dependence from Z to Y. so the graph for MH и is instead the bottom graph of Fig. 2(b).
In words, a partial SCM projection of M over W is essen- tially a smaller version of M defined only on the variables of W C V, where each W; E W is only partially repre- sented in the projection. A function ō splits W;'s domain into its observed (W9) and unobserved (W?") portions. Eq. 6 ensures that any value of W obtained from intervention on the original SCM Mx will match the corresponding out- put from M', when the observed portion of the intervention x" is applied to MM, while the unobserved portions of x" and w" are passed as unobserved arguments to the func- tions. A comparison between regular SCM projections and partial SCM projections is shown in Fig. 2. The definition of projected abstractions follow.
It is easy to see that Eq. 6 holds in this example. For instance, fix Uz = 21, U = I2, U = 1. Clearly, evaluating Mr with these values results in Z = 21. X = 22, Y = 1. Note that 22 = 5(zc, x2), and this is the only set of values of XH, X that map to T2. Indeed, on the high level, with Uz 21, U = 22, U2 XI = 22, it must also be the case that Z = z1, XH = zc. =.
Definition 6 (Projected Abstraction). An SCM Mи is projected abstraction of Mr if and only if it is a partial SCM projection of a r-abstraction (Beckers & Halpern, 2019, Def. 3.13) (also Def. 14 in App. A) of Mr.
Projected abstractions make an important step to working around the AIC as Eq. 6 allows for quantities to be well- defined between low and high-level variables by simply obtaining partial projection of the original SCM M, over the high-level variables Vy. However, unlike full SCM projections, partial SCM projections are not unique in terms of the induced PCH distributions. Prop. guarantees its existence but is underspecified in a couple of ways. First, P(U') is not fully defined, and it is not clear how W should be sampled (e.g., it is not clear how Eq. 8 is chosen in Ex 2). Second, Eq. 6 does not specify what behavior M' should follow when z" does not match w (e.g.. How should Y depend on X% in Ex. 2 if X = Tg?).
To provide intuition for projected abstractions, consider the following example.
Example 2. Continuing Example 1, given the setup of Eq. 3, suppose X E (F is given the function
1 (,,,,
ac
u E (2)
(7)
F
ux = 53
and define Xi-- [2) as a random variable with distri- bution
The specific choice of partial SCM projection that best serves as an ubstraction can be determined by understand- ing how low-level interventions relate to high-level inter- ventions. In other words, given high-level intervention Xi <- XH,, it is important to define the corresponding low- level soft-intervention ox,, which is a distribution over all possible interventions x,, that map to X The consequence of the underspecification of partial SCM projections is that there are many possible choices of defining xx,- For full
- = = P(X, = x; Xr. E {a1, 2)).
(8)
Suppose now Y is now given a high-level function
uy IH = rc, Xy 21
$ (MH u mg) =
up IH Ic, 1/1 22 (9)
up
IH = FE
5

---

Causal Abstraction Inference under Lossy Representations
discussion how xx, should be decided, see App. B. A useful general form of xx, is defined as follows. Split xx, into individual soft interventions oc; for each intervariable cluster C C Xy. Then define each dc. as
Algorithm 1 Constructing Mr from Mr.
input Mr (Uz, V2, Ft, P(Uz)), constructive abstraction
function T from clusters C and D
1: Us + U2, P(Ux) + P(UL)
2: Vи <- C, Dv,, f B
Plac. = c.) P(c, r(c.) UHH paya a
3: for W E V. do
(11)
4: W W < projjct((W) [construct 5 from Prop. 1]
5: Un + U UU
In words, high-level intervention should be equivalent to distribution over the corresponding low-level interventions that assigns probability to each possible intervention based on their prior probabilities given their parents. 2
6: end for
7: for C, E C (and conrespondigg V2 E VR) do
8: P(ő(e), CT) = c; U2) < P(C, = a T(c.) V1 P,,pay, u n (from Eq. 11)
9: 1.r + T +TV (ó(pa,, (appaaapaa.......
Example 3. Continuing Example I, suppose the data scien- tist is interested in the causal effect of choosing a cheap insurance plan on claim approval. In other words, she would like to study the intervention Xμ Zc, which is ambiguous on the low-level as it could refer to either X + 21 or XL t Z2. More specifically, according to Eq. 11, XH + IC corresponds to a soft intervention oxe on the low level, defined as
10: end for
II: Fu < ( C.E
12: returnMa (UH, Vn, FH, P(Un))
where each Y [ corresponds to the potential out- comes of the variables YL, under the (possibly soft) in- tervention æx,. over X1.55. Each Yr.;; and X must be unions of clusters from C (i.e. Y Ucer C for some C' C) such that r(Y,,)) and r(X,,s) are well-defined (i.e. r(,),) (Acec' Tc(C)))CC))))))))) For the high-level coun- terpart, denote
21 w.p. P(a1 Xx E (21,
Ox,- =
(12)
22 w.p. P(x2 X2 E (21,
While there are many ways to disambiguate whether ze is referring to 1 or z2, this choice of xx,, will assign proba- bilities based on the prior probabilities of Xr. being one of T1 or Z2. Moreover, the probabilities change depending on the value of z. This makes intuitive sense, since under the intervention X" +- TC. we expect that if Z 21, then Xr. is more likely to be æ1 than 22, or vice- versa when Z = 22- From a query perspective, this implies that
Hr T(Y = (Y (YH.()ma)) (m .. (16)
such that YH = T(YL,), and XH.s = ((L,,) for all i. For any value yH,, E 'DY.... denote
Dy....yy, Ly : Vr,, E DY 7 )
(17)
that is, the set of all values yL,- such that r(yL,.) = -HH
P(Yxw=Re 1 7 =11
(13)
For any high-level query
= P(Yox, (x0,2) = 1 21)
T(Q) P(Y H,- VH,)),
(18)
P(ra PPas Xx (x1, 21) P(), =1 0.74
of the form of Eq. 16, its lewel counterpart is
Likewise, P(YX==Re = 1| 7 = =22)= =
22)==2000 -0.26
(14)
Q = Σ
P(Y = =LL,),
(19)
EL (yn..)
While projected abstractions are defined over the entire SCM, the mapping between low and high-level interven- tions are more clear at the query-level (i.e., individual inter- ventional and counterfactual distributions of interest). Such quantities can be defined as follows.
of the form of Eq. 15.
This query definition connects the distributions of La(MH) to corresponding distributions of L2(M).) Compared to earlier definitions. Eq. 15 has been generalized to ac- count for soft interventions in addition to hard interven- tions. Under constructive abstractions functions т, a no- tion of Q-T consistency was established for certain queries Q L3(M2) (App. A Def. 17), which still apply under this generalized definition. In short, given low level query Q (Bq. 19) and its high-level counterpart 7(Q) (Eq. 18), M is said to be Q-T consistent with Mr if QM,, = MM)) One can then say that MH is an abstraction of Mr specif- ically for the query Q, even if Mи may not be Q'-t con- sistent with Mr for other query choices Q'. If M is Q-7
Definition (Generalized Query). Denote YL,. as a set of counterfactual variables over Vt. That is,
YL,, = (X (Yr.) 1.....x..........
(15)
'Here, uZ can informally he thought of as the confounded exogenous parents of VH.a. The full definition is somewhat in- volved, and the subtleties are discussed in App. B.2. Due to space constraints, the main body provides intuition in Markovian settings, where unohserved confounding not present.
6

---

Causal Abstraction Inference under Lossy Representations
consistent with My for all r(Q) E (.M then Ma is said to be Li-T consistent with ML.
g
g C
C
With xx,,, defined in Eq. 11, one can then algorithmically construct a projected abstraction consistent in all queries. Given MM and a constructive abstraction function 7 (which may not satisfy the AIC), Alg. I can be used to construct the high-level abstraction Mr . In line each W VL is split into its observed and unobserved counterparts W and W Line 8 assigns each W a distribution based on Eq. 11. Line 9 builds the high-level function using the low-level function with inputs reconstructed using ó. Finally, the full high-level model Mr is assembled and returned in line 10. Under these inputs, Alg. constructs a projected abstraction Mи that is Q-t consistent with Mr for all possible high-level C3 queries, as shown by the following result.
Z
X
Y
Z
X
Y
(b)
Z
X
Y
Z
X
Y
(c)
Z
X
Y
Z
X
Y
Figure 3: Examples of C-DAGs (left) and their correspond- ing projected C-DAGs (right), with AIC violation variables V/ H outlined in red.
Theorem The SCM Mи constructed by Alg. I is a projected abstraction of MM that is Q-T consistent with for all r(Q) E (M
introduce new dependencies between SCM functions, there- fore implying new edges in the graph. Ex. 3 explains this dependence numerically. Since no variables are clustered together in the example, both the original causal diagram and the C-DAG Gc are represented by the top graph in Fig. 3. However, this graph implies that P(Y.,, 2) = P(Y.n) ). Evidently, this is not true since Eq. 13 is not equal to Eq. 14. As hinted by the construction in Alg. 1, the high-level func- tion 1 requires some additional information from Z to decide between interpreting IC as I1 or Ø2. This informa- tion adds a dependence from Z to the function of f& which requires adding directed edge from Z Y.
As an example, can be verified that running Alg. I on My in Ex. 1 results in the SCM Mи from Ex. 2.
3. Projected Abstraction Inference
Alg. 1 finds an abstraction model Mи that is consistent with its low-level counterpart Mr for all queries, but it requires the full specification of MM. In practice, ML typically represents the true model of reality and will not be observed. Inferences of Le and L3 queries must be made through limited available data, usually observational (C1).
While the original C-DAG construction is not valid for pro- jected abstraction inferences, one can use a modified version that adds the new required dependencies into the C-DAG.
The Causal Hierarchy Theorem (Bareinboim et al., 2022, Thm. I) states that cross-layer inference, or inferring higher layer quantities (e.g., L2, La) from lower layer data (e.g., C1), is generally impossible without additional assumptions. Many such assumptions take the form of graphical model, such as causal diagram (Pearl, 1995), which imply con- straints between causal distributions from causal (Barein- boim et al., 2022) and counterfactual Bayesian networks (Correa & Bareinboim, 2024). In the context of abstractions, when 7 is a constructive abstraction function that satisfies the AIC, it has been shown that one can avoid assuming the entire causal diagram of the low-level model in favor of cluster causal diagram (C-DAG) (Anand et al., 2023) w.r.t. the intervariable clusters C. Unfortunately, this graphical model is insufficient for the case when the AIC is violated.
Definition 8 (Partially Projected C-DAG), Let T Dvz + Dvv be constructive abstraction function w.r.t. intervari- able clusters C and intravariable clusters D. Let 9c (VH, Eς) be C-DAG (with nodes VH and edges Ec), of graph w.r.t. C. Let V) I S VH be the set of AIC violation variables (App. Def. 19). Then, construct gf (Vn, EE)) follows. Start by setting E <- Ec.
Then apply the following rules for all X E V
(1) If Z + x + Y in Eg, then add Z + Y into E!.
(2) If Z +-+ X + Y in Ec, then add Z + + Y and
x > Y into Et.
(f) Z t X + Y in Ec, then add Z < Y into E. Repeat iteratively to accommodate new edges.? 3 g. is called the partially projected C-DAG of G w.r.t. C and Vt H.
Proposition 2 (C-DAG Insufficiency (Informal)). For con- structive abstraction function т over intervariable clusters C in which the AIC does not hold, the C-DAG GG implies constraints that may be unsound.
The steps correspond to the intuition discussed when performing a partial projection, parts of the variables in V1
To witness why this is the case, Fig. 2(b) shows the issue clearly. Attempting an abstraction in violation of the AIC akin to performing a partial SCM projection, which may
PProceuure can be applied algorithmically in one pass by ap- plying rules for each node Vt. in topological order.
7

---

Causal Abstraction Inference under Lossy Representations
are projected into the exogenous space, resulting in addi- tional dependences that require additional edge connections. Examples of C-DAGs and their corresponding projected C-DAGs are shown in Fig. 3. In the figure, rows (a), (b), and (c) correspond to examples of steps 1, 2, 3 respectively. It turns out that this new definition is precisely what is needed for abstraction inference in the absence of the AIC.
have different effects on Y. Hence, V = (X#): ). and the C-DAG Ge and projected C-DAG g are the two graphs in Fig. 3(a). To answer the query of interest P(Yx, mac m== = 1), one can apply Thm. 3 to simply identify the quantity w.r.t. P(VH) and G In this case, note that the causal effect of H on Y can be computed via backdoor adjustment on 2, so P(Yxn=re = 1) is equal to
Theorem 2 (Projected C-DAG Sufficiency and Necessity (Informal)). Let Mr. be an SCM over variables VI. T Dv= + , be constructive abstraction function w.r.t. clusters C and ID, and V be the AIC violation set. The partially projected C-DAG g w.r.t. C and V'd completely describes all constraints over H.
Σ P(Y = =1 XH = Ic, 2 = = = =2)
(20)
P(Y = 1 Xr. E { 22)
(21)
= (0.7) + (0.3) = 0.596.
(22)
In other words, the projected C-DAG provides exactly the constraints necessary solve the task of performing causal inferences across abstractions, even when the AIC is vio- lated. In particular, certain interventional and counterfac- tual distributions may be inferrable from a combination of the projected C-DAG gf and the available datasets from Mr. Determining precisely which queries can be inferred known as the identification problem, which is defined below in the context of abstract identification.
Thm. implies that, in practice, r-ID can be checked by performing any classical ID procedure on the high-level space. This may include algorithmic approaches or other optimization-based approaches.
4. Experiments
We perform two experiments to demonstrate the bene- fits of projected abstractions. The models in the ex- periments leverage Neural Causal Models (NCMs) (Xia et al., 2021; 2023), specifically the generative adversar- ial implementation called GAN-NCMs. Details of the experiment setup can be found in App- D, and code can be found at https: //gttbb//////b///// wtm Caom/CaasllAlaa Pro wweejeeeeaasllllaetaaacaacaaaaaa
Definition 9 (Abstract Identification (General)). Let 7 DV + D be constructive abstraction function. Con- sider projected C-DAG g and let Z- be collection of available interventional (or observational if Z5 0) distributions over Vr.. Let S2, and Rn be the space of SCMs defined over Vz and V H. respectively, and let 22 ( and 2t (9 be their corresponding subsets that induce G A query Q is said to be r-ID from g and Z iff for every Mr E (G)) MH E QH such that Mr is Z-7 consistent with ML. MH is also Q-T consistent with ML.
In the first experiment, we test the necessity of the projected C-DAGs when the AIC does not hold. The high-level query r(Q) = P(yz =) is estimated in the graph setting shown in Fig. 3(a), where Z is a digit from 0 to 9, X is corre- sponding colored MNIST image, and Y is a label denoting the color prediction of X. 7(X) maps the image to a binary variable representing the shade (light or dark) of X.
In words, query Q is considered 1-ID if, for any pair of models Mr and Mи such that both are compatible with of and Z. they also match in Q. In contrast, Q is not 7-DD if there exist Mr and Mи that are compatible with both G and Z but disagree on Q (i.e., Q.Mi. & ((Q).Mn). Abstract identification may seem like difficult property to check, but it turns out that there is a natural connection with the classical identification problem, as shown below.
The results are shown in Fig. Three different GAN-NCMs are trained: one directly on the low-level data that does not use abstractions (red), abstracted one constrained by the C-DAG (yellow), and an abstracted one constrained by the projected C-DAG (blue). 95% confidence intervals of the errors are plotted in the figure. Note that the abstractionless model and the projected C-DAG model have decreasing error with more samples, but the regular C-DAG model is unable to learn the correct query. The abstractionless model has higher error than the projected C-DAG model since it operates in higher-dimenssonal space.
Theorem 3 (Dual Abstract ID (General)). Consider a coun- terfactual query Q over L. a constructive abstraction function I w.r.t. clusters C and ID, projected C-DAG G), and data Z from VL. Q is T-ID from G. and Z if and only (Q) is ID from G. and rZ))
In words, 7-identification across abstractions is equivalent to classic identification on the high-level space.
In the second experiment, we test an interesting consequence of the projected abstraction theory: the soft intervention def- inition in Eq. 11 can be directly modeled and sampled if attempting to reconstruct the low-level data. We call this
Example 4. Continuing Ex. 1, note that X4H is the only AIC violator in V H. since I1 and x2 both map to De but
8

---

Causal Abstraction Inference under Lossy Representations
Image Samples
P(I
D
=0
D(D=
p
D = 5)
Non-Causal
0 3 3
5 5
O 2
9 7
8 2 Q 4 0 0 0 0 3 0 0
4
3 - 4 8
7 70 0 Ø 8 5 3 9 6 0 2 3
5 2 1 0 9 D
0 2
9
3 2 8
5
0
9 0
0 4 0 0
0 3
5
8 5
6
3
RNCM (R'))
T
3
0 O 0
0
9 3
2
O 0 9
0
4 9 8
0
0
5 0 8
RNCM ()
g g
e g
8 9 8 B
8 9 8 9 8
8 8
9 8
g g
8
g
8
8 9
9 6
8
8
8 8 8
8
9
8
8
8 8
8 8 5 9 6 3 3
88
9
8
RNCM+ Projected
O 0 7 L
{
2 0
R
4 3 S O
&
Sampling ((0.1))
6
6 0
7
7 2
0
9 9
6
Ground Truth
1 0
6
2
6
2 2
0 0 0
O
0 0 0
0
0
5 0
0 0 0
O
0
O
9 5
/
8
2
9
0 4
0 0
0
0
Figure 4: Colored MNIST results. Samples from different causal queries (top) are collected from competing approachess (left). The expressions in parentheses are the representation sizes. The left column shows direct image samples from each of the models, while the second, third, and fourth columns show samples generated from an L1, L2, and L3 query, respectively.
0 1 3 5 9
g-NCM
D
MAE
10-1
3 NCCM
Figure 5: (Left) Graph of Colored MNIST experiment.
(Right) Correlation shown between color and digit.
8 NNCM
10
10a
10 10a
n
approach projected sampling and explain it in more detail in App. B.3. We show this in the causal colored MNIST exper- iment (Xia & Bareinboim, 2024). In the model, digit D and color C' both cause the image I, but they are confounded (c.g., O's are red, 5's are cyan, see Fig. 5). Three differ- ent queries are tested (the right three columns of Fig. 4). P(I D 0) is an Li query representing images condi- tioned on digit 0, resulting in red O's. P(ID=o) is an C2 query representing images with the digit intervened as O, cutting the confounding and resulting in O's of all colors. P(T D= 5) is an L3 query representing images with digit intervened as O, conditioned on the digit originally being This results in O's with colors of images that were originally 5's, resulting in eyan O's.
Figure 6: Mean absolute error (MAE) v. number of samples for the MNIST estimation task. Comparisons between an abstractionless approach (red), C-DAG approach (yellow), and a projected C-DAG approach (blue).
5. Conclusion
This paper introduced projected abstractions (Def. 6), which can be constructed algorithmically (Alg. 1, Thm. 1), to over- come the AIC limitation. When the full model was not avail- able, we leveraged new graphical model (Def. 8, Thm. 2) that allowed for causal inferences through the abstract-ID problem (Def. 9, Thm. 3). Finally, we demonstrated the ability of projected abstractions to leverage representation learning within difficult causal inference settings through high-dimensional image experiments.
Four methods are compared these queries in Fig. 4, with the ground truth shown on row 5. The non-causal approach (row 1) simply directly models the conditional distribution between digit and image and therefore fails to model any- thing higher than C1. The representational NCM or RNCM (Xia & Bareinboim, 2024) (row 2) is able to decently repro- duce all queries, but uses 16-dimensional representation space, which cannot shrink much further due to AIC limi- tations. When forced to take a binary representation (row 3), the RNCM clearly lacks the representation power to properly generate images. In contrast, using a projected sampling approach (row 4) can reproduce the images even with representation size as small binary digit.
Impact Statement
This paper presents work whose goal is to advance the field of causal inference, subfield of machine learning. The re- sults in this paper may have implications bringing together strong practical results in representation learning and com- puter vision research with the explainability and generaliz- ability of causal inference results. The trend is that this will lead to smarter AI, which itself has many consequences out

---

Causal Abstraction Inference under Lossy Representations
of the scope of this work, but the benefit of understanding causal inference is that it can lead to less bias and more
Massidda, R., Geiger, A., Icard, T., and Bacciu, D. Causal abstraction with soft interventions. In van der Schaar, M., Zhang, C., and Janzing, D. (eds.), Pro- ceedings of the Second Conference on Causal Learn- ing and Reasoning, volume 213 of Proceedings of Ma- chine Learning Research, pp. 68-87, PMLR, 11-14 Apr 2023. URL httpl:psss/ppcceedinnge mlr prrsss/
accountability of Al models.
Acknowledgements
This research is supported in part by the NSE, ONR, AFOSR, DoE, Amazon, JP Morgan, and The Alfred Sloan Foun- dation.
html.
Pearl, J. Causal diagrams for empirical research. Biometrika, 82(4)6669-688 1995.
References
Anand, T. V., Ribeiro, A. H., Tian, J., and Bareinboim, E. Causal effect identification in cluster dags. In Proceed- ings of the 37th AAAI Conference on Artificial Intelli- gence. AAAI Press, 2023.
Pearl, J. Causality: Models, Reasoning, and Inference. Cambridge University Press, New York, NY, USA, 2nd edition, 2000.
Pearl, J. and Mackenzie, D. The Book of Why. Basic Books, New York, 2018.
Bareinboim, E., Correa, J. D., lbeling. D., and Icard, T. On pearl's hierarchy and the foundations of causal inference. In Probabilistic and Causal Inference: The Works of Judea Pearl, pp. 507-556. Association for Computing Machinery, New York, NY, USA, 1st edition, 2022.
Rubenstein, P. K., Weichwald, S., Bongers, S., Mooij, J., Janzing, D., Gtrosse-Wentree M., and Schölkopf, B. Causal Consistency of Structural Equation Models. In Proceedings of the Thirty-Third Conference on Uncer- tainty in Artificial Intelligence, 2017.
Beckers, S. and Halpern, Y. Abstracting causal mod- els. In Proceedings of the Thirty-Thirl AAAI Confer- ence Artificial Intelligence and Thirty-First Innova- tive Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, MAN19/AATF11/AATT AAAI Press, 2019. ISBN 978-1-57735-809-11 doi: 10. 10.1609/aaai.vvo /t.p:/aaai.........3331000101000 URL https: /doi. org 10. 609 aaai v33101 33012678.
Steinberg, D. Copyright. In The Cholesterol Wars.
Academic Press, Oxford, 2007. ISBN 978- 0-12-373979-7. B977-0-12-3333779-5500033 arc.oleepp1//9919127779993333339
doi: https://oo..or//10
URL https:
Truswell, A. Cholesterol and Beyond: The Research on Diet
Beckers, S., Eberhardt, F., and Halpern, J. Y. Approximate causal abstraction. In Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence. 2019.
and Coronury Heart Disease 1900-2000. 01 2010. ISBN 978-90-481-88744 doi:
Bengio, Y., Courville, A., and Vincent, P. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1798-18) aug 2013.
Xia, K. and Bareinboim, E. Neural causal abstractions. In Proceedings of the 38th AAAI Conference on Artificial Intelligence. AAAI Press, 2024.
ISSN 0162-8828. doi: 10. 20.11099/AAMM.203....... URL https: http:/oo..oorr/
Xia, K. and Bareinboim, E. Causal Abstraction Inference under Lossy Representations. Technical Report Techni- cal Report R-124, Columbia University, Department of Computer Science, New York, 2025.
1109 /TPAMI. 2013. 50.
Correa, J. and Bareinboim, E. Counterfactual graphical mod- cls: Constraints and inference. Technical Report R-115, Causal Artificial Intelligence Lab, Columbia University, August 2024.
Xia, K., Lee, K.-Z., Bengio, Y., and Bareinboim, E. The causal-neural connection: Expressiveness, learnability, and inference. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 10823-10836. Curran Associates, Inc., 2021.
Felekis, Y., Zennaro, F. M., Branchini, N., and Damoulas, T. Causal optimal transport of abstractions. In Conference on Causal Learning and Reasoning, CLeaR 2024, 2024.
Geiger, A., Potts, C., and Icard, T. Causal abstraction for faithful model interpretation, 2023.
Xia, K., Pan, Y., and Barcinboim, E. Neural causal models for counterfactual identification and estimation. In Pro- ceedings of the 1 11th International Conference on Learn- ing Representations (ICLR-23). 2023.
Lee, S. and Bareinboim, E. Structural Causal Bandits with Non- monimuaipee Variables. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019.
10

---

Causal Abstraction Inference under Lossy Representations
Zennaro, F. M., Drávucz, M., Apachitei, G., Widanage, W. D., and Damoulas, T. Jointly learning consistent causal abstractions over multiple interventional distributions. In van der Schaar, M., Zhang, C., and Janzing, D. (eds.), Conference on Causal Learning and Reasoning, CleaR 2023, 11-14 April 2023, Amazon Development Center, Tübingen, Germany, April 11-14, 2023, volume 213 of Proceedings of Machine Learning Research, pp. 821 PMLR, 2023. URL https: proceedings mlr. 20sss//213/zeenarro/2 html.


Reviews:
[{"review_id": "Xc9B38PfZ4", "reviewer": "Reviewer_8K54", "summary": "Existing causal abstraction frameworks often struggle with lossy abstraction functions, where different low-level interventions produce distinct effects but map to the same high-level intervention. To address this, the authors propose projected abstractions, a new framework that extends previous definitions to handle lossy representations more effectively. The construction of projected abstraction and inference is shown.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful questions and positive review. We appreciate the multiple suggestions and will address them as follows:\n\n> L110-118 could be put into Def 2, as I got confused by undefined notations when reading Def 2.\n\n**Response:** We will adjust Def. 2 to accommodate this and make it clearer; thanks! \n\n> Def 5: $\\widetilde{\\mathbf{pa}}_V$ is not used.\n\n**Response:** Thanks for pointing this out. It was originally used in settings in which the AIC was not violated, but we do not consider such cases in the main body, so we will omit this from the definition to improve legibility. \n\n>Def 6: Definition of $\\tau$-abstraction is missing in the main text (please move or point to Def 10 in Appendix A).\n\n**Response:** We will include this as suggested.\n\n>The paper is notational heavy [...]\n\n**Response:** Thank you for the feedback. We do assume some background of the previous literature, but indeed, our efforts to fit all of our content into the page requirement have unfortunately led to the paper being a bit dense. To remedy this problem, we have included Appendix B for further discussion and Appendix C for clarifying examples. If accepted, we will use the extra page to clarify each definition and make the theory easier to follow. We will also add a table in the appendix to clarify all of the notation we use throughout the paper. \n\nFinally, to answer your questions:\n\n> 1. The details of the theoretical contribution are unclear. It seems like projection has been proposed in Lee & Bareinboim, 2019 [...]\n\n**Answer:** Yes, SCM projections were proposed in Lee & Bareinboim (2019). We draw inspiration from this result to generalize to partial SCM projections (one of our contributions). In contrast to Lee & Bareinboim (2019), which solves a causal RL problem to find optimal intervention sets in a causal bandits setting, we work in an entirely different space, applying projections to define a causal abstraction formalism. Def. 6 and Thm. 1 (both contributions of our paper) establish this intriguing duality between partial SCM projections and causal abstractions, which is surprising and may otherwise seem unrelated. The idea that a lossy abstraction that violates the AIC can be thought of as partially projecting away the abstracted variables is the main insight that allows us to generalize previous abstraction inference results to new settings where the AIC does not necessarily have to hold. This insight leads to further results about how to perform more general causal inferences across abstractions (Sec. 3) and a new type of sampling procedure for high-dimensional causal generative modeling (Sec. 4). In short, the generalization of SCM projections to the partial form is indeed a contribution of the paper, but the main motivation for this contribution is to generalize challenging instances of causal abstraction inference. We thank the reviewer for pointing this out, and we will include this discussion in the paper's appendix.\n\n> 2. For experiments, what does the variance in each method (Fig 5) come from? [...]\n\n**Answer:** The variance for each approach arises from the sampling of the training data and the randomness in the training process (e.g., parameter initialization). One explanation for the seemingly higher variance in the projected C-DAG approach may simply be that the plot somewhat exaggerates it due to the log-log formatting. Another deeper reason may be that there is a tradeoff between granularity and constraints when we convert from the original C-DAG to the projected C-DAG (as we are potentially adding more edges). With more edges, there are fewer constraints; therefore, converging may be more challenging despite the higher accuracy. Of course, this more nuanced view is just brought up by the new machinery developed in this paper. This paper is the first to study relaxed abstractions under violations of AIC, which are very likely to occur in almost any scenario.  \n\n> 3. What is the scalability of the proposed method in a more complicated, real-world scenario? [...]\n\n**Answer:** One issue that is resolved by our paper is that the AIC can be a challenging thorn when attempting to apply out-of-the-box representation learning tools in causal generative modeling contexts. We would like to be able to learn an NCM on top of any representation, but the AIC prevents us from doing so. The goal of the MNIST experiment is to show a proof of concept that, regardless of how the representation arises, we can still perform causal inferences and sample causal images using the new projected-sampling approach. This is powerful as it means that the method can achieve robust performance regardless of the method of representation learning used. Scaling to even higher-dimensional settings may require larger architectures, but fortunately, one can leverage the vast research of the general deep learning community to accomplish this while still enjoying the causal benefits from this work.", "claims_and_evidence": "The claim that the Abstract Invariance Condition (AIC) can be easily violated and invalidate causal conclusion is well supported by the example and explanations. The discussion on when and how lossy transformation should be used is interesting. This highlights the importance of the problem that the paper is tackling.", "methods_and_evaluation_criteria": "The abstraction projection looks like a practical method to address the problem of AIC violation.", "theoretical_claims": "I have checked until Def 6. \n\n\nL110-118 could be put into Def 2, as I got confused by undefined notations when reading Def 2.\n\n\nDef 5: $\\tilde{pa}_V$ is not used.\n\n\nDef 6: Definition of $\\tau$-abstraction is missing in the main text (please move or point to Def 10 in Appendix A).", "experimental_designs_or_analyses": "I checked the two experiments and they look correct to me.", "relation_to_broader_scientific_literature": "The key contribution seems to be a generalization from the prior literature. I think it is a useful addition to the direction of causal representation learning.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The paper is notational heavy and contains a number of definitions that require the readers to consume, especially in Sec 1.1 and 2.1. They are not the easiest to follow. It would be helpful for the authors to state briefly why those definitions are needed and how they will be used. Aside, it is hard to cross-reference the notations from all different parts in the paper to Alg 1. I suggest the authors add explanations to it.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. The details of the theoretical contribution are unclear. It seems like projection has been proposed in Lee & Bareinboim, 2019, but partial SCM projection seems novel in this paper. However, is definition 6 original or discussed in prior work? Could the authors articulate what is novel in the paper in comparison Lee & Bareinboim, 2019? Since the paper’s main contribution is a formalism framework that generalizes from previous literature, at a higher level, I think it would be better to have a separate paragraph to summarize the theoretical contribution.\n\n2. For experiments, what does the variance in each method (Fig 5) come from? It seems like the projected C-DAG approach has a larger variance than other methods. Is it some sort of bias-variance trade off and/or can authors discuss it?\n\n3. What is the scalability of the proposed method in a more complicated, real-world scenario? MNIST is a classic but relatively simple dataset.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mZk3fj7Ual", "reviewer": "Reviewer_b1c8", "summary": "This work addresses the problem of causal abstractions, providing an intriguing approach that could extend traditional fine-grained causal applications to more general scenarios. It emphasizes higher-level causal relationships and inferences. A key contribution of this work is the relaxation of the function class limitation in abstraction functions, bridging low-level causal models to high-level ones, and enabling lossy representations known as projected abstractions. Specifically, the work introduces three rules, as outlined in Definition 8 (Partially Projected C-DAG), which play a critical role in handling violations of the abstract invariance condition. Experiments on color MNIST demonstrate the benefits of using projected abstractions.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive review and detailed suggestions. Addressing the reviewers points:\n\n> The Abstraction Function, as defined in Definition 4, is required to satisfy certain assumptions [...]\n\n**Response:** Yes, broadly, the paper is focused on a specific family of abstractions called constructive abstractions. Most works found in this literature, including ours, focus on this family because there is a natural mapping between low- and high-level interventions and distributions, allowing for well-defined inferences. These abstractions are most easily realizable in practice as well, since the construction of the high-level model naturally follows from the declaration of the clusters, which are interpretable and can also sometimes be learned (see “Neural Causal Abstractions” (Xia & Bareinboim, 2024).) If one would prefer to allow unrestricted abstraction functions, the results of the paper may still be applicable, but one would have to define how low-level interventions correspond to high-level interventions on a case-by-case basis (i.e., it is not necessarily the case that an intervention on $do(\\mathbf{X}_L = \\mathbf{x}_L))$ on the low-level is equivalent to an intervention on $do(\\tau(\\mathbf{x}_L))$ on the high-level, or even well-defined). \n\n> The three rules in Definition 8 are key factors in the proposed method. I am curious about the underlying motivation [...]\n\n**Response:** Indeed, that’s a cool question, thank you. The three rules demonstrate the core insight of allowing lossy abstractions – by losing some information in the variables, we must make up for it by removing causal constraints (i.e., adding more edges). These rules are necessary for guaranteeing the validity of the downstream causal inferences, as shown through Thm. 2. Without adding these edges, one risks claiming that a causal query is identifiable when it is not. For example, in Fig. 1(a), if the edge from $Z$ to $Y$ is not included, one may mistakenly assume that $P(Y \\mid do(X), Z) = P(Y \\mid do(X))$, which is the mistake made in Ex. 2. The proof of Thm. 2 shows the math behind what could go wrong when each of these rules are ignored.\n\n> Overall, this work mainly addresses AIC violations, but I am concerned whether the results hold regardless of the degree of AIC violations [...]\n\n**Response:** This is a great question. In fact, this is precisely the issue that we are solving. In the past, approaches that rely on the AIC would deteriorate in performance as the AIC becomes more and more violated. In contrast, the projected abstraction approach does not care how much the AIC is violated and performs identically regardless. Note, for example, in Fig. 4 of our MNIST experiment, reducing the representation size to a mere binary variable strongly violates the AIC and therefore results in very poor performance from the RNCM (3rd row), but with projected sampling, we can produce images as usual (4th row).\n\n> My final concern is regarding the experiment settings. [...]\n\n**Response:** Our goal with the MNIST experiment was to provide a proof-of-concept of how one could potentially learn a causal generative model without worrying about AIC violations. More generally, this would allow for a unification of out-of-the-box representation learning methods and causal generative modeling methods since it would not matter how the representation is learned once we move to the causal generative modeling phase. With this unification understood, scaling to more complex datasets is a matter of scaling the architecture and representation learning methods using state-of-the-art techniques from general deep learning research, which can be done separately from the causal results guaranteed by our work.\n\n\n> While I can understand the overall story, I feel that the writing style may not be very accessible [...]\n\n**Response:** We aimed for a more rigorous analysis of this highly technical topic for the sake of concreteness, so its more mathematical nature is somewhat to be expected. We do appreciate the suggestion and will add some clarifying sentences in the final report and include an appendix section discussing some of the key prior works to help bring readers up to speed with the contents of the paper. \n\n> It would be better to introduce, from an intuitive (high-level) standpoint, why consistency estimation is still possible when faced with lossy representations, and how the proposed method addresses this issue.\n\n**Response:** Yes, thank you for the suggestion. We will include the discussion mentioned above on the tradeoff between the lossiness in the abstracted variables and the loss of constraints in the projected C-DAG and why inferences in the new model still apply.", "claims_and_evidence": "The proposed method aligns with the claims made in the paper.", "methods_and_evaluation_criteria": "Empirical evaluations on toy data validate the benefits of the proposed method. Here, in the case of color MNIST, which is relatively simple and serves as toy data, the significant advantages of the proposed method are not fully demonstrated. While the importance of relaxing the abstraction function is clear, especially for real-world applications, the simplicity of the dataset limits the verification of these benefits.", "theoretical_claims": "I reviewed the theoretical aspects at a high level but did not rigorously verify the correctness of the theorems.", "experimental_designs_or_analyses": "The provided analyses are generally well-conducted.", "relation_to_broader_scientific_literature": "The paper tries to work on an important problem.", "essential_references_not_discussed": "Related works are thoroughly discussed in this paper.", "other_strengths_and_weaknesses": "Strengths:\n\nThe organization is well-structured, and the presentation is relatively clear.\n\nIt addresses a very important theoretical problem, and the problem is also significant in practice.\n\nI really like the three rules in Definition 8, as they play an important role in 'correctly' constructing high-level causal models.\n\nSome examples are provided, which greatly aid in understanding from an intuitional viewpoint.\n\n\n\nProblems:\n\n1) The Abstraction Function, as defined in Definition 4, is required to satisfy certain assumptions to maintain specific structures for inter/intravariable clusterings. This implies that the function class of the Abstraction Function is constrained within an equivalent class. While this is clear from the definition, I am concerned about how such a function class is restricted.\n\n2) The three rules in Definition 8 are key factors in the proposed method. I am curious about the underlying motivation for introducing these three rules. What is the reasoning behind their inclusion, and how do they contribute to the overall approach?\n\n3) Overall, this work mainly addresses AIC violations, but I am concerned whether the results hold regardless of the degree of AIC violations. Does the method remain effective when AIC violations are more significant, or is its performance sensitive to the extent of these violations?\n\n4) My final concern is regarding the experiment settings. While I understand that this work is primarily theoretical, the motivation for addressing lossy representations stems from a practical viewpoint. In this context, the experiments are conducted at a relatively simple level, which raises concerns about the effectiveness of the proposed method in more complex scenarios. \n\n\n\nSuggestions:\n\n1) While I can understand the overall story, I feel that the writing style may not be very accessible for those who are not familiar with this area. It would be helpful to provide some intuitive interpretations for the basic and key definitions. For example, a brief discussion of Definitions 3 and 4 from an intuitive standpoint could enhance clarity.\n\n2) It would be better to introduce, from an intuitive (high-level) standpoint, why consistency estimation is still possible when faced with lossy representations, and how the proposed method addresses this issue.", "other_comments_or_suggestions": "none", "questions_for_authors": "none", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Nv5QCVFxVD", "reviewer": "Reviewer_cdvd", "summary": "This paper presents a theory of causal abstractions that generalizes them beyond the usual abstract invariance condition. It formalizes the idea of general causal abstractions through the concepts of partial SCM projections, soft interventions and generalized queries. An algorithm to construct these is general abstractions is obtained. Results regarding inference on the general causal abstractions (through a notion of project cluster causal diagram) are obtained. Some experimental validation of the theory is presented with inference tasks on coloured MNIST datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive review and valuable insight in our experimental analysis. We address your comments below. \n\n> I would like to see a discussion of the limitations of the proposed theoretical framework, as well as well as future perspectives.\n\n**Response:** Indeed, one of the main limitations of the theoretical framework is that, as a tradeoff for violating the AIC, it is possible that more causal dependencies are introduced, as illustrated through the definition of the projected C-DAG in Def. 8 (more edges are added when there are AIC violators). We will emphasize this point further in the paper. This naturally implies that there is a challenge of determining how to balance the tradeoff between lossy representations and loss of causal constraints, and we will also make a point of this as a potential future direction of work. \n\n> \"The abstractionless model has higher error than the projected C-DAG model since it operates in a higher-dimensional space.\" I don't find this explanation fully satisfying [...]\n\n**Response:** In this particular experiment, the dimensionality reduction is quite extreme. In the $\\mathcal{G}$-NCM, the model is trying to sample the entire image $X$ as an intermediate step, while in the $\\mathcal{G}_{\\mathbb{C}}^{\\dagger}$-NCM, the variable $X$ has been abstracted into a binary variable. Since the high-level model has much less information in the input dataset, its only serious advantage is the dimensionality reduction. That said, the error of the $\\mathcal{G}$-NCM still trends downward with higher data, since it is theoretically still a sound method of estimating the query. With much more parameters and compute, it is possible that the approach could achieve similar errors. We will include more discussion on this in Appendix D, as well as discussion on the tradeoff of performing such a large dimensionality reduction. \n\n> In the coloured MNIST experiment one thing is not clear to me [...]\n\n**Response:** Indeed, we chose the binary representation as the most egregious example of an AIC violation, and we illustrate one of its clear flaws – one should not be able to achieve any good reconstruction of the image with such a lossy representation. This highlights the strength of our approach of projected sampling – whatever we lose in the representation, we can represent it in the exogenous space. An example analogy is this: suppose we have an image of 10 bits, but we represent it with only 1 bit. We certainly cannot reconstruct a 10-bit image with only 1 bit, but we can reconstruct samples by sampling the other 9 bits. The particular distribution of 9 bits is based on the theory discussed earlier in Sec. 2: we sample conditioned on the 1 available bit and the parents of the variable as if we are translating a high-level intervention to the low-level. We will include this discussion in the paper.\n\n> We see an improvement with respect to the non-causal model, but the digits themselves don't look right compared to the 16 dimension RNCM model [...]\n\n**Response:** Thank you for the suggestion, we will run the two methods at different representation sizes to see how the performance scales. Our expectation is that while the original RNCM degrades in performance as the AIC is more and more violated (translating to worse performance with lower dimensionality), the RNCM with projected sampling does not suffer from this issue. To continue the bit analogy from above, the reconstruction of a 10 bit image would look better if we had 9 of the bits compared to only having 1 bit. However, with projected sampling, we are sampling the remaining bits anyway, so we always have the full 10 bits. Generally speaking, performance of the projected sampling approach could potentially be improved by simply improving the base architecture of the model, although this is out of the scope of our work. \n\n> I suggest to nuance/precise this type of statement: \"Combining these two modes of reasoning is vital for building more advanced AI systems.\" [...]\n\n**Response:** Thank you for the suggestion, we will change it to “Combining these two modes of reasoning unlocks great potential for building more advanced AI systems.”\n\n> In definition 7, is there a consistent generalization of the PCH to soft interventions? [...]\n\n**Response:** Yes, this is indeed an interesting point. We frame Def. 7 in the context where the high-level model is performing hard interventions to emphasize where the noise in the corresponding low-level soft interventions arises as a response to AIC violations. That said, the method could certainly be generalized to handle cases where the high-level model is performing soft interventions as well. Then, the corresponding low-level interventions would be soft interventions that aggregate the results of all of the possible sampled interventions in their high-level counterparts. We will add a discussion on this in the appendix, as you suggested. Thank you!", "claims_and_evidence": "- The contributions of the paper are mainly theoretical. The theoretical framework is very well supported by useful and insightful results, clear explanations and examples.\n- The experimental validation is somewhat convincing although not totally. I do not think that this is a significant weakness of the paper, it is known that the learning problem for causal models is difficult. However, I would like to see a more nuanced discussion of the experimental results (see below). I would also like to see a discussion of the limitation of the experimental setup.\n- I would like to see a discussion of the limitations of the proposed theoretical framework, as well as well as future perspectives.", "methods_and_evaluation_criteria": "The proposed evaluation is appropriate.", "theoretical_claims": "The defintions and statements are clear and sound. I however did not check the proofs in detail.", "experimental_designs_or_analyses": "- \"The abstractionless model has higher error than the projected C-DAG model since it operates in a higher-dimensional space.\" I don't find this explanation fully satisfying. As far as I understand, the fact the the baseline operates on a higher-dimensional space could be both beneficial or detrimental. The reduction in dimension is also not dramatic. It is necessary to understand better if this is indeed the cause of the improvement or if there is another explanation.\n- In the coloured MNIST experiment one thing is not clear to me, should it even be possible to sample from the right distribution using a binary representation?\n- We see an improvement with respect to the non-causal model, but the digits themselves don't look right compared to the 16 dimension RNCM model. I could like to see versions of both models with intermediate numbers of dimensions (for example going from R^16 to R) to see how to performance degrades with less dimensions.", "relation_to_broader_scientific_literature": "As far as I know, this paper is a very significant contribution to the literature on causal abstractions. I am not aware of previous works that dealt with relaxing the AIC condition. The results of the paper open the door to applying causal abstractions to much wider problems.", "essential_references_not_discussed": "Not that I know of.", "other_strengths_and_weaknesses": "**Other strengths**\n- The work tackles an important problem in an original way\n- The paper is very well written and there is a nice effort to make it understandable", "other_comments_or_suggestions": "I suggest to nuance/precise this type of statement:\n\"Combining these two modes of reasoning is vital for building more advanced AI systems.\"\nThe word \"vital\" is strong, so either replace with something weaker or elaborate on what is meant.", "questions_for_authors": "In definition 7, is there a consistent generalization of the PCH to soft interventions? If yes, would be interesting to discuss/include it.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qDEEXfHhZ8", "reviewer": "Reviewer_iNbN", "summary": "This paper introduces a new notion of causal abstraction called \"projected abstraction\" that extends causal abstraction theory to handle lossy representations—situations where multiple low-level interventions with different effects map to the same high-level intervention. The authors show how to construct projected abstractions from low-level models, translate causal queries between levels, identify high-level causal relationships from limited low-level data, and demonstrate the effectiveness of their approach in high-dimensional image settings. Broadly, this work bridges causal reasoning and abstraction.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and review. We are happy that the work was understood. To answer your concerns:\n\n>The experimental section seems to be good to me, but it will be difficult for readers not familiar with NCMs. I would recommend trying to give the experimental section some more room to breath.\n\n**Response:** Thank you for the suggestion. We will expand the experimental section to contextualize NCMs for unfamiliar readers, and we will include a section in the appendix discussing NCMs and their relevant results to the discussion in this paper.\n\n>However, given that causal abstraction has been applied to mechanistic interpretability, it might be good to situate the paper relative to that literature and explain to readers from that field how this relates.\n\n**Response:** Thank you for these citation suggestions; we will be sure to include them. Mechanistic interpretability is a prominent issue in the field of causal abstractions, and we will certainly add an appendix section to discuss it in more detail. Although this work focuses more on the foundational aspects of abstraction theory, particularly performing cross-layer inferences, we find that many of the results are strongly applicable to both problem settings. Relating this work to the mechanistic interpretability problem is a clear next step.\n\n>How does this relate to causal feature learning? I don't think you cite any work from that area, but it seems deeply related to me.\n\n**Response:** Actually, the problem of causal feature learning is highly relevant. Papers such as “Visual Causal Feature Learning” (Chalupka et al., 2014) leverage data-identifiable forms of the AIC to learn ideal representations and variables that best preserve the causal relationships between variables. This connection is briefly discussed in “Neural Causal Abstractions” (Xia & Bareinboim, 2024) Appendix D.2. We will also include a section in our paper complete with these citations.", "claims_and_evidence": "The paper claims to address the limitation named the \"Abstract Invariance Condition\", a common scenario where two variables cannot be abstracted together because they have different downstream impacts. The proposed notion of projected abstraction deals with this issue by representing the loss of information from collapsing two variables that have different downstream impacts in terms of exogenous variables in the high-level causal model. \n\nThis is an important direction for causal abstraction theory to be developed in. Exact transformations or abstractions should be very rare, and understanding how to model lossy abstraction will be key in all practical efforts.", "methods_and_evaluation_criteria": "The MNIST task in this paper seems appropriate for the proposed methods", "theoretical_claims": "I can confirm that the main text is coherent and the definitions make sense given prior work in the area. No obvious issues stick out. However, I cannot attest to the proofs or appendix material as it would take too much time as a reviewer to go through them all in detail.", "experimental_designs_or_analyses": "The experimental section seems to be good to me, but it will be difficult for readers not familiar with NCMs. I would recommend trying to give the experimental section some more room to breath.", "relation_to_broader_scientific_literature": "This paper is written for theoretical researchers in the field of causality. Within that context, this paper is very strong and will be of great interest. However, I believe that researchers outside this field will find it difficult to work through. \n\nI don't think this is a real issue, and I think ICML should accept theoretical work on causality!", "essential_references_not_discussed": "These aren't essential citations, I believe that the authors did a good job citing the relevant work in causality. However, given that causal abstraction has been applied to mechanistic interpretability, it might be good to situate the paper relative to that literature and explain to readers from that field how this relates. \n\nhttps://proceedings.mlr.press/v162/hu22b.html\n\nhttps://proceedings.neurips.cc/paper/2020/hash/92650b2e92217715fe312e6fa7b90d82-Abstract.html\n\nhttps://arxiv.org/pdf/2502.20914v1\n\nhttps://arxiv.org/abs/2106.02997", "other_strengths_and_weaknesses": "I think introducing the notion of identifiability into causal abstraction is an important aspect of this work, as it helps bridge existing theory on causal abstraction to other research in causality.\n\nAdditionally, developing methods for inferring abstract structure from limited concrete low-level data is crucial for causal abstraction to be applied in real world settings.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "How does this relate to causal feature learning? I don't think you cite any work from that area, but it seems deeply related to me.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GFLt5J8kgL", "reviewer": "Reviewer_jmSk", "summary": "This paper introduces projected abstractions and an algorithm to compute them from a low-level SCM.\n\n### Update after rebuttal\nFirst of all apologies for an initial review that may have read harsh.\n\nSo I carefully read the other reviewers' opinions and the rebuttals. It didn't really bring much more clarity. My impression is that there is even some divergence in what the reviewers think the contribution of the paper is. In parts it seems that even the formalization is considered as part of the contribution, where I think that notation-wise most is taken from what \"Neural Causal Abstractions\", which I believe is the authors' previous work.\n\nI really do not know what to do with this work. I am convinced that the general research direction of the paper is important, and maybe so is even the contribution. But, at least in the way how it is written down, I can hardly imagine that the paper will have a huge impact beyond a very limited audience that will really make the effort to understand the work in detail. I think that the paper might even be relevant not only for theoretical scientists, so the potential contribution is beyond mere theory I would say. I think it is interesting for me and wouldn't consider myself a theoretic researcher in this field. \n\nSo my concerns about clarity prevail and I am going to leave my assessment as is, but apparently I am the outlier. If for the other reviewers the paper not only makes a relevant technical contribution but also manages to properly convey it in their opinion, I have no trouble with the paper being accepted.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's feedback and believe some misunderstandings may have led to a harsh evaluation. We respectfully ask for reconsideration based on the clarifications below.\n\n> In general the paper is very difficult to read [...]\n\n**Response:** We recognize that the reviewer’s primary concern is the paper’s density. While we aimed for clarity through diagrams and examples, some content had to be relegated to Apps. B and C, preventing us from elaborating as thoroughly as we would have liked. The density arises from building upon several works that already involve complex semantic systems, including:\n1. Pearl Causal Hierarchy (PCH): Our approach is general, encompassing all 3 layers of the PCH. While we chose to present examples using simpler interventional quantities from Layer 2, we present all definitions generally to convey the full scope of our contribution (Layer 3).\n2. Abstractions: We follow established abstraction theory, using concepts like inter/intravariable clusters, $\\tau$-abstractions, AIC, and C-DAGs, with notation aligned to prior literature.\n3. Causal Generative Modeling: Our work extends neural causal models (NCMs), forming the basis for Sec. 4 experiments.\n\nIn the main paper, we prioritized our new contributions over reviewing prior work, referencing established concepts rather than restating them.\n\nThat said, we are committed to improving clarity by:\n- adding a notation summary table,\n- expanding discussion on related works,\n- using the extra page to elaborate on each definition.\nWe appreciate your willingness to work with us in evaluating the contributions presented.\n\n>I am unsure what significant claims this paper wants to make.\n\n**Response:** We emphasize in our paper that our primary goal is to develop a causal abstraction framework robust to violations of the abstract invariance condition (AIC). When two low-level interventions produce different effects but map to the same high-level intervention, existing frameworks fail to address the resulting ambiguity. This problem is particularly prevalent when working in the representation space after performing representation learning, as a learned, lossy representation is unlikely to avoid such ambiguity. Working in a representation space offers substantial advantages for high-dimensional causal inference, a direction we aim to advance in this paper. We hope this motivation is clearly conveyed in the introduction (lines 58–75, left), supplemented by the example on HDL and LDL cholesterol, as is our proposed solution (discussed from lines 76-95 (left)). We will refine the introduction for clarity and welcome any specific suggestions.\n\n>the approach starts from the premise to have a low-level SCM already available [...]\n\n**Response:** The relaxation of this requirement is the entire contribution of Sec. 3. Recognizing that we are unlikely to have the whole low-level SCM, we introduce identification results (Def. 9, Thm. 3) that specifically aim to infer causal quantities given limited data from the low-level SCM, such as observational data.\n\n> if I already had a low-level representation (input of your algorithm) [...]\n\n**Response:** The benefits of working in the abstraction space are similar to those motivating the use of representation learning in non-causal contexts: reducing dimensionality for improved tractability, transforming data into spaces with desirable mathematical properties (e.g., the linearity of Word2Vec), and better interpretability.\n\nThe AIC challenge is unique to the causal setting, as non-causal contexts do not require consideration of how causal relationships between variables are preserved or altered when the representation is lossy. Previously, the trade-off of working in the abstracted space, as you mentioned, meant accepting AIC violation risks for the benefits of representation learning. Our work resolves this dilemma by enabling the advantages of representation learning without incurring any drawbacks from AIC violations.\n\n> even if I am interested in summarizing variables [...]\n\n**Response:** Certainly, but doing so would prevent leveraging the computational benefits of representation learning in the causal modeling process.\n\n>in a similar line: If you admit dependency between U-variables [...]\n\n**Response:** This is a good question. Recognizing that AIC violations can be reinterpreted as projections into the exogenous space is one of the key insights of our paper. Previously, the prevailing view was that if a high-level intervention was ambiguous, performing the abstraction was futile since it meant ignoring details that were relevant to the setting (resulting in mathematical contradictions).\n\nThe natural connection to SCM partial projections enabled a generalization of previous frameworks, accommodating AIC violations by interpreting the lost details as exogenous variables. Establishing this formal connection and generalization constitutes a key and non-trivial set of contributions presented in Sec. 2.", "claims_and_evidence": "I am unsure what significant claims this paper wants to make. There is the introduction of the computation of projected abstractions, which is probably a straight forward algorithm once one buys the ton of definitions of projections. There is also the claim that those projected abstractions \"make an important step around the AIC\", but it is not entirely clear to me neither what this is supposed to mean exactly and how the algorithm assures it. I am not saying that the experiments don't deliver on a certain expectation, I am just saying that the paper fails to make the contribution understandable, even to someone with fairly solid knowledge in causal inference.", "methods_and_evaluation_criteria": "n/a", "theoretical_claims": "There are some theoretical statements (even though some or just definitions, like Prop 1). Based on the density of the paper and the lack of clarity, I was not even able to assess what the theorem statements want to say. Sorry, I really made an effort.\n\nBesides, I am not sure about the relevance of the whole question, because the approach starts from the premise to have a low-level SCM already available, which is of course usually not the case. I am partially willing to revise my review (even though I have spent quite a significant time on this paper already to understand what is going on) if the authors answer to my below questions.", "experimental_designs_or_analyses": "n/a", "relation_to_broader_scientific_literature": "Good.", "essential_references_not_discussed": "None I am aware of.", "other_strengths_and_weaknesses": "The presentation of the paper is in my view not top-tier conference-ready. In general the paper is very difficult to read, even for someone who is fairly familiar with the field. On about 3 pages, the paper introduces over a 100 symbols, which in itself is already a challenge. I think that it should be possible to somehow reduce the formal burden of the paper. But what is worse, I believe that there are mistakes in the notation. So when the reader arrives at Proposition 1 and the 4 millionth symbols W_i^o and W_i^v, then I think it should be W_i^u instead. In that same proposition, there is a syntactic definition of \\delta(W^o, W^u) that has no semantic attached, or at least none that I would understand. Usually explanations follow after the formal definitions, but the intellectual burden for the reader is really enormous, and the authors make little effort to wrap it into didactically well elaborated portions.", "other_comments_or_suggestions": "Let someone familiar with causal inference (and maybe abstractions) read your paper before you submit to ICML and check on clarity.\n\nI beg the authors to try to make things simpler notation-wise or be at least more didactic and structured in the write-up. I understand that there are two layers of abstraction going on, and defining mappings between them considering all relevant details can be tough. But my experience tells me that probably some definitions could be merged and even some symbols ommitted.\n\nIt is urgently necessary that language is used more carefully. In the motivation (line 70-72 left), the authors say \"it may be desirable to have a formalism in which these kinds of ambiguous abstractions are well-defined\". I strongly doubt that the word \"well-defined\" is the correct choice here. My current intuition is that with a different type of abstraction you have other *properties* that allow you to do something that you cannot do with previous approaches. If your contribution is not more than making something well-defined, then we seriously have an issue.\n\nIn this line, the contribution needs to become clearer. And with this I do not mean to make an explicit bullet list of 3 un-understandable points, but to very concretely formulate a limitation of previous approaches and how you overcome it. I believe that example 1 could help a lot in this regard if one introduces it on an intuitive (already somewhat formal) level without the need of having the AIC conditioned formalized already at that point. It is really weird that we have a 2.5 page introduction and the relevant example only follows in section 2.", "questions_for_authors": "- if I already had a low-level representation (input of your algorithm), I clearly am in danger to suffer from the potential problem of AIC violation if I map into the abstraction space. Which advantage can I gain from the projection to compensate this risk other than shortcuts (summarizing certain situations in low-level setup)?\n- even if I am interested in summarizing variables, couldn't I avoid running into the AIC problem altogether by reasoning on the low level and doing an abstraction only post-hoc?\n- in a similar line: If you admit dependency between U-variables, it seems that you start right away from the premise that even the low-level SCM is not described at such a level of detail that noises are independent (as they should be ideally). To me, the projections just seem to make a step into another SCM where there are possibly yet more of those dependencies, and we give it a special name (AIC). The question is: Why is the AIC violation more severe for reasoning than having dependencies among variables in U?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel framework for causal abstraction under lossy representations by introducing projected abstractions, generalizing causal abstraction models to handle violations of the Abstract Invariance Condition (AIC). 

The paper was well received by most reviewers for its originality, theoretical depth, and connection with practical causal inference challenges. While the empirical evaluation was somewhat limited in scope, it served as a convincing proof of concept. A few reviewers noted that the notation and density of definitions made the paper challenging to follow, and suggested improvements in clarity, intuitive explanation, and experimental elaboration. Reviewer jmSk provided the only negative review, due to the presentation issues. 

Despite concerns about the presentation raised by multiple reviewers, and one reviewer providing a negative score primarily due to clarity,  the majority of reviewers agreed that the paper addresses an important gap in causal abstraction and offers meaningful theoretical and practical advances while finding the ideas well-motivated and clearly articulated. 

Given the significance of the problem, the originality of the proposed solution, and the overall positive reception, the AC recommends accepting this paper. The authors should incorporate the reviewers’ constructive suggestions on the presentation to improve readability in the final version.
Decision: Accept (poster)