Mechanistic Independence: A Principle for Identifiable Disentangled Representations
Authors: Stefan Matthes
Zhiwei Han
Hao Shen
Venue: ICLR 2026 Poster (2026)
Keywords: Identifiability
Disentangled Representation
Mechanistic Independence
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: *Disentangled representations* seek to recover latent factors of variation underlying observed data, yet their *identifiability* is still not fully understood. We introduce a unified framework in which disentanglement is achieved through *mechanistic independence*, which characterizes latent factors by how they act on observed variables rather than by their latent distribution. This perspective is invariant to changes of the latent density, even when such changes induce statistical dependencies among factors. Within this framework, we propose several related independence criteria -- ranging from support-based and sparsity-based to higher-order conditions -- and show that each yields identifiability of latent subspaces, even under nonlinear, non-invertible mixing. We further establish a hierarchy among these criteria and provide a graph-theoretic characterization of latent factors as connected components. Together, these results clarify the conditions under which disentangled representations can be identified without relying on statistical assumptions.

Published as a conference paper at ICLR 2026
MECHANISTIC INDEPENDENCE:
A PRINCIPLE FOR
IDENTIFIABLE DISENTANGLED REPRESENTATIONS
Stefan Matthes
Zhiwei Han Technical University of Munich zhi wei w.n@tum.de
Technical University of Munich stefan. tee @tmmm..mm...mm de
Hao Shen
Technical University of Munich hao. shene tum. de
ABSTRACT
Disentangled representations seek to recover latent factors of variation underlying observed data, yet their identifiaiility is still not fully understood. We introduce a unified framework in which disentanglement is achieved through mechanistic independence, which characterizes latent factors by how they act on observed variables rather than by their latent distribution. This perspective is invariant changes of the latent density, even when such changes induce statistical dependen- cies among factors. Within this framework, we propose several related indepen- dence criteria ranging from support-based and sparsity-based to higher-order conditions and show that each yields identifiability of latent subspaces, even under nonlinear, non-invertible mixing. We further establish a hierarchy among these criteria and provide graph-theoretic characterization of latent subspaces as connected components. Together, these results clarify the conditions under which disentangled representations can be identified without relying on statistical assumptions.
INTRODUCTION
Disentangled representations capture the underlying explanatory factors that generate observed data. They are widely believed to promote compositionality, enable controllable generation, and facilitate transfer (Bengio et al. 2013 Higgins et al. 2017 [Schökkopf et al. 2021 Locatello et al. 2019 9 Gr- eif et al. 2020 Goyal & Bengio 2)22 From scientific perspective, disentanglement aligns with the goal of discovering the causal or mechanistic structure of data-generating processes (Schölkopf ett al.) 2021). The question of whether such representations can be consistently recovered is ad- dressed by identifiability. If a model class lacks identifiability, different training runs may encode incompatible factors, thereby undermining interpretability and transfer.
classical route to identifiability is to posit statistical independence of the latent factors, as in independent component analysis (ICA) (Comon 1994, Hyvärinen Ojal 2000) and independent subspace analysis (ISA) (Cardoso) 1998 Hyvärinen & Hoyer (2000.. Early work focused on lin- ear mixing, where identifiability can be obtained under mild conditions. For general nonlinear mixing, however, identifiability is impossible without further assumptions (Hyvärinen & Pajunen, 1999 Locatello al., 2019). motivating a large body of work that augments statistical assumptions with temporal cues (Hyvärinen & Morioka, 2016 2017] Klindt et al.] (020)) auxiliary variables
HHvärinen Morioka, | Hyvärinen et al. [219]] Khemakhem et al. (000a), multiple views
Khemakhem al.) 2020b Gresele et al. 20200 Von Kügelgen al. [021 Zimmermann et al.)
2021 Matthes et al. 2023), or interventions (Locatello et al. 2020; Lachapelle et al,
2022 Ahuja 2022 Brehmer al. 2022 Ahuja et al. 2023 Jiang & Aragam 2023 Yao al.
al..
2023 Zhang
et al.
2024 Ng et al. 2025)
A complementary strategy constrains the mechanism that maps latents to observations, such post-
nonlinear mixtures (Taleb Jutten 1999] Jutten & Karhunen 2004: Zhhang & Hyvärinen 2012
Lyu & Fu 2021), near-linear mixtures (Zhang al. 2008), local isometries (Horan et al.)
2021

---

Published as a conference paper at ICLR 2026
piecewise-affing mixtures (Kivva al.
2022),
sparsity (Moran et al. 2021
Zheng al. 2022
Zheng & Zhang 2023), additive structures (Lachapelle et 20.]) conformal maps and orthogo- nal coordinate transformations (Gresele al.
Ghosh et al. 2023), ) and Jacobian-based constraints (Brady et al. 2023 2024 Nguyen & Hu 2025 Independent Mechanism
[202]] Buchholz et al. 2022
Analysis (IMA) (Gresele et al. | proposes to address nonlinear ICA by restricting the mixing function so that its Jacobian has orthogonal columns. This couples statistical independence of the latents with a mechanistic constraint on the generator. In contrast, we pursue mechanistic indepen- dence as stand-alone organizing principle: factors are defined by how they act on observations (through the generator), not by how they are distributed. This shift yields identifiability statements that are invariant to reweightings of the latent density and allows the true factors to be misaligned with any statistically independent subspaces.
This work presents a family of mechanistic independence criteria spanning support-based sepa- ration, sparsity gaps first-order action, and higher-order (cross-derivative) constraints. Similar to ISA that shows identifiability with respect to minimal decomposition into independent subspaces, each criterion comes with a corresponding notion of irreducibility that rules out spurious internal splits of a factor and yields identifiability theorem. Our framework covers multi-dimensional factors and partial disentanglement.
Our framework generalizes and unifies recent identifiability results based on mechanistic con- straints: object-centric disentanglement via disjoint supports (Brady et al.) 2023). interaction asym- metry (Brady al. 2024), and additive decoders (Lachapelle et | [002]) and it partially subsumes sparsity-based nonlinear ICA results (Zheng et 20,2 Zheng & Zhang 2023) (the parts that do require statistical independence). Moreover, defining independent mechanisms by Jacobian- orthogonality as in IMA (Gresele et al. (02TT) appears in our laxonomy as one instance within a broader class of mechanistic constraints. Unlike approaches that rely primarily on distributional assumptions (e.g., temporal structure or auxiliary variables), our results hinge on properties of the generator and therefore remain valid under broad latent densities. The main contributions of this work are as follows.
We define a notion of local disentanglement and prove that under mild topological assump- tions (such as path-conneetedness of the source space) local disentanglement extends to global disentanglement even for generators that are not fully invertible.
We introduce a family of mechanistic independence criteria for subspaces and prove for each identifiability (up to block-wist invertible transforms and permutations).
We discuss how the independence criteria are related and show that the independent and ir- redueible factors coincide with connected components of graphs derived from mechanistic assumptions of the generator.
Notation We write [n] [1, (l....,,r] n for E N. Scalars are denoted by lowercase letters, vectors by bold lowercase, and matrices by bold uppercase (e-g., E R. E Th A E R Scalar- valued functions are written f. fs, while general maps are written fi- For p S1 X Sn, we set D.fp Dfp for the differential in the i-th argument (t, the canonical inclusion), and more generally Fp D fp o (4, 4, id).
DISENTANGLEMENT AND IDENTIFIABILITY
We now formalize the data-generating assumptions and the notion of disentanglement used through- out the paper, before turning to identifiability. Our goal is to explain when decoder (or encoder) recovers, up to natural ambiguities, the underlying factors of variation that compose the observa- tions.
2.1 DATA GENERATING PROCESS
We model latent factors of variation as subspaces a product manifold, reflecting the often compo- sitional nature of observed data. Let the set of generative (latent) configurations be oper subset
'The condition that S is open implies that each factor can vary independently and without restriction any point within the space and common assumption.
2

---

Published as a conference paper at ICLR 2026
S S S₁ x... x SK. where each factor space S; has positive dimension. We assume the latent distribution Pe is strictly positive on S.
In line with the manifold hypothesis in representation learning (though assuming that observations lie on rather than merely near a manifold), we posit that observations are produced via a generator (also called ground-truth decoder mixing function)
g:S - RRd
We denote the observation manifold by == g()), where typically d, dim(() is much smaller than dz.
Notably, instead of characterizing the underlying factors through (conditional) statistical indepen- dence or latent-space group actions (Higgins et al. 2018)) we characterize them by their action on the observation manifold via g. While these notions may align, they do not necessarily have to. Several possibilities for making this precise are discussed in Section
2.2 DISENTANGLED REPRESENTATIONS
To discuss how a learned representation may or may not reflect the underlying generative factors, we consider a target representation space Z S I1 Zj. In a disentangled representation, each component Zj is intended to capture a single latent factor, or at most a restricted subset of factors. We formalize this with the notion of decomposable map.
Definition 1 (Decomposable map). Let S <II,=, S, and 2 S I[ Zj. map h: S + Z is decomposable if there exists a surjection a: [K] + [L] and maps h,: Ilie- 1() S + Zj such that, for all 8 E S.
h(s) (()))(((()))))))))))))))) 101)
(1)
In other words, target factor 2j Zj depends only on the subset of source factors {s; a(i) j}. Definition (Disentanglemennt. A decoder 9: 2 + X' is disentangled w.r.t. a generator g: + X there exists a decomposable map h: S Z such that g - 9 h.
Disentanglement asserts that varying a single factor of the learned representation changes the de- coded observation exactly as varying the corresponding source factors would. It can also be defined in terms of encoder f: X + 2 (e.g., fo exist or may lack desirable properties such as continuity? Notably, an oracle generator would be trivially disentangled w.r.t. itself, even if not invertible. Under mild regularity assumptions, disen- tanglement forms an equivalence relation (see Propositions [nd meaning that g and I represent equivalent generative models.
h). However, when g is not invertible, f may not
More generally, ğ is locally disentangled if, for every E S, there exists neighborhood of s where the restriction of g admits such a disentangled representation (see Defn. [3]. first glance, local disentanglement may appear less significant than the global property. However, under mild topolog- ical constraints the two notions coincide, even when g is not fully invertible (see next section).
2.3 IDENTIFIABILITY
Identifiability asks whether (locally) disentangled description is essentially unique given only ob- servations in X. It characterizes when a learned representation must be disentangled. The following global result shows that, under mild topological assumptions, local disentanglement implies global disentanglement. The key condition is connectedness of slices in the source space. A k-slice the subspace obtained by holding all but factors constant (see Defn. 14). Note that path-connectedness of space and of its slices are related but independent notions (see Remark
Theorem 1 (Global Identifiabilityy. Let S be an open subspace of the product manifold TTK Sü where each factor S, has positive dimension. Then local disentanglement extends to global disen- tunglement if:
AA practical example where continuity breaks is the responsibility problem which arises when learning representations of unordered data, such as objects within image (Zhang al. 2019 Hayes et al. 2023 Mansouri al. 2023). The permutation invariance makes the generator non-invertible.
3

---

Published as a conference paper at ICLR 2026
(1) g: - is locally injective.
(2) S is path-connected.
(3) Every (K-1)-slice of S is path-connected.
A proof is given in Appendix A. each factor can vary independently (by openness and soteecooneett)nss)), and local injectivity pre- vents branching, local decompositions extend globally.
Informally, local disentanglement propagates along paths: since
In many practical cases (e.g.. convex open sets R"), the topological conditions hold automatically, and local injectivity follows from standard regularity assumptions. Thus, the main challenge is usually to establish local disentanglement, and the remainder of the paper therefore focuses on local identifiability.
IDENTIFIABILITY VIA INDEPENDENT MECHANISMS
We now establish a general framework that certifies local disentangiement by analyzing how latent factors acI on the observation manifold through the generator g. The key difference from classical approaches is that independence is formulated at the level of the generative mechanism rather than the latent probability law. As a result, it accommodates almost arbitrary distributions, including those with statistical dependencies between and within subspaces. Importantly, there is universal notion of mechanistic independence comparable to statistical independence. Instead, we present a family of independence criteria disjointedness (Type D). mutual non-inclusion (Type M), spar- sity gap (Type S), and higher-order separability (Type H,,) each which leads to disentangled representations when mirrored in the learned representation.
3.1 LOCAL IDENTIFIABILITY OF TYPE D
We begin by slightly extending the result of Brady et al. (2023) and rephrasing it within our frame- work.
Definition 3 (Mechanistic Independence of Type D). We say that S, and Sj (equivalently, s; and 8j) are mechanistically independent of Type D for all s E S, E Sa and v E Sa,Sj.
(u) D,go (v) 0,
(2)
where S; denotes the tangent space of S at s; and denotes the element-wise (Hadamard) product in Ed.
We call this Type D independence since Hadamard orthogonality expresses that different factors act on disjoint set of observation coordinates. For example, in images, each factor controls non-overlapping set of pixels. Independence among the Z, is defined analogously via g.
To ensure disentanglement, independence alone is insufficient: if source factor S, can be de- composed into smaller, mutually independent components, a learned representation may split and recombine them arbitrarily. This motivates the notion of reducibility.
Definition 4 (Reducibility of Type D). We say that S; is reducible of Type D if there exists s E S such that Te, S, admits a nontrivia F direct- decomposition T,, S, U @ V with the property that, for all u E U and v E V.
(ug.(u) (v) =
If no such decomposition exists, we call S; irreducible of Type D.
This coincides with reducibility defined in (Brady et al.) 2023) (see Proposition 5). but makes the connection to Type D independence explicit. If S; is reducible we could split it at a point into smaller independent subspaces, and if a factor one-dimensional should always be irreducible.
Throughout this work, we identify T (a) X' with its natural inclusion in R=
"Notriviall" means dim(U), dim(V)
4

---

Published as a conference paper at ICLR 2026
Theorem 2 (Local Identifiability of Type D). Let g: 8 - X and g: Z + X be local diffeomor- phisms with g(S) - 9(Z). Then 9 locally disentangled w.r.t g if:
(1) S S K S is open, and all factors are Type D independent and irreducible.
(2) 3 I Z is open with L S K, and the factors are independent of Type D.
Intuitively, if each source factor influences disjoint set of observation coordinates, and no finer decomposition is possible, then any learned representation that also acts on disjoint coordinates recovers the true source factors (up to block invertible transformations and permutations).
This result generalizes Theorem of (Brady al., 2023) partial disentanglement and non- invertible generators (when taking Theorem [ into account). A proof is given in Appendix A.3 Interestingly, all local identifiability proofs this paper follow common template: starting from the local reconstruction identity 9 g (where = g I 9 exists locally since both maps are local diffeomorphisms), one applies the independence conditions to constrain interactions between source and target factors. If a source factor interacted with multiple target factors, their indepen- dence would force a decomposition of the source factor, contradicting irreducibility. Occasionally, additional assumptions needed to further restrict the function class.
Since Type independence requires that no observation coordinate is affected by two factors, a natural question is whether this be relaxed allow limited overlap while still achieving identifi- ability. We next express this via supports (the index set of nonzero elements, denoted with supp(-)) of Jacobians.
Select a product basis (ui...., ua,) for TAS; define (4(8) = supp (upp(()a(u)) for the i-th basis vector; and let Cj be the index set of basis vectors of Tx, Sy. Then Type D independence can be reformulated as
Vi J. Va e, vo vbee, e,: (s)s) ( =
As long the basis respects the product structure, the particular choice does not matter. In the next two sections, we show how this condition can be relaxed, either via mutual non- con-inclusion or through sparsity gap.
3.2 LOCAL IDENTIFIARILITY OF TYPE M
Define the mutual non-inclusion relation between sets A, C [dg] as A B A that the sets may intersect, but neither is contained in the other.
Definition 5 (Mechanistic Independence of Type M). We say that S; and Sj are mechanistically independent of Type M if, for every s E S,
Vi # Va C, Vb ebee, (s(s) rh (s)
Type M independence allows observation coordinates to be influenced jointly by multiple factors as long as neither support fully contains the other. In image data, for example, different factors may affect intersecting sets of pixels, allowing partial occlusion, shadows and reflections. Unlike Type D independence, this notion depends on the choice of basis for T.S. To make it meaningful, we restrict to S dd (only for Type M), where T. carries canonical basis that aligns with the product structure. Reducibility is then expressed directly in these fixed coordinates.
Definition (Reducibility of Type M). The component S, is reducible of Type M if there exist E S and a partition C; AUB B such that
Va E A Q2(8) th (g()).
Theorem (Local Identifiability Type M). Let g: S + X and g: Z - X be local diffeomor- phisms with g()) 9()). Then ğ is locally disentangled w.r.t. g if:
(1) S C ddd is open, and the factors are Type M independent and irreducible.
A diffeomorphism is smooth bijection between manifolds with smooth inverse. A local diffcomorphism map that restricts to diffcomorphism some neighhorhood of each point.
5

---

Published as a conference paper at ICLR 2026
(2) Z C R is open, and the factors are independent of Type M.
(3) For all E S and Z with g(s) = (2)), [| v [
(5)
(4) For all such pairs,
(x(2)= = U 24(s),
(6)
(Bsupp(B...)
where B = Jg-log( and Sig mirrors fi g.
This theorem generalizes Theorem 3.1 of (Zheng & Zhang 2023) (itself an extension of (Zheng al., 2022)) multidimensional factors (see Proposition 4 for detailed comparison). Statistical independence of the sources is not required. Assumptions (2)(2) mirror those Theorem[2 condi- tion (3) motivates sparsity regularizer; and condition (4) rules out pathological cases and is implied by condition (i) in (Zheng & Zhang 2023). It usually holds when g is sufficiently nonlinear, though a failure mode is illustrated Example 1 case B, where the Jacobian constant on
3.3 LOCAL IDENTIFIARILITY OF TYPE
We now return to the setting where S is a smooth manifold and replace the mutual non-inclusion assumption with sparsity gap criterion. Among all coordinate systems, the basis aligned with the true factor decomposition yields the sparsest first-order action of the generator.
For s E S, let Pi (s) be the minimal tu norm of the matrix representing Dg.: T&S Ta(a when the domain basis is aligned with the decomposition
B
① St <E[K]
Conversely, let Pig (s) be the infimum of the to no-m over all bases of T&S that do not respect B. Definition 7 (Mechanistic Independence of Type S). The subspaces {S:) are mechanistically independent of Type S if. for every 8 E S,
Pas
< Pzg (s).
(7)
dictionary that maps infinitesimal latent directions to observation di- rections, Type S independence states that the sparsest such dictionary (in the lo sense) is attained precisely when the basis aligns with the true factorization. Any misalignment necessarily incurs strict sparsity gap.
Viewing the Jacobian
If the supports of different components disjoint, any mixing of partial derivatives can only enlarge the support, since no cancellations are possible. In this case, Equation holds trivially. Thus Type D independence is a special case of Type S independence. The sparsity gap, however, is considerably stronger: it remains valid even when the supports substantially overlap. For instance, suppose we have one-dimensional sources where each support (2, overlaps with the others by less than half of its elements. Even if misaligned basis were tuned so that every shared element canceled perfectly (if all possible), the total number of nonzeros would still increase. Thus, the sparsity gap persists under this optimal misaligned (but still suboptimal) basis transformation. In higher-dimennsional subspaces, the situation becomes more intricate, since inter-cancellations within block columns are possible. In sense, the sparsity gap captures all such potential cancellations and characterizes the theoretical limiting case. As before, irreducibility rules out internal decompositions (see Defn. 20) Example discusses Type M/S independence and reducibility in detail.
Theorem (Local Identifiability of Type S). Let g - X and ğ Z - X be local diffeomor- phisms with gS)) S g(). Then ğ is locally disentangled w.r.t. if:
(1) S C I S; is open, and the factors S; are Type S independent and irreducible.
(2) Z C I1 Z, is open with L S K, and the factors Z, are independent of Type S.
Intuitively, identifiability follows by exploiting the strict sparsity gap in equation 7 While fairly general, Equation] / is intractable to optimize in practice. In Section we investigate whether com- positional contrast (Brady al. 2023) serve suitable surrogate loss.

---

Published as a conference paper at ICLR 2026
3.4 LOCAL IDENTIFIABILITY OF TYPE H
Lastly we simplify and generalize the asymmetric interaction principle of (Brady et al. 2024 subsuming special case the additive setting of (Lachapelle et al. 2023).
Definition 8 (Mechanistic Independence of Type H,). Let S II K f=1 S be smooth manifold, and let g: S X be of class C" with n 2 2. We say that S, and Sj are mechanistically independent of Type H, if. for all E S,
9 0.
(8)
For n = 2. this requires that all cross-Hessian blocks vanish, implying additivity as in Lachapelle al. 2023). Irreducibility is defined analogously (see Defn. 22
To derive disentanglement, we additionally constrain the function class via separability. Definition 9 (Separability of n-th Order). We say that g: S + X is separable of order n 2 2 if there exists
8 such that, for all i [K], the image of D2,9a intersects trivially with
span P j * i; D*g ga: 1 2 1
Separability is closely related to sufficient independence in (Brady et al. 2024) and sufficient non- linearity in (Lachapelle et al., 2023), but is slightly weaker: allows arbitrary interactions among lower-order derivatives and within each block D
Theorem 5 (Local Identifiability of Type H,). Let g: S - X and g: Z X be local C".
diffeomorphisms with n 2 2 satisfying (S)) C ((2). Then 9 is locally disentangled w.r.. g if:
(1) S S I S, is open, and the factors are Type Hn independent and irreducible.
(2) S I1 2j is open with L S K, and the factors are independent of T'Typ Hn.
(3) g is separable of order
Compared to (Brady et al. 2024), our formulation highlights that source factors should be taken as irreducible, which we argue is a necessary and natural requirement. This perspective eliminates any dependence on (n+1)-th derivatives (which may not exist) and avoids the use of equivalent generators. As with our other results, the conclusion also applies to non- -on-invertible generators, and provide explicit proof for n 3 (corresponding
2 in their slightly different notation).
4 DISCUSSION
4.1 HIERARCHY OF INDEPENDENCE
The different independence criteria form a natural hierarchy (see Figure [ Type D independence is the strongest: it implies all others. Differentiating Type D yields Type H2, and further differentiation gives Type Hg, and on. Type M follows since disjointness is special case of mutual non- inclusion. Type S is also implied: in the sparsest product-respecting basis, Type D ensures that supports are disjoint, and any linear combination of column vectors from different blocks strictly enlarges the support, creating sparsity gap. Finally, Type S implies Type M independence when working in the sparsest product-splitting basis (but not an arbitrary product-aligned basis).
Since reducibility describes whether a factor can be split into smaller independent subspaces, the im- plication relations among reducibility types largely mirror those among independence types, except for Type M, which depends on the choice of basis.
This reveals tradeoff between the identifiability results for Type and Type S: by enforcing tolerate stronger interactions between different fac-
stronger coherence within each factor, tors. Relations among the other identifiability results are less direct, since they require additional assumptions (cf. the asymmetric interaction principle of (Brady al. 2024
As with statistical independence, one must distinguish between pairwise and mutual independence. For Types D, M, and H,, the two coincide, but for Type S they differ in general. While mutual independence always implies pairwise independence, Example case B, shows a Jacobian where factors are pairwise Type independent but not mutually so.
7

---

Published as a conference paper at ICLR 2026
Type D
Type
Type O
Type
Type H2
Type H3
Relations among mechanistic independence types. Arrows indicate logical implications. The dotted arrow holds only in the sparsest product-splitting basis.
Figure 1:
4.2 FACTORS OF VARIATION AS CONNECTED GRAPH COMPONENTS
The factors of variation also be viewed through graph structures.
Definition 10 (Graph structures). Let g: + X be sufficiently smooth, and let = (U1..... Us.) be a basis for TyS. Define the following graphs:
(1) gD (8, B) = ([d.), ED) with
- = = [d.]²] Dga(u:) (u,) # #0 = {(i.)) [d.]3 92, n * #0}.
(2) g (s, B) = ([d,), ( with HH = {(i,j) E [d,] gu; uy) # 0}.
(3) GM (s, B) ([d,], EM) with MM {(i.j) e [d.]² Ω4 # 12
Consider gD. In any product-splitting basis, the index sets C₁ and C₁ for i j are disconnected sub- sets the vertex set. Type D irreducibility ensures that no C₁ can be further split into disconnected components by using different basis for Sr Thus, the Type independent and irreducible factors correspond exactly to the connected components of G Moreover, under the assumptions of Type D independence and irreducibility, GD cannot have more than K connected components in any basis (see Proposition 5}, and in any non-aligned basis it has strictly fewer. Hence. Type D independence and irreducibility could alternatively be characterized by gap the number of con- nected components between aligned and misaligned bases, paralleling the sparsity-gap perspective of Type
A similar statement holds for Ha, If g is second-order separable and satisfies Type H2 indepen- dence and irreducibility, then no basis change increases the number of connected components, and any misaligned basis strictly reduces it.
For GM, no analogous conclusion can drawn, since definition depends on a specific basis. Nevertheless, the identification of factor subspaces with connected components still applies, though only in the standard basis of Idd.
This graph-based perspective also connects recent work on identifiability for local (Euclidean) isometries (Horan et 20011, conformal maps, and orthogonal coordinate transformations (Gresele al. 2021 Buchholz
al. 2022 Ghosh et al. 2023 Each of these function classes can be characterized in terms of their Jacobians: the columns of the Jacobian are mutually orthogonal, differing only in whether they have unit norm, equal norm, arbitrary norms. By analogy with Type D independence, we may define Type O independence through orthogonality in the inner- product sense:
H Digo(u) (D) =.
Constructing graph analogous gD, but replacing the Hadamard product with the inner product, yields totally disconnected graphs for these maps when the source factors one-dimensional.
However, without additional statistical assumptions, identifiability remains limited: even in the smallest class (local isometries), it holds only up affine transformations. Therefore, to achieve the stronger notion identifiability pursued this paper, extra assumptions on the latent distribution required, even for ane-dimensional factors. Nevertheless, such graph constructions may provide a useful tool when combining mechanistic and stochastic independence to recover multidimensional factors.
8

---

Published as a conference paper at ICLR 2026
4.3 APPLICABILITY AND LIMITATIONS OF MECHANISTIC INDEPENDENCE
We illustrate the requirements for Type D/M/S/H, mechanistic independence in the context of image data. Assume that individual latent factors s; encode distinct objects a scene (e.g., position, shape, color), and let g denote the rendering process.
Type D independence fails whenever two latent factors influence the same pixel. This excludes shadows, reflections, transparency, and partial occlusions.
Type H₂ independence fails when the generator cannot be decomposed additively, i.e,, when g(s) * EiE[K] g (s.) for any set of functions g(t), Although this assumption is strictly weaker than Type D independence, it still generally disallows partial occlusions, shadows, and reflections. In principle, it permits cmii-rransparency, but only in the absence of refraction and only when colors mix exactly additively. This condition is further weakened for r > 2, but in practice, the calculation of higher-order derivatives is not feasible.
Type M independence fails when the set of pixels affected by latent coordinate one group strictly contained in the set affected by latent coordinate in another group; for example, when an object is visible solely through its reflection.
Type S independence is more subtle. For one-dimensional slots (i.e., when each object is parame- terized by a single latent variable), can fail only when the fraction shared affected pixels across slots exceeds one half (lower bound). As already mentioned, it is difficult to convey a similarly strong intuition for multidimensional slots.
5 EXPERIMENTS
In an experiment mirroring Brady et al. (2023). we investigated whether the compositional contrast
K
0gn Oz.
Ogn Oz,
Commp (g. 2)
(z)
(z)
can serve as an effective surrogate loss for enforcing Type S independence. This question is moti- vated by the observation that some generators have latent components that are Type S independent but not Type D independent, yet minimizing Cromp can nonetheless enforce Type S independence the learned representation (see Example 2}. argued in Section 3.3] Type independence likely to hold when only a small number of observation dimensions are influenced by multiple latent factors (slots).
To examine this, generate synthetic datasets with varying degrees of overlap between the sets of observation dimensions affected by different slots, as illustrated on the right in Figure 2 Latent variables sampled from standard normal distribution, and observations them through an invertible MLP whose Jacobian is constructed to have the desired support structure. Only when the overlap is 0% does the generator satisfy Type independence.
produced by passing
We train an autoencoder with reconstruction loss and compositional contrast, L Lreeon + XCromp- across five random seeds, using L = K E {2, 3, 5] slots and regularization strengths À E {10-2 2,1) For comparability across hyperparameters,
normalize Coomp (see Appendix (or details).
Figure indicates that, for sufficiently small overlaps, Ccomp acts as a reliable proxy for Type independence. However, as the overlap ratio increases, the likelihood of convergence bad local minima also grows. Identifying more robust surrogate losses remains an open challenge, which we leave for future work. Further experiments can be found in Appendix D
RELATED WORK
Beyond the already mentioned approaches, a number other works establish identifiability by imposing structural constraints, Moran et al. (2021) prove identifiability in sparse VAEs by enforcing sparsity in the decoder; while our framework does not subsume theirs, their synthetic dataset can also be shown to satisfy Theorems and 4 Rhodes & Lee (202)) provide empirical evidence that

---

Published as a conference paper at ICLR 2026
1.0
100
0.8
Compositional Contrast
Slot lddettfiiaably Score
0.6
1/3 Overiap
10-1
0,4
overlap 5% overlap
0.2
20% overlap
-0-2-
50% overlap
0.0
10-5
10-4
10
Jg
Reconstruction Error
Figure 2: Slot Identifiability Score (SIS) over reconstruction loss and compositional contrast for different support overlaps.
penalizing the decoder Jacobian with an mnorr helps break rotational symmetries in VAEs our results can be seen as offering the corresponding theoretical justification. contrast, Lachapelle et al. (2022) obtain identifiability of latent factors by enforcing sparsity causal mechanisms, while Reizinger et al. (203) connect sparsity patterns in the Jacobian identifiable causal graphs in nonlinear ICA.
distinctive aspect of our work is that we establish identifiability at the subspace level, whereas most prior results assume that each latent factor is captured in single dimension. Recent research has also examined -ineiiiiiiiii of latent variables under paired observations. These include content-style separation via data augmentation (Von Kügelgen et al. (02I)) multiple views (Daun-
hawer et al. 2023), block-disentanglemen under sparse perturbations (Fumero et af.) 2021 Ahuja al. 2022 Mansouri al. (023), and temporal formulations leveraging causal graphs (Lachapelle
& Lacoste- Julien 2022 1022:L222a Lachapelle et al. 2024
CONCLUSION
In this work, have developed unifying framework for disentanglement and identifiability based on mechanistic independence. By formulating independence at the level of generative mechanisms rather than distributions, assumptions on the latent density and extend to nonlinear, non-invertible generators. Our analysis revealed a hierarchy of independence criteria ranging from disjointness (Type D) to mutual non- inclusion (Type M) to sparsity (Type S) and higher-order separability (Type H,). We also showed how connected components in graphs naturally characterize the structure of latent factors. Overall, the results establish when disentangled representations are identifiable without relying on statistical assumptions, providing a theoretical foundation for future work that explores other mechanistic independence criteria or combines mechanistic and stochastic assumptions.
obtained identifiability results for subspaces that hold under minimal
REFERENCES
Kartik Ahuja, Jason Hartford, and Yoshua Bengio. Weakly supervised representation learning with sparse perturbations. arXiv preprint arXiv:2206.01101, 101, 2022.
Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. Interventional causal representa- tion learning. In International conference machine learning, pp. 372-407. PMLR, 2023.
10

---

Published as a conference paper at ICLR 2026
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-1828, 2013.
Jack Brady, Roland S Zimmermann, Yash Sharma, Bernhard Schölkopf, Julius von Küigelgen, and Wieland Brendel. Provably learning object-centric representations. arXiv preprint
arXiv:2305.14229, 2023.
Jack Brady, Julius von Kügelgen, Sébastien Lachapelle, Simon Buchholz, Thomas Kipf, and Wieland Brendel. Interaction asymmetry: A general principle for learning composable abstrac- tions. arXiv preprint urXiv:2411.0778. 2024.
Johann Brehmer, Pim De Haan, Phillip Lippe, and Taco S Cohen. Weakly supervised causal repre- sentation learning. Advances in Neural Information Processing Systems,
2022.
Simon Buchholz, Michel Besserve, and Bernhard Schölkopf. Function classes for identifiable non- linear independent component analysis. Advances in Neural Information Processing Systems, 35: 16946-16961, 2022.
J-F Cardoso. Multidimensional independent component analysis. In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP'98 '98 (Cat. No. 98CH36181), volume 4, pp. 1941-1944 IEEE, 1998.
Pierre Comon. Independent component analysis, a new concept? Signal processing, 36(3):287-314, 1994.
Imant Daunhawer, Alice Bizeul, Emanuele Palumbo, Alexander Marx, and Julia E Vogt. Identifia- bility results for multimodal contrastive learning. arXiv preprint arXiv:2303.09160 2023.
Marco Fumero, Luca Cosmo, Simone Melzi, and Emanuele Rodolà. Learning disentangled repre- sentations via product manifold projection. In International conference on machine learning, pp. 3530-3540. PMLR, 2021.
Shubhangi Ghosh, Luigi Gresele, Julius von Kügelgen, Michel Besserve, and Bernhard Schölkopf. Independent mechanism analysis and the manifold hypothesis. arXiv preprint arXiv:2312.1343, 2023.
Anirudh Goyal and Yoshua Bengio. Inductive biases for deep learning of higher-level cognition. Proceedings of the Royal Society A, 478(22266:202100663 2022.
Klaus Greff, Sjoerd Van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial neural networks. arXiv preprint arXiv:2012.05208, 2020.
Luigi Gresele, Paul K Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Schölkopf.
The incomplete rosetta stone problem: Identifiability results for multi-view nonlinear ica. In Uncertuinty in Artificial Intelligence, pp. 217-227. PMLR, 2020.
Luigi Gresele, Julius Von Kügelgen, Vincent Stimper, Bernhard Schölkopf, and Michel Besserve. Independent mechanism analysis, new concept? Advances in neural information processing systems, 34:28233-288 28248, 2021.
Ben Hayes, Charalampos Saitis, and György Fazekas. The responsibility problem in neural networks with unordered targets. arXiv preprint vrXiv:2304.094999 2023.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. In International conference on learning representations, 2017.
Irina Higgins, David Amos, David Pfau, Sehastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. Towards a definition of disentangled representations, arXiv preprint arXiv:1812.02233., 2018.
Daniella Horan, Eitan Richardson, and Yair Weiss. When is unsupervised disentanglement possible? Advances in Neural Information Processing Systems, 34:5150-566,, 2021.

---

Published as a conference paper at ICLR 2026
Aapo Hyvärinen and Patrik Hoyer. Emergence of phase-and shift-invariant features by decomposi- tion of natural images into independent feature subspaces. Neural computation, 12(7):1705-1722) 2000.
Aapo Hyvärinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems, 29, 2016.
Aapo Hyvärinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources.
In Artificial Intelligence and Statistics, pp. 460-469. PMLR, 2017.
Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):411-430 2000.
Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12(3):429 439, 1999.
Aapo Hyvärinen. Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning, In The 22nd International Conference Artificial Intelligence and Statistics, pp. 859-868. PMLR, 2019.
Yibo Jiang and Bryon Aragam. Learning nouparametric latent causal graphs with unknown inter- ventions. Advances in Neural Information Processing Systems, 36:60468-6035 2023.
Christian Jutten and Juha Karhumen. Advances in blind source separation (bss) and independent component analysis (ica) for nonlinear mixtures. International journal of neural systems, 14(05): 267-292, 2004.
Hyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvärinen. Variational autoen- coders and nonlinear ica: A unifying framework. In International Conference on Artificial Intel- ligence and Statistics, pp. 2207-2217. PMLR, 2020a.
Ilyes Khemakhem, Ricardo Monti, Diederik Kingma, and Aapo Hyvärinen. Ice-beem: Identifiable conditional energy-based based deep models based on nonlinear ica. Advances in Neural Information Processing Systems, 33:12768--2777 12778, 2020b.
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, and Bryon Aragam. Identifiability of deep generative models without auxiliary information. Advances in Neural Information Processing Systems, 35:1 1687-1100, 2022.
David Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding. arXiv preprint arXiv:2007. 2020.
Sébastien Lachapelle and Simon Lacoste-Julien. Partial disentanglement via mechanism sparsity. arXiv preprint 2rXiv:220..07732. 2022.
Sébastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, Rémi Le Priol, Alexandre Lacoste, and Simon Lacoste-Julien. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ica. Conference on Causal Learning and Reasoning, pp. 428-484. PMLR, 2022.
Sébastien Lachapelle, Divyat Mahajan, Ioannis Mitliagkas, and Simon Lacoste-Julien. Additive decoders for latent variables identification and partesian-produee extrapolation. arXiv preprint arXiv:2307.025598 2023.
Sébastien Lachapelle, Pau Rodríguez López, Yash Sharma, Katie Everett, Rémi Le Priol, Alexan- dre Lacoste, and Simon Lacoste-Julien. Nonparametric partial disentanglement via mecha- nism sparsity: Sparse actions, interventions and sparse temporal dependencies. arXiv preprint arXiv:2401.04880, 2024.
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. international conference on machine learning, pp. 41 4114-4124 PMLR, 2019.
12

---

Published as a conference paper at ICLR 2026
Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschannen. Weakkyysuuperviee disentanglement without compromises. In International Confer- on Machine Learning, pp. 6348 6359.635,.60 PMLR, 2020.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Confer- ence on Learning Representations, 2017. URL https:// semanticscholaar.
CorpusID:535922
Qi Lyu and Xiao Fu. Identifiability guarrfaebllty-ggaaatteeedd simplex-structune nost-nonlinear mixture learning via autoencoder. IEEE Transactions on Signal Processing, 69:4921-4936, 2021.
Amin Mansouri, Jason Hartford, Yan Zhang, and Yoshua Bengio. Object-centric architectures en- able efficient causal representation learning. arXiv preprint arXiv:2310.19054, 2023.
Stefan Matthes, Zhiwei Han, and Hao Shen. Towards unified framework contrastive learning for disentangled representations. Advances in Neural Information Processing Systems, 36:67459- 67470, 2023.
Gemma E Moran, Dhanya Sridhar, Yixin Wang, and David M Blei. Identifiable deep generative models via sparse decoding. arXiv preprint arXiv:21 10..:::110..00000 10804, 2021.
Ignavier Ng, Shaoan Xie, Xinshuai Dong, Peter Spirtes, and Kun Zhang. Causal representation learning from general environments under nonparametric mixing. In The 28th International Con- ference on Artificial Intelligence and Statistics, 2025.
Hoang-Son Nguyen and Xiao Fu. Diverse influence component analysis: A geometric approach to nonlinear mixture identifiability. arXiv preprint arXiv:2510.17044 2025.
Patrik Reizinger, Yash Sharma, Matthias Bethge, Bernhard Schölkopf, Ferene Huszár, and Wieland Brendel. Jacobian-based causal discovery with nonlinear ica. Transactions on Machine Learning Research, 2023.
Travers Rhodes and Daniel Lee. Local disentanglement in variational auto-encoders using jacobian 1.1 regularization. Advances in Neural Information Processing Systems, 34:22708-227199 2021.
Bernhard Schölkopf. Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612-65) 2021.
Anisse Taleb and Christian Jutten. Source separation in post-nonlinear mixtures. IEEE Transactions signal Processing, 47(10):2807-282) 1999.
Julius Von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf. Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances in neural information processing systems, 34:1 16451-16467, 2021.
Dingling Yao, Danru Xu, Sébastien Lachapelle, Sara Magliacane, Perouz Taslakian, Georg Martius, Julius von Kügelgen, and Francesco Locatello. Multi-view causal representation learning with partial observability, arXiv preprint arXiv:2311.06m 1.04666, 2023.
Jiaqi Zhang, Kristjan Greenewald, Chandler Squires, Akash Srivastava, Karthikeyan Shanmugam, and Caroline Uhler. Identifiability guarantees for causal disentanglement from soft interventions. Advances in Neural Information Processing Systems, 36, 2024.
Kun Zhang and Aapo Hyvärinen. On the identifiability of the post-nonlinear causal model. arXiv preprint arXiv: 1205. 2299. 2012.
Kun Zhang, Laiwan Chan, et al. Minimal nonlinear distortion principle for nonlinear independent component analysis. Journal of Machine Learning Research, 9885)2245-22877 2008.
Yan Zhang, Jonathon Hare, and Adam Prügel-Benneti Bennett. Fspool: Learning set representations with featurewise sort pooling. arXiv preprint arXiv: 1906.02795.... 2019.
13

---

Published as a conference paper at ICLR 2026
Yujia Zheng and Kun Zhang. Generalizing nonlinear ica beyond structural sparsity. Advances in Neural Information Processing Systems, 0011336613333 2023.
Yujia Zheng, Ignavier Ng, and Kun Zhang. On the identifiability of nonlinear ica: Sparsity and beyond. Advances in neural information processing systems, 35:16411-16422 2022.
Roland Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive learning inverts the data generating process. In International Conference on Machine Learning. pp. 19979-12990. PMLR, 2021.
14

---

Published as a conference paper at ICLR 2026
NOTATION INDEX
a
A scalar
4
The i-th canonical inclusion map
A vector
K Number of latent factors
A
A matrix
L
Number of factors in learned representation
A
A set
L,
Elimination matrix for matrices A probability distribution
i-th coordinate of (index starting at i-th factor a a lives in product space aij j-th coordinate of the i-th factor of a
dj
IP
a,
8
Ground truth latent variable
S
Ground-truth latent space
dr Dimensionality of observations
S
i-th latent subspace (S S S1 x.... Sx) Support of matrix (index set of nonzero elements) or support of a func- tion (set of elements mapped values not equal to zero)
d, Dimensionality of ground-truth latents supp(.)
d, Dimensionality of the i-th latent factor
d₂ Dimensionality the learned representa- tion
D Duplication matrix for matrices Differential of g
T.S
Tangent space of S at
Dg.
V
Mapping from learned to ground-truth la- tents
D.ga
Partial derivative w.r.t. the i-th factor
Observation or measurement
Dg, 04
Di,99 Mixed derivative Dags (4, id)
X
Data manifold (a: E X' C (R))
e,
Standard basis vector with at position
Learned representation (or encoding)
f Ground-truth encoder
Z
Learned representation space
f Learned encoder
X
Direct product
+ Direct sum
Ground-truth decoder
Hadamard product (element- wise product)
g Learned decoder
Kronecker product
G
EV.E) A graph G defined by set of ver-
tices V and edges
Row-wise Kronecker product (also face- splitting product)
Mapping from ground-truth to learned la- tents
Set subtraction
Identity matrix with implied size from con-
rh
Mutual non-inclusion (A B A B)
text
Al
Cardinality of set A (the number of ele- ments in AA)
I,
Identity matrix of size X
[n]
The set 21.2...., n} for E
Jf Jacobian matrix of f R. + I Th (Jf g
f fog
Composition of the functions f and
||=]oo
Eg norm
15

---

Published as a conference paper at ICLR 2026
A PROOFS
Before we turn to the theorems and proofs, let us recall the following definitions.
Definition 11 (Decomposable map). Let 8 K St and Z 11 Zj. We say that
map h: S + 2 is decomposable if there exists surjection [K] [L] and maps
h,: Ilico S + 2, such that, for all E,
h((s) (hi((S))iez 16)))
Definition 12 (Disentanglement), A decoder 9: 2 > X is said to be disentangled w.r. generator g: X there exists a decomposable map h: S - 2 such that g = 9 o h.
Remark (Partial/full and local/global disentanglement). If K and a is bijection (i.e., local full disentanglement), Defn. 12 gives
g g( (11))
To distinguish the cases gled, respectively.
K from L K, we say 9 is fully disentangled or partially disentan-
Definition 13 (Local disentanglement). A decoder 9: 2 X is locally disentangled w.r.t. a erator g: + X for every 8* E S and z² E with g(s") g(=*) there exist a neighborhood U C S of s" and a decomposable map h: U + Z such that
9lu = g
and h(s") z"
Definition 14 (k-factor slice). Let (O...., K}. and let I C [K] be an index with [Z] K k. S is subset of the product space S₁ x ... x Sk, k-factor slice (or simply k-slice) of any set of the form
u = {s E S
C. for all i I).
where e E St i E I are fixed constants.
Put simply, k-slice a subspace in which all but k factors are held constant.
Remark 2. Puth-comnectednes of C II S; and path-connectednes. of its (K 1)-slices are related but independent properties: neither one implies the other (see Figure 3). More generally, for K >> 2, connectedness of 1-slices and 2-slices are likewise independent (for K they coincide trivially). ). A further related notion is orthogonal convexity, which can be interpreted as the property that all 1-slices are path-connected (when each factor is one-dimensional)
(a)
(b)
(c)
(d)
Figure 3: Examples illustrating independence slice- and set-level connectedness. (a) path- connected. but not every 1-slice is connected. (b) S is not path-connected, though every 1-slice is connected. (c) Some 1-slices are disconnected, but every 2-slice connected. (d) Some 2-slices are disconnected, but every 1-slice is connected.
16

---

Published as a conference paper at ICLR 2026
A.1 PROOF OF THEOREM 1
Lemma 1. Lei be an open subspace of the product manifold II Si with each factor St posi- live dimension. Suppose 9: 2 - X locally disentangled w.r.t g S + X. If is locally injective and S is path-connected, then the surjection a from the definition of disentanglement (Defn. 3 is
globally unique.
Proof. The proof proceeds two steps. First, we show that the surjection from the definition of disentanglement is unique on sufficiently small neighborhoods, using local injectivity of g. In the second step, we extend this uniqueness to all of S by path-connectednes.
Step 1. The surjection a is locally unique.
Let C be open such that g u injective and 9 disentangled with respect to g llr- Then there exist a surjection
[K] [L] and a map h: u + Z that decomposes into
hy: :
II S. - Zz. E [L] iEO-1(5)
such that for all s E U,
g(s) = of hi( (()( (r((::))
(9)
Let V = -()). From Equation 9 it follows that both h and g) are injective.
Now suppose that for the same g. 9 another representation on exists with a different surjection ō. Fix any E [K] and basepoint U. Consider the one-factor slice
ut {s e:::: = P, for all j i).
Since (S,) > O, (1) contains at least two distinct points. By Equation 9 variation along ((()) affects exactly the component indexed by (i). If (i)) * (i)), then the same variation would be forced to appear two different components. Thus, on the right side of Equation 2(() mapped to different sets for a and o, while on the left side g maps 21(3) to the same set independently a. Therefore, ā(i) (i). Since was arbitrary, we get 0 on ld
Step The surjection a is globally unique.
Let 8a 8¹ S and let 7 [0, 1] - S be continuous path between them. By Step 1, every point 8 E <(0, 1J) admits a neighborhood U, on which is uniquely determined. The compact >(0, 1])}. By compactness, there exists finite subcover
set ((0, 1]) is covered by (Ug U1 M4....,,yy
Using the Lebesgue number lemma, choose partition
0= lo < <1 ... < LM = such that ((m C Um for each m.
Then 7(tm) E Um n num+11 so consecutive sets intersect. By Step 1, is unique on each Um and therefore must agree on overlaps. Induction along the chain implies that the same a applies to UM Um 2 7(0, 1)). Since s", 8 were arbitrary and is path-connected, there exists a single global surjection [] + [L] valid all S.
Theorem (Global Identifiability). Let S be an open subspace of the product manifold II S, where each factor S, has positive dimension. Then local disentanglement extends to global disen- tanglement if:
(1) S ++X X' is locally injective.
(2) S is path-connected.
(3) Every (--I)-slice of S is patt-connected.
17

---

Published as a conference paper at ICLR 2026
Proof. From Lemma it follows that there is unique surjection [K] + [L] such that locally, for all j E [L]. 2, depends only on the source components 8, with i (j).
Now fix E [L] and tuple 1G) I1 St Consider the slice
A ()) = { E 8, = 8, for all i E (i)-
This slice is path-connected since by assumption all (K- (K-I)-slices of S are path-connected. Along any path in A (s.-1))) local disentangled representations agree on overlaps (see Step 2 in Lemma D. and the j-th component remains constant since only coordinates outside o (j) vary. Thus the j-th component well defined on the slice.
Therefore,
define
h
II S. S, iEa iEr-L(j)
where (5, (()) is the common value the j-th target component on (8.-1))
Finally, fix E S and choose U open such that glu is injective and I disentangled with respect to 9 u- On U, local representation has the form
g(s) of h1 hi((s)) 1ay), ..... hz(((s)veo- -w))
By construction of h, for all s U the local maps h, agree with h.j Hence
g hı h1 hh. h=,
SEu.
Since was arbitrary, this identity holds globally. Thus local disentanglement extends to global disentangled representation with surjection and maps (h, [
Remark If T. K, not all (K-1) (--I)sslices need to be path-connected. suffices that only the slices corresponding to indices mapped to common target component are path-connected.
A.2 PROOF OF PROPOSITION
Lemma 2. Let S S IT S S I Zj. and suppose g: + X and g: 2 + are local homeomorphisms, Assume that for every 8* E S there exists x" E Z with g(s)) ) g(z"). Moreover, suppose that for each such z" there exist
neighborhood L S of2² 2"
surjection a: [K] + [T.], and
maps v:: Zo(() for i [K].
such that for all x E U,
g(z) u (zo (x()))),,,,,(((()))) (x(x((K))
(10)
Then g is locally disentangled with respect to g.
Proof. Fix an arbitrary E S and pick a neighborhood = 11 Ug C Z, a surjection o, and maps v, giving Equation 10 on 24
with g(=*) = gs*). By hypothesis at =", there is
Shrink to a neighborhood W C S of s* on which g: W + g(W) is homeomorphism, and shrink UI if necessary so that ((4) C gWV). Define
9 U - W. Then db is homeomorphism onto its image with v0(2)) s"
/b
For each j E [L] set
Φ
Ug
II S;
0,(a) (v:(a)) 16)
iEa 1()
18

---

Published as a conference paper at ICLR 2026
Then for 4 Equation 10 is equivalent to
ex({(=))
(1I)
where &a a reindexing homeomorphism + ((0)))E--)))))) Therefore, each Φ must be injective, because the left hand side of Equation is homeomorphism onto its image.
Since (U)) open neighborhood of s" in the product space II; Si, we can choose product neighborhoods Vi C S with
I S ((T).
i=1
Then for each j we have Ilio 1(1) Vi S Φ (U,), and we set
h = $5 11co-1o0 n
II V; Uj. VEO-1())
Finally, for any II. V,, define 2 = (()((())) Then, by construction and Equa-
tion 11 + (8) hence
g(s) g(z) ( h
Therefore, 9 locally disentangled with respect to on a neighborhood the arbitrary point 8* S.
Proposition Let g: > X and 9: 2 - X be surjective local homeomorphisms, where and Z are open subsets of their respective product spaces. Then local full disentanglement defines equivalence relation ~1d 9.
Proof. We verify that the relation is reflexive, transitive and symmetric.
Reflexivity: If g = 9. we can set each h; as the identity map and take a as the identity permutation. Then the definition is trivially satisfied.
Transitivity: Follows directly from composition of functions. ~ia 9 via hi, a and ğ ~ia via h, 5. then g ~id via h h3-1()), 7 da.
Symmetry: Follows from Lemma
Proposition Let II S, and Z C II Z, be open, and let g: - X and 9: Z - X be surjective. Then disentanglement defines equivalence relation g ~ 9 if one the following conditions hold:
(1) g and 9 are bijective and (equivalently Z) itself a product space.
(2) g and 9 are locally injective and every -K-lie S and is path-connected.
Proof. The proof is analog to Proposition 1
A.3 PROOF OF THEOREM 2
Definition 15 (Mechanistic Independence of Type D). We say that S; and S, (equivalently, s; and 8j) are mechanistically independent of Type D if. for all E S, & E S, and E Ta Sy
Dig.(E) D = 0,
(12)
where denotes the element-wise (Hadamard) product in Rd
Independence of the 2, is analogously defined based on ğ.
19

---

Published as a conference paper at ICLR 2026
Definition 16 (Reducibility of Type D). We say that S, is reducible of Type D if there exists E S such that Te, St admits a nontrivial direct-sum sum decomposition Te, S; + with the property that, for all & U and V.
D5g=(E) Dig,(n) = 0.
(13)
no such decomposition exists, we call S, irreducible of Type D.
Lemma 3. Let (2, and { be smooth manifolds of positive dimension with I. S K, and
let Z _ 2, x 3g and S C St x ... x Sk open subsets. Suppose 2 S
diffeomorphism such that for every E Z there exists surjection 0z [K] + [4] satisfying
TT v)= 0, for all i E [K], j oz)),
where n;: S S S; denotes a canonical projection. Then for every z E Z there exists a neighbor- hood U of z such that oe' o² for all z' E L1 and moreover v,(2') depends only on the component Zati) for each i E [K].
Proof. At each 2 E Z, the differential Du, has block form
Du,
①
S z. j=1
2z.: 2,
k (
Since v is a diffeomorphism, Du, is an isomorphism. Hence each block Øz.j must also be an isomorphism, and in particular
dim(
Σ iE (
dim(S.).
The maps = (TT o v)2 vary smoothly with Thus, if d æz.j is an isomorphism at remains so in neighborhood of z, since invertibility is an open condition. This implies oz'(i) (j) for all j E [Z] as we assumed the S; have positive dimension. Because each σ2' is surjective, we must have σz' a= in neighborhood U of z.
or (f)
As 2 is open in the product manifold, we may shrink so that U1 ... x4r with each M4 path- connected. Fix E [K] and let å U satisfy žo(i) =a(s)- Choose smooth path 7 [0, 1] U with (0) and (11 Σ. By the fundamental theorem of calculus,
(; (=) = S.'
d di (x(((() dt.
By the chain rule,
di (x(((()) D(T, o 00) (t)
- to()()+
Σ Dj(T, o) (t).). (+
The first term vanishes because (x())) (t) is constant, and the second vanishes by the structural as- sumption Dv. Thus the integral is zero, and we conclude v2(2) (2). Hence v, depends only on the coordinate Zo((), completing the proof.
Theorem 2 (Local Identifiability of Type D). Let g: S + X and g: Z - X be local diffeomor- phisms 6 with gSS) C (Z). Then 9 locally disentangled w.r.t g if:
(1) S IT S, is open, and all factors are Type D independent and irreducible.
(2) 2 I is open with L < K, and the factors are independent of Type D.
A diffeomorphism is smooth bijection between manifolds with smooth inverse. A local diffcomorphism map that restricts to diffcomorphism some neighhorhood of each point.
20

---

Published as a conference paper at ICLR 2026
Proof. Fix arbitrary point s' E S. By the range assumption g(S) 9(2), there exists least one x= E such that
g(s")= 9(²))
Since both g and 9 are assumed to be local diffeomorphisms, there exists a neighborhood 24 S of z" such that, for all U.
g(z) u(2),
(14)
where we define
w= g uu U + (g g))
and g denotes the local inverse satisfying v(=") s" Differentiating gives
Dg Dz)(z) Dv>
(15)
To obtain matrix representations, choose product-aligned bases on Te( (i), and (1 2,), and identify x and T (((())) with their natural inclusions into Rdd,
By Type D independence for g. the row supports of the partial derivatives Dig. and Djgs are disjoint whenever * j. Thus there is a partition of observation coordinates [dz] = R Rx such that rows in R. depend only S, Permuting rows by P to group R....., R consecutively makes
Dgu(=) block-row diagonal. Set
A = P Dgw(z)
B = Duz
C=PDD
So that C AB.
For E [L], let B denote the block-columns note the block-columns corresponding + corresponding block-columns of C. Then
corresponding to Tep Zu. and let B.. de- Dzzk Tay 2j. Define C. and C, analogously as the
A1.1 0
0
B1,
B
A
0
B
B
CC
C.,+l =
(16)
A Ak,
BR
BR,-
By Type D independence for 9, the column supports of from different target slots are disjoint in observation coordinates, which is preserved by left-multiplication with P. Hence the supports of the columns C. are disjoint from those of C., so all pairwise Hadamard products between them vanish. Denoting the Kronecker product by @ and the row-wise Kronecker product (also known as the face-splitting product) by O we obtain
0 C...& C.,-k
= (AB..) O (A
(AO A) B. B..
B 3 B1.-k B2, & B2.-
= [A., O A.
2, A
A, 2 A.
BK. & Bk.-k
(Ai.1 O AI.)
B & @ Bi,-k) B2 B2.
(A2. A2.2)
(AK.K @ AK.K) (BK. @ Bk.-k)
Here, the third equality uses the mixed-product property, the fourth expands and reorders terms, and the last exploits the block-diagonal structure of A. Reversing the mixed- -roxuutt property yields, for all i [K] and & [L].
(A,, 2 (A,,
(17)
Suppose, for contradiction, that both B., and B nonzero. Since v is a composition of diffeomorphisms, is invertible and each B., has full row rank. Let us consider two cases (note that (S,) = 0 and dim( Z,) were categorically excluded in advance):
21

---

Published as a conference paper at ICLR 2026
Case I (S) 1). Here B,: consists of single Choose nonzero scalars Bi and b E Bi,-k. From Equation 17
(A1,5a) ④ (A,,b) = 0,
which implies A, = 0, contradicting the assumption that g is a local diffcomorphism..
Case 2 (S, 1). In this case, select columns from B and B that together form an invertible square matrix (Bi, B.), with B, consisting of columns of Bi and B, of B Then Equation 17 gives
(A, Bi (A,, =.
This implies that S; is reducible, since there exists a basis in which Te, S; decomposes into subspaces where all pairwise directional derivatives vanish in the Hadamard product. Hence, either Bi, k or B must be zero.
Repeating the argument for all [K] and & [L] shows that each block-row of B contains most one nonzero block. Since B is invertible, each block-row must contain exactly one nonzero block. Hence, there exists surjection σ: [K] - [L] such that
(at * and B for j + o(i).
By Lemma it follows that on U, the component vz(2) depends only on za(1) for every i E [K]. Equivalently, there exist functions
ú,: Zo(1) - S
such that locally
g 9 g(z)) 01 (=o(1)), .... ëx (Zo(K))).
Since s* was arbitrary and the constructions hold for any x" satisfying g(8") = =g(z"), Lemma implies that ğ is locally disentangled with respect to g.
A.4 PROOF OF THEOREM 3
Denote with (2)ss) C [da] the support of the i-th column of (s) Dga: Rda Rdd in the standard basis (i.e., (s) = (s))(J)(s)..),. Similarly, we use (2) for (z) Let C, denote the column index set of the i-th source factor.
For sets A, [ [m], write A rh iff & and A 2 B (mutual non non-inclusion).
Definition 17 (Mechanistic Independence of Type M). We say that S and Sj are mechanistically independent of Type M if. for every s E S,
Va E C, Ab e;: Ps(8) th (s(s).
Definition 18 (Reducibility of Type M). We say that the component S, is reducible of Type M if there exist a point 8 E S and a partition C, AUR such that
Va E A. AbE
(a(ss (s)
Lemma 4. Let AB, where A Ruxn E Raxn and C E Rmxn are all of full column rank. Define G = ([n), ES) with E (i..) E [n] supp(A..)) Hi (upp(A..)); If
||C|lo I |Allo and for all l E [n]
supp(C,,) 2
U
supp(A.))
(18)
iEsupp(B..) B..)
then |Cllo ||Allo and (C) is isomorphic to (AA).
Proof. Write 2, = supp(A A. 1.), R₂ Bu and U4 = supp(C.,k) Since C.
EiER, Bi, we have 24 C U 2i, while Equation 18 gives the reverse inclusion; hence
Uu
Vk [n].
iER
22

---

Published as a conference paper at ICLR 2026
Because B is invertible, the Leibniz formula for det(B) # yields permutation [n] + [n] with (i))()) * 0 for all i, ie., i E Ro(i). Thus
Q Ho
e [n].
Summing sizes and using ||C|o I
||A|lo
E - Σ Mo(i)
Σ U4 = | S ||lloo 1lo = Σ 12
so equality holds throughout, which forces (o) = | and hence Mo(s) - Q for all i. This says the column supports of C are exactly those of up to a relabelling of indices. Since the edge relation in (-) depends only on mutual non-in -on-inclusion of these supports, the bijection + o(i) preserves adjacency:
Q = Uo(s) th Mo())
Hence (C) = (A).
Theorem (Local Identifiability of Type M). Let g: S X and g: 2 X be local diffeomor- phisms with g(S) C g( Then ÿ is locally disentangled w.r.t. if:
(I) S S Rdd open, and the factors are Type independent and irreducible.
(2) S Rd is open, and the factors are independent of Type M.
(3) For all E S and z E Z with g(s) = =]()), ||Jg(=)llo s |J
(5)
(4) For all such pairs,
ix(z)=
U
2,(s),
(6)
iEsupp(B..)
where B = (z))og(z) and fg mirrors Q4 for g.
Proof. As before, we begin with the identity
go
defined on neighborhood / C 2, where
= g 9 b U + ag (()
diffeomorphism that maps unique z" l some initially chosen arbitrary point s" E S. Thus, after differentiation we get
(z(z)= = J
which we write C AB. Since both g and ğ are local diffeomorphisms into the same observa- tion manifold, B is square and invertible, and A, C' have full column rank.
Let R, C [d,] be the column-index set in the i-th source block, and define C, [d.] analogously for the target blocks. Then (R, partitions the columns of A and (C.) partitions the columns of C.
Step 1. Each column of B has support contained in a single source block.
Suppose not: then for some column index k. the support supp(B..) intersects distinct blocks R2 * Ry. By independence of the S,, B would mix mutually non-inclusive column supports of A. Thus, Equation[))
force a strict increase in the support, which contradicts the assumption that |[C| ||Aloo. Hence supp( B..). C R; for some Define Q; := {g: supp(B.,)) C Ri), i... the column set of B supported in R;,
Step 2. For each the columns of supported in R₁ land in single target block.
23

---

Published as a conference paper at ICLR 2026
Assume otherwise: then Qi meets two distinct C-blocks Ca and Cg. Pick qa Ca n Q and da E Cg n 2, By Lemma there are Uo and ug such that supp(C..an .g. = supp(A,,.. and supp(C..a) supp(A By Equation 6 for every & supp(B..or C Ri,
supp (...) supp(C.
U
supp (A..)) 2 supp(A..))
(19)
jEsupp( (..))
This implies ua R2 due independence of the source factors, If Ua were not in R, then (upp(A.uu) would contain a column support from different block by Equation 19 Analogously, we get Ug E R.
If Ua = ug then supp( = supp(C contradicting independence of the target blocks. Thus ua U..
Define g with vertex R, and edge set & {(a,b) R2 R2 supp(A..)) (A.)() By irreducibility of S,, G is connected. Thus, there is path vaa 1o: v1:-.., Ur ug with each consecutive pair comparable (i.c., either supp(A,,,) S supp(A,...n)) or supp(A,..) 2 supp(A,, e(+1)) )) Let p be the first index where the image of Up (in C)) leaves C., Then Up-1 and Up are comparable but land in different C-blocks, giving containment across C'-blocks. This contra- dicts independence of the target factors. Therefore, for each all columns of B supported in R; belong to single target block. Since B is invertible, repeating the argument for all i E [K] shows that each block-row of B contains exactly one nonzero block.
Finally, Lemmas 3 and respect to g.
(as in the proof of Theorem imply that is locally disentangled with 2
Proposition 3. Let A E mmx. For k E [m]. write Ra == supp((..... S [m]] and i E [m]] write C, == supp(Ar.) S [n]. The following are equivalent:
(1) (Mutual non-inclusiveness) For all lo + e, RR dh Ra (or equivalently, neither RR S Re nor R₂ C Rx).
(2) For every k E [n],
(A) n C..
iE
Proof. Fix lt E [n]. Observe the identity
{j E [n] : R₂ S R2 } = {j [n] jEC,vi R+
= E [n] Aij + E R+)
0 C..
RR
Thus (2) is equivalent {k} = {j Rr S R2J. That is, the only column whose support contains R2 is & itself. This rules out R2 C R2 for any j + k, and by symmetry across pairs (k, {) yields (1).
Conversely, if (1) holds, then for each & there is no j + k with Rr S Rg. So by the above identity we get NiER. C {k}, which is (2).
Remark 4. Under the usual convention that N.cE C, = [m], both conditions in Proposition 3 forbid zero columns (unless
1, in which case both true regardless if the column contains nonzero
elements or nol).
Proposition 4. Type M identifiability generalizes Theorem 3.1 from Zheng Zhang 2023
Proof. We will show that the assumptions of Theorem 3.1 in Zheng & Zhang 2023 imply the
assumptions of Theorem 3 when we pick S; = IR.
Zheng & Zhang 2023 show that condition (i) in Theorem 3.1 implies Equation 14 in their ap- pendix (v(i.))
(i T3 t F), which can be reformulated as Equation 6 Furthermore,
24

---

Published as a conference paper at ICLR 2026
Proposition Setiriooiieesttalsishhhhh that structural sparsity (condition (ii) in Theorem 3.1) is equivalent to mu- tual non-inclusion. Thus, structural sparsity implies Type M independence of the source factors. The sparsity gap (Equation[5) [5) is not explicitly listed in Theorem 3.1 but required throughout their entire work. Finally, for one-dimensional factors, Type M irreducibility vacuously true, and by 4 M independence of the target factors holds automatically.
A.5 PROOF OF THEOREM 4
For s E S, denote by Pig (s) the minimal fo-norm (i.e. the number of nonzero entries) of the matrix representing Dgs: : T,S -> a when expressed basis of TyS that is aligned with the de- composition and the canonical basis of g induced by its embedding in R Conversely, define Pa (s) as the infimum of the 4o-norm of Dg. taken over all choices of basis of T&S that do not respect the decomposition B. Analogously, we define Piz, (s) and Prs, (s) based on Dige, where 2; is a decomposition T St.
Definition 19 (Mechanistic Independence of Type S), We say that the subspaces S; mechanisti- cally independent Type if. for every & E S,
Pig (s) < (s)( where B =
sE|K]
Definition 20 (Reducibility of Type S). We say that the component St is reducible of Type S there exist 8 E S and a nontrivial decomposition S = Φ V B; such that
Pos. (s) < Pzz (s).
Otherwise, we call S; irreducible of Type S.
Theorem (Local Identifiability Type S). Let g : - X and 9 Z + X be local diffeomor- phisms with gS)) C (Z). Then ğ is locally disentangled w.r.t. if:
(1) S _ I S, is open, and the factors S₁ are Type independent and irreducible.
(2) 2 C II Z open with L S. K, and the factors Z, are independent of Type S.
Proof. On neighborhood U C define the diffeomorphism
v g 9 uu U + (g ())
so that و = g o on U. Hence
Dg= Dgv(z) DUz. Fix product-splitting bases for (o III S,) and T2 (II, 2,) that minimize the fu-sparsity of Dgo(z)
(20)
and Dg, respectively. In these bases, write Equation 20 as C = AB. Since both and local diffcomorphisms into the same observation manifold, B is square and invertible. Let R2 S [d.] be the column-index spanning Sr and define C₁ C [d.] analogously for Tzy Zj-
Step Each column of B has support contained in a single source block.
Suppase not: then for some column index k, the support supp( B..k) intersects distinct blocks R2 * Ryy- By independence of the S, any basis change of Dg, that mixes coordinates from different source blocks worsens the so-ssiiyy after multiplication. Equivalently,
|| ||AB% ||C||
This contradicts the assumption that the chosen basis for Dÿz is fo-minimal, since independence of the Z, implies that the lowest nn-mr is achieved in product-splitting basis (up reordering of the basis vectors). Hence supp(B. x)) S R; for some i.
Step 2. For each i, the columns of B supported in R, land in single target block.
Assume otherwise: then there exists E [K] and columns E Cs and q E C_x == Ujya C, such that both B., and B.g are supported in R,, Now consider two cases (with dim dim(S,) = and
dim (2,) = 0 excluded priori):
25

---

Published as a conference paper at ICLR 2026
Case (dim(S,) = 1). Then R; {r) and for nonzero scalars Br.p- Br, have supp(C.,p) = ,,, = supp(A.,,,ry = supp C..)).
However, a necessary requirement for independence of the target factors supp(C..p) rh supp(C:g):
since otherwise cross-block mixing can be constructed involving C.p and C..g which leaves the overall support unchanged. This contradicts the earlier result that suppi C..)) = (upp((C.g).
Case (S:) > 1). The full row rank of BR... yields an invertible square submatrix B formed from columns in Cs and C_ such that
R.R, B [Ai Az),
where A₁ and Ag submatrices of C,C, and C. respectively. By independence of the 2,.
(z) P (2), where
B
T2 2,
LE[L]
This forces
/ (z) = ||C||o = || + (z) S
inf G$ respecii))))
A2] +c,
where is the number of nonzero entries of C outside [Ai, A2). Since C' has minimal support, there is no basis transformation reducing the no-morm of A, or A2 individually. Thus
Pb. (v(2)) - |A, B 1110-
inf G# (bbckkrssssp)))))ggg
||| A2] Psz, (v(z)),
contradicting irreducibility of S.
Hence, for each i, all columns of B supported in R2 belong to single target block. Repeating the argument for all E [K] shows that each block-row of contains exactly one nonzero block (since B is invertible).
Finally, Lemmas and (as in the proof of Theorem 2 imply that 9 is locally disentangled with respect g.
A.6 PROOF OF THEOREM 5
Definition 21 (Mechanistic Independence of Type H,). Let S S I. S; be a smooth manifold, and let g: X be of class CT with S, and S, are said to be mechanistically independent of Type H,, if. for all 8 E S,
Dr,9.=0 0.
(21)
Definition 22 (Reducibility of Type H.)). We say that the component S; is reducible of Type H, if there exists E S such that either D = 0 or there exists a nontrivial splitring Sr = U + V
such that for all E E U, V, and Ç. T,S for k E [n - 2],
De,g.(E n. S........ Sn-2)) (0-2)=00 =0
(22)
Definition 23 (Separability of n-th Order). We say that g: S X is separable of order if there exists E S such that, for all i E [K], the image of D intersects trivially with
span (P j + D D*g..
ga. 1 1 1sksn 1kknn
Lemma 5. Let V be fiitte-imensioonal vector space with dim(V) 2 2, and suppose W WI,..., W
with 2 2 are subspaces of V such that W1 +... + Wn. = V. Assume that there exist indices i that satisfy W, + (0) and W, {0). Then there exist nonzero subspaces U1 and U2 of V such that
V = U1 ( U2.
with U W; and U2 C Ek Wt-
26

---

Published as a conference paper at ICLR 2026
Proof. Set C' == Ex Wi and Vo = W; C.
Then choose complements
W; = Vo + Vi
and C= Vo + 1/2
for some subspaces V1 S W, and V/2 - C. Then
= W; + C (V, @ Vi) + (Vo @ V22 = Vo Vi V/2
and the sum is direct because V n 1/2 - {0) and V n (Vi + 1/2 - {0).
We now choose U1 and U2 case by case.
Case Vi + {0) and 1/2 + {0}. Set U1 == Vi S W, and U2 == Vo e V/2 C C.
Then
U1 @ U2 V1 @ (Vo @ V2) V, and both U1. U2 nonzero.
Case 2: V/2 * {0) and V22 = {0). Then C Vo and, since W, C C with W, * {0) we have
Vo + {0). Set U1 = Vi C W; and U2 == Vo C C. Again U1 @ U2 = Vi @ Vo = V, with both nonzero.
Case 3: 1/2 {0) and V2 f {0]. Then W, = Vo. hence Vo + {0) because W * (}) Set
U1 Vo [ W, and U2 1/2 C. We have U1 U2 Vo V22 = V. both nonzero.
Case 4: VI = {0} and V22 = (0). Then W; = C = Vo- particular W, = C = V. Since
dim V) > 2. choose decomposition V A B with A, B o {0). Taking U1 = A W; and
U2 = B C C yields the claim.
In all cases obtain nonzero subspaces U1 S W, and U2 S C Exss We with V required.
U1 + U2.
Theorem 5 (Local Identifiability of Type H,). Let g: S + X and g: 2 + X be local C".
diffeomorphisms with n 2 2 satisfying gSS) (Z)). Then 9 locally disentangled w.r.t g if:
(1) S IT S, is open, and the factors are Type Hn independent and irreducible.
(2) _ II Z is open with L S K, and the factors are independent of Type Hn.
(3) g is separable order
Proof. Let 8* E S be arbitrary, and choose z' E Z such that
g(s") = g(z").
Since g and و are local diffeomorphisms, there exists a neighborhood U Z of x" on which may write
9 = 9 u
where
1. = g o gguu: u - (g (2f satisfies v(=*) 8"
Fix 2 2. For E U, the higher-order chain rule gives
D"
Σ Di Dimlgu(z) gu(z) Dl BEX*
(23)
*EP((N])
where P([n] denotes the set of partitions of [I....,n).
On the left-hand side of Equation 23 mechanistic independence of the Z, implies that all mixed derivatives ğ vanish:
ワū =
i {
Now restrict Equation 23 to this mixed derivative and consider the right-hand side. Mechanistic independence of the S, implies that the highest-order term (corresponding to f1..... n)) can be split up, and all mixed derivatives D (m(()(()) vanish:
D go(z)) (Diuz, D,v2, Duz...., Dv2)
Σ RE(K]
D DDAA (T Dvz. ......... Duz)
n- 2
2 times
27

---

we then have, Published as a conference paper at ICLR 2026
where vy denotes the projection onto the k-th slot.
By separability (Defn. 23). the image of (.o(()))) intersects the images of all other derivative terms on the right-hand side Equation 23 only zero. Hence they cannot cancel and each individual term in the sum must be zero. Therefore, for each k E [K], we obtain
1 (Di( o D) o v)x DUz...., DU, O.
(24)
Now assume, for contradiction, that there exist E 22 and B T2; 2, such that (TT (T ()))(()) and D 0)= + 0.
We distinguish two cases (recall that (S,) = and dim( = 0 were excluded by assumption):
Case 1; (S) 1. Then Equation 24
implies D (,, contradicting irreducibility.
Case 2: dim Sx)) > 1. Define
W; im (D: o v)=).
Since v is a composition local diffeomorphisms, D(T,, v) is surjective, hence
(x) = W1 +... Wz.
By Lemma
decompose
T (e = U1 U2 with nontrivial tangent subspaces U1 C Wi and U2 S Ejji i Wj. From Equation 24 for all & E U1 and E U2.
1 (E. n, Si...., Sn-2) 0, where Çe E S are arbitrary. This implies that S₃ is reducible, contradiction.
Therefore, for each & E [K] there at most one E [4] such that
Dj Di(TR o v)_ 0.
Since Dv, is isomorphism, least such must exist. Applying Lemmas and 2 in the proof of Theorem we obtain surjection a: [K] + [L] with the disentanglement property.
Hence g is locally disentangled with respect g.
A.7 PROOFS GRAPH-THEORETICA RELATIONS
Proposition 5. Let A E RMxx have full column rank and define (AA) (Ir),, E), = {(i,5) e
[n]
A A.j + 0}. For a fixed integer K 2 1 the following are equivalent:
(i) For any invertible Raan the maximal number of connected components of G(AB) K.
(i) There are a permutation matrix P and an invertible matrix B such that
PAB diag (
and no other P', B' such that P' AB' is block-diagonal with K K+1 blocks on the diagonal.
(iii) There exists invertible B such that AB is compositional with K irreducible mechanisms in the sense of Definitions I and 5 of Brady et al. (2023).
(iv) There is a partition [m] = 21 U...UQ QK with 2x + Ø such that
K
rank(A) Σ rank(Aq..)
rank(A 2 Vk.
and no partition of [m] into K + non-empty sets satisfies this equality.
Proof. Throughout, ranks are column-rand For a matrix X, let row denote its row space and let (X) be the set of row indices whose corresponding by an invertible matrix or a permutation matrix preserves rank and does not change the edge-relation that defines the graph g(.).
meroo. Multiplication
28

---

Published as a conference paper at ICLR 2026
(i)
Statement (i) asserts that there exists _ax= such that G(AB) possesses exactly K con- nected components. Let Ci,.... CK C [n] be the vertex sets of these components and put R3 == Uiec, supp((AB).. S [m]. Without loss of generality we can assume that Cr Ck appear in contiguous order. Otherwise, permute the columns of B first.
Because different components have disjoint row supports (otherwise there would be connecting edge), the sets R.....; R mutually disjoint. Permute the rows that R.,..., Rk appear contiguously and denote the corresponding permutation matrix by P. Then PAB is block-diagonal with exactly K diagonal blocks. Note that any zero rows of AB be placed arbitrarily.
If, contrary to the minimality clause of (ii), another pair P' B' produced K + 1 diagonal blocks, the graph (AB')) would contain at least K connected components, contradicting (i). Therefore (ii) holds.
(ii) (iii)
Write PAB- diagf A ((K)). as in (ii) and set M == (AB) R2:: (k 1,... K) with R2 as before. The matrices (kk) have pairwise disjoint row supports, so they constitute K mechanisms and AB is compositional.
Assume that one mechanism, say M were reducible. Then its row support could be partitioned into two non con-empy sets whose row spaces are independent, yielding another decomposition of P' AB' into + diagonal blocks. This contradicts the minimality property (ii). Thus every mechanism is irreducible and (iti) follows.
(iii)
(iv)
Since AB has K compositional mechanisms, there are disjoint Ri....,RA C [m]. Add zero rows of AB arbitrarily R, denoted by 2, (i.e., R₂ 2,) such that .I...., 2xx partition [m]] Then rank( (BB) = Ex rank ((AB) e...) and rank
2 1.
Suppose a refinement [m] = 2 U... 2x+1 also satisfied the same rank identity. Then there is B' E Raxx such that AB' has K 1 compositional machanisms. Next, we show by contradiction that if AB has K compositional and irreducible mechanisms then there is invertible B' pa*= such that AB' has more than K compositional mechanisms establishing (iv).
Suppose such a B' existed. Denote with { (with K' > K) the row sets that constitute the compositional mechanisms AB', respectively.
According to the pigconhole principle there is at least one R, which has elements in multiple R/-- Denote with Uij R, n R,. Then rank(AR,.) rank(ARR.: Σ, rank( Au,,, BB) Ej rank(Au,,..) Ju.,.) which contradicts the irreucibilityy assumption. Thus, there is basis in which A has more than K compositional mechanisms.
(iv) (i)
Assume (iv) with partition [m] = Q U... Qx-
Permute rows so that QI...., Qx are consecutive; call the permutation matrix P. Because the row spaces row( A (AQ.:) are pairwise independent, one may choose a column basis aligned with them, yielding B E g with PAB diag (1), ..... (K)). Consequently G(PAB) C(AB)
has at least K connected components.
Now, let B be arbitrary and suppose ((AB) had K + 1 connected components with vertex sets Ci,... CC'- As before set R/ == User supp((AB)..) C [m]. Disjointness of components implies [m] R2 U... U RR and, before.
K+1 rank (ARsa);
rank(A) = Σ
k=1
29

---

Published as a conference paper at ICLR 2026
contradicting the minimality clause in (i). Therefore every invertible B produces at most K con- nected components.
We have established the chain of implications
ii
(iv)
(ii))
(1),
hence all four statements equivalent.
B EXAMPLES
Example (Type M and Type S mechanistic independence vs. reducibility). This example illus- trates Type M and Type S mechanistic independence and reducibility. We display four Jacobians, each written in a basis aligned with a given product decomposition of the source tangent space. Block columns (corresponiing distinct source components) are separated by vertical rules:
0
0
1
-2
1
0
-1
1
0
I
B
0000
1
0
2
i
0
O
-1
0
0
0
1
1 1 1000
0
0
DOUD
0000
2
0
C
D
-1
000
1000
2 2
0
For a Jacobian J displayed in a basis aligned with the product decomposition B TT, let [|o denote the number of its nonzero entries. In this aligned basis, we have
Pr
|
For a (right) change of source basis G E GL(T,S) that respects B, the transformed Jacobian is JG, and
p
min GE (block-
|
Conversely, for change of basis G E GL(T,S) that does not respect B,
Pog Ge (pock-rrss)))))))
inf
Likewise, for single component with a (nontrivial) split B, UVV= S& we compare Pas. Pvs using changes of basis that do (or do not) respect B, while fixing basis elements spanning + )EEKKT Sy
+
Mechanistic independence and irreducibility of Type M. Since no column support contains or contained in the support of a column from a different block, Type M mechanistic independence holds in all cases. For A, B, C, each component is one-dimensional, so Type M irreducibility holds vacuously. The first block of D further reducible since D D.2 while the second block is irreducible as D.3 3 D. Note that in the sparsest product-splitting basis multi dimensidmmnsi factors cannot be Type M reducible.
30

---

Published as a conference paper at ICLR 2026
Irreducibility of Type S. Again, since each component of A, B, C is me-dimensional. Type S irreducibility holds automatically. For the Jacobian D, each component is pro-dimensional: thus must verify that no 2D block be internally split to reduce sparsity compared with all other possible splits.
First block (columns 1 1-2). Consider the displayed split B1 Ts,Sa and any other nontrivial internal split B1 = U @ V. Since both U and V are one-dimensional, no further -rrrepectingg basis transformation can reduce the support. Counting nonzeros yields P = 8, and since
0 0 1
[D. = =1..,
=
we obtain Pis. -'
For distinct split B + B, we have Ps. P', since we can always revert to the current split. Moreover, P Pb.. the current split already achieves minimal support, Hence, the first block irreducible. (We could construct an alternative Jucobian with reducible first component by setting both -1 entries in D to O; modifying only one is insufficient.)
Second block (columns 3-4 4). Here a local simplification possible: by mixing the third and fourth columns appropriately, we reduce the third column by one nonzero. After this adjustment, the same argument as above shows that the second block is also Type irreducible.
Mechanistic independence of Type We now check mechanistic independence for each Jaco- bian individually.
Case A. Columns (blocks) have exclusive rows: rows 1,2 are nonzero only in the first block, and 7,8 only in the second. Any non-respecting change of basis mixes the one-dimensional components, introducing nonzeros into these exclusive rows while at most one of the four shared rows in the middle can be canceled. Thus, any genuine mixing strictly increases the total to-norm, so Pog > Pag = A|
Case B. Pairwise, B behaves analogously C: for each column pair there are four exclusive and only one shared. This enforces a lower bound under any 2 x 2 mix, so all pairwise checks pass.
However, there exists full G support (thus violating strict inequality in Def. 19::
R mixing all three columns without increasing the overall
0 1
G
1
0
|B ||
0 0
Hence, B is pairwise but not fully mechanistically independent.
Case C. As in B. all pairwise checks pass. The key difference is that C the three shared rows (1st, 3rd, 5th) cannot be simultaneously eliminated by any invertible G E R3x3 Thus, any combination involving all three blocks necessarily preserves the three exclusive rows (2nd, 4th, 6th) and increases fo- Therefore, Pig Pag
[| i.e., C is fully mechanistically independent.
Case D. A local simplification inside the second block (mixing the third and fourth columns) reduces the third column by nonzero. After this, the first, second, and third columns each have four nonzeros (the fourth remains at two), giving P = 14
+ 4 2
To break Type S independence, one would need a cross-block mix: there must exist vector (a, b, C, d) with either a or b nonzero and either e or d nonzero such that
D ba, c.
has at most four nonzero entries (matching (Tz). This is impossible: any such combination has at least five nonzeros, even under careful cancellations. Hence, every cross-block mixing strictly increases the norn and is Type mechanistically independent.
In summary, all components of A. B, C, D are Type S 'irrduucble: A, C. D are Type mechanis- tically independent; B is pairwise but not fully mechanistically independent.
31

---

Published as a conference paper at ICLR 2026
Example 2 (Minimizers of compositional contrast yield Type S independence in some generators). This example shows that there exist generators whose latent components are Type S independent but not Type D independent, yet for which the compositional contrast Ceomp recovers the sources up to permutation and element-wise transformations.
Let E TR2 and g: R2 + R with g(s) As, where
0
1
0
1
1/
We immediately observe that and 82 Type. S but not Type D independent.
Consider now a learned decoder 9: TR² - R5. If 9 minimizes the reconstruction error, then its Jacobian at some z" E ?R² takes the form [z(z") AB for nonsingular matrix B. Equivalently. | (de((B) 2 q ffror some > O. Writing
aa B
c d).
obtain
b
b
AB
a
+ +
and
Ccomp 2|a||b| + 2|clld] + +la c|lb + d].
c
d
d
will show that
min de
Ceomp
(B) q.
and that every global minimizer of Ceomp is a generalized permutation matrix (i.e., matrix with exactly one nonzero entry each row and each column). This means that the learned latent factors Type S independent after joint minimization of the reconstruction error and Ceomp
Proof. We prove the claim in three steps.
Step 1. Reduction the case det(B)) (B) q.
For > 0 we get
Ceomp (I.B) 12 Coomp B);
dee(tB |et (B)l
[et) q. choose ///l det B)] < 1. Then
|ctt(tB)| = 9,
Ccomp (tB) = t Ceomp B Ccomp ( (B)-
Thus any minimizer must satisfy det [B) q. It therefore suffices prove
Coomp B 2 (det(B) for all B,
and to identify the matrices for which equality holds.
Step 2. chain of inequalities.
Let
x= la),
[6],
u lcl.
v= |d].
By the triangle inequality,
la cl > lx u/.
bb + d > ly vl.
Hence,
Ccomp
( > 2ny + 2uv + Ixc
usl ly F(x,y, u.
32

---

Published as a conference paper at ICLR 2026
We now claim
F(a,y,u,0 2 xv + yu for all x. y. O.
Define
D: F(x,y.u,0) (xv + yu) = 2xy + 2uo + |x - ulyy vl yu.
We analyze by cases on the signs of x - u and v.
Case 1: 2 and
Then |x
by v| y v, and
D 2ry + 2wv xo yu + (x u) v)
3ry 3wv 2xv 2yn
= (xy + uv) + 2(x u) - -v) 2 0.
Case 2: x 2 u and < U. Then |z ul = x. - u. ly - v = v y, and
2xy + 2uv Tv yu (a u)(v y) =
2ry + 2uv 2V yu + (rv ry wv + uz)
xy 0.
Case 3: and y By symmetry with Case 2 (interchanging (xr, u)), again obtain D = ry + uv 2 O.
Case 4: and y U. Then |z
v. ly v|
y, and
D 2ry + 2uy zv yu + (u z) y)
2xy + 2uv zv - yu + (uv - uy xu + xy)
3ry + 3wv 2rv 2yu
(ry + uv) + 2(x u) v) 2 0.
In all cases have D 2 0, indeed
F(z,y,u,v) 2 xv + yu = |a||d| + [b[lc].
Finally, the determinant satisfies
det( (B) lad bel lad] +
|dc] |a|ld| + [b||c]
xv yu
by the triangle inequality. In summary, we have the chain
Coomp (B) 2 F(z,y,u,)) xv xu yu |dct
In particular, if del (B)) = g. then
Ceomp B 229.
Step 3. Equality conditions and classification of minimizers.
To attain the minimum under det(BB) 2 q. we must have det(BB] = q and equality throughout the chain
Coomp B 2 F(r, u, v) 2 xu + yu 2 det (B)
(i) Equality in Coomp (B) 2 F(I,y,u,e))
We used
la cl |||| - |c||,
b d Z |||| -dd||
This requires
ac S O,
bd S O.
(ii) Equality in F(x, y, v) 2 yu.
33

---

Published as a conference paper at ICLR 2026
From the case analysis above, equality F (z, y, u) + yu forces
Fzx,
ry 0 and 0,
that is,
||||6| = 0,
|c|ld| = O,
so
either = or b = 0, and either e = 0 or d = 0.
(iii) Equality in zv + yu 2 dde(BB))
We used
det((A) lad bcl laal + |bc] xv + yu,
which requires
(ad) S O.
From (ii) get four structural patterns, of which are incompatible with det(B) + 0. The remaining possibilities are
0 b) d) or B = c 0
B
For these matrices,
Ccomp
B = det [B] E |ad), [bc] }
so equality holds everywhere.
These precisely the 2 xx2 2 generalized permutation matrices. Consequently, the global minimizers of Ceomp under the constraint det B)] 2 are exactly the generalized permutation matrices with det((B) = q, and the minimum value of Coomp is q.
C DISENTANGLEMENT FOR NON-INVERTIBLE GENERATORS
In some applications the underlying generator is non-invertible when modelling the latent space as product space (or subset thereof). For two such scenarios statements about disentanglement: (1) local invertibility, and (2) invertibility on an open subset.
nevertheless make meaningful
For the former category, example image data containing multiple objects with identical appear- ance, since the generator is then (block- pereut-pemutttioonn invariant. Another example is the angle of a rotary joint with multiple revolutions in robotic arm, + n(2x) maps to the same physical state. More generally, these situations involve symmetries such as permutation or rotational symmetry.
For the second category, example arises from occlusions in image data. Here the generator is also non-invertible, but now entire regions of the latent space map to the same observation (e.g., when one object hidden behind another).
In such cases multiple latent codes map to the same observation, which makes the encoder inherently ambiguous. The learning algorithm must make choice about how to represent those observations. This choice may lead to defects in the encoder, such as discontinuities (as discussed later). However, decoder need not suffer from this issue. As long as we can learn a decoder that generates the observation manifold and whose components are mechanistically independent, we can still obtain disentanglement case (1), and at least on the invertible subsets case (2).
We now consider two illustrative examples.
Example 1. Consider images depicting two balls of identical appearance at arbitrary positions in the image, but without occlusion. this case the latent space can be modelled using an ordered configuration space, representing tuples of pairwise distinct object states:
Conf (Ω) {s E K
8: #
#
Vi Vi [K]]
(25)
34

---

Published as a conference paper at ICLR 2026
where Ω denotes the state space of single object (e.g.. position, color), and K is the number of objects. Since any permutation the factors (i.e., objects) yields the same observation, the ground- truth decoder must be permutation invariant. The observation manifold can therefore be viewed as an unordered configuration space, obtained by quotienting out permutations.
Assuming soft rasterizer, the generator be modelled local diffeomorphism: the map is locally invertible, and small latent perturbations produce small and reversible changes in the ob- servations. direct check verifies that have Type D independence and, by implication, also Type M/S/H,, independence. Because a single ball cannot be itself represented as an additive func- tion with two or more components, we have Type H2 irreducibility, and thus also Type D irreducibil- ity. The generator is also second-orde separable, since all first- and second-order partial derivatives are linearly independent at every point in the latent space. Considering the affine equivariance of positional encoding, the generator must also be Type S irreducible. Thus the local identifiability results for Type D, S, and H2 apply.
Furthermore, any configuration space with K 2 2 and dim(A)) > 2 is connected. Its 1-slices are also connected: fixing one ball, the other can be moved continuously to any other position in the image while avoiding collisions. Hence Theorem [ applies, and local disentanglement extends to global disentanglement.
Example 2. Consider images of two balls, one large and one small, placed at different locations, with possible occlusion (the smaller potentially disappearing behind the larger). The latent space can be modelled as S R²2 R22, describing the (zr, -posiitoosn of both balls. The generator then maps a hyper-tube of latent codes (corresponding to positions of the smaller ball behind the larger one) to the same observations. With soft rasterizer, the generator becomes differentiable, but it not invertible (not even locally invertible) on the full domain. However, at any point outside the hyper-tube it is locally invertible.
Following the same reasoning as in Example I, we obtain local identifiability of the decoder at all points outside the hyper-tube. Restricted to this region, the model is even globally identifiable, since the resulting space is identical to that of the previous example. Therefore, if we train a decoder with mechanistically independent components that can generate the observation manifold, it must be globally disentangled outside the hyper-tube.
Whether disentanglement holds within the hyper-tube is undecidable: if the large ball occludes the smaller one and moves by a small amount, we cannot determine from the observation alone whether the smaller ball behind moved as well.
D EXPERIMENTAL DETAILS AND FURTHER EXPERIMENTS
D.1 COMPOSITIONAL CONTRAST AS SURROGATE FOR TYPE INDEPENDENCE
This experiment closely follows the setup of Brady et al. (2023). We first sample latent variables from a standard normal distribution and then generate observations by passing them through an invertible MLP. The outputs are concatenated as
g(s) (s (1.2) (81, 82), (81, (2) (52), (2.3) (82, 83), ..... (K) (*K))-
For each g (9), the slot dimension is fixed dim(S,) 3. and the slot-output dimension is set to 20. The overlap ratio is determined by the output dimensions of (1) and g (1.2); if they have the same number of output dimensions, the overlap is 50%. Strictly speaking, for K > 2, this implies that in slots exhibit 66% overlap.
Figure 2
We train models with K E [2, 5) slots and regularization parameters À E (10-2, (0-2,11] 1]. where the loss is Creeon + ACcomp- For each configuration, we run five random seeds across overlap levels (0%, 5%, 20%, 50% resulting in 120 models in total. To ensure comparability across different numbers of slots and regularization parameters, we apply the same normalization procedure to all experiments. In addition, within each group of models sharing the same overlap ratio, we normalize Coump by dividing by the group mean, since the achievable minimum of C Cromp varies substantially with overlap.
35

---

Published as a conference paper at ICLR 2026
D.2 EXPERIMENTS ON
GENERATORS
We now consider images depicting two balls whose colors lie between green and red and which may appear at any position in the image, but without occlusion (K 2, d, 6); see Figure We train an autoencoder with an additive decoder (i.e., Type Hg independence), defined as
((z)
Σ 1 (z,). kE[K]
using the standard Mean Squared Error (MSE) reconstruction loss (implementation details are pro- vided below).
Rec
Slot
Slot
Figure 4: Image reconstructions for the entire latent code and for individual slots. Note that the per-slot reconstructions appear brighter because the offset is undetermined in an additive model.
To evaluate disentanglement quantitatively in this setting. we cannot directly predict the ground- truth latent code from the learned representation using the Slot Identifiability Score (SIS) as before. Because multiple latent codes always correspond to the same observation, the prediction target is ambiguous. Instead, we first determine the best-fitting fundamental bestefittnnfnunaannnn domain of the latent space under permutations. A fundamental domain is any connected subset of the configuration space containing exactly one representative of each latent orbit under permutations. Restricting the generator to fundamental domain renders it invertible, making the prediction target unique.
However, there exist infinitely many choices of fundamental domains, and most of them the leamed regressor would need to approximate discontinuous function. To avoid this, for each reconstructed image we compute the centers mass of both balls in image space and select, among all permutations of the ground-truth factors, the closest match. This produces partition of the latent space that aligns as closely as possible with the encoder's s (arbitrary) convention. We then compute the SIS on this selected fundamental domain and denote the resulting metric by SIS*. Table shows that the model achieves nearly perfect disentanglement.
Table 1: Slot Identifiability Scores after selecting the best-fitting fundamental domain over 5 random seeds.
RMSE
SIS*
1.30 + 0.18 99.6 = 0.05
Next, we examine the encoder more detail. As noted by Zhang et al. (2019): Hayes et al. (2023). the encoder must approximate discontinuities in this setting, phenomenon known the responsi- bility problem. Such discontinuities necessarily arise whenever we traverse a path in the latent space that connects point to its pereuttemutttion.
Figure Sluuee]sshww shows the learned latent variables for several latent traversals: in each row, only one ground- truth latent variable is varied while the others remain fixed. The corresponding images and recon- structions are shown in Figure 6 When we vary the coordinates of the first ball 8,, only the second encoded slot 22 (22,1, 22,2, 22,3) changes; modifying 82 analogously affects only 21, with one
36

---

Published as a conference paper at ICLR 2026
exception: The lower-left traversal in Figure 5 reveals (approximately) discontinuous jump in the middle. On either side of this jump, changes to the ground-truth latent affect only one slot. This discontinuity is unavoidable and can, in principle, hinder autoencoder training by trapping the optimization in poor local minima. However, this instance, training succeeds without issue.
Z
21,2
22,1
22.2
22,3
15
2.0
1.0
0.5
Predicted
4.5-
-3.6
05-
4.5
3.0
Ball x-position
y-pasition
Ball color
1.5
1.5
1.0
1.0
1.0
0.3
0.3
0.-
Predicted
0.5
0.1
-3.8
-.3
2.5
3.4
3.0
0.2
x x-pnsition
Ball y position
Ball
Figure 5: Latent traversals: in each subplot, one ground-truth latent variable varies while all others remain fixed. The curves depict the learned latent codes.
In Figure 6 consider the traversal the z-position of the second ball: the image reconstructions align perfectly with the ground truth, and nothing in the visual output betrays the latent discontinuity.
GT
Rice
9
GT
9
Fice
ST
GT
GO
Flce
GT
ST
Ried
Figure 6: Image reconstructions for latent traversals: in each row, a single ground-truth latent vari- able is varied while all others remain fixed. From top to bottom, vary the z-position, y-position, and color of the first ball, followed by the same for the second.
Implementation Details
37

---

Published as a conference paper at ICLR 2026
Dataset. We use images of size 64 64 x 3 with pixel values in [0, 255). The dataset contains 300,000 training images, 10,000 validation images, and 50,000 test images. Images with occlusions removed, introducing mild dependencies between object positions. Apart from this, the latent variables are sampled uniformly over their support.
Model Architecture. The encoder consists of:
(1) A ResNet-18 backbone with the final classification layer removed (output dimension: 512).
(2) A linear layer mapping 512 - 4096, followed by Batch Normalization and Leaky ReLU (slope 0.01 for negative inputs).
A fully connected layer of size 4096 4096, again followed by Batch Normalization and a Leaky ReLU.
(4) A final linear layer mapping 4096 to the total ground-truth latent dimension, followed by Batch Normalization.
We use additive decoder g(z) = ΣiE[K] (zi). where each subdecoder ( has the same architecture (with no shared weights):
A linear layer mapping from d; d, to 1024, followed by Batch Normalization and Leaky ReLU.
(2) Four fully connected layers of size 1024 x 1024, each followed by Batch Normalization and Leaky ReLU. The output is reshaped into 64 feature channels over 4 x 4 grid.
(3) A stack of deconvolutional layers, each followed by Leaky ReLU:
(a) Deconvolution: 64 -> 1024, kernel size 4, stride 2, padding 1.
(b) Deconvolution: 1024 - 512, kernel size 4, stride 2, padding 1.
(c) Deconvolution: 512 - 128, kernel size 4, stride 2, padding 1.
(d) Deconvolution: 128 3, kernel size 4, stride 2, padding 1.
Hyperparameters. We use the AdamW optimizer (Loshchilov & Hutter 2017 with:
Batch size: 1024,
Learning rate: 5 x 10-5
Weight decay: x 10-5,
Number of training epochs: 1000.
LLM USAGE DISCLOSURE
In accordance with the ICLR policy on large language model (LLM) usage, we disclose that LLM (OpenAl's ChatGPT) was used solely for minor language polishing. This included limited grammar correction and rephrasing for clarity. All research ideas, technical content, analyses, and conclusions were generated entirely by the authors, who remain fully responsible for the paper's content. For full transparency, this very disclosure note also drafted with the help of ChatGPT.
38


Reviews:
[{"review_id": "M7VV6Wdhi1", "reviewer": "Reviewer_aEKR", "summary": "The paper addresses the challenge of identifiability in disentangled representation learning, particularly for nonlinear generative models, by introducing a unified framework based on mechanistic independence. This principle defines latent factors not by their statistical distribution but by how they act on the observation manifold through the generator function. \nThe authors propose a hierarchy of mechanistic independence criteria (and corresponding irreducibility notions) ranging from\nType D (Disjointness), Type M (Mutual Non-inclusion), Type S (Sparsity Gap), to Type H_n (Higher-Order Separability).\nFor each criterion, the paper provides identifiability theory. Furthermore, this framework successfully generalizes and unifies several existing mechanistic constraints based or sparsity based identifiability results.", "questions": "Practical Losses for Type M/S: Can the authors propose initial or speculative ideas for a robust surrogate loss that is more reliably minimized in practice?\n\nIndependence Hierarchy: Could the authors include in the figure what are the precise conditions under which one criterion is strictly weaker or stronger than another?\n\nExperimental setting: It is not very clear to me that which types of independence are considered in the experiments? By following the setup of Brady et al. (2023) are you saying that Type D is considered? Have you considered any preliminary empirical validation for the other types like $H_n$?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Clarity: The paper is very well-written.\n\nRelevance: The paper is very relevant to the field and the contribution is significant. The framework represents a fundamental conceptual advance, providing a robust, distribution-agnostic foundation for identifying latent factors. \n\nGenerality: The framework is broad, covering multi-dimensional factors, partial disentanglement, and non-invertible generators. The Theorem 1 result, extending local to global disentanglement under mild topological assumptions, looks like a strong tool.\n\nUnification: By establishing identifiability for a range of mechanistic constraints under nonlinear, non-invertible mixing and statistically dependent latent factors, the paper generalizes and expands several very recent, disparate results.", "weaknesses": "The experimental setting looks limited (and not clear to me).  Plus identifying robust surrogate losses for the relaxed criteria, specifically Type M (mutual non-inclusion) and Type S (sparsity gap), remains an open problem. However, given the solid theoretical contribution, these weaknesses are not major.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3Z8RS0o4Sm", "reviewer": "Reviewer_7EYq", "summary": "This paper proposes a framework for identifiability of disentangled representation learning based on mechanistic independence. The authors start from properties of the generator from latent variables to observations, rather than from the distribution of latent variables. A family of independence criteria is discussed: Type D (disjoint support via Hadamard orthogonality of Jacobian columns), Type M (mutual non-inclusion of supports in a fixed basis), Type S (a strict sparsity-gap condition on Jacobian representations), and Type H (vanishing cross *n*-th-order derivatives plus *n*-th-order separability). The paper proves identifiability from the local to the global level and provides a graph-theoretic view where factors correspond to connected components. But the assumptions are not tested in experiments or used to guide new methods. Overall, this paper offers a different view of identifiability of disentangled representation learning.", "questions": "- How can these independence criteria be deleted or used in cases such as a synthetic dataset?\n- Can mechanistic independence be linked with current theories? Like in intervations: the second derivative for different components $i, j$ is zero like Type M somehow.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proof of identifiability is detailed and step-by-step, going further.The proof looks right and kind of makes sense.\n- The whole paper is connected by mechanistic independence, which is an interesting view for identifiability.\n- Graph-based characterization of factors as connected components aids intuition and potential diagnostics.\n- The paper considers a more general generating process than previous works.", "weaknesses": "- Several key assumptions are strong, hard to verify, or basis-dependent: Type M depends on a fixed canonical basis; Type S requires the existence and uniqueness of a sparsest product-splitting basis.\n- The assumptions are not tested in well-controlled experiments or real-world cases, and the theory doesn't contribute to methods. There is little evidence on real datasets or complex architectures.\n- The limitations of mechanistic independence are not specifically discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iqwbMkS9Hy", "reviewer": "Reviewer_UZdZ", "summary": "This paper introduces a unified framework for achieving identifiable disentangled representations through \"mechanistic independence\", a principle that characterizes latent factors by how they act on observed variables (via the generator) rather than by their statistical distribution. The authors propose several related independence criteria (Type D, M, S, and H_n) and prove that each yields identifiability of latent subspaces under nonlinear, non-invertible mixing. They establish a hierarchy among these criteria and provide a graph-theoretic characterization of latent factors.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The novel theoretical perspective shifting from statistical to mechanistic independence is interesting and practically motivated (e.g. by compositionality and transfer learning). This paper is an interesting step connecting disentanglement with discovering the mechanistic structure of data-generating processes.", "weaknesses": "The main weakness of the paper is that the theoretical ideas are not especially intuitive. For example, the paper could better discuss when the various independence assumptions are likely to hold in practice. It might help to include more empirical results, not so much because experiments are required to validate the theory, but because experiments might give intuition about settings where the theory is actually relevant and which assumptions hold in which experiment settings.\n\nAs a more minor note, the main text would benefit from a short description of the experimental setup (without reading Appendix C, I didn't understand what Figure 1 shows at all).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hL82XXu5KZ", "reviewer": "Reviewer_xBHk", "summary": "This paper discusses a new set of principles for the identifiability problem of disentangled representations. Distinguished from existing works that usually assume statistical independence, this paper proposes mechanistic independence instead, to achieve identifiability even when the mixing function is non-invertible. Various related mechanistic independence conditions are proposed, most of which advance some branches of existing literatures.", "questions": "1.\tThis work does not rely on global invertibility of mixing function, how much is this condition be relaxed? I find that local diffeomorphism is still need. Together with path-connectivity, can global invertibility be derived? If not, please provide examples, and discuss how possible is it for such cases to happen in real applications.\n2.\tHow to determine L (the number of blocks in Z) in practice? Setting $L=1$ clearly leads to trivial results.\n3.\tIn Line 86, how to understand the symbol $D_{ij}^n$? Should “id” be 0 instead?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThis paper proposes a novel perspective for latent variable identification, i.e., consider disentanglement directly as the goal instead of traditional slot/block-wise equivalence. This is intuitive since global invertibility is not required here.\n2.\tThis paper gives a thorough discussion on the proposed mechanistic independence, providing several options for the identifiability principle. Most options have close relation to related works from different branches of identifiability research, and are unified nicely into one framework.\n\nI am particularly interested in the novel results that global invertibility of mixing function is not required in this paper. Since almost all works on identifiability rely on this assumption, this would be an important advancement if its implication is properly discussed. See my question 1 for discussion.", "weaknesses": "1.\tAlmost no experiment is provided in this paper, which is my major concern for this paper. I understand this is mainly a theoretical paper. However, at least experiments on synthetic data should be provided (Fig. 1 is not enough), to validate each of your main theorems. Without experimental results, it is hard to understand the application scenario of different mechanistic independence principles, and reliability of the theorems are also weakened.\n2.\tLack of explanation for the applicability of proposed assumptions in real applications. It is nice to have so much different independence principles as choices, but all principles lead to one common learning method (Fig. 1). I think it is infeasible to select one principle by validating each assumption in practice, and lacking customized methods significantly blurs the boundary among different principles.\n\nI do not insist that identifiability works *must* include experimental results. But for this paper, I think it is helpful and necessary to demonstrate the superiority given that large part of results are upgrades of existing works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1zFsGAS3VD", "reviewer": "Reviewer_Crp8", "summary": "This paper proposes mechanistic independence, which enables identifiable disentangled representations without strong statistical assumptions.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. As a theoretical paper, the presentation is rigorous.\n2. The theorem proofs appear solid.\n3. The authors seem familar with recent identifiability literature and compare their results with prior work across the types D, M, S, H.", "weaknesses": "1. The abstract and main text repeatedly state that the paper’s identifiability results allow a non-invertible g, and they give the practical example of the “responsibility problem” (lines 160–161) to justify relaxing the global invertibility assumption. From my reading, permitting non-invertibility appears to be one of the paper’s contributions/novelties. However, all of the identifiability results actually rely on local diffeomorphism, which is stronger than local invertibility. Therefore, I think it is necessary to justify that this relaxation is non-trivial. By “trivial,” I mean the situation where one advertises that the framework does not require Condition A, but actually assumes Condition A holds in all the relevant region in the space (i.e. only not hold on a set of measure 0). It would be nice if you can add a discussion clarifying the gap between global invertibility and local diffeomorphism. For example, could you analyze a practical case like the “responsibility problem” that fails to be globally invertible on the space of interest but still satisfies the local-diffeomorphism condition?\n\n2. In the theoretical framwork of this paper, S is the souce factors space, Z is the representation/target factor space, X is the observational space. But before def 1 (line 120-123), you only introduce S and X. Notation Z and term 'target factor' first appear in def 1 without any explanation. Maybe you can introduce Z and target factor before def 1 like what you have done with S and X. I think this may make the paper easier to read.\n\n\n\n3. The theorem numbering in the appendix does not match the numbering in the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)