Nonparametric Identification of Latent Concepts
Authors: Yujia Zheng
Shaoan Xie
Kun Zhang
Venue: ICML 2025 poster (2025)
Keywords: Concept Learning
Compositional Learning
Nonparametric Identification
Primary area: general_machine_learning->representation_learning
TL;DR: We establish theoretical guarantees for learning latent concepts in general settings, without assuming specific concept types, functional relations, or parametric generative models.

Abstract: We are born with the ability to learn concepts by comparing diverse observations. This helps us to understand the new world in a compositional manner and facilitates extrapolation, as objects naturally consist of multiple concepts. In this work, we argue that the cognitive mechanism of comparison, fundamental to human learning, is also vital for machines to recover true concepts underlying the data. This offers correctness guarantees for the field of concept learning, which, despite its impressive empirical successes, still lacks general theoretical support. Specifically, we aim to develop a theoretical framework for the identifiability of concepts with multiple classes of observations. We show that with sufficient diversity across classes, hidden concepts can be identified without assuming specific concept types, functional relations, or parametric generative models. Interestingly, even when conditions are not globally satisfied, we can still provide alternative guarantees for as many concepts as possible based on local comparisons, thereby extending the applicability of our theory to more flexible scenarios. Moreover, the hidden structure between classes and concepts can also be identified nonparametrically. We validate our theoretical results in both synthetic and real-world settings.

Nonparametric Identification of Latent Concepts
Yujia Zheng" Shaoan Xie Kun Zhang 12
Abstract
We are born with the ability to learn concepts by comparing diverse observations. This helps us to understand the new world compositional manner and facilitates extrapolation, as objects naturally consist of multiple concepts. In this work, we argue that the cognitive mechanism of comparison, fundamental to human learning, is also vital for machines to recover true concepts underlying the data. This offers correctness guar- antees for the field of concept learning, which, de- spite its impressive empirical successes, still lacks general theoretical support. Specifically, we aim to develop theoretical framework for the identi- fiability of concepts with multiple classes of ob- servations. We show that with sufficient diversity across classes, hidden concepts can be identified without assuming specific concept types, func- tional relations, or parametric generative models. Interestingly, even when conditions not glob- ally satisfied, we can still provide alternative guar- antees for as many concepts as possible based on local comparisons, thereby extending the appli- cability of our theory to more flexible scenarios. Moreover, the hidden structure between classes and concepts can also be identified nonparametri- cally. We validate our theoretical results in both synthetic and real-world settings.
Figure 1: The class "shark A has concepts like "predator x," "slcek body so," and "ocean A2" The class "turtle & has concepts like "shell "" and "ocean 22" A child may learn to distinguish between these classes by focusing on the unique concepts specific to each as "predator SE" and "sleek body = for "shark A," and "shell t for "turtle &.."
ferences between various species, thereby identifying the unique concepts that define each group (e.g., Fig. 1). This mechanism of learning through comparison has been exten- sively studied and verified across various fields, including psychology and neuroscience, affirming its universality and effectiveness (Bruner et al., 1957).
Meanwhile, in machine learning, the extraction of concep- tual features is crucial for the development of robust and interpretable models, illustrating the integration of cognitive principles into machine intelligence (Valiant, 1984; Mitchell, 1997). Recent research has achieved notable success in deriving buman-interpreede interpretable concepts from various data modalities with different formulations of the problem (Bau et al., 2017: Radford et al., 2017; Alvarez Mclis & Jaakkola, 2018: Kim et al., 2018; Zhou al., 2018; Yeh al., 2020; Koh al., 2020; Du et al., 2021; Bai al., 2022; Achtibat et al., 2022; Crabbé & van der Schaar, 2022; Liu et al., 2023; Park et al., 2023; Jiang et al., 2024). These concepts have proven beneficial for tasks such as extrapolation (Janner et al., 2022; Lachapelle et al., 2023; Du & Kaelbling, 2024), explanation (Alvarez Melis Jaakkola, 2018; Sreedharan et al., 2020; Leemann et al., 2023; Poeta et al., 2023), and docision-making (Grupen et al., 2022; Zabounidis al., 2023; Delfosse et al., 2024). Furthermore, advancements in this domain have significantly contributed to scientific discovery, particularly in healthcare (Jia et al., 2022).
1. Introduction
Humans possess an innate ability to learn concepts, i.e., conceptual factors underlying the observational world, by comparing diverse classes of observations, process foun- dational to cognitive development (Rosch, 1973; Fodor & Pylyshyn, 1988). For example, child distinguishes be- tween different types of animals not by memorizing each species separately, but by observing and comparing dif-
'quual contribution 'Carnegic Mellon University Mohamed bin Zayed University of Artificial Intelligence.
While numerous methods have been developed to extract concepts from data, most provide only empirical support and lack theoretical guarantees concerning the correctness
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Nonparametric Identification of Latent Concepts
of the recovered concepts (Marconato et al., 2024b). With the help of specific parametrie assumptions, few studies have explored the identifiability of concept learning. For example, with the linear representation hypothesis that con- cepts are linearly related, recent research (Rajendran et al., 2024; Reizinger et al., 2024; Marconato et al., 2024a) has shown that the concept space can be identified up to linear transformation. Another line of research has tackled object- centric learning, attempting to identify individual objects as groups of pixels (slots), such as trees dogs, while ex- cluding more abstract concepts like lighting and styles. In addition to these concept type restrictions, further assump- tions are also required for identifiabilitty such as no occlu- sion between objects (each observed variable is influenced by only one concept) (Brady et al., 2023; Wiedemer al., 2024) or the additivity of the generating process (Lachapelle et al., 2023; Wiedemer et al., 2024). These studies mark significant exploration toward understanding concept learn- ing, At the same time, the constraints imposed on concept types and functional relationships may limit the confidence to fully account for the empirical success observed in con- cept learning from real-world scenarios. Therefore, despite significant empirical progress, a fundamental question in concept learning remains unanswered:
ing concepts (Thm. 1). This pairwise comparison' serves as a foundational prototype for learning concepts, enabling the flexible identifiability of as many concepts as possible, given that they exhibit enough diversity, even when others do not. We then extend the pair-wise identifiability to learn unique concepts from an arbitrary subset of classes (Cor. D), Given that most works rely on global assumptions for all concepts and fail offer guarantees when assumptions are partially violated for some concepts, the proposed flexible identifiability by local comparisons provides unique practi- cal value, since real-world scenarios often do not perfectly conform to ideal conditions for all concepts.
Furthermore, with sufficient diversity across different classes of observations, we prove the nonparametric identifi- ability for all class-related hidden concepts up an element- wise transformation and permutation (Thm. 2). For other invariant background concepts, such as "chrromatc"" that re- main consistent across all classes, we can also identify them under appropriate structural diversity conditions (Prop. 1). Consequently, we introduce, to the best of our knowledge, one of the first frameworks for concept identifiability the general setting that does not confine itself to specific con- cept types or parametric generative models. Moreover, the connective structure between classes and concepts can also be recovered in nonparametric way (Prop. 2). Our theo- retical results are substantiated through empirical validation on synthetic data and five different real-world datasets.
In the general setting, which concepts can we reliably recover with theoretical guarantees?
We address this question by grounding our approach in a fundamental cogmitive mechanism: humans learn concepts by contrasting diverse classes of observations. Classic stud- ies have shown that concept formation relies on detecting distinctions across examples: Bruner et al. (1957) empha- sized learning through contrasts between exemplars and non-exemplars Gibson (Gibson, 1963; 1969) proposed dif- ferentiation as a basis of perceptual learning in infants; and Gentner & Namy (1999) demonstrated that direct compari- son enables children to abstract category-defining features. This principle is further supported by extensive literature across cognitive science, reinforcing that it is only through discerning the differences between classes that humans can unravel and understand previously unseen concepts. As result, in the most general setting, the essential information for provably learning hidden concepts must pertain to the diversity present among different classes
2. Preliminaries
We first introduce the problem setting as well some essen- tial notation. Fig. 2 illustrates the key notation and relations of the considered setting. We also provide structured summary of notation in Appx. A for a quick reference.
-ata-gennrating process. Let x = (xi,...,Xx)) , E X C F" be a vector representing observed variables. We assume that the observation x is generated by latent concept vari- ables (Z Z E Z C R The generating process is as follows, with Fig. 2 serving as a concrete example:
x x f(x,,
where consists of class-dependent part ZA (z) (Z)...., Zna) E ZA C Ra and class-independent part ZB = (Znat1...., 2n) E ZB S R"2, nB = n - n1 These two parts are conditionally independent given ob- served class variables c = (CI..... c.) S R", i.c. p(z|c) = ())(z)) Since a depends on the classes c, we represent Z, == g(c, 6), where denotes other fac- tors including potential noise. All densities are smooth and positive, and domains are path-connected.
Inspired by this cognitive process of learning by comparison, we establish a set of theoretical guarantees on concept learn- ing in the general setting. We show that hidden concepts can be identified without relying on assumptions about the na- ture of the concepts or specific parametric models, provided there is sufficient diversity across classes. Specifically, we first prove that for any pair of classes, the unique part of the concepts for each class can be disentangled from the remain-
Learning by comparison serves as an inspiration for identifia- bility theory, rather than being a specific estimation method.
2

---

Nonparametrie Identification of Latent Concepts
The support of the matrix S is supp(S) {(1, (1, (2, (3, For the matrix-valued function S(0), the support supp(S) remains the same as long as some 0 * #0 makes those entries nonzero.
C
Classes
M
Structure
Connective structure. Based on these, we define the structure M as binary matrix with the support D -a, The class-dependent part z can be further represented as
L
Class deesedee Concepts A
Class-independent Concepts B
RA p(zA() = II p(z, [M, Oc),
(2)
where M,, the i-th row of M. The operator @ denotes the element-wise (Hadamard) product. Since classes c connected class- glass-independent part z M illustrates the connective structure between classes с and concepts z. The conditional independence provides a form of modularity commonly adopted in prior work on identifiable latent variable models (Hyvärinen & Morioka, 2016; Khemakhem et al., 2020a; Sorrenson et al., 2020; Lachapelle et al., 2022: Hyvärinen al., 2024). It may be particularly natural in class-concept framework; for example, while the concepts "wings" and "feathers" are related, they become conditionally independent given the class variable "bird." Moreover, we define A, as the index set of concepts corresponding to class C₁ with the associated concepts represented as ZA, Likewise, A, \A, refers the difference in the concept sets between classes c; and Cj.
X Observed Variables (e.g., pixels)
The problem setting. Consider images of aquatic animals, where the observed variables x represent image pixels. The different animal types (e.g.. "shark A and "turtle *h"" correspond to class variables c. Class-dependent concept variables 4 might include attributes like "predator s,," "sleek body =²."" "ocean #s" and "shell t" (see, e.g., Fig. 1), while class-independent concept variables z could be "lighting #" and "temperature J." The hidden generative process of each image depends on all of these concepts, though only some are specific to each class (encoded by the structure M, which is binary adjacency matrix). The goal is to identify z based on observed variables x and classes c.
Figure 2:
The generating function f an unknown diffcomorphism onto its image, capturing complex mixing in observational data without imposing specific distributional assumptions, such as Gaussian, on latent concepts z. This allows for a broad formulation that encompasses diverse concepts and nonparametric generative models. Equation can also be extended to the additive noise setting through a standard deconvolution technique (Khemakhem al., 2020a).
Example 2. Consider the example in Fig. 2. we have A = (Z] ãे, Z Z3 Z.1 tb) and classes c = (CiA, c2th). Then we have ZA, {21, Z2, 2z and Z = {zz, 21). and AA (21,22) 22). The connective structure M a binary matrix:
1 0 0
0
M
Observational equivalence. All of our identifiability re- sults are rooted in the observational equivalence between the ground truth and the estimated model. This equivalence can be achieved in the large-sample limit using maximum likelihood estimation, which is extimutor-aynosicc and can be facilitated by methods such as normalizing flows.
Technical notation. Throughout this work, for any matrix S, we use St, for its i-th row and S.g for its j-th column. We denote the first & dimensions of the i-th row as S and the remaining as S1,+1.. Similarly, the first ls dimen- sions of the j-th column are S.k.j and the remaining Sk+1-5- For any set of indices I (1.....m) m] {1,....n), n). analogously, we have I, = (j (i,j) Z} and
Definition 1 (Observational Equivalence). Two models (g, Pz, M) and (g.Pa, and only if (xfe(z(c) = (x)c)
I.j. {i (i,j) I}. We define the support of S aabb supp(S) = ((i,) Siy * 0}. and also extend supp(-) to matrix-valued valued function S(.), defining SSp = (i,,)) 30 E e S(0),, + 0}. Then we de- fine D as the support of Deg. where Deg represents the partial derivative of g w.r.t. c.
Pa. M) are observationally equivalent if
3. Identifiability Theory
Without any assumptions specific concept types, func- tional relations, or parametric generative models, to what extent can we provably learn hidden concepts from diverse classes of observations?
Example 1. Consider the matrices
1 0 3
8
0
3
S =
0 0 5
S(0)
0
0
50
To answer this, in Sec. 3.1, we first prove that the unique concepts in any pair of classes can be disentangled from
4
0 0
4
0
0
3

---

Nonparametric Identification of Latent Concepts
the remaining ones (Thm. 1). Based on this, we can fully leverage the diversity in the data and provide flexible identifiability for any subset of concepts, as long as there exists sufficient diversity for local comparison (Cor. 1). For the global identification, in Sec. 3.2, we prove the nonparametric identifiaiility for all class-dependent hidden concepts (Thm. 2) under the structural diversity condition (Assump. 1). Together with a sparsity condition for the remaining class- daas-inndependen part, all hidden concepts can be identified up to trivial indeterminacy (Prop. 1). Furthermore, in Sec. 3.3, we recover the hidden connective structure between classes and concepts (Prop. 2), providing further insights into the latent compositional relations.
consists of vectors of the form (81, 83), where $1, 83 E R are free to take any real values.
Theorem 1 (Learning by pairwise comparison). Consider two observationally equivalent (Defn. 1) models (g. Pe. M) and (g. Pe, M) as in Sec. 2. Suppose, with an to regu- larization on D&9 (1D) S |DD, there exist set of points {(c, (),00)(0)) IP..l such that
0))((( 1P.al are linearly independent: /t=1
2. (TDeg((e,0)))) ()))) TR D.
Then any pair of classes c, and Cj there exists a per- mutation x such that the unique concepts of each class are disentangled with other concepts, i.e., D2 B2A;
To enhance technical under- standing, we provide de- tailed discussion in each see- tion on connections to tech- niques from the broader identi- fiability literature. Additional concrete examples and discus- sions are available Appx. D.
C1
C2
C3
C4
and
(B equal to zero.
Uea,
21
22
Insights. For Example 3, given classes C₁ and ca. Thm. I ensures that for any pair of classes, such C₁ and c2. the concept unique to C1 Z1, can be disentangled from the other concepts { za Similarly, the concept unique to C32 Z2, can be disentangled from the other concepts {z1, 23), up to a standard permutation indeterminacy.
Figure 3:
Structure in Running Example 3.
Example We introduce a running example (Fig. 3) to illustrate key implications and insights throughout the paper. The lines depict the connective structure between classes and concepts, distinguishing dlass-dependent con- cepts, ZA = (Zz, Z2, Zz), from class-independens vari- ables, n, enclosed within the blue dotted square.
We have also extended the guarantees for pairwise com- parisons to arbitrary class sets, facilitating more efficient learning in complex scenarios with its proof in Appx. B.2
Corollary (Learning by local comparison). Consider two observationally equivalent (Defn. 1) models (g.
(g.Pz. Pz. M) and (g. Pz. M) as in Sec. 2. Suppose that the assumptions in Thmm. 1 hold. Then, for a set of classes e₁ and its corre- sponding concept sets ZA, with set of indices 1, there exists a permutation π such that the unique concepts for the class c, are disentangled with concepts associated with other classes, i.e., (belA,/App 0.
3.1. Learning Concepts by Local Comparison
Humans learn concepts by leveraging the diversity across classes. We argue that the fundamental mechanism in this cognitive process is learning through pair-wise comparison, since any two classes can only be distinguished by identify- ing their unique concepts. Pairwise comparison thus serves as the basic unit for concept learning across multiple classes, as comparisons among any set of classes can be reduced to pairs. In the following theorem, we prove that the unique concepts between any pair of classes can be disentangled from the remaining concepts. The proof is in Appx. B.1.
BAry
Insights For Example 3, given set of classes {c2, C3, ca Cor. I ensures that for any class in the set, e.g., C2 E Cı, the unique concept Z1 can be disentangled from the concepts associated with others, i.e., (22, (22,23). }3}. where C₁ can be arbitrary subset of all classes.
We first introduce some additional notation. Let us define T as matrix with the same support of T(-) in Değ = TDeg. where T-)) is a matrix-valued function. In addition, given a subset E {l....,,7 n2, the subspace R2 is defined as:
Discussion on nondegenerate sample space. A key imprint of the nonparametric generating process in obser- vational data its structure. The assumption helps ensure the possibility of capturing the connective structure by the Jacobian in the nonlinear cases, following the similar spirit in (I (Lachapelle et al., 2022; Zheng et al., 2022). In general, it avoids pathological cases where all samples originate from highly restricted sub-population. that only cover a degener- ate subspace. The first part makes sure that there are at least (D.n.al data points such that the Jacobian function spans the
RS (s E R 8, 0 i S},
where is the i-th element of the vector 8.
Example 4. Intuitively, P3S consists of all vectors in R. where only the coordinates indexed by S can vary, while the
remaining coordinates are fixed at zero. For Example 3, if
n 3 and S D.g = (1,3), then Rd R3
(1.3)
D.
4

---

Nonparametric Identification of Latent Concepts
support space, which is almost always guaranteed asymptot- is also TR. ..
Theorem 2 (Learning by global comparison). Consider two models (g. Pa, M) and (g. Pa, M) as in Sec. 2. Under the assumptions in Thm. I and Assump. 1, suppose that for any set A₂ S Z with non-zero probability measure and which cannot be expressed as Bag x 2. for any Bag S ZB, there exist two values of c, i.e., clt) and clv) (which may vary across different Ag), that
ically. The second part (TDeg((c,
mild since D.,, = TDeg((c, ))) always resides in R Even in some rare cases where the matrix does not fit the support due to some generic combination of values, the assumption is still almost always satisfied asymptotically, since it only necessitates the existence of one matrix in the entire space with the same support of T(.). Meanwhile, the non-degeneracy assumption excludes linear transformations with constant Jacobians, as lincar Gaussian models are non-identifiable and linear non- non-Gaussian models have already been extensively studied. An illustrative example is provided as Example 5 in Appx. D.2.
J p(z) c )dz # [ p(z) dl
JZEA2
(f Pa, M) and (ه. Pa, M) are observationally equivalent (Defn. 1), then:
Partial identifiability benefits applicability. Besides being the foundation for the learning process, the principles of local comparisons in both Thm. 1 and Cor. 1 also enable partial identifiability for a subset of concepts when diversity not universally satisfied for all. Previous theoretical stud- ies concept learning often assume universal conditions like linearity or additivity. Nevertheless, such assumptions rarely hold for all in complex, unpredictable real-world sce- narios and fail to guarantee identifiability under any degree of violation. This challenge of handling partial assumption violations persists in the broader identifiability literature (Hyvärinen & Morioka, 2016; Khemakhem et al., 2020a; Taeb et al., 2022: Zheng et al., 2022; Hyvärinen et al., 2024; Zhang et al., 2024; Zheng et al., 2025). Fortunately, with the proposed theory based on local comparisons (Thm. and Cor. 1), we can leverage the structure to recover the hidden system as much as possible, even when the degree of diversity does not support global identifiability. For in- stance, while shared concepts across similar classes remain unidentifiable, sufficient diversity ensures identifiability for other concepts. Crucially, these new flexible guarantees require additional restrictive assumptions on concept types, functional forms, or parametric models.
(Element-wiss Identifiability) For any Z₁ E ZA, there exists permutation and invertible functions hę I- R.s. 2, ha h=(Zn(())
2. (Block-wise Identifiabilitt)) For ZB. there exists an in- vertible function h T - R s.t. ăg h(zB).
Insights. For Example 3, element- identifiability ensures all class-dependent concepts can be identified up permutation, i.e., concept Z1, Z2, and Z3 can be indi- vidually disentangled from each other. The block-wise indeterminacy makes sure that the class-independens con- cepts Z = (Z4,... Zn) as a block (group of variables) can be disentangled from the class- dlass-dependent concepts ZA. These two types of identifiabilities are standard in the literature (Hyvärinen & Morioka, 2017: Lachapelle al., 2022: Von Kügelgen al., 2021; Zheng & Zhang, 2023).
Discussion on structural diversity. Assumption 1, termed Structural Diversity, ensures sufficient diversity across dif- ferent classes of observations for the nonparametric identifi- ability of all deeendee concepts. Without imposing parametric constraints on concept types, functional rela- tions, or generative models, the only exploitable informa- tion is the inherent connective structure between classes and concepts. As discussed earlier, when classes lack diver- sity on their corresponding concepts, identifying individual class-dependent concepts becomes impossible without extra knowledge. Thus, Structural Diversity is crucial for guar- anteeing correctness across all concepts without relying on parametric assumptions or auxiliary information.
3.2. Learning Concepts by Global Comparison
Inspired by the mechanism of local comparison, we have shown that it is possible to fully leverage the diversity among different classes of observations to recover hidden concepts as much as possible. This naturally leads us to consider the conditions required for identifying all hidden concepts in a global manner. We first prove that, under the condition of Structural Diversity (Assump. 1), all class-dependent concepts are identifiable up to composition of permu- tation and an element-wise invertible transformation (Thm.
Insights. Structural diversity suggests that for each class- dependent concept z;, there exists a set of classes such that z, is unique to one of these classes. For Example 3, the corresponding matrix M is as Fig. 4. Consider (21), there exists set of class indices J = [1,3) s.t. M1,1 # 0 and M O. Meanwhile, M, M1 is the only row with all zero entries in M. M.
2). The proof is included in Appx. B.3.
Assumption 1. (Structural Diversity) For any 2; E ZA. there exists a set of indices J ([J) > 1) and j E J where M. #0 and M = 0 for all & E J, f j. and M,N..) the only row with all zero entries in M. (13
Thus, the structural diversity holds for concept Z1.
5

---

Nonparametric Identification of Latent Concepts
The structural difference in the example ahove implies that 2 can be distinguished by con- sidering these classes. Simul- taneously, we have sufficient information for all the remain- ing concepts, as the submatrix M encompasses the other concepts. Then it is possible to uniquely identify among all the class-dependent hid- den concepts. Coupled with this sufficient diversity for other concepts, we have the Structural Diversity assumption for the nonparametric identifiability of all class- deasndee hidden concepts in a structural view.
Zg. It necessitates least two classes with differing condi- tional distributions. As discussed and empirically verified in (Kong et al., 2022), the likelihood of all classes having identical probability measures is exceedingly slim. Interest- ingly, these two classes may also vary across different Ag. Therefore, this assumption is highly likely to be satisfied in real-world scenarios, as it is virtually impossible for the measures corresponding to all classes (c.g., all kinds of ani- mals in a z00) to be almost identical. A concrete example (Example 6) is provided in Appx. D.2. Furthermore, prior proof techniques typically require at least n. + 1 domains or classes to establish block-wise identifiability (Zheng & Zhang, 2023; Li et al., 2024), and Kong et al. (2022) require 2n + 1. In contrast, two domains often suffice in Thm. 2, an interesting technique that can also benefit other tasks.
C2
C3
C4
21
1
0
0
B2
0
1
1
1
Z3
1 0
0
Figure 4: Example of Structural Diversity.
Comparison with pervious conditions. Different from various assumptions encouraging the sparsity of the struc- ture in the identifiability literature (Rhodes & Lee, 2021; Moran et al., 2021; Zheng et al., 2022: Zheng & Zhang, 2023), our assumption only ensures necessary variability on the dependency structure and could also hold true with relatively dense connections. At the same time, we per- mit arbitrary structures between the class-dependent hidden concepts and the observed variables, while previous work has to assume sparse structure the generating process between latent and observed variables. This flexibility ac- commodates a general generative process, thereby distin- guishing our assumptions from others. Additionally, another standard identification strategy requires 2n + distinct domains or classes to achieve latent variable identifiabil- ity (e.g., (Hyvärinen & Morioka, 2017; Khemakhem al., 2020a; Kong et al., 2022)), condition we do not impose.
Generalized concept learning and beyond. Extending the results on subset of concepts (Thm. and Cor. 1), Thm. 2 provides guarantees for learning all class class-dependent hidden concepts. Unlike prior work restricted to parametric constraints (e.g.. disjointness, linearity), our framework re- lies on Structural Diversity between classes and concepts, enabling broader applicability with sufficient diversity. This aligns with cognitive processes of learning by compari- son, ensuring nonparametric recovery of latent structures. Beyond hidden concept learning, our theory also offers insights into latent variable models without prior knowl- edge, as it builds solely on their basic generative struc- ture. Consequently, some findings may interest disentangle- ment (Hyvärinen et al., 2024), causal representation learning (Schölkopf et al., 2021), object-centric learning (Mansouri et al., 2024), and generalization (Du & Kaelbling, 2024).
Limitations. Of course, since we aim for the general non- parametric identifiability for all class-dependent concepts, there are scenarios where it is impossible to fully recover every hidden concept, even with the help of the Structural Diversity condition. For example, if all dog breeds are defined by overlapping features like "barks" and "furry," a learner could not distinguish breeds without unique distinguishing traits. Here, the lack of unique concepts violates Structural Diversity, preventing identifiability from observational data alone. In such scenarios, prior assumptions from provable concept learning-like disjoint concept representations (c.g., non-overlapping Jacobians), linearity, or additive generating funetions-remain essential to resolve ambiguities (Brady et al., 2023; Lachapelle et al., 2023; Wiedemer et al., 2024). Therefore, our assumption does not supersede the previous ones; rather, it offers new direction that can be helpful for learning hidden concepts with minimal prior knowledge about the system.
Identify ing class-independent concepts. After identify- ing class-dependent concepts, one may still be interested in how to provably uncover the remaining daaseinndeeendnnt concepts, even though they may not stand out the cognitive process due to their invariance. To address this, we present the following informal result with its formal version and proof in Appx. B.4, which identifies all concepts-boot class-dependent and class-ndeppnddent-l nonparametric manner.
Proposition 1 (Learning dlass-independemt concepts; Infor- mal). Consider two observationally equivalent (Defn. 1) models (g. Pz, M) and (ğ. P2, M) as in Sec. 2. Under the assumptions in Thm. 2. furthe assume structural conditions on the connective structure between 2g and and an to reg- ularization. Then for any 2; there exists permutation т and invertible functions h; IR + IR s.s. 2, = h2 (2((())))
To avoid introducing parametric assumptions, we still mainly rely on conditions on the connective structures. Since classes e are not connected to those iaaseinndeen concepts zp. the proposed structural condition on M does not help identify ZB. Thus, we leverage the structural condi-
Discussion on distributional variability. The other as- sumption introduced in Thm. 2 requires the existence of distributional variability for the dise identifiabilty of
6

---

Nonparametric Identification of Latent Concepts
tion between these concepts and the observed variables, as proposed in (Zheng et al., 2022). Consequently, if needed, we can provide nonparametric guarantees under appropriate structural conditions for both Z and z in general settings.
has been open for decades (Spirtes al., 2000; Zheng et al., 2023; 2024b;a). Although this is not the focus of our work, the insights provided here are of independent interest to researchers exploring this longstanding challenge.
3.3. Learning Structure Between Classes and Concepts
4. Experiments
Furthermore, we show that the hidden structure M, which encodes the dependency relations between classes and con- cepts, can also be identified based on multiple classes of ob- servations (Prop. 2). This process parallels human learning, where distinguishing between classes involves recovering underlying structures, such as aligning concepts with their corresponding classes. While hidden structure identifica- tion in complex systems remains an open challenge (Spirtes et al., 2000), our findings provide potential insights toward its resolution. The proof is included in Appx. B.5.
In order to show the identification of hidden concepts based the proposed nonparametric identifiability theory, we con- duct experiments on both synthetic and real-world datasets. It is noteworthy that an extensive body of research has em- pirically verified the ability to learn hidden concepts from various data modalities (Bau et al., 2017; Radford et al., 2017; Alvarez Melis & Jaakkola, 2018; Kim et al., 2018; Yeh et al., 2020; Koh et al., 2020; Bai et al., 2022; Achtibat et al., 2022: Crabbé & van der Schaar, 2022: Liu al., 2023). Furthermore, the application range of concept learn- ing is expanding significantly with recent advancements in foundation models (Park et al., 2023; Rajendran et al., 2024; Jiang et al., 2024). Our results complement previous empiri- cal findings by verifying the proposed theory, and we refer to the extensive previous research outlined above for more applications of concept learning across various scenarios.
Proposition 2 (Learning two observationally equivalent (Defn. 1) models (g,Pz, M) and (g. P2. M) ) as in Sec. 2. Suppose all assumptions in Thm. 1 hold, except Assump. 1. Then M- = PM for a permutation matrix P.
structure). Consider
Insights. In Example 3, the theory from previous sections ensures the identifiability of all latent concepts. With Prop. 2, we can now provably identify the connective structure M between classes and concepts remains unknown, align- ing the identified concepts with the corresponding classes,
Setup. In the considered setting, different samples may correspond to different classes selected by a mask. We structure the dataset as { {(x, c) Ji=1, where N denotes the sample size, and c( is a multi-hot vector represent- ing the classes for the data point x(6), A mask M,, @ cli) is applied to account for the specific class for each sam- ple. We employ regularized maximun-likellhood method during estimation, following the standard approach in (Sor- renson et al., 2020). The objective function is defined as [(0) = [ [- (x | M,,: @c) + XR], where À is the regularization parameter, and R. represents the & norm applied to AM and, if estimating class-independent concepts, also to Daff Following previous work, we use Mean Corre- lation Coefficient (MCC) to measure the alignment between the ground-truth and the recovered latent concepts. The results are from 10 random trials. Additional details and results are provided in Appx. C, such as identification with general noises and supplementary evaluation metrics across various settings.
All assumptions have been discussed in the previous sec- tions. Compared to the previous theories on the identifiabil- ity of latent concepts, the recovery of the hidden connective structure does not necessitate the structural diversity assump- tion (Assump. 1). This allows to uncover the structure in even more general scenarios, if the identification of latent concepts might not be of particular interest.
Connection with structure learning. In addition to re- vealing the hidden class-concept structure, if we consider the class variables с as exogenous to the system and the underlying concept variables z as general hidden variables, the dependency structure between exogenous noises and hidden variables encodes most of the structural information in the system, even if dependencies exist among hidden variables (c.g., a hidden directed acyclic graph (DAG)). In structure learning, similar strategies have been applied to recover the DAG among observed variables by first recov- ering the structure of how exogenous noises influence the system in both linear (Shimizu et al., 2006) and nonlinear (Reizinger et al., 2022) cases the DAG constraint ensures the correspondence between the Jacobian of the mixing function (Zheng et al., 2022) and the adjacency matrix. It is worth noting that identifying the hidden structure in a general nonlinear system from purely observational data (i.e., without interventions) a challenging problem that
Synthetic datasets. We conduct experiments on various synthetic datasets verify the proposed identifiability theory. Specifically, we focus on two settings: learning all class-dependent concepts (Fig. 5) and learning all concepts including llass-independent ones (Fig. 6), under appropriate conditions. For Ours, the observations are generated accord- ing to the assumptions required for the theory; while for Base, no structural conditions on either M or have been imposed, and the other settings stay the same. Comparison with other baselines and more details are included in Appx. C. To measure the element- wise identifiability, we use the
7

---

Nonparametrie Identification of Latent Concepts
Ours
Base
0.8
MCC
0.4
0.2
4
Dimensionality
Dimensionality
Figure 7: Results on Fashion-MNIST. The rows correspond to different concepts of a pullover: "sleeve length," "torso length," and "shoulder width," respectively.
Figure 5: Identification of dess-deeendent concepts
w.r.t. different numbers of concepts.
Ours
Base
1.0
0.8
MCC
0.6
0.4
0.2
4
6
8
10
4
10 Dimensionality
Dimensionality
Figure 8: Results on Fashion-MNIST The rows correspond to different concepts of ankle boot: "heel height," "ankle width," and "toe box width," respectively.
Figure 6: Identification of all concepts w.r.t. different numbers of concepts.
align with our understanding of the corresponding classes. For Flower102, we test the robustness of the recovered con- cept by comparing the same concept across different angles and environments. As seen in Fig. 10, the concept can be consistently identified from the same class across var- ious conditions, further supporting our theory. Therefore, these results indicate that hidden concepts can be identified from observational data alone without the need to specify the generative model, underscoring the practical viability. Moreover, certain concepts are inherently entangled across classes (e.g.. Figs. 13, 14, and 15 in Appx. C.2), making it impossible to disentangle them individually without addi- tional assumptions. This further highlights the practical sig- nificance of our partial identification by local comparison.
standard Mean Correlation Coefficient (MCC) between the ground-truth and estimated concepts.
The results (Figs. and demonstrate that our models achieve higher MCCs compared to the base model in both settings. This suggests that it is possible to identify hidden concepts from purely observational data without making assumptions about the concept type, functional relationships, or parametric generative models. Meanwhile, our models also provide lower variances across different runs, which further verifies theoretical findings. As indicated by these results, hidden concepts can be identified up element- wise transformation and permutation under our conditions, while the base model fails to disentangle and recover most concepts from data, further suggesting the necessity of the proposed conditions.
5. Conclusion
Real-world datasets. To assess the applicability of our pro- posed structural condition in complex practical contexts, performed experiments on five different real-world datasets, i... the Fashion-MNIST (Xiao et al., 2017), EMNIST (Co- hen et al.. 2017). AnimalFace (Si & Zhu, 2011). I). Flower 102 (Nilsback & Zisserman, 2008). and FFHQ (Karras et al., 2019) datasets. We highlight the identified concepts with the largest standard deviations (SDs) for Fashion-MNIST (Figs. 7 and 8), EMNIST (Fig. 12 in Appx. C.2), Animal- Face (Fig. 9), and FFHQ (Figs. 13, 14, and 15 in Appx. C.2). Each row in the figures shows reconstructed images with the corresponding concept value varying to illustrate its effect. Additionally, the rightmost column features a heat map depicting the absolute pixel differences to visualize the influence. Clearly, the semantics of the identified concepts
Drawing inspiration from the fundamental cognitive mechanism of learning through comparison, establish set of theoretical guarantees for learning concepts in general nonparametric settings. We provide a theoretical framework that potentially explains the impressive empirical successes in many previous works. Specifically, we prove that hidden concepts can be identified up to trivial indeterminacy from diverse classes of observations without any assumptions on the concept types, functional relations, or parametric generating models. Interestingly, even in scenarios where the structural conditions do not universally hold, we can still provide appropriate identifiability for a subset of concepts with sufficient diversity based on the mechanism of local comparison, thereby greatly broadening the applicability of the proposed theory. Furthermore, the connective structure
8

---

Nonparametrie Identification of Latent Concepts
Bai, A., Yeh, C.-K., Lin, N. Y., Ravikumar, P. K., and Hsieh, C.-J. Concept gradient: Concept-based interpretation without linear assumption. In The Eleventh International Conference on Learning Representations, 2022.
Figure 9: Results on Face The rows correspond to "Ursid" and "Monochrome" of panda, respectively.
Bau, D., Zhou, B., Khosla, A., Oliva, A., and Torralba, A. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6541-6549, 2017.
Brady, J., Zimmermann, R. S., Sharma, Y., Schölkopf, B., Von Kügelgen, J., and Brendel, W. Provably learning object-centric representations. International Confer- ence on Machine Learning, pp.
The same concept ("Blooming") consistently identified from different environments in Flower102.
Figure 10:
PMLR, 2023.
between classes and concepts can also be recovered in nonparametric manner. As current limitation, future work involves exploiting the theory to wider range of practical problems, such as compositional generalization, decision- making, controllable generation, and foundation models.
Bruner, J. S., Goodnow, J. J., and Austin, G. A. A study of thinking. AIBS Bulletin, 7(1):40, 1957.
Chen, R. T., Li, X., Grosse, R. B., and Duvenaud, D. K. Isolating sources of disentanglement in variational au- toencoders. Advances in neural information processing systems, 31, 2018.
Impact Statement
Cohen, G., Afshar, S., Tapson, J., and Van Schaik, A. Em- nist: Extending mnist to handwritten letters. In 2017 in- ternational joint conference on neural networks (IJCNN) pp. 2921-2926. IEEE, 2017.
This paper presents work whose goal is advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Crabbé, J. and van der Schaar, M. Concept activation re- gions: A generalized framework for concept-based ex- planations. Advances in Neural Information Processing Systems, 15:2590-2600 2022.
Acknowledgment
We appreciate the anonymous reviewers and the area chair for their constructive feedback. We would also like to acknowledge the support from NSF Award No. 2229881, AI Institute for Societal Decision Making (AI- SDM), the National Institutes of Health (NIH) under Con- tract R01HL159805, and grants from Quris AI, Florin Court Capital, and MBZUAI-WIS Joint Program.
Delfosse, Q. Sztwiertnia, S., Stammer, W., Rothermel, M., and Kersting, K. Interpretable concept bottlenecks to align reinforcement learning agents. arXiv preprint arXiv:2401.0582) 2024.
Du, Y. and Kaelbling, L. Compositional generative model- ing: A single model is not all you need. arXiv preprint arXiv:2402.011 103, 2024.
References
Du, Y., Smith, K. A., Ullman, T., Tenenbaum, J. B., and Wu, J. Unsupervised discovery of 3d physical objects from video. In International Conference on Learning Representations, 2021.
Achtibat, R., Dreyer, M., Fisenbraun, I., Bosse, S., Wie- gand, T., Samek, W., and Lapuschkin, S. From" where" to" what": Towards human-understandable explanations through concept relevance propagation. arXiv preprint arXiv:2206.000 03208, 2022.
Eastwood, C. and Williams, C. K. I. A framework for the quantitative evaluation of disentangled representations. ICLR, 2018
Alvarez Melis, D. and Jaakkola, T. Towards robust in- terpretability with self-explaining neural networks. Ad- vances in neural information processing systems, 31, 2018.
Eimas, P. D., Siqueland, E. R., Jusczyk, P., and Vigorito, J. Speech perception infants. Science, 171(3968): 303-306, 1971.
Ardizzone, L., Bungert, T., Draxler, F., Köthe, U., Kruse, J., Schmier, R., and Sorrenson, P. Framework for Eas- ily Invertible Architectures (FrEIA), 2018-2022. URL https: /tts:///ithiil cwwcccoo//iii...eeeeeooooeeccess http/viallea
Fodor, J. A. and Pylyshyn, Z. W. Connectionism and cogni- tive architecture: A critical analysis. Cognition, 28(1-2): 3-71, 1988.
9

---

Nonparametric Identification of Latent Concepts
Gentner, D. and Namy, L. L. Comparison in the devel- opment of categories. Cognitive Development, 14(4): 487-513, 1999.
Karras, T., Laine, S., and Aila, T. A style-based genera- tor architecture for generative adversarial networks. In CVPR, 2019.
Gibson, E. J. Perceptual learning. Annual Review of Psy- chology, 14(1):29-56, 1963.
Khemakhem, I., Kingma, D., Monti, R., and Hyvärinen, A. Variational autoencoders and nonlinear ICA: unifying framework. In International Conference on Artificial Intelligence and Statistics, pp. 2202-2217. PMLR, 2020a.
Gibson, E. J. Principles of Perceptual Learning and Devel- opment. Appleton-Century -roofts, New York, 1969.
Khemakhem, I., Monti, R., Kingma, D., and Hyvarinen, Ice-beem: Identifiable conditional energy energy-based deep models based nonlinear ica. Advances in Neural Infor- mation Processing Systems, 33: 11222776777777 2020b.
Grother, and Hanaoka, K. Nist special database 19. Handprinted forms and characters database, National Institute of Standards and Technology. 10:69, 1995.
Kim, B., Wattenberg, M., Gilmer, J., Cai, C.,, Wexler, J., Viegas, F., et al. Interpretability beyond feature attribu- tion: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pp. 2668-26666666 2677. PMLR, 2018
Grupen, N., Jaques, N., Kim, B., and Omidshafiei, S. Concept-based understanding of emergent multi-agent behavior. In Deep Reinforcement Learning Workshop NeurlPS 2022, 2022.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by two time-scale update rule converge to a local nash equilibrium. In NeurlPS, 2017.
Kingma, D. P. and Dhariwal, Glow: Generative flow with invertible 1x1 convolutions. Advances neural information processing systems, 31, 2018.
Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, Concept bottleneck models. In International conference on machine learning, pp. 5338- 5348. PMLR, 2020.
Hyvärinen, A. and Morioka, H. Unsupervised feature ex- traction by time-contrastive learning and nonlinear ICA. Advances in Neural Information Processing Systems, 29: 3765-3773, 2016.
Kong, L., Xie, S., Yao, W., Zheng, Y., Chen, G., Stojanov, P., Akinwande, V., and Zhang, K. Partial disentanglement for domain adaptation. In International Conference on Machine Learning, pp. 11455-11472, PMLR, 2022.
Hyvärinen, A. and Morioka, H. Nonlinear ICA of tem- porally dependent stationary sources. International Conference on Artificial Intelligence and Statistics, pp. 460-469. PMLR, 2017.
Lachapelle, S., López, P. R., Sharma, Y., Everett, K., Priol, R. L., Lacoste, A., and Lacoste-Julien, S. Disentangle- ment via mechanism sparsity regularization: A new prin- ciple for nonlinear ICA. Conference Causal Learning and Reasoning, 2022.
Hyvärinen, A. and Pajunen, P. Nonlinear independent com- ponent analysis: Existence and uniqueness results. Neural networks, 12(3):429-453 1999.
Hyvärinen, A., Khemakhem. I., and Monti, R. Identifiability of latent-variable and sruetural-equation models: from linear to nonlinear. Annals of the Institute of Statistical Mathematics, 76(1):1-33, 2024.
Lachapelle, S., Mahajan, D., Mitliagkas, I., and Lacoste- Julien, S. Additive decoders for latent variables identifi- cation and cartesian producct extrapolation. Advances in Neural Information Processing Systems, 36, 2023.
Janner, M., Du Y., Tenenbaum, J. B., and Levine, S. Plan- ning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.0990 2022.
Leemann, T., Kirchhof, M., Rong, Y., Kasneci, E., and Kasneci, When are post-hoe conceptual explanations identifiable? In Uncertainty in Artificial Intelligence, pp. 1207-1218. PMLR, 2023.
Jia, Y., McDermid, J.. Lawton, T., and Habli, I. The role of explainability in assuring safety of machine learning in healthcare. IEEE Transactions on Emerging Topics in Computing, (7(4)):746-1766 2022.
Li, Z., Cai, R., Chen, G., Sun, B., Hao, Z., and Zhang, K. Subspace identification for tion. Advances Neural Information Processing Systems, 36, 2024.
domain adapta-
Jiang, Y., Rajendran. G., Ravikumar, P., Aragam, B., and Veitch, On the origins of linear representations in large language models. Proceedings of the 41nd International Conference on Machine Learning (ICMI. 2024), 2024.
Liu, N., Du, Y., Li, S., Tenenbaum, J. B., and Torralba, A. Unsupervised compositional concepts discovery with text-to-image generative models. In Proceedings of the
10

---

Nonparametric Identification of Latent Concepts
IEEE/CVF International Conference on Computer Vision, pp. 2085 20952 2023.
Reizinger, P., Bizeul, A., Juhos, A., Vogt, J. E., Balestriero, R., Brendel, W., and Klindt, D. Cross-entropy is all you need to invert the data generating process. arXiv preprint arXiv:2410.22 241.:299,,,,169, 2024.
Locatello, F.. Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Schölkopf, B., and Bachem, O. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference machine learning, pp. 4114-4124, PMLR, 2019.
Rhodes, T. and Lee, D. Local disentanglement in varia- tional auto-encoders using jacobian l_1 regularization. Advances in Neural Information Processing Systems, 34: 22708-22719, 2021.
Mansouri, A., Hartford, J., Zhang, Y., and Bengio, Y. Ob- ject centric architectures enable efficient causal repre- sentation learning. In The Twelfth International Confer- ence on Learning Representations, 2024. URL https: //openreview net
Rosch, E. H. Natural categories. Cognitive psychology, (3):3283350, 1973.
net http://www.ne//ooomiiea F9 ixxxxxx..
Schölkopf. B., Locatello, F., Bauer, S., Ke. N. R., Kalch- brenner, N., Goyal, A., and Bengio, Y. Toward causal representation learning. Proceedings of the IEEE, 109(5):
Marconato, E., Lachapelle, S., Weichwald, S., and Gresele, L. All or none: Identifiable linear properties of next- token predictors in language modeling. arXiv preprint arXiv:2410.23501, 2024a.
2021.
Shimizu, S., Hoyer, P. O.., Hyvärinen, A., Kerminen, A., and Jordan, M. A linear non- con-gaussian acyelic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
Marconato, E., Teso, S., Vergari, A., and Passerini, A. Not all neuro-symbolic concepts are created equal: Analysis and mitigation of reasoning shortcuts. Advances Neural Information Processing Systems, 36, 2024b.
Si, Z. and Zhu, S.-C. Learning hybrid image templates (hit) by information projection. IEEE Transactions on pattern analysis and machine intelligence, 1477):1354 1367, 2011.
Mitchell, T. M. Machine learning, volume 1. McGraw-hill New York, 1997.
Moran, G. E., Sridhar, D., Wang, Y., and Blei, D. M. Identi- fiable deep generative models via sparse decoding. arXiv preprint arXiv:21 10804, 2021.
Sorrenson, P., Rother, C., and Köthe, U. Disentanglement by nonlinear ICA with general incompressible-ffoo net- works (GIN). arXiv preprint arXiv:2001.04872 2020.
Nilsback, M.-E. and Zisserman, A. Automated flower clas- sification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp. 722-729. IEEE, 2008.
Spirtes, P., Glymour, C. N., Scheines, R., and Heckerman, D. Causation, prediction, and search. MIT press, 2000.
Sreedharan, S., Soni, U.. Verma, M.. Srivastava, S., and Kambhampati, S. Bridging the gap: Providing post-hoc symbolic explanations for sequential decision- making problems with inscrutable representations. arXiv preprint arXiv:2002.01080 2020.
Park, K., Choe, Y. J., and Veitch, V. The linear represen- tation hypothesis and the geometry of large language models. arXiv preprint 1.XXiv:3311.066633 2023.
Poeta, E., Ciravegna, G., Pastor, E., Cerquitelli, T., and Bar- alis, E. sancept-based explainable artificial intelligence: A survey. arXiv preprint arXiv:2312.129366 2023.
Taeb, A., Ruggeri, N., Schnuck, C., and Yang. F. Provable concept learning for interpretable predictions using vari- ational autoencoders. arXiv preprint 2rXiv:2204.00992 2022.
Radford, A., Jozefowicz, R., and Sutskever, I. Learning to generate reviews and discovering sentiment. arXiv preprint 1rXiv:1704.01444,, 2017.
Valiant, L. G. A theory of the learnable. Communications of the ACM, 27(11):1134 1142, 1984.
Rajendran, G., Buchholz, S., Aragam, B., Schölkopf, B., and Ravikumar, P. K. From causal to concept-based representation learning. In The Thirth-eighth Annual Conference on Neural Information Processing Systems, 2024.
Von Kügelgen, J., Sharma, Y., Gresele, L.,, Brendel, W., Schölkopf, B., Besserve, M., and Locatello, F. Self- supervised learning with data augmentations provably isolates content from style. Advances in neural informa- tion processing systems, 34:16451-16461 2021.
Reizinger, P., Sharma, Y., Bethge, M., Schölkopf, B., Huszár, F. and Brendel, W. Jacobian- causal dis- covery with nonlinear ica. Transactions on Machine Learning Research, 2022.
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The caltech-ucsd report, California Institute of Technology, 2011.
dataset. Technical

---

Nonparametrie Identification of Latent Concepts
Wiedemer, T., Brady, J., Panfilov, A., Juhos, A., Bethge, M., and Brendel, W. Provable compositional gener- alization for object-centric learning. In The Twelfth International Conference on Learning Representations, 2024. URL wttp://eeppeeeeeeeeee.eeemmeeeee forum? id 1d=VVPIUWKiDQ
Zhou, B., Sun, Y., Bau, D., and Torralba, A. Interpretable
basis decomposition for visual explanation. In Proceed- ings of the European Conference Computer Vision (ECCV), pp. 119,134, 2018.
Xian, Y., Schiele, B., and Akata, Z. Zero-shot learning the good, the bad and the ugly. In CVPR, 2018.
Xiao, H., Rasul, K., and Vollgraf, R. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms, 2017.
Yeh, C.-K., Kim, B., Arik, S., Li, C.-L.,, Pfister, T., and Ravikumar, P. completeness-aware concept-based explanations deep neural networks. Advances in neural information processing systems, 33:20554-20565, 2020.
Zabounidis, R., Campbell, J., Stepputtis, S., Hughes, D., and Sycara, K. P. Concept learning for interpretable multi- agent reinforcement learning. In Conference Robot Learning, pp. 1828-1888 1837. PMLR, 2023.
Zhang, K., Xie, S., Ng, I., and Zheng, Y. Causal repre- sentation learning from multiple distributions: A general setting. In International Conference on Machine Learn- ing, pp. 60057-60075. PMLR, 2024.
Zheng, Y. and Zhang, K. Generalizing nonlinear ica be- yond structural sparsity. Advances in Neural Information Processing Systems,
2023.
Zheng, Y., Ng, I., and Zhang, K. On the identifiability of nonlinear ICA: Sparsity and beyond. Advances in Neural Information Processing Systems,
1-16422, 2022.
Zheng, Y., Ng. I., Fan, Y., and Zhang, K. Generalized precision matrix for scalable estimation of nonparamet- ric markov networks. arXiv preprint arXiv:2305.11379 2023.
Zheng, Y., Huang, B., Chen, W. Ramsey, J., Gong, M., Cai, R., Shimizu, S., Spirtes, P., and Zhang, K. Causal-learm: Causal discovery in python. Journal of Machine Learning Research, 25(60):1-8, 2024a.
Zheng, Y., Tang. Z. Qiu, Y., Schölkopf, B., and Zhang, K. Detecting and identifying selection structure in sequential data. In International Conference on Machine Learning, pp. 11498-61522 PMLR, 2024b.
Zheng, Y., Liu, Y., Yao, J., Hu, Y., and Zhang, K. Non- parametric factor analysis and beyond. In International Conference on Artificial Intelligence and Statistics, pp. 424 432. PMLR, 2025.
12

---

Nonparametrie Identification of Latent Concepts
Appendices
Table of Contents
Summary of Notation
13
B Proofs
15
B.1 Proof of Theorem
15
B.2 Proof Corollary
B.3 Proof of Theorem 2
18
Proof Proposition
21
B.5 Proof of Proposition 2
23
Experiments
C.I Supplementary Experimental Sctup
26
C.2 Supplementary Experimental Results
26
D Supplementary Discussion
29
D.I Challenges, Motivations, and Implications
................................
29
D.2 Supplementary Examples for Assumptions
31
A. Summary of Notation
We summarize the key notation used throughout the paper to provide a quick reference for readers.
Variables and Functions
x (x Xm) X R"
Observed variables.
(ZA, 2B)
Z
C R where + as Latent concept variables.
Z R Class-dependent concepts influenced by the classes c.
Zg E Re
Class-independent concepts, unaffected by c.
(C). .... C.) Class variables represented vectors, with classes.
f Z Injective generative function mapping latent concepts observations.
Z = gg 0)
Class-dependent concept function parameterized by and (other factors).
0 Additional influencing factors in the function g.
Probabilities and Densities
p(z c) p(za c)p(ZB) Conditional density of latent concepts z given classes c, assuming conditional independence.
p(z | c) = lI PA p(z Mr O c) Factorized density of class-dependent concepts ZA-
E|:] Expectation operator.
P Probability measure.
13

---

Nonparametrie Identification of Latent Concepts
Indices and Sets
A; Index set of concepts corresponding class C;.
2,, Concepts associated with class C;.
ZA,AA, Difference in concept sets between classes and Cj.
I m! x (I,...,7) n}: Set of indices for matrix elements.
L, = [(()) E 2}: Indices corresponding to row i I.
I {i (i,j) E I}: Indices corresponding column j in I.
S c (1.....n Subset of indices.
R E R = Of i S): Subspace of IR where components not in S are zero.
Matrices and Operations
S E Rexb An arbitrary matrix with the shape (a, b).
S,., J i-th row, j-th column of matrix S.
supp(S) = ((.,)) S. # 0}: Support of matrix S.
supp(S(O)) ((t.)) 30 e, S(O),) + 0} Support of a matrix-valued function S(e).
Deg Partial derivative of g with respect to class labels c.
D = supp(Deg) Support of the Jacobian of g with respect to
T Matrix-valued function representing transformation between Deg and Değ.
T: A matrix sharing the same support as T.
M {0,1)"A**; Binary structure matrix showing connections between classes and concepts.
G Element-wisc (Hadamard) product.
spanf-} Linear span of a set of vectors.
rank(-): Rank of matrix.
Data and Parameters
((x,c) (2 Dataset of N samples with observed variables and corresponding classes.
M: Mask applied to classes in the dataset.
A Regularization parameter used in the estimation objective.
Regularization term (e.g., l norm applied to estimated supports).
Permutation function used to align estimated concepts.
Parameter space.
14

---

Nonparametrie Identification of Latent Concepts
Conventions
Bold lowercase letters (c.g., x) denote variables, with their realizations denoted by the plain version (e.g., for x); uppercase letters (e.g., S, M) denote matrices.
Calligraphic letters (e.g., X, Z) denote sets or spaces.
Subscripts with dots denote slicing: St, represents the i-th row; S.j represents the j-th column; Si. denotes the first k dimensions of the i-th row and S1,++1- dentoes the remaining; S.k.j denotes the first dimensions the j-th column and 1k+ denotes the remaining.
Estimated quantities are denoted with hats (e.g., 2 for estimated latent concepts).
B. Proofs
B.1. Proof of Theorem
Theorem (Learning by pairwise comparison). Consider two observationally equivalent (Defn. models (g.Pa,
Pa. M) and
(g. Pa, M) as in Sec. 2. Suppose, with an ln regularization on DeO ((D) S [D)), there exist set of points {(c, 0) (0))))) such that
IP.d
/t1
1. ())(((,,)00)),)) IP.1, are linearly independent:
2. ()) R D..
Then for any pair of classes C₂ and Cj, there exists permutation such that the unique concepts of each class BRALA and (he(A))As) equal to zero.
disentangled with other concepts, i.e. OZA,
oza,
Proof. Because of the observational equivalence, we have
Pxjc Pxle = Pf(a)ee Pf(x)(e
(4)
Based the change-of-variable formula, and the invertibility of the generating function there exists invertible function t = f- f such that 2 t(z). By taking the derivatives of both sides w.r.t. we have
DeO TDe9,
where T is matrix-value function that is invertible. According to the assumption, the span is nondegenerate in the sense that
Deg((e, 0) Deg((c, 0)
(6)
Deg((c, 0)(1).11))
are linearly independent. Then construct one-hot vector eio E RD,,, for any in E D.., linear combination of vectors (Deg((c, ()))))))) 1P.al i.e., Cip Eeep., BeDeg((c, ())))))))))))))))))))))))))))))) where Be denotes some coefficient. Note that we
define D as the support of Deg. We also define T as matrix with the support of T in Deg TDag. Then have
T. = Teio
Σ LED.
()
According to the assumption, we have
(Deg((c, () DD.
(8)
which is equivalent to Therefore, Eq. (7) implies T. E Raa, D.
Vi E D.y, T. E R D.
(9)
15

---

Nonparametrie Identification of Latent Concepts
This further indicates
V(LJ) (5) DD
(10)
Since T is invertible, we have
RA II
(Tt(T)
Σ
sgn(a)
#0,
(11)
dES
where Son is set of perpmuuutations. Then there must exist at least one non-zero term in the summation, which indicates that
3o E SuAS Vj E fl..... nAi, sgn(a) II T
Tamss *
(12)
J=1
Clearly, there cannot be any term in the product that equals zero, so we have
3 E Snai Vj (L.......A ToC)s # 40.
(13)
Denote T as the support of T.. Thus, it follows that
vi (I,.... (A),(())))),,(() ET,,
(14)
Then it yields
E ET, x (i}.
(15)
Because of Eq. (10), we have
v(i,) E D, (o(1) D. Let us denote #DD) as a row permutation of D, where V(i,j) E D, there must be
(16)
(o()), E (D)
(17)
and
[[D] = [D].
(18)
Furthermore, Eq. (16) indicates that
(DD C D,
(19)
We have the following relation based on the sparsity regularization:
|D] |P]-
(20)
Therefore, we have the following relation: Together with Eq. (19), follows that
|m()] [D] N [D]
(21)
D (D).
(22)
Let us denote the permutation indeterminacy in goal as π s.t.
D = ((T(()))) (i,5) D).
(23)
Given classes c; and cy. for any 2 ZA, have
E. E.
(24)
Because of Eq. (10), this further implics
T. (i) D.
(25)
For any (u) where zu E Z,, suppose we have
(T(1), ET,
(26)
16

---

Nonparametrie Identification of Latent Concepts
which is equivalent
=(v) E T.
(27)
Then according to Eq. (25), we have
(π((),,) T. x (i) ED.
(28)
Based on Eq. (23), Eq. (28) is equivalent to which indicates a contradiction since 2, E EAAAA
(v, ED,
(29)
As result, there must be (T(D), k) & T. Similarly, for any Zu E ZA,,, we can also show by contradiction that there must be (m( & TT Therefore, for any two classes e; and cy, there exists permutation that
(ZA(A))) OZA,
(30)
Similarly, there is also
(Ar(A)) O.
(31)
OzA.
which is the goal.
B.2. Proof of Corollary
Corollary 1 (Learning by local comparison). Consider two observationally equivalent (Defn. 1) models (g. Pa. M) and (g. Pa. Pa.M) as in Sec. 2. Suppose that the assumptions in Thm. hold. Then, for set of classes and its corresponding concept sets AA, with set of indices 1, there exists permutation such that the unique concepts for the class c₁ are disentangled with concepts associated with other classes, i.e., (Be(A,)Aps)) 0.
Bays
Proof. Because all assumptions in Theorem hold, according to the proof of it, know that, for permutation of DD
i.., (D) where
#(D) = ((o(i))))((i))) E D).
(32)
There must be relationship that
==
(33)
Then we want to show that, there exists a permutation x that
(Z.(AA)Ana)) 0.
(34)
ZAp
For any zy ZAy. and its corresponding class Cy c, and q + i, have
(k, E D.
(35)
According to the proof of Theorem 1, we have
TDeg((c, ()) E ER
(36)
Therefore, Eq. (35) further indicates that
T. {a) E D.,
(37)
where T denotes the support of T. Define the permutation
D = ((T(()))) (i.1) e D).
(38)
Then we consider any (v) where we have
Zu AAAAAA
(39)
17

---

Nonparametrie Identification of Latent Concepts
Suppose we have
(#()), ET.
(40)
This also implies that Based on Eq. (37), we further have
<(v) ET,
(41)
(m(v), E T, x (a) E
(42)
According to the definition of D, this is equivalent to
(0. q) E D,
(43)
Because Zu AAre the above equation indicates that there must be Cy = C.. which is contradiction since # i. Therefore, we have
(T( TT.
(44)
This implies that there exists a permutation that
02 O.
(45)
ZAp
This completes the proof.
B.3. Proof Theorem
Theorem (Learning by global comparison). Consider two models (g. Pa. M) and (g, Pa, M) as Sec. Under assumptions in Thm. 1 and Assump. 1, suppose that for any set A2 S with non-zero probability measure and which cannot be expressed as Ban X ZA for any Bun ZB, there exist values of c, ie., c(k) and cl (which may vary across different Au), that
[ p(z cl) +
[ p(z clo) cl
DEEA,
JAEA.
If (g, Par M) and (g.P2, M) are observationally equivalent (Defn. 1), then:
1. (Element-wise Identifiability) For any z, ZA. there exists a permutation x and invertible finntiool h4 : R. + R s.f.
2; -=,(zz()):
2. (Block-wise Identifiability) For 2n, there exists invertible munctibblfunnciio h: R -> R s.I. 2B -((2)))
Proof. Because of the observational equivalence, have
Picc Pxle + Pf(a)ee Pf(x)ee-
(46)
Based on the change-of-variable formula, and the invertibility of the generating function f, there exists an invertible function h = f-1 o such that h()). Using the chain rule, the derivative of with respect be expressed as:
De9 DzhDeg.
(47)
The Jacobian of can be written as:
02A
02
Dzh
Oz
Oz
(48)
JZB
0ZB
OZA
OzB
According steps 1, 2, and in the proof of Theorem 4.2 Kong al. (2022), the bottom-left block of Dzh, i.e.
Dyhnn t consists of only zero entries. As result, the Jacobian equivalent to:
a Oz
0 A
Dzh
Üz
(49)
Ü2 8
Oz
18

---

Nonparametrie Identification of Latent Concepts
Since h is invertible, the determinant of Dyh is non-zero. Together with the structure of the Jacobian matrix, have
OZA A
d2B
del(Dzl = det det
(50)
dz.
which further implies
det 02,1 * 0,
(51)
Öz,
02.
det
* 0.
(52)
Oz.
Since det( (2a izg Sn, = 0, there exists an invertible function hB : ZB - 2B s.t.,
+ 0 and
82g
2B = hB(28).
(53)
Since A is independent of žg and ăg = (zg))), we further have
02 0.
(54)
dz
Then the Jacobian can be represented
0
Dzh
Oza
(55)
0
U2
Üz
Thus, 2B is identifiable up to block-wise invertible transformation, and we have
82.
iE n.a...,,,A,s. E fMa 1,..., n),
dx:
(56)
D2g
0
k (nA +1, n,,
8%.
This implies that
Deÿ = D Delt Deg.
(57)
According to the assumption, we have a set linearly independent vectors as follows
Deg((e, 0) Deg((e, ())))
(58)
Deg((e, 0)(RR.)))
Then we can construct an one-hot vector eio E R2g, for any in Di as a linear combination of vectors (Deg((e, ()))) g.g /|Ppsl iee, Co = EEED, BeDeg((e, 0)(0)) where 83 denotes some coefficient. Note that we de- fine as matrix with the same support of T Deî TDeg, where a matrix-valued function. Then we have
T = Te Σ (TTDes((e,0)0))
(59)
EED.,
According to the assumption, we have
(())((c,))())). E Ra D.,"
ERA
(60)
Therefore, Eq. (59) implies T. E RRA. D.J""
which is equivalent to
Vi D.j. T. E RRA D.
(61)
This further indicates
V(Lj) x (1) DD,
(62)
19

---

Nonparametrie Identification of Latent Concepts
where 7 denotes the support of T. Since T invertible, we have
PA II
det (T)
Σ
sgn
T
# 0,
(63)
d'E
where S is a set of n pprmuutations Then there must exist at least one non-zero term in the summation, which indicates that
RA
Jo E SaAi Vj E fl....,,AAl nAJ. sgu(a)
II
T To) #
(64)
j=1
Clearly, there cannot any term in the product that equals zero, so we have
FF E SNA Aj E (1,....nAA), Totis #0.
(65)
Thus, it follows that
ViE (l.....,nA),,0()) E T.
(66)
Then yields
v(i,)) E (.(((i), TT, {i}.
(67)
Because of Eq. (62), we have
v(i,) E D, (a E D. Let us denote (D) as row permutation of D, where v(t,j) E D, there must be
(68)
(o(1).)) E #D),
(69)
and
[#D) = [D].
(70)
Eq. 68 indicates that According to the sparsity regularization, we have the following relation based on the sparsity regularization:
(D) C D.
(71)
|D] |P]-
(72)
Therefore, we have
[iD]] |D] Z [D}
(73)
Together with Eq. (71), follows that
D D= (D))
(74)
Let us denote the permutation indeterminacy in our goal as π s.t.
D = ((T(1)))) (i.5) e D).
(75)
For latent concept 2;, according the structural diversity assumption (Assump. 1), there exists set of column indices J. where Mi,J only has one non-zero entry. Let denote that non-zero entry Mij. Since M is binary matrix with the support D, we have (i,j)= E DD and (i, k) & DD for any lo J \.
Then, according to the assumption, for any other concept z, where v + i, there must be a class cg s.t. q E jss.
(v.g) DD
(76)
Because of Eq. (62), it follows that
T.p x fgt e D.
(77)
For any (i), suppose we have
(T()),)) ET,
(78)
20

---

Nonparametrie Identification of Latent Concepts
which is equivalent
(() E T.,.
(79)
Then according Eq. (77), we have
(x(1). E T.p x (1) ED.
(80)
Based on Eq. (75), Eq. (80) is equivalent to
(i.9) D.
(81)
This is contradiction since (i,g) $ D for any a E J \j. Thus, for any i E (I,...,naa) and E nA..,,nAA) {i). there must be
(T)),)) TT.
(82)
Because T is invertible, all row must have at least one non-zero entry. Thus, Eq. (82) further implies
(1).))
(83)
Combining both Eqs. (82) and (83) for each E {1, nA J. the transformation between 2 and 3 must be composition of an element-wise invertible transformation and permutation, which is our goal.
B.4. Proof of Proposition 1
Proposition (Learning Mass-independenn concepts; Informal). Consider two observationally equivalent (Defn. 1) models (g. Pa. M) and (g. Pa, M) in Sec. 2. Under the assumptions in Thm. 2, further assume structural conditions on the connective structure between zn and x and an &n regularization. Then for any 2; E 2. there exists a permutation and invertible functions h; R IR s.f. 2; = h;(2x()))
We first present formal version. For brevity, let and f denote the support of Jacobian DzJ and DaJ. respectively, Additionally, T refers a matrix with the same support of Ty D2f D₂ TT where T matrix-valuce function. Generally, the condition on the structure supp (rop( encourages sparsity in the Jacobian of the generating function As verified empirically in previous work (Zheng& & Zhang, 2023), this condition is likely to hold in our setting where the number of observed variables x exceeds the number of class-independent concepts Zn-
Proposition 1 (Learning class- independddnn concepts; Formal). Consider observationally equivalent (Defn. 1) models (g. Pa, M) and (ğ. Pa. M) generated as in Sec. 2. Under the assumptions in Thm. 2, further suppose that, for all z, E with an to regularization
Daf (IFI < {Fl). there exists C₁ s.t. Nkec. supp Dz. Dz.f)i fi,ntti- = {i). Meanwhile, for each E {nA + 1,...,n), n), there exist (emaill s.1. (2a)(z()) jmatti-ee Fiam 1.l are linearly independent, and (2 s.mat
ug FumAtt." Then for any z, E z, there exists a permutation and invertible functions h; lR E
such that 2; h;(Zz(t) for all i.
Proof. Because of the observational equivalence, we have
Pxjc Px|c = Pf(x)]e Pf(x)ee
(84)
Based the change-of-variabe formula, and the invertibility of the generating function f. there exists an invertible function h = f-1 o such that à h(z). According to the proof in Theorem 2, the Jacobian of h w.r.t. z is as follows:
a
0
Dzh
Oz.
(85)
0
82.B B
Oz
At the same time, by using the chain rule h f-1 f, we have
Daf DDh
(86)
which is equivalent to
D&f Daf.mnat..
DDfDDgh mat1-
(87)
21

---

Nonparametric Identification of Latent Concepts
Based Eq. 85, this further indicates that
Dzf mat11 DeS mat+1 Deh nAtl-nAth Then, according the assumption, there must be set of vectors
(88)
Def(a!)) j,nat DaJ Dz (2) iin
(89)
D (zll Dinatt
that are linearly independent. Then we can construct one-hot vector ejo Rag combination of vectors (D.f(z(")) mn J1
for any E Fina+1. linear
1
[6=1
i.e.
eju =
E (oDzf(z(())inatt))
(90)
REFimAti
where Bg denotes some coefficient. Then we have
T jo. Ejo Ty mm
Σ
BeDnS(z BeDa mutt E R t.att
(91)
RED.,
This further implies that, for any j E Finati, we always have TJj. E support as follows:
R Finatti
Thus, we have the connection between
(i.j) F. ma {1 Tzy. f Fmtti
(92)
Then, because of the invertibility of y, its determinant must not equal to zero, i.e.,
n
sgn(o (r)
# 0,
(93)
ES,.
where S is the set of n-permutations. Therefore, there must be least term in the summation that does not equal to zero, i.e.,
ng
3o Sn Vi (1, (1.....nB)) sgn(o)
II ( (1,0()
(1))))) # 0.
(94)
Because sg) # O, every term in the production must not equal to zero, i.e.,
3o E S., Vi .n...,,n)...... T (z )i0(1)) #0.
(95)
This follows that
(I,.,,.., o(j) E T jmj-
(96)
Based on Eq. (92), Eq. (96) further implies that, for any (i,j) E F. Fnu++i we have (i, a(j)) E f. Let us denote a(F) = {ii ai)) (i,j) E FJ. the above connection implies a (F) C F. Together with the sparsity regularization on the estimated Jacobian, we have
| < F
(97)
Because of the definition of oFF), there must be
F (o(F),
(98)
which follows that
(oF)) μ.
(99)
Together with the relation that (F) f. fhhee must be
$ F=o
(100)
22

---

Nonparametrie Identification of Latent Concepts
Suppose T, mma+1. is not composition of permutation matrix and a diagonal matrix, then
3j1 + j2, Ti, 0 T2 + 0.
(101)
Additionally, consider js {l......0 nB. for which
o(ja) E T m.. n Ja.
(102)
Since ji + j2, we can assume ja + ji without loss of generality. Based on assumption, there exists Cj2 9 ji such that niEc, Fi,m++1- = {. Because
js # {ii
n
Fematlo
(103)
iECn
there must exists is Cj, such that Since ji E +1. follows that (is.J1) F. mmt+1. Therefore, according Eq. (92), we have
js $ F Fitm
(104)
{is} Tnmat1 C F. Notice that o(js) E Tima+ti n 7 +1 +1 implies
(105)
(is. {is} x Tama+
(106)
Then by Eqs. (105) and (106), we have
(iz, (())) E F
(107)
This further implies (ia+.aa) F. by Eq. (100), which contradicts Eq. (104). Therefore, we have proven by contradiction that T.. is composition of a permutation matrix and diagonal matrix, which means that the invariant part 2 is identifiable up an element-wi wise invertible transformation and permutation. Together with the element-wise identifiability for concepts in the changing part A given by Theorem 2, we have proved that all latent concepts z (Z. ZB) are identifiable up to an element-wise invertible transformation and permutation.
B.5. Proof of Proposition 2
Proposition 2 (Learning class-concept structure). Consider two observationally equivalent (Defn. 1) models (g.Pa, M) and (g. Pa. M) as in Sec. Suppose all assumptions in Thm. hold, except Assump. 1. Then M = PM for a permutation matrix P.
Proof. Because of the observational equivalence, we have
pxjc Pxle = Pf())] Pf(x)ee
(108)
Based on the change-of-vvaiabhe formula, and the invertibility of the generating function f, there exists an invertible function = f-1 f such that l((z).
Using the chain rule, the derivative of both sides with respect to e can be expressed as:
De9 = DzhDeg.
(109)
The Jacobian of can be written as:
Dzh
Oz
Uz
(110)
02
02,3
ZzA
0z.
According to steps 1, 2, and 3 in the proof of Theorem 4.2 in Kong et al. (2022), the bottom-left block of Dah, i.e.
+ consists of only zero entries. As result, the Jacobian is equivalent to:
a
0z A
Dzh
Oz
(111)
Ü2 8
Oz #
23

---

Nonparametrie Identification of Latent Concepts
Since h is invertible, the determinant of Dyh is non-zero. Together with the structure of the Jacobian matrix, have
82 Az
82B
det(Dxh) det(
det
(112)
dz
which further implies
OZ, + 0,
det
(113)
Öz
d2.
det
* 0.
(114)
Oz
Since det( d u2n
A 0 and
dan C2A
0, there exists an invertible function hB : 2B - 2B s.t.,
ßB hz(2)).
(115)
Since A is independent of 2g and éB = hB(ZB), we further have
Až O.
(116)
Oz
Therefore, the Jacobian of h is
AZ
0
Dzh
Oz
(117)
0
82
Oz
Note that we have
Deî = DohDoas
(118)
which is equivalent to
Deî (DehDeg) = Deh Deg.
(119)
ds Because Sz, of only zero entries. It further indicates that
0 for i E {1...., nA and & (NA + 1, m...., the upper-right block of Deh, ie., D t consists
Deg = manm Deg.
(120)
According to the assumption, we have
Deg((e, ()0)
Deg((c, ()(3))
(121)
Deg((c, (11P))))))
set of linearly independent vectors.
Then we can construct an one-hot vector Cio E R.p., for any in D., as a linear combination of vectors (Deglle, (3) |Paal i.e., eio EiED., BeDol((e, ()(0())))))).).) where denotes some coefficient. Then have
T -Tei
Σ (/T)a LED.,
(122)
Note that we define as the support Deg. Additionally, define T matrix that share the same support T the equation Değ TDeg, where T is matrix-valued function and T T.
According to the assumption, we have
(Deg((c, () E 1844 D.."
(123)
Therefore, Eq. (122) implies T,o 12 D.,""
which is equivalent to
Vig E D.j. T. R D.J
(124)
24

---

Nonparametrie Identification of Latent Concepts
This further indicates
V(L.) x (5) cD
(125)
Since T is invertible, we have
nA II
det(T)=
Σ
sgn(() o))
Tobis
T
(126)
o
where Sn is a set of maa-permutations. Then there must exist at least one non-zero term in the summation, which indicates that
MA
3o E SUAi Vj E fl...., nai, sg)
II
Totos *
T
(127)
==1
Clearly, there cannot be any term in the product that equals zero, so we have
Fo Snas Vj (U.....na), Totlis #
(128)
Thus, it follows that
vi oa), T.,.
(129)
Then it yields
v(i,j) E D. T, x (i}.
(130)
Because of Eq. (125), we have
v(i,j) E D, (o) e Let us denote TDD) as row permutation of D, where V(i,j) E D, there must be
(131)
(o(1).) E (D).
(132)
And it also implies
| (D) = [D]
(133)
Furthermore, Eq. 131 indicates that
=(D) c D
(134)
We have the following relation based the sparsity regularization:
[D] [D]-
(135)
Therefore, have
[T( [D] > [D}.
(136)
Together with Eq. (134), it follows that
D = (D)
(137)
Thus, we have proved the identifiability of up to permutation on row indices. Since M a binary matrix with support of D. have proved the connective structure between classes and concepts up to a row permutation.
25

---

Nonparametrie Identification of Latent Concepts
Ours
Base (a)
Base (b)
0.8
MCC
0.6
0.4
0.2
3
4
4
6
3
4
5
Dimensionality
Dimensionality
Dimensionality
Figure 11: Identification of concepts w.r.t. different numbers of concepts and different settings.
C. Experiments
In this section, we provide more details regarding the experimental setup as well as additional experimental results to further support our theoretical findings.
C.1. Supplementary Experimental Setup
We generate the data following the process outlined in our theorems. For our model that identifies only class-dependent concepts (Fig. 5), the connective structure between classes and concepts is generated according to the Structural Diversity condition. For dass-dee concepts, we sample from two multivariate Gaussian distributions with zero means and variances drawn from uniform distribution on [0.5, 3). consistent with parameters used in previous work (Khemakhem al., 2020b; Sorrenson al., 2020). For model that identifies all hidden concepts, including class-independent (Fig. 6), the connective structure between class-independent concepts and observed variables follows the structural condition in Prop. 1. These class- paass-indepeneent concepts are sampled from single multivariate Gaussian distribution with zero means and variances drawn from a uniform distribution on [0.5, 3). In the base model, we remove the structural constraints on both types of connective structures to verify the necessity of the proposed conditions. All other settings remain the same as ours.
In our model evaluation, we employ the Mean Correlation Coefficient (MCC) measure the alignment between the ground-truth and the recovered latent concepts, which is standard in the literature (Hyvärinen & Morioka, 2016). To calculate MCC, we first compute the pairwise correlation coefficients between the true concepts and the recovered concepts after applying component-win transformation via regression. Following this, we solve an assignment to match each recovered concept to the corresponding ground-truv concept with the highest correlation.
We use Generative Flow (Kingma & Dhariwal, 2018) as the nonlinear generating function. For synthetic settings, the sample size set as 10, 000. Experiments are conducted using the official implementation of GIN2 (Sorrenson al., 2020) with an additional regularization on the Jacobians and FrEIA (Ardizzone al., 2018-2022) for the flow-based generative function. The regularization parameters À is according a search in {0.01, 0.1, 1), and select À 0.1 according to the average MCCs of experiments conducted synthetic datasets. Moreover, all experiments conducted on 12 CPU cores with 16 GB RAM.
C.2. Supplementary Experimental Results
Partial violation of previous conditions. We also conduct experiments evaluate the identification under partial violations of previously established assumptions in the literature of latent variable models. Specifically, we generated datasets with the following conditions:
1. Base (a): The structural sparsity assumption on the mixing structure between latent concepts and observed variables, as outlined in (Zheng et al., 2022), is partially violated for a subset of concepts, with the size randomly selected from all integers in the range to n/2.
26

---

Nonparametric Identification of Latent Concepts
00000000
0
0
0
0
0
1
1
1
1
1
1
/
1
1
1
1
/
2 2 2
2
2
2
2
2
2
222
2
2
2
2
2 3 4
2 3 4
2
2
2
2
2
2
2
2
2
2
2.
3 3
3
3
3
3
3
3
3
3
333
3333
3
3
3
3
33
3
3
3
3
3
3
3
3
3 4 6
3 47
4 1/41/ 4
4
4
4
44
4 4
4
4
4
4
4
4
4
4
4
4
4
4
V 5
5 5 5
5
555555
5
5
5
555
55552
5
5
5
5
6 6 6966666
6
6 66 66666
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
7 7 7 7 7
7777777
7
7
7
7
7
7
7
7
7
7
77
7
7
7
7
7
7
8 8 8
8
8 8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8 9 9 9 9
9 9
9
9
9
9
9
9
9
9
9
9
9
9
9
(a) Angle
(b) Height
(c) Thickness
Figure 12: Results for each digit class in the EMNIST dataset, showing the identified concepts with the top three standard deviations (SDs). Each subfigure represents concept identified by our model, with values ranging from -4 to +4 SDs to demonstrate their impact. The rightmost column features heat map of the absolute pixel differences between -1 and +1 SDs. These concepts can be interpreted as variations in angle, height, and thickness.
Figure 13: Multiple concepts (e.g., skin, eyes, face shape, etc.) corresponding to "Age" are entangled.
2. Base (b): The 2n + domain requirement in (Khemakhem et al., 2020b; Kong et al., 2022) is partially violated. Instead, latent concepts are generated from n + multivariate Gaussian distributions, each with zero mean and variances drawn from uniform distribution over [0.5, 3).
3. Ours: The data-generatiin process adheres to our proposed structural diversity condition. While there are no constraints on the mixing structure between latent concepts and observed variables, the structure between classes and concepts satisfies the required structural diversity.
The results, shown in Fig. 11, indicate that when assumptions from previous works are partially violated, the recovery of latent concepts becomes unreliable, shedding light on the necessity of the proposed flexible guarantees based on learning by comparison. All results are from 10 runs with different random seeds.
Some concepts are naturally entangled. As discussed Sec. 4, we include the results on EMNIST and FFHQ datasets here in the appendix. The EMNIST dataset (Cohen al., 2017) an extension the classical MNIST, which consists of much larger set of handwritten digits derived from the NIST Special Database 19 (Grother & Hanaoka, 1995).
The results are shown in Fig. 12. Similar to the other datasets, we select the identified components with the top three standard deviations and vary the value of the identified components to visualize their potential semantics. According to the results, clear that the hidden concepts can be identified by only learning from diverse classes of observations. This further indicates that the proposed nonparametric identifiability, which is based on the basic cognitive mechanism of learning by comparison, has potential applicability in real-worle scenarios.
27

---

Nonparametric Identification of Latent Concepts
Figure 14: Multiple concepts (e.g., lipstick, eye shadow, powder, etc.) corresponding to "Makeup" are entangled.
6
Figure 15: Multiple concepts (e.g., hairstyle, head shape, eye, etc.) corresponding to "Gender" are entangled.
To explore scenarios where not all concepts can be identified element-wise, we conduct additional real-world experiments on a more complex scenario, i.e., the FFHQ dataset (Karras et al., 2019). The dataset contains 70, 000 human face images, which is more complicated than the datasets in our other experiments.
From Figs. 13, 14, and 15, it evident that some concepts remain entangled and cannot be fully recovered. For instance, for the class "Age", concepts like "skin," "eye," and "face shape" are all entangled together, suggesting that assumptions for component-wise identifiability may not be fully satisfied in this scenario. However, these class-dependent concepts can still be identified as a group, consistent with our theorem based on local or pairwise comparisons. This suggests that, even in complex scenarios where theories fail to guarantee identifiability for all individual concepts due to assumption violations, our alternative identifiability framework based on pairwise comparisons may still provide an alternative theoretical basis for recovering class-dependent concepts collectively, even if they remain entangled. This sheds light on the necessity of our alternative identifiability guarantees in some complicated real-world scenarios.
Learning concepts with noise. We evaluate the robustness of concept recovery in noisy environments. Specifically, introduce non-additive random noise to the Fashion-MNIST images, since additive noise has been extensively studied and can often be removed via deconvolution. The more challenging case with non-additive noise is also shown to be identifiable with rather general conditions (Zheng et al., 2025). Example noisy samples are shown in Fig. 16. Despite the corruption, our method still recovers meaningful concepts (Fig. 17), demonstrating the framework's 's robustness to general noise.
Quantitative evaluation on images. To quantitatively evaluate our framework on images, we conducted series of experiments. Specifically, we report Mutual Information Gap (MIG) (Chen et al., 2018) and DCI scores (Eastwood & Williams, 2018) for synthetic datasets (Figures 18 and 19), and Fréchet Inception Distance (FID) (Heusel et al., 2017) and Perceptual Path Length (PPL.) (Karras et al., 2019) for real-world datasets, RUB-200-2011 (Wah et al., 201 1) and AWA2 (Xian et al., 2018). CUB-200 CUB-200-2011 I contains 200 bird classes with 11,788 images; AWA2 includes 50 animal categories with 37,322 images. We report a Mutual Information Gap (MIG) of 8.135, a DCI score of 16.533, a Fréchet Inception Distance (FID) of 8.11, and a Perceptual Path Length (PPL) of 30.938, indicating strong disentanglement and high-quality generation under our framework, We also visualize the recovered concepts from CUB-200-201 and AWA2 (Figs. 20 and 21).
28

---

Nonparametrie Identification of Latent Concepts
Figure 16: Samples of Fashion-MNIST with noise.
Figure 17: Results noisy Fashion-MNIST. The rows correspond different identified concepts: "sleeve length," "torso
length," and "shoulder width," respectively.
Ours
Partial
Base
0.6
0.4
MIG
0.0
3
4
6
3
4
5
Dimensionality
Dimensionality
Dimensionality
Figure 18: Mutual Information Gap (MIG) w.r.t. 10, 000 samples and different number concepts. All results are from 10 runs with different random seeds. The settings for Ours and Base follow those used the main manuscript. For Partial, we intentionally violate the structural diversity assumption for one randomly selected concept and vary the total number of concepts to control the proportion of the violated concept.
D. Supplementary Discussion
D.1. Challenges, Motivations, and Implications
Challenges of nonparametric identifiability. Theoretical and empirical evidence consistently demonstrates that, without additional assumptions, achieving identifiability in nonparametric generative processes is nearly impossible (Hyvärinen & Pajunen, 1999; Locatello et al., 2019). This challenge extends beyond concept learning to fields such as independent component analysis and causal representation learning. Without additional conditions, it is impossible to ensure that the recovered concepts align with the ground truth, the solution space vastly underconstrained, allowing non-trivial mixtures
29

---

Nonparametrie Identification of Latent Concepts
Ours
Partial
Base
0.8
0.6
D
0.4
0.2
0.0
3
4
6
3
4
Dimensionality
Dimensionality
Dimensionality
Figure 19: Disentanglement, Completeness, and Informativeness (DCI) Disentanglement score w.r.t. 10, 000 samples and different number of concepts. All results are from 10 runs with different random seeds. The settings for Ours and Base follow those used in the main manuscript. For Partial, we intentionally violate the structural diversity assumption for one randomly selected concept and vary the total number of concepts to control the proportion of the violated concept.
Figure 20: Results on CUB-200-2011. The rows correspond to different identified concepts: "color," "orientation," and "size," respectively.
of the true concepts to generate observationally equivalent data. Parametric assumptions, such as linearity, disjointness, or additivity, constrain the solution space, reducing ambiguity and identifying the true generative process up to certain indeterminacies. In contrast, focus is understanding what concepts can be recovered in the general setting, Our nonparametric identifiability results provide insights into this general setting and suggest promising direction.
Learning by pairwise comparison. Theorem demonstrates that for any given pair of classes and their corresponding sets of hidden concepts, the unique concepts in each class can be disentangled from all the remaining concepts. This process is fundamental to the cognitive mechanism learning through comparison. Consider an infant with no prior experience of the world: when presented with two classes, such as cat and a dog, the infant learns and memorizes the unique concepts associated with each class, such as "meows" for the and "barks" for the dog. The invariant concepts, like "furry" or "four-legged," cannot be distinctly learned because they do not provide distinguishing information between the classes. From cognitive science perspective, infants and young learners rely heavily on contrastive features to form distinct categories and concepts (Eimas et al., 1971). For instance, if an infant repeatedly hears cat meow and dog bark, they begin to associate these unique sounds with the respective animals. In contrast, shared attributes like fur or four legs do not stand out because they do not help differentiating between the two animals. This emphasizes the role of unique concepts in early learning and memory, highlighting how pair-wise comparisons are essential in the process of discovering the new world. For machines learn without prior knowledge, we argue that similar mechanisms also help.
Learning by local comparison. Corollary extends these theoretical guarantees from pair-wise comparisons to local comparisons among multiple classes. Although pairwise comparison fundamental to the learning mechanism, local comparison is more efficient in complex scenarios. For instance, when an infant is exposed to a variety of stimuli, they do not learn by isolating pairs indefinitely. Instead, they begin to discern patterns and unique features within a broader context, comparing multiple classes simultaneously. For example, a child distinguishing between a cat, dog, and a bird must identify unique concepts such as "meows," "barks," and "chirp." As the child interacts with these animals different contexts Sontexts-perhaps hearing bird chirp in the park, dog bark at home, and cat meow in the neighbor's yard- they learn to
30

---

Nonparametrie Identification of Latent Concepts
Figure 21: Results on AWA2. The rows correspond to different identified concepts: "number of animals," " "orientation," and "size," respectively.
associate specific sounds and behaviors with each animal. This local comparison ensures that even as the number of classes increases, the child can efficiently disentangle and learn the unique concepts of each class, providing a more complete understanding of the new environment.
Class-concept alignment. Proposition indicates that, the recovered hidden structure between classes and concepts an isomorphism the ground-truth structure. Intuitively, this helps the machine understand which concepts correspond to a given class of observations. While this process may seem straightforward to us, can be challenging for infants or machines without prior experience, it aligns with essential step of learning through comparison. For instance, consider infant presented with set of objects like cat, a dog, and bird (the classes) and set of concepts like "furry," "barks," and "flies." Without proper knowledge, the infant might incorrectly assign "barks" to the cat or "flies" to the dog, lacking the experience to accurately match these concepts with the correct classes. The concept of "furry" might also be mistakenly assigned to the bird, despite its inapplicability. Therefore, to distinguish different classes by their concepts and learn unique concepts through comparison, the machine must first recover the underlying connective structure. This is essential for provably leamning from multiple classes of observations.
D.2. Supplementary Examples for Assumptions
We first include a concrete example for assumptions Thm. and Cor. as follows:
Example 5. Suppose there exist two samples with their corresponding Jacobians given by Deg((e, ()))),)))),,) (0, 1,2) and (eg((c, (2)) = (0, 3, 4). Clearly, these two vectors span 2-dimensional subspace. We can also find matrix T (e.g.. a binary matrix with the same support as T) s.t. [TDeg((e, ())),,,, E P. for l E [1,2). Since identifiability theory considers an infinite number of samples, the requirement for several non-degenerate samples is almost always satisfied asymptotically.
Then we introduce an illustrative example for the distributional variability condition in Thm. 2. Intuitively, the condition is a generic faithfulness assumption, ruling out special parameter combinations that would make the two integrals equal.
Example 6. Consider as a 2-dimensional vector with (²)) 0] and cl*) [0, 1). Let Z R2. and Ag {(21, 22)
R S 21 2,0 < 22 < 1}. The conditional densities are p(z c [1, 0]) 2- (10-1)=-100001 and p(z c
[0, 1]) = 27 -12- 11? 1)? Evaluating the integrals over Az, we have
S J.
(5) (=)=1)=+(
dzy # (( (( 2x
( dz dz2.22
2m
Note that (ks v) can even be different for different Ag, which further weakens the assumption.
31


Reviews:
[{"review_id": "azZ1qUz3vR", "reviewer": "Reviewer_DXpa", "summary": "The authors tackle the important problem of identifying latent classes from data. They claim that given some constraints, it should be possible to do given that different classes (e.g. different animals), will produce different sets of observable concepts (i.e. different shapes, colors, etc); and that as long as there is at least one difference, it should be possible to identify them. The formalize these notions via several theorems and definitions. Finally, they validate these empirically by fitting simple non-parametric matrices corresponding to each pair of class and related concepts. Specifically, they fit a sparse model that attempts to fit concepts that are generated according to their theoretical assumptions by each class. The compare these to other data generating mechanisms that are entangled or do not have sparse structure and show they are harder to fit.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your constructive feedback. Accordingly, we have included several new discussions in the updated version. Please see our point-by-point responses below:\n\n**Q1:** More intuition on Theorem 2 and Figure 4.\n\n**A1:** Thanks for the suggestion. We have added more discussion in the revised version. Theorem 2 guarantees the identifiability of each class-dependent concept under the structural diversity condition. Figure 4 illustrates this condition: for example, for concept $\\mathbf{z}_1$, there exist classes $\\mathbf{c}_1$ and $\\mathbf{c}_3$ s.t. $\\mathbf{z}_1$ is unique to $\\mathbf{c}_1$, allowing that concept to be disentangled. Meanwhile, class-independent concepts can also be disentangled as a block by leveraging distributional variety.\n\n**Q2:** What models did the authors use for the real-world datasets?\n\n**A2:** Thanks for the question. We used a general incompressible-flow network (GIN) following Sorrenson et al., 2020, with an additional sparsity regularization (L365-370 right). We have further highlighted this in the updated manuscript according to your constructive comments.\n\n**Q3:** Relation of theoretical claims with previous work, especially in the ICA literature.\n\n**A3:** Thank you for the insightful comment. We truly appreciate the opportunity to further illustrate how our theoretical results relate to prior work. While Thm. 2 shares ICA’s goal of identifying latent variables, it differs in assumptions, techniques, and scope. We would be very grateful for any further feedback if any part of the following differences is unclear or could be better articulated.\n\n- ***Objective:*** While ICA aims to recover all latent variables individually, our focus is on identifying which concepts are reliably recoverable, grounded in the cognitive process of comparison. Our main contribution lies in learning by local comparisons—showing that unique concepts can be disentangled from any class pair. This naturally extends to global identifiability (Thm. 2), but the local view enables more flexible, practical analysis. In contrast, most previous work in ICA cannot provide any guarantee with partial violation of their assumptions for some latent variables.\n\n- ***Techniques:*** ICA methods are built on elegant and powerful assumptions—such as auxiliary variables or sparse mixing—that enable strong identifiability results. For example, iVAE needs more than 2n+1 environments with sufficiently varying distributions; sparsity-based methods assume sparse concept–observed mixing structure. Our method avoids these by relying on class–concept structures, offering partial identifiability when global assumptions may not fully apply.\n\nIn summary, our theory targets a different question: determining which concepts are identifiable under minimal, localized assumptions. We do not claim generality over ICA, but rather provide complementary insights, especially in settings where global assumptions may be partially violated.\n\n**Q4:** How to extract the concepts from raw input?\n\n**A4:** Thank you for the great question, and we have added more details in the updated version. After estimation (detailed in A2), we recover a latent vector from observed data and class labels. Each dimension of this vector corresponds to a concept. To aid interpretation, we rank dimensions by their variances—based on the heuristic that higher-variance components are often more semantically meaningful.\n\n**Q5:** Rewrite the paragraph in lines 82-92, and add a reference from the cogsci literature.\n\n**A5:** Thanks so much for the great suggestion. We have rewritten the paragraph as follows and added a line of references.\n\n> We address this question by grounding our approach in a fundamental cognitive mechanism: humans learn concepts by contrasting diverse classes of observations. Classic studies have shown that concept formation relies on detecting distinctions across examples—Bruner et al. (1956) emphasized learning through contrasts between exemplars and non-exemplars; Gibson (1963, 1969) proposed differentiation as a basis of perceptual learning in infants; and Gentner & Namy (1999) demonstrated that direct comparison enables children to abstract category-defining features. This principle is further supported by extensive literature across cognitive science, reinforcing that learning by comparison is the key underlying engine.\n\n---\n\nReferences:\n\n*Bruner, J. S., Goodnow, J. J., & Austin, G. A. (1956). A Study of Thinking. Wiley.*\n\n*Gibson, E. J. (1963). Perceptual learning. Annual Review of Psychology.*\n\n*Gibson, E. J. (1969). Principles of perceptual learning and development. Appleton-Century-Crofts.*\n\n*Gentner, D., & Namy, L. L. (1999). Comparison in the development of categories. Cognitive Development.*\n\n---\n\n**Q6:** Move intuitions before the theorems.\n\n**A6:** Thank you very much for the helpful suggestion. We have revised the layout accordingly, placing the intuitions before the theorems for better flow.", "claims_and_evidence": "Yes. The authors theoretically define their model and compare it empirically to one that does not follow their regularization schemes.", "methods_and_evaluation_criteria": "Yes. The authors extensively test their theoretical approach on several datasets.", "theoretical_claims": "I didn't check all the theorems. Some are intuitive, while I admit that others are a bit hard to follow given the notation. For example, theorem 2 and the corresponding figure (Figure 4), were not completely clear to me.", "experimental_designs_or_analyses": "For the most part yes. The simple toy experiments are easy to follow. I have questions about the real world datasets. Specifically, what models did the authors use? I could not find this.", "relation_to_broader_scientific_literature": "I think the authors do a good job to refer to the related literature. I feel like some of their theoretical claims are very similar to previous work, especially in the ICA literature, but they are very thorough about formalizing their ideas.", "essential_references_not_discussed": "Not that I can think of.", "other_strengths_and_weaknesses": "The main strength of the paper is rigorous the authors try to be. I don't know if all the theorems are correct though. Main weakness is that for the real world datasets it is not clear how the concepts related to each class are extracted. A common story I see in these articles is that they abstract away the process of extracting the relevant entities (in this case the class-related concepts) and operate directly on those. If that is what's happening here then that makes the whole article much less useful since part of what makes this task hard is that we need to extract these concepts from raw input.", "other_comments_or_suggestions": "I suggest the authors re-write the paragraph in lines 82-92. If two classes of observations share an identical set of concepts are they really two different classes? It seems like the very set of concepts in the observation is what defines a class (from the perspective of the AI/Human) and not the other way around. To put it another way, if the observations are really generated by different classes, eventually they should differ, if they don't they are effectively the same (assuming we can observe the differences). Also, add a reference from the cogsci literature to the actual mechanisms you are taking inspiration from.\n\nI really like that the authors explain the intuitions behind their theorems and definitions. My only suggestion would be to introduce these intuitions first and the theorems later. A general structure could be: \"Intuitively, structural diversity is important ....\" and then \"We can formalize this notion through theorem ....\". I feel like this is a more natural way to read things than just suddenly having to wade through formulas.", "questions_for_authors": "Can you provide more details about the models used to test on real world data?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H0llpLLXMW", "reviewer": "Reviewer_VdCS", "summary": "This paper proposes a nonparametric framework for identifying latent concepts by leveraging structural diversity across observation classes, inspired by human cognitive mechanisms of learning through comparison. The authors establish theoretical guarantees for concept identifiability without relying on parametric assumptions about concept types, functional relationships, or generative models. Key contributions include:\n\n1. **Pairwise and local comparison theorems** (Thm. 1, Prop. 1) showing that unique concepts in classes can be disentangled when sufficient diversity exists.\n2. **Global identifiability** (Thm. 2) under structural diversity conditions, recovering class-dependent concepts up to permutation/element-wise transforms.\n3. **Structure recovery** (Prop. 3) of class-concept relationships.\n4. Empirical validation on synthetic data and real-world datasets (Fashion-MNIST, AnimalFace, FFHQ), demonstrating alignment between identified concepts and semantic attributes.\n\n\n## update after rebuttal\n\nThanks for the authors' rebuttal which addressed my concern. So I increase my score to clear accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are profoundly thankful for your insightful feedback. In light of it, we have included new discussions and conducted **new experiments** (https://anonymous.4open.science/r/0-518C/rebuttal_new_results.pdf). Please find our point-by-point responses below:\n\n**Q1:** Thm. 1 in high-dimensional space or under noisy observations.\n\n**A1:** Thank you for the question. Theoretically, the assumption only requires Jacobians to be linearly independent at some points to ensure meaningful variation—which can be expected from large populations. We have also added new experiments in light of your suggestions:\n\n- **High-dimensional space:** Beyond the five image datasets used before, we added two more (Figs. 5–8, Table 1 in the link). Since images are naturally high-dimensional, these results across seven datasets support applicability in such settings.\n\n- **Noisy observations:** We added experiments with noisy images (Figs. 3-4 in the link), showing meaningful concepts remain identifiable.\n\n**Q2:** Practical applicability of structural diversity.\n\n**A2:** Thanks for the suggestion. We have added new experiments under partial violations (Figs. 1–2 in the link), showing robustness. Intuitively, if a concept (e.g., “furry”) is absent in some classes, it can be disentangled. If all classes share it, it is unidentifiable under Thm. 2. However, our local comparison framework (Thm. 1, Prop. 1) still ensures identifiability even without global structural diversity.\n\n\n**Q3:** Reproducibility details for Jacobian regularization and synthetic data.\n\n**A3:** In light of your suggestion, we have highlighted details further in the updated version. Due to the rebuttal policy, we can only include figures in the anonymous link. We approximate ℓ₀ regularization with an ℓ₁ norm (L369 right) since ℓ₀ norm is known to be indifferentiable. We use a flow-based estimator (Sorrenson et al., 2020) to recover latent concepts (L365–367 right), though our theory is estimator-agnostic—as long as the observed distribution is matched and regularization applied. For synthetic data, we randomly construct a class-concept structure satisfying our conditions and sample each latent variable conditioned on its class labels.\n\n\n**Q4:** Suggestions on empirical evaluations.\n\n**A4:** Thank you so much. We added the following new experiments accordingly:\n\n- **Q4.1:** Partial violation of structural diversity.\n\n   **A4.1:** Evaluated in Figs. 1–2 in the anonymous link; results show robustness.\n\n- **Q4.2:** Quantitative metric for real-world evaluations.\n\n   **A4.2:** We added Fréchet Inception Distance (FID) and Perceptual Path Length (PPL). As shown in Table 1 in the link, ours consistently outperforms baselines.\n\n- **Q4.3:** More metrics for simulation.\n\n   **A4.3:** We added DCI score and Mutual Information Gap (MIG) (Figs. 1–2 in the link); same conclusion holds.\n\n**Q5:** Entangled concepts on FFHQ.\n\n**A5:** Thanks for your question. Learning by local comparisons focuses on the unique part of each class (e.g., a group of dependent concepts), enabling partial identification even when full disentanglement is impossible. FFHQ illustrates this: despite global violations, concept groups remain identifiable (L1460–1467). In addition to the full identifiability (shown in other datasets), this partial identifiability is also one of our main focuses.\n\n**Q6:** Baseline comparisons lack details.\n\n**A6:** Thanks. As noted in L407 left, all models share the same setting except the data-generating process. We have highlighted this further in the updated manuscript.\n\n\n**Q7:** Some steps (e.g., Eqs. 82–83) lack intuition.\n\n**A7:** Thank you—we have added more intuition accordingly. For any $i \\in \\\\{1,\\ldots,n_A\\\\}$ and $v \\in \\\\{1,\\ldots,n_A\\\\} \\setminus i $, $(\\pi(i), v) \\notin \\mathcal{T}$ (apologies for a typo—$k$ in L1053 should be $v$) implies that all entries in row $\\pi(i)$ of matrix $\\mathrm{T}$ are zero except possibly at index $i$. Since $\\mathrm{T}$ is inveritble, each row must have a non-zero entry, implying $(\\pi(i), i) \\in \\mathcal{T}$.\n\n\n**Q8:** Measuring structural diversity in practice.\n\n**A8:** Thanks for the excellent point. We have added a discussion in the updated version. Structural diversity can be assessed via concept–class mutual information (as you kindly suggested) and domain knowledge (e.g., “sunshine” may be universal, “furry” is class-specific). Even if violated or unsure, our local comparison framework ensures alternative identifiability.\n\n**Q9:** Compare with Delfosse et al., 2024 on differences in identifiability guarantees.\n\n**A9:** Thanks for the suggestion. For Delfosse et al. (2024), did you mean *“Interpretable concept bottlenecks to align reinforcement learning agents”*? We are not sure if we understood correctly, as we did not find identifiability theories in that paper. Please let us know if we missed or misunderstood anything—we would be more than happy to include a discussion accordingly.", "claims_and_evidence": "The proofs for Thm. 1 (pairwise disentanglement) and Thm. 2 (global identifiability) were examined.\n\nWhile the linear algebra arguments and Jacobian-based reasoning are logically structured, two concerns arise:\n\n- **Thm. 1** assumes linear independence of Jacobian vectors (Eq. 6). The proof does not address whether this holds in high-dimensional spaces or under noisy observations, potentially limiting practical applicability.\n- **Thm. 2** relies on the structural diversity assumption (Assump. 1), which requires each concept to be uniquely tied to a class subset. The proof does not quantify how often this condition fails in real-world scenarios (e.g., overlapping concepts like \"furry\" for cats/dogs).", "methods_and_evaluation_criteria": "**Methods**: The nonparametric framework is novel and well-motivated by cognitive mechanisms. However, the paper does not provide an explicit algorithm for estimating concepts (e.g., how to enforce ℓ0ℓ0 regularization in practice). This limits reproducibility.\n\n**Evaluation**:\n\n- **Synthetic experiments** validate identifiability under ideal conditions but do not test robustness to structural assumption violations (e.g., partial failures of Assump. 1).\n- **Real-world evaluations** rely on qualitative interpretation (e.g., heatmaps in Fig. 8–9) without quantitative metrics for concept interpretability (e.g., concept-accuracy scores via human trials or downstream tasks).\n- The MCC metric measures alignment but does not assess disentanglement quality (e.g., mutual information between recovered concepts).", "theoretical_claims": "I checked the correctness of proofs. All proofs are correct.", "experimental_designs_or_analyses": "Experiments on synthetic data validate identifiability under controlled structural conditions. However:\n\n- Real-world results (e.g., FFHQ in Fig. 13–15) show entangled concepts (e.g., \"age\" involving skin/eyes), yet the paper claims partial identifiability via local comparisons. This discrepancy is not rigorously analyzed.\n- Baseline comparisons (e.g., \"Base\" models) lack details on architecture/optimization parity, raising questions about fairness.\n- The MCC metric measures alignment but does not assess semantic interpretability, a critical aspect of concept learning.", "relation_to_broader_scientific_literature": "The work extends identifiability theory in concept learning, contrasting with parametric approaches (e.g., linear ICA [Hyvarinen & Morioka, 2016], additive models [Lachapelle et al., 2023]). It aligns with cognitive-inspired methods [Brady et al., 2023] but diverges by eliminating parametric constraints. The structural diversity condition parallels sparsity assumptions in nonlinear ICA [Zheng et al., 2022], though the focus on class-concept relationships is novel.", "essential_references_not_discussed": "No essential references are not discussed.", "other_strengths_and_weaknesses": "see above.", "other_comments_or_suggestions": "1. Include a discussion on how structural diversity could be measured/validated in practice (e.g., via concept-class mutual information).\n2. Compare with concurrent work on concept bottlenecks (Delfosse et al., 2024) to highlight differences in identifiability guarantees.\n3. Release code for reproducibility, especially for Jacobian regularization and synthetic data generation.", "questions_for_authors": "The questions are listed above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u29h3TVvuD", "reviewer": "Reviewer_kdtg", "summary": "The paper studies the identifiability of latent concepts. The key assumption is that we observe class labels $c$ and observations $x$ which are mediated through concepts $z$, i.e., $c\\to z\\to x$ and they in addition allow for class independent features. Then they show that the difference in concepts for different classes can be identified and under a diversity condition the class dependent features are identifiable and the remaining features are block identifiable. Then they conduct experiments on synthetic and real data using established methods to investigate their identifiability results.\n\n## update after rebuttal\n\nAs explained in the rebuttal comment,  I maintain my (overall positive) opinion", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your valuable insights. In light of these, we have carefully added several new discussions and conducted **new experiments** (https://anonymous.4open.science/r/0-518C/rebuttal_new_results.pdf). Please find our detailed responses below:\n\n**Q1:** More clarifications on Eq. 5.\n\n**A1:** Thanks for your suggestion, we have added further clarifications accordingly. $\\mathbf{T}$ in Eq. 5 is invertible because the mapping $t$ is invertible. The invertibility of $t$ follows from Eq. 4 (L743-746), independent of $g$. Eq. 5 simply represents the derivatives w.r.t. $\\mathbf{c}$, and $\\theta$ denotes other factors independent of $\\mathbf{c}$.\n\n\n**Q2:** Is the comparison based on the same method under violated assumptions?\n\n**A2:** Yes. The estimation method (previous work + regularization) is unchanged; only the data-generating process is altered to violate structural conditions (L407 left). We have now emphasized this more explicitly in light of your great question.\n\n**Q3:** The technical relation to previous methods such as iVAE and sparsity approaches.\n\n**A3:** Thanks a lot for the insightful comment. iVAE (and related methods) require sufficient distributional change (e.g., $2n+1$ environments), while we focus on how classes affect concepts—structure, not magnitude, of change. Our theory is closer in spirit to sparsity-based methods, as both leverage structural relationships. However, there are two key differences:\n- We study class–concept structures, while prior work focuses on concept–measurement structures. This relaxes assumptions on mixing function by weak supervision from class labels.\n\n- Our pairwise comparison framework supports identifiability under partial violations, unlike prior work that requires full structural assumptions.\n\nWe hope this makes the distinctions clearer and would be more than happy to further refine it based on your feedback.\n\n**Q4:** Possible missingness in L117 left.\n\n**A4:** Thanks. We were not certain we understood correctly, so we assumed you suggested moving L124-125 ($p(\\mathbf{z}|\\mathbf{c}) = p(\\mathbf{z}_A|\\mathbf{c}) p(\\mathbf{z}_B)$,  $\\mathbf{z}_A \\coloneqq g(\\mathbf{c}, \\theta)$) alongside L117 (Eq. 1). We've made this change, but please let us know if this wasn't your intention.\n\n\n**Q5:** The set $A$ and the matrix $M$ should be related explicitly.\n\n**A5:** Thank you–we now explictly highlight that $A_i$ is the support of $M_{i,\\cdot}$ in the updated manuscript.\n\n**Q6:** Abuse of notation with $\\theta$ in Defn. 1.\n\n**A6:** Addressed—we removed $\\theta$ in Defn. 1, keeping only $(g, p_{\\mathbf{z}}, M)$, and likewise for $\\theta’$. Thanks for raising this.\n\n**Q7:** More clarification on $\\mathbf{T}$ and $\\mathrm{T}$.\n\n**A7:** Thank you for your suggestion. We have added further highlights as follows:\n\n- $\\mathbf{T}$: a matrix-valued function in $D_{\\mathbf{c}} \\hat{g} = \\mathbf{T} D_\\mathbf{c} g$.\n\n- $\\mathrm{T}$: a matrix with the same support as $\\mathbf{T}$.\n\n**Q8:** Rename Prop. 1 to Cor. 1.\n\n**A8:** Thanks, we have renamed it accordingly.\n\n**Q9:** L266 left — Do you mean “and which cannot be expressed as”? Do you allow singular distributions?\n\n**A9:** Thanks for the suggestion. We meant “and which cannot be expressed as” and have revised the wording. The condition excludes singular distributions by requiring the subset $A_{\\mathbf{z}} \\subseteq \\mathcal{Z}$ to have non-zero probability measure.\n\n**Q10:** L360 right — How exactly are the $\\mathbf{c}^{(i)}$ created? What is a multi-hot vector? Is it sufficient to use one-hot vectors?\n\n**A10:** Thank you for the great suggestions. Accordingly, we have added a detailed explanation in the updated manuscript. Specifically, $\\mathbf{c}^{(i)}$ denotes the classes of sample $i$: a one-hot vector if single-labeled, or a multi-hot vector if multi-labeled (e.g., $[0,1,1]$ for two active classes).\n\n**Q11:** Learning by comparison could be connected closer to the experimental methodology and the literature\n\n**A11:** Thanks for the helpful advice. We have rewritten L82–99 left for closer grounding:\n\n> We address this question by grounding our approach in a fundamental cognitive mechanism: humans learn concepts by contrasting diverse classes of observations. Classic studies have shown that concept formation relies on detecting distinctions across examples—Bruner et al. (1956) emphasized learning through contrasts between exemplars and non-exemplars; Gibson (1963, 1969) proposed differentiation as a basis of perceptual learning in infants; and Gentner & Namy (1999) demonstrated that direct comparison enables children to abstract category-defining features. This principle is further supported by extensive literature across cognitive science, reinforcing that learning by comparison is the key underlying engine.\n\nExperimentally, FFHQ (Figs. 13–15) illustrates the need for local comparisons due to entangled concepts. We also added new experiments on two new datasets (see anonymous link), in light of your great insights.", "claims_and_evidence": "The key contributions of the paper are theoretical, see below.", "methods_and_evaluation_criteria": "Yes, the datasets make sense, for evaluation see below.", "theoretical_claims": "The results generally seem plausible. However, I have one question/complaint. The treatment of the $\\theta$ variable is a bit unclear. So the initial relation $(4)$ is clear. But then a relation for $g$ is used in (5) but $g$ is not injective and depends on $\\theta$. In my opinion this part needs to be clarified. I did not check the rest of the argument in detail but it seems to follow the standard arguments.", "experimental_designs_or_analyses": "My understanding is that the authors do the following:\nThey use (up to minor changes) a method from prior work and the comparison is not to another method but to a setting where their assumptions are violated. Is this correct? In this case (especially the second part) should be made a bit clearer in the text.", "relation_to_broader_scientific_literature": "The specific setup of the paper is novel to my knowledge and I find it quite convincing, essentially it seems to be, loosely spearking, a combination of the iVAE paper with structural sparsity. The theoretical results are also new, but the used techniques date back to the first works on identifiability of linear ICA and the iVAE work and the sparsity approaches have also similarly appeared before.", "essential_references_not_discussed": "One could consider citing more works from the causal representation learning literature that relies on similar techniques, but the key original works seem to be cited.", "other_strengths_and_weaknesses": "Strengths:\n\nThe paper is well motivated and overall well written. The examples are well-chosen, and it combines a setting of relevance (concept learning) with rigorous theoretical results that rely on reasonable assumptions. In particular, the results do not rely on parametric assumptions which makes them quite general. The discovered concepts in the experiments are quite convincing.\n\nWeakness:\n\nThe technical novelty is limited. The experimental methodology is not new. The results require quite some supervision (class labels) this is a weakness compared to some existing work.", "other_comments_or_suggestions": "l 117 left: There seems to be something missing.\n\nThe set $A$ and the matrix $M$ should be related explicitly\n\n$\\theta$ in Definition 1 is a abuse of notation because $\\theta$ was used before\n\nl.197 Definition of $T$ is unclear. Do you define $\\mathbf{T}$ by the equation that follows? Make this clearer\n\nProposition 1 might be renamed to Corollary 1\n\nl 266 left: This is hard to understand, do you mean 'and which cannot be expressed as'? Do you allow singular distributions on $z_A$?\n\nl 360 right: How exactly are the $c^{(i)}$ created? What is a multi-hot vector? Is is sufficient to use one-hot vectors?\n\nLearning by comparison could be connected closer to the experimental methodology and the literature", "questions_for_authors": "See theoretical claims and experimental design. Depending on the answer to the former my assessment would change.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "46DlX8BcuU", "reviewer": "Reviewer_CTeq", "summary": "The paper proposes that when there is an unknown concept-to-class matrix, the concepts can be identified under certain assumptions on the structure of the matrix. Namely, it formalizes in which sense each concept needs to have characteristic classes it belongs to, or vice versa, how every class needs to have characteristic attributes that distinguish it from the other classes. The paper is primarily theoretical, with some synthetical experiments.\n\n## Update after the rebuttal\n\nI am raising my score to 3 because $f$ is non-linear (though still bijective), $z$ is continuous, and because of the inclusion of DCI/MIG, FID/PPL, and CUB/AWA experiments.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are genuinely grateful for your insightful comments. In light of these, we have introduced **new experiments** (https://anonymous.4open.science/r/0-518C/rebuttal_new_results.pdf) and new discussions. Please find our detailed response below:\n\n**Q1:** Strong assumptions / limited generative process space.\n\n**A1:** Thank you so much for the comment. It seems we may not be fully aligned on the core setting and assumptions. We sincerely appreciate the opportunity to clarify the following:\n\n- **Q1.1:** There is a very strong assumption that the generative process $f$ is bijective.\n\n  **A1.1:** As defined in L129-136 left, $f$ is an unknown diffeomorphism onto its image—a very *standard* assumption in identifiability literature (e.g., most works mentioned in the survey [Hyvärinen et al. 2024]). It aligns with the manifold hypothesis, which posits that high-dimensional data (e.g., pixels) lie on a lower-dimensional latent manifold (e.g., concepts). In overcomplete settings (latent dim > observed dim), much stronger assumptions (e.g., linearity) are typically required.\n\n  - *Hyvärinen et al., Identifiability of latent-variable and structural-equation models: from linear to nonlinear, 2024*\n\n- **Q1.2:** $f$ may be linear (if it denotes $M$); unclear spaces that $f$ maps between.\n\n  **A1.2:** Thanks for raising this. To clarify, **$f$ and $M$ are unrelated**.\n\n   - As defined in L113–121 left, $f$ maps latent concepts $\\mathbf{z}$ to observations $\\mathbf{x}$ (Eq. 1: $\\mathbf{x} = f(\\mathbf{z})$) and is a general diffeomorphism, **without any linearity constraint**.\n\n   - As defined in L159 left–133 right, $M$ is a binary structural mask between classes $\\mathbf{c}$ and concepts $\\mathbf{z}$, independent of $f$.\n\n- **Q1.3:** All concepts are binary.\n\n  **A1.3:** As defined in L116 left, all concepts are **continuous variables** ($\\mathbf{z} \\subseteq \\mathbb{R}^{n}$), with no restriction to binary values.\n\n- **Q1.4:** Unclear how strong the structural diversity is.\n\n  **A1.4:** Thank you for the great question. In a general nonparametric setting (*no functional/distributional constraints*), identifiability is known to be ill-posed without additional assumptions. Structural diversity characterizes when *every concept* is identifiable in such settings. It is expected that ambiguous cases must be excluded, as in most prior identifiability theories.\n\n  A key contribution of our work is going *beyond* this assumption. The local comparison framework (Thm. 1, Prop. 1) provides identifiability guarantees even when global condition is *violated*—allowing recovery of many concepts as groups (e.g., in FFHQ). In contrast, most prior work cannot provide any guarantees under any degree of violation.\n\n  In light of your question, we have also added new experiments under partial violations (Figs. 1–2 in the link), confirming empirical robustness.\n\n**Q2:** Additional metrics for synthetic and real-world datasets.\n\n**A2:** Great suggestion. In light of it, we have added evaluations on DCI and Mutual Information Gap (MIG) for synthetic datasets (Figs. 1–2 in the link), and Fréchet Inception Distance (FID) and Perceptual Path Length (PPL) for real-world images (Table 1 in the link). Our method consistently outperforms baselines.\n\n**Q3:** More baselines.\n\n**A3:** Thanks for your advice. We have added two more baselines (Table 1 and Figs. 7-8 in the link), further supporting the effectiveness.\n\n**Q4:** Code is not provided.\n\n**A4:** Thanks. Due to rebuttal policy, we can only include figures in the link. Based on the flow-based method (L1372), we added an ℓ₁ norm (L369 right) since ℓ₀ norm is known to be indifferentiable. \n\n**Q5:** Test on CUB or AwA2.\n\n**A5:** Thank you for the advice. We have conducted new experiments on both datasets (Figs. 5–8, Table 1 in the link), showing that identifiable concepts remain recoverable. With these additions, we now have *seven* different real-world datasets to support the practical applicability.\n\n**Q6:** Comparison with two related papers.\n\n**A6:** Thanks for the suggestion. Please find the detailed comparison below:\n\n- **Brady et al., 2023:**\n\n   (This work has been discussed in the introduction (L62-76 left).)\n\n  - ***Theory:*** Assumes each observed variable (pixel) maps to only one latent slot.\n\t- In contrast, we allow arbitrary mixing.\n\n\n  - ***Practice:*** Targets recovering concrete objects as pixel groups, assuming no occlusion or overlap among them.\n\t- In contrast, we focus on general concepts (concrete or abstract), with no constraints on observed object composition.\n\n\n- **Leemann et al., 2023:**\n\n  - ***Theory:*** Assumes linear generative process or orthogonal Jacobians.\n\t- In contrast, we allow general nonlinear generative process, where the identifiability is much harder.\n\n  - ***Practice:*** Targets post-hoc explanation assuming access to linear combinations of latent factors.\n\t- In contrast, we focus on general concept learning from observation in a nonparametric setting.", "claims_and_evidence": "Under the given assumptions, concepts are identifiable\n\n* This seems to be theoretically backed, but I did not check the proofs\n\nThe theoretical identifiability holds in practice\n\n* I have doubts here since the experimental setup is very small-scoped, see below.", "methods_and_evaluation_criteria": "The paper uses only one metric to benchmark whether the captured concepts are ground-truth concepts, the ECC. While widely used, there are other popular metrics like the Disentanglement-Completeness-Informativeness score (DCI) or the Separated Attribute Predictability (SAP). It would be more convincing to test on multiple criteria, since they are known to all be noisy and imperfect.\n\nThe paper does not compare to any baselines, except the base model. \n\nAll quantiative experiments are only on synthetical datasets. Real datasets only have qualitative plots, without baselines.", "theoretical_claims": "I do not consider myself an expert in the very nuanced proofs of identifiability / nonlinear ICA, so I abstain from guaranteeing correctness of the proofs.", "experimental_designs_or_analyses": "Synthetical experiments lack metrics, models, and baseline approaches.\n\nThe paper lacks quantitative real-life dataset experiments.", "relation_to_broader_scientific_literature": "The authors make connections between concept learning, nonlinear ICA, and structure learning.", "essential_references_not_discussed": "The paper discusses most of the relevant works in nonlinear ICA.", "other_strengths_and_weaknesses": "Strengths\n\n* Bridging concept learning literature with nonlinear ICA / identifiability literature is overdue\n* There are no distributional assumptions on the latent concepts, in particular no non-Gaussianity (though I am not sure, because the paper seems to exclude fully Gaussian data)\n\n\nWeaknesses\n\n* There are very strong assumptions / the space of generative processes this can be used for is very reduced\n    * There is a very strong assumption that the generative process $f$ is bijective. \n    * It may be that $f$ is linear (if it denotes M), though I am not sure, because the spaces that f works in are not defined (class to concept, class to image, image to concept, ...) and no comment on linearity / non-linearity is made\n    * All concepts are binary\n    * There is the structural diversity assumption, and I cannot judge how strong it is. It assumes that for each concept, there exists at least one class that has it (so far so good) and there exists a set of classes that does not have it, and this concept is the only concept that is not found in any of these classes. I understand that this prevents block matrices (which are by default non-identifiable) but cannot judge how many other realistic matrices it assumes away.\n* The experiments are on only one model, with one evaluation metric, and only on synthetical datasets (all evaluations on the real-life datasets are purely qualitative and without a baseline). It can thus be very overfitted to that setup, since, as put in the appendix “We generate the data following the process outlined in our theorems.”. Showing that the approach works on a synthetical setup which exactly follows its the paper’s assumptions is not a very general result, especially when the assumptions are as strict as in this paper.\n* The experiments do not compare to any baseline approaches, except the base model, see also below.\n* Code is not provided.\n\nScore justification:\nThe paper seems to make progress on the exact matrix structure that is needed to allow identifiability, but I cannot quite judge how much it differs from current approaches since I know enough about ICA to not consider myself an expert. Hence, in terms of novelty and theoretical contribution, the score can be anywhere between 2-4. I am currently more on the sceptic end of this range for two reasons: 1) I believe the assumptions are rather strong / the applicability is limited to binary matrices (since f needs to be bijective and M binary). This is understandable for structure learning, but limiting for concept learning. I hope that the other reviewers can comment on the strength of the assumptions and novelty over previous approaches. 2) the experiments are very limited, to one evaluation metric and one model and only on synthetical datasets that exactly follow the assumptions the approach exploits. This limits the generality of the results. More results, as outlined above and below, can convince me otherwise here.\n\nPost rebuttal score: I have raise to score 3 due to the additional experiments and due to clarifications on the assumptions, which are still restrictive, but not as restrictive as I initially believed.", "other_comments_or_suggestions": "I did not find typos or formatting issues.", "questions_for_authors": "If I understand correctly, your approach should work quite well on attribute-based classification, such as on CUB or AwA2? There you have a (binarizable) attribute-per-class matrix. Can you comment on whether it applies, and test your algorithm on it? This would help judge if it works in higher dimensions.\n\nI am concerned about how far the results span in terms of width of generative processes covered. Could you compare to the following two neighboring approaches (https://openreview.net/pdf?id=mGUJMqjDwE, https://proceedings.mlr.press/v216/leemann23a/leemann23a.pdf) in structural identifiability in terms of...\n\n* Theoretical scope, i.e., the assumptions placed on the generative process to guarantee identifiability\n* Practical applicability, i.e., which datasets it can currently discover concepts on", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers unanimously acknowledge the quality of the contribution and recommend acceptance. I concur that this is a solid contribution to the program and recommend acceptance.
Decision: Accept (poster)