Title: Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability
Venue: NeurIPS 2025
Accepted: yes
Keywords: identifiability

Jacobian

component analysis

nonlinear mixture
Abstract: Latent component identification from unknown *nonlinear* mixtures is a foundational challenge in machine learning, with applications in tasks such as self-supervised learning and causal representation learning. Prior work in *nonlinear independent component analysis* (nICA) has shown that auxiliary signals---such as weak supervision---can support *identifiability* of conditionally independent latent components. More recent approaches explore structural assumptions, like sparsity in the Jacobian of the mixing function, to relax such requirements. In this work, we introduce *Diverse Influence Component Analysis* (DICA), a framework that exploits the convex geometry of the mixing function’s Jacobian. We propose a *Jacobian Volume Maximization* (J-VolMax) criterion, which enables latent component identification by encouraging diversity in their influence on the observed variables. Under suitable conditions, this approach achieves identifiability without relying on auxiliary information, latent component independence, or Jacobian sparsity assumptions. These results extend the scope of identifiability analysis and offer a complementary perspective to existing methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article addresses the problem of identifying latent features from their nonlinear mixtures. The approach proposed is based on exploiting a data model where the the rows of the Jacobian matrix of the nonlinear mapping are assumed to lie in a scaled $\\ell_1$ norm ball. The article proposes the use of two neural networks as parametric nonlinearities, whose parameters are obtained through a volume maximization on the gradients of the nonlinear mixing to exploit their pressumed scattering within the scaled $\\ell_1$ norm ball. The article offers numerical experiments to illustrate the performance of the proposed approach which look convincing.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n - The article addresses the hard problem of obtaining latent factors from their nonlinear mixtures and offers a solution with proven identifiability under some assumptions about the data model\n\n## Weaknesses\n\nA. One significant concern is the lack of proper attribution to prior work from which key definitions and methodology appear to be adopted. Although the referenced article is included in the general bibliography, the paper fails to cite it at the precise points where the reused ideas are introduced. As a result, the presentation gives a misleading impression of originality. This omission is not merely stylistic—it impairs the reader's ability to understand the origin and novelty of the contributions and violates standard scholarly norms of credit and transparency. Furthermore, this makes it difficult to assess whether the paper's contributions are genuinely novel or derivative.\n\n\nThe main reference under consideration is [48] G. Tatli et.al. “Polytopic Matrix Factorization: Determinant Maximization Based Criterion and Identifiability”. In: IEEE Transactions on Signal Processing 69 (2021), pp. 5431–5447.\n\nAlthough the authors very briefly list this reference at the end of Section 4, the current work appears to closely extend the polytopic matrix factorization (PMF) introduced in [48] for linear mixtures to nonlinear mixtures. In this endaveour, the author(s) appeared to use definitions, criterion and the proof steps in [48] without any precise referals.\n\nTo clarify this point, I checked [48] and summarize main relevant points here: [48] proposes  PMF for the extraction of latent factors from their mixtures, which doesn't assume independence of latent factors but assumes a generative model (in Section II.A of [48]) for the input matrix $\\mathbf{Y}$:\n\n$$ \\mathbf{Y}=\\mathbf{H}_g\\mathbf{S}_g $$\n\nwhere $\\mathbf{H}_g$ is unknown full-column-rank factor matrix and $\\mathbf{S}_g$ is the unknown matrix whose columns are in a polytope $P$. [48] proposes the use of the following volume maximization  problem (in Section II.B)  to identify these factors  from the input matrix $\\mathbf{Y}$:\n\n*PMF Problem*\n\\begin{array}{ll}\n\\displaystyle \\max_{\\mathbf{H} \\in \\mathbb{R}^{M \\times r},\\, \\mathbf{S} \\in \\mathbb{R}^{r \\times N}} & \\det(\\mathbf{S}\\mathbf{S}^T)  \\tag{PMF}\\\\\n\\text{subject to} & \\mathbf{Y} = \\mathbf{H} \\mathbf{S}, \\\\\n& \\mathbf{S}_{:,j} \\in \\mathcal{P}, \\quad j = 1, \\dots, N.\n\\end{array}\n\n\nFor the identifiability condition it introduces the *sufficient scattering condition* (in Section II.c) for the columns of $\\mathbf{S}$ as\n\n*{Definition II.2: Sufficiently Scattered Factor:}* \n$\\mathbf{S} \\in \\mathbb{R}^{r \\times N}$ is called a sufficiently scattered factor corresponding to $\\mathcal{P}$ if\n\n  - (PMF.SS.i) $\\mathcal{P} \\supseteq \\mathrm{conv}(\\mathbf{S}) \\supset \\mathcal{E}_{\\mathcal{P}},$ and\n  - (PMF.SS.ii) $\\mathrm{conv}(\\mathbf{S})^{*, \\mathbf{g}_{\\mathcal{P}}} \\cap \\mathrm{bd}(\\mathcal{E}_{\\mathcal{P}}^{*, \\mathbf{g}_{\\mathcal{P}}}) = \\mathrm{ext}(\\mathcal{P}^{*, \\mathbf{g}_{\\mathcal{P}}}),$\n\nwhere $\\mathcal{E}_{\\mathcal{P}}$ is the MVIE of $\\mathcal{P}$, centered at $\\mathbf{g}_{\\mathcal{P}}$.\n\nSection III.B of [48] focuses on the special case $\\mathcal{P}=\\mathcal{B}_1$ ($\\ell_1$ norm ball), i.e. the sparse PMF case. Then Theorem 3 of [48] proves that the optimal solution of the (PMF) optimization problem above under sufficiently scattered condition (Definition II.2) recovers the factors of the PMF generative model upto sign and permutation ambiguity.\n\nFrom this perspective, the contribution in the current article is an extension of the Sparse-PMF of [48]:\n\n- The Sufficiently Diverse Influnce definition (Assumption 3.1) is exactly the sufficiently scattered condition (Definition II.1 above) in [48] applied to the rows of the Jacobian for the special case of polytope $\\mathcal{P}=C \\mathcal{B}_1$. Figure 1 is the two dimensional version of Fig. 5 in [48].\n\n- Jacobian Volume Maximization problem in Page 5 is the extension of the Sparse PMF problem above, imposing sparsity to the Jacobian matrix, replacing the linear reconstruction constraint in Sparse-PMF with the nonlinear construction constraint.\n\n- The proof of Theorem 3.2 (Identifiability of J-VolMax) in Appendix B, shares exact same steps as the proof of Theorem 3 for Sparse PMF in [48]:\n    * the treatment between equations (22)-(27) in the proof of Theorem 3.2 are the same as the treatment around (15)-(17) in the proof of Theorem 3 in [48],\n    *  Lemma B.1 in Appendix (from a nonexisting reference [55]) is the same as the second stage of the proof of Theorem 3 of [48],\n    * the treatment between equations (28)-(33) in the proof of Theorem 3.2 are the same as the treatment in the third stage of the proof of Theorem 3 (around equation (24)) in [48].\n\nIn summary, the current article appears to be the extension of the sparse-PMF for linear mixing to nonlinear mixings.\nDespite this tight  extension relationship, and  the reuse/generalization of results in  [48],  \n\n- there is no mention of the reference [48] in Section 2 (Background). Section 2 should also discuss the reference\nK. Huang, N. D. Sidiropoulos, and A. Swami, “Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition,” IEEE Transactions on Signal Processing, vol. 62, no. 1, pp. 211–224, October 2013\nwhich appears to be the first article where the sufficiently scattered condition for latent extraction, the root of the sufficient diverse influence condition, was introduced,\n- there is no discussion about how the current work extends Sparse-PMF for linear mixings to  nonlinear mixings, \n- there are no precise references at the appropriate points to the original determinant maximization based sparse PMF problem and the corresponding sufficiently scattered identifiability condition definitions, Theorem 3, and the proofs of Theorem 3 in [48]. \n\n\n\n\nB.  The assumptions about the data model (about the Jacobian), their applicability to practical problems could be better motivated and explained.\n\n\nB. Section 5.2 \"Inferring Transcription Factor\" could be better explained in terms of the problem statement and the performance measure.\n\nQUESTIONS:\n- what is the motivation for the assumption about the Jacobian (inclusion in the l-1 norm ball) besides identifiability? Do you see that it is a practically valid assumption? can we also consider other sets alternative to l-1 norm ball (such as other polytopes in [48])?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of nonlinear mixture model identification (NMMI), i.e., how to provably recover d-dim. latent sources $s$ from m-dim. observed nonlinear mixtures $x=f(s)$ thereof. (If the sources are jointly independent, this problem is known as nonlinear ICA.) NMMI is hard in general, but has been addressed in prior work under additional assumptions such as auxiliary information or sparsity. \n\nThe paper instead proposes a different approach based on the assumption of sufficiently diverse influences (SDI). SDI is a non-trivial condition involving the convex geometry of the gradients $\\nabla f_1, ..., \\nabla f_m$. Intuitively, it states that globally, i.e., for all $s$: \n- (i) for each source $s_j$ there exists an $x_i$ that is positively influenced by $s_j$ and an $x_k$ that is negatively influenced by $s_j$\n- (ii) for each source $s_j$ there exists an $x_l$ such that $\\nabla f_l$ is dominated (in magnitude) by its $j$th component $\\partial x_l / \\partial s_j$\n\nAlgorithmically, the SDI condition suggests an approach based on maximising the volume of the Jacobian $J_f$ subject to invertibility of $f$ (enforced via an autoencoder) and a bounded norm constraint on the rows of the Jacobian, i.e., the gradients $\\nabla f_i$. \n\nTheoretically, it is shown that SDI enables identification of the latent sources up to permutation and element-wise reparametrization, without requiring statistical independence or sparsity. For a finite sample setting, it is shown that, under additional conditions, the identification error can be bounded. \n\nEmpirically, the proposed DICA approach is compared against a vanilla autoencoder and a baseline with sparsity regularization on the Jacobian. On synthetic and semi-synthetic gene expression data, DICA is shown to outperform these baselines in terms of latent variable recovery.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- the paper is well written and (apart from the mathematical complexity) easy to read\n- the proposed approach is original and novel\n- the paper provides contributions along several axes: a new principle, identifiability results, an estimation methods, and experiments\n- the finite sample analysis (which is not common in the identifiability literature) is a nice addition\n- the paper does a good job at covering related work and discussing its relations to the proposed approach\n- the paper does not just consider sythetic data, but also provides results on a semi-synthetic dataset from a biological simulator\n\n### Weaknesses\n- the SDI assumption is quite difficult to understand: while the \"Physical Meaning of SDI\" paragraph is helpful to gain some intuition and interpret the condition, it is unclear how the authors came up with or arrived at the precise mathematical formulation in Assumption 3.1; it does not help that definitions and explanations of key concepts (convex hull, polar set, MVIE) are deferred to the appendix (these should be moved to the main paper);\n- the precise relation between the SDI assumption and the Jacobian volume maximisation (J-VolMax) approach, as well as that between SDI and orthogonality maximization of the Jacobian columns (IMA) remain a little unclear; \n- it seems that the proposed approach requires a very large number of observations relative to the number of latent sources (in section 3, it is stated that $m\\geq 2d$ is required, but in the experiments it is much larger, $m\\gg d$) \n- the empirical evaluation is somewhat limited in that it only considers relatively naive baselines; comparison with other nonlinear ICA methods (e.g., IMA or based on sparsity) and reducing the number of observations $m$ (resp. increasing the number of sources) could be useful additions to get a better understanding of empirical performance\n- quantifiers (for all, there exists) are used too sparingly and inconsistently; including these for all formal statements would improve clarity\n\nQUESTIONS:\n### Questions\n- How did you arrive at or come up with the SDI assumption? Can it be derived or motivated \"from first principles\"? \n- Thm. 3.2 establishes that J-VolMax under Asm. 3.1 yields identifiability, but is there a more direct connection between the SDI condition and volume maximisation?\n- For fixed magnitude vectors, the volume of a parallelotope is maximal iff the vectors are orthogonal, as enforced in IMA [11]. This seems related to the J-VolMax objective, but there the norm constraint in Eq. (7b) appears to be on the gradients (i.e., rows of the Jacobian), rather than on the partial derivatives (i.e., columns of the Jacobian) as in IMA. Could you please comment further on the relationship between the two approaches? \n- is the SDI condition invariant to reparametrisation of the sources? i.e., if SDI holds for $f$ and $f$ is replaced by $f\\circ \\rho$ for an element-wise function $\\rho$, does SDI still hold? \n- what is relation between the notion of \"one source dominating the derivatives\" entailed by SDI to that of anchor features/hard sparsity [e.g., 41]? can one think of the former as a relaxation of the latter? \n- Since [4] appears to be a key reference and is used as a baseline, are the authors aware of [R1] and [R2] which relax the sparsity assumption on the Jacobian of [4]? You may want to consider these works and discuss their relation to SDI as well. \n\n### Comments and Suggestions\n- There appear to be some inconsistencies regarding $\\mathcal{B}_\\infty^{1/w(s)}$ vs. $\\mathcal{B}_\\infty^{w(s)}$ in Asm. 3.1., the caption of Fig. 1, and lines 125-126. Is it possible that there is a typo here? \n- Eq. (7a) seems to miss a closing bracket.\n- $\\hat{\\Pi}$ is undefined in eq. (9)\n- Thm 3.3 mentions \"three regularity conditions\" but four are given.\n- The claim that IMA [11] does not have identifiability underpinning appears a little too strong: while full identifiability for IMA has not been established (to the best of my knowledge), e.g., [5] proves local identifiability.\n- Since optimization of the Jacobian is highlighted as a computational challenge, the authors may be interested in [R3].\n- The authors may be interested in [R4] which remarks on some potential issues with some proofs of reference [50] in their manuscript. \n\n### References\n[R1] Lachapelle, Sébastien, et al. \"Additive decoders for latent variables identification and cartesian-product extrapolation.\" Advances in Neural Information Processing Systems 36 (2023): 25112-25150.\n\n[R2] Brady, Jack, et al. \"Interaction Asymmetry: A General Principle for Learning Composable Abstractions.\" International Conference on Learning Representations (2025).\n\n[R3] Gresele, Luigi, et al. \"Relative gradient optimization of the jacobian term in unsupervised deep learning.\" Advances in neural information processing systems 33 (2020): 16567-16578.\n\n[R4] Buchholz, Simon. \"Some remarks on identifiability of independent component analysis in restricted function classes.\" Transactions on Machine Learning Research (2023).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose an alternative way to tackle the nonlinear mixture identifiability problem. The setting is as follows: unknown latent components are transformed by an unknown nonlinear function into observation and the challenge is to recover the unknown latent components from the observations. In general, the problem might be ill-posed, therefore most work are restricted to scenarios where identifiability can be shown. One way to do so is to make stronger assumptions on the nonlinear transformation, e.g. by imposing sparsity on the Jacobian matrix of the transformation. Instead, the authors build a geometric argument as to why it is sufficient to penalize the Jacobian log determinant. This allows them to treat functions with dense jacobians as well.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- the mathematical foundation seems solid and the solution proposed is elegant\n\nWeaknesses:\n- even though it is left for future work, the proposed approach is not viable in practice because training requires computing the Jacobian determinant, which is cubic if the function is unconstrained (see questions below)\n\nQUESTIONS:\n- the proposed approach employs an unconstrained neural network and forces its inverse to exist with a loss that acts on training points. While this approach has been previously used in other contexts as well, it only requires the learnt map to be invertible locally. Especially in high-dimensional spaces, this doesn't enforce invertibility in any systematic way. This might not be visible in the experiments since a very small network is used (1 hidden layer with 64 units) in a very low-dimensional space\n- Maybe I am missing something, but have you considered defining f as invertible by construction like normalizing flows? this would also make the proposed method computationally much more efficient when it comes to computing the Jacobian determinant (from cubic to quadratic/linear)\n- If this is a viable approach, one could easily [1] even constrain the function to lie on a L1 norm ball by construction (in place of the regularization term of your approach)\n- I have a general doubt concerning the experiment section. In the synthetic experiment the nonlinearity is chosen at random so I expect its jacobian not to be sparse (which is also a benefit of the proposed approach). On the other hand, for the single-cell experiment, do we know something about the degree of sparsity of the Jacobian of the \"true\" f to be learnt?\n- If the underlying Jacobian is however sparse, should the method Sparse provide better results than the one obtained? Did you also try with higher sparsity? (I saw in the appendix you selected the best parameter via cross-validation)\n- In case both experiment have a dense/non-sparse Jacobian, how does your method compare to current approaches that assume some form of sparsity?\n\n[1] Negri et al., Injective flows for star-like manifolds, ICLR 2025\n\nLIMITATIONS:\nLimitations are clearly stated and discussed in the conclusions. The main limitation is that in practice it is really expensive to compute the Jacobian log-determinant at every iteration, which makes the method not viable in practice.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-14"}