Identifiability Challenges in Sparse Linear Ordinary Differential Equations
Authors: Cecilia Casolo
Sören Becker
Niki Kilbertus
Venue: ICLR 2026 Poster (2026)
Keywords: dynamical systems
identifiability
sparsity
Primary area: learning on time series and dynamical systems
TL;DR: We investigate the identifiability challenges of  linear ordinary differential equations for sparse system matrix starting from a single trajectory.

Abstract: Dynamical systems modeling is a core pillar of scientific inquiry across natural and life sciences. Increasingly, dynamical system models are learned from data, rendering identifiability a paramount concept. For systems that are not identifiable from data, no guarantees can be given about their behavior under new conditions and inputs, or about possible control mechanisms to steer the system. It is known in the community that "linear ordinary differential equations (ODE) are almost surely identifiable from a single trajectory." However, this only holds for dense matrices. The sparse regime remains underexplored, despite its practical relevance with sparsity arising naturally in many biological, social, and physical systems.
 In this work, we address this gap by characterizing the identifiability of sparse linear ODEs. Contrary to the dense case, we show that sparse systems are unidentifiable with a positive probability in practically relevant sparsity regimes and provide lower bounds for this probability. We further study empirically how this theoretical unidentifiability manifests in state-of-the-art methods to estimate linear ODEs from data. Our results corroborate that sparse systems are also practically unidentifiable. Theoretical limitations are not resolved through inductive biases or optimization dynamics. Our findings call for rethinking what can be expected from data-driven dynamical system modeling and allows for quantitative assessments of how much to trust a learned linear ODE.

Published as a conference paper at ICLR 2026
IDENTIFLABILITY CHALLENGES IN SPARSE LINEAR ORDINARY DIFFERENTIAL EQUATIONS
Cecilia Casolo, Sören Becker, Niki Kilbertus Technical University of Munich
Helmholtz Munich
Munich Center for Machine Learning (MCML) first. www.llloootlnee
ABSTRACT
Dynamical systems modeling is a core pillar of scientific inquiry across natural and life sciences. Increasingly, dynamical system models are learned from data, rendering identifiability a paramount concept. For systems that are not identifiable from data, guarantees can be given about their behavior under new conditions and inputs, or about possible control mechanisms to steer the system. is known in the community that "linear ordinary differential equations (ODE) are almost surely identifiable from a single trajectory." However, this only holds for dense matrices. The sparse regime remains underexplored, despite its practical relevance with sparsity arising naturally in many biological, social, and physical systems. In this work, we address this gap by characterizing the identifiabiliyy of sparse linear ODEs. Contrary to the dense case, we show that sparse systems are unidentifiable with positive probability lower bounds for this probability. We further study empirically how this theoretical unidentifiability manifests in state-of-the-arr methods to estimate linear ODEs from data. Our results corroborate that sparse systems also practically unidentifiable. Theoretical limitations are not resolved through inductive biases or optimization dynamics. Our findings call for rethinking what can be expected from data-drived dynamical system modeling and allows for quantitative assessments of how much trust learned linear ODE.
practically relevant sparsity regimes and provide
INTRODUCTION AND RELATED WORK
The field of dynamical systems has emerged early corner stone of scientific modeling, primarily due to the high demand for modeling temporally evolving systems the natural and life sciences. For the most part, dynamical systems have been modeled "manually," i.e., by differential equations or simulators inspired by, and validated in real-world systems. With the advent of machine learning and large data collection efforts, dynamical systems increasingly learned from data. This new perspective also brought about a shift from an original focus on the forward poblem-solvingg given differential equations from different initial conditions for predictions- the inverse problem-finding the governing differential equation from observed trajectories. Inferring the underlying dynamical laws governing system solely from observational data a longstanding aspiration across numerous scientific disciplines. A fundamental prerequisite to realize this goal is the identifiability of the governing laws, i.e., that the true dynamics can, principle, be uniquely determined from the available observations. If multiple different laws could result in the exact same observations, we are stuck. This the problem identifiability, as illustrated Fig. T. could the observed data have arisen from only single unique dynamic? While solving the inverse problem poses a host of practical challenges, no guarantees can ever be given for unidentifiable systems.
prescribed
The identifiability of different types of dynamical systems from different types of observed data has been studied in a variety of domains like natural science (Donà et al., 2022 Muñoz-Tamayo et al. 2018), control theory (Ding & Toulis 22200) Gargash & Mital] 1980), experimental design (Raue et al.
2010) and many more. MMao et al. (2011) provide well-structured overview of how identifiability analysis of different types of (non-linear) dynamics is approached in practice while Scholl et al. 2002

---

Published as a conference paper at ICLR 2026
True dense system
F0.35
0.54
1.05
State variables
0-
statte vatiables
A=
0.10
0.34 00.29
0.42
0.38 0.30
0.1
F0:35
0.54
-1.06
F0.35
0.54
-1.05
True sparse system
0.5
A
0:10
0.31
-0.29
A=
0.10
0.34
-0.29
0
-10.74
0.7
0:42
0.38
0.50
-3
0.42
0.58
0.30
B
0
-1.85
0.9
0.00
0,215
0.55
0.75
1.00
0.00
0.2.
0.50
0.75
1.00
-10.89
Lime a
Lime
1500
state variabls
states varfables
statos
Count
1000
-1
-84
11
3.2
34.2
-25.3
500
17.3
+5.6
-6.5
-2.1
50.4
-19.8
-32.5
227.1
10.5
-00.0
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
Lime
time
Normalized Hamming distance
Figure 1: Identifiability differs between dense and sparse systems. The trajectories in a and correspond solutions for two different initial conditions of the dense system in e; similarly, trajectories in d and e correspond to solutions of the sparse system in c. All estimated trajectories (colored dashed curves) fit the respective observations (thick gray curves) well, yet only the estimate of the dense system is correct. The histogram in f shows that the normalized Hamming distance between the estimated and true system matrix systematically increases as sparsity p increases.
2023) have characterized necessary and sufficient conditions about what needs to be observed for identifiability in different functional classes of dynamics. The bio-mathematics community has been among the first to develop rigorous approaches to study identifiability (Bellman & Aström 1970), referring it "structural identifiability." Similarly, researchers in epidemiology have a strong track record in analyzing identifiability for domain-specific compartment models (Saccomani] [2011] Miao al., 2011 I Xia & Moog, 2003) Tuncer et al. 20166). Cunniffe et al. (2024) recently analyzed the connection between identifiability and observability, as a critical aspect of epidemiological models, where we typically cannot observe the entire state. Stochastic differential equations often enjoy stronger theoretical identifiability guarantees, typically because the stochastic component is assumed to have full support, allowing to "probe the entire space" (Bellot et al. 202T Wang et al.) 2023).
Irrespective of theoretical considerations, new practical methods for learning dynamical systems from data are proposed continuously. These range from traditional parameter estimation techniques for ODEs (Lavielle et al.] 2011 Commenges et al. 2011 HHung et al.] 2006 Li et al.] 2005 Brunton ctt al. 2016), to neural-networke -wt-ennewwwweeeeee based parameter estimation (Rubanova et al. 2019 Qin et al. 2019)
as well as deep-learning based approaches that learn the dynamics as neural net (Chen al. 2018
or predict symbolic expressions (Becker et al. 2023 d' Ascoli et al. 2024). While these methods are reported to achieve strong reconstruction performance, i.e., they find dynamics that, when integrated from the same initial condition, recover the observed trajectories well, it is rarely reported to which extent they actually identify the originally underlying dynamical law. Due to unidentifiability, these methods could learn dynamical systems that do not generalize beyond the observed time spans or to new initial conditions, let alone warrant claims of scientific insights.
In this work, we will focus on linear, homogeneous, autonomous ordinary differential equations (ODE), a(()= = Az(t), heavily relied upon in many domains while also amenable to rigorous theoretical analysis. Pioneering works StSthhoe et al. (2014) and, more recently. Qiu et al. (2022) established as a key result that "almost all such linear ODEs are identifiable from a single solution trajectory." Here, it is assumed that we have observed the entire trajectory without any observation noise in continuous time. This is often paraphrased as "linear ODEs are identifiable" and summoned by practitioners as justification to assume that whenever a linear ODE is estimated well from data, one has actually found the unique underlying governing law of the system with probability 1. The core contributions in this work build on the observation that this, rightfully celebrated, result only holds for dense matrices, i.e., they assume a measure over systems R that absolutely continuous with respect to the n-dimensional Lebesgue measure A
This is not satisfied by sparse systems. Sparsity in dynamical systems means that not all variables depend (or interact with) all other variables (Aliee al., 2021 2022). Hence, sparsity allows
2

---

Published as a conference paper at ICLR 2026
us understand and interpret complex networks pervasive in nature, society, and technology. In fact, dynamical models in biology (Lu et al. 2011) or social networks (Ravazzi et al. 2017) typically exhibit high degrees of sparsity. For example, Liben-Nowell (2005) asserts that "on average, person has on the order of at most a thousand friends." Similarly, interactions in gene regulatory networks known be sparse, with only tiny fraction of possible pair-wise interactions being nonzero (Carey et al. 2018). Hence, in many settings researchers only attempt to learn dynamical models from data when they assume interactions to be sparse, because meaningful interpretation only possible in sparse models in the first place (Xu et al., 2023). The resulting models are also believed to have lower risk of overfitting, particularly when data limited (Bartoldson al., (020).
Contributions. This work puts an asterisk on the result that "almost all linear ODEs are identifiable" proving positive lower bounds on the probability of sparse systems being unidentifiable. We also quantify theoretically to what extent "near unidentifiability" poses a challenge for practical identi- fication identifiable cases. Finally, we bridge the gap from theory to practice and demonstrate that (near) unidentifiability is not circumvented in practical methods by potential inductive biases or optimization dynamics. Our work clearly characterizes the regimes (in terms of dimensionality and sparsity) in which unidentifiability is an issue, and what observed time horizons required to escape near umidentifiability.
2 BACKGROUND AND PROBLEM SETTING
We focus autonomous, homogeneous, linear, noise-free ordinary differential equations (ODE), i.e., initial value problems (IVP) of the form
dxx(t) = Azo(1), a=(0) = 10 +(1) = dt
where A E R2x" is also called the system or simply referred to the "ODE," x [0, R" denotes the trajectory, 2n E IR" is the initial condition, and denotes time. In autonomous systems, can assume without loss of generality that the initial time is / 0 autonomy implies that the system A is not time-dependent. Homogeneity ensures that the dynamics are not externally forced or perturbed. The system matrix A is also referred to as the adjacency matrix following a graph theoretic view on the dynamics. The key novelty in our work is in considering sparse systems, i.c., sparse matrices... A.
Discussion of assumptions and limitations. Linearity, autonomy, homogeneity and the absence of noise heavily restrict the types of systems studied in our work. The linearity assumption is in line with existing work (Stanhope et al.) 2014: Qiu et al., [022)] Fedoryuk 2012) Ovchinnikov et al. 2022) and required for the feasibility of theoretical analysis. Autonomy and homogeneity are adopted to represent our focus on passive observations of naturally evolving systems without external forcing, which corresponds closely to the idea of an observational distribution in the causal inference literature and differentiates our work from approaches towards aantrollabiltyy/obeevabbbil control and system identification literature. Some of the existing work on identifying linear ODEs from a single trajectory have been extended to post-nonlinear models (breaking linearity) (Miao et al.) 201 affine systems (extending homogeneity) (Duan et al. 2020), systems with hidden confounder Wang al. (2024a), and include discrete, noisy observations (Wang al. 20246). We believe these works offer fruitful pointers towards extending our analysis of sparse systems here beyond its current limitations. At the same time, [Scholl et al. (2023) show that in more general non-linear function classes identifiability requires observing trajectories that essentially "cover the entire state space", putting theoretical limits on what one can hope for in generic non-linear settings. More implicitly, we also assume full observations, i.e., that the entire state relevant for the evolution of the system can be observed. In many practical settings, one may be limited by partial observability. Existing results indicate that identifiability is generally impossible in such settings without strong assumptions on the observation function (Cobelli & Romanin-Jacur 2007). which is why partial observations are beyond the scope of this work. We discuss our assumptions and their implications in detail in Section A
in the (optimal)
Main goal. Our main goal is to answer the following question: Given a trajectory æ [o, T] - IR that solves the IVP in Eq. (1), (when) can uniquely infer A under the assumption that A is sparse?
Let us first formalize and concretize this identifiability problem.
3

---

Published as a conference paper at ICLR 2026
Definition 1 (identifiability from in Ω S R=x*), Ler A Ω S R and let [0, - R= be solution of (t)) dæ (t) (t). We call A identifiable from in Q if there exists no B E Ω2 with B + A and
(I)().
Remarks. The IVP in Eq. (1) has unique solution all of R for all A and Zy by the Picard- Lindelöf theorem and the fact that linear functions are globally Lipschitz ( (Amolld 1992). Therefore, instead of [0, 7] we can use any interval (including all of IR) as time interval in Definition 1 In this work, we are specifically interested in identifying A for a given observed trajectory. Related but different notions of identifiability studied in the literature include, for example (Stanhope et al.) 2014 Qiu et al. ) (a) For fixed initial condition oo E I2", all pairs of distinct systems A, B E lead to different trajectories. (b) For all pairs of distinct systems A, B E Q, there exists some initial condition Zn E IR" leading to different trajectories. (c) For all pairs of distinct systems A, B E S2, every initial condition 2n IR" leads to distinct trajectories. It is easy to see that notion (b) is true "globally." i.e., even for Ω = R.** (Stanhope et al. 2014). This relates closely to the typical linear time invariant (LTI) setting in system identification including controls: when we can "probe" the system by choosing different initial conditions as controls, Instead, we concretely focus on identifiability from merely observational data in the form of a single solution trajectory. On the flip side, (c) is not true for Ω Re*x (and does not even hold for "most" Ω S TR=x" see Qiu et al. 2022 Sec. S1). This is primarily due to the existence of "unlucky" initial conditions, which we will make rigorous below.
always able to identify it fully,
To decide when a system can be identified from a given trajectory, it is useful to characterize whether there are systems for which this is never possible. Throughout this work, will primarily focus on the setting where Ω = Rax= reflecting the fact that we typically cannot exclude any systems from potentially being the ground truth, while still allowing for different probability measures on Ω.
Definition 2 (global unidentifiability). We call A E R=x= globally unidentifiable in Ω. if there exists no Do E I such that A is identifiable from the corresponding solution trajectory x in Ruxa, We denote the set of all globally unidentifiable A by U C _
In words, system A is globally unidentifiable if any solution trajectory is also the solution some other system B (this need not be a single alternative B, but for each trajectory there must exist at least one). A necessary and sufficient condition for global unidentifiability that characterizes these "hopeless" " systems reads as follows.
Theorem 1 (Stanhope et al. 2014 Thm 2.5 + Thm 3.6). A system A E RR is globally unidentifiable if and only it has more than one Jordan block corresponding to the same eigenvalue (for at least one eigenvalue) in the Jordan normal form of A.
Additionally, they also characterized when A is identifiable from x.
Theorem (Stanhope et al. 2014 Lem 3.4 + Thm 3.4). A system A E R=x" is identifiable from æ if and only if (0)) is not contained in any A-invariant proper subspace of TR..
Whenever any point along the solution trajectory (e.g., the initial condition) lies in an A-invariant proper subspace of A, the entire trajectory will be confined to this subspace. The trajectory is thus not "probing" the complement of this subspace in B We can then construct another system B that agrees with A (viewed as linear maps) on the A-invariant subspace, but differs on the complement.
3 SYSTEM-LEVEL UNIDENTIFIABILITY
We will refer to A being in 2/ as "system level unidentifiabilityy" because when A E l it is unidentifi- able regardless of the observed trajectory. For all & 24, there exists at least one solution trajectory x, such that A is identifiable from in gax= However, there may still be "unlucky" æ, namely those confined to an A-invariant proper subspace, for which remains unidentifiable. For any system A, we define UA C TR be the set of initial conditions 2n R from which A is not identifiable. We call this "trajectory level unidentifiability.
Ultimately, main goal aims at quantifying whether we should expect to be able to identify "a random system" from "a random solution trajectory." To formalize this question, consider a joint probability distribution PA PA.., over pairs of systems and initial conditions (A, no) RRan x R
4

---

Published as a conference paper at ICLR 2026
(equipped with the Borel-o-algebra). We are then interested in
PA.a, (A not identifiable from zo) = PA.Eo (u ()
PA(l) XAHu (UA |) A)dPa(A).
where X is the indicator function, i.c., (A) 1 if and 0 otherwise. The partitioning in the last equality corresponds to a two-step reasoning: What is the probability of hitting a globally unidentifiable system (on the system identification level) and, separately, for any non globally unidentifiable model, what is the probability of hitting "unlucky" initial condition
Before developing a lower bound for Eq. (2) we introduce the assumed sparsity model.
Definition 3 (Sparse-Contimune Continuous Ensemble). We call matrix
(Big with Bej Ber(1 p) independent Bernoulli random variables for some p E [0, 1], and Xij Px independent continuous random variables (i.e., their distribution is absolutely continuous with respect to the Lebesgue measure on R., a sparse-continuous random matrix (or system) with sparsity level p.
p)
Examples include any iid sub-Gaun Gaussian random matrix masked by iid Bemoulli variables.
Lemma 1. For a sparse-continuon random matrix A, the probability that A has repeated nonzero eigenvalues is zero. In other words,
PA(IAE R E rank(A- AI) < n 13) = PA(AA E Rax (Aan < 1}).
Proof. Let the random matrices B,X that make up the sparse-continuous random matrix A be defined on a probability space (S2, A,P). P). For a fixed zero-pattern S S [m|] (specifying which entries are nonzero) we define
Es (w E Ω Biy(w) iff (i.j) E S} n {w e Ω A(w) has a repeated nonzero eigenvalue)-
We have (([((w) S}) = p S p) -181 and conditionally on S(w) S. A(w) is vector of [S] continuous random variables (Xij) that is absolutely continuous with respect to the Lebesgue measure RISI. The matrix A has repeated nonzero eigenvalue if there exists an eigenvalue À > 0 of algebraic multiplicity at least 2, i.e., det(A A - Aʃ) = and Zx det(A - A/) = 0. These conditions define a nontrivial algebraic subset in R=*,, imposing polynomial equations on these |S| nonzero entries. These polynomials vanish only on measure-zero subset RRIS (since the polynomials are not identically zero). Thus, P(Es) 0. And by the union bound,
d
P([A has repeated nonzero cigenvalue))
Σ P(Es) 0. SCIN
The second statement follows since the two events characterize having some repeated eigenvalue and zero being repeated eigenvalue, respectively.
With these preliminaries, we find the following lower bound.
Lemma 2. A sparse-continuong random matrix with sparsity is globally not identifiable with probability least -1-
>- for (and p for n
The proof provide in Section B is driven by the presence of zero rows/columns. These correspond to sink or source nodes in the corresponding adjacency graph, which are typically frequent in sparse graphs and dominate the probability of rank deficiency. However, neither zero rows/columns nor an iid Bernoulli sparsity structure are required. In Lemma 5 we prove a non-zero lower bound for more general sparsity patterns that also allow for different degree distributions (e.g.. accommodating hubs) and also empirically study a sparsity model that excludes zero rows/columns in Section D.6
Next, we prove sharp threshold on the dimension and sparsity level p for global unidentifiability.
'The negation of "globally unidentifiable" is not called "globally identifiable" as this would convey the wrong impression of it being "always" identifiable, i.c., from any initial condition.
5

---

Published as a conference paper at ICLR 2026
Lemma 3 (sharp threshold for global unidentifiabilitty.. Let A be a sparse-continuout matrix with n-dependent sparsity level p(n). Then, for Po(n) = = ln(n) and any function w(n) + 30 we have that if p(n) -p(nn)+ + -(m), then P P(rank(A) < - n. - 3c and if p(n) pn( _(m)
n
then P(rank(A) S. n 2) + 0 for n + oc. That is, there is a threshold at p = In(n)/ n decisive for whether A is asymptotically globally unidentifiable with high probability or not.
The proof can be found /eectio This is consistent with known results on Bernoulli matrices (e.g.. Frieze & Karoński 2015] Basak & Rudelson (02T) where there exists a similar sharp transition at p = 1 In(n)/n for the probability of singularity. Our proof extends the same threshold to two-fold rank deficiency in sparse-continuous matrices and thus to global unidentifiability. In practice, numerically evaluating matrix ranks requires thresholding. By the Eckart-Young Young-Mirsky theorem the second smallest singular value d2 of A corresponds to the distance of A the closest matrix A' with rank(A') < n - 1. As such it provides a robust continuous numerical measure of unidentifiability.
Remark sparsity, rank-deficiency, and non-identifiability. (Stanhope al. (2014); Qiu et al. (2022) establish a tight connection between global non-identifiability and rank deficiency to then argue that rank deficiency (and thus unidentifiability) rare. One of our key contributions is to demonstrate and quantify how sparsity, commonly assumed in dynamical models, leads to rank deficiency that ultimately results in unidentiffabiiltyy Crucially, this link does not rely on sparsity inducing entire zero rows/columns (obviously giving rank deficiency), but also holds for sparse systems without zero rows/columns, see also also [Section D.6 Finally, we link to systems studied in the literature to corroborate that the found phenomena are relevant for real-world modeling SSection C
Crucialll
4 TRAJECTORY UNIDENTIFIABILITY
We now turn to "trajectory unidentifiabilityy.. i.e. the situation where A is not globally unidentifiable, yet it may be unidentifiable from specific observed trajectories Let (A,zo) ~ PA @ Pa. with an absolutely continuous Pa, (w.r.t. the Lebesgue measure on R²) and PA the distribution of a sparse-continuous matrix A. Since, according to Thheremm 2. A is unidentifiable from if and only if xo is in a proper invariant subspace of A, and the fact that any proper subspace of TR. has zero probability under any absolutely continuous probability measure (with respect to the Lebesgue measure), we conclude that PA. (MA A = A') O for all A' $ U. Therefore, the second term in Eq. (2) is zero. The probability of a "random" " A being identifiable from a "random" solution trajectory is thus given entirely by the probability of A being globally unidentifiable the probability of "unlucky" initial conditions zero.
However, 2o being close to a proper A-invariant subspace can still be problematic in practice, which we later demonstrate empirically. To measure this "closeness to unidentifiabilityy. we define
1
d RR" + [0. 1), Zo + a
min min(dist(ao. V) | V E I(A)).
where I(A) is the set of proper A-invariant linear subspaces of R" and dist V) == minyev ||
y/12 is the Euclidean distance of xn from the subspace V. We have that d (xo) = 0 and only æo lies in proper A-invariant subspace, and A provides continuous measure for the "level of
unidentifiability from the trajectory." formalized as follows.
Lemma 4. Let A' E gaxn and assume there is an A-invariant subspace V such that (A A')V= = [0]. Then, for every? t 2 and Do E RR, we have that
||eA1x0 xo- -e''xxol
C(t,,, A' (x(
with C(t,A, = So lalle (tt I12lle (1 lldds. Further, for any = >> 0, the condition
l1eerr - a'r) zoll Se for all 0 SiSs
holds whenever
8 T 15-Wo W A M d da(zu))
where W-)) denotes the Lambert function and constants E R, M 2 1 such that lle || s at for all 2 O.
Generally sparse matrices can have full rank (e.g., identity) and full rank matrices can be dense (all ones).

---

Published as a conference paper at ICLR 2026
Rn-
=an
Zero Eigenvalue
50
50
p 1 In(n)/n
40
40
40
unidee Mibebeeee
30
30
20
20
20
10
10
0.95
0.2
0.5
0.8 0.9
0.95
0.2
0.95
sparsity p
sparsity p
sparsity p
Figure 2: Proportion of matrices satisfying the conditions i), ii) and iii) at different system dimensions n and sparsity levels p. All three conditions exhibit sharp increase in frequency with higher sparsity, consistent with the threshold p 1 la(n)/n.
The proof can be found in Section BJ Assume A is identifiable from 2g. but 3g is close proper A-invariant subspace V* with distance (20)) Then there exists another A' +f A that agrees with A on V*, i.e., A' could practically be mistaken for the underlying true dynamic. [Lemma gggurrnttsss e-closeness of A and A' up to time T, which scales inversely with both the distance d (Zo) and the norm | 1- A'Il2, so trajectories remain indistinguishable for longer horizons when the initial condition lies closer to invariant subspace when competing matrices A and A' differ only slightly. While lemma 4 holds for general matrices, it is particularly relevant for sparse matrices as these more likely have large invariant subspaces and are consequently more likely unidentifiable in practice.
EMPIRICAL RESULTS
Our empirical validation experiments include three distinct viewpoints: firstly, assume the true system matrix to be available in order to confirm our theoretical results on unidentifiability in sparse matrices in principle. Secondly, we assume that only have observed trajectories at hand and evaluate trajectory-level identifiability criteria. While these results assess whether system is (or is not) identifiable, finally also empirically assess the performance of two widely used estimators that attempt to learn the underlying system from data directly, hence zooming in on identifiability challenges in practice.
Data generation. We focus on systems of dimensions n E D = (3,5, 10. 20, 30, 40, 50} and sparsity levels p E P = {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.95, 1.0) For each (n. p) D x P. we generate 100 matrices according to the sparse sparse-contiuouus model in Definitio 3 and take standard normal for the continuous distribution Px = (O, 1). For each matrix, we sample 100 initial conditions uniformly from the unit circle in R and solve the corresponding trajectories numerically over the interval E [0, 1] using a standard explicit RK45 solver with 512 homogeneous time steps. Results for different random matrix models, for example disallowing zero rows/columns, can be found in Section
5.1 SYSTEM LEVEL UNIDENTIFIABILITY
In this section, perform exploratory analysis of global unidentifiability conditions by examining properties at the system level. The contour map in Fig. 2 iillustrates for varying values of sparsity p and dimension n, the empirical frequency with which the generated system matrices A violate at least one of three structural criteria: (i) rank(A) < n 1; (ii) there exists an eigenvalue À for which rank(A AI) < n 1; or (iii) the spectrum contains the zero eigenvalue. As matrices become sparser (i.e., as increases), the proportion of globally unidentifiable matrices sharply increases. Notably, the nearly identical contour plots for conditions and (ii) strongly support our theoretical result stated in Lemma Moreover, the condition regarding the existence of zero eigenvalues aligns closely with the theoretical threshold p ln(a) (depicted by the red curve).
Discussion on realistic sparsity levels. Although global (system-level) unidentifiability becomes less critical as the dimension increases (consistent with the threshold p In(n)) seemingly
Recall that higher p for corresponds "more zeros."

---

Published as a conference paper at ICLR 2026
p-valne
10
10
10
10
10
10
10
90
10
10
10
30
1.0
0.8
Log(SCN)
20
0.6
10
2,, min
0.2
02, min
22
max
0.0
5
20
30
3
5
10
20
30
40
Box- Box-plots of smoothed condition numbers (SCN) and destance-to-umidetiiiee da for the least and most identifiable groups of systems at different dimensions n.
Figure 3:
only leaving systems of negligibly high sparsity- unidentifiable, we emphasize that such high sparsity regimes are not merely theoretical but are indeed realistic and frequently encountered in real-world scenarios. For instance, the P. trichocarpa PEN gold standard network (Walker al. 2022), introduced earlier, has sparsity level of approximately 0.997 for graph comprising 1690 nodes (genes), exceeding the theoretical threshold p" 1 in(2) ~ 0.996. Similarly, the E. coli gene regulatory network (Walker et al. 2022), consisting of 1222 genes, exhibits sparsity of about 0.999, surpassing its corresponding threshold of p" 0.995.
5.2 TRAJECTORY-LEVEL UNIDENTIRABILITY
Following the argument in Section 3, the second-smallest singular value 02 of a system relates to its global unidentifiability. Here, we ask whether systems with worse global unidentifiability metries also give rise to trajectories that are statistically less identifiable. To address this question, we rank the generated system matrices by their second-smallest singular value d2 for each combination of dimension n and sparsity p. Matrices with falling into the top- top-10% of ranked singular values are subsequently referred to Aoz.mazi similarly, matrices the bottom- 10% are referred as A maim
To determine whether the trajectories produced by these two matrix groups differ identifiability, we compared the sample means of two complementary trajectory-level identifiability metrics: our closeness-ue ceseenss-tt-unndee un--ueniiiiibbbttt distance d' (2o) from Section (larger implics more likely identifi- able) as well as the smoothed condition number (SCN) as established by QQu et al. (2022) (smaller implies more likely identifiable). Specifically, we approximate d (xx)) by the normalized distance of the initial state to ker(A) (A), using the metric
d (wo) 2 da,o(=0)) = |!rO
dist(xo, ker(A)) E [].
Compared to A (Tro), dA.0( allows us to estimate ddiisance-to-umideeniiibbil all proper A-invariant subspaces. As a second criterion, we evaluate the invertibility of the pairwise inner product matrix, as established by Qiu et al. (2022), who showed that trajectory æ(t A, xp) is
without analyzing
identifiable and only if the pairwise inner product matrix Exg (x(t), a(t)),, So 2 (t) (t) is invertible. Qiu et al. (2022) offer a straightforward method for approximating Exz- and introduce the Smoothed Condition Number (SCN) k(E..)) as the condition number of Exa-
Results. Boxplots of SCN and d values are depicted in Fig. 3 at different dimensions n. For each In in(n//n, which corresponds
n we focus on the sparsity parameter p" closest the threshold to the critical sparsity value at which a system becomes unidentifiable. We use Welch's d t-test with one-sided alternatives that reflect the expected direction of the difference:
SCN:
Hu Aug.
2
SCN Aog
H1 : SCN
Aogg
SCN Poogmmin
d
Ho 1 Aeg.mmn
2 A'ggmmma
H1 : p Aeggomn
da
1l Aog.mma
Here MAog.maz and mmig.mnn denote the population means in the two subgroups. The goal of the statistical test is to establish whether the two subgroups truly differ performance. We find that systems in subgroup Ao..min significantly less probable to identify (p < 0.01) for all system dimensions and both criteria. These results are consistent with our theoretical derivations and also in line with the previously analyzed systems- criteria
iesectoryyidentifabbliig
8

---

Published as a conference paper at ICLR 2026
SINDY
NODE
30
Formmalli ham distance
1..
10
0.7
0.0
20
10
10
0.4
0.1 0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.90.95
0.1 0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.90.95
sparsity p
sparsity p
Figure 4: Normalized Hamming distance of reconstructed systems using SINDY (left) and NODE (right) across varying dimensions and sparsity levels. Reconstruction accuracy declines noticeably with increasing dimensionality and sparsity.
5.3 EMPIRICAL UNIDENTIFIABILITY
We evaluate the performance of two state-of-the-art models for identifying dynamical systems: Neural ODEs (NODEs) (Chnn al. 2018). which approximate dynamics through neural-nettork--bsee function fa((()) as well as the Sparse Identification of Nonlinear Dynamics algorithm (SINDy) (Brunton et al. 2016), which employs Ly-regularized linear regression predefined set of basis functions. To effectively model sparse linear systems, we utilize an L -reruularize neural network without activation functions for NODEs, and linear basis functions for SINDy. Hyper-parameters for both models are tuned per system. More details on the methods can be found Secciin D.3
Since are less interested in the question of whether the model can fit the observed data overall, but rather in the question of whether the models correctly identify the underlying system matrices A in cases where they do fit the data well, filter estimates for which the corresponding reconstruction of the observed trajectory does not satisfy prescribed R² and MSE thresholds, where these metrics compare observed and reconstructed trajectories. This best-case filtering ensures that subsequent empirical results can be tied to identifiability properties rather than potential issues with model optimization or architecture.
Results. As perfect point estimates of all coefficients of system matrix A are unreasonable to expect (e.g. due to numerical issues), we instead use the Hamming distance to compare binarized ground truth and estimate Å, which we additionally normalize by n² for comparability across dimensions. The Hamming distance corresponds to the number of matching symbols (here: zero one) and thus captures the overlap of the predicted and true sparsity patterns. The heatmaps in Fig. 4 show for both SINDY and NODE clear left-right gradient, indicating that sparse systems lead larger Hamming distance consistent with our theoretical finding that higher sparsity increases the probability of unidentifiabilltty In addition, results for SINDy also show an unexpected up-down gradient and high sparsity levels, which were not predicted by our theory. This behavior can be tied to the (default) sequentially thresholded least- sauared optimizer that is used fit SINDy the data: Following least- squared optimization, this optimizer sets any coefficient that falls below a ussered magnitude threshold zero hence promoting sparsity aggressively in the fitted model. cases of very high sparsity and low system dimensionality. most coefficients of the true system matrix will be zero so that thresholding coefficients towards zero biases the model towards smaller Hamming distances. As the dimensionality increases the sparsity reduces towards moderate sparsity levels, this effect vanishes as there are multiple non-zero coefficients.
Exemplary model estimates and corresponding trajectory fits are displayed for selected dense and sparse system in Fig. which further illustrates the phenomenon.
CONCLUSION
In this work, we focus on identifiability of sparse linear ODEs A from a single observed trajectory We first partition the probability unidentifiability of from z into "system level unidentifiability" "unlucky" initial condition. After
(A E U)) and "trajectory level unidentifiability".. i.e., hitting showing that the latter scenario almost surely does not happen, we show that in sparse systems the probability of unidentifiability be lower bounded by a positive quantity and exhibits sharp

---

Published as a conference paper at ICLR 2026
asymptotic threshold at p In(n)/n for global unidentifiability. This puts important asterisk on the celebrated result that "almost all linear ODEs are identifiable," which ceases to hold under sparsity, which is often a underlying assumption in dynamical system modeling. We go to empirically verify that theoretical unidentifiability is a serious practical challenge and extend the "trajectory level," demonstrating that the theoretically
our analysis near-unidentifiabliiy negligible case of unlucky initial conditions further exacerbates the problem. Important directions for future work include extending our results to affine and post-nonlinear models, discrete, noisy or partially observed systems, multiple observed trajectories, and deriving computable metrics for practical unidentifiability from data alone.
REPRODUCIBILITY STATEMENT
Significant effort was made to ensure reproducibility, both for the theoretical and experimental results.
The complete proofs of Lemma 2 Lemma 3 Lemma 4 can be found in Section B.1 Section B.2 Section B.3 respectively. Additionally, we add discussion on the assumptions and limitations in Details regarding implementation and metrics can be found in |Section D specifically software resources in Section D.I metries details in Section and methods implementation details The data generating mechanism is described in Section 5 Code is publicly available
Section 2
in Section D.3
at httpt:: http:::/ggituuu.. com/ ceci www.ciiiaaaalooooooooooooooo http:///h....ccceeeeeeaaaaaaaaaa ideetiddddifff ability
THE USE LARGE LANGUAGE MODELS
In this work, Large Language Models (LLMs) were used writing assistant to polish the text and improve clarity of exposition.
ACKNOWLEDGMENTS
This work is supported by the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research. This work has been supported by the German Federal Ministry of Education and Research (Grant: 011S24082). The authors gratefully acknowledge the Gauss Centre for Supercomputing e. V. (www.gauss www.s-ccentre...eemmm for funding this project by providing computing time through the John von Neumann Institute for Computing (NIC) on the GCS Supercomputer JUWELS at Jülich Supercomputing Centre (ISC).
REFERENCES
Hananeh Aliee, Fabian Theis, and Niki Kilbertus. Beyond predictions in neural odes: Identification and interventions. arXiv preprint arXiv:2106.1244 2021.
Hananeh Alice, Till Richter, Mikhail Solonin, Ignacio Ibarra, Fabian Theis, and Niki Kilbertus. Sparsity in continuous-depth neural networks. Advances in Neural Information Processing Systems, 35:901-914, 2022.
Vladimir Arnold. Ordinary differential equations. Springer Science & Business Media, 1992.
Brian Bartoldson, Ari Morcos, Adrian Barbu, and Gordon Erlebacher. The generalization-staaitte tradeoff in neural network pruning. Advances in Neural Information Processing Systems, 33: 20864, 2020.
Anirban Basak and Mark Rudelson. Sharp transition of the invertibility of the adjacency matrices of sparse random graphs. Probability theory and related fields, 180:233-308, 2021.
Sören Becker, Michal Klein, Alexander Neitz, Giambattista Parascandolo, and Niki Kilbertus. Predicting ordinary differential equations with transformers. In International conference on machine learning, pp. 1978-2002. PMLR, 2023.
Ror Bellman and Karl Johan Åström. On structural identifiability. Mathematical biosciences, 7(3-4): 329-339, 1970.
Alexis Bellot, Kim Branson, and Mihaela van der Schaar. Neural graphical modelling in continuous- time: consistency guarantees and algorithms. arXiv preprint arXiv:2105.025:: 2021.
10

---

Published as a conference paper at ICLR 2026
James Bradbury. Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL
http:/ http//ggtttbbbbb jammmjjxxmm/aa
Steven L. Brunton, Joshua Proctor, and Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy 15173841 151.///....1118411111111
of Sciences, 1 113(11))39322333333 2016. ISSN 0027-8424. doi: 10.1073/pnas.
URL
https: www. pnas org wonttnt http://tnnnennn///39999/9 http://wwwwppsooooooocoooooo/a//
Michelle Carey, Juan Camilo Ramírez, Shuang Wu, and Hulin Wu. A big data pipeline: Identi- fying dynamic gene regulatory networks from time-course gene expression omnibus data with applications to influenza infection. Statistical methods in medical research, 27(7):133--1555 2018.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Claudio Cobelli and Giorgio Romanin-Jacur, Controllability, observability and structural identifia- bility of multi input and multi output biological compartmental systems. IEEE Transactions on Biomedical Engineering, 2007.
Daniel Commenges, D Jolly, Julia Drylewicz, Hein Putter, and Rodolphe Thiébaut. Inference in hiv dynamics models via hierarchical likelihood. Computational Statistics Data Analysis, 55(1): 146-456, 2011.
Nik Cunniffe, Frédéric Hamelin, Abderrahman Iggidr, Alain Rapaport, and Gauthier Sallet. Identifia- bility and Observability in Epidemiological Models. Springer, 2024.
Stéphane d' Ascooli Sören Becker, Alexander Mathis, Philippe Schwaller, and Niki Kilbertus. Ode- former: Symbolic regression of dynamical systems with transformers. In International Conference on Learning Representations, 2024.
Brian de Silva, Kathleen Champion, Markus Quade, Jean- -han-Chrittophe Loiseau, J Nathan Kutz, and Steven Brunton. PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data. Journal of Open Source Software, 5(49): 2020.
Yi Ding and Panos Toulis. Dynamical systems theory for causal inference with application to synthetic control methods. In International Conference on Artificial Intelligence and Statistics, pp. 1888-1898, PMLR, 2020.
Jérémie Donå, Marie Déchelle, Marina Levy, and Patrick Gallinari. Constrained thysical-statistinn models for dynamical system identification and prediction. In ICLR 2022-The Tenth International Conference on Learning Representations, 2022.
Duan, Rubin, and D Swigon. Identification of affine dynamical systems from single trajectory. Inverse Problems, 36(8):085004, 2020.
Mikhail V Fedoryuk. Asymptotic analysis: linear ordinary differential equations. & Business Media, 2012.
Springer Science
Alan Frieze and Michal Karoński. Introduction to random graphs. Cambridge University Press, 2015.
BC Gargash and DP Mital. A necessary and sufficient condition of global structural identifiability of compartmental models. Computers in biology and medicine, 10(4):237-242, 1980.
Aric A. Hagberg, Daniel A. Schult, and Pieter J. Swart. Exploring network structure, dynamics, and function using networkx. Gaäl Varoquaux, Travis Vaught, and Jarrod Millman (eds.), Proceedings of the 7th Python in Science Conference, pp. 11 15, Pasadena, CA USA, 2008.

---

Published as a conference paper at ICLR 2026
Charles R Harris, K Jarrod Millman, Stéfan J van der Walt, Ralf Gommers, Pauli Virtanen, David Courapeaa,, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Marchann, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 2020.
Yangxin Huang, Dacheng Liu, and Hulin Wu. Hierarchical bayesian methods for estimation of parameters in longitudinal hiv dynamic system. Biometrics, 62(2):413-42) 2006.
John D Hunter. Matplotlib: A 2D graphies environment. Computing in Science & Engineering, 2007.
Mare Lavielle, Adeline Samson, Ana Karina Fermin, and France Mentré. Maximum likelihood estimation of long-term hiv dynamic models and antiviral response. Biometrics, 67(1)2250-,59,
2011.
Zhengfeng Li, Michael Osborne, and Tania Prvan. Parameter estimation of ordinary differential equations. IMA Journal of Numerical Analysis, 25(2):264-25 285, 2005.
David Liben-Nowell. An algorithmic approach to social networks. PhD thesis, Massachusetts Institute of Technology, 2005.
Marten Lienen and Stephan Günnemann. torchode: A parallel ODE solver for pytorch. In The Symbiosis of Deep Learning and Differential Equations II, NeurlPS, 2022. URL https://
openreview net/
net/ Sreenteriww.net//ooumttttiiiiaie
Tao Lu, Hua Liang, Hongzhe Li, and Hulin Wu. High-dimensional odes coupled with mixed-effects modeling techniques for dynamic gene regulatory network identification. Journal of the American Statistical Association, 106(496):
2011.
Daniel Marbach, James Costello, Robert Küffner, Nicole M Vega, Robert J Prill, Diogo M Camacho, Kyle R Allison, Manolis Kellis, James J Collins, et al. Wisdom of crowds for robust gene network inference. Nature methods, 98))796-804, 2012.
Hongyu Miao, Xiaohua Xia, Alan S Perelson, and Hulin Wu. On identifiability of nonlinear ode models and applications in viral dynamics. SIAM review, 53(1):3-39, 2011.
Muñoz-Tamayo L Puillet, JB Daniel, D Sauvant, O Martin, M Taghipoor, and P Blavy. Review: To be or not to be an identifiable model. is this a relevant question in animal science modelling? animal, 12, 701-712, 2018
Alexey Ovchinnikov, Gleb Pogudin, and Peter Thompson. Input-output equations and identifiability of linear ode models. IEEE Transactions on Automatic Control, 68(2):812 122228822-824 2022.
The pandas development team. Landaa-dev/pandass Pandas, February 2020. URL https://dol org/10. (tl1110.2212/eeeeee 2
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Detito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performanee deep learning library. In NeurlPS, 2019.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jak Vanderplas, Alexan- dre Passos, David Cournapeau, Matthieu Brucher, Matthicu Perrot, and Édouard Duchesnay. Scikit-learn: Machine learning in Python. JMLR, 2011.
Tong Qin, Kailiang Wu, and Dongbin Xiu. Data driven governing equations approximation using deep neural networks. Journal of Computational Physics, 395:620-633 2019.
Xing Qiu, Tao Xu, Babak Soltanalizadeh, and Hulin Wu. Identifiability analysis of linear ordinary differential equation systems with a single trajectory. Applied Mathematics and Computation, 430: 127260, 2022.
12

---

Published as a conference paper at ICLR 2026
A Raue, V Becker, U Klingmüller, and J Timmer. Identifiability and observability analysis for experimental design in nonlinear dynamical models. Chaos: An Interdisciplinary Journal of Nonlinear Science, 20(4), 2010.
Chiara Ravazzi, Roberto Tempo, and Fabrizio Dabbene. Learning influence structure in sparse social networks. IEEE Transactions on Control of Network Systems, 5(4):1976-1986, 2017.
Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. Latent ordinary differential equations for irregularly-smmpled time series, Advances in neural information processing systems, 32, 2019.
Maria Pia Saccomani. An effective automatic procedure for testing parameter identifiability of hiv/aids models. Bulletin of mathematical biology, 73:1734-1753, 2011.
Philipp Scholl, Aras Bacho, Holger Boche, and Gitta Kutyniok. Symbolic recovery of differential equations: The identifiability problem. arXiv preprint arXiv:2210.08342, 2022.
Philipp Scholl, Aras Bacho, Holger Boche, and Gitta Kutyniok. The uniqueness problem of physical law learning. In ICASSP 2023-2023 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Shelby Stanhope, Jonathan E Rubin, and David Swigon. Identifiability of linear and linear-in- parameters dynamical systems from single trajectory. SIAM Journal on Applied Dynamical Systems, 13(4):1792-188)) 2014.
Necibe Tuncer, Hayriye Gulbudak, Vincent I. Cannataro, and Maia Martcheva. Structural and practical identifiability issues of immuuo--pidemmologicall vector-host models with application to rift valley fever. Bulletin of mathematical biology, 78:1796-1827, 2016.
Guido van Rossum and Fred I. Drake. Python 3 Reference Manual. CreateSpace, 2009.
Roman Vershynin. Hligh-dimensional probubility: An introduction with applications in data science, volume 47, Cambridge university press, 2018
Pauli Virtanen, Ralf Gommers, Travis Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J van der Walt, Matthew Brett, Joshua Wilson, Jarrod Millman, Nikolay Mayorov, Andrew R Nelson, Eric Jones, Robert Kern, Eric Larson, C Carey, ilhan Polat, Yu Feng, Eric W Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, A Quintero, Charles R Harris, Anne M Archibald, Antônio H Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 2020.
Angelica M Walker, Ashley Cliff, Jonathon Romero, Manesh B Shah. Piet Jones, Joao Gabriel Felipe Machado Gazolla, Daniel A Jacobson, and David Kainer. Evaluating the performance of random forest and iterative random forest based methods when applied to gene expression data. Computational and Structural Biotechnology Journal, 20:3372-3386, 2022.
Benjie Wang, Joel Jennings, and Wenbo Gong. Neural structure learning with stochastic differential equations. arXiv preprint arXiv:2311.03300 2023.
Yuanyuan Wang, Biwei Huang, Wei Huang, Xi Geng, and Mingming Gong. Identifiability analysis of linear ode systems with hidden confounders. Advances in Neural Information Processing Systems, 37:59054 59092, 2024a.
Yuanyuan Wang, Wei Huang, Mingming Gong, Xi Geng, Tongliang Liu, Kun Zhang, and Dacheng Tao. Identifiability and asymptotics in learning homogeneous linear ode systems from discrete observations. Journal of Machine Learning Research, (5(114))1-50, 2024b.
Wes McKinney. Data Structures for Statistical Computing in Python. Stéfan van der Walt and Jarrod Millman (eds.), Proceedings of the 9th Python in Science Conference, pp. 56 61, 2010. doi: 10.25080/Majora-9900 920-rr9-999199220022.aa9
Xiaohua Xia and Claude H Moog. Identifiability of nonlinear systems with application to hiv/aids models. IEEE transactions on automatic control, 48(2):330-336 2003.
13

---

Published as a conference paper at ICLR 2026
Shiyun Xu, Zhiqi Bu, Pratik Chaudhari, and Ian J Barnett. Sparse neural additive model: Interpretable deep learning with feature selection via group sparsity. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 349-359. Springer, 2023.
Andy B Yoo, Morris A Jette, and Mark Grondona. Slurm: Simple linux utility for resource man- agement. In Workshop on job scheduling strategies for parallel processing, pp. 44-60. Springer, 2003.
14

---

Published as a conference paper at ICLR 2026
A DISCUSSION OF OUR ASSUMPTIONS AND THEIR IMPLICATIONS
For our theoretical work we assume continuous, noise-free, fully-observed trajectories, which may at first glance appear limiting. Indeed, those assumptions would be limiting if this was a method proposal, where the method only works on such observations. However, for analyzing unidentifiability, they are the right starting point: Without these assumptions, no system would ever be identifiable, i..., unidentifiability would be trivially given. Only once we have eliminated unidentifiability due to, e.g., undersampling. finite samples, and partial observations, can we make non-trivial claims about the inherent unidentifiability of the underlying systems. We split the discussion of our assumptions into two parts.
1. The assumptions of continuous trajectories, noise-free observations and full observability are assumptions about our ability to observe/measure the system. For these, our setting corresponds to the "best case" idealized setting where (in principle) infinite data (both in terms of sampling frequency as well as in terms of noise instantiations) are available about all relevant parts of the system. As we will discuss one-by-one below, violating any of those assumptions will add additional identifiability issues, which are due to limitations of our observation process (not inherent to the system) that one could in principle overcome. Hence, instead of "our results only apply under those assumptions," our results show unidentiffabiliiy "despite these assumptions", even when all solvable sources of unidentifiability due to limited measurement processes have been overcome. In short, these form the appropriate starting point for an identifiability analysis.
2. The assumption of linear systems and sparsity are indeed assumptions about the underlying dynamics. We elaborate on the relevance of these assumptions below.
1a. Continuous trajectories. Any real-world measurement device will have maximum sampling frequency, so all real-world data is necessarily discrete in time. Discrete observations add trivial unidentifiability issues around aliasing (ef. aliasing in the Nyquist sampling theorem). Hence, the continuous observation case is the best-case scenario the limit of infinite sampling frequency- where we expect any remaining unidentifiability be due to the fundamental problem setting, not our insufficient sampling rate.
Again, real-world data are almost always noisy. And again, in the presence of noise, system identification becomes inherently more difficult, namely probabilistic: multiple different systems may be compatible with the observed data, each with a different likelihood under the assumed noise model (for example, the standard assumption of i.i.d. additive Gaussian noise in classical machine learning often used to model "measurement errors"). Within this probabilistic framework, our noise-free setting can be interpreted as the best case scenario, where unidentifiabiliiy is not just due to suboptimal measurement devices. It can also be interpreted as the asymptotic limit of observing infinitely many trajectories for single initial condition such that the "noise can be averaged out" to recover the noise-free setting. From this perspective, our results state under which conditions collecting more observations allows (in the asymptotic limit) the identification of the system. Or put differently, we show that there are cases in sparse systems where, even if more data was collected where the noise level is reduced by other means, the true system remains unidentifiable.
lb. Noise-free observations.
1c. Full observations. Akin to discrete or noisy observations, only having partial observations of the system state will naturally render identifiability more difficult. Unlike discrete observations and noise, partial observations add fundamental degree of unidentifiability as there is no "asymptotic limit" of more frequent or precise observations to recover our setting. Intuitively. when we allow for the presence of (an arbitrary number of) unobserved state variables, both the full system as well as the "observed part" (viewed as submatrix of the full system) are inherently unidentifiable. As an extreme (and somewhat pathological) counterexample, one could imagine an exact copy of each observed variable among the latents such that all observed variables may either depend on the unobserved copy or the observed variable, which is indistinguishable from data. Hence, to study the identifiability under partial observations, one would have to provide additional limiting assumptions the allowed type and nature of confounding. Exploring the palette of possible assumptions is an interesting direction for future work.
2a. Linear systems. There are two main reasons for this assumption: (a) Linear differential equation models are core pillar of scientific modeling and widely used in practice due their analytical
15

---

Published as a conference paper at ICLR 2026
tractability, relative simplicity, and inherent interpretability. While the real underlying dynamics of complex systems are arguably rarely perfectly linear, linear models also turn out to be surprisingly useful and predictive (akin to linear regression still being competitive and reliable tool in many settings). (b) Non-linear ODEs are known to be unidentifiable from single trajectories if one does not impose heavy constraints on the allowed non-linear function family (Scholl et al. (023). Hence, the previous proclaimed state in the literature was a clear separation: "non-linear systems are unidentifiable while linear systems are almost surely identifiable." Our work adds important nuances to the part about linear systems.
2b. Sparsity, If were somehow known that real-world systems never evolve according to dynamies below the However, the true sparsity of real-world dynamies remains inherently unknown. While is impossible to determine whether a given unknown system is sparse, the examples in our submission indicate that this may not be unlikely in practice. Our results are thus important in that modeling practitioners should consider them when interpreting their results, e.g., despite perfectly-fit model, the inferred weights might not reflect the true causal relations between variables due to unidentifiability issues. As pragmatic step to highlight that high degrees of sparsity may occur in practically relevant settings, we examined published sparse systems, which we discuss i5 In both reported cases, the published systems are on the sparser side of the threshold required for identifiability, which further supports the practical relevance of our findings. Nonetheless, we emphasize that the relevance of our results in terms of how frequently they show up cannot be empirically proven, since the notion of sparsity in our analysis refers to that of the true but unknown dynamics, rather than the model dynamics (models cannot be proven "true"). However, a practical implication for future method development efforts aimed at inferring the "ground truth" networks is for instance that such methods may produce estimates that fit the observed data equally well as the ground truth, yet are different. Importantly, this discrepancy might not stem from the limitations of the developed method itself, but rather from the nature of the underlying dynamics being estimated.
our findings would indeed be of purely theoretical interest,
B THEORETICAL RESULTS
B. PROOF OF LEMMA 2
Lemma 2. A sparse-continuont random matrix with sparsity is globally not identifiale with 11 for 2 2 (and p for =1)
probability at least - -1- - nip
Proof. Following Theorem 1. the model Eq. (1) is globally unidentifiable if and only if A has more than one Jordan normal block for one of its eigenvalues, say ); which then has geometric
multiplicity (A.)) > 1, i.e., rank(A X; < n 1. According Lemma I it follows that (2 = Pa([A E Raxn (A) < n 1}). Hence, we are looking for the probability of dependencies among columns in A that drop the rank. For a fixed zero structure B := (bL))<JE(M) E {0, 1] nxx the set on which X := (Ii.))i.e[() E Rax= introduces additional lincar dependencies among the rows/columns of A has Lebesgue measure zero and thus zero probability under A. Therefore, P(rank(A) < n - 1)= = P(rank( B) < n - 1). We focus on the sufficient event that B has multiple zero columns. Since the entries of B are independent, P(B.:
0) p" for all E [n] where B. represents the i-th column, it follows that the random variable Z representing the count of zero columns follows a Binomial distribution Z (in q) with q == p". With the probability mass function P(Z~ k) ( gt((1 n** we find
P(rank( A) < n - 1) > P(Z 2 2) = 1 P(Z E (0. = - (1 p")" -np"(L (1-p-)~-1 p²)
provides non-zero lower bound the probability of system level unidentifiability for the simple iid Bernoulli sparsity pattern. However, by analogous reasoning the proof extends to a much broader class of structured sparsity patterns that also allow for the existence of hubs, i.e., for heavy tailed degree distributions.
Lemma 2
Lemma 5 (General sparstsy
lower bound). Let Pn be a probability measure on {0, 1 and let B ~ Pn. Consider sparse-coninnuos random matrix A == (Bij (.))...)).=p where (X E
16

---

Published as a conference paper at ICLR 2026
independent continuous random variables (i.e., their joint distribution is absolutely continuous with respect to the Lebesgue measure on R=*")) and independent of B. Assume that for some
k {0, 1, ................................ 2} there exists set of columns J S [n] with [J 2 k 2 and a set of rows
R C [n] with R S such that the event
Ex B (0. 1)
Bij = 0 for all E J, i $ RR
(5)
has positive probability q = PB (B(Ex) O. Then is globally unidentifiable with probability least i.e. U)) 2 q 0.
Proof. The key idea is that every deterministic matrix M E IR whose zero pattern belongs to Eg has rank(M) I n 2 and is therefore system level unidentifiable. For such matrices, for every j E J the j-th column M.., is supported only on rows in R. Hence each M. with j E lies in an |R -imennsional subspace. We thus found at least & + 2 columns of M that lie in the same |R| k dimensional subspace, which decreases the rank of M by at least and by [heorem IJ M is globally unidentifiable. Now consider the random matrix A (B.y Xij) Since the above argument only depends on B. we conclude that rank(A) S n 2 whenever B E Eg.. Hence P(A E 4 B) = for all B Es and by total probability
P(A u1 EB (PAEE u B)] 2 (B) B)] (B = 2
The main takeaway is that as long there may be some variables that are sparsely connected among each other with sufficient degree of sparsity there will be a non-zero probability of system level identifiability- in contrast the dense setting.
B.2 PROOF OF LEMMA 3
Lemma 3 (sharp threshold for global unidentifiability). Let A be a sparse-continuous matrix with n-dependent sparsity level p(n). Then, for Pe(n) = la(n) and any function w()) co we have that p(n) = pe(n))+ + w(n), then P(rank(A) 2 - for n - oc and if p(n) = p.(nn) w(x) then P(rank(A) (A) S n -2) 0 for + That is. there is a threshold at p In(n)/ /n decisive for whether is asymptotically globally unidentifiable with high probability or not.
Proof. From A form the random bipartite graph Go,n.e with A the corresponding adjacency matrix. Edges are present independently with probability s = s(n) = 1 p(2).
For every bipartite graph Gn,n, let
mG)) = ma] M is matching),
d(G) = max (IS) - (N())
(6)
SCMN
(Hall deficiency). From Hall- Höll-K theory for graph matching, the rank of the adjacency matrix A m(G) < rankA < n - dG)).
Given
s(n) = Inn + a(n)
n
we have that (Frieze& & Karoński 2015) P(m(G) = n) + 1 a(n) + oo and P(m(G) = n) + if a(n)
Case ann) + oc. Let Z be the number of isolated vertices in Gn.,.- Without loss of generalization, we will consider isolated vertices from rows. Any fixed vertex isolated with probability (1 x)" so
Z (nn (1 )))",,
E|Z] n1 - =s) - -a()) (1 o(L),
Var[ = -
Chebyshev's 's inequality therefore yields P(Z 2 2) - for n - co. Two isolated vertices form a set
S with [N( |S| 2, so d(G)) 2 2 and consequently PrfrankA $ 2] 1.
Case (n) + +oo. Then (Gm = n) + 1, hence P(rank(A) ()) S n - 2) + 0. Substituting
1 p concludes the proof.
17

---

Published as a conference paper at ICLR 2026
B.3 PROOF OF LEMMA 4
Lemma 4. Let A'
A' E Ruxn and assume there is an A-invariant subspace V such that (A
A')V* [0). Then, for every 1 and wo IR", we have that
| xo E' eo'tx0l122 S
A (x
with C(t, A, == lolle (llledl-0))la |aaalell' 112ds. Further, for any e > 0, the condition
l1eeAe - eA')) xoll = for all 0 sts T
holds whenever
w ((a |A- A M d da(zo)
8
Ts a
where (-) denotes the Lambert function and constants E R, M 2 1 such that ||e ||e - M for all 2
Proof. Let assume without loss of generality (otherwise simply get loser bound) that V is the closest proper A A-invariant subspace to Ty- We can then decompose Zy as Zu = III To + w with |w||| da (w0) (by definition, all norms are 2-norms). Given the assumption
and A', we have
(e e )zyg (e e't) (IT (II + ux) (eAt c't))..
From a variation of constants approach and the triangle inequality, we have
Ie-i- -14-11491111 Alt (A - A A's ds 1 $[[ [|ed(0=)) (lle I11A A' A'Ilds,
which ultimately gives the result
||e exltzull | ed'll| ||/|| S IIA A'(lds(z0)) ||odit->)||loott [1e
For the second statement, we note that
wlll s I|A A'll 2² ||ll (f cat ds) -11A ||I-A'1) A'l d a
l|ee )
which is bounded by for t E [0, T] with
T Te
|I - A'll M d (xo)
Using the definition of the Lambert W function W--) we obtain
T w(a
a [ A'll M (
C EMPIRICAL EVIDENCE OF NON-IDENTFFABLITY IN GRNS
We elaborate the practical relevance of non- non-identifiability by discussing real-world gene reg- ulatory networks (GRNs) that may exhibit structural features leading to non-identifiable dynam- ics. To illustrate this, we examined the gold-standard E. coli network from Marbach et al. (012) (Supplementary Data 1). The adjacency matrix derived from this network contains several zero columns, indicating genes with outgoing regulatory edges. This can be checked by analysing the TREAM5_Netwos www.wookkwfffffcccoooooo www.feeenneeeolldstteeee SooldStaddare Net.orrrddeetorrrrrrrrro tav data. As discussed in our main text, columns in the adjacency matrix key source of non-identifiability. A second example provided by Marbach et al. (2012), the S. cerevisiae network (Network 4), also contains zero columns and thus similar issues.
For some other networks, such as P. trichocarpa and A. thaliana, we were not able to obtain the full gold-standard network, and definitive identifiability analysis is not possible. However,
18

---

Published as a conference paper at ICLR 2026
Table 1: Summary real-world gene regulatory networks and their sparsity levels. The sparsity threshold corresponds to log from Lemma 3.
Name
# Nodes (n)
Edges
Sparsity (p)
Threshold (1 log
A. thaliana
2,864
18,663
0.9977
0.9972
P. trichocarpa
1,690
9,268
0.9968
0.9957
summarize in Table I the available statistics from Walker et al. (2022). including the number of nodes, number edges, sparsity level, and the theoretical sparsity threshold from Lemma 2 of our work.
In both cases, the sparsity level lies below the identifiability threshold, suggesting high probability of non-identifiabiiity. More broadly, the key point is that identifiability cannot be assessed purely from data. For any given system, even if it is theoretically identifiable, one cannot determine from observed data alone whether the inferred model corresponds to the ground truth. When fitting linear ODE model to data from sparse system, may recover model that exactly reproduces observations, yet may be structurally incorrect. Thus, regardless of whether the true system is globally identifiable, it remains unidentifiable from data alone.
Moreover, the datasets discussed above should not be viewed as perfect ground truths. noted by Walker et al. (2022) see $2.5). these "gold standard" networks incomplete and potentially inaccurate. As such, both the exact sparsity patterns (E. coli) and sparsity levels (P. trichocarpa) may be unreliable, and conclusions about identifiability must be treated cautiously.
D EXPERIMENTAL DETAILS
D.1 SOFTWARE
We provide the resources with the corresponding licenses used in this work in Table 2
Table Overview of resources used in our work.
Name
Reference
License
Python
van Rossum & Drake
2009
PSF License
PyTorch
Paszke et al
2019)
BSD-style license
Numpy
Harss et al.
2020
BSD-style license
Pandas
pandas development team
2020
Wes McKinney 2010
BSD-style license
Matplotlib
Hunter 2007)
modified PSF (BSD compatible)
Scikit-learn
[Pedregosa al.
2011
BSD 3-Daala
SciPy
Virtanen et al.
2020)
BSD 3-Clause
SLURM
YOO al. [0033
modihed GNU GPL v2
networkx
Hagberg et al.
2008
3-Cluuse
JAX
BBradbury
2018
Apache-2.0
D.2 METRICS
System-level identifiability metrics. To compute system level identifiability metrics, we perform batched singular-value decomposition on every system matrix A using jax.numpy numpy linallg...... svd. Subsequently, any singular value with |a| < 10-6 is treated as numerically zero. Eigenval- ues are computed with jax. numpy. finall g ei digvall and the matrix rank is computed via jax. numpy. linalgg matrix rank with tolerance level set to 1
Trajectory-Level identifiability metrics. Smoothed condition number (SCN) analysis begins by constructing the empirical Gram matrix E5z = YSY E Raxd where Y = [axtt)... (r(tz)....{t,,)) collects one simulated trajectory and where the diagonal "smoothing" matrix S contains the numerical quadrature weights (trapezoidal rule by default jax. numpy. trapz). The condition number is estimated with jax, numpy. lina dinill.c... cond.
19

---

Published as a conference paper at ICLR 2026
Normalized Hamming distance. We use the normalized Hamming distance computed on binary input matrices compare the true system matrix A with the empirically estimated matrix A. To this end we first binarize A and Å via B II (A) and B (A) with threshold T -10 and where Il, is the indicator function that acts elementwise on matrix entries as
I (aaa)) = =(0
if as > Ti
else
The normalized Hamming distance between two matrices B, B R is then defined as dHMD [B, B) = + Eis (R,, * B.y) where By * B, has to be understood as a boolean comparison which evaluates to one under inequality and zero otherwise.
D.3 EMPIRICAL ESTIMATORS
SINDy. SINDy, short for Sparse Identification of Nonlinear Dynamics (Brunton et al. 2016). is widely adopted algorithm for system identification. It leverages a user-defined set of basis functions to execute Iz-regularized linear regression, mapping observations of the solution trajectory x(t:) to their corresponding temporal derivatives x(()). In practical applications, temporal derivatives are often unobservable, and SINDy estimates them through numerical finite difference approximations. We adopt the implementation available in S INDD (de Silva et al. (020), restrict the basis set to linear functions, and use the default optimization algorithm (sequentially thresholded least squares) which sets any coefficient whose magnitude falls below the user-defined threshold À to zero. Model and optimizer come with several hyper-pane syper-parameters out of which we tune the T-g-regularization strength (a). coefficient threshold (A), finite difference approximation order and maximum number of iterations separately for each sample.
Regularization of SINDY. For each trajectory we select the pruning threshold À E {10-6, - 10-1} that enforces the sparsity gate: after every ridge-regression step any coefficient whose magnitude falls below À is hard-set to zero, so increasing À enforces progressively sparser candidate systems. Complementing this, the ridge weight a E (0.001, 0.05, 0.13 continuously shrinks the surviving coefficients toward the origin: larger values thus promote numerical stability without directly changing the zero pattern. For every trajectory, we select the optimal parameters (X,a) argmax R where the R² score is measured between the observed trajectory and the trajec- tory obtained by numerically solving the system estimate Å for the observed initial value. Finally, the identifiability analysis is based on systems with well-fitted trajectories only, which we define as trajectories for which the estimate achieves R² > 0.99 and MSE < 10-1 This regularization sets any coefficient that falls below X threshold to zero-hence aggressively promoting sparsity, which might lead to problems in low dimensional settings, see Fig. 4 In cases of very high sparsity and low system dimensionality, most coefficients of the true system matrix will be zero. In this case thresholding coefficients to zero biases the model towards a smaller Hamming distance. As the dimensionality increases the sparsity reduces, this effect vanishes as there multiple non-zero coefficients.
Neural ODEs. Neural ODEs (NODEs) (Chen et al. 2018) parameterized function fo()(() to approximate the dynamics underlying the observed trajectories. Instead of using finite difference schemes to estimate temporal derivatives, NODEs numerically integrate fo obtain a solution that can be directly compared to the observed trajectory. In practice fa is implemented as a neural network; since we focus on linear systems, we use model with multiple linear layers and no activation functions. To promote sparsity, we incorporate an LI regularization term into the loss function. The total loss consists of the mean squared error (MSE) of the trajectories, augmented by the regularization parameter À multiplied by the I.1 norm of the network's 's weights. To optimize the model we use the ode solvers implemented in torchode (Lienen & Günnemann 2022), specifically the Dopri5 solver in combination with the with relative and absolute tolerances set to 1c-3 and 1e-6, respectively. Neural network parameters 0 are optimized with PyTorch's RMSprop optimizer with learning rate of 1e-3 to minimize the mean absolute error between the observed and predicted solution trajectory as in Chen et al. (2018). Optimization proceeds for 10000 iterations or until the loss falls below 10-5
for adaptive step size selection,
of Neural ODEs. For the LinearNODE experiments we first identify well- fitted trajectories which we define as trajectories for which the empirical estimate (after numerical
20

---

Published as a conference paper at ICLR 2026
integration from the ground truth initial value) achieves R² > 0.99 MSE 10-4. The proportion of well-fitted trajectories (among all trajectories) for different system dimensionalities 72 and sparsity
levels p is displayed for different values of regularization weight À E {0, 10 10 2) in Fig. 5 Among the X-values that yield well-fitted trajectories, we then select the one that most faithfully reproduces the sparsity pattern of the ground-truth system matrix A. Specifically, for every trajectory we count the zero entries in the matrix estimate Å and in the true A, compute the absolute difference in these counts, and use this sparsity-mismatch mismatch score as our selection metric. The optimal for given p is selected the value that minimizes the average sparsity-mismatee -mimmatch across all well-fitted trajectories. The effect of regularization weight X, system dimensionality n and sparsity on the sparsity sparsity-mismatee mismatch between model estimate A and ground truth system matrix A is illustrated in Fig. 6 Our arguably very permissive model selection strategy reflects the idea that we are only interested in well-fitted models (as measured on the trajectory-leve)) in order to draw conclusions about (empirical) system identifiability rather than about optimization. model architecture or numerical issues.
0.8
0.8
0.8
0.6
0.6
0.6
prspaccia
0.0
0.0
0.0
0.5
0.6
0.7
1.0
20
1.0
30
1.0
0.0
0.8
0.88
0.001
0.4
0.4
0.1
0.2
0.2
0.2
0.0
0.1 0.2
O.1
0.4
0.5 0.8 0.7 0.8
0.90099
0.0
0.1
0.2
0.3
0.4
0.5.
0.5
0.7
0.8
0.90.95
0.0
0.1 0.2
0.3
0.S
D.8
0.7 0.B U.S.UM
Figure 5: Proportion of trajectories that have been well-reconstruutte by Sparse Neural ODEs for different regularization parameters À and different dimensions n and sparsity levels p. For sparse systems, the model recovers only a smaller fraction of the trajectories.
D.4 ADDITIONAL RESULTS ON TRAJECTORY-LEVEI IDENTIFIABIITYY METRICS
We extend the results on trajectory-level identifiability metrics a and SCN to broad range of system dimensions n and sparsity levels Box-plots for the two subgroups Az2.m. and Agz.ux introduced in Section a.e displayed in Fig. and Fig. 8 We observe the consistent trend that subgroup A Le., the subgroup with smaller second smallest singular value d2 leads to lower identifiability scores for both metrics (lower value for d and higher value for SCN) in line with theoretical results.
21

---

Published as a conference paper at ICLR 2026
10
relotvv diffecnnce sparrity
differnce appareltyy
00.8
02.6
00.8
0.4
0.001
90.95
0.0
0.10.2
0.90.55
20
30
40
Welstwe BlSEeossee spartty
0.8
00.0
0.6
0..6
0.001
0.01
0.3 0.6 00.90.55
Figure 6: Relative difference in sparsity count (lower the better) between the true and reconstructed system matrices using Sparse Neural ODEs for different regularization parameters À and different dimensions and sparsity levels Lower regularization recovers dense matrices better, while higher values suit sparse ones.
5
10
0.4
0.4
0.79
n. 20
= 30
40
0.65
0.50
6.50
0.60
0.76
0.50
Figure 7: Box-plots of distarce---uumidnniiiiilll da for the least and most identifiable groups of systems for different p and n values. Trajectories generated with Ag, lead to smaller dA than those produced with Agg.a.
the distance subspace of dimension n from random vector on the unit sphere. We now empirically validate the glose-t-uunidentifabblle an initial condition Zg and the kernel of the corresponding matrix A. Since we sample the initial conditions uniformly from the unit sphere, theoretical expected distance between a random unit vector and a dg da-dimensional subspace of IR... We partition the trajectories by the null-space dimension do dim(ker(A)) of their corresponding generating matrix A and display the resulting the (mean) empirical measure closely matches the theoretical expected distance between a random unit vector æo and do do-dimensional subspace of IR" (Vershynin 2018), given by
metric d'A which measures the distance between
compare the empirical distribution of d to the
d in Fig. 9. As expected,
r(n/2)I((n do 1)/2)
E ((d) nz, do]
I((n do)/2)l((n 1)/2)
hence (in expectation) validating the computation of disance--o-mmideeiffabllyyy dA.
22

---

Published as a conference paper at ICLR 2026
n 5
n. 10
TegrScos
LegiSCSS
30
0.00
0.79
0.80
0.0
0.00
0.0
0.30
0.00
0.70
0.00
0.83
0.00
20
30
1.=000
4.2
$7.5
33.0
LegiSoN0
27.5
0.50
10.00
18.70
0.80
01.03
0.00
0.70
0.80
0.85
0.80
0.50
16.00
0.50
0.80
01.33
0.00
Figure Box-plots of smoothed condition numbers (SCN) in log-scale for the least and most identifiable groups of systems for different p and n. values. Trajectories generated with A Aggaai lead to higher SCN than those produced with A Arg.amx
10
1.0
0.0
0.8
0.8
0.6
0.4
0.4
0.4
0.2
0.0
0.0
0.0
do
do
do
20
30
40
0.8
0.8
0.8
0.6
0.6
0.4
0.2
0.2
2
10
12
14
16
18
20
3
9
12
15
10
21
24
27
4
8
12
16
20
24
28
34
do
Figure 9: Box-plots of coaanncett-umidentiiiaabll da different n and different dimensions of ker(A)
(dg dim(ker( (A))) together with the expected distance TEdd (3r)) no, (black line).
D.5 ADDITIONAL EXPERIMENTS
A consequence of our findings in Lemma 4 is that the trajectories generated by two candidate matrices A and A' remain indistinguishable over longer time horizons when the initial condition lies closer to an invariant subspace. Here examine how this behavior manifests in the perfor- mance of empirical estimators. To this end, we consider multiple configurations (p, dim), where p {0.1.0 0.3, 0.7, 0.8, 0.9, 0.95) and dim (3. 5, 10, 20). For each configuration we sample 10 matrices following the same data generation procedure as Section 5.3 We then generate four initial conditions at varying distances d = (0.1, 1} from the corresponding invariant subspace to assess how the estimators' performances change with increasing d As described in (Section D. we restrict the performance evaluation to well-fitted trajectories in order to investigate the effect of the distance on identifiability rather than model optimization properties. In Fig. 10, we display the Hamming distance and mean squared error (MSE) of the estimated matrices compared to the ground truth one. In line with theory, the performance of both the Neural ODE as well as SINDy scales inversely with the distance to the invariant subspace, i.c., the smaller the distance d the larger the estimation emror.
23

---

Published as a conference paper at ICLR 2026
A second factor that Lemma predicts to affect empirical performance is the length of the observation interval: since trajectories remain indistinguishable up to time horizon T, reducing the observation window should degrade estimator performance. To investigate this aspect empirically, same data from the previous experiment but restrict input to the estimator to only a fraction of the previous observation window. Our results in Fig. 11 show that indeed, predicted, performance both in terms of Hamming distance as well as MSE decreases in almost all cases as the observation interval reduces. The exception to this observation occurs the smallest distance to the invariant subspace, d 0.1, where the performance remains constant across different trajectory lengths. As described before, d 0. also leads the largest errors overall, hence in this case the initial value is presumably already so close to the invariant subspace that unidentifiability even occurs the longest observation window.
the
Hamming Distance
04
MSE Distance
03
0.2
0.1
1.0
0.1
4A
A
0.5
Hamming Distance
MSE Distance
0.3
0.1
1.0
AA
A
Figure 10: MSE and Hamming distance between the ground truth matrix and the matrix estimated from Neural ODE (top) and SINDy (bottom). The estimators' performance degrades with lower values of d'-
10
0.
08
30"
Homming
022
Taw 2rava
10
0.5
4
4,
Figure 11: Hamming (left) and MSE (right) distance between the ground truth matrix and the matrix estimated from Neural ODE on trajectories spanning different time intervals.
24

---

Published as a conference paper at ICLR 2026
D.6 EXTENSION TO FURTHER MATRIX MODELS
We focus on further random matrix models and present here the results.
Fixed number of zeros per ensemble. In this random matrix model fix the number of zeros per row such that each row contains exactly d(n)= = _[mp] zeros. The remaining non-zero coefficients
add [(0, 1). We generate 100 system matrices and solve each of the for 100
are sampled i..d. as aij initial values which are sampled uniformly at random from the unit circle in R". Subsequently, carry out the system-level identifiability and empirical identifiability analyses, following the same procedures described in the main text.
Results for both analyses are in line with the results reported in the main paper: system-level unidentifiability shows are sharp increase sparsity increases (Fig. 12) and empirical identifiability shows a clear left-right gradient in Hamming distance for both SINDy Fig.. 15) and NODE (Fig. 16 confirming the expected rise in unidentifiability as sparsity increases.
Sparse-Conte Continuous ensemble with no zero rows or columns. In this random matrix model we explicitly exclude matrices with zero rows or zero columns. For this, matrix entries are generated
dt Sky big. where bij idd Ber(p) and Itü matrices, however, not all dimensionality n and sparsity p combinations permit any system matrices with zero rows or columns. (E.g. a 3 X 3 matrix with sparsity level 0.9 will have ss 8 zeros and hence always multiple zero rows and/or columns.) We hence cap the number of attempts to generate single matrix that fulfills the zero-rows zero-columns constraints at 100 attempts. For every valid generated system matrix, we sample 100 initial values randomly from the unit circle in I and numerically solve the initial value problem obtain 100 trajectories per system.
sid (0, 1). We again attempt to generate 100 system
We perform system-level identifiability analysis as well as an empirical identifiability analysis. System-level metrics are provided in Fig. 13] coamming-distancee for different system dimensions and sparsity levels are reported in Fig. for SINDy and in Fig. 16 for NODEs. For both estimators the average Hamming distance increases as sparsity rises, confirming the trends observed for other matrix models as well as the theoretical underpinnings.
Rank(A-Al)<n-)
Zero Eigenvalue
Pappoott noo smidenti Eafbblic Anstemes
50
50
50
In(n)/n
0,
40
40
n
30
20
20
20
10
10
0.2
0.5
0.8
sparsity p
sparsity p
sparsity p
Figure 12: Proportion matrices satisfying the conditions conditions rank(A AI) < n 1 (left), YE R rank(A- AI) < (center), and presence of zero eigenvalues (right) at different system dimensions n and sparsity levels for fixed number of zeros per row ensemble.
25

---

Published as a conference paper at ICLR 2026
Ran
Rank(A-Al))a-11
Zero Eigenvalue
bibbbbebbbeee spsterm
p 1 In(n)
p=1- In(n)]n
p In(n)/n
40
40
40
n
20
20
20
0.2
0.5
0.8
0.9
0.95
0.2
0.5
0.8
0.9
0.95
0.2
0.5
0.8
0.9
0.95
sparsity p
sparsity
sparsity
Figure 13: Proportion of matrices satisfying the conditions conditions rank(A - XI) 1 (left),
YY R rank(A XI) < n (center), and presence of zero eigenvalues (right) at different system dimensions and sparsity levels p for sparse-continoous ensemble with zero
Rank(A)<a-1
Rank(A-Al)<n-1
Zero Eigenvalue
Broportion of Indenninable Systems
p
In(n]/n
p In(n)in
40
40
20
10
0.2
0.8
0.9
0.95
0.2
0.5
0.8
0.9
0.95
0.2
0.5
0.8
0.9
0.95
sparsity p
sparsity
sparsity p
Figure 14: Proportion of matrices satisfying the conditions conditions rank - AI) < n- (left), YE RR system dimensions and sparsity levels for sparse-continnous ensemble with no zero columns.
rank(A- Al) <
(center), and presence of zero eigenvalues (right) at different
fixed number zeros
no zero columns
no zero rows
0.6
0.4
5A
0,2
sparsity
sparsity p
sparsity p
Figure 15: Hamming distance for different generating settings for SINDy on trajectories generated from fixed number of zeros row ensemble (left), sparse-continuous ensemble with no zero columns (center), sparse-continuous ensemble with no zero rows (right).
26

---

Published as a conference paper at ICLR 2026
fixed number zeros
zero columns
0.2
920000
sparsily
sparsily p
sparsily P
Figure 16: Hamming distance for different generating settings for NODE on trajectories generated from fixed number of zeros per row ensemble (left), cparse-continuous ensemble with no zero columns (center), sparse-continuous ensemble with no zero rows (right).
27


Reviews:
[{"review_id": "yrxquSBJlP", "reviewer": "Reviewer_i8tC", "summary": "The authors prove an identifiability result for a class of linear dynamics governed by a sparse-continuous random matrix. I went through the proofs in detail and they are reasonable, although this is outside my personal area of research so I can just say that the techniques and arguments seem reasonable. My only feedback is whether the authors can better motivate this class of sparse-continuous matrices and in what way they are relevant to machine learning. Of course identifying dynamics from time series is of interest, but a bit more discussion about why one should care about this is necessary.", "questions": "no further questions", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Seems like a correct theoretical paper.", "weaknesses": "See  my response to the summary - the relevance of this class of matrices needs to be better motivated. I couldn't tell whether this is an impactful result and the authors chose this problem because its important, or if the authors just chose a class of matrices for which they knew how to prove something. if they can help me understand that better i will raise the score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "odP21LECEl", "reviewer": "Reviewer_h45Z", "summary": "The paper presents a theoretical study of the identifiability of autonomous, linear and noise-free ODEs from a single trajectory, focusing on systems where the drift matrix is *sparse*. The authors distinguish between two notions of unidentifiability: (i) system-level (or global) unidentifiability, where the system is unidentifiable regardless of the initial condition, and (ii) trajectory-level unidentifiability, which occurs only for initial states lying in invariant subspaces of the system. \n\n*For system-level unidentifiability*, they show that in sparse systems it is implied by rank deficiency (i.e., zero-eigenvalue degeneracy, Lemma 1), derive a lower bound on its probability via the occurrence of system matrices with two zero columns (Lemma 2), and extend the analysis (Lemma 3) using random graph theory to identify a dimensionality-dependent sparsity threshold governing unidentifiability. \n\n*For trajectory-level unidentifiability*, they prove that the probability of “unlucky” initial conditions lying in invariant subspaces is zero, so identifiability depends solely on the system-level property. They further analyze near-invariant initial conditions, introducing a notion of distance to invariant subspaces, and showing that the indistinguishability time between trajectories of two systems agreeing on such a subspace increases inversely with this distance (Lemma 4). \n\nTheoretical findings are finally supported by numerical experiments, which rely on neural network (Neural ODE) and symbolic regression (SINDy) methods.", "questions": "1. Why did you title Section 3 “Global Unidentifiability” instead of “System-level Unidentifiability”?\nPut differently, why introduce the term system-level unidentifiability later in Section 3 rather than consistently using global unidentifiability throughout? This shift in terminology is somewhat confusing.\n\n2. In Appendix B, you discuss the sparsity of real-world gene regulatory networks.\nWhat are the node degree statistics of these networks, and can they be reproduced by your sparse–continuous random matrix model? If not, how might structured or heavy-tailed degree distributions affect your theoretical results?\n\n3. Within your setup, is there a way to study how the closeness of an initial condition to an invariant subspace influences the performance of Neural ODE and SINDy? In particular, does the length of the observed trajectory or the inter-observation interval play a significant role in this regime, as suggested by Lemma 4?\n\n4. In Figure 4, SINDy appears to match the true sparsity pattern almost perfectly for the 3-dimensional case across all sparsity levels.\nHow should these empirical findings be interpreted in light of your sharp sparsity threshold for global unidentifiability, and the corresponding empirical verification in Fig. 2? In other words, why does SINDy succeed in this low-dimensional setting even in regimes predicted to be unidentifiable?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper provides strong context through a clear discussion of related work, assumptions, and limitations. This situates the contribution well within the system identification and machine learning communities and makes its relevance easy to grasp.\n2. The work extends classical results on the unidentifiability of linear systems to the practically important case of sparse systems, filling a notable gap in the existing theory.\n3. The proofs are well structured and technically interesting. \n4. The experimental section is well designed and directly supports the theoretical claims. The authors also apply two widely used system-identification methods (Neural ODE and SINDy) to test the practical implications of their results. The exposition of these empirical findings is clear and coherent (albeit limited, see Weekness 2 below).\n5. The appendix is comprehensive, providing detailed proofs, additional empirical results, and clear information for reproducibility.\n\nOverall, the paper is well written, technically sound, and contributes meaningful theoretical insight into the identifiability of sparse linear systems.", "weaknesses": "1. The most immediate limitation lies in the restriction to linear and noise-free systems. While the authors are transparent about this assumption, it considerably narrows the scope of applicability to real-world or data-driven settings.\n\n2. The results on empirical unidentifiability (Section 5.3) are interesting but somewhat limited. The authors do not explicitly connect their theoretical findings on trajectory-level unidentifiability with their empirical unidentifiability results. For example, it would be interesting to examine how the two system-identification methods considered behave for initial conditions close to invariant subspaces, especially since the trajectory length should influence distinguishability in that regime (as suggested by Lemma 4).\n\n3. While the use of a Bernoulli mask to model sparsity is theoretically convenient, the authors neither discuss nor acknowledge (or at least I don't find such a discussion/acknowledgement) that real-world sparse networks often exhibit \"structured sparsity\" (e.g., hub nodes or community structure). The Bernoulli mask (i.e. Erdős–Rényi) model does not capture such degree statistics, which may limit the generality of the conclusions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8wHziOaRxx", "reviewer": "Reviewer_7pZ9", "summary": "This paper revisits the identifiability of linear ODE systems $dx(t)/dt = Ax(t)$ when the system matrix $A$ is sparse. The authors show that, unlike the dense case where almost all systems are identifiable from a single trajectory, sparsity introduces a positive probability of unidentifiability. They define a sparse–continuous ensemble and prove a sharp phase transition: when sparsity exceeds roughly $1-ln n /n$, systems become globally unidentifiable with high probability. The paper also introduces a trajectory-level metric to quantify how close a trajectory is to being unidentifiable. Simulations confirm that identifiability deteriorates with higher sparsity, both theoretically and in practice, using SINDy and Neural ODE estimators", "questions": "1. In Fig. 4, for small dimensions (e.g., $n=3,5$) SINDy sometimes achieves low normalized Hamming distance even at very high sparsity. Could you comment on this regime?\n2. In Section 2 (“Discussion of assumptions and limitations”), you might consider citing a related work on identifiability under hidden confounders: Wang et al. (2024) Identifiability analysis of linear ODE systems with hidden confounders.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Provides a clear theoretical characterization of when sparse linear ODEs lose identifiability.\n2. Decomposing failure probability into global vs. trajectory unidentifiability (Eq. 2) is conceptually clarifying, and the distance $d_A(x_0)$ is a helpful practical proxy.\n3. The sharp threshold result is elegant and connects to known results in random matrix theory.\n4. Writing is well structured, and assumptions are transparently stated.", "weaknesses": "1. The strong assumptions (noise-free, single continuous trajectory, full observability) limit real-world applicability.\n2. For readers less familiar with random matrix theory, it would be helpful to include a brief intuition box or paragraph below Lemma 3 explaining why the identifiability transition occurs precisely at $1-ln n/n$. A short, high-level explanation would make this elegant result more accessible and highlight its connection to classic random graph thresholds.\n3. In the main text, the normalized Hamming distance is described as divided by $n$, but Appendix C.2 defines it as normalized by $n^2$. Please clarify which version was used, and make it consistent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)