Identifiability Analysis of Linear ODE Systems with Hidden Confounders
Authors: Yuanyuan Wang
Biwei Huang
Wei Huang
Xi Geng
Mingming Gong
Venue: NeurIPS 2024 poster (2024)
Keywords: Linear ODEs
Identifiability analysis
Hidden confounders
Causality
Primary area: causal_inference

Abstract: The identifiability analysis of linear Ordinary Differential Equation (ODE) systems is a necessary prerequisite for making reliable causal inferences about these systems. While identifiability has been well studied in scenarios where the system is fully observable, the conditions for identifiability remain unexplored when latent variables interact with the system. This paper aims to address this gap by presenting a systematic analysis of identifiability in linear ODE systems incorporating hidden confounders. Specifically, we investigate two cases of such systems. In the first case, latent confounders exhibit no causal relationships, yet their evolution adheres to specific functional forms, such as polynomial functions of time $t$. Subsequently, we extend this analysis to encompass scenarios where hidden confounders exhibit causal dependencies, with the causal structure of latent variables described by a Directed Acyclic Graph (DAG). The second case represents a more intricate variation of the first case, prompting a more comprehensive identifiability analysis. Accordingly, we conduct detailed identifiability analyses of the second system under various observation conditions, including both continuous and discrete observations from single or multiple trajectories. To validate our theoretical results, we perform a series of simulations, which support and substantiate our findings.

Identifiability Analysis of Linear ODE Systems with Hidden Confounders
Yuanyuan Wang
Biwei Huang
The University of Melbourne yuanyuan20stuudent. unimelb. edu.
University of California, San Diego bih007@ucsd. edu
Wei Huang
Geng The University of Melbourne xi geng@unimelb edu au
The University of Melbourne wei. hwwnguunimelb. edu.au au
Mingming Gong The University of Melbourne
mingming gongdunimelb edu. au
Abstract
The identifiability analysis of linear Ordinary Differential Equation (ODE) systems is a necessary prerequisite for making reliable causal inferences about these sys- tems. While identifiability has been well studied in scenarios where the system is fully observable, the conditions for identifiability remain unexplored when latent variables interact with the system. This paper aims address this gap by presenting a systematic analysis of identifiability in linear ODE systems incorporating hidden confounders. Specifically, we investigate case, latent confounders exhibit no causal relationships, yet their evolution adheres specific functional forms, such polynomial functions of time Subsequently, we extend this analysis to encompass scenarios where hidden confounders exhibit causal dependencies, with the causal structure of latent variables described by a Directed Acyelic Graph (DAG). The second case represents more intricate variation of the first case, prompting a more comprehensive identifiability analysis. Accordingly, we conduct detailed identifiability analyses of the second system under various observation conditions, including both continuous and discrete ob- servations from single or multiple trajectories. To validate our theoretical results, we perform series of simulations, which support and substantiate our findings.
cases of such systems. In the first
Introduction
Understanding the dynamics of systems govemed by Ordinary Differential Equations (ODEs) is fundamental in various scientific disciplines, from physics (9 23 24 47]. biology [66 27 29
33 36)
to economics 13 38 39 43 These ODE systems provide natural framework for modeling causal relationships among system variables, enabling us to make reliable interpretations and interventions
25 30 31. Central to unraveling the causal mechanisms of such systems is the concept of identifia- bility analysis, which aims uncover conditions under which system parameters can be uniquely determined from error-free observations. Identifiability is crucial for ensuring reliable parameter estimates, thereby guaranteeing reliable causal inferences about the system [41. The motivation for our research the identifiability analysis of ODE systems arises from the necessity of making reliable causal inferences about these systems.
Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Our research focuses on the homogeneous linear ODE system, represented as:
(tt) Ax(t),
z(0) =
(I)
where [0, oo) denotes time, (+t) _d represents the system's state time 6, (+t) denotes the first derivative of æxt) w.r.t. time, and Zy represents the initial condition of the system. The solution (trajectory) of the system, denoted <t; Zu. A) for t [0, 20). single d' d-dimensional trajectory initialized with Zo.
Existing literature has extensively examined the identifiability of linear ODE systems under the assumption of complete observability, where all state variables are directly observable [5]
[4]
15 17
28
34
22
Specifically, researchers have investigated identifiability of the ODE system from
single whole trajectory [28] 34], and extended analysis to discrete observations sampled from the trajectory 42).
However, practical scenarios often entail systems with latent variables, rendering them not entirely observable. In this paper, we explore the identifiability analysis of this ODE system under latent confounders, particularly examining cases where no causal relationships exist from observable variables to latent variables, commonly assumed condition in causality analysis with
hidden variables [10, 20 22 44 45
In this paper, we focus on two scenarios:
1. Independent latent confounders: Latent variables exhibit no causal relationships among themselves, leading the following linear ODE system:
(t)
A
B
(x))) = + [(()
(x
(
0
0
( =
2. Causally related latent confounders: Latent variables exhibit causal relationships among themselves, specifically, they follow a DAG structure, represented as:
A
B
xtt) x(t))
(0)
xD
2
0
G
==())
2
In these two ODE systems, (t) E Rd denotes the state of observable variables (I1 T2. ... zd). while ztt) E R denotes the state of latent variables z = (21 22...., 2p). Example causal structures of these two ODE systems are illustrated in Figure T It is noteworthy that the structure may include cycles and self-loops within the observable variables. Additionally, two real-world examples are provided in Appendix B
21
2
Zp
Z1
Z2
Zp
X1
X²2
Xd
X1
X2
x
ODE system (2)
ODE system (3)
Figure Example causal structures of the ODE system and
This paper provides an identifiability analysis for the ODE system 2 under specific latent variable evolutions, such as polynomial functions of time t. Additionally, we conduct systematic identifiabil- ity analysis of the ODE system 3 when the causal structure of the latent variables can be described by a DAG.
Background
2.1 Causal interpretation of the ODE system
When an ODE system describes the underlying causal mechanisms governing dynamic system, provides a natural framework for modeling causal relationships among system variables. The causal

---

structure inherent in such systems can be directly read off [25] 31]. For instance, in the ODE system (T) where the ij-th entry of the parameter matrix A is denoted as A.j. the presence of A, * signifies that the derivative of (4) is influenced by (t). thus indicating causal link from z; to ry- Here, 24 denotes the i-th variable the ODE system (T), and x:(() represents its state time
does not explicitly depend on time 1, the causal t. Since the right hand side of the ODE system
[
structure of this ODE system is time-invariant.
An essential prerequisite for reliably inferring the causal structure and effects of an ODE system, for purposes of interpretation or intervention, is the identifiability analysis of such systems. To underscore this necessity, we provide an illustrative example. Consider the ODE system (3. Set
1
H
1 1 1
0 0
B
G
1
1
0
1
0
0
A
B
A'
B
A
A' =
M =
M'
G
G
Calculations reveal that the solutions (trajectory) of the ODE system or M' are identical, i.e.,
with parameter matrices M
xx( ((t)
e
M'
250
20
This indicates that using observations sampled from this trajectory to estimate parameter matrix M may end up yielding M', which exhibits a fundamentally distinct causal relationship between ø and zz, see Figure 2 This discrepancy in parameter estimation, wherein M' is obtained instead
Z1
Z2
Z1
Z2
X1
X2
X1
X2
M
M
Figure 2: Causal structures of the ODE system with parameter matrix M and M'.
of the true underlying parameter matrix M, may lead to misleading interpretations and causal inferences, potentially influencing decision-making particularly regarding interventions. For instance, intervention with x (4) 1, under the true underlying parameter matrix M, yields the trajectory
(t) 4e - t - 3 (post-intervention). whereas under matrix MM, the trajectory becomes 22 (t)
f2/2+ + +
Detailed calculations are provided in Appendix C
2.2
Identifiability analysis of the linear ODE system
The identifiability analysis of the ODE system 11 has been well studied. Here, we present fundamental definition and theorem essential for understanding identifiability the ODE system 1 Denoting its solution (1;; my A), it is noteworthy that the system is fully observable, without latent variables interacting with it. We present the identifiability definition and theorem as follows.
Definition 2.1. For Zo E IRd A E Raxd, the ODE system 1 is said to be (æo, A -)iieentif,able, if for all x/ E Rd and all A' E gddd with (xo, A) # (a/, A'), holds that x(:; No, A) ( T/y A')
Lemma 2.1. For 2a E Rdd A E Rdxd, the ODE system is (xo, A)-identifiable if and only if condition AO is satisfied.
AO the set of vectors (wo, Awo.... Ad-1 } is linearly independent.
<e(:; wo, A) = (x(t; A) 0 & << oo). this inequation means that there exists at least one > 0 such that (11; 20, A) * 21::266 A').
3

---

Definition and Theorem 2.1 are adapted from [42 Definition 1] and 42 Lemma2]. We use ag and A' to distinguish other system parameters from the true system parameters By and A: æ/ and A' can represent any d-dimensional initial conditions and any d x d parameter matrices, respectively. Here instead of describing a collective property of set of systems, we describe an intrinsic property of single system with parameters (Zo. A). In practice, the aim is to ascertain whether the true underlying system parameter (æo, A) is uniquely determined by observations. Hence, (æo, A)-identifiability offers a more intuitive description of the identifiability of the ODE system from a practical perspective.
From a geometric perspective, condition AO stated in Lemma 2.1 indicates that the initial condition 2p is not contained in an A-invariant proper subspace of R Intuitively, this means the trajectory of this system started from 2e spans the entire d-dimensional state space. That is, our observations cover information on all dimensions of the state space, thus rendering the identifiability of the system. Additionally, condition AO generic, as noted in [41], meaning that the set of system parameters violating this condition has Lebesgue measure zero. Thus, condition AO is satisfied for almost all combinations of Eg and A.
3 Identifiability analysis of the linear ODE system
In this section, we present the identifiability condition for the linear ODE system 2 the function ft) 2 as a specific function of time Here we first define ft) as a r-deeee polynomial function of time t, expressed as follows:
We consider
f(1) = vul
Uk IR..
(4)
k=0
Simple calculations show that
xtt) =
Uk
k + 20
k +
k=0
Thus,
g Bus x+1 + Bzg
(+t) Ax(() + Bx(t) Az(t)
(5)
k+1
We denote the unknown parameters of the ODE system 2 as O, specifically, (Fo, 2o, A, B, {v=))), where [vx]ū denotes all the v& for k rr. Let xx' (t: 0)) 0))"
denote the solution of the ODE system (2). It is important to note that under our hidden variables setting, only <t:: 0) observable. Based on Equation (5). we present the following identifiability definition.
Definition 3.1. For To E Rd. Rd, 20 E RP. A E Raxd E TRaxp and (uajō E RR, for all 26 Rd, all z11 E RP, all A' Raxd all B' gdxp, and all {wil5 E IR, we denote O'
(2)) 2 A', B', {x) we say the ODE system (2) is 0-identifiable: if (2o, A, Bzo, (Box)o) # (26) A', B'z 20, (B'ok))), it holds that w(:; 0) + x');0').
In the ODE system (2). where only variables æ observable, will, with some terminological leniency, refer to (=: 0) as the trajectory of the ODE system (2) with parameters 9. According to Definition 3.1 if the ODE system ( with a polynomial ftt is O-identifiable, then the trajectory of the system can uniquely determine the values of (Ia. A, Bzo. { {Bux 18). This determination is sufficient to identify the causal relationships between observable variables æ described by Equation (5) Consequently, one can safely intervene in the observable variables of the ODE system and make reliable causal inferences, despite the fact that matrix B cannot be identified under this definition.
Theorem 3.1. For E Rdd, zo E RP, A E Raxd, B E Rd P, {vx) E RP, the ODE system 2 is o-identifiable if and only if assumption Al is satisfied.
AI the set of vectors (3, AB....,., Ad-1 B} is linearly independent, where B = A (Ax, + Bzn) j(- Ju,, and j! denotes the factorial of j.
The proof of Theorem 3.1 can be found in Appendix D.I Condition A1 is both sufficient and
necessary, indicating, from geometric perspective, that the vector (3 is not contained in an A- invariant proper subspace of [ (34 Lemma 3.1].

---

The key point of the proof is the introduction of an augmented state y(t) = [x)(t), 1,,, 1111 with corresponding ODE system:
A
Bzo
Bu
Bir-
Bu, 1)
0₂
0
0
0
0
itt)
1
0
ytt),
:
(6)
0
0
denoted as F
(00) = [ 1, 0, 0J yo
where O2 is a d-dimensional zero row vector, and matrix F' E (2(++++))((d---2)) The ODE system (6) is a homogeneous linear ODE system analogous to ① but with fully observable variables y- In other words, we transform our system of interest, (2). which includes hidden confounders, into
fully observable ODE system (6). This allows us to leverage existing identifiability results for
homogeneous linear ODE systems, specifically Lemma 2.1 to derive the identifiability condition for the ODE system
Based on this approach, if the state of the hidden variables =(t), as determined by the function f()) in the ODE system (2) can be deseribed by some linear combinations of observable functions of time t, then the identifiability condition of the ODE system (2) can be derived. For an illustration, in the Appendix E we provide identifiability conditions for the ODE system (2) when f(t ve and f(t) U2 sin(() + Vycos(t). While we do not enumerate all functions ((t that meet this condition, our primary objective is to demonstrate a method for deriving the identifiability condition for the ODE (2) when the evolution of its hidden variables conforms to certain specific functions. Researchers can apply this approach find appropriate functions ftt according to their specific requirements.
4 Identifiability analysis of the linear ODE system
3
In this section, we extend the identifiability analysis to linear ODE systems with causally related latent confounders. Specifically, we assume that the causal structure of latent variables satisfies the following latent DAG assumption.
Latent DAG: the causal structure of latent variables can be described by DAG.
The DAG assumption is commonly employed in causality studies [I0 22 26 40
Under the latent DAG assumption, the matrix G can be permuted to be a strietly upper triangular matrix, i.c., an upper triangular matrix with zeros along the main diagonal [9]. Without loss of generality, we set G as strictly upper triangular matrix.
Since G is strictly upper triangular matrix, by the Cayley- Hamilton theorem matrix with an index & p. Consequently, G = 0 for all k 3 p.
35
G is nilpotent
Based on
34 37 the solution of ztt) can be expressed as:
60
G 20,k
p
Gh 20,k
x(t) = Gt 20
hd
k=0
k=0
Thus,
p-1
BG 20
+(t) = Ax(t) Bz(() Axx(t) +
We observe that Equation 7 has the same function form as Equation (5. but with different coeffi- cients (system parameters) for the polynomial of time Therefore, the ODE system B under the latent DAG assumption can be considered a more complex version of the ODE system (2) when f(1) follows a polynomial function related latent confounders, which is a more interesting and practical case, we will provide a more comprehensive identifiability analysis of the ODE system 3 The derived identifiability results can be easily generated to the case of the ODE system 2).
time L. Since the ODE system 3 incorporates causally
5

---

4.1
Identifiability condition from a single whole trajectory
We denote the unknown parameters of the ODE system (3) as 77, that is. 77 := (Zo, Zu. A, B, We
further denote the solution of the ODE system 3) as (a (t; m)) x (t:m) note that under our latent variables setting, only æ(t; m) is observable. Thus, based on Equation 77. we present the following identifiability definition.
Definition 4.1. For 2n E IRd, 20 E RR, E Rdxd B E Rdxp and G E TRXX, under the la- tent DAG assumption, for all z E Rd all # E RR, all A' Rddd all B' E &Rxx, and all
G' E RPXP, we denote n/ == (T zo, A', J', G'), we say the ODE system (3) is n-identifiable: if (zo, A, Bzg, BG BG 20) # (c/ A', B'zb, B'G "G, ax:: m) # 2':
z)). it holds that
Similar to the case of the ODE system 2). we refer to x(:;7) as the trajectory of the ODE system 33 with parameters 77 Definition 4. defines the identifiability of the ODE system (3) from single whole trajectory ()) Once the ODE system (3) is n-identifiable, the causal relationships among the observable variables æ can be determined through Equation (7). We then establish the condition for the identifiability of the ODE system
3 based on Definition 4.1
Theorem 4.1. For Ty E IRd, Za E IR., A Raxd B E Rddx and G E RPxP, under the latent DAG assumption, the ODE system (3 is 17 in-iddttifiable if and only if assumption B1 is satisfied.
Bl: the set of vectors
Ad-1y} linearly independent, where Y = AP +
Σj 18-1-111111 BG' zg-
The proof of Theorem 4.1 can be found in Appendix D.2
Condition B1 is both sufficient and necessary, and from geometric perspective, it indicates that the vector y is not contained in an A-invariant proper subspace of Id 34 Lemma 3.1].
4.2 Identifiability condition from discrete observations sampled from a single trajectory
In practice, we often have access only sequence of discrete observations sampled from trajectory rather than knowing the whole trajectory. Therefore, we also derive the identifiability conditions under the scenario where only discrete observations from a trajectory are available. Firstly, extend the identifiability definition of the ODE system (3) as follows.
Definition 4.2. For Do E R2,, 20 E RP, A E Rdxd E Raxp and G E RPxP. For any n. 3 1, let tj.j 1....,n be any time points and x; := æ(t;; n) be the error-free observation of the trajectory (0)5))) at time 1j- Under the latent DAG assumption, we say the ODE system 3 is n-- identifiable from æ1...., _m,, if for all xo6 E Rd all 2/1 E RR, all A' E Rdxd all B' E RRRxP, and all o (a/g) A', J, B' B/2 B'G' p' B'G 2%).
G' E RPxP with (mo, A, B2o, BG20. ...
holds that 3j E (1, (1.....,,, n such that (ty; n) (t;;)))
Definition 4.2 defines the identifiability of the ODE system (3) from n observations sampled from the trajectory n)) Then we establish the condition for the identifiability of the ODE system (3) from discrete observations based on Definition 4.2
Theorem 4.2. For Zo Rdd 20 RR,AE Raxd B E Rdxp and G E RPxP_ We define new
observation y To 1, t. .t t-- Rd+p, L...., Under the latent DAG
assumption, the ODE system (3) is n-ideetiifable from discrete observations æ1,... Eno if and only assumption CI is safisfied.
Cl: there exists (d p) yj 's with indices denoting as (11.j2.... ja+p3 S f1,2..... n2, such that the set of vectors {y y "............................... Wjatg } is linearly independent.
The proof of Theorem can be found in Appendix D.3 Condition C1 is both sufficient and necessary. This theorem states that as long as there are + p observations x 's such that the corresponding augmented new observations y 's are linearly independent, the ODE system (3) is te 1--dentiiiable from these discrete observations. Under the latent DAG assumption, we can transfer the ODE system (3). which includes hidden confounders, into a (d + dimeesionall fully observable ODE system (T) through the augmented state y(). Condition Ci indicates that our observations span the entire (d + p) -dimensional state space, thus rendering the system identifiable.

---

Both Definition 4.1 and Definition 4.2 define the identifiability of the ODE system 3 to some extent of the unknown parameters. In other words, given the available observations. under Definition [4.] and Definition 4.2 one can only identify the values of (æo, A, Bzo, BG2o,..- BG 20), but not the values of (2o, B, G). Based on Equation (7). this level of identifiability is sufficient to identify the causal relationships between observable variables æ. enabling safe intervention on the observable variables with reliable causal inferences. However, in scenarios where practitioners can intervene in the latent variables and require inferring the causal effects of the intervened system, identifying the matrices B and G becomes essential for reliable causal references. For instance, in chemical kinetics, where the evolution of chemical concentrations over time can often be modeled by ODE system [8, 12), some chemicals may not be measurable during the reaction, rendering them latent variables. Nonetheless, practitioners can intervene in these latent variables by setting specific initial concentrations. Therefore, we provide an identifiability analysis of the linear ODE system (3) when practitioners control the initial condition of the latent variables: 20-
4.3 Identifiability condition from p controllable whole trajectories
Assuming the initial condition of the latent variables zg is controllable, which means that the values of zg can be treated as given values, we denote it as z In the following, we provide the identifiability condition of the ODE system when we are given p initial conditions zÿ. denoting as 2 We first present the definition.
RR for i -,...., P. for Ea Rd A Raxd B Rdxd and G' E RexP, under the latent DAG assumption, for all a E R2, all A' E Raxdd all B' E Rdxp, and all G' E RexP, we denote n, == (xo, 20 A, B, G) and o := (a 20 A', A', we say the ODE system is (m.M ma.--iddetiffablee (xo, A, B, G) + (x' A', B', G"), J, it holds that Ei such that v)))
Definition 4.3. Given zō¹
((: * x
Definition 4.3 defines the identifiability of the ODE system 3 from whole trajectories x(::7,) with i ......P. and under this definition, matrix B and G are also identifiable. Based on this definition, we provide the identifiability condition.
Theorem 4.3. Given 2 E Rd for = 1..... PP, for Do E Rd, E Bddd B E Raxp and G E TRPx,, under the latent DAG assumption, the ODE system 33 is (n: -identiiiie if assumptions B2, B3 and Ba are all satisfied.
B2: each zū¹ for i I....,p. satisfies assumption BI. That is, if we set Y AFxo Σí AP-1-1 zūi then the set of vectors (76 AYo..., Ad-lay is linearly indepen- dent for all i 1, ..
B3: the set of vectors {zgl z0² 20 is linearly independent.
B4: the matrix composed by vertically stack the matrices {B, BG,.... BGP-1} has rank p.
The proof of Theorem 4.3 be found in Appendix Assumption B2 ensures that the ODE system (3) is n, identiiiiee for all i 1, I....,P. Consequently, (æo, A. Bz BG ..... BGG-1 20 for all I....,P is identifiable. Then, under assumption B3, the identifiability of matrix B is established. To identify matrix G. assumption B4 is required. While the ability to control the initial condition of the latent variables may appear strict, it is a reasonable assumption in our context, This is because identifying matrices B and G necessary only when practitioners can intervene in the latent variables, thereby allowing control over their initial conditions. An alternative approach to identifying B and G involves intervening in the initial condition of each latent variable z; independently, rather than controlling the initial condition of all latent variables simultaneously. This method draws inspiration from the "genetic single-node intervention" " proposed by (32), where one can intervene at each latent node individually. Further details of this method can be found in Appendix F
4.4 Identifiability condition from discrete observations sampled from controllable trajectories
We also extend the identifiability analysis of the ODE system B to cases where only discrete observations from p controllable trajectories are available.
Definition 4.4. Given 20 E TRP for i I,...,P.
, Ps for an RRd A E Raxd, B E Rdd and G E RPXP For any 2 1, let j = 1,..., n be any time points and Dij == æ(tzi m.) be the
7

---

error-free observation of the trajectory z.); at time ig. Under the latent DAG assimption, we say the ODE system 3 is {n, -idettideetiiiable from Ra...., Din i = ,,...,P. P, for all a's E , all A' E Rdddd all B' E Rdxp, and all G' E RPxP with (Zo, A, B, G) + (æ A', B', G'), it holds that Di (.) and j E El,....,7 such that (t)
Based on Definition 4.4 we present the identifiabilityy condition.
Theorem 4.4. Given z E IR for i = 1,...,P. for 2n E Rd A E Raxd B E RddD and G RPXP, We define new observation yı == [m ,, j,, 2.....,, 10-111 E RRdt,, gert,fori= fori=1..... and j = l....,, n. Under the latent DAG assumption, the ODE system discrete observations Fel..... Fins ........ .P, if asssumptions C2, B3 and B4 are all satisfied.
(3) is {n. H iffidentifiable from
C2: for each i E {L...,,p} there exists (d + p) Di 's with indexes denot- ing (in, : C (1,2,...,n). such that the set of vectors {y Vatua "........aaaa. :Diican)) is linearly independent.
The proof of Theorem can be found in Appendix [.5] Assumption C2 ensures that the ODE system (3) is -iiienntiiablle from discrete observations T41,.... .... ................................ Din for all I.....P. As in Subsection 4.3 under assumptions B3 and B4, the matrices B and G are also identifiable.
Simulations
To evaluate the validity of the identifiability conditions established in Section 3 and 4, we present the results of simulations. As previously indicated, the ODE system (3) can be treated as a more intricate centered the former.
version of the ODE system (2) : hence, our simulation experiments
Simulation design. We conduct four sets of simulations, which include one identifiable case and one unidentifiable case for both the 77-identifiable check and the {n, J -ddeetntiieable check. The dimensions of both observable variables, d, and latent variables, p, are set to 3. The true underlying parameters of the systems case, with n equally-spaced observations generated from the time interval [0, 1] for each trajectory, and we only keep the values of the observable variables æc
provided below. Observations simulated from the true ODE systems for each
2
-2
1
- -2
2
0
2
17
0
0
=
1
-1
=
0
-1
-2
G
-2
A'
1
0
2
-
-1
-2
O
0
0
0
0
1
--17
[1]
T
0
20
1
20
-2
20'
0
2
0
17-dentifiable: 77 (op, 20. A, B, G), unidentifiable: r (wo, 2o. A.
[n:} iidettiieitii 7. = (wo. 2 A, B, G), unidentifiable: n7; = (Io. ~ū B,B. 1,2,,3
Parameter estimation. The Nonlinear Least Squares (NLS) method is employed for parameter estimation, widely used technique for estimating parameters in nonlinear regression models, including ODEs 7 21 [4]. The "least_squares" sqquaes" function from the "scipy.optimize" Python module, with default hyperparameter settings, is utilized for implementation. Given that the NLS loss function for our simulation is non-convex, parameter initialization is performed near the true values to promote convergence
the global minimum. Specifically, for the 7 (-nuuiddettffiab cases, initial parameter values are set to the true parameters plus a random value drawn from a uniform distribution U-0.1, 0.1) for each replication. For {n. (unniddnntiie cases, initial parameter values are set to the true values plus a random value from U (/003, 0.3).
Evaluation metric. Mean Squared Error (MSE) is adopted as the metric assess the accuracy of the parameter estimator. To ensure the reliability of the estimation results, 100 independent random replications are run for each configuration, and we report the mean and variance of the squared error.
Results analysis. Table and present the simulation results for the 7 (un)iddnnttiee cases and the (n, (n.) (un))ddenttialll cases, respectively. According to Definition 4.1 and Definition 4.3 for the (un(ddddeettiiaall cases, the identifiability of (aro, A, Bzg, BGz,, BG*zo) needs to be checked, while for the {n. ff. (uddiidiiiifiiblee cases, we need to check the identifiability of (æo, A, B, G).
8

---

Since Zo is consistently identifiable (with MSE less than 1.00E-10) across all (unjidentifiable cases, its results are not presented.
In both Tables, for identifiable cases, as the number of samples n increases, the MSEs for all parameters of interest decrease and approach zero. However, in the unidenfiable cases, where the identifiability condition B1/B2 stated in Theorem 4..3 is unmet, the MSEs for certain parameters remain high irrespective of sample size. These results offer strong empirical support for the validity of the identifiability conditions outlined in Theorem 4.1 and Theorem 4.3 It is noteworthy that in the { case, where observations are sampled from p = 3 controllable trajectories, remarkably accurate parameter estimates can be obtained even with limited number of samples.
Table I: MSEs of the (nnddddddetii cases of the ODE
Identifiable
Unidentifiable
A
Bzo
BGzo
BG²z0
A
Bzo
BGzo
BG
6.00E-05
0.0004
0.0044
0.0007
0.0994
0.0494
0,9185
0.6482
10
(:.5.40E-0))
(1:345E:-6)
(1:0.0004)
(133918:06)
(1.0.015))
(1:01144))
(18.3148)
(1:14306)
4.15E-05
0.0003
0.0029
0.0005
0.0372
0.0174
0.3517
0,5767
100
(d: 1.62--88)
(12.5524)7
(1:942E-05)
(4:2.901-06)
(4000003))
(4:0.0087)
(2003460)
(d:14055)
2.65E-05
0.0002
0.0019
0.0002
0.0461
0.1071
0.5783
0.3648
500
(#88 (1F-71
(#4438F-07)
(4:484FF-05)
(188.18F007)
(#:0.0099)
(4:0.1768)
(#2.574
(4:04507)
Table 2: MSEs of the H (un)iddnnnnet cases of the ODE
Identifiable
Unidentifiable
n
A
B
G
A
B
G
5.83E-22
2.85E-21
2.27E-21
0.6349
0.1913
0.0044
10
(17.41E-42)
(12.75E-40)
(1.5699B-1)
(1.0.7464)
(L00.0686)
(10.0011)
1,50E-22
7.80E-22
5.76E-22
0.6169
0.1850
0.0045
30
(+3.23E-43)
(+1.14B-41)
(+5.28B-42)
(++07199)
(+:0.6577)
(+0.0007)
5.16E-23
3.01E-22
2.39H-22
0.5876
0.1761
0.0045
50
(+6.20FF44)
(+3.277-42)
(+8.466)43)
(+-06895)
(+0006277)
(+:0.0088)
For the n-nunnidentiffable cases, assumption C1 stated in Theorem 4.2 holds true for all values of n in the identifiable cases, while it is violated across n the unidentifiable cases. In the (m,)f-(um lunieeiiiieee cases, condition C2 stated in Theorem (.4 is satisfied for all values of n in the identifiable cases, but is found be violated for all values in the unidentifiable cases. These findings provide strong empirical evidence supporting the validity of the identifiability conditions proposed in Theorem 4.2 and Theorem 4.4
In Appendix G we present additional simulation results for higher-dimensional cases, along with simulations that incorporate a variety of ground-truth parameter configurations. These results consistently affirm the validity of our proposed identifiability conditions. For further details, please refer Appendix G
6 Related work
Identifiability analysis of linear ODE systems. Within control theory, extensive research has been conducted on the identifiability analysis of linear dynamical systems governed by ODEs (5) 14 17]
In the applied mathematics area, Stanhope et al. 33 and Qiu et al. (8 have systematically investigated the identifiability of linear ODE systems based on a single trajectory. Furthermore, Wang et al. 42 have extended these findings to scenarios where only discrete observations sampled

---

from single trajectory are available. However, existing studies primarily concentrate on linear ODE systems with fully observable variables. To the best of our knowledge, our work represents the inaugural endeavor systematically analyze the identifiability of linear ODE systems in the presence of hidden confounders.
Connection between causality and differential equations. Differential equations provide a nat- ural framework for understanding causality within dynamic systems, particularly in the context of continuous-time processes [ 31]. Consequently, significant efforts have been directed towards establishing theoretical link between causality and differential equations. In the deterministic case, Mooij et al. [25] and Rubenstein et al. [30 have established a mathematical connection between ODEs and Structural Causal Models (SCMs). Wang et al. [42] have proposed a method for inferring the causal structure of linear ODEs. In the domain of neural ODEs, Aliee et al. [2] 3 have applied various regularization techniques to enhance the recovery of the causal relationships. Turning the
stochastic case, Hansen et al. [B] and Wang et al. [4] have proposed causal interpretations and identifiability analysis of Stochastic Differential Equations (SDEs). Additionally, Bellot al. [6] have introduced method for consistently discovering the causal structure of SDE systems using penalized neural ODEs. These works aim to establish a theoretical connection between causality and differential equations in various ways. Our contribution to this scholarly landscape lies in the systematic analysis of the identifiability of linear ODEs, particularly in the presence of hidden confounders.
Conclusion
This paper presents a systematic identifiability analysis of linear ODE systems incorporating hidden confounders. Specifically, we establish a sufficient and necessary identifiability condition for the linear ODE system with independent latent confounders. Additionally, provide four identifiability conditions for the linear ODE system with causally related latent confounders, wherein the causal structure of the latent confounders adheres a DAG.
notable limitation of our work lies in the practical verification of these identifiability conditions, given that the true underlying system parameters are often unavailable in real-world scenarios. However, our study significantly contributes to the understanding of the intrinsic structure of linear ODE systems with hidden confounders. By providing insights into the identifiability aspects, findings empower practitioners to utilize models that adhere to the proposed conditions (e.g., through constrained parameter estimation) for learning from real-world data while ensuring identifiability.
Acknowledgements
YW was supported by the Australian Government Research Training Program (RTP) Scholarship from the University of Melbourne. BH was supported by NSF NMMS-2428058 XG was supported by ARC DE21010135 MG was supported by ARC DE210101624 and ARC DP240102088.
10

---

References
[1] O. O. Aalen, K. Røysland, J. M. Gran, and B. Ledergerber. Causality, mediation and time: a dynamic viewpoint. Journal of the Royal Statistical Society: Series A (Statistics in Society), 175(4):831-861, 2012.
[2] H. Aliee, T. Richter, M. Solonin, I. Ibarra, F. Theis, and N. Kilbertus. Sparsity in continuous- depth neural networks. Advances in Neural Information Processing Systems, 35:901-914, 2022.
[3] H. Alice, J. Theis, and N. Kilbertus. Beyond predictions in neural odes: interventions. arXiv preprint arXiv:2106. 12430, 2021.
Identification and
[4] M.-C. Anisiu. Lotka, volterra and their model. Didáctica mathematica, 32(01), 2014.
[5] R. Bellman and K. J. Aström. On structural identifiability. Mathematical biosciences, 7(3- 4):329-338 1970.
[6] A. Bellot, K. Branson, and M. van der Schaar. Neural graphical modelling in continuous-ting time: consistency guarantees and algorithms. In International Conference on Learning Representa- tions, 2021.
[7] H. G. Bock. Recent advances in parameteriddentffcatioo techniques for ode. In Numerical Treatment of Inverse Problems in Differential and Integral Equations: Proceedings of an International Workshop, Heidelberg, Fed. Rep. of Germany, August 30-September 3, 1982, pages 95-121. Springer, 1983.
[8] S. C. Burnham, D. P. Searson, M. J. Willis, and A. R. Wright. Inference of chemical reaction networks. Chemical Engineering Science, 63(4):862-8)) -23822322222222 2008
[9] I. Bzhikhatloy and S. Perepelkina. Research of robot model behaviour depending model parameters using physic engines bullet physics and ode. In 2017 International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM), pages 1-4. IEEE, 2017.
[10] R. Cai, F. Xie, C. Glymour, Z. Hao, and K. Zhang. Triad constraints for learning causal structure of latent variables. Advances in neural information processing systems, 32, 2019.
[11] W. Chen, K. Zhang, Cai, Huang, Ramsey, Z. Hao, and C. Glymour. Fritl: hybrid method for causal discovery in the presence of latent confounders. arXiv preprint arXiv:2103. 14238, 2021.
[12) G. Craciun and M. Feinberg. Multiple equilibria in complex chemical reaction networks: I. the injectivity property. SIAM Journal on Applied Mathematics, 65(6):1526-1566, 2005.
[13] E. Dockner. Differential games in economics and management science. Cambridge University Press, 2000.
[14] B. Gargash and D. Mital. A necessary and sufficient condition of global structural identifiability of compartmental models. Computers biology and medicine, 10(4):237-242, 1980.
[15] K. Glover and J. Willems. Parametrizations linear dynamical systems: Canonical forms and identifiability. IEEE Transactions Automatic Control, 19(6):400-66 646, 1974.
[16] D.-E. Gratie, B. lancu, and I. Petre. Ode analysis of biological systems. Formal Methods for Dynamical Systems: 13th International School on Formal Methods for the Design of Computer, Communication, and Software Systems, SFM 2013, Bertinoro, Italy, June 17-22, 2013. Advanced Lectures, pages 29-62. 2013.
[17] M. Grewal and K. Glover. Identifiability of linear and nonlinear dynamical systems. IEEE Transactions automatic control, 21(6):833-837 837, 1976.
[18] N. Hansen and A. Sokol. Causal interpretation of stochastic differential equations. Electronic Journal of Probability, 19:1-24, 2014.

---

[19] P. O. Hoyer, S. Shimizu, A. J. Kerminen, and M. Palviainen. Estimation of causal effects using linear non -onn-gassiin causal models with hidden variables. International Journal of Approximate Reasoning, 49(2):362-378, 2008.
[20] B. Huang, C. H. Low, F. Xie, C. Glymour, and K. Zhang. Latent hierarchical causal structure discovery with rank constraints. Advances in Neural Information Processing Systems, 35:5549- 5561, 2022.
[21] R. I. Jennrich. Asymptotic properties of non-linear least squares estimators. The Annals of Mathematical Statistics, 40(2):633-64 1969.
[22] D. Kaltenpoth and J. Vreeken. Causal discovery with hidden confounders using the algorithmic markov condition. In Uncertainty in Artificial Intelligence, pages 1016-1026. PMLR, 2023.
[23] M. N. Koleva and L. G. Vulkov. Two-grid quasilinearization approach to odes with applications to model problems in physics and mechanics. Computer Physics Communications, (3):3366 670, 2010.
[24] V. Mandelzweig and F. Tabakin. Quasilincarization approach to nonlinear problems in physics with application to nonlinear odes. Computer Physics Communications, 141(2):268-25) 1208882881111 2001.
[25] J. Mooij, D. Janzing, and B. Schölkopf. From ordinary differential equations to structural causal models: the deterministic case. In 29th Conference on Uncertainty in Artificial Intelligence (UAI 2013), pages 440-448 AUAI Press, 2013.
[26] J. Pearl. Causality. Cambridge university press, 2009.
[27] A. Polynikis, S. Hogan, and M. Di Bernardo. Comparing different ode modelling approaches for gene regulatory networks. Journal of theoretical biology, 261(4):51 2009.
[28] X. Qiu, T. Xu, B. Soltanalizadeh, and H. Wu. Identifiability analysis of linear ordinary differential equation systems with a single trajectory. Applied Mathematics and Computation, 430:127260, 2022.
[29] M. Quach, N. Brunel, and F. d' Alché Buc. Estimating parameters and hidden variables in non-linear state-space models based on odes for biological networks inference. Bioinformatics, 23((3):3209-322 -216,, 2007.
[30] Rubenstein, S. Bongers, B. Schölkopf, and J. Mooij. From deterministic odes to dynamic structural causal models. In 34th Conference on Uncertainty in Artificial Intelligence (UAI 2018), pages 114-123. Curran Associates, Inc., 2018
[31] B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612 634, 2021.
[32] C. Squires, A. Seigal, S. Bhate, and C. Uhler. Linear causal disentanglement via interventions. In International Conference on Machine Learning, pages 32540-32560. PMLR, 2023.
[33] Städter, Y. Schälte, L. Schmiester, J. Hasenauer, and P. L. Stapor. Benchmarking of numerical integration methods for ode models of biological systems. Scientific reports, 11(1):2696, 2021.
[34] S. Stanhope, J. E. Rubin, and D. Swigon. Identifiability of linear and Annear-in-paramee dynamical systems from single trajectory. SIAM Journal on Applied Dynamical Systems, 13(4):1792-1815 2014.
[35] H. Straubing. A combinatorial proof of the cayley-hamilton theorem. Discrete Mathematics, 43(2-3):233-79, 1983.
[36] W.-H. Su, C.-S. Chou, and D. Xiu. Deep learning of biological models from data: applications to ode models. Bulletin of Mathematical Biology, 83:1-19, 2021.
[37] G. Teschl. Ordinary differential equations and dynamical systems, volume 140. American Mathematical Soc., 2012.
12

---

[38] A. Tsoularis. On some important ordinary differential equations of dynamic economics. Recent developments in the solution of nonlinear differential equations, pages 147-153, 2021.
[39] N. Tu. Dynamical systems: an introduction with applications in economics and biology. Springer Science & Business Media, 2012.
[40] T. Verma and J. Pearl. Causal networks: Semantics and expressiveness. In Machine intelligence and pattern recognition, volume 9, pages 69-76. Elsevier, 1990.
[41] Y. Wang. X. Geng, W. Huang, B. Huang, and M. Gong. Generator identification for linear sdes with additive and multiplicative noise. Advances in Neural Information Processing Systems, 36, 2024.
[42] Y. Wang, W. Huang, M. Gong, X. Geng, T. Liu, K. Zhang, and D. Tao. Identifiability and asymptoties learning homogeneous linear ode systems from discrete observations. Journal of Machine Learning Research, 25(154):1-50, 2024.
[43] T. A. Weber. Optimal control theory with applications in economics. MIT press, 2011.
[44] F. Xie, R. Cai, B. Huang. C. Glymour, Z. Hao, and K. Zhang. Generalized independent noise condition for estimating latent variable causal graphs. Advances in neural information processing systems,
2020.
[45] Xie, B. Huang, Z. Chen, Y. He, Geng, and K. Zhang. Identification of linear non-gaussian latent hierarchical structure, In International Conference on Machine Learning, pages 24370- 24387. PMLR, 2022.
[46] H. Xue, H. Miao, and H. Wu. Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error. Annals of statistics, 38(4):2351, 2010.
[47] Y. D. Zhong, B. Dey, and A. Chakraborty, Symplectic ode-net: Learning hamiltonian dynamics with control. arXiv preprint arXiv: 1rXhv:1909.12077, 2019.
13

---

Appendix for "Identifiability Analysis of Linear ODE Systems with Hidden Confounders"
A Summary of notations and proposed identifiability conditions
Table Summary notations
Notation
Description
x/2
observable/latent variables
///2
the i-th observable/latent variable time
the j-th time point
e (1)/2(()
state of observable/latent variable at time t atl;), observable state time Lj initial condition of observable/latent variable first derivative x w.r.t. time dimension of observable variables dimension latent variables
20/20
att)
P
A. B.B.G
constant parameter matrices defined in Eq. and Function of time defined Eq.
f())
U
constant parameter vector defined in Eq.44) all the 24 for 0,...,
(u
(æo, Zo, A, B, (vx))) the system parameter of ODE system 2 vector defined Thm 3.1 A1
(())
augmented state
9o
initial condition augmented variable
(Zo, &0, A, B3, G)), the system parameter of ODE system
vector defined Thm 4.1 B1
20
given initial condition of latent variable
zsi
the i-th given initial condition of latent variable
= (Zog z A, B, G). the system parameter of ODE system
vector defined in Thm 4.3 B2
Jij
== x (ty; n7), observable state of ODE system 33 with parameter v7; at time ty
Vij
augmented state of æ,y at time tú the alternative counterpart corresponding to do......
A A/o......
Table 4: Summary of proposed identifiability conditions
ODEs
Conds.
Traj.
Obs.
Def/Thm.
Necessity
Eq.
A1 latent DAG,
continuous
3.1
Yes
Eq.
continuous
Yes
Eq.
latent DAG, CI
discrete
4.2
Yes
Eq.
latent DAG, B2, B3, B4
continuous
4.3
No
Eq.
latent DAG, C2, B3, B4
P
discrete
4.4
No
14

---

B Real world examples
In this section, we present two real-world examples that correspond to the ODE systems 2 and (3) These examples initially assume fully observable systems, with latent variables introduced by us based prior experience established physical laws.
B.1 Damped harmonic oscillators model
Consider one-dimensional system comprising D point masses m. for i = I,..., D with positions Qi(t) E I and momenta (t(t) E R. These masses are interconnected by springs characterized by
spring constants & and equilibrium lengths 4, and each mass is subject to friction with coefficient b;- The system's boundary conditions are fixed at Qu(t) = 0 and @D+1(t) =.
The dynamics of this system are described by the following linear ODE system
25
Pt( (.(Qi4(()) - - 00010000--000000000000000-0-000-
b.P.(tt/mm;
(8)
(.(1) P.(1//m;
where Qu(t) = 0 and @t+1(t) = L represent the fixed boundary conditions. External forces (t (e.g., wind force or varying magnetic field) may influence the entire system of coupled oscillators. These external forces can modeled here as latent variables with constant derivatives. Consequently, the system can be reformulated as follows:
P.(() = (2 (t) Q.((t)
(t) - 4) - k- (Q.(t) (t- 4-1) - b,P.(t/mm - Fo.
(1(() P.(1)/m; (1) =
where Oaj is a constant determining the effect the external force (t) on the i-th mass, and ej is the constant rate of change of the external force ((). This model aligns with our ODE system (2), and an illustrative causal structure for this model is provided Figure
F1
F2
P1
P2
P3
Q1
Q2
Q3
Figure 3: Example causal structure of the damped harmonic oscillators model with 3 oscillators and 2 latent variables.
In regions with predictable wind patterns, such as during monsoon seasons or controlled experi- mental settings, wind force can be approximated with constant rate, making this an ideal context for modeling external forces with constant derivatives. Furthermore, constant forces or those represented as polynomial functions of time align well with our ODE system structure. For instance, uniform magnetic field acting on the system would produce constant force. These examples demonstrate that various latent factors can effectively fit within our ODE structure.
15

---

B.2 Population model
The growth of population P(t) can be described by linear ODE
P(t) = aP((t,
where a constant representing the population growth rate. This system may also influenced by latent variables Ii, such as environmental factors or food supply. By incorporating these latent influences, the system can be expressed as:
P(() aP(I) + bL,(()) + cl2(()
(t (L2
L2(1) m
where a, b, c, and m are constants. Here, (t) represents the food supply, which is influenced by the environmental factor (2((). (1) corresponds an environmental factor, such as temperature or pollution level, that changes steadily over time. This model aligns well with our ODE system (3), and an illustrative causal structure for this model is provided in Figure 4
L1
L2
P
Figure 4: Causal structure of the population model.
example of an environmental factor changing at a constant rate is pollution from industrial plant that continuously releases a fixed amount of pollutants, or from a wastewater treatment plant that discharges a specified amount of treated wastewater into a river on an hourly basis.
16

---

C
An example of an unidentifiable case of the linear ODE system
3
Recall that the parameters of the ODE system S
are:
0
Co
1
Zg
B
1 1
G=
0
0
1 0
0
[
B
A''
B G
1 =
A' =
M
M'
0 1
0
0
G
0
We first calculate the solution of =()),
(t) = e e²=20 00
Gh 20
G z0,
[1+t]
k!
k!
0=
k=0
We intervene (t) then under matrix M:
(t) x2 (f) + (t) 22 = (t) + 1 2.
To solve this differential equation, we rewrite it in the standard linear form and multiply both sides by the integrating factor e e_t,
( - (t) = (t 2)
The left-hand side of this equation is the derivative of E (r²
d _ fe (x))) (t 2)e
Next, integrate both sides w.r.t.
f di x2())dt
/ (t ddt
The left-hand side integrates to:
exx((
Next, we use integration by parts to find the integral on the right-hand side:
[(++)e 2)= - 2)e- - / 'et
=-(t+ 2)e -t -t
(t+3)e
Thus:
(r2(( = - (1+ 33e + C,
where C' the constant of the integration.
Multiplying both sides by et to solve for (t)(:):
(2 -t - Ce'
Now, use the initial condition z2) = 1, we get
C = 4.
Therefore,
xt = 4e - -t- - 3.
Whereas under matrix M':
b2(() (t) + (t)) (t)
-.
Simple calculations show that
x2 = 22 2 11
17

---

D Detailed proofs
D.1 Proof of Theorem 3.1
Proof. Recall that the first derivative of (t)) can be expressed as:
a<t) = Ax(() + Bz(()
: Bus k+1 + Bzo
= Ax((t) + Σ
k k+1
k=
Set
v(() =
f2
(+++
we see that gt)) E RR++++2 and the first derivative of yt) w.r.t. time be expressed as
(t))
ú(t)
2t
(r + 1)E")
A
Bzu
Bu
Boi
B
2
r+1
0d
0
1
O₂
1
0
0
0
t
0g
2
0
t2
Og
0
0
0
1
t
denoted as F
((
where O₂ denotes d dimensional zero row Obviously,
y = [x 1,0
denote it yo- Therefore, yt) follows homogeneous linear ODE system that can be expressed as:
(t) Ft((),
(10)
v(0) Yo
where F E ((d++4)) Worth noting that all state variables in the ODE system 100 are observable. Then according to Lemma [2.1] the identifiability of the dynamical system de-
scribed by the ODE system (10) is contingent upon the linear independence of the vectors
(yos Fyo F'yyo.... pdtr+igo y- Specifically, the system is (yo, -ddiinitiieeble and only this set of vectors linearly independent, indicating that the matrix formed by these vectors, denoted by H. has rank of + + In the following, we will elucidate that and only assumption A satisfied, the rank of this matrix H equals d + + 2
18

---

Some calculations show that,
_1* (Axo + B2o) + Ei=0 j!AA 22- BBy
F you
for = 1,,,... (11)
k!
where k! the (d + &o- - 1) element.
And
( + BB0 + -ia
F= yo
for - + d 1.
(12)
According to assumption A1 Theorem 3.1
Ar+2 Eu + Bzu
Σ )=0
jIA' jIAA- Buj,
therefore, Fxy can also be expressed as
A A (r+2) 0
F royo- you
for k: r +....,++++1.
(13)
We denote the matrix
H
yoo
Fyo
F2ya
prt1yo yo
F
H11
H12
H21
Hza
as block matrix. Then, based on Equations 11I) and [3). one obtains that
H11 2o Ano - Bio
A'xo + ABz, + Be
A mo Bgg + 11A-
E Rdx(++2)
H1
[a A3
1d113]
E Rdxd
1
0
0
H21
0
0
2!
0
R (R(+2))(+++))
(r+ 1)!
Hz2 xx E (
Some calculations show that
(an rank H12) + rank(Ha1).
apparent that
rank(H21) = 2.
19

---

To achieve rank(H) = + the rank of H12 must be d. The rank of H12 equals if and only if the set of vectors (3, 1,8....,4441131 is linearly independent, that is, assumption A satisfied.
Now that we have proved that the ODE system (IO is (yo, Fididtitiie if and only if assumption Al is satisfied. That is, under assumption A1, the trajectory y(: yo: F)) uniquely determines both Yo and matrix F. Consequently, it also uniquely determines (æa, A, Bzg, Bwo. ... Bu,), thus establishing that the ODE system 2 is i-identifiable if and only if assumption A is satisfied. C
D.2 Proof of Theorem 4.1
Proof. Recall that the first derivative of a+t) can be expressed as:
(+t) At(()) Bz(()
p-1
BG* 20,4
= Axt(t) + Σ
k!
A=
Set
x
v())
we see that yt)) E RR+,,, and the first derivative of yt)) w.r.t time can be expressed as
(tt)
0
i(t)
2/
(p
2
A
Bzo
BGzo
BG sg
BGP-2a0
BGP-1 11 2n
x
(p-2)1
(p-1)) 1J]
0a
0
0
0
0a
0
0
0
O₂
0
0
0
t2
:
0a
0
0
0
denoted as
2/(1)
where O₂ denotes d dimensional zero row vector. Obviously,
0 [o] 1, 1,0,0.....00
we denote it as yo- Therefore, yt) follows a homogeneous linear ODE system that can be expressed
(t) F (t))), v(0) yo
(14)
where F (dd+)) Worth noting that all state variables in the ODE system [4] are observable. Then according to Lemma the identifiability of the dynamical system described by the ODE sys- tem 14 is contingent upon the linear independence of the vectors (yn F F2 yos...., pd+p-lyn) Specifically, the system is (Yo- F)-identifiable if and only if this set of vectors is linearly independent, indicating that the matrix formed by these vectors, denoted by H. has rank of d p. In the following, will elucidate that if and only assumption B1 is satisfied, the rank of this matrix H equals d +
20

---

Some calculations show that,
A
k-1 1 BG'
0
Fk you
for k: =
(15)
k!
where k! the (d + k - 1)-th element.
And
Ak-P + E5=d 1p 1P-1-iB''z
F* yo
for k = = ...., + d.
(16)
According to assumption B1 in Theorem 3.1
7 = A² +
A
20.
==0
therefore, yo can also be expressed as
AARPP
F* voy0= you
for k: = - L..., p 1.
(17)
We denote the matrix
: = yo
Fyo
Foyo
F F Yo
F
FDid yn
H1
H12
H21
H22
as block matrix. Then, based on Equations (15) and (17,, one obtains that
H11
4o
Aua
Bzo
A
ABzo BCzo
A no
E Edxp
H1 = b
A7 ...
1
E dxd
1
0
0
0
0
1
0
H21
0
0
2!
0
Rpxp
(p 1)!
Hzg O Opxd E Rpxd
Some calculations show that
rank(Hf) = (ank(H12) + rank(H2)).
apparent that
rank(HH21) p.
21

---

To achieve rank(H) = + p. the rank of H12 must be d. The rank of H12 equals d if and only if the set of vectors {7, , Ad-1yy is linearly independent, that is, assumption B1 satisfied.
Now that have proved that the ODE system (4) is (yo -'dieeitifiialle if and only if assumption B1 satisfied. That is, under assumption B1, the trajectory y(-; yox F) uniquely determines both Yo and the matrix F. Consequently, it also uniquely determines (æo, A, Bzu, BG ... BGF-1zu zu), thus establishing that the ODE system 3 is ny-ieentifiable if and only if assumption B1 is satisfied. [
D.3 Proof of Theorem
Before providing the main proof, we first present two lemmas we will use for our proof.
Lemma D.1. (34 Theorem 3.4] The ODE system ① is (æo, A)-identifiable if and only if the Fo. Fo A) is not confined to a proper subspace of Fd..
trajectory z=
Lemma D.2. (34 Lemma 6.1] Trajectory ( A) is not confined to a proper subspace of R if and only there exists 11, 12.-... la such that 21, -2,---.3 y Da linearly independent.
Proof. In the proof of Theorem 4. we demonstrated that the ODE system B under latent DAG assumption, can be transformed into fully observable homogeneous linear ODE system (44. According to Lemma D.1 the ODE system (14) is (yo. Fi-ideniife if and only if trajectory ( Vo- F) is not confined to proper subspace of Rd+P, Furthermore, based on Lemma D.2 this condition holds if and only if there exists time points 12,/2,..... Ld+p such that the vectors
91 ,2...., Ddtpp are linearly independent (i.e., assumption C1). Therefore, if and only if assumption C1 is satisfied, the trajectory y(: F) is not confined to proper subspace of RR+,,, ensuring that the ODE system (14) is (yo: -iideeiiiiiieb.. Consequently, the ODE system (3) is 17-identifiable.
D.4 Proof of Theorem
Proof. Under assumption B2, since each z satisfies assumption B1, Theorem implies that the ODE system (3) is -idieentteeeal for all ... That is, one can identify
(æo, A, Bz BG BG 2
for I....,p.
Next, we will prove that matrix B is identifiable under assumption B3. Define the matrix
S [B
B2y²
Bz 1P)
know that E Rd and S is identifiable. The matrix S can also be expressed as:
= [zo]
26²
20 ap
BZ
where under assumption B3, the matrix Z is invertible. Therefore,
B= S2
Since Z is known matrix, B is identifiable.
Similarly, prove that B for j 1, ,.....P 1 is also identifiable.
We now show that, under assumption B4, the matrix G is identifiable. Define the matrix
B BG
W =
BGP-1
know that W e Rddxx, and W is identifiable.
22

---

Since G as nilpotent matrix, G 0, thus BG = 0 If we define the matrix
BG
BG²
LB
then V Rdxx,, and V is identifiable. The matrix V can also be expressed as:
B
BG
V
G= WG.
(18)
[GG-1
Under assumption B4, one can find p linearly independent rows in matrix W. Denote the matrix composed of these p linearly independent rows Wp, which is invertible. Denote the matrix composed of the corresponding rows of V as Vp we have
V = WGG.
Since Wp is invertible, then
G= W
Because both V,, and W, are identifiable, G is also identifiable.
D.5 Proof of Theorem 4.4
Proof. Under assumption C2, for each E {l,..., p), the corresponding observations satisfy assumption C1. Based on Theorem 4.2 the ODE system is n7, -ddienntiiialle for all i ......P p. This implies that one can identify
(mo, A, B2 BG BGP-1
for all 1, l. p.
According to the proof of Theorem 4.3 under assumptions B3 and B4, matrices and G are also identifiable.
23

---

E
Identifiability conditions of the linear ODE system with other f(t)
In this section, we provide identifiability conditions for the linear ODE system 2 with f() ve and ft) sin(t) + Uycos(t) For notational simplicity, we slightly abuse notation by using the same symbols as in Section
When f(1) follows an exponenial function of time E.1
We define ft) in the ODE system 2 as:
f(t) = ve" 1 IR.
Simple calculations show that
ztt) ve' + 20 v.
Thus,
a(t) Ax(t) + Bz(()) At(t) + Bue' + Bzo Br.
(19)
We denote the unknown parameters of the ODE system with this f(t) 0, specifically, 0 == (oo. 20, A, B, v). Let [x: (t; x² O1: (t: denote the solution the ODE system (2). It important note that under
hidden variables setting, only a=t; 0) is observable. Based Equation (T9). we present the following identifiability definition.
Definition E.1. For œy E Rd 20 E R,, A Rdxdd B E Raxp and v R., for all x'g E Rd all #o E IR, all A' E Rdxdd all B' Rdd,, and all v' E IR, we denote 0' := =aa z A', B', ,'',') v'), we say the ODE system (2 is f-identifiable: if (Io. A, Bzo. Bv) + (x A', B' zn B' v'), it holds that ((:0) + x(:6").
According to Definition E.1 if the ODE system 2 with exponential (() is 0-identifiable, then the trajectory of the system can uniquely determine the values of (mog A, Bzg, Bv). This determination sufficient to identify the causal relationships between observable variables as described by Equation (19). Consequently, one can safely intervene in the observable variables the ODE system and make reliable causal inferences, despite the fact that matrix B cannot be identified under this definition.
Theorem E.1. For an E TR.d Za E RD, A E Rdxdd RRdx,, and v E RP, the ODE system is θ D-identifiable if and only if assumption DI is satisfied.
DI the set of vectors (yo Fyg...... and
pd+1 yob is linearly independent, where yo = [x 1
A
B
Bzg
Bo
F
0g
1
O0
0
0
0g denotes d dimensional zero row vector.
The proof Theorem E.1 is presented below. Condition D1 both sufficient and necessary, indicating, from geometric perspective, that the vector yo is not contained in F-invariant proper subspace of pd+2
Proof. Set
[x(t)
v(()
e²
that (t) E dR2+2 and the first derivative of yt) w.r.t. time can be expressed as
T()(()
A
Bv
Bzo
Bv
T (
0(1)= =
e'
=
0₂
1
e
0
0
v(1)

---

where Og denotes d dimensional zero row vector. Obviously,
y(0) ( 1, -Yo-
Therefore, y(t follows homogeneous linear ODE system that be expressed as:
( Fy(t),
(20)
y y(0)
where F E (dd++))(d+2),, Worth noting that all state variables in the ODE system (2) are observable.
Then according to Lemma 2.1 the system (20) is (Yo. -i)-identiiieeeee if and only if condition D1
stated in Theorem E. satisfied. That is, under assumption DI, the trajectory y(: yo. F)) uniquely
determines both Yo and matrix F. Consequently, it also uniquely determines (æo. A, Bzo, Bv), thus
establishing that the ODE system 2 is 0-identifiable if and only if assumption D1 is satisfied.
C
When f(i) follows an trigonometric function of time E.2
We define (() in the ODE system (2) as:
f(t) sin((t)) U2cos(t), 01 U2 R
Simple calculations show that
z(t) v2sin(t) vic((s() 20 Uu
Thus.
+(()= (t( + Bz(()
(21)
(tt + Begsin(t) - Brycos(() + Bzg + B
We denote the unknown parameters of the ODE system 2 with this f() as 0, specifically, 0 (wo, 20, A, B, U1, U2). Let [x (t; 0). (t: 0))" denote the solution of the ODE system (2). It is important to note that under our hidden variables setting, only ætt; 0) is observable. Based on Equation 21). present the following identifiability definition.
Bdxp and U1. V1.02 RP, for all 26 Rd. all z1 E RP, all A' gaxd all B' gdxp, and all vi.vzz E IRP, we denote O' := (atg) o1 A', B', v/y V2), say the ODE system ( is 0-identifiable: if (æo, A, Bzo, Bv Bu2) + (x), A', B' 2/6, B'u B' v')),, holds that x:; ;) + xx:;0")
Definition E.2. For Eo E Rdd 20
R2, A E gaxd
According to Definition E.2 if the ODE system ( with trigonometric f() is 0-identifiable, then the trajectory of the system can uniquely determine the values of (æo, A, Bzg, B Bv2). This determination is sufficient to identify the causal relationships between observable variables as described by Equation (2I). Consequently. one can safely intervene in the observable variables of the ODE system and make reliable causal inferences, despite the fact that matrix B cannot be identified under this definition.
Theorem E.2. For ao E Rd, 20 E RP, A E IRddd d B E Rd and U1, R', the ODE system is O-identifiable if and only if assumption El satisfied.
El the set of vectors (yo, Fy..... ..., pd+2 yot is linearly independent, where yo ( 1, 1, and
A
Buz
-B
Bzo + Bvi
1
0
F
Oa
-1
0
0 0
Oa
0
Oa denotes d dimensional zero row vector.
The proof of Theorem E.2
is presented below. Condition E1 is both sufficient and necessary, indicating, from geometrie perspective, that the vector yo not contained in an F-invariant proper subspace of pd+3
25

---

Proof. Set
a(t)
(t
v()) =
(t)
we see that y(t) E IRd+3, and the first derivative of y(t) w.r.t. time can be expressed
a(t) 1
A
Buz
Bu
Bzn
Bvi
x (t(
cos(l)
Oa
0
1
sin (t)
ú(t)
(t)(()
Oa
-1
0
cos (t)
0a
0
where Og denotes d dimensional zero row vector. Obviously,
y [= 0, Y/o
Therefore, y() follows a homogeneous linear ODE system that can be expressed as:
(() Fy((),
(22)
y(0) Y/u
where F E (dd+3) Worth noting that all state variables in the ODE system (2) are ob- servable. Then according Lemma 2.1 the system (22) is (y/- F'diidentifii if and only condition E1 stated in Theorem E.2 is satisfied. That is, under assumption E1, the trajectory ( Vo. F) uniquely determines both Yd and matrix F. Consequently, it also uniquely determines (Zo. A, Bzg, Bv1, Buz), thus establishing that the ODE system is 0-identifiable if and only if assumption E1 is satisfied.
26

---

F An alternative approach to identifying matrices B and G in the ODE system
E.1
Identifiability condition from 2p controllable whole trajectories
Recall that Zg denotes the initial condition of the latent variables in the ODE system 33. We further specify the initial condition of the latent variable 2j as 2nj for 1....,, Assume that it is possible to control the initial condition of each latent variable, žej. independently. Specifically, for each experiment, researchers can intervene in the initial condition of a latent variable, denoted as z0z. The value of Fūy is treated as a given value. Under this intervention, the initial conditions of the latent
variables are adjusted to [z01...., 26j......26..................... 20pp IT= which we denote as øoy.
To identify matrices B and G, it is necessary have least two intervened initial conditions for each
latent variable, denoted as 20;
=t! and =0 for the latent variable 2j. Consequently, the corresponding
intervened initial conditions for all latent variables can be represented as æ0; and z0j. Under these conditions, we present the definition of the identifiability of the ODE system
Definition F.1. Given zūj. 203 IR for j == 1...... ... Zo E Rd zu RP, A E Raxd B E Rd and G E Rexp, under the latent DAG assumption, for all z/ E Rdd all z/1 E RP, all +1 A' E Rdddd all B' E Rdxx, and all G' & IRXX, we denote øbj [zo1,..., 0ff 0fs..., z0pl l and (%/,)'' (=01,-..,,00 -0 -0j..... =opp² 1T we further denote n == (ar, zo. A, G) and (m/)* (a%) (R A',,B',"' for i = 1,2, we say the ODE system 33) is --ddenniiii if (æo, A, G) * (æ', A' B', G'). it holds that 3i {1,2} and j (l..... ,p) such that x(:mn) 7 ar: (.)
Definition F.1 establishes the identifiability of the ODE system 3 from 2p whole trajectories (::)))
with = 1, and j = 1, , p. According to this definition, both matrices B and G are identifiable. Based on this definition, we present the identifiability condition.
Theorem F.I. Given =ū], = E l with = + =0} for j = I,..., for 2o E Rdd, 20 E RP, A
Raxd B E Raxp and G E Rexp, under the latent DAG assumption, the ODE system is {m 1.2 yp- identifiable if ussumptions B5 and Ba are both satisfied.
B5: each Z0g for i = 1,2 and j = 1,..., ,.. satisfies assumption BI. That is, if we set Et= AP-1- 1 BG soy then the set of vectors {r', AY..... Ad Ad is linearly independent for all i = 1.2 and = I..
= A' =
The proof of Theorem F.1 is presented below. Assumption BS ensures that the ODE system 3 n
identifiable for all i 1,2 and j -l...... Consequently, (æo, A, Bih BG .... BG 2nj for all = 1, 2 and =;...... p is identifiable. Through straightforward calculations, the identifiabil- ity of matrix B is established. To identify matrix G, assumption B4 is required.
The assumption that the initial condition of each latent variable 24 can be controlled independently is inspired by the "genetic single-node intervention" proposed in [32), where interventions can be made each latent node individually. This assumption is relatively more relaxed compared to controlling
the initial condition of all latent variables simultaneously, as discussed in Subsection 4.3 However, this method requires p more trajectories, totalling 2p trajectories, to identify matrices B and G.
Proof. Under assumption B5, since each b/ satisfies assumption B1. By Theorem 4.1 the ODE system is -identiitiee for all = 1,2 and j = 1,....p. Consequently,
(xo, A, B BGE BGP- bby)
for all i 1,2 and] j = 1,.... . is identifiable.
27

---

We express oz as
B11
B13
Bip
B
Bav
Báj
Bap
Lpp
We know that Bž E IRd is identifiable for i = 1, Thus, the first entry of Bigy. denoted as (B%b))1, is identifiable and can be expressed
(B 1 B11 201 + +... 2 ... + B1p20y
(B2 11 B11 201 + B15 +2 +... + B1p
=11 Since 20j and 20 are given values, we can easily calculate the value of B₁j. Similarly, one can calculate the values of Bmj for all m l...., and j I.....P. thereby establishing the identifiability of matrix B.
=2
In similar manner, matrices BG, BG2, ..... BG also identifiable. Then, according to the proof D.4 Theorem 4.3 the matrix is identifiable under assumption B4.
F.2 Identifiability condition from discrete observations sampled from 2p controllable trajectories
We further extend the identifiability analysis of the ODE system 3 cases where only discrete observations from 2p controllable trajectories are available.
Definition F2. Given z)) 203 for 1,... for zo E Rd 20 IR, Raxd B E R and G E Rexy, For any n > let 4k, ,,...,11 TI be any n time points and æ) 0042i m2) be the error-free observation of the trajectory x0::n;) at time tk. Under the latent DAG assumption,
say the ODE system (3) is {nj Jf-identifiabll from xr 1111--- ..) 1,2 and j ,,...,P.
E R., all A' E Raxd all B' E Rddx, and all G' E RRxD with (Do, A. B,B. G) + (I)) A', B' G"), ), it holds that 3i (1.2).j (I.....PP and k E {1...., such
if for all xo E Rd, all z/
that æ(ta: m) f (T)
Based on Definition F.2 we present the identifiability condition.
Theorem F.2. Given 20j 20 E I with =nj z0j for = 1....p for Rd E RP,A A Rdxd, B E Raxp and G Rexp. We define new observation V/s
=2
(( 1, t........ k..... 16-11T E Betp, for i = 1,2,j = 1.....p and k = 1, 1,..., Under the latent DAG assumption, the ODE system L is {n) 1nl221p -ddentiffable from discrete observations zi.,
2)n..... ",,,,.., i 1,2 and 1.... p. if assumptionss C3 and B4 are both satisfied.
C3: for each E {1,23,j E [l.....p} there exists (d + p) 8y 's with indexes denoting as (k),, ki k........................... k k),ppp) S (1,,,...,n), such that the set of vectors linearly independent.
(W'Ng, y " -- Vjng
The proof of Theorem F2 is presented below. Assumption C3 ensures that the ODE system is n -jiiddentefiable from discrete observations x .j....., ",),,, for all = 1, and I,..., p. As in Subsection F.I matrix is identifiable. Then, under assumption B4, matrix also identifiable.
Proof. Under assumption C3, for each E [1.2] and E (1....,p)) .P. the corresponding observations satisfy assumption C1. Based on Theorem 4.2 the ODE system ( is v/ -identfiible for all i = 1, and 1, ,.....p. Consequently,
(æo. A, B BG BGP 120,)
for all 1,2 and j ..... p identifiable.
Following the proof of Theorem F.I identifiable.
matrix B is identifiable. Under assumption B4, matrix G also
28

---

G More simulation results
In this section, we present additional simulation results for higher-dimensional cases, along with simulations that incorporate a variety of ground-trute parameter configurations.
G.1 Higher dimensional cases
In this subsection, for the nunjidddeetii cases of the ODE system (3 we provide case with and p 5. The true underlying parameters of the systems are provided below. Initial parameter values are set to the true parameters plus a random value drawn from uniform distribution U(-0.14, 0.14) for each replication. To ensure reliability in the estimation results, we perform 50 independent random replications for each configuration, reporting the mean and variance of the squared error Table
2
2
1
1
1
-2
-1
1
0
-2
-2
-1
-1
A
-2
2
0
-1
-2
B
-2
2
1
-1
-1
-2
-1
2
0
0
-2
-2
-2
1
-2
0
2
-2
2
-1
TO
0
0
-2
-17
2
2
0
1
-2
G
0
0
0
2
A' = 15
30
2
0
0
2
0
0
0
0
-2
n- n-identifiable: 17 (Do, 20, A, B, G), unidentifiabee: 1)
(Do, 20, A, B.G.
I denotes a j x identity matrix.
Table 5: MSEs of the (indeeiiiiii cases of the ODE (3) with d = 5. =
n
Bzu
BGzo
BG zu
BG 20
BG z0
0.0148 (#0.0006)
0,3911
0.9624
0.7316
0.1037
0.0096 (=00.0033
10
Identifiable
(=:055899
(#3.9249)
(+118971)
(=0003374)
0.0059
0.1529
0.1726
0.2447
0.0212
0.0012
100
(14.01E-055
(4.0.0277)
(1.0.0541)
(L00,0748)
(1.0.0007)
(...00
0,0053
0.1394
0.1241
0,2119
0.0164
0.0004
1000
(42.92E-05)
(10.0200)
(100.025))
(10..049)
(1.0.0004)
(16.00E-07)
0.0853
1,0067
3.7422
2.7696
0.9229
0.0508
10
Unidentifiable
(+0000075)
(+: (+.51)
(+555.8402)
(+24.5043)
(+.2.7559)
(+-0.0111)
0.0357
0.4091
1.0428
0.9782
0.3871
0.0256
100
(+-0.0019)
(+-0.3112)
(+2.1792)
(+5536544)
(+-0.6747)
(+-0.0032)
0.0332
0.3286
0.7123
0,9782
0,5487
0.0393
1000
(+000017))
(#:0.124)
(+1.8836)
(1:231633)
(#:0,220)
(#:0.0047)
For (un)idennntiee cases of the ODE system (3), we consider case with d 10 and p To accelerate estimation, sparsity is introduced in the parameter matrices by randomly setting 70, 35. and entries matrices A, B and G, respectively, as zero. The true underlying parameters of systems are provided below. Initial parameter values are set to the true parameters plus random value drawn from a uniform distribution 0.(00.1,0.1) for each replication. ensure reliability in the estimation results, perform 50 independent random replications for each configuration.
29

---

reporting the mean and variance of the squared error in Table 6
0
0
0
0
0000000000000
0
2
0
-2
2
0
10000
2
0
0
0
00000000000
00
1
1
00000
0
0
00000000000
00000000000000
1
0
-2
0
2
0
-2
0
0
-1
B
0
2
0
0
0
2
0
-2
0
2
0
0
0
0
0
0
0
0
2
0
0
0
2
-2
O
0
0
-1]
0
-1
0
20000
0
0
00000 0
G=
0
0 0
-1
A' I10:
0
0
0
0
= 2-2
0
0
-2
2
-1 1101 1) 20 = , 1,...,..
{ -i-iientifiiabl: 7, (mo, =0 A, G) unidentifiable: 7; (mo, = A', B,G).
e; stands for a p-dimensional vector, with the i-th entry being and the other entries being 0.
Table 6: MSEs of the {n.)- (nn)--(uniddentfiaale cases of the ODE (3) with 10, 5
Identifiable
Unidentifiable
n
A
B
G
A
B
G
1.53E-11
2.49E-10
3,01E-10
0.8345
0.2118
0.0037
10
(422.36B221)
(16,30E-19)
(19.200-19)
(1.0.2668)
(+00.0260)
(1:0.0022
9.15E-13
1.49E-11
1,80E-11
0.7216
0.1952
1.25E-21
30
(+:444B-24)
(+1.18E-21)
(+1773B-21)
(=0.4099)
(+0.0156)
(+5,18E-41)
9.64E-14
1.57E-12
1.90E-12 12
0.6510
0.2211
0.0042 (1:0.0003)
50
(112299-25)
(13.43E-23)
(15.02EB-33
(10.2251)
(100.0778
Tables and 6 present results similar those in Tables [ and 2 providing strong empirical support for the validity of our proposed identifiability conditions.
G.2 Various true parameters
To further support our proposed identifiability conditions, we conduct additional simulations incorpo- rating a variety ground-truth parameter configurations, rather than a fixed underlying parameter set. Specifically, for each simulation run, unique ground-truth parameter configuration using different random seeds, and we subsequently reported the mean and variance of the squared error across all results. For the low-dimensional and {n, ) (un)identifiable cases, we perform 100 replications, while for the higher-dimensional cases, we perform 50 replications. Additionally, in the ) (un)ieentife cases, we initialize the parameter values the parameters plus random value drawn from U 0. for the d = 3, p = 3 case and from U (-0.055, 0.05) for the 10, p 5 cases. For the (indiiedenti cases, the initialization settings the same those used in the faxed-parameter configurations.
generated
The simulation results are presented in Tables estimates in the identifiable cases are notably more accurate than in the unidentifiable cases, providing strong empirical support for the validity of our proposed identifiability conditions.
and 10
Across all these tables, parameter
is noteworthy, however, that even in theoretically identifiable cases, certain scenarios emerge where parameter identification is challenging in practice; we refer to these hard estimate cases. In these instances, estimates may deviate significantly from satisfactory values, similar to challenges encoun- tered in fully observable ODE systems as discussed in 28 Consequently, for identifiable cases
30

---

with varying true parameter configurations, the results are less precise than those for corresponding pard-parreeter cases, due the inclusion of some hard estimate instances. Investigating the practical identifiability of the ODE system
remains an intriguing direction for future research.
Table 7: MSFs of the (un(uniddentffabll cases of the ODE with various true parameters
Identifiable
Unidentifiable
n
A
Bzo
BGzo
BG zo
A
Bzo
BGzo
BG zo
0.0060
0.0157
0.1698
0.2297
0.0691
0.2720
1.3133
0.6622
10
(#000008)
(#:0.003)
(#:056665)
(#111053)
(#0.0203)
(#005914)
(#7.4471)
(#885348)
0.0026
0.0108
0.0820
0.1287
0.0283
0.1003
0,4880
0.2649
100
(#-9227B-05)
(=:0.0022)
(#0.1119)
(#:0.7042)
(=0.0031)
(#000044)
(#:2.6547)
(#116631))
0.0020
0.0092
0.0870
0.0705
0.0227
0.1061
0.5015
0.2574
500
(=6448B-05)
(+000233)
(#:1,194))
(#:01117)
(=0.0018)
(=000))))
(#:3,0111)
(+22.0779)
Table 8: MSEs of the (n. (nnDd-aannideetfffaalll cases of the ODE
(3)
with various true parameters
Identifiable
Unidentifiable
B
G
A
B
G
0,0006
1.89E-5
0.0009
0.0861
0.0088
0.0101
10
(12.21E-5)
(13.55E-8)
(1:6.71B-5)
(1:0.1773)
(+0.0020)
(10.0045)
0,0006
1.87E-5
0.0010
0.0789
0.0092
0.0104
30
(42220E-55)
(#3.47B-8)
(1-6.64F-5)
(+-0.1280)
(+:0.0028)
(+:0.046)
0.0006
1.88E-5
0.0009
0.0503
0.0063
0.0114
50
(+:2.11E55)
(#3.51E-8)
(1-6.67E-5)
(+:0.0430)
(+:0.0006)
(4:0.0047)
Table MSEs of the n-nundiident cases of the ODE (3) with d 5, p = 5 with various true parameters
A
Bzo
BGzD
BG zu
BG321
BG 20
0.0144
0.1215
1.4643
2.1890
1.8254
0.4826
10
(10.0004)
(+0,0757)
(18.3976)
(1:54.9706)
(1:48.7033)
(d5.71277)
Identifiable
0.0041
0.0395
0.2850
0.3891
0,2078
0.0239
100
(+:455E--05)
(+0.0092)
(=+0.1739)
(+:04933)
(++02250)
(+000022))
0.0032
0.0337
0.1934
0.2242
0.1197
0.0181
1000
(+:3266E-05
(+0.0049)
(=006)
(10.2180)
(=0000712)
(+000014)
0,0740
0.4599
2.8628
1.8743
0.4834
0.0334
Unidentifiable
(=0.0047)
(+0.4841)
(=9.5476)
(+:86653)
(+1.2600)
(+000147)
0.0263
0,2142
1,1678
1.2354
0.2878
0.0193
100
(+0.0031)
(+..1869)
(=8.2277)
(+9.44770)
(+0.8655)
(+00.0552)
0.0142
0.1389
0,6979
0.6701
0.0732
0.0062
1000
(10.0003)
(10004633
(=1.20800
(+.15528)
(100.0336
(100.0003)
31

---

Table 10: MSEs of the (n, iindnfiianddeetiffaable cases of the ODE (3) with d 10, 5 with various true parameters
Identifiable
Unidentifiable
B
G
A
B
0.0044
0.0350
0.0287
0.6266
0.1310
0.0054
10
(+0.0001)
(#0.0098)
(#0.0053)
(=0.1524)
(+0.0269)
(=0.0004)
0.0067
0.1258
0.0315
0.5833
0.1058
0,0021
30
(1.0,0005)
(1005597))
(100.0104)
(1:0.2885)
(100.0114)
(18..7E--55)
0.0033
0.0323
0.0354
0.5193
0.1108
0.0021
(1:566EB-05)
(1:0.1033
(10.0084)
(+0.0982)
(1000014)
(19.02E-05)
32

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the contributions and scope of our paper?
Answer: [Yes]
Justification: The main claims presented in abstract and introduction accurately reflect our paper's s contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of work are discussed in Section 7
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
33

---

We provide the full set of assumptions and complete proof for each theoretical result presented in our paper.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide all experimental details in Section and include the code in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
34

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We provide our code in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ priips.cc/publiccguiieee CoodeSumeeieeeeee ssi cessonPolll mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all experimental details in Section 5 and it is noteworthy that our experiments do not require any training phase.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We conduct 100 independent random replications for each configuration and report the mean and variance of the squared emror.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
35

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [No]
Justification: Since our experiments solely consist of simulations designed to validate our theoretical findings, the computational resources employed not consideration for our research objectives.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: atttss:/neuurpp: http:///www.n.ppppcppppppeaa/a// httppp/bllcccttticccciieaaaaaaa.
Answer: [Yes]
Justification: Our research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA].
Justification: We claim that this work does not present any foresecable positive or negative social impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
36

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA].
Justification:
This paper does not require safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA].
Justification:
This paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
37

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA].
Justification:
We did not use crowdsourcing conduct research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA].
Justification:
We did not use crowdsourcing or conduct research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
38

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
39


Reviews:
[{"review_id": "M1BMtoNfH3", "reviewer": "Reviewer_1wF2", "summary": "The paper focuses on the (parameter) identifiability problem of linear ODE systems. The identifiability results of the existing work has been limited to fully-observable systems, i.e., with no latent variables. The paper analyzes the parameter identifiability of partially-observable linear ODEs with certain structure, that is $\\dot{\\mathbf{z}}(t) = \\mathbf{0} \\mathbf{x}(t) + G \\mathbf{z}(t)$: (i) the observables don’t affect the time derivative of the latents, and (ii) the latent transition matrix $G$ is strictly upper-triangular (“DAG structure”). For this setup, they characterize the identifiability conditions for a single trajectory and multiple trajectories; in addition to the cases where these trajectories are observed in discrete-time steps. They evaluate the validity of the proposed identifiability conditions on a simulation study, by comparing the parameter estimation errors for identifiable and unidentifiable data generating processes.", "questions": "* To show the practical value of the paper, can you provide linear ODE examples having practical value in some fields, e.g., chemistry, etc? On these, the structural constraints could be assessed. Then, it could be checked if the identifiability conditions hold for the typical ranges of the real-world variables. In addition, these can be added to the simulation study where the identified parameters have real-world meaning, explaining certain intervention effects.\n* Even though I understand what you mean by the \"DAG structure\", I think the graph considered here is not well defined. The variables do not affect each other as demonstrated in Figure 1, they affect each other's time derivatives. The nodes in the graphs in Figure 1 represent two things at the same time: the variable states and the time derivatives.\n* What happens if you increase the system dimensionality in the simulation study? Currently, it is set to $d=3$.", "limitations": "* The main limitation seems to be the verification of the identifiability conditions in practical scenarios, as noted by the authors.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper extends the identifiability conditions of the previous work from fully-observable systems to partially-observable systems, which have practical value in real-world scenarios.\n* The paper is very well-written and easy-to-follow despite being a theoretically heavy.", "weaknesses": "* The paper motivates the identifiability analysis with latent variables by its importance to practical scenarios in causal inference. Yet, the examples in the paper and its simulation setup are far away from being practical.\n* In lines 142-143, the DAG assumption for the latent relationships is motivated as being common in causality studies. However, these studies only consider a static setup. From the provided references, it is not possible to see how feasible this assumption is for an ODE system.\n* The simulation setup seems to be contrived, where the parameter means ($|x| \\in {0,1,2}$) are chosen by hand with small uniform perturbations, $U(-0.1, 0.1)$ and $U(-0.3, 0.3)$. It is hard to say how much randomness this scenario creates. The simulation study would support the claim better if its setup shows more randomness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. We have addressed each of your comments as follows. Additionally, we have revised our manuscript in accordance with your suggestions, and we believe that the quality has been significantly enhanced as a result of your insightful input.\n\n## Answers to weaknesses:\n\n>**W1:** Provide practical examples.\n\n**A:** Thank you for your comment. Since both this and your first question (Q1) address the practical value of our paper, we will respond to them together here.\n\nIn response to your suggestion, we have included two real-world linear ODE examples. Due to the 6000-character limit, detailed descriptions of these two models are provided in the following comment, and the corresponding causal graphs are presented in the uploaded PDF file. As illustrated by the graphs, the structure of these models aligns well with our structural constraints.\n\nRegarding the applicability of the proposed identifiability condition and real-world dataset experiments, we kindly refer you to our detailed response to Q1 from Reviewer LnTR, which addresses this query comprehensively. Thank you for your understanding.\n\n>**W2:** Feasibility of the latent DAG assumption.\n\n**A:** Thank you for your comment. In response to your second question, we believe there may be some confusion regarding how the causal graph is defined within the context of an autonomous (time-invariant) ODE system. To address this, we will first clarify the causal graph in the context of an ODE system, which we hope will provide a clearer understanding of the feasibility of the latent DAG assumption.\n\nTo enhance understanding, consider a general fully observable time-invariant ODE system  $\\dot{\\boldsymbol{x}}(t) = f(\\boldsymbol{x}(t))$. In such systems, the derivatives of state variables $\\dot{\\boldsymbol{x}}(t)$ do not explicitly depend on time $t$. For an ODE system like this, we state there is a direct causal relationship from variable $x_j$ to variable $x_i$ if $\\dot{x_i}$ is dependent on $x_j$, expressed as $\\dot{x}_i(t)= f(x_j(t))$. As detailed in [24], the causal graph for such an ODE system is defined such that **each node represents a variable $x_i$, and there is a direct edge from $x_j$ to $x_i$ if and only if $\\dot{x}_i$ depends on $x_j$**. This causal graph remains invariant over time in a time-invariant ODE system. Both ODE systems (2) and (3) in our manuscript are time-invariant, and the graphs in Figure 1 are well-defined within this context.\n\nWe assert that the latent DAG assumption is reasonable for several reasons:\n1.  Consider a particle of mass $m$ in a uniform gravitational field where the gravitational field exerts a constant force $F$ on the particle. The evolution of the particle's velocity (denoted as $v$) and position (denoted as $r$) can be described by a linear time-invariant ODE system:\n\n    $$ \\dot{v}(t) = F/m, \\dot{r}(t) = v. $$\n\n    The corresponding causal graph of this ODE system is $ v \\rightarrow r$, which is a DAG. Hence, a DAG is a reasonable causal structure to describe ODE systems.\n\n2. Since we focus on time-invariant ODE system analysis, the causal graph remains invariant with respect to time. Therefore, treating the causal graph as static and making a DAG assumption is a natural extension of traditional static causal studies.\n\n3. Deriving identifiability conditions for causal models is a challenging problem. This is why the DAG assumption is adopted in static setups, even in fully observable cases. Similarly, deriving the identifiability conditions for linear ODE systems with hidden variables is difficult, and this field of study is still in its early stages. To our knowledge, our work is the first to systematically derive identifiability conditions for such ODE systems. Referring to well-established assumptions in traditional causal studies is a prudent starting point. Additionally, we allow for cycles and self-loops among observable variables and only assume that latent variables follow a DAG structure, which is relatively less restrictive compared to the classic DAG assumption in static setups. Without the latent DAG assumption, it is currently not feasible to derive identifiability conditions for linear ODE systems with hidden confounders.\n\n>**W3:** Regarding randomness of the simulation study.\n\n**A:** Thank you for your comment. Upon reviewing our uploaded codes in the supplementary material, you will see that all parameters in our simulation study are entirely randomly generated. Specifically, we used `randint(-2,3)` to generate the parameters for simplicity. \n\nWe chose to set the initial parameter values close to the true parameter values with uniform perturbations $U(-0.1,0.1)$ and $U(-0.3,0.3)$ because the Nonlinear Least Squares (NLE) loss function associated with our simulation is non-convex. Initializing the parameters close to the true values helps the NLE converge to the true global minimum. Introducing more randomness into the initial parameter values would necessitate using a computationally intensive and time-consuming global optimization technique or another parameter estimation method.\n\nThe primary objective of our simulation is to validate the proposed theoretical results rather than check or develop parameter estimation methods or techniques for ODEs. We believe that the current simulation results robustly support our theoretical claims. However, to address your concern and further substantiate our findings, we have included two additional higher-dimensional cases.\n\n## Answers to questions:\n\n>**Q1:** Provide practical examples.\n\n**A:** Please refer to our response to W1.\n\n>**Q2:** Explain causal graph for an ODE system.\n\n**A:** Please refer to our response to W2.\n\n>**Q3:** Increase dimension in simulations.\n\n**A:** Thank you for your question. Due to the 6000-character limit, we kindly refer you to our response to W2 from Reviewer Te42, as it addresses the same query in detail. We appreciate your understanding.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0dqWSEUybj", "reviewer": "Reviewer_Te42", "summary": "This paper studies the problem of identifiability analysis of linear Ordinary Differential Equation (ODE) systems. It focuses mainly on the scenarios where some variables in the system remain latent to the learner. This paper aims to address this challenge by studying identifiability analysis in two classes of linear ODE systems with latent confounders. The first case considers latent confounders that are mutually independent, and the second case includes correlated latent confounders. The authors conduct detailed identifiability analyses for both systems and propose sufficient identification conditions. Simulation results support the theoretical findings.", "questions": "- How does the agent obtain the matrix $G$ from the latent DAG assumption? Could the author elaborate on this?", "limitations": "The authors have adequately addressed the limitations of the paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and clearly organized. The authors clearly stated all the necessary assumptions.\n- Identifiability analysis is an important problem in causal inference and control theory. Most of the existing methods in causal inference literature focus on the acyclic systems without feedback loops. On the other hand, methods in control theory often assume there are no unobserved confounders in the system. This paper attempts to close this gap by studying causal identification in linear ODE systems with latent confounders. It could have a significant impact across disciplines, including AI, econometrics, and environmental science.", "weaknesses": "- This paper is dense, and it could be difficult for readers unfamiliar with ODE analysis. It could be recommended that the authors could include an additional table summarizing the notations. Also, a table summarizing the identification conditions would also be helpful.\n- Simulations are performed on relatively simple synthetic instances. It would be interesting to see how the result scale to a more complex system.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. We have addressed each of your comments as follows. Additionally, we have revised our manuscript in accordance with your suggestions, and we believe that the quality has been significantly enhanced as a result of your insightful input.\n\n## Answers to weaknesses:\n\n>**W1:** Include tables that summarize the notations and identifiability conditions.\n\n**A:** Thank you for your valuable suggestion. In accordance with your recommendation, we have included a table to summarize all the notations (please refer to the following comment) and another table to summarize all the proposed identifiability conditions (available in the uploaded PDF file). We believe that these additions significantly enhance the clarity and readability of our manuscript.\n            \n>**W2:** Scale the simulations to a more complex system.\n\n**A:** Thank you for your comment. We would like to clarify that, theoretically, our proposed identifiability conditions are applicable to any finite dimensions $d \\geqslant 1$ and $p \\geqslant 1$. In practical scenarios, as the system dimensions increase, the complexity of the system also escalates. Consequently, larger sample sizes and more advanced parameter estimation methods may be required to achieve satisfactory parameter estimates.\n            \nIn response to your suggestion, we have included additional simulation examples with increased complexity in our updated manuscript. For single trajectory identifiability validation (Theorems 4.1 and 4.2), we have added an example with $d=5, p=5$ (i.e., 5-dimensional observable variable and 5-dimensional latent variables, totalling 10 dimensions). For $p$ trajectory identifiability validation (Theorems 4.3 and 4.4), we have added an example with $d=10, p=5$ (i.e., 10-dimensional observable variables and 5-dimensional latent variables, totalling 15 dimensions). The results of these simulations are presented in the uploaded PDF file and provide empirical evidence supporting the validity of our proposed identifiability conditions in more complex systems.\n\n## Answer to question:\n\n>**Q1:**  Elaborate on how the agent obtains matrix $G$.\n\n**A:** Thank you for your question. As discussed in our manuscript, the identifiability of matrix $G$ is established under the conditions stated in Theorem 4.3 and Theorem 4.4. The proof of Theorem 4.3, detailed in Appendix B.4, provides a comprehensive derivation of how matrix $G$ can be obtained. Due to the complexity and extensive use of notations, it is challenging to fully elaborate on this proof in a few plain sentences.\n\nTo summarize, obtaining matrix $G$ requires not only the latent DAG assumption but also the assumptions B2, B3, and B4 outlined in Theorem 4.3. A crucial aspect of the latent DAG assumption is  that it allows the matrix $G\\in \\mathbb{R}^{p\\times p}$ to be permuted into a strictly upper triangular form, such that $G^k = 0$ for all $k \\geqslant p$, where $p$ is the dimension of the latent variables. Consequently, the states of hidden variables can be expressed as polynomial functions of time $t$, from which the identifiability conditions are derived. \n            \nWe hope this brief explanation provides some insight into the latent DAG assumption. For a detailed derivation of matrix $G$, we encourage you to refer to our proof in Appendix B.4.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nmOajlusEi", "reviewer": "Reviewer_LnTR", "summary": "This paper provides the identifiability analysis of linear Ordinary Differential Equation (ODE) systems, particularly in scenarios where latent variables interact with the system. In detail, it investigates two specific cases. In the first scenario, latent confounders do not exhibit causal relationships, but their evolution follows specific functional forms, such as polynomial functions of time. The analysis is then extended to a second, more complex scenario, where hidden confounders have causal dependencies described by a Directed Acyclic Graph (DAG). The authors perform a series of simulations to substantiate their theoretical results.", "questions": "How practical are the proposed assumptions in the paper? Can the authors discuss their validity and design experiments to test their validity for real datasets?", "limitations": "See the weaknesses above.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper makes a significant contribution by extending the understanding of identifiability in linear ODE systems to include cases with latent variables, thereby enhancing the reliability of causal inferences in more complex systems. The simulated experimental results provide strong support for the theoretical findings, making this a robust and valuable study in the field.", "weaknesses": "1. The symbols $x'$ and $A'$ in the paper are not clearly defined. This may lead to confusion for readers when understanding the derivation process and the results. It is recommended to clearly define and explain these symbols in the paper.\n2. The assumptions and theorems lack intuitive explanations. It would be better to provide some intuitive explanations or examples after each assumption and theorem to help readers better understand the essence of these theories and their roles in practical applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. We have addressed each of your comments as follows. Additionally, we have revised our manuscript in accordance with your suggestions, and we believe that the quality has been significantly enhanced as a result of your insightful input.\n\n## Answers to weaknesses:\n\n>**W1:** Explain the symbols $\\boldsymbol{x}_0'$ and $A'$.\n\n**A:** Thank you for pointing this out. The symbols $\\boldsymbol{x}'_0$ and $A'$ are first introduced in our paper in Definition 2.1. To enhance clarity, we have included an explanation in the subsequent paragraph, which reads as ``We use $\\boldsymbol{x}_0'$ and $A'$ to distinguish other system parameters from the true system parameters $\\boldsymbol{x}_0$ and $A$; $\\boldsymbol{x}_0'$ and $A'$ can represent any $d$-dimensional initial conditions and any $d\\times d$ parameter matrices, respectively.\"\n\n>**W2:** Provide intuitive explanations for the assumptions and theorems.\n\n**A:**  Thank you for your valuable suggestion. We have provided intuitive explanations for each assumption as they are introduced. The added explanations are as follows (_please refer to **Table 1** in the uploaded PDF file for the summary of the proposed conditions_):\n\n1.  **For condition **A1** in Theorem 3.1 and condition **B1** in Theorem 4.1:** These conditions are the same as the one stated in Lemma 2.1 for the fully observable ODE system (1), but with a different initial condition $\\boldsymbol{\\beta}$ or $\\boldsymbol{\\gamma}$. We have added the intuitive explanation of this condition in the paragraph following Lemma 2.1, since it is where this condition was first introduced. The added explanation reads: \n\n    \"From a geometric perspective, the set of vectors stated in Lemma 2.1 being linearly independent indicates that the initial condition $\\boldsymbol{x}_0$ is not contained in an $A$-invariant proper subspace of $\\mathbb{R}^d$. Intuitively, this means the trajectory of this system started from $\\boldsymbol{x}_0$ spans the entire $d$-dimensional state space. That is, our observations cover information on all dimensions of the state space, thus rendering the identifiability of the system.\"\n\n2. **For condition **C1** in Theorem 4.2:** The added explanation reads: \n    \n    \"Under the latent DAG assumption, we can transfer the ODE system (3), which includes hidden confounders, into a $(d+p)$-dimensional fully observable ODE system (1) through the augmented state $\\boldsymbol{y}(t)$. Condition **C1** indicates that our observations span the entire $(d+p)$-dimensional state space, thus rendering the system identifiable.\"\n\n3. **For conditions **B2, B3, B4** in Theorem 4.3:** Condition **B2** is the same as **B1**, for which the explanation has been added. Conditions **B3, B4** are straightforward and do not require additional intuitive explanations.\n\n4. **For condition **C2** in Theorem 4.4:** This condition is the same as **C1**, for which the explanation has been added.\n            \nIn addition, we have included a table to summarize all the notations (please refer to our comment to reviewer Te42) and another table to outline the proposed identifiability conditions (available in the uploaded PDF file). We believe these additions significantly enhance the clarity and readability of our manuscript, aiding readers in better understanding the essence of our proposed theories.\n\n## Answer to question:\n\n>**Q1:** Practical validity of the proposed theoretical results.\n>\n**A:** Thank you for your question. The primary assumption in our manuscript is the **latent DAG** assumption, which is standard in causality studies. For a detailed discussion on the practicality and reasonableness of this assumption, please refer to our response to W2 from Reviewer 1wF2.\n\nIn addition to the **latent DAG** assumption, the other assumptions are quite mild:\n1. **Conditions in Theorem 3.1, 4.1, 4.2:** These conditions are both sufficient and **necessary** and cannot be relaxed further in our linear ODE setup. As we have mentioned conditions **A1** and **B1** align with the condition stated in Lemma 2.1, which is the set of vectors $\\\\{\\boldsymbol{x}_0, A\\boldsymbol{x}_0, \\ldots, A^{d-1}\\boldsymbol{x}_0\\\\}$ being linearly independent (denote it as condition **A0**). This condition is **generic** as noted in [40], meaning that the set of system parameters that violate this condition has Lebesgue measure zero. Intuitively, condition **A0** is satisfied for **almost all** combinations of $\\boldsymbol{x}_0$ and $A$. Once condition **B1** is satisfied, then one can always find observations on the same trajectory satisfying condition **C1**, rendering condition **C1** also mild.\n2. **Conditions in Theorem 4.3 and 4.4:** These conditions are sufficient but not necessary. Identifying parameter matrices $B$ and $G$ requires additional observations (trajectories) and assumptions. Condition **B2** and **C2** mirror the mild conditions **B1** and **C1**, respectively. Condition **B3** is experimentally controllable and trivially satisfied, while condition **B4**, similar to **A0**, is **generic**. Thus, all proposed conditions are practical and not restrictive.\n \nRegarding real-world dataset experiments, we were unable to include them due to the unavailability of suitable data. However, we have added two real-world linear ODE examples (see the comment to Reviewer 1wF2) and additional higher-dimensional simulation cases to better support our theoretical results. The corresponding simulation results are presented in the uploaded PDF file. These simulations align with current studies on theoretical identifiability of linear ODE or SDE systems, which also **rely solely on simulations** (see [27, 33, 40, 41]).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to express our gratitude to all the reviewers for their thoughtful and constructive feedback on our manuscript. Below, we summarize the modifications made to the manuscript based on your comments:

- To Reviewer 1wF2:
    - We have added two real-world linear ODE examples that align well with our interested ODE (2) and (3).
    - We have provided a clear defination of the causal graph in the context of autonomous (time-invariant) ODEs.
    - We have included two higher-dimensional simulation examples.

- To Reviewer Te42
    - We have included a table summarizing all the notations.
    - We have added a table summarizing all the proposed identifiability conditions.
    - We have included two higher-dimensional simulation examples.

- To Reviewer LnTR
    - We have provided explanations for symbols $\boldsymbol{x}_0'$ and $A'$.
    - We have included intuitive explanations for each of our assumptions as they are introduced.
    - We have added two real-world linear ODE examples.

We deeply appreciate the reviewers' insights, which have been invaluable in refining our work.

Meta-review: This paper provides an analysis of identifiability in linear ODE systems incorporating hidden confounders. The reviewers appreciated the theoretical contributions, and the authors provided extensive new experiments in the rebuttal phase in response to reviewer concerns. Authors: please incorporate the promised additions to the paper and revise the paper according to the reviewer suggestions.
Decision: Accept (poster)