Generator Identification for Linear SDEs with Additive and Multiplicative Noise
Authors: Yuanyuan Wang
Xi Geng
Wei Huang
Biwei Huang
Mingming Gong
Venue: NeurIPS 2023 poster (2023)
Keywords: Linear SDE
Identification
Causal inference

Abstract: In this paper, we present conditions for identifying the generator of a linear stochastic differential equation (SDE) from the distribution of its solution process with a given fixed initial state. These identifiability conditions are crucial in causal inference using linear SDEs as they enable the identification of the post-intervention distributions from its observational distribution. Specifically, we derive a sufficient and necessary condition for identifying the generator of linear SDEs with additive noise, as well as a sufficient condition for identifying the generator of linear SDEs with multiplicative noise. We show that the conditions derived for both types of SDEs are generic. Moreover, we offer geometric interpretations of the derived identifiability conditions to enhance their understanding. To validate our theoretical results, we perform a series of simulations, which support and substantiate the established findings.

Generator Identification for Linear SDEs with Additive and Multiplicative Noise
Yuanyuan Wang
Xi Geng The University of Melbourne geng@umimelb edu. au
The University of Melbourne yuanyuanv20student www.meeeennttttnb.aeaaemmmmm
unimelb. edu. au
Wei Huang
Biwei Huang
The University of Melbourne
University of California, San Diego bih007 Sacdd.ucddde.e edu
wei huangounimelb edu au
Mingming Gong
The University of Melbourne mingming gongdunimell edu. au
Abstract
In this paper, we present conditions for identifying the generator of linear stochastic differential equation (SDE) from the distribution of its solution process with a given fixed initial state. These identifiability conditions crucial in causal inference using linear SDEs as they enable the identification of the cestiintervention distributions from its observational distribution. Specifically, we derive sufficient and necessary condition for identifying the generator of linear SDEs with additive noise, as well as a sufficient condition for identifying with multiplicative noise. We show that the conditions derived for both types of SDEs are generic. Moreover, we offer geometric interpretations of the derived identifiability conditions to enhance their understanding. To validate our theoretical results, we perform a series of simulations, which support and substantiate the established findings.
generator linear SDEs
Introduction
Stochastic differential equations (SDEs) are powerful mathematical tool for modelling dynamic systems subject to random fluctuations. These equations are widely used in various scientific disciplines, including finance [ 30, 40), physics [53] [3] biology 8 61] and engineering
[18 44 (55. In recent years, SDEs have garnered growing interest in the machine learning research
community. Specifically, they have been used for tasks such as modelling time series data [19
and estimating causal effects [5 36 47].
To enhance understanding we first introduce the SDEs our interest, which are multidimensional linear SDEs with additive and multiplicative noise, respectively. Consider an m-dimensional standard Brownian motion defined on a filtered probability space (2, IP, (Fe)), denoted by W := (W, = [W......, Wm.l
< oo). Let X, R be the state time and let xo E Rd be a constant vector denoting the initial state of the system, we present the forms of the aforementioned two linear SDEs.
"Corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

1. Linear SDEs with additive noise.
dX, AX,dt + GdW,, Xo = 20 =0
(I)
where 0 & t < 0o, A E pddd and G E _dx= are some constant matrices.
2. Linear SDEs with multiplicative noise.
dX.- AX,dl >k=1 G&X,dWa.e Xo - 2o,
where <<< 00, G2 E Raxd for k = m are some constant matrices.
Linear SDEs are wildly used in financial modeling for tasks like asset pricing, risk assessment, and portfolio optimization [ [0] [4]. Where they are used to model the evolution of financial variables, such as stock prices and interest rates. Furthermore, linear SDEs are also used in genomic research, for instance, they are used for modeling the gene expression in the yeast microorganism Saccharomyces Cerevisiae [17. The identifiability analysis of linear SDEs is essential for reliable causal inference of dynamic systems governed by these equations. For example, in the case of Saccharomyces Cerevisiae, one aims to identify the system such that making reliable causal inference when interventions are introduced. Such interventions may involve deliberate knockout of specific genes to achieve optimal growth an organism. In this regard, identifiability analysis plays a pivotal role in ensuring reliable predictions concerning the impact of interventions on the system.
Previous studies on identifiability analysis of linear SDEs have primarily focused on Gaussian 35 42 These studies are typically based on observations located on one trajectory of the system and thus require restrictive identifiability conditions, such as the ergodicity of the diffusion or other restrictive requirements on the eigenvalues of matrix A. However, in practical applications, multiple trajectories of the dynamic system can often be accessed [5 45 54). In particular, these multiple trajectories may start from the same initial state, e.g., in experimental studies where repeated trials or experiments are conducted under the same conditions [9 12 26 29] or when the same experiment is performed on multiple identical units [48). To this end, this work presents an identifiability analysis for linear SDEs based on the distribution of the observational process with given fixed initial state. Furthermore, our study is not restricted to Gaussian diffusions (I). but also encompasses linear SDEs with multiplicative noise ( Importantly, the conditions derived for both types of SDES are generic, meaning that the set of system parameters that violate the proposed conditions has Lebesgue measure zero
diffusions, as described by the SDE 11) 6 16 23 28
Traditional identifiability analysis of dynamic systems focuses deriving conditions under which unique set of parameters can be obtained from error-free observational data. However, our analysis of dynamic systems that are described by SDEs aims to uncover conditions that would enable unique generator to be obtained from its observational distribution. Our motivation for identifying generators of SDEs is twofold. Firstly, obtaining unique set of parameters from the distribution of a stochastic process described by an SDE is generally unfeasible. For example, in the SDE (I). parameter G cannot be uniquely identified since one only identify GG based on the distribution of its solution process 7 28 Secondly, the identifiability of an SDE's generator suffices for reliable causal inferences for this system. Note that, in the context of SDEs, the main task of causal analysis is to identify the post-intervention distributions from the observational distribution. As proposed in [17], for an SDE satisfying specific criteria, the post-intervention distributions are identifiable from the generator of this SDE. Consequently, the intricate task of unraveling causality can be decomposed into two constituent components through the generator. This paper aims to uncover conditions under which the generator of a linear SDE attains identifiability from the observational distribution. By establishing these identifiability conditions, we can effectively address the causality task for linear SDEs.
In this paper, we present a sufficient and necessary identifiability condition for the generator of linear SDEs with additive noise ( along with a sufficient identifiability condition for the generator of linear SDEs with multiplicative noise
2 Background knowledge
In this section, we introduce some background knowledge of linear SDEs. addition, we provide a concise overview of the causal interpretation of SDEs, which is critical aspect of understanding the nature and dynamics of these equations. This interpretation also forms a strong basis for the motivation of this research.

---

2.1 Background knowledge of linear SDEs
The solution the SDE can be explicitly expressed as (cf.
(5)):
X, == xt; = = xo + ! ell GdW.
Note that in the context of our study, the solution stands for the strong solution, refer detailed definition.
22 for its
In general, obtaining an explicit expression for the solution to the SDE is not feasible. In fact, an explicit solution can be obtained when the matrices A, GI,...,C. GR commute, that when
AGk = G and G = G.G,
holds for all k, = 1,.... m (cf. (25]) However, the conditions described in are too restrictive and impractical. Therefore, this study will focus on the general case of the SDE (
We know that both the SDE [ and the SDE (2) admit unique solutions that manifest as continuous stochastic processes 22 A d-dimensional stochastic process is collection of -vdllleed random variables, denoted as X {X,; 0 < < x defined on some probability space. When comparing two stochastic processes, X and X, that are defined the same probability space F,, (PP,, various notions equality may be considered. In this study, we adopt the notion of equality with respect to their distributions, which is weaker requirement than strict equivalence, see [22 for relevant notions. We now present the definition the distribution of a stochastic process.
Definition 2.1. Let X be random variable measurable space (S, B(S),, i.e., the function X :Ω - S is distribution the random variable X is the probability measure px on (S, B(S)) given by
probability space F,P) with values in
Then, the
(x) P(X E =) E Ω E B}, BEB(S)
When x= (x;;0 < )o) is a continuous stochastic process (R, IP), and S = C'O, oo), such an X can be regarded as random variable on F,P) with values in (C(0, oo), B(C(O, oo))), and px is called the distribution of X. Here C(O, oo) stands for the space of all continuous, real- valued functions on [0, oo).
It is noteworthy that the distribution of continuous process can be uniquely determined by its finite-diment dinee--imensional distributions. Hence, if two stochastic processes, labelled X and X, share identical finite-dimensional distributions, they are regarded as equivalent in distribution, denoted by
X X. Relevant concepts and theories regarding this property be found in 22
The generator stochastic process is typically represented by a differential operator that acts on functions. It provides information about how function evolves over time in the context of underlying stochastic process. Mathematically. the generator of stochastic process X, can be defined as
(c)) lim Eif(X.-)) f(X)))
where f is suitably regular function.
In the following, we present the generator of the SDEs under consideration. Obviously, both the SDE ( and the SDE (2) conform to the general form:
dX;; b(X,)dl + (X.)dW, Xo 2o
(5)
where b and locally Lipschitz continuous in the space variable æ. The generator L of the SDE
5) can be explicitly computed by utilizing Itô's formula (cf. [50]).
Proposition 2.1. Let be stochastic process defined by the SDE 5). The generator L of X on
Cf (Rd) ) is given by
0f(x) 1 d
0 f(x)
(cf)))) = 1b(12
+ Σ
cis(2)
(6)
Ox:
dz,0x,
RJ=1
for I E CRR) and x E Rd, where c(x) o olx) (x) is d x d matrix, and CRR)) denotes the space of continuous functions on Rd that have bounded derivatives up order two.
3

---

2.2 Causal interpretation of SDEs
An important motivation for the identification of the generator of an SDE lies in the desire to infer reliable causality within dynamic models described by SDEs. In this subsection, we aim to provide some necessary background knowledge on the causal interpretation of SDEs. Consider the general SDE framework described as:
dx, = (d, Xo = To,
where Z is p-dimensional semimartingale and a Fd - dxd is a continuous mapping. By writing the SDE (7) in integral form
X - Ez= at (X,) i i d.
(8)
The authors of [17] proposed mathematical definition of the SDE resulting from intervention to the SDE (8) In the following, X denotes the (d 1)-dimensional vector that results from the removal of the I-th coordinate of X E R.d
Definition 2.2. [17 Definition 2.4.] Consider some < d and ç: R R. The SDE arising from 8 under the intervention X == [(X! is the (d - 1)-dimensional equation (Y(-))) = - + ΣE So by
JdZz itl,
where b : IRd-1 (dd-))xp defined by bis(y) azs a:y(y ... (Uy)....(((,,...... ya) for i + and p and the (()) is on the 1-th coordinate.
Definition 2.2 presents natural approach to defining how interventions should affect dynamic systems governed by SDEs. We adopt the same notations as used in [17]. Assuming ( and have unique solutions for all interventions, we refer to (8) as the observational SDE, to its solution as the observational process, to the distribution of its solution as observational distribution, to (9) as the inssinntervention SDE, to the solution of (9) the gost-intervention process, and to the distribution of the solution of (9) as the post-intervention distribution. The authors in [17] related Definition 2.2 to mainstream causal concepts by establishing mathematical connection between SDEs and structural equation models (SEMs). Specifically, the authors showed that under regularity assumptions, the solution to the post-intervention SDE is equal to the limit of a sequence of interventions in SEMs based on the Euler scheme of the observational SDE. Despite the fact that the parameters of the SDEs are generally not identifiable from the observational distribution, the post-intervention distributions can be identified, thus enabling causal inference of the system. To this end, Sokol and Hansen [17] derived condition under which the generator associated with the observational SDE allows for the identification of the post- interrvneenn distributions. We present the corresponding theory as follows.
Lemma 2.1. 117 Theorem 5.3.] Consider the SDEs
d.X. a(X,)dZe, Xo =0,
(10)
dx, = =(X:)(X)dd2, Xo = uo
(11)
where Z is a p-dimensional Lévy process and 2 is p-dimensional Lévy process. Assume that (T)) and (IT) have the same generator, that a .dd - Rdxp and ( : + IR are Lipschitz and that the initial values have the same distribution. Then the post-intervention distributions of doing x = ((X(-)) in [0p and doing X == ((x(-)) in [1 are equal any choice of Ç and
main task in the causality research community is to uncover the conditions under which the post-intervention distributions are identifiable from the observational distribution. In the context of dynamic systems modelled in SDEs, similar conditions need to be derived. Lemma 2. establishes that, for SDEs with Lévy process the driving noise, the iost-intervention distributions can be identifiable from the generator. Nevertheless, a gap remains between the observational distribution and the SDE generator's 's identifiability. This work aims to address this gap by providing conditions under which the generator is identifiable from the observational distribution.
3 Main results
In this section, we present some prerequisites first, and then we present the main theoretical results of our study, which include the condition for the identifiability of generator that is associated with the SDE (I) SDE 2 from the distribution of the corresponding solution process.

---

3.1 Prerequisites
We first show that both the SDE T and the SDE satisfy the conditions stated in Lemma 2.1
Lemma 3.1. Both the SDE ① and the SDE can be expressed the form of 110 with Z being
p-dimensional Lévy process, and a: R R xxp being Lipschitz.
The proof of Lemma 3.1 can be found in Appendix A..] This lemma suggests that Lemma 2.1 applies to both the SDE L and the SDE 2 given that they meet the specified conditions. Therefore, for either SDE, deriving the conditions that allow for the generator to be identifiable from the observational distribution is sufficient. By applying Lemma 2.1] when the intervention function is Lipschitz, the post-intervention distributions can be identified from the observational distribution under these conditions.
We then address the identifiability condition of the generator L defined by (
Proposition 3.1. Let C and È be generators of stochastic processes defined by the form of the SDE
(5) on Cj (IR)) where C is given by (6) and É is given by the same expression, with b(x) and o(x) substituted for b(x) and c(x). It then holds that the two generators = É and only if b(z)) = b(x) and c(x) = ((x) for all E Rdd.
The proof of Proposition 3.1 can be found in Appendix A.2 This proposition states that for stochastic
processes defined by the SDE (5. the generator is identifiable from functions associated with its coefficients: b(x)) and c(x) = (z) o
3.2 Conditions for identifying generators of linear SDEs with additive noise
Expressing the SDE T in the form given by (5) yields b(z) Ax and c(x) GG Therefore, based on Proposition 3.1 we define the identifiability of the generator of the SDE (I as follows.
Definition 3.1 ((ro, A, G--identifiabbiitty) For zo E IRd A E Raxdd and G E _dxm the generator of the SDE (T) is said to be identifiable from Zo. for all E Rdxd and all E Rdxm, with (A, GG * (A, GG it holds that (-; vo A. G ((= A,
In the following, we begin by introducing two lemmas that serve as the foundation for deriving our main identifiability theorem.
Lemma 3.2. For Do E IRd A,,,AA E Rddd and G, G E Rdxm, let X, == X(t; A,G). X, :=
X ((1; xo, A, G), then (:;; G, 4 X(; A, G) if and only if the mean B[X,]] TR [x and
the covariance [E{ - (e[x++))((X) ] E{((++ B[x++))(X, E[X.)] for all 0 < t < and 0 < h
The proof of Lemma 3.2 can be found in Appendix A.3 This lemma states that for stochastic processes modelled by the SDE (I. the equality of the distribution of two processes can be deconstructed as the equality of the mean and covariance of the state variables at all time points. Calculation shows
E E[X,] = Do
V(t,t h) = E[(x+h - - -EX
(12)
AlV())
where V(t): ; tt, Please refer to the proof A.5 of Theorem 3.4 for the detailed calculations. It can be easily checked that [E[X,] follows the linear ordinary differential equation (ODE)
mt() Am(t), m(0) Zo
(13)
where rt( denotes the first derivative of function m(() with respect to time t. Similarly, each column of the covariance V (t, + h) also follows the linear ODE (13) but with different initial state: the
corresponding column of (t). This observation allows us to leverage not only the characteristies of the SDE [D. but also the established theories
4 577 on identifiability analysis for the ODE [3 to
derive the identifiability conditions for the generator of the SDE ①
2x A, G) (X A, A,G) :00 60)
5

---

We adopt the same setting as in 443, discussing the case where A has distinct eigenvalues. Because random matrix theory suggests that almost every E I2d has d distinct eigenvalues with respect to the Lebesgue measure on Rdxd 43 And the Jordan decomposition of such matrix A follows a straightforward form which helpful for deriving the geometric interpretation of the proposed identifiability condition. The Jordan decomposition can be expressed as A = QAQ where
J1
A ax
i = 1,..., K1: if = K + 1..... K.
A
with Ju
bb
JK]
bk
ak
Q 1Q11..... 1xx]
[1i]] (oll.... [ull....u]
Qk
{
Uk.
if I..... K1,
[U2X- K1 -1 |22--x] k,].
if k K1
K
where 02 is a real eigenvalue of A and Up is the corresponding eigenvector of λς. for k 1...., K1.
For k: K1 + 1...., , [U2z-KK Ki |22--K K2 are the corresponding "eigenvectors" of complex eigenvalues ag + bgi. Inspired by Definition 2.3., Lemma 2.3.], we establish the following Lemma.
Lemma 3.3. Assuming A E Rdxd has d distinct eigenvalues, with Jordan decomposition A = -AQ-1. Let Tij E Rd and 9/ == Q-17g IR for all I...., ,....,7 n. with 2. We define
7i.k E TR. WHsh =
for h = 1, 1,..., K1 for h: K1 + +1..... K,
(f5,2t Kii--KK-11 112k- K;) E R.?
where 7i. denotes the k-th entry of 73. rank([rn| (Aml... [A n1l.. ... mmlmAml ... |Ad-1 < d
if and only if there exists k E (l...., K}, such that [wy, for all j = I,..., n, where (wj, is
the absolute value of f.r k 1, ..... K1, and the Euclidean norm of wor k = K1 11,tl.... K.
The proof of Lemma 3.3 can be found in Appendix decomposed into linear combination of Qs'S
A.4 From a geometric perspective, Y3 can be
7/ = Qh = ΣK k= QKIGA
Let L5 := (Qan According to [43] Theorem 2.2], each La is an A-invariant subspace of Rdd Recall that space L is called A-invariant, if for all 7 E L, Ay E L. We say L is a proper subspace of Rd if L C Rd and L. * IR.. If |w,, = 0 (i.e., Wj, = 0 in IR or R22,, then % does not contain any information from Lu. In this case, Yj contained in an A-invariant proper subspace of T&d that excludes Lx,, denoted as L-k- It is worth emphasizing that L_ C RRd is indeed a proper subspace of Idd This further implies that the trajectory of the ODE (3) gen- crated from initial state Y, is confined to L-k [57] Lemma 3.2]. Lemma 3.3 indicates that (ank(n)|A7)/)) 1A*-?r/l... /ns/An/l.....
< d then all Y for = I,.... n are confined to an A-invariant proper subspace of T&d, denoted as L. Therefore,
trajectories of the ODE (1) generated from initial states 7, are also confined to I.. Furthermore, based on the identifiability conditions proposed in [57], the ODE (13) is not identifiable from observational data collected in these trajectories. This lemma provides approach to interpreting our identifiability conditions from a geometric perspective.
Now we are ready to present our main theorem.
Theorem 3.4. Let To E RRd be fixed. Assuming that the matrix A in the SDE 1 has d distinct eigenvalues. The generator of the SDE (1) is identifiable from 3o if and only
rank([zo]Aann ... |o.. 1A H. AH |A H.a) d, (14)
where H GG and H. standsfor the j-th column vector of matrix H. for all j = =, d.
The proof of Theorem 3.4 can be found in Appendix A.5 The condition in Theorem 3.4 is both sufficient and necessary when the matrix A has distinet eigenvalues. worth noting that almost every A E Rdxd has d distinct eigenvalues concerning the Lebesgue measure on Rdxd, Hence, this condition is both sufficient and necessary for almost every in dddd However, cases where A has repetitive cigenvalues, this condition is solely sufficient and not necessary.

---

Remark. The identifiability condition stated in Theorem 3.4 is generic, that is, let
S = ((2u, A, A,G) R ddd+d+dmm condition
14 is violated),
has Lebesgue measure zero in 1gd+d++dm Refer to Appendix B.2 for the detailed proof.
From the geometric perspective, suppose matrix has distinct eigenvalues, the generator of the SDE (I) is identifiable from zo when not all of the vectors: Ep, H ..... H.a confined to A-invariant proper subspace IR A key finding is that when all the vectors H. j = 1,.... d are confined to an A-invariant proper subspace L of R4, each column of the covariance matrix ((t) in Equation (12) also confined to L., for all
f < oo. Thus, the identifibiilty of the generator of the SDE (1)
be fully determined by zo and the system parameters (A, GGT). Further details can be found in the Aroof of Theorem 3.4
By rearranging the matrix in the identifiability condition can also expressed as
rank([rroAAa) ... | "zolGG" |AGG" ! - d.
(15)
Based on the identifiability condition (5).. we derive the following corollary.
Corollary 3.4.1. Let zo E be fixed. If rank([G]AG]] ... - GJ) then the generator of the SDE ① is identifiable from
The proof of Corollary 3.4.1 can be found in Appendix A.6 This corollary indicates that the generator of the SDE (I) is identifiable from any initial state E R when the pair [A, G] is controllable (rank( [G] AG| A = d). Notably, this identifiability condition stricter than that proposed
Theorem 3.4 as it does not use the information of zo.
3.3 Conditions for identifying generators of linear SDEs with multiplicative noise
Expressing the SDE in the form given by yields 6(x) = Ar and (xx) = [G ... |Gm 2| e Rdxm, thus, c(x) a (z) = Dk= k=1 GLE G Let (t; mo. A, [G denote the solu-
tion to the SDE then based on Proposition 3.1 we define the identifiability of the generator of the
SDE (2) as follows.
Definition 3.2 ((zo, A, (G )m
\ -identifiiabilityy) For zo E IR Red,A A, G₁ E Bdxd for all k 1,.... the generator of the SDE ( said be identifiable from Fy. if for all A, Gg E =Rdxd, there exists E Rdd, such that LA, Gyaæ GT) # (A, GgEx G ). it hhlls that
X (5500 A, (G+) 7 X(550, A, &.xx))
Based on Definition 3.2 we present the identifiability condition for the generator of the SDE (2). Theorem 3.5. Let xa Rdd be fixed. The generator of the SDE (2 identifiable from To if the following conditions are satisfied:
Al rank([zo|AAro) ... AAd-1 200) d,
A2 rank([v|Au).. IA +2-2)/200) (el d)/2.
where A = A & E && @ denotes Kronecker sum and @ denotes Kronecker product, is di -d-mmensionnl vector defined by vec(sor)) where vec(M) denotes the vectorization of matrix M.
The proof of Theorem 3.5 can be found in Appendix A.7 This condition is only sufficient but not necessary. Specifically, condition A1 guarantees that matrix A is identifiable, and once A is identifiable, condition A2 ensures that the identifiability of GAæ G holds for all x: E Rdd.
Remark. The identifiability condition stated in Theorem 3.5 generic, that is, let
S = { A, (G+ Hm E (gd+(m+1)dt either condition Al or A2 in Theorem 3.5 5 is violated).
S has Lebesgue measure zero in Rd+(m+1)d This signifies that the conditions are satisfied for most of the combinations of ro. A and G3 's except for those that lie in a set of Lebesgue measure zero The corresponding proposition and detailed proof can be found in Appendix B.]
7

---

Since obtaining an explicit solution for the SDE 2 is generally infeasible, we resort to utilizing the first- and second-order moments of this SDE to derive the identifiability conditions. Let m(t) == [[X,] and P(t) == E X,", it known that these moments satisfy ODE systems. Specifically, m(t) satisfies the ODE 13 while P(t) satisfies the following ODE (cf. [56]):
P(t) = At(t) + P(t)A + E G&P(t)GT P(O) 1o40
(16)
An important trick to deal with the ODE
16 to vectorize P(t), then it can be expressed as:
vec( (()) Avecc veci P(O))
(17)
where A and u are defined in Theorem 3.5 In fact, the ODE 117 follows the same mathematical
structure as that of the ODE (33) which is known homogeneous linear ODEs. Thus, in addition to the inherent properties of the SDE (), we also employ some existing identifiability theories for
homogeneous linear ODEs to establish the identifiability condition for the generator of the SDE (
From the geometric perspective, condition A1 indicates that the initial state To not confined to A-invariant proper subspace of R [57] Lemma 3.1.]. And condition A2 implies that the vectorization of z is confined an A-invariant proper subspace of W, with W C B& and dim(W)= = (d2 + d)/2, where dim (W) denotes the dimension of the subspace W, that is the number of vectors in any basis for W. In particular, one can construct basis for W as follows:
(vec(En1) vec E21), vec( (Fzz), vec) Ead)
where Eig denotes a d x d matrix whose ij-th and ji-th elements are 1. and all other elements are 0, for all j I...., d and i > j. Refer to the proof A.7 of Theorem 3.5 for more details.
4 Simulations and examples
In order to assess the validity of the identifiability conditions established in Section 3 results of simulations. Specifically, we consider SDEs with system parameters that either satisfy or violate the proposed identifiability conditions. We then apply the maximum likelihood estimation (MLE) method to estimate the system parameters from discrete observations sampled from the corresponding SDE. The accuracy of the resulting parameter estimates serves as an indicator of the validity of the proposed identifiability conditions.
we present the
Simulations. We conduct five sets of simulations, which include one identifiable case and one unidentifiable case for the SDE (T), and one identifiable case and two unidentifiable cases with either condition A1 or A2 in Theorem 3.5 unsatisfied for the SDE 2 We set both the system dimension, d, and the Brownian motion dimension, m. 2. Details on the true underlying system parameters for the SDEs can be found in Appendix C We simulate observations from the true SDEs for each of the five cases under investigation. Specifically, the simulations carried out for different numbers of trajectories (N), with 50 equally-spaced observations sampled on each trajectory from the time interval [0, 1]. We employ the Euler-Maruyama (EM) method (34]. widely used numerical scheme for simulating SDEs, to generate the observations.
Estimation. We use MLE 38 50 to estimate the system parameters. The MLE method requires knowledge of the transition probability density function (pdf) that governs the evolution of the system. For the specific case of the SDE ( the transition density follows Gaussian distribution, which can be computed analytically based on the system's 's drift and diffusion coefficients (cf. (50)). To compute the covariance, we employ the commonly used matrix fraction decomposition method 4 49) 50)
However, in general, the transition pdf of the SDE 2 cannot be obtained analytically due to the lack of a closed-form solution. To address this issue, we implement the Buler-Maruyama. approach 32 34), which has been shown to be effective in approximating the transition pdf of SDEs.
Metric. We adopt the commonly used metric, mean squared error (MSE), to assess the accuracy of the parameter estimates. To ensure reliable estimation outcomes, we perform 100 independent random replications for each configuration and report the mean and variance of their MSEs.
Results analysis. Table T and Table 2 present the simulation results for the SDE 1 and the SDE
(2). respectively. In Table the simulation results demonstrate that in the identifiable case, as the number of trajectories N increases, the MSE for both and GGT decreases and approaches zero However, in the unidentifiable case, where the identifiable condition (14) stated in Theorem 3.4 is not
8

---

Table 1: Simulation results of the SDE (
Identifiable
Unidentifiable
MSE-A
MSE-GG
MSE-A
MSE-GG
0.0117 = 0.0115
5.28E-05 + 4.39E-05
3.66 = 0.10
0.05 + 0.03
10
0.0063 + 0.0061
2.39E-05
1.82E-05
3.88 = 0.06
0.64 + 0.59
20
0.0029 + 0.0027
1.87E-05 + 1.51E-05
3.70 8.006
0.09 + 0.07
50
0.0013 + 0.0010
8.00E-06
5.68E-06
3.76 2.760000
0.11 0.08
100
0.0007 + 0.0004
4.34E-06 + 2.70E-06
3.66 1: 0.02
2.09 + 1.98
satisfied, the MSE for both A and GG remains high regardless of the number of trajectories. These findings provide strong empirical evidence supporting the validity of the identifiability condition proposed in Theorem 3.4 The simulation results presented in Table 2 show that in the identifiable case, the MSE for both and GSæ decreases and approaches zero with the increase the number of trajectories N. Here, Gar
k== GyÆr"' T Gg. where x is randomly generated vector from R2 (in these simulations, x [1.33, 0.72] Interestingly, even in unidentifiable case 1, the MSE for both A and G&E decreases with an increasing number of trajectories N, indicating that the generator of the SDE utilized in this particular case is still identifiable, although larger number of trajectories is required compared to the identifiable case to achieve the same level of accuracy. This result is reasonable, because it aligns with our understanding that condition Al I is only sufficient but not necessary for identifying A, as the lack of an explicit solution for the SDE (2) results in condition A1 not incorporating any information from G₂ 's. The identifiability condition derived for the SDE (I) in Theorem leverages the information of G, similarly. if information regarding G 's is available, a weaker condition for identifying A could be obtained. For illustration, in Appendix E we present such a condition assuming the SDE 22 has a closed-form solution. In the case of unidentifiable case 2, the MSE for decreases with an increasing number of trajectories N: however, the MSE for Gsr remains high, indicating that A is identifiable, while GSæ is not, albeit requiring more trajectories compared to the identifiable case achieve the same level of accuracy of A (since the Gsr is far away from its true underlying value). This finding is consistent with the derived identifiability condition, as condition Al is sufficient to identify A, whereas condition A2 governs the identifiability of Gaa. Worth noting that in cases where neither condition A1 nor condition A2 is satisfied, the estimated parameters barely deviate from their initial values, implying poor estimation of both and Gar. These results indicate the validity of the identifiability condition stated in Theorem 3.5
Table 2: Simulation results of the SDE 2
Unidentifiable
Identifiable
casel: A1-False, A2-True
case2: A1-True, A2-False
N
MSE-A
MSE-GGr
MSE-A
MSE-Gsr
MSE-A
MSE-Gst
10
0.069 1: 0.061
0.3647 1 0.3579
0.509 1: 0.499
0.194 1: 0.140
2.562 1 2.522
9763 1 8077
20
0.047 1: 0.045
0.1769 0.1694
0.195 1 0.180
0.088 0.058
0.967 1 0.904
8353 1 6839
50
0.018 + 0.018
0.1703 + 0.1621
0.132 + 0.131
0.081 + 0.045
0.423 = 0.410
4779 + 4032
100
0.006 + 0.006
0.0015 + 0.0012
0.065 = 0.065
0.068 + 0.036
0.207 = 0.198
3569 = 3150
500
0.001 0.001
0.0004 0.0001
0.008 0.008
0.059 0.004
0.046 = 0.046
4490 + 3991
Illustrative instances of causal inference for linear SDEs (with interventions). To illustrate how our proposed identifiability conditions can guarantee reliable causal inference for linear SDEs, we present examples corresponding to both the SDE (T) and the SDE ( these examples, we show that under our proposed identifiability conditions, the post-intervention distributions are identifiable from their corresponding observational distributions. Please refer to Appendix D.I and D.2 for the details of the examples.
5 Related work
Most current studies on the identifiability analysis of SDEs are based on the Gaussian diffusion processes that conform to the form described in the SDE I In particular, the authors of [27 28 42

---

have conducted research on the identifiability or asymptotic properties of parameter estimators of Gaussian diffusions in view of continuous observations of one trajectory, and have highlighted the need for the diffusion to be ergodic. A considerable amount of effort has also been directed towards the identifiability analysis of Gaussian diffusions, relying on the exact discrete models of the SDEs 16, 23 (35] 41] Typically, these studies involve transferring the continuous-time system described in the SDE I to a discrete-time model such as a vector autoregressive model, based on equally- spaced observations sampled from one trajectory, and then attempting to determine conditions under which (A, GG is identifiable from the parameters of the corresponding exact discrete models. These conditions often have requirements on eigenvalues of A among other conditions, such as requiring the eigenvalues to have only negative real parts, or the eigenvalues to be strictly real. Due to the limitation of the available observations (continuous discrete observations located on one trajectory of the SDE system), the identifiability conditions proposed in these works are restrictive.
Causal modelling theories have been well-developed based on directed acyclic graphs (DAGs), which do not explicitly incorporate a time component [39]. In recent years, similar concepts of causality have been developed for dynamic systems operating in both discrete and continuous time. Discrete-time models, such as autoregressive processes, can be readily accommodated within the DAG-based framework [13 14. On the other hand, differential equations offer natural framework for understanding causality in dynamic systems within the context of continuous-ting processes 52). Consequently, considerable effort has been devoted to establishing theoretical connection between causality and differential equations. In the deterministic case, Mooij et al. [37] and Rubenstein et al. [66] have established a mathematical link between ODEs and structural causal models (SCMs). Wang et al. [60] have proposed a method to infer the causal structure of linear ODEs. Turning to the stochastic case, Boogers and Mooij have built bridge from random differential equations (RDEs) to SCMs (7), while Hansen and Sokol have proposed a causal interpretation of SDEs by establishing connection between SDEs and SEMs [17].
Conclusion and discussion
In this paper, we present an investigation into the identifiability of the generators of linear SDEs under additive and multiplicative noise. Specifically, we derive the conditions that are fully built on system parameters and the initial state zo. which enables the identification of linear SDE's generator from the distribution of its solution process with given fixed initial state. We establish that, under the proposed conditions, the cost-intervention distribution is identifiable from the corresponding observational distribution for any Lipschitz intervention Ç.
The main limitation of our work is that the practical verification of these identifiability conditions poses a challenge, as the true underlying system parameters are typically unavailable in real-world applications. Nevertheless, our study contributes to the understanding of the intrinsic structure of linear SDEs. By offering valuable insights into the identifiability aspects, our findings empower researchers and practitioners to employ models that satisfy the proposed conditions (e.g., through constrained parameter estimation) to learn real-world data while ensuring identifiability. We believe the paramount significance of this work lies in providing systematic and rigorous causal inter- pretation of linear SDEs, which facilitates reliable causal inference for dynamic systems governed by such equations. It is worth noting that in our simulations, we employed the MLE method to estimate the system parameters. This necessitates the calculation of the transition pdf from one state to the successive state at each discrete temporal increment. Consequently, as the state dimension and Brownian motion dimension increase, the computational time is inevitably significantly increased, rendering the process quite time- consumin.. To expedite parameter estimation for scenarios involving high dimensions, alternative estimation approaches are required. The development of a more efficient parameter estimation approach remains an important task in the realm of SDEs, representing promising direction for our future research. We claim that this work does not present any foreseeable negative social impact.
Acknowledgements
YW was supported by the Australian Government Research Training Program (RTP) Scholarship from the University of Melbourne. XG was supported by ARC DE210101352. MG was supported by ARC DE210101624
10

---

References
[I] O. O. Aalen, K. Røysland, J. M. Gran, and B. Ledergerber. Causality, mediation and time: a dynamic viewpoint. Journal of the Royal Statistical Society: Series A (Statistics in Society). 175(4))831-8611 2012.
[2] I. J. Allen. An introduction to stochastic processes with applications to biology. CRC press, 2010.
[3] I. P. Arribas, C. Salvi, and L. Szpruch. Sig-sdes model for quantitative finance. In Proceedings of the First ACM International Conference on Al in Finance, pages 1-8, 2020.
[4] P. Axelsson and F. Gustafsson. Discrete-time solutions to the continuous-time differential Iyapunov equation with applications to kalman filtering. IEEE Transactions on Automatic Control, 60(3):632- 643, 2014.
[5] A. Bellot, K. Branson, and M. van der Schaar. Neural graphical modelling in continuous-time: consistency guarantees and algorithms. In International Conference on Learning Representa- fions, 2021.
[6] J. R. Blevins. Identifying restrictions for finite parameter continuous time models with discrete time data. Econometric Theory, 33(3):739-754, 2017.
[7] S. Bongers and J. M. Mooij. From random differential equations to structural causal models: The stochastic case. arXiv preprint arXiv:1803.0m 883.08884, 2018.
[8] C. A. Braumann. Introduction stochastic differential equations with applications modelling in biology and finance. John Wiley & Sons, 2019.
[9] A. P. Browning, D. J. Warn, K. Burrage, R. E. Baker, and M. J. Simpson. Identifiability analysis for stochastic differential equation models in systems biology. Journal of the Royal Society Interface, 17(173):2020055, 2020.
[10] M. Capinski and E. Kopp. The Black-Schole Model. Cambridge University Press, 2012.
[11] G. C. Chow. Optimum control of stochastic differential equation systems. Journal of Economic Dynamics and Control, 122):143-175, 1979.
[12] Collins, Howard, and J. Leitner. Quantifying the reality gap in robotic manipulation tasks. In 2019 International Conference on Robotics and Automation (ICRA), pages 7706-6712 IEEE, 2019.
[13] M. Eichler. Granger causality and path diagrams for multivariate time series. Journal of Econometrics, 137(2):334-353, 2007.
[14] M. Eichler and V. Didelez. On granger causality and the effect of interventions in time series. Lifetime data analysis, 16:3-32, 2010.
[15] A. L. Goldberger, L. A. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley. Physiobank, physiotoolkit, and phys- ionet: components of a new research resource for complex physiologic signals. circulation, 101(23)=e215e220) 2000.
[16] L. P. Hansen and T. Sargent. The dimensionality of the aliasing problem in models with rational spectral densities. Econometrica: Journal of the Econometric Society, pages 377-387, 1983.
[17] N. Hansen and A. Sokol. Causal interpretation of stochastic differential equations. Electronic Journal of Probability, 19:1-24, 2014.
[18] D. Henderson and P. Plaschko. Stochastic Differential Equations In Science And Engineering (With Cd-rom). World Scientific, 2006.
[19] lacus al. Simulation and inference for stochastic differentiall equations: with R examples. Springer, 2008

---

[20] M. Jacobsen. Homgeneous Gaussian Diffiusions in Finite Dimensions. Preprint, Institute of Mathematical Statistics University of Copenhagen, 1991.
[21] Jia and A. R. Benson, Neural jump stochastic differential equations. Advances in Neural Information Processing Systems, 32, 2019.
[22] I. Karatzas, S. Shreve, and S. E. Shreve. Brownian motion and stochastic calculus. Springer Science Business Media, 1991.
[23] M. Kessler and A. Rahbek. Identification and inference for multivariate cointegrated and ergodic gaussian diffusions. Statistical inference for stochastic processes, 7:137-151, 2004.
[24] Klein. Pricing black-scholes options with correlated credit risk. Journal of Banking & Finance, 201)):111-1229, 1996.
[25] P. E. Kloeden and E. Platen. Numerical Solution of Stochastic Differential Equations. Springer Berlin, Heigelberg, 1992.
[26] Kügler. Moment fitting for parameter inference in repeatedly and partially observed stochastic biological models. PLOS ONE, 7:1-15, 2012.
[27] Y. A. Kutoyants. Statistical inference for ergodic diffusion processes. Springer Science & Business Media, 2004.
[28] A. Le-Breton and M. Musiela. Some parameter estimation problems for hypoelliptic homoge- neous gaussian diffusions. Banach Center Publications, 1(16):337-356, 1985.
[29] R. Levien and S. Tan. Double pendulum: An experiment in chaos. American Journal of Physics, 61(11):1038-104, 1993.
[30] L. Lima and L. Miranda. Price dynamics of the financial markets using the stochastic differential equation for a potential double well. Physica A: Statistical Mechanics and its Applications, 490:828 833, 2018
[31] L. Lu, P. Jin, G. Pang. Z. Zhang, and G. E. Kariiadakis. Learning nonlinear operators via deep- onet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-225 2021.
[32] X. Mao. The truncated euler-maruyane method for stochastic differential equations. Journal of Computational and Applied Mathematics, 290:370-384, 2015.
[33] M. C. Mariani and O. K. Tweneboah. Stochastic differential equations applied to the study of geophysical and financial time series. Physica A: Statistical Mechanics and its Applications, 143:170-178, 2016.
[34] G. Maruyama. Continuous markov processes and stochastic equations. Rendiconti del Circolo Matematico di Palermo, 4:48-90 90, 1955.
[35] Mccrorie. The problem of aliasing identifying finite parameter continuous time stochastic models. Acta Applicandae Mathematicae, 79(1-2):9, 2003.
[36] S. W. Mogensen, D. Malinsky, and N. R. Hansen. Causal learning for partially observed stochastic dynamical systems. In UAI, pages 350-360, 2018.
[37] J. M. Mooij, D. Janzing, and B. Schölkopf. From ordinary differential equations to structural causal models: the deterministic case. In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, pages 440-448, 2013.
[38] J. N. Nielsen, H. Madsen, and P. C. Young. Parameter estimation in stochastic differential equations: an overview. Annual Reviews in Control, 24:83 94, 2000.
[39] Pearl. Causality. Cambridge university press, 2009.
[40] Phillips. The structural estimation of stochastic differential equation system. Econometrica:
Journal of the Econometric Society, pages 1021-1041, 1972.
12

---

[41] P. C. Phillips. The problem of identification in finite parameter continuous time models. Journal of Econometrics, 144):351-32, 1973.
[42] A. Prior, M. Kleptsyna, and P. Milheiro-Olliverr, On maximum likelihood estimation of the drift matrix of a degenerated o-u process. Statistical Inference for Stochastic Processes, 20:57-78, 2017.
[43] X. Qiu, T. Xu, B. Soltanalizadch, and H. Wu. Identifiability analysis of linear ordinary differential equation systems with single trajectory. Applied Mathematics and Computation, 430:127260, 2022.
[44] S. Rong. Theory of stochastic differential equations with jumps and applications: mathematical and analytical techniques with applications to engineering. Springer Science & Business Media, 2006.
[45] Y. Rubanova, R. T. Chen, and D. K. Duvenaud. Latent ordinary differential equations for irreuulary-samppled time series. Advances in neural information processing systems, 32, 2019.
[46] P. Rubenstein, S. Bongers, B. Schölkopf, and J. Mooij. From deterministic ODEs to dynamic structural causal models. In 34th Conference on Uncertainty Artificial Intelligence (UAI 2018), pages 123. Curran Associates, Inc., 2018.
[47] Sanchez and S. A. Tsaftaris. Diffusion causal models for counterfactual estimation. In Conference Causal Learning and Reasoning, pages 647-668. PMLR, 2022.
[48] K. Sankaran, R. Perez, and K. Jata. Effects of pitting corrosion on the fatigue behavior of alu- minum alloy 7075-16: modeling and experimental studies. Materials Science and Engineering: A, 297((12):2223-2) 229, 2001.
[49] S. Särkkä et al. Recursive Bayesian inference stochastic differential equations. Helsinki University of Technology, 2006.
[50] S. Särkkä and A. Solin. Applied stochastic differential equations. Cambridge University Press, 2019.
[51] K.-i. Sato. Stochastic integrals with respect to lévy processes and infinitely divisible distributions. Sugaku Expositions, 27(1):19-42 2014.
[52] B. Schölkopf, F. Locatello, Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612 634, 2021.
[53] Z. Schuss. Singular perturbation methods stochastic differential equations of mathematical physics. SIAM Review, 22(2):1 11(2):119-155, 19-155, 1980.
[54] I. Silva, G. Moody, J. Scott, L. A. Celi, and R. G. Mark. Predicting in-hospital mortality of icu patients: The physionet/compputng in cardiology challenge 2012. In 2012 Computing in Cardiology, pages 245-248. IEEE, 2012.
[55] K. Sobezyk. Stochastic differential equations: with applications to physics and engineering. Springer Science & Business Media, 2001.
[56] L.. Socha. Linearization methods for stochastic dynamic systems. Springer Science & Business Media, 2007.
[57] S. Stanhope, J. E. Rubin, and Swigon. Identifiability of linear and linear-in parameters dynamical systems from single trajectory. SIAM Journal on Applied Dynamical Systems, 13(4):1722-8815, 2014.
ineearri-parrmeteers
[58] N. G. Van Kampen. Stochastic differential equations. Physics reports,
1976.
[59] Vatiwutipong and N. Phewchean. Alternative way to derive the distribution of the multivariate umstein-uhlenbee process. Advances in Difference Equations, 2019(1):1-7, 2019.
13

---

[60] Y. Wang, W. Huang, M. Gong, X. Geng, T. Liu, K. Zhang, and D. Tao. Identifiability and asymptotics in learning homogeneous linear ODE systems from discrete observations. arXiv preprint arXiv:2210.05955, 2022.
[61] D. J. Wilkinson. Stochastic modelling for quantitative description of heterogeneous biological systems. Nature Reviews Genetics, 10(2):122-13 133, 2009.
[62] M. Zakai and J. Snyders. Stationary probability measures for linear differential equations driven by white noise. Journal of Differential Equations, 8(1):27-33, 1970.
14

---

Appendix for "Generator Identification for Linear SDEs with Additive and Multiplicative Noise"
A Detailed proofs
A.1 Proof of Lemma 3.1
Proof. We start by presenting the mathematical definition of Lévy process. (cf. (51)
Definition A.1. A stochastic process X == (X, : 0 & t < oo) is said to be a Lévy process if it
satisfies the following properties:
1. Xo = O almost surely;
2. Independence of increments: For any
< 1 < 122 <.
<ha < xc, X12 Xxy X12 X
X,, Xt.- are independent;
3. Stationary increments: For any < t, X, X, is equal in distribution to Xt-si
4. Continuity in probability: For any e > 0 and 0 < t < oo it holds that lim,; [P(hh x 0.
the following, we first show that the SDE T can be expressed as the form (O). with Z being p-dimensional Lévy process and Rd Ra being Lipschitz. The first equation in the SDE can be rearranged as
dX: AX,dt GdW,
dt
[AX. G] dW,
(18)
=d(X:)dZe:
with
a(X,) = [AX,
G] E (dd(m)1))
and
dt dW.
Oixi 0mx 1
01xm
doo, dW.
dZ,
+
(19)
0mxi 1
Imxm
where Dix denotes an i x zero matrix, let W = (We 0 < t < oo} with W, = (W,,, Wie....., W ( denote a (m + -)-dimensional standard Brownian motion, then one can find a process Z = (2, 0 < 20) with
Z, rt EW:
(20)
Zo = 0,
satisfying dZ, described Equation (19). Then we will show that the process 7 descrihed in 20 a Lévy process, that is, it satisfies the four properties stated in Definition A.1
Property 1: The first property readily checked since Zo 0.
Property 2: For any < t < 12 < ts < 00,
Zi - Zz, = r((2 - -t) + E(We, - We,)
1/2 - 11]
0
0mx1
+
W Wt,
15

---

Similarly,
1/ 12
Zts Z1
+ Wss [e
0 mmx1
Since Wt, - We, and Wts - We, are independent, Ztg
Zea and Z₁ - Z,, are independent.
Property 3: when < 4,
t
Ze 2s
0 0mx1
+
We
Wo
NV
- s
01x1 0mx1
01xm
0mx1
(t 8) =m
And
Zi- r(t
s) + EWi-s
1
+ Wt-s
0mx1
[-8 s)
O1xi
01 Xmxm
N
0mx1
0mx1
(1 8) Fmxm
Therefore, property 3 is checked.
Property 4: Obviously, process Z described in (20 is continuous with probability one at é for all 0 / xc, therefore, Z has continuity in probability.
Now that we have shown that process a p-dimensional Lévy process with p = m + 1. Then we will show that a(X,) [AX, G] is Lipschitz.
|| a a(X.) ||I =Il [A(X, -xs) 0] I|f
=|| A(X, -x,) ||2
<|| A ||||| Xx- Xx ll2
where || M |e denotes the Frobenius norm of matrix M and |l2 denotes the Fuclidean of vector Now it is readily checked that function Rdd - Rd is Lipschitz.
Similarly, we will show that the SDE can also be expressed the form of (TO). with Z being p-dimensional Lévy process and R Raxx being Lipschitz. Let us rearrange the first equation the SDE 2
m
dX, = AX,dt Σ
GxX,dWk.t
dW1,
[AX, G,X,
GmXi]
LdW m.t_
a((x)ddZz
Since the dZ, here has the same form as that of the SDE (T. we use the same process Z described in Equation 20 which has been shown to be Lévy process.
As for the function a(X)). a(X,) a( (X.)) ||r =Il [A(X, XX,)
Gi(X, -X.)
Gm Xx)l l|r
< A(X, -X)) 11 + Σ || Gx -X)
km
< A || X, X, 1|2 +
Gx ||||| X, X, ll2
1=1
-(11 ||F Σ
GR |F
1| X, 1|z.
16

---

it is readily checked that function a R - Raxp is Lipschitz.
A.2 Proof of Proposition 3.1
Proof. For the backward direction, when b(x) = b(x) and c(x) = Ō(x) for all E R it obviously that (Lf)(z) = (z)) for all f E C% (TR.) and æ E Rd, that is L L.
For the forward direction, since (Lf)(z1) = (a1) for all f E (R()) and 21 e RR.
We first set
[(x) = Tp:
where Tp denotes the p-th component of variable It is readily checked that
= bx(x))=
for all E Rd and 1....,d.. d. As result.
b(x) = b(x), for all x E RRd
Then we set
f(x) (xp Tip) rig), where 2ip denotes the p-th component of æ1. It is readily checked that
Cpg (x1) (x1),
for all Rd and p.q 1,.... d. Consequently,
c(x) Ō(x), for all x E IRd.
A.3 Proof of Lemma 3.2
Proof. For the forward direction, since
((500, A, G) - ((;00. A, A,G),
one has
B|X,] = E[X)), vo < 00 20.
Thus,
(X, E[X,)D0G - (X, - E[K
in particular, one has
((XX= E(X.) BI -(X,)) for all 0 & 1, co.
For the backward direction, we know that the solution of the SDE (I) is Gaussian process. The distribution of a Gaussian process can be fully determined by its mean and covariance functions. Therefore, the two processes have the same distribution when the mean and covariance are the same for both processes for all 0 & f, h < Co.
A.4 Proof of Lemma
Proof. For the forward direction, since
rank(/7 |Aal | 1al...l/./ |..1A(-)))))) < d,
then for all / = .l......................... 4n] E R",
rank [[3 [5] ... .........111111 -A- 18) d,
where B = 1171 +... + In. Consequently, the corresponding ODE system
*(t) Az(t),
(21)
20) = 3
17

---

is not identifiable from 3 by [57] Theorem 2.5.], where ftt) denotes the first derivative of x(t) with respect time t.
Let
: R
and
Br E Ri wk =
for k = I,.... K1 for k = K1 + 1, K.
(32t-Ki B2k-Ki) K1) 1 E RR,
Simple calculation shows that
=Q -4-( +... 4a7)
-1 + hinn
therefore, one has
= 1111 +... Wn,k, for all &s E {1,.... K).
(22)
By 43 Theorem 2.4], we know that for any E IR", there always exists & E {l,..., such that wk (E R- or RR) since the ODE (2T) is not identifiable from initial state 3. Next, will
show that this result is satisfied only when there exists a le such that Wi.k = 0 (E R or R?) for all
j ...... n. Let us rearrange the Equation (22 as
W1,1
Wn.1
W1
W1,K
Wn,K
WK
assume for any E [1,..., K). wikk.... Wp,s] T * O, then there always exists a E IR" such that we & for all k {1...., K.. The reason is that under this circumstance, for any k E {l...., K} the set such that W/ = has Lebesgue measure zero in IR.. Therefore, the set of l's such that there exists & such that wy 0 has Lebesgue measure zero in R". This result creates
contradiction. Thus, there must exist k, such that [W1,k.... & 0, that | = 0 for all j == 1=
For the backward direction, there exists such that |w,.el for all j I,..., n, that is Wi.K (E g¹ or I22) for all j I,..., Simple calculation shows that
k-1 7/ = Q9, Σ p=1
K
pp + Σ pot1
Opwip
and
T3 = QA'Q-1Q1 '7
QA
Σ Qndfwjo
Σ p=k+1
recall that
Ak
k = 1,... K
Jk =
Okk
- bb
if l: Ki 1,.... K.
by
ak
Then matrix
M In An/
|A ((d-\/al1
71l hm/ . ( mml
= Q-+C,
where
Q- = - 1.0
18

---

and matrix C denotes:
11,1
J1W1,1
J J W111
Wn,1
Juwn,1
J
1 Wn
:
wi,k-1
1
J 11
wnkk-1
J
d k
Wokk-1
w1,-+1
JK+1
k+1
w1,++1
wn,t+1
J
x+1
Wn,
Wi.K
JK
Jk w1.K
wn.K
JKWNnK
Jk Wn.K
We know that
rank(M) (+n)) & min( When & fl....,KKK Q_x E Rxxx(d-)) (d-1), and rank(Q d 1, while when E {K} 1...., K}, Q-x E [Rdx(d-2) and rank(Q = d In both cases, rank(Q-+)) < d, thus vank(M) < d.
A.5 Proof of Theorem 3.4
Proof. Let Ā E Rdxd and & E Rddmm such that ( G, - ((; Å,O), we denote as
X X. For simplicity of notation, in the following, we denote A := A, A2 = A, G. = G and
G2 = G. and denote X X as X x².
Sufficiency. We will show that under the identifiability condition 14). one has (Ai, G,G. = G.G.).
We first show that H1 = H2 (H₁ == G,G,). Indeed, since X x² have the same distribution, one has
[[S(x))] [[X(
(23)
for all 0 < 0o and f C (Rd). By differentiating 23 at one finds that
(Fof)(zo) (C2))
(24)
where Li is the generator of X (i = 1,2). Based on the Proposition 2.1
05 (2O)
82
(C.f)(zo)= Σ> (A) (
(Hi)at
(oo)
Oak
Oxz
k=11=1
k.1=1
where (M) denotes the kl-entry of matrix M, and æos is the 1-th component of ro. Since (4 is true for all f. by taking
f(x) (x. [Tp xng).
it is readily checked that
(H1)pu (H2)pg
for all p. - I,..., d. As result, Hi H2. Let call this matrix H.
Next, we show that A₁ = A2. We first show the relationship between A, and no. and then show the relationship between A, and H. To this end, one first recalls that
x; e No + L:d G.dW.
Set mt(t) = E[X;), we know that (t) satisfies the ODE
m, (t) = Atm,(t), VO < m, (0) wo
(25)
where j(t) denotes the first derivative of function f(t) with respect to time f.
Simple calculation shows that
mt(t) Attzo-
19

---

Since X 4 x2, one has
E|
B|X
for 0 && 1 < co. That is Taking k-th derivative of Adtoo 2o with respect to t, one finds that
e e xo s0 4A 7
d*
eAt
xo AN
dt 1=1
for all k = 1,,,.... Consequently, Let us denote this vector A zg. Obviously, one gets
Af50 Al30
1 A₁ A -- 20 = A21 20 for all = 1,2,....
(26)
In the following, we show the relationship between A, and H. Let us denote
Yi = 1/ eillt = x; - E[x]]
and
= -xx)))
Simple calculation shows that
Vi(1+++)) = each S( He (t-s))
(27)
Vi(t)
where Vi(1) = Vi(1, 1).
Since X 4 X2, by Lemma 3.2 one has
(I, L + h) = (2((++1) YO 10,4,
To obtain information about A,, let us fix for now and take k-th derivative of 27 with respect to h.
One finds that
d
(28)
dh h-o
for all & = 1,2,.... On the other hand, the function (t( satisfies the ODE (56
(i(t) = A,Vi(t) + vi(t)A + H. OEt<O, (i(0) 0.
In partieular,
(i(0) A,V((0) + V.00), + H.
By differentiating 28 at t = 0, it follows that
d d
Vi(t, - h) = H,
dt
t=o t h=
for all & 1,2,.... ... Consequently,
A = A H.
Let us denote this matrix * H. Obviously, by rearranging this matrix, gets
A₁ A 1 = 1- H for all & 1,,, ........
(29)
Recall our identifiability condition is that rank (M) = d with
M [ro|Arol ... -.d1 2o H AMH ... A H
H. AHH.)
-.- H.H
(A--1
20

---

If we denote the j-th column in M as M. one gets
A M, J
for all j= 1....... dd by Equations 26) and
This means one can find a full-rank matrix B E R ddd by horizontally stacking d linearly independent columns of matrix M, such that A, A, Since B is invertible, one thus concludes that A1 A2. Hence, the sufficiency of the condition is proved.
Necessity. In the following, we will show that when A has distinct eigenvalues. The condition (4) stated in TheoremB.4pp also necessary. Specifically, we will show that when the identifiability condition (14) not satisfied, one can always find with (A, GG + (A, GGT) such that X X. Recall that for simplicity of notation, we denote A A, A2 Á, and denote X - X as x - X² where process X = {X; 0 & t 20), and X; (t;; zo. A, G) following the form described in the solution process (3). the following, we may use both and A interchangeably
according to the context.
By Lemma
3.2 to guarantee X x2 one only needs to show that
B| E[X|] E[X²], 4A & 00,0 (i1 (11, = 1224, 1O < 1,4 < x0.
That
Aitio Aulzo, V0 <00 ( (A))((), V0 < &,h < 20,
(30)
Vi(t) = 12((), YO <t<<<,
where (tt) = (t( V2(t)).
Recall that H GG For simplicity of notation, abusing notation bit, we denote H. To- Let i H.j for all j .d...,d,
and
{
E (H. H. H, x, x,) T R2, for K1 + K.
for k 1, Ki
Wj.k
When the identifiability condition 14 is not satisfied, that is
1A- H. H.HHH |A -d H.l H. A...A.HH...... AA- -.- H.l)) < d,
by Lemma
3.3 there exists is such that |wy.al = O, i... Wj,E = (E IR' R?), for 0,
Recall that
L' Alt-s) He (t-a) (t-a)
V(1) - Vi(1)
(' (e - 1(...11..... Hl (T (t-s)
(31)
Q S All
A Hal... .........HHHee HH (e-
1/" We (1-o)
Q
where
W c A(t-s) ( .H.a,
and some calculation shows that
(b- edil wa.1
W
e [x(( (x((()))
Wi.K
ed (t- W
21

---

recall that
Ak+
if k 1, K1,
Jk
- bk
if k = K1 + 1, I....,K. K
bb
ax.:
Since wi.k (E RR or R2), for all j = 0, l.... . d, then if ls E {I,..., K1). the k-th row of W
W4. 0;
and if (Ki + ,,... Kj, then the (2k K1 1 and the (2k: Ki) --t rows
W(2x K1-1) W(2x- (()-))). 0, where Wg. denotes the k-th row vector of matrix W.
we denote
Q(t),==Q-VV()) for all 1, I.....d,
and
w(t),A {
v()) E RR
for k: = 1,..., Ki for k = K₁ + 1,.... K
(V(t),,.s-ki11 K1 (()) E R?,
Then by multiplying Q-1 in both sides of Equation 31 one obtains that
w (E R or R?)
for 1, ...... and all <t< oo. This indicates that when all the vectors H., j..j=,..... confined A-invariant proper subspace of Rd. denotes as L, then each column of the covariance matrix (II) in Equation (30) is also confined to L, for all 0 < L < co. Therefore, under condition (14). Io. H., (for all j = 1...., d) and each column of the covariance matrix V (for all < (0) are confined an A-invariant proper subspace of Rd. Thus, matrix A2 exists, with A2 + A₁ such that the first two equations in Equation (30) are satisfied.
In partieular, by 43 Theorem 2.5], when f1.....
K1}, there exists matrix D E Rdxd with the kk-th element D&s = c 0 and all the other elements of D are zeros. Let
A₂ = A + QDQ- + A1.
then A, and A2 satisfy the first two equations in Equation (30). Then we will show that such a A2 also satisfy the third equation in Equation (30).
Some calculation shows that
(i(
Al(t-o) He (t-s)
[ Q- -ett-s)00
(32)
S'
ds,
where P = ====="((Q") And
/ A2(t-e) e) A (t-s)
(t(t)
(AL++DD (e( (1-s))
(33)
Pacl QT ds,
22

---

where Pg = e T
If one can show that P1 = P2, then it is readily checked
that (t(t) V2(t) for all t < co. Recall that
Q' H
where = [H.1)..... [...). And when condition
14 is not satisfied, the k-th row of H:
H 0.
Since
Pi = = --Q' H1
therefore, the k-tth row of P1: Simple calculation shows that matrix Pi symmetric, thus, the k-th column of Pi:
(Pi) = 0.
(Pi)
is easy to obtain that ((t-a) is a diagonal matrix expressed as
((t-s)
ce(c-s)
where (t(t-)) is the kk-th entry. Then, simple calculation shows that = D(t-s) Pie D(ts)) P
p== D(t 1
*H(0") (H(Q)))) -(t(t)s)))) D(t-s)
Therefore, one obtains that
Vi(1) V2(1), 4O 1 < oo. Hence, when & E {1,.... K1}. find Ag, with Ag + A such that Equation When & E {Ki + 1,..,,
30 is satisfied.
......,K, K). there exists matrix D' E Rdd with
D2R- 1,28 -12- D 22
D2u K1 1,2k
c1
C2
D' D22--K. 21
C3
C4
where Ma,j denotes the ij-th entry of matrix M. [0,, C3 ca) of D' are zeros. Let
T f O. and all the other elements
A A1 + QD'2- A1
then Ai and A2 satisfy the first two equations in Equation (30). Similar to the case where k {1..... K1}, one can also show that such A2 also satisfies the third equation in Equation (30).
Therefore, assuming has distinct eigenvalues, then when the identifiability condition (1] is not satisfied, satisfied, i.e., X1 d x², Hence, the necessity of the condition is proved.
always find A₂ with (A1, GGT) * (A2, GGT) such that Equation 30 is
A.6 Proof of Corollary 3.4.1
Proof. There are two ways to prove this corollary, we will present both of them in the following.
Wayl. By [62 Lemma 2.2].
|AGGT .A. = span([GGT |AGGGGIAAGTT J...
IA J... I),
where (M) denotes the linear span of the columns of the matrix M. therefore, when
rank(GG|AG)...
A AA = d,
then
[G..AA).. Edd
23

---

thus,
span([GG' |AGGT| ... ...1144 1.-11---GGG]] GGTT = R
Therefore,
rank([GG' |AGG
....IIA--GG" d,
since the rank of a matrix is the dimension of its span. Then by Theorem 3.4 the generator of the SDE ① is identifiable from z0.
Way2. Let A gaxd and G Rdxm such that (; A, A, _ X A, G), we denote as X X, we will show that under our identifiability condition GAT) (A, GG ). By applying the same notations used in the proof of Theorem 3.4 in the following, denote A = A, A2 = A, G1 = G and G2 == G.
In the proof of Theorem 3.4 we have shown that GjG = G2G thus, we only need to show that under the condition stated in this corollary, A A2. According to the proof of Theorem 3.4 for all & t < 0o, we have
(t) V2(t), (t) = A21((1).
Let V(()= = Vi(t)( = 1,2), one gets
A (t) = (t)( 4A Et<<on 00.
Therefore, if there exists 0 < f < oc, such that (t)) is nonsingular, then one can conclude that
A1 A2.
By 20 Theorem 3.2], the covariance V(t) nonsingular for all if and only if rank(GG (G) A A = d,
that is the pair [A, G] is controllable. Therefore, under the condition stated in this corollary, A = A2. thus the generator of the SDE (1) is identifiable from zn-
A.7 Proof of Theorem 3.5
Proof. Let A. Gk E Rdxd for all
1...., m. such that X(= A, (Gx) 4
((500o ,, {Gx) k= we denote as X d X, will show that under identifiability condi- tion, for all x E Rdd (A, Em GLA GT) (Á, Ex= Ggax" G For simplicity of notation, in
the following, we denote A1 A, Ag A, G1 = G2 and G2 = Gk, and denote X X as
X d x
We first show that A₁ Ag. Set m, (t) = E[X;], we know that mt(t) satisfies the ODE
m, (t, = (tm, VO <
(34)
(0) Fo
where ft) denotes the first derivative of function f(t) with respect to time t.
Simple calculation shows that
mt(t) Aitoo
Since x d X2, one has
E|X,] =[
for all 0 < co. That is
c Al' 100000 A2tra
6A < <t
Taking j-th derivative of e Art
As' 2o with respect to t, one finds that
d'
Ail xo
AQ
dti
1=1
24

---

for all j = 2,2..... Consequently,
A/20 Ag.0-
Let us denote this vector mo. Obviously, one gets
A,4 = A2A xo for all j = 2,.....
(35)
By condition A1, it is readily checked that A A2 from Equation (35)
In the following, we show that under condition A2, for all Rd
GIaTT G
Σ k=1
G2 G2,
We know the function (t( [ satisfies the ODE
P((() A,P.(() + P.(1) + Σ
G. P.))G 6 co:
(36)
1=1
P.(0) =o=o
Since X 4 X2,
(t(t) = Pa(t), 4A < < 00,
let us call P(t). By differentiating (t( one also gets that
(t) P2(t), 4A <<<<0
Since we have shown that A
A2 under condition A1, from Equation (36) one observes that
4A < / 00. G1, - C2sP(()EE.
-
(37)
By vectorizing P(t), some calculation shows that the ODE (36) be expressed
(e)()(()) Avee P())),
(38)
vec P(()) = vec(
with an explicit solution
vec( (ec(P(t)) el vec(raa)))
where A A A Σn - Gk Ga E xd and vec(M) denotes the vector by stacking the columns matrix vertically.
By definition,, P(() E pdxd is symmetric, thus vec(P(()) for all 0 < < oo is confined to proper subspace of edd let denote this proper subspace W, simple calculation shows that
dim(W) = (d2 + d)/2.
where dim(W) ( denotes the dimension of the subspace W, that the number of vectors any basis for W. In particular, one can find a basis of W denoting
(vec(E11), vec (E1),, rec(Ezz)..... vec(Ead)} (e
where Eg stands for a x d matrix, with the ij-th and ji-th elements are 1 and all other elements 0, for all i, 1....,d d and >
Suppose there exists t;'s for i 1..... (d d)/2, such that vec(P(tz)) ...,, vec ve((P(t((44a)2) are linearly independent, then for all x E Rd.
vec(xx T) = Livee + +... + ((2
that is
TT =1p(tt) +... ((000/2((((000001))
25

---

where == f4,.... (240)/2 E +d)/2 According to Equation 337. it is readily checked that for all x E IR4.
Σ Giaer T G
> G2 G 2,
By [57 Lemma 6.1], there exists (df + d2/2 tt's such that re.(P vet linearly independent, if and only if the orbit of vec(P(t)) (i.e., the trajectory of ODE (38 started from initial state v), denoting as r(A, v) with v vec( ), is not confined to a proper subspace of W.
Next, we show that under condition A2, orbit 7(A, v) is not confined to a proper subspace of W.
Assume orbit r(A, v) is confined to proper subspace of W. Then there exists wr & 0 E W such that
TT A = 0, vO & < x0.
By taking j-th derivative with respect to t, we have
A'e = =0 44 < 00, 0...., (d² 2)/2.
In particular, for t = 0, w A'v for ....., (d² d 2)/2.
Therefore,
w w [ul ... ( +d 21/2 =
(39)
Since E W, E (pen[vee(Ejng vec (21(E2)),.. vec (daa) set vec( = then : is d x d
symmetric matrix. Since P(t) is symmetric for all 0 < < co, according to Equation (36). simple calculation shows that the j-th derivative of P(t) is also symmetric for all 0 & 0o, for j = 0, 1..... Recall that
vec(P(()) eτro
vec(P()(1)) = A
where (1) denotes the j-th derivative P(t) with respect to In particular, when 0, one has vec(P(O)) = v,
vec vec(p))((0) -A',,
then if we denote matrix Ale by setting vec(AAr) = A'v, matrices A are symmetric for all j = 0, I.... Therefore, w, U. Av, A(f (dd2222))222 in Equation (39). Moreover, since these vectors correspond to d symmetric matrices, the repetitive elements in each vector appear in the same positions in each vector.
say that there are only (d + d)/2 distinct elements in each of the vectors:
Hence, we can focus on checking those distinct elements in each vector, that is Equation 39 can be expressed as
Au
(dee +d +d22)/2 =0,
(40)
where w E IR (df +d)/2 denotes as
2w
2wa. U22 20 ... Wad
where Wi denotes the i-jth element of w. with i, = 1,..., and i > j. When i A j. the element multiplied by a 22, that is, w only keeps the distinctive elements w, and for each of the repetitive element, we multiply v2. We define u, Au... using the same way.
Under condition A2, matrix
Av
LA +d 22/22 E IR
(22
is easily to be checked to be invertible, then = O, thus = O. This contradicts to w of 0, therefore. under condition A2, orbit (A4, v) is not confined proper subspace of W. Hence, the theorem is proved.
26

---

B Genericity of the derived identifiability conditions
B.1 The identifiability condition stated in Theorem 3.5 is generic
We will show that the identifiability condition stated in Theorem 3.5 is generic. Specifically, we will show that for the set of (zo, A, {Gx) E (dd(m+1)dd such that either condition A1 or A2 stated in Theorem 3.5 is violated, has Lebesgue measure zero in [ In the following, first present lemma we will use to prove our main proposition.
Lemma B.1. Let p : IR" + R be non-zero functinni Let Z == for E R" p(x) = 0}. Then Z has Lebesgue measure zero in IR".
Proof. When n = 1, suppose the degree is k > 1, then by the fundamental theorem of algebra, there most l =t's such that Z. Therefore, Z has Lebesgue measure zero, since finite set has measure zero in TR.
Suppose the lemma is established for polynomials in variables. Let be non-zero polynomial in n variables, say of degree in Fn, then we can write
p(x,
(Z)))
................................ In-1) and ....... . Pk are polynomials in the variables {.1,..., , and there exists j E (O....,Ke k} such that Py is non-zero polynomial since is non-zero polynomial. Then we can denote Z
where æ {o1......
In
Z = (x, a..) p(æ, Fn. = 0}.
Suppose (æ, .T.) E Z, then there are two possibilitics:
case po( = (x) = O.
case there exists i E (O....,k) ..) such that pa( # O.
Let
= ((x,0n) E Z case is satisfied) B ((x, 7 case is satisfied).
then Z AUUU B.
For case 1, recall that there exists j E {O....,k} such that pj is non-zero polynomial, let
Aj == for E Rn=1 p)(2)
0)
then by the induction hypothesis. A, has Lebesgue measure zero in Rn-1, Therefore, A, x R has Lebesgue measure zero R Since A C Aj x IR, A has Lebesgue measure zero in R="
For case 2, let A be Lebesgue measure on R", then
(BB =
(n(a,,,)d
R
1 Lx on daddx
(41)
.fmm ( 11 2x) dx.
where
{ 1, (x, B
LB(x, xx) = 0. if (x, B
The inner integral in Equation 41 is equal to zero, since for fixed there are finitely many (indeed,
most k) x, such that p(æ, F.)) = 0 under the condition case Thus, X"(B) = O, that is, B
has Lebesgue measure zero in g Then it is readily checked that Z has Lebesgue measure zero. D
27

---

Now we are ready to present the main proposition. Proposition B.1. Let
: ((ro, A, {G+) Re+(m+ either condition Al or A2 in Theorem 3.5 violated).
then S has Lebesgue measure zero in (d+(m=
Proof. Let
SA == ((x0, A) E Rd+d" condition Al is violated):
we first show that SA has Lebesgue measure zero in RRd Suppose (zo. SA, then (2o, A) satisfies
rank rank([zo)Axx)) ... 1.. mn)) d,
that is the set vectors (zo, Aro. Ad-1 xo} 10) are linearly dependent, this means that det [ro Axol [A4-11x0) O.
(42)
is simple matter of algebra that the left side of (42 can be expressed as some universal polynomial of the entries of and entries of A, denotes p( A) p(Zu1, ... Fud: 011, 012. ..2...... add), where Toj denotes the j-th entry of xo and ai denotes the ij-th entry of A. Therefore, one concludes that
p( A) p(ro1,,..... 3od, a11, 012,
add) 0.
Thus, SA be expressed as
SA = ((10, A) Id :((x, A) = =0}
Some calculation shows that
p A)
Σ
3o0 ... Foi, dettd((""") --.((((-))-))).))
(43)
where (M).j denotes the j-th column vector of matrix M. Obviously, p(zp, A) is a non-zero polynomial function of entries of To and entries of A, therefore, by Lemma B.1 SA has Lebesgue measure zero d Let
S (Fr, A, (G+)
(ddddm+ (m+ 1)d"
condition A1 is violated),
then it is readily checked that S₁ has Lebesgue measure zero in pd+
Let
S2 = (fro A, (G E (m+-1- condition A2 violated)
we then show that S2 has Lebesgue measure zero in (m+ Suppose (zo, A, ( S2. then (Fo, A, { satisfies
rank([u|Au)...... (dd
(d2 + dd/22
recall that = = A A A+ k= G5 & Gb E of x and == vec(Euz) Rdd According to the proof A.7 of Theorem 3.5 we obtain that the set of vectors {v, Au, AAd +d-2)/2) are linearly transferred from vectorizing d x d symmetric matrices, thus each of these vectors has only (d? + d)/ /2 distinct elements and the repetitive elements appear in the same positions all vectors. Hence, abuse notation little bit, we can focus checking those distinct elements in each vector, that is
dependent. Because all of these vectors
fe. (
(44)
where v E +d)/2 denotes the vector of deleting the repetitive elements of v. Since the set of vec- tors {E, Au,.... A +d-2)/2) are linearly dependent, the set of vectors {u]Ave] are linearly dependent, that
det([L) det([u)AAl.... ... dd 0.
(45)
28

---

Each entry of can be written a non-zero polynomial function of entries of since = vec(2000)) Each entry of A can be written as a non-zero polynomial function of entries of and G₁. with
k I..... m, since A = A @ + E Gk @ G3 E IR xd² Hence, the left side of Equation (45) can be expressed some universal polynomial of the entries of zo. A and G3 denotes
p A, (G+ = p(ro1,.... Fod, 011. Odd: .1. Gi.dd: ...d...... Gm, 1l....... Gm,daa,
where G3.ú5 denotes the ij-th entry of matrix Gk- Therefore, one concludes that
p(ro, A, (G+))==) = -.
Thus, S2 can be expressed
S =
A, A, E gd+(m+1)d p( A, (G+)==) 0}.
Similar to the calculation of p(zg. A) in Equation 443, p( A, {G can be expressed as non-zero polynomial function of entries of u and A, thus it can also be expressed as a non-zero polynomial function of entries of un, A and G ss. Therefore, by Lemma B.1 S2 has Lebesgue measure zero in Rd+(m+1))
We know that S S S1 U.S.S S2, let À be Lebesgue measure on (d+(++))+ then one has
X(S) ((Si) A(S2) 0.
Thus S has Lebesgue measure zero in (d++++1)dd
B.2 The identifiability condition stated in Theorem 3.4 is generic
We will show that the identifiability condition stated in Theorem 3.4 is generic. Proposition B.2. Let
S = (FF, A,G) G) 1 condition 14 in Theorem 3.4 violated),
then S has Lebesgue measure zero in +d
Proof. Suppose (xo, A,G) S. then (Zo. A, G) satisfies
rank([zo) Axol |........A__ |Ad 2o H. AH. ... - | ....|A.-IHH.. H.al) < d, recall that GGT, and H.j stands for the j-th column vector of matrix H, for all j 1, d.
Let
S' ( E TR (azol Amol (4--1
d),
one observes that 5 [ S². According to the proof of Proposition B.1 it is readily checked that S has Lebesgue measure zero in Rd Thus, S has Lebesgue measure zero in R 2
29

---

C Simulation settings
We present the true underlying system parameters along with the initial states of the SDEs employed in the simulation experiments. We randomly generate the true system parameters that satisfy or violate the corresponding identifiability conditions.
For the SDE
1. identifiable case: satisfy condition
14) stated in Theorem 3.4
1.87
1.76 0.98
-0.1
-0.11
-0.14
id = 0
A
G
0.98
0.229
-0.22
unidentifiable case: violate condition
(4):
[
2
0.11
0.22
00
A
G
-0.11
-0.22
For the SDE
1. identifiable case: satisfy both Al and A2 stated in Theorem 3.5
1.87 -0.98
1.76
-0.1
-0.11
-0.14
-0.17
0.59
10
A
0.98
G
- -0.99
-0.2
G
0.81
0.18
2. unidentifiable casel: violate Al satisfy A2:
[]
2
0.11
-0.14
-0.17
0.59
un-AI 0
qun-Al
G
Gun-A1
0
0.29
-022
0.81
0.18
unidentifiable case2: satisfy Al violate A2:
A2
[
Aun-A2_
[
-2] 0 0
Gun-A2
-0.3
0.4
Gun-A2
0.8
0.2
20
0.7
0.2
0.2
-0.4
We have discussed in Section 4 that we use MLE method to estimate the system parameters from discrete observations sampled from the corresponding SDEs. Specifically, the negative log-likelihood function was minimized using the 'scipy.optimize mmmtimizemmiiiie library in Python.
For all of experiments, initialized the parameter values the true parameters plus 2. In the case the SDE (I) we utilized the 'trust-constr' method with the hyper- arper-parameter "gtol'= = 1e-3 and 'xtol'= = 1e-3. On the other hand, for the SDE ( applied the 'BFGS' method and set the hyper-parameter 'gtol'= 1e-2. The selection of the optimization method and the corresponding hyper-parameters was determined through series of experiments aimed identifying the most suitable configuration.
30

---

D Examples for reliable causal inference for linear SDEs
D.1 Example for reliable causal inference for the SDE
This example is inspired by 17 Example 5.4]. Recall that the SDE is defined as
dX, AX,dt + GdW,, Xo = Zu.
where 0 < t 0R, A E Rdxd and G E _d are constant matrices, W is an m-dimensional
standard Brownian motion. Let X(t; zo. A, G) denote the solution to the SDE IT. Let A E Rdxd and G Rdxm define the following SDE:
dX, AX,dt + GdW, Xo = zu.
such that
( A, G) (1;00, A, G) Then under our proposed identifiability condition stated in Theorem 3.4 we have shown that the generator of the SDE (T) is identifiable, i.e., (A, ) (A, GGT) Till now, we have shown that
under our proposed identifiability conditions, the observational distribution identity the generator of the observational SDE. Then we will show that the post-intervention distribution is also identifiable. For notational simplicity, we consider intervention on the first coordinate, making the intervention X and X { for 0 < t < non-intervened coordinates (i.e., x(-1) and x(-1), note the superscripts do not denote reciprocals, but denote the (d -ocooddiinss without the first coordinate). Express the matrices of A and G in blocks
It will suffice to show equality of the distributions of the
Au
A12
G G2
A =
G=
A21
A23
where An E Rix1 Az2 E [(d-1)x(d-1), Gj E Rl and G2 E (R 1)xm Also, consider corresponding expressions of matrices Á and G. By making intervention X E, one obtains the post-intervention process of the first SDE satisfies:
ax(-") = (Az1E + A22 dt G2dW, x (-1) 0
which is multivariate OrnstiinUUlleneed process, according to [59 Corollary 1]. this process is
Gaussian process, assuming A22 is invertible, then the mean vector can be described as
E|x = Azat s/ (I Azat ) A21E.
And
and based on [59 Theorem 2), the cross-covariance can described as )) B[(X - E(X -
v(X 2th
Amatth- (T2(t+))
Similarly, one can obtain that the mean vector and cross-covariance of the distribution of the post- intervention process the second SDE by making intervention X; & satisfy:
E|X!
A x (-1) (I ) A A21E.
and
E(( - <F
V(X(-) Eth
= - A 5 G (t-(t ds
Then we will show that E(X!-1) E\X! and v(X(-1)
(Ah x(-1)) v(X!-1) , x((11) for all < f, h < oc. Recall that we have shown (A, GG = (A, GG thus, A22 = Aez and Aq1 Agi, then it is readily checked that E[X(- E|X, for all 0 < co.
31

---

Since
GG c G2
G G G
= GGT
thus, G2G. = G2GJ. then it is readily checked that V(X ( = 44h 0 & 1 < co. Since both of these two post-intervention processes Gaussian processes, according to Lemma 3.2 the distributions of these two post- -nsterntervnntoon processes are the same. That is, the post-intervention distribution is identifiable.
,x x for all
D.2 Example for reliable causal inference for the SDE 2
Recall that the SDE 2 is defined as
dX, AX,dt Ek=1 GrX,dWWa. kets Xo zo.
where
0O, A.G.& E Rddd for k 1,... < < co) is an m-dimensional standard Brownian motion. Let
some constant matrices, W
(W, = [......., W m
(1; A, (G k= denote the solution to the SDE 2 Let A, GR Raxd for k = 1,....7. define the following SDE:
dx, = AX,dt Ets GrX,dWan. Xo = zo.
such that
X(: A, (G+ (;: A, (G& 1
Then under our proposed identifiability condition stated in Theorem 3.5 we have shown that the generator the SDE 2 is identifiable, i... (A, Em GLAT G = (A, Ex= GgaZ T GT) for all x E Rd. Till now, we have shown that under our proposed identifiability conditions, the observational
distribution identity, the generator of the observational SDE. Then we aim to show that the post- intervention distribution is also identifiable. For notational simplicity, we consider intervention on the first coordinate, making the intervention X = & and X; for 0 < t < Jo. It will suffice to show equality of the distributions of the non-intervened coordinates (i.e., x(-1) and X(-1)). Express the matrices of A and G2 for k = I,.... m in blocks
An
A12
GA,11 GR,12
A =
Gk =
A21
A22
Gk,21 Ga,22
where An, GA.11 E RIIx, A22, Gk,z2 E (dd- Also consider corresponding expressions
of matrices A and G₁ for k 1,..., By making intervention X! &, one obtains the post- intervention process the first SDE satisfies:
ax(-") = (A21 A22X!-1)
1 Gr,22X ((=22x(-")))a ddW x(-" = (-1)
Since this post-intervention process is not a Gaussian process, one cannot explicitly show that the post-intervention distribution is identifiable. Instead, we check the surrogate of the post-intervention distribution, that the first- and second-order moments of the post-intervention process x(-")
Which denote as (-t)(-1) = E[x(-") and P(t)(-1) E[X!
(1x 11 (x((-") respectively. Then
m(t)(-1) and P(t) satisfy the following ODE systems:
dm(())-11 A21 + Az2m(1)(-)) m(0) = x1
(-1)
dt
and dP(()(-1)
m({) AT (1) P()))))))) A
dl
GT + Gr.z2m(t) -1 GGa (46) Gz. P(t) (-1) GT ((0)(-1) = -1) "T)
Dmn GT
Similarly, one can obtain the ODE systems describing the m(t (-1) and ((t) Then we will
show that m(t)(-1) m(t)(-1) and ((t)(-1) (())(-1) for all 0 < t < Recall that have
32

---

shown = A, thus A21 A21 and A22 = A22, then it is readily checked that m(t)(-1) -((t(-1))
for all 0 < 00.
In the proof of Theorem 3.5 we have shown that
E G&P()GG = GRP(I)GT
for all 0 E 00, where P(() ][
Simple calculation shows that
G&P() Σ(
GR.11 GR.21
G
EET m((t)--
{(m(()(-1)) P())(1))
G
G G2,22
G
Then one can get that the (2, 2)-th block entry of the matrix Σx G&P(IIGT is the same as (.-.(...) part in the ODE corresponds to P(t)(-1) (i.e., Equation (46)) since Σk=1 G2 P(tGGT =
Ex=1 Pt)(t) then the (.. part in the ODEs correspond to both P(t) and P(t)(-1) the same. Thus, it is readily checked that P(t)(-1) P(t)(-1) for all 0 t co.
Though we cannot explicitly show that the post-intervention distribution identifiable, showing that the first- and second-order moments of the post-intervention process is identifiable can indicate the identification of the post-intervention distribution to a considerable extent.
33

---

E Conditions for identifying the generator of a linear SDE with multiplicative noise when its explicit solution is available
Proposition E.1. Let xo Rd be fixed. The generator of the SDE 2 is identifiable from zo if the following conditions are satisfied:
C1 rank([zo| Amol ......AA--11 200 H.1|HHH H. ... H. .. = d,
C2 rank([u|Aul.... 1 (22)22)) - -cf² _ + +d//2,
C3 AG2 = Gk and G1 = G2Gx for all k. 1...=1,..... 1,...,
where H : Σk= G Gb,, and H.j stands for the j-th column vector of matrix H. for all j 1,...,,, d. And A A A + Ek= G2 Gy Rd @ denotes Kronecker sum and & denotes Kronecker product, v is d² d--mensiionll vector defined by v vec( ), where vec(M) denotes the vectorization of matrix M.
Proof. Let A, G2 E Raxd and AGr- GLA, GRG, = G.G. for all k, = 1,... m. such that X, we will show that under
(; A, ( X(; A, {Gx 1m ) we denote as our identifiability condition, for all æ E Rd, (A, >k=1 G& G = (Á, Et= GRAX" GT) By applying the same notations used in the proof of Theorem 3.5 in the following, denote A = A,
A2 == Å, G1. = G5 and G2.A = GR. and denote X _ X as X x².
We first show that H1 = H2 (H, = Ex= Gi G.,)). Indeed, since X X2 have the same distribution, one has
(47)
for all 0 < t oo and f CO By differentiating 47 at t 0, one finds that
(LI)) = (zo)
(48)
where Li is the generator of X (i = 1,
Based on the Proposition 2.1
d d
0S (zo) 2 k=1=1
OS
(Caf) = -E M (A))a
Hi)a
(oo),
Ozk
azzdzi
k=1 1=1
where (M)x denotes the kl-entry of matrix M, and fos is the 1-th component of ap. Since 488 is true for all f, by taking
f(x) (xp [ xng).
is readily checked that
(H1)pq (H2)
for all p.g = 1....,d. d. As aesult, H1 = Hg. Let call this matrix H. That is
H:=
Σ
Giazon G
Σ k=1
G2 G Hig
1=1
In the proof of Theorem 3.5 we have shown that
A A A2 xo for all j = 11:,,,2 1,2,...
(49)
next, we will derive the relationship between A, and H. Under condition C3, the SDE system ( has an explicit solution (cf. (25)):
/ (
m
m
X, X(t: Fo. A, (Gx) =
exp
G2 t
G&
(50)
2
k=1
1=1
34

---

then the covariance of Xt, P h) E[XxX, ++1 can be calculated as
1 m
m
Ge rony exp {
AT AAT {ATT
m (c)) (t+
E exp
A
G2
2
(51)
m
G Wheth
k=1
E Eis GitE &- GeWan e S Gf Waethlee - (CE) (L+h) (T(th)))
where
E c Di GeWe.
Di G. Wk.eth]
= E Dt_1 GeWa 2o
Di-1 G., Wkit Ei-1 Gj WeitgE Gf Wo
(52)
1E Ee G.W voz
Di= C Wi. W Gg (We,et Seenn-WW.)) We.l]
E C E e G.W 202g e G Wh.
of (Waeia Wa.e]]
because the Brownian motion Wk, r has independent increments. It is known that, for Z (O, 1), we have that the jth moment is
B(Z²) {
0
is odd,
2-1/2
j is even.
0/2
Since Wa. ~ (0, t), we have
DC (WWW
Ele GaWNe)
J!
(G.)E((WE.)
j!
(t22(1/2)112
(/2)
(Qtt22)²
d!
1=0
Similarly, we have
Ele (Wr (We We,)) (..
Simple calculation shows that
E
E[e GJ (Wo (Wa.s4n-W..)) = (
(53)
By combining Equations
51
52 and (3). one readily obtains that
P(L h) = P(t, P(t,t)edAA
(54)
we denote P(t) = P(t, Set (tt, h) (X 1 since X - x², it follows that
E| E[
P2((,t+h) Vt, 0. ) I -
To obtain information about A, let us fix I. for now and take j-th derivative of 54 with respect to
One finds that
d
(55)
dhi h=0
35

---

for all j =
1,2,.... 1,
It is readily checked that
P( P2(()AA)) VO
(56)
We know the function (t( satisfies the ODE
(t( = (0) + P.(0) + Σ G...(()GG.. V
(57)
(0) Zo T
In particular,
P.(0) = A,toog + 2oz0 A! + Σ
Giaarry G.,k.
By differentiating 56 with respect to at t 0, it follows that
A1 Ag (AT) + xoni _ (A +
GI GI, (AT)
=2 ( + xo20 (AI) +
GrrZard G22 (ANY.
Since have known that A/350 Aluro for all j 1,2,..... it is readily checked that
Σ GI G. (ATY
Σ G2 G2 (ATY,
that is A/H = A for all j j 1, 2, Let us denote this matrix A'H. Obviously, by rearranging this matrix, one gets
A H = A24 H for all j = 1, 2...
Therefore, under condition C1, that is rank(M) = with
M =
|Ad-1 H., H.H.HH,
- H.1 ... J.. ...I...1111111 IAd-I H.
(58)
we denote the j-th column in M as M.j. one gets A₁ M.j = A2 for all j = 1,...,, d by equations (49) and (88.
This means one can find a full-rank matrix B E igddd by horizontally stacking d linearly independent columns from matrix M, such that B A, Since B is invertible, one thus concludes that A A2.
In the proof of Theorem 3.5 we have shown that when A Ag, under condition C2, for all z E Rd
m
GI GT = Σ
G2 T G
k=1
Thus the proposition is proved.
It is noteworthy that Proposition E.1 is established on the explicit solution assumption of the SDE
(2) which requires both sets of vectors {4, (G& [G and (Á, [G+)m,) to satisfy condition C3. As aforementioned, condition C3 is very restrictive and impractical, rendering the identifiability condition derived in this proposition unsatisfactory. Nonetheless, this condition is presented to
illustrate that condition CI is more relaxed compared to condition Al stated Theorem 3.5 when identifying A with the incorporation of G 's information.
36


Reviews:
[{"review_id": "JaIFHQFiDF", "reviewer": "Reviewer_eRyW", "summary": "As I am not an expert in causal inference nor in SDEs, I will start with summarising the goal of the paper how I understood by reading it and some references therein. This summary will certainly reveal my ignorance regarding this research area, but I prefer to give this transparency and hope my review will be weighted accordingly. \nOrdinary and stochastic differential equations (SDEs) can be interpreted as modelling the influence among elements of a dynamical system. Therefore, they offer a tool for causal inference. For example, when making an intervention to a dynamical system, this intervention can be defined by a change to the corresponding SDE, resulting in “post-intervention” SDE. Previously, it was shown that if we can identify the generator of the original SDE, then we can use this result to obtain the distribution of the post-intervention SDE. \n\nThis paper derives sufficient conditions for the identifiability of the generator of linear SDEs with both additive and multiplicative noise. The authors conduct simulation experiments to assess the validity of their theoretical results.", "questions": "You are using MLE to estimate the system parameters (let’s call them $\\theta$) given discrete observational data $x$, i.e., you are obtaining a point for $\\theta$. Given there is uncertainty in the system and due to limited amount of observational data, there should be uncertainty w.r.t. to the estimate of $\\theta$ as well. Have you considered performing fully Bayesian inference over $\\theta$ instead, i.e., obtaining $p(\\theta | x)$ instead of performing $\\hat{\\theta}$ = MLE? \nFor example, you could use simulation-based inference (SBI) to obtain a approximate posterior from simulated data. \n\nThis would enable you add the uncertainty estimates to your analysis of the different “Identifiable” and “unidentifiable” cases reported in section 4.", "limitations": "The authors discuss the limitations of their work. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is generally well-written and clearly structured and, although technical, well readable. \n\nThe theoretical results which seem to be the major contribution of the paper are tested in an intuitive simulation study. The simulation experiments and results are described clearly and concisely.", "weaknesses": "The beginning of the introduction is good, but in the following paragraphs, the motivation for the importance of identifiability in SDE from a causal inference point of view is not well-explained. \nI had to refer to [16] to find a clear explanation of how SDE can be used for causal inference. \n\nSDE (1) is referenced in the introduction but appears only later in section 2.1.\n\nThe simulation experiment nicely illustrate the derived identifiability conditions. However, an example of how this can used for causal inference is not shown. Given that performing causal inference with SDE is one of the central motivations of this work, I think it would be essential to provide a concrete example of how this can be achieved using the contributions of this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful comments.  We address your comments point by point as follows. We have also updated our paper according to your comments, and we believe that the quality has been significantly improved thanks to your insightful comments.\n\n**Answers to weaknesses:**\n>1. **W:** The beginning of the introduction is good, but in the following paragraphs, the motivation for the importance of identifiability in SDE from a causal inference point of view is not well-explained. I had to refer to [16] to find a clear explanation of how SDE can be used for causal inference.\n\n **A:** Thank you for bringing this matter to our attention. In order to enhance the clarity of the connection between the identifiability of the SDE's generator and causal inference, we have made the necessary adjustments to lines 41-46 of the manuscript. The updated version now reads as follows: \"Secondly, the identifiability of an SDE's generator suffices for reliable causal inferences for this system. Note that, in the context of SDEs, the main task of causal analysis is to identify laws of the post-intervention processes from the law of the observational process. As proposed in [16], for an SDE satisfying specific criteria, laws of the post-intervention processes are identifiable from the generator of this SDE. Consequently, the intricate task of unraveling causality can be decomposed into two constituent components through the generator. This paper aims to uncover conditions under which the generator of a linear SDE attains identifiability from the law of the observational process. By establishing these identifiability conditions, we can effectively address the causality task for linear SDEs.\" The detailed interpretation of this connection, involving complex mathematical formulations and explanations, is presented in Section 2. Specifically, we elucidate the way of making an intervention on an SDE model and the arising SDE after intervention (i.e., the post-intervention SDE) in Definition 2.2 and the subsequent paragraph.\n\n>2. **W:** SDE (1) is referenced in the introduction but appears only later in section 2.1.\n\n**A:** Thank you for pointing this out. We have made the necessary adjustments to our manuscript to get rid of the forward references. Specifically, we have relocated the concise formulations of the two linear SDEs (i.e., SDE (1) and SDE (3)) to the appropriate position of the Introduction section. \n\n>3. **W:** The simulation experiment nicely illustrate the derived identifiability conditions. However, an example of how this can used for causal inference is not shown. Given that performing causal inference with SDE is one of the central motivations of this work, I think it would be essential to provide a concrete example of how this can be achieved using the contributions of this paper.\n\n**A:** Thank you for bringing this matter to our attention. We have updated our manuscript and given two examples for both SDE (1) and SDE (3), respectively, to show how our proposed identifiability conditions can ensure reliable causal inference for SDE models. Specifically, we have shown how under our proposed identifiability conditions, \"the law of the observational process $\\xrightarrow{\\text{identify}}$ the generator of the observational SDE $\\xrightarrow{\\text{identify}}$ the laws of post-intervention processes\". \nPlease note that since we can provide examples with analytic solutions, we do not conduct simulation experiments where the results may be affected by numerical errors. In addition, explicitly presenting the entire calculation process affords readers a comprehensive understanding of how our proposed identifiability conditions facilitate reliable causal inference for SDEs.\nIn Section 4, we have modified the title to \"Simulations and examples\", accompanied by an additional subsection titled \"Illustrative instances of causal inference for linear SDEs\". Within this subsection, we present a concise introduction to these two examples. Due to constraints related to space, the detailed calculation processes are presented in the Appendix.\n\n**Answers to questions:**\n\nThank you for your question. As you know that the simulations we used in Section 4 were intended to validate our proposed theoretical results (i.e., the identifiability conditions established in Section 3) rather than propose a new estimation method. Our objective was to demonstrate that, when the proposed identifiability conditions are satisfied, the estimated parameters will converge to the true underlying parameters as the number of observations goes to infinity. For parameter estimation, we opted for the MLE method, as it is widely used for parameter estimation for SDEs and straightforward to implement. In addition, we find that the results obtained from using the MLE method well supported the validity of our theoretical findings. \n\nWe truly appreciate your suggestion regarding the Bayesian inference method. In our future work, we plan to explore Bayesian inference for estimating system parameters in a finite sample setting. By doing so, we expect to obtain valuable uncertainty estimates for the estimated parameters, thus further enriching our analysis.\n\n"}, {"review_id": "5NtQzV4a8u", "reviewer": "Reviewer_b8xq", "summary": "The present work considers the problem of identifying the generators of autonomous linear\nstochastic differential equations (SDEs) with an additive or multiplicative\nnoise term from the law of its solution process (e.g., in case of additive noise\nthis is a Gaussian process) with a given fixed initial state. The authors derive\ngeneric sufficient conditions for this and use the knowledge gained to expand\nthe model interpretability from a geometric point of view. The applicability of\nthe theoretical results is shown in the context of causal inference using linear\nSDEs and empirical evidence is provided by a (small) set of simulations.", "questions": "- l. 34f. Check wording.\n- l. 63 What type of solution?\n- l. 73 Can you provide a reference here?\n- l. 75 Can you provide a definition for the notion of a stochastic process\n  here?\n- l. 82 What do you mean by: \"..., we recover the notion of stochastic\n  process,...\" ?\n- l. 85 Do you mean: \"..., $S$-valued functions on $[0,\\infty)$?\n- l. 124 \"... can be identifiable from their ...\", check wording.\n- l. 129 What is that gap you are mentioning here?\n- l. 142 Eq. 10: how are $b$ and $\\sigma$ defined?\n- l. 147 Prop. 2.1: \"... denotes the space of **continuous** functions ...\"\n- l. 185 Can you provide reference for this statement?\n- l. 195 Lemma 3.2: How is $Q$ defined?", "limitations": "Limitations are addressed (see e.g. Chapter 6) and mainly concern the practical\nverification of the identifiable conditions. From my perspective there are important aspects that need to be\ndiscussed additionally, e.g., computational limitations in the case of system\ndimension $d$, and Brownian motion dimension, $m$, greater 2. Further, the\nauthors consider the class of linear SDEs with either additive or multiplicative\nnoise in the autonomous setting (i.e., drift and diffusion do not explicitly\ndepend on time); can the presented results be transferred to the non-autonomous case?\n\nNo evidence was found that the authors addressed potential negative societal impact. Please make the effort to include this mandatory information in your work.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "In recent years, modeling random dynamics had great impact into deep learning by\nproposing so called Neural SDE (NSDE) models. Since SDEs are ubiquitously used\nfor modelling real world data, research interest in NSDEs rapidly grew.\nHowever, we still know very little about the often hidden systematics of these\nmodels and in particular the design of problem-specific function classes is a\nchallenge without a deeper understanding of it. Therefore, I appreciate this\nwork's attempt to make a systematic contribution to solving the mystery behind\nrandom dynamic modeling.\n\nFurther, the content is well organized, with very few exceptions, has no\nspelling or grammatical flaws and is original to the best of my knowledge. Empirical evidence is accessed through\nexperiments on toy data. Code examples are included in the supplementary\nmaterial. ", "weaknesses": "It is not the value of the approach I am concerning here, but the shortcomings\nrelated to the clarity of the submission and the technical soundness which I\nwill address with examples in the following:\n\n- W1: Although the use of SDEs has rapidly gained prominence in the NeurIPS\n  contributor landscape in recent years, I would argue it still represents a\n  rather subtopic, not easily accessible to every contributor. On the other hand, the generator theory for SDEs is more of\n  an advanced topic. Hence, in the context of its central role in this paper, it is\n  desirable to add a (gentle) introduction to the topic for the wider NeurIPS\n  audience. To illustrate this deficiency with an example: In Sec. In Section\n  2.3 the authors talk about SDEs generators without giving a definition, and\n  later in Section 2.1 the reader discovers that it is an operator on function\n  spaces.\n\n- W2: Starting from l. 184 including Lemma 3.2 and the interpretation of\n  identifiability conditions from a geometric perspective, i.e., the generator\n  of the SDE (1) is identifiable from $x_0$ when not all the given vectors\n  are confined to an $A$-invariant proper subspace of $\\mathbb{R}^d$, I found\n  difficult to access; similar in the case of multiplicative noise (l. 269ff.).\n  Can you phrase the idea in an easier manner? I think what I'm saying\n  is that the amount of technical detail presented diminishes the value of the\n  result.\n\n- W3: In my opinion, the presentation of the causal interpretation of SDEs in\n  Section. 2.3 as a subsection to \"Model and Problem Formulation\" is confusing\n  since this topic is more an example of the application of generator\n  identification of SDEs than a contribution to the problem definition of the\n  latter in general.\n\n- W4: Identifying the solution process of autonomous linear SDEs with additive\n  noise term as in Eq. (1) by Gaussian processes is necessary for the results in\n  the present work, e.g., cf. Lemma 3.1. This statement is justified by the\n  identification of generators by properties of the law of such Gaussian\n  solution processes with a given fixed initial state. I am wondering if the\n  authors also assume the solution process to be Gaussian in case of linear SDEs with multiplicative noise as in Eq.\n  (3) (because the method of solving for m(t) and P(t) is involved in the proof\n  of Theorem 3.4 in Sec. A.7 in the appendix) because this proves wrong in the general case; e.g., in case of a geometric\n  Brownian motion which is a solution to a linear SDE with multiplicative noise\n  term.\n\n- W5: Simulations are only presented in a very limited scenario on (very) low\n  dimensional toy datasets. In real world applications, e.g., modeling (multivariate) time\n  series data, the state variable dimension $d$ is often significantly larger\n  than 10 (cf. I. Silva et al. [52]). In order to create added value for such models, e.g., NSDEs, the\n  approach has to be tested in this extended framework.\n\nHowever, I would like to reiterate my appreciation for the authors' interest in\nexpanding the field of modelling random dynamics. I'm sure that by working on\nthe deficiencies mentioned, the work will gain in value\nand will deliver an enriching contribution to the community.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful comments.  We address your comments point by point as follows. We have also updated our paper according to your comments, and we believe that the quality has been significantly improved thanks to your insightful comments.\n\n**Answers to weaknesses:**\n\n>**W1:** introduce generator\n\n**A:**  Thank you for bringing this matter to our attention. We have added an introduction to the generator of a stochastic process to our manuscript. Reads as follows: \"The generator of a stochastic process $X_t$ can be defined as $(\\mathcal{L}f)(x)= \\lim_{s\\rightarrow 0}\\cfrac{\\mathbb{E}[f(X_{t+s})-f(X_t)|X_t= x]}{s}$, where $f$ is a suitably regular function.\"\n\n>**W2:** geometric interpretation\n\n**A:** Thank you for your comment. In order to elucidate the geometric explanation of the condition, we would like to draw your attention to the two fundamental concepts (refer to lines 203-204) \"$A$-invariant subspace\" and \"proper subspace\".  In other words, an $A$-invariant proper subspace of $\\mathbb{R}^d$ can be described as a non-empty subspace of the vector space $\\mathbb{R}^d$ that remains unchanged under the transformation induced by matrix $A$ and is not the entire space itself.\n   \nMoreover, it is crucial to recognize that this portion primarily constitutes an expression of the fundamental Jordan decomposition of matrix $A$.  We have made necessary adjustments to make the manuscript more reader-friendly.\n\n>**W3:** presentation of Section 2\n\n**A:**  We have made necessary adjustments to enhance the presentation of Section 2. Moreover, we have made necessary adjustments to various sentences and thoughtfully incorporated gentle introductions and relevant citations to elucidate the concepts presented throughout the paper, thereby ensuring a more coherent and reader-friendly presentation.\n\n>**W4:** Regarding the proof of Theorem 3.4\n\n**A:** Thank you for this comment. We would like to clarify that for linear SDEs with multiplicative noise, as depicted in Eq. (3), we did not assume the solution process to be Gaussian. \n\n>**W5:** simulation dimensions $d$ and NSDEs.\n\n**A:** Thank you for this comment. We would like to clarify that our proposed sufficient conditions for both linear SDEs with additive noise and linear SDEs with multiplicative noise are applicable to any dimension $d\\geqslant2$.  \n\nThis work primarily focuses on establishing the foundational theory for the identifiability analysis of linear SDEs.  The simulations presented in this study are intended to validate the proposed theoretical results.  As for NSDEs, given their neural network-based structure and non-parametric nature, they differ from our model settings, and no identifiability guarantees currently exist. Consequently, NSDEs are predominantly employed for forecasting tasks. However, for SDEs satisfying our proposed identifiability conditions, not only perform well in forecasting tasks but also enable reliable predictions about the SDE system's behavior under **intervention**.\n\n**Answers to questions:**\n>1. **Q:** l. 34 wording.\n\n**A:** We have updated the sentence, reads as follows: \"Traditional identifiability analysis of dynamic systems focuses on deriving conditions under which a unique set of parameters can be obtained from error-free observational data.\"\n\n>2. **Q:** l. 63 type of solution\n\n**A:** In the context of our study, the solution to the linear SDEs is the strong solution. \n\n>3. **Q:** l. 73 reference\n\n**A:** We have provided an inference [25].\n\n>4. **Q:** l. 75 notion of a stochastic process\n\n**A:** We have included the notion. \n\n>5. **Q:** l. 82 \"$\\ldots$, we recover the notion of stochastic process,$\\ldots$\"\n\n**A:** We would like to clarify that the mentioned sentence means that we obtain the definition of the law of a continuous stochastic process. We have updated our manuscript to the wording. \n\n>6. **Q:** l. 85 $S$-valued functions\n\n**A:** Here when $X$ is a continuous stochastic process, $S = C[0, \\infty]$, in other words, one can say that a stochastic process is an $S$-valued random variable, where $S = C[0, \\infty]$ stands for the space of all continuous, real-valued functions on $[0, \\infty]$. But we cannot say $S$-valued functions.\n\n>7. **Q:** l. 124 wording\n\n**A:** We have updated the sentence\n\n>8. **Q:** l. 129  gap\n\n**A:** In the context of SDEs, the primary goal of causal inference is to uncover conditions under which: \"the law of the observational process $\\xrightarrow{\\text{identify}}$ the generator of the observational SDE $\\xrightarrow{\\text{identify}}$ the laws of post-intervention processes\". The latter aspect has been established in reference [16], while the former part represents the gap that we mention here and address in this paper.\n\n>9. **Q:** l. 142 Eq. 10: $b$ and $\\sigma$\n\n**A:**  $b$ and $\\sigma$ are locally Lipschitz continuous in the space variable $x$. \n\n>10. **Q:** l. 147 **continuous**\n\n**A:** We have updated our manuscript accordingly.\n\n>11. **Q:** l. 185 reference\n\n**A:**  We have added a reference ([41]).\n\n>12. **Q:** l. 195 $Q$\n\n**A:** In Lemma 3.2, $Q$ is defined in lines 189-193, such that matrix $A$ can be expressed as $A = Q\\Lambda Q^{-1}$ through Jordan decomposition.\n\n>**Answers to limitations:**\n\nThank you for this comment. We would like to reiterate that the goal of this work is to establish the foundational theory for the identifiability analysis of linear SDEs, rather than developing a specific parameter estimation method or algorithm for SDEs. Our proposed theories are applicable to any $d\\geqslant 2$ and $m\\geqslant 2$. Therefore, we maintain the stance that it is unnecessary to introduce this computational limitation, which are primarily of concern when proposing a new estimation algorithm.\n\nRegrettably, our proposed method cannot be readily extended to the non-autonomous case. Accordingly, we have updated our manuscript to reflect this constraint.\n\nRegarding the potential negative social impact, we have added relevant information.\n\n\n\n\n\n"}, {"review_id": "JV4IOz7oFb", "reviewer": "Reviewer_GdBe", "summary": "This paper derives some sufficient conditions for identifying linear stochastic differential equations (SDEs). The validity of the identifiability conditions is clarified on synthetic simulation data.", "questions": "What are the situations where the derived conditions for identifying linear SDEs so useful in practice?", "limitations": "N/A", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The identification problem of ordinary/stochastic differential equations has been a hot topic in machine learning community, but the condition for identification seems to be relatively overlooked. The paper could raise a good question about it. \n- The paper is well-motivated and makes a technically solid contribution.", "weaknesses": "- The paper focuses on linear SDEs. They are very simple models and, to the best of my knowledge, are rarely applied to modern problems of system identification. Thus, the paper’s contribution is very limited and not impactful. \n- The paper emphases its contribution to reliable causal inference, but the experiments don’t include causal inference task. How the derived sufficient conditions benefits the causal inference task remains to be clear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful comments.  We address your comments point by point as follows. We have also updated our paper according to your comments, and we believe that the quality has been significantly improved thanks to your insightful comments.\n\n**Answers to weaknesses:**\n\n>1. **W:** The paper focuses on linear SDEs. They are very simple models and, to the best of my knowledge, are rarely applied to modern problems of system identification. Thus, the paper’s contribution is very limited and not impactful.\n    \n**A:** Thank you for your comment. We would like to clarify that there are also some real-world applications used linear SDEs. For example, linear SDEs are wildly used in financial modeling for tasks like asset pricing, risk assessment, and portfolio optimization. Where they are used to model the evolution of financial variables, such as stock prices and interest rates. And linear SDEs are also used in genomic research, for instance, they are used for modeling the gene expression in the yeast microorganism Saccharomyces Cerevisiae [16], where one aims to identify the system such that making reliable causal inferences when interventions are made. For example, what genes to knock out to achieve optimal growth of an organism. \n    \nIdentifiability analysis of SDEs is essential for reliable causal inferences of dynamic models governed by SDEs. However, the inherent complexity arising from the stochastic nature of variable evolution poses challenges in addressing identifiability tasks. Consequently, the identifiability analysis for SDE systems is still at the **early stage**, even for linear SDEs (especially for multiplicative noise cases), remain understudied. This is one of our motivations for starting from the linear cases.\n    \nOur other motivation is that the evolution from simpler linear cases to more intricate nonlinear instances is a common progression in research studies. This journey is shared across various model classes, including state space models, Ordinary Differential Equations (ODEs), among others. The foundation typically rests on comprehending linear cases first. Consequently, linear cases serve as a good starting point for identifiability analysis of SDEs.\n    \nWe believe that our proposed work not only serves to enhance reliable causal inferences for linear SDEs but also paves the way for a comprehensive identifiability analysis of more intricate SDE models. This advancement holds the potential to significantly elevate the accuracy of causal inferences within dynamic systems governed by SDEs.\n\n>2. **W:** The paper emphasizes its contribution to reliable causal inference, but the experiments don’t include causal inference task. How the derived sufficient conditions benefit the causal inference task remains to be clear.\n\n**A:** Thank you for bringing this matter to our attention. We have updated our manuscript and given two examples for both SDE (1) and SDE (3), respectively, to show how our proposed identifiability conditions can ensure reliable causal inference for SDE models. Specifically, we have shown how under our proposed identifiability conditions, \"the law of the observational process $\\xrightarrow{\\text{identify}}$ the generator of the observational SDE $\\xrightarrow{\\text{identify}}$ the laws of post-intervention processes\". \n\nPlease note that since we can provide examples with analytic solutions, we do not conduct simulation experiments where the results may be affected by numerical errors. In addition, explicitly presenting the entire calculation process affords readers a comprehensive understanding of how our proposed identifiability conditions facilitate reliable causal inference for SDEs.\n\nIn Section 4, we have modified the title to \"Simulations and examples\", accompanied by an additional subsection titled \"Illustrative instances of causal inference for linear SDEs\". Within this subsection, we present a concise introduction to these two examples. Due to constraints related to space, the detailed calculation processes are presented in the Appendix.\n\n**Answers to questions:**\n>**Q:** What are the situations where the derived conditions for identifying linear SDEs so useful in practice?\n\n**A:** Thank you for your question. We would like to clarify that an important application of our derived identifiability conditions is to ensure reliable causal inferences for dynamic models governed by linear SDEs. Specifically, for linear SDEs under our proposed identifiability conditions, laws of the post-intervention processes are identifiable from the law of the observational process. In other words, our derived conditions ensures the identification of intervention effects on linear SDE systems using observational data. Note that in cases where the SDE system lacks identifiability, observational data would only suffice for predictive tasks, rendering it unsuitable for observational causal inference tasks. Thus, when dealing with real-world phenomena governed by linear SDEs, the application of our proposed identifiability conditions guarantees reliable causal inferences within the systesm. The example of gene expression in the yeast microorganism Saccharomyces Cerevisiae, as presented in our response to weakness 1, serves as a good practical situation for the use of our derived conditions."}, {"review_id": "1Zao1GXhxZ", "reviewer": "Reviewer_5yfh", "summary": "This article derives relationships for identifying generators of linear SDEs in both the case of additive and multiplicative noise. It is written in a very classically statistical manor, with a great number of citations from the statistical research community.", "questions": "Why is this article relevant to the machine learning community?", "limitations": "This article is written for pure-statisticans and it is not really legible for many in the ML community.", "rating": 4, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "This article identifies two apparently unresolved issues from stochastic differential equations and solves them.\n\nI thank the authors for their thoughtful discussion in the rebuttal process.", "weaknesses": "This article assumes a high understanding of statistics, and unfortunately this is not my field. I do not have the expertise to review the proofs in this article. I think the article could do a better job at explaining the relevance of this topic to the broader machine learning community. Excluding the unpublished (arxiv) citations, it looks to me like there are only two published articles cited in the ML community, and both of those articles seem only tangentially related to the work in this article. The presentation of this article appears to be quite good for a pure-statistical research community.\n\n- All of the proofs claimed in this article are in the appendices of their article. While I know this is standard practice, technically NeurIPS appendices are un-reviewed (and in practice they are criminally under-reviewed). In my own field, this has led to an increased number of incorrect results being published, cited, and propagated through the research community. Because reviewers like me cannot assess the validity of the results claimed in this article, I worry that this may also happen in statistics. For this reason, I am giving a low-confidence rejection. If I see that other reviewers make good points attesting to the validity of the theory, I will change my recommendation.\n- Throughout the article there are many forward-references (e.g., on line 25, the reader has no idea what the matrix \"A\" means, in lines 31 and 32, the equations (1) and (3) are referenced without being introduced first, etc.) which make for a hard experience for the reader.\n- Clerically, one could order the citations so that they appear in increasing order (e.g., in line 19, [4, 34, 45])\n- Line 39 - 41: The argument for uniqueness is not clear from this statement.\n- Line 69: I think you mean \"are commutative\"\n- The sectioning in this article does not quite make sense to me. The pagraph \"We know that both the SDE (1) and SDE (3) ...\" which appears in Section 2.2, actually talks about both the content in Sections 2.1 and 2.2.\n- Inconsistent citations throughout. For instance, some articles are cited using the lowercase \"volume\" while others do not use the abbreviation for a volume at all (e.g., 61(11): 1038-1044) -- this citation style should be consistent.\n- The citation also contain many mis-capitalizations (odes vs ODEs, SIAM vs Siam), typos, duplicated authors [21], and clerical errors.\n- There are a lot of citations of unpublished material; a small number of recent arxiv publications is fine in this community, but some are missing any references alltogether, e.g.,  [13] and [24].", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful comments.  We address your comments point by point as follows. We have also updated our paper according to your comments, and we believe that the quality has been significantly improved thanks to your insightful comments.\n\n**Answers to weaknesses:**\n\n>1. **W:**  Proofs concern.\n    \n**A:** We acknowledge your concern regarding the correctness of the proofs presented in this article, considering that you may not possess the specialized expertise to thoroughly review them. We would like to assure you that all the proofs have been checked multiple times to ensure their accuracy. Additionally, we wish to highlight that the simulation results presented in Section 4 serve as a substantial validation of the proposed theoretical results.\n    \n>2. **W:**  Regarding forward references.\n    \n**A:** Thank you for bringing this matter to our attention. We have made the necessary adjustments to our manuscript. Specifically, we have relocated the concise introduction of the two linear SDEs (i.e., Eq.(1) and Eq.(3)) to the appropriate position of the introduction. In addition, we conducted a thorough review of the entire manuscript to ensure the elimination of any further instances of forward references.\n    \n>3. **W:** Order the citations in increasing order\n\n**A:** Thank you for pointing this out. We have conducted a thorough review of the entire manuscript to ensure all multiple citations have been arranged in increasing order.\n\n>4. **W:** Line 39 - 41: The argument for uniqueness is not clear from this statement.\n\n**A:** Thank you for pointing this out. We have made the necessary adjustments to our manuscript. The updated version reads as follows: \"For example, in the SDE (1), parameter $G$ cannot be uniquely identified since one can only identify $GG^{\\top}$ based on the law of the observational process [16, 26].\"\n    \n>5. **W:** Line 69: I think you mean \"are commutative\"\n\n **A:** Thank you for pointing this out. We have updated our manuscript accordingly.\n    \n>6. **W:** The sectioning concern in Section 2.\n    \n**A:** Thank you for bringing this matter to us. As we have mentioned that in order to get rid of the forward references, we have relocated the introduction of the two linear SDEs (i.e., Eq.(1) and Eq.(3)) to the  Introduction section. Here, within this section, we have included a new subsection titled \"Background knowledge of linear SDEs\" to introduce the solutions of the linear SDEs, as well as the paragraph you mentioned and beyond.\n    \n>7. **W:** citation problems\n    \n**A:** Thank you for bringing this matter to our attention. We have made a thorough review of all the citations and rectified the mentioned inconsistent issues and errors. Regarding those citations lacking proper references, we have added the relevant sources.\n\n**Answers to questions:**\n\n>**Q:** Why is this article relevant to the machine learning community?\n\n**A:** Thank you for your question. SDEs are a powerful mathematical tool for modeling dynamic systems subject to random fluctuations. These equations find wide-ranging applications in the modeling of real-world data, spanning fields such as finance, physics, biology, and engineering, among others. They are used in the machine learning field in various ways to model and analyze complex and dynamic systems affected by randomness and uncertainty. These applications include (1) time series modeling and forecasting; (2) generative models; and (3) causal inference for dynamic models affected by randomness. \n\nIt is essential to acknowledge that our current understanding of the identifiability analysis of these models remains limited. This analysis is crucial for reliable causal inferences of these models and is the main motivation of our work. Through this work, we endeavor to pave the way for a comprehensive understanding of the identifiability of linear SDE models, ultimately enabling reliable causal inferences regarding system behavior under interventions, thus facilitating the aforementioned (1) and (3) applications of SDEs in the machine learning community. \n\n**Answers to limitations:**\n\n>**L:** This article is written for pure-statisticans and it is not really legible for many in the ML community.\n\n**A:** Thank you for your comment. we would like to clarify that this paper is aiming to target the ML and causality community. We have made necessary adjustments to enhance the readability and accessibility of this paper to the ML/causality community. We would like to summarize some of these improvements as follows:\n- We have made significant revisions to the original third paragraph in Introduction section. Specifically, we focused on conveying the intuitions and enhancing the clarity of why our proposed work can facilite reliable causal inference in dynamic models governed by linear SDEs, and how the machine learning community can benefit from our proposed theories.\n- In Section 4, we have modified the title to \"Simulations and examples\", accompanied by an additional subsection titled \"Illustrative instances of causal inference for linear SDEs\". Within this subsection, we present two examples for both SDE (1) and SDE (3), respectively, to show how our proposed identifiablity conditions can ensure reliable causal inference for SDE models. Specifically, we have shown how under our proposed identifiability conditions, \"the law of the observational process $\\xrightarrow{\\text{identify}}$ the generator of the observational SDE $\\xrightarrow{\\text{identify}}$ the laws of post-intervention processes\". By providing these examples afford readers a comprehensive understanding of how our proposed identifiability conditions are used for reliable causal inference.\n- Furthermore, we have reduced unnecessary mathematical concepts and formulas, instead we incorporated gentle introductions and relevant citations to elucidate these concepts, thereby ensuring a more reader-friendly presentation.\n\n"}, {"review_id": "IQQtOHcYcu", "reviewer": "Reviewer_8vBF", "summary": "This paper addresses the interventional identifiability of two classes of stochastic differential equations. Theoretical results are illustrated with simulation of identifiable and non-identifiable cases. ", "questions": "How the continuous time identifiability results relate to identifiability of equivalent discrete time models (Markov chains with additive/multiplicative noise in general state space)? ", "limitations": "Limitations are addressed in the discussion. However, how restrictive is the interventional framework for concrete problems is not addressed (see \"weaknesses\" section).", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper is carefully written and the theoretical results appear sound, although I could not check the proofs in detail. ", "weaknesses": "My main concern is whether the target audience for this paper is really NeurIPS. Considering myself one of relatively few in the NeurIPS community with some familiarity with both causality and (to a limited extent) with SDE, I have a hard time making sense of the contribution. Although the presentation of the framework of reference [16] for causality with SDE is interesting, it is quite challenging to get an intuition of the concrete cases for which this framework is relevant: the defined interventions are static (not time dependent) and I have a hard time to imagine in which application we need the heavy toolbox of SDEs to model static interventions, which moreover are constrained to always start from the exact same point in state space. The paper does not really help, neither with justifying this framework nor with providing some intuitions. \n\nMoreover, the paper is written in a quite confusing way, at least for a non expert in SDEs: after introducing \"standard\" SDE based on Brownian motion in Eqs. (1) and (3), we are introduced in Lemma 2.1 and above with SDEs based on integrating Levy processes (and not being explained what is this). Then we move on to talking about Ito diffusion processes (Propostion 2.1) with an equation of the generator exhibited, although we have not being told at any time, even roughly, what a generator is for a stochastic process. \n\nI can imagine the results may be relevant for a mathematical journal, but if the authors what to target the ML/causality community, there is a lot of work to do on the manuscript to keep only the essential mathematical concepts, explain them well with reduced formalism, and convey intuition of why this is relevant to the NeurIPS readership interested in causality and time series.\n\nAdditionally, if I risk myself on evaluating the importance/novelty of the key results of this work, they resemble standard results on characterizing the set of states reachable by some classes of Markov chains in general state space. I can imagine there are also fairly standard results for this question in the context of SDEs, and the rank constraint of Theorem 3.3 is not very surprising when considering the dimension of the subspace reachable from a fixed initial point. Given the whole state space is explored, again I can imagine identifying the generator boils down to exploiting standard results. I am left wondering whether this result really deserves a new paper that addresses it exclusively (meaning: without exhibiting any concrete problem where it is relevant and helpful).  ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful comments.  We address your comments point by point as follows. We have also updated our paper according to your comments, and we believe that the quality has been significantly improved thanks to your insightful comments.\n\n\n**Answers to weaknesses:**\n\n>1. **W:** target audiance of this paper and interventions.\n\n**A:** With regard to the concern of our target audiance, please refer to our response to the third weakness, where we present the adjustments we have made to enhance the readability of this paper to the ML/causality community.\n    \nRegarding interventions, we would like to clarify that although the interventions considered in this work are not explicitly dependent on time, they are not static since they implicitly depend on time  thorough the state variable $X_t$. Specifically, we define interventions in Definition 2.2, where they are described as $X_t^l := \\zeta(X_t^{-l})$. Here, $X_t^l$ represents the $l$-th coordinate of the state variable $X\\in \\mathbb{R}^d$ at time $t$, and $X_t^{-l}$ denotes the $(d-1)$-dimensional vector resulting from the removal of the $l$-th coordinate of state variable at time $t$. In addition, in Lemma 2.1 we state that the intervention function $\\zeta: \\mathbb{R}^{d-1} \\rightarrow \\mathbb{R}$ only needs to be Lipschitz, which is a rather mild condition. In other words, an intervention of a coordinate of the state variable can be any Lipschitz function of the other $(d-1)$-dimensional coordinates, since each coordinate (e.g., $X_t^l$) depends on time, the intervention also implicitly depends on time.\n\n>2. **W:**  presentation of the paper.\n\n**A:** Thank you for bringing this matter to our attention. We have made necessary adjustments to enhance the presentation of our manuscript. Specifically, we have changed the title of Section 2 to \"Background knowledge\", which now consists of two subsections. In Subsection 2.1, now titled \"Background knowledge of linear SDEs,\" includes:\n\n- The solutions of the two linear SDEs of interest (lines 64, 67-78).\n- Definition 2.1 (lines 79-89)\n- A gentle introduction to the generator of a stochastic process.\n- The general form (Eq. (10)) and its generator (lines 141-148).\n\nIn Subsection 2.2, we have retained the (formerly Subsection 2.3).\n\nwe have introduced a new subsection, 3.1, titled \"Prerequisites\", includes\n- Lemma 2.2 (lines 133-141).\n- Proposition 2.2  (lines 151-157).\n\n>3. **W:**  target audience \n\n**A:** Thank you for this comment, we would like to clarify that this paper is aiming to target the ML and causality community. We have made necessary adjustments to enhance the readability and accessibility of this paper to the ML/causality community. Includes:\n- In Introduction, added practical scenarios to motivate our proposed work.\n- In Section 4,  added a new subsection titled \"Illustrative instances of causal inference for linear SDEs\" to present two examples for both SDE (1) and SDE (3), respectively, to show how our proposed identifiability conditions can ensure reliable causal inference for SDE models.\n-  We have reduced unnecessary mathematical concepts and formulas, instead, we incorporated gentle introductions and relevant citations to elucidate these concepts, thereby ensuring a more reader-friendly presentation.\n\n>4. **W:** novelty\n\n**A:** Thank you for your comment. Since the linear SDEs with additive noise (SDE (1)) is a relatively simple model, the rank constraint of Theorem 3.3 may not be very surprising under our settings. However, to the best of our knowledge, there are no fairly standard results for this question under the same setting as ours. We would greatly appreciate it if you could share some relevant references. \nMoreover, we have derived the identifiability conditions for the linear SDEs with multiplicative noise (SDE (3)). Incorporating state variables into the diffusion coefficient significantly complicates the SDE model and the evolution of the state variables. As a result, the identifiability of this particular model remains significantly under-explored. Our work presents the first systematic study of the identifiability analysis of this particular model. We believe that our proposed identifiability conditions can effectively facilitate reliable causal inference for dynamical systems governed by linear SDEs.\n\n**Answers to questions:**\n\n>**Q:**  relation to discrete time models\n\n**A:** Thank you for your question. We would like to highlight that our proposed continuous-time identifiability conditions are considerably more relaxed when compared to the existing identifiability conditions based on discrete-time models. Specifically, we have demonstrated that our proposed identifiability conditions for both SDEs with additive or multiplicative noise are generic (lines 219-221 and lines 256-257). However, in the \"Related Work\" section, we have discussed that the present identifiability conditions based on discrete-time models tend to be restrictive. \n\nFurthermore, continuous-time models and discrete-time models represent two distinct approaches used to address different problems. Continuous-time models are employed to study systems that evolve continuously over time and are described by differential equations. While discrete-time models are designed to analyze systems that evolve in discrete steps and are described using difference equations. As a result, there is no direct correlation between our proposed continuous-time identifiability conditions and the current identifiability conditions of relevant discrete-time models. \n\n\n**Answers to limitations:**\n\n>**L:** interventions\n\n**A:** Thank you for your comment. Please refer to our answer regarding the concern related to interventions in the \"weakness\" section. In our explanation, we have elucidated that the definition of intervention is rather generic, which leads us to maintain our position that it is unnecessary to discuss the restriction of the interventional framework as a limitation. \n\n\n\n\n"}, {"review_id": "VZ9eiTIGMz", "reviewer": "Reviewer_Huoq", "summary": "This paper presents conditions for identifying the generator of a linear stochastic differential equation (SDE) with additive and multiplicative noise. The authors derive sufficient conditions for identifying the generator of both types of SDEs and offer geometric interpretations. \n", "questions": "1. Could you please give some examples or practical scenarios of the proposed method \n2. Could you please provide DAGs or any graphs that describing the data generating process? I want to see graphical settings that enables identification of post-interventional processes.", "limitations": "1. I believe that the paper's writing lacks motivation because it lacks examples or practical scenarios to describe practical importance.\n2. Without a graphical description of the data generating processes, it's difficult to assess how this result is related to causality.", "rating": 7, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "I think this paper is theoretically sound. ", "weaknesses": "1. It would be great if there are practical examples for motivating readers. It’s hard to evaluate the papers’ practical importance because there are no practical scenarios and simulations. \n2. In the introduction, the link between causal inference and the identiability of the SDE is weak. Is the goal to identify the “generator” of the post-interventional stochastic process when only observational process is available?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful comments.  We address your comments point by point as follows. We have also updated our paper according to your comments, and we believe that the quality has been significantly improved thanks to your insightful comments.\n\n**Answers to weaknesses:**\n\n>1. **W:** practical examples\n\n**A:** Thank you for your comment. We have made necessary adjustments to our manuscript, incorporating practical and illustrative examples that effectively demonstrate the importance of our proposed identifiability conditions. \n- We have added practical scenarios where the identifiability analysis of linear SDEs is important in Introduction section. \n- In Section 4, we have modified the title to \"Simulations and examples\", accompanied by an additional subsection titled \"Illustrative instances of causal inference for linear SDEs\". Within this subsection, we presented two examples for both SDE (1) and SDE (3), respectively, to show how our proposed identifiability conditions can ensure reliable causal inference for SDE models.\n\n>2. **W:** link between causal inference and the identifiability of the SDE \n\n**A:** Thank you for your comment. The goal of this paper is to uncover the conditions under which the \"generator\" of the observational SDE is identifiable from the law of the observational process. The main motivation behind this pursuit lies in causal inference. Specifically, within the domain of dynamic systems modelled in SDEs, the primary goal of causal inference is to uncover the conditions under which \"the law of the observational process $\\xrightarrow{\\text{identify}}$ the laws of post-intervention processes\". This intricate task can be addressed by examining the generator of the observational SDE, thereby establishing a connection between \"the law of the observational process $\\xrightarrow{\\text{identify}}$ the generator of the observational SDE $\\xrightarrow{\\text{identify}}$ the laws of post-intervention processes\". The latter aspect of this connection has been established in reference [16], while the former part represents the gap that we address in this paper. In the introduction section, we have made necessary adjustments to enhance the clarity of this connection. The detailed interpretation of this connection, involving complex mathematical formulations and explanations, is presented in Section 2.\n\n**Answers to questions:**\n\n>1. **Q:** examples/practical scenarios\n\n**A:** Thank you for your question. We would like to clarify that an important application of our derived identifiability conditions is to ensure reliable causal inferences for dynamic models governed by linear SDEs. Specifically, for linear SDEs under our proposed identifiability conditions, laws of the post-intervention processes are identifiable from the law of the observational process. In other words, our derived conditions ensure the identification of intervention effects on linear SDE systems using observational data. Note that in cases where the SDE system lacks identifiability, observational data would only suffice for predictive tasks, rendering it unsuitable for establishing reliable causal inferences. Thus, when dealing with real-world phenomena governed by linear SDEs, the application of our proposed identifiability conditions guarantees reliable causal inferences within the system. \n    \nAs an illustration, linear SDEs are used for modeling the gene expression in the yeast microorganism Saccharomyces Cerevisiae [16], where one aims to identify the system such that making reliable causal inferences when interventions are made. For example, what genes to knock out to achieve optimal growth of an organism. By applying our proposed identifiability conditions ensures reliable predictions of intervention effects on the system.\n\n>2. **Q:** DAGs\n\n**A:** Thank you for your question. In the classic DAG-based framework, the DAG directly represents the causal structure of a system. However, in the context of SDEs, We do not directly provide such a graph representation of causality. This distinction arises because, for SDE models, it is possible to identify laws of the post-intervention processes from the law of the observational process, even when the **signature (graph) cannot be identified** from the law of the observational process. Please refer to [Example 5.5, 16] for the illustration of this statement. This statement also indicates that, when formulating a successful causality theory for SDEs, the relevant concept to consider is **the laws of post-intervention processes**, rather than the signatures, since the former is identifiable from the law of the observational process while the latter is not. This contrasts with the DAG-based case. \n\n**Answers to limitations:**\n\n>1. **L:**  lacks motivation and examples\n    \n**A:** Thank you for your comment. We would like to take this opportunity to provide further clarification regarding the motivation behind our work. A great motivation for our work is to uncover identifiability conditions of linear SDEs to ensure reliable causal inferences for dynamic models governed by these systems. Please refer to our response to your first question, where we presented practical scenarios and an illustrative example to show where our proposed conditions are important. In addition, please refer to our response to the first weakness, where we summarized the adjustments we have made to our manuscript, to incorporate practical and illustrative examples.\n    \n    \n>2. **L:** graphical description\n\n**A:** Thank you for your comment. Please refer to our responses provided for question 2. We would like to reiterate that under our proposed identifiability conditions, the laws of the post-intervention processes are identifiable from the law of the observational process. This capability essentially allows for the identification of causal effects resulting from interventions within the system, thereby establishing a foundation for making reliable causal inferences for SDE models.\n\n\n"}]

Author rebuttal:

Meta-review: This work provides a theory of generator identification for linear SDEs, which has relevance to the blossoming theory for identifying causal models. The theoretical contribution is solid and relevant, but most reviewers found the discussion of the motivation and setting to be difficult to follow for an ML audience. That said, the authors have been extensively engaged in the rebuttal period and have promised specific, concrete changes to the manuscript that should satisfactorily address this concern. To quote one reviewer,

"Multiple reviewers, myself included, have raised concerns about how accessible this work is to the broader NeurIPS audience. In that vein, I'd like to underscore the importance of refining the introduction and background sections. It would be great to see these sections become more reader-friendly, providing a smoother entry point into the topic and shedding light on the motivating factors behind this research endeavor."

In this vein, it will be crucial for these edits (and others mentioned in the reviews below) to be included in the camera ready revision prior to publication.
Decision: Accept (poster)