Variational Inference for Continuous-Time Switching Dynamical Systems
Authors: Lukas Köhs
Bastian Alt
Heinz Koeppl
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: variational inference
hybrid processes
stochastic processes
continuous time
stochastic differential equations
Markov jump processes

Abstract: Switching dynamical systems provide a powerful, interpretable modeling framework for inference in time-series data in, e.g., the natural sciences or engineering applications. Since many areas, such as biology or discrete-event systems, are naturally described in continuous time, we present a model based on a Markov jump process modulating a subordinated diffusion process. We provide the exact evolution equations for the prior and posterior marginal densities, the direct solutions of which are however computationally intractable. Therefore, we develop a new continuous-time variational inference algorithm, combining a Gaussian process approximation on the diffusion level with posterior inference for Markov jump processes. By minimizing the path-wise Kullback-Leibler divergence we obtain (i) Bayesian latent state estimates for arbitrary points on the real axis and (ii) point estimates of unknown system parameters, utilizing variational expectation maximization. We extensively evaluate our algorithm under the model assumption and for real-world examples.

Variational Inference for Continuous-Time Switching Dynamical Systems
Lukas Köhs
Bastian Alt
Heinz Koeppl
Department of Electrical Engineering and Information Technology
Technische Universität Darmstadt
[lukas. kuukss.oehs,, aastian.att, weinz.koeppl hut
tu-darmstadt de
Abstract
Switching dynamical systems provide a powerful, interpretable modeling frame- work for inference in time-series data in, e.g., the natural sciences or engineering applications. Since many areas, such as biology or discrete-event systems, are naturally described in continuous time, we present model based a Markov jump process modulating subordinated diffusion process. We provide the exact evolution equations for the prior and posterior marginal densities, the direct solu- tions of which are however computationally intractable. Therefore, we develop a new continuous-tima variational inference algorithm, combining a Gaussian process approximation on the diffusion level with posterior inference for Markov jump processes. By minimizing the path-wise Kullback-Leein obtain (i) Bayesian latent state estimates for arbitrary points on the real axis and (ii) point estimates unknown system parameters, utilizing variational expectation maximization. We extensively evaluate our algorithm under the model assumption and for real-world examples.
divergence we
1 Introduction
Many natural and engineered dynamical systems can be understood terms of dontinuous-diserete hybrid models, in which given system switches between discrete modes exhibiting continuous dynamics. Examples include neuro- muuuo-mchhanccll models of locomotion (T). transition dynamics between different brain states (2 and brain-state dependent dicision-making in neuroscience (4);
grimee-swithhngg volatility dynamics (5) and risk assessment (6] 7 in financial analysis or electric
power systems and phenotype differentiation in systems biology (
In a discrete-time setting, a widely used class of stochastic hybrid models are switching linear dynamical system (SLDS) [10), which have received considerable attention in recent years [1]
12 13 14 [5]. Since real-world physical and biological systems naturally evolve in continuous time, discrete-time description of such systems is however limiting. In biological experiments and discrete-event systems in engineering [16), for instance, one typically (i) is interested in the system behavior at any given point in time and (ii) not easily determine discretization. To overcome these limitations, a continuous time analog to SLDS models has been put forward termed switching stochastic differential equations (SSDEs) (17), which augment set of diffusion processes with an underlying Markov jump process (MJP). Hybrid models of this kind have long tradition in statistics [8] and have been analyzed in particular in applications to biological
appropriate time
systems 19
For each process class individually, exact expressions for the posterior paths given some set observations can be obtained (22). However, for diffusion processes in particular, these expressions quickly become intractable as they entail solving multi-dimensional partial differential equations (PDEs). This is
Diffusion processes and MJPs have been treated extensively in the literature 200 21
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

1311
201
Eitz
14
p
100
X
82
Time
Time
Figure Sketch of the three layers of the hybrid process model. A two-state MJP (() (blue) evolves freely in the time interval E [0. 7], see Eq. (I) The MJP controls the dynamics of the SSDE (1) (() (green), see Eq. (2). From these continuous dynamies, only noisy observations X1, --,--- (red) are available for inference at irregularly-space time points l........ Left: Graphical model. Right: Sample path (vertical dashed lines indicate the Z-transitions).
aggravated the diffusion is coupled to an underlying jump process, because both processes then have be solved jointly.
One way to circumvent this issue are Monte Carlo approaches, which have been devised for both dif- fusion 23 and jump processes (24). Sampling methods may however suffer from slow convergence, and, as we shall show, still face computational intractabilities for hybrid systems. An established approach avoiding these problems variational inference (VI) methods, which approximate the exact posteriors by optimization [25). For inference and parameter learning in diffusion processes. continuous- time VI frameworks have been developed utilizing, c.g.. Gaussian processes (GPs) [26] 27]. and general exponential family distributions (28). Similar methods have also been devised for inference in MJPs (29 33]. To the best of our knowledge, inference framework for continuous-tin hybrid processes is however lacking. We draw on these previous works and present a generalized VI framework for hybrid systems which recovers existing diffusion and MJP approximations special cases. We specifically focus on meta-stable systems which remain in distinct, qualitatively different regimes over extended periods of time, which are of special interest, e.g., in computational structural biology 31 An implementation of our proposed method is publicly available
2
Mathematical Background
2.1 The Model
In this work, we consider three joint stochastic processes (22)) (Y) and {Xi A continuous-time adaptation of probabilistic graphical model and realization of the processes shown in Fig.
The Switching Process. The discrete-valued process Z(t) E Z C is given latent Markov jump process (MJP) freely evolving in time t. An MJP is continuous-tine Markov process (3) on countable state space Z and is completely characterized by initial probability (o(=) (Z) 2), Vz E Z and transition rate function
P(Z(t + h) z' Z(t) = 2)
Alz, 2), lim 6
(I)
h
for z' 2 \ with the exit A(z, EEE
The Subordinated Diffusion Process. The freely evolving MJP controls the dynamics of continuous-valued process (()) E y C IR"", which is given as latent switching stochastic differential equation (SSDE) in an Itō sense, [17
( (t) ((f, + (+) (t),
(2)
where f yx
x + an arbitrary drift function, the dispersion Q y x determines the noise characteristics of the system and W (t) I is a standard Brownian motion.

---

We define the system noise covariance (t), ((),, t) as := QQ". Note that the only difference conventional stochastic differential equation (SDE) the <(tedeepene of the drift function and dispersion; hence, Eq. (2) can be understood collection of individual SDEs between which the systems switches via (7). For an accessible introduction to SDEs, see (33
The Hybrid Process. In the following, refer the continuous value (1) as the state and to the discrete value (() as the mode of a hybrid process. The hybrid process (2()), (()) is fully
characterized by its time-point wise marginal density p(y, 2,t) == Oz -g. PY y, Z(t) = 2).
where "<" has to be interpreted element-wisse This density is given as the solution to the hybrid master equation (HME) 34 35
Oeply, 2, Ap(y.
(3)
with initial condition p(y, 2, = po(2)po(U, where po(y, 2) := a,. ... P(Y(0) I y Z(0) z) denotes the initial density of the Y -process and po(=) the initial probability mass function of the Z-process. The operator A(-) = F(-) + T()) is given via
a n
FØ(y,2,1)= == - Sy
2,
ΣΣ (=1
To(U,2,t) := Σ A(z', 2, t)oly, .t - A(z, 2),
z'EZ/=
for an arbitrary test function o R" x 2 Rz0 R. We provide a detailed derivation in Appendix A.1. As the discrete process (()) is independent of Y (l), further obtain the dynamics
of the marginal distribution p(z, () == (1) =)) by integrating over the continuous variable Y in the HME. This recovers the traditional master equation 32]
T )(,,1))
Σ 2 (pz,))
Z'EZ/2
with p(z, Po(2) V2 E Z: for details, see Appendix A.1.1.
Note that a general, analytical solution to the PDE (3) does not exist. Numerical solvers utilizing schemes such the finite differences finite element method suffer from the curse of dimensionality and can in principle only be applied to very low-dimensional state spaces (36) Even in low dimensions however, solvers need to be adapted the problem hand and may struggle due to, e.g., slow step-size adaptation. On the other hand, sampling trajectories from a hybrid process (t), (t)} straightforward: One can draw the process Z(t) by utilizing the Doob-Gillespie algorithm (37, 38). Given this trajectory, the SSDE (t) (tt) can be simulated using, e.g., an Euler- Mally-Maruyama stochastic Runge- Kutta method 33].
The Observation Process. Lastly, we denote with {Xihien the countable set of observed data points at times (4iiie. The observations X; E X are generated X; p(a; 3t). where p(z; ya) := am,1 ... Oz,, P(X; S x; ((4) - = y:), i N, by conditioning on the diffusion process (().. The observation space X' can be either discrete, X' C N, or continuous, X' C R. Note that in our model, a continuous-tim time description for the latent processes is assumed, while the observations recorded discrete time points. It therefore belongs to the class of cutinuous-discrete models, which have a long history in the filtering community [33 40 41]. This type of description is of great practical relevance as data is often recorded at discrete time points while the system of interest in fact evolves continuously in time, see, e.g., [6].
2.2 Exact State Inference
We now show how the exact posterior inference problem is solved in principle; the detailed derivations can be found in Appendix A.2.
The inference problem consists of finding the posterior hybrid process (Z(t), (()) xp. where condition finite set Fp,a] {.i...., IN of N observations obtained at time points (ti....,ty (N } in the interval [0, T). The posterior process is fully specified by its marginal density
3

---

2yy,2,t 21I,N) :=
... 0r., S y. Z(t) = 1 =:1=1...... 21...., XN = IN). which is known
as the smoothing distribution. The smoothing distribution given by
p(u. 2, T(I.N)) = --- 1))0,, 2,
(5)
with the filtering density a(y, t) = 0g ... (t) S y, [(t = X1 = X = IRR).
the backward density 2,), :- 1I: -[ Oz P(Xx+1 S 1k+11...,, XN S aN (t)
y. (t) = 2) and time-dependent normalizer C(t) = Σ, aay, 2, t) 2, t) dy. where max(k' N to I t). The components 3 and C' are continuous-time analogs to the quantities of the forward-backe backward algorithm for discrete-time hidden Markov models (HMMs) [IO].
It is a standard result for continuous-discret filtering problems (33) that the filtering distribu- tion between observation time points follows the prior dynamics, dra(y. 2, 2, = Aoc(y. 2, t). with initial condition a(y, 2, 0)
po(=)po(V, z). At the observation times, it is reset as 2,,,2,t,) = Ci 2, t (p(z, y). with the normalizer C, Σ. ay, 2, t )p(x, y) dy
and a(y, 2, ) lime Lo (1, 2, t. - h). Similarly, the backward distribution between observations
is given as the solution to another PDE [22]
0g 2, = - A 2)
(6)
with end point condition B(y, 2, T) = and adjoint operator At, see Appendix A.1. The reset conditions at observation times are given as (y,, 2,t,) B(y, 2, tt)p(ar y).
By calculating the time derivative of Eq. (5). can be shown (see Appendix A.2.3) that the smoothing distribution itself follows HME
a.ply, 2,6 T(I.N)) = Ap(u. 2, TTI,N),,
with initial condition p(y, 2, 0 f(),N)) po(=)po(U,=)))8 0). The operator Á contains the posterior drift function f(y, z, t) = 2,
t)a,, (log ( 2, t)}, the dispersion
matrix Dyy z, t) = Dyy, t) and the posterior rate function A(z' 2, = A(z', 2, B(y.21.))
((xa..)
Approximate Inference
Since the smoothing distribution is governed by the HME 77. which depends the solution of Eq. (6) the exact inference problem amounts to solving two PDEs, which is computationally intractable already for toy systems. Similarly, na've posterior sampling scheme would still require solving the backward PDE Eq. (6) and hence suffers from the same issue [42]. To address this challenge, we adopt a VI approach: aim to find approximate path measure Qy,z that minimizes the path-wise Kullback-eibler (KL) divergence
dQy,z log dlyy x
KT. (Qr,z Pr,iıx) Earz
d0y.z where dFy.ix is the Radon-Nikodym derivative between Qy,z and the exact posterior measure Pr.zix over paths Mo.rj (Y(1)) and Z = (Z(t) [o For details on the path-wise
KL divergence between stochastic processes, see, e.g., 28 43 44 a standard result for VI
methods that Eq. (8) can be recast as 25 42
KL (Qy.2 || Py,z = KL (Qy.z Py,z) -EQ In p Mo.r))l + +og logp(zp)))), (9)
with the expected log- likelihood Eqy.z [In p( Mo.ry)]
Σ. E [In p(a: y:)]. The minimization problem over Eq. ( can then be cast as a maximization problem over the evidence
lower bound (ELBO) 25
C = E (nn((pp Morr1) ) - KL (Qyzz Py:z),
(10)
which does not include the computationally intractable marginal bog-likelihood log M() and can hence be evaluated.
Optimizing Eq. (OPp requires explicitly computing the path-wise KI. divergence KT, (Qy.z || Py.z). For two hybrid processes of the same class obeying Eq. (3), this expression can formally be derived

---

using Girsanov's 's theorem for diffusion processes [45 and MJPs [46]. A more intuitive derivation can however be carried out using a limiting argument, similar to [9]. The detailed derivation can be found in Appendix A.3.1. Assuming a constant, state- and mode-independent dispersion /) as done also in (27) and drift g 2, t) and rate function A(2,z', 2 pertaining to the variational measure Qy the path-wise wise KL. divergence is obtained as
KL (Qr,z || Pr,z) KI. (0), |1 P2:2) + /// E [1(g(V,,,tt) - f(y, 2(,,2,
Σ ( (In ( A(, (A(2,)) dl, S/E2/3
(11)
with the weighted norm | :- xT Ar, the KL of the initial distributions, KL (Qy,2 2 || P. and the expectation is carried out with respect to the variational time-point marginal q1,, 2, 0g ... 0g (t) S y, Z(t) = =). We note that extensions to mode- and time-dependent D = t)z,t) [45) state-dependent D D(y) (28) are also possible. Additionally, in the absence of any coupling between (t) and (it), i.e., fly. 2, f t), Eq. (11) reduces to the sum of the known individual path-wise KL divergences for diffusion processes and MJPs.
3.1 The Constrained Objective
Since Eq. (TO) is mere reformulation, is still optimized by the true, intractable posterior distribution Py,zIx- To arrive at computationally tractable expressions, we restrict the class Q of admissible variational processes. Making a structured mean-field ansatz, we approximate the exact joint posterior density p(u, 2, t (l.N)) as
ply, 2, (p.,N)) = p(z,1 [TLNN) ply. (0, 2,,,1))
(12)
= qz,ttt 4r (y. t 2) =: q(y, 2, t),
with p(y.t f 2, == 0y, ... 0..00...PPPP (t) - ((t) 2, 21 We approximate the exact conditional ply,t t 2, (),N)), which in general does not have simple parametric form, by one fixed parametric expression per mode qr (y. / =) = oy Og Q(Y(() Sy 2 = =) via the introduction of the time-inden inneeinndpendent random variable Z. This results in a point- wise mixture
distribution (v,, 2, f) with weights 42(2,t) t) (Z((t) = 2) and mixture densities qy (v, & 2). Note that this similar in spirit to amortized inference techniques [47]. because we utilize the same parametric form for all times t.
To ensure the mixture distribution structure Eq. (12) to hold every time point I, impose separate constraints on the dynamics of qz (z, t) and gr (g. 2). Firstly, we require the marginal (2(2, t) to obey a master equation
d
dt 2z (z((t))=
Σ A(z', 2, 22)2)))))))))) (2,,t) V Z'EZ'=
(13)
This reproduces the structure of the exact posterior marginal p(2, / x1,,)), which can be seen by integrating out the continuous variable in the HME 7 c.f. Eq. (4). Secondly, constrain the variational factor qy (y, 1 =) to follow a Fokker-Planck equation (FPF) (2] with linear variational drift g(y, 2, t) A(z, t)y 6(2,t) for every mode individually,
n >>>,
n
Oggy (y.
0y. (,,,,,,))) (g. 22)
Or, (Dugy 2)). (14)
Equation describes the marginal density of a classical SDE, which, under linear drift, is equivalent
to a GP 48 This PDE is hence solved by a time-dependent Gaussian distribution gr (y. 2) =
Ny plz, t), E)),t)) (33), where the dynamics of the parameters is described by two ordinary
differential equations (ODEs)
((z,t) = A(2, ()) +)2, E(2,t) A() ((2,)))) (z, (D, Vz e Z. (15)
Our approach hence amounts to a mixture of GPs: this approximation will accurate whenever the distribution q2(2, t) is peaked at one E Z. In this case, the HME separates into FPE and
5

---

master equation for the processes (t) Z and (t),, respectively; see Appendix A.3.2 for details. Accordingly, the approximation eπror over the whole interval [0. T] will be small if the original system dynamics are linear in each mode and the modes are well discernible, that is, if the exact posterior concentrates on one mode. Since we are interested specifically in meta-stable systems which, by definition, transition between qualitatively different regimes and exhibit a separation of time scales between the intra-mode diffusive dynamics and the inter-mode transitions, we expect these criteria to be met reasonably well for our systems of interest.
The constraints 13) and (15) can be included into the objective Eq. (10) via Lagrange multiplier functions, yielding an augmented objective, the Lagrangian. We define the multipliers (2,tt). (2, f), (22 i) for the variational mean /u(2, t) and covariance E(2, 1) and variational rates A(2, (). respectively. Writing the ELBO as [[Qr,z] fo (t(t) dt, the full Lagrangian L to be maximized reads
L //' la(t)+ + (2, (anee,t))((((,,t)))))
b(2, + tr {w (2, ($l2,t)
2E2
((2, () (2))) ((ztt) (z(2,t)
Σ Ar ( dt,
ZEZZ
(16)
where we used the shorthand (2==())== - 2, for x' + and Z2z = - A(z, 1) else. Note that the dependency of on the variational measure Qy,z is fully captured by the variational factors, == [[Qy.z] = L|qz, E].
2,
3.2 Optimizing the Variational Distributions
The optimization problem consists in finding the optimal variational factors 92- k²; E' and parameters A b" "" maximizing Eq. 16). where á summarizes the variational imitial conditions. Our structured mean-field assumption Eq. (12) allows us maximize Eq. (T6) individually with respect to qz (z, t) and gy (y, z) [0), guaranteeing an increase in the ELBO due to convexity in each individual argument (25 As consequence of Pontryagin's 's maximum principle [49], the solutions to the maximization problems (2, t) arg max, gz (2.1) L and qÿ (y. f 2) arg ma (s.tta) L have to fulfil the respective constraint equations (13) and (15) as well as the Fuler-Lagrang. (EL. equation de agll- 0gt, where L = {(t) dt. The latter is giving rise to ODEs for the Lagrange multiplier functions: Firstly, the EL equation with respect to (z(2, () yields
Σ ((,,,', z'EZ/=
(),2()).
(17)
Secondly, the EL. equations hold separately for both Gaussian parameters ulz, E(, We obtain
A(2, 1) = - (z,
(18)
(2, = Ox(..))f - A (z, -
Both Eqs. (17) and (8) hold between observations; at the observation time points, reset conditions follow from the respective observation likelihoods. For the detailed derivations, including the explicit expressions for the gradients Oig for linear prior models [v, 2,1) () = Ap(z, Q)y + b, L) as well as the reset conditions, see Appendix A.3.3, Note that this constitutes a set of impulsive ODEs scaling linearly in the number of observations [50). For more details the scaling behavior of these ODEs, see Appendix A.3.3. As the parameters of both Eq. (7) and Eq. (88) are time-dependemt, general, analytic solutions does not exist [5I]. Instead, we resort to established numerical solvers (52)
The full optimization problem requires the Lagrange multiplier ODEs and (8) and the constraint Eqs. (13) and 15) to be solved jointly as boundary- problem with terminal conditions ((,T),A (.,, ( T) 0 and initial conditions on the distribution parameters (49). The variational
parameters have to be optimized simultaneously. A standard approach to this problem is an iterative forward-backward sweeping algorithm (53) [4]:
: (i) solve the Lagrange multiplier ODEs Eqs. (17)
and (18 backward in time, starting from the terminal conditions X, V 0. Next, (ii) update the

---

variational parameters acting on the constraints, in our case A, Φ. Here, we employ a simple gradient ascent scheme: for each w(t) E {A(z, t), b(z, A(2, f), 6}, we update
(l)) - u(t) + x(t) Bucc)
(19)
where we use a back-tracking line search 55 for the step size k(t), see Appendix A.3.4. Then, (iii) solve the constraint equations (33) and (5 forward in time, starting from initial conditions
e (zz(2, = (Z(2), ((2,, = μ (=), (1
(2)). Finally, (iv) repeat until convergence.
Note that our results generalize the findings of [26 29 :: for f(v, fv, t), gly, 2, g t). i.e. in the absence of coupling between the Y- and Z- Z-processes, the MIP and diffusion contributions to the prior KL Eq. [II separate and the derivative ( reduces to the result of (29) between observations. Furthermore, for the special case |Z| = 1, our result recovers the conventional GP approximation (26).
3.3 Parameter Learning
To learn the model parameters, that is, the prior tran- sition rate matrix A, the dispersion D, the prior ini- tial conditions, the parameters of the drift function f(y, 2, f) and the parameters of the observation like- lihood p(#, y:), we employ a variational expecta- tion maximization (VEM) scheme [10]. After con- verging onto variational distributions qz (z, t) and qy (y, t z), we perform gradient ascent with respect to these parameters on the Lagrangian L, Eq. (T)). see Appendix A.3.5. Note that we opt for this basic approach for simplicity so as to focus on the general inference framework. For an in-depth discussion on parameter optimization, see, e.g.. (56). The complete optimization scheme summarized in Algorithm| [ where we subsume all model parameters under θ for conciseness. This strategy yields a local optimum due to convexity in each individual argument (25). Note that to alleviate potential issues with local optima it is straightforward to utilize, e.g., multi-start approaches [57].
inpurrobbvvvattin data r (t ii=l.....
Initialize 92, 14, A, b, A, θ
while L not converged do
while L not converged do
Compute multiplier functions X, Vr V via Eqs. (17 and 18]
Update variational parameters A,
b, A via Eq.
19)
Compute variational factors μr, E,
qz via Eqs. (I3) and (15)
Update lower bound C
end
Update prior parameters θ via gradient ascent
Update lower bound C
end
A Algorithm 1: VI for hybrid processes
Experiments
4.1 Model Validation on Ground-Truth Data
We validate our method on synthetic data generated from a ID, two-mode hybrid system with observations corrupted by Gaussian noise, p(z; y.) A(x; Ue, Ents), where the observation times are drawn from a Poisson point process [20). Both mode dynamics are given by time-independent linear drift functions
f(y, = 0= - -),,
(20)
with set points 32 and dynamics a= > 0, = E 2 = (1,2)- For |2| = this would recover the well-known
process [33]
As shown in Fig. 2 A, the inferred posterior distributions gr (y. = qr (y. 2) and (z, t) both faithfully reconstruct the respective latent ground-truth trajectories. This is also reflected by the maximum a a-posteriori (MAP) paths y (t), 2 MAP (t) = arg max,, g(y, z, t). In regions around mode transitions, one can observe artifacts from the variational approximation as mixture of GPs: in the time interval Attraus between the last observation before and the first observation after the ground- truth mode transition at t = 12.5, the marginal qy (v. t) does not exhibit smooth transition across Y = O, but splits the probability density between the independently evolving Gaussian distributions 4r (y, /. z). This is not surprising, as we have argued (c.f. Section 3.1) that our approximation will be accurate in regions where (2, 1) Rs 1, which is the case at the beginning and the end of the interval At,. but not in between. Since the relaxation onto the mode set points 3g is fast compared to the mode
7

---

B
42
XAA
0
Time
Time
Figure 2: A: Model validation on ground-truth data from a ID, two-mode hybrid process. Top: True discrete path =(t) with observation times (vertical lines below). Middle: inferred marginal q2 (2(2,)) 1) and 2AP (t) (dashed). Bottom: true SSDE path (gray line), observations (crosses), the inferred marginal qy (y. t) and y (MAP(() (t) (dashed). Brighter colors indicate higher probability density, B: Diffusion in a 1D four-well potential. Top: inferred marginals q2 (2(2)t). t). Bottom: true SDE and observations with the inferred qy (y, t) and learned set points 3g (arrows). Right: potential landscape.
remain times, these transition regions short, yielding high approximation quality. This is further highlighted by comparison with an experiment which this condition is violated, which we show in Appendix B.1. As pursue generative modeling approach, we can verify the approximation quality by sampling full trajectories from the variational posterior. The empirical distribution over paths closely resembles the latent continuous trajectory. We provide a plot of the sampling distribution in Appendix B. along with a version of Fig. A showing the dynamics of the individual modes. Furthermore, the model parameters
identified with high accuracy: the learned set points, for instance, 3 = 0.70, 32 = 0..2,, where the ground-truth values are +1. The exhaustive list of both the learned and ground truth model parameters is also provided in Appendix B.I.
4.2 Diffusions in Multi Well Potentials
In many real-world scenarios, ground-truth discrete modes driving continuous dynamics do not exist, but continuous dynamics often exhibit qualitatively different regimes. Transitions between different regimes typically occur on vastly longer time scales than the relaxation dynamics within each regime, as is observed, e.g., for the folding dynamics of complex biomolecules (58). For such meta-stable systems in particular, explicit probabilistic modeling of set of underlying discrete modes can greatly aid interpretability and enable targeted interventions on the system. To demonstrate the capability of our model to yield sensible representations of distinct dynamic regimes, we apply it to latent Itō diffusions driven by ID and 2D benchmark potentials widely used in computational biology [59] 60
61 62) We model these system via hybrid processes with linear drift, c.f. Eq. (20). and Gaussian observation noise in both the ID and 2D case, as before. We assume mode mode-dependent dispersion,
D = Dzz). The observation time points are regularly spaced and we fix the observation covariance Eghs-
In both the 1D and 2D case, the mode reconstructions accurately capture the global transitions between distinct potential minima, see Fig. B and Fig. B. We note that in the ID example (Fig. 2 B), the true latent continuous trajectory exhibits a particularly clear separation of time scales between the inter- and intra-well dynamics, which is reflected in sharp transitions in the mode reconstruction and accordingly particularly accurate results. On the other hand, as shown in Fig. 3 in the 2D case, more pronounced transition regions exist, where it is not possible unambiguously assign the state at given time t to one of the three minima. The posterior marginals q2 (2, t) sensibly capture this uncertainty, which also reflected in a high quality mode-assignment [[LN]] shown in Fig. 3... A. Furthermore, the asymmetry of the learned mode-dependent dispersions accurately reflects the topology of the underlying potential. An overview over parameters given in Appendix B.2.
the observed data points

---

Vi
5
10 Time
15
20
1
Figure 3: Diffusion in three-well potential. A: Potential landscape with the inferred set points B (diamonds) and dispersions (zz) (ellipses, 30-region) with observations (crosses); colors according to MMPP (4:) for each observation 24, Top: inferred marginals gz(2, Bottom: components of y MAP (L) (thick lines). the ground-truth path (thin lines) and the observations (crosses). Shaded region: transition region with high ambiguity.
4.3 Switching Ion Channel Data
We apply our method to a structural molecular biology problem: we aim identify the switching behavior of the viral ion channel WCV exhibiting three different channel conformations (63) Different conformations yield different ion permeabilities and hence di lifferent conductivities which can be directly detected by applying voltage across the cell mem- brane and measuring the trans- mans-membram current. We model the conformation dynamics discrete process Z and the current, passing through an analog (continuous-tine filter and incurring amplifier noise, as continuous process Y. The observations X, are sampled at fixed rate and subject to quantization errors from an analog-to-digital converter. The obser- vation noise is known property of the used setup; we hence fix E-obe We reconstruct a highly plausible switching behavior and filter individual outliers, as depicted in Fig. 4 see Appendix B.3 for list of all learned parameters and experimental details. We note that very similar problem setups can be found, e.g., in nanopore sequencing technologies [64].
MAP 92
Current [pA]
2
Time [s]
Figure 4: Switching behavior of viral ion channel KCV Top: marginals qz corresponding to three channel conformations. Middle: MAP (t). Bottom: data (white line), marginals qy (3, f) and learned set points B. (arrows). Brighter colors indicate higher prob- ability density.
4.4 Learning Complex Latent Continuous Dynamies
After having demonstrated the applicability of our model to hybrid systems with different time scales for the discrete and continuous dynamics, we lastly show that it also works well when this criterion is not met. In areas such as automation and robotics, hybrid models are ubiquitously used to, e.g., encode highly complex continuous movements via discrete set of movement primitives [65]. which are non-stationary processes. To demonstrate that our method able to reconstruct such complex latent continuous dynamics, dynamics f(y. 2, t) = f(y, 2) are given as two counter-rotating vector fields, see Fig. B. We fix the observation covariance,
employ a 2D version of Eq. (20) where the mode
assume its value to be known and small compared to the system volatility for applications such as robotics.
As shown in Fig. A A and C, the mode and state reconstructions accurately recover the true paths. Accordingly, also the underlying mode dynamics are correctly learned, exhibiting the counter-rotating behavior of the ground-trun ttrnthhttuthhhe model. We provide a list of all parameters in Appendix B.4.

---

A
10
-5
Time
X1
Figure 5: Inference of complex structured continuous dynamics. A: True mode path (t)) (top) and inferred marginal qz t) (bottom). B: True state path y(t) (coloring according to associated
mode; = 1: blue, Z = 2: red), observations (crosses) and true mode dynamics fy. z) (arrows) in the phase plane. C: Reconstructed MAP path yMAP (t) (coloring according 2 zAP (t) and reconstructed mode dynamics (arrows).
Conclusion
We presented, the best our knowledge, the first variational inference framework for continuous time hybrid process models: since the exact filtering and smoothing distributions computationally intractable, we proposed a variational approximation to the exact model. The key assumption is that the true discrete posterior is peaked at any Z for extended periods of time, allowing for straightforward, easily interpretable mixture of GPs to be used as approximation. We have evaluated framework various benchmark tasks including real-world biological data and demonstrated its ability to faithfully reconstruct complex latent dynamics and to learn the unknown system parameters, in particular in applications to meta-stable systems. While implemented parameter learning via point estimates, we aim to extend this to fully Bayesian framework in the future, enabling the integration of prior domain knowledge and the associated uncertainty about the system hand. Furthermore, due to the scaling behavior of the ODEs (13) and (15), additional approximations need be worked out to be able apply the method high-dimensional state spaces and large numbers of modes, sec, c.g., [66]. As many natural and engineered systems can be described as hybrid systems, we think that extending the toolbox for inference will be of great utility for the analysis and control of such systems.
Acknowledgments
We thank Gerhard Thiel and Kerri Kukovetz for providing the ion channel voltage data and helpful discussions and the anonymous reviewers for their useful comments and suggestions. This work has been funded by the German Research Foundation (DFG) as part of the project B4 within the Collaborative Research Center (CRC) 1053 MAKI, and by the European Research Council (ERC) within the CONSYN project, grant agreement number 773196.
References
[1] Joshua L. Proctor, Raghavendra P. Kukillaya, and Philip Holmes. "A phase-reduced neuro- mechanical model for insect locomotion: cerw-forward stability and proprioceptive feedback". In: Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineer-
ing Sciences 368.1930 (2010), pp. 5087-5104.
[2] Gustavo Deco, Josephine Cruzat, Joana Cabral, Enzo Tagliazucchi, Helmut Laufs, Nikos K. Logothetis, and Morten L. Kringelbach. "Awakening: Predicting external stimulation to force transitions between different brain states". In: Proceedings of the National Academy of Sciences 116.36 (2019), pp. 18088-18097.
[3] Morten L. Kringelbach and Gustavo Deco. "Brain states and transitions: insights from compu- tational neuroscience". In: Cell Reports 32.10 10 (2020), 108 128.
10

---

[4] Jalil Taghia, Weidong Cai, Srikanth Ryali, John Kochalka, Jonathan Nicholas, Tianwen Chen, and Vinod Menon. "Uncovering hidden brain state dynamics that regulate performance and decision-making during cognition". In: Nature communications 9.1 (2018), pp- 1-19.
[5] Carlos M. Carvalho and Hedibert F. Lopes. "'imullttion-asse" sequential analysis of Markov switching stochastic volatility models". In: Computational Statistics & Data Analysis 51.9
(2007), pp.
[6] Thibaut Duprey and Benjamin Klaus. "How to predict financial stress? An assessment of Markov switching models". In: ECB Working Paper Series (2017).
[7] Khaled Salhi, Madalina Deaconu, Antoine Lejay, Nicolas Champagnat, and Nicolas Navet. "Regime switching model for financial data: Empirical risk analysis". In: Physica A: Statistical Mechanics and its Applications 461 (2016), pp. 148-157
[8] Alan S. Willsky. "A survey of design methods for failure detection in dynamic systems". In: Automatica 12.6 (1976), pp. 601-611.
[9] Paul C. Bressloff. "Stochastic switching in biology: from genotype to phenotype". In: Journal of Physies A: Mathematical and Theoretical 50.13 (2017), p. 133001.
[10]
David Barber. Bayesian reasoning and machine learning. Cambridge University Press, 2012. [11] Emily Fox, Erik Sudderth, Michael Jordan, and Alan Willsky. "Nonparametric Bayesian learn- ing of switching linear dynamical systems". In: Advances in Neural Information Processing Systems 21 (2008), pp. 457-464.
[12] Matthew J. Johnson, David Duvenaud, Alexander B. Wiltschko, Sandeep R. Datta, and Ryan P. Adams. "Composing graphical models with neural networks for structured representations and fast inference". In: Advances in Neural Information Processing Systems 29 (2016), pp. 2954- 2962.
[13] Scott W. Linderman, Matthew J. Johnson, Andrew C. Miller, Ryan P. Adams, David M. Blei, and Liam Paninski. "Bayesian learning and inference in recurrent switching linear dynamical systems". In: 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017. 2017.
[14] Joshua Glaser, Matthew Whiteway, John Cunningham, Liam Paninski, and Scott Linderman. "Recurrent Switching Dynamical Systems Systems Models for Multiple Interacting Neural Populations". In: Advances in Neural Information Processing Systems. Ed. by H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin. Vol. 33. Curran Associates, Inc.. 2020.
[15] David Barber and Bertrand Mesot. "A novel Gaussian sum smoother for approximate inference in switching linear dynamical systems". In: Advances in Neural Information Processing Systems 19 (2007), p. 89.
[16] Christos G. Cassandras and Stephane Lafortune. Introduction to discrete event systems. Springer Science & Business Media, 2009.
[17] Xuerong Mao and Chenggui Yuan. Stochastic differential equations with Markovian switching. Imperial college press, 2006.
[18] Mark H. A. Davis. Markov models & optimization. Routledge, 2018.
[19] Paul C. Bressloff. "Switching diffusions and stochastic resetting". In: Journal of Physics A:
Mathematical and Theoretical 53.27 (2020), p. 275003.
[20] Stewart N. Ethier and Thomas G. Kurtz. Markov processes: characterization and convergence.
John Wiley & Sons, 2009.
[21]
Bernt Øksendal. Stochastic differential equations. Springer, 2003.
[22] Brian D. O. Anderson and Ian B. Rhodes. "Smoothing algorithms for nonlinear finite- dimensional systems". In: Stochastics: An International Journal of Probubility and Stochastic
Processes 9. 9-1 (1983), pp. 139-165.
[23] Hans-Christian Ruiz and Hilbert J. Kappen. "Particle smoothing for hidden diffusion processes: Adaptive path integral smoother" In: IEEE Transactions on Signal Processing 65.12 (2017), pp. 3191-3203.
[24] Vinayak Rao and Yee Whye Teh. "Fast MCMC Sampling for Markov Jump Processes and Extensions." " In: Journal of Machine Learning Research 14.11 (2013).
[25] David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. "Variational inference: A review for statisticians". In: Journal of the American statistical Association 112.518 (2017), pp. 877.

---

[26] Cédric Archambeau, Dan Cornford, Manfred Opper, and John Shawe- Shawe-Taylor. "Gaussian process approximations of stochastic differential equations". In: Journal of Machine Learning Research 1 (2007), pp- 1-16.
[27] Cédric Archambeau, Manfred Opper, Yuan Shen, Dan Cornford and John Shawe-Taylor "Variational Inference for Diffusion Processes". In: Advances in Neural Information Processing Systems 20 (2007), pp. 17-24.
[28] Tobias Sutter, Arna Ganguly, and Heinz Koeppl. "A Variational Approach to Path Estimation and Parameter Inference of Hidden Diffusion Processes". In: Journal of Machine Learning Research 17.190 (2016), pp. 1-37.
[29] Manfred Opper and Guido Sanguinetti. "Variational inference for Markov jump processes".
In: Advances in neural information processing systems 20 (2007). pp. 1105-1112.
[30] Christian Wildner and Heinz Koeppl. "Moment-based variational inference for Markov jump processes". In: International Conference on Machine Learning. PMLR. 2019, pp. 6766-6775. [31] Weinan and Eric Vanden- Eijnden. "Metastability, conformation dynamics, and transition pathways in complex systems". In: Multiscale modelling and simulation. Springer, 2004, pp. 35-68.
[32] James R. Norris. Markov Chains. Cambridge Series Statistical and Probabilistic Mathemat- ics. Cambridge University Press, 1997.
[33] Simo Särkkä and Arno Solin. Applied stochastic differential equations. Cambridge University
Press, 2019.
[34] Robert Pawula. "Generalizations and extensions the Pokker-Planck- Solnck-Kolmogorvv equa-
tions". In: IEEE Transactions on Information Theory 13.1 (1967), pp- 33-41.
[35] Derya Altintan and Heinz Koeppl. "Hybrid master equation for jump-diffusion approximation of biomolecular reaction networks". In: BIT Numerical Mathematics (2019), pp. 1-34.
[36] Christian Grossmann, Hans-Görg Roos, and Martin Stynes. Numerical treatment of partial differential equations. Springer, 2007.
[37] Joseph L.. Doob. "Markoff denumerable case". In: Transactions of the American Mathematical Society 58.3 (1945), pp. 473-
[38] Daniel T. Gillespie. "A general method for numerically simulating the stochastic time evolution of coupled chemical reactions". In: Journal of computational physics 22.4 (1976), pp. 403- 434.
[39] Peter E. Kloeden and Eckhard Platen. Numerical Solution of Stochastic Differential Equations. Springer, 1992.
[40]
Peter S. Maybeck. Stochastic models. estimation, and control. Academic press, 1982.
[41] Frederick E. Daum. "Exact finite dimensional nonlinear filters for continuous time processes with discrete time measurements". In: The 23rd IEEE Conference on Decision and Control. IEEE. 1984, pp. 16-22.
[42] Ramon Van Handel. "Filtering, stability, and robustness". PhD thesis. California Institute of Technology, 2007.
[43] Sanjoy K. Mitter and Nigel J. Newton. "A variational approach to nonlinear estimation". In:
SIAM journal on control and optimization 42.5 (2003), pp. 1813-1833.
[44] Alexander G. de G. Matthews, James Hensman, Richard Turner, and Zoubin Ghahramani. "On sparse variational methods and the Kullback-Leible divergence between stochastic processes". In: Artificial Intelligence and Statistics. PMLR. 2016, pp. 231-239.
[45] Tommi Sottinen and Simo Särkkä. "Application of Girsanov theorem to particle filtering of discretely observed continuous-time non-linear systems". In: Bayesian Analysis 3.3 (2008), pp. 555-55544444
[46] Claude Kipnis and Claudio Landim. Scaling limits of interacting particle systems. Vol. 320. Springer Science & Business Media, 1998.
[47] C. Zhang, J. Butepage, H. Kjellstrom, and S. Mandt, "Advances in Variational Inference". In:
IEEE Transactions on Pattern Analysis Machine Intelligence 41.08 (2019). pp. 2008-2026. ISSN: 1939-3539.
[48] Carl E. Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press, 2005. ISBN: 0262182553 [49] Daniel Liberzon. Calculus of variations and optimal control theory: a concise introduction.
Princeton university press, 2011.
12

---

[50]
Anatolii Samoilenko and Mykola Perestyuk. Impulsive differential equations. World scientific, 1995.
[51] Michael Baake and Ulrike Schlaegel. "The Peano-Bakes series". In: Proceedings of the Steklov Institute of Mathematics 275.1 (2011) pp. 159-
[52] Ernst Hairer, Syvert P. Nørsett, and Gerhard Wanner. Solving Ordinary Difserential Equations
I: Nonstiff problems. Springer, 1993.
[53]
Robert F. Stengel. Optimal control and estimation. Courier Corporation, 1994.
[54] Michael McAsey, Libin Mou, and Weimin Han. "Convergence of the forward- backward sweep method in optimal control". In: Computational Optimization and Applications 53.1 (2012), pp. 207-226.
[55] Dimitri Bertsekas. "Nonlinear programming". In: Journal of the Operational Research Society 48.3 (1997), pp. 334-334.
[56] David Barber, A. Taylan Cemgil, and Silvia Chiappa. Bayesian time series models. Cambridge University Press, 2011.
[57]
Rafael Martí. Multi-start methods. Springer, 2003, pp. 355-368.
[58]
Jiri Sponer, Giovanni Bussi, Miroslav Krepl, Pavel Banáš, Sandro Bottaro, Richard A Cunha, Alejandro Gil-Ley, Giovanni Pinamonti, Simón Poblete, Petr Jurečka, al. "RNA structural dynamics captured by molecular simulations: comprehensive overview". In: Chemical reviews 18.88 (2018), pp. 4177-4338.
[59] Jan-Hendrik Prinz, Hao Wu, Marco Sarich, Bettina Keller, Martin Senne, Martin Held, John D. Chodera, Christof Schütte, and Frank Noé. "Markov models of molecular kinetics: Generation and validation". In: The Journal of chemical physics 134.17 (201 1). p. 174105.
[60] Hao Wu, Andreas Mardt, Luca Pasquali, and Frank Noe. "Deep Generative Markov State Models", In: Advances in Neural Information Processing Systems 31 (2018), pp. 3975-3984. [61] Frank Noé, Hao Wu, Jan- Hendrik Prinz, and Nuria Plattner. "Projected and hidden Markov models for calculating kinetics and metastable states of complex molecules". In: The Journal of chemical physics 139.18 (2013).
[62] Hao Wu, Feliks Nüske, Fabian Paul, Stefan Klus, Péter Koltai, and Frank Noé. "Variational Koopman models: slow collective variables and molecular kinetics from short off-equilibrium simulations". In: The Journal of chemical physics 146.15 (2017), 154104.
[63] Sabrina Gazzarrini, Ming Kang, Svetlana Epimashko, James L. Van Etten, Jack Dainty, Gerhard Thiel, and Anna Moroni. "Chlorella virus MT325 encodes water and potassium channels that interact synergistically". In: Proceedings of the National Academy of Sciences 103. (2006), pp. 5355-5360.
[64] Miten Jain, Sergey Koren, Karen H Miga, Josh Quick, Arthur C. Rand, Thomas A. Sasani, John R. Tyson, Andrew D. Beggs, Alexander T.. Dilthey, Ian T. Fiddes, al. "Nanopore sequencing and assembly of human genome with ultra-long reads". In: Nature biotechmology 36.4 (2018), pp. 338-345.
[65] Debora Clever, Monika Harant, Katja Mombaur, Maximilien Naveau, Olivier Stasse, and Do- minik Endres. "COCoMoPL: A novel approach for humanoid walking generation combining optimal control, movement primitives and learning and its transfer to the real robot HRP-2". In: IEEE Robotics and Automation Letters 2.2 (2017). pp. 977-984.
[66] Michail D. Vrettas, Manfred Opper, and Dan Cornford. "Variational mean-field algorithm for efficient inference in large systems of stochastic differential equations". In: Physical Review E 91.1 (2015), P. 012148.
[67] Cedric Archambeau and Manfred Opper. "Approximate inference for continuous Markov processes".. In: Bayesian Time Series Models (2011) 1), pp. 125-140.
13


Reviews:
[{"review_id": "ruceTfqWl3v", "reviewer": "Reviewer_naZJ", "summary": "The manuscript extends the literature of statistical inference in switching linear systems to the continuous domain. Starting from the specifics of the generative model (a discrete valued latent stochastic Markov Jump process modeling the \"mode\" of the hybrid process and a continuous valued latent diffusion process modeling the \"state\" of the process and finally an observation model connecting the latent space to discretely sampled observations) and then provides the expressions for equations governing the joint, posterior (or smoothing) and marginal distributions. The authors then show why exact inference is intractable and provide the methodology needed for approximate (structured mean-field) variational inference. The variational family considered is a mixture of GPs (which turns out to be the case because of the structural PDE equations enforced on the variational family to mimic the true posterior distribution) where the time-specific mixture weights follow a master equation and variational factor q(y,t|z) follows a a Fokker-Planck equation. The ELBO cost function is then derived and optimized using forward-backward sweeping algorithm. To optimize the model parameters (transition rate function, parameters of the drift function, and dispersion) a variational EM approach is taken where the parameters are updated iteratively after satisfying structural constraints and updating variational parameters. Experiments are performed on toy examples as well as multi-well potentials and ion channel data.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n=======\nThe question under investigation is timely, significant, and relevant to NeurIPS community in my opinion. \n\nThe background section includes a nice summary of the prerequisites needed for following the ideas throughout the paper. \n\nThe introduction provides a nice methodological literature as well as the application of the methods in various fields. \n\nAlthough the content of the manuscript is fairly complex I found the flow of the writing very sensible and easy to follow. \n\nThe amount of mathematical details included in the main text seems appropriate and every details that is mentioned in the supplementary is properly referenced in the main text. \n\nFinally the experiments are diverse and the visualizations are very helpful for digesting the outputs of the model and its inner workings.\n\nSuggestions\n==========\nAdding some analytical intuitions (not derivations) would help the reader to have a better sense about the equations.\n\nIt would be nice to supplement the visualizations with the observations x(t) to show what type of data is being considered.\n\nWeaknesses\n==========\nThe experiments are performed mostly on the cases where the analytical form of the drift function is known. However in real world scenarios it's rarely the case and in many cases the function is approximated using another parametric family. For discrete cases to the best of my knowledge neural networks are used to model the drift function (e.g. https://arxiv.org/abs/1605.08454). It would be useful to investigate if the model can recover dynamical regimes even in the presence of mismatch in the drift function.\n\nAlmost all the model validation that is done in the experiments section is qualitative. Even for a low-D system the parameter error (beta) seems relatively large. A more thorough investigation of the error as a function of dimension, number of modes, and separability between different dynamical regimes would help practitioners to apply the method with some prior caution.\n\nSince the field of statistical inference in discrete switching dynamical systems is fairly stablished it would be nice to perform comparisons between the fit given by the continuous model (proposed) and discrete models.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "For limitations, see above.\n\nAs the authors mentioned, the work is theoretical and hence there are not trivial societal and ethical impacts subject to discussion.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "Za6ImtjACL", "reviewer": "Reviewer_QSy5", "summary": "In this paper, the authors propose a variational inference framework for continuous time hybrid process models. The particular assumption on the true posterior allows for a mixture of GPs as an approximation to it. The proposed framework is tested on various benchmark tasks including real-world biological data and the results demonstrate its ability to faithfully reconstruct complex latent dynamics and to learn the unknown system parameters, in particular in applications to meta-stable systems. It extends the toolbox for inference will be of great utility for the analysis and control of such hybrid systems.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The proposed method is new for continuous-time SLDS. The assumed variational posterior enables inference via optimization. The relevant previous contributions were clearly discussed and cited.\n\nQuality: The submission is technically sound. The methodology is rigorously derived and empirically tested with various synthetic and real data. The work in this manuscript is a complete piece of work. The weaknesses of the work was not substantially evaluated, especially for non-meta-stable systems like continuous attractors. The experiments shown in the manuscript do not include a case that the number of state $Z$ is mismatched. Since the optimization of ELBO is generally a non-convex problem, the authors better talk about the convergence, e.g. empirically convergence reached and converging to the same or different optima with different initial values, and if the optimas are equivalent.\n\nClarity: Overall, the manuscript was well organized and written clearly. See itemized comments.\n- Line 202: Where are $A_p$ and $b_p$ defined? What does the subscript $p$ represent, prior? Are they assumed constant as $A$ and $b$ latter on.\n- Fig2 B: I suppose the curve attached to the right is the wells, but it is not described in the caption.\n\nSignificance: This work extends SLDS to continuous-time regime and proposes easy approximate inference via optimization. SLDS is well known in e.g. the field neuroscience for modeling meta-stable dynamics. This work would have high impact in such fields and be of great utility for the analysis and control of such hybrid systems.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors do not talk much about limitations or societal impact. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "YeNqUQLDKw", "reviewer": "Reviewer_d46C", "summary": "The paper proposes a variational inference for continuous-time switching dynamical systems, whose inference is believed to be lacking in need of biology and engineering applications. The form of the variational distributions is constrained by a master equation for the jump process and a Fokker-Planck equation for the diffusion process. These constraints are encoded in the Lagrangian. Both model parameters and variational distribution are inferred by a variational EM algorithm. The validity of the model is demonstrated with low-dimensional experiments.", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The derivation is very fundamental, and the statement is mathematically rigorous. It starts from the Markov process formulation and arrives at the inference algorithm for switching dynamical systems. My main concern is that whether such lower-level formulation is required for applications. The following are specific concerning points and please correct me if my understanding is wrong.\n- By the formulation, since continuous variables in both variational processes and the prior processes have linear drift, can we have explicit mean and variance representations for the diffusion process, e.g., Ornstein-Uhlenbeck process has an explicit time-dependent mean and variance? If this is possible, then we can work with a parametric Gaussian as the diffusion process and are free from the constraints.\n- The experiments are conducted in low-dimensional space, does it work in a high-dimensional setup? There are some works using diffusion processes for deep models or high-dimensional data. \n[Kurle, Richard, et al. \"Continual learning with bayesian neural networks for non-stationary data.\" International Conference on Learning Representations. 2019.]\n[Bamler, Robert, and Stephan Mandt. \"Dynamic word embeddings.\" International conference on Machine learning. PMLR, 2017.]\n ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I didn't find the limitation statements, although the checklist indicates there is one in a quite large page range. It would be better to describe the computation aspects and whether it fits modern high-dimensional applications.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "TGe3mI_8L1A", "reviewer": "Reviewer_czKb", "summary": "This manuscript concerns a continuous-time variant of switching state-space models, aka switching stochastic differential equations (SSDEs). Such hybrid models have a Markov jump process $z_t$ that modulates the diffusion process $y_t$. The contribution of this work is a new and unifying mean-field variational framework that leads to scalable approximation. In particular, time evolutions of the approximate posteriors over the switch variables $q(z,t)$ and the conditional $q(y,t|z)$ are computed individually. The final optimization objective is a Lagrangian consisting of the evidence lower bound (ELBO) and implicit-form ordinary differential equation (ODE) constraints describing the above-mentioned time evolutions. As usual, Lagrange multipliers are given by the optimality conditions defined by the Euler-Lagrange equations. The method is demonstrated on simple toy and real-world datasets with irregular observations. It is shown to consistently learn the continuous dynamics and infer accurate approximate posteriors over the switch $z_t$.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "With the neural ODE breakthrough and succeeding impressive NODE-based methodologies, continuous-time models have gained huge popularity. So, the manuscript studies a theoretically interesting and timely topic. Unlike existing approaches, the real-world applications of this work might be a little limited (perhaps for the time being) but the rigorous theoretical investigation easily outweighs it. \n\nThe proposed variational inference framework leads to computationally convenient expressions as shown by the authors, which is often rare in such hierarchical models. Therefore, although the inference framework overlaps with [20] in large parts, I find the contribution significant.\n\nThe authors motivate and describe the problem very well. Despite the heavy use of mathematics, I was able to follow the main arguments of the manuscript. So, I could say that it is written as clear as possible. Also, please note that I was not able to follow all the equations in the Appendix line by line but there should be no issue in the ones in the main paper.\n\nOverall, I recommend an accept as the submission is above average in all acceptance criteria. That being said, I am slightly concerned that the entire framework might rely on a wrong assumption. More specifically, is the factorization in eq.(12) correct? Should not one of the t's be on the rhs (to be conditioned to)? I wonder what the authors (and other reviewers) think about this.\n\nBelow are more detailed (minor) comments:\n- The approximations typically hold when the posterior concentrates on a single value of $z$. What if it does not? When does it tend to hold?\n- I wish to see the examples in which the framework starts failing, which would help us understand the limits of the method. Also, comparing the method with baselines would even further highlight its merits.\n- It would be much better if the authors can state which ideas are inspired by which papers. This would allow the reader to check reference papers more easily.\n- How important is the linear drift choice? How does the model behave with less/more data?\n- The derivation in Section3.2 is deeply connected to Pontryagin's maximum principle, which goes uncited. I suggest the authors check out the connections, which would allow the model to draw more attention.\n- Why is back-tracking line search needed? In general, how long does the optimization take? How sensitive is the optimization?\n- In line 242, do the left hand side expressions resemble Viterbi path? If so, the relationship should be shown.\n\n**Post-rebuttal comment:** Thanks for the rebuttal. All in all, I believe the paper should be accepted. It would be better if the following are addressed in the final version (or any arxiv submission or similar):\n- Lack of quantitative comparisons\n- References should be improved (I don't very closely follow the \"Switching Dynamical Systems\" literature but at least the VI framework is heavily inspired from [20], which is not explicitly mentioned in the text).\n- I still believe the factorization in eq.(12) is invalid. I'm starting to think this is just a notation issue.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "All the limitations and potential negative societal impacts are addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a model for continuous-time temporal datasets. The model takes the form of a Markov jump process that modulates a diffusion process. The work addresses posterior inference by applying a continuous-time variational inference algorithm that combines Gaussian processes for the diffusion process and posterior inference for the Markov jump process.

The reviewers agreed the paper is timely, significant, and relevant for the community, and that it deserves acceptance.

I encourage the authors to address the concerns raised by the reviewers. In particular:
+ Include a more thorough investigation of the parameter error in different regimes (i.e., dimensions, modes, separability of the modes). A discussion of which of these might be the limiting factor from a practical perspective would be useful for the readers.
+ Improve the references [R1-R9], including the ones in the discussion with reviewer d46C and specially the references about the variational inference method, e.g., mention [20] in the text.
+ Clarify the issue with the factorization in Eq. (12). The authors argued that it's just an attempt to make the notation uncluttered, but I agree with reviewer czKb that it creates confusion and it makes it look like the factorization is incorrect. Consider changing the notation accordingly.
+ Address the other changes that were discussed during the review period (e.g., extend Section 3.3, include an example of ELBO vs iterations, clarify the points about the drift function and about Figure 2, etc.).
Decision: Accept (Spotlight)