On the Stochastic Stability of Deep Markov Models
Authors: Jan Drgona
Sayak Mukherjee
Jiaxin Zhang
Frank Y Liu
Mahantesh Halappanavar
Venue: NeurIPS 2021 Poster (2021)
Keywords: markov models
deep neural networks
stochastic stability
dynamical systems

Abstract: Deep Markov models (DMM) are generative models which are scalable and expressive generalization of Markov models for representation, learning, and inference problems. However, the fundamental stochastic stability guarantees of such models have not been thoroughly investigated. In this paper, we present a novel stability analysis method and provide sufficient conditions of DMM's stochastic stability.  The proposed stability analysis is based on the contraction of probabilistic maps modeled by deep neural networks. We make connections between the spectral properties of neural network's weights and different types of used activation function on the stability and overall dynamic behavior of DMMs with Gaussian distributions. Based on the theory, we propose a few practical methods for designing constrained DMMs with guaranteed stability. We empirically substantiate our theoretical results via intuitive numerical experiments using the proposed stability constraints.

On the Stochastic Stability of Deep Markov Models
Ján Drgoña', Sayak Mukherjee', Jiaxin Jiaxxnn Zhang", Frank Liu?, Mahantesh Halappanavar"
Pacific Northwest National Laboratory
Richland, Washington, USA
Oak Ridge National Laboratory Oak Ridge, Tennessee, USA
(jan.drgona, sayak. mukherjee, mahantesh, halappanavar] @pnn1. gov (zhangī, Slufy)oonn..r
Abstract
Deep Markov models (DMM) are generative models that are scalable and expres- sive generalization of Markov models for representation, learning, and inference problems. However, the fundamental stochastic stability guarantees of such models have not been thoroughly investigated. In this paper, we provide sufficient condi- tions of DMM's stochastic stability as defined the context of dynamical systems and propose stability analysis method based on the contraction of probabilistic maps modeled by deep neural networks. We make connections between the spec- tral properties of neural network's 's weights and different types of used activation functions on the stability and overall dynamic behavior of DMMs with Gaussian distributions. Based on the theory, propose few practical methods for design- ing constrained DMMs with guaranteed stability. We empirically substantiate our theoretical results via intuitive numerical experiments using the proposed stability constraints.
Introduction
Modeling, analysis, and control of dynamical systems are of utmost importance for various physical and engineered systems such as fluid dynamics, oscillators, power grids, transportation networks, and autonomous driving, to name just few. The systems are generally subjected to uncertainties arising from a plethora of factors such as exogenous noise, plant-model mismatch, and unmodeled system dynamics, which have led researchers model the dynamics in stochastic frameworks. One of the most commonly used probabilistic frameworks to model dynamical system the Hidden Markov Models (HMMs) |Rabiner and Juang 1986 Eddy 1996) which in their vanilla form have been extensively investigated for representation, learning, and inference problems [Ghahramani and Jordan 1997 Cappé al. 2006 Beal et al.] 2202). One of its variants, the Gaussian state space models, have been used in the systems and control community for decades [Beckers and Hirche 2016 Eleftheriadis et al. 2017].
has been shown that expressivity of Markov models to emulate complex dynamics and sequential behaviors is greatly improved by parametrizing such models using deep neural networks, giving rise to Deep Markov Models (DMMs) [Krishnan et 2017). Main research activities have been focused the inference of these models. In particular, works such as Awiszus and Rosenhahn 2018) Liu et al. 2019) Musttaa et al. [0199] Qu et al. [2019) proposed parametrizing the probability distributions using deep neural networks for modeling various complex dynamical systems. Despite the rising popularity of DMMs, many of their theoretical properties such as robustness to perturbations, and stability of the generated trajectories remain open research questions. Many natural systems exhibit complex yet stable dynamical behavior that is described by converging trajectories towards an attractor set BBaaton and Tong 1979]. Additionally, safety-critical systems such autonomous
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

driving call for formal verification methods to ensure safe operation. Thus the ability to guarantee stability in data-driven models could lead to improved generalization or act as safety certificate in real-world applications.
In this paper, we propose new analytical method assess stochastic stability of DMMs. More specifically, we utilize spectral analysis of deep neural networks (DNNs) modeling DMM's distri- butions. This allows us to make connections between the stability of deterministic DNNs and the stochastic stability of deep Markov models. As a main theoretical contribution we provide sufficient conditions for stochastic stability of DMMs. Based on the proposed theory we introduce several practical methods for design of constrained DMM with stability guarantees. In summary, the main contributions of this paper include:
1. Stability analysis method for deep Markov models: we base the analysis on the operator norms of deep neural networks modeling mean and variance of the DMM's distributions.
2. Sufficient conditions for stochastic stability of deep Markov models: we show the suffi-
ciency of the operator norm- norm-based contraction conditions for DMM's deep neural networks by leveraging Banach fixed point theorem.
3. Stability constrained deep Markov models: we introduce a set of methods for the design
of provably stable deep Markov models.
4. Numerical case studies: we analyze connections between the design parameters of neural networks, stochastic stability, and operator norms of deep Markov models.
2 Related Work
Deep Markov models (DMMs) have been used as a scalable and expressive generalization of Hidden
Markov Models (HMM) for learning probabilistic generative models of complex high-dimensional
dynamical systems from sequential data (Rezende et al. 2014 Krishnan et al. 2017 Fraccaro et al.
2016). DMMs have been successfully applied speech recognition problems [i et al. [013
Prasetio et a control problems [Shashua and Mannor 2017], human pose forecasting Toyer et al.
2017]. fault detection [Wang et al.) 2018), climate data forecasting [Che al.) 2018). molccular
dynamics [Wu et al.] 2018]. or as internal models in model-based deep RL applied to automatic
trading [Ferreira| 2020]. Several modifications of DMMs have been proposed handle incomplete
data [Tan et a 2019]. or multi-rate time series [Che et al., 2018 multivariate time series [Montanez]
al. 2015], training DMMs in unsupervised settings [Trn et al 2016). or architectures inspired by
et al., 2019). However, works focusing on formal analysis to ensure stability guarantees for DMMs are missing.
Kalman Filters Krishhan et al.] [2015] [Shashua and Mannor
Stability notions and analysis for stochastic dynamic systems have been studied in the automatic control literature in various forms, depending on the representation of the system dynamics. Some classical results on stochastic stability for analysis and control can be found in McLanc [1971].
Willems and Willems [1976). These results are presented mainly for stochastic differential equation (SDE) models. Khasminskin kii [2012) discusses different notions of stochastic stability, among them mean-square based stability notions have gained interest in works such as Lu and Skelton (2002),
Farmer et al. 2009), EElia et al. [2013). Nandanoori et al. 2018). Wu et al. [2019]. We resort to such mean-square based stability notions when analyzing probabilistic state transition models parametrized by deep neural networks. We show that when the stochastic transitions are modeled by DNNs, the probabilistic stability requirements can be translated to deterministic stability notions of nonlinear discrete-time dynamics (Khalil] 2002).
In recent years, deep neural networks have been extensively studied from the viewpoint of dynamical systems [Chen et al. 2018 Raissi et al.) 2017 Ciccone et al. 2018)) allowing for the application of stability analysis methods to DNNs. For instance, Manck and Kolter (2019 proposed neural Lyapunov function to stabilize learned neural dynamics of autonomous system. Haber and Ruthotto [2177) interpret residual connections in neural networks as Euler discretization of ODEs and provide
stability guarantees of ResNet. Goel and Klivans [2017] makes connections between eigenvalue
decay and learnability of neural networks. [Engelken et al. (2020), Vogt et al. [2020] studies the
Lyapunov spectrum to of the input-output Jacobian of recurrent neural networks (RNNs) to assess
RNN's stability. this work we leverage advances the spectral analysis of deep neural networks
and apply them to derive stochastic stability guarantees for DMMs.
2

---

Besides analytical methods, many authors introduced provably stable neural architectures HHabe
al. 2019 Greydanus al. 2019 Cranmer al. 2020] stability constraints [John et al. 2017
Another popular strategy is to employ stabilizing regularizations. This can be achieved by minimizing eigenvalues of squared weights [Ludwig al., 2014], using symplectic weights [Haber and Ruthotto 2017], orthogonal parametrizations [Mhammedi et al., 2017]. Perron-Frobenius theorem [Tuor ct al. 2020], Gershgorin dics theorem [Lechner et al., 2020J, or via singular value decomposition (SVD) Zhang et al. 20188) In this paper we leverage different weight factorization empirically validate the proposed theoretical guarantees on stochastic stability DMMs.
Methodology
This section presents stochastic stability analysis method for deep Markov models (DMM). First, we demonstrate the equivalence of DNNs with pointwise affine (PWA) functions. Next, we recall definition of DMM with transition probabilities modeled by deep neural networks (DNNs). We introduce definitions of stochastic stability and show how can we leverage deterministic stability analysis in the probabilistic context. Finally, we will leverage the equivalence of DNNs with PWA maps to pose sufficient stability stochastic conditions for DMMs based on contraction of PWA maps.
3.1 Equivalence of Deep Neural Networks with Pointwise Affine Maps
Let us consider deep neural network (DNN) z/b
IR +
R m parametrized by 0g =
{A A, bo....,by b with hidden layers S l S L with bias given as follows:
uo,, (x) A 1/ hy + bb,
(la)
h v (AA h + b;-1)
(Ib)
with hn x. and v R. R. representing element-wise application of an activation function vector elements such that vzz) (.))))... v.2n,))
Lemma 1. For multi-layer feedforward neural network wo 3.2 with arbitrary activation futivationfuunctinn
v, there exists an equivalent pointwise affine map (PWA) parametrized by x which satisfies:
th (x) = (x)x (x).
(2)
Where As(x) is a parameter varying matrix given as:
Ax(x)== A A
A
(3)
And (x) is parameter varying vector, both parametrized by input vector given by following recurvent formula:
bx(x) = bY: by + (0) bi, E N byd A
(4)
(5)
with by bb, and representing index of the network layer. Here A represents parameter varying diagonal matrix of activation patterns defined as:
==)))=o(0)
To)
(z)) =
z +
A A
(6)
0 (3,) (0)
(10))
Zn
Proof. First lets observe the following:
Ta(21)
(2(0(2))-o(0)+
(=)))o(())
[0)
(z)) =
Z +
(7)
(2m))
za(o(2n. (0)
a c 1
((0))
2m
3

---

Remember (0) o(0) = 0, and 21 = are identity elements of addition and multiplication,
2;
respectively. Thus 7 demonstrates the equivalence (z) A (0) as given in (6). Then if
let Z = A x3 + bj, we can represent a neural network layer in a parameter varying affine form:
o. x bi) = A (A)x x + bi) + (0) = xxx + A + (0
(8)
Now for simplicity of exposition lets assume only activations with trivial null space, i.e. a
Thus (z)) z By composition, a DNN thg (x) can now be formulated parameter-varying affine map Aw (x)xx + by (x), parametrized by input
2 (x) = (xxx + (x)
bo) + +b).... bL-1)x bL A AW, (A AN (A A (AXx bo)
A (x)x = AY A A% A
(9)
bΦ (x) = AY A
A
A
A bo + +A
AT AW b AL (
AT DL-1 A AL br.
Hence, each input feature vector generates unique affine map A (x)x (x) of the DNN Vop (x). Thus proving the equivalence DNN map (3.2) with the form (2). The case with (0) + can be derived following the same algebraic operations above.
3.2 Deep Markov Models
We consider dynamical system with latent state variables x, E R", and the observed variables Yr E IR" The transition from X, to the next time step Xt+1, and the outputs yr are modeled by probabilistic transitions. Over horizon of T time steps with step size Al, assume the Markov property to embed structural independence conditions in the dynamic state evolution, i.c.
X 1 X X
(10)
Thus having latent state dynamics characterized by one-time-stey conditional distribution P(x++1|x)). The joint distribution over the latent states and the observations is given by,
T-1
P(xo.T, Yor) = P(xx) xo) II
P(x t=
(11)
More explicitly, we consider the following probabilistic transition and the emission maps:
Xx+1 -N(Ka At), Lx(xx, At))
(Transition) (Emission)
(12a)
Yt ~ ((((((())))
(12b)
with the initial condition X4=0 Xg. Here, .N and M denote the probability distributions. For the transition mapping, N' denotes Gaussian distribution with the mean vector K2 (x,, At), and covariances La(X,, At). The distribution M can be arbitrary with its distribution characterized by the map F,, (x.). In this paper, we are interested in the stability characterizations the latent state dynamics given by transition map 12a thereby assuming full state observability.
We interested in expressing the dynamics of complex systems using (12). therefore it is suitable to expand the expressivity of our model by parametrizing the conditional distribution P(x x,) by deep neural networks (DNNs) given as,
(x At) = fe, (xx), vec(L At)) ga, (x+),
(13)
(14)
where RD + R and g R R are two deep neural networks parametrized by 0r and 0g. respectively. And vec(.) denotes standard vectorization operation. Therefore, the probabilistic transition dynamics (Tab) can be characterized by analysing the stability and boundedness of deep neural networks f and g.

---

3.3 Stability of Deep Markov Models
To this end. we bring forth a few stability notions in the context of stochastic state transitions. In stochastic dynamics and control literature [Khasminskii, 2012]. various different notions of stability for stochastic state transitions, such as square stability, almost-sure stability and stability via convergence in probability, have been discussed. In this article, since we are interested in the latent state trajectories of the dynamic systems, we consider the mean-squar stability as defined in [Willems) and Willems) 1976. Nandanoori et al. 2018). The dynamic system is said to be mean-square stable if the first and the second moment converge over time.
Definition 2. The stochastic process X E R" is mean-square slable (MSS) if and only if there exists 1 RR Σ E RTx", such
such that 1o Rx(x)) μi, and lim [x(xx};) ) Σ.
The MSS condition from Definition 2 requires the dynamies (13 have stable equilibrium x, - μ, where & denotes the mean state vector. The definition also requires the covariance to converge. To express the dynamic behavior of complex system, the second moment convergence criterion can be relaxed by only requiring it to be norm bounded in order to ensure stochastic stability given as, (15)
Igo (xx)lpp < K, K 0, Vt.
||p denotes any appropriate vector norm, e.g., L. 12-norm. The bound on the covariances will depend on the extent of stochasticity that the dynamic system encounters in an uncertain environment. However, in the later parts, we consider the convergence scenario for as in Definition 2 rather than merely boundedness, in prescribing the sufficient conditions for the MSS-type stability.
Here ||
Let us first analyze the mean dynamics characterized by (33), and its equilibrium x, = which satisfies the stationarity condition for (Xe) = x.. We have the mean state vector x; evolving under the following dynamics:
Re+1 for (xt).
(16)
(6) allows to analyze the asymptotic stability of the DMM mean dynamics. Now in the main
result of this section we leverage the fact that the dynamic characteristics of the deep neural networks fo.. go, around point x, can evaluated by obtaining their exact pointwise affine forms (PWA) (
Based on this equivalence we formulate Theorem and Corollary S as follows with sufficient conditions for the stability of deep Markov models.
Theorem 3. The deep Markov model [12 which is parametrized by deep neural networks (3
remains globally stable in the mean-square sense if the following holds. The mean neural network for (x) is a contractive map for any x in the domain of for (x). The variance network go, (x) is bounded for any x in the domain of gig gxg(x).
(x). Or more formally:
|A((x)]pp <1
(17a)
|| (x) 1ipp= + [Ibg(x)lle < 1,
(17b)
||x||
Vx E Domain (x), gug (x)).
(17c)
Proof. First we prove the sufficiency of the contraction condition of the mean dynamics [7a). We
base the proof on the equivalence of multi-layer neural networks with pointwise affine maps An affine map a contraction if the 2-norm of its linear part bounded below one, i.e. |
< 1.
Thus it follows that the condition (7a) and equivalence (2 imply contractive mean neural network fa, (x). The sufficiency of the contraction condition on mean square stable (MSS) equilibrium in 2 follows directly from the Banach fixed-point theorem, which states that every contractive map converges towards single point equilibirum. Hence condition (17a) implies convergent mean transition dynamics:
the sense of Definition
f = fer lim for (Xx)
(18)
t-000
Now we show the sufficiency of (17b) to guarantee the boundedness of the covariance matrix elements (15) by bounding the p-norm of the covariance neural network ||g (x) Please note that
using the form 2 gives us [g (x)]lp ||Ag(x)x + (x(x))lp yielding following inequalities:
ggg (x)llp < (AAg())(]] + (b)(x)l
(19a)
|gg, (x)]lp
< ||A (x) + [x)lb
(19b)
|x|||
|x||
5

---

We show that (9bb gives in fact local Lipschitz constant of the variance network g0g (x). We
exploit the point-wise affine nature of the neural network's 's form 2 and the fact that the norm of
linear operator A is equivalent to its minimal Lipschitz constant K = A|Al|p [Huster et al. 2018).
Thus we can compute the local Lipschitz constants of a neural network go, (x) as:
[x]le
<²(()) =|Ag(())|e
(20)
||x||p
Applying the upper bound (T7b) on the local Lipschitz constant (20) guarantees the contraction of the variance neural network fog (x) towards fixed steady state E.
Remark 4.
To guarantee stochastic stability, the condition (17b) can be relaxed as given in 15 to
bounded second moment (15) with maxx (x*(x) < K, where K 7.0. O.
Corollary 5. The deep Markov model 12) which is parametrized by deep neural networks 13 14
remains globally stable in the mean-square sense if the following holds: All weights A of the mean network are fe, contractive maps. All activation scaling matrices AE, of the mean network non non-expanding. Norms of all weights A and activation scaling matrices A%, of the variance network gox are upper bounded by 1. Or more formally:
| lpp 1, ||4 ||p S Sii N
(21a)
| p < 1, | |p 1, jijjEN
(21b)
Vx E Domain [or (x), 89 (x)).
(21c)
Proof. First we show the sufficiency ||Ae|p < 1 of contractive weights and non-expanding activa- tion scaling matrices | ll guarantee the contractivity of arbitrary deep neural networks.
Assuming general non non-square weights A; R Ru we use the submultiplccatiiity of the induced p-norms to upper bound the norm of products of m matrices given as:
|| ...... ......Amly, lpp S ||Al|p || llp
(22)
Now by applying (22) to the linear parts (3) of the mean neural network fo, in the pointwise affine form (2) with ||A < 1, Vi E No. ||A lp < 1, Vj E Nf. it yields |Ar(x)]lp < 1 over the entire domain of fo, (x), thus with 2 satisfying the contraction condition ||A+(x)]la < 1 for affine maps. The submultiplicatiitty
22 naturally applies also the variance network go, (x) thus
implying the contraction towards fixed point given the conditions (T..
Remark 6. To guarantee stochastic stability, we can relax the upper bound of the second moment as K = II cAcA, where A > O. and сA > O represent the relaxed upper bounds of the operator
norms in condition 21b Thus satisfyyng the relaxed boundedness condition the variance via (15).
Assuming the contraction conditions (17) or (1 hold and the mean neural network for has zero bias,
then the DMM's mean network fo. (x) is equivalent with stable parameter varying linear map (3) with equilibrium
the origin, i.e. x = In the case with non-zero bias fer, the corresponding
PWA map (2 has non-zero equilibrium x O. Both conditions (17) (2I) are sufficient for
convergence of a DMM 12) to a stable equilibrium &. However, they do not provide bounds of the
admissible values of the equilibrium X. The corresponding equilibrium bounds are provided in the supplementary material.
3.4 Design of Stable Deep Markoy Models
In this section, we provide set of practical design methods for provably stable DMM (12). Based on the Corollary5]thee use contractive activation functions together with contractive weights for both mean and variance network will guarantee the stability of DMM by design. In particular, the conditions (T) on bounded norm of transitions' activation scaling matrices ||A S 1. | |l
implies Lipschitz continuous activation functions with constant < 1. Conveniently, this condition is satisfied for many popular activation functions such as ReLU, LeakyReLU, or tortan tanh. The
contractivity conditions (2T) on weight matrices || < 1, || 1I, < 1, respectively, can be enforced by employing various matrix factorizations proposed in the deep neural network litertature. Examples include singular value decomposition (SVD) [Zhang et al.) (018). Berron-Frobening
(PF) Tuor et al. 2020). and Gershgorin discs (GD) Lechner et al. 2020 factorizations given below.

---

PF weights: This factorization applies Perron-Frobenice theorem for constraining the dominant eigenvalue the square nonnegative matrices. Based on this theorem, we can construct the weight matrix A with bounded eigenvalues as follows:
M Amax (Amax (M)')
(23a)
AIj exp(A'ss) (
M.j.
(23b)
exp(AAas)
here M represents the damping factor parameterized by the matrix M' E RR=xx while A' E I2,x*, represents the second parameter matrix encoding the stable weights A. The lower and upper bound of the dominant eigenvalue are given by Amin and Amax. respectively.
SVD weights: Inspired by singular value decomposition (SVD), this method decomposes possibly non-square weight matrix A = UEV into two unitary matrices U and V, and a diagonal matrix Σ with singular values its diagonal. The orthogonality of U and V is enforced via penalties:
Lrgg
UU l2 + ||| U U. U + | VV T 1l2 + ||T V
Vila
(24)
An alternative approach penalties introduced in Zhang et al. [0188] is to use Houscholder reflectors to represent unitary matrices and V. The constraints Amio and Amax on the singular values can be implemented by clamping and scaling given as:
Σ diag( (Amax (Amax Amin) oA))
(25)
GD weights: This method supporting square matrices leverages the Gershgorin dics theorem [Varga 2004). It says that all eigenvalues A of the weight A can be bounded in the complex plane with center Ả and radius given by the formula:
= diag : :., M + diag (A,...,))
(26)
Where M E R2xn with ma, U(0, 1), except m4,i 0 is learnable parameter matrix. While diagonal matrices diag (5 ... 2.), and diag (A, X) represent radii and centers of the bounded cigenvalues, where 8j Zizj m.g-
The disadvantage of enforcing the global stability conditions as given via Corollary 55 their negative effect on the expressivity of the DMM, resulting in dynamics with a single point or line attractors. This will effectively prevent the DMM from expressing more complex attractors such limit cycles or chaotic attractors. As a more expressive alternative we introduce the use of parameter varying bounds in the conditions 17 and 21 such as:
Parametric stability constraints:
p(x) < ||r(x))l p(x)
(27)
Where p(x) RT* - R, and p(x) R + IR are scalar valued functions parametrizing lower and upper bounds of operator norm of the DMM's mean transition dynamies. Similar parametric constraints can be applied to the variance bounds in (17). . or weight norm constraints in (2T). This approach allows us control the contractivity of the DMMs depending on the position the state space. This allows us to increase the expressivity of the DMM, c.g., by partitioning the state space into constrained and unconstrained regions resulting in DMM with hybrid switching dynamics. In particular, we could divide the state space to outer contractive regions (where conditions (17) hold) and inner relaxed regions allowing for more complex trajectories emerge. This parametrization will effectively generate non-empty attractor set in which it is possible to learn arbitrary attractor shape. The proposed state space partitioning method is inspired by the Bendinxon- -ullc criteria on periodic solutions of differential equations [McCluskey and Muldowney 1998
4 Numerical Case Studies
In this section we empirically validate the conditions given in Theorem B and Corollary 5 by
investigating the dynamics of DMM's transition maps (12) whose mean fo, (x) and variance gog (x)
are parametrized by neural networks with different spectral distributions of their weights and activation
scaling matrices (6). We apply spectral analysis to the PWA forms (2) of neural networks modeling
the mean and variance maps to obtain the corresponding spectra DMMs. We performed the
experiments using the probabilistic programming language Pyro [Bingham et al. 2019).
7

---

4.1 Design of the Experiments
Since the stability DMM (12) depends on the transition dynamcis, in all of the case studies we consider a fully observable model with identity as emission map. We parametrize the mean and variances of the transition map fa, (x) 13 and ge, (x) (14) by feedforward neural networks. Given the mean neural network (x) we generate set of different transition dynameis by changing activation functions (ιr E {ReLU, Tanh, Si cmo cl SELU, Softplus} layer depth L E {1, 2,4, 8), and presence of bias E (True, Fal False). For the variance network geg (x) we use ReLU activations. For both, mean and variance networks we initialize their weights with desired spectral properties via design methods described Section 3.4 In particular we use SVD, PF, and GD factorizations to bound the weight's singular values in a prescribed range. We generate three categories of weights A stable with operator norm strictly below one ||A p < 1, marginally stable with norm close one ||Aillp 1, and unstable with norm larger than one | > 1.
4.2 Stability Analysis of Deep Markov Models
Stable mean
1
||
Marginal mean ||Ar|p R 1
SOERD
Unstable mean
|Arlp > 1
Stable var. Alg
Marginal var. ||Ag
Unstable var. |Agll
Figure 1: Phase portraits of DMMs demonstrating the effect of norm bounds on mean faş (x) and variance fo (x) networks modeling transition dynamics. Thin lines are samples of the stochastic dynamics with bold lines representing mean trajectories. Colors represent different initial conditions.
In order to provide intuitive visualisations of the dynamics in the phase space, in this section, we focus dimensional system. Fig. visualizes the phase portraits of randomly generated DMM's probabilistic transition maps of the mean fg, (x) and variance feg (x) networks with constrained operator norms enforced using PF weights from Section 3.4 Figures in the first row demonstrate that DMMs with asymptotically stable mean transition dynamics A|lr| < with bounded variances ||Agllp K, K generate stable single point attractors. Hence they validate the sufficient conditions of Theorem 3 Figures in the second row display dynamics of DMM with marginally stable mean ||Arl|p
Due the non-dissipativeness of the mean transition dynamics, the trajectories converge to line attractor only if the variance is converging map ||Aglp < 1, thus having a dissipative second moment. In case of marginally stable variance, ||A Ipp the energy conserving nature of the mean and variance together generate random walk type trajectories along the direction of the mean's line attractor. For the cases with unstable variance | > the overall dynamics behaves close to Brownian motion with degree of randomness, which is positively

---

correlated with the variance network's ' operator norm. Figures in the third row show diverging dynamics DMM with unstable mean | With converging variance ||Aglp the
diverging stochastic trajectories stay close to the mean direction. While, for both marginal and unstable variances the stochastic trajectories diverge all directions.
4.3 Effect of Biases and Depth the Stability of Deep Markov Models
(a)
(b)
(c)
Figure 2: Left panels show the effect of biases using PF regularization and ReLU activation ((a) w/o bias, (d) w bias). Right panels show the effect of network depths with SVD regularization and ReLU (b) layer, (c) layers, (e) layers, (f) 8 layers.
In Fig. we experiment with biases and depths of mean (x) and variance geg (x) networks.
Effeet of biases: In the left panels of Fig. 2 we demonstrate the dynamics of DMM with tanh activations and SVD factorized weights resulting asymptotically stable transition maps, thus ||A+(x)]| < 1, and [x)]] < 1. Fig. 2 a) shows the scenario without any bias whereas Fig. 4d) shows the scenario where both fo, (x), go (x) have bias terms. It demonstrates that the general contractive nature of the stable behavior, as given via conditions (17), does not change with addition
of biases. Instead, the biases shift the region of attraction by generating non-zero equilibrium points.
This shift is correlated with absolute value of the aggregate bias term of the PWA form (2). For the norm bounds on the equilibria stable DMM see supplementary material.
The right panels of Fig. demonstrate the dynamics of DMMs with increasing number of layers using ReLU activations and SVD weights close marginal stability 0.99 < ||A[()]| 1. It can be seen that with increase in the number of layers, the convergence of trajectories toward origin becomes less uncertain. In this case, larger number of mildly contractive layers results in stabilizing behavior, demonstrating the effect of the norm submultiplicativity operator norm the mean transition dynamics (22). However, one needs to careful about the delicate balance between stability, and ability to efficiently train the parameters of DMM with gradient- optimization. With increasing depth, the norm product of the contractive layers (22) will eventually result in network with very small operator norm thus causing the vanishing gradient problem. Analogously, the exploding gradient problem will occur for DMM parametrized with very The use of parametric stability constraints (27) as a function of depth could be an efficient strategy for avoiding the vanishing and exploding gradients by keeping the overall dynamics norm bounded.
Effect of depth:
deep neural networks with non-contractive layers, i.e. [AA,()))
4.4 Deep Markov Models with Parametrized Stability Constraints
In Fig. we demonstrate the use of parametrized stability constraints 27 the design stable DMMs without compromising the expressivity as it is in the case of restrictive single point attractors

---

enforced via 17 and 21 In particular, we design two DMMs with randomly generated weights with three phase space regions with different mean transition dynamics, (i) an inner expanding region [x)+(x)|| > 1, x' E R1, (ii) middle marginal region || II| ~ 1, x' E R2, and (iii) outer contractive region Ar(x*) II < 1, x² E Rg. Where the regions are given as R1 (x[0 - ||x||2 < 20). R2 [x|20- & |
(0 I ||x|2] respectively. The variance dynamics in both cases is kept being contractive |[Ag(x') (xa)l < 1. From Fig. is apparent that the overall dynamics of the DMMs with parametrized constraints (27) is able generate stochastic periodic behavior while remaining bounded within prescribed region of attraction, thus providing high degree of expressivity while being provable stable. On the left (Fig. (a) and (c)) show phase plots, and on the right (Fig. (b) and (d)) corresponding time series trajectories. As a potential extension, we envision learning the constraints bounds p(x), and p(x) in 27 using penalty methods.
40), and R3 (x|40
30
20
23
-40
-20
20
(b)
kgth
20
30
55
-40
-20
20
40
(c)
(d)
Figure 3: Phase plots (left panels), and time series trajectories (right panels) of DMMs with parametrized stability constraints exhibiting periodic behavior within bounded regions of attaction. Both cases consider SVD regularizations with softplus (top) and SELU(bottom) respectively.
5 Conclusion
In this paper, introduce As the main result, we provide sufficient conditions for the stochastic stability and introduce a of practical methods for designing provably stable DMMs. In particular, of contractive weight matrices factorizations and stability conditions for the activation functions. Furthermore, we propose using novel parametric stability constraints allowing expression of more complex stochastic dynamics while remaining contractive towards non-empty region of attraction. The proposed theory is supported by numerical experiments, with design guidelines considering weight factorizations, choices of activation functions, network depth, or use of the bias terms for guaranteed stability. future work, we aim derive the stability guarantees for broader family of probability distributions modeled by normalizing flows. We also aim to expand the theory to partially observable Markov decision processed (POMDP) to derive closed-loop stability guarantees in the context of deep reinforeement learning.
stability analysis method for deep Markov models (DMMs).
discuss the use
10

---

Acknowledgments and Disclosure of Funding
We acknowledge our colleagues Aaron Tuor, Mia Skomski, Soumya Vasisht, and Draguna Vrabie for their contributions related topics that served as base for developing the method presented in this paper. We would like to thank Craig Baker and David Rolnick for fruitful discussions that helped improve the technical quality of the presented ideas. Also, want thank anonymous reviewers for their constructive feedback and suggestions.
This research was supported by the U.S. Department of Energy. through the Office of Advanced Scientific Computing Research's "Data-Driven Decision Control for Complex Systems (DnC2S)" project. Pacific Northwest National Laboratory is operated by Battelle Memorial Institute for the U.S. Department Energy under Contract No. DE-ACOS-76RLO188 Oak Ridge National Laboratory is operated by UT-Battelle LLC for the U.S. Department of Energy under contract number DE-AC05- 00OR22725.
6 Limitations and Broader Impact
Stability is the major concern of many safety-critical systems. The significance of the presented work lies in the proposed stability conditions, analysis, and design methods allowing the generation of provably stable DMMs. Furthermore, the presented methods for the design of stable DMMs could be of large significance in the context of stochastic control systems with stability guarantees. Thus, the authors believe that besides academic relevance, the methods presented in this paper have the potential for practical impact in many real-worke applications such unmanned autonomous vehicles, robotics, or process control applications.
The authors are aware of the limitations of the presented numerical case studies focusing on small- scale DMMs with two-dimensional state space. This choice was made for the sake of the visualizations of the state space trajectories allowing us to provide intuitive examples the presented theoretical results. As part of the future work, the authors plan to use the proposed stability analysis and design methods for DMMs on real-world datasets with higher dimensional state space.
The presented work falls into the basic research category. As such, authors are not aware of any potential direct negative societal impact of the proposed work. On the contrary, authors of this paper believe that the presented theory is a minor contribution towards general knowledge, which accumulation has been historically proved to inherently benefit all humanity.
References
M. Awiszus and B. Rosenhahn. Markov chain neural networks. In Proceedings of the IEEE Conference on Computer Vision and Patlern Recognition Workshops, pages 2180-2187, 2018.
M. Beal, 7. Ghahramani, and C. E. Rasmussen. The infinite hidden markov model. Advances in neural information processing systems, 1:577-584, 2002.
Becker, H. Pandya, G. H. W. Gebhardt, Zhao, C. Taylor, and G. Neumann. Recurrent kalman networks: Factorized inference in high-dimensionall deep feature spaces. CoRR, 0bbs/1995.0775 2019. URL/ttpp://arxt
wwtp/r///abbbbaaeeeeeeee /aabs//99.... C7357
T. Beckers and S. Hirche. Stability of gaussian process state space models. In 2016 European Control Conference (ECC), pages 2275-221522 22211. IEEE, 2016.
E. Bingham, J. P. Chen, M. Jankowiak, F. Obermeyer, N. Pradhan, Karaletsos, R. Singh, P. Szerlip. P. Horsfall, and N. D. Goodman. Pyro: Deep universal probabilistic programming. The Journal of Machine Learning Research, 20( (0(1:9733-978, 2019.
R. Brayton and C. Tong. Stability of dynamical systems: constructive approach. IEEE Transactions on Circuits and Systems, (6(4):224-234 1979. do. 109/TT...
197999 1979119999991 1084637.
O. Cappé, E. Moulines, and T. Rydén, Inference hidden Markov models. Springer Science Business Media, 2006.

---

Z. Che, S. Purushotham, G. Li, B. Jiang, and Y. Liu. Hierarchical deep generative models for multi-rate multivariate time series. In Dy and A. Krause, editors, Proceedings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 784-793, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR. URL
.mlr, press/ /V00 /B0/ che Ba html
R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31, pages 6571 Cur- ran Associates, Inc., 2018. URL
wewpeeeeiiiinss.eneeeeeeeeeeeeee wwtpriiss.ccppppppeeepmp cc/ppppee//11////// pdf
21le/693866
M. Ciccone, M. Gallieri, J. Masci, C. Osendorfer, and F. J. Gomez. Nais-net: Stable deep networks from non-autoonomous differential equations. CoRR, 0bs//104.07209, 2018. URI. http://
arxiv. wtt.vv/.rgg/bb//////oe/ 804 07209
M. Cranmer, S. Greydanus. S. Hoyer, P. Battaglia, D. Spergel, and S. Ho. Lagrangian neural networks. arXiv preprint arXiv:2003.04633 046630 2020.
S. R. Eddy. Hidden markov models. Current opinion in structural biology, 6(3):361-365, 1996.
S. Eleftheriadis, T. Nicholson, M. P. Deisenroth, and J. Hensman. Identification of gaussian process state space models. In NIPS, pages 5309-5319, 2017.
N. Elia, J. Wang, and X. Ma. Mean square limitations of spatially invariant networked systems. In Control of Cyber-Physical Systems, pages 357-378. Springer, 2013.
R. Engelken, F. Wolf, and L. Abbott. Lyapunov spectra of chaotic recurrent neural networks. arXiv preprint Xii:22006.02427,, 2020.
R. E. Farmer, D. F. Waggoner, and Zha. Understanding markov-switching rational expectations models. Journal of Economic theory, 144(5):1849-186) 2009.
T. A. Ferreira. Reinforced deep markov models with applications in automatic trading. arXiv preprint wxXiv:2011.0439) 2020.
M. Fraccaro, S. K. Sønderby, U. Paquet, and O. Winther. Sequential neural models with stochastic layers. arXiv preprint arXiv: 1605.0051, 2016.
Z. Ghahramani and M. I. Jordan. Factorial hidden markov models. Machine learning. 2922):45--77,, 1997.
S. Goel and A. Klivans. Eigenvalue decay implics polynomial- -ime lcarnability for neural networks. In Advances in Neural Information Processing Systems, pages 2192-2202, 2017.
S. Greydanus, M. Dzamba, and J. Yosinski. Hamiltonian neural networks. CoRR, abs/1906.01 996.00163. 2019. URL/ttps///errx..orr//////1.... C1563
E. Haber and L. Ruthotto, Stable architecctures for deep neural networks. Inverse Problems, 34(1): 014004, 2017.
03341
E. Haber, K. Lensink, E. Treister, and L. Ruthotto. Imexnet: forward stable deep neural network. CoRR, abs/1903.02633, 2019.
org/ /tggabs/////eee...////////////a/ 1903. 02639
T. Huster, C.-Y. J. Chiang, and R. Chadha. Limitations of the lipschitz constant as defense against adversarial examples. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 16-29. Springer, 2018.
V. John, A. Boyali, and S. Mita. Gabor filter and gershgorin disk-based convolutional filter constrain- ing for image classification. Ini. Mach. Learn. Comput, 7(4):55-60, 2017.
H. Khalil. Nonlinear Systems. Centice-Hall, New York, 2002.
R. Khasminskii. Stochastic Stability of Differential Equations, volume 66. Springer, 2012.
12

---

R. Krishnan, U. Shalit, and D. Sontag. Structured inference networks for nonlinear state space models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
R. G. Krishnan, U. Shalit, and D. Sontag. Deep kalman filters. arXiv preprint arXiv: 2015.
M. Lechner, R. Hasani, D. Rus, and R. Grosu. Gershgorin loss stabilizes the recurrent neural network compartment of an end-to-end robot learning scheme. In 2020 International Conference on Robotics and Automation (ICRA). IEEE, 2020.
Li, Zhao, D. Jiang. Y. Zhang, F. Wang, I. Gonzalez, E. Valentin, and H. Sahli. Hybrid deep neural network-hidden markov model (dnn-hmm) based speech emotion recognition. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, pages 312-317, 2013. 1o0110.109/ACI122000
D. Liu, A. Honoré, S. Chatterjee, and L. K. Rasmussen. Powering hidden markov model by neural network based generative models. arXiv preprint 1rX1v.1910.05744, 2019.
J. Lu and R. E. Skelton. Mean-square small gain theorem for stochastic control: discrete-time case. IEFE Transactions Automatic Control, 47(3):490-494 2002.
O. Ludwig, U. Nunes, and R. Araujo. Eigenvalue decay: A new method for neural network regularization. Neurocomputing, 124:33-42, 2014.
G. Manek and J. Z. Kolter. Learning stable deep dynamics models. In Advances in Neural Information Processing Systems 32, pages 11126-11134 2019. URL /tap::r//ppprs..... nips. cc/ paper/ mode pddf
9292-1earni htaprr/ggeettbbeeeeeeeeeeeaaaaee
C. C. McCluskey and J. S. Muldowney. Bendixson-dulac criteria for difference equations. 10(4):
567-575, 1998. ISSN 1572-9222, dbi100.1023/A1102267003333 URL URLhttts::/doi.crr
10.1023/A: 1
P. McLane. Optimal stochastic control of linear systems with state-and control-dependen disturbances.
IEEE Transactions on Automatic Control, 16(6):793-798, 1971.
Z. Mhammedi, A. Hellicar, A. Rahman, and J. Bailey. Efficient orthogonal parametrisation of recurrent neural networks using householder reflections. In International Conference on Machine Learning, pages 2401-2409. PMLR, 2017.
G. Montanez, S. Amizadeh, and N. Laptev. Inertial hidden markov models: Modeling change in multivariate time series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
M. K. Mustafa, T. Allen, and K. Appiah. A comparative review of dynamic neural networks and hidden markov model methods for mobile on-device speech recognition. Neural Computing and Applications, 31(2):991-899 2019.
Nandanoori, A. Diwadkar, and U. Vaidya. Mean square stability analysis of stochastic continuous- time linear networked systems. IEEE Transactions on Automatic Control, 63(12):4323 4330, 2018.
B. H. Prasetio, H. Tamura, and Tanno. Deep time-delay markov network for prediction and model- ing the stress and emotions state transition. 10(1):18071. ISSN 2045-2322. 10.1//4403//41133 020-75155-w URL http::tt/:/// hww. org. 1tt.::...1111111111111 020-1555---277755----
M. Qu, Y. Bengio, and J. Tang. Gmnn: Graph markov neural networks. International conference machine learning, pages 5241-5250. PMLR, 2019.
L. Rabiner and B. Juang. An introduction to hidden markov models. ieee assp magazine, 3(1):4-16. 1986.
M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics informed deep learning (part I): data- driven solutions of nonlinear partial differential equations. CoRR, abs/1711.10055 2017. URL http: httt:///rxivve arxiv. org/ argg/bs//ie abs/1711 10561
13

---

D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning. pages 1278-1286 PMLR, 2014.
S. D. Shashua and S. Mannor. Deep robust kalman filter. CoRR, abs/1703.02310, 2017. URL wwtp////g////////..//.e/eee...//
abs/1 72s./011000
Z. Tan, H. Soh, and D. C. Ong. Factorized inference in deep markov models for incomplete multimodal time series. CoRR, abs/1905.13570, 2019. URL http::/arxiv.or//bbba
1905. 13570
Toyer, A. Cherian, T. Han, and S. Gould. Human pose forecasting via deep markov models, 2017.
K. M. Tran, Y. Bisk, A. Vaswani, D. Marcu, and K. Knight. Unsupervised neural hidden Markov models. In Proceedings of the Workshop Structured Prediction for NLP. pages 63-71, Austin, TX, Nov. 2016. Association for Computational Linguistics. doi 10.18653/v 16-5907 URL
https: // www aclweb org/ /athhooohoogyy. /w R16-1 5907
A. Tuor, J. Drgona, and D. Vrabie. Constrained neural ordinary differential equations with stability guarantees. arXiv preprint WXXv:2004.10883 2020.
R. Varga. Gersgorin and His Circles, volume 36. Springer, Berlin, Heidelberg, 01 2004. 11011101007/98--6442221111
R. Vogt, M. P. Touzel, F. Shlizerman, and G. Lajoie. On Iyаpunov exponents for mnss: Understanding information propagation using dynamical systems tools. arXiv preprint arXiv:2006. 14123, 2020.
S. Wang, J. Xiang, Y. Zhong, and Y. Zhou. Convolutional neural network-based hidden markov models for rolling element bearing fault identification. Snowledge-Based Systems, 144:65 76, 2018. ISSN 0950-7051. /tsptt/s//oooorr///000/////ooo./
URL http:
//ww.scceencdirecs com/ wtt.:///w............//////////0
L. Willems and J. C. Willems. Feedback stabilizability for stochastic systems with state and control dependent noise. Automatica, 12(3):277 283, 1976. ISSN 0005-1098. doi http:://o..ooo///000000000000100 com/ sci
URL http: / www. www.seeenceeieettt
B. Wu, M. Cubuktepe, and U. Topeu. Switched linear systems meet markov decision processes: Stability guaranteed policy synthesis. 2019 IEEE 58th Conference on Decision and Control (CDC), pages 2509-2516. IEEE, 2019.
H. Wu, A. Mardt, L. Pasquali, and F. Noe. Deep generative markov state models. arXiv preprint arXiv: 07601, 2018
J. Zhang, Q. Lei, and Dhillon. Stabilizing gradients for deep neural networks via efficient svd parameterization. In International Conference on Machine Learning, pages 5806-5814, 2018.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] We believe the abstract and the introduction are accurate representation of the paper and that the claims are supported by the presented
methods and case studies.
(b) Did you describe the limitations of your work? [Yes] Please see the limitations section.
(c) Did you discuss any potential negative societal impacts of your work? [N/A] We presents fundamental theoretical work, which by itself has no negative implications towards societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
14

---

(a) Did you state the full set of assumptions of all theoretical results? [Yes] Please see the definitions in the methodology section and in the supplementary material.
(b) Did you include complete proofs of all theoretical results? [Yes] Please see the methodology section and supplementary material. 3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material plan to open-source the code with the camera ready version of the paper. The reason why we do not disclose the case study code during the review process is the of custom open- open-source libraries that could reveal the affiliation of the authors.
URL)? [No] The authors
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We provide the summary of the experiment hyperparameters in the supplementary material.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [N/A]
4. If you are using existing assets code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We performed the experiments using the probabilistic programming language Pyro Bingham et al. [2019]
which we cite in the paper.
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you're 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "oHvglrNkwUR", "reviewer": "Reviewer_arwr", "summary": "The paper discusses stable deep Markov models (DMM), which can be used to model stochastic dynamical systems, restricting to the case where state transition probabilities are given by normal distributions with mean and covariance matrices parameterized by two deep neural nets.\nOne of the main contributions of this paper is to study sufficient conditions under which such DMMs are stable (in the mean-square sense), the second is to provide a list of practical (re)parameterization of the DNNs that satisfy the sufficient conditions. \nThe paper concludes with a series of small scale numerical experiments aimed at showing the possible behaviours of the class of proposed models.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Clarity:\nI found the paper very well written (found only one typo in Eq (7), missing t=0) and generally easy to follow.  \nThe work is well motivated; I appreciated that the proofs of the main results are in the paper, and  I found section 3.4., particularly informative and well organized, linking theoretical findings to practical design principles.\nA few comments regarding clarity:\n- In the proof of lemma 1, in eq (4), it is not clear whether the bias terms are included or not, especially since in line 95 there is written $z_{l+1} = A_l h_l$ whereas I would have expected to see also the bias in $z$.\n- In theorems 4 and 5, since the statements are about mean-square stability, perhaps it would be better to state in the hypotheses that K<1 and $c^A c^{\\Lambda}<1$ and then in the proof (or in a remark after those) relax these conditions referring to eq (11).\n- (Question) It is not clear to me how the parametric stability constraints (also used in sec 4.3) relate to the hypotheses of the theorems. \n- The experimental section is the only section that I found rather difficult to follow, this is because: (i) there is no description, however short, of the experimental settings, and (ii) I think the discussion could be a bit more focused, better accompanying the reader on what the should look at. For (i), I would recommend making a bit of space for short descriptions of the setting in the main paper.\n\nOriginality: \nI find the contributions of this work overall novel, although not surprising, as they are in a way adaptations of known facts that have been studied in other circumstances (as per related section and also see added references below, which could be added to the related work). I think however the results are not obvious, and hence they deserve publication. In section 3.4, the paragraph which I found most original (and interesting) is the last one, which probably deserves some more discussion. \n\nSignificance:\nI believe the theoretical results and especially the design principles will be useful in the ML community and beyond; this paper may become a handy reference for DMM designers.\n\nQuality:\nThe mathematical treatment is clear and precise, and the results look correct to me. \nThe experimental section could be improved, by describing the experimental settings (as in the clarity section) and by including some quantitative comparative results about optimizing with various parameterizations (ideally this should be addressed in the rebuttal).\nRelated literature is discussed well enough, but perhaps the authors could consider including also works with deterministic models e.g. [1] and [2] (in particular, in [2] there is a discussion about equilibrium models and stability thorough projections which could complement the discussion of section 4.3.)\n\n\nReferences:\n[1] Miller, J. and Hardt, M.  Stable recurrent models. ICLR, 2019.\n[2] Grazzi, Riccardo, et al. \"On the iteration complexity of hypergradient computation.\" International Conference on Machine Learning. PMLR, 2020.\n\n____\nPost-rebuttal\n\nThanks for your reply. I am keeping my score and recommend acceptance.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Addressed ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "C58Ba2SNC9P", "reviewer": "Reviewer_BjNt", "summary": "This paper analyzes deep Markov models from a theoretical perspective. More specifically, it derives conditions on the construction/parameters of models which allow for a specific notion of stochastic stability. This is accomplished by first proving that a multilayer perceptron can be equivalently represented as a pointwise affine map (PWA). It is then proven that achieving specific properties on the norms of these PWAs leads to global stability, and that bounding the norms of the layers in neural networks leads to achieving these desired properties within the corresponding PWAs. There is a discussion on how one can achieve these properties when training a neural network, followed by empirical analysis demonstrating the connections between the properties of DMMs and their stability.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Originality: \nI am unaware of prior work which analyzes DMMs in this manner. Connections to prior work are made clear.\n\n### Quality: \nAs far as I could tell, the analysis done in this paper is sound. The conclusions regarding the analysis, and suggestions are made/analyzed regarding how to construct/train networks that achieve the required properties. The required assumptions are made very clear, and the authors are open and honest about the limitations of the current analysis.\n\n### Clarity: \nThe paper is clearly written and straightforward to follow. All important properties are defined, and the proofs are organized in a logical order. The empirical analysis is well-visualized, and the points being made are conveyed well by the plots.\n\n### Significance: \nAssessing the properties of deep neural networks and their use in a variety of modeling frameworks is an open field, and understanding the theoretical properties of networks and how to achieve desired configurations can help lead to better performing models. The analysis presented here is another step in this direction, and thus I imagine many will find it interesting.\n\n### Miscellaneous Comments\nEquation 7 needs to be rewritten slightly - for example, $P(\\mathbf{y}_0|\\mathbf{x}_0)$ appears twice on the right side the way it is currently written.\n\n# POST-REBUTTAL UPDATE\nThanks for the responses. After reading other reviews and the responses to each of them, I am maintaining the original score I gave. I think this paper makes a valuable contribution and should be accepted.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations are presented clearly and thoroughly. I agree with the authors that this work is basic science and does not have any direct connections to societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5 hours"}, "extra_text": {}}, {"review_id": "7Pi3VVHXCX", "reviewer": "Reviewer_fyK8", "summary": "This paper analyses stochastic stability of Deep Markov Models (DMMs) based on the contraction of probabilistic maps modeled by Deep Neural Networks (DNNs). To this end, DMMs are modeled using latent space representations of a dynamical system, and Gaussian distribution is used in modeling the transition and emission dynamics through DNN representation. With this modeling assumption, the authors perform stochastic stability analysis of DMM focused on bounding the norm of the model parameters of DNN. They state sufficient conditions for achieving stability and outline techniques for doing so. Finally, their claim is substantiated through small-scale experimentation of DMMs.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper analyses the stability of Deep Markov Models (DMMs) and provides sufficient conditions for stochastic stability of DMM. The authors perform mean-squared based stability analysis by using Deep Neural Networks (DNNs) to parameterize the probabilistic state transitions. In doing so, deterministic stability analysis is used in a probabilistic setting.\n\nTheir proof is based on modeling DMMs as a dynamical system with latent space variables and using the property that a dynamical system is said to be mean-square stable if its first and second moment converge with time. This leads to equivalent stability conditions outlined in Theorem 4 and Corollary 5 relying on contractive mapping of the DNN (modeling the DMM). These conditions are sufficient for the convergence of a DMM to a stable equilibrium. The proof is simple and naturally flows from existing literature dealing with probabilistic transitions in a dynamical system. \n\nIn my opinion, the main crux of the paper lies in using the sufficient conditions (in Theorem 4 and Corollary 5) to outline the methods for designing stable DMMs by enforcing the contractivity of the weight matrices through different matrix factorizations. More importantly, the discussions on how some design considerations can have a negative impact on the expressivity of DMMs and can be ameliorated using parametric stability constraints have value in real-life applications. Going along with this, the numerical case studies in Section 4 shows a nice intuitive illustration of the ideas that have been theoretically proven earlier.\n\nWhile the paper has potential in motivating design of (mean)-stable DMMs for real-life applications, there is little discussion connecting how much validity mean-stability analysis has in real-life applications. Also, there could have been more literature review and basics of spectral analysis of DNNs. This is not very clear at the moment. \n\nIn spite of the strong points, my main concern about the paper remains about the applicability of the design choices (Section 3.4) with respect to large-scale dynamical systems. It would have been better to discuss the limitations with more reference to the literature and state the possible use cases where this kind of design might not be feasible.\n\nMinor comments: The paper requires proofreading as there are typos, repeated abbreviations, and grammatical errors.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "While the authors have done a good job in outlining a good theoretical framework for their work, the practical implications are somewhat lacking. From a practical point of view, it would have been good if they would have shown how their findings could be applied to a DMM in real-life. My main concern is that in large-scale systems where it might be computationally challenging to apply the design principles that have been stated in the Section “Design of Stable Deep Markov Models”. It would have been better if there was a section titled “Threats to Validity” for discussing the above issue.", "review_text": "", "extra_scores": {"ethical_concerns": "This paper is a basic research paper and does not raise any ethical concern that I am aware of.\n", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper analyzes the stability properties of _deep Markov models_ (DMMs), which are nonlinear dynamical systems that are parameterized by deep neural networks (basically, replacing the transition/emission functions of an HMM with neural nets). The form of stability used in this paper is _stochastic stability_, which essentially means that the first and second moments of a dynamical process asymptotically converge. (Note that this is different from the notion of _algorithmic stability_ used in learning theory.) The paper provides sufficient conditions under which a DMM is stochastically stable, and then analyzes how various architectural considerations (activation, weights, depth, etc.) affect stability. A numerical study rounds out the picture.

The reviews were unanimously positive, with 7s (Accept) across the board. The theory is sound; the work feels complete; and the paper is well written and easy to follow, with minor typos. None of the reviewers were all that confident in their assessments (perhaps this is a niche subject for the NeurIPS audience), but all agreed that it's definitely a solid paper.

In my opinion, the main weakness (echoed by Reviewer fyK8) is that not much discussion is devoted to motivating stability. Why is stochastic stability important in applications of dynamical systems? The introduction ignores this question and starts from the assumption that stability is important and, therefore, worth studying. As far as I can tell, the only discussion of why stability is important comes up at the end, in the Broader Impact statement, which only says, "Stability is the major concern of many safety-critical systems. ... [T]he methods presented in this paper have the potential for practical impact in many real-world applications such as unmanned autonomous vehicles, robotics, or process control applications." OK, but why? What would happen if an _unstable_ system were deployed in an autonomous vehicle? How specifically does stochastic stability ensure safety in an autonomous vehicle? I can kind of fill in the blanks myself, but I'd like the paper to be more explicit about how this highly theoretical study has practical impact.

One other thing I'll note is that the term "stability," in the context of ML Theory, typically implies something different from stochastic stability (which comes from Control Theory). Readers familiar with _algorithmic stability_ (see Bousquet & Elisseeff, JMLR 2002) may be confused or misled by the title/abstract. I recommend adding something in the abstract and intro that (a) briefly describes stochastic stability and (b) makes it clear that it is not the same as algorithmic stability.
Decision: Accept (Poster)