Learning Neural Contracting Dynamics: Extended Linearization and Global Guarantees
Authors: Sean Jaffe
Alexander Davydov
Deniz Lapsekili
Ambuj Singh
Francesco Bullo
Venue: NeurIPS 2024 poster (2024)
Keywords: contraction theory
learning from demonstration
dynamical systems
Primary area: deep_learning_architectures
TL;DR: We learn globally gauranteed contractng dyanmical systems by parameterizing the extended linearization of the vector field.

Abstract: Global stability and robustness guarantees in learned dynamical systems are essential to ensure well-behavedness of the systems in the face of uncertainty. We present Extended Linearized Contracting Dynamics (ELCD), the first neural network-based dynamical system with global contractivity guarantees in arbitrary metrics. The key feature of ELCD is a parametrization of the extended linearization of the nonlinear vector field. In its most basic form, ELCD is guaranteed to be (i) globally exponentially stable, (ii) equilibrium contracting, and (iii) globally contracting with respect to some metric. To allow for contraction with respect to more general metrics in the data space, we train diffeomorphisms between the data space and a latent space and enforce contractivity in the latent space, which ensures global contractivity in the data space. We demonstrate the performance of ELCD on the high dimensional LASA, multi-link pendulum, and Rosenbrock datasets.

Learning Neural Contracting Dynamics: Extended Linearization and Global Guarantees
Sean 12+ Alexander Davydov', Deniz Lapsekili" Ambuj K. Singh", and Francesco Bullo'
Center for Control, Dynamical Systems and Computation, University of California, Santa Barbara. Department of Computer Science, University of California, Santa Barbara.
Abstract
Global stability and robustness guarantees in learned dynamical systems essen- tial to ensure well-behavedness of the systems in the face of uncertainty. We present Extended Linearized Contracting Dynamics (ELCCD, the first neural network-based dynamical system with global contractivity guarantees in arbitrary metrics. The key feature of ELCD is parametrization of the extended linearization of the nonlinear vector field. In its most basic form, ELCD is guaranteed to be (i) globally exponentially stable, (ii) equilibrium contracting, and (iii) globally contracting with respect to some metric. To allow for contraction with respect to more general metrics in the data space, we train diffeomorphisms between the data space and a latent space and enforce contractivity in the latent space, which ensures global contractivity in the data space. We demonstrate the performance ELCD on the high dimensional LASA, multi-link pendulum, and Rosenbrock datasets.
Introduction
Due their representation power, deep neural networks have become popular candidate for modeling continuous-tin catiiuoous-tmme dynamical systems of the form
dx dt = f(x),
(I)
where f(x) is an unknown (autonomous) vector field governing a dynamical process. Beyond approximating the vector field f. it desirable to ensure that the learned vector field well-behaved. In many robotic tasks like grasping and navigation, a well-behaved system should always reach fixed endpoint. Ideally, learned system will still stably reach the desired endpoint even when pushed away from demonstration trajectories. Additionally, the learned system should robustly reach the desired endpoint in the face of uncertainty. In tasks such as manufacturing, animation, and human-robot interaction, functionality and safety require the learned system must smoothly follow specific trajectory its target.
To enforce stability guarantees, popular approach has been to ensure that the learned dynamics admit a unique equilibrium point and have Lyapunov function, c.g. [30]. While popular, approaches based on Lyapunov functions typically struggle to provide general robustness guarantees since global asymptotic stability does not ensure robustness guarantees in the presence of disturbances. Indeed, input-to-state stability of global asymptotically stable dynamical systems needs to be separately established, e.g., see Chapter in [21].
To ensure robustness and to allow for smooth trajectory following, there has been increased interest in learning contracting dynamics from data [6]. A dynamical system is said to be contracting if any
Conresponding author: a sjaffceuesb.edu
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

two trajectories converge to one another exponentially quickly with respect to some metric [27]. If a learned contracting system that admits a demonstration trajectory pushed off that trajectory, it will follow new trajectory that exponentially converges to the demonstration trajectory. Additionally, if a system contracting, is exponentially incrementally input-to-state stable [39]. If the system is autonomous, it also admits a unique equilibrium, is input-to-state stable, and has two Lyapunov functions that establish exponential stability of the equilibrium [9]. Since establishing contractivity globally requires satisfying matrix partial differential inequality everywhere, prior works have focused on establishing contractivity in the neighborhood of training data 36].
1.1 Related Works
Stable, but not necessarily contracting dynamics. Numerous works have aimed to learn stable dynamical systems from data including [7, 15, 22, 30, 32, 40, 45]. In [22], the authors introduce the Stable Estimator of Dynamical Systems (SEDS), which leverages Gaussian mixture model to approximate the dynamics and enforce asymptotic stability via constraints on learnable parameters. In [30]. the authors jointly learn the dynamics and a Lyapunov function for the system and project the dynamics onto the set of dynamics which enforce an exponentially decay of the Lyapunov function. This projection is done in closed-form and establishes global exponential convergence of the dynamics to the origin. In [40], the authors introduce Imitation Flow where trajectories are mapped to a latent space where states evolve in time according a stable SDE, In [32], Euclideanizing Flows is introduced where the latent dynamics are enforced to follow natural gradient dynamics [2]. A similar approach is taken in [45] where additionally collision avoidance is considered.
Existing works on learning contracting dynamics. Learning contracting vector fields from demon- strations has attracted attention due to robustness guarantees [6, 33, 35, 37, 39.. In [33], the dynamics are defined using a Gaussian mixture model and the contraction metric is parametrized to be a symmetric matrix of polynomial functions. Convergence an equilibrium is established using partial contraction theory [41]. In [35], the dynamics are defined via an optimization problem over a reproducing kernel Hilbert space; the dynamics data points, i.e., there may be points in state space where the dynamics are not contracting. In [37] and [39], the authors study controlled dynamical systems of the form J f(x u), where is con- trol input and train neural networks to find a feedback controller such that the closed-loop dynamics approximately contracting, i.e., contractivity is not enforced but instead lack of contractivity is penalized in the cost function.
constrained to be locally contracting around the
Recent work, [6], has proposed Neural Contractive Dynamical System (NCDS) which learns dynamical system which is explicitly contracting to a trajectory. NCDS parametrizes contracting vector fields by learning symmetric Jacobians and performing line integrals to evaluate the underlying vector field. NCDS is computationally costly because of this integration. Additionally, the Jacobian parametrization used is overly restrictive, because not all contracting vector fields have symmetric Jacobians. Constraining the vector field to have symmetric Jacobian is equivalent to enforcing that the vector field is a negative gradient flow and contracting with respect the identity metric [42]. For scalability to higher dimensions, NCDS leverages a latent space structure where an encoder maps the data space to diwen-dimeniional space and enforces contracting dynamics in this latent space. Then decoder "projects" the latent-space dynamics to the full data space. It is then argued that on the submanifold defined by the image of the decoder, the dynamics are contracting.
1.2 Contributions
In this paper, we present novel model for learning deep dynamics with global contraction guarantees. We refer to this model as Extended Linearized Contracting Dynamics (ELCD). To the best of our knowledge, ELCD is the first model to ensure global contraction guarantees. To facilitate the development of this model, we provide a review of contracting dynamics and extended linearization of nonlinear dynamics. Leveraging extended linearization, we factorize our vector field as f(z) A (z, x² (x= - x²), where x² the equilibrium of the dynamics. We enforce negative definiteness of the symmetric part of ((xx²) everywhere and prove (i) global exponential stability of x*, (ii) equilibrium contractivity of our dynamics to x², and (iii) using a converse contraction theorem, contractivity of the dynamics with respect some metric.

---

Since negative definiteness of the symmetric part of A is not sufficient to capture all contracting dynamics, we introduce a latent space of equal dimension as the data space and learn diffeomorphisms between the data space and this latent space. The diffeomorphisms provide additional flexibility in the contraction metric and allow learning of arbitrary contracting dynamies compared to those which are solely equilibrium contracting.
Our example in Section 3.4 provides theoretical justification for why the diffeomorphism and the learned contracting model must be trained jointly. In summary, if the diffeomorphism is trained first, and transforms the data to a latent space, the model class may not be expressive enough accurately represent the true latent dynamics. If the diffeomorphism and dynamics are trained simultaneously. this limitation is overcome. This is in contrast to [6] which trains the diffeomorphism independently as variational autoencoder and then trains the model after on the transformed data.
Our model, ELCD, directly improves on NCDS in several ways. We parameterize the vector field directly instead of parametrizing its Jacobian. Doing so prevents us from needing to integrate the Jacobian to calculate the vector field and thus speeds up training and inference. ELCD is also more expressive than NCDS because it can represent vector fields with asymmetric Jacobians. Additionally, ELCD guaranteed be contracting to an arbitrary equilibrium point, either selected or learned, at all training steps. NCDS, in contrast, must learn the equilibrium point over the course of training. Additionally, NCDS learns an encoder and decoder for a lower- diwver-dmensionall latent space and thus can only be contracting on the submanifold defined by the image of the decoder. From initial conditions that are not on this submanifold, NCDS may not exhibit contracting behavior. In contrast, since the latent space of EL.CD is of the same dimension as the data space, we train diffeomorphisms that ensure global contractivity the data space. ELCD exhibits better performance than NCDS at reproducing trajectories from the LASA [26]. n-link Pendulum, and Rosenbrock datasets. We additionally compare against the Euclideanizing Flow [32], and Stable Deep Dynamics [30] models.
2 Preliminaries
We consider the problem of learning a dynamical system à = fz) using a neural network that ensures af
that the dynamics are contracting in some metric. Going forward, we denote by Df öz (x) the Jacobian of f evaluated at x. To this end, we define what means for a dynamical system to be contracting
Definition 1. A contracting dynamical system is one for which any two trajectories converge
exponentially quickly. From [27], for a continuously disserentiable map I R + Bd,, the dynamical system i = f(x) is contracting with rate e >> O if there exists continuously continouuslyddfffrentttbb valued map M : IRd - RRdd and two constants ao, 01 > 0 such that for all x E RR,," M(r)= = M(x) and agla = M(x) = la and additionally satisfies for all x
matrix-
M(x)Df(x) + D)(x) M(x) + Mx)) 3 (x).
The map M is called the contraction metric and the notation (x) is shorthand for the n x matrix whose (i,j) entry is Mx), Ma)(( f(x). A central result in contraction theory is that any dynamical system - = f(x) satisfying (2) has any two trajectories converging to one another exponentially quickly [27, 29, 39]. Specifically, there exists L 2 1 such that any two trajectories
21, of the dynamical system satisfy
(t= (1)l2 S 11 (0) (0))-
In other words, contractivity establishes exponential incremental stability [4].
We note that any contracting dynamical system enjoys numerous useful properties including the existence of unique exponentially stable equilibrium and exponential input-to-state stability when perturbed by a disturbance. We refer to [27) for more useful properties of contracting dynamical systems and [9] for a recent monograph on the subject.
The problem under consideration is as follows:
given a set of demonstrations D = {(x,, i;) ) consisting of a set of n state, 2i E IRd, and velocity, à; E RR, pairs, we aim to learn a neural network f(x;)) = z that parametrizes globally contracting dynamical system with equilibrium point x", such that f(x:) = 0. essence, this task requires learning both the vector field and the contraction metric, M such that they jointly satisfy (2).
3

---

2.1 Contracting Linear Systems
Suppose we assumed that the dynamical system we aimed learn was linear, i.e., å Ar for some matrix A E Raxd and we wanted to find contraction metric M which we postulate to be constant ((x) := M for all The contraction condition (2) then reads
( + cI,) + (A) (A M
which implies that A has all eigenvalues with Re(X(A)) v cc, see Theorem 8.2 [18]. In other words, for linear systems, contractivity is equivalent to stability.
Although the condition (4) is convex in M fixed A, the task of learning both (A, M) simultaneously from data is not (jointly) convex. Instead, different methods must be employed such as alternating minimization. A similar argument is made in [30] in the context of stability and here we show that the same is true of contractivity.
2.2 Contractivity and Exponential Stability for Nonlinear Systems
In the case of nonlinear systems, contractivity is not equivalent to asymptotic stability. Indeed, asymptotic stability requires finding a continuously differentiable Lyapunov function V RR + R2o which is positive everywhere except at the equilibrium, x², and satisfies the decay condition
VV(x) f(x) O, Var Rd {x*)-
One advantage of learning asymptotically stable dynamics compared to contracting ones is that the function is scalar-valued, while the contraction metric M is matrix-valued. disadvantage of learning asymptotically stable dynamics compared to contracting ones is that we are required to know the location of the equilibrium point beforehand and robustness property of the learned dynamics is automatically enforced. To this end, there are works in the literature that enforce an equilibrium point the origin and learn the dynamics and/or the Lyapunov function under this assumption [30]. In the case of contractivity, existence and uniqueness of an equilibrium point is implied by the condition (2) and it can be directly parametrized to best suit the data.
3 Methods
3.1 Motivation
The motivation for our approach comes from the well-known mean-valus value theorem for vector-valued mappings, which we highlight here.
Lemma 2 (Mean-value theorem (Proposition 2.4.7 in (1]).. Let f Rdd > TRd be continuously differentiable. Then for every x, y E RRd
f(x) ((v) = (f, (1-r)))d y)-
If y x² satisfies f(x*) 0, then the continuously differentiable map f admits the factorization
f(x) = A(x, x (xx -x"), where
A(x,0*) [. Df(TT (1-T)x* "dd""
(8)
This factorization referred to as extended linearization in analogy to standard linearization, i.c., for x close to x², f(x) s Df(x*)( - x²). We remark that when d > 2, extended lincarization not unique, and for a given f. there may exist several A such that f(x) = A(r, x= x²) - - x²). In other words, (8) showcases
valid choice for A such that this factorization holds. Indeed, this nonuniqueness has been leveraged in some prior works, e.g. Section 3.1.3 [39], to yield less conservative contractivity conditions.
Since we know that a contracting vector field admits a unique equilibrium point, x², we restrict our attention to learning matrix-valuee mapping A : pd Rd - gddd and ensuring that this mapping

---

has enough structure so that the overall vector field satisfies the contraction condition (2) for some contraction metric M. This task nontrivial since f(x) A(x, x=) x²) implies that
04 (a, x xx =x),,
Df(x) = A(x,x")
dx
where 2A Rd x R Raxdxd is a third-order tensor- mapping. In what follows, we will 2z show that more simple condition will imply contractivity of the vector field. Namely, negative definiteness of the symmetric part of 2(x will be sufficient for contractivity of the dynamical system å A(x, xx)(x=""" x²).
3.2 Parametrization of A(z,
We show a simple example of our model, the Extended Linearized contracting Dynamics (ELCD). Let R be the state variable and f RR + I. be vector field with equilibrium point x², As indicated, we parametrize our vector field by its extended lincarization
a f(x) A(x, (x -x=xx"), x"),
(10)
where now
A(x,x") -P, P "'"(,,"""'P,,,,""""" 2= PP(lz,x") Pa
(11)
alı
Pa. Rd Rd + Rddd are neural networks, La is the d-dimensional identity matrix, and a > a constant scalar. Note that the symmetric part of A(z, x²) is negative definite since
A(z, x²) + A(I, TT) Px, T (x x²) ala
2
and ( Pa x²) is guaranteed to be positive semidefinite.
We prove that vector field parametrized this way is guaranteed to be (i) globally exponentially stable, (ii) equilibrium contracting as defined in [10], and (iii) contracting in some metric. The key tools are partial contraction theory [41] and a converse contraction theorem.
Theorem 3 (Equilibrium Contraction and Global Exponential Stability). Suppose the dynamical system & = f(x)) is parametrized as f(x) A(x,, x= (x x²) where A(x) is in (11). Then any trajectory zt) of the dynamical system satisfies
|(((1) - x 22 S at [|x(0) x=|12.
(12)
Proof. We use the method of partial contraction in [41]. Let æ(t) be a solution to &(t) A(z(t))=)))zt) x²) with initial condition (0) = zo. Then, define the time-varying virtual system
(t) A(x(t), - x²).
(13)
We will establish that this virtual system contracting in the identity metric, i.e., (2) is satisfied with ((x) Ia. We that the Jacobian for this virtual system is simply A(x(t), x²) and
A(z(t),=)) + A(z(t,,z²)) -=2P((x(()xx)) P..(t)))) 2ald
J --2al4-
In other words, in view of (3), and since (xx) = La, any two solution trajectories yı (t) and v2(() of the virtual system satisfy
(l)) (2)) < é - [|yn (0) y2 (0)l22
Note that we can pick one trajectory be y1 (t) x² for all and we can pick y2()) x(t). Since To was arbitrary, this argument establishes the claim.
Clearly, the bound (12) implies exponential convergence trajectories of the dynamical system (10)
to xx. Moreover, this bound exactly characterizes equilibrium contractivity. as was defined in [10].
Notably, using the language of logarithmic norms, Theorem in [10] establishes a similar result to Theorem 3 without invoking virtual system.
5

---

Note that although Theorem 3 establishes global exponential stability and equilibrium contraction, it does not establish global contractivity. Indeed, the contractivity condition (2) not guaranteed to hold with (z) la without further assumptions on the structure of A(x, x²) in (10). Remarkably, however, due to converse contraction theorem of Giesl, Hafstein, and Mehrabinezhad, turns out that one can construct state-dependent M such that the dynamics are contracting. Specifically, since trajectories of the dynamical system satisfy the bound (12), for any matrix- -valued mapping C': Rd + R2dd with C(x) = C(x) T > 0 for all x, the matrix PDE
(x))Df(x) + Df(x) M(x) + M(z) (xx)
(14)
admits unique solution for each [16]. In other words, the unique M solving this matrix PDE serves as the contraction metric and will satisfy (2) with suitable choice for c. The following proposition provides the explicit solution for M in terms of the solution to the matrix PDE.
Proposition (Theorem 2.8 [16]). Let C:: Idd + Rddd be smooth and have C(x) - C(x) T be
positive definite matrix at each z. Then M Ed - Idddd given by the formula
M(x) - L. (T,x)) ()((T))))((TT
(15)
is a contraction metric for the dynamicul system (10) on any compact subset containing x², where t + ((T, x) is the solution to the ODE with (00,x) x) = x and T + ((T,x) is the matrix-valued solution to
Y= D)) Y()) la.
(16)
While it is challenging. in general, to compute the metric (15), numerical considerations for approx- imating arbitrarily closely are presented in [17]. In practice, one would select C(a) Ia and evaluate all integrals numerically. For ELCD, unless one directly needs to know the contraction metric for application purposes, not required to compute the contraction metric at any point during either training or inference.
We remark that our parametrization for A in (11) is similar to the parametrization for the Jacobian of f that was presented in [6]. There are however a few key differences. Notably, A(x,x") may be asymmetric while the Jacobian in equation (3) in [6] is always symmetric. Notably, since the Jacobian in [6] is symmetric and negative definite, the vector field à = f(x) is a negative gradient flow, - V (z). for some strongly convex function V. On the contrary, our dynamics (10) can exhibit richer behaviors than negative gradient flows in view of the asymmetry in A(x, x²). Additionally, since the dynamics in [6] with their parametrization can be represented as
(V)(x) for some
strongly convex V, it is guaranteed to be contracting in the identity metric, M(x) Ia [42]. On the other hand, our dynamies (10) is not necessarily contracting in the identity metric and instead is contracting in a more complex metric given in (15).
3.3 Latent Space Representation
Realistic dynamical systems and their flows including the handwritten trajectories found in the LASA dataset, are often highly-nonlinear and may not be represented in the form (10) or obey the bound (12). One solution to these challenges space and learn an ELCD in the latent space.
transform the system to a latent, possible lower-dimensional.
Latent space learning is possible because contraction is invariant under differential coordinate changes. From Theorem 2 of [29]. given a dynamical system á = f(x), f : Rd - Rd, with f satisfying (I), the system will also be contracting under the coordinate change = = <xx) if d Rd - Rd is smooth diffeomorphism. Specifically, if à f(x) is contracting with metric M, then the system that evolves is contracting well with metric given by M D M (=) where 2 = ((x) and DØ is the Jacobian of the coordinate transform. In other words, We can learn vector fields that are contracting in arbitrary metric by training vector field f which is parametrized as (10) and using a coordinate transform ó.
NCDS [6], treats the coordinate transform as a Variational Autoencoder (VAE) [23]. Their training procedure consists of two steps: first training the coordinate transform with VAE training, then training the function in the new learned coordinates.

---

Figure 1: The learned vector field and corresponding trajectories of ELCD with transform (left) and model with a transform (right) when trained on data that is generated from a vector field that contracting in general metric.
3.4 On the Interdependence of Diffcomorphism and Learned Dynamics
To demonstrate the need for coordinate transform, we consider the task of learning a vector field that fits trajectories generated by the linear system ă Az with A
(o' -)- Clearly, this linear system cannot be represented in the form (10) with parametrization (11). To see this fact, observe that A + A not negative definite and thus no choice of P, or Pa can represent this linear system. To remedy this issue, one take the linear coordinate transform ((x) Px, where
1 0)
P
4)
Then the z-dynamics read
-
PAP-1
(17)
and now the symmetric part PAP-1 is negative definite and thus we can find suitable choices of Ps Pa. Specifically, we can take (0, 1/2), let Pz(z)
! and let Pz(z) Q2 where
1/2 0
Q is the matrix square root of (-1 - 1/2 a.22. Note that this toy example, asymmetry
1/2 1
essential to exactly represent these dynamics in the latent space, If we used the parametrization [6], we would not be able to represent these dynamics since they have an asymmetric Jacobian.
demonstrate this routine in Figure (I), We generate two trajectories starting at (0, 2) and (0. 22), and use that data to train two ELCDs, one without and one with a learned, linear transform. Figure (a) shows the vector field and trajectories corresponding to ELCD with no transform. The trajectories are forced to the center sooner than the actual data as a consequence of the bound (12). Learning coordinate transform allows the ELCD to learn system that contracting in arbitrary metric and exhibit overshoot. This is demonstrated by the learned system in Figure (b)), which perfectly matches the data.
3.5 Choice of Diffeomorphism
There are several popular neural network diffeomorphisms including coupling layers [12, 32], nor- malizing flows [40], M-flow [6, 8), spline flows [14], and radial basis functions [5].
coupling layer R Rd consists of neural network 0 R R with 1 < k: and
neural network g : T x + R. The transform & maps input x E Rd to E Rdd with the following procedure:
(i) Set y: = x, for 1 Sis for some 1 < k < d.
(ii) Set y: a.) for l: sisd
Coupling layers are invertible by doing the above process reverse, so long as is invertible. A coupling layer can exhibit wide variety of behaviors depending on the choice of g.
Polynomial spline curves are another diffeomorphism that are commonly used as g in coupling layers. A polynomial spline has restricted domain which is divided into bins. A different linear, quadratic

---

[31], or cubic [13] polynomial covers each bin. [14] introduce rational -uaaratiq splines, which constructs a spline from functions that are the quotient of two quadratic polynomials.
In practice, is the composition of several of transforms. Because the coupling transform only alters some coordinates, they are often used in conjunction with random permutations. If the latent space dimension is smaller than the data space dimension [6] makes the last composite function of ó an 'Unpad" function, which simply removes dimensions to match the latent space. The decoder is then prepended by 'Pad' function, which concatenates the latent variable with the necessary number of 'zeros' to match the data space.
Of course, the latent dimension is smaller than that of the data dimension, then Φ can not be bijective. However, [6] argue that long being contracting in the latent space implies that the learned dynamics are contracting on the submanifold defined by the image of .b_ In other words, NCDS cannot be globally contracting. The consequences of this are shown in Figure (2). While NCDS can learn to admit an equilibrium point when on the submanifold, trajectories that fall off the submanifold may diverge. ELCD, contrast, is contracting to the correct equilibrium point during all phases of training.
is injective and has range over the dataset, then
Figure Plots of the vector fields and induced trajectories learned by ELCD (Top) and NCDS (Bottom) after different training epochs. ELCD is contracting always while NCDS may admit multiple equilibrium or diverge.
Epoch
Epoch
Epoch
3.6 Training
Our task is simultaneously learn the contracting system f(x) and the metric in which f contracts. As previously discussed, the metric is implicitly determined by o(x). [6] uses two-step method. First, they treat @ as a variational autoencoder and maximize the evidence lower bound (ELBO). They then fix the encoder and train the model to evolve the state in latent space. Note that the VAE objective to make the encoded data match a standard-normal distribution. Notably, VAE training does not encourage the encoder to transform the data to space in which the data corresponds to trajectories of a contracting system. the data is not contracting in the transformed space, a contracting model will not be able to fully fit the data. This explains why, in our implementation of the two step-training, we are unable reasonably learn the data. For further comparison with NCDS, we will train the encoder and model jointly.
4 Experiments
The code for our model and data can be found here: https://github. wwt.cooo//c/aaee cow/seanjaafel// For all datasets we compare our method against NCDS [6], Stable Deep Dynamies (SDD) [30] and Euclideanizing Flow (EFlow) [32]. See A.2 for detailed discussion of the methods. We report the dynamic time warping distance (DTWD) [20] (see A. and standard deviation between predicted and data trajectories. See A.5 for model implementation details. See also the Appendix for more details on these models.

---

Figure 3: Plots of the 2D LASA data. Demonstrations in black. The learned ELCD trajectories in magenta are plotted along with the learned vector field. The vector field velocities have been normalized for visibility.
Table I: Mean DTWD + one standard deviation across 10 runs on LASA, multi-link pendulum, and
Rosenbrock datasets
SDD
EFlow
NCDS
ELCD
LASA-2D
0.37 + 0.32
1.05 0.25
0.59 + 0.61
0.12 + 0.11
LASA-4D
2.49 2.4
2.24 + 0.12
2.19 + 1.23
0.80 0.54
LASA-8D
5.26 0.50
2.66 0.63
5.04 + 0.77
1.52 + 0.61
Pendulum-4D
0.49 + 0.11
0.17 0.01
1.35 + 2.26
0.03 0.01
Pendulum-8D
0.75 + 0.08
0.33 0.01
2.88 + 0.69
0.14 + 0.03
Pendulum-161)
1.86 + 0.14
0.45 + 0.01
1.65 + 0.31
0.44 + 0.09
Rosenbrock-81
NaN
1.90 - 0.16
2.74 + 0.15
1.22 + 0.01
Rosenbrock-161
NaN
3.57 + 0.66
3.68 + 0.12
2.57 + 0.09
4.1 Datasets
We experiment with the LASA dataset [26]. which consists of 30, two-dimensional curves. We use three demonstration trajectories of each curve. As in [6], the first few initial points of each trajectory are omitted that only the target state has zero velocity. we stack two and four LASA trajectories together to make datasets and 8-dimensional trajectories, respectively. All data is standardized to have mean of zero and variance of one. We use in total 10 2D curves, 4D curves, and 6 8D curves. Some 2D-LASA trajectories and their respective trained ELCD trajectories and vector fields visualized in Figure (3).
We also experiment with simulated datasets. We simulate trajectories of a 2,4, and 8-link pendulum (4D, 8D, and 16D respectively) each and trajectories of 8D and 16D Riemannian gradient descent dynamics on a generalization of the Rosenbrock function (see Appendix A.4). Each model trained on all trajectories of the same dimension, and then predictions are made starting from every initial point.
4.2 Results
Table presents our results. ELCD performs the best in all tasks. These results shows the benefit of the increased expressiveness allowed by the skew symmetric component of our parametrization. These benefits are more apparent the pendulum dataset. The oscillatory behavior of the pendulum dynamics is product of complex eigenvalues in its Jacobian. Our model's skew symmetric com- ponent is what enables it to learn the pendulum dynamics so well. sample of demonstration and leameed pendulum trajectories is shown in figure (4).
The Rosenbrock dynamics are stiff and difficult to learn. In our training of SDD, we observed lack of stability and convergence, resulting in very large DTWDs. Hence, we report NaN for the results of SDD models on the Rosenbrock dataset. While our model performs the best, there is still room for improvement. Handling such dynamics with multiple time scales is still open challenge.

---

Figure 4: Phase plots of the two link- link-pendulum trajectories (blue) and the trajectories produced by ELCD (red). Each column is different trajectory. The top row is the first link and the bottom row the second link. The x-axis is angle and the y-axis is angular velocity.
Limitations
Our method assumes that the underlying dynamics of the data trajectories are contracting. However, the diffeomorphism allows for wide class of stable systems to be represented. Our method is currently implemented for trajectories that converge to the same fixed point. But, this can be overcome by manually changing the fixed point, depending on which trajectory the input data came from. Also, right now our method is limited to dynamics in I2, but we imagine that an extension to more general manifolds should be possible.
As are motivated by applications in imitation learning and robotics, in this work, are primarily interested in problems where the underlying systems should be robustly stable, especially away from training data. For this reason, we focus on learning dynamics which are guaranteed be globally contracting and do not address learning other types of systems, such as those with multiple fixed points, limit cycles, or chaotic attractors. We imagine that extensions of ELCD could capture these richer dynamical behaviors by leveraging weaker notions of contraction including local contraction (i.e., contractivity the region of attraction of stable equilibrium), k-contraction (contraction of k-dimensional bodies) [43]. transverse contraction [28] and contraction in the Hausdorff dimension [44]. Moreover, the notion of translation invariance mentioned could be studied using semicontraction theory [11], i.., contraction subspace.
6 Conclusions
In this paper, we introduce ELCD, the first neural network-based dynamical system with global contractivity guarantees in arbitrary metries. The main theoretical tools extended linearization, equilibrium contraction, and a converse contraction theorem. To allow for contraction with respect to more general metries, use latent space representation with dimension of the latent space equal to the dimension of the data space and train diffeomorphisms between these spaces. We highlight key advantages of ELCD compared NCDS as introduced in [6]. including global contraction guarantees, expressible parametrization, and efficient inference. We demonstrate the performance of ELCD on nigh-dimensional LASA datasets and simulated multi-link pendulum and Rosenbrock dynamics. Our method shows consistent performance across all datasets.
Acknowledgments and Disclosure of Funding
This work was supported part by the NSF Graduate Research Fellowship under grant 2139319, the AFOSR under award FA9550-22-1-0055 the NSF under grant 2229876, the Department
10

---

of Homeland Security, and by IBM. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF or its federal agency and industry partners.
References
[1] R. Abraham, J. F. Marsden, and T. S. Ratiu. Manifolds, Tensor Analysis, and Applications, volume 75 of Applied Mathematical Sciences. Springer, 2 edition, 1988. ISBN 0387967907.
[2] Amari. Natural gradient works efficiently in learning. Neural Computation, 10(2):251-276
1998. 166 12016697669990000
[3] B. Amos, L. Xu, and J. Z. Kolter. Input convex neural networks. In International Conference on Machine Learning. 2017. URL wttpttt:ppppppeeeeeeeeeeee www.:innsmm..r....mmmmm mlr S.l..ppees/vv00 amosl7b. html.
[4] D. Angeli. A Lyapunov approach to incremental stability properties. IEEE Transactions on Automatic Control, 47(3): 10-4211, 2002. doi:10.1
[5] H. Beik-Mohammadi, S. Hauberg, G. Arvanitidis, G. Neumann, and L. Rozo. Reactive motion generation on learned Riemannian manifolds. International Journal of Robotics Research, 42 (10):729-754, 2023.
193046.
[6] H. Beik-Mohammadi, S. Hauberg, G. Arvantidiis, N. Figueroa, G. Neumann, and L. Rozo. Neu- ral contractive dynamical systems. In International Conference on Learning Representations, 2024. URL https: //wpr//eeeeeeeeeeeee//me net feet/forum? wtt.rummmmmiiAIIIROYYP0PP
[7] P. Bevanda, M. Beier, S. Kerz, A. Lederer, S. Sosnowski, and S. Hirche. Diffeomorphi- cally learning stable Koopman operators. IEEE Control Systems Letters, 6:3427-3432, 2022. 10i10..109/LCCSSS 202..S..2...1.19227
[8] J. Brehmer and K. Cranmer. Flows for simultaneous manifold learning and density estimation. Advances Neural Information Processing Systems, pages 442-453, 2020. URL https:
/ /aaxiii..oo .wwp//org/.... abs 2o03/2003..m 139133.
[9] F. Bullo. Contraction Theory for Dynamical Systems. Kindle Direct Publishing, 1.1 edition, 2023. ISBN 779-8836646806. URL. https: / fbullolloooooooo wwt.o.iittubee lo/ctds.
[10] A. Davydov, S. Jafarpour, and F. Bullo. Non-Euclidean contraction theory for robust nonlinear stability. IEEE Transactions on Automatic Control, 67(12):6677-668, 2022. N00:00.11099TAA222222333339
[11] G. De Pasquale, K. Smith, F. Bullo, and M. E. Valcher. Dual seminorms, ergodic coefficients. and semicontraction theory. IEEE Transactions on Automatic Control, 6965):3040-3053, 2024.
10f:100.1109/TAA.22233330
[12] I.. Dinh, Sohl-Dickstein, and S. Bengio. Density estimation using real NVP. In Interna- tional Conference on Learning Representations, 2017. URL http:::/ net/
Forum?id-Hkpporae
[13] C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios. Cubic-spline flows. /CML Workshop on Invertible Neural Networks and Normalizing Flows, 2019. httpttt:: http:::s////xx...orro/o org/
abs /1906...... 02145.
[14] C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios. Neural spline flows. In Advances in Neural Information Processing Systems, 2019. URL 04032.
[15] F. Fan, B. Yi, D. Rye, G. Shi, and I. R. Manchester. Learning stable Koop- man embeddings. In American Control Conference, pages 2742-2747, 11110002919/AAC333388222222222.
2022.
[16] P. Giesl, S. Hafstein, and I. Mehrabinezhad. Computation and verification of contraction metrics for exponentially stable equilibria. Journal of Computational and Applied Mathematics, 390:
113332, 2021. 10. 11t.:.00116...mm22....... 11332........
[17] P. Giesl, S. Hafstein, and I. Mehrabinezhad. Contraction metrics by numerical integration and quadrature: Uniform error estimate. In 20th International Conference on Informatics Control, Automation and Robotics, 2023. 100.://2200000021110000000000000

---

[18] J. P. Hespanha. Linear Systems Theory. Princeton University Press, 2009. ISBN 0691 140219. [19] N. Jaquier and L. Rozo. High-dimensional Bayesian optimization via nested Riemannian manifolds. In Advances in Neural Information Processing Systems, volume 33, 2020. URL https: / arxii. org/ abs 2012001000 10904.
[20] E. Keogh and C. A. Ratanamahatana. Exact indexing of dynamic time warping. Knowledge and Information Systems, 7(3):358-385 2005.
1007/s11111-0004
[21] H. K. Khalil. Nonlinear Systems. Prentice Hall, 3 edition, 2002. ISBN 0130673897.
[22] S. Mohammad Khansari -hahhh and Aude Billard. Learning stable nonlinear dynamical sys- tems with Gaussian mixture models. IEEE Transactions on Robotics, 27(5):943-955 957, 2011. Noi:10.119//RR 201 1.2159412.
[23] D. P. Kingma and M. Welling, Auto-encoding variational Bayes. In International Conference on Learning Reprexentations, 2014. URL h
6114.
[24] L. Kozachkov, P. M. Wensing, and J.-J. Slotine. Generalization dynamical robustness- The role of Riemannian contraction in supervised learning. Transactions on Machine Learning Research, 2023, URL https: /app//epeeeeeeeeeeeee net forum? Sitiummii-SSia 6p5cc
[25] Lan and I. Mezić. Linearization in the large of nonlinear systems and Koopman operator spec- trum. Physica Nonlinear Phenomena, 242(1):42-53, 2013. doi: 101 pht.:/.pp...200000 2012.08.01
[26] A. Lemme, Y. Meirovitch, M. Chansari-Zadehl T. Flash, A. Billard, and Steil. Open- source benchmarking for learned reaching motion generation in robotics. Paladyn, Journal of 101.1115///--001551.
Behavioral Robotics, 6(1 (()):33411, 41, 2015.
[27] W. Lohmiller and J.-J. E. Slotine. On contraction analysis for non-linear systems. Automatica, 34(6):683-696, 1998. doi: 10. (001///0000-11989988800000
[28] I. R. Manchester and J.-J. E. Slotine. Transverse contraction criteria for existence, sta- bility, and robustness of limit cycle. Systems & Control Letters, 63:32-38, 2014. (011000016//.ssscooee220111 10.005.
[29] I. R. Manchester and J.-J. E. Slotine. Control contraction metrics: Convex and intrinsic criteria for nonlinear feedback design. JEEE Transactions Automatic Control, 6246)3304633053,
2017. doi: 11/////C.20...2666333300
[30] G. Manek and J. Z. Kolter. Learning stable deep dynamics models. In Advances in Neural Information Processing Systems, 2019. URL https: / /arxivivvv/v/////////ee//aaaaava org/
wttp/////////////////e////////// pdtppdff/20//// 06116. 6..6.
[31] T. Müller, B. McWilliams, F. Rousselle, M. Gross, and J. Novák. Neural importance sampling. ACM Transactions on Graphics, 38(5):1-19, 2019. doi: 145/3341156.
[32] M. A. Rana, A. Li, D. Fox, B. Boots, F. Ramos, and N. Ratliff. Euclideanizing flows: Diffeomor- phic reduction for learning stable dynamical systems. In Conference on Learning for Dynamics and Control, pages 630-633-- 639, 2020. URL https: http:::s//ppoceeeiiiie rana20a. html.
ml press/ /v120/
[33] H. Ravichandar, I. Salehi, and A. Dani. Learning partially contracting dynamical systems from demonstrations. In Conference Robot Learning, pages 369-378, 2017. URL https:
/ ppppeediiggs
.ww.sng.ss......m............... mlrr www.pppess///i het77//raviccanaa..am
html.
[34] H. H. Rosenbrock. An automatic method for finding the greatest or least value of a function. The Computer Journal, 3(3): 1960. doi: 10. 1093/comjnl/3.3.1777
[35] V. Sindhwani, S. Tu, and M. Khansari. Learning contracting vector fields for stable imitation learning, 2018. URL https::p///o /ttpvv.xivvvvv////////e////..av. wttp /8044 1804. 04878. Arxiv e-print.
[36] S. Singh, B. Landry, A. Majumdar, J-J. E. Slotine, and M. Pavone. Robust feedback motion planning via contraction theory. International Journal of Robotics Research, 42(9):655-688, 186165
2023. doi: 10. 0
[37] D. Sun, S. Jha. and C. Fan. Learning certified control using contraction metrie. In Conference on Robot Learning, volume 155, pages 1519-1539, 2021. URL https:://prcceeedng mlr. aalr.press/ve svieess/vl55/ h155/sun211133 sun2lb html.
[38] T. Teshima, I. Ishikawa, K. Tojo, K. Oono, M. Ikeda, and M. Sugiyama. Coupling- based invertible neural networks are universal diffeomorphism approximators. In Advances Neural Information Processing Systems, 2020. URL https://arxiv org/ abs/2006. ....1.1... 11469.
12

---

[39] H. Tsukamoto, S.-J. Chung, and J.-J. E Slotine. Contraction theory for nonlinear stability analysis and bearning-based control: tutorial overview. Annual Reviews in Control, 52: 135-169, 2021. doi:10.10 00::10.101/// wff..arcontroo.2221....00
[40] J. Urain, M. Ginesi, D. Tateo, and J. Peters. ImitationFlow:
Learning deep stable stochastic dynamic systems by normalizing flows. In IEFF/RSJ International Conference Intelligent Robots and Systems (IROS), page 5231 2020, doi: 10.
[41] W. Wang and J. J. Slotine. On partial contraction analysis for coupled nonlinear oscillators. Biological Cybernetics, 92(1):38(8--- 53, 2005. doi: N00::00.107//000 (0.10077//00222-000--0222
[42] P. M. Wensing and J.-J. E. Slotine. Beyond convexity - Contraction and global convergence of gradient descent. PLoS One, 15(8):1-29, 2020. doi: 1371/omall
/journal. www.ur.al..poe.......m.. pone.0236661.
[43] C. Wu, Kanevskiy, and M. Margaliot. k-contraction: Theory and applications. Automatica, 136: 10041, 2022. doi: 10.10066..utommtii..2.....1 10044
[44] C. Wu, R. Pines, M. Margaliot, and J.-J. E. Slotine. Generalization of the multiplicative and additive compounds of square matrices and contraction in the Hausdorff dimension. IEEE Transactions on Automatic Control, 2022. 1oi: 109/TAC..022.331633
[45] W. Zhi, T. Lai, L. Ott, and Ramos. Diffeomorphic transforms for generalised imitation learning. In Learning for Dynamics and Control Conference, pages 508-519, 2022. URL https: httes://proceee /pwp/eediiggg....../e///.o....ae mlr, www.spessvvvv.... zhi22a, html.
13

---

A Appendix
A.1 Metric
We evaluate trajectories by comparing them against their reference trajectory using the normalized dynamic time warping distance (DTWD) [20]. For two trajectories (I...e
{ (i....))))))))) and some distance function d(.,)). let DTWD be
1 t
f1 min d(z,, 2,)) Ryea
1 12
DTWD 2z, (2,
Σ 12
min d(T., 2))) ayez
A.2
Comparisons to Other Models
In this section, we describe the models that we compare to, namely EFlow [32], SDD [30], and NCDS [6].
Euclideanizing Flow: Starting with EFlow, they parametrize their dynamical system by
c = G (x: (+)((x)))
(18)
where v0 12d + Rdd is diffeomorphism, + Rd + R is convex potential function, and Gz(xx) = ( Dab(ar) the induced Riemmanian metric. The dynamics (18) then has the interpretation of being the steepest descent for & 16 on the Riemmanian manifold defined by metric Gg.- Specifically, in [32], the convex potential function defined to be ((y) = lly y" ll2, where y" w(*") and the diffeomorphism is parametrized via a coupling layer [12]. Note that contraction properties of dynamics of these form were studied in [42]. Since V is chosen to be convex but not strongly convex, these dynamics are only weakly contracting, i.c., satisfy (2) with 0.
Stable Deep Dynamics: In [30], the authors parametrize both an unconstrained vector field Rd + I220 via neural networks. Then to enforce global exponential stability, at every point x E Rdd they project f(x) onto the convex set of points {u E Rd Vz(x) u S -aV(x)) They do this projection in closed-form so that the dynamics have the representation
f I2d + Rdd and a Lyapunov function
i fz) VV (cLU(VV(=)TTFCC +z))+=+++())))) +vV())
(19)
[VV(2)
Under smart parametrization of V using input-convex neural networks [3], the authors guarantee global exponential stability of (19) to the origin with rate a. Note that the dynamics (19) are continuous, but not necessarily differentiable. Due to this potential nonsmoothness, it is theoretically unknown whether these dynamics are contracting (since Theorem 2.8 in [16] requires at least some differentiability)..
Neural Contractive Dynamical Systems: In [6], the authors enforce contractivity by directly enforcing negative definiteness of the Jacobian matrix of the vector field by parametrizing
Df(x) = - (J) T Jo(x) + cIa),
(20)
where Jo Rdd Rdxd is a neural network. Under this parametrization of the Jacobian, it is straightforward to see that (2) holds with c = and Mxx)= Ta for all æ. To recover the vector field from the Jacobian, the fundamental theorem of calculus for line integrals (which can be seen as version of the nean-value theorem) is utilized to express
+ S. DS((1 - 1)=o + tx)( zo)dl,
;= fx) =
(21)
where xp and f(x) denote the initial condition and its velocity, respectively.
To express richer dynamics, NCDS also consists of an encoder and decoder for a lower- diveer-dmmesiinall latent space. The dynamics in the latent space are constrained to be (21) and the encoder and decoder are parametrized using VAE. Since, generally, the latent space is of lower dimension than the data space, the resulting dynamics are only guaranteed be contracting on the submanifold defined by the image of the decoder.
14

---

A.3 Universal Representation
Our ELCD model equipped with coupling layer-based transformation can represent any contracting dynamical system. It is known that any smooth nonlinear system with hyperbolically stable fixed point can exactly linearized inside basin of attraction via suitable diffeomorphism [25]. Effectively, our extended linearization parameterization is tasked with learning the stable linear part, while the coupling layers aim to learn and approximate this suitable diffcomorphism. As our ELCD parametrization can universally approximate any linear model, we simply need our coupling layer to universally approximate all diffeomorphisms. Indeed, [38] proved that the coupling layers are universal approximators for diffeomorphisms.
A.4
Additional Details on Datasets
In this section, we provide additional elaboration on the multi-link pendulum dataset and the Rosen- brock dataset.
Multi-link pendulum: The multi-link pendulum is a simple mechanical system which is the interconnection of rigid links under the force of gravity. We specifically assume that there is damping on each of these links, which makes the resulting dynamical system stable and almost all trajectories converge to the downright equilibrium point. In this case, the state of the pendulum is described by the n pairs (0,, 0.). where is the angle of the i-th link and 0, is its angular velocity. Thus, this dynamical system is 2n dimensional.
As was done in [30], we adapt the code from http: / jakevdp github. ho/ollog//////// 03/08 htttttrrpppppppneeeee cmmoo / to generate data for n links.
Rosenbrock datset: The classical Rosenbrock function, see [34]
f(z, v) = (1 - x²² + 100(y x²)²
(22)
is nonconvex function with global minimum at (1, Despite its apparent nonconvexity, it known that a Riemannian gradient descent dynamics, is contracting with respect to suitable Riemannian metric, see Example 3 in [42] for details. In optimization and in machine learning, the Rosenbrock function is a benchmark for the design of various optimizers and the study of Riemannian optimization [19, 24].
For higher-dimennsional generalization of the Rosenbrock function, we consider
f(x) A1 (1)=)) 21)²+ + Σ MaZ - -
(23)
where A; are parameters that affect the conditioning of the function. This generalization is still nonconvex and admits several saddle points. Note that the classic Rosenbrock corresponds to 2, A 1, A2 100. The unique global minimum of this generalization is at (1, 1,.,, 1). The corresponding contracting Riemmanian gradient descent dynamics that find this global minimum are
i = -==((x))V((t))
(24)
where G(x) (i) (()))))D(((())) Dib( where R Th is the mapping (<x) (VAi(1
11). V .?)...., ..... Vnn =²-1)) )). Note that these dynamics were designed via gen- eralization of the procedure proposed in [42]. To generate data, we choose four initial points and evolved them according to (24).
A.5
Model Details
For the encoder o, we use the composition of two coupling layers composed of rational-quadrate spline. The splines cover the range (-10, 10) with 10 bins, and linearly extrapolated outside that range. The parameters of each spline are determined by residual networks, each containing two transform blocks with hidden dimension of 30. permutation and linear flow layer is placed before, in the middle, and after the two quadratic spline flow layers. For ELCD, we let the latent dimension size equal the data dimension size. P. and P. are implemented as two-layer neural networks with hidden dimension of 16.
15

---

As the authors of NCDS have not yet made their code publicly available, we implemented NCDS the best of our abilities. We used latent dimension of size 2 for all datasets. This in-line with the description in [6], and necessary given the extra cost from integrating the Jacobian in higher-dimensional latent spaces. We use the same encoder Φ as for ELCD with an added un-padding layer as the last function which removes the last d dimensions.
A.6 Training details
All experiments are trained with batch size of 100, for 100 epochs, with an Adam optimizer and learning rate of 10-3. All computation is done on CPUs.
NCDS [6] trained the model and the encoder output x++11 1 from F2 using a single Fuler integration step: It+1 = o ()(x))) + dt f()(x=)). Their loss was the MSE between the predicted and true It-1- We empirically find better performance by training
model to directly predict the velocity vector ø1+1 oj (f)))()(z))) where oj (ax) is the Jacobian inverse of o(zz). Calculating the
gradient of 4 (ate) efficiently required slight alteration the original M-flow [8] code, which have included in our repository.
16

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes] Replace by [Yes], [No], or [NA].
Justification: We introduce new model has global contraction gaurantess and more expressible than prior work. We discuss the contraction properties and demonstrate our the expressibility experimentally.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
(ii) Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Replace by [Yes], [No], or [NA].
Justification:
See section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model cell-specffiaatinn. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
(iii) Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer: [Yes] Replace by [Yes], [No], [NA]
Justification: Our main theorem in section (3.2) is supported by complete proof. Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
(iv) Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes] Replace by [Yes], [No] [NA].
Justification: Data details and model details provided in the appendix. Code is also provided.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path reproducing verifying the results.
Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Replace by [Yes], , [No], or [NA].
Justification:
Code reproduce experiments provided.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https: /tapps://niee //tp//nipps.//////////// cc/
publ / guides/ CodeSulomi ss onPol mlyyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips. cc/
CooeeeoeeSaaaeeee ProdeSubmis Submi iesionPoliceee for more details. The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
(vi) Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes] Replace by [Yes], [No], or [NA].
Justification:
See Appendix A.6
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
(vii) Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes] Replace by [Yes], [No], or [NA].
Justification: Standard Deviations are reported.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
19

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
(viii)
Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes] Replace by [Yes], [No] [NA]
Justification:
We explain in the appendix that we use CPUs.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
(ix)
Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htt http///eerippppeee cc
Answer: [Yes] Replace by [Yes], [No] [NA].
Justification: We have reviewed and comply with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
(x) Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA] Replace by [Yes), [No], or [NA].
There is no direct link between our work and a possible negative societal impact.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
(xi) Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA] Replace by [Yes], [No], [NA].
Justification:
This paper poses such risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
(xii) Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes] Replace by [Yes], [No], [NA].
Justification: Data and code from prior work are all cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papers aw! thcode com da set. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
21

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
(xiii) New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes] Replace by [Yes], [No] [NA]
Justification: Alongside the new ELCD model, we also provide the n-link pendulum dataset and Rosenbrock dataset in the supplementary material of the submission. Upon acceptance, we will release the datasets with a permissive license.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
(xiv)
Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA] Replace by [Yes], [No]. or [NA].
Justification:
The paper does not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
(xv)
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA] Replace by [Yes], [No], or [NA].
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22


Reviews:
[{"review_id": "x7tYmN2IX9", "reviewer": "Reviewer_2hrV", "summary": "This paper proposes a novel parameterization of the extended linearisation form of a dynamical system that guarantees global contractivity. Whilst the most basic form of this parameterization only ensures contractivity in some (implicitly defined) metric, a latent space version is also proposed that enables more flexible learning of dynamics that are contractive in more general metrics. Experiments are provided that show superiority over related methods.", "questions": "In the partial contraction analysis of Eq 13, it might be worth stating that x can be any function of time (as opposed to a constant). This seems critical to the eventual conclusion, where you say \"we can pick y2(t) = x(t)\", as this statement implicitly relies on a trajectory of the original system also being a valid trajectory of the virtual system. I'm guessing this is very standard but as someone not familiar with this type of partial contraction analysis I thought I'd mention it.\n\nA few typos\n- l.41 “enforce an exponentially decay the Lyapunov function”\n- l.52 “are constrainted”\n- l.53 “there are points [...]” → there may be points [...]\n- l.161 “prior words” → prior works\n- l.258: diffeopmorphisms\n- l.287 minimize the ELBO → maximize the ELBO\n- l.290 the data is corresponds\n- l.456: metrc\n- just a suggestion for your acronym: \"ELCoD\" would be easier to pronounce (\"ell-cod\") than \"ELCD\"", "limitations": "Could the authors please comment on whether requiring contraction behaviour (and in particular, the existence of a unique stable equilibrium) might be too constraining in some applications?\n\nI wonder if the authors could also say a few words about the controlled case -- i.e. driven systems as opposed to autonomous systems. E.g. if the autonomous system $\\dot{x} = f(x)$ obeys the guarantees provided by this work, what guarantees might be automatically inherited by the corresponding control affine system $\\dot{x} = f(x) + B u$?", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper is very well written; it provides a very good intro / recap of contractivity which was very useful to me at least, and a clear articulation of what was missing from previous works on the subject (e.g. NCDS is computationally expensive and has overly constrained (symmetric) Jacobians, no global contraction guarantees, etc). The idea of directly parameterising the extended linearization of the system is very neat, and so is the use of the converse contraction theorem of Giesl et al to prove the main contractivity result. Although the experiments are a bit thin, I find them convincing enough.", "weaknesses": "The authors very quickly gloss over the motivations for learning contractive dynamics; I had to go read the NCDS paper to understand the real-world relevance of such dynamics. Let me write down below what occurred to me when reading the intro (which mostly betray my lack of experience in potential practical applications of this paper), and the authors may or may not want to modify their introduction to preempt this potential confusion.\n\nMy initial thoughts when reading the first paragraph of the intro, in particular “it is desirable to ensure [...]”:\n- For one, one might actually be learning from data that comes from a dynamical system with multiple stable fixed points, so why is it desirable to enforce a single one in the learned model?\n- Relatedly, why does the stability of the _learned_ dynamical system matter? I would assume that what typically matters in robotics/control is that the actual physical system implemented (and in particular, the closed control loop) be stable and robust? What does that have to do with the learned model? Is model stability important because an unstable model (where the ground truth is stable) would extrapolate very badly outside the region where there is data?\n\n(I was thinking of a situation where deep learning is used to learn an internal model of a plant from observational data, e.g. for subsequent control purposes; in which case (i) autonomous dynamics are not necessarily very relevant and (ii) it's unclear why those dynamics would be contractive in general; now I understand that there are applications in e.g. imitation learning where it might be useful to ensure that the learned dynamics have such guarantees, but it was hard to guess from the current intro; the intro of the NCDS paper helped).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Introduction**\n\nThank you for clarifying your thought process. We will make the following changes to the introduction to make things clearer:\n\nWe will replace line 17 onward in the first paragraph with:\n\n\"Beyond approximating the vector field f, it is desirable to ensure that the learned vector field is well-behaved. In many robotic tasks like grasping and navigation, a well-behaved system should always reach a fixed endpoint. Ideally, a learned system will still stably reach the desired endpoint even when pushed away from demonstration trajectories. Additionally, the learned system should robustly reach the desired endpoint in the face of uncertainty. In other tasks, such as manufacturing, animation, and human-robot interaction, the learned system must smoothly follow a specific trajectory to its target. Following a trajectory is necessary to ensure function, as is the case with a manufacturing robot dynamically interacting with its environment, and safety, as is necessary with a robot occupying the same space as humans.\"\n\nWe will additionally replace lines 27-28 in paragraph three with:\n\n\"To ensure robustness and to allow for smooth trajectory following, there has been increased interest in learning contracting dynamics. A dynamical system is said to be contracting if any two trajectories converge to one another exponentially quickly with respect to some metric [25]. If a learned contracting system that admits a demonstration trajectory is pushed off that trajectory, it will follow a new trajectory that exponentially converges to the demonstration trajectory. \n\"\n\n\n\n\n\nTo address those specific concerns:\n\n**One might actually be learning from data that comes from a dynamical system with multiple stable fixed points**\n\nThank you for bringing this point up. Indeed, contracting dynamics can only\nhave one stable fixed point. In this work, we are assuming that the\nunderlying dynamics are globally contracting (hence with a unique fixed\npoint) as this assumption ensures strong convergence and robustness\nguarantees for the learned dynamics, even away from training data.\n\nAs a possible extension to systems with multiple fixed points, one could\nimaging learning multiple ELCD models, one for the neighborhood of each\nfixed point and then (i) either devising a strategy to blend them for\nglobal problems, or (ii) depending on the initial condition, leveraging the\nspecific relevant model. \n\n\n\n**Why does the stability of the learned dynamical system matter?**\n\nLike you mention, we envision applications in imitation learning where the\nresulting learned model has global guarantees of contraction, even away\nfrom the training data. We will update the introduction to be more specific\nas to why we want to learn globally contracting dynamics (and mention\nextensions to multiple equilibria and multiple local ELCD models).\n\n\n\n**In the partial contraction analysis...**\n\nOf course you are completely correct. We will be more precise and state\nthat $x(t)$ is meant to be a specific solution trajectory of the original\nsystem $\\dot{x}(t) = A(x(t),x^*)(x(t) - x^*)$.\n\n\n\n**Words about the controlled case...**\n\nThank you for your curiosity on this topic. Our method can be naturally\nextended to learning control systems $\\dot{x} = f(x) + Bu$ which are\ncontracting in $x$ at fixed $u$. These systems automatically inherit an\ninput-to-state stability property and an entrainment property, i.e., a\nperiodic input $u$ results in a stable periodic solution $x$,\nsee [6]. Moreover, control systems with a contracting\ndrift term can be amenable to computationally efficient tools for optimal\ncontrol, see [7]. We are actively studying the extension of\nELCD to closed-loop contractivity guarantees, i.e., learning both $f$ and a\nclosed-loop controller $u$ that offers global contraction guarantees (not\njust at the training data as is frequently studied in existing works).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SUprDmbes0", "reviewer": "Reviewer_HPgs", "summary": "The paper presents a method for learning contracting representations from a dynamical system. The novelty of the method lies in learning a linear map and a coordinate transform map (Diffeomorphism), which extends the types of contracting systems that can be learned. The experiments show that the proposed methods performs better empirically than Neural Contractive Dynamical System and other approaches for learning contracting dynamics.", "questions": "The caption in Table 1 misses which metric is represented here and also what the +- means (this should not be implicitly assumed)", "limitations": "The paper discusses some extensions to having multiple fixed points.\nHowever, the paper lacks a true discussion on limitations and scenarios where the approach may fail (not necessarily due to the approach but also including due to learning it)", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well structured and easy to follow\n- The problem motivation beyond the vast amount of methods on learning stable dynamics is well formulated\n- The relation to recent literature on this topic is also well done", "weaknesses": "The main weakness of the paper is the experimental part. \nThere are details missing and it feels \"rushed\", e.g., table caption unclear (see below).\nAdditionally, he set of tasks are very limited and don't go beyond the \"usual suspects\" in stable dynamical system learning (e.g. pendulum chains).\n\nBesides the experiments, a point of limitation would be that the approach itself is by no means \"groundbreaking\" but rather incremental in the context of learning contracting dynamics (whether empirically it is truly groundbreaking cannot be assessed from the provided experiments)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**The novelty of the method lies in learning a linear map...**\n\nThank you for your time in reviewing our paper. Respectfully, we would like to clarify that we do not learn a linear map, but rather a nonlinear matrix-valued map\n$x \\mapsto A(x,x^*)$ via the parametrization (11) to allow for nonlinear\ncontracting dynamics in the latent space.\n\n\n\n**The main weakness of the paper is the experimental part...**\n\nWe do focus on the ``usual suspects\" in stable dynamics learning since they\nare common benchmarks studied in other works. These examples allow us to\nshowcase that ELCD outperforms other state-of-the-art methods for learning\nstable dynamics in established benchmarks or well-known systems. \nBeyond LASA and the multi-link pendulum, we argue\nthat the Rosenbrock dataset is novel and valuable (while the Rosenbrock\ndataset has not been featured in other works on learning dynamics, the\nRosenbrock function is a classic function in optimization and ML). The Rosenbrock dynamics are novel in that they are stiff and evolve at different timescales. In each case we are able to showcase the superiority of ELCD compared to prior alternative methods.  \n\n\n\n\n**The approach itself is by no means ``groundbreaking\" but rather incremental...**\n\nWe would argue that the approach is more than incremental in nature as we\nallow for learning a larger class of contracting dynamics compared to NCDS\nby our novel parametrization which captures asymmetric Jacobian\nmatrices. The benefits of our parameterization are clearest in our updated results on the multi-link pendulum using continuous-time training. The pendulum has inherently oscillatory dynamics, whose Jacobian has complex eigenvalues. No learning model with symmetric Jacobian can represent this system, regardless of what diffeomorphism is used. Our parameterization is key towards learning this system. The commonality of the multi-link pendulum (and more geneally oscillatory dynamics) in the dynamical systems literature demonstrates the value of our contribution. Moreover, by directly parametrizing the extended linearization,\nwe avoid the expensive line integral necessary in NCDS. \nFinally, we feel that the recent improvement from discretized-time to continous-time training is also valuable. \nAs a testament to\nbenefits of these changes, we can see the notably improved performance of ELCD\ncompared to existing benchmarks in our numerical experiments.\n\n\n\n**The caption in Table 1 misses which metric is represented here...**\n\nYou are correct that we were not precise enough in our caption of Table 1. To\nclarify, we are reporting the dynamic time warping distance (DTWD) between\nthe trajectories of the learned model and the trajectories of the ground\ntruth dataset. The $\\pm$ corresponds to one standard deviation from the\nmean. We will clarify these details before the final submission.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Bnmsjxjec7", "reviewer": "Reviewer_UYYA", "summary": "The paper presents Extended Linearized Contracting Dynamics (ELCD), a dynamical system with neural network components that has global contraction guarantees. They demonstrate improved efficiency and performance on the trajectory fitting LASA dataset (upto 8 dim), pendulum dataset (upto 16 dim), Rosenbrock dataset (upto 16 dim).", "questions": "Please see above.", "limitations": "NA", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Strong theoretical basis for the ELCD model.\n\n2. Well-written and easy-to-read paper.\n\n3. Reasonably sized experiments for a paper with a primarily theoretical contribution.", "weaknesses": "1. If the authors would like to present more of a empirical contribution, it would help to scale the experiments to something of the order of 100s of dimensions such as in common RL environments like mujoco-humanoid.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First, we would like to thank you for finding our work to have a strong theoretical basis, well-written,\nand easy-to-read with reasonable experiments. Regarding larger-scale experiments, we have been unable to\nperform them at this time. We agree that they would add value and we envision executing them as future\nwork.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kxfWQF1RPi", "reviewer": "Reviewer_Njwt", "summary": "The paper proposes a novel neural network architecture, Extended Linearized Contracting Dynamics (ELCD), for learning dynamical systems with global contractivity guarantees. The authors leverage the concept of extended linearization to factorize the vector field and enforce negative definiteness of its symmetric part, ensuring global exponential stability and equilibrium contraction. To extend contractivity to more general metrics, they introduce a latent space representation and learn diffeomorphisms between the data and latent spaces. The method is evaluated on various datasets, demonstrating superior performance compared to existing models, including Neural Contractive Dynamical Systems (NCDS).", "questions": "1. Is the ELCD model expressive enough to represent all contracting dynamical systems? If not, what are the limitations, and can they be formally characterized?\n\n2. The paper states that \"contraction is invariant under differential coordinate changes...\" (L227) but also mentions that \"If the data is not contracting in the transformed space...\" (L291). These statements seem contradictory. Can the authors clarify this discrepancy and explain how the choice of diffeomorphism affects the contractivity of the learned dynamics?\n\n3. The computational complexity of approximating the contraction metric is mentioned as a potential limitation. Can the authors elaborate on the scalability of the method to high-dimensional systems and suggest possible strategies to mitigate this issue?\n\n4. The paper focuses on trajectories converging to a fixed point. How can the method be extended to handle more complex dynamical behaviors, such as limit cycles or chaotic attractors?\n\n5. Why is joint training of the encoder and the model possible for ELCD but not in NCDS?", "limitations": "The limitation sections address some of the limitations of the current approach, but is insufficient. \n\nThe current implementation of ELCD is limited to trajectories converging to a fixed point and dynamics in Euclidean space. This restricts its applicability to problems where these assumptions hold. For example, while ELCD is applied to model handwriting in the LASA dataset, it struggles to capture the translation invariance inherent in human handwriting, where the end-point location is not fixed. It would be beneficial to  discuss the specific types of problems where ELCD would be most applicable and where it might not be suitable.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper addresses the important problem of learning stable and robust dynamical systems, which has significant implications in various fields.\n\nThe proposed ELCD model is theoretically sound, with guarantees of global contractivity, a property that ensures robustness to perturbations.\n\nThe use of extended linearization and latent space representation is innovative and allows the model to capture a wider range of contracting dynamics.\n\nThe experimental results are promising, showing that ELCD outperforms existing methods on several benchmark datasets.\n\nELCD addresses some limitations of NCDS. Specifically, ELCD parameterizes the vector field directly, avoiding the computationally expensive Jacobian integration step in NCDS. ELCD also allows for asymmetric Jacobians, leading to increased model expressivity compared to NCDS, which is restricted to symmetric Jacobians.\n\nELCD trains the encoder jointly with the dynamics model, allowing the encoder to learn representations specifically tailored for capturing contracting dynamics. This is in contrast to NCDS, which trains the encoder separately as part of a Variational Autoencoder (VAE) before training the dynamics model. The joint training in ELCD potentially leads to better performance by optimizing the encoder for the specific task of representing contracting dynamics.", "weaknesses": "The assumption of contractivity in the underlying dynamics might limit the applicability of the model to real-world systems where this assumption may not hold.\n\nThe current implementation is restricted to trajectories converging to a fixed point and dynamics in Euclidean space, which could be extended to more general settings. This limitation is particularly evident in the LASA handwriting dataset, where the model struggles to capture the translation invariance inherent in human handwriting. In reality, the end-point location of handwriting is not fixed, and the entire letter can be translated without affecting its validity.  Also, handwriting can often exhibit overlapping trajectories, which cannot be modeled by the current implementation. Moreover, the learned model can only generate one handwriting.\n\nIt is unclear why the joint training of the encoder and the model is possible for ELCD but not in NCDS. The authors could address this point to clarify the advantages of their approach.\n\nThe paper lacks a clear discussion of the specific types of problems where ELCD would be most applicable and where it might not be suitable. A more detailed analysis of the model's strengths and weaknesses in different scenarios would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**It is unclear why the joint training of the encoder and the model is possible for ELCD but not in NCDS.**\n\nIt is possible to jointly train the encoder and the model for NCDS as well. The authors of NCDS treat the encoder as a Variational Autoencoder (VAE) and train it by minimizing the evidence lower bound. The authors claim that the VAE training helps deal with the complexity of high dimensional systems. We argue, however, that the encoder and model must be trained jointly. If the encoder is trained independently of the model, then it is not necessarily true that the dynamics will be $l_2$-contracting in the learned transformed space. We confirmed this lack of contractivity experimentally, finding that VAE training of the encoder followed by training of the model did not produce reportable results. These poor results explain why we used joint training for NCDS for our baseline results. Since the NCDS code is not available, we have reproduced it to the best of our abilities.\n\n\n**The paper lacks a clear discussion of the specific types of problems where ELCD would be most applicable...**\n\nWe envision applications in imitation learning where the resulting learned\nmodel has global guarantees of contraction, even away from the training\ndata. Additionally, we imagine learning control systems of the form $\\dot{x} = f(x) +\nBu$, where $f$ is contracting. We could then leverage recent results on\noptimal control of contracting systems to design effective controllers for\nthese systems [7].  We will update the introduction,\nmotivation, and discussion about future research to highlight these points.\n\n\n**Is the ELCD model expressive enough to represent all contracting dynamical systems?**\n\nThank you for your question.\n\nIt is known that any smooth nonlinear system with a hyperbolically stable\nfixed point can be exactly linearized inside its basin of attraction via a\nsuitable diffeomorphism [8].\n\nIn essence, our extended-linear parameterization is tasked with learning\nthe stable linear part, while the coupling layers aim to learn and\napproximate this suitable diffeomorphism.\n\nSo your question can be transcribed into the following: are coupling layers\ncapable of approximating any diffeomorphisms?\n\nSome results in this direction were obtained\nby [9,10]. The survey [11] confirms that those universality conditions hold for our choice in coupling layer [12]. \n\nHowever, the universality results in [9,10] apply to learning transformations between cumulative distribution functions. At this time, we cannot directly answer the universality question about diffeomorphisms.\n\n\n**Can the authors clarify... how the choice of diffeomorphism affects the contractivity of the learned dynamics?**\n\nYou raise a good point. If we have a contracting dynamical systems, then any diffeomorphism applied to the system ensures that the resulting systems remains contracting (in a different Riemannian metric, however). In L291 we were commenting on the specific parametrization of NCDS [1]. Since the NCDS parametrization ensures contraction in the $l_2$ metric, it is possible that an arbitrary diffeomorphism applied to data coming from some contracting dynamics is not contracting in the $l_2$ metric (although it would still be contracting in some other metric). We realize this point was not fully clear in the first version of our paper and\nthe revision will include a clearer more detailed explanation.\n\n\n\n**Complexity of approximating the contraction metric...**\n\nThis is a good question. In our opinion, ELCD scales well independently of the complexity of approximating the contraction metric. This is because we do not compute the contraction metric at any step during training or evaluation. The complexity of approximating the metric would only arise if the end user needed to have an expression for it; something we do not need in our tasks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**Explanation of Updated Results**

After submission, we improved the performance of our method by implementing a change in the training loss. We had previously trained the model $f$ and the encoder $\phi$ to output $x_{t+1}$ from $x_{t}$ using a single Euler integration step: $x_{t+1} = \phi^{-1}(\phi(x_t) + dt * f(\phi(x_t))$. Our loss was the MSE between the predicted and true $x_{t+1}$. This training method, which we call discretized-time training, is the same method used by [1] to train their NCDS model.

We have now updated our training method to, instead, directly predict the velocity vector $\dot{x}_{t+1} = \phi^{-1}_J(x_t)f(\phi(x_t)$, where $\phi^{-1}_J(x_t)$ is the Jacobian of the inverse of $\phi(x_t)$. We call this new training method continuous-time training. To avoid slowdowns, the computation of this Jacobian requires a careful implementation in pytorch; this will be documented in the final code. This change is purely technical and does not impact the parameterization of the model, the parametrization of the diffeomorphism, or the overall content of the paper. Directly predicting the velocity vector through continuous-time training allows us to better demonstrate the efficacy of our method. The new results are shown in table (1) in the attached document, which is the same as table (1) in the submission, with an additional column for the ELCD trained via continuous-time training. In the submission, we will remove the column from table (1) with ELCD trained by the original discretized-time training method. Our ELCD model with the new, continuous-time training loss has overall improved performance. Additionally, we visualize the performance of ELCD with continuous-time training on the 2-link pendulum in figure (1) in the attachment. 

**Regarding learning non-contracting dynamics and limitations**

Several reviewers asked us to comment on limitations and learning tasks
where the true dynamics have multiple fixed points or limit cycles /
chaotic attractors. Motivated by applications in imitation learning and
robotics, in this work, we are primarily interested in problems where the
underlying systems should be robustly stable, especially away from training
data, following the same reasoning in [1]. For
this reason, we focus on learning dynamics which are guaranteed to be
globally contracting. We imagine that extensions of ELCD could capture
these richer dynamical behaviors by leveraging weaker notions of
contraction including local contraction (i.e., contractivity in the region
of attraction of a stable equilibrium), $k$-contraction (contraction of
$k$-dimensional bodies) [2], transverse
contraction [3] and contraction in the Hausdorff
dimension [4]. Moreover, the notion of translation
invariance mentioned by Reviewer Njwt could be studied using
semicontraction theory [5], i.e., contraction to a
subspace. Since we do not study these extensions here, we will add more
details in the limitations section elaborating on this point.

**References**

[1] H. Beik-Mohammadi, S. Hauberg, G. Arvanitidis, N. Figueroa, G. Neumann, and L. Rozo, “Neural
contractive dynamical systems,” in International Conference on Learning Representations, 2024.

[2] C. Wu, I. Kanevskiy, and M. Margaliot, “k-contraction: Theory and applications,” Automatica, vol. 136,
p. 110048, 2022.

[3] I. R. Manchester and J.-J. E. Slotine, “Transverse contraction criteria for existence, stability, and
robustness of a limit cycle,” Systems & Control Letters, vol. 63, pp. 32–38, 2014.

[4] C. Wu, R. Pines, M. Margaliot, and J.-J. E. Slotine, “Generalization of the multiplicative and additive
compounds of square matrices and contraction in the Hausdorff dimension,” IEEE Transactions on
Automatic Control, 2022.

[5] G. De Pasquale, K. D. Smith, F. Bullo, and M. E. Valcher, “Dual seminorms, ergodic coefficients, and
semicontraction theory,” IEEE Transactions on Automatic Control, vol. 69, no. 5, 2024. To appear.
[6] H. Tsukamoto, S.-J. Chung, and J.-J. E. Slotine, “Contraction theory for nonlinear stability analysis
and learning-based control: A tutorial overview,” Annual Reviews in Control, vol. 52, pp. 135–169,
2021.

[7] S. Taoufik and B. Missaoui, “Method of successive approximations for stochastic optimal control: Con-
tractivity and convergence,” arXiv preprint 2405.07048, 2024.

[8] Y. Lan and I. Mezi ́c, “Linearization in the large of nonlinear systems and Koopman operator spectrum,”
Physica D: Nonlinear Phenomena, vol. 242, no. 1, pp. 42–53, 2013.

[9] C.-W. Huang, D. Krueger, A. Lacoste, and A. Courville, “Neural autoregressive flows,” in International
Conference on Machine Learning, pp. 2078–2087, PMLR, 2018.

[10] P. Jaini, K. A. Selby, and Y. Yu, “Sum-of-squares polynomial flow,” in International Conference on
Machine Learning, pp. 3009–3018, PMLR, 2019.

[11] I. Kobyzev, S. J. D. Prince, and M. A. Brubaker, “Normalizing flows: An introduction and review of
current methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 11,
pp. 3964–3979, 2021.

[12] C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios, “Neural spline flows,” in Advances in Neural
Information Processing Systems (H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch ́e-Buc, E. Fox,
and R. Garnett, eds.), vol. 32, Curran Associates, Inc., 2019

Meta-review: ### Summary

This paper proposes a method for learning a contractive dynamical system from demonstrations, with global contractivity guarantees. The system is flexible and parameterized by a simple neural network architecture that relies only on mild constraints. Contractivity is achieved under an implicit metric. Additionally, a diffeomorphism enables the modeling of the dynamics, as the system is learned in the latent space, where the representations help to ensure contractivity, while the mapping to the input space handles the expressivity.

$$$$

### Strengths

**Contribution**: This paper considers an important problem and addresses some limitations of the seminal NCDS model (efficiency, expressiveness, and global contraction guarantees). It introduces an innovative combination of latent space learning and extended linearization, enabling the capture of contracting dynamics through learned representations. The provided results are promising.

**Theoretical Clarity**: The paper is well-written with a clear structure (easy to follow). The theoretical guarantees and insights were appreciated by most reviewers. The concept of extended linearization is neat as well as the use of the converse contraction theorem to prove the main contractivity result.

$$$$

### Concerns

**Experiments**: Most reviewers found the experimental section to be somewhat limited, noting that it does not sufficiently demonstrate the claimed advantages of the proposed model.

**Novelty**: Concerns were raised by Reviewer HPgs, who described the proposed methodology as "incremental," noting that the benefits cannot be adequately assessed from the provided experiments.

$$$$

### Conclusion
Considering the contributions of the paper --particularly the modeling choices and theoretical results-- along with the efforts made to address the reviewers' questions and the additional experiments/extensions provided in the rebuttal, I believe this paper is a reasonable contribution to the conference and recommend acceptance as a poster. While the experimental section may be lacking, the other aspects were well-received by most reviewers, even though the approach appears somewhat incremental to previous works.
Decision: Accept (poster)