Original: A Nonparametric Spatio-temporal SDE Model
Revised: A Nonparametric Spatio-temporal SDE Model

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: We propose a nonparametric spatio-temporal stochastic differential equation (SDE)model that can learn the underlying dynamics of arbitrary continuous-time systemswithout prior knowledge.
Revised: We propose a nonparametric spatio-temporal stochastic differential equation (SDE) model that can learn the underlying dynamics of arbitrary continuous-time systems without prior knowledge.

Original: We augment the input space of the drift function ofan SDE with a temporal component to account for spatio-temporal patterns.
Revised: We augment the input space of the drift function of an SDE with a temporal component to account for spatio-temporal patterns.

Original: Theexperiment on a real world data set demonstrates that the spatio-temporal modelis better able to ﬁt the data than the spatial model and also reduce the forecastingerror.
Revised: The experiments on a real world data set demonstrate that the spatio-temporal model is better able to ﬁt the data than the spatial model and also reduce the forecasting error

Original:  Introduction
Revised:  Introduction

Original: Dynamical systems modeling is a cornerstone of experimental sciences.
Revised: Dynamical systems modeling is a cornerstone of experimental sciences.

Original: Modelers attempt to capturethe dynamical behavior of a stochastic system or a phenomenon in order to improve its understandingand make predictions about its future state.
Revised: Modelers attempt to capture the dynamical behavior of a stochastic system or a phenomenon in order to improve its understanding and make predictions about its future state.

Original: Stochastic differential equations (SDEs) are an effectiveformalism for modelling systems with underlying stochastic dynamics, with wide range of applications [5].
Revised: Stochastic differential equations (SDEs) are an effective formalism for modelling systems with underlying stochastic dynamics, with wide range of applications [5].

Original: The key problem in SDE’s is estimation of the underlying deterministic driving function,and the stochastic diffusion component.
Revised: The key problem in SDE’s is estimation of the underlying deterministic driving function, and the stochastic diffusion component.

Original: In this work, we are interested in a multivariate system governed by Markov process x t described byan SDE
Revised: In this work, we are interested in a multivariate system governed by Markov process x t described by an SDE

Original: [Equation]
Revised: [Equation]

Original: where x t ∈ R D is the state vector of a D -dimensional dynamical system at continuous time t ∈ R ,f
Revised: where x t ∈ R D is the state vector of a D -dimensional dynamical system at continuous time t ∈ R , f

Original: ( x , t ) ∈
Revised: ( x , t ) ∈

Original: R D is a deterministic state evolution, σ ( x t , t ) ∈ R is a scalar magnitude of the stochasticmultivariate Wiener process W t ∈ R D .
Revised: R D is a deterministic state evolution, σ ( x t , t ) ∈ R is a scalar magnitude of the stochastic multivariate Wiener process W t ∈ R D .

Original: The Wiener process has zero initial state W 0 = 0 , and theindependent increments W t + s − W t ∼ N ( 0 , sI ) follow a Gaussian with standard deviation √ s .
Revised: The Wiener process has zero initial state W 0 = 0 , and the independent increments W t + s − W t ∼ N ( 0 , sI ) follow a Gaussian with standard deviation √ s .

Original: Thestate solutions of SDE are given by the Itô integral [9]
Revised: The state solutions of SDE are given by the Itô integral [9]

Original: [Equation]
Revised: [Equation]

Original: (2) where we integrate the system state from an initial state x 0 for time t forward, and where τ is anauxiliary time variable.
Revised: where we integrate the system state from an initial state x 0 for time t forward, and where τ is an auxiliary time variable.

Original: We assume the states are observed with additive noise y ( t ) =
Revised: We assume the states are observed with additive noise y ( t ) =

Original: x ( t ) + ε t withε t ∼ N ( 0 , Ω) and Ω = diag( ω 21 , . . .
Revised: x ( t ) + ε t with ε t ∼ N ( 0 , Ω) and Ω = diag( ω 21 , . . .

Original: , ω 2 D ) .
Revised: , ω 2 D ) .

Original: With σ ( x τ , τ ) = 0 , we obtain the ordinary differentialequation (ODE) solution.
Revised: With σ ( x τ , τ ) = 0 , we obtain the ordinary differential equation (ODE) solution.

Original: There is a vast literature on inferring SDEs [5] where a parametric form for drift and diffusionfunctions are assumed to be known, and the parameters of those functions are optimized.
Revised: There is a vast literature on inferring SDEs [5] where a parametric form for drift and diffusion functions are assumed to be known, and the parameters of those functions are optimized.

Original: Suchmethods cannot be readily applied to real world data sets, where the underlying dynamics areunknown.
Revised: Such methods cannot be readily applied to real world data sets, where the underlying dynamics are unknown.

Original: To tackle such data sets, non-parametric drift and diffusion functions have been proposedin several works [16, 3, 6, 13].
Revised: To tackle such data sets, non-parametric drift and diffusion functions have been proposed in several works [16, 3, 6, 13].

Original: This work extends the method proposed in [16] to account forspatio-temporal patterns.
Revised: This work extends the method proposed in [16] to account for spatio-temporal patterns.

Original:  Nonparametric SDE Model
Revised:  Nonparametric SDE Model

Original: In this section, we review the sparse Gaussian process (GP) modeling of differential equationsdescribed in [16].
Revised: In this section, we review the sparse Gaussian process (GP) modeling of differential equations described in [16].

Original: The model introduces two GP priors over the vector valued drift f ( x ) : R D → R Dand scalar valued diffusion σ ( x ) : R D → R functions
Revised: The model introduces two GP priors over the vector valued drift f ( x ) : R D → R D and scalar valued diffusion σ ( x ) : R D → R functions

Original: [Equation]
Revised: [Equation]

Original: Drift and diffusion functions have zero mean, the drift kernel K f [Equation] x , x
Revised: Drift and diffusion functions have zero mean, the drift kernel K f ( x , x

Original: [Equation]
Revised: [Equation]

Original: ∈ R D
Revised: ∈ R D

Original: × D is matrix valued,and the diffusion kernel k σ ( x , x (cid:48) )
Revised: × D is matrix valued, and the diffusion kernel k σ ( x , x (cid:48) )

Original: ∈ R is univariate.
Revised: ∈ R is univariate.

Original: Covariance matrices are deﬁned using squaredexponential kernel function
Revised: Covariance matrices are deﬁned using squared exponential kernel function

Original: [Equation]
Revised: [Equation]

Original: where θ f = { σ f , (cid:96) f 1 , . .
Revised: where θ f = { σ f , (cid:96) f 1 , . .

Original: , (cid:96) f D } and θ σ = { σ σ , (cid:96) σ 1 , . . .
Revised: , (cid:96) f D } and θ σ = { σ σ , (cid:96) σ 1 , . . .

Original: , (cid:96) σD } stand for the function-speciﬁc kernelparameters.
Revised: , (cid:96) σD } stand for the function-speciﬁc kernel parameters.

Original: Following [16], the identity decomposable kernel K f ( x , x (cid:48) )
Revised: Following [16], the identity decomposable kernel K f ( x , x (cid:48) )

Original: =
Revised: =

Original: k ( x , x (cid:48) ) · I D is usedfor the matrix valued drift kernel [2], whereas k σ ( x , x (cid:48) )
Revised: k ( x , x (cid:48) ) · I D is used for the matrix valued drift kernel [2], whereas k σ ( x , x (cid:48) )

Original: =
Revised: =

Original: k ( x , x (cid:48) ) .
Revised: k ( x , x (cid:48) ) .

Original: Given a set of states X =( x 1 , . . .
Revised: Given a set of states X = ( x 1 , . . .

Original: , x N ) T ∈ R N × D , the values of the drift function F = ( f ( x 1 ) , . . .
Revised: , x N ) T ∈ R N × D , the values of the drift function F = ( f ( x 1 ) , . . .

Original: , f ( x N ))
Revised: , f ( x N ))

Original: T ∈ R N × D andthe diffusion function σ = ( σ ( x 1 ) , . . .
Revised: T ∈ R N × D and the diffusion function σ = ( σ ( x 1 ) , . . .

Original: , σ ( x N ))
Revised: , σ ( x N ))

Original: T ∈ R N follow normal prior distributions
Revised: T ∈ R N follow normal prior distributions

Original: [Equation]
Revised: [Equation]

Original: where K f ( X, X )
Revised: where K f ( X, X )

Original: =
Revised: =

Original: ( K f ( x i , x j ))
Revised: ( K f ( x i , x j ))

Original: Ni,j =1 ∈ R ND × ND is a block matrix of matrix-valued kernels
Revised: Ni,j =1 ∈ R ND × ND is a block matrix of matrix-valued kernels

Original: K f ( x i , x j ) and diffusion kernel is given by K σ ( X, X ) =
Revised: K f ( x i , x j ) and diffusion kernel is given by K σ ( X, X ) =

Original: ( k σ ( x i , x j ))
Revised: ( k σ ( x i , x j ))

Original: Ni,j =1 ∈ R N × N .
Revised: Ni,j =1 ∈ R N × N .

Original: In standard GP regression, the posterior over the function values are obtained by conditioning the GPprior on the data [11].
Revised: In standard GP regression, the posterior over the function values are obtained by conditioning the GP prior on the data [11].

Original: On the other hand, in differential equation models, the conditionals f ( x ) | Yand σ ( x ) | Y are intractable due to the integral mapping between observed states and differentials.
Revised: On the other hand, in differential equation models, the conditionals f ( x ) | Y and σ ( x ) | Y are intractable due to the integral mapping between observed states and differentials.

Original: To overcome this, two sets of inducing variables
Revised: To overcome this, two sets of inducing variables

Original: U f
Revised: U f

Original: = ( u f 1 , . .
Revised: = ( u f 1 , . .

Original: u f M ) T ∈ R M × D and u σ =( u σ 1 , . .
Revised: u f M ) T ∈ R M × D and u σ = ( u σ 1 , . .

Original: . , u σM ) T ∈ R M are introduced [10].
Revised: . , u σM ) T ∈ R M are introduced [10].

Original: U f and u σ contain the values of the drift and diffusionfunctions at inducing locations Z
Revised: U f and u σ contain the values of the drift and diffusion functions at inducing locations Z

Original: = ( z 1 , . . .
Revised: = ( z 1 , . . .

Original: , z M ) , which live in the same space as the states X .
Revised: , z M ) , which live in the same space as the states X .

Original: Finally, the drift and diffusion functions are interpolated from the locations and variables:
Revised: Finally, the drift and diffusion functions are interpolated from the locations and variables:

Original: [Equation]
Revised: [Equation]

Original: where u f = vec U f .
Revised: where u f = vec U f .

Original: The SDE model is determined via the inducing locations
Revised: The SDE model is determined via the inducing locations

Original: Z , the inducing values
Revised: Z , the inducing values

Original: U f and u σ , theobservation noise variance Ω , and the kernel parameters θ f
Revised: U f and u σ , the observation noise variance Ω , and the kernel parameters θ f

Original: and θ σ of the drift and diffusion kernels.
Revised: and θ σ of the drift and diffusion kernels.

Original: The model learns the underlying system to induce state distributions with high expected likelihood 
Revised: The model learns the underlying system to induce state distributions with high expected likelihood p ( y i | f , σ, Ω) =

Original: [Equation] ] , which is intractable.
Revised: E p ( x | t i ; f ,σ ) [ N ( y i | x , Ω)] , which is intractable.

Original: The posterior of the model combinesthe likelihood p ( y i | f , σ, Ω) and the independent priors p ( U f ) and p ( u σ ) using Bayes’ theorem as
Revised: The posterior of the model combines the likelihood p ( y i | f , σ, Ω) and the independent priors p ( U f ) and p ( u σ ) using Bayes’ theorem as

Original: [Equation]
Revised: [Equation]

Original: where x ( s ) ∼ p ( x 0 ...t | U f , u σ , Z ) denotes a path sample x ( s ) t that is drawn from the time dependentstate distribution p ( x 0 ...t | U f , u σ , Z ) by sampling a Brownian motion path W ( s ) t .
Revised: where x ( s ) ∼ p ( x 0 ...t | U f , u σ , Z ) denotes a path sample x ( s ) t that is drawn from the time dependent state distribution p ( x 0 ...t | U f , u σ , Z ) by sampling a Brownian motion path W ( s ) t .

Original: The true expectedlikelihood is approximated by unbiased Monte Carlo averaging.
Revised: The true expected likelihood is approximated by unbiased Monte Carlo averaging.

Original: The likelihood estimate with N ssamples turns out to be a kernel density estimator with Gaussian bases.
Revised: The likelihood estimate with N s samples turns out to be a kernel density estimator with Gaussian bases.

Original: We draw the sample paths using Euler-Maruyama (EM) [9]:
Revised: We draw the sample paths using Euler-Maruyama (EM) [9]:

Original: [Equation]
Revised: [Equation]

Original: 9) where we discretise time into N T subintervals t 0 , t 1 , . ..
Revised: where we discretise time into N T subintervals t 0 , t 1 , . .

Original: , t N T of width ∆ t = t N T /N T , and samplethe Wiener coefﬁcients as ∆ W ( s ) i ∼ N ( 0 , ∆ t · I ) with standard deviation √ ∆ t .
Revised: , t N T of width ∆ t = t N T /N T , and sample the Wiener coefﬁcients as ∆ W ( s ) i ∼ N ( 0 , ∆ t · I ) with standard deviation √ ∆ t .

Original: We set x ( s ) 0 to theinitial observation and use (9) to compute state path x
Revised: We set x ( s ) 0 to the initial observation and use (9) to compute state path x

Original: ( s ) ≡ ( x ( s ) 0 , x ( s ) 1 , . . .
Revised: ( s ) ≡ ( x ( s ) 0 , x ( s ) 1 , . . .

Original: , x ( s ) T N ) .
Revised: , x ( s ) T N ) .

Original: The number oftime steps N T > N is often higher than the number of observed timepoints to achieve sufﬁcient pathresolution.
Revised: The number of time steps N T > N is often higher than the number of observed timepoints to achieve sufﬁcient path resolution.

Original: Finally, the maximum a posteriori (MAP) estimate of the model parameters are foundusing sensitivity equation based gradients (see [16] for details).
Revised: Finally, the maximum a posteriori (MAP) estimate of the model parameters are found using sensitivity equation based gradients (see [16] for details).

Original:  Spatio-Temporal SDE Model
Revised:  Spatio-Temporal SDE Model

Original: So far, the formulation relies on SDE states x ( t ) and the inducing locations living in the same space,71 R D , which remains valid as long as the input to the drift function is only the state.
Revised: So far, the formulation relies on SDE states x ( t ) and the inducing locations living in the same space, R D , which remains valid as long as the input to the drift function is only the state.

Original: In order todeﬁne drift functions explicitly parameterized by time, we augment the space to include a temporalcomponent, that is, f ( x , t ) :
Revised: In order to deﬁne drift functions explicitly parameterized by time, we augment the space to include a temporal component, that is, f ( x , t ) :

Original: R D + → R D where R D + = ( R D × R + ) .
Revised: R D + → R D where R D + = ( R D × R + ) .

Original: The GP prior over the driftfunction then becomes
Revised: The GP prior over the drift function then becomes

Original: [Equation]
Revised: [Equation]

Original: The matrix valued kernel is again deﬁned to be the identity decomposable kernel 
Revised: The matrix valued kernel is again deﬁned to be the identity decomposable kernel K f (

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: t ) ,
Revised: t ) ,

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: , t
Revised: , t

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: k (( x , t ) , ( x (cid:48) , t (cid:48) )) · I D , where we redeﬁne the kernel function of the drift as
Revised: k (( x , t ) , ( x (cid:48) , t (cid:48) )) · I D , where we redeﬁne the kernel function of the drift as

Original: [Equation]
Revised: [Equation]

Original: x ,

Original: [Equation]
Revised: [Equation]

Original: New kernel function turns out to be the product of squared exponential kernel functions over spaceand time and requires one additional parameter (cid:96) t modeling the smoothness in temporal domain.
Revised: New kernel function turns out to be the product of squared exponential kernel functions over space and time and requires one additional parameter (cid:96) t modeling the smoothness in temporal domain.

Original: In the limit (cid:96) t → ∞ , we recover the spatial model.
Revised: In the limit (cid:96) t → ∞ , we recover the spatial model.

Original: Because the kernel function is deﬁned in theextended space, inducing locations Z must have an additional component: z i ∈ R D + .
Revised: Because the kernel function is deﬁned in the extended space, inducing locations Z must have an additional component: z i ∈ R D + .

Original: Time variantdiffusion functions can also be deﬁned in a similar way, which we do not investigate in this work.
Revised: Time variant diffusion functions can also be deﬁned in a similar way, which we do not investigate in this work.

Original: Spatio-temporal drift function boosts the model power and ﬂexibility without any additional parameterother than temporal lengthscale.
Revised: Spatio-temporal drift function boosts the model power and ﬂexibility without any additional parameter other than temporal lengthscale.

Original: In spatial systems, state trajectory follows the same dynamics (up torandom perturbations due to diffusion) when a particular state is visited at different points in time.
Revised: In spatial systems, state trajectory follows the same dynamics (up to random perturbations due to diffusion) when a particular state is visited at different points in time.

Original: With the proposed spatio-temporal formulation, temporal proximity is also taken into account whendeﬁning state dynamics, meaning that different representations can be learned over time.
Revised: With the proposed spatio-temporal formulation, temporal proximity is also taken into account when deﬁning state dynamics, meaning that different representations can be learned over time.

Original: The model is implemented in Tensorﬂow [1].
Revised: The model is implemented in Tensorﬂow [1].

Original: We ﬁnd the MAP estimates of the inducing variables
Revised: We ﬁnd the MAP estimates of the inducing variables

Original: U f , u σ and the noise variance Ω using the Adam [8] optimizer with learning rate 0.001 and staircase(every 20th iteration) exponential decay with rate 0.99.
Revised: U f , u σ and the noise variance Ω using the Adam [8] optimizer with learning rate 0.001 and staircase (every 20th iteration) exponential decay with rate 0.99.

Original: Kernel parameters in complex GP modelsare usually hard to optimize since they appear in the covariance matrix.
Revised: Kernel parameters in complex GP models are usually hard to optimize since they appear in the covariance matrix.

Original: In our case, the covariancematrix itself is connected to the likelihood through an integral, further complicating the optimization.
Revised: In our case, the covariance matrix itself is connected to the likelihood through an integral, further complicating the optimization.

Original: Therefore, we perform a grid search to optimize lengthscales { (cid:96) f 1 . . .
Revised: Therefore, we perform a grid search to optimize lengthscales { (cid:96) f 1 . . .

Original: (cid:96) f D , (cid:96) t , (cid:96) σ 1 . . .
Revised: (cid:96) f D , (cid:96) t , (cid:96) σ 1 . . .

Original: (cid:96) σD } , and setthe signal variance terms to one, σ f = σ g = 1 .
Revised: (cid:96) σD } , and set the signal variance terms to one, σ f = σ g = 1 .

Original: The inducing points are scattered around the data andkept ﬁxed during optimization.
Revised: The inducing points are scattered around the data and kept ﬁxed during optimization.

Original:  Experiments
Revised:  Experiments

Original: The experiments aim at demonstrating that the spatio-temporal model is better able to ﬁt the datathan the spatial model, and also better at capturing the data generating dynamics.
Revised: The experiments aim at demonstrating that the spatio-temporal model is better able to ﬁt the data than the spatial model, and also better at capturing the data generating dynamics.

Original: The experimentsare performed on a benchmark dataset of human motion capture data from the Carnegie Mellon
Revised: The experiments are performed on a benchmark dataset of human motion capture data from the Carnegie Mellon

Original: University motion capture (CMU mocap) database.
Revised: University motion capture (CMU mocap) database.

Original: We followed the data preprocessing technique in[14], which results in 50 -dimensional pose measurements from a person swinging a golf club where each pose dimension records a measurement in different parts of the body during the action 1 .
Revised: We followed the data preprocessing technique in [14], which results in 50 -dimensional pose measurements from a person swinging a golf club where each pose dimension records a measurement in different parts of the body during the action 1 .

Original: Weconsider four input sequences, each having 350 data points to train the model.
Revised: We consider four input sequences, each having 350 data points to train the model.

Original: In order to tackle theproblem of dimensionality, we project the original dataset with PCA to a three dimensional latentspace where the system is speciﬁed, following [4] and [15].
Revised: In order to tackle the problem of dimensionality, we project the original dataset with PCA to a three dimensional latent space where the system is speciﬁed, following [4] and [15].

Original: We also investigated whether incorporating temporal information into the drift function enhancesnon-parametric ordinary differential equation (ODE) model presented in [7], which is tested bydiscarding the diffusion component from (1) and ﬁtting the same input sequences.
Revised: We also investigated whether incorporating temporal information into the drift function enhances non-parametric ordinary differential equation (ODE) model presented in [7], which is tested by discarding the diffusion component from (1) and ﬁtting the same input sequences.

Original: Overall fourmodels are examined, which we call in short sODE, stODE, sSDE and stSDE, lower case lettersrepresenting if the system is spatial or spatio-temporal.
Revised: Overall four models are examined, which we call in short sODE, stODE, sSDE and stSDE, lower case letters representing if the system is spatial or spatio-temporal.

Original: Each model is optimized 750 times usingdifferent lengthscale and inducing locations.
Revised: Each model is optimized 750 times using different lengthscale and inducing locations.

Original: In order to select the hyperparameters that preventoverﬁtting, we compute the root mean square error (RMSE) between 3 holdout input sequences andstate paths (at observed time points) that are estimated using the EM integration from the measuredinitial states.
Revised: In order to select the hyperparameters that prevent overﬁtting, we compute the root mean square error (RMSE) between 3 holdout input sequences and state paths (at observed time points) that are estimated using the EM integration from the measured initial states.

Original: The models are then retrained using 4 training and 3 holdout sequences, and ﬁnally
Revised: The models are then retrained using 4 training and 3 holdout sequences, and ﬁnally

Original: RMSE between 3 test input sequences and the estimated state paths are reported.
Revised: RMSE between 3 test input sequences and the estimated state paths are reported.

Original: Figure 1 visualizes ﬁts for four models on the same training data.
Revised: Figure 1 visualizes ﬁts for four models on the same training data.

Original: In both ODE and SDE settings, wesee that inferred spatio-temporal trajectories are closer to the data than the spatial counterparts.
Revised: In both ODE and SDE settings, we see that inferred spatio-temporal trajectories are closer to the data than the spatial counterparts.

Original: Forthe same reason, random state trajectory cloud needs to be more voluminous in sSDE to achieve agreater posterior whereas stSDE can ﬁt individual trajectories.
Revised: For the same reason, random state trajectory cloud needs to be more voluminous in sSDE to achieve a greater posterior whereas stSDE can ﬁt individual trajectories.

Original: Also, the bottom left corners in theﬁgures show that temporal information helps capturing rapidly changing spatial ﬁelds.
Revised: Also, the bottom left corners in the ﬁgures show that temporal information helps capturing rapidly changing spatial ﬁelds.

Original: Quantitative results are presented in Figure 2.
Revised: Quantitative results are presented in Figure 2.

Original: Unsurprisingly, time-variant drift functions boost thetraining performance with more than 10 inducing points, which is compatible with the plots in Figure1.
Revised: Unsurprisingly, time-variant drift functions boost the training performance with more than 10 inducing points, which is compatible with the plots in Figure 1.

Original: We also observe that increasing the number of inducing points yields better training posteriors in
Revised: We also observe that increasing the number of inducing points yields better training posteriors in

Original: SDE model.
Revised: SDE model.

Original: Figures 2c-d clearly illustrate that time-variant drift function leads to reduced predictionerror, which hints that the dynamics underlying a golf swing motion are learned better.
Revised: Figures 2c-d clearly illustrate that time-variant drift function leads to reduced prediction error, which hints that the dynamics underlying a golf swing motion are learned better.

Original: We also seethat the error consistently decreases as the number of inducing points M is increased, and reaches theminimum at M = 80 .
Revised: We also see that the error consistently decreases as the number of inducing points M is increased, and reaches the minimum at M = 80 .

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: We propose an approach for learning non-parametric spatio-temporal drift and diffusion functions ofstochastic differential equation (SDE) systems such that the resulting simulated state distributionsmatch data.
Revised: We propose an approach for learning non-parametric spatio-temporal drift and diffusion functions of stochastic differential equation (SDE) systems such that the resulting simulated state distributions match data.

Original: The experiment on a real world data set shows that our model can better ﬁt complexdynamics than the spatial counterpart.
Revised: The experiment on a real world data set shows that our model can better ﬁt complex dynamics than the spatial counterpart.

Original: An interesting future research direction is the study of variousvector ﬁeld kernels, such as divergence-free, curl-free or spectral kernels [12].
Revised: An interesting future research direction is the study of various vector ﬁeld kernels, such as divergence-free, curl-free or spectral kernels [12].

Original: The model couldbe extended to have an observation model, e.g., GPLVM or deep neural network, rather than PCA.
Revised: The model could be extended to have an observation model, e.g., GPLVM or deep neural network, rather than PCA.

Original: Including inputs or controls to the system would allow precise modelling in interactive settings, suchas robotics
Revised: Including inputs or controls to the system would allow precise modelling in interactive settings, such as robotics.

Original: Acknowledgments
Revised: Acknowledgments

Original: The data used in this project was obtained from mocap.cs.cmu.edu .
Revised: The data used in this project was obtained from mocap.cs.cmu.edu .

Original: The database was created withfunding from NSF EIA-0196217.
Revised: The database was created with funding from NSF EIA-0196217.

Original: This work has been supported by the Academy of Finland grantsno.
Revised: This work has been supported by the Academy of Finland grants no. 260403, 299915, 275537,