Original: A Nonparametric Spatio-temporal SDE Model
Revised: A Nonparametric Spatio-temporal SDE Model

Original: Abstract
Revised: Abstract

Original: We propose a nonparametric spatio-temporal stochastic differential equation (SDE) model that can learn the underlying dynamics of arbitrary continuous-time systems without prior knowledge.
Revised: We propose a nonparametric spatio-temporal stochastic differential equation (SDE) model that can learn the underlying dynamics of arbitrary continuous-time systems without prior knowledge.

Original: We augment the input space of the drift function of an SDE with a temporal component to account for spatio-temporal patterns.
Revised: We augment the input space of the drift function of an SDE with a temporal component to account for spatio-temporal patterns.

Original: The experimentson a real world data set demonstrate that the spatio-temporal model is better able to ﬁt the data  than the spatial model and  also reduce the forecasting error
Revised: The experiments demonstrate that the spatio-temporal model is better able to ﬁt a real world data set that has complex dynamics than the spatial model, and can also reduce the forecasting error

Original:  Introduction
Revised:  Introduction

Original: Dynamical systems modeling is a cornerstone of experimental sciences.
Revised: Dynamical systems modeling is a cornerstone of experimental sciences.

Original: Modelers attempt to capture the dynamical behavior of a stochastic system or a phenomenon in order to improve its understanding and make predictions about its future state.
Revised: Modelers attempt to capture the dynamical behavior of a stochastic system or a phenomenon in order to improve its understanding and make predictions about its future state.

Original: Stochastic differential equations (SDEs) are an effective formalism for modelling systems with underlying stochastic dynamics, with wide range of applications [5].
Revised: Stochastic differential equations (SDEs) are an effective formalism for modelling systems with underlying stochastic dynamics, with wide range of applications [5].

Original: The key problem in SDE’s is estimation of the underlying deterministic driving function, and the stochastic diffusion component.
Revised: The key problem in SDEs is estimation of the underlying deterministic driving function, and the stochastic diffusion component.

Original: In this work, we are interested in a multivariate system governed by Markov process x t described by an SDE
Revised: In this work, we are interested in a multivariate system governed by Markov process x t described by an SDE

Original: [Equation]
Revised: [Equation]

Original: where x t ∈ R D is the state vector of a D -dimensional dynamical system at continuous time t ∈ R , f
Revised: where x t ∈ R D is the state vector of a D -dimensional dynamical system at continuous time t ∈ R , f

Original: ( x , t ) ∈
Revised: ( x , t ) ∈

Original: R D is a deterministic state evolution, σ ( x t , t ) ∈ R is a scalar magnitude of the stochastic multivariate Wiener process W t ∈ R D .
Revised: R D is a deterministic state evolution, σ ( x t , t ) ∈ R is a scalar magnitude of the stochastic multivariate Wiener process W t ∈ R D .

Original: The Wiener process has zero initial state W 0 = 0 , and the independent increments W t + s − W t ∼ N ( 0 , sI ) follow a Gaussian with standard deviation √ s .
Revised: The Wiener process has zero initial state W 0 = 0 , and the independent increments W t + s − W t ∼ N ( 0 , sI ) follow a Gaussian with standard deviation √ s .

Original: The state solutions of SDE are given by the Itô integral [9]
Revised: The state solutions of SDE are given by the Itô integral [9]

Original: [Equation]
Revised: [Equation]

Original: where we integrate the system state from an initial state x 0 for time t forward, and where τ is an auxiliary time variable.
Revised: where we integrate the system state from an initial state x 0 for time t forward, and where τ is an auxiliary time variable.

Original: We assume the states are observed with additive noise y ( t ) =
Revised: We assume the states are observed with additive noise y ( t ) =

Original: x ( t ) + ε t with ε t ∼ N ( 0 , Ω) and Ω = diag( ω 21 , . . .
Revised: x ( t ) + ε t with ε t ∼ N ( 0 , Ω) and Ω = diag( ω 21 , . . .

Original: , ω 2 D ) .
Revised: , ω 2 D ) .

Original: With σ ( x τ , τ ) = 0 , we obtain the ordinary differential equation (ODE) solution.
Revised: With σ ( x τ , τ ) = 0 , we obtain the ordinary differential equation (ODE) solution.

Original: There is a vast literature on inferring SDEs [5] where a parametric form for drift and diffusion functions are assumed to be known, and the parameters of those functions are optimized.
Revised: There is a vast literature on inferring SDEs [5] where a parametric form for drift and diffusion functions are assumed to be known, and the parameters of those functions are optimized.

Original: Such methods cannot be readily applied to real world data sets, where the underlying dynamics are unknown.
Revised: Such methods cannot be readily applied to real world data sets, where the underlying dynamics are unknown.

Original: To tackle such data sets, non-parametric drift and diffusion functions have been proposed in several works [16, 3, 6, 13].
Revised: To tackle such data sets, non-parametric drift and diffusion functions have been proposed in several works [16, 3, 6, 13].

Original: This work extends the method proposed in [16] to account for spatio-temporal patterns.
Revised: This work extends the method proposed in [16] to account for spatio-temporal patterns.

Original:  Nonparametric SDE Model
Revised:  Nonparametric SDE Model

Original: In this section, we review the sparse Gaussian process (GP) modeling of differential equations described in [16].
Revised: In this section, we review the sparse Gaussian process (GP) modeling of differential equations described in [16].

Original: The model introduces two GP priors over the vector valued drift f ( x ) : R D
Revised: The model introduces two GP priors over the vector valued drift f ( x ) : R D

Original: → R D and scalar valued diffusion σ ( x ) : R D
Revised: → R D and scalar valued diffusion σ ( x ) : R D

Original: → R functions
Revised: → R functions

Original: [Equation]
Revised: [Equation]

Original: Drift and diffusion functions have zero mean, the drift kernel K f ( x , x
Revised: Drift and diffusion functions have zero mean, the drift kernel K f ( x , x

Original: [Equation]
Revised: [Equation]

Original: ∈ R D
Revised: ∈ R D

Original: × D is matrix valued, and the diffusion kernel k σ ( x , x (cid:48) )
Revised: × D is matrix valued, and the diffusion kernel k σ ( x , x (cid:48) )

Original: ∈ R is univariate.
Revised: ∈ R is univariate.

Original: Covariance matrices are deﬁned using squared exponential kernel function
Revised: Covariance matrices are deﬁned using squared exponential kernel function

Original: [Equation]
Revised: [Equation]

Original: where θ f = { σ f , (cid:96) f 1 , . .
Revised: where θ f = { σ f , (cid:96) f 1 , . .

Original: , (cid:96) f D } and θ σ = { σ σ , (cid:96) σ 1 , . . .
Revised: , (cid:96) f D } and θ σ = { σ σ , (cid:96) σ 1 , . . .

Original: , (cid:96) σD } stand for the function-speciﬁc kernel parameters.
Revised: , (cid:96) σD } stand for the function-speciﬁc kernel parameters.

Original: Following [16], the identity decomposable kernel K f ( x , x (cid:48) )
Revised: Following [16], the identity decomposable kernel K f ( x , x (cid:48) )

Original: =
Revised: =

Original: k ( x , x (cid:48) ) · I D is used for the matrix valued drift kernel [2], whereas k σ ( x , x (cid:48) )
Revised: k ( x , x (cid:48) ) · I D is used for the matrix valued drift kernel [2], whereas k σ ( x , x (cid:48) )

Original: =
Revised: =

Original: k ( x , x (cid:48) ) .
Revised: k ( x , x (cid:48) ) .

Original: Given a set of states X = ( x 1 , . . .
Revised: Given a set of states X = ( x 1 , . . .

Original: , x N ) T ∈ R N × D , the values of the drift function F = ( f ( x 1 ) , . . .
Revised: , x N ) T ∈ R N × D , the values of the drift function F = ( f ( x 1 ) , . . .

Original: , f ( x N ))
Revised: , f ( x N ))

Original: T ∈ R N × D and the diffusion function σ = ( σ ( x 1 ) , . . .
Revised: T ∈ R N × D and the diffusion function σ = ( σ ( x 1 ) , . . .

Original: , σ ( x N ))
Revised: , σ ( x N ))

Original: T ∈ R N follow normal prior distributions
Revised: T ∈ R N follow normal prior distributions

Original: [Equation]
Revised: [Equation]

Original: where K f ( X, X )
Revised: where K f ( X, X )

Original: =
Revised: =

Original: ( K f ( x i , x j ))
Revised: ( K f ( x i , x j ))

Original: Ni,j =1 ∈ R ND × ND is a block matrix of matrix-valued kernels
Revised: Ni,j =1 ∈ R ND × ND is a block matrix of matrix-valued kernels

Original: K f ( x i , x j ) and diffusion kernel is given by K σ ( X, X ) =
Revised: K f ( x i , x j ) and diffusion kernel is given by K σ ( X, X ) =

Original: ( k σ ( x i , x j ))
Revised: ( k σ ( x i , x j ))

Original: Ni,j =1 ∈ R N × N .
Revised: Ni,j =1 ∈ R N × N .

Original: In standard GP regression, the posterior over the function values are obtained by conditioning the GP prior on the data [11].
Revised: In standard GP regression, the posterior over the function values are obtained by conditioning the GP prior on the data [11].

Original: On the other hand, in differential equation models, the conditionals f ( x ) | Y and σ ( x ) | Y are intractable due to the integral mapping between observed states and differentials.
Revised: On the other hand, in differential equation models, the conditionals f ( x ) | Y and σ ( x ) | Y are intractable due to the integral mapping between observed states and differentials.

Original: To overcome this, two sets of inducing variables
Revised: To overcome this, two sets of inducing variables

Original: U f
Revised: U f

Original: = ( u f 1 , . .
Revised: = ( u f 1 , . .

Original: u f M ) T ∈ R M × D and u σ = ( u σ 1 , . .
Revised: u f M ) T ∈ R M × D and u σ = ( u σ 1 , . .

Original: . , u σM ) T ∈ R M are introduced [10].
Revised: . , u σM ) T ∈ R M are introduced [10].

Original: U f and u σ contain the values of the drift and diffusion functions at inducing locations Z
Revised: U f and u σ contain the values of the drift and diffusion functions at inducing locations Z

Original: = ( z 1 , . . .
Revised: = ( z 1 , . . .

Original: , z M ) , which live in the same space as the states X .
Revised: , z M ) , which live in the same space as the states X .

Original: Finally, the drift and diffusion functions are interpolated from the locations and variables:
Revised: Finally, the drift and diffusion functions are interpolated from the locations and variables:

Original: [Equation]
Revised: [Equation]

Original: where u f = vec U f .
Revised: where u f = vec U f .

Original: The SDE model is determined via the inducing locations
Revised: The SDE model is determined via the inducing locations

Original: Z , the inducing values
Revised: Z , the inducing values

Original: U f and u σ , the observation noise variance Ω , and the kernel parameters θ f
Revised: U f and u σ , the observation noise variance Ω , and the kernel parameters θ f

Original: and θ σ of the drift and diffusion kernels.
Revised: and θ σ of the drift and diffusion kernels.

Original: The model learns the underlying system to induce state distributions with high expected likelihood p ( y i | f , σ, Ω) =
Revised: The model learns the underlying system to induce state distributions with high expected likelihood p ( y i | f , σ, Ω) =

Original: E p ( x | t i ; f ,σ ) [ N ( y i | x , Ω)] , which is intractable.
Revised: E p ( x | t i ; f ,σ ) [ N ( y i | x , Ω)] , which is intractable.

Original: The posterior of the model combines the likelihood p ( y i | f , σ, Ω) and the independent priors p ( U f ) and p ( u σ ) using Bayes’ theorem as
Revised: The posterior of the model combines the likelihood p ( y i | f , σ, Ω) and the independent priors p ( U f ) and p ( u σ ) using Bayes’ theorem as

Original: [Equation]
Revised: [Equation]

Original: where x ( s ) ∼ p ( x 0 ...t | U f , u σ , Z ) denotes a path sample x ( s ) t that is drawn from the time dependent state distribution p ( x 0 ...t | U f , u σ , Z ) by sampling a Brownian motion path W ( s ) t .
Revised: where x ( s ) ∼ p ( x 0 ...t | U f , u σ , Z ) denotes a path sample x ( s ) t that is drawn from the time dependent state distribution p ( x 0 ...t | U f , u σ , Z ) by sampling a Brownian motion path W ( s ) t .

Original: The true expected likelihood is approximated by unbiased Monte Carlo averaging.
Revised: The true expected likelihood is approximated by unbiased Monte Carlo averaging.

Original: The likelihood estimate with N s samples turns out to be a kernel density estimator with Gaussian bases.
Revised: The likelihood estimate with N s samples turns out to be a kernel density estimator with Gaussian bases.

Original: We draw the sample paths using Euler-Maruyama (EM) [9]:
Revised: We draw the sample paths using Euler-Maruyama (EM) [9]:

Original: [Equation]
Revised: [Equation]

Original: where we discretise time into N T subintervals t 0 , t 1 , . .
Revised: where we discretise time into N T subintervals t 0 , t 1 , . .

Original: , t N T of width ∆ t = t N T /N T , and sample the Wiener coefﬁcients as ∆ W ( s ) i ∼ N ( 0 , ∆ t · I ) with standard deviation √ ∆ t .
Revised: , t N T of width ∆ t = t N T /N T , and sample the Wiener coefﬁcients as ∆ W ( s ) i ∼ N ( 0 , ∆ t · I ) with standard deviation √ ∆ t .

Original: We set x ( s ) 0 to the initial observation and use (9) to compute state path x
Revised: We set x ( s ) 0 to the initial observation and use (9) to compute state path x

Original: ( s ) ≡ ( x ( s ) 0 , x ( s ) 1 , . . .
Revised: ( s ) ≡ ( x ( s ) 0 , x ( s ) 1 , . . .

Original: , x ( s ) T N ) .
Revised: , x ( s ) T N ) .

Original: The number of time steps N T > N is often higher than the number of observed timepoints to achieve sufﬁcient path resolution.
Revised: The number of time steps N T > N is often higher than the number of observed timepoints to achieve sufﬁcient path resolution.

Original: Finally, the maximum a posteriori (MAP) estimate of the model parameters are found using sensitivity equation based gradients (see [16] for details).
Revised: Finally, the maximum a posteriori (MAP) estimate of the model parameters are found using sensitivity equation based gradients (see [16] for details).

Original:  Spatio-Temporal SDE Model
Revised:  Spatio-Temporal SDE Model

Original: So far, the formulation relies on SDE states x ( t ) and the inducing locations living in the same space, R D , which remains valid as long as the input to the drift function is only the state.
Revised: So far, the formulation relies on SDE states x ( t ) and the inducing locations living in the same space, R D , which remains valid as long as the input to the drift function is only the state.

Original: In order to deﬁne drift functions explicitly parameterized by time, we augment the space to include a temporal component, that is, f ( x , t ) :
Revised: In order to deﬁne drift functions explicitly parameterized by time, we augment the space to include a temporal component, that is, f ( x , t ) :

Original: R D + → R D where R D + = ( R D × R + ) .
Revised: R D + → R D where R D + = ( R D × R + ) .

Original: The GP prior over the drift function then becomes
Revised: The GP prior over the drift function then becomes

Original: [Equation]
Revised: [Equation]

Original: The matrix valued kernel is again deﬁned to be the identity decomposable kernel K f (
Revised: The matrix valued kernel is again deﬁned to be the identity decomposable kernel K f (

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: t ) ,
Revised: t ) ,

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: , t
Revised: , t

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: k (( x , t ) , ( x (cid:48) , t (cid:48) )) · I D , where we redeﬁne the kernel function of the drift as
Revised: k (( x , t ) , ( x (cid:48) , t (cid:48) )) · I D , where we redeﬁne the kernel function of the drift as

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: New kernel function turns out to be the product of squared exponential kernel functions over space and time and requires one additional parameter (cid:96) t modeling the smoothness in temporal domain.
Revised: New kernel function turns out to be the product of squared exponential kernel functions over space and time and requires one additional parameter (cid:96) t modeling the smoothness in temporal domain.

Original: In the limit (cid:96) t → ∞ , we recover the spatial model.
Revised: In the limit (cid:96) t → ∞ , we recover the spatial model.

Original: Because the kernel function is deﬁned in the extended space, inducing locations Z must have an additional component: z i ∈ R D + .
Revised: Because the kernel function is deﬁned in the extended space, inducing locations Z must have an additional component: z i ∈ R D + .

Original: Time variant diffusion functions can also be deﬁned in a similar way, which we do not investigate in this work.
Revised: Time variant diffusion functions can also be deﬁned in a similar way, which we do not investigate in this work.

Original: Spatio-temporal drift function boosts the model power and ﬂexibility without any additional parameter other than temporal lengthscale.
Revised: Spatio-temporal drift function boosts the model power and ﬂexibility without any additional parameter other than temporal lengthscale.

Original: In spatial systems, state trajectory follows the same dynamics (up to random perturbations due to diffusion) when a particular state is visited at different points in time.
Revised: In spatial systems, state trajectory follows the same dynamics (up to random perturbations due to diffusion) when a particular state is visited at different points in time.

Original: With the proposed spatio-temporal formulation, temporal proximity is also taken into account when deﬁning state dynamics, meaning that different representations can be learned over time.
Revised: With the proposed spatio-temporal formulation, temporal proximity is also taken into account when deﬁning state dynamics, meaning that different representations can be learned over time.

Original: The model is implemented in Tensorﬂow [1].
Revised: The model is implemented in Tensorﬂow [1].

Original: We ﬁnd the MAP estimates of the inducing variables
Revised: We ﬁnd the MAP estimates of the inducing variables

Original: U f , u σ and the noise variance Ω using the Adam [8] optimizer with learning rate 0.001 and staircase (every 20th iteration) exponential decay with rate 0.99.
Revised: U f , u σ and the noise variance Ω using the Adam [8] optimizer with learning rate 0.001 and staircase (every 20th iteration) exponential decay with rate 0.99.

Original: Kernel parameters in complex GP models are usually hard to optimize since they appear in the covariance matrix.
Revised: Kernel parameters in complex GP models are usually hard to optimize since they appear in the covariance matrix.

Original: In our case, the covariance matrix itself is connected to the likelihood through an integral, further complicating the optimization.
Revised: In our case, the covariance matrix itself is connected to the likelihood through an integral, further complicating the optimization.

Original: Therefore, we perform a grid search to optimize lengthscales { (cid:96) f 1 . . .
Revised: Therefore, we perform a grid search to optimize lengthscales { (cid:96) f 1 . . .

Original: (cid:96) f D , (cid:96) t , (cid:96) σ 1 . . .
Revised: (cid:96) f D , (cid:96) t , (cid:96) σ 1 . . .

Original: (cid:96) σD } , and set the signal variance terms to one, σ f = σ g = 1 .
Revised: (cid:96) σD } , and set the signal variance terms to one, σ f = σ g = 1 .

Original: The inducing points are scattered around the data and kept ﬁxed during optimization.
Revised: The inducing points are scattered around the data and kept ﬁxed during optimization.

Original:  Experiments
Revised:  Experiments

Original: The experiments aim at demonstrating that the spatio-temporal model is better able to ﬁt the data than the spatial model, and also better at capturing the data generating dynamics.
Revised: The experiments aim at demonstrating that the spatio-temporal model is better able to ﬁt the data than the spatial model, and also better at capturing the data generating dynamics.

Original: The experiments are performed on a benchmark dataset of human motion capture data from the Carnegie Mellon
Revised: The experiments are performed on a benchmark dataset of human motion capture data from the Carnegie Mellon

Original: University motion capture (CMU mocap) database.
Revised: University motion capture (CMU mocap) database.

Original: We followed the data preprocessing technique in [14], which results in 50 -dimensional pose measurements from a person swinging a golf club where each pose dimension records a measurement in different parts of the body during the action 1 .
Revised: We followed the data preprocessing technique in [14], which results in 50 -dimensional pose measurements from a person swinging a golf club where each pose dimension records a measurement in different parts of the body during the action 1 .

Original: We consider four input sequences, each having 350 data points to train the model.
Revised: We consider four input sequences, each having 350 data points to train the model.

Original: In order to tackle the problem of dimensionality, we project the original dataset with PCA to a three dimensional latent space where the system is speciﬁed, following [4] and [15].
Revised: In order to tackle the problem of dimensionality, we project the original dataset with PCA to a three dimensional latent space where the system is speciﬁed, following [4] and [15].

Original: We also investigated whether incorporating temporal information into the drift function enhances non-parametric ordinary differential equation (ODE) model presented in [7], which is tested by discarding the diffusion component from (1) and ﬁtting the same input sequences.
Revised: We also investigated whether incorporating temporal information into the drift function enhances non-parametric ordinary differential equation (ODE) model presented in [7], which is tested by discarding the diffusion component from (1) and ﬁtting the same input sequences.

Original: Overall four models are examined, which we call in short sODE, stODE, sSDE and stSDE, lower case letters representing if the system is spatial or spatio-temporal.
Revised: Overall four models are examined, which we call in short sODE, stODE, sSDE and stSDE, lower case letters representing if the system is spatial or spatio-temporal.

Original: Each model is optimized 750 times using different lengthscale and inducing locations.
Revised: Each model is optimized 750 times using different lengthscales and inducing locations.

Original: In order to select the hyperparameters that prevent overﬁtting, we compute the root mean square error (RMSE) between 3 holdout input sequences and state paths (at observed time points) that are estimated using the EM integration from the measured initial states.
Revised: In order to select the hyperparameters that prevent overﬁtting, we compute the root mean square error (RMSE) between 3 holdout input sequences and state paths (at observed time points) that are estimated using the EM integration from the measured initial states.

Original: The models are then retrained using 4 training and 3 holdout sequences, and ﬁnally
Revised: The models are then retrained using 4 training and 3 holdout sequences, and ﬁnally

Original: RMSE between 3 test input sequences and the estimated state paths are reported.
Revised: RMSE between 3 test input sequences and the estimated state paths are reported.

Original: Figure 1 visualizes ﬁts for four models on the same training data.
Revised: Figure 1 visualizes ﬁts for four models on the same training data.

Original: In both ODE and SDE settings, we see that inferred spatio-temporal trajectories are closer to the data than the spatial counterparts.
Revised: In both ODE and SDE settings, we see that inferred spatio-temporal trajectories are closer to the data than the spatial counterparts.

Original: For the same reason, random state trajectory cloud needs to be more voluminous in sSDE to achieve a greater posterior whereas stSDE can  ﬁt individual trajectories.
Revised: For the same reason, stochastic state trajectory cloud needs to be more voluminous in sSDE to achieve a greater posterior whereas stSDE can better ﬁt individual trajectories.

Original: Also, the bottom left corners in the ﬁgures show that temporal information helps capturing rapidly changing spatial ﬁelds.
Revised: Also, the bottom left corners in the ﬁgures show that temporal information helps capturing rapidly changing spatial ﬁelds.

Original: Quantitative results are presented in Figure 2.
Revised: Quantitative results are presented in Figure 2.

Original: Unsurprisingly, time-variant drift functions boost the training performance with more than 10 inducing points, which is compatible with the plots in Figure 1.
Revised: Unsurprisingly, time-variant drift functions boost the training performance with more than 10 inducing points, which is compatible with the plots in Figure 1.

Original: We also observe that increasing the number of inducing points yields better training posteriors in
Revised: We also observe that increasing the number of inducing points yields better training posteriors in

Original: SDE model.
Revised: SDE model.

Original: Figures 2c-dclearly illustrate that time-variant drift function leads to reduced prediction error, which hints that the dynamics underlying a golf swing motion are learned better.
Revised: Figures 2c-d illustrate that time-variant drift function can also reduce prediction error, at least for golf swing trajectories that span approximately the same part of the state space.

Original: We also see that the error consistently decreases as the number of inducing points M is increased, and reaches the minimum at M = 80 .
Revised: We also see that the error consistently decreases as the number of inducing points M is increased, and reaches the minimum at M = 80 .

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: We propose an approach for learning non-parametric spatio-temporal drift and diffusion functions of stochastic differential equation (SDE) systems such that the resulting simulated state distributions match data.
Revised: We propose an approach for learning non-parametric spatio-temporal drift and diffusion functions of stochastic differential equation (SDE) systems such that the resulting simulated state distributions match data.

Original: The experiment on a real world data set shows that our model can better ﬁt complex dynamics than the spatial counterpart.
Revised: The experiment on a real world data set shows that our model can better ﬁt complex dynamics than the spatial counterpart.

Revised: This increase in model capacity, however, results in larger data set requirements and makes the model more vulnerable to overﬁtting, which could be better accounted for using e.g. variational inference.

Original: An interesting future research direction is the study of various vector ﬁeld kernels, such as divergence-free, curl-free or spectral kernels [12].
Revised: An interesting future research direction is the study of various vector ﬁeld kernels, such as divergence-free, curl-free or spectral kernels [12].

Original: The model could be extended to have an observation model, e.g., GPLVM or deep neural network, rather than PCA.
Revised: The model could be extended to have an observation model, e.g., GPLVM or deep neural network, rather than PCA.

Original: Including inputs or controls to the system would allow precise modelling in interactive settings, such as robotics.
Revised: Including inputs or controls to the system would allow precise modelling in interactive settings, such as robotics.

Original: Acknowledgments
Revised: Acknowledgments

Original: The data used in this project was obtained from mocap.cs.cmu.edu .
Revised: The data used in this project was obtained from mocap.cs.cmu.edu .

Original: The database was created with funding from NSF EIA-0196217.
Revised: The database was created with funding from NSF EIA-0196217.

Original: This work has been supported by the Academy of Finland grants no. 260403, 299915, 275537,
Revised: This work has been supported by the Academy of Finland grants no. 260403, 299915, 275537,