Paper:
Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC
State-space models are successfully used in many areas of science, engineering and economics to model time series and dynamical systems. We present a fully Bayesian approach to inference and learning (i.e. state estimation and system identification) in nonlinear nonparametric state-space models. We place a Gaussian process prior over the state transition dynamics, resulting in a flexible model able to capture complex dynamical phenomena. To enable efficient inference, we marginalize over the transition dynamics function and, instead, infer directly the joint smoothing distribution using specially tailored Particle Markov Chain Monte Carlo samplers. Once a sample from the smoothing distribution is computed, the state transition predictive distribution can be formulated analytically. Our approach preserves the full nonparametric expressivity of the model and can make use of sparse Gaussian processes to greatly reduce computational complexity.

Review:
This paper presents a Bayesian approach to state and 
      parameter estimation in nonlinear state-space models, while also learning 
      the transition dynamics through the use of a Gaussian process (GP) prior. 
      The inference mechanism is based on particle Markov chain Monte Carlo 
      (PMCMC) with the recently-introduced idea of ancestor sampling. The paper 
      also discusses computational efficiencies to be had with respect to 
      sparsity and low-rank Cholesky updates. This is a technically 
      sound and strong paper with clear and accessible presentation. The online 
      marginalisation of the transition dynamics and the use of ancestor 
      sampling to achieve this is novel. The consideration of computational 
      issues such as sparsity and low-rank updates/downdates to the Cholesky 
      factors of covariance matrices strengthens the paper further. The 
      empirical results, while brief, are sufficient (further suggestions 
      below). In addition to its stated aims, the paper will likely 
      stimulate discussion around inference methods for non-Markovian 
      state-space models and the potential advantages/disadvantages of learning 
      the transition dynamics in this way rather than specifying a parametric 
      model a priori. While space is slight, the authors may like to 
      consider some further discussion around the differences between using a 
      parametric transition model given a priori against the use of a similar 
      model as the mean function of the GP. For example in out of sample 
      prediction (e.g. forecasting). The results in Table 1 and the 
      description in the preceding paragraph are slightly unclear to me. I am 
      unsure as to whether the RMSE is against a withheld set of data points or 
      the same set of data points that is conditioned upon (the *|data in the 
      column headings). My main interest would be an RMSE against an 
      out-of-sample prediction, especially a forecast forward in time against 
      withheld data. It is in this scenario that I would expect to see the 
      largest differences between the learnt dynamics and the ground truth 
      model. If Table 1 is not already showing this, an extra column that does 
      so would be a great addition. One minor point: the abbreviation 
      CPF-AS is used in Algorithm 1 before being defined in the first paragraph 
      of Section 3.3.1 below.