Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This is a strait forward paper that aims to address the lack of scalability of prior methods on system  identification.  The idea is simple yet the assumptions required are significant and restrict the applicability of the proposed algorithm to systems that are closed to deterministic.  Below are some pros and cons for this paper:

Cons:
(+) The proposed method is simple and it could be  quickly adopted by robotics researchers and practitioners
(+) The method seems to outperform previous algorithms

Pros:
(-) The idea of working with deterministic systems is restrictive.  While the process noise may appear in some robotics applications to be small  I see no way of how one could reduce the effect of the observation noise.    Also in the context of systems that navigate in terrestrial environments  the process noise is definitely more present that in the case of flying vehicles. 

(-) It would definitely strengthen the paper if the authors could elaborate more on the concepts of controllability, observability and how these concepts relate to their method etc. How does the existing methods handles these properties? How is the performance of the method affected  when the true system is not controllable but it is stabilizable?


(-) There seems to be no assumption on the shape of the trajectories. What kind of trajectories you need to perform system identification? Can you mathematically represent these assumptions? In the adaptive control literature there exists conditions related to "sufficient rich signals" see the book by Ioannou on Robust Adaptive Control. These are signals that can excite dynamics so that to ensure proper identification.  




review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper proposes a novel way to address the well studied system identification problem. More specifically, this paper focuses on non linear systems with partial observations. To this aim, the authors propose the "System-indetification via Iterative Smoothing and Learning" (SISL) algorithm. The gist of the approach is to formulate an ML problem to estimate the system model parameters from a parametric family of models to estimate both the dynamical system evolution (state updates) and the observation process. The exact optimization for this ML problem is not tractable. This work proposes to solve a surrogate point-estimate  ML problem. This optimization is solve using an alternate approach (between smoothing and learning). 

The paper also presents experimental results both on synthetic  and real data aiming to demonstrate that SISL provides unbiased estimates in deterministic settings. The synthetic setting is also used to motivate the need for methods able to handle large system dimensionality for non-linear and partially observable systems. 
 
The real data setting compares favorably the SISL approach with well established baselines. 

I believe that the paper can be improved in the following ways: 
1) There are intermediate steps in the derivation of the maths that could be better explained. For example, in equation 2, the “Markovian” aspect of the system is key to derive the formula (x_{t+1} only depends on x_{t}). Equation 4 is not really derived from Equation 3. The factorization shown in Equation 2 is relevant for Equation 4 although the 4) is for the surrogate objective function. 

Also, I did not understand why the naive approach is not plotted on Figure 3 (a and b) (but it comes back in the appendix).  Is it just an oversight?
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: 

#######################
Rebuttal Response:
Thanks for these clarifications and updating the paper. I adapted my score to weak accept. To increase my score to accept, I would like to see evaluations of control experiments rather than just comparing the MSE. 

#######################
Review: 

Summary: 
The paper introduces an iterative learning scheme to perform system identification on partially observable systems. The proposed approach iteratively optimizes the unobserved state and the model parameters. The proposed approach is demonstrated on a 'high'-dimensional Lorenz oscillator and existing quadcopter data. Furthermore, the authors show that existing approaches don't scale to 'high' dimensions. 

Conclusion;
The paper introduces a simple iterative scheme to learn a model with a hidden state. The approach is sound and reasonable and the paper is well written but I am uncertain whether this paper fits the scope of ICLR. Maybe R:SS would be a better fit? Furthermore, the experiments can be improved, which is especially important as this paper does not propose a fundamental new approach. It would be really interesting to see this model controlling an actual quadcopter. My current vote is borderline reject but can be improved by further highlighting why quadcopter dynamics are only partially obsevable and further experiments.

Experiments:
My main concern with the paper is the experiments of more complex systems. The initial experiments demonstrating that the approach works and previous methods don't scale to high dimensions are very good and highlight important points. However, the more complex experiments might be not indicative of the performance. Especially for the quadcopter experiments, the proposed method only learns a non-linear noise model while the state transition is still linear and compares it against the purely linear case. Both methods seem qualitatively comparable and the quantitative improvement is only marginal. Therefore, I am doubtful whether the quadcopter data really requires to incorporate the partially observable model and I would prefer to see an evaluation of the proposed iterative scheme using a linear model. Within your evaluation you should compare the different optimization approaches (i.e., naive vs iterative) using the same model representation as your proposed algorithm only proposes a new optimization scheme but is not fixed to a representation. Furthermore, it would be interesting to see a non-linear state model and a non-linear observation model. Does the iterative approach also learn this model? Learning the robot dynamics model with contacts would be the best experiment to validate your approach, as this is only partially observable using the joint measurements and is non-linear. Could you setup an experiment with a simulated two-link robot, generate data containing samples with & without contact and learn the model just using q, qd, qdd, and tau. This experiment would clearly highlight that your iterative scheme can identify a partially observable model.  

Minor Comments:
"We let ̃yt=yt−NNθn(ut)be the prediction errors of the trained naive model." -> it is a bit unclear whether the linear model approximates just the noise or the complete model? From the following text I assume that the complete model is modelled as linear model. Then the introduction of the naive model seems unnecessary as the naive model is not used. Could you please clarify this point?
review_title: Official Blind Review #2

Meta-review: The paper is about nonlinear system identification in an EM-style learning framework. The idea is to use nonlinear programming for the E step (finding a MAP estimate) and then refine the model parameters. In flavor, this approach is similar to the work by Roweis and Ghahramani. 

However, this paper does not offer any new insights whatsoever and the (very short) methods section arrives at proposing to compute the maximum a posteriori estimate (eq. 5). While the motivation for this given in the paper is a bit hard to understand it is of course a very well-known and useful estimator. Besides the maximum likelihood estimator this is one of the most commonly used point estimators, see any textbook on statistical signal processing. There has been quite a bit of work in the signal processing community over the last 10 years, and a good overview can be found here:
https://web.stanford.edu/~boyd/papers/pdf/rt_cvx_sig_proc.pdf
This should give evidence that this is indeed a standard way of solving the problem and it does work really well. Given that we have so fast and good optimizers these days it is common to solve Kalman filtering/smoothing problems via this optimization problem.
The paper does not contain any analysis at all. The experiments do of course show that the method works (when there is low noise). Again, we know very well that the MAP estimate is a decent estimator for unimodal problems. The MAP estimator can also be made to work well for noisy situations.

As for the comments that the sequential Monte Carlo methods do not work in higher dimensions that is indeed true. However, there are now algorithms that work in much higher dimensions than those considered by the authors of this paper, e.g.
https://ieeexplore.ieee.org/document/8752074
which also contains an up-to-date survey on the topic. Furthermore, when it comes to particle smoothing there are also much more efficient smoothers than 10 years ago. The area of particle smoothing has also evolved rapidly over the past years. 

Summary:
The paper makes use of the well-known MAP estimator for learning nonlinear dynamical systems (states and parameters). This is by now a standard technique in signal processing. There are several throw-away comments on SMC that are not valid and that are not grounded in the intense research of that field over the past decade.

Decision: Reject