Paper:
System identification is the process of building a mathematical model of an unknown system from measurements of its inputs and outputs. It is a key step for model-based control, estimator design, and output prediction. This work presents an algorithm for non-linear offline system identification from partial observations, i.e. situations in which the system's full-state is not directly observable. The algorithm presented, called SISL, iteratively infers the system's full state through nonlinear optimization and then updates the model parameters. We test our algorithm on a simulated system of coupled Lorenz attractors, showing our algorithm's ability to identify high-dimensional systems that prove intractable for particle-based approaches. We also use SISL to identify the dynamics of an aerobatic helicopter. By augmenting the state with unobserved fluid states, we learn a model that predicts the acceleration of the helicopter better than state-of-the-art approaches.

Review:
#######################
Rebuttal Response:
Thanks for these clarifications and updating the paper. I adapted my score to weak accept. To increase my score to accept, I would like to see evaluations of control experiments rather than just comparing the MSE. 
#######################
Review: 
Summary: 
The paper introduces an iterative learning scheme to perform system identification on partially observable systems. The proposed approach iteratively optimizes the unobserved state and the model parameters. The proposed approach is demonstrated on a 'high'-dimensional Lorenz oscillator and existing quadcopter data. Furthermore, the authors show that existing approaches don't scale to 'high' dimensions. 
Conclusion;
The paper introduces a simple iterative scheme to learn a model with a hidden state. The approach is sound and reasonable and the paper is well written but I am uncertain whether this paper fits the scope of ICLR. Maybe R:SS would be a better fit? Furthermore, the experiments can be improved, which is especially important as this paper does not propose a fundamental new approach. It would be really interesting to see this model controlling an actual quadcopter. My current vote is borderline reject but can be improved by further highlighting why quadcopter dynamics are only partially obsevable and further experiments.
Experiments:
My main concern with the paper is the experiments of more complex systems. The initial experiments demonstrating that the approach works and previous methods don't scale to high dimensions are very good and highlight important points. However, the more complex experiments might be not indicative of the performance. Especially for the quadcopter experiments, the proposed method only learns a non-linear noise model while the state transition is still linear and compares it against the purely linear case. Both methods seem qualitatively comparable and the quantitative improvement is only marginal. Therefore, I am doubtful whether the quadcopter data really requires to incorporate the partially observable model and I would prefer to see an evaluation of the proposed iterative scheme using a linear model. Within your evaluation you should compare the different optimization approaches (i.e., naive vs iterative) using the same model representation as your proposed algorithm only proposes a new optimization scheme but is not fixed to a representation. Furthermore, it would be interesting to see a non-linear state model and a non-linear observation model. Does the iterative approach also learn this model? Learning the robot dynamics model with contacts would be the best experiment to validate your approach, as this is only partially observable using the joint measurements and is non-linear. Could you setup an experiment with a simulated two-link robot, generate data containing samples with & without contact and learn the model just using q, qd, qdd, and tau. This experiment would clearly highlight that your iterative scheme can identify a partially observable model.  
Minor Comments:
"We let ̃yt=yt−NNθn(ut)be the prediction errors of the trained naive model." -> it is a bit unclear whether the linear model approximates just the noise or the complete model? From the following text I assume that the complete model is modelled as linear model. Then the introduction of the naive model seems unnecessary as the naive model is not used. Could you please clarify this point?