Paper:
Partial differential equations (PDEs) play a prominent role in many disciplines such as applied mathematics, physics, chemistry, material science, computer science, etc. PDEs are commonly derived based on physical laws or empirical observations. However, the governing equations for many complex systems in modern applications are still not fully known. With the rapid development of sensors, computational power, and data storage in the past decade, huge quantities of data can be easily collected and efficiently stored. Such vast quantity of data offers new opportunities for data-driven discovery of hidden physical laws. Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models. The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses. Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses. A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network. These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory). We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet). Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.

Review:
Authors propose a neural network based algorithm for learning from data that arises from dynamical systems with governing equations that can be written as partial differential equations.  The network architecture is constrained such that regardless of the parameters, it always implements discretization of an arbitrary PDE.  Through learning, the network adapts itself to solve a specific PDE.  Discretization is finite difference in space and forward Euler in time.  
The article is quite novel in my opinion.  To the best of my knowledge, it is the first article that implements a generic method for learning arbitrary PDE models from data.  In using networks, the method differs from previously proposed approaches for learning PDEs.  Experiments are only presented with synthetic data but given the potential for the method and its novelty, I believe this can be accepted.  However, it would have been a stronger article if authors have applied to a real life model with real initial and boundary conditions, and real observations. 
I have three main criticism about the article: 
 1.   Authors do not cite Chen and Pock’s article on learning diffusion filters with networks that first published in CVPR 2015 and then authors published a PAMI article this year.  To the best of my knowledge, they are the first to show the connection between res-net type architecture and numerical solutions of PDEs. I think proper credit should be given. [I need to emphasize that I am not an author in that article.] 
 2.   Authors emphasize the importance of interpretability, however, the constraint on the moment matrices might cripple this aspect.  The frozen filters have clear interpretations. They are in the end finite difference approximations with some level of accuracy depending on the size and the number of zeros.  When M(q) matrix is free to change, it is unclear what the effect will be on the filters.  Are the numbers that replace stars in Equation 6 for instance, will be absorbed in the O(\epsilon) term?  Can one really interpret the final c_{ij} for filters whose M(q) have many non-zeros? 
 3.   The introduction and results sections are well written.  The method section on the other hand, needs improvement.  The notation is not easy to follow due to missing definitions.  I believe with proper definitions — amounting to small modifications — readability of the article can substantially improve. 
In addition to the main criticisms, I have some other questions and concerns: 
  1.  How sensitive is the model?  In real life, one cannot expect to get observations every delta t.  Data is most often very sparse.  Can the model learn in that regime?  Can it differentiate between different PDEs and find the correct one with sparse data? 
  2.  The average operations decreases the interpretability of the proposed model as a PDE.  Depending on the filter size, D_{0}u can deviate from u, which should be the term that should be used in the residual block.  Why do authors need this?  How does the model behave without it? 
  3.  The statement “Thus, the PDE-Net with bigger n owns a longer time stability.” is a very vague statement.  I understand with larger n, training would be easier since more data would be used to estimate parameters.  However, it is not clear how this relates to “time stability”, which is also not defined in the article. 
 4.  How is the relative error computed?  Values in relative error plots goes as high as 10^2.  That would be a huge error if it is relative.