user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Scaling up Deep Learning for PDE-based Models

Paper content:
Across numerous applications, forecasting relies on numerical solvers for partial differential equations (PDEs).
Although the use of deep-learning techniques has been proposed, the uses have been restricted by the fact the training data are obtained using PDE solvers.
Thereby, the uses were limited to domains, where the PDE solver was applicable, but no further.
We present methods for training on small domains, while applying the trained models on larger domains, with consistency constraints ensuring the solutions are physically meaningful even at the boundary of the small domains.
We demonstrate the results on an air-pollution forecasting model for Dublin, Ireland.
Solving partial differential equations (PDEs) underlies much of applied mathematics and engineering, ranging from computer graphics and financial pricing, to civil engineering and weather prediction.
Conventional approaches to prediction in PDE models rely on numerical solvers and require substantial computing resources in the model-application phase.
While in some application domains, such as structural engineering, the longer run-times may be acceptable, in domains with rapid decay of value of the prediction, such as weather forecasting, the run-time of the solver is of paramount importance.In many such applications, the ability to generate large volumes of data facilitates the use of surrogate or reduced-order models BID3 obtained using deep artificial neural networks BID11 .
Although the observation that artificial neural networks could be applied to physical models is not new BID14 BID15 BID16 BID25 BID9 BID20 BID26 BID16 BID27 , and indeed, it is seen as one of the key trends BID2 BID13 BID31 on the interface of applied mathematics, data science, and deep learning, their applications did not reach the level of success observed in the field of the image classification, speech recognition, machine translation, and other problems processing unstructured high-dimensional data, yet.
A key issue faced by applications of deep-learning techniques to physical models is their scalability.Even very recent research on deep-learning for physical models BID32 BID12 BID34 ) uses a solver for PDEs to obtain hundreds of thousands of outputs.
The deep learning can then be seen as means of non-linear regression between the inputs and outputs.
For example, BID12 have recently observed a factor of 12,000 computational speedup compared to that of a leading solver for the PDE, on the largest domain they were able to work with.
Considering the PDE solver is used to generate the outputs to train the deep-learning model on, however, the deep-learning model is limited to the domain and application that it is trained on.We present methods for training Deep Neural Networks (DNNs) on small domains, while applying the trained models on larger domains, with consistency constraints ensuring the solutions are physically meaningful even at the boundaries of the small domains.
Our contributions are as follows:• definition of the consistency constraints, wherein the output for one (tile of a) mesh is used to constrain the output for another (tile of a) mesh.•
methods for applying the consistency constraints within the training of a DNN, which allows for an increase in the extent of the spatial domain by concatenating the outputs of several PDE-based models by considering boundary conditions and state at the boundary.•
a numerical study of the approach on a pollution-forecasting problem, wherein we lose accuracy from 1 to 7 per cent compared to the unconstrained model, but remove boundary artefacts.We note that the methods can be applied both in terms of "patching" multiple (tiles of a) meshes, and in terms of "zooming" in multi-resolution approaches, where lower-resolution (e.g., city-, countryscale) component constrains higher-resolution components (e.g., district-, city-scale), which in turn impose consistency constraints on the former.
We have presented consistency constraints, which make it possible to train DNN on small domains and apply the trained models to larger domains while allowing incorporation of information external to the domain.
The consistency constraints will ensure the solutions are physically meaningful even at the boundary of the small domains in the output of the DNN.
We have demonstrated promising results on an air-pollution forecasting model for Dublin, Ireland.The work is a first that makes possible numerous extensions.
First, one could consider further applications of the consistency constraints, e.g., in energy conservation, or in consider merging the outputs of a number of PDE models within multi-physics applications.
Second, in some applications, in may be useful to explore other network topologies.
Following BID34 , one could use long short-term memory (LSTM) units.
Further, over-fitting control could be based on an improved stacked auto-encoder architecture BID35 .
In interpretation of the trained model, the approach of BID7 may be applicable.Our work could also be seen as an example of Geometric Deep Learning BID5 , especially in conjunction with the use of mesh-free methods BID28 , such as the 3D point clouds BID23 , non-uniform meshing, or non-uniform choice of receptors within the meshes.
Especially for applications, where the grids are in 3D or higher dimensions, the need for such techniques is clear.
More generally, one could explore links to isogeometric analysis of BID6 , which integrates solving PDEs with geometric modelling.Finally, one could generalise our methods in a number of directions of the multi-fidelity modelling, e.g., by combining the reduced-order and full-order models using adaptation, fusion, or filtering.
Overall, the scaling up of deep learning for PDE-based models seems to be a particular fruitful area for further research.Within the domain of our example application, recent surveys BID2 suggest that ours is the first use of deep learning in the forecasting of air pollution levels.
Following the copious literature on PDE-based models of air pollution, one could consider further pollutants such as ground-level ozone concentrations BID18 , and ensemble BID17 or multi-fidelity methods.
One may also consider a joint model, allowing for traffic forecasting, weather forecasting, and air pollution forecasting, within the same network, possibly using LSTM units BID7 , at the same time.

assistant: text: We present RNNs for training surrogate models of PDEs, wherein consistency constraints ensure the solutions are physically meaningful, even when the training uses much smaller domains than the trained model is applied to.