An Efficient Bayesian Method for Advancing the Application of Deep Learning in Earth Science Despite their wide and successful applications, deep learning (DL) models are prone to overfitting for small training datasets, produce a poor predictive performance for uncertain data, and provide point estimations without any indication of the accuracy and credibility. These limitations of the deterministic DL models hinder their effective application in Earth system science where the labelled data are sparse, noisy and incomplete with large uncertainty and where the predictive uncertainty quantification is needed for scientific understanding and policy decision making. Integration of Bayesian inference into DL models adds an estimate of uncertainty and regularization in the predictions. However, traditional Bayesian methods are computationally unaffordable and inflexible for highdimensional problems, which limits their application to DL systems that typically have millions of model parameters. In this effort, we propose an efficient and generalpurpose Bayesian inference method to advance DL model optimization and uncertainty quantification, so as to facilitate the adoption of DL in Earth sciences. In a demonstration, we integrate the proposed Bayesian method with a feedforward neural network (NN) to build a fasttoevaluate surrogate of the complex Energy Exascale Earth System Land Model for efficient modeling. The formulated Bayesian NN, using a small number of training data, produces an accurate prediction with high credibility, whereas with the same small training size, the deterministic NN cannot yield a reasonable estimation and does not provide confidence information. The proposed Bayesian method is computationally efficient and flexible, capable of integration with diverse network variants such as convolutional NNs and recurrent NNs to advance the application of DL in Earth sciences.

Nonlinear system identification using a recurrent network in a Bayesian framework Modern deep neural networks are being widely exploited to solve challenging learning tasks, including nonlinear system identification. Bayesian system identification intrinsically encapsulate uncertainty in model parameters and provides forecasting distribution enabling enhanced analysis, simulation and control system design. Nevertheless, the application of the full Bayesian approach to articulated models as deep neural networks results quite challenging in practice. In this work we propose an identification technique for nonlinear dynamic systems exploiting a deep recurrent neural network with LongShort Term Memory (LSTM) units retaining a Bayesian framework. To such an aim, we stacked the recurrent neural network with a probabilistic layer, decomposing the nonlinear dynamic model into a combination of flexible functions. Hence, deterministic and stochastic layers are trained jointly, forcing the learning algorithm to transform the input data sequences into a deterministic feature space encoded by the LSTM, useful for predictions. Besides, we deployed a scalable technique based on Variational Inference to deal with the exact inference intractability. We show the effectiveness of the proposed approach by the application to a widely exploited open benchmark for nonlinear system identification.