Measuring System Entropy with a Deep Recurrent Neural Network Model In this paper, a methodology for assessing the unpredictability of systems with memory was developed. The proposed approach consists in approximating the probability distribution exhibited by the response of a system, understood as a stochastic process, with a deep recurrent neural network; such networks offer increased forecasting capability by exploiting an accumulative register of previous system states. Once the probability distribution is computed, the uncertainty or entropy of the underlying process is measured. This measure determines the degree of regularity in the system, and identifies how atypical the system dynamics are. The proposed model was validated by identifying industrial gas turbine engine faults from recorded sensor data.

Nonlinear system identification using a recurrent network in a Bayesian framework Modern deep neural networks are being widely exploited to solve challenging learning tasks, including nonlinear system identification. Bayesian system identification intrinsically encapsulate uncertainty in model parameters and provides forecasting distribution enabling enhanced analysis, simulation and control system design. Nevertheless, the application of the full Bayesian approach to articulated models as deep neural networks results quite challenging in practice. In this work we propose an identification technique for nonlinear dynamic systems exploiting a deep recurrent neural network with LongShort Term Memory (LSTM) units retaining a Bayesian framework. To such an aim, we stacked the recurrent neural network with a probabilistic layer, decomposing the nonlinear dynamic model into a combination of flexible functions. Hence, deterministic and stochastic layers are trained jointly, forcing the learning algorithm to transform the input data sequences into a deterministic feature space encoded by the LSTM, useful for predictions. Besides, we deployed a scalable technique based on Variational Inference to deal with the exact inference intractability. We show the effectiveness of the proposed approach by the application to a widely exploited open benchmark for nonlinear system identification.