## 2 Methodology
### Combined DBN and Particle Filter-Based Approaches for RUL Prediction  
The ultimate objective of the combined DBN and particle filter approach is to estimate the pdf of \(P(\mathbf{x}_{t+L}|\mathbf{z}_{1:t})\), where \(L\in\{1,2,\ldots N-t\}\) and \(N\) represents the length of the signal. It should be noted that no new (future) information about the system is provided at the generic time step \(t\), and only information provided from time steps \(\{0,1,\ldots t\}\) may be used for the prediction. There are two functions that are of interest in modeling; the state transition distribution Equation (14) (sometimes referred to as the proposal distribution) and the measurement distribution Equation (15). Both of these functions can be modeled by the DBN, since neural networks in general are universal function approximators [34]. In the prediction step, the state transition model can simply be approximated by first reconstructing the time series of the measurements (\(\mathbf{z}_{t}\) for \(t=1,2,\ldots N\)) into a matrix, where each feature (column) represents a lagged order of the time series, and whose output is an \(L\)-step ahead into the future state value, and each row represents an index in time. Formally, the input can be denoted as:  
\[\left[\mathbf{z}_{t},\mathbf{z}_{t-1},\ldots,\mathbf{z}_{t-d+1}\right],\;\epsilon\;\mathbb{ R}^{d} \tag{16}\]  
and treat the state \(\mathbf{x}_{t}\) as the actual RUL of the bearing at time \(t\) as the output:  
\[\left[\mathbf{x}_{t},\mathbf{x}_{t+1},\ldots,\mathbf{x}_{N}\right] \tag{17}\]  
where \(d\) represents the embedding dimension, and determines the size of the first visible layer in the DBN. Thus, the network requires training tuples of the form (\(\left[\mathbf{z}_{t},\,\mathbf{z}_{t-1},\,\ldots,\,\mathbf{z}_{t-d+1}\right]\), \(\mathbf{x}_{t}\)), \(t=d-1,\ldots,N\). Using this training set, the state transition model is instead actually modeling the pdf of \(p(\mathbf{x}_{t+L}^{i}|\mathbf{z}_{t},\mathbf{z}_{t-1},\ldots,\mathbf{z}_{t-d+1})\). The \(L\) step ahead prediction, \(\mathbf{x}_{t+L}^{i}\), is simply made by subtracting the network's output \(\mathbf{x}_{t}\) by \(L\). The advantages of treating the state \(\mathbf{x}_{t}\) (instead of treating it as the crack depth for instance) as the actual RUL allow for a simpler model to be developed. One does not need to utilize another neural network to determine the RUL, given the crack depth, or to recursively compute the state transition model multiple times until it exceeds such a threshold, which needs to be defined. By training the network directly on the RUL, the network is minimizing the error of predicting the RUL, rather than an intermediate value. This results in a network that is more easily trainable and reduces the potential problem of having the network's eventual forecast of the state die off (converging to a single value). The disadvantage of this approach however, is that the parameters of the state transition model will be updated/relearned less frequently. The measurement \(z_{t}\) (often a multi dimensional vector) can be constructed into a mono-dimensional feature vector by setting the input of the DBN as \(z_{t}\) and then setting the size of the last hidden layer to one. The output of the DBN in its unsupervised stage of learning is then the reconstructed mono-dimensional feature vector of \(z_{t}\). Equations (16) and (17) define a method often called the sliding window technique [35]. This windowing approach is illustrated in Figure 4.  
Figure 4: The windowing approach. The first plot in Figure 4 is the complete series of a fault feature. The second plot highlights the first window of size \(d=100\). The last figure simply shows a zoom-in of the first window. The first window contains points from \(t=0,\ 1,\ 2,\ \dots\ 98,\ 99\) (first row of features), and the second window would contain points from \(t=1,\ 2,\ 3,\dots\ 99,\ 100\) (second row of features), and so on.