## 3 The Proposed Methodology for Prediction of Bearing Defect Degradation Using Deep Neural Network
### Adam Optimization-Based Backpropagation Algorithm and Xavier Weight Initialization  
To train the network, it is necessary to gain optimal operation parameters in DNN. In this study, the backpropagation algorithm was a good choice owing to the fact that it is a common method of training artificial neural networks used in conjunction with an Adam optimization method [26]. Backpropagation calculates the gradient of a loss function with respect to all the weights in the network. Suppose we have a fixed training set \(\{(x_{1},y_{1}),(x_{2},y_{2}),\ldots,(x_{M},y_{M})\}\) of M training datasets. Then, the output target of the \(x_{i}\) is \(y_{i}\). In detail, the loss function with the model's parameters can be defined by calculating the mean square error (MSE), as in the following equation:  
\[ \begin{split} J(W,b)&=\frac{1}{M}\sum\limits_{i=1}^{M }J(W,b;x_{i},y_{i})=\frac{1}{M}\sum\limits_{i=1}^{M}\left(\frac{1}{2}\left| \left|h_{W,b}(x_{i})-y_{i}\right|\right|^{2}\right)\\ J(W,b)&=\frac{1}{2M}\sum\limits_{i=1}^{M}\left|\left| h_{W,b}(x_{i})-y_{i}\right|\right|^{2}\end{split}  \tag{8}\]  
where J(W,b) is the loss function, M is the number of training dataset samples, \(y_{i}\) is the ith target sample, and \(h_{W,b}(x_{i})\) is the ith output sample of the neural network. Calculation of the loss function using MSE is simple and effective for linear regression problems. In this paper, the loss function for estimating the undesired error ratio was calculated by average squaring the reconstruction error term, as given in Equation (8), which is widely used in measuring the difference between predicted and actual values. The main objective of DNN training is to minimize this loss function, making the error of the DNN outputs close or equal to zero. Hence, an Adam optimization algorithm was utilized and the pseudocode of method was given as  
* Set \(\alpha\) the learning rate \(\beta_{1},\beta_{2}\in[0,1)\): exponential decay rates for the moment \(J(\theta)\): the loss function with the model's parameters \(\theta=\{W,b\}\). * Start at time step \(t=0\): initialization of \(\theta_{0}\), 1 st moment \(m_{0}=0\), and 2 nd moment \(v_{0}=0\). While \(\theta_{t}\) do not converge, do \(t=t+1\) \(g_{t}=\Delta_{\theta}J(\theta_{t-1})\): calculating gradient of the loss function at \(t\) \(m_{t}=\beta_{1}.m_{t-1}+(1-\beta_{1}).g_{t}\): updating the first moment estimation \(v_{t}=\beta_{2}.v_{t-1}+(1-\beta_{2}).g_{t}^{2}\): updating the second moment estimation \(\hat{m}_{t}=\frac{m_{t}}{1-\beta_{1}^{2}}\): calculating the bias-corrected first moment \(\hat{v}_{t}=\frac{v_{t}}{1-\beta_{2}^{2}}\): calculating the bias-corrected second moment. Update the parameters: \(\theta_{t}=\theta_{t-1}-\alpha.\frac{\hat{m}_{t}}{(\sqrt{v_{t}}+\epsilon)}\)end while, return \(\theta_{t}\). Adam optimization is derived from adaptive moment estimation. This algorithm works better in practice compared to other learning methods due to its fast convergence that makes the learning speed of a model quite fast and efficient. The key step is computing the first and second moments of the gradients above. Backpropagation is a popular technique for neural network training, which gives an efficient way to compute these moments. We first propagated forward calculating the activations for all layers up to the output layer using the equations defining the forward propagation steps. Then, we propagated backward in the network carrying the squared error terms and updating the W, b values using the Adam algorithm, in which we computed the first and second moments of the gradients of the loss function with respect to the model's parameters and updated their values in the descent direction of the gradient to obtain the loss function minimum.