## 3 The Proposed Methodology for Prediction of Bearing Defect Degradation Using Deep Neural Network
### Adam Optimization-Based Backpropagation Algorithm and Xavier Weight Initialization
Once the moments of the gradient were calculated, the DNN parameters were updated as follows:  
\[\mathrm{w}_{ij}^{l}=\mathrm{w}_{ij}^{l}-\alpha.\frac{\hat{m}_{t}}{\left( \sqrt{\hat{v}_{t}+\varepsilon}\right)} \tag{9}\]  
\[\mathrm{b}_{i}^{l}=\mathrm{b}_{i}^{l}-\alpha.\frac{\hat{m}_{t}}{\left(\sqrt{ \hat{v}_{t}+\varepsilon}\right)}. \tag{10}\]  
Thus, to approximate properly the output targets, the backpropagation algorithm was used to minimize the undesired error at output via a loss function by adjusting the model's parameters on the backward procedure of DNN. The process of calculating the gradient of a loss function and updating the model's parameters was performed in a DNN training epoch. Training and adjusting the network parameters were an over and over again process until the error ratio of the DNN outputs had reduced to the minimum values. The synaptic weights of a DNN are usually initialized random values and its biases are set to zero. A proper initialization of the weights in a neural network is critical to its convergence. If these weights are set too small, the variance of the input data starts to diminish as it goes through each layer of the neuron network. Hence, the input finally falls to a very low value that can be no longer useful. If the weights are set too large, the variance of input data has a tendency to rapidly increase after passing each layer. Consequently, the input gets so large that it also becomes useless. Because the nonlinear activation function might saturate for larger values, the network activations become saturated. As a result, random initialization does poorly with DNN. Actually, the assignment of the initial network weights is usually a random process for DNN training because nothing is known about the data and how to assign the weights that are suitable for the particular cases. An appropriate initialization of the weights is very important for a neural network to function properly. Thus, we made certain that the weights were set in a reasonable value range. Xavier initialization is derived from the idea of keeping the input and output of a layer with the same variance and the zero mean [27]. Supposing a neuron layer and its variance (Var) are considered as  
\[\mathrm{z}=\sum_{i=1}^{n}\mathrm{w}_{i}\mathrm{x}_{i}+\mathrm{b} \tag{11}\]  
\[\mathrm{Var}(\mathrm{z})=\mathrm{Var}(\sum_{i=1}^{n}\mathrm{w}_{i}\mathrm{x}_ {i}), \tag{12}\]  
when each signal passes through a layer, we want to keep the same variance. Assume that the inputs and weights are independent and identically distributed Gaussians of zero mean, to obtain \[ \begin{split}\mathrm{Var}(\mathrm{z})&=\sum\limits_{ \mathrm{i=1}}^{\mathrm{n}}Var(w_{\mathrm{i}}x_{\mathrm{i}})\\ \mathrm{Var}(\mathrm{z})&=\sum\limits_{\mathrm{i=1}}^{ \mathrm{n}}\left[\mathrm{E}(w_{\mathrm{i}})\right]^{2}Var(x_{\mathrm{i}})+ \mathrm{E}[(x_{\mathrm{i}})]^{2}Var(w_{\mathrm{i}})+\mathrm{Var}(x_{\mathrm{ i}})\mathrm{Var}(w_{\mathrm{i}})\\ \mathrm{Var}(\mathrm{z})&=\sum\limits_{\mathrm{i=1}}^{ \mathrm{n}}Var(x_{\mathrm{i}})\mathrm{Var}(w_{\mathrm{i}})=\mathrm{nVar}(w) \mathrm{Var}(x)\end{split}  \tag{13}\]  
The above equation shows that the variance of the output is the variance of the input but scaled by \(\mathrm{nVar}(\mathrm{w})\). To keep the variance of the input and output unchanged, set \(\mathrm{nVar}(\mathrm{w})=1\), which results in \(\mathrm{Var}(\mathrm{w})=\frac{1}{\mathrm{n}}\). Hence, we have arrived at the final Xavier initialization formula. The weights should be initialized using Gaussian distributions with a zero mean and a variance of \(\frac{1}{\mathrm{n}}\), where \(\mathrm{n}\) denotes the number of neurons in a layer. This method works extremely well in many cases that are extensively used in the initialization of neural networks.