## 3 DLSTM Based Indirect RUL Prognosis
### DLSTM for RUL Prognosis  
In this section, the LSTM [33] is introduced for Li-ion battery RUL prognosis. Compared with the recurrent neural network (RNN), the LSTM adds a memory cell structure which can solve the vanishing gradient problem and the exploding gradient problem. The LSTM is more suitable for time series prediction. The basic LSTM architecture predictor is shown in Figure 4. Here, \(x_{t}\) is the input at the current time step, \(h_{t-1}\) stands for the output at the previous time step, and \(C_{t-1}\) is the cell memory at the previous time step; \(h_{t}\) stands for the output at the current time step, and \(C_{t}\) is the cell memory at the current time step. The red line (Figure 4) can maintain information transfer and not change the information through the whole cell state, which is the key to LSTM. The LSTM mainly contains a forget gate, input gate, input node, and output gate. The forget gate is important; it can decide which information is discarded. For Li-ion battery RUL prognosis, the forget gate can discard some previous redundant information which does not influence the prediction result of next time step. The input gate and input node can determine which new information is stored. The output can realize the information output. Therefore, according to Figure 4, the essential formulas of LSTM are expressed as follows:  
\[f_{t}=\mathbf{\sigma}(W_{f}\cdot[h_{t-1},x_{t}]+b_{f}) \tag{4}\]  
\[i_{t}=\mathbf{\sigma}(W_{i}\cdot[h_{t-1},x_{t}]+b_{i}) \tag{5}\]  
\[\widetilde{C}_{t}=\tanh(W_{C}\cdot[h_{t-1},x_{t}]+b_{C}) \tag{6}\]  
\[C_{t}=f_{t}*C_{t-1}+i_{t}*\widetilde{C}_{t} \tag{7}\]  
\[o_{t}=\mathbf{\sigma}(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}) \tag{8}\]  
\[h_{t}=o_{t}*\tanh(C_{t}) \tag{9}\]  
where \(\mathbf{\sigma}\) is the sigmoid function; \(W_{f}\), \(W_{i}\), \(W_{C}\) and \(W_{o}\) are weight matrices; \(b_{f}\), \(b_{i}\), \(b_{C}\) and \(b_{o}\) are bias vectors. When the LSTM is applied for Li-ion battery RUL prognosis, there are three problems. First, the traditional gradient descent methods, such as stochastic gradient descent (SGD), mini batch gradient descent, and RMSprop, cannot guarantee rapid convergence. They are also not suitable for on-line  
Figure 4: Basic LSTM architecture. RUL prognosis. The second problem is the overfitting phenomenon. The last problem is dynamic prediction with high accuracy. In order to solve these problems associated with the deep learning in LSTM, this paper contributes three possible solutions. (1) Based on the resilient back-propagation algorithm [34] (Appendix A), this paper adopts the adaptive moment estimation (Adam) method [35] so that LSTM can have rapid convergence. The Adam method can avoid learning-rate loss, slow convergence, and significant change of loss function. The literature [35] has demonstrated that the Adam method can be applied for practical deep-learning problems, using large models and datasets. Therefore, we do not repeat it in this paper (2) The L2 regularization method [36] is utilized to avoid overfitting. (3) Combining the incremental learning idea with the advantage of the forget gate, a dynamic long short-term memory neural network (DLSTM) is developed. The major steps of DLSTM for RUL prognosis can be summarized as follows:  
Step 1: Partition the training data and test data according to the acquired indirect HI time series in Section 3.1.