## 3 Prediction Models
### LSTM with an Attention Mechanism  
Although the standard recurrent neural network is an extension of the conventional feed-forward neural network, it has the problem of gradient vanishing or explosion. LSTM was developed to solve these problems and achieve excellent performance. It has unique memory and forgetting modes and can be flexibly adapted to the timing characteristics of network learning tasks. The units of the LSTM model includes a forget gate, input gate, and output gate [17; 18]. The forget gate is designed to determine whether it needs to be discarded from the cell state. The input gate is designed to determine whether new information should be stored in the cell state. The output gate is designed to determine what information will be transferred from the cell state to the current hidden layer data. These gating units are derived by  
\[f_{t}=\sigma\Big{(}W_{f}X_{t}+R_{f}h_{t-1}+b_{f}\Big{)} \tag{1}\]  
where \(\sigma\) is the sigmoid function to keep the output value between 0 and 1; \(h_{t-1}\) and \(X_{t}\) are previous layer data and the current input layer data; \(\Big{(}W_{f},R_{f},b_{f}\Big{)}\) are the input weight, the recurrent weight, and the bias of a forget gate.  
\[i_{t}=\sigma(W_{i}X_{t}+R_{i}h_{t-1}+b_{i}) \tag{2}\]  
where \((W_{i},R_{i},b_{i})\) are the input weight, the recurrent weight, and the bias of an input gate. A tanh layer is chosen to form the new memory as \(g_{t}=\tan h(W_{g}X_{t}+R_{g}h_{t-1}+b_{g})\), where \(\Big{(}W_{g},R_{g},b_{g}\Big{)}\) are the input weight, the recurrent weight, and the bias of a new memory. Then, the cell state is updated by \(C_{t}=C_{t-1}\times f_{f}+g_{t}\times i_{t}\).  
\[o_{t}=\sigma(W_{o}X_{t}+R_{o}h_{t-1}+b_{o}) \tag{3}\]  
where \((W_{o},R_{o},b_{o})\) are the input weight, the recurrent weight, and the bias of an output gate. Finally, we multiply it by the output of the sigmoid gate as \(h_{t}=o_{t}\times\tan h(C_{t})\). Figure 1 shows the architecture of the LSTM cell.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Cell Barcode** & **Cell** & **Cycle Life** & **Charging Policy** & **Experiment** \\ \hline EL150800460655 & 2\_5 & 444 & 3.6C (22\%)—5.5C & 1 \\ EL150800460635 & 2\_6 & 480 & 3.6C (2\%)—4.85C & 1 \\ EL150800460208 & 2\_11 & 483 & 4C 40\%)—6C & 1 \\ EL150800460449 & 2\_12 & 485 & 4C (4\%)—4.85C & 1 \\ EL150800460480 & 2\_14 & 487 & 4.4C (47\%)—5.5C & 1 \\ EL150800460603 & 2\_18 & 513 & 4.65C (44\%)—5C & 1 \\ EL150800460501 & 2\_19 & 527 & 4.65C (19\%)—4.85C & 1 \\ EL150800460514 & 1 & 1852 & 3.6C (80\%)—3.6C & 2 \\ EL150800460486 & 2 & 2160 & 3.6C (80\%)—3.6C & 2 \\ EL150800460623 & 3 & 2237 & 3.6C (80\%)—3.6C & 2 \\ EL150800460597 & 2\_25 & 495 & 4.8C (80\%)—4.8C & 2 \\ EL150800463611 & 2\_26 & 461 & 4.8C (80\%)—4.8C & 2 \\ EL150800460596 & 2\_27 & 471 & 4.8C (80\%)—4.8C & 2 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of the thirteen cells.  
Recently, the attention mechanism is usually used to analyze images and time-series data. Compared with other ordinary deep learning models, combining attention with LSTM can obtain better results. Note that the attention layer only helps to select the output of earlier layers that are critical to each subsequent stage of the model. It allows the network to focus on specific information selectively. It is accomplished by building a neural network focused on appropriate tasks. Detailed information on the attention-based LSTM model can be found in [21; 22; 23; 24]. In this study, the attention-based LSTM model is used as a single model. The best model parameters are obtained from the DE algorithm.