## 3 Prediction Models
### Propose Model  
Figure 2 illustrates a novel framework for cycle life prediction using an ensemble model. In the first level, two machine learning models, such as LSTM with attention and GBR models, are used to generate predictions. These predicted values are used as input features for the final prediction  
Figure 1: Long short-term memory (LSTM) memory cell architecture redrawn from [19; 20].  
with actual features. In the second level, the SLSTM model with a sliding window method is used to predict the final predicted value. All hyperparameters of each model are derived using the DE algorithm. The detail of the DE algorithm can be found in [28; 29; 30]. For example, five parameters need to be determined in advance, such as lookback, batch size, neuro, steps per epochs, and epochs in the LSTM model. The following steps can obtain the best hyperparameters.  
Step 1. Extract the features such as cycle, capacity, internal resistance, and average temperature.  
Step 2. Use mean absolute percentage error (MAPE) as the fitness function, which is obtained by  
\[\text{MAPE}=\frac{\sum_{t=1}^{n}\left|\left(C_{t}-\hat{C}_{t}\right)/C_{t} \right|}{n}\ast 100 \tag{5}\]  
where \(C_{t}\) is the actual capacity at cycle \(t\), \(\hat{C}_{t}\) is the predicted capacity at cycle \(t\), and \(n\) is the prediction length.  
Step 3. Select the range of LSTM and GBR hyperparameters, and use the specified model to calculate the MAPE value.  
Step 4. Decide the values of DE algorithms such as NP, CR, and F, which are 50, 0.9, and 0.8, respectively, in this study.  
Step 5. Output the best value. For example, the optimal values for the lookback, batch size, neuro, steps per epoch, and epochs of the proposed model in cell 2_5 are 8, 21, 15, 45, and 59, respectively.  
The LSTM model should include dropout parameters to reduce overfitting and enhance model performance. Dropout is a regularization method. In this method, the cyclic connection with the input of the LSTM cell may not be excluded from the activation and weight update in network training. That is to say, two parameters (such as dropout and recursive dropout) are used for linear transformation of the recursive state. Therefore, the Monte Carlo (MC) dropout technique is used to obtain the variance and bias of the proposed model, and the sliding window method is used to  
Figure 2: The flowchart of the proposed model.  
construct its prediction interval. Converting a conventional network to a Bayesian network through MC dropout is as simple as using the dropout technique for each layer during training and testing. It is equivalent to sampling from the Bernoulli distribution and provides predictive stability across samples [31]. The idea is to run the model several times with random dropout, which will produce different output values. Then, we can calculate the empirical mean and variance of the output to obtain the prediction interval for each time step. The sliding window is a temporary approximation of the actual value of the time-series data. The window and segment sizes will increase until a smaller error approximation is reached. After selecting the first segment, we select the next segment from the end of the first segment; repeat the process until all time-series data are segmented.