# Lithium-Ion Battery Prognostics through Reinforcement Learning Based on Entropy Measures
## 5 Results and Findings
### Numerical Results  
The observed and predicted battery capacities results from ARIMA and LSTM models are shown in Figures 11-13. Based on the graphs obtained, it can be seen that in all three datasets the ARIMA model predictions are following the trends in the test data, and so, yields better results as compared to the LSTM model for predicting the time series of battery capacities.  
The early battery-life prediction, which includes a prediction of the battery cycles at earlier cycles, is performed, and the results are displayed in Figures 14-16. It is observed that the deviation between the predicted capacities and the actual capacities are not significant, indicating that the proposed ARIMA and LSTM models are capable of predicting battery capacities at earlier cycles.  
Figure 11: Train, test, and predicted data results from ARIMA and LSTM models for PL19.  
Figure 12: Train, test, and predicted data results from ARIMA and LSTM models for PL11.  
Figure 13: Train, test, and predicted data results from ARIMA and LSTM models for PL09.  
In the RL model, as demonstrated in Figure 17, the reward values have an impressive increase and immediately become stable with some noise. The loss values increase at first; however, after approximately 250 epochs, they decline to 0, which verifies the procedure of Reinforcement Learning.  
To find the best data split ratio, our proposed RL approach is initially trained using shuffled datasets with five different training ratios (70%, 75%, 80%, 85%, and 90%). Afterwards, Mean Squared Error (MSE) is utilized as a loss function to evaluate the obtained results. Based on Table 4, the best accuracy is gained by using 90% of each dataset for training purposes and using the rest for the testing process (Figure 18). Finally, this ratio is applied to training the other two models (LSTM and ARIMA). To save space, the results from the LSTM and ARIMA models are not reported here. The results from the other two models are consistent with those from RL (i.e., the best training ratio of 10%).  
Figure 16: Train, test, and predicted data results from ARIMA and LSTM models for PL09.  
Figure 14: Train, test, and predicted data results from ARIMA and LSTM models for PL19.  
Figure 15: Train, test, and predicted data results from ARIMA and LSTM models for PL11.  
From Tables 5-7, it is observed that the ARIMA model yields smaller errors compared to the LSTM model. ARIMA, which is a mean-reverting process, has the ability to predict battery capacities with smaller deviations. However, the LSTM model--which is a recurrent network--attempts to avoid the long-term dependency by storing only necessary information, and thus, it is unable to probabilistically exclude the input (i.e., previous permutation entropy of battery voltage sequences) and the recurrent connections to the units of the network from the activation and weight updates while the model is being trained. Consequently, the deviations between the actual battery capacities and the predicted capacities resulting from the LSTM model are greater than those resulting from the ARIMA model. The results displayed on Figures 11-13 are consistent with the Tables.