# Lithium-Ion Battery Prognostics through Reinforcement Learning Based on Entropy Measures
## 4 Methodology
### Predictive Models
#### 4.2.2 Arima  
The Autoregressive Integrated Moving Average (ARIMA) method is proposed as a technique for statistical analysis in time series data. An ARIMA model is a combination of the autoregressive (AR) and moving average (MA) models. The ARIMA model can be explained according to three notations--\(p\), \(d\), and \(q\)--which define the type of the ARIMA model:  
* \(p:\) order of auto-regression
* \(d:\) order of difference
* \(q:\) order of moving average  
For AR \((p)\), we have:  
\[\hat{y}_{t}=\varnothing_{1}y_{t-1}+\varnothing_{2}y_{t-2}+\cdots+\varnothing _{p}y_{t-p}+\varepsilon_{t} \tag{11}\]  
MA \((q)\) can be described as follows:  
\[\hat{y}_{t}=\varepsilon_{t}-\theta_{1}\varepsilon_{t-1}-\theta_{2}\varepsilon _{t-2}-\cdots-\theta_{q}\varepsilon_{t-q} \tag{12}\]  
ARMA \((p.q)\) is a combination of AR \((p)\) and MA \((q)\), and is described as below:  
\[\hat{y}_{t}=\varnothing_{1}y_{t-1}+\cdots+\varnothing_{p}y_{t-p}+\varepsilon_{ t}-\theta_{1}\varepsilon_{t-1}-\cdots-\theta_{q}\varepsilon_{t-q} \tag{13}\]  
where \(y_{t}\) and \(\hat{y}_{t}\), respectively, are the observed and estimated values; \(\varnothing\) and \(\theta\), respectively, are coefficients; and \(\varepsilon_{t}\) is a normal white noise process with zero mean. ARIMA is an advanced version of ARMA, which also works well for non-stationary time series data. To convert the non-stationary to stationary data, a data transformation is needed using a \(d\)-order difference equation [32]. Consequently, ARIMA \((p.d.q)\) can be described as Equation (14). \[\hat{w}_{t}=\varnothing_{1}w_{t-1}+\cdots+\varnothing_{p}w_{t-p}+\varepsilon_{ t}-\theta_{1}\varepsilon_{t-1}\cdots-\theta_{q}\varepsilon_{t-q} \tag{14}\]  
where \(w_{t}=\
abla^{d}y_{t}\) and \(\
abla\) is the gradient operator. When \(d=0\), Equation (14) is the same as Equation (13) and, thus, ARIMA acts the same as ARMA. \(p\) and \(q\) are initialized using the autocorrelation function (ACF) and partial autocorrelation function (PAFC). AFC measures the average correlation between data points in a time series and previous values of the series measured for different lag lengths. PACF is the same as ACF, except that each correlation controls for any correlation between observations of a shorter lag length [32]. Figure 9 demonstrates the ARIMA framework from the input data stage through the prediction stage. In this study, an ARIMA model is proposed to predict future battery capacities. Since we are working with a non-stationary time series, we have made a data transformation with \(d=1\). \(p\) and \(q\), respectively, are set to 5 and 0, and thus, predictions were made with ARIMA \((5.1.0)\). The rationale behind choosing the order of the ARIMA model is as follows. We compare the results from a range of non-negative integers, \(p=[1,10]\) (extracted from the existing literature), and select the optimal number of time lags for the autoregressive model, which results in minimal errors compared to other orders in that range. The results from the optimal model are displayed and reported here. There is a battery voltage sequence at each cycle (i.e., a time series of voltages at each cycle). We first compute the permutation entropy of each voltage sequence according to the  
Figure 9: ARIMA framework. corresponding algorithm; then, we use the time series of the permutation entropy measures (i.e., one entropy measure at each cycle) as an input in the ARIMA model, compare them with the deviations in the battery capacities, and predict the next-cycle battery capacity as an output of the model. An algorithm for the ARIMA model is presented as follows.