# Power Battery Scheduling Optimization Based on Double DQN Algorithm with Constraints
## 1 Introduction
### Capacity Estimation Model Based on GPR  
Gaussian process regression is a nonparametric probability estimation model that extends multivariate Gaussian distribution and has advantages in solving regression and  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Battery**} & \multicolumn{2}{c}{**Var**} & \multicolumn{2}{c}{**Ske**} & \multicolumn{2}{c}{**Kur**} \\ \cline{2-7}  & **PCC** & **SCC** & **PCC** & **SCC** & **PCC** & **SCC** \\ \hline CH5 & 0.9867 & 0.9789 & 0.9998 & 0.9878 & 0.9764 & 0.9777 \\ CH11 & 0.9986 & 0.9686 & 0.9999 & 0.9799 & 0.9915 & 0.9837 \\ CH17 & 0.9976 & 0.9966 & 0.9954 & 0.9733 & 0.9824 & 0.9968 \\ CH22 & 0.9971 & 0.9974 & 0.9877 & 0.9843 & 0.9962 & 0.9695 \\ CH32 & 0.9846 & 0.9732 & 0.9865 & 0.9812 & 0.9921 & 0.9999 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Numerical results of correlation coefficients between health indicators and capacity.  
Figure 4: (a) LSTM network structure; (b) LSTM memory cell.  
classification problems due to Bayesian\(-\)based optimization [29; 30]. GPR has many applications in battery SOH and charging state estimation, all of which have yielded satisfactory results [31; 32]. Therefore, this paper adopts Gaussian process regression to build a capacity estimation model. In general, the relationship can be written as follows:  
\[y=f(x)+\varepsilon_{s}\varepsilon\sim N(0,\sigma_{n}^{2}) \tag{6}\]  
where \(x\) is input, \(y\) is output, \(\varepsilon\) is the noise variable with variance \(\sigma_{n}^{2}\), \(f(x)\) is a linear function, and its probability distribution is:  
\[f(x)\sim GP(\mu(x),\kappa(x,x_{*})) \tag{7}\]  
where \(\mu(x)\) is the mean function \(E[f(x)]\) and \(\kappa(x,x_{*})\) is the kernel function. The expression is as follows:  
\[\kappa(x,x_{*})=\sigma_{f}^{2}\exp(-\frac{||x-x_{*}||_{2}^{2}}{2l^{2}}) \tag{8}\]  
where \(l\) is the characteristic length scale. The kernel matrix of the training set is calculated according to the kernel function. For the test set \(X_{*}\), the prediction vector \(f_{*}(x)\) is expected to be obtained. The model assumption for Gaussian process regression is:  
\[\begin{bmatrix}y\\ f_{*}(x)\end{bmatrix}\sim GP\bigg{(}0,\begin{pmatrix}\kappa(x,x)+\sigma_{n}^{2 }I_{n},\kappa(x,x_{*})\\ \kappa(x_{*},x),\kappa(x_{*},x_{*})\end{pmatrix}\bigg{)} \tag{9}\]  
where \(I_{n}\) is an n\(-\)dimensional unit matrix. According to the Bayesian regression method, the posterior probability is found and the expression for the prediction point is determined. The right side of Equation (9) is simplified as \(GP(0,\begin{bmatrix}\kappa_{y}&\kappa_{*}\\ \kappa_{*}^{T}&\kappa_{**}\end{bmatrix})\):  
\[P(f_{*}(x)|x_{*},x,y)=GP\big{(}\mu_{*},\sum*\big{)} \tag{10}\]  
where  
\[\mu_{*}=\kappa_{*}^{T}\kappa_{y}^{-1}y_{*}\sum*=\kappa_{**}-\kappa_{y}^{-1} \kappa_{*}\]  
The predicted mean \(\bar{y^{*}}\) and predicted covariance values \(cov(y^{*})\) are expressed as follows:  
\[\bar{y^{*}}=\kappa_{*}^{T}[\kappa_{y}+\sigma_{n}^{2}I_{n})]^{-1}y \tag{11}\]  
\[cov(y^{*})=\kappa_{**}-\kappa_{*}^{T}[\kappa_{y}+\sigma_{n}^{2}I_{n})]^{-1} \kappa_{*} \tag{12}\]  
Finally, the final predicted value can be obtained by calculating \(\bar{y^{*}}\) and 95% confidence interval. Based on this, the GPR model of capacity estimation is established. The input of this model is the HI of each cycle in the battery aging data, and the output is the corresponding battery capacity.  
In establishing the battery aging model, the HI is first sought and the correlation coefficient and experiment validated the selected HI. Then, construct the HI degradation prediction model based on LSTM, with the original dataset as input and the corresponding HIs as output. Build the battery capacity estimation model based on GPR, input the predicted HIs, and output the battery capacity. We use this battery aging model as a reinforcement learning environment. The process of constantly training the battery aging model is the process of constantly finding the Equation (3) function \(g(m)\).