# Power Battery Scheduling Optimization Based on Double DQN Algorithm with Constraints
## 5 Experimental Evaluation and Discussion
### Cycle Life and Cumulative Rewards  
In the first part, the performance of the proposed methods is compared from the perspective of battery cycle life. In this experiment, six different battery states are input into each algorithm, and the experimental results are shown in Table 3. We find that the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Variables** & **Description** & **Value** \\ \hline \(\gamma\) & Discount factor & 0.99 \\ \(\eta_{\pi}\) & The learning rate of neural network & 0.001 \\ \(\epsilon\) & \(\epsilon\) greed strategy, exploration, and exploitation & 0.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Double DQN hyperparameters. Figure 9: Overall scheduling framework. PCA\(-\)DDQN agent has better results under all input settings. At the same time, other heuristic scheduling algorithms have achieved good results in only one or several cases because the state space is too ample and easy to fall into the local optimum. DDQN and Q\(-\)learning have too large a search space. They may fall into the risk of the \"curse of dimensionality\" without principal component analysis for dimensionality reduction, and the performance is sometimes even worse than heuristic scheduling algorithms. The overall inspection shows that the proposed method can provide appropriate charging and discharging methods with a number of cycles comparable to the optimal heuristic scheduling method. The same reward function is set for the DDQN algorithm and the Q\(-\)learning algorithm to compare the reward and convergence of the proposed method. Figure 10 shows the experimental results. The average reward during training is an essential and decisive factor reflecting the convergence process, training efficiency, and learning ability [41], as shown in Figure 10a. Maintaining a stable average return in the training process implies reliable convergence. The results show that PCA\(-\)DDQN converges much faster than Q\(-\)learning and DDQN due to the former's additional PCA dimensionality reduction capability. Because DRL adds neural networks and the experience pool, DDQN converges faster than Q\(-\)learning and can handle continuous problems and stabilize the training effect to some extent. In addition, combining Equation (28) and Figure 10b reveals that the PCA\(-\)DDQN algorithm has a higher reward value than the other two algorithms. This may be due to the excellent convergence speed and more opportunities for the agent to try. With the agent's continuous learning, the experience accumulation process makes the agent constantly adjust the control strategy quickly. It can be explained that the Q\(-\)learning algorithm exhibits the worst training efficiency and learning ability due to the inherent defects of the Q\(-\)table output. The comparison in this section shows that the proposed PCA\(-\)DDQN method in this paper has an excellent performance in terms of cycle life and cumulative reward, which provides ideas for the following experiments. According to Equation (1), it can be seen that the temperature and internal resistance differences inside the battery are assumed to be within a specific range. The comparison of the operating points of different reinforcement learning algorithms' violating constraints is shown in Figure 10c,d. An observation of the original dataset shows that the battery's temperature range, after the beginning of cyclic aging, is 30-40 \({}^{\circ}\)C, and that the internal resistance increases continuously in the aging process. The change accuracy of internal resistance is too small, which cannot sufficiently reflect the strengths and weaknesses of violating the constraint. The internal resistance difference \(\Delta IR\) is used, and its range, \(0\sim 1\times 10^{-4}\)\(\Omega\), is represented by an orange solid line. Suppose the point is not within the range of two orange lines, meaning the agent violates the constraint.