# Power Battery Scheduling Optimization Based on Double DQN Algorithm with Constraints
## 1 Introduction
Although the previous work in the literature has proposed battery scheduling optimization schemes in terms of hardware, it is still a challenge to realize these schemes in real life, owing to the ever\(-\)popular new technology and the development of the general trend of data exchange. Since non\(-\)heuristic algorithms blindly search for information and do not consider the characteristics of the problem, many studies are based on heuristic algorithms. Heuristic algorithms can provide feasible solutions for each instance of the combinatorial optimization problem to be solved at an acceptable expense and are more economical and efficient in algorithm design compared to hardware design [12]. Currently, heuristic algorithms are more widely applied in battery scheduling optimization. [PERSON] et al. [13] proposed a day\(-\)ahead dispatch optimization model based on power cells, using an integrated grid containing wind farms, solar PV, and battery storage for problem simulation, and an improved artificial bee colony algorithm (ABC), to study the optimal dispatch of battery storage with operational constraints in high\(-\)voltage power systems, improving the performance of the battery. [PERSON] et al. [14] proposed a set of optimal battery pack charging control strategies based on the leader\(-\)follower framework, obtained the optimal average charging state trajectory based on the nominal battery model through multi\(-\)objective optimization, established online model deviation compensation, and effectively improved the robustness. [PERSON] et al. [15] proposed a synergistic optimization method for the battery's entire life cycle and life maximization, established a scheduling model consisting of a host of nonconvex nonlinear programming problems, designed equivalent problem transformation and acceleration algorithms, promoted the lifetime profit. Traditional optimization methods and heuristic algorithms, such as linear programming and genetic algorithms, have high convergence speed stability and solving effect. However, they cannot effectively deal with high\(-\)dimensional space, and the algorithms are too much influenced by their parameters and easily fall into the local optimum. A robust battery scheduling technique should be able to respond to different applications and different underlying architectures and determine the optimal management policy on time. A promising approach is dynamic battery scheduling optimization based on reinforcement learning (RL); by estimating the time\(-\)out feedback from the external environment, the agent learns to augment itself from environmental transactions. RL has shown its potential in scheduling optimization and has been successfully applied to power battery scheduling management [16; 17]. [PERSON] et al. [18] formulated the problem of using the battery to provide flexibility as a sequential decision problem under uncertainty, while the article proposed a batch reinforcement learning algorithm based on Q\(-\)fitting iterations to help solve the battery supply flexibility problem for electricity demand. [PERSON] et al. [19] took different battery types and application scenarios as environmental models. They proposed a battery scheduling framework based on deep reinforcement learning (DRL) to solve the problem of battery scheduling and prolong the service life of the battery as much as possible. [PERSON] et al. [20] coordinated the security control algorithm using the serial strategy with the PPO algorithm in deep reinforcement learning. PPO agent safely implemented the capacity scheduling strategy of the photovoltaic cell system and increased the cumulative net revenue. [PERSON] et al. [8] implemented the deep deterministic policy gradient algorithm (DDPG) to design a model to determine the optimal real\(-\)time charging and discharging power of the battery to achieve scheduling control, which brought lower operating costs while providing sufficient exchangeable battery.