## 3 Method
### Automatic Optimization Model Based on DDPG  
Through matrix processing, resource planning can serve as an intelligent agent for reinforcement learning. Another advantage of the resource planning matrix is that it can achieve overall planning actions through matrix operations. Therefore, this paper improves matrix \(X^{\prime}\) by adding a new column and initializing it to 0 to form final resource matrix \(M\), as shown in Equation (15). Additionally, a new action matrix \(N\) is established. The first few rows of the action matrix correspond to the planned quantity rows of resource matrix \(M\), and each row has only one pair (+1/\(-\)1), as shown in Equation (16). \[M= \begin{bmatrix}a_{1}&\cdots&a_{7}&0\\ \vdots&\ddots&&\vdots\\ z_{1}&&z_{7}&0\\ A_{1}&&A_{7}&0\\ B_{1}&&\cdots&B_{7}&0\\ C_{1}&&C_{7}&0\end{bmatrix}  \tag{15}\]  
\[N= \begin{bmatrix}-1&\cdots&+1&0\\ \vdots&\ddots&&\vdots\\ 0&&-1&+1\\ 0&&0&0\\ 0&\cdots&0&0\\ 0&&0&0\end{bmatrix}  \tag{16}\]  
Matrix \(N\) enables resource planning agents to take action through paired (+1/\(-\)1) operations. We simulate the situation where the plan for this week is called out through the eighth column of the matrix. This paper incorporates a plan adjustment constraint, which means that the plan for this week should be completed as much as possible. If it cannot be completed during the week, it must be arranged on the first day of the next week. The paper adds matrix \(M\) to matrix \(N_{i}\) to obtain \(M_{i}\), forming an adjusted resource plan. \[M_{i}=M+N_{i}\ =\ \begin{bmatrix}a_{1}&\cdots&a_{7}&0\\ \vdots&\ddots&&\vdots\\ z_{1}&&z_{7}&0\\ A_{1}&&A_{7}&0\\ B_{1}&&\cdots&B_{7}&0\\ C_{1}&&C_{7}&0\end{bmatrix} + \begin{bmatrix}-1&\cdots&+1&0\\ \vdots&\ddots&&\vdots\\ 0&&-1&+1\\ 0&&0&0\\ 0&\cdots&0&0\\ 0&&0&0\end{bmatrix}  \tag{17}\]  
Our goal is to train a strategy to automatically coordinate and balance resource plans. However, from matrix \(N\), it can be seen that the action space of the resource planning matrix is relatively complex and diverse, and the combination of +1/\(-\)1 can randomly appear at any position in matrix \(N\). It is difficult to obtain the optimal strategy using only deep reinforcement learning models. This paper proposes the use of the DDPG (deterministic policy gradient) algorithm, with the actor-critic algorithm as its basic framework, deep neural networks as approximations of the policy network and action value function, and the use of the stochastic gradient method to train the parameters of the policy network and value network models. The DDPG algorithm framework is displayed in Figure 3. The value of proxy operations is accurately evaluated by the critical network. There will be continuous interaction between agents and the environment. This interaction is also an iterative and trial and error process. Reward \(r\), observed state \(s\), selected action \(a\), and new state \(s^{\prime}\) in replay buffer \(D\) are saved in the network. The agent trains the critic network from small-batch sampleddata in the replay buffer. By using this training method, the difference in output from the target neural network is reduced. \[L_{i}(\theta_{i})=\mathbb{E}_{(s,a,r,s^{\prime})\sim u(D)}[(r+\gamma\mathcal{Q}( s^{\prime},\mu\big{(}s^{\prime};\varphi^{-}\big{)};\theta^{-}-\mathcal{Q}(s,a; \theta_{i}))^{2}] \tag{18}\]  
\(L_{i}(\theta_{i})\) is the expected value of the difference between the \(Q\)-value of the target critic network and the \(Q\)-value of the training critic network. \((s,a,r,s^{\prime})\sim u(D)\) is the sampling through mini-batch data from replay buffer \(D\). \(\gamma\) is the discount factor. \(\theta_{i}\) is the training network parameter, and \(\theta^{-}\) is the target critic network parameter; these parameters give the weights and bias. \(i\) is the parameter index. \(\mu\) is the policy of the actor network, and \(\varphi^{-}\) is the parameter of the target actor network. The actor network uses the state of the environment as the input and actions as the output to calculate the policy.