# A Small-Sample Scenario Optimization Scheduling Method Based on Multidimensional Data Expansion
## 3 XGBoost PV Power Prediction Method Considering Data Imbalance in Small-Sample Scenarios
### DDPG Algorithm  
The DDPG algorithm is currently one of the more mainstream approaches; its advantage lies in its ability to handle high-dimensional and continuous action spaces, making it suitable for complex control tasks. This paper applies the above concept to the Markov decision process, solving for the optimal scheduling of the integrated energy system to mitigate the power imbalance in real-time scheduling, as well as to reduce costs through the optimization of charging and discharging actions for the integrated energy storage system. DDPG is an algorithm based on the structure of Actor-Critic [23], and its basic framework is shown in Figure 4.  
Deep neural networks approximate the policy network and action value function. The parameters in the policy network and value network models are trained using the stochastic gradient method, which can solve the problem of continuous action space. It represents the policy as a form of a function with parameters that map a state directly to a definite action value and, therefore, a deterministic policy, which [PERSON]'s equations can solve to obtain the Q-value:  
\[Q_{\alpha}(s_{t},a_{t})=E[r(s_{t},a_{t})+\gamma Q_{\alpha}(s_{t+1},a_{t+1})] \tag{32}\]  
Solving the maximum value of [PERSON]'s equation, the optimal action value function can be calculated, and the optimal policy can be obtained. The DDPG algorithm approximates the action value function through the Critic network and the policy function through the Actor network. At the same time, to ensure that the training process is stable, the DDPG algorithm, the Critic network is divided into two independent networks, the current Critic network and the target Critic network, and the Actor network is divided into two independent networks, the current Actor network and the target Actor network. The principle of the DDPG algorithm is shown in Figure 5.  
**Current Critic Network Parameter Update**  
The current Critic network Q updates the parameters by using the timing differential error (TD error), minimizing the loss function L as  
\[L=\frac{1}{N}\sum_{I}[y_{i}-Q(s_{i},a_{i}\,|\,\theta_{Q})]^{2} \tag{33}\]  
where \(y_{i}=r_{i}+\gamma Q^{\prime}(s_{i+1},\alpha^{\prime}(s_{i+1}|\theta_{\alpha^ {\prime}})|\theta_{Q^{\prime}})\) is computed by the target Critic network and the target Actor network. Based on the loss function, the gradient of \(L\) with respect to \(\theta_{Q}\), denoted as \(\
abla_{\theta_{Q}}L\), is calculated, and the parameters \(\theta_{Q}\) are updated along the gradient direction.  
**Current Actor Network Parameters Update**  
The current Actor network updates the parameter \(\theta_{\alpha}\). The goal of the current Actor network is to obtain a larger \(Q\)-value, so, first, find the gradient \(\
abla_{\alpha}Q\) of the current Critic network over the current Actor network action, and then calculate the gradient \(\
abla_{\theta_{\alpha}}\alpha\) of the current Actor network \(a\) with respect to the parameter \(\theta_{\alpha}\):  
Figure 4: Actor–Critic architecture.  
\[\
abla_{\theta_{a}}\alpha|s_{i}\approx\frac{1}{N}\sum_{i}\
abla_{a}Q(s,a|\theta_{ Q})|_{s=s_{i},a=\alpha(s_{i})}\
abla_{\theta_{a}}\alpha(s|\theta_{a})|_{s=s_{i}} \tag{34}\]  
**Target Network Parameter Update**  
The parameters \(\theta_{a^{\prime}}\) of the target Actor network \(\alpha^{\prime}\) and the parameters \(\theta_{Q^{\prime}}\) of the target Critic network \(Q^{\prime}\) are updated using a soft update method, expressed as  
\[\theta_{Q^{\prime}}\leftarrow\tau\theta_{Q}+(1-\tau)\theta_{Q^{ \prime}} \tag{35}\] \[\theta_{a^{\prime}}\leftarrow\tau\theta_{a}+(1-\tau) \tag{36}\]  
where \(\tau\) is the soft update coefficient, a weighted average of the original target and current network parameters, and its value ranges from 0 to 1.