# A Small-Sample Scenario Optimization Scheduling Method Based on Multidimensional Data Expansion
## 3 XGBoost PV Power Prediction Method Considering Data Imbalance in Small-Sample Scenarios
### Markov Decision Modeling and Solution Methods  
During the operation of an integrated energy system (IES), the state of each power component in each scheduling period is determined solely by the state of that component in the previous scheduling period and changes in the external environment. Therefore, it can be modeled using a Markov decision process (MDP). A MDP can be represented by a quintuple: (\(S\), \(A\), \(P\), \(R\), and \(\lambda\)), where \(S\) denotes the system's state variables, \(A\) representsthe action space, \(P\) is the state transition probability, \(R\) is the reward function, and \(\lambda\) is the discount factor.  
State Variables: In this formulation, the energy system operator can be modeled as a reinforcement learning (RL) agent. State information provides a critical basis for the operator to schedule generation units. Thus, the state is defined as \(s_{t}=(P_{t}^{V},P_{t}^{L},P_{t-1}^{G},SOC_{t})\), where \(s_{t}\in S\).  
Action Variables: The actions for scheduling DG units and ESSs are defined as \(a_{t}=(P_{t,t}^{G},P_{t}^{B})\), where \(at\in A\). The RL agent does not directly control transactions between the energy system and the main grid. Instead, after executing any action, power is imported/exported from the main grid to maintain the power balance. Therefore, a maximum power capacity constraint must be enforced, i.e., Equation (26).  
Given the state \(s_{t}\) and the action at time step \(t\), the energy system transitions to the next state \(s_{t+1}\), defined as  
\[\mathcal{P}_{ss^{\prime}}^{a}=\Pr\{s_{t+1}=s^{\prime}\mid s_{t}=s,a_{t}=a\} \tag{28}\]  
where \(\mathcal{P}_{ss^{\prime}}^{a}\) corresponds to the leapfrog probability, which models the dynamics and uncertainty of the energy system. In model-based algorithms, uncertainty is predicted by determining values or sampling from a priori probability distributions. In contrast, DRL is a model-free approach that can learn uncertainty from historical data and interactions.  
Reward function: Environmental offers reward \(r_{t}\) as an indicator guiding the direction of policy updates. In the optimal scheduling problem of the energy system, the reward function should guide the RL agent to take actions that minimize the operating cost while implementing the power balance constraint. This can be achieved by using the next reward function:  
\[r_{t}(s_{t},a_{t})=-\sigma_{1}\left[\sum_{i\in G}C_{i,t}^{G}+C_{t}^{E}\right] -\sigma_{2}\Delta P_{t}, \tag{29}\]  
where \(\sigma_{1}\) and \(\sigma_{2}\) control the trade-off between cost minimization and the penalties incurred when there is a power imbalance. \(\Delta P_{t}\) corresponds to the power imbalance at time step \(t\) and is defined as  
\[\Delta P_{t}=|\sum_{i\in\mathcal{G}}P_{i,t}^{G}+\sum_{m\in\mathcal{V}}P_{m,t} ^{V}+P_{t}^{N}+\sum_{j\in\mathcal{B}}P_{j,t}^{B}-\sum_{k\in\mathcal{L}}P_{k,t }^{L}| \tag{30}\]  
The goal of the RL algorithm is to find an optimal scheduling policy \(\pi^{\star}\) (stochastic or deterministic) that maximizes the total expected discounted rewards of the formulated MDPs.  
\[\pi^{\star}=\underset{\pi}{\text{argmax}}\mathbb{E}_{(s_{t},a_{t})\sim\rho_{ \pi}}\left[\sum_{t\in\mathcal{T}}R(s_{t},a_{t})\right] \tag{31}\]