## 3 Improved Stochastic-Batch Gradient Pile Descent
### Training Is Conducted Using the Stochastic-Batch Gradient Descent Method  
The parameter training of models based on deep learning mainly consists of two steps. (1) Pre-training the parameters of each hidden layer in a feedforward manner to obtain initial values for each hidden layer's parameters. (2) On the basis of pre-training, applying an algorithm to uniformly train all parameters to adjust the hidden layer parameters and ultimately obtain the output layer parameters. In the context of deep learning algorithm structure, this study adopts a six-layer hidden layer configuration, with each hidden layer having 100, 80, 60, 40, 30, and 20 neurons respectively. The activation function used for the hidden layers is the sigmoid function, while the output layer utilizes a linear function as its activation function. The sigmoid function is a typical activation function used in neural networks and has proven to be effective. Its mathematical expression is given by Equation (33) as follows:  
\[\text{sigmoid}(z)=\frac{1}{1+\text{e}^{-z}} \tag{33}\]  
The model structure is illustrated in Figure 4.  
In Figure 4, \(\{d,y\}\) represents the input variables and output variables, where \(d=[d_{1},d_{2},d_{3},d_{4}]\) represents the influencing parameters such as power flow data, and y represents the critical point active power. \(f_{i},\mathbf{w}_{i},m_{i},b_{i}\), respectively, denote the number of neurons, activation function, weight, and bias of the \(i\)-th layer. \(I_{i}\) represents the output of the \(i\)-th layer, which also serves as the input for the I + 1-th layer. \(I_{0}\) represents the input to the model, and the output of the model is the output of the last layer \(I_{7}=y\). There are two methods for layer-wise pre-training: unsupervised learning and supervised learning. Research has shown that the supervised learning approach for layer-wise pre-training yields better fitting results. Therefore, in this paper, we adopt the supervised learning method for layer-wise pre-training. Below, we illustrate the first step of pre-training, focusing on the parameters \((\mathbf{w}_{1},\mathbf{b}_{1})\) of the first hidden layer in the predictive model, as shown in Figure 4. The training structure for the first and second layers is depicted in Figure 5.  
Figure 4: Model structure. First, construct a three-layer neural network, as shown in Figure 5a, where \(\mathbf{w^{\prime}}_{1},b^{\prime}_{1}\) represent the weight matrix parameters and biases of the output layer. The training optimization model is given by Equations (34)-(36) as follows:  
\[l_{1}^{j}=\text{sigmoid}(\mathbf{w}_{1}\mathbf{d}^{j}+\mathbf{b}_{1}),\;j=1,\ldots,S \tag{34}\]  
\[L_{\text{oss}j}(\mathbf{w}_{1},\mathbf{w^{\prime}}_{1},\mathbf{b}_{1},b^{\prime}_{1})= \left(\mathbf{w^{\prime}}_{1}\mathbf{\hat{t}}_{1}^{j}+b^{\prime}_{1}-y^{j}\right)^{2} \tag{35}\]  
\[\text{min}L_{\text{oss}}(\mathbf{w}_{1},\mathbf{w^{\prime}}_{1},\mathbf{b}_{1},b^{\prime} _{1})=\sum_{j=1}^{S}L_{\text{oss}j} \tag{36}\]  
In Equations (34)-(36), \(\mathbf{d}^{j}\) and \(\mathbf{y}^{j}\), respectively, represent the input variable and output variable of the \(j\)-th sample. \(S\) is the total number of training samples, and \(L_{\text{oss}}\) represents the training error or loss function. By employing the stochastic-batch gradient descent method for training, we can obtain the parameters \(\mathbf{w}_{1},\mathbf{w^{\prime}}_{1},\mathbf{b}_{1},b^{\prime}_{1}\) that satisfy our objectives. Then, based on Equations (34)-(36), we can compute the output \(\mathbf{l}_{1}\) for the first hidden layer. The specific implementation steps are as follows. (1) Firstly, randomly initialize the initial values of the parameter variables \(\mathbf{w}_{1},\mathbf{w^{\prime}}_{1},\mathbf{b}_{1},b^{\prime}_{1}\). Then, perform iterative calculations using a combination of the SGD and GP methods.