## 3 Improved Stochastic-Batch Gradient Pile Descent
### Training Is Conducted Using the Stochastic-Batch Gradient Descent Method
In the \(k\)-th iteration of the \(j\)-th sample, the update for \(\mathbf{w}_{1}\) is as follows:  
\[\mathbf{a}_{k}^{\mathbf{w}_{1}}=-\left.\frac{\partial L_{\text{oss}j}}{\partial\mathbf{w} _{1}}\right|_{\mathbf{w}_{1}=\mathbf{w}_{1k}} \tag{37}\]  
\[\mathbf{v}_{k}^{\mathbf{w}_{1}}=\lambda_{2}\mathbf{v}_{k-1}^{\mathbf{w}_{1}}+\lambda_{3}\mathbf{a }_{k}^{\mathbf{w}_{1}} \tag{38}\]  
Figure 4.0  
where \(\mathbf{w}_{1k}\),\(\mathbf{a}_{k}^{\mathbf{w}_{1}}\), and \(\mathbf{v}_{k}^{\mathbf{w}_{1}}\) represent the values of \(\mathbf{w}_{1}\) and its negative gradient and gradient accumulation at the \(k\)-th iteration. The iterative update calculations for \(\mathbf{w^{\prime}}_{1},\mathbf{b}_{1},b^{\prime}_{1}\) can be found in Appendix B. The methods for computing gradients of parameter variables can be referred to in references [21; 22; 23; 24; 25; 26]. Figure 4.1 The pre-training structures for the first and second hidden layers. (2) Subsequently, the values obtained from step (1) for \(\mathbf{w}_{1},\mathbf{w}^{\prime}_{1},\mathbf{b}_{1},\mathbf{b}^{\prime}_{1}\) are used as initial values, and the iterative calculation is performed using the BGD method and GP method combined. At the \(t\)-th iteration, the iterative update for \(\mathbf{w}_{1}\) is as follows:  
Figure 4.2  
Figure 4.3  
Figure 4.4  
The iterative update calculations for \(\mathbf{w}^{\prime}_{1},\mathbf{b}_{1},\mathbf{b}^{\prime}_{1}\) can be found in Appendix B. After the iterative calculations are completed, we obtain the pre-training values for \(\mathbf{w}_{1},\mathbf{b}_{1}\), as well as the output values \(\mathbf{l}_{1}\) of the samples at the first hidden layer. With the pre-training of \(\mathbf{w}_{1},\mathbf{b}_{1}\) completed, we construct a three-layer neural network, as shown in Figure 5b. We use \(\mathbf{l}_{1}\) as the input for the second hidden layer and apply the same training approach as in the first layer to train the parameters of the second hidden layer's neurons. We repeat this process iteratively until the pre-training of all six hidden layers' neuron parameters is completed. After completing the pre-training of hidden layers 1 to 6, we proceed with the second step of training, which involves using the algorithm to perform unified training on all parameters. Through this process, we obtain the weight matrix parameters \(\mathbf{w}\) and bias \(\mathbf{b}\) for all the neurons in the model. Solve the Boundary of the Safety Domain by Utilizing the Stochastic-Batch Gradient Pile Descent (S-BGPD) Method  
Step One: Initialization in the simulation system involves selecting three stable operating points of the system as starting points for the search process. Taking the two-dimensional search as an example, we determine the search range and generate search directions. We perform searches in three directions, \(15^{\circ}\), \(45^{\circ}\), and \(75^{\circ}\), from the baseline operating point. We also conduct searches along the ray direction from the spatial origin to the baseline operating point. These searches enable us to obtain the initial critical points. Step Two: We use the stochastic gradient descent (SGD) method to iteratively calculate the parameter variables and apply gradient accumulation to adjust the calculation results. This process helps us find the region where the global optimum is located. Step Three: Within this region, we employ the batch gradient descent (BGD) method to perform iterative calculations and apply gradient accumulation to adjust the calculation results. We obtain the hyperplane coefficients and credible hyperplane. Step Four: Following the corresponding search methods in Step Two and Step Three, sequentially search for critical points until the number of critical points exceeds 'n'. At this point, we can employ the least squares method for fitting operations. Step Five: Validate whether the error of the obtained hyperplane meets the required criteria. If the criteria are met, proceed to Step Six. If not, return to Step Two for recalibration.