## 4 Control Design  
### ADP Trajectory Control  
ADP is a newly emerging approximate optimum method in the field of optimal control, which is an important branch of machine learning. It has been applied in blast furnace gas systems [27], reusable launch vehicles [28], manipulators [29], road intersection path planning [30] and other fields, and achieved good results, especially in improving the tracking accuracy of reusable dynamic systems. The structure of the traditional ADP control system consisting of action network, critic network and model network is shown in Figure 4, and the model network is not used when dynamic model is derived. In SSS, the trajectory tracking controller, whose optimal feedback control depends only on the gradient of the optimal cost function output from the critic network obtained by online iteration [31; 32; 33], is designed based on ADP with critic network only. The controller that not only simplifies the training process, but also eliminates the approximation error between the two networks, can always improve the tracking accuracy. Considering the uncertainties and disturbances of the actual system, let \(x\ =\ [x_{1},x_{2}]^{\rm T}\ =\ [\theta,\dot{\theta}]^{\rm T}\), then SSS (30) is organized as  
\[\left\{ \begin{array}{rl}\dot{x}_{1}\ =\ x_{2}\\ \dot{x}_{2}\ =\ f(x)+g(x)\,I_{s2}\\ y\ =\ x_{1}\end{array} \right. \tag{34}\]  
where \(f(x)\ =\ -M_{1,s2}^{-1}(\theta,0)\ddot{\theta}[G_{1,s2}(\theta,\dot{\theta},0,0)+ \widetilde{A}^{-1}\widetilde{B}\dot{\theta}+\psi]\), \(g(x)\ =\ M_{1,s2}^{-1}(\theta,0)\widetilde{A}^{-1}\widetilde{C}\), and \(f(x)\) and \(g(x)\) are both Lipschitz functions. **Assumption 3**. : _The sum of uncertainties and disturbances \(\psi\) in the second slow subsystem has an unknown upper bound \(\psi^{*}\), which is \(\|\psi\|\leq\psi^{*}\)._  
Let the desired trajectory and actual trajectory of the subsystem be \(x_{d}\) and \(x\), respectively, then the trajectory tracking error is  
\[e\ =\ x\ -\ x_{d} \tag{35}\]  
We define the performance index as follows  
\[J(e(\tau))\ =\ \int_{0}^{\infty}N(e(\tau),I_{s2}(e(\tau)))d\tau \tag{36}\]  
where \(N(e(\tau),I_{s2}(e(\tau)))\ =\ e^{\rm T}Qe+I_{s2}^{\rm T}RI_{s2}\) is the utility function. There is \(N(0,0)\ =\ 0\), \(N(e,I_{s2})\ >\ 0\) holds for all \(e\) and \(I_{s2}\), and \(Q\in R^{n\times n}\), \(R\in R^{m\times m}\) are positive definite matrixes [27]. Let \(u_{d}\) denote the desired control law, then  
\[I_{s2d}\ =\ g^{+}(x_{d})\big{(}\dot{x}_{d}-f(x_{d})\big{)} \tag{37}\]  
Therefore,  
\[\dot{e}\ =\ \dot{x}-\dot{x}_{d}\ =\ f(x)-f(x_{d})+g(x)I_{s2}-g(x_{d})I_{s2d} \tag{38}\]  
For SSS, the control law \(I_{s2}\) includes two parts, desired control \(I_{s2d}\) and optimal control \(I_{s2v}\), which is  
\[I_{s2}\ =\ I_{s2d}\ +\ I_{s2v} \tag{39}\]  
Therefore,  
\[\dot{e}\ =\ f_{e}+[g(x)-g(x_{d})]I_{s2}+g(x_{d})I_{s2v} \tag{40}\]  
The optimal control \(I_{s2v}\) can ensure that the trajectory tracking error of the subsystem converges to the steady state in an optimal manner. Figure 4: Basic idea of traditional ADP. Equation (30) can be rewritten as  
\[J(e(\tau))\ =\ \int_{0}^{\infty}N(e(\tau),I_{s2v}(e(\tau)))d\tau \tag{41}\]  
where \(N(e,I_{s2v})\ =\ e^{\rm T}Qe\ +\ I_{s2v}^{\rm T}RI_{s2v}\) is the utility function, \(N(0,0)\ =\ 0\), \(N(e,I_{s2v})>0\) holds for all e and \(I_{s2v}\). \(I_{s2v}\in\Phi(\Omega)\) and \(\Phi(\Omega)\) is a set of allowable control sequences. **Definition 1**.