## 4 Two-Time Scale Control
### Slow Subsystem and the Corresponding Controller
: _The optimal approximation error \(\varsigma(\mathbf{x})\) is bounded and satisfies:_  
\[\mathbf{z}=\dot{\mathbf{e}}+\mathbf{\Lambda}\mathbf{e} \tag{28}\]2  
_where \(\varsigma_{M}\) is an unknown positive constant._  
Next, the weight adaptive law of neural network can be further designed as:  
\[\mathbf{z}=\dot{\mathbf{e}}+\mathbf{\Lambda}\mathbf{e} \tag{28}\]3  
\[\mathbf{z}=\dot{\mathbf{e}}+\mathbf{\Lambda}\mathbf{e} \tag{28}\]4  
where \(\mathbf{K}_{b}\), \(\mathbf{K}_{c}\) are positive definite diagonal matrix. \(\eta\) is a positive constant. Defining estimation error as \(\widetilde{\mathbf{W}}_{c}=\mathbf{W}_{c}-\hat{\mathbf{W}}_{c}\), and it satisfied \(\dot{\widetilde{\mathbf{W}}}_{c}=-\dot{\mathbf{W}}_{c}\). **Theorem 1**. : _For the dynamic equation of the slow subsystem (26) with unknown nonlinear terms, supposing that Assumptions 1 and 2 hold and adopting the weight adaptive law (36) and (37), the control law (32) based on reinforcement learning signals (35) can ensure that the trajectory tracking error \(\mathbf{e}\) converges to zero asymptotically._  
**Proof of Theorem 1. ** Introducing the Lyapunov function:  
\[\mathbf{z}=\dot{\mathbf{e}}+\mathbf{\Lambda}\mathbf{e} \tag{28}\]5  
Differentiating Equation (13), we have:  
\[\mathbf{z}=\dot{\mathbf{e}}+\mathbf{\Lambda}\mathbf{e} \tag{28}\]6  
Substituting Equations (34)-(37) into Equation (39) yields:  
\[\mathbf{z}=\dot{\mathbf{e}}+\mathbf{\Lambda}\mathbf{e} \tag{28}\]7  
\[\mathbf{z}=\dot{\mathbf{e}}+\mathbf{\Lambda}\mathbf{e} \tag{28}\]8  
and combing Assumption 2, Equation (40) is rewritten as:  
\[ \begin{array}{l}\dot{V}\leq-z^{\mathrm{T}}\mathbf{K}_{z}z+\|z\|_{\xi M}+\|z\|\{(W _{aM}+W_{cM}\|\mathbf{\Phi}\|^{2})\|\widetilde{\mathbf{W}}_{a}\|\\ \quad+(W_{cM}+W_{aM}\|\mathbf{\Phi}\|^{2})\left\|\widetilde{\mathbf{W}}_{c}\right\|- \eta\left\|\widetilde{\mathbf{W}}_{a}\right\|^{2}-\eta\left\|\widetilde{\mathbf{W}}_{ c}\right\|^{2})\end{array}  \tag{41}\]  
Considering the regression vector \(\mathbf{\Phi}\) is bounded, it can be set \(\left\|\mathbf{\Phi}\right\|^{2}\leq\ \widetilde{\Phi}\.\ K_{\mathrm{zm}}\) is minimum eigenvalue of \(\mathbf{K}_{z}\). Equation (41) satisfies:  
\[\dot{V}\leq-K_{z\mathrm{m}}\|z\|^{2}-\eta\|z\|\left\{\left(\left\|\widetilde{ \mathbf{W}}_{a}\right\|\!-\!a_{1}\right)^{2}+\left(\left\|\widetilde{\mathbf{W}}_{c} \right\|\!-\!a_{2}\right)^{2}-\left[a_{1}^{2}+a_{2}^{2}+\frac{\xi M}{\eta} \right]\right\} \tag{42}\]  
where \(a_{1}=\frac{W_{aM}+W_{cM}}{2\eta}\frac{\widetilde{\Phi}}{a_{2}}\), \(a_{2}=\frac{W_{cM}+W_{aM}}{2\eta}\frac{\widetilde{\Phi}}{\widetilde{\Phi}}\). To assure \(\dot{V}\leq 0\), we only require one of the following conditions:  
\[\|z\|>\eta[a_{1}^{2}+a_{2}^{2}+\frac{\xi M}{\eta}]/K_{z\mathrm{m}} \tag{43}\]  
\[\left\|\widetilde{\mathbf{W}}_{a}\right\|>a_{1}+\sqrt{a_{1}^{2}+a_{2}^{2}+\frac{ \xi M}{\eta}} \tag{44}\]  
\[\left\|\widetilde{\mathbf{W}}_{c}\right\|>a_{2}+\sqrt{a_{1}^{2}+a_{2}^{2}+\frac{ \xi M}{\eta}} \tag{45}\]  
Based on the analysis results of the above steps, and combing with the Lyapunov stability theorem, which implies that the whole closed-loop system is stable the trajectory tracking error \(\mathbf{e}\) converges to zero asymptotically from the stability analysis in Theorem 1. The proof is thus completed. \(\square\)