## 3 Controller Design and Stability Analysis
### Stability Analysis
\begin{array}{c}0\,\ if\big{(}\hat{d}_{i}\leq 0 \big{)}\ or\ \big{(}\Omega_{i}\geq\overline{\Omega}_{i}\big{)}\\ \\ \dot{v}_{i}\Big{(}\Gamma_{i}^{*}-\hat{d}_{i}\Big{)},\ if\big{(}\hat{d}_{i}>0 \big{)}\ and\ \big{(}\Omega_{i}<\overline{\Omega}_{i}\big{)}\end{array} \right. \tag{19}\]  
where \(\mathfrak{r}_{i}=\big{|}\dot{q}_{i}\big{|}\big{(}1+k_{yi}C_{i}\big{)}+\frac{ \dot{q}_{i}}{\big{|}\big{(}1+k_{yi}C_{i}\big{)}}\). Clearly, \(\mathfrak{r}_{i}\geq 2\sqrt{\mathfrak{r}_{i}}>0\) holds. In addition, the \(V_{i}\) defined in (16) can be written as:  
\[V_{i}=\Omega_{i}+\frac{1}{2\psi_{i}}\dd_{i}^{2} \tag{20}\]  
where \(\Omega_{i}\) is defined in (14). To prove the Lyapunov function, \(V_{i}\) is bounded by a positive constant and we assume a sufficiently large constant \(V_{i}^{*}\). Clearly, the sufficiently large \(V_{i}^{*}\) requires at least one of the terms (\(\Omega_{i}\) or \(\dd_{i}^{2}\)) to be sufficiently large. If \(\Omega_{i}\) is sufficiently large such that \(\Omega_{i}>\overline{\Omega}_{i}\), then \(\dot{V}_{i}\leq 0\) holds according to (19). If \(\dd_{i}^{2}=\big{(}\Gamma_{i}^{*}-\hat{d}_{i}\big{)}^{2}\) is sufficiently large such that \(\dd_{i}^{2}>(2\Gamma_{i}^{*})^{2}\), then \(\dot{d}_{i}>3\Gamma_{i}^{*}\) will hold because of the facts of \(\Gamma_{i}^{*}>0\) (Lemma 1) and \(\hat{d}_{i}\geq 0\) (remark 3), which further means \(\Gamma_{i}^{*}-\hat{d}_{i}<0\) holds. As a result, according to (19) and the fact of \(\mathfrak{r}_{i}\geq 2\sqrt{\delta_{i}}>0\), \(\dot{V}_{i}\leq 0\) can hold by \(\Omega_{i}>\overline{\Omega}_{i}\) or \(\dd_{i}^{2}>(2\Gamma_{i}^{*})^{2}\), which means \(\dot{V}_{i}\leq 0\) holds if \(V_{i}\geq\overline{\Omega}_{i}+\frac{1}{2\Psi_{i}}(2\Gamma_{i}^{*})^{2}\) holds. It is thereby easy to conclude that \(V_{i}\) is bounded by \(V_{i}^{*}\) such that \(V_{i}\leq V_{i}^{*}=\max\Big{\{}V_{i}(0),\ \overline{\Omega}_{i}+\frac{1}{2\Psi_{i}}(2 \Gamma_{i}^{*})^{2}\Big{\}}\). Therefore, the tracking error of the \(i^{th}\) subsystem is bounded because of the fact of \(\frac{1}{2}k_{pi}\dot{e}_{i}^{2}\leq V_{i}\). \[|e_{i}|\leq\sqrt{\frac{2}{k_{pi}}}\max\{V_{i}(0),\ (\overline{\Omega}_{i}+\frac{1}{2\psi_{i}} \big{(}2\Gamma_{i}^{*})^{2})i=1,2..,n \tag{21}\]  
Then, the norm of the vector of the tracking error is thereby bounded. \[||e(t)||\leq\sqrt{\sum\limits_{i=1}^{n}\frac{2}{k_{pi}}}\max\{V_{i}(0),[ \overline{\Omega}_{i}+\frac{1}{2\psi_{i}}\big{(}2\Gamma_{i}^{*})^{2}] \tag{22}\]Moreover, the bounded \(V_{i}\) implies all terms in \(V_{i}\) are bounded. The (23) therefore holds, which means \(|q_{i}|<\varepsilon_{i}\) and \(|\dot{q}_{i}|<\Lambda_{i}\) hold. \[\frac{\varepsilon_{i}^{2}z_{i}^{2}}{2(\varepsilon_{i}^{2}-q_{i}^{2})},\ \frac{\dot{q}_{i}^{2}}{ \Lambda_{i}^{2}-\dot{q}_{i}^{2}}\in L_{\infty}i=1,2,\ldots,n \tag{23}\]  
\(\square\)  
_3.3. Fuzzy \(Q\) Reinforcement Learning Mechanism Determining Parameters of Controller_  
In this section, a fuzzy \(Q\) reinforcement learning mechanism is designed to tune the parameter \(\Lambda_{i}\) to improve the tracking accuracy. The motivations of RL are detailed in remark 7. **Lemma 2. **_If (24) holds, the tracking error \(e_{i}\) defined in (3) will asymptotically converge to zero with the converging rate satisfying \(\big{|}\dot{q}_{i}\big{|}<\Lambda_{i}\)._  
\[0=\dot{q}_{i}+\Lambda_{i}tanh\bigg{(}\frac{e_{i}}{\dot{\xi}_{i}}\bigg{)},\ i=1,2,..n \tag{24}\]  
**Proof. ** A simple Lyapunov function is given in (25). \[\mathcal{V}_{i}=\frac{1}{2}e_{i}^{2} \tag{25}\]  
Combining (24) and derivative of (25), we can obtain (26). \[\dot{\mathcal{V}}_{i}=-\Lambda_{i}e_{i}tanh\bigg{(}\frac{e_{i}}{\dot{\xi}_{i} }\bigg{)}\leq-\Lambda_{i}\dot{\xi}_{i}tanh^{2}\bigg{(}\frac{e_{i}}{\dot{\xi}_ {i}}\bigg{)}\leq 0 \tag{26}\]  
\(\dot{\mathcal{V}}_{i}\leq 0\) implies \(\mathcal{V}_{i}\in L_{\infty}\). And then, (27) can be obtained by integration on both sides of (26).