## 4 Design of Controller  
The controller consists of a PMU, an ASN, and an ACN. Since the primary reinforcement signal is used to design the control torque directly, it easily leads to failure of stability control. Therefore, the controller obtains the primary reinforcement signal through the PMU, in turn using the ACN to construct a more informative signal than the primary reinforcement alone in order to tune the ASN realize stability control of the hybrid system. The control based on fuzzy logic RL is shown in Figure 4.  
**Property 1**. : _The matrix \(\dot{\mathbf{M}}_{\rm h}(\mathbf{q}_{\rm h})\) and \(\mathbf{H}_{\rm h}(\mathbf{q}_{\rm h},\dot{\mathbf{q}}_{\rm h})\) are skew symmetric matrix such that \(\frac{1}{2}\mathbf{z}^{\rm T}\dot{\mathbf{M}}_{\rm h}(\mathbf{q}_{\rm h})\mathbf{z}=\mathbf{z}^{ \rm T}\mathbf{H}_{\rm h}(\mathbf{q}_{\rm h};\dot{\mathbf{q}}_{\rm h})\mathbf{z}\), where \(\mathbf{z}\in\mathbf{R}^{n+1}\) is the arbitrary column vector._  
**Assumption 1**. : _The matrix \(\mathbf{M}_{\rm h}(\mathbf{q}_{\rm h})\) and \(\mathbf{H}_{\rm h}(\mathbf{q}_{\rm h},\dot{\mathbf{q}}_{\rm h})\) are bounded such that \(\left\|\mathbf{H}_{\rm h}(\mathbf{q}_{\rm h},\dot{\mathbf{q}}_{\rm h})\dot{\mathbf{q}}_{\rm h }\right\|\leq\lambda_{\rm H}\big{\|}\hat{\mathbf{q}}_{\rm h}\big{\|}\), \(\lambda_{\rm m}\leq\left\|M_{\rm h}(\mathbf{q}_{\rm h})\right\|\leq\lambda_{\rm M}\), where \(\lambda_{\rm m}\), \(\lambda_{\rm M}\) and \(\lambda_{\rm H}\) are positive constants._  
The position error and performance measurement signal are defined as follows:  
\[\mathbf{e}=\mathbf{q}_{\rm d}-\mathbf{q}_{\rm h} \tag{14}\] \[\mathbf{S}=\dot{\mathbf{e}}+\mathbf{\Lambda}\mathbf{e} \tag{15}\]  
where \(\mathbf{\Lambda}=\text{diag}(\Lambda_{1},\Lambda_{2},\cdots,\Lambda_{n+1})\) is a diagonal matrix. The primary reinforcement signal can be obtained from Equations (14) and (15):  
\[\mathbf{r}(t)=\mathbf{L}\mathbf{\sigma}(t) \tag{16}\]  
where \(\mathbf{L}=\text{diag}(\frac{1}{\rho_{1}},\frac{1}{\rho_{2}},\cdots,\frac{1}{\rho _{n+1}}),\rho_{i}>0\), \((i=1,2,\cdots,n+1)\), \(\mathbf{\sigma}(t)=[\sigma_{1}(s_{1}(t)),\sigma_{2}(z_{2}(t)),\)\(\cdots,\sigma_{n+1}(s_{n+1}(t))]^{\rm T}\), \(\mathbf{\sigma}_{i}(s_{i}(t))=\frac{e^{\rho_{2}(t)}-e^{-\rho_{2}(t)}}{e^{\rho_{3}(t)} +e^{-\rho_{2}(t)}}\), \((i=1,2,\cdots,n+1)\). Then,  
\[\mathbf{M}_{\rm h}\dot{\mathbf{r}}=-(\mathbf{H}_{\rm h}+\mathbf{D}_{\rm Lh})\mathbf{r}-\mathbf{\tau}_{\rm h }+\mathbf{\chi}(\mathbf{x}) \tag{17}\]  
where \(\mathbf{\chi}(\mathbf{x})=\mathbf{M}_{\rm h}[\hat{\mathbf{q}}_{\rm d}+\mathbf{\Lambda}\dot{\mathbf{e}}- \text{diag}(\sigma^{2})(\tilde{\mathbf{e}}+\mathbf{\Lambda}\dot{\mathbf{e}})]+\mathbf{H}_{\rm h }(\hat{\mathbf{q}}_{\rm d}-\dot{\mathbf{e}}+\mathbf{r})\), \(\mathbf{x}=[\mathbf{q}_{\rm h},\dot{\mathbf{q}}_{\rm h},\ddot{\mathbf{q}}_{\rm h},\dot{\mathbf{q}}_{ \rm d},\dot{\mathbf{q}}_{\rm d},\dot{\mathbf{q}}_{\rm d}]^{\rm T}\) is the uncertain item in system. In order to eliminate the influence of uncertain item on control accuracy, ASN is used to estimate the controller of wavelet neural network. It is assumed that the estimation of uncertain item by wavelet neural network is as follows:  
\[\hat{\mathbf{\chi}}=\hat{\mathbf{W}}_{\text{S}}^{\text{T}}\mathbf{\psi}(\mathbf{x},\hat{l},\hat{ \mathbf{\omega}}) \tag{18}\]  
where \(\hat{\mathbf{W}}_{\text{S}}=[\hat{\mathbf{w}}_{\text{S1}},\hat{\mathbf{w}}_{\text{S2}}, \cdot\cdot\cdot,\hat{\mathbf{w}}_{\text{Sm}}]\) is the estimates of the weight matrix. Based on this, the control signals are designed as follow:  
\[\tau_{\text{h}}=\mathbf{K}_{\text{r}}\mathbf{r}+\hat{\mathbf{\chi}}+\tau_{\text{H}} \tag{19}\]  
where \(\mathbf{K}_{\text{r}}\in R^{(n+1)\times(n+1)}\) is a positive definite diagonal matrix, \(\tau_{\text{H}}=\frac{1}{K_{\text{d}}}\mathbf{r},k_{\text{d}}>0\).