## 4 Simulation Results and Analysis
The \(\zeta_{1,\boldsymbol{\Delta}}^{(i)}\) and \(\zeta_{2,\boldsymbol{\delta}}^{(i)}\) are suggested to be evenly distributed among (\(\zeta_{1,1}^{(i)}\), \(\zeta_{1,\boldsymbol{\Delta}}^{(i)}\)) and (\(\zeta_{2,1}^{(i)}\), \(\zeta_{2,\boldsymbol{\delta}}^{(i)}\)), respectively. The great \(\boldsymbol{\Delta}\) and \(\boldsymbol{\Phi}\) increase the potential to well present \(e_{i}\) and \(\hat{q}_{i}\) at the cost of increasing the computational load. Therefore, the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Value (\(i\)=1)** & **Value (\(i\)=2)** \\ \hline \(k_{yi}\) & 10 & 10 \\ \(k_{zi}\) & 1 & 1 \\ \(k_{zi}\) & 0.1 & 0.1 \\ \(\hat{\Phi}_{i}\) & 0.1 & 0.1 \\ \(k_{yi}\) & 0.1 & 0.1 \\ \(\beta_{i}\) & 0.1 & 0.1 \\ \(\frac{\psi_{i}}{\Omega_{i}}\) & 0.01 & 0.01 \\ \(\frac{\delta_{i}}{\Omega_{i}}\) & 0.05 & 0.05 \\ \(\delta_{i}\) & 0.1 & 0.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Parameters of proposed controller. trial of selecting \(\alpha\) and \(\delta\) could start at some great values (e.g., 20), and then users could decrease them until a reasonable computational load._  
**Remark 17**. : \(\sigma_{i}\) _is the threshold of obtaining rewards. The inappropriately small values of \(\sigma_{i}\) could result in a difficulty in obtaining high values of reward, while the over-great values of \(\sigma_{i}\) could result in the different action candidates to be offered with the similarly high rewards even if they lead to the different control performances. Therefore, both of the over-great and over-small values of \(\sigma_{i}\) will negatively influence the convergence of \(q\)-values in (43) and thereby compromise the performance of reinforcement learning. The selection of \(\sigma_{i}\) could start at a small value, and the users could increase \(\sigma_{i}\) until a satisfying convergence of \(q\) values (the convergence of \(q\)-values can be also reflected by the convergence of obtained rewards). The selection of mutation probability \(\rho\), learning rate \(\Gamma_{i}\) and discount factor \(\alpha_{2}\) can be based on the strategy mentioned in [36]._  
**Remark 18**_Action candidates \(\mathfrak{U}_{i,\boldsymbol{n}}=\left\{\mathfrak{u}_{i,1}^{(\boldsymbol{n})},...\mathfrak{u}_{i,p}^{(\boldsymbol{n})}\ldots,\mathfrak{u}_{i,p}^{( \boldsymbol{n})}\right\}\) are the most important parameters in the proposed control scheme because the optimal \(\Lambda_{i}\) that brings up a satisfying control performance is calculated based on them. To make the optimal action included in the group of action candidates, \(\mathfrak{u}_{i,1}^{(\boldsymbol{n})}\) should be given a small value (e.g., 0) while \(\mathfrak{u}_{i,p}^{(\boldsymbol{n})}\) should be given a great value. The rest candidates \(\mathfrak{u}_{i,p}^{(\boldsymbol{n})}\) (\(p=1,2,..,p\)) are suggested to be evenly distributed between \(\mathfrak{u}_{i,1}^{(\boldsymbol{n})}\) and \(\mathfrak{u}_{i,p}^{(\boldsymbol{n})}\). Users could initially give \(\mathfrak{u}_{i,p}^{(\boldsymbol{n})}\) with a small value (but greater than \(\mathfrak{u}_{i,1}^{(\boldsymbol{n})}\)), and an increase of \(\mathfrak{u}_{i,p}^{(\boldsymbol{n})}\) until a sufficient improvement on the tracking performance is achieved. \(p\) is the amount of action candidates of each fuzzy rule for each subsystem. The selection of \(p\) could start at a great number (e.g., 20), and then users could decrease \(p\) until the computational load is acceptable._  
According to _Remark 16-Remark 18_, the parameters of the fuzzy Q learning mechanism are given as follows. The amount of fuzzy sets in (17) are \(\alpha=\delta=8\). Therefore, the amount of fuzzy rules are \(\boldsymbol{n}=64\). The parameters of membership function to do the fuzzification are: \(Lin(e_{1})=\left\{\mathfrak{z}_{1,1}^{(1)},\ \mathfrak{z}_{1,2}^{(1)},\ldots, \mathfrak{z}_{1,8}^{(1)}\right\}=\)\(\{-0.0017,\ -0.0014,\ -0.001,\ -0.0006,\ -0.0002,\ 0.0002,\ 0.0006,\ 0.001,\ 0.0014,\ 0.0017\}\).