# Single-Parameter-Tuned Attitude Control for Quadrotor with Unknown Disturbance
## 4 Training and Testing in the Simulation
### Performance in the Simulation and Details
#### 4.2.2 Performance of the RL Controller and Training Details  
Taking the roll and pitch attitude control \((cgp\approx 66.0)\) as an example, the step response and sine function tracking performance is shown in Figure 4:  
The performance is excellent with a high convergence rate and nearly no overshoot and steady-state error. In theory, the RL controller grows close to the global optimal controller with sufficient proper training, which performs better than any other traditional controller.  
The controller in this paper has great robustness for different types of quadrotors with different CGPs. We test the performance of quadrotors with different CGPs (cgp = 6.0, 66.0, 100.0, 180.0) in Figure 5. The quadrotors start with random states (attitude angles and angular velocities) and recover to zero points with the RL controller, and this process is tested for 100 times. The network here is fully trained for 1000 episodes. All of the quadrotors perform excellently and we can conclude that the rate of convergence improves with larger CGP value.  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Parameter** & **Value** \\ \hline Mass \(m\) & 1.311 kg \\ Length of arm \(d\) & 0.24 m \\ Moment of inertia \(J_{xx}\) & \(1.762*10^{-2}\) kg\(\cdot\)m\({}^{2}\) \\ Moment of inertia \(J_{yy}\) & \(1.796*10^{-2}\) kg\(\cdot\)m\({}^{2}\) \\ Moment of inertia \(J_{zz}\) & \(2.805*10^{-2}\) kg\(\cdot\)m\({}^{2}\) \\ Lift coefficient \(C_{T}\) & \(9.138*10^{-6}\) N/(rad/s)\({}^{2}\) \\ Torque coefficient \(C_{M}\) & \(1.368*10^{-7}\) N\(\cdot\)m/(rad/s)\({}^{2}\) \\ Motor dynamic response constant \(T_{m}\) & 0.0157 s \\ Motor linear control constant \(C_{R}\) & 973.6 rad/s \\ Control scale in x and y axes \(scale_{x}scale_{y}\) & \(6.5*10^{5}\) \\ Control scale in z axis \(scale_{z}\) & \(7.3*10^{5}\) \\ CGP of x and y axes & 66.0 \\ CGP of z axis & 3.6 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Physical parameters of the quadrotor.  
Figure 4: Step response and sine function tracking performance.  
The training process improves the performance gradually, and the performance of controllers in different episodes (ep = 0, 100, 200, 1000, respectively) is tested in Figure 6. The same quadrotor (cgp = 66.0) is tested here to recover from random initial states 100 times. The controller functions better during the training process as a result.  
The reward shaping and mirror sampling method greatly accelerates the training process and improves the control quality. In Figure 7, the reward function of this RL controller is a traditional  
Figure 5: The robustness of quadrotors with different CGPs.  
Figure 6: The performance of the controller in different episodes.  
squared error function \(r\left(s\right)=-\theta^{2}-0.1\omega^{2}\), and the controller is fully trained with 1000 episodes. The performance of this RL controller without the reward shaping and mirror sampling method is not so good in the \(\pm 30^{\circ}\) step response task and sine function tracking task as compared to the RL controller in Figure 4; the response rates are too low and the steady-state errors are obvious.  
The total training process took approximately 12 min. The hyperparameters of the training process are listed in Table 2.