# An Intelligent Control Method for Servo Motor Based on Reinforcement Learning
## 5 Experiment and Analysis
### Comparison Experiment for Control Performance  
In each control task, the same control target is used to compare the control performance of all comparison algorithms. The total length of each test was 10,000 time steps, and each control algorithm repeated the process under 10 identical random seeds. The mean absolute error (MAE) during the test was used as a measure of the control accuracy.  
Figure 6 shows the control performance of all the control algorithms in each control task under the same target trajectory when the random seed is 0, where the horizontal coordinate represents the number of time steps in the test process, and the vertical coordinate represents the actual size of the target state quantity of the control task. By measuring the distance between the actual running state and the target trajectory of each control algorithm in the control process, we can roughly see the difference in the control effect among various algorithms. The blue lines represent the control targets, while the lines of other colors represent the actual control results of various algorithms. The smaller the gap, the stronger the ability of the algorithm to control the motor to track the reference trajectory; and the larger the gap, the weaker the ability of the algorithm to control the motor to track the target trajectory. Table 2 shows the mean and standard deviation of the mean absolute error in tests based on 10 different random seeds.  
As shown in Figure 6, for the torque control, the performance difference between control methods is minimal. This may be because the response speed of the current loop is fast, and so no algorithms can easily open the gap. In the task of speed control, it can be clearly seen that the distance between the motor running trajectory controlled by the proposed algorithm and the target trajectory is the closest. Compared with other algorithms, our algorithm has certain advantages regarding control accuracy. The difference in control accuracy between algorithms may be more highly reflected in control trajectory changes due to the slow response speed of the speed control. In general, the proposed soft actor-critic algorithm based on target trajectory coding maintains the closest distance to the target trajectory. No matter how the target trajectory changes, the proposed algorithm can respond to the change quickly and adjust the running state of the motor in time.  
As can be seen from Table 2, in both torque control and speed control, our algorithm obtains the smallest mean absolute error in all tasks, indicating that compared with the other algorithms, the proposed algorithm has the highest control accuracy under a regular control trajectory, although it has a higher standard deviation in individual tasks. This shows that the performance of the control algorithm may not be as stable as that of similar algorithms, but, in general, it achieves the best effect.  
\begin{table}
\begin{tabular}{l l l l l l} \hline
**Control Task** & **TD3-SI** & **TD3-DVI** & **DDPG-SI** & **DDPG-DVI** & **Ours** \\ \hline TC-ExtExDc & 0.029 \(\pm\) 0.002 & 0.021 \(\pm\) 0.006 & 0.022 \(\pm\) 0.004 & 0.035 \(\pm\) 0.006 & 0.016 \(\pm\) 0.002 \\ TC-PMSM & 0.320 \(\pm\) 0.014 & 0.474 \(\pm\) 0.016 & 0.312 \(\pm\) 0.012 & 0.434 \(\pm\) 0.019 & 0.308 \(\pm\) 0.014 \\ TC-SeriesDc & 0.035 \(\pm\) 0.004 & 0.048 \(\pm\) 0.002 & 0.045 \(\pm\) 0.003 & 0.029 \(\pm\) 0.004 & 0.013 \(\pm\) 0.002 \\ SC-ExtExDc & 13.509 \(\pm\) 1.351 & 16.785 \(\pm\) 1.262 & 14.827 \(\pm\) 1.316 & 18.949 \(\pm\) 1.401 & 12.60 \(\pm\) 1.229 \\ SC-PMSM & 29.591 \(\pm\) 4.053 & 38.271 \(\pm\) 4.231 & 32.670 \(\pm\) 4.122 & 43.365 \(\pm\) 5.032 & 25.590 \(\pm\) 3.653 \\ SC-SeriesDc & 12.478 \(\pm\) 1.133 & 19.503 \(\pm\) 1.231 & 18.549 \(\pm\) 1.393 & 21.798 \(\pm\) 1.538 & 11.251 \(\pm\) 1.513 \\ \hline \end{tabular}
\end{table}
Table 2: Mean and standard deviation of control error.  
Figure 6: The control performance on six tasks.