# Control Strategy of Speed Servo Systems Based on Deep Reinforcement Learning
## 4 Result and Discussion
### Experimental Results
Test case #5  
The tracking command signal is set with a signal amplitude of 20, a frequency of 200 Hz sinusoidal signal, an amplitude of 0.1, zero mean, variance of 1, and random white Gaussian noise signal. When tracking 0.02 s, system 1 is selected as the servo system. The torque mutation system is switched to system 3 at 0.01 s, and agent 2_2 is tested. The reinforcement learning adaptive PID current compensation method is compared with Improved_PID and Classic_PID. The MODEL_3 control and the performance comparison of the PID approaches are shown in Figure 11.  
Figure 11a presents the tracking effect diagram of 0.02 s for the reinforcement learning adaptive PID current compensation method. A good tracking performance is obtained, and the circle-part amplification is shown in the figure. From the comparison between the tracking effect (RL_PID), Improved_PID and Classic_PID tracking effect, the tracking effect of the proposed method is still better than that of the no current compensation PID control structure. Figure 11b is the error graph of the adaptive PID, Improved_PID, and Classic_PID control algorithms. The error value of the adaptive PID control algorithm is significantly smaller than that of the Improved_PID algorithm and the Classic_PID algorithm under the same system. The Improved_PID has a better PID parameter that can achieve a smaller tracking error than Classic_PID when the load is a mutation. The results prove that the presented control strategy under the condition of torque mutation has a better tracking performance than that of Improved_PID and Classic_PID. The applicability and superiority of the proposed method are verified. Test case #6  
The proposed control strategy based on reinforcement learning is compared with fixed parameter PID control performance of servo system structure under the circumstances of different signal types, signal amplitudes, torques, and inertia mutations, with its IAE as the evaluation index. For the system  
Figure 11: Result of test case #5. (**a**) Tracking signal comparison diagram; (**b**) Tracking error comparison diagram. The use of reinforcement learning cannot only make the control parameters of the servo system have fast and accurate tuning, but also can perform current online compensation on the basis of the state of the servo system. The system can effectively overcome the influences of external factors. This study proposes a servo system control strategy based on reinforcement learning in different types of instructions, torque disturbances, and inertia cases. A satisfactory performance of the control system is obtained. The experimental results demonstrate the effectiveness and robustness of the proposed method. In this paper, the tuning of PID parameters of the servo system and the compensation of current are two agents. Multi-task learning may solve this problem and this will be the topic of a future work. In addition, we find that L2 regularization can improve the generalization ability of reinforcement learning agents. The contributions in this paper will be beneficial to the application of deep reinforcement learning in a servo system. [PERSON] contributed to the conception of the reported research and helped revise the manuscript. [PERSON] contributed significantly to the design and conduct of the experiment and the analysis of results, as well as contributed to the writing of the manuscript. [PERSON] and [PERSON] helped design the experiments and perform the analysis with constructive discussions. This research is supported by the National Natural Science Foundation of China (61663011) and the postdoctoral fund of Jiangxi Province (2015 KY19). The authors declare no conflicts of interest.