# Control Strategy of Speed Servo Systems Based on Deep Reinforcement Learning
## 2 Model Analysis of a Speed Servo System  
The speed servo system consists of current control loop, servo motor, load, and feedback system. According to the analysis of the component of speed loop, the current closed loop, motor, and speed detection are assumed as first-order systems. Using the PI control algorithm employed as a speed regulator, the speed servo system structure diagram can be obtained in Figure 1.  
Where \(K_{s}\) is the proportional gain of current loop. \(T_{s}\) is the integral time constant. \(K_{t}\) is the servo system torque current coefficient. \(K_{s}\)\(=60m/p_{f}T\) represents the speed feedback coefficient, \(m\) is the number of detected pulses, \(p_{f}\) represents the resolution of encoder, \(T\) is the speed detection cycle. \(2T_{\Sigma}\) is the time constant of current closed-loop approximate system. \(T_{sf}\) is the time constant of speed feedback approximate system. \(K\) is the stiffness coefficient of mechanical transmission. \(B\) is the conversion coefficient of load friction. When the impact of resonance on a servo system is ignored, the motor rotor and load object can be assumed as a rigid body and the inertia of the servo system can be expressed as \(J=J_{m}+J_{L}\). Where \(J_{m}\) represents motor inertia, \(J_{L}\) is load inertia. The external load torque of the servo system is assumed to be \(T_{L}\), and the speed-loop-controlled model can be expressed as:  
\[G_{so}(s)=\frac{K_{t}K_{sf}}{((J_{m}+J_{L})s+B)(T_{\Sigma n}s+1)}. \tag{1}\]  
where \(T_{\Sigma n}=2T_{\Sigma}+T_{sf}\). Equation (1) can be rearranged into:  
\[G_{so}(s)=\frac{B_{1}}{A_{1}s^{2}+A_{2}s+A_{3}} \tag{2}\]  
where \(A_{1}=(J_{m}+J_{L})T_{\Sigma n}\), \(A_{2}=(BT_{\Sigma n}+(J_{m}+J_{L}))\), \(A_{3}=B\), and \(B_{1}=K_{t}K_{sf}\). The control time interval is used to discrete Equation (2), and the transfer function \(z\) of the controlled object of the speed servo system is obtained as follows:  
Figure 1: Diagram of the speed servo system structure. \[G_{so}(z)=\frac{b_{1}z^{-1}+b_{2}z^{-2}}{1+a_{1}z^{-1}+a_{2}z^{-2}}. \tag{3}\]  
Equation (3) can be rewritten into the following system difference equation for convenience:  
\[\omega(t)=-a_{1}\omega(t-1)-a_{2}\omega(t-2)+b_{1}i_{q}(t-1)+b_{2}i_{q}(t-2) \tag{4}\]  
where \(\omega(t)\) is the speed at time \(t\), \(i_{q}\) represents the q axis component of the current. The load torque disturbances can also be described by:  
\[\Delta\,\omega(k)=\Delta t\times\Delta T_{L}/(J_{m}+\Delta J_{L}). \tag{5}\]  
Equation (5) presents that the speed fluctuation caused by changes in load torque is associated with the system inertia and load torque. A small inertia indicates great impacts of the same load torque fluctuations on the speed control process. The system inertia and load torque should be carefully considered during the servo system operation to obtain a stable velocity response. For example, during the movement of the arm of a robot, its inertia is constantly changing. Thus in the CNC machining process, the quality of the processed parts will reduce, which will lead to the change of converted inertia of the feeding axis. When the characteristics of the load object changes, the characteristics of the entire servo system will change. If the system inertia increases, the response of the system will slow down, which is likely to cause system instability and result in climb. On the contrary, if the system inertia decreases, dynamic response will speed up with speed overshoot as well as turbulence. According to Equations (3)-(5), the relevant parameters of a specific type of servo system are given as follows: \(T_{\Sigma n}\) of \(1.25\times 10^{-4}\), motor inertia of \(6.329\times 10^{-4}\) kg \(\cdot\) m\({}^{2}\), and friction coefficient of \(3.0\times 10^{-4}\) N \(\cdot\) m \(\cdot\) s /rad.