# Data-Driven Control Algorithm for Snake Manipulator
## 3 Simulation of DDPG Control Based on 2-Link Model
### Environmental Object Establishment  
A modular design method was adopted for the snake manipulator in this study to facilitate equipment maintenance. In order to improve the flexibility of the manipulator, a 1-joint 2-degrees-of-freedom structure was adopted to enable the manipulator to move in both the yaw and pitch directions, and the three-dimensional motion of the manipulator was realized. A structure diagram of the snake manipulator model is shown in Figure 3. Due to the large number of structural joints, a 2-joint structure was used as an example in this study to establish the dynamic model. (1) Definition of input signal: In this experiment, the joint torque of the 2-link manipulator was used as the input signal of the environment object. (2) Establishing a dynamic equation to evaluate the environment: The dynamic model of the two connecting rods was established according to the Lagrangian dynamic method. The joint coordinate system of the manipulator is shown in Figure 4. The yaw angle is \(\alpha\). The pitch angle is \(\beta\). Figure 3: Structural model diagram of snake manipulator. (**A**) is the connecting rod structure and (**B**) is the universal joint structure. Figure 2: Control block diagram based on DDPG control system. Suppose that the mass of the connecting rod is concentrated at the center of mass; the inertia tensor of the connecting rod is set to 0. Therefore, according to the Lagrange formula, the kinetic energy and potential energy of each connecting rod are expressed as follows. \[E_{k1}=\frac{1}{2}m_{1}\dot{P}_{c1}^{\rm T}P_{c1}^{i} \tag{6}\]  
\[E_{p1}=m_{1}\!gL_{c1}sin(\beta_{1}) \tag{7}\]  
\[E_{k2}=\frac{1}{2}m_{2}\dot{P}_{c2}^{\rm T}P_{c2} \tag{8}\]  
\[E_{p2}=m_{2}g(asin(\beta_{1})+L_{c2}sin(\beta_{1}+\beta_{1})) \tag{9}\]  
In Formulas (6)-(9), \(E_{k1}\) denotes the kinetic energy of the first connecting rod; \(E_{p1}\) denotes the potential energy of the first connecting rod; \(P_{c1}\) denotes the velocity at the center of mass of the first connecting rod; \"\(g\)\" denotes gravitational acceleration; \"\(a\)\" represents the length of the connecting rod; and \(L_{c1}\) denotes the length of the center of mass of the first connecting rod, etc. The definitions are consistent for all formulas in this paper. \[P_{c1}=\left[ \begin{array}{c}L_{c1}cos(\alpha_{1})cos(\beta_{1})\\ L_{c1}sin(\beta_{1})\\ -L_{c1}sin(\alpha_{1})cos(\beta_{1})\end{array} \right] \tag{10}\]  
\[P_{c1}=\left[ \begin{array}{c}a*cos(\alpha_{1})cos(\beta_{1})+L_{c2}cos( \alpha_{1}+\alpha_{2})cos(\beta_{1}+\beta_{2})\\ a*sin(\beta_{1})+L_{c2}sin(\beta_{1}+\beta_{2})\\ -a*sin(\alpha_{1})cos(\beta_{1})-L_{c2}sin(\alpha_{1}+\alpha_{2})cos(\beta_{1 }+\beta_{2})\end{array} \right] \tag{11}\]  
\(\theta_{1}=\left[ \begin{array}{cc}0&\alpha_{1}&\beta_{1}\end{array} \right]^ {\rm T}\) denotes the rotation angle of joint 1. \(\dot{\theta_{1}}=\left[ \begin{array}{cc}0&\dot{\alpha_{1}}&\dot{\beta_{1}} \end{array} \right]^{\rm T}\) denotes the angular velocity of joint 1. \(\dot{\theta_{1}}=\left[ \begin{array}{cc}0&\ddot{\alpha_{1}}&\dot{\beta_{1}} \end{array} \right]^{\rm T}\) denotes the angular acceleration of joint 1. The angular value, angular velocity, and angular acceleration of joint 2 are the same as those of joint 1. Set the center of mass of the connecting rod to the center of the connecting rod, such that \(L_{c}=\frac{1}{2}a\). According to the position of the centroid point, the expressions of Formulas (6) and (8) can be obtained, which are \(P_{c1}^{\rm T}P_{c1}\) and \(P_{c2}^{\rm T}P_{c2}\) of Formulas (6) and (8), as shown in Formulas (12) and (13). \[P_{c1}^{\rm T}P_{c1}^{i}=L_{c1}^{2}cos(\beta_{1})^{2}\dot{\alpha_{1}}^{2}+L_{ c1}^{2}\dot{\beta_{1}}^{2} \tag{12}\]  
Figure 4: Joint coordinate system. \[