# Velocity Control of a Multi-Motion Mode Spherical Probe Robot Based on Reinforcement Learning
## 2 Materials and Methods
### Dynamic Modeling  
The dynamic model of MMSPR in this paper is based on the following assumptions:  
* During movement, MMSPR rolls without slipping, and its mass is decomposed into the mass of the shell and the pendulum: \(M_{s}\), \(m_{p}\);
* The centroid of MMSPR coincides with its geometric center;
* The two degrees of freedom do not mutually interfere, and the longitudinal axis does not rotate.  
Figure 1: The structure schematic of the MMSPR: (**a**) structural diagram of MMSPR; (**b**) simplified model of MMSPR.  
The schematic diagram of MMSPR during rolling is shown in Figure 2a. Specifically, \(O_{A},O_{B}\), and \(O_{C}\) represent the coordinate systems \(O_{A}-X_{A}Y_{A}Z_{A}\), \(O-X_{B}Y_{B}Z_{B}\), and \(O-X_{C}Y_{C}Z_{C}\), and \(O_{A}\) is the world coordinate system. \(O_{B}\) is the coordinate system fixed at the geometric center of the sphere, which can only translate relative to \(O_{A}\). Additionally, \(O_{C}\) is the coordinate system fixed at the geometric center of the sphere, which can only rotate relative to \(O_{A}\). \(R\) and \(L\) represent sphere's radius and the pendulum's length, respectively. \(\theta\) and \(\phi\) represent the rolling angles of MMSPR along its \(X_{B}\) and \(Y_{B}\). \(\alpha\) and \(\beta\) refer to the rotation angles of the pendulum around the \(X_{B}\) and \(Y_{B}\). \(\tau_{x}\) and \(\tau_{y}\) are the driving torques acting on the \(X_{B}\) and \(Y_{B}\).  
To facilitate the controller design, MMSPR's system needs to be decoupled. Referring to Kayacan [24], we decoupled the robot into the forward motion and the steering motion (Figure 2b,c). According to the Lagrange equation, taking the state variable \(x=\begin{bmatrix}\theta&\alpha&\phi&\beta\end{bmatrix}^{T}\), the dynamic model of MMSPR can be formulated as:  
\[\begin{gathered} M(x(t))\ddot{x}(t)+V\big{(}x(t),\dot{x}(t)\big{)}=u (t)\\ \begin{bmatrix}M_{11}&M_{12}&M_{13}&M_{14}\\ M_{21}&M_{22}&M_{23}&M_{24}\\ M_{31}&M_{32}&M_{33}&M_{34}\\ M_{41}&M_{42}&M_{43}&M_{44}\end{bmatrix}\begin{bmatrix}\dot{x}_{1}\\ \dot{x}_{2}\\ \dot{x}_{3}\\ \dot{x}_{4}\end{bmatrix}+\begin{bmatrix}V_{1}\\ V_{2}\\ V_{3}\\ V_{4}\end{bmatrix}=\begin{bmatrix}\tau_{x}\\ \tau_{y}\\ \tau_{y}\\ \tau_{y}\end{bmatrix}\end{gathered} \tag{1}\]  
Figure 2: Establishment of spherical robot coordinate system: (**a**) spatial coordinate system of spherical robot; (**b**) projection of coordinate system on \(Y_{A}Z_{A}\) plane; (**c**) projection of coordinate system on \(X_{A}Z_{A}\) plane.  
where:  
\[\begin{array}{ll}M_{11}=&M_{s}R^{2}+m_{p}R^{2}+m_{p}L^{2}+I_{s}+I_{p}+2m_{p}RLcos( x_{2}-x_{1}),\\ M_{12}=&-m_{p}L^{2}-I_{p}-m_{p}RLcos(x_{2}-x_{1}),\\ M_{13}=&M_{14}=0,\\ M_{21}=&-m_{p}L^{2}-I_{p}-m_{p}RLcos(x_{2}-x_{1}),\\ M_{22}=&m_{p}L^{2}+I_{p},\\ M_{23}=&M_{24}=M_{31}=M_{32}=0,\\ M_{33}=&M_{s}R^{2}+m_{p}R^{2}+m_{p}L^{2}+I_{s}+I_{p}+2m_{p}RLcos(x_{4}-x_{3}), \\ M_{34}=&-m_{p}L^{2}-I_{p}-m_{p}RLcos(x_{4}-x_{3}),\\ M_{41}=&M_{42}=0,\\ M_{43}=&-m_{p}L^{2}-I_{p}-m_{p}RLcos(x_{4}-x_{3}),\\ M_{44}=&m_{p}L^{2}+I_{p},\\ V_{1}=&m_{p}RLsin(x_{2}-x_{1})\dot{x}_{1}^{2}+m_{p}RLsin(x_{2}-x_{1})\dot{x}_{2 }^{2}\\ &-2m_{p}RLcos(x_{2}-x_{1})x_{1}x_{2}-m_{p}gLsin(x_{2}-x_{1}),\\ V_{2}=&m_{p}gLsin(x_{2}-x_{1}),\\ V_{3}=&m_{p}RLsin(x_{4}-x_{3})\dot{x}_{3}^{2}+m_{p}RLsin(x_{4}-x_{3})\dot{x}_{4 }^{2}\\ &-2m_{p}RLsin(x_{4}-x_{3})x_{3}x_{4}-m_{p}gLsin(x_{4}-x_{3}),\\ V_{4}=&m_{p}gLsin(x_{4}-x_{3})\\ I_{s}=&\frac{2}{3}M_{s}R^{2},I_{p}=m_{p}L^{2}\end{array} \tag{2}\]  
where \(I_{s}\) and \(I_{p}\) are the moments of inertia for the sphere and the pendulum, respectively. \(\tau_{x}\) and \(\tau_{y}\) represent the input torques on the \(X_{B}\) and \(Y_{B}\) axes, respectively.  
It is worth noting that the \(X_{B}\) is perpendicular to the ground when \(\beta\) equals \(\pi/2\) or \(-\pi/2\), which causes the robot to rotate in place. For this reason, the range of \(\beta\) in the model is limited to \((-\pi/3,\,\pi/3)\).