# Velocity Control of a Multi-Motion Mode Spherical Probe Robot Based on Reinforcement Learning
## 1 Introduction  
The exploration of extraterrestrial planets is widely regarded as a cutting-edge pursuit in contemporary science and technology, owing to its remarkable foresight and innovative potential. However, since the planets have much uncertainty, it is dangerous for humans to explore them directly. Therefore, replacing humans with unmanned detection equipment (such as mobile robots and detection vehicles) is the main development trend in deep space exploration missions. Due to their geometrically symmetrical structure, spherical robots have better balance and flexibility [1]. Thus, they are highly resistant to the rollover problems that wheeled or tracked robots suffer. Consequently, they have attracted many researchers and have beenwidely used in the detection of unknown environments [2], environmental monitoring [3,4], surveillance [5], and other fields [6]. Based on the single-motion mode [7, 8, 9], researchers have gradually developed spherical robots with multi-motion modes, such as rolling and jumping, rolling and crawling, and others [10, 11, 12, 13]. The multi-motion mode further expands the application of spherical robots in environmental detection tasks. During exploration missions, the spherical robot is commonly outfitted with specialized, internally contained optical functional modules designed to capture images and record video. However, owing to structural discrepancies in the MMSPR, its mass distribution is non-uniform. Any deviation in the MMSPR's movement during the rolling process could trigger an excessive motion in its rotational direction, resulting in a disruption of the robot's postural stability. This instability could severely impact the quality of photos and videos captured during exploration missions. Additionally, such motion instability can exacerbate the force exerted on the optical modules and other associated equipment, leading to reduced service life and higher chances of maintenance costs. Therefore, maintaining the motion stability of the MMSPR during its rolling process is of great significance. In addition, the MMSPR could switch to jump mode to cross obstacles while performing exploration missions. In such a situation, it is crucial for the MMSPR to brake rapidly and adjust its postures after landing to allow for user intervention for the next course of action. This prevents the MMSPR from reentering obstacles, ensuring the efficiency and safety of the task. In conclusion, the successful implementation of the MMSPR in deep space exploration hinges on addressing two critical issues: motion stability and rapid braking. The spherical robot is a strong-coupled and underactuated system with nonholonomic constraints [14], which poses considerable challenges to its control. Moreover, the un-modeled dynamics of the system and environmental measurement errors will affect the closed-loop stability of the system [15]. There are few studies on the postures control and trajectory tracking of spherical robots. The primary control algorithms used include the sliding mode algorithm [16, 17, 18], the PID algorithm [19], the fuzzy algorithm [20, 21], and others [22, 23]. [PERSON] decoupled the spherical robot into two subsystems and designed a feedback controller with fuzzy logic to achieve linear motion [24]. The fuzzy controller enhanced the system's adaptability to nonlinear characteristics and uncertainties, resulting in superior control performance compared to traditional proportion-differentiation (PD) control. [PERSON] designed a fuzzy PD controller to address the trajectory tracking issue of a spherical robot [25]. The controller uses fuzzy logic to adjust the parameters of the PD controller, which effectively eliminates the overshoot and improves the stability time, thus ensuring more accurate control of the robot's position. Considering the asymmetry of the spherical robot prototype, [PERSON] proposed an adaptive hierarchical sliding mode controller (AHSMC) [26].