# Applying Reinforcement Learning for AMR's Docking and Obstacle Avoidance Behavior Control
## 2 Deep Q-Network Methodology  
### Localization and Differential Wheel Control  
Navigation technology and machine learning have rapidly developed in recent years. The most common kinematics model for mobile robots is differential control, as depicted in Figure 1. The equations for differential control, centered on the two-wheel axis, are shown in (1)-(3). The distance from the center of the arc motion trajectory to the wheel centers is described by Equations (4)-(6), accomplished through odometry and Map-Server synchronization. The motion equations translate odometry data to the robot's coordinates, as shown in Equation (5). Figure 1: Differential control. Current SLAM technologies heavily rely on cameras and LiDAR sensors for environmental perception and map construction. Common mapping methods include Hector SLAM, Gmapping SLAM, Cartographer SLAM, and ORB-SLAM, with ORB-SLAM specifically designed for visual SLAM. Self-localization techniques, such as odometry, accumulate positional data using wheel encoders [6; 7]. For navigation and localization, Adaptive Monte Carlo Localization (AMCL) is a widely adopted approach that utilizes particle filtering to estimate the robot's pose by maintaining and updating a set of weighted particles representing possible locations. The AMCL algorithm dynamically adjusts the number of particles based on localization uncertainty, refining the robot's pose estimation. Moreover, LiDAR sensors improve localization accuracy by offering high-resolution environmental perception. Path tracking and stabilization are crucial for ensuring the accuracy and efficiency of autonomous navigation. The authors of [8] demonstrated that an effective path-tracking algorithm significantly improves the stability and accuracy of autonomous guided vehicles (AGVs). This study integrates machine learning with Monte Carlo Localization (MCL), along with the Deep Q Network (DQN) algorithm, to enhance the robot's trajectory refinement and navigation precision. Specifically, MCL provides a probabilistic estimation of the robot's pose, ensuring a stable localization foundation, while the DQN model learns optimal navigation strategies based on input velocity and angular adjustments. By combining probabilistic inference with reinforcement learning-based decision-making strategies, this approach effectively enhances the robot's trajectory tracking and navigation accuracy. The robot's average speed. \(v\_r\) and \(v\_l\) are the speeds of the right and left wheels respectively. By adding the speeds of the two wheels and dividing by 2, we obtain the robot's average speed. \[\mathbf{v}=(\mathbf{v\_r}+\mathbf{v\_l})/\mathbf{2} \tag{1}\]  
The difference in speed between the two wheels is related to the robot's wheelbase (the distance between the two wheels). \(\mathbf{w}\) is the robot's angular velocity, and \(\mathbf{l}\) is the robot's wheelbase. This formula indicates that the greater the difference in speed between the two wheels, the greater the robot's angular velocity. \[\frac{\mathbf{v\_r}}{\mathbf{w}}-\frac{\mathbf{v\_l}}{\mathbf{w}}=\mathbf{l} \tag{2}\]  
The robot's turning radius. The turning radius is equal to the ratio of the robot's average speed to its angular velocity. When the speeds of the right and left wheels are different, the robot moves along an arc, and the radius of this arc is the turning radius. \[\mathbf{\omega}=(\mathbf{v\_r}-\mathbf{v\_l})/\mathbf{l} \tag{3}\]  
The robot's turning radius \(r\). The turning radius is equal to the ratio of the robot's average speed to its angular velocity. When the speeds of the right and left wheels are different, the robot moves along an arc, and the radius of this arc is the turning radius. \[\mathbf{r}=\frac{\mathbf{v}}{\mathbf{\omega}}=\frac{(\mathbf{l}\times( \mathbf{v\_r}+\mathbf{v\_l}))}{(\mathbf{2}\times(\mathbf{v\_r}-\mathbf{v\_ l}))} \tag{4}\]  
The robot's displacement in the x and y directions are described below, where is the robot's forward speed, \(\theta_{t}\) is the current direction angle, and \(\Delta t\) is the time step.