# Research on Dynamic Path Planning of Multi-AGVs Based on Reinforcement Learning
### Literature Review and Contribution  
Path planning is further divided into local and global planning, depending on the environment [4]. Global path optimization algorithms are suitable for static environments, as they require complete map information [5]. Local path planning algorithms collect information about the environment through sensors and dynamically generate planning tasks [6]. The main objective of AGV path planning is to select the least costly and collision-free path from the starting point to the end point. Traditional path planning methods for multi-AGVs include the artificial potential field method [7], fuzzy logic algorithm [8], genetic algorithms [9], particle swarm optimization algorithms [10], etc. Traditional path planning algorithms have good search capability, but they require accurate maps or high-precision sensors in dynamic environments, resulting in unsatisfactory actual planning results. Based on the above problems, in the field of local path planning, many scholars have applied Q-learning algorithms as the research direction because the autonomous learning capability of Q-learning algorithms can be used to accomplish pathfinding and avoid dynamic obstacles in unknown environments in the absence of accurate maps and high-precision sensors [11]. Q-learning algorithms are used to obtain the reward value of the environmental feedback through the continuous interaction between the agent and the environment, and then obtain the ideal policy by maximizing the long-term compensation. Traditional Q-learning algorithms are applicable to simple environments; however, AGVs work in a complex 3D space. For Q-learning algorithms, complex environments lead to more complex actions, resulting in huge Q-tables (a Q-table is a mapping table of actions to reward values). Because AGVs traverse the Q-table for each step forward, this leads to slow convergence and poor planning results, which is unacceptable for practical planning tasks. Scholars have made a series of improvements to alleviate this problem. [PERSON] [12] proposed an adaptive heuristic critic (AHC) algorithm, which classifies and generalizes the input information through neurons so that the continuous state can be discretized. However, AHC converges slowly and faces the problem of \"the curse of dimensionality\". [PERSON] [13] used a hierarchical reinforcement learning algorithm to reduce the number of states, but this method requires considerable prior knowledge in order to build a hierarchical structure. [PERSON] [14] proposed heuristic Q-learning based on experience replay for three-dimensional path planning (HQ), which reduces the number of useless explorations by constructing heuristic functions with RL. In summary, although many scholars have used traditional neural networks or an a priori knowledge base to optimize Q-learning algorithms so that the number of states is reduced, in the absence of training samples, a classification effect cannot be achieved; therefore, the problem of a large state space cannot be solved by Q-learning algorithms, and the algorithm optimization effect is poor. Moreover, the \"explore-and-use\" strategy is static, and the algorithm does not receive timely feedback when there are dynamic obstacles, resulting in low learning efficiency. In a dynamic environment, Q-learning algorithms obtain the same environmental information, such as static cargo containers, charging piles, walls, etc., with high probability, and different environmental information only when dynamic obstacles are present. Therefore, clustering algorithms should be used to classify dynamic environmental information and reduce the size of the Q-table as much as possible to speed up the convergence of the algorithm.