## 3 Macro-Control and Micro-Autonomy Pathfinding Strategy
### Distributed Planning Strategy
#### 3.3.2 \(\varepsilon\)-DAgger Imitation Learning Strategy  
\(\varepsilon\)-DAgger Imitation Learning is based on Dataset Aggregation (DAgger) [45] and uses a parameter \(\varepsilon\) to control whether to use expert demonstrations (A* pre-planned path) for training. \(\varepsilon\)-DAgger blends imitation learning with reinforcement learning, dynamically interleaving them during training. In training, when an AGV selects an action, it performs the following decision-making process. 1. With probability \(1-\varepsilon\): It executes the standard reinforcement learning process and uses \(\varepsilon\)-greedy policy to select an action; with probability \(\varepsilon\), it selects a random action from the action space; with probability \(1-\varepsilon\), it selects the best action according to the evaluation network. Consequently, the overall probability that the final model takes a random action is \(\varepsilon(1-\varepsilon)\). 2. With probability \(\varepsilon\): It imitates the expert demonstrations (A* pre-planned path), uses A* to plan path, and chooses the action from the planned path. After receiving feedback from the environment, the expert demonstrations will be stored in the experience pool for learning. In the early stage of training, AGVs heavily rely on A* to avoid random inefficient movements. As training progresses, AGVs increasingly rely on their own learned policy, gradually surpassing the performance of A*. Compared to traditional DAgger, \(\varepsilon\)-Dagger avoids overfitting to the expert strategy and can adapt to flawed expert strategy. It allows AGVs to discover more efficient paths while still benefiting from expert guidance. The AGV action space adopts the previously mentioned AGV-referenced action space. It is defined as follows: move forward, rotate 90\({}^{\circ}\) left, rotate 90\({}^{\circ}\) right, rotate 180\({}^{\circ}\) back, and then wait. If an AGV attempts an invalid action (such as colliding with an obstacle), the action will be rolled back and replaced with a \"wait\" action. The limited field of view mechanism simulates the constraints of the AGV's LIDAR detection range, reduces the dimensionality of the input data, facilitates the generalization of the policy to maps of any size, and helps the model focus on resolving local AGV conflicts while minimizing interference from irrelevant factors. The observation space is primarily composed of information from the AGV's own LIDAR detection, the segmentation of the global map within a limited field of view, and communication with the neighboring AGVs. Specifically, the observation representation consists of the following four feature layers, as shown in Figure 5:  
1. Goal Layer: This layer represents the relative position of the key waypoint that the AGV must reach. Using the AGV's own frame of reference, the matrix center corresponds to the AGV's location. If the waypoint lies within the AGV's nearby cells, the corresponding matrix entry is set to 1; otherwise, it remains at 0. 2. Obstacles Layer: This layer depicts all potential obstacles around the AGV by fusing the global map provided by the central system with the AGV's LiDAR data. Cells containing obstacles have a value of 1, and free cells are 0. In addition, the loaded AGVs treat shelves as obstacles, whereas the unloaded AGVs freely pass beneath shelves, which are then not considered obstacles. 3. A* Pre-Planned Path Layer: This layer encodes the pre-planned path from the central system to guide the AGV. Starting from the center of matrix (the AGV position) and extending outward, each cell on the planned path is assigned the number of time steps required to reach it. If a single grid cell appears in multiple time steps (e.g., during a turn), the maximum value is used. 4. Neighbor AGV Layer: This layer conveys the relative positions and expected trajectories of the surrounding AGVs. Similarly to the A* Pre-Planned Path Layer, each neighboring AGV's A* pre-planned path radiates outward from its current position.