## 2 Related Work
### Deep Reinforcement Learning for Robot Navigation Tasks  
There has been extensive research in robot indoor navigation tasks [36; 37; 38] using SLAM. The work [39] focuses on improving the navigation accuracy of AVG in an indoor environment via SLAM, i.e., first positioning the AVG via multiple sensor readouts and then performing navigation. They improve the positioning accuracy via multiple sets of measurements and multilaterations. DRL has been investigated for the task of robot navigation in recent years. The contribution of [40] proposes virtual-to-real DRL for mapless navigation on mobile robots with continuous actions. In their work, the mobile robots acquire LiDAR observations, relative angles and distances from the current robot pose to the goal. They trained the agent using asynchronous advantage actor-critic and demonstrated both success and generalizability in new environments in Gazebo simulator [41]. However, their state and reward formulation is impractical for training directly in real environments, as they assume the knowledge of goal position and current robot pose, which are expensive to acquire in the real world. The work of [42] explores the potential of using discrete actions for navigating and they adopted the similar problem setting as [40]. In their findings, training discrete action space using double deep Q-network and PER is more efficient than a continuous action space via deep deterministic policy gradient and proximal policy optimization. However, their approach has restricted the degree of freedom in trajectory space due to the choice of discrete actions and also encounters the same problem in real application as [40]. The authors of [43] apply the similar problem formulation on a multi-agent scenario, where a swarm of robots learn to navigate to their own targets (in group formation) without colliding with each other. Despite the impressive result in simulation, the information on relative pose to goal is still required. We also see such settings in [44; 45]. Some other work [46] also proposes deep learning approaches for traffic flow prediction. Some other work implements a target-driven approach for visual navigation, where an image of the target is also provided as a part of the observation. In [47], they used a pre-trained network ResNet-50 [48] to transform both the current and target visual observation into the embedding space and afterwards mapped to policy and critic values. Their work mainly addresses the generalizability among different scenes and the learned agent demonstrates the ability to reach manifold targets in various interior environments. However, the actor outputs only four discrete high-level actions, which greatly alleviatesthe difficulty of DRL training. The work of [49] also exploits the same idea with two major improvements. Firstly, they resorted to additional auxiliary tasks, e.g., learning meaningful segmentation and reward prediction, for performance boost, where the convolutional encoders are learned end-to-end. Secondly, they mitigated partial observability by keeping a longer historical observation using long short-term memory (LSTM) [50] instead of frame stacking. The performance boost could be seen with their proposed approach. Both of the two works used asynchronous advantage actor-critic as DRL algorithm. A third category implements map-based DRL, where the map is either given or generated online. The work of [51] generates egocentric local occupancy maps for local collision avoidance via SLAM. A second component local planner proposes local goals given the final target position. This is ensued by a DRL algorithm that maps the agent's velocity, the planned local goal and local occupancy maps to 28 discrete actions. They used dueling double deep Q-network [52] with PER and randomized the number of obstacles and initial position to facilitate learning.