## 2 Related Work  
Existing robotic autonomous exploration methods can be divided into two categories: the traditional approaches and the machine learning-based approaches.  
The standard pipeline for traditional approaches is based on a frontier-based exploration strategy proposed by [PERSON] et al. [11]. They find the frontier, the junction of known open areas and unknown areas on the map. The points on the frontier are served as frontier waypoints, from which the robot selects the best one as a local target and navigates to the target point autonomously by path planning. For a frontier-based exploration strategy, many approaches mainly focus on environmental uncertainty evaluation and optimal target point selection. [PERSON] et al. [12] proposed a utility function considering information gain, distance cost, and covariance matrix-based localization effect, and use the function to evaluate the candidate points of the robot. [PERSON] et al. [13; 14] proposed to compute the expected entropy gain of an occupancy grid map and further applied it to select the target points. [PERSON] et al. [15] proposed Shannon entropy for map uncertainty and Renyi entropy for robot pose uncertainty to estimate the cost of candidate points. [PERSON] et al. [16] assessed the candidate points using mutual information obtained by a Bayesian optimization algorithm, which improved computational efficiency and significantly reduced the map entropy. The frontier waypoints, however, are not the only choice. [PERSON] et al. [17] proposed to use the points with the most observation information as the local goal. However, it results in a larger search space, which increases the computational burden. Traditional approaches have achieved positive results in some specific scenarios. However, the common problem of such approaches is hard to balance exploration effectiveness and computational burden as the exploration area increases. Moreover, these methods highly depend on human-designed experts' experience. While the environment is changing, the algorithm needs to be redesigned.  
The learning-based approaches employ machine learning to replace the expert algorithm for target selection. [PERSON] et al. [9] measured the quality of mapping using Shannon entropy and defined the action space on a grid map. The candidates in the action space are evaluated by DRL using a fully convolutional Q-network. [PERSON] et al. [10] combined traditional boundary-based exploration methods with DRL to choose the best target points. Some other learning frameworks focus on learning the exploration actions in an end-to-end fashion. For instance, [PERSON] et al. [5] employed CNN-based DRL to learn the exploration strategy, which is obtained end-to-end from RGBD camera data. Furthermore, [PERSON] et al. [6] directly used a neural network to learn the mapping from the maps created during exploration to actions. [PERSON] et al. [7] used the distances in ten directions, the targets, and previous actions as the inputs to obtain the robot's velocities. Usually, these learning-based methods may afford high calculation burdens and cannot be generalized to new environments with different obstacles, scales, etc.  
Recently, GNN shows powerful capabilities in solving problems in non-Euclidean spaces [18]. Since the exploration problem is closely related to the spatial structure of the environment, [PERSON] et al. [8] firstly proposed a generalized graph representation for mobile robot exploration and solved the exploration problem by using Deep Q-Learning (DQN) [19] with graph convolutional network (GCN) [20] as the policy and/or value networks. In addition, recurrent neural networks (RNNs) have achieved great success in processing sequential data [21], such as trajectory prediction [22; 23] and speaker identification [24]. Inspired by them, based on the exploration graph constructed by a more general environment with obstacles, we propose a novel network to extract spatiotemporal information to generate a more effective exploration strategy.