## 4 Experimental Evaluation
### Results and Analysis  
In Graph-STNN, the first MLP only maps the initial feature into high-dimensional space, so we use one fully connected layer for reducing computational complexity. Similarly, the output MLP maps the hybrid features into the evaluation of each frontier. It is constructed from one linear layer with Gaussian noise (adding uncertainty to training and improving training efficiency) and one fully connected layer. In recurrent neural networks, the addition layer increases the level of abstraction of input observations over time [36]. However, the increase in layers will lead to an exponential growth in time and memory overhead and a sharp decline in convergence effect and efficiency. Therefore, combined with our actual computing capacity, we design the state encoder as a two-layer GRU. Finally, to determine the structure of the spatial encoder, we test one to three layers of GCN, and the training performance is shown in Figure 7. The test results are shown in Figure 8. As we see, Graph-STNN with a two-layer GCN has the best performance with the fast convergence, the minimum trajectory error, and the highest efficiency. Figure 6: An example of our simulation environment, in which the exploration range is 20 m \(\times\) 20 m, the light blue circular area the robot detection range, the light gray grid the unknown area, the gray grid the obstacle grid, the ellipse in the grid the virtual landmark uncertainty, the black plus the actual position of the landmark in the map, the yellow plus the estimated landmark position by the robot, the black trajectory the robot’s actual trajectory, the green trajectory the robot’s estimated trajectory, the green ellipse the robot’s pose uncertainty, the purple point the candidate frontier waypoints (action space), and the red point the selected frontier waypoint (action). Figure 7: The training performance of Graph-STNN with different-layer GCN in the spatial encoder in our setting with the size of 20 m \(\times\) 20 m.  
The entropy-based method [15] and the uncertainty-based methods [25] are the most popular traditional methods in this field, which perform well in autonomous exploration. Very recently, the learning-based method proposed in reference [8] achieved great progress on exploration performance, which is also inspired us. Therefore, to verify our approach, we compare our methods with them. Specifically, we train the GCN network proposed in reference [8] in the same experimental setting (with obstacles) as a comparison, which is denoted as GCN-obs for shorting. In addition, we also test the original GCN network of [8] in our experimental environment, which is trained in the environment without obstacles and denoted as GCN-no_obs. The following figure shows the convergence of rewards between Graph-STNN and the GCN-obs under the same reward function settings. The rewards are averaged every ten thousand steps. As shown in Figure 9, the proposed Graph-STNN is able to converge more quickly. Figure 8: The test results of Graph-STNN with different-layer GCN in the spatial encoder of 50 exploration trials, with the same randomly initialized landmarks and robot start locations, on 20 m \(\times\) 20 m maps (the results shown are plotted per time-step of the simulation). (**a**) One of the test result of 50 exploration trials. (**b**) Average exploration rate (left), trajectory error (middle), and exploration efficiency (right). Figure 9: The training performance of Graph-STNN and GCN-obs in our setting with the size of 20 m \(\times\) 20 m.  
We test the trained strategies in randomly generated maps with sizes 20 m \(\times\) 20 m and 40 m \(\times\) 40 m. We test 50 episodes, and the results of the comparison are shown in Figures 10 and 11. To evaluate the performance, we use three metrics: the average exploration speed, \(e\) ( the average trajectory error, which measures the mapping accuracy), and \(\eta\) (the average exploration efficiency, which measures the reduction of map uncertainty per unit time).