# Fast Loop Closure Selection Method with Spatiotemporal Consistency for Multi-Robot Map Fusion
## 5 Experiments Evaluation
### Simulations in Synthetic Datasets  
The main goal of the first set of simulations is to demonstrate the effectiveness of the proposed method and to further qualitatively evaluate the performance of the methods on varying numbers of outliers. To simulate the scenarios of two pose graphs, we divided a complete robot trajectory released with dataset CASIL [39] into two independent segments representing pose graphs. In this way, a part of intra-robot loop closures is converted to inter-robot loop closures. Therefore we can quickly determine whether an inter-robot constraint is accurate or not as the relative pose between two robots is known. The newly inter-robot measurements that link two pose graphs were used as inliers. We added randomly generated inaccurate measurements to the pose graphs to evaluate the performance of methods under the variable number of outliers, where the number of outliers was from 0 to 250 with a step of 50. It is necessary to explain that some of the outliers are the false positive loop closures corresponding to perceptual aliasing, and the other is randomly produced with random mean and covariance. In our simulations, the  
Figure 4: Diagram of system framework. threshold \(t_{g}\) was set to 50 to distinguish cluster. Lastly, based on the selected trust subset of inter-robot measurements, the pose graphs were optimized with open-source iSAM [33] and the concept of anchor node [24]. Figure 5 presents the maps estimated by our method. Table 1 quantitatively shows the numerical results of the estimated maps shown in Figure 5. According to the results, we use three metrics to evaluate the performance of the methods in detail, including time, TPR, and FPR. In addition, there are also two related metrics, the maximum size cluster and number of clusters, to describe clusters generated by our method in Table 1. The maximum size cluster is the one that contains the most inter-robot measurements  
As the number of outliers increases, the number of clusters gradually increases to 21 first, and then reduces to 19 when the number of outliers is 250. The reason is that those originally isolated clusters are associated with gradually increasing outliers, and then multiple discrete measurements are merged into a new cluster, so the number of clusters first increases and then decreases. In terms of computation time, our method takes clusters as the basic unit to estimate the topologically correct and consistent subset, dividing the entire measurement set into multiple sub-matrix blocks for solving. Compared to existing methods for building a high-dimensional consistency matrix over the entire measurement set, our method reduces the optimization time by 82%. The TPR of our method is lower than PCM, and this case is in accordance with the assumption proposed in this paper. Our method selects the pairwise internally consistent set from the region where the two trajectories overlap the most rather than the entire measurement set. In the following simulations, we will show that this assumption is feasible and meaningful. This set of simulations is used to compare the performance of the methods in the presence of dense inter-robot measurements. We evaluate the consistency and accuracy of methods by running the manhattan dataset [40], which simulates a world with 3500 nodes and 2099 ground truth loop closures. Similarly, we split the single complete trajectory generated on a full dataset into two pose graphs. The number of inter-robot loop closures \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **Outliers** & **Ave Time (s)** & **Max Size of Cluster** & **Num of Cluster** & **TPR** & **FPR** \\ \hline \multirow{5}{*}{Our} & 50 & 0.09 & 28 & 14 & 0.35 & 0.00 \\  & 100 & 0.10 & 30 & 20 & 0.35 & 0.00 \\ \cline{1-1}  & 150 & 0.13 & 36 & 21 & 0.33 & 0.00 \\ \cline{1-1}  & 200 & 0.15 & 40 & 20 & 0.29 & 0.00 \\ \cline{1-1}  & 250 & 0.24 & 62 & 19 & 0.26 & 0.00 \\ \hline PCM & 250 & 1.35 & \multicolumn{2}{c}{–} & 0.70 & 0.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation Results: CASIL.