# Small-Scale Zero-Shot Collision Localization for Robots Using RL-CNN
## 5 Experiment Results and Discussion  
### Experiment Results  
In this work, we use the dataset described in Section 3 for the evaluation of the proposed methods. In this dataset, we only use collision point samples in different cases for training. In order to select the optimal model, the collision point samples are divided into a training set and a validation set, with a ratio of 8:2, and then the optimal parameters are selected according to the prediction accuracy of the model on the validation set. For testing, all of the zero-shot collision point samples from different cases are used, and the information from these samples will not be leaked during training. To illustrate the effectiveness of the proposed RL-CNN method, we compared the results with the baseline methods, both with and without batch normalization. For convenience, we named the baseline method using batch normalization as BN-DCNN, and the plain baseline method as DCNN. As mentioned in Section 2, since the RNN-based model has natural advantages for processing classification problems in time series, we also use LSTM, GRU, Bidirectional LSTM (BiLSTM), and Bidirectional GRU (BiGRU) as methods for comparison. All of these compared methods are trained in a similar way to the proposed method. However, it should be noted that since the compared methods only have one stage, the data for training the compared methods only needs radial region labels. To clearly describe the hyper-parameters of the mentioned methods, the relative settings are listed in Table 4.  
Table 5 shows the prediction accuracy of different radial regions using the different methods. Each result is the average accuracy of five cross-validations. Overall, the accuracy of the proposed RL-CNN is higher than the compared methods in all cases. This is especially true in case 4; the proposed method has a considerable improvement over the next most accurate method (up to 7.77%), which is also the largest gap between the first and second place across all of the cases. Moreover, the addition of the BN layer makes the model better at predicting the location of zero-shot collision points than the shallow network without using the BN layer. For example, the average prediction accuracy of the CNN-BN method is 7.01% higher than that of the CNN method. Meanwhile, it is worth noting that the effect of the CNN-based model is better than that of the RNN-based model, which means that when the structure is relatively simple, the CNN-based model has more advantages for addressing the zero-shot collision problem. The experiments were performed using a Linux-based system equipped with an Intel (R) Core (TM) i7-10700K CPU @ 3.80 GHz, 31.9 GiB Memory, and an NVIDIA GeForce RTX 3080 GPU. Table 5 also presents the run time metrics for different models. Notably, the run time was measured with a batch size of 1. Table 6 shows the F-score of each region for each different method. The results of each region are the average of the F-score from five cases. In terms of the F-score, the proposed RL-CNN is not always the best. For example, when predicting R1 samples, the CNN-BN method performs better than all other methods. In addition, although the RNN-based model has poor overall accuracy, it is more effective in locating the zero-shot collision points in some areas, such as R2 and R3. In these eight regions, the effect of all of the methods for localization of the zero-shot collision points in R4-R7 is obviously lower than that for localization of the zero-shot collision points in R1, R2, R3, and R8. Table 7 compares the localization effects of the different methods on the zero-shot collision points of the upper and lower regions. The proposed method still performs best in this binary classification task, for which the F-scores are as high as 94.53% and 93.60% for the upper region and the  
lower region, respectively. In contrast to the eight-region localization, there is a very small gap in the F-scores between the upper and lower regions. The highest gap is 0.92%, which happens when using RL-CNN.