# Staircase Detection, Characterization and Approach Pipeline for Search and Rescue Robots
## 3 Results
### Training of the Detector  
The configurations used and the results on the test set are shown in Tables 1 and 2. Seven different YOLOv4-tiny configurations were trained and for comparison purposes, YOLOv4 (test 2) and YOLOv3-tiny (test 3) were also trained. In the tables, each test shows the configurations and evaluation metrics obtained for the best detector in that specific run. Google Colaboratory Hosted Jupyter notebook service (colab.research.google.com) was used for training and evaluation of the models. Test number 1 was used to obtain an initial detector and, in order to improve it, in each test there was a change in parameters or data used. According to [35], detection may be improved by:  
* Selecting the appropriate number of iterations and subdivisions of the batch. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**No.** & **Total Iterations** & **Subdivisions** & **Total Layers** & **Detection Layers** & **Random Flag** & **Negative Images** & **[EMAIL_ADDRESS]** \\ \hline
1 & 2000 & 24 & 38 & 2 & 1 & NO & 59.25\% \\ \hline
2 & 2000 & 24 & 127 & 3 & 0 & NO & 76.74\% \\ \hline
3 & 4000 & 24 & 31 & 3 & 1 & NO & 45.24\% \\ \hline
4 & 4000 & 24 & 38 & 2 & 0 & NO & 66.20\% \\ \hline
5 & 2000 & 24 & 38 & 2 & 0 & NO & 63.08\% \\ \hline
6 & 4000 & 24 & 38 & 2 & 0 & YES & 65.53\% \\ \hline
7 & 10,000 & 8 & 38 & 2 & 0 & YES & 70.93\% \\ \hline
8 & 10,000 & 4 & 45 & 3 & 0 & YES & 68.17\% \\ \hline
9 & 10,000 & 8 & 45 & 3 & 0 & YES & 65.32\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Configurations of YOLOv4-tiny used and [EMAIL_ADDRESS] IoU obtained. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**No.** & **Approximate Training** & **Approximate Inference** & **Precision** & **Recall** & **F1 Score** & **Avg. IoU** & **[EMAIL_ADDRESS]** \\ \hline
1 & 40 & 5.3 & 0.76 & 0.49 & 0.59 & 55.09\% & 59.25\% \\ \hline
2 & 186 & 44.1 & 0.80 & 0.69 & 0.74 & 63.62\% & 76.74\% \\ \hline
3 & 104 & 5.6 & 0.85 & 0.17 & 0.29 & 61.86\% & 45.24\% \\ \hline
4 & 100 & 5.2 & 0.73 & 0.62 & 0.67 & 52.76 \% & 66.20\% \\ \hline
5 & 60 & 5.2 & 0.74 & 0.56 & 0.64 & 53.62\% & 63.08\% \\ \hline
6 & 75 & 5.47 & 0.73 & 0.62 & 0.67 & 54.00 \% & 65.53\% \\ \hline
7 & 175 & 5.4 & 0.81 & 0.62 & 0.70 & 61.36\% & 70.93\% \\ \hline
8 & 160 & 6.1 & 0.88 & 0.52 & 0.65 & 67.57 \% & 68.17\% \\ \hline
9 & 154 & 6.2 & 0.77 & 0.57 & 0.65 & 58.22\% & 65.32\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Test results of the different YOLOv4-tiny configurations used. Setting random flag = 1 in config file will increase precision by training YOLO for different resolutions. * To detect both large and small objects use modified YOLO versions, which include more detection stages (more YOLO layers). * Use non-labeled images during training. From the results obtained after training, it can be seen that the YOLOv4-tiny versions trained in this work were not able to match the performance (in terms of mAP) of the full size YOLOv4 version, but they were clearly increased through the use of negative images, and also by choosing the appropriate parameters for training. Furthermore, even though it is recommended to train YOLO at different resolutions (setting random flag = 1), our results showed that disabling this flag resulted in a better mAP on the test set. Considering the results in Table 2, the detectors 7, 8 and 9 (obtained in tests 7, 8 and 9, respectively) were chosen to be implemented in the different pipelines. The main differences between theses detectors is the fact that detectors 8 and 9 contain 3 detection layers and detector 7 only contains 2. Furthermore, for detector 8, a recalculation of the anchor boxes was done using the labeled dataset. On the other hand, detector 9 used the original bounding boxes specified by [35]. The three detectors were tested in the two pipelines, and it was found that the 3 YOLO layers in detectors 8 and 9 helped to detect staircases that were far from the robot, meaning that the staircase in the RGB image was small and other objects were also present.