# Staircase Detection, Characterization and Approach Pipeline for Search and Rescue Robots
## 3 Results
### Training of the Detector
Additionally, the recalculated bounding boxes improved [EMAIL_ADDRESS], but decreased recall. Considering the results obtained, detector 9 was chosen to be used in our proposed pipeline. The images in Figure 7 show the predictions obtained using the detector in test 9 using images taken from the test set. From the images, it can be seen that the detector can make correct predictions for ascending and descending staircases with different textures and viewing angles and even staircases whose step edges are not completely parallel. Figure 7 also shows that multiple staircases in an image can be detected and staircases that are partially occluded by fences or handrails. Before testing the complete pipeline on a physical robot, we implemented it in simulation, mainly to adjust the clustering parameters and to obtain initial parameters for the membership functions that represent each fuzzy set. The use of simulation in this work  
Figure 7: Results obtained using detector 9 in images taken from test set. also allowed us to test different fuzzy rules and helped us obtain the fuzzy controller implemented on the physical robot. Figure 8 shows the staircase detection and segmentation of smooth regions on a simulated point cloud and the path the robot takes to arrive at the first step of the staircase. Using the deep learning model obtained in test 9, we proceeded to test both of the pipelines described. For both tests we mounted the camera on top of the differential drive robot and placed it in front of a staircase. The results obtained with the D435i pipeline are shown in Figure 9. From Figure 9 it is possible to observe that the pipeline is able to detect the staircase in the RGB image, obtained from the D435i camera, and using the bounding box we extract the points, from the initial point cloud (points in white), corresponding to the ROI (points in green). Due to the quality of the point cloud, mainly dependent on the disparity levels and the use of an infrared pattern projector, the segmentation was done using only region growing and Figure 9 shows the planes extracted (red polygons). Figure 10 shows the results obtained using the OAK-D camera pipeline. Figure 10 shows the detection obtained using the NCEs of the OAK-D camera and Figure 10 shows the initial point cloud (white points), the points corresponding to the ROI (green points) and the smooth regions obtained after the region growing algorithm (yellow polygons). Compared to the D435i camera, the point cloud is rather noisy and has less levels of disparity, making it necessary to implement the smooth region clustering algorithm. Figure 10 shows the clustered smooth regions (red polygons) and, therefore, the segmented staircase risers. Considering all the components in the staircase detection and characterization pipeline, the staircase parameters were published at a rate of approximately 2 Hz for both the D435i and OAK-D camera. Also, taking into account the low computing resources available in the test robot (it only contains a second generation Intel(r) Core(tm) i5 CPU @ 2.60 GHz), it was decided to use the OAK-D pipeline to test the fuzzy logic controller. The fuzzy logic controller used the staircase parameters to perform fuzzy inference and publish linear and  
Figure 8: Pipeline test in simulation: (**a**) detection, (**b**) smooth region segmentation, (**c**) approach and alignment using Fuzzy controller. Figure 9: Results obtained using the D435i camera pipeline: (**a**) staircase detection, (**b**) smooth region segmentation. angular velocity commands, needed by the differential drive robot to approach and align with the staircase. To test the controller, the robot was placed in different positions, with respect to the staircase and at different distances from the staircase. Then, the complete pipeline was executed using the teleoperator computer and the robot moved, autonomously, towards the staircase until the orientation could be obtained (at approximately 1.5 m) and then it aligned itself with the staircase.