# Staircase Detection, Characterization and Approach Pipeline for Search and Rescue Robots
###### Abstract  
Currently, most rescue robots are mainly teleoperated and integrate some level of autonomy to reduce the operator's workload, allowing them to focus on the primary mission tasks. One of the main causes of mission failure are human errors and increasing the robot's autonomy can increase the probability of success. For this reason, in this work, a stair detection and characterization pipeline is presented. The pipeline is tested on a differential drive robot using the ROS middleware, YOLOv4-tiny and a region growing based clustering algorithm. The pipeline's staircase detector was implemented using the Neural Compute Engines (NCEs) of the OpenCV AI Kit with Depth (OAK-D) RGB-D camera, which allowed the implementation using the robot's computer without a GPU and, thus, could be implemented in similar robots to increase autonomy. Furthermore, by using this pipeline we were able to implement a Fuzzy controller that allows the robot to align itself, autonomously, with the staircase. Our work can be used in different robots running the ROS middleware and can increase autonomy, allowing the operator to focus on the primary mission tasks. Furthermore, due to the design of the pipeline, it can be used with different types of RGB-D cameras, including those that generate noisy point clouds from low disparity depth images. object detection; point cloud segmentation; ROS; fuzzy control system +
Footnote †: journal: Medical  
with the outer edges of the staircase and thus slippage can occur, causing abrupt changes in direction, which are difficult to control. For this reason, this work focuses on implementing a staircase detection, characterization and approach pipeline that would allow the teleoperator to gain useful information about the size and orientation of the staircase and also allow the robot to align itself with the staircase. This information obtained can also help determine whether or not the staircase is traversable or could be used to implement an autonomous staircase crossing algorithm. The implementation of the pipeline in this work consists of three main parts: the detection of the staircase in the RGB image and the characterization of the staircase using the point cloud corresponding to the region of interest and the design of the fuzzy controller for the approach and alignment. The related works in this area have focused mainly on the detection of the staircase using data from RGB images, depth images or point clouds. For RGB images, the work presented in [3; 4; 5; 6; 7] first apply a filtering stage to reduce noise and lighting effects and later apply edge detection algorithms, such as a Canny operator, to obtain the edges present in the image. After edge extraction, non-candidate stair edges are eliminated and a linking algorithm is used to determine the lines corresponding to the staircase's parallel edges. Using these lines, certain staircase characteristics can be extracted, such as orientation with respect to the camera. Additionally, some state of the art staircase localization systems have been implemented using RGB images, deep learning and point clouds. Deep learning methods to detect staircases use neuronal networks such as YOLOv3-tiny [8; 9], YOLOv4 [8] and SSD [10; 11] or implement a stair step detection using YOLOv3-tiny [12] and staircase segmentation using AlbuNet [13]. The use of CNNs to detect an object in an image allows further processing in the detected area or region of interest (ROI). In [9], they fit lines to the stair edges by using an edge detector and Hough Transform. With these lines, a PID controller is implemented that allows the robot to align with the stairs. On the other hand, in [11] they present a complete pipeline where SSD is used to detect the staircase and later, using a 3D point cloud, obtain the planes within the detected region. By doing so they are able to estimate the orientation of the stairs in a simulated environment.