# Staircase Detection, Characterization and Approach Pipeline for Search and Rescue Robots
## 2 Materials and Methods
### ROS Pipeline  
To evaluate our proposal we implemented two different ROS pipelines using the two different cameras previously described. The main difference between the two were in the image acquiring and detection steps and also in the necessary steps for plane extraction. fixed minor labeling mistakes in the Figures 5 and 6 show the two pipelines and the  
Figure 4: Membership functions for the output variables: (**a**) linear velocity, (**b**) angular velocity.  
different nodes used for the image acquisition and staircase detection. For the D435i camera we used a pipeline that needed access to a GPU to implement the model, and to obtain the detections. Furthermore, communication with the teleoperator's computer was needed at all times. In comparison, the OAK-D camera pipeline was processed on the robot and the communication with the teleoperator's computer was mainly for monitoring and launching nodes. Another important different is that the RealSense has more disparity levels than the OAK-D and, for this reason, only the region growing based segmentation was needed. In comparison, the OAK-D camera required an extra clustering step (outlined in Algorithm 1), to perform the segmentation.  
The D435i pipeline uses the _raalsense_ros_ nodes and nodelets to acquire the RGB and depth images at a resolution of 848 \(\times\) 480 and they were published at 30 FPS. The RGB image was sent to the detection node (darknet_ros) which published the bounding boxes at approximately 100 FPS. For the OAK-D camera we used the depthai-ros wrapper [31], which allowed the use of the depthAI library. Using this wrapper, we implemented the node required to obtain the RGB and depth images and the spatial detection from the camera. The node configured the RGB camera with a 1920 \(\times\) 1080 resolution and at 30 FPS, and it was compressed to 416 \(\times\) 416 for the neural network. Additionally, the stereo cameras were configured with a resolution of 1280 \(\times\) 720 and at 30 FPS, and the depth image was synchronized with the detections obtained on the RGB image.  
The generation of the point cloud from the depth images was done using the _depth_img_proc_[32] nodelets and, for both cameras, a reasonable amount of noise is generated. To decrease the effect of noise, we downsampled the point clouds generated using Voxel filters with a leaf size of 0.11 \(\times\) 0.11 \(\times\) 0.11. Furthermore, we only used the points that were in the range of 0 to 5 m and eliminated the ground points. Also, for both pipelines, the bounding box and the point cloud were synchronized before starting the ROI extraction and characterization algorithm. The point cloud handling and processing was done using the Point Cloud Library [33].  
The staircase and characterization node published the staircase parameters and the node with the fuzzy logic controller used these parameters to determine the linear and angular velocity commands necessary for the staircase approach and alignment. Additionally, to adjust the membership function values, we first used a simulation of our pipeline  
Figure 5: ROS pipeline using the RealSenseTM D435i camera.  
Figure 6: ROS pipeline using the OAK-D camera.  
in Gazebo [34] and later performed minimal adjustments on the physical differential drive robot.