# A Depth-Based Hybrid Approach for Safe Flight Corridor Generation in Memoryless Planning
## 4 Free Space Detection
### Safe Flight Corridor Constitution  
We define our SFCs in a pyramid-shaped form to take advantage of the depth image characteristics. One of the features of depth images is that, when deprojecting it into space, a segment connecting the optical center to any pixel does not intersect any segment connecting the optical center to any other pixel. Therefore, any plane parallel to the sensor frame and not obscured by any pixels will be collision-free. Additionally, any segment connecting the optical center (the robot's position) to any point on that plane will also be collision-free. If the plane is rectangular, we can constitute a rectangular pyramid SFC with its apex located at the robot's location \(\mathcal{O}\), and the base perpendicular to the z-axis of the depth camera-fixed frame, as visualized in Figure 3.  
To archive the mentioned pyramid SFCs, we need to generate its rectangular base in the space. Since we predict SFC only in the current depth image, the pyramid base will also be in the camera's FoV and can be specified by its projection in the image frame. Let \(bb=(x_{1},y_{1},x_{2},y_{2})\) denotes the bounding box that represents the SFC base's projection where in this case, \(x_{1}\), \(y_{1}\), \(x_{2}\), and \(y_{2}\) are the pixel coordinates of its left, top, right, and bottom in the image frame. Our object detector aims to predict \(bb\) so that the corresponding  
Figure 1: Local planner architecture.  
Figure 2: Free space detection planning flow.  
constituted SFCs are collision-free. The flow of the SFC constitution from a predicted bounding box in the image frame is visualized in Figure 4.  
It is noted that the network learns to perceive safe corridors purely from the depth values of neighborhood pixels. Therefore, the quality of generated SFCs and the underlying planning algorithm does not depend on the features or materials of the environment if depth estimation is accurate enough. In other words, foliage in a rainforest or mirrored objects will be perceived as the form of depth frames and will be treated equally when they are fed to the model. That also means the network can label a 2D bounding box containing faraway obstacles as free space. This makes sense because we can still extract sufficient traversable SFCs between the robot and unreachable obstacles. This generalization turns out to be effective as the trained model can generate a safe corridor from both obstacle-dense and sparse situations with the same amount of computation.  
On the other hand, the training also benefits from that formulation as it requires only one depth image and corresponding SFC annotations to constitute a data point. Specifically, the training data only needs to tell how deep and big the SFCs are via a depth image and corresponding annotations. The trained model will take one depth image as an input and output bounding boxes \(bb\) representing projections of SFCs.  
This work assumes that the accuracy of the depth estimation is perfect, and the discussion on it is beyond this project's scope. In reality, the depth estimation accuracy of popular commercial cameras is reliable enough for this application. For example, an Intel(r) RealSense(tm)Depth Camera D435 has an accuracy of under 2% of depth at 2 m (Intel(r) RealSense(tm)Depth Camera D435 Tech Specs [[https://www.intelrealsense.com/depth-camera-d435/](https://www.intelrealsense.com/depth-camera-d435/)]([https://www.intelrealsense.com/depth-camera-d435/](https://www.intelrealsense.com/depth-camera-d435/)), accessed on 1 June 2023) in the z-axis measured as out of the factory. This amount  
Figure 4: An SFC constitution from a predicted bounding box.  
Figure 3: Visualization for the top view of the depth point cloud inside the FoV.  
of uncertainty in depth estimation is acceptable and can be easily eliminated by adding safety margins on training data.