# A Depth-Based Hybrid Approach for Safe Flight Corridor Generation in Memoryless Planning
## 4 Free Space Detection
### Data Collection and Training  
We collect training data from both the flight simulator and the actual environment. We run the planner in our previous work [37] in the Flightmare drone simulator [38] and capture depth images at the resolution of 320 \(\times\) 240 from a simulated onboard camera. We then label a bounding box representing the base of a rectangular pyramid SFC by a modified version of the depth-based inflating algorithm in [12] in which we brute-forced search all pixels in the frame. That exhaustive inflating method generally outputs overlapping bounding boxes in the frame without concerning obstacle density, as illustrated in Figure 5.  
This annotation type cannot be used for training because it does not imply an explicit feature for the SFC object. Labeled SFCs in an image should not be overlapping and represent separated free space at a constant and specific depth. Therefore, we label only one bounding box in an image. We modify the depth-based inflating algorithm to exhaustively search all the pixels at a depth of 1 m, inflate all the possible rectangular pyramids, and  
Figure 5: Depth-based inflating methods produce highly overlapped SFCs (**left**) unsuitable for training data points. We label only one largest bounding box (**right**) for training the free space detection model. then peak the one with the largest base. The largest base will be extracted as a bounding box representing an SFC. We also collect actual depth images with a RealSense D435, a depth camera from Intel. The camera has an FoV of 87\({}^{\circ}\times\) 58\({}^{\circ}\) for depth sensing. The maximum frame rate at the resolution 320 \(\times\) 240 is 90 frames per second (fps). Actual depth frames are also captured at the resolution of 320 \(\times\) 240. A person holding the camera walks around the indoor and outdoor environments to obtain sample depth frames, including clear spaces and obstacle-filled scenarios, as shown in Figure 6.  
For each synthetic and real image collection, we produce 36,000 frames, split into a train, validation, and test set with a ratio of 90:8:2. Applying an autonomous labeling mechanism, we can save a huge resource in producing a hand-engineered data process used in [17; 18]. Our method does not need to encrypt any robot's dynamic information into training data. Therefore, we can collect data from various scenarios and configurations, simulated and real frames, without worrying about synchronization, estimation inaccuracy, or vehicle damages. Furthermore, our free space detection approach accepts data points containing only one image and its annotations, much lighter than other sequence data points in collision probability prediction approaches. Training hardware must support the Pytorch framework (Pytorch [[https://pytorch.org/](https://pytorch.org/)]([https://pytorch.org/](https://pytorch.org/)), accessed on 1 June 2023) as required by Yolov7. We train the model in our available local server with an NVIDIA GeForce GTX 1080 Ti 11 GB GDDR5X. Our model will be transfer-learning trained from the yolov7-tiny.pt, which is the checkpoint of a Yolov7 tiny pre-trained model with the COCO dataset. The training configuration will be 64 samples per batch and 150 epochs. Given the Intersection over Union (IoU) threshold of 0.5, the trained model achieves a prediction accuracy of 94.9%, recall of 92.9%, mean average precision at IoU 0.5 ([EMAIL_ADDRESS]) of 96.1% and [EMAIL_ADDRESS]:0.95 of 80.9% on the test dataset. Free space prediction results on the test set are visualized in Figure 7. Predicted SFCs are reasonably similar to the ground truth. As illustrated in Figure 8, the precision, recall, and mAPs converged to stable values over training epochs. Since we have only one class of object, the precision-recall (P-R) curve for the free space class coincides with the global P-R curve for all classes. The P-R curve profile shows that the precision approached 1 when the recall approached 0 and vice versa, maximizing the integral of the area under the P-R curve, which is the [EMAIL_ADDRESS] of 96.1%.