# Part-Based Obstacle Detection Using a Multiple Output Neural Network
## 5 Results and Evaluation  
The same hardware setup was used for evaluating as well as training the model. The experimental setup is based on the Python programming language, the project being implemented using open-source deep learning software. The proposed multiple output network was evaluated, using not only existing popular datasets (as mentioned in Section 4) but also our own datasets [17; 33]. For the semantic segmentation output head, the evaluation was performed using the CityScapes validation dataset; the results were published in our previous work [16], where  
Figure 15: Extracting the obstacle quarters from the original segmentation mask and 2D obstacle bounding boxes. Image source [36]. we used the free road space in order to detect on-road obstacles and the segmented road surface was integrated into our monocular perception system, which is able to track the detected obstacles using a particle filter. For the road class, we obtained an IoU score of 0.90. The obstacle detection output head that predicts the obstacle quarters represents the main information used for detecting the obstacles from the road traffic scene. Therefore, the obstacle detection module was also evaluated using the dataset presented in [17], which features road-traffic images captured with different camera systems from the ones used in the training datasets. For this dataset, the ground truth is considered to be the information obtained from a stereo-vision camera setup. We compared our results with our previously proposed system and previously published results in [39; 40]. On our own dataset, which features over 1000 images of object-bounding boxes extracted from the stereo-vision data, we obtained an IoU score of 0.74. On the CityScapes validation dataset, we obtained an IoU score of 0.83 and, on the KITTI [41] dataset, a 0.70 IoU score. The results were also compared with a Yolo V3 [5] detector featuring a DarkNet backbone, which was trained on KITTI using the same loss as the initial Yolo paper. We also compared the results with a Yolo V3 detector with the same ResNet-50 backbone, trained on the same datasets as our proposed model. We have also compared our results with the bounding boxes from the Mask R-CNN and Yolact; the results are similar to ours, with minor differences (1-2% in favor of the Mask R-CNN or Yolact). The bounding box obstacle detection results are presented in Table 1.  
The stereo-vision database from Table 1 used images acquired with a pair of cameras from which we used only the left image from the setup. The ground truth data was considered to be the result of a stereo-tracking algorithm [42]. The main advantage of a stereo-vision setup is that in the case of overlapping objects, they can be easily detected based on their depth information. Still, our proposed system, using a monocular camera setup, obtained a high accuracy. The evaluation in terms of complex city scenarios was proven to be highly accurate, especially using the CityScapes dataset. When testing on the KITTI database, our results were poorer due to the different aspect ratios of the dataset images. To address this issue, we had two possibilities: we either resized the KITTI image to fit the 256 \(\times\) 256 pixels input of the CNN, or we cropped the KITTI image and then resized it. Both variants would affect our evaluation results; the first choice would produce a severe deformation of the objects from the traffic scene, which would highly impact the detection process, whereas the second choice (cropping the input image), would most likely prevent various objects from the scene being considered for detection. For this evaluation, we used the first option. For the 2D bounding box (object detection) evaluation, we obtained a marginal improvement (1%) from previously published work [40], due to the image fusion of the output heads (as described in Section 3.5). The pixel-wise evaluation of the detection was also performed.