# Part-Based Obstacle Detection Using a Multiple Output Neural Network
## 1 Introduction
This combination of semantic segmentation and the geometric post-analysis of the semantic segmentation results is able to achieve the real-time identification of individual objects at the pixel level, with a significant reduction in the required resources when compared with instance segmentation networks. In this paper, we propose a convolutional neural network that is trained to perform multiple prediction tasks, along with a geometry-based post-processing step to extract additional relevant data. The multi-output CNN prediction is achieved using a single encoder module and multiple decoders. The encoder part will extract the relevant features from the input images that are fed into the CNN and is usually referred to as the \"backbone\" network. The decoders will make use of these features to generate the predictions (outputs) of the network and are usually called network \"heads\". The presented CNN architecture is able to produce three different output heads related to the road traffic scene: the obstacle parts for instance segmentation (the top-left, top-right, bottom-left, and bottom-right quarters, which will be used later by the clustering algorithm), the semantic segmentation of the scene for detecting the drivable road area, and the vanishing point of the image. The backbone network (the encoder) features convolution layers that are used by the semantic segmentation head in order to construct images of the scene using the extracted features by using various upsampling layers and direct connections with the encoder. The first head outputs the semantic segmentation of the object parts, which will be clustered using the algorithm presented in Section 3.4 to produce the instance segmentation results. The second head produces the obstacle vs. free area semantic segmentation of the scene; this result is used as a mask to refine the object regions after they are clustered. The free space segmentation result can also be used as a standalone result for simpler driving assistance applications, such as collision avoidance. The third head produces the gradient voting maps for vanishing point computation, a result that can be used for automatic camera orientation calibration and, therefore, for estimating the three-dimensional (3D) position of the objects detected by the previous two heads. This step is beyond the scope of this paper. Thus, our proposed solution achieves obstacle reconstruction and individual instance extraction. We were able to perform semantic segmentation of the road traffic scene, detect obstacles and extract instances, and determine the vanishing point in an efficient manner, with results that were comparable to the ones produced by instance segmentation networks without the added memory requirements, high computational requirements, and network complexity. The multi-output neural network is trained using the existing publicly available datasets. The training process is fully automated and does not require any annotation work, as the object quarters are extracted from labeled instances by our training pre-processing tool. The main contributions of this work are the following:  
1. A multiple-head network architecture that is able to produce the geometric parts of the objects, which can be then grouped into individual instances, free and occupied pixel classification for results refinement, and voting maps for vanishing point computation;
2. A lightweight object-clustering algorithm, based on the results of the multiple-head semantic segmentation network;
3. An automated training solution for the multiple-head network, which uses publicly available databases without the need for manual annotation of the object parts.