# Part-Based Obstacle Detection Using a Multiple Output Neural Network
## 3 Solution Description
### Obstacle Reconstruction Using Part-Based Semantic Segmentation  
The detected obstacles from the decoder module will be reconstructed using the methodology presented in this section. The CNN will provide an output wherein the obstacles are split into four individual quarters (parts), each represented by a binary channel of the output image (see Figure 6). Based on the semantic segmentation results of the neural network, which labels each obstacle pixel with a \"quarter\" label, we have designed an algorithm to extract the individual objects. First, each pixel of the whole image space is labeled with a 4-bit code, each bit corresponding to a quarter that overlaps the pixel. Some pixels can be overlapped by more than one quarter, as the four semantic segmentation images produced by the CNN are not mutually exclusive (see Figure 7). The meaning of the four bits is as follows:  
* Bit 0--top-left quarter;
* Bit 1--top-right quarter;
* Bit 2--bottom-left quarter;
* Bit 3--bottom-right quarter. Figure 5: Examples of the global semantic segmentation output: the first image is the color input image of the road scene, while the second image represents the drivable road area, the third represents the dynamic objects, and the fourth image is static objects (sidewalk or fence). Figure 6: The artificial neural network, inferring the obstacle quarters. The coded pixels will have a value of 0 if they belong to the free space (they are not obstacle points), or have a value between 1 and 15 if they belong to an object. For each of the 15-obstacle pixel values, a binary image will be generated, and the image will be labeled using the connected component labeling algorithm. The labels for each of the 15 images will be unique (the labels for image 2 will start from the maximum value of the labels of image 1, plus 1, and so on), meaning that at the end, they can be joined together into a common label image, as shown in Figure 8. The resulting regions are similar to the superpixels described in [11], but these have the added advantage that they are the result of semantic segmentation and have a high probability that each region belongs to a single object--they are grouped by meaning and not simply by color or texture properties, as in the case of superpixels. The problem is now re-stated as the problem of assigning a unique object identifier to each region. Basically, the problem becomes a problem of region-grouping but, again, we can use the advantage of semantics because we can restrict the position of the region inside the whole object, based on the quarter codes. The grouping of the regions will be based on generating multiple object hypotheses, based on the individual quarters. Because we have four types of quarters, we can generate a maximum of four hypotheses for each real-world object. Each quarter will generate a  
Figure 8: Labeled regions, based on connected pixels of the same quarter code. Figure 7: Object pixel coding, based on segmented quarters. Some pixels can belong to more than one quarter. complete object by extending its size to cover the other three missing quarters, assuming that the quarters will be of a similar size. For example, a bottom-left quarter can be extended upwards and to the right to generate the possible complete object to which the quarter belongs. If the object is completely visible (not covered by another object and not at the edge of the image), four complete rectangular hypotheses will be generated, as seen in Figure 9.  
Due to the fact that the hypotheses outnumber the objects by a factor of 4 to 1, the algorithm will compute a pixel fitness score for all hypotheses, so that the best-fitting one can be selected. The pixel score S(R) for each region R is computed by counting the pixels overlapping the region that fit with their corresponding quarter: the quarter defined by the region must correspond to the quarter label assigned to the pixel by the CNN-based semantic segmentation.