Bounding Box based Annotation Generation for Semantic Segmentation by Boundary Detection This paper proposes a new method to generate pseudoannotations from manual bounding boxes for semantic segmentation. Different from traditional local data driven based methods such as Conditional Random Field (CRF) and GrabCut, we aim at using classagnostic bounding box based segmentation models. To this end, we propose a new segmentation network, which formulates segmentation task as a sparse boundary point detection task rather than dense pixel label prediction task, and therefore can provide new type of pseudoannotations. Furthermore, we detect object boundary based on direction, and use multiple directions to handle various shapes of objects. Moreover, we further enhance the pseudo generation by combining different types of segmentation masks. Classical Fully Convolutional Networks (FCN) network based on dense prediction is also modified to generate diverse foreground masks. A simple fusion method based on intersection operation is proposed to combine the two types of pseudoannotations. We verify the effectiveness of our method on PASCAL VOC 2012 validation dataset. The mIoU value is 67.9%, which outperforms the stateoftheart method by 1.1%.

Seeing Behind Things: Extending Semantic Segmentation to Occluded Regions Semantic segmentation and instance level segmentation made substantial progress in recent years due to the emergence of deep neural networks (DNNs). A number of deep architectures with Convolution Neural Networks (CNNs) were proposed that surpass the traditional machine learning approaches for segmentation by a large margin. These architectures predict the directly observable semantic category of each pixel by usually optimizing a crossentropy loss. In this work we push the limit of semantic segmentation towards predicting semantic labels of directly visible as well as occluded objects or objects parts, where the networkxe2x80x99s input is a single depth image. We group the semantic categories into one background and multiple foreground object groups, and we propose a modification of the standard crossentropy loss to cope with the settings. In our experiments we demonstrate that a CNN trained by minimizing the proposed loss is able to predict semantic categories for visible and occluded object parts without requiring to increase the network size (compared to a standard segmentation task). The results are validated on a newly generated dataset (augmented from SUNCG) dataset.