Semantic Segmentation for Aerial Imagery with Convolutional Neural Networks

Aerial imagery has been utilized as important input for many applications such as map making, disaster relief, real estate management, and urban planning. In those applications, extracting ground objects such as buildings and roads from aerial imagery is necessary as a pre-processing. This task is to give an object class label for each pixel in an aerial image, so that it is highly attentional task for human and very costly and time-consuming process. Therefore, various methods have been proposed for automating this task. However, conventional methods usually need hand-crafted image features and specially designed classifiers for each target area. Furthermore, the extraction accuracy is not enough to be alternative to human experts. On the other hand, some recent studies show that this task can be performed automatically with relatively high accuracy and no feature engineering by utilizing a convolutional neural network (CNN) trained on a large scale dataset that includes many pairs of aerial imagery and ground object mask images. However, those recent achievements still require independent networks for each target object, so that simultaneous multiple object extraction using a CNN has not been evaluated. In this thesis, a method to extract multiple objects simultaneously with a single network is proposed. To train a single network efficiently for multiple object extraction task, a novel output function called channel-wise inhibited softmax that focuses on the specialty of the background class that is a class to represent all the other classes other than classes of interest, e.g., buildings and roads, is proposed. Then, how to improve seamline determination by utilizing the result of semantic segmentation for aerial imagery is shown. In Chapter 1, example applications of aerial imagery interpretation are introduced. Then problems in conventional methods for automated ground object extraction are presented, and our approaches to address the problems are overviewed to clarify the research goal. In Chapter 2, the details of the most related work and the relationship to our approach are presented to show the difference between them. In Chapter 3, the details of our methods are presented. A method to extract buildings and roads simultaneously and accurately from aerial imagery by training a single convolutional neural network with a novel output function on a large scale dataset that has aerial images and corresponding building and road mask images is proposed. The output function is designed by focusing on the specialty of the background class. In Chapter 4, empirical evaluation of the proposed method is performed on the large scale dataset. Then the effectiveness of the proposed method is compared to the conventional method. In Chapter 5, the results of the evaluation performed in Chapter 4 are discussed in details by visualizing the middle layer outputs and analyzing the trained parameters. Then the size of dataset that is required to train a convolutional neural network for this task is discussed. In Chapter 6, the fact that automatic seamline determination for aerial imagery mosaicking can be performed with optimal results by utilizing building extraction results that are acquired by semantic segmentation is presented. Then evaluation of the resulting seamlines by comparing to the conventional method based on non-semantic segmentation is performed. In Chapter 7, the proposed methods are summarized and the most important findings are clarified. Then conclusion and future work of this research are shown.