Semantic annotation and object extraction for very high resolution satellite images

Mit den vielen hochauflosenden SAR- (Radar mit synthetischer Apertur) und optischen Satelliten, die sich im Orbit befinden, werden auch die zugehorigen Bildarchive standig groser und aktualisiert, da taglich neue hochaufgeloste Bilder aufgenommen werden. Daraus ergeben sich neue Perspektiven und Herausforderungen an eine automatische Interpretation von hochaufgelosten Satellitenbildern zur detaillierten semantischen Annotation und Objekt-Extraktion. Dazu kommt, dass das florierende Gebiet des maschinellen Lernens die Leistungskraft von Computer-Algorithmen gezeigt hat, die ihre "Intelligenz" zur Losung zahlreicher und verschiedenartiger Anwendungsfalle (wie visuelle Objekterkennung, inhaltsbasierte Bildsuche etc.) bereits allgemein demonstriert haben. Allerdings konnen die vorgeschlagenen und bereits existierenden Methoden momentan nur eine begrenzte Anzahl von Bildern verarbeiten. Daher wird in dieser Dissertation versucht, Informationen aus grosen Mengen von Satellitenbildern zu extrahieren. Wir bieten Losungen zur halbautomatischen Interpretation von Satellitenbildinhalten auf der Ebene von Bild-ausschnitten und von Pixeln, bis hin zur Objekt-Ebene mit hochaufgelosten Bildern von TerraSAR-X und WorldView-2. Hierbei wird das Analyse-Potential von nicht uberwachten Lernverfahren zur Verarbeitung von grosen Datenmengen genutzt. With a number of high-resolution Synthetic Aperture Radar (SAR) and optical satellites in orbit, the corresponding image archives are continuously increasing and updated as new high-resolution images are being acquired everyday. New perspectives and challenges for the automatic interpretation of high-resolution satellite imagery for detailed semantic annotation and object extraction have been raised up. What’s more, the booming machine learning field has proved the power of computer algorithms by presenting the world their "intelligence" to solve numerous and diverse applications, visual object recognition, content-based image retrieval, etc. However, till now, the proposed and already existing methods are usually able to process only a limited amount of images. Hence, this dissertation tries to extract information from large amounts of satellite imagery. We provide solutions for the semi-automatic interpretation of satellite image content from patch-level and pixel-level to object-level, using the high-resolution imagery provided by TerraSAR-X and WorldView-2. The mining potential of unsupervised learning methods is utilized for the processing of large amounts of data. With large amounts of data, our solutions try to simplify the problem at the first step based on a simple assumption. A Gaussian distribution assumption is applied to describe image clusters obtained via a clustering method. Based on the already grouped image patch clusters, a semi-supervised cluster-then-classify framework is proposed for the semantic annotation of large datasets. We design a multi-layer scheme that offers a great opportunity to describe image contents from three perspectives. The first perspective represents image patches in a hierarchical tree structure, similar patches are grouped together, and are semantically annotated. The second perspective characterizes the intensity and SAR speckle information in order to get a pixel-level classification for general land cover categories. The third perspective allows an object-level interpretation. Here, the information of location and similarity among elements are taken into account, and an SVM-based active learning concept is implemented to update iteratively the so-called "non-locality" map which can be used for object extraction. A further exploitation of our approach could be to introduce a hierarchical structure for SAR and optical data in the way the patch-level, pixel-level and object-level image interpretation are connected to each other. Hence, starting from a whole scene, general and detailed levels of information can be extracted. Such fusions between different levels have achieved promising results towards an automated semantic annotation for large amounts of high-resolution satellite images. This dissertation also demonstrates up to which level information can be extracted from each data source.