DOI: 10.5281/zenodo.4767771
title: Thermal Bridges on Building Rooftops - Hyperspectral (RGB + Thermal + Height) drone images of Karlsruhe, Germany, with thermal bridge annotations

description: <strong>Overview:</strong> The dataset of <strong>Thermal Bridges on Building Rooftops (TBBR dataset)</strong> consists of annotated combined RGB and thermal drone images with a height map. All images were converted to a uniform format of 3000x4000 pixels, aligned, and cropped to <strong>2680x3370</strong> to remove empty borders. See the "Usage" section below for details about the stored formats made available here. The raw images for our dataset were recorded with a normal (RGB) and a FLIR-XT2 (thermal) camera on a DJI M600 drone. They show six large building blocks of around 20 buildings per block recorded in the city centre of the German city Karlsruhe east of the market square. Because of a high overlap rate of the images, the same buildings are on average recorded from different angles in different images about 20 times. All images were recorded during a drone flight on March 19, 2019 from 7 a.m. to 8 a.m. At this time, temperatures were between 3.78 ° C and 4.97 ° C, humidity between 80% and 98%. There was no rain on the day of the flight, but there was 2.3mm/m² 48 hours beforehand. For recording the thermographic images an emissivity of 1.0 was set. The global radiation during this period was between 38.59 W / m² and 120.86 W / m². No direct sunlight can be seen visually on any of the recordings. The dataset contains <strong>926 images</strong> with a total of <strong>6,927 annotations</strong> of thermal bridges on rooftops, split into train and test subsets with 723 (5,614) and 203 (1,313) images (annotations), respectively. The annotations only include thermal bridges that are visually identifiable with the human eye. Because of the aforementioned image overlap, each thermal bridge is annotated multiple times from different angles. For the annotation of the thermal images the image processing program <em>VGG Image Annotator </em>from the Visual Geometry Group, version 2.0.10, was used. The thermal bridge annotations are outlined with polygon shapes. These polygon lines were placed as close as possible but outside the area of significant temperature increase. If a detected thermal bridge was partially covered by another building component located in the foreground, the thermal bridge was also marked across the covering in case of minor coverings. Adjacent thermal bridges, which affect different rooftop components, were annotated separately. For example, a window with poor insulation of the window reveal located in the area of a poorly insulated roof is annotated individually. There is no overlap between annotated areas. While each image contains annotations, they also include thermal bridges present that are not annotated. <strong>Usage:</strong> Each compressed archive file represents one of the six flight paths. For the related publication the final path (Flug1_105Media) was used as a hold-out test sample. The archives contain Numpy files (one per image) of shape (2680, 3370, 5), where the final dimension is the colour channel in the format [B, G, R, Thermal, Height]. Archives were compressed using ZStandard compression. They can be decompressed in a terminal by running e.g. <pre><code class="language-bash">tar -I zstd -xvf Flug1_105Media.tar.zst</code></pre> these will be decompressed into the file structure: <pre><code>images/ └── Flug1_105Media/ └── DJI_0004_R.npy └── DJI_0006_R.npy └── ...</code></pre> Corresponding annotations are provided in the COCO JSON format. There is one file for training (Flug1_100Media - Flug1_104Media blocks) and one for test (Flug1_105Media block). They contain a single class (thermal bridge) and expect the folder structure shown below. Note: The annotation files contain <em>relative</em> paths to numpy files, in case of problems please convert to <em>absolute</em> paths (i.e. insert the containing directory before each file path in the JSON annotation files). We provide the <strong>TBBRDet software</strong> which includes a dataloader and dataset inspection tools which make use of the Detectron2 and MMDetection libraries. We recommend the following folder structure for use: <pre><code>├── train/ │ ├── Flug1_100-104Media_coco.json │ └── images/ │ ├── Flug1_100Media/ │ │ ├── DJI_XXXX_R.npy │ │ └── ... │ ├── ... │ └── Flug1_104Media/ │ ├── DJI_XXXX_R.npy │ └── ... └── test/ ├── Flug1_105Media_coco.json └── images/ └── Flug1_105Media/ ├── DJI_XXXX_R.npy └── ...</code></pre> <strong>Metadata:</strong> The experimental metadata was structured with the <strong>Spatio Temporal Asset Catalog (STAC)</strong> specification family. This specification provides a standardized way for describing geospatial assets. It defines related JSON object types of Item, Catalog, and Catalog, extending on Collection as the basis. One STAC Collection JSON object provides information about the recorded images and the environmental conditions during recordings. It also contains information about the overall bounding box of the entire area in which images were recorded. This object links to related STAC Item JSON objects containing information about the recorded city blocks and the cameras. The objects for the city blocks contain the GeoJSON geometry of the respective block and the<br> corresponding bounding box. The objects containing the camera information are based on an existing STAC extension for camera related metadata. Metadata of the archived NumPy files for each image was structured using the <strong>Data Package</strong> schema from the <strong>Frictionless Standards</strong>. This standard describes a collection of data files. Therefore, metadata about all containerized NumPy files of the six flight paths (Flug1_100Media - Flug1_104Media blocks and Flug1_105Media block) is provided within a JSON-based file. Note that <strong>camera1</strong> corresponds to the <strong>RGB camera</strong> and <strong>camera2</strong> the <strong>thermal</strong>. <strong>FAIR Digital Objects:</strong> All files are represented in a standardized way as <strong>FAIR Digital Objects<br> (FAIR DOs)</strong> to enable machine actionable decisions on the data in spirit of<br> the FAIR principles. <strong>Persistent Identifier (PID):</strong> Persistent Identifiers (PIDs) are resolvable with the Handle.Net Registry (HNR). File Persistent Identifier (PID) Flug1_100-104Media_coco.json 21.11152/6ea60288-d895-414e-80c0-26c9fdd662b2 Flug1_105Media_coco.json 21.11152/58d43ddc-5e29-4980-8675-ae579b50a1e2 Flug1_100.tar.zst 21.11152/6858a0b5-cc60-40e9-afef-8c2dd8b35e8e Flug1_101.tar.zst 21.11152/e670f510-7e00-4d3a-9b90-3bac7a7c069e Flug1_102.tar.zst 21.11152/3ab9f444-05f6-445e-a691-62fae4021bea Flug1_103.tar.zst 21.11152/365fd8cf-8e86-41b8-9d0e-b816fdd01d29 Flug1_104.tar.zst 21.11152/041a6111-644a-4617-afb3-3c421a88e8e3 Flug1_105.tar.zst 21.11152/f48bf4e7-3879-4216-8f64-45a060b8f658 Flug1_100-105_frictionless_standards.json 21.11152/7b58b3b5-75eb-4417-ac4d-abe025e159f6 Flug1_collection_stac_spec.json 21.11152/ba370aa3-6422-428c-9ff7-c2ef429df603 Flug1_100_stac_spec.json 21.11152/09cb76fc-b8cb-4116-a22a-68c5bdfa77b0 Flug1_101_stac_spec.json 21.11152/24a55398-b96b-43dd-b0fb-cd8ce302c7ce Flug1_102_stac_spec.json 21.11152/721234ac-4b5a-4d02-9944-82a08ef2db35 Flug1_103_stac_spec.json 21.11152/ebaeb5bc-0514-47c9-bcd2-98f0253843d8 Flug1_104_stac_spec.json 21.11152/9854677c-77c5-4a0b-916b-57dd9ec20198 Flug1_105_stac_spec.json 21.11152/cfd0fc0e-f5ea-464e-a57f-28e882924860 Flug1_camera1_stac-spec.json 21.11152/976fcf28-f924-4a21-b53d-5d054ad8198d Flug1_camera2_stac-spec.json 21.11152/37833c54-1d36-42e4-858d-831447122863