SpaceNet 7 Results: ‘Tis the Season
By: Adam Van Etten
Tags: Computer Vision, Machine Learning, Data Science, Satellite Imagery, Open Source

2. Winning Approach

The winning team of lxastro0 (consisting of four Baidu engineers) improved upon the baseline approach in three key ways.

First, they swapped out the VGG16+U-Net architecture of the baseline with the far more advance HRNet, which maintains high-resolution representations through the whole network. Given the small size of the SpaceNet 7 buildings, mitigating the downsampling present in most architectures is highly desirable.

Second, the small size of objects if interest is further mitigated by upsampling the imagery 3× prior to ingestion into HRNet. The team experimented with both 2× and 3× upsampling, and found that 3× upsampling proved superior, see Figure 2.

Figure 2. Various segmentation models in the right three columns. The top row shows an entire (AOI), while the bottom row zooms in on a sub-region.

Finally, and most crucially, the team adopted an elaborate post-processing scheme they term “temporal collapse” which we detail in the following section.

3. Temporal Collapse

In order to improve post-processing for SpaceNet 7, the winning team assumed:

(a) Buildings will not change after the first observation

(b) In the 3× scale, there is at least a one-pixel gap between buildings.

(c) There are three scenarios for all building candidates:

c.1. Always exists in all frames

c.2. Never exists in any frame

c.3. Appears at some frame k and persists thereafter

The data cube for each AOI can be treated as a video with a small (~24) number of frames. Since assumption (a) states that building boundaries are static over time, lxastro0 compresses the temporal dimension and predicts the spatial location of each building only once, as illustrated in Figure 3.

Figure 3. Visualization of temporal collapse for ground truth (top row) and predictions (bottom row). The left frame is the compressed probability map.

Building footprint boundaries are extracted from the collapsed mask using the watershed algorithm and an adaptive threshold, and taking into account assumption (b). This spatial collapse ensures that predicted building footprint boundaries remain the same throughout the time series.

With the spatial location of each building now determined, the temporal origin must be computed. At each frame, and for each building, the winning team averaged the predicted probability values at each pixel inside the pre-determined building boundary. This mapping is then used to determine at which frame the building originated, as illustrated in Figure 4.

Figure 4. Method for determining the temporal origin of an individual building. Top row: The three possible scenarios of assumption (c). Bottom row: The aggregated predicted probability for the building footprint at each time step (blue) is used to map to the final estimated origin (red).

The techniques adopted by lxastro0 yield marked improvements over the baseline model in all metrics, but most importantly in the change detection term of the SpaceNet Change and Object Tracking (SCOT) metric. See Table 1 for quantitative improvements, and Figures 5 and 6 for qualitative examples. Further details on the winning algorithm (and all of the top-5 submissions) will be provided in the upcoming SpaceNet 7 model release blog post.