  The task of building footprint segmentation has been well-studied in the
context of remote sensing (RS) as it provides valuable information in many
aspects, however, difficulties brought by the nature of RS images such as
variations in the spatial arrangements and in-consistent constructional
patterns require studying further, since it often causes poorly classified
segmentation maps. We address this need by designing a joint optimization
scheme for the task of building footprint delineation and introducing two
auxiliary tasks; image reconstruction and building footprint boundary
segmentation with the intent to reveal the common underlying structure to
advance the classification accuracy of a single task model under the favor of
auxiliary tasks. In particular, we propose a deep multi-task learning (MTL)
based unified fully convolutional framework which operates in an end-to-end
manner by making use of joint loss function with learnable loss weights
considering the homoscedastic uncertainty of each task loss. Experimental
results conducted on the SpaceNet6 dataset demonstrate the potential of the
proposed MTL framework as it improves the classification accuracy considerably
compared to single-task and lesser compounded tasks.
