HighRes-net: Multi-Frame Super-Resolution by Recursive Fusion
Generative deep learning has sparked a new wave of Super-Resolution algorithms that enhance single images with impressive aesthetic results, albeit with imaginary details.Multi-frame Super-Resolution offers a more grounded approach to the ill-posed problem, by conditioning on multiple low-resolution views.This is important for satellite monitoring of human impact on the planet -- from deforestation, to human rights violations -- that depend on reliable imagery.To this end, we present HighRes-net, the first deep learning approach to MFSR that learns its sub-tasks in an end-to-end fashion: co-registration, fusion, up-sampling, and registration-at-the-loss.Co-registration of low-res views is learned implicitly through a reference-frame channel, with no explicit registration mechanism.We learn a global fusion operator that is applied recursively on an arbitrary number of low-res pairs.We introduce a registered loss, by learning to align the SR output to a ground-truth through ShiftNet.We show that by learning deep representations of multiple views, we can super-resolve low-resolution signals and enhance Earth observation data at scale.Our approach recently topped the European Space Agencys MFSR competition on real-world satellite imagery.
The first deep learning approach to MFSR to solve registration, fusion, up-sampling in an end-to-end manner.This paper proposes an end-to-end multi-frame super-resolution algorithm, that relies on a pair-wise co-registrations and fusing blocks (convolutional residual blocks), embedded in a encoder-decoder network 'HighRes-net' that estimates the super-resolution image.", 'This paper proposes a framework including recursive fusion to co-registration loss to solve the problem of super-resolution results and high-resolution labels not being pixel aligned.