  Optical flow is a regression task where convolutional neural networks (CNNs)
have led to major breakthroughs. However, this comes at major computational
demands due to the use of cost-volumes and pyramidal representations. This was
mitigated by producing flow predictions at quarter the resolution, which are
upsampled using bilinear interpolation during test time. Consequently, fine
details are usually lost and post-processing is needed to restore them. We
propose the Normalized Convolution UPsampler (NCUP), an efficient joint
upsampling approach to produce the full-resolution flow during the training of
optical flow CNNs. Our proposed approach formulates the upsampling task as a
sparse problem and employs the normalized convolutional neural networks to
solve it. We evaluate our upsampler against existing joint upsampling
approaches when trained end-to-end with a a coarse-to-fine optical flow CNN
(PWCNet) and we show that it outperforms all other approaches on the
FlyingChairs dataset while having at least one order fewer parameters.
Moreover, we test our upsampler with a recurrent optical flow CNN (RAFT) and we
achieve state-of-the-art results on Sintel benchmark with ~6% error reduction,
and on-par on the KITTI dataset, while having 7.5% fewer parameters (see Figure
1). Finally, our upsampler shows better generalization capabilities than RAFT
when trained and evaluated on different datasets.
