  Space-time video super-resolution (STVSR) aims to increase the spatial and
temporal resolutions of low-resolution and low-frame-rate videos. Recently,
deformable convolution based methods have achieved promising STVSR performance,
but they could only infer the intermediate frame pre-defined in the training
stage. Besides, these methods undervalued the short-term motion cues among
adjacent frames. In this paper, we propose a Temporal Modulation Network
(TMNet) to interpolate arbitrary intermediate frame(s) with accurate
high-resolution reconstruction. Specifically, we propose a Temporal Modulation
Block (TMB) to modulate deformable convolution kernels for controllable feature
interpolation. To well exploit the temporal information, we propose a
Locally-temporal Feature Comparison (LFC) module, along with the Bi-directional
Deformable ConvLSTM, to extract short-term and long-term motion cues in videos.
Experiments on three benchmark datasets demonstrate that our TMNet outperforms
previous STVSR methods. The code is available at
https://github.com/CS-GangXu/TMNet.
