  Most recent video super-resolution (SR) methods either adopt an iterative
manner to deal with low-resolution (LR) frames from a temporally sliding
window, or leverage the previously estimated SR output to help reconstruct the
current frame recurrently. A few studies try to combine these two structures to
form a hybrid framework but have failed to give full play to it. In this paper,
we propose an omniscient framework to not only utilize the preceding SR output,
but also leverage the SR outputs from the present and future. The omniscient
framework is more generic because the iterative, recurrent and hybrid
frameworks can be regarded as its special cases. The proposed omniscient
framework enables a generator to behave better than its counterparts under
other frameworks. Abundant experiments on public datasets show that our method
is superior to the state-of-the-art methods in objective metrics, subjective
visual effects and complexity. Our code will be made public.
