  Video interpolation aims to generate a non-existent intermediate frame given
the past and future frames. Many state-of-the-art methods achieve promising
results by estimating the optical flow between the known frames and then
generating the backward flows between the middle frame and the known frames.
However, these methods usually suffer from the inaccuracy of estimated optical
flows and require additional models or information to compensate for flow
estimation errors. Following the recent development in using deformable
convolution (DConv) for video interpolation, we propose a light but effective
model, called Pyramid Deformable Warping Network (PDWN). PDWN uses a pyramid
structure to generate DConv offsets of the unknown middle frame with respect to
the known frames through coarse-to-fine successive refinements. Cost volumes
between warped features are calculated at every pyramid level to help the
offset inference. At the finest scale, the two warped frames are adaptively
blended to generate the middle frame. Lastly, a context enhancement network
further enhances the contextual detail of the final output. Ablation studies
demonstrate the effectiveness of the coarse-to-fine offset refinement, cost
volumes, and DConv. Our method achieves better or on-par accuracy compared to
state-of-the-art models on multiple datasets while the number of model
parameters and the inference time are substantially less than previous models.
Moreover, we present an extension of the proposed framework to use four input
frames, which can achieve significant improvement over using only two input
frames, with only a slight increase in the model size and inference time.
