Framer: Interactive Frame Interpolation
Authors: name: Wen Wang; name: Qiuyu Wang; name: Kecheng Zheng; name: Hao Ouyang; name: Zhekai Chen; name: Biao Gong; name: Hao Chen; name: Yujun Shen; name: Chunhua Shen
Published: 2024-10-24 17:59:51

Abstract: We propose Framer for interactive frame interpolation, which targets
producing smoothly transitioning frames between two images as per user
creativity. Concretely, besides taking the start and end frames as inputs, our
approach supports customizing the transition process by tailoring the
trajectory of some selected keypoints. Such a design enjoys two clear benefits.
First, incorporating human interaction mitigates the issue arising from
numerous possibilities of transforming one image to another, and in turn
enables finer control of local motions. Second, as the most basic form of
interaction, keypoints help establish the correspondence across frames,
enhancing the model to handle challenging cases (e.g., objects on the start and
end frames are of different shapes and styles). It is noteworthy that our
system also offers an "autopilot" mode, where we introduce a module to estimate
the keypoints and refine the trajectory automatically, to simplify the usage in
practice. Extensive experimental results demonstrate the appealing performance
of Framer on various applications, such as image morphing, time-lapse video
generation, cartoon interpolation, etc. The code, the model, and the interface
will be released to facilitate further research.