: From a set of sparsely captured images of a user, we construct our image-based dynamic avatar, consisting of a face blendshape and a hair morphable model, which represent the corresponding coarse geometry. The captured images are warped and blended under the guidance of the coarse geometry to generate real-time facial animation with fine-scale details. From left to right: input images, the reconstructed coarse geometry for face and hair, rendering results of our avatar with different poses and expressions. We present a novel image-based representation for dynamic 3D avatars, which allows effective handling of various hairstyles and headwear, and can generate expressive facial animations with finescale details in real-time. We develop algorithms for creating an image-based avatar from a set of sparsely captured images of a user, using an off-the-shelf web camera at home. An optimization method is proposed to construct a topologically consistent morphable model that approximates the dynamic hair geometry in the captured images. We also design a real-time algorithm for synthesizing novel views of an image-based avatar, so that the avatar follows the facial motions of an arbitrary actor. Compelling results from our pipeline are demonstrated on a variety of cases.
REF drive dynamic image-based 3D avatars based on a real-time face tracker.
Real-time facial animation with image-based dynamic avatars
TOGS
Computer Science