Patent ID: 6735566
Filing Date: 2004-05-11
Classification: G06K,G06T,G10L

Abstract:
A method for generating realistic facial animation directly from speech, comprising the steps of:learning dynamical models of facial and vocal action from observations of a face and facial gestures made while an individual is speaking whereby voice is analyzed with regard to learned categories of facial gestures and whereby long term dependencies including facial-co-articulation are implicitly modeled; generating sequences of facial gestures to accompany newly input audio; and, converting said sequences into optimal sequences of facial parameters; and, driving facial animation with said facial control parameters; and wherein the step generating sequences of facial gestures includes using a hidden Markov model to learn the dynamics and essential states of the face as said individual speaks, to learn mappings from states to facial poses, to learn mapping from said states to acoustic features extracted from said individual's speech, to analyze newly input speech to obtain an optimally compatible sequence of facial states, and to convert said sequence of facial states into an optimal trajectory through the possible facial poses of an animated face corresponding to how the individual's face would move while speaking said newly input audio.