Patent Document ID: 9818024
Application ID: 14680977
Patent Status: 1

Claim One:
1. A method for mirroring a user facial expression in a three-dimensional (3D) in-game avatar in real-time, the method comprising: generating a 3D in-game avatar and displaying the 3D in-game avatar within a game display of a game by performing: capturing an image of a user as the user participates in the game; based on contents of the image, determining a plurality of user facial features in the image depicting a user facial expression of the user, wherein the plurality of user facial features comprises a left eye region, a right eye region, and a lips region; applying an active appearance face model, which includes a plurality of facial feature parameters, to the left eye region, the right eye region and the lips region to determine similarities between the left eye region and a reference left eye feature defined by one or more facial feature parameters of the plurality of facial feature parameters, similarities between the right eye region and a reference right eye feature defined by one or more facial feature parameters of the plurality of facial feature parameters, and similarities between the lips region and a reference lips feature defined by one or more facial feature parameters of the plurality of facial feature parameters; generating a left eye sub-model based on the similarities between the left eye region and the reference left eye feature; generating a right eye sub-model based on the similarities between the right eye region and the reference right eye feature; generating a lips sub-model based on the similarities between the lips region and the reference lips feature; generating a global model by projecting and fitting the left eye sub-model onto the left eye region, projecting and fitting the right eye sub-model onto the right eye region, and projecting and fitting the lips sub-model onto the lips region; and based on the global model, generating a 3D face model of the 3D in-game avatar; and causing the 3D in-game avatar to mimic the user facial expression in the game display in real-time.