Patent ID: 11941736
Assignee: GOOGLE LLC
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 12:
13. A computing system for motion-controlled animation, system comprising:
one or more processors; and
one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining a vector-based surface illustration and a rigging skeleton at an initial pose, wherein the rigging skeleton comprises a plurality of rigging vectors that define a plurality of rigging vertices, and wherein correspondence data describes one or more correspondences between one or more rigging vectors or rigging vertices and one or more portions of the vector- based surface illustration;
obtaining a plurality of image frames that depict an entity in motion; and
for each image frame of the plurality of image frames:
processing the image frame with one or more machine-learned models to obtain landmark positions within the image frame for a plurality of facial or body landmarks of the entity;
updating one or more respective positions of the plurality of rigging vertices of the rigging skeleton based on the landmark positions for the plurality of facial or body landmarks within the image frame; and
performing a skinning algorithm to transform and render the vector-based surface illustration based on the respective positions of the plurality of rigging vertices of the rigging skeleton and the correspondence data.