Patent ID: 11951384
Assignee: TAKE-TWO INTERACTIVE SOFTWARE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC A  G | IPC A  G

Claim 18:
19. A non-transitory computer-readable storage medium for motion capture of an actor to estimate a hand pose prediction of the actor for rendering a three-dimensional (3D) virtual environment, wherein the non-transitory computer-readable storage medium includes instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
receiving raw motion capture data from at least one wearable sensor and at least one optical marker-based sensor;
normalizing the received raw motion capture data via one or more sensor pre-processors;
receiving the normalized motion capture data from the sensor pre-processors via a convolutional neural network;
generating a learning model via the convolutional neural network to detect one or more patterns over a predetermined time window, the detected patterns being encoded by a matrix;
receiving a measurement of bone length from the actor and the matrix of the convolutional neural network via a hand pose dense neural network;
determining an intermediate hand pose prediction at a selected point in time via the hand pose dense neural network based on the received measurement of bone length and the matrix of the convolutional neural network;
learn a mapping of the received measurement of bone lengths and the determined intermediate hand pose prediction via a hand scale dense neural network based on the outputs from the convolutional neural network; and
rescaling and offsetting the determined intermediate hand pose prediction based on the learned mapping via a post processor to generate the hand pose prediction.