Patent ID: 11951384
Assignee: TAKE-TWO INTERACTIVE SOFTWARE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC A  G | IPC A  G

Claim 0:
1. A system for motion capture of an actor to estimate a hand pose prediction of the actor for rendering a three-dimensional (3D) virtual environment, comprising:
one or more sensor pre-processors to receive raw motion capture data from at least one wearable sensor and at least one optical marker-based sensor and to normalize the received raw motion capture data;
a convolutional neural network to receive the normalized motion capture data from the sensor pre-processors and to generate a learning model to detect one or more patterns over a predetermined time window, the detected patterns being encoded by a matrix;
a hand pose dense neural network for receiving a measurement of bone length from the actor and the matrix of the convolutional neural network to determine an intermediate hand pose prediction at a selected point in time;
a hand scale dense neural network for receiving outputs from the convolutional neural network to learn a mapping of the received measurement of bone lengths and the determined intermediate hand pose prediction; and
a post processor for rescaling and offsetting the determined intermediate hand pose prediction based on the learned mapping to generate the hand pose prediction.