Patent ID: 11954258
Assignee: SONY GROUP CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 6:
7. The system according to claim 1, wherein the circuitry is further configured to execute the third neural network-based analysis that includes:
an extraction of frames from the immersive content, and
an extraction of spatiotemporal features associated with the digital avatar from the immersive content,
an application of a second Bi-LSTM model on a temporal portion of the spatiotemporal features to generate a first result,
an application of a convolutional neural network (CNN) with an attention mechanism on a spatial portion of the spatiotemporal features to output a second result,
an application of a transformer encoder on the first result and the second result to generate a transformer result, and
an application of a classifier on the transformer result to generate a label that indicates the activity that the digital avatar is engaged in at the time-instant.