Patent ID: 11922356
Assignee: SNAP INC.
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 0:
1. A computer-implemented method, the method comprising:
receiving, by one or more processors, an audio stream comprising a conversation between at least two individuals;
detecting a voice feature comprising a speaking rate of a first individual of the at least two individuals;
determining that the voice feature comprising the speaking rate of the first individual corresponds to a speech emotion of a plurality of speech emotions, the determining including comparing the voice feature comprising the speaking rate to a reference voice feature, and selecting the speech emotion based at least on the comparing;
identifying an emotional status for the first individual based on the speech emotion that corresponds to the voice feature of the first individual, the identifying an emotional status for the first individual further comprising;
detecting the first individual in one or more of frames of a video stream;
dynamically determining, over the one or more frames, a deformation of a virtual face mesh of the first individual;
locating feature reference points of the first individual;
aligning the virtual face mesh to the first individual in one or more of the frames based at least in part on the feature reference points;
determining that the deformation refers to at least one facial emotion selected from a plurality of reference facial emotions; and
evaluating the identified emotional status for the first individual based on the at least one facial emotion of the first individual within the video stream and the speech emotion;
storing a timestamped work quality parameter representing the identified emotional status in a database in a profile associated with the first individual;
associating the timestamped work quality parameter with the audio stream;
aggregating timestamped work quality parameters in the database over a predetermined period to produce a work performance characteristic and determining a value of the work performance characteristic; and
generating an alert when one or more work quality parameters associated with the first individual exceed a determined threshold value.