Patent ID: 11881010
Assignee: VOOMER, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. A system comprising:
one or more computers; and
one or more computer-readable media storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
training a machine learning model to classify videos into performance level classifications based on characteristics of image data and audio data in the videos, wherein the model is trained based on training data that includes (i) videos recorded in response to prompts to users, wherein the videos that include speech of the users and image data of the users, and (ii) classification labels assigned to the videos;
receiving, over a communication network, video data captured by a device of a user following a prompt that the device provides to the user, wherein the video data depicts a face of the user and includes speech of the user that responds to the prompt provided by the device;
determining, based on the video data, a set of feature values that describe audio and video characteristics of the video data;

providing the set of feature values as input to the trained machine learning model to generate output that classifies the video data with respect to the performance level classifications based on the set of features provided to the trained machine learning model;
updating a user interface of the device based on the performance level classification for the video data, including transmitting to the device over the communication network user interface data that indicates the performance level classification and one or more characteristics of the video data that contributed to the classification of the video data; and
providing user interface data for a simulation user interface having interactive user interface controls that enable a user to specify different values for audio characteristics and video characteristics, wherein the simulation user interface is configured to present a simulation result indicating a performance level classification determined based on the values specified using the user interface controls.