Patent ID: 11861317
Assignee: EDUCATIONAL TESTING SERVICE
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 11:
12. A system for characterizing human-machine dialog comprising:
at least one data processor;
memory for storing instructions which, when executed by the at least one data processor, result in operations comprising:
receiving data comprising a recording of an individual interacting with a dialog application simulating a conversation;
parsing the received data using automated speech recognition to result in text comprising a plurality of words;
extracting features from the parsed data;
inputting the extracted features into an ensemble of different machine learning models each trained to generate scores corresponding to each of a plurality of different dialog constructs, a first of the ensemble of different machine learning models being a stacked bi-directional long short term memory (BiLSTM) network, a second of the ensemble of different machine learning models being an End to End Memory Network (MemN2N), a third of the ensemble of different machine learning models being a transformer network;
fusing, for each of the dialog constructs, scores generated by the different machine learning models;
generating, based on the fused scores, a performance score characterizing a conversational proficiency of the individual interacting with the dialog application; and
providing data comprising the generated score.