Patent ID: 9652991
Date: 2017-05-16
CPC Classifications: G09B

Claim:
1. A computer-implemented method of scoring a response to a prompt, comprising: receiving an audio recording of the responses; performing automatic speech recognition of the response to convert the text from a spoken sample to text and one or more acoustic features using automatic speech recognition using an acoustic model; determining, using a processing system, a first expression similarity feature for the response by applying one or more regular expressions to the response, wherein the one or more regular expressions are determined based on one or more first training responses associated with the prompt; determining, using the processing system, a second expression similarity feature for the response by applying one or more context free grammars to the response, wherein the one or more context free grammars are determined based on one or more second training responses associated with the prompt; determining, using the processing system, a third expression similarity feature for the response by applying a keyword list to the response, wherein the keyword list is determined based on the prompt; determining, using the processing system, a first word usage similarity feature for the response by applying one or more probabilistic n-gram models to the response, wherein each of the one or more probabilistic n-gram models is associated with a proficiency level and is determined based on one or more third training responses associated with the prompt and with that proficiency level; determining, using the processing system, a second word usage similarity feature for the response by comparing a POS response vector to one or more POS training vectors, wherein the POS response vector is determined based on the response, wherein each of the one or more POS training vectors is associated with a proficiency level and is determined based on one or more fourth training responses associated with the prompt and with that proficiency level; determining, using the processing system, a third word usage similarity feature for the response by comparing a response n-gram count to one or more training n-gram counts using an n-gram matching evaluation metric, wherein the response n-gram count is determined based on the response, wherein each of the one or more training n-gram counts is associated with a proficiency level and is determined based on one or more fifth training responses associated with the prompt and with that proficiency level; determining, using the processing system, a dissimilarity feature for the response by comparing the test response to one or more sixth training responses using a dissimilarity metric, wherein each of the one or more sixth training responses is associated with the prompt and with a proficiency level; and automatically determining, using the processing system, a score for the response by using a scoring model to evaluate content correctness of the response based on the first expression similarity feature, the second expression similarity feature, the third expression similarity feature, the first word usage similarity feature, the second word usage similarity feature, the third word usage similarity feature, and the dissimilarity feature of the response, wherein the score is displayed on a graphical user interface; wherein the scoring model is trained based on predetermined analytic content correctness scores assigned to a plurality of training responses and a set of features extracted from the plurality of training responses, the set of features comprising: the first expression similarity features, the second expression similarity features, third expression similarity feature, the first word usage similarity feature, the second word usage similarity feature, the third word usage similarity feature, and the dissimilarity feature.