Patent Document ID: 9471667
Application ID: 13848837
Patent Status: 1

Claim One:
1. A computer-implemented method of scoring a response to a character-by-character highlighting task, comprising: comparing, using a processing system, the response to the character-by-character highlighting task to a correct response to the task, the comparison generating a dissimilarity value that indicates an extent of a dissimilarity of the response and the correct response; selecting, using the processing system, a threshold dissimilarity value for the task, the threshold dissimilarity value being based on (i) a first set of responses to the task, each response of the first set differing from the correct response and being scored by a human grader as being correct or incorrect, and (ii) a second set of responses to the task, each response of the second set being scored by a human grader as being correct, the threshold dissimilarity value being further based on filtering the first set to remove null responses from the first set; comparing each response of the first set to each response of the second set, each comparison generating a dissimilarity value that is associated with the response of the first set, and populating a data structure comprising M×N dissimilarity values, where M is a number of non-null responses in the first set and N is a number of responses in the second set, M and N each being an integer that is greater than 1, for each response of the first set, determining a minimum dissimilarity value associated with the response from the data structure, and determining the threshold dissimilarity value further based on (i) the minimum dissimilarity values associated with the responses of the first set, and (ii) the scoring of the responses of the first set as being correct or incorrect by the human grader; comparing, using the processing system, the dissimilarity value for the response to the threshold dissimilarity value; and scoring the response based on the comparing of the dissimilarity value for the response to the threshold dissimilarity value, wherein the response is scored as being correct if the dissimilarity value is less than or equal to the threshold dissimilarity value, and wherein the response is scored as being incorrect if the dissimilarity value is greater than the threshold dissimilarity value.