Patent ID: 7149468

Claim:
A method of allocating resources to score a test-taker's response to an open-ended item of an assessment comprising: a. estimating the test-taker's performance on the assessment, wherein estimating the test-taker's performance on the assessment comprises: 1. estimating the test-taker's overall score for all items of the assessment; 2. utilizing one or more cutoff scores which separate the set of all possible overall scores into discrete non-overlapping ranges; 3. treating each of said ranges of scores as an indicator of a performance level on the assessment; 4. assigning to the test-taker the performance level based on the range of scores which includes the test-taker's estimated overall score; and 5. using the assigned performance level as the estimate of the test-taker's performance on the assessment; b. establishing a specified level of certainty required for a final determination of the test-taker's performance; c. estimating a level of certainty for the estimated performance of the test-taker; d. allocating an additional scoring resource to the test-taker's response to at least one open-ended item when the estimated level of certainty fails to meet the specified level of certainty and e. accepting the assigned performance level as the final determination of the test-taker's performance when the estimated level of certainty meets the specified level of certainty.