Patent ID: 7693808

Claim:
A computer-implemented method for ordinal ranking comprising the steps of: defining a case set containing a plurality of cases to be ranked; defining an attribute set containing a plurality of attributes for which a score can be assigned with regard to each case; assigning a weight to each of the plurality of attributes, wherein the sum of the weights equals a total weight; defining a plurality of raters; assigning decision rights to each of the raters, wherein the decision rights govern the raters participation in the steps of defining an attribute set and assigning weights; independently for each of the raters, generating an evaluation result set comprising one evaluation result for each one of the plurality of cases and wherein each evaluation result comprises scores, assigned by the rater, for each of the attributes in the attribute set with regard to the case; independently for each of the raters, generating and displaying to the rater, referent data as each evaluation result is generated in the step of generating an evaluation result set, wherein the referent data comprises a comparison group of cases for at least one attribute of the plurality of attributes; independently for each of the raters, monitoring and providing warnings to the rater if any of a plurality of pre-defined thresholds is violated; independently for each of the raters, repeating the step of generating an evaluation result set to generate one or more additional evaluation result sets, comparing the results of the evaluation result set and the one or more additional evaluation result sets, and providing the outcome of the comparison to the rater, wherein repeating the step of generating an evaluation result step comprises re-rating each one of the plurality of cases to obtain an additional evaluation result for each one of the plurality of cases and wherein each additional evaluation result comprises scores, assigned by the rater, for each of the attributes in the attribute set with regard to the case; collecting the evaluation result sets generated by each of the raters; aggregating and analyzing the collected evaluation results sets; and generating, responsive to a specified set of the evaluation result sets, a final ordinal ranking of the cases based on the scores contained in the specified set of evaluation result sets and the weights assigned to each of the plurality of attributes.