Patent ID: 9223858
Filing Date: 2015-12-29
Classification: G06F

Abstract:
1. A method of determining the quality of a selected set of documents resulting from a document review process, said method executed by a computer system comprised of at least one processor connected by a data network to at least one data storage device, comprising: receiving data representing the selection of a set of documents from a document review process into a first and a second subsets, where the first subset of documents is associated with a first criteria indicator data value indicating that the document meets a predetermined document selection criteria and the second subset associated with a second criteria indicator data value indicating that it does not meet the predetermined document selection criteria; storing in a computer memory a first and a second pre-determined statistical sampling rates, each sampling rate corresponding to the first and second subsets, said statistical sampling rates being values representing a statistical frequency of checking the accuracy of the respective first and second subset selection where the first and second rates are determined by a sampling model; determining using the computer system for the first and the second subset a corresponding first and second sample set of selected documents using the corresponding first and second sampling rate; for each document in the first and second sample set of selected documents, receiving data representing a determination whether the associated criteria indicator is erroneous and a corresponding error type and storing in computer memory a data value representing the corresponding error type; determining using the computer system for the first and second sample sets, a corresponding first and second error rate; in dependence on the first error rate being equal to zero, storing in computer memory for each document in the first subset a data value representing that the document is correct and in dependence on the first error rate being greater than zero storing in computer memory a data value representing the logical condition that the entire first subset is rejected; in dependence on the error rate corresponding to the second sample set being equal to or less than a predetermined threshold value, storing in computer memory for each document in the second subset a data value representing that the document is correct and in dependence on the second error rate being greater than the predetermined threshold value, storing in computer memory a data value representing the logical condition that the entire second subset is rejected; and generating a data structure representing a report on the quality of the selection of the set of documents into the first and second subsets.