Patent Document ID: 8165974
Application ID: 12479972

Base Claim:
1. A method of reviewing documents comprising: partitioning a collection of documents into sets of documents for review by a plurality of reviewers; for each set: organizing the documents in the set into a plurality of groups, displaying documents in the set on a display device for review by a reviewer, receiving the reviewer's labels for the displayed documents, based on the reviewer's labels, assigning a class from a plurality of classes to each of the reviewed documents, progressively training a classifier model stored in computer memory based on features extracted from the reviewed documents in the set and their assigned classes; and prior to review of all documents in the set, identifying documents in the set for which the classifier model assigns a class different from the one assigned based on the reviewer's label and returning a subset of the identified documents for a second review by a reviewer.

---

Claim 19:
19. The method of claim 1 , wherein there are at least three sets of documents and wherein at least three classifier models are trained, one classifier model for each of the sets of documents, the method including comparing the classes assigned to the documents in the first set based on the labels of the reviewer with classes assigned by at least two classifiers to the first set of documents, which classifiers have not been trained on the first set of the documents.