Patent ID: 9269053
Filing Date: 2016-02-23
CPC Classification: G06F,G06N

Claim Text:
1. A method for reviewing a collection of documents, the method comprising: forming a set of scored documents by scoring all documents in the collection of documents using an artificial intelligence model, wherein every document in the collection of documents has not been scored previously; forming a subset of scored documents by selecting a subset of documents meeting a certain criteria from the set of scored documents; forming a set of randomly-selected documents by randomly-selecting one or more documents from the collection of documents; forming a set of documents for review, the set of documents for review including both the subset of scored documents and the set of randomly-selected documents; presenting the set of documents for review by a reviewer in a manner that does not differentiate between the set of randomly-selected documents and the subset of scored documents; outputting, by the artificial intelligence model, a class label encoding a prediction of whether a particular document in the set of documents for review is responsive or nonresponsive; and if the particular document is responsive, outputting, by the artificial intelligence model, a prediction of a secondary class label associated with the particular document, wherein the secondary class label encodes a prediction of the particular document's membership in one or more classes including: areas of law or type of document, wherein the type of document includes marketing document, sales document, and technical document.