Patent ID: 11860920
Assignee: OPEN TEXT HOLDINGS, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. A non-transitory computer readable medium, storing instructions for:
controlling automated assisted review of a plurality of documents within a data store of a document system by:
initiating generation of a document map for the plurality of documents using a topic-related generative model for the plurality of documents;
initiating transmission of a control set of documents from the plurality of documents to a user, the control set of documents determined based on selected documents, based on selecting the selected documents from a first strata of the plurality of documents and a second strata of the plurality of documents; and
based on a control set metric regarding the control set of documents provided by the user, the control set metric including an indicator of responsiveness for each of the documents of the control set of documents, initiating:
determining a responsiveness score for each of the plurality of documents based on a scoring algorithm;
determining a set of responsive documents and a set of non-responsive documents of the plurality of documents based on the responsiveness score determined for each of the plurality of documents and a decision boundary score;
determining a confidence score for the document system using the responsiveness score for each of the documents of the control set and the indicator of responsiveness for each of the control set documents provided by the user;
selecting one or more of the plurality of documents based on the responsiveness scores of the plurality of documents, wherein the responsiveness score of each of the one or more selected documents is at or near the decision boundary score;
providing a presentation of the one or more selected documents to the user;
based on an indicator of responsiveness by the user for each of the selected documents, refining the scoring algorithm based on the indicator of responsiveness for each of the selected document; and
generating a desired confidence score for the document system, and providing a presentation of the set of responsive documents to the user when the desired confidence score for the document system is achieved, wherein the confidence score for the document system is an F1 score determined based on a comparison of the responsiveness score for the documents of the control set with the indicator of responsiveness for the documents of the control set received from the user.