Patent ID: 8635172
Filing Date: 2014-01-21
Classification: G06K

Abstract:
1. A method of determining performance of a machine learning apparatus of the type that generates plural groupings each from a plurality of documents, comprising: using at least one processor to perform each of the following steps: associating a precision probability distribution to one of said groupings based on Markov Chain Monte Carlo analysis of at least another of said plural groupings; receiving review information about a first document associated with said one of said groupings and assigning that first document a first document rating indicating whether the machine learning apparatus correctly or incorrectly assigned that first document to said first one of said groupings; applying Bayesian analysis using said first document rating to update said precision probability distribution; performing the following steps (a) and (b) until termination by a stopping rule that includes analyzing the relation of a grouping precision to at least one predetermined threshold: (a) successively processing subsequent documents associated with said one of said groupings by assigning that subsequent document a subsequent document rating indicating whether the machine learning apparatus correctly or incorrectly assigned that subsequent document to said one of said groupings; (b) applying Bayesian analysis using said subsequent document rating and the updated precision probability distribution to further update the precision probability distribution and to thereby generate said grouping precision; using the grouping precision as a metric to assess performance of said machine learning system.