Patent ID: 11893772
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A system, comprising:
one or more computing devices;
wherein the one or more computing devices include instructions that upon execution on or across one or more processors implement cause the one or more computing devices to:
obtain an indication, via one or more programmatic interfaces, of a source data set, wherein the source data set comprises a plurality of records to be used to train a classification model for one or more target classes;
divide the source data set into a plurality of buckets, wherein individual ones of the buckets comprise one or more records;
execute a plurality of two-phase learning iterations on selected computing resources to train successive versions of the classification model using the source data set until a training completion criterion is met, wherein individual ones of the two-phase learning iterations include (a) a bucket group selection phase and (b) a class boundary refinement phase, and the training completion criterion limits execution of training by a computing resource consumption limit or execution time limit,
wherein the bucket group selection phase comprises identifying one or more positive-match buckets from among unlabeled buckets in the plurality of buckets, wherein respective ones of the positive-match buckets include at least one record labeled by one or more bucket annotators as a member of a target class of the one or more target classes, wherein the bucket group selection phase eliminates one or more other buckets in the plurality of buckets for annotation to avoid wasting computing resources, wherein at least one bucket is selected as a candidate for annotation based at least in part on output obtained from a first version of a classification model generated in the class boundary refinement phase of an earlier two-phase learning iteration and results of a search query; and
wherein the class boundary refinement phase comprises:
selecting, from among unlabeled records of the one or more positive-match buckets identified in the bucket group selection phase, a set of labeling-candidate records which meet a model enhancement potential criterion;
obtaining, from one or more record annotators, class labels for the set of labeling-candidate records; and
training, using a training data set comprising at least the labeled records obtained from the one or more record annotators, a second version of the classification model;

stop execution of the training when the training completion criterion is met and store a trained version of the classification model produced by the training; and
run the trained version of the classification model to classify one or more records which are not in the source data set.