Patent ID: 11875230
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A system, comprising:
one or more computing devices comprising respective processors and memory configured to implement an artificial intelligence-based classification service;
wherein the one or more computing devices are configured to:
initiate a labeling feedback session with a label provider;
cause, during the labeling feedback session, one or more visualization data sets to be presented to the label provider via an interactive programmatic interface, including a particular visualization data set which comprises an ordered representation of one or more data items for which labeling feedback is requested, wherein the order in which the one or more data items are arranged is based at least in part on a respective rank, with respect to estimated learning contribution, associated with individual ones of the one or more data items in a training set for a particular training iteration of one or more classification models, and wherein the one or more data items for which the labeling feedback is requested include a first data item selected based on a determination that a variation among predictions for the first data item from a plurality of classification models exceeds a threshold;
obtain, during the feedback session, indications from the label provider via the interactive programmatic interface of (a) respective labels for the one or more data items of the one or more visualization data sets and (b) a filter criterion to be used to select one or more additional data items out of one or more data sources for which one or more additional visualization data sets are added to the visualization data sets presented to the label provider such that the selected additional data items are newly displayed via the interactive programmatic interface as new data items to be labeled for use in the training set; and
provide, to one or more destinations, a classification prediction corresponding to a data item, wherein the classification prediction is obtained from a classification model trained using a training set which includes at least one label of the respective labels.