Patent ID: 11868941
Assignee: WORKFUSION, INC.
Field: IT methods for management (Electrical engineering)
Classification: CPC G | IPC G

Claim 18:
19. One or more computer-readable non-transitory storage media embodying software comprising instructions operable when executed to perform operations including:
receiving, by one or more processors of an information processing system, one or more first results of a first task, wherein behavioral features of a worker system, observed while the worker system performs each of the one or more first results of the first task, are provided;
determining, by the one or more processors, an accuracy confidence score corresponding to each of the one or more first results of the first task, each accuracy confidence score representing a likelihood for the corresponding first result of being fraudulent, wherein an ensemble algorithm for combining an unsupervised machine learning model and a supervised machine learning model is used for determining each determined accuracy confidence score, and wherein the unsupervised machine learning model compares, for each of the one or more first results, the observed behavioral features for the first result to behavioral features of a group of worker systems;
adding, by the one or more processors, for each of the one or more first results having an accuracy confidence score higher than a threshold score, the observed behavioral features for the first result to the behavioral features of the group of worker systems;
determining, by the one or more processors, that a number of the one or more first results having an accuracy confidence score higher than the threshold score exceeds an accuracy quality threshold set for the first task based on the accuracy confidence scores determined for the of the one or more first results;
providing, by the one or more processers, a determination for the first task to a customer system, in response to the determination that the number of the one or more first results having an accuracy confidence score higher than the threshold score exceeds the accuracy quality threshold;
receiving, by the one or more processors, one or more labels indicating whether one of the one or more first results of the first task is fraudulent, wherein the one or more labels are generated based on either manual or programmatical review of the one or more first results of the first task; and
updating, by the one or more processors, the supervised machine learning model based on the one or more first results and the one or more labels.