Patent ID: 11886892
Assignee: AUTOMATION ANYWHERE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 3:
4. A tangible storage medium, having stored thereupon computer program code for execution on a computer system, the computer program code executing on a server processor to cause the computer system to perform a computer-implemented method for detecting one or more user interface objects contained in a screen image of a user interface generated by an application program, the computer-implemented method comprising:
selecting a dataset that comprises a plurality of object classes, each object class corresponding to a type of user interface object and comprising at least one variance parameter;
training a machine learning model with a portion of the dataset designated as a training portion of the dataset to cause the machine learning model to detect, in the training portion of the dataset, objects in each of the plurality of object classes;
testing the machine learning model with a portion of the dataset designated as a testing portion of the dataset to determine whether the machine learning model can detect objects in the plurality of object classes with an accuracy above a predetermined set of test metrics;
wherein based on the machine learning model exhibiting an accuracy that does not meet the predetermined set of test metrics, the method further comprises the operations of:
processing the testing portion of the dataset to generate, for each object class in the testing portion of the dataset, a numerical value that represents failure of the machine learning model with respect to each variance parameter of a corresponding object class,
generating a first set of variance parameters by selecting from among the numerical values, variance parameters that fall within a predetermined variance range,
generating an updated dataset comprising updated object classes by, for each variance parameter within the first set of variance parameters, and
retraining the machine learning model with a portion of the updated dataset designated as an updated training portion of the dataset to cause the machine learning model to detect, in the updated training portion of the dataset, objects in each of the plurality of object classes.