Patent ID: 9555544
Date: 2017-01-31
CPC Classifications: B25J,G05B,G06F,G06T

Claim:
1. A computer-implemented method comprising: identifying a process that (i) is manually performed by a user interacting with a computer, and (ii) is to be automated for performance by a robot that is configured to interact with the computer or another computer; obtaining images taken of a display of the computer while the user is interacting with the computer in manually performing the process; applying a computer vision technique, to the images taken of the display of the computer while the user is interacting with the computer in manually performing the process, to determine that a change in images taken of the display of the computer while the user is interacting with the computer in manually performing the process satisfies a predetermined threshold corresponding to a scene change; in response to determining that the change in images taken of the display of the computer while the user is interacting with the computer in manually performing the process satisfies the predetermined threshold corresponding to a scene change, identifying a first activity corresponding to receipt of input from the user during the process followed by a second activity corresponding to a scene change during the process; for each of the identified activities, generating activity information associated with the activity; generating a process definition for use in causing the robot to automatically perform the process by interacting with the computer or the other computer, where the process definition indicates the first activity corresponding to receipt of the input from the user followed by the second activity corresponding to the scene change and, for each indicated activity, indicates the activity information associated with the activity; and storing the process definition for later use in causing the robot to automatically perform the process by interacting with the computer or the other computer.