Patent ID: 7831542

Claim:
A computer-implemented method comprising: capturing a selected playback image, the playback image being one of a series of display images generated over time by a graphical user interface (GUI) of an application program being tested by a cognitive control framework (CCF) system in a playback phase in response to simulated user inputs to the application program; detecting at least one active display object in a recorded image, the recorded image being one of a series of display images generated over time by the GUI of the application program being tested and captured during a recording phase of the CCF system based on user inputs to the application program, the user inputs, time interval between user inputs, and resulting display data forming an execution scenario script for use in testing the GUI of the application program during the playback phase for generating playback images, the at least one active display object being a portion of the recorded image being acted upon as a result of user input; searching subsets of hypotheses of locations of display objects in the captured playback image for a matching display object corresponding to the detected at least one active display object of the recorded image according to predetermined criteria; recalculating old actions for the matching display object in the playback image by applying actions according to the execution scenario script and loading a next set of data, when the corresponding matching display object is found; and checking dynamic conditions, including the time between user inputs.