Patent ID: 11954507
Assignee: NIPPON TELEGRAPH AND TELEPHONE CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 4:
5. A screen recognition method comprising:
a marking step of marking a graphical user interface (GUI) component in an active display screen provided by an application software, wherein the marking step distinguishes a background of the display screen from the GUI component in the display screen of the application software;
an extraction step of extracting the GUI component included in the display screen provided by the application software from the display screen;
a superimpose step of superimposing the mouse cursor over the GUI component in the display screen based on at least one of a form and a position of the GUI component;
an identification step of identifying a change in a displayed symbol image of the mouse cursor;
a determination step of determining that a kind of the GUI component cannot be identified based only on the change in the displayed symbol image of the mouse cursor, wherein in response to determining, determining the kind of the GUI component extracted in the extraction step and an operation that can be performed to the GUI component, based on the change in the displayed symbol image of the mouse cursor and a change in the GUI component at a same time the mouse cursor is superimposed on the GUI component; and
moving the mouse cursor sequentially to each of the plurality of GUI components in the display screen based on the form and the position of each GUI component, wherein the mouse cursor is moved to and superimposed on each GUI component to determine the kind and the operation of the GUI component.