Patent ID: 11954507
Assignee: NIPPON TELEGRAPH AND TELEPHONE CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 8:
9. A method for automatically recognizing a graphical user interface component in an active display screen, the method comprising:
having a display screen includes a plurality of graphical user interface components and a background provided by an application software, wherein the plurality of graphical user interface components incudes at least two graphical user interface components and the display screen is in active operation;
extracting a graphical user interface component of the plurality of graphical user interface components from the display screen provided by the application software;
identifying a first form and a position of the graphical user interface component extracted in the display screen;
superimposing a mouse cursor on the graphical user interface component in the display screen based on at least one of the first form and the position of the graphical user interface component in the display screen, wherein when the mouse cursor is superimposed on the graphical user interface component occurs at least one of the following occurs: a change in a display form of the mouse cursor and change in a display form of the graphical user interface component, wherein the display form of the mouse cursor includes a symbol image of the mouse cursor;
identifying a change in the display form of the mouse cursor;
determining that a kind of the graphical user interface component cannot be identified based on only the change in the display form of the mouse cursor;
in response to the determining, identifying the kind of the graphical user interface component along with an operation associated with the kind of the graphical user interface component in the display screen based on the change in the display form of the mouse cursor and the change in the display form the graphical user interface component; and
moving the mouse cursor sequentially to each of the plurality of graphical user interface components in the display screen based on the form and the position of each graphical user interface component, wherein the mouse cursor is moved to and superimposed on each graphical user interface component to determine the kind and the operation of the graphical user interface.