Patent ID: 11947752
Assignee: REALWEAR, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A system for enabling hands-free operation of a binary touch-based application, the system comprising:
one or more processors; and
one or more computer storage media storing computer-useable instructions that, when executed by the one or more processors, implement a method comprising:
detect a user interface tree generated by the binary touch-based application during initialization of the binary touch-based application, the binary touch-based application having a GUI with at least a GUI element;
extract a reference that corresponds to the GUI element of the GUI from the user interface tree, the reference including a user interface label of the GUI element;
determine that the reference corresponding to the GUI element is associated with a customized user interface label, the customized user interface label being different than the user interface label of the GUI element and being user-defined within a label index that is customized based on the user interface tree;
provide for display an overlay adjacent to the GUI element, the overlay including the customized user interface label based on the determination that the reference is associated with the customized user interface label within the customized label index; and
emulate a touch input that corresponds to the GUI element based on a determination that an audio input corresponds to the customized user interface label.