Patent ID: 11960636
Assignee: MAGIC LEAP, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC F  G

Claim 0:
1. A system comprising:
a head-mounted display (HMD) of a wearable system configured to present three dimensional (3D) virtual content to a user;
two or more user input components comprising at least an audio sensing device configured to capture sound, an eye gaze tracking device configured to detect eye gaze direction, and a gesture tracking device configured to acquire data indicating hand gestures of the user; and
a hardware processor communicatively coupled to the head-mounted display and the two or more user input components, the hardware processor programmed to:
receive, from the audio sensing device, speech data encoding an utterance of one or more words spoken by the user;
obtain a transcription for the one or more words spoken by the user based at least on the received speech data;
control the head-mounted display to present a string of textual characters representative of the obtained transcription to the user;
receive, from the eye gaze tracking device, an eye gaze direction;
receive, from the gesture tracking device, hand gesture data;
identify a particular subset of the textual characters for editing by the user, the hardware processor identifying the subset of the textual characters utilizing a process comprising:
determining a first confidence score for the particular subset of the textual characters utilizing eye gaze direction data received from the eye gaze tracking device;
further determining a second confidence score for the particular subset of the textual characters utilizing the hand gesture data received from the gesture tracking device;
calculating an aggregated confidence score for the particular subset of textual characters based at least in part on the first confidence score and the second confidence score; and
selecting the particular subset of textual characters selected based on the aggregated confidence score;

following identification of the particular subset of textual characters, modify the particular subset of textual characters in accordance with a user input command received via the system.