PATENT DOCUMENT

Publication Number: US-9075783-B2
Application Number: US-201313948053-A
Country: US
Kind Code: B2

Title: Electronic device with text error correction based on voice recognition data

Abstract:
During operation of an electronic device such as a cellular telephone with a touch screen display or other electronic equipment, a voice recognition engine may gather data on spoken words. Data on the spoken words that are recognized may be maintained in a spoken word database maintained by an input processor with an autocorrection engine. A user may supply text input that contains mistyped words to the electronic device using the touch screen or a keyboard. The input processor may use the autocorrection engine to automatically replace mistyped words with corrected versions of the mistyped words. The corrected words may be displayed in real time as the user supplies the text input. The autocorrection engine may make word correction decisions based at least partly on information in the spoken word database.

Claims:
What is claimed is: 
     
       1. A method of operating an electronic device with a storage, processing circuitry and input-output circuitry including at least a microphone and a display, the method comprising:
 recognizing spoken words; 
 storing spoken word data in a database corresponding to the recognized spoken words; and 
 after storing the spoken word data in the database:
 receiving one or more text input characters; 
 performing an automatic correction process on at least a subset of the one or more text input characters based at least in part on the spoken word data stored in the database; and 
 after performing the automatic correction process, displaying on the display corrected text corresponding to at least one of the one or more text input characters. 
 
 
     
     
       2. The electronic device method of  claim 1 , wherein the spoken word data includes information on when the spoken words were spoken. 
     
     
       3. The method of  claim 1 , wherein:
 the input-output circuitry comprises a touch screen; and 
 receiving the one or more text input characters comprises gathering the one or more text input characters as the user enters the one or more text input characters with the touch screen. 
 
     
     
       4. The method of  claim 3 , wherein the electronic device comprises a cellular telephone. 
     
     
       5. The method of  claim 1 , wherein performing the automatic correction process comprises automatically replacing a mistyped word in the one or more text inputcharacters with a replacement word without obtaining replacement word choice confirmation input at the electronic device from the user. 
     
     
       6. The method of  claim 1 , wherein performing the automatic correction process comprises automatically substituting a replacement word on the display in response to receiving a space character as part of the one or more text input characters. 
     
     
       7. The method of  claim 1 , wherein performing the automatic correction process comprises automatically replacing a mistyped word in the one or more text input characters with a replacement word in response to receiving a user-supplied confirmation of which of a plurality of displayed potential word replacements is to be used as the replacement word. 
     
     
       8. An electronic device, comprising:
 a storage; 
 processing circuitry; 
 input-output circuitry including at least a microphone and a display; and 
 one or more programs, wherein the one or more programs are stored in the storage and configured to be executed by the processing circuitry, the one or more programs including instructions for:
 recognizing spoken words; 
 storing spoken word data in a database corresponding to the recognized spoken words; and 
 after storing the spoken word data in the database:
 receiving one or more text input characters supplied by a user of the electronic device; 
 performing an automatic correction process on at least a subset of the one or more text input characters based at least in part on the spoken word data stored in the database; and 
 after performing the automatic correction process, displaying on the display corrected text corresponding to at least one of the one or more text input characters. 
 
 
 
     
     
       9. The electronic device of  claim 8 , wherein the spoken word data includes information on when the spoken words were spoken. 
     
     
       10. The electronic device of  claim 8 , wherein:
 the display comprises a touch screen; and 
 receiving one or more text input characters comprises gathering the one or more text input characters as the user enters the one or more text input characters with the touch screen. 
 
     
     
       11. The electronic device of  claim 10 , wherein the electronic device comprises a cellular telephone. 
     
     
       12. The electronic device of  claim 8 , wherein performing the automatic correction process comprises automatically replacing a mistyped word in the one or more text input characters with a replacement word without obtaining replacement word choice confirmation input at the electronic device from the user. 
     
     
       13. The electronic device of  claim 8 , wherein performing the automatic correction process comprises automatically substituting a replacement word on the display in response to receiving a space character as part of the one or more text input characters. 
     
     
       14. The electronic device of  claim 8 , wherein performing the automatic correction process comprises automatically replacing a mistyped word in the one or more text input characters with a replacement word in response to receiving a user-supplied confirmation of which of a plurality of displayed potential word replacements is to be used as a replacement word. 
     
     
       15. A non-transitory computer-readable medium storing instructions for operating an electronic device, which, when executed by the device with a storage, processing circuitry and input-output circuitry including at least a microphone and a display, cause the device to perform operations comprising:
 recognizing spoken words; 
 storing spoken word data in a database corresponding to the recognized spoken words; and 
 after storing the spoken word data in the database:
 receiving one or more text input characters; 
 performing an automatic correction process on at least a subset of the one or more text input characters based at least in part on the spoken word data stored in the database; and 
 after performing the automatic correction process, displaying on the display corrected text corresponding to at least one of the one or more text input characters. 
 
 
     
     
       16. The non-transitory computer-readable medium of  claim 15 , wherein the spoken word data includes information on when the spoken words were spoken. 
     
     
       17. The non-transitory computer-readable medium of  claim 15 , wherein:
 the input-output circuitry comprises a touch screen; and 
 receiving the one or more text input characters comprises gathering the one or more text input characters as the user enters the one or more text input characters with the touch screen. 
 
     
     
       18. The non-transitory computer-readable medium of  claim 17 , wherein the electronic device comprises a cellular telephone. 
     
     
       19. The non-transitory computer-readable medium of  claim 15 , wherein performing the automatic correction process comprises automatically replacing a mistyped word in the one or more text input characters with a replacement word without obtaining replacement word choice confirmation input at the electronic device from the user. 
     
     
       20. The non-transitory computer-readable medium of  claim 15 , wherein performing the automatic correction process comprises automatically substituting a replacement word on the display in response to receiving a space character as part of the one or more text input characters. 
     
     
       21. The non-transitory computer-readable medium of  claim 15 , wherein performing the automatic correction process comprises automatically replacing a mistyped word in the one or more text input characters with a replacement word in response to receiving a user-supplied confirmation of which of a plurality of displayed potential word replacements is to be used as the replacement word.

Description:
RELATED APPLICATIONS 
     This is a continuation of U.S. application Ser. No. 12/891,720, filed Sep. 27, 2010, which is hereby incorporated by reference in its entirety. 
    
    
     BACKGROUND 
     This relates generally to systems that assist users in entering text correctly and, more particularly, to systems in which text corrections are made based on voice recognition data. 
     Users of computing equipment such as computers and handheld devices may enter text using input-output equipment. In a typical scenario, a user of a desktop computer may enter text into a word processing program or other software application using a keyboard. In devices with touch screens such as cellular telephones and tablet computers, user may enter text by tapping on on-screen keys. 
     Users are not perfectly accurate when entering text, particularly in situations in which a user is using small keys or when a user is distracted. As a result, user-entered text may contain errors. 
     Error correction capabilities are provided in many devices to assist a user in accurately entering text. Automatic error correction functions are generally implemented using a dictionary of known words. When a user enters a character sequence that does not correspond to a known word, the character sequence may be corrected. In some situations, a mistyped word may be automatically replaced with a corresponding correct word. In other situations, such as when a user pauses after entering a word, the user may be presented with a list of possible alternatives from which the correct word may be selected. 
     Error correction functions such as these are unaware of the user&#39;s environment. As a result, autocorrection accuracy is limited. Mistyped words are generally replaced with commonly used words. While this type of approach may be satisfactory in some circumstances, there are limits to the accuracy level that may be achieved. It would therefore be desirable to be able to provide improve systems for assisting a user in making corrections when entering text. 
     SUMMARY 
     Electronic devices such as cellular telephones and other computing equipment may contain touch screens and other displays. A user of an electronic device may supply text input that contains mistyped words. For example, a user may press an incorrect sequence of letter keys when typing a message or other document. 
     A microphone in an electronic device may be used to gather sound. The microphone may be used, for example, to gather spoken words in the vicinity of the electronic device. A voice recognition engine may be used to process signals from the microphone. Information on recognized spoken words may be stored in the electronic device. For example, a database that reflects the identity and frequency of various spoken words that have been recognized by the voice recognition engine may be maintained by an autocorrection engine. 
     As the user supplies the text input to an electronic device, the autocorrection engine may use information in the spoken word database to determine how best to perform autocorrection operations. The presence of a particular spoken word may, for example, help the autocorrection engine determine which of several possible replacement words would be most suitable to use to replace a mistyped word. By using contextual information from the spoken word database in addition to other information, autocorrection accuracy can be enhanced. 
     Autocorrection operations may be fully automated. With this type of arrangement, the autocorrection engine may make a word replacement automatically in response to receiving a space character or other such separator character from a user that indicates that the user has finished entering a particular word. Semi-automated correction techniques may also be used. For example, an autocorrection engine may display a list of potential word replacements for a user when a user pauses after entering a mistyped word. The order of the potential word replacements in the list may be influenced by the information in the spoken word database. For example, potential replacement words that match frequently spoken words may be provided with elevated prominence in the list. 
     Further features of the invention, its nature and various advantages will be more apparent from the accompanying drawings and the following detailed description of the preferred embodiments. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a perspective view of an illustrative electronic device that may be used to gather voice recognition information while a user is making text entries in accordance with an embodiment of the present invention. 
         FIG. 2  is a diagram of an illustrative electronic device of the type shown in  FIG. 1  in accordance with an embodiment of the present invention. 
         FIG. 3  is a diagram of software components that may be run on a system of the type shown in  FIG. 2  to implement automatic correction features in accordance with an embodiment of the present invention. 
         FIGS. 4A ,  4 B, and  4 C are tables showing illustrative autocorrection data that may be maintained by an autocorrection system in accordance with an embodiment of the present invention. 
         FIG. 5  is a diagram showing the operation of an illustrative device that is using voice-recognition-assisted autocorrection features in accordance with an embodiment of the present invention. 
         FIG. 6  is a flow chart of illustrative steps involved in using spoken word input and text input in performing text autocorrection operations in accordance with an embodiment of the present invention. 
     
    
    
     DETAILED DESCRIPTION 
     Electronic devices may be provided with the ability to process text and voice input. Text input may sometimes contain errors. An electronic device may be provided with autocorrection capabilities to handle erroneous text. The autocorrection capabilities may be implemented using an autocorrection engine. The autocorrection engine may gather voice data using a voice recognition engine. The voice data may include words that have been spoken and recognized by the voice recognition engine. During autocorrection operations, these spoken words may be used to provide context to the autocorrection engine that enhances autocorrection accuracy. 
     Autocorrection operations may be performed using equipment of the type shown in  FIG. 1 . As shown in  FIG. 1 , a user may interact with an electronic device such as device  10  and may interact with additional equipment such as computing equipment  86 . Computing equipment  86  may be, for example, an accessory that operates with device  10 , a server that device  10  can communicate with over a communications path such as a wired or wireless link such as link  84 , or other suitable electronic equipment. Autocorrection software in the system of  FIG. 1  may run on the hardware of device  10 , on the hardware of device  86 , on both device  10  and device  86 , or other suitable computing equipment. Arrangements in which autocorrection functions are implemented using the resources of device  10  are sometimes described herein as an example. This is merely illustrative. Any suitable computing equipment or combination of local and online resources may be used in implementing autocorrection functions if desired. 
     Device  10  may be a desktop computer, a laptop computer, a tablet computer, a handheld computing device such as a cellular telephone, media player, gaming device, or navigation device, a pendant or wristwatch device, consumer electronics equipment such as a set-top box, television, or stereo system, or other electronic equipment. Illustrative examples of devices such as cellular telephones, media players, and tablet computers are sometimes described herein as an example. 
     As shown in  FIG. 1 , device  10  may include a housing such as housing  12 . Housing  12  may be formed from metal, plastic, or other materials. Openings may be formed in housing  12  to form ports. Device  10  may include, for example, ear speaker port  18 , speaker port  22 , electrical connector port  88 , and microphone port  20 . Device  10  may also include one or more buttons such as button  16 . 
     User text input may be provided to device  10  using an integral keyboard or an external keyboard, using a soft set of keys on a touch screen, or using other suitable input equipment. In the example of  FIG. 1 , device  10  includes a touch screen display  14 . Display  14  includes portion  14 T for displaying text  26  to a user and portion  14 B for accepting text entry from a user. On-screen keys (character selection options)  24  may be displayed within region  14 B using a layout such as a QWERTY layout or an alphabetical layout. When a user&#39;s finger touches a particular one of on-screen keys  24 , device  10  can recognize which key was selected and can display an associated character as part of text  26  in region  14 T. 
     The arrangement of  FIG. 1  is merely illustrative. If desired, a wireless or wired keyboard may be used to provide device  10  with text input, a user may enter characters using keys in a keypad on device  10 , etc. 
       FIG. 2  is a schematic diagram showing illustrative components of equipment such as device  10  of  FIG. 1 . As shown in  FIG. 2 , device  10  may include storage and processing circuitry (control circuitry)  74 . Storage and processing circuitry  74  may be implemented using one or more integrated circuits. Storage  76  may include flash memory, hard disk drive memory, solid state storage devices, other nonvolatile memory, random-access memory and other volatile memory, etc. Processing circuitry  78  may include digital signal processors, microcontrollers, application specific integrated circuits, microprocessors, power management unit (PMU) circuits, circuits for processing touch sensor input, audio codec circuits that include analog-to-digital converter circuitry and digital-to-analog converter circuitry, and processing circuitry that is part of other types of integrated circuits. 
     Device  10  may include input-output devices  80  such as displays, speakers, microphones, status indicator light-emitting diodes, sensors such as proximity sensors and accelerometers, touch screens, data port circuits coupled to data ports, analog input-output circuits coupled to audio connectors and other analog signal ports, track pads and other pointing devices, keyboards, key pads, buttons, wireless communications circuitry for forming links such as link  84  with equipment  86 , etc. 
     The components of storage  76 , processing circuitry  78 , and input-output devices  80  need not be mutually exclusive. For example, processing circuitry  78  may contain memory, storage  76  may include circuitry that performs processing functions, and input-output circuitry  80  may include storage circuits and processing circuits. 
     During operation of device  10 , device  10  may gather input. For example, a finger such as finger  81  or other external object may be used to touch keys  24  (e.g., on-screen keys) or keys in a keyboard in circuitry  80 . Text may be entered one character after another to form text strings (e.g., words). For example, text may be entered when a user is composing a document such as an email message, a text message, a spreadsheet, or a word processing document, or when a user is otherwise interested in providing software on device  10  with text input (e.g., when filing in a dialog box, when entering a search term into a browser, when entering information into a calendar application, when performing operating system functions such as file search functions and command line functions, etc.). 
     In addition to gathering text input from a user, device  10  may use a microphone in circuitry  80  (e.g., a microphone within port  20 ) to gather audio input (sound) from a source such as sound source  82 . Source  82  may be a person other than the user who is in the vicinity of device  10  and the user, may be the user of device  10 , may be a person who is having a voice telephone call with the user over a cellular network or wireless local area network, or other suitable source of spoken word content (voice information in audio form). Device  10  may use input-output circuitry  80 , processing circuitry  78 , and storage  76  to perform voice recognition operations. For example, voice recognition engine code for performing voice recognition operations may be stored on storage  76 , the code may be executed using processing circuitry  78 , and, during execution of the voice recognition code, input-output circuitry  80  may use a microphone to gather sound such as spoken words and may digitize and recognize the gathered sound as words using an analog-to-digital converter and the resources of storage and processing circuitry  74 . 
     Words that are recognized using voice recognition may be maintained in a database. The words that are detected in this way may be used to provide an autocorrection engine in device  10  with additional context to use in performing text autocorrection operations. If, for example, a user misspells a word, device  10  can consult a list of recently recognized words in the database to determine whether a correct version of the misspelled word is present. In situations in which multiple correctly spelled words are similar in spelling to a misspelled word, the use of the words in the voice recognition database may help device  10  decide which word is correct. 
     Autocorrection functions may be implemented using an input processor having an autocorrection capability. An input processor with autocorrection capabilities (i.e., an input engine with an autocorrection engine) may be implemented using code that is stored in storage  76 , that is executed using processing circuitry  78 , and that uses input-output circuitry  80  (e.g., to gather user input and to display text and other output for the user). 
     Software that may be executed on device  10  of  FIG. 2  is shown in  FIG. 3 . The software components of  FIG. 3  may be implemented as applications, as parts of applications, as online services, as operating system functions, as blocks of code that are called by applications, operating system functions, or online services, as processes that run on localized or distributed equipment, as hardware-accelerated software, or as any other suitable software. As shown in  FIG. 3 , user text input  38  may be supplied to input processor  30 . User text input  38  may be gathered from a user using text processing interface  34 . Interface  34  may use hardware resources such as a keyboard and/or touch screen (on-screen) keyboard to gather text character entries by a user. Spoken input  40  (e.g., sound containing words) may be gathered using voice recognition engine  36 . Voice recognition engine  36  may maintain a database of word sounds and words to allow gathered sound to be identified as particular words. Engine  36  and interface  34  may run on circuitry such as storage and processing circuitry  74  of  FIG. 2  and may use the resource of circuitry  80 . 
     Input processor and autocorrection engine  30  may maintain one or more database such as database  32 . Database  32  may contain a dictionary of correctly spelled words, may contain mappings from common misspellings to correct spellings, and other data that input processor and autocorrection engine  30  may use in correcting errors in text input  38 . Text that is supplied as input  38  (possibly containing errors) may be corrected by input processor  30  and the corresponding corrected version of the text input may be provided to software such as application  28  that uses text input. Software  28  may be an operating system function, a word processor that displays text in the form of text documents, an email client that displays text for the body of email messages, a spreadsheet program that displays text as part of a spreadsheet, an instant messaging client that displays text associated with text messages, or other suitable software. 
     Spoken words that are recognized by voice recognition engine  36  and information on their frequency and timing may be stored in autocorrection database  32 . If, for example, a user of device  10  is in the vicinity of a lecturer in a lecture hall, input processor  30  may store words from a lecture that is being presented by the lecturer in database  32 . As input processor  30  analyzes errors in text input  38  to determine how best to correct these errors, input processor  30  may consult the contents of database  32 . The recognized spoken words from the lecture that are stored in database  32  may provide useful contextual information that assists input processor  30  in accurately determining how to proceed in correcting errors. If, for example, a word such as “hit” has recently been spoken, input processor  30  may use this information in determining that a mistyped word such as “het” should be corrected to read as “hit” rather than being corrected to read as “the.” 
     The amount of influence that the recognized words in database  32  have on the operation of input processor  30  may vary as a function of time and other factors. If, for example, a particular word was recently spoken, that word may be given more influence in determining the outcome of a particular error correction operation than if the word was not recently spoken. Illustrative time frames for maintaining spoken words in database  32  include ten seconds, less than ten seconds, 30 seconds or less, 1-5 minutes or less, 20 minutes or less, 1-10 hours, 1-10 days, etc. 
     User agreement (e.g., in the form of user selection of a particular correction from a list of possible corrections) may be used to help train input processor  30 . For example, if a spoken word that was recognized by voice recognition engine  36  and that was stored in database  32  is presented to the user as a possible replacement for a mistyped word, the user may select that word to use as a replacement for the mistyped word (e.g., by selecting an on-screen option on display  14 ). Input processor  30  may use database  32  to maintain information on user-confirmed corrections of this type (sometimes referred to as semiautomatic corrections) and may use this information to improve autocorrection accuracy (e.g., by making the user-selected correction more prominent in subsequently displayed lists of possible corrections, by using the user-selected correction when appropriate during operation in a fully automatic correction mode, etc.). 
     Autocorrection database information such as information in database  32  that includes voice recognition data from voice recognition engine  36  (e.g., recognized versions of the words in spoken input  40 ) may be stored in one or more tables or other suitable data structures. Illustrative database tables of the type that may be used in database  32  are shown in  FIGS. 4A ,  4 B, and  4 C. 
     In the scenario of  FIG. 4A , device  10  is being operated in a quiet environment, devoid of spoken words. As a result, there are no voice-recognition entries from voice recognition engine  36  in database  32 . Table  42  of the example of  FIG. 4A  has three columns. Column  44  contains entries corresponding to potential text input (e.g., typed text entries). In an environment in which spoken words are recognized by voice recognition engine  36 , column  46  would be populated with the words that were recognized by voice recognition engine  36  (e.g., words that were recognized within a suitable period of time before the present time, as described in connection with  FIG. 3 ). In the example of  FIG. 4A , the operating environment of device  10  is quiet, so column  46  is empty. Column  48  contains suitable corrected versions of the words appearing in the same row of column  44 . 
     During operation of device  10  input processor  30  may use table  42  of  FIG. 4A  to determine how to correct text input  38 . If, for example, a user types the string “the” into device  10 , the string “the” will be received by text processing interface  34  ( FIG. 3 ) and processed by input processor  30  to remove errors. As shown in the first row of table  42  in  FIG. 4A , the word “the” in column  44  is a valid (correct) word, so the corresponding corrected version of the word “the” in column  48  (i.e., the string “the” in the first row of column  48 ) is unchanged. If a user mistypes the word “the,” however, the user&#39;s mistake may be corrected by the autocorrection engine of input processor  30 . For example, if the user types “het” (see, e.g., the second row of column  44  in table  42  of  FIG. 4A ), input processor  30  may replace the text “het” with the corrected version of this string (i.e., the text “the” from the second row of column  48  in table  42  of  FIG. 4A ). 
     In an environment in which voice recognition engine  36  is able to recognize the presence of words of spoken input  40 , the presence of the recognized words may be reflected in database  32 . As shown in the scenario of  FIG. 4B , for example, database  32  may contain information such as the information in column  46  that reflects the recent detection of the spoken word “hit” by voice recognition engine  36 . This information may be stored in a database table or using any other suitable format. In situations in which the word “hit” has recently been spoken, it is appropriate to perform autocorrection operations differently than in situations in which no words have been spoken. If the user types a word correctly (as with “the” in the first row of column  44  in table  42  of  FIG. 4B ), processor  30  may perform the same type of correction as in the scenario of  FIG. 4A  (i.e., by leaving the “the” entry unchanged as shown by the entry in the first row of column  48  in table  42  of  FIG. 4B ). If, however, the user types “het” (as shown in the second row of column  44  of table  42  of  FIG. 4B ), input processor  30  may change “het” to “hit.” In the presence of recent occurrences of the spoken word “hit,” it is more appropriate to change “het” to “hit” (as in  FIG. 4B ) than to change “het” to “the” (as in  FIG. 4A ). The use of recognized spoken word data (i.e., the “hit” information shown in column  46  of table  42  in  FIG. 4B ) by input processor  30  may therefore allow processor  30  to correct errors in text input  38  with enhanced accuracy. 
       FIG. 4C  shows how autocorrection accuracy may be improved in another scenario. In the example of  FIG. 4C , spoken input  40  such as the spoken word “ETA” (which may be pronounced, “eta” or as individual letters in an acronym such as “e” . . . “t” . . . “a”) has been detected by voice recognition engine  36  and stored in database  32  (depicted as the entry in column  46  of table  42  in  FIG. 4C ). In this type of environment (i.e., because “eta” was recognized rather than “hit” as in the  FIG. 4B  example), input processor  30  may correct the mistyped word “het” by replacing “het” with “eta” as shown in the first row of columns  44  and  48  of table  42  in the  FIG. 4C  example, rather than by replacing “het” with “hit” as in the  FIG. 4B  example. The presence of different spoken words in database  32  may therefore influence the types of real time correction that are made on text input  38 . The prevalence of each word may be represented by weighting factors, using histograms (e.g., word count totals in a given time period), using time-weighted averages, or may otherwise be used to influence which correction is made by input-processor  30 . The simplified examples of  FIGS. 4B and 4C  in which only a single detected spoken word is being maintained in database  32  are merely illustrative. 
       FIG. 5  shows how the process of correcting a mistyped word (“het”) with a corrected word (“hit,” as in the example of  FIG. 4B ) may be performed by input processor  30 . 
     Initially, a user may enter the character “h” (shown in box  50  of  FIG. 5 ). This letter may be displayed immediately for the user on display  14 . When the user types “e” into device  10 , device  10  may display the “e” adjacent to the “h” to form the string “he” (shown in box  52 ). If the user continues typing and enters the character “t,” device  10  may display “het” on display  14  (see box  54  of  FIG. 5 ). 
     The behavior of device  10  after the user types the misspelled word “het,” may depend on whether the user continues typing or whether the user pauses (as an example). If the user continues typing by typing a space character, the incorrect word “het” and the space character may be displayed on display  14  (see box  56 ) in response to detection of the space character (or other such separator character). In this situation, the fully automatic correction capabilities of the autocorrection engine in input processor  30  may be used to automatically correct the incorrect word “het” by replacing “het” with “hit” on display  14  in real time (see, e.g., box  58 ). 
     If, however, the user pauses (e.g., for a fraction of a second or longer) after typing “het” (box  54 ), input processor  30  may present the user with a list of possible choices for replacement words. As shown in box  60  of  FIG. 5 , for example, the user may be presented with the suggested corrections of “hit” and “the.” The user may select a desired correction from the displayed list (e.g., by using a touch command such as a tap on the desired correction or by moving a movable highlight followed by an enter command). 
     In the example of  FIG. 5 , the user has selected the replacement word “hit” to use in correcting the incorrect text “het.” In presenting suggested corrections in the correct words list of box  60 , input processor  30  may more prominently display words that are more likely to be correct than words that are less likely to be correct. For example, words that are more likely to be correct (i.e., “hit” in this example) may be displayed in list  60  above words that are less likely to be correct (i.e., “the” in this example). The ordering of the words in list  60  may be influenced by the contents of the recognized words in database  32  (i.e., the presence of a recently spoken word “hit” in this example). Depending on which words were most recently spoken (and/or which words were spoken most frequently), certain words may be moved up or down in list  60 . 
     Illustrative steps involved in operating device  10  in an environment in which spoken words are present while a user is supplying text to the device are shown in  FIG. 6 . 
     At step  64 , device  10  may gather input from the environment. During the operations of step  66 , for example, voice recognition engine  36  of input processor  30  ( FIG. 3 ) may use a microphone and other input-output circuitry  80  in converting spoken input  40  from sound source  82  into recognized words in database  32 . Any suitable type of information may be maintained on words that have been recognized by voice recognition engine  36  in database  32 , such as information on when the words were recognized (e.g., in the form of a time stamp or data indicating detection within a moving time window), how frequently the words occur (e.g., in the form of raw data or a normalized histogram or other data structure that reflects whether the prevalence of a particular spoken word within a certain time period is higher or lower than would normally be expected), whether the spoken words grammatically agree with other spoken and/or typed words, or other information regarding the expected popularity of the spoken words relative to typed text and/or spoken word context. This data may be used in enhancing auto correction accuracy. 
     During the operations of step  68 , text input may be gathered from a user. For example, a touch sensor with soft keys or a keyboard in circuitry  80  may be used to gather text input  38  corresponding to touch screen touch events and/or key presses by a finger or other external object (object  81  of  FIG. 2 ). Characters may, as an example, be entered one at a time as described in connection with  FIG. 5 . Entered text may be gathered using text processing interface  34  of input-processor  30  ( FIG. 3 ). Words, parts of words, and other character strings may be gathered at step  68 . 
     During the operations of step  70  (some or all of which may be performed simultaneously with the operations of step  64 ), input processor  30  may use the gathered input from step  64  to update one or more databases in device  10  (represented as autocorrection database  32  in  FIG. 3 ). Database  32  may be updated by include information identifying each spoken word that has been recognized, timing information on each spoken word, information on typed text, information on the timing of typed text, etc. To reduce storage requirements, data may be compressed. For example, spoken word data may be represented using frequency data rather than time-stamped information, may be represented by moving averages or other weighting information, may be maintained only for events within a moving time window (e.g., 5 minutes before the present time to the present time), etc. 
     Input-output circuitry  80  may display text as it is entered by the user on display  14  (e.g., within a window associated with application  28 , within an overlay window controlled by input processor  30 , etc.). The boxes of  FIG. 5  show how device  10  may display one entered character at a time in real time as the user makes text character entries. 
     While the user is entering text, the autocorrection functions of input processor and autocorrection engine  30  may be active. Automatic corrections may be made as the text is entered. 
     As described in connection with boxes  54 ,  56 , and  58  of  FIG. 5 , autocorrection operations may be fully automated so that mistyped words are replaced automatically with correct words as the user types (i.e., without need for the user to select from a group of proposed replacement words or otherwise supply replacement word choice confirmation input). Fully automated corrections may be made, for example, when a user continues typing after entering a word, as evidenced by entry of one or more spaces, carriage returns, or other separator characters. 
     As described in connection with boxes  54 ,  60 , and  62  of  FIG. 5 , autocorrection operations may also be implemented using a semi-automatic approach in which suggested replacement words are displayed on display  14  for selection by the user (i.e., for the user to select by tapping or otherwise highlighting and thereby confirming a selected word as a desired replacement word choice). The prominence of each suggested replacement word in the list of suggested replacement words that is displayed may be varied as a function of the likelihood that each word is a suitable replacement. The words that are the best possible replacements for a mistyped word may, for example, be placed at the beginning of an ordered list rather than at the end of an ordered list or may otherwise be indicated to be the most suitable replacements for the mistyped word. Suggested word prominence on display  14  (e.g., the location of the suggested word within the list of box  60  of  FIG. 5 ) may be determined based at least partly on spoken word information maintained in database  32 . For example, if the word “eta” has been recognized as a recent spoken word, the suggested replacement of “eta” for the mistyped word “het” may be given prominence over the suggested word replacement of “the.” 
     Word corrections may be made automatically as complete words are typed or as parts of words are typed. As indicated by line  72  in  FIG. 6 , operations may loop back to step  64 , so that text input processing operations, voice recognition operations, and autocorrection operations may be performed continuously (e.g., until device  10  is turned off, until application  28  of  FIG. 3  is deactivated, etc.). 
     The foregoing is merely illustrative of the principles of this invention and various modifications can be made by those skilled in the art without departing from the scope and spirit of the invention.

Metadata:
Filing Date: 20130722
Publication Date: 20150707
Grant Date: 20150707
Priority Date: 20100927
Inventors: WAGNER OLIVER P.
Assignee: APPLE INC
CPC Classifications: [{"code": "G06F40/166", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F40/232", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L15/26", "inventive": true, "first": false, "tree": "[]"}, {"code": "G10L15/26", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F17/273", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F17/24", "inventive": true, "first": true, "tree": "[]"}, {"code": "G10L15/26", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F40/232", "inventive": true, "first": true, "tree": "[]"}]
Family ID: 45871524